The algorithm and data structures conundrum
Hacker news denizens have a penchant with blog posts around recruiting programmers. Every once in a while, a post appears on the home page around this topic followed by a passionate and sometimes violent debate. People debating on this can usually by divided into two camps, one camp argues that math, algorithm and data structures are very important and candidates should be quizzed on them and the other takes a stand that it is not. I find these discussions amusing. If you have not taken a look at these dog fights before, I highly encourage you to. See one example here.
These discussions most of the times miss the real essence of the issue, the question is not whether algorithms and data structures are a requirement for a programmer, it goes without saying it is, the topic that should really be introspected is, is it important for candidates to be interviewed on these and to what extent is this knowledge needed for your day to day programming tasks. Your interview should reflect the fact that how deep an understanding one should have on this topic to excel at their tasks. Are you fine if some one knows the different sorting algorithms and has a superficial understanding of their trade offs or do you want someone who can go to the board and write the algorithm from scratch and explain the math behind why such and such an algorithm has an average time complexity of nlogn? I can very confidently say that for most of the day to day work, you do not need to ponder on the mathematics behind algorithms and complex data structures like red black trees or skip lists. What is really needed as a programmer is neat coding skills, good design, tenacious curiosity and passion. I will expand on each of them below, I can write tomes on these subject, but will try to keep it as short as possible.
Neat coding skills should be a no brainer but still let me expand on it. What do I mean when I say neat coding skills? The very basics, splitting long code into small functions, appropriate comments in the code, test cases, following code conventions and the spirit of the language. This is the foundation of a great project and has a tremendous impact on the future of your project. If these basic tenets of programming are not followed, scaling the project and modifying it in future becomes another project on it’s own.
Good design is what helps you scale your application as the business needs change, more users come in etc. To give an example of what I consider good design in a web project would be moving all your business logic into a separate layer and not tying it up with your controllers/views. I have personally seen this helping us at burrp! when we completed our mobile app in a month’s span as our design was good enough to support a new UI with minimal changes to the back end. If we had scrambled all the business logic in the controller itself instead of abstracting it into a different layer, there was no way we could have developed a mobile app which had most of the functionality of the main site in such a short span of time.
Now coming to the most important part, curiosity. Why is this needed? Without curiosity we stagnate. We become complacent. We do not invent. We do not try to become better. We think what we know is the ultimate truth and what ever language/framework/methodology that we use is the end of the world, there is nothing better than that. If you do not look around and explore how will you know that there are other frameworks in java to write a webapp other than the usual suspects of struts/springmvc etc? How will you know that OOPS is not the only principle behind programming? Or for that matter, there are programming languages out there other than Java? Do not laugh at this one, I have talked to professional programmers who did not even know there was a programming language called PHP.
If you are tenaciously curious, when faced with a problem, you always start looking around. Say, for example, if you feel that a part of your code is slow, you dig into it and explore other options. In this process you come across people who have faced similar issues and resolved the problem using a better algorithm or data structure. If you are in a problem domain which requires the novel use of algorithms and data structures on a regular basis, someone with curiosity will develop that on the job. But the sad fact is for most of the programming tasks these days, you do not need to wrangle algorithms and data structures on a daily basis. This is the reason why most of the programmers are rusty on these subjects and in interviews when you start drilling them on this, you see the look of the deer suddenly confronted by traffic lights in their eye. The fact that you are being constantly monitored and evaluated adds to the downward spiral.
I would say interviews focused heavily on data structures and algorithms are rigged towards new college students and people who are just preparing for interviews as most of these question are in no way novel. They tend to be a common set of questions which keep getting regurgitated. Go to one of these interview question sites, spend a couple of days trying to solve these question and suddenly you become an algorithm and data structures expert in the eyes of the interviewer. Just because google does this, everyone does it and it has got to a point where interviews are more about showing off your data structure and algorithm skills rather than evaluating whether the person being interviewed is a good fit for the job and can handle the day to day tasks of your business. I am not even sure as to whether even google, facebook etc need to do this for all their departments. Also, an unhealthy side effect of this sort of interviewing is that in a fury to concentrate on your algorithm and data structure skills, everything else is relegated to a corner. Companies hardly put in the effort to see how good your coding fundamentals are, have you taken the effort to know the philosophy behind the language that you have been using for the past two years, how many times have you re factored your code in your previous project, have you put in the effort to learn your IDE/editor etc etc.
Unless you are working on some high end cutting edge computer science research project, you do not need people who can solve algorithm and data structure problems in interviews. Hiring people who are infinitely curious will give you a much better bang for your buck.