Month: February, 2013

The algorithm and data structures conundrum

Hacker news denizens have a penchant with blog posts around recruiting programmers. Every once in a while, a post appears on the home page around this topic followed by a passionate and sometimes violent debate. People debating on this can usually by divided into two camps, one camp argues that math, algorithm and data structures are very important and candidates should be quizzed on them and the other takes a stand that it is not. I find these discussions amusing. If you have not taken a look at these dog fights before, I highly encourage you to.  See one example here.

These discussions most of the times miss the real essence of the issue, the question is not whether algorithms and data structures are a requirement for a programmer, it goes without saying it is, the topic that should really be introspected is, is it important for candidates to be interviewed on these and to what extent is this knowledge needed for your day to day programming tasks. Your interview should reflect the fact that how deep an understanding one should have on this topic to excel at their tasks. Are you fine if some one knows the different sorting algorithms and has a superficial understanding of their trade offs or do you want someone who can go to the board and write the algorithm from scratch and explain the math behind why such and such an algorithm has an average time complexity of nlogn? I can very confidently say that for most of the day to day work, you do not need to ponder on the mathematics behind algorithms and complex data structures  like red black trees or skip lists. What is really needed as a programmer is neat coding skills, good design, tenacious curiosity and passion. I will expand on each of them below, I can write tomes on these subject, but will try to keep it as short as possible.

Neat coding skills should be a no brainer  but still let me expand on it. What do I mean when I say neat coding skills? The very basics, splitting long code into small functions, appropriate comments in the code, test cases, following code conventions and the spirit of the language. This is the foundation of a great project and has a tremendous impact on the future of your project. If these basic tenets of programming are not followed, scaling the project and modifying it in future becomes another project on it’s own.

Good design is what helps you scale your application as the business needs change, more users come in etc. To give an example of what I consider good design in a web project would be moving all your business logic into a separate layer and not tying it up with your controllers/views. I have personally seen this helping us at burrp! when we completed our mobile app in a month’s span as our design was good enough to support a new UI with minimal changes to the back end. If we had scrambled all the business logic in the controller itself instead of abstracting it into a different layer, there was no way we could have developed a mobile app which had most of the functionality of the main site in such a short span of time.

Now coming to the most important part, curiosity. Why is this needed? Without curiosity we stagnate. We become complacent. We do not invent. We do not try to become better. We think what we know is the ultimate truth and what ever language/framework/methodology that we use is the end of the world, there is nothing better than that. If you do not look around and explore how will you know that there are other frameworks in java to write a webapp other than the usual suspects of struts/springmvc etc? How will you know that OOPS is not the only principle behind programming? Or for that matter, there are programming languages out there other than Java? Do not laugh at this one, I have talked to professional programmers who did not even know there was a programming language called PHP.

If you are tenaciously curious, when faced with a problem, you always start looking around. Say, for example, if you feel that a part of your code is slow, you dig into it and explore other options. In this process you come across people who have faced similar issues and resolved the problem using a better algorithm or data structure. If you are in a problem domain which requires the novel use of algorithms and data structures on a regular basis, someone with curiosity will develop that on the job. But the sad fact is for most of the programming tasks these days, you do not need to wrangle algorithms and data structures on a daily basis. This is the reason why most of the programmers are rusty on these subjects and in interviews when you start drilling them on this, you see the look of the deer suddenly confronted by traffic lights in their eye. The fact that you are being constantly monitored and evaluated adds to the downward spiral.

I would say interviews focused heavily on data structures and algorithms are rigged towards new college students and people who are just preparing for interviews as most of these question are in no way novel. They tend to be a common set of questions which keep getting regurgitated. Go to one of these interview question sites, spend a couple of days trying to solve these question and suddenly you become an algorithm and data structures expert in the eyes of the interviewer. Just because google does this, everyone does it and it has got to a point where interviews are more about showing off your data structure and algorithm skills rather than evaluating whether the person being interviewed is a good fit for the job and can handle the day to day tasks of your business. I am not even sure as to whether even google, facebook etc need to do this for all their departments.  Also, an unhealthy side effect of this sort of interviewing is that in a fury to concentrate on your algorithm and data structure skills, everything else is relegated to a corner. Companies hardly put in the effort to see how good your coding fundamentals are, have you taken the effort to know the philosophy behind the language that you have been using for the past two years, how many times have you re factored your code in your previous project, have you put in the effort to learn your IDE/editor etc etc.

Unless you are working on some high end cutting edge computer science research project, you do not need people who can solve algorithm and data structure problems in interviews. Hiring people who are infinitely curious will give you a much better bang for your buck.


Programmers and movies

I just finished watching Looper and really loved it. In the beginning, for sometime, my brain was scrambling around like a headless chicken trying to fit things together. After settling into the movie for about half an hour, things started to make sense like a mist evaporating from a glass pane as the sun rises up in the sky. Movies are meant to be relaxing, but this was no where close to relaxing, my brain was working over time trying to figure out what was going on and still I relished it. This got me thinking, why do programmers love movies and shows like Looper, Inception, Matrix, Lost etc.

None of the movies/shows listed above are straight forward to grasp, they move around in tangles with plots with in plots. Take for example, Friends with benefits, you could detach your brain, keep it aside and still the movie would make sense to you(not to say I did not enjoy it :)). Movies like Looper,  you have to watch them with unparalleled concentration so that you do not miss something subtle the whole movie hinges on or some key plot which is the glue that holds the whole movie together or that one dialogue from the hero which is so mind numbing that you start thinking the director is the Buddha reincarnated. Sometimes, even after assuming you have got it, you read a blog where a person has a total different interpretation of the movie and you start doubting your own grasp of the movie. Was this what the director really trying to portray would be the conversation running in your brain for sometime. Is this person correct or am I in the right?

If you think deeply enough, the answer is crystal clear.  Why did you spend that weekend diving deep inside the code written by someone else, chasing that one bug which used to happen once in a while in your app? Or spend a couple of days trying to break into stripe’s servers? Or rewrite a piece of code the nth time thinking that finally you have the most legible and maintenable piece of code ever? A programmer’s brain snatches on to things which are not easy enough for it to grasp, which need reasoning and logic to untangle it. Many of these movies are not just about the movie, it is about the whole mind fuck experience, the digging in through internet trying to figure out that one scene in the movie which you did not understand, the endless debate with your friends as to what the director is really portraying in the movie and to some extent also the knowledge of being in that elite minority who really got what the movie was about. Last but not the least, these movies are like jigsaw puzzles where some of the pieces are missing and there is no one one to supply you with the missing pieces. It is upto you to craft these missing tiles, stitch them together and complete the puzzle. Show me a programmer who does not like a good puzzle to solve.