The Algorithm and Data Structures Conundrum

Hacker news denizens have a penchant for articles on hiring. Every once in a while, a post appears on the home page followed by a passionate and vituperative debate. People conversing on this can be divided into two camps; one argues that math, algorithm and data structures are critical and interview candidates should be quizzed on them while the other group takes the opposite view. If you are not privy to these conversations, I highly encourage you to go through them.

In my opinion, these discussions miss the real essence of the issue. The question is not whether algorithms and data structures are a requirement for a programmer, it goes without saying it is. What we should debate is the depth of knowledge someone should have on these to be effective at their day to day work. Are we after someone who knows the different sorting algorithms and has a fair understanding of their trade-offs? At work, if this person faces a tricky problem, she knows where to start and then reaches out to Google to aid her? Or, do we want someone who can go to the whiteboard and code the algorithm from scratch and explain the math and computational complexity right in the interview? I can very confidently say that for most of the day to day work, you do not need to ponder on the mathematics behind algorithms and complex data structures. What is required is sound coding and design skills, tenacious curiosity, and a passion for excelling.


What do I mean when I say sound coding skills? The very basics; splitting lengthy code into modules, appropriate comments in the code, test cases, following code conventions and the spirit of the programming language. The above is the foundation of a great project and has a tremendous impact on the future of the project. If you do not follow these basic tenets of programming, scaling the project and modifying it in future becomes a project of its own.

Good design is what helps you scale your application as the business needs change and user base expands. In a web project, a good design would be to have business logic in a separate layer and not tying it up with controllers and views. I have personally seen this helping us at burrp! where we rolled out our mobile app in a month as our backend was flexible enough to support a new interface with minimal changes. If our design was not good, there was no way we could have developed a mobile app which had most of the functionalities of our website in such a short period.

Now coming to the most critical part, curiosity. Why is curiosity such an important trait? Without curiosity, we stagnate. We become complacent. We do not innovate. We do not try to become incrementally better. We think what we know is the ultimate truth and whatever language/framework/methodology that we use is the best. I have interacted with professional programmers who were not aware of PHP which powers a substantial number of websites.

If you are tenaciously curious, when faced with a problem, you start obsessing about it. You see it as a puzzle that has to be solved. For example, if you feel that a part of your code is slow, you dig deep into it and try to figure out what is happening. In this process, you come across people who have faced similar issues and solved the problem using a better algorithm or data structure.

I would say interviews focused heavily on data structures and algorithms are biased towards new college students and people who are just preparing for interviews. Most of these questions are in no way novel. Go to one of the interview preparation sites, spend a couple of days trying to solve the problems and suddenly you become an algorithm and data structures expert. Just because FANGs ask these kinds of questions in their interviews, everyone has adopted this interviewing style, and it has got to a point where interviews are more about showing off the interviewer’s data structure and algorithm skills rather than evaluating whether the person on the other side of the table is a good fit for the job. Also, an unhealthy side effect of this is that in a fury to concentrate on algorithms and data structure skills, everything else gets sidelined.

Unless you are working on some cutting-edge niche project, you do not need people who can whiteboard algorithm and data structure problems in interviews. Hiring people who are infinitely curious and can get incrementally better will give you a much better bang for your buck.

Programmers and movies

I just finished watching Looper and really loved it. In the beginning, for sometime, my brain was scrambling around like a headless chicken trying to fit things together. After settling into the movie for about half an hour, things started to make sense like a mist evaporating from a glass pane as the sun rises up in the sky. Movies are meant to be relaxing, but this was no where close to relaxing, my brain was working over time trying to figure out what was going on and still I relished it. This got me thinking, why do programmers love movies and shows like Looper, Inception, Matrix, Lost etc.

None of the movies/shows listed above are straight forward to grasp, they move around in tangles with plots with in plots. Take for example, Friends with benefits, you could detach your brain, keep it aside and still the movie would make sense to you(not to say I did not enjoy it :)). Movies like Looper,  you have to watch them with unparalleled concentration so that you do not miss something subtle the whole movie hinges on or some key plot which is the glue that holds the whole movie together or that one dialogue from the hero which is so mind numbing that you start thinking the director is the Buddha reincarnated. Sometimes, even after assuming you have got it, you read a blog where a person has a total different interpretation of the movie and you start doubting your own grasp of the movie. Was this what the director really trying to portray would be the conversation running in your brain for sometime. Is this person correct or am I in the right?

If you think deeply enough, the answer is crystal clear.  Why did you spend that weekend diving deep inside the code written by someone else, chasing that one bug which used to happen once in a while in your app? Or spend a couple of days trying to break into stripe’s servers? Or rewrite a piece of code the nth time thinking that finally you have the most legible and maintenable piece of code ever? A programmer’s brain snatches on to things which are not easy enough for it to grasp, which need reasoning and logic to untangle it. Many of these movies are not just about the movie, it is about the whole mind fuck experience, the digging in through internet trying to figure out that one scene in the movie which you did not understand, the endless debate with your friends as to what the director is really portraying in the movie and to some extent also the knowledge of being in that elite minority who really got what the movie was about. Last but not the least, these movies are like jigsaw puzzles where some of the pieces are missing and there is no one one to supply you with the missing pieces. It is upto you to craft these missing tiles, stitch them together and complete the puzzle. Show me a programmer who does not like a good puzzle to solve.