Void

Nothing is sacrosanct

There is an interesting bug opened against Kafka. For those of you too lazy to click on the link and read through the description, I am reproducing it here in full.

It appears that validation of configuration properties is performed in the ConsumerConfig and ProducerConfig constructors. This is generally bad practice as it couples object construction and validation. It also makes it difficult to mock these objects in unit tests.

Ideally validation of the configuration properties should be separated from object construction and initiated by those that rely/use these config objects.

http://misko.hevery.com/code-reviewers-guide/flaw-constructor-does-real-work/

It links to an article by Misko Hevery on writing testable code. If you have not read posts by Misko Hevery, I urge you to. For an open source project like Kafka, it might make sense(Jay Kreps, the person behind Kafka does not agree with the bug as visible in the comment) to follow all the guidelines of writing testable code but for your project it might not. If you are a small company with a two or three person team, do not blindly follow practices because someone on the internet says so.

Follow rules and guidelines only if it helps you to make your code more secure, performant, easier to maintain etc, do not ape guidelines without understanding why they are laid out in the first place. To go back in history, checked exceptions were all the rage in the Java land a couple of years ago, but these days, after frameworks like Spring sprang up, people look down upon checked exceptions. Same with TDD. TDD was expounded as the next best thing after sliced bread, but now programmers are raising their doubts about TDD.

A lot of times, mocking objects, interfaces etc takes more work than writing the actual functionality. In many projects, there might not be an ROI in writing/maintaining this elaborate test framework/infrastructure. It is true that injecting dependencies into an object makes it easier to test, but it also comes with the downside of having to take care of injecting those dependencies every time you create that object. If you are injecting dependencies by hand, object creation becomes an elaborate exercise each time, else you have to delegate this to some framework like Spring, Guice etc, now your project is bloated. Maybe your should side step dependency injection and create the object with the dependencies inside it.

The situations and background under which these rules/guidelines crop up might be radically different that the one in your organisation/team. As with most other things, taking the call of what to follow and what not to is more of an art than science. Your instincts, past experiences etc help you in formulating them.

At what cost?

I happened to read the comic by zenpencil on Jim Henson yesterday while reminiscing on the comic by the same artist on Bill Waterson’s advice. Both the comics have the same essence of escaping corporate drudgery and following your dreams. Also yesterday, I watched Inside Llewyn Davis. Inside Llewyn Davis is a melancholic movie that depicts the daily struggles of a musician who has given up his job as a seaman to become a musician. Serendipitous right?

Most of the time, I get a feeling of disenchantment from people about their occupation. To put it bluntly, it is fashionable to be cynical about one’s job and complain about it. A lot of people feel that they want to do something different but most of them do not know what is this different path they want to take. And to fuel this fire, you have hundreds of books and blog posts which extol you to give up your job and follow your passion.

You go on a scuba diving holiday and all of a sudden you want to be a professional scuba diver. You do a couple of hikes and a la, you aspire to be a travel writer. You have started to cycle to office and you have this strong urge to start a bicycle touring company. You purchased this new DSLR and now your single aim in life is to be a wildlife photographer. An article about a so and so who gave up his cushy corporate job to start a local bike store or became a wildlife photographer reinforces these thoughts.

Even though these thoughts are romantic and warm the cockles of our heart, the reality is a bit different. When a hobby becomes a job, the fun aspect of the hobby goes out of the window and the boring part kicks in. When you do something repetitively, the novelty wears off. Some paragraphs from this article on becoming a travel writer:

But while free trips, global travel and your name in print sound glamorous, there are down sides. It’s hard work, hugely competitive and – unless you are the second Bryson – you won’t earn much. Roving overseas with a notebook, a deadline and a pack of other journalists can also take the fun out of travelling altogether. Not put off? Read on to find out how you can get this dream job.

Below is a paragraph from an article by a scuba diving instructor:

The job duties of an instructor aren’t what most newcomers expect, either. And to many it comes as a sad surprise. The reality of being an instructor at least full-time is that teaching is only a small portion of what you’ll do. Mainland-based instructors often work 40 hours a week at a dive store counseling customers or repairing equipment, then teach one or two classes a week on top of that. That means 60-hour weeks are commonplace. Think it’s easier at a resort? The norm for resort-based instructors is six days a week, and during busy periods seven days isn’t uncommon. Here, too, teaching is only a minor part of the job, but an instructor ticket is essential, if for no other reason than to get a permit to work in a foreign country. Many have left the industry disappointed that their dream of spending their days primarily as teachers never materialized.

And to top it all, you have to read this from a person who started his own micro brewery.

A lot of times, we misunderstand novelty for passion. You have a sedentary desk job and travelling once in a while looks like life’s calling, but the question to ask is, would it still be your life’s calling if you had to do it 24 by 7 while earning a substantially lower income? Would you not be more happy earning a good salary, enjoying the material comforts that your day to day job provides and travelling once in a while to break the routine?

In most of these offshoot jobs, the number of slots where you can be comfortable with the income is limited and the aspirants for these slots are unlimited. Also, in a majority of these, you have to be at the pinnacle to earn really well. We all love to think that if we are talented, success naturally follows, but I call this specious. There is more to success than just talent, success is mostly a factor of being in the right place at the right time and luck, not to say that talent and other factors do not help, but it is for sure not only talent. Thinking fast and slow by Daniel Kahneman expounds a bit on this.

The melancholy that sets in with a day to day job is our own making. If you are really interested in spicing up your work, there are innumerable ways to do it, it is just that effecting change is boring while cribbing about it is romantic.

Requirements

We had an admin interface from which people could download an Excel report. One day, we got a mail saying that the report format is Excel version so and so and it does not work with new Excel versions. The scramble began to find out which version of Excel was used, which version our app produced, how we can upgrade the library the app uses to create Excel documents to the latest version etc. For people not in the know, it is a pain to programatically work with any Microsoft office format in Java, you invariably feel like pulling your hairs out. In the middle of all this madness, I asked a simple question, instead of creating the document in excel format, why do not we create it as CSV? The report consumers were neutral, they were like we do not care as long as we can open it in excel.

As programmers, a lot of times, we blindly work on product requirements without questioning the intention behind them. As it was in our case, the person who gave the Excel requirement did not have any idea as to the difference in the amount of work one would have to put in to create an Excel report versus a CSV, nor the technical debt of the two approaches. From his perspective, it was all the same.

When a requirement comes to you, take a step back and go through them, see if there are chances to simplify. If some of the requirements sound absurd to you, tell it to the stake holder, if some of them might take a long time to implement, talk to him about this. The person providing the requirement has absolutely no idea that some of these features might be technically difficult to pull off or take a long time to implement or there is a better alternative that might not meet the criteria strictly but will not take eons to implement. Bias towards action is good, but a trigger happy one hurts everyone.

Designing for failure

In the world of software, failure is a certainty. Servers go kaput, databases go down, processes go out of memory, things break all the time. You can categorize software as good or bad based on how they behave in these adverse scenarios. I am not trying to imply that software has to be resilient to all these, on the other hand, I believe that it is perfectly fine to crap out when shit hits the fan. The question is how do you handle this crapping out.

Whenever architecting components, devote ample amount of time to failure scenarios. Let us take the case of a piece of code which interacts with an external, third party API. What are the questions you should be asking when designing this component? What happens if the API suddenly stops responding one day? Can I hold my main thread hostage to the API response time? What happens if the API takes eons to respond? In case there is an exception, am I logging enough data to debug? If there are performance issues, do I have enough diagnostic data? Diagnostic data might be in terms of graphing the API response time, no of times the code path was executed, etc. Do I need to send out an alert when something goes wrong? All these question revolve around failure handling. These questions should be second nature to you as a software engineer.

I have seen a tendency among developers to devote inordinate amount of time in making their code adhere to the latest programming fad, trying to use the best possible library etc, but not to failure scenarios. Logging data might not be as sexy as debating which design pattern to use, but once things break, logs are your only friend. Next time when you are furiously pounding on the key board, take a step back and ask these questions. In the future, the developer who maintains the code that you wrote today, will thank you for doing this.

Poor cannot eat roads

Rahul Gandhi allegedly made this statement. It is sad that an armchair, untrained economist like me understands the significance of good roads while someone who is poised to lead India does not. Check out this vice documentary on truckers in West Africa to get the connection between roads and economy. Jim Rogers, author of the excellent book Investment Biker also alludes to how Africa is rendered poor due to bad road connectivity. In spite of so much evidence suggesting good road connectivity being essential for a healthy vibrant economy, Rahul Gandhi makes this asinine statement and our so called independent media is busy discussing his dimples and his charm working up pubescent teenage girls.

A road is something that transcends socio economic, caste and religious boundaries. A road does not discriminate between a rich man driving his BMW or a poor milk seller riding his bicycle or a pandit riding his Luna or a moulvi riding his scooter. I do not for a second doubt the ingenuity of our so called secular politicians to come up with statement on the lines of Minorities have the first right to our roads, but until that happens, for all purposes, we can take rest in the fact that a road is a great unifier. 

How do good roads benefit the poor? Good connectivity makes the transportation of goods efficient there by negating the cost that would be introduced due to transportation inefficiencies. This is an indirect benefit that is enjoyed by everyone not just the poor, but let us take a specific case of how good roads will make the life of a poor auto driver better. If an auto driver drives his auto on pot holed roads all day long, imagine the toll it takes on his auto. This will directly reflect in the efficiency of the auto as well as the money he has to spend on maintenance, not to mention the umpteenth lost opportunity to make more money by ferrying more customers due to the reduced travel time. All in all, an auto driver has to gain a lot with smooth roads. Instead of working on these, our government is hell bent on extending the economic black hole of NREGA to urban poor and our media is a silent spectator to this theater of absurd.

 

Moving up the value chain

In the gym that I go to, there is a guy whose job is to sanitize(spray, clean, rub etc) the equipment after each person uses them. I always pity him and think how boring his job is. How can this person move up the value chain?

The other day, people at the gym were opening up the machines and lubricating and tuning them. This guy was taking a keen interest in the job and helping them out. This sparked a wave of thoughts in my mind where I put myself in his shoes and thought how I could move up the value chain if I were him. I would learn how to tune and lubricate these machines properly. If the norm is to do this activity once a month, I would volunteer to do this every week. Everyone wins, the machines are smoother to be used by the members on a regular basis and I get a good opportunity to hone by skills. Once I am adept at doing this, I would go to the other gyms and offer my services to them. There are a lot many gyms in Indiranagar that I know of and probably some more that I do not know of. I am sure all these people are looking for someone who knows his way around the equipment because such people are usually scarce these days. Once I have a good number of gyms under my belt, I would probably bring more people from my native village and teach them the job so that I can build a team and expand my operations throughout Bangalore. In this way, I am not only bettering my life, but also providing employment opportunities to the people of my village. Once I have Bangalore under my belt, you know the drift, expand more etc.

This is a very hypothetical scenario, but a belief that I very much hold on to, if you try hard enough, there is always a way to move up the value chain.

Consumer tech

I was reading this article about a startup that is launching satellites for imaging the earth. A line from the article stuck with me, the startup is leveraging technology developed for consumer devices like laptops and mobile phones in their satellites. This is a reversal from the olden days where technology that was developed for defense/space would find their way into consumer tech. Now a days, you see technology that was specifically targeted at consumer tech making ways into defense/space. Another specific example that I can think of is Palantir. Ways found out by engineers during their days at Paypal to fight fraud shaped into Palantir which now focuses on hunting down terrorists/hackers based on data like call records, etc.

Another point of note is consumer tech driving futuristic technology. Take google’s driverless cars for example, where is the money coming from? From consumer tech like search. Where did Elon Musk get his money from to bootstrap Tesla motors? Again consumer tech like paypal etc. As of today, I see consumer tech driving research and development more than defense and space which in a way is good for us as consumers because we get a taste of the shiny new thing without having to wait for decades for it to be made mainstream.

Go lang

Last round of the recently concluded stripectf was in Go lang. This gave me a good opportunity to familiarize myself with the language. Even though my native programming language is Java, I have worked professionally in JavaScript, Perl and PHP; dabbled in Python for my personal projects and can manage to read Ruby, Lisp(and it’s dialects), Erlang and Scala with some Google help. When I ruminate on programming languages, I do not see any of these replacing Java as the de facto lingo of the enterprise world but I see the promise in Go lang.

1. Go lang is strongly and statically typed. This means that a lot of mistakes that could potentially cause your code to blow up in production would be caught at compile time. Apart from this, if the language crosses a critical mass of adoption, someone will come up with an IDE that can possibly match Eclipse, IntelliJ, et al. Also, one of the principles behind the language’s design is to aid tooling which means that a lot of tools could possibly crop up which would help to make code more secure, performant etc.

2. Syntax of Go lang is not revolutionary. I consider this a virtue. I am strongly of the opinion that if a language has to gain mass adoption, it’s syntax should be very close to the prevalent languages. Go lang does not deviate much from the Cish syntax but has subtle improvements which improves programmer productivity.

What makes Java a good programming language for the enterprise? Syntax of Java is very close to C which means that you could possibly train a lot of the computer science graduates out there to code in Java. Try Lisp with an average Joe programmer and you will know what I am talking about. With the people supply problem solved let us move on to other factors. Enterprises gravitate towards stability, security and viewing programmers as replaceable components of a machine. Java gives enough constructs to prevent a reasonably sane headed person from shooting themselves in the foot. Static/strong typing, code analysis tools and IDEs go a long way in helping with this. Without tearing your hairs out, try working with a code base in a dynamic language designed by an architect and then handed to a team for coding and then passed on to a testing team and then shifted over to an offshore team for maintenance. And to top it all, you do not have Eclipse or IntelliJ to help you with this mess.

Go lang has all the pluses of Java minus the verbosity. Of course it has a lot of other features which you can read about in the website.

Startup test

Where does your startup stand on the below?

  • Do you have version control?
  • Do you back up data(server data, db data, etc)?
  • Can you build using a single command or prosaically, do you have build scripts?
  • Can you deploy code to production in a single click/command?
  • Do you have different staging environments like dev, qa, pre production etc?
  • Do you have a script to bootstrap the database so that any one can run the app locally?
  • Are your configuration files in version control?
  • Do you use a system provisioning tool?
  • Do you use a database versioning tool?
  • Are your system and business metrics graphed?
  • Do you have alerting systems in place when your business or system metrics go out of whack?
  • Do you have a document listing all the ips/machines in use, external software services used along with their user name and password?
  • Do you have unit and integration testing suites?
  • Do you use a task/project management tool?
  • Does your team swear by code review?
  • Suckers for simplicity

    Let me lay it out straight in the beginning, we Indians are big time suckers for simplicity in others. It is not that we ourselves aspire to be simple but attribute unnecessary importance to it in others. Just that virtue is enough for us to weave magical stories around a person and see him as a messiah who will emancipate us from all evils. In this child like wide eyed innocent chase of simplicity, we forget the qualities that are really necessary to do someone’s job.

    Case in point, Arvind Kejriwal and AAP. Folklores were weaved around AK’s simplicity and his party’s candidates, on a daily basis we were bombarded with their spartan lifestyle stories. How come it occurred to no one to ask the questions that really mattered? Is being a simpleton really enough to be an effective administrator? How well did these people perform in their previous jobs? What extraordinary achievements did they unlock in their past endeavors? Do they have the skills to govern and deliver? Instead of asking these pertinent questions we were wallowing in their hallowed simplicity.

    A simpleton(aam admi) should not try to be a leader. Leadership is not for the faint hearted, it is a game of thrones. A leader is someone who tries to move a collective group of people towards a single goal and when there is a group of people there will always be mismatches, heart burns and politics. An effective leader should know how to deal with this and make the right calls most of the time, an aam admi is not suited for this role. If someone plays this role well, he is not an aam admi because not everyone is born a leader nor has the leadership skills.

    Yes, leading a simplistic lifestyle trumps a hedonistic one governed by avarice any day but put the full stop there, it does not translate into one being an able administrator or being at the top of one’s job. I would any day prefer a Ferrari driving, Armani wearing, Gucci slipping chief minister who has a vision and track record than a slipper wearing one who has absolutely no idea how to govern and deliver.

    Follow

    Get every new post delivered to your Inbox.

    Join 422 other followers