Void

Category: Uncategorized

Shoe Dog

nike

Phil Knight, founder of Nike, pens down the early days of Nike in this part memoir, part biography and part business book “Shoe Dog”. The book starts with Phil Knight’s idea to sell Japanese made sneakers in USA, his business school thesis supporting the same, his journey to Japan to ink the deal and the subsequent world travel post which he dedicates his life to building Nike and the struggles there after.

In today’s interconnected world where communication is instant and automation is the norm, reading about running a business when these were non existent is fascinating. Nike is closer to today’s startup than to any old school business as they stressed on growth and expansion than on security and capital accumulation. His anecdotes regarding dearth of capital, constant tussle with the bank on rapid expansion, the bank asking him to scale down his ambition and focus on capital accumulation gives us insights on the business mentality in the yesteryears.

Throughout the book, Phil Knight talks about his hands off management style. He styles his management on the below quote from Antoine de Saint-Exupery.

If you want to build a ship, don’t drum up the men to gather wood, divide the work and give orders. Instead, teach them to yearn for the vast and endless sea.

Profile of the Nike founding team is far fetched from the general imagery of traditional old school businessmen. They seem to be misfits who found a home in Nike, at least Phil Knight thinks so. No wonder the whole company rallied around doing iconoclastic things that many business would not even think of.

All in all, the book is a great read. I was hoping to read more about Nike’s marketing strategy and how it evolved, but there is hardly any mention of this. Probably for another book I guess.

Reading Shoe Dog brought back fond memories of reading “Made in Japan” years back.

Advertisements

Unit test – purist versus practical

building-blocks-615239_640

Whenever you ask a question on unit testing in a forum there is always that one person whose only job is to point out that what you are doing is not unit testing but integration testing. It is important to know this difference but it is more important to not lose sight of the goal. Also, you need to adopt a terminology that works for you and your team rather than what purists think or say.

In absolute terms, if a test depends on anything that is not in your control, then it is not a unit test. For example, if a method that you are testing uses a public function or a function from an included library or a database or an external API, then it is not a unit test but an integration test. For a test to qualify as a unit test, you need to mock all these dependencies and get them under control, only then you can claim your test as unit test. Now that we have the purists happy, let us move to a more practical world view.

When a regular joe developer refers to a test as unit test, what she means is she is trying to test a functionality in a massive gnarly application that she thinks is a small independent unit. This unit might have some components that is not under her control. A regression test would test how these units behave when interconnected. Instead of debating whether what she is doing is unit testing, a better discussion in my opinion is trying to figure out what is the intention of the test and what needs to be controlled and not. Figuring this out and working on achieving this will add more value than debating whether something is a unit test or not.

Fighting change

Ninja Fighter Sword

In my new workplace, I was given a brand new shiny MacBook Pro. My first reaction was to ask for a Ubuntu laptop. My brain justified by giving several reasons, it is developer friendly, softwares that you run on a server can be run as is in ubuntu, etc etc. I was almost about to voice this opinion, but system 2 took over from system 1. It started asking questions along the lines of is this really the reason you want a Ubuntu machine or are you just trying to avoid the unfamiliar. You have not used a Mac before and are you trying to run away from something new? The two debated for sometime and settled on giving the Mac a chance. So far, the experience has been wonderful and I am learning some cool things like using gestures for different actions etc.

Whenever something new and unfamiliar comes across us, the first reaction for most of us is to fight against it. Take a step back, analyse whether this is the primal part of the brain trying to fight against the unfamiliar or you really have a valid reason not to.

Altruism FTW

Have you observed the way Google maps asks for info about local joints and places? They word it in such a manner that it sounds like you are helping others to make an informed decision along the lines of “Give us more info to help others”. What they are doing in effect is appealing to the altruism in all of us to generate more info to make their product better.

I think this is a great way to ask for more data in this world of user generated content. Instead of asking to review a restaurant how about wording it as “Review this place so that others can discover great food”. Instead of asking people to rate your app how about saying “Help your friends discover the app on playstore, rate us”. It would be interesting to A/B test this and see the result.

A little extra effort

I was sauntering on Church Street and came across a used book store. It had been long since I had been to a physical book store, hence ventured in. I started browsing around. I have been wanting to read Shoe Dog for quite sometime and hence asked the proprietor did he have a used copy of that. He answered in the negative and got back to doing what he was doing before. I continued my aimless browsing and got out.

Down the road, there was another used book store. Again got in and repeated the question. The person searched, said no, but he put a new copy of the book in my hand and said he can give me a 20% discount. I am sure he knew before hand that he did not have a used copy of the book, searching was just a ruse. I made the purchase.

We tend to neglect the impact of putting that little extra in. Sometimes all it takes is a little extra effort for a great outcome.

Selfie

Let us say that you want to execute a job periodically, what comes to your mind first? If you are familiar with Linux, I can hear your screaming cron. Well, no doubt about that, cron is great, but what if I told you there is another approach which you can take to execute periodic jobs? Our good old continuous integration server Jenkins can supplant cron as a tool to execute periodic jobs and it kicks ass in doing so.

What makes Jenkins such a gem for executing periodic jobs?

1. You get a great web front end which is comfortably accessible from the browser.

2. The front end gives you a complete history of the previous runs with detailed information like when did the last execution occur, how long it took, what was the output during this execution, historic trend of the execution time and other diagnostic information.

buildHistory

buildTimeTrend

3. You can leverage the Jenkins plugin eco system and do some nifty things. For example, you can use log parser plugin to parse the execution output and alert if a specific format is found in the output. The great part here is that your job need not have alerting logic baked in, your job concentrates on doing what it does best, let Jenkins take care of the rest.

4. You can configure regular Jenkins build rules like alerting on execution failure, preventing subsequent executions if the current one fails, etc.

5. You can chain multiple jobs and the chain is very obvious thanks to the great Jenkins UI.

All this is great, but one problem I faced with Jenkins is that you cannot have a Job call itself recursively with a delay in between, you have to schedule the job execution using cron expression. The difference is subtle, but there are implications of this limitation which I will expound with an example. Let us say that I have a job which ideally should run every 15 minutes, but sometimes this job execution takes more than 15 minutes to complete, in that case what happens is, job executions queue up and fire successively one after the other. The way I want this scenario to pan out is, once the execution finishes, it should wait for 15 minutes before the next execution starts. I could not find a way to do this in Jenkins and hence selfie was born.

Selfie is a Jenkins build trigger plugin which lets a project to trigger itself after a configured delay. The plugin appears as one of the build triggers while configuring a new project.

selfie

This is my first attempt at writing a Jenkins plugin, pull requests and code reviews are more than welcome.

SQS versus Kinesis

There is some confusion around SQS versus Kinesis, both are QAAS™(queue as a service) provided by AWS(this statement is not entirely true, you will know why as you read on). This is an attempt at defogging this confusion.

SQS is a queue in the old fashion sense, it promises at least once delivery. You create a queue, enqueue items, dequeue items. One point to note is that, dequeueing an item does not delete that item from the queue, you have to explicitly delete the item post dequeue, this sort of goes against the intuitive understanding of dequeue, at least for me, but, you can configure that once you dequeue an item, that item should not be available for dequeueing again for a specified period of time.

Kinesis is a queue but it is not a queue and it is not a paradox, it is a high throughput stream handler. A conceptual way to think of Kinesis is like one huge log file, items that you enqueue as lines in this log file. You have a pointer to this log file, when you read one line from this log file(dequeue), the pointer points to the next line. Kinesis is stateless, as in, it does not maintain the pointer for you, it is upto you to maintain this. What this means is that, say you are reading off a Kinesis stream and your process goes down, when you bring it up again, it will start over the processing from where it started originally, not from the last line before crash. There is no concept of taking items out of Kinesis, the data is always there(retention period of a day), you manipulate the pointer to this data. Hence, if you want to re process the stream, you can replay. AWS provides a client library for Kinesis which maintains the state for you. This client library uses dynamodb to persist the state.

This should give you a fair idea of when to use Kinesis and when to opt in for SQS.

Release early, release often

Releasing early and often can make the difference between life and death for new age internet companies. Most of the successful dotcoms like Amazon, Google, Etsy etc do hundreds of deployments per day. If you are a small organization, your magnitude and frequency of deployments might not rival these big organizations, but it is always a good idea to release early and often.

If you plan to carry out multiple deployments a day, it is critical that you do not have downtime during these deployments. When your shiny new code is getting deployed, your end users should be able to access the site. This is usually done by routing all your traffic through a load balancer. The end user does not directly hit your application server, he/she hits the load balancer and it is the load balancer’s responsibility to route traffic to your application servers. In addition to this being the recommended architecture for server side software, it gives you the agility to deploy code without having to worry about down time. You take a server out of the load balancer, deploy your code on it, add it back to the load balancer and do the same with the other servers that are part of the group. Two of the most popular load balancers out there, Nginx and HAProxy, allow you to do this dynamically(while the load balancer is up and running you can add and remove back end servers) and I am sure that other load balancers let you too. If you are running on AWS, Elastic Load Balancer lets you do this.

Also, your deployments should be as simple as possible, even a monkey should be able to deploy your code to production. More the complicated it is to deploy software, less the enthusiasm of developers to do it. Using a continuous integration tool like Jenkins helps to make this as painless as possible.

Enable a kill switch for all new features. Your app should have the ability to turn on and off features in seconds. This gives you the power to turn off a feature if you detect any problems with it in the early days.

Also, gradually releasing features is a good idea. This lets you whet out performance and other issues on a small scale before it becomes a site wide issue. Release to 1% of your users and then slowly ramp up to 100% keeping a close eye on the feature all the time.

If you are working on a feature, you do not have to wait for feature completion to release. As and when you finish off logical steps, keep releasing. Your end users might not see the feature yet but this helps you to get away from the situation of one big bang release and everything going down. These incremental releases help to detect bugs early and build confidence for the final release.

Set alerts for all your key performance and business metrics. Post deployment, if any of these metrics go awry, you get an alert and you can set things right. In addition to alerting, having the ability to graph these is tremendous. Post a deployment, you can check your dashboard to see whether the deployment has had an adverse affect on your response times, key business metrics, etc. This adds to your confidence to do multiple deployments without having to worry about your new code degrading performance or adversely affecting business.

These are some of the simple tips that help you deploy code to production on a regular basis. I have not touched on more advanced topics like automated testing, integration testing, automated provisioning etc.

The Expectation Test

I got a phone lying down on road. Since I could not unlock the phone, I waited for the owner of the device to call. A couple of hours later, he did call and in his opening sentence started pleading to return it back. Even though he was the rightful owner of the phone, he expected me to never return it back to him.

I was stuck in a traffic jam caused due to BWSSB(Bangalore Water Supply and Sewerage Board) closing a very small section of the road to fix a sewer. This was on an afternoon on a main road in Bangalore and no wonder, there was a traffic pile up for close to 3 kilometers. I cursed myself for taking that road instead of some other alternate route. I did mentally spew venom at the people digging the road, but it never occurred to me to hold the authorities accountable for this and expecting better from them.

The curious case in both the incidents is the expectation of the people involved in them. Even though the right thing to do when you find something that is not yours is to return it back to the owner, people nowadays expect that not to happen. It is the responsibility of the government to execute it’s task in such a manner that it causes least annoyance to the citizens but we no longer expect that from our administration. If things do happen the right way, we are mesmerized and think of ourselves as lucky, it does not occur to us that this is how it should be. This is not a good sign because when expectations hit rock bottom, it becomes the norm and from there it is a vicious downward spiral.

The same is true for organizations as well. Holding people to higher expectation and making them accountable for it is the key to success. If in an organization, you are happy that your team members are not taking impromptu leaves, there is something really rotten. There is nothing stupendous about people not taking unplanned leaves, this should be the norm and if someone following this norm is an aberration in your organization, then this a warning signal to you that there is something deeply wrong which needs immediate fixing.

A test for any organization to see whether it is on the path to greatness would be the expectation test. Mentally go through your expectations from employees/company and see whether these are really higher expectations or something that should have been the norm. If these are expectations that breach the bar, well and good, if not, do a reality check and fix them.

We are a startup

Being part of a startup is not an excuse for you to shoot first and ask questions later. “We are a startup” is the most common phrase people spout when you ask them about their sloppy processes or development practices. Being a startup is not an excuse for you to have zero process or letting your developers be trigger happy. It is also not an excuse for condoning sloppy employee behavior like taking leave without adequate notice or working from home on a short notice.

I have heard of stories in startups where everyone writes components in a language of their own choosing. If today’s fad is Node, then the developer uses Node and if tomorrow hacker news has a post touting the next greatest language, the developer switches to that. We are a startup, we do not have a QA and pre production environment for testing, we test code directly in production. We are only a couple of people, hence we do not need version control. Since I am a single guy who handles servers, configuration files are not in version control, my brain is my GIT repository.

A start up is an opportunity for you to move fast without being burdened by the bureaucracies/politics of big organizations. Say for example, if you find an employee kicking ass, as a manager in a start up you can give him a raise by having a quick chat with the CEO. In a big organization, you have to make a case for that, present it to a committee, justify why someone should be given an out of turn raise etc etc. It is also an opportunity for you to trust people and call them out when you see something odd without processizing it. For example, if you see someone working from home on a very regular basis and it does not work for you, instead of floating a blanket rule saying work from home is prohibited there by inconveniencing other employees who use it judiciously, you talk to the person and see what is the problem that person is facing and try to solve it with him. This is very difficult to pull off in a big organization that has thousands of employees but highly doable in a startup.

A startup is an opportunity for you to create an atmosphere based on trust rather than draconian processes. Let us take technology choice for example. Big organizations have lots of rules and regulations on what technology can be used, what cannot be used, how to buy licenses etc etc. This is made with the central assertion that individuals are not responsible enough, so let us decide by committee. In a startup, you do not need to introduce a process for this, you can trust your employees to make the right technology choices as long as they do not go bonkers with it. If something goes wrong with a technology choice, you can talk to that person and figure out where he went wrong instead of mandating a policy for all employees from then on. Individual responsibility trumps rules any day, it is difficult to implement and follow this in a large company but can be easily pulled off in a small organization.

Next time when you use the phrase “We are a startup”, think whether you are trying to mask your inefficiencies in the guise of being a start up.