“Intentionally” Leaky Abstractions

Software abstractions that developers create should leak enough to let non-developers design the right product experience.

Eons back in internet history, Joel Spolsky wrote a post on leaky abstractions. The gist of the writing is that even though we create software abstractions to make life easy, they somehow unintentionally leak through.

With technology getting a lot more complicated, I believe that the software abstractions that developers create should intentionally leak to some extent to let non-developers design the right product experience.


Taking a concrete example, let us say that you are creating a search utility which goes over a lot of data. Your algorithm has two steps, one that works on recent data and the other, older data. Recent search is blazing fast while searching through old data is slow. You do both in parallel, combine the results and display it to the user. You have two options here; you can abstract this two-pronged approach from the product teams and act as if the search is a single step process or you can leak this abstraction so that they can take this into account while designing the product.

What can the product team do with this information?
They have multiple options.

Show the faster results first without waiting for the slower ones. Where the older results are displayed, show a loading placeholder, and then populate the results as and when they become available.

Another option is to make this a two-step process by showing only the recent results and then asking the users explicitly whether they want to search through old data too.

They can do this only if your abstraction leaks through informing them of the internal search nuances.

You need to be judicious with what you expose, and what not, so that the product teams are not burdened with unnecessary details – striking the right balance is essential.

Image by Jesse Bridgewater from Pixabay


Amara’s law says – “We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.”

I see a lot of shared two-wheelers on Bangalore roads these days. The expectations of the companies which buy these for shared mobility would be very different from individual buyers of two-wheelers. As of today, I do not know of any company designing vehicles keeping shared mobility in mind; but I am sure it is not far away. Someone is going to do a deep re-think on this and come up with a design that is amenable to mass public ride sharing.


Samsung opened a large experience center in Brigade road, a high street shopping destination in Bangalore. MI has a similar one in Indiranagar, another bustling area of Bangalore. The emphasis in these places is not on selling but letting consumers experience the product. These centers would roll up to marketing functions than to sales. If you think about it, sales and logistics are no longer the thorny problems they used to be before; technology has taken the pains out of these to a large extent. I believe physical retail is going to go through a deep re-think where it will be more about product experience than selling.

Boston Dynamics has been releasing jaw-dropping videos of their robots in action. These robots seem to be getting better by the day. We are not far from a time when a lot of the hard and mundane physical labor will be carried out by these automated agents than humans. But none of our current products have been designed keeping these automated agents in mind. For example, if you want an army of drones to fix and replace the street lights, the light assembly needs to be designed in a way that is drone friendly. As of today, it is not; these have been designed keeping in mind a human carrying out these maintenance tasks. The same applies to almost everything that we see around. Someone is going to start re-thinking on these designs and changing them to be more automated agent friendly.

I firmly believe the trifecta of technology, automation and sharing economy is going to bring about a massive re-think in how we look at the design and assembly of almost everything.

Photo by Tincho Franco on Unsplash

Competitive Advantage

Are you focussing enough on developer productivity? If not, read on why you should.

Technology is getting commoditized these days. In the yesteryears, you could have counted on your software frameworks and technologies, hardware prowess, access to capital and talent pool as significant competitive advantages. In a world, which is getting commoditized day by day, the access to these has been democratized, thus leveling the playing field. Today, it does not matter whether you are a single person startup or a large multinational corporation; everyone has access to almost the same resources.


Cloud providers ensure you have enough servers and hardware at your disposal to meet your growing demands. Open source frameworks and technologies provide the required technological backbone to build stuff. Umpteen services exist to recruit talent and develop products; co-working spaces take care of your physical office requirement at an amenable cost; social media gives you access to a broad marketing audience. In spite of these, if you are short on capital or want to splurge, there is the ever-growing VC industry to take care of your capital needs. The icing on the cake is internet and sites like Stackoverflow and Coursera which have levelled the playing field when it comes to access to knowledge.

In light of this, what constitutes your business competitive advantage? One short answer, developer productivity.

Developer productivity dictates how well you can effectively and efficiently leverage all the resources at your disposal to build a great product and trounce your rivals. Developer productivity is a trifecta of having the right engineering tools, structure and processes. When you excel in these three, developers can move fast, and business can conquer markets with ease.

Developer productivity is too vast a topic to be covered in one post. I will be writing more on this in the coming months; it is an area which is extremely close to my heart, and I firmly believe companies do not focus enough on this.

Photo by Erwan Hesry on Unsplash

Daily Standup

Are you confused as to whether to do a daily standup or not for your team? Read on why you should.

Daily standups are a hotly contested and controversial topic in the software industry. There are some who actively champion it, and then there are others who detest it. I belong to the former camp, and firmly believe every team should do a daily standup.

One of the primary problems that teams face is information asymmetry. Some members tend not to know what their teammates are up to; this results in individuals becoming islands and a breakdown in efficiency. I can write an entire post on the problems this causes but for some other day. One of the reasons for this is not everyone in the team communicates at the same level; all teams face communication asymmetry.


There is a vast diversity in humans when it comes to communication. Some excel in it; some do not. Efficient and timely communication can make or break teams and are a huge indicator of a team’s success. Standups force everyone to communicate. You do not have the option of not communicating during a standup.

A format I have seen work is going around with each one telling:
1. What they did yesterday.
2. What they plan to do today.
3. Any blockers they are facing which needs immediate attention.

This ensures everyone in the team knows what their teammates are working on.

Apart from information proliferation, this also helps in surfacing blockers. A lot many people do not know when to surface blockers, and they wait for things to go down before yelling for help. Daily standup gives everyone a forum to do it before shit hits the fan. It also allows others in the team to intervene and course correct when needed.

As with any meeting, it is paramount to keep daily standups short and sweet. Do not allow anyone to hijack the daily standup for lengthy or off-track discussions. Do it outside the standup.

One downside of standup is that people who communicate well and efficiently feel penalized and do not see the worth of it. I was in this camp when I was new to the industry, but as I have built teams and scaled organizations, I have become an acolyte of daily standups. If it is of any solace, think of it like taking one for the team and the organization.

Photo by Margarida CSilva on Unsplash

All In

As a young fledgling startup, you should not be building layers of abstraction over the technology that you are using; on the contrary, you should be going all in.

I have talked to multiple startups doing this. The intention is to future proof the product; this should be the least of your concerns during the early days. In your gestation period, you should be spending your waking hours building features to acquire customers and grow. That is what will make or break your startup, not the abstraction layers you create for the distant future.


If you are using MySQL, then go all in on it. MySQL has a lot of features that are above and beyond the ANSI SQL standards. Do not try to be standard compliant and avoid the esoteric non-standard features; try to leverage them as much as possible. It will take you faster to market at the cost of being locked in on MySQL for the foreseeable future. If your startup survives to see the day when MySQL becomes the bottleneck, you will have enough resources and expertise to make the switch. Same applies to cloud providers. Do not try to be cloud agnostic; utilize the features given by your cloud provider to the maximum.

This might sound orthogonal to some of the advice that you might get on the internet as well as sound engineering practices. A lot of these are written for mature companies with established product and revenues, not for young, early-stage startups. As with everything, it is all about context.

If you think about this, it boils down to a simple decision; whether you live in the present and try to maximize for today and the immediate future or plan out for the distant, uncertain future. A young startup, trying to make its mark on the world, should do the former.

This is a bit counter-intuitive and becomes challenging to do if you have a mindset that is more attuned towards a systematic way of thinking and doing. A lot of things with startups are counter-intuitive and go against the grain. If not, would it not be a cake walk?

Photo by Nancho from Pexels.

Make It Small

“Make it large” is the mantra these days, but when it comes to releasing software, think the opposite, make it small. The secret to a successful release is to break it into numerous small deployments; this serves a dual purpose, minimizes risk as well as gives you enough leeway to fix bugs before it negatively impacts end users.

Never wait for a feature to be complete for deployment. Break it into multiple smaller chunks and keep deploying these to production even though no one will be using it. Smaller deployments always win over one big bang deployment. Longer the code runs in production, more data points you have to analyze and take corrective steps.


If you can test with real users without them knowing, there is nothing like it. If you put your mind to it, you will always figure out a way to do this. Taking an example, let us say you are building a chat feature for your application. You can stealthily release it without informing your users and send dummy messages on their behalf. The user does not see any chat related functionalities, but behind the scene, you create dummy messages and send them to one another. Apparently, this is how Facebook tested their chat before launch; this allows you to check your feature realistically in ways it will be ultimately used.

Another trick is to log the action without actually carrying out the intended side effect. Say for example; you are introducing an API rate limiter. As a first step, start logging whenever an API call hits the rate limit threshold. During this phase do not block the calls, the idea is to emulate the result; this lets you test out the feature with real traffic and fix any bugs before it starts negatively affecting your API consumers. Once you are confident of your approach, you can roll out the feature where you block offending callers.

Phased rollouts is a life saver. Release features to a small set of users, keenly observe your application and gradually ramp up to 100%; this minimizes the impact of bugs and lets you catch them early.

Another lifesaver is feature flags. The idea behind a feature flag is simple; you put all your features behind an on-off flag which you can toggle if and when needed. The ability to turn off features during their early lives gives you a lot of flexibility and room to maneuver and improvise.

If you are working on refactoring or replacing an infrastructure component, never do a stop the world cut off. Always run both the systems in parallel and only once you are confident of the new approach, do the switch over. Say, for example, you are moving your data source from MySQL to Mongo, start by writing to both the datastores in parallel. Then change your application to work with Mongo while still keeping the option to fall back to MySQL if needed. Put this behind a flag which you can toggle. Only once you are confident of everything working with Mongo, pull the plug on MySQL.

The common thread that holds all these strategies together is to plan your deployments such that they are small and gradual; if something goes wrong, it is not catastrophic. You increase the scope gradually as you observe and gain confidence.

As they say – Risk is what’s left over when you think you’ve thought of everything. A bit of planning goes a long way in reducing risk as well as giving an excellent experience to your users.

Photo by Magda Ehlers from Pexels

Knowing Versus Doing

Over-engineering is ripe in the software industry; this mainly manifests in three ways:
1. Needlessly complicated design.
2. Lift and shift engineering – Picking up technology and practices from other places without understanding the context in which it was developed.
3. Being trendy – Using frameworks and languages that are currently in fashion irrespective of whether one needs it or not.

I have written about this before.

Please do not take this prose as an argument for being sloppy and building crappy products.


I firmly believe one of the reasons why this occurs is that people confuse knowing with doing. You being aware of something should not force you into using or implementing it. You might have a personal inclination towards a language or a framework; you might even believe that it is the best in the world, but that does not imply you rewrite your currently well-working production application in the said technology. You come across something new, and shiny does not mean it has to be part of your application stack.

When we see something novel and trendy, our brain actively tries to figure out ways in which we can make it a part of our lives.

Another arena where this plays out is in doing things if and when needed. Because you read a blog post touting Redis as the next best thing to sliced bread should not make you go about slapping Redis as a cache in front of your data stores. Your data size might be tiny that it entirely fits into main memory. Achieving five nines of reliability is a daunting task which takes a lot of engineering and effort. As a geek, it is a fascinating problem to work on, but your application may not need it. Just because you are aware of what others are doing to achieve this does not imply you too embark on this path.

If you ponder over this, you will realize this behavior is not only restricted to work but plays out in our personal lives too. When we come across something new and exciting in any domain of life, we try to adopt it irrespective of whether we truly need it or not, and when we do it, we go way out of line in justifying its usefulness to ourselves.

Open Source and Revenue

This is the second part in a series on open source software. In the first part, we examined why equating open source with “just” free is fool’s errand. In this post, we will explore the different avenues for revenue from open source software.


Get essays about software development, management, productivity, and more by subscribing to the blog

Join 15 other followers

The first one is pretty straight forward – charge for support, maintenance, consulting, and custom development. Software takes an effort to understand and maintain. Either you can do it in-house or outsource it to an external firm. Big enterprises have specific needs which require custom enhancements. They also need consistent support and suggestions. There are a lot of companies which have used this model to generate revenue from open source software like Redhat, Percona, etc.

The SAAS option for open source software has gained immense traction in the last decade or so especially since the advent of cloud. Instead of you taking the pain to host and maintain, the company behind the software deploys and manages it for a recurring fee. Most of the popular open source software is available under this model nowadays. WordPress, MongoDB, ElasticSearch are some prime examples of this strategy.

Another revenue strategy is the open core model. The core is open and free, but features which enterprises need like security, high availability and user management are part of the commercial offering. For example, the core database could be open, but clustering and high availability might be available only in the retail version. InfluxDB uses this model.

Then there is the licensing play. Software licensing is nuanced and comes with a lot of baggage and restrictions. The open source version of the software is released under a restrictive and commercial antagonistic license. If you want to use the software in a business setting, you have the option of buying a more permissive and commercial friendly license; this is very prevalent in software bundled in commercial products.

It is not uncommon for a company to use a mixture of the above strategies.

In the next part of the series, we will go through some recent developments in the open source world in an attempt to ward off the threat from big cloud providers like AWS.

Image Credit: lucas Favre

Open Source != Free

This is the first post in a series on open source software. You can read the second post here.

One of the most common conflations I see people making is mistaking open source software for free software; both are not the same. Being free is just icing on the cake, the more significant advantage is the freedom and flexibility that comes with open source software.

Let us say you are an enterprise with millions of dollars that has built its entire product on a closed source database. Your business is now profoundly entwined with the success of the database company. What happens if the database company goes kaput and shuts down? You now have to replace the database. Depending on the complexity of the product and the business, this might take significant effort and end up derailing your entire business. Open source software severely mitigates this problem.



Get essays about software development, management, productivity, and more by subscribing to the blog

Join 15 other followers

There is no concept of shutting down in open source world. Open source software development is inherently decentralized. In a lot of cases, committees govern open source software development. These committees have many stakeholders whose best interest is in keeping the software alive. Apart from this, many boutique firms provide development and maintenance services. All this leads to a robust eco-system that prevents a project from abruptly shutting down and taking you hostage.

Commercial closed source software reminds me of a famous line from the Eagles song Hotel California – You can check out any time you like, but you can never leave. Once you are locked into a piece of software, it is not easy to get out. During pricing discussion, there is an asymmetry. As a locked in customer, it is difficult for you to leave, there is no BATNA. Open source software does not have this problem.

Having access to source code is a huge advantage. When I was building Kwery, I used Apache Derby as the database. I started seeing a weird bug in Kwery which lead me to tinker with the Apache Derby source and finally unearthing a bug in the Derby database. A couple of mail exchanges on the Derby mailing list confirmed this. If I did not have access to the source code, there would be no way for me to figure this out.

I am not saying that open source software is a panacea to all problems and that you should completely shun commercial closed source software. Each has its place but equating opensource software with only free is folly.

You can read the second post here.

Image credit: Photo by Adailton Batista from Pexels

The Source

I think the minute you have a backup plan, you’ve admitted you’re not going to succeed.

Elon Musk said so. Chew on this for sometime before you read the rest.


I was not honest with the above. It was not Musk who made the statement but Elizabeth Holmes, at the peak of her popularity. It has been a hard fall for Elizabeth from then to now; she is now accused of fraud.

Did your opinion of the quote change with the source? Did you go from awe to retching?

I believe we give as much importance to the source of a quote as to the quote itself. We should be internalizing quotes and aphorisms by divorcing the source. A quote should be evaluated solely on its content, not who it came from. When we do not do this, the aura of the person shadows the import of the saying, reducing it to personality worship. The significance of the quote tends to get lost.

The act of viewing a quote objectively also acts like a shit umbrella against famous people getting away with baloney like the above from Holmes. If a successful person makes a statement, we take it at face value thinking if she says it, it must be true. We should guard ourselves against this attitude.

Internalizing a quote just on its content is not easy to do as we all love narrative fallacy, but it is worth trying. As with everything, we get better at it with practice.

Image credit: Ilyass SEDDOUG