Void

Category: Uncategorized

The Source

I think the minute you have a backup plan, you’ve admitted you’re not going to succeed.

Elon Musk said so. Chew on this for sometime before you read the rest.

ilyass-seddoug-667396-unsplash

I was not honest with the above. It was not Musk who made the statement but Elizabeth Holmes, at the peak of her popularity. It has been a hard fall for Elizabeth from then to now; she is now accused of fraud.

Did your opinion of the quote change with the source? Did you go from awe to retching?

I believe we give as much importance to the source of a quote as to the quote itself. We should be internalizing quotes and aphorisms by divorcing the source. A quote should be evaluated solely on its content, not who it came from. When we do not do this, the aura of the person shadows the import of the saying, reducing it to personality worship. The significance of the quote tends to get lost.

The act of viewing a quote objectively also acts like a shit umbrella against famous people getting away with baloney like the above from Holmes. If a successful person makes a statement, we take it at face value thinking if she says it, it must be true. We should guard ourselves against this attitude.

Internalizing a quote just on its content is not easy to do as we all love narrative fallacy, but it is worth trying. As with everything, we get better at it with practice.

Image credit: Ilyass SEDDOUG

Déjà Vu

You have been trying to solve a problem for quite some time; the solution appears to be elusive. As you grapple more with the problem, a seed of a solution germinates which sprouts into an answer. In hindsight, the resolution appears obvious. You kick yourself telling why did I not think of this sooner?

How many times has the above happened to you?

I believe almost everyone goes through these in life.

diego-ph-249471-unsplash

One simple hack to get better and faster at problem solving is to backtrace through your thinking. Once you successfully arrive at a solution, replay your unsuccessful attempts and figure out what you could have tweaked in your thinking process to arrive at the answer faster.

High performance teams do post-mortem analysis after a critical issue. Members create RCA(root cause analysis) document which contains what went wrong, what could have been done to prevent the untoward incident from occurring and what are the steps to be taken to avoid a relapse. We should be applying the same steps to our thought process when we do not arrive at solutions on time; think of this as an RCA of your thinking process.

This simple trick I believe helps us in getting better and faster at problem-solving.

Image credit: Diego PH

Now You See Me

In the modern software world, where micro-services are de rigueur, observability of systems is paramount. If you do not have a way to observe your application, you are as good as dead.

w-a-t-a-r-i-776085-unsplash

W A T A R I

The first step towards embracing observability is figuring out what to track. Broadly, we can categorize software observability into:
1. Infrastructure metrics.
2. Application metrics.
3. Business metrics.
4. Distributed tracing.
5. Logging.
6. Alerting.

Infrastructure metrics:
Infrastructure metrics boil down to capturing the pulse of the underlying infrastructure where the application is running. Some examples are CPU utilization, memory usage, disc space usage, network ingress, and egress. Infrastructure metrics should give a clear picture as to how well the application is utilizing the hardware it is running on. Infrastructure metrics also aid in capacity planning and scaling.

Application metrics:
Application metrics help in gauging the efficiency of the application; how fast or slow the application is responding and where are the bottlenecks. Some examples of application metrics are the API response time, the number of times a particular API is called, the processing time of a specific segment of code, calls to external services and their latency. Application metrics help in weeding out potential bottlenecks as well as in optimizing the application.

Infrastructure metrics give an overall picture whereas application metrics help in drilling down to the specifics. For example, if the infrastructure metric indicates more than 100% CPU utilization, application metrics help in zeroing in on the cause of this.

Business metrics:
Business metrics are the numbers which are crucial from a functionality point of view. For example, if the piece of code deals with user login and sign-up, some business metrics of interest would be the number of people who sign up, number of people who log in, number of people who log out, the modes of login like social login versus direct. Business metrics help in keeping a pulse on the functionality and diagnosing feature specific breakdowns.

Business metrics should not be confused with business reports. Business metrics serve a very different purpose; they are not to quantify numbers accurately but more to gauge the trend and detect anomalous behavior.

It helps to think of infrastructure, application and business metrics as a hierarchy where you zoom in from one to the other when keeping a tab on the health of the system as well as diagnosing problems. Keeping a check on all three ensures you have hale and hearty application.

Logging:
Logging enables to pinpoint specific errors. The big challenge with logs is making logs easily accessible to all in the organization. Business metrics help in tracking the overall trend and logging helps to zero in on the specific details.

Distributed Tracing:
Distributed tracing ties up all the microservices in the ecosystem and assists to trace a flow end to end, as it moves from one microservice to another. Microservices fail all the time; if distributed tracing is not in place, diagnosing issues which span microservices feels like searching for a needle in a haystack.

Alerts:
If you have infrastructure, application and business metrics in place, you can create alerts which should be triggered when they show abnormal behavior; this pre-empts potential downtimes and business loss. One golden rule for alerts is, if it is an alert, it should be actionable. If not, alerts lose their significance and meaning.

Both commercial, as well as open source software, are available to build observability. NewRelic is one of the primary contenders on the commercial side. StatsD, Prometheus and the ilk dominate the open source spectrum. For log management, Splunk is the clear leader in the commercial space. ELK stack takes the crown on the open source front. Zipkin is an open source reference implementation of distributed tracing. Most of the metrics tracking software have alerting capabilities these days.

If you already have microservices or are moving towards that paradigm, you should be investing heavily on observability. Microservices without observability is a fool’s errand.

Market Size

When I was working on Kwery, a lot of people used to ask me – Why are you not trying to build a team and raise money? The niche market that Kwery served was not amenable to VC money or building a team. Kwery was an experiment in creating a single person lifestyle business.

Market size is defined as – The number of individuals in a certain market who are potential buyers and/or sellers of a product or service.

carnival-carousel-circus-992763

When building any business, gauging the potential market size is the most critical step. The size of your team and the amount of money you want to raise should correlate with the market size. If the intention is to create a big business, you should be going after a big market. A lot of would-be startup founders are fixated on their idea and raising money that they do not analyze the market size or fool themselves into thinking that the market is vast. Peter Thiel, in his book Zero To One, beautifully explains the significance of market size.

The story that you build around your startup should also reflect the market size that you are after. In 2014, professor Aswath Damodaran estimated the value of Uber at ~6 Billion dollars with the story that Uber is an urban car service company. Bill Gurley, the lead investor in Uber, refuted this saying that Uber is not an urban car service company but a logistics company that is going to be everywhere, not just urban areas and is going to exploit global networking benefits, not local networking benefits. With this story, Bill Gurley was valuing Uber at ~53 Billion, a ~9X increase from Aswath Damodaran’s valuation. With a change in the story, the market size that Uber is after went up. This incident illustrates the importance of framing your startup story for big market size.

Some parting words on market size from Kunal Shah, founder of FreeCharge – Market > Product > Team.

Poor Man’s Anomaly Detection

You have a feature where if someone signs up on your product, you create a wallet for that person and top it up with complimentary money. Your organization swears by micro-services; hence sign-up logic is in one service and wallet creation and crediting is in another service. Once a user signs up, sign up service sends a message to the wallet service so that it can create the wallet and do the credit. To ensure the sanctity of the system, you have to make sure that the number of signups, wallets created and credits done match one another. Also, if these go out of sync, alerts need to be in place to take corrective action.

Since the two are disparate distributed systems, one way to achieve the above is to use an anomaly detector. There are off the shelf products for this as well as open source projects. If you do not have the time, need and resources to invest in deploying an anomaly detection system, having a reconciliation system is the way to go.

black-and-white-child-connected-265702

Reconciliation is deeply entrenched in the financial domain where it is a way of life. The technology world can borrow this and use it as a poor man’s anomaly detector. For the scenario that we started with, we run queries on the data repository of the sign-up and wallet systems at regular intervals. These queries fetch the count of sign-ups, wallet creations, and credits that occurred during the period. Once we have the numbers, all we have to do is ensure that they match. One can do this with a simple bash script; this is extremely simple to develop and deploy.

Reconciliation can play a role in all places where two-phase commit flows are involved. For example, most payment flows follow a two-phase commit process. You first deduct money from the user’s payment instrument and then fulfill the commitment. There is a good chance that post payment debit, your system dies not doing the fulfillment. Having a reconciliation system in place helps you to take corrective action in these scenarios.

Reconciliation is a simple way to achieve anomaly detection until you have the need and the resources to invest in a more robust distributed anomaly detector.

Startups and VC La La Land

With daily morning newspapers dedicating pages and front lines to startups and venture capital(VC) investments, they are no longer the niche they used to be. Whenever a startup investment is in the news, I hear questions along the lines of:

  1. Is the VC crazy to invest so much money in that startup?
  2. Why does that startup need massive amounts of money?

In my opinion, these questions stem from not having an understanding of how the VC business works.

Let us say that there is a contest with prize money of 1 billion dollars. You have to form teams, do some tasks; whoever wins, walks away with the prize money. There are a hundred individuals who are eager to participate but do not have the financial means and resources to recruit people to form teams and get to work. There is a wealthy benefactor who is ready to fund these hundred individuals with 10,000 dollars each. He strikes a deal with each of these hundred people saying, if you win, you give me 50% of the prize money; if you lose, you do not have to repay the 10,000 dollars I gave you. The benefactor is sure that one of them is going to bag the prize, she does not know who. The benefactor has invested 1 million dollars in total(10000 * 100) to ensure a return of half a billion dollars, i.e., 500x returns.

The above is a crude explanation of how the VC industry works. The individuals are the startup founders, and the wealthy benefactor is the VC. The only difference is, in the real world, there is no guaranteed success, i.e., there is no way for the VC to tell with cent percent certainty that someone is going to win for sure; it is all about probabilities, data, reading the trend and gut instincts. To bring a semblance of certainty to returns, VC spreads her bet on a wide array of startups in different domains; at least a handful of them is going to hit the jackpot.

dollar-499481_640.jpg

Why does this startup need so much money and are the VCs crazy to give them that?
Startups that VCs invest in are not chasing organic growth. Organic growth is steady, sustainable growth which compounds to big returns over decades. VCs are chasing exponential returns in a few years; not steady growth. The money that VCs give to startups helps them with this exponential growth.

Why so?
Double-digit returns are achievable by investing in public companies and debt. Why take the risk of investing in startups with absolutely no guarantee of returns when there are better ways to get there? For the chance that VCs are taking, the gains have to be comparable, i.e., exponential returns.

Why do startups need so much money?
To short-circuit the whole organic growth curve and achieve exponential growth so that investors get the returns in a relatively short period. The money is spent on scaling teams, building technology and more importantly in acquiring users through discounts and advertising. In short, the VC capital infusion empowers startups to compress multi-decades of growth to a decade or less.

Why is this startup spending money on discounts when they are not even profitable?
Big spending circles back to the concept of exponential growth and building market dominance for a winner take all effect. The idea is that once the startup dominates the market or has enough users, they will eventually figure out a way to make money, dominate the market and turn profitable. Case in point; Amazon, Google, and Facebook. When VCs invested in these firms, none of them were revenue positive, but look at where they are today. Of course, there is a risk of startups never figuring out how to make money, but VC investment is about calculated risks and spreading the bet.

You might have read of startups of previous successful founders raising astronomical capital without even having a product. VCs are not crazy to do this; they are betting and playing with probabilities and sentiments. The chance of the founder hitting the lottery again is high, hence the clamor to invest. VCs know for sure that not all the startups they invest in are going to be home runs, some are going to go dud and die, and they take this into account while investing. Only a very handful of the startups they invest in turn out to be successes and these handfuls generate the returns. A minority of the successes cover up for the majority of the failures. Returning to our lottery example, out of the hundred, the VC is betting on only one person to win, she knows the rest of them are going to fail, but she gets 500x returns with just one win and 99 failures.

Treat this as a dummies guide to startups and venture capital. I have watered down a lot ignoring VCs and funds who invest for more extended periods. Like any business, the VC industry is varied and diverse.

Deviation From Expected

Someone sitting at a distance asks for the water bottle near me. I pick up the bottle and throw it at that person. Surprisingly, the cap is not screwed. Water splashes all over. When a bottle has its cap on, we usually expect it to be tightly screwed. When something deviates from the expected, unless there is an indication saying so, it creates trouble and confusion.

arrows-1834859_640

The same principle applies to systems and application design. For example, let us say that you have a development server where someone is running a production cron job. Since this is a development server, someone might take it down for experimentation. No one expects the non-availability of a development server to have untoward consequence.

Whenever you deviate from the expected, ensure you scream from the top of your voice so that no one misses it. Documentation, common conventions and putting in the right processes are some of the ways to mitigate this. The best is not to do it. Whatever you are doing, it always helps to ask, is this a deviation from the expected? If I am not part of the inner circle, would I expect it to be like this?

Conventions

Most programming languages have conventions. These could be for naming or code patterns.

rule-1752415_640

How does this help?

A simplistic view is that it helps to keep code consistent, especially when multiple people work on it.

A deeper way to look at this I believe is in reducing the cognitive load.

In cognitive psychology, cognitive load refers to the effort being used in the working memory.

If you have conventions, it is one less thing to think about. You do not have to spend mental capacity on thinking whether to name variables small case, capital case, camel case, with hyphen, underscore etc. You blindly rely on the convention. Same applies to code patterns. You look at the pattern and automatically grok the idea; without expending grey cells.

I strongly believe that all tech teams should have conventions wherever possible; outside code too. Freeing up any amount of working memory for things that matter will go a long way towards increasing productivity.

 

Anti features

When evaluating new technology, framework or library; a lot of importance is given to the salient features. While it is very important to know the positives, the negatives usually tend to be glossed over. Being aware of the shortcomings of a framework gives one the ability to anticipate problems down the road.

feedback-3239454_640

For example, let us take NoSQL databases. A lot of time is spent on singing paeans to the scalability, malleability etc of NoSQL databases while hardly thinking about the negatives that come with it.

Two simple techniques which give a good visibility on anti-features:
1. The very obvious one, Google for the shortcomings. Someone would have written a blog post on the interwebs highlighting how a framework or technology let them down. For example, take this post by Uber on how Postgres did not work as expected for them.
2. Comb through Github and/or JIRA peeking at the bugs raised and enhancements requested.

Both of the above will provide a good picture of the shortcomings. If you are evaluating a closed source proprietary technology, the above may not make the cut.

Once a mental note is made of the negatives, ponder on the scenarios where this might affect your usage. It helps to spend quality time on this as this will save one from a lot of future trouble.

If you think about this, this might sound very obvious but tends to be highly neglected. We get so caught up in the positives of something that the negatives tend to be ignored and this usually comes biting us back later.

Luck

I read an interesting article by Richard Wiseman on luck, which I would highly encourage everyone to read. The gist of the article is that people make their own luck and being lucky is something that can be learned.

An excerpt from the article:

Lucky people generate their own good fortune via four basic principles. They are skilled at creating and noticing chance opportunities, make lucky decisions by listening to their intuition, create self-fulfilling prophesies via positive expectations, and adopt a resilient attitude that transforms bad luck into good.

horseshoe-504821_640

Patrick O’Shaughnessy, in his podcast “Invest like the best“, talks to interesting people. He mainly concentrates on investors who have made it big, but once in a while he also chats with people from other walks of life. A common theme that keeps repeating in his interviews is how these people jumped at opportunities which others had shunned, their optimism and an attitude that stresses on continuous learning and development. These qualities eerily match with what Richard Wiseman says makes one lucky.

In the recent Farnam Street podcast, behavioral economist Dan Ariely says the following – “I gamble with my time. I take risks, I do things that do not seem like the right things to do”.

Two of the most successful and rich people of our times, Bill Gates and Warren Buffett are gung-ho about the future. Bill Gates actively champions positive thinking and wants all of us to cultivate this.

Probably luck is not luck after all. I am sure it is more nuanced than this, but something to ponder about.