Security By Obscurity

Security by obscurity is one of the most common ills that plague the software industry.

The most cited analogy for this practice is the act of burying your gold under a tree versus locking it in a safe. Hiding your gold under a tree is security by obscurity. You are relying on the secrecy of the act for the safety of your asset. If someone uncovers this act, you are busted. Also, this gives you a false sense of security.

tom-roberts-Xbx4h70MIcU-unsplash.jpg

Get articles on coding, software and product development, managing software teams, scaling organisations and enhancing productivity by subscribing to my blog

Recently my mesh router went kaput. I had to put in place a temporary alternative as internet connectivity is more important than oxygen these days. I had an old router given by an internet service provider. I tried to get it working with my current internet provider, but I could not. The router was locked to the provider. It would not work with any other internet providers. I tried to install OpenWRT on the router, but the hardware specs of the router were too low to host OpenWRT.

The other option was to change the custom firmware of the router to the router manufacturer’s default firmware. The manufacturer’s website had the steps to upgrade the firmware. I tried to follow the same. Strangely, the firmware upgrade section was missing in the router’s configuration interface. A bit of searching on the internet revealed that the internet company had removed this functionality from the interface, but if you directly go to the endpoint of the upgrade, it does work. They had just hidden this functionality in the interface. I did the update, and voila, unlocked the router.

This is textbook security by obscurity in practice. The internet company relied on only removing the upgrade link from the interface instead of eliminating the entire functionality. Anyone who discovers this is free to unlock the router. The right way to do it would have been to disable the feature as a whole. Not just obscure it in the interface.

Whenever you want to secure something, do not only rely only on obscurity. You can add obscurity as an additional security measure. For example, it is a common practice to change the ssh port on servers from the default port. Server admins do this in addition to a lot of other security lockdowns.

Photo by Tom Roberts on Unsplash

Distributed Tracing

Distributed tracing means different things to different people – depends on whom you ask.

In this post, we will look at distributed tracing in the context of debugging web application errors using logs in a microservices environment. We will also figure out the way to implement distributed tracing in the least disruptive manner. The aim is to leverage components that are already part of your application stack without introducing anything new for the sake of distributed tracing.

There are many closed as well as open-source products to get distributed tracing off the ground. There is nothing wrong in using these; in a lot many cases, the investment may not be worth it.

The idea of distributed tracing is straightforward. Every request that lands on your application should have a trace id – a random unique alphanumeric string. The trace id is usually called a request id. From there onwards, the request id should be part of whichever code path the request invokes within the application as well as any external calls it makes. The request id should also be part of the logs generated in all these paths. If the code path invokes external services, those calls should have the request id in the header. The application serving the external calls should follow the same pattern as discussed here.

There you go, the simplest possible distributed tracing implementation.

distributed-tracing-copy.jpg

Get articles on coding, software and product development, managing software teams, scaling organisations and enhancing productivity by subscribing to my blog

One decision to make is where to generate the request id?

If you are serving a user-initiated action, generate the request id at the entry point of the request. Post that, all the subsequent code paths and dependent services will use the same request id.

The best place to generate the request id is the web server fronting the application server. All web servers have a way to add a custom header to an incoming request.

Below is how you would generate a request id in Nginx:


location / {
  proxy_pass http://upstream;
  proxy_set_header X-Request-Id $request_id;
}

Nginx doc – http://nginx.org/en/docs/http/ngx_http_core_module.html#var_request_id

We should generate a request id only if the request does not already have one. This sort of conditional generation can be achieved using the below configuration:


map $http_x_request_id $reqid {
  default $http_x_request_id;
  "" $request_id;
}

location @proxy_to_app {
  proxy_set_header X-Request-ID $reqid;
  proxy_pass http://backend;
}

Helpful links:

https://stackoverflow.com/questions/17748735/setting-a-trace-id-in-nginx-load-balancer

https://stackoverflow.com/questions/13583501/nginx-how-to-add-header-if-it-is-not-set/44761645

Also, make the request id part of web server access log.

Now that we have guaranteed all incoming requests have an id, we need to log the request id along with the application logs.

Most logging/application frameworks have a way to pass a thread-local custom context between layers. You can inject the request id in this context.

If you are on Java using Logback, MDC can help you to achieve this – https://logback.qos.ch/manual/mdc.html
If you are on Python and Django – https://github.com/dabapps/django-log-request-id
If you are on Go and Gin – https://github.com/atarantini/ginrequestid

Now that the access log, application log as well as any external calls made has request id, you can trace a request’s entire journey throughout the application and debug errors and pinpoint the cause.

Even if you are not trying to tie up micro service calls together, even within an application, having a request id helps in debugging issues faster – you can trace a request as it traverses through the different layers of an application.

Now that the strategy and tactic are clear, some enhancements; this is the right time to talk about span ids.

Let us say you have service A calling service B. Service A generates and sends a trace id as part of the call to service B. In addition to logging A’s trace id, service B should create a trace id of its own and log that too; this is the concept of a span id – an id owned by each service.

You can expand the span concept to a unit of work within a single service. For example, if your application has three different logical units, you can generate a span id for each of them too; depends on your use case and what is it that you are trying to achieve.

Having distributed tracing in place is a huge productivity boost. It is a low investment high ROI activity.

Designing A Great Meeting Room Experience

This post is a thought experiment in designing a great meeting room experience. This post will not go into the productive ways to conduct meetings but will deal with the mundane yet essential logistics part of meetings.

We will try to design the experience based on a couple of simple, timeless principles:
1. Nudging people towards proper behavior – Libertarian Paternalism.
2. Not relying on one’s will power to do the right thing – Ulysses Contract.
3. Designing the environment to influence productive behavior.

drew-beamer-9uX5cX1l3bw-unsplash.jpg

Get articles on coding, software and product development, managing software teams, scaling organisations and enhancing productivity by subscribing to my blog

Here we go; this is more of a listicle than structured prose.

If you want someone to do something, make it easy for them to do it – following this principle, it should be effortless to book a meeting room. The booking interface should list all the meeting rooms and the available free slots. It should also inform all the facilities a meeting room has like video conferencing, the capacity of the room, etc.

Outside every meeting room, there should be a display of the schedule for the day.

One of the often irritations is you landing up for a meeting in a room that you booked well in advance to find a paper sticking on the door saying the room is blocked for some critical visitor; this sort of overriding should be restricted to only a very few rooms.

How many times has it happened to you that you get up during a meeting to write something on the whiteboard to find the marker and the duster missing? A meeting room should be well-stocked with stationaries so that people do not have to step out in the middle of a meeting to fetch them.

There should be a large wall clock in every meeting room so that everyone is conscious of time. Even better would be a countdown timer which gives an auditory signal when the end of the meeting is near.

There should be a designated place in all meeting rooms to dump phones and laptops. One of the biggest distractions during a meeting is the constant barrage of notifications on devices and the pavlovian reaction to them. Do not rely on people controlling their will power to overcome this but design the right environment for people to achieve this.

Even though meetings have come to be associated with unproductivity, I believe collaboration and brainstorming are essential for crafting the right product. It is paramount that you do everything possible to facilitate communication and discussion between teams; this might make or break your product.

Photo by Drew Beamer on Unsplash