by on

I keep thinking about abstractions all the time ever since I begun to understand large scale software development. In the early days of programming I took a lot of pride in understanding code in depth and writing the most efficient code. I was fortunate to start learning programming from the rudimentary steps like creating adders using logic gates. From there on progressed to assembly, BASIC and C.

If we take a step back, late 90s and early 2000s we are talking about super expensive hardware in Indian currency terms. To give an idea, a simple desktop pc setup costed the entire year’s income of a middle class household. On top of it internet had dial up charges and subscription charges. This meant people had to be frugal and extract the maximum out of their machines. For many years I have never been able to run an IDE on my machine because of limited hardware and ended up using text editors. Also efficiency was at the back of my mind of writing small footprint programs which consumed a lot more time than producing software that is reasonably performant but needs good hardware.

Collectively this scarcity based mindset made a lot of people approach with frugality, often having a lower specced computer for their personal use. Even when hardware became more affordable and software development technologies leapfrogged, I witnessed that people fought against upgrades and learning newer ways to do. Developers refused to learn to work with IDEs which made development significantly faster, test automation was in no one’s land as both devs and QAs avoided adopting it.

Photo by Andrea De Santis on Pexels.com

The difference between the teams using IDEs, continuous integration, build and release automation, test automation all which consumed a lot of hardware was around 3-4x. It was very visible that we removed dedicated roles like infra engineer, DBA, reduced the dev-qa ratio, reduced the number of leads and managers. A 40 member team was replaced with a 12 member one when high degree of automation was used with an added expense of hardware costs but overall it was much much better than running a 40 member team. One thing that changed which I did not expect was it increased the number of Business Analysts needed now to define the requirements better.

Only a few companies succeeded in taking advantage of this and it took more than a decade for people to catchup. We are at a similar point now to take advantage of new tools that can change the team composition and get more out of less in software development. Just like in previous waves of development, it is not going to eliminate people entirely from this. Instead it frees up a lot of people to solve more higher order problems. Companies can now focus exponentially on what was traditionally expensive with less people able to do more. Examples I can think of is accelerating development of new medicines and treatments to the extent it can be custom made for every person; predict, fight and prevent large scale natural disasters with precision; strengthen the security posture making it difficult more expensive and difficult to cheat; revolutionise entertainment by localising with actors, accents and languages than traditional dubbing and subtitles. The list goes on, jump on the new wave and ride new highs.

by on

A lot of have been talked about stress at the workplace, those are visible and bold like toxic managers, unrealistic deadlines etc. A very few people speak about silent stressors that are very much invisible but damages more in the long run as very less is done to eliminate them. When I had begun working, hardware was very expensive. In some companies they used to use the machine across multiple people in shifts. I got a machine assigned to me and over the course of 4+ years it was never replaced. It was impossible to run an IDE so I had to rely on text editors to code. What this meant is a constant loop of editing code, going to the console to compile and come back to figure out errors in the editor by comparing compiler outputs. The days just dragged on in endless loops.

Photo by SHVETS production on Pexels.com

One of the other types of stressors were unplanned working days. At the hands of a novice manager, days get difficult to plan. Developers need unbroken time and reduced ambiguity, but constant interruptions and ambiguity feels like a constant droning in your head. Detachment from family and social life, this stressor builds up slowly and is a secondary stressor often a result of other ones. In my younger days, I have missed many of my friends’ weddings, found it difficult to be at the side sick parents/grand parents at home town and slowly the immediate circle is only acquaintances from the office which is a deep echo chamber.

Identifying these stressors early is a key to good mental and physical health. I did not realise this for a long time, once I figured out and eliminated it then it improved my quality of life. Ask these simple questions, a single no requires you to reconsider what you are doing?

  1. Are you able to finish your work week within ~40 hour window and still find it fulfilling/ having a sense of achievement/ being recognised? (Some lean weeks and some tight weeks will be fine, but the average should be around 40)
  2. Do you find time for friends and family?
  3. Are you able to eat, sleep and exercise well?
  4. Do you like going to your office, is your 1 way commute less than 30 minutes or if it exceeds do you have a commute routine that is engaging (like reading, audio books, podcasts or simply enjoy the drive)

Ask yourself these questions? Find out the root silent stressor and figure out a way to eliminate it. The stressor could be anything from poor hardware, bad planning, noisy environment, gossip culture, poor management, tough competition etc. Don’t cope up with any kind of stress, just remove it.

by on

A doctor whom I met, was upset about dealing with a lot of people coming to clinic equipped with wikipedia and LLM generated knowledge. Years of education and practice puts a doctor in the zone of unconscious competence, but for an expert beginner with no formal education or practice it is just a fact in some context without reflecting what is in hand. The doctor’s intuition will be right and often arrived without conscious thought, asking them to explain in detail may prove counter effective, in some case make the doctor doubt their judgement and end up treating poorly.

Let us take a few other examples from other domain. If you have come across the Monty Hall problem, where a host in a game shows 3 doors to a contestant, behind 2 doors are goats and the remaining 1 door with a huge reward. Once a door has been chosen, the host will open a door which has a goat behind it and give the contestant an option to switch the door if they think they would have made the wrong choice. If we do not think deeply, we think the odds are always 1/3rd irrespective of switching or not. In reality the odds are 2/3 if you switch and that has even stumped degree holders in math.

Photo by Pavel Danilyuk on Pexels.com

Another counter intuitive one is bank teller queue management. I read at this blog, which mentions that an average five hour waiting time to service a customer can be reduced to 3 minutes to just by adding one teller extra. That is 90x productivity jump by doubling cost yet a lot of decision makers won’t believe the expert who analyses the situation and recommends them the solution because it does not make sense for a non expert. The example is dramatic but can happen in real situations as well.

When I observe a lot of people with expertise, their unconscious competence helps them navigate with ease without even thinking about it. The moment you question their judgement, their instincts take a back seat and suddenly their competency goes down. I was at a restaurant, I requested a cook for fried eggs. The cook asked if I wanted both sides to be cooked, and I said “yes, but don’t break the yolk”. The cook left out a nervous laughter and when turning the eggs to the other side, the yolk broke. I disrupted the cook’s flow just by doubting their ability.

This happens frequently at work. Decision makers who are expert beginners often want to get into the details, what this does is, it interrupts the flow mode of expertise. Explaining the solutions and decision making process which would otherwise be unconscious nature, requires a good deal of effort and often leads to sub optimal solutions. If you have an expertise on some area and are tasked with solutions, then keep in mind that you have to explain your decisions to people who know details at a surface level. People are naturally curious and LLMs feed their curiosity a great deal.

Photo by Julia M Cameron on Pexels.com

For this reason, whenever I come up with solutions, I make it a multi step approach. This is I learnt from many different sources which help you harvest your unconscious competence.

Step 1 – Read the problem statement, re-read and think very hard to solve the problem. Too often no solution emerges, but all of a sudden when you are at a break, a solution emerges and when it happens immediately write it down. Beware, this solution is ephemeral and its details vanishes within a few minutes. Keep noting down the solutions that pop up at odd times like driving, cleaning etc.

Step 2 – Find reference material from internet and previous assignments to back your solutions. If it is a novel solution, dive deeper to explain but do not change the solution because you can’t find explanations.

Step 3 – KYEB (Know Your Expert Beginners) and be equipped with ELI5 answers to anticipated questions to impart confidence of the solution.

Before presenting your solution, establish your credentials which helps setting the right expectation using info from the KYEB research. This helps to present your views as an expert without getting into a loop of explanations and doubts. For the doctor’s case I discussed at the beginning, I recently see a few doctors have a dossier to quickly explain their decisions and cut short the questioning from the patients. We also will be expert beginners at many topics, the best we can help there is to let the experts do their job.