Medicine

[At the outset, i'll clarify that i have the utmost respect for the doctors and the profession in general. They are in the profession of health, life, survival, healing and more importantly - alleviating suffering. What can be more important to humanity than this? ]

Medicine is a wide area. There are areas where cause and effect is easily discernible. If there is a fractured arm - we know what to do. There are many aspects of the body that are mechanical and electrical in nature and hence greatly helped by advances in technology - say the hearing aid or progressive lenses or prosthetic limbs

It is important to remember that medicine is in the business of survival, NOT TRUTH. Because seeking truth means understanding 'Life' and the very hard problem of 'Consciousness'. Only the sages have those answers because of their direct experience. Most of the medical science eschews this problems (in)conveniently and look only at the effect to make inferences.

There are many areas in our body that directly falls under the ambit of 'Complex systems'. Our BS detector should vibrate vigorously when someone makes sweeping statements in those  areas. 

Having said about complex systems earlier, the puritanical (or sensible) approach to our body and health would be

• Use prevention: In medicine, stopping someone from smoking has fewer adverse effects than giving pills and treatments. 
• Approach via Negativa:

        Why? Addition to complex system is intervention. You do not intervene a complex system because you will not know the results. Avoidance is simply not messing with the existing system and a more robust way of handling your health

        The Hippocratic oath says, 'First, do no harm' and do we know for sure?

• Instead of disease, look at health and work on that
• Instead of diet, look at fasting

• Look for Lindy: Check on the habits people who survive long. They must be doing something correct
• Be Anti Fragile: Develop equanimity and strength of mind that will help you deal with tail events that you’ll anyway be exposed to in life. 
• Avoid harm and ruin from tail events: for the patient this means, to avoid treatment when he or she is mildly ill, but use medicine for the “tail events,” that is, for rarely encountered severe conditions. 

        (The problem is that the mildly ill represent a much larger pool of people than the severely ill—and are people who are expected to live longer and consume drugs for longer. Pharma has incentives to target the large pool. I'll touch upon Bad Pharma in a bit)

Conflating medical advances in simple areas to that of complex areas

Commonly used refrains are: 'Look at the advances made in the last 20 years. At this rate we can produce consciousness.'

We will become increasingly adept at replacing body parts and will make tremendous progress in various other areas of the body. But, it's a mistake to extrapolate this to the complex areas of the body. There this statement is self-delusional. We will still make new theories, reject old theories, bring back a old theory. It will be a fertile area of wallowing in BS and self congratulation. If you have any doubts about it, check nutritional claims

So to all the complex areas of the body ... apply the BS Detector ruthlessly however rosy a picture one may paint. Even when it seems to work, it is just randmomness at play and you can be ruined by unforeseen event. What is needed is anti-fragility, not some deceptive smart solution in randomness

This is where one has to be careful. We can use very simple theories - they NEED NOT BE THE TRUTH, as long as it achieves the purpose (survival, healing, reducing pain). That's why we needn't look for truth in medicine because one can't do that in complex areas. What is important are the outcomes. And that puts us directly in the area of Statistics and Probability

Allopathic and Western Medical science in general

• Treats the symptoms not the cause. In most of the cases, cure comes from our body's own self healing mechanism. The healer is inside. The external intervention is just to quieten the mind and body to the discomfort and let nature take its course
• Uses a reductionist approach. As long as there is not a holistic approach for a system like our body, we will wallow in BS. It will be an endless game of finding new chemicals, correlation and new theories. Why? Because we are Complex systems ... we are sampling various marker signals in a complex system to try to effect a cure ... that is BS
• Causality is a bitch and direction of causation even more so
• Have Side effects. Why? If you intervene a complex system you don't understand, you will mess with things
• Long term usage is unknown. This is dangerous ...because the statistics which is the cornerstone on which efficacy is predicted, becomes useless if you look at a longer horizon because of the confidence intervals.
• Higher order effects - this is because of the myopic view and simply the lack of unifying thesis of the body

        As if all of the above things are not enough we have three big villains

• Bad Pharma

Pharmas have wrong incentives and hence their ethics can become questionable

The science behind them is shaky - make chemicals, test, prove efficacy. This kind of intervention in a complex system is the equivalent of spray and pray.

Pharmas can be bad actors. It's an important thing to remember and something we can't do anything unless we change their incentives

• Bad Doctor

Another Bad actor. For all the respect i have for the doctors and the profession, many are aligned on the wrong incentives. So the interventions they suggest can also be wrong

• Bad Science

Use of statistics and probability to make random inferences. This combined with the ethics and incentives can become an intractable problem to solve.

Establishing causation is at the heart of it. As we have seen causation is a bitch in complex areas

The only reasonable way we can get confidence in an intervention is to do a randomised control trial (RCT). If you are convinced that this is infallible, ask yourself why Fat was implicated as a culprit for more than a century? 

        It's a combination of Bad Actors (Sugar industry), Bad science and Bad Doctors 

Randomised Control Trials - RCT

Why RCT - our best bet, can be derailed in different ways, 

• Selecting control group

When you don't know causality. On what basis will you select? 

You can only hope whatever that is 'causal' is available randomly and equally in the control group as well as the test group. 

There is an element of chance involved here

Let's say that the actual truth about immune response to any medication is actually governed by the enterotype of the biome in the gut. If by chance we get a skewed distribution in the control group and test group, we are looking at bias

Random assignment helps reduce the chances of systematic differences between the groups at the start of an experiment and, thereby, mitigates the threats of confounding variables and alternative explanations.

This is the best that we can do here. 

• Heisenberg effect - Complex system can behave in a different way when observed. 

• Our body is not an automaton with fixed states, triggers and transitions. Mind is a big variable. Double blind tests are designed to mitigate this

• Duration: This is an intractable problem. Doing a long term statistical study on a considerable population worthy of statistical analysis is virtually impossible. Even if you are ok with the costs, the environment control, the real world connection and simply handling the habits of the population over an extended period means we will have to deal with larger confidence intervals and errors in estimates.

• Unknown independent variable causes bias in the model - can't do anything here. Hope for the best

• Model fitting in the higher dimension - can become the game of connecting the dots and finding patterns in randomness when none exist

• Higher order effects and feedback in the system can cause the system to behave in unpredictable ways - can't do anything here

• Causality direction - this is super difficult to establish. 

• Controlling variables in a complex systems means the absence of some of the stressors that work in a complex system. These can lead to unwanted effects that can affect the outcome

• Random inference: With all the above difficulties we have to have good interpretation and inference and be able to deal with a larger margin for error.

So the upshot of all of this is the equivalent of someone saying, 'Well, i don't know how the system works. But i will  intervene it in many different ways, simplify it at will to reduce complexity and interpret the results and feel good about it'. 

If someone says this even to an external inanimate system - say investing money in a financial system - we'll be petrified ... but we don't blink twice about doing this to our body. 

Finally

I'm not saying i have all the answers. As outlined, i can only take the puritanical approach and try to avoid harm.  What i do expect from medicine though is a high dose of humility to understand and accept that we have no clue of the system. We have to respect nature and the innate intelligence of the body to heal itself. 

Have kindness, compassion, and ethics to the highest level and work with a service mindset.

Managing Cost

[Cost Optimisation Pillar from AWS offers a fantastic introduction to managing costs on AWS] 

Cost Optimisation

While evaluating costs for your solution, use this checklist

• Managed Service vs Self hosted solution: By selecting the appropriate building blocks and managed services, you can reduce or remove much of your administrative and operational overhead, lower operating costs and free you to work on applications and business activities.
• Licensing Costs
• User based vs Bullk
• Pay per usage vs Standard costs. 
• Networking costs
• Data Transfer (within and across DC)
• Elastic IP
• API calls
• HTTP vs HTTPS
• Storage costs
• How much data are you going to store or generate?
• What is the retention period?
• Backup and Restore
• Hot / Cold data structuring.
• Replication
• IOPS
• Caching
• Snapshotting
• Choose the right Instance Type
• Memory,
• IOPS,
• CPU,
• Disk instances are available to choose from.
• Capacity Planning: Right capacity planning avoids wasting costs.
• Choose right instance type - Spot, OnDemand Reserved, EC2 Fleet

Tracking Costs

Cost optimisation is an ongoing effort. Tracking costs helps tune the cost model, figure out optimisation areas, understand total cost of ownership and recommend pricing options for Business

• Choose right instance type - Spot, OnDemand Reserved, EC2 Fleet
• Consistently tag all the resources used.
• Setup alerts to detect daily and monthly breach and act with alacrity
• Use Costs Explorer to understand the costs split across services, instance types, business tags.
• Tracking monthly costing trend month helps to project future costs, understand basic changes in operating structure
• Include stakeholders and plan cost reduction by working on cost

Further Cost Optimisation

After your solution is deployed and tracked, consider further optimisations. 

• Prepare Cost model for the Service 
• Expected use in terms of CPU / Memory / Disk
• Track Expected vs Actual and modify cost model based on observation - understand how cost varies
• Cost model should help identify what parameter varies the cost. e.g,
• Cost / No: of users, 
• Cost / No: of documents,
• Cost / 1GB of storage iv.  Cost / 1M API calls
• Tools
• Tools like Botmetrics can recommend right instance type based on usage
• Some providers bid for instances and pass on the savings as well
• Automation
• AWS instance scheduling - shutdown and bring up instances automatically based on time
• Dev / QA clusters can be set up and torn down on demand. Reduce the need for long running clusters
• Architecture
• Use Autoscaling design where possible
• Serverless - Lambdas
• Docker / K8S - Goal: Max resource utilisation,  Datacenter as OS
• Other architectural changes 

Finally,

all the areas should be working together in a continuous cycle. What items you pick in each stage depends on your particular situation. Feel free to move items to different stages - the goal is to get into a groove on cost cycle.

Cost Awareness -> Cost Optimisation -> Deploy -> Track Cost -> Further Cost Optimisation -> More Cost Awareness 

This cycle deepens when done consistently over a period of time. You will gain deeper understanding of the cost implications and a systematic way of handling costs

Product Requirement Document

A simple PRD template

What is it?

<< Your one sentence pitch on what this release is >>

This is ________ that solves/helps ________ by improving _____ by _____%.

What is the customer problem?	Why are we solving this problem?	How did we validate this problem exists?
• What customer problem are we solving? (your explanation needs to be as clear as mid day sun)	• Why are we solving this problem?  (talk about why is this important to be solving now?)	• Does data show there's a problem? • Did customers come back and ask (if yes, mention the customer name with quotes)

Solution

What's in scope?	What's not in scope?	What are the assumptions?	Are we building the right 'solution'?
• Mention the scope of the proposed solution • Clarify if we would be building in stages	• Clearly mention what this isn't so that stakeholders don't make wrong assumptions	• Outline the BIG assumptions we made in coming up with the solution	• How did we validate our solution? Include any links to customer conversation  (ex. Provide a quote from a customer  that describes how they experienced the benefit.,  user research etc.)

What does success look like?

Metric	Current Metric (before release - dd/mm/yy)	Target Metric (after release + x days/months)	Qualitative aspects
List of metrics you will be measuring • P999 Response time < 1s • Conversions	What is the current value of these metrics?	What are we aiming for?	1. Customer Happiness 2."Your app is loading fast" 3. "I love this new feature, makes my life easy" etc.

Release date & Launch plan

Release plan (beta-release)	Release plan (full-release)	Customer Success Education Plan	Sales Education Plan	PR plan
• Date of Release  • List of accounts • Customer Education plan • Email on <date> • In-app tour on login • Blog on <date>	• Date of Release  • List of accounts • Customer Education plan • Email on <date> • In-app tour on login • Blog on <date>	• Date of Education <date> • Links to training material	• Date of Education <date> • Links to training material	• Date of Education <date> • Link to Press Release

User Stories

#	Title	User Story	Importance	Notes
1	Short identifier for the story	Describe the user and what they are trying to achieve	Must Have	• Additional considerations or noteworthy references (links, issues)
2

User interaction and design

Include any mockups, diagrams or visual designs relating to these requirements.

Learnings & Action Plan

What went well?	What didn't go well?	Learnings	Action Plan
Metrics - did we hit our targets? Customer Feedback	Metrics - why did we not hit our targets? Customer Feedback	Unique insights we gathered	Next steps - ideally the Product Requirements doc for next version

FAQs

Below is a list of questions customer might ask and our answers:

Latency

 

Hiring

Goal of Hiring

The ultimate purpose of interviewing is to make a judgment about how the candidate would perform in your company’s environment. This is at odds with how performance reviews are done after a candidate joins. While you need to avoid the “potential” trap in performance reviews, when hiring, you must judge potential contribution. Within the hour or so at your disposal, you must move between the world of the past employer and your own, and project the candidate’s future performance in a new environment based on their own description of past performance. 

This managerial task is clearly tricky and high-risk, but unfortunately unavoidable. So in the end careful interviewing doesn’t guarantee you anything, it merely increases your odds of getting lucky - It increases the probability of the candidate being successful in your company 

Ultimately, though you would like to avoid false negatives and not miss out on great candidate, focussing on avoiding and managing false positives should be the goal of the hiring process

At a high level, 

• Hiring is the highest leverage activity that you can do as a manager
• Hiring is manager's responsibility
• You should come out of every interview with a clear sense of whether the person could improve the probability of your company’s success.
• Great interviewing is work. It takes time to prepare, conduct the interview and then de-brief in an effective way. If you don’t want to do the work, don’t interview.
• You should always take copious notes during interviews so you can make a cogent argument for or against a candidate.
• In most cases, the “best and the brightest” already have jobs, so you’re really just on the lookout for the best available. Plus there’s no way to prove that your hiring process resulted in the right people because you can’t A/B test hiring decisions.
• Always strive to hire superstars but realise that not all hires need to “walk on water.” This person should be better along some dimensions than a majority of the current staff and have the potential to have a long-term impact.
• You want to hire people who are smart, that get stuff done and have the functional set of skills you need for the role.

What to look for (and the Mindset)?

• You have to know what Excellent looks like
• Hire for strength, not apparent lack of weakness
• Look for excellence in what they do - excellent coding is a must, look for depth and tools
• Look for impact
• Look for learning, drive, energy and enthusiasm apart from talent
• Do not hire with hope - Be aware of the margin for error
• Ask how the candidate is better than us and what can we learn from them.
• Are we excited about candidate and would be disappointed to lose them?

Beware of Bias 

These are some of the do's and dont's, pay attention so that you avoid the pitfalls

                

Dont

• Anchoring / Halo / Horns: tendency to rely heavily on one data point while making a decision (CGPA, College) 
• Bandwagon effect: Everyone follows the group's hiring decisions
• Ostrich effect: Ignoring red flags during the interview process
• Outcome bias: To avoid it, focus on the process and criteria instead of the end result
• Stereotype: racism, sexism 
• Inferior / Superior perceptions of the interviewer: To avoid it, Ensure A hires A (they have nothing to fear and are confident of their abilities)
• Overconfidence: Interviewer's subjective confidence that their judgment is greater than their objective accuracy.
• Confirmation bias: Making perceptions in the first few minutes
• Intuition: Relying on vague feelings instead of concrete facts

Do

• Make hiring decisions based on evidence rather than on subjective assumptions.
• Be consistent and transparent in your hiring process.
• Have multiple panelists (diverse preferably)
• Shadow interviews and give feedback during debrief process
• Once you form an initial impression of someone - which usually happens within the first 60 seconds - beware of the confirmation bias and try invalidating it.

Be aware of your Evaluation criteria

• Distinguish between good and great answers: know extension points and follow up questions 
• Your whole team should sit down and discuss what’s a good coding question, what’s a good answer and why you should use this one.”

Creative and in-depth answers indicate the thought process of the   candidate. For example,

Q: Is a leader born or made?

A: I am persuaded that a leader is not made in one life. He has to be born for it. For the difficulty is not in organisation and making plans; the test, the real test of the leader,  lies in holding widely        different people together along the line of their common sympathies. And this can only be done unconsciously, never by trying - Swami Vivekananda

• Learn to validate a resume: How to read a resume, identify the areas worth deep diving, and determining how to probe into them.

• Great interview questions focus on specific examples of the candidate’s unique contributions, actions, decisions and impact, 
• Ask: 
• unique contributions, actions, decisions and impact
• Probe: who, what, where, when, why and how on every accomplishment or project
• Differentiate: we vs. I, good vs. great, exposure vs. expertise, participant vs. owner/leader, depth vs breadth
• Differentiate between glib talker vs person with substance

• STAR questions: Look for past projects and accomplishments that seem to have enough weight and depth so that you can apply STAR questions — STAR stands for situation, task, actions and results.

• What was the background of what you were working on?
• What tasks were you given?
• What actions did you take?
• What results did you measure?

• Learn to ask structured questions: Idea is to understand at what level the candidate is operating.
• In your experience, what is the key to developing a good team?
• What do you look for when hiring a candidate?
• What would you have done differently at your last company if you had been CEO?
• Tell me about a time your behaviour had a positive impact on your team.

• Give allowance that a candidate might have a bad day
• Do not ask questions which are usually not a test of fitment to a job
• Do not ask for things that can be Googled
• Look for the questions the candidates ask? Do they just need a job or are they curious about your company
• "What is the most technically challenging software problem your company is facing right now?"
• "What would you expect me to accomplish in my first 90 days?"
• "How will my performance be evaluated?"
• "How do your best employees grow within the organisation?"
• "What is your company's short-term / long-term business strategy? What is your competitive advantage?"
• "What do you (the manager) love most about your job?"
• "What are your company's biggest weaknesses? How do you intend to improve on them?"

Some Tips

• “Tell me about your background” is not a useful question for a Tech interview. It's an invitation to digress and ramble
• If the interview lands up in a unintended place, it's an indication of under preparation and lack of moderation to avoid digression
• Pick specifics out of a resume to determine what the candidate actually did. Remember, you want people who gets stuff done. Period.
• Probe when you see a resume with a long list of skills. Separate the truth from filler.
• Don’t “try out” new questions on candidates. Know what a good answer sounds like.
• Make sure you have them write code! This is too often skipped.
• Dig into algorithms, data structures, code organisation, simplicity.
• Use some questions that are vague and open-ended. See if they ask you questions to find out more.
• Ask a design question. See how people think about a bigger picture problem.
• Create core competences for your company. Make sure candidates measure up well.
• Make it tough but fun. Good developers want to know they’re talking to smart folks.

Finally,

• Care for candidate experience.
• Calm the nerves for the candidate before you start the interview. Ensure they aren’t nervous or intimidated
• Prepare precise questioning and structure, so that the answer can be evaluated predictably
• Allocate specific time / day so that you and your team doesn’t get randomised with interviews
• Hiring speed is important: Remember, the candidate is interviewing elsewhere
• Have referral checks. In fact they should have a lot of weightage (particularly the blind referrals)
• Have post offer engagement and keep in touch. Offer: Join ratio is usually is 3:1, you wan’t to reduce it.
• Have a kick-ass on-boarding plan to integrate the candidate into your environment
• Evaluate your Hiring process 3 - 6 months with HR after the candidate joins and make change to reduce false positives
• Evaluate your team's interviewing proficiency. This is important for assembling a great team.

On Complexity

Complex system

The main idea behind complex systems is that the ensemble behaves in ways not predicted by its components. The interactions matter more than the nature of the units. One cannot perform reductionism.By definition they are chaotic and random. There are myriad interactions and what appears out of it should simply be seen as is without naive explanations. These are multi-dimensional systems interacting at various levels. Finding order or explanation in this is a fools errand.

By definition it is not possible to discern Causality in a complex system. Consequently, we cannot make any prediction and any attempt to predict is Fooled by Randomness (FBR)

Some complex systems

• Brain, Mind
• Human body, 
• Macroeconomics, 
• Universe, 
• Financial markets, 
• Ecosystem

These have properties such as such as nonlinearity, emergence, spontaneous order, adaptation, and feedback loops, among others which are observed but cannot be explained

Guidelines on dealing with Complex systems

Beware of the Bull Shitter

Anyone claiming to know / predict / theorise in complex systems is a bull shitter because no one knows causality. Without understanding causality, prediction is farcical. So, beware of anyone making predictive statements in this area. They don't know anything. You can treat those predictions as pure entertainment, otherwise they are useless

Do not intervene with a complex system

Why? Because you don't have a clue as to what's happening. You might disturb the intricate balance. Taleb fondly calls them as Interventionistas because they don't have a frigging clue but still want to intervene. More importantly they don't have skin in the game

Look for Survival a.k.a Lindy Effect

The only lessons you can draw from a complex system is to look at something that has Survived. It's a sure indication that it's doing something correct to survive in a complex system in which it exists. Otherwise it would have become extinct. Lindy effect is in the risk dimension, not the truth dimension.

"Not everything that happens happens for a reason, but everything that survives survives for a reason" -- Taleb

Understand Ecological Rationality

Taleb argues many supposedly irrational beliefs in a complex system and behaviours may have an 'ecological rationality', that is they are rational when viewed in terms of the environment they exist in.

Rather than purely looking at things as true/not true, as the naïve rationalist does, we should look at behaviours through the lens of pay-off. When we view things this way, the world becomes a different place - 'irrational' things can become rational when we consider their outcome. 

An action, superstition or attitude that reduces harm either to oneself or to a collective is arguably entirely rational, even if it seems illogical or ridiculous. 

He argues there must be benefits of irrational long-lasting beliefs otherwise they would have died out.

Beware of the Black Swan 

Platonification (When people, especially the learned and authoritative, create categories and oversimplify) missing central but hidden elements of a thing in the process of transforming it into an abstract construct, then causing a blowup. This is because of our inherent nature to simplify, categorise and explain everything. 

The Devil is in the details and the complex interactions. Simplfying a complex system is to miss its intricacies and be ruined when a rare event happens. Not understanding the stressors of a complex system, removing, reducing and simplifying is an invitation for trouble

Look for Skin in the Game

Arm chair analysts do not have any skin in the game. If you want to understand financial markets - go talk to a trader who lives in that unpredictable system and has to systematically avoid ruin. He cannot form fancy theories, discern patterns and use unbaked ideas because a single mistake can cause ruin. 

Anyone else who doesn't have Skin in the Game, who hasn't faced systematic ruin and survived are removed from ground reality. Their opinions are worthless

Look for Antifragility

Antifragility is the property of all those natural (and complex) systems that have survived, It is far easier to figure out if something is fragile than to predict the occurrence of an event that may harm it. That is the only reasonable thing that we can do in an unpredictable world. 

It's about Non Prediction - we can build a systematic and broad guide to non-predictive decision making under uncertainty in business, politics, medicine, and life in general— anywhere the unknown preponderates, any situation in which there is randomness, unpredictability, opacity, or incomplete understanding of things.

This is important ... don't try to predict / theorise in a complex system. Try and be Anti Fragile. Anything that has more upside than downside from random events (or certain shocks) is anti-fragile; the reverse is fragile.

The mother of all Anti Fragility is given by the Sages: The external world is a bitch and completely random. You can't predict or make sense of it and more likely you will get an oddball. 'So, take your mind off it; strengthen your mind to not depend on it and make sense of it. Otherwise, you will be only deluding yourself and unprepared when the oddball arrives'. 

That is the only reasonable thing to do with the world.

Approach via Negativa

This is very important. Why?

Addition to a complex system is intervention. Without knowing causality, it's asking for trouble. You don't know the outcome. While avoidance is simply not messing with the existing system and a more robust way of handling the situation. 

The Golden rule approached via Negativa is the best

"Do not treat others in ways that you would not like to be treated" rather than 

"Treat others as you would like others to treat you"

If you get this, you get the concept

On Reality and Truth

 

Over the years, i've had many conversations with various people on the nature of reality and truth.

What i've found is that people look at it from various directions with differing assumptions. I've tried to arrange the view points and the assumptions therein.

I've expanded a bit on Advaita (which is my position) on why it provides a Unified Theory of Reality as it exists now.