So here I am again, wanting to write something. My head and hands feel so numb lately for keeping so many things in my head, unorganised. Do you know what happen when you keep too many things in your drawer, unorganised? You'll find it hard to try to pull something out of it. You even forget that you've kept anything specific on it. In the end, that drawer will become a full but useless one because you think it's just filled with unimportant stuff. Alright, that's a perfect metaphor which I just figured out in my brain as I'm writing this, for the injustice that I think I have done lately to my brain. I feel so excited all of sudden because I'm able to finally jot everything down like this! So here is the stuff that keeps my brain full in the past few weeks:

The concept of mental model

I first discovered this term from my colleague who has his PhD in decision science (a very cool program, I think, since it does not only give you the "theoritical" knowledge, but also gives you the wisdom you need for your own life). What is a mental model, exactly? I also have difficulties in trying to grasp the concept at first. I'm just gonna quote the brief explanation from my new favourite site, Farnam Street, about the definition:

"Mental models are how we understand the world. Not only do they shape what we think and how we understand but they shape the connections and opportunities that we see. Mental models are how we simplify complexity, why we consider some things more relevant than others, and how we reason. A mental model is simply a representation of how something works. We cannot keep all of the details of the world in our brains, so we use models to simplify the complex into understandable and organizable chunks."

Why am I so amazed by this? I always wonder how some people can be so smart, having broad knowledge, and can see things from various perspective. I'm always a BIG fan of these people, and now I kind of understand why they can be so. It's not only because they are "naturally smart", but they are exposed to or they learn deep about many mental models which shape the way they see the world. And I find that SO amazing.

Again, another quote:

"The quality of our thinking is proportional to the models in our head and their usefulness in the situation at hand. The more models you have—the bigger your toolbox—the more likely you are to have the right models to see reality. It turns out that when it comes to improving your ability to make decisions, variety matters."

It's amazing to me because I now figure out the reason why I love discovering new knowledge, especially from disciplines that I never got exposed to before. It might be because having knowledge of various disciplines can lead us to have a well-rounded understanding and eventually allow us to make better decision, and become wiser in life. And maybe, just maybe, this is also the reason why I aspire to have a life partner who has different area of expertise than me. It is amazing to dream how we would talk about different things and I will be first-hand realise how my partner see single thing from different perspective. LOL. Am I dreaming too high?

As mentioned in Farnam Street blog, building latticework of mental model is a lifelong project. Working on it would enable us to improve our ability to better understand of our surroundings, make better decisions, and help those around us. The blog has a list of useful mental models here for you to learn more.

The concept of second-order-thinking. 

When I heard "first-order"or "second-order", the thing that popped to my mind is numerical optimisation (You know, first order method which requires first derivative a.k.a gradient, second order method which requires second derivatives, and so on). Apparently, the term can also be used in thinking. First order thinking is when you can think of the immediate consequence of some kind of act, like the first layer of it. Second order thinking is when you can also think further about the subsequent impact of that act, holistically. Mastering second-order thinking is what makes you different with the general population because naturally, human have the ability to do first-order thinking in all situation, but not second-order thinking. It's also amazing to me how I observe my nephews growing his second-order thinking. He started to know that doing x does not only impact y, but he also knows if y happens that it means he will observe z (Amazing and sad at the same time, because that means he's not so little anymore :') ok why am I getting emotional all of a sudden?)

Derivative of Occam's Razor.

When people mention Occam's Razor, I will always remember my favourite lecturer back from my Bachelor study, Pak Ruli. I could not specifically recall which example that he used in one of his classes when he briefly joked about this razor, but what I remembered from his explanation is that out of all possible events, the thing that is most likely to happen next would be the one having least difference to what just happened before that i.e. the simplest one. It's very unlikely that a major complex change will happen all of a sudden without any prior events which lead to that. That's the only understanding about Occam's Razor that I believed in before, and I might not really grasp the concept accurately. So recently, I read about the details of this Razor, and turned out it's much more meaningful that that. Occam's Razor is not only about "the simplest solution is correct", but in its heart, it's a principle of minimum energy. One of the derivatives of this can be used to scientifically explain why is it that when a person repeats the same action on a regular basis in response to the same cue and reward (or practice something over and over again), it will become a habit as the corresponding neural pathway is formed. According to Occam's razor, it's because from then on, our brain will use less energy to complete the same action. This is just one of derivates of Occam's Razor that I find so mind-blowing.


And those are the three concepts that get my brain excited lately. I also realise now that mostly I'm so amazed when I find a scientific reason of anything that I usually observe without much context, things that I previously thought as only natural to happen. After all, it all boils down to science. Crazy, how this world works.

Ciao!

Just few months after I (re-)joined the Data Team in TVLK, I learned a new concept called Causal Impact. It's the concept used by statisticians or analysts from various domains to understand the impact of an event or action (i.e. intervention) to the outcome they're interested in. Why is this important in various domains, even in business?

It's a common phenomenon in Business to observe various correlation between factors or variables, whether it's a positive or a negative one. The problem is, we would not know if this correlation also means the presence of causation between the variables that we're observing. Causation is harder to discover than correlation because there's a direction in causation. we can say A correlates with B positively, and that would mean the same thing if we say B correlations with A positively. However, A causes B is not the same with B causes A. In causation, we also have to figure out direction of the causation, or in simpler terms, we need to know which one causes which.

In other situation, we are faced with a problem where we would like to implement or impose something that we know would impact several variables, and we would like to measure how big the impact is. Since we are planning to put a change / new treatment in place, one thing we can do to estimate the impact is to plan out a randomised experiment where we split our population (or parts of it) into two groups: control and treatment, randomly. Control group would not experience the intervention while treatment group does. In short, what we need to do is running the experiment for a specified period of time and by the end of the experiment we perform a statistical test (e.g. t-test) to see the difference of the outcome (using our metric of interest) between the two groups and see if both are having statistically significantly difference or not.

However, we don't always have the bandwidth to do experiment. For example, marketing team or product team demanded to release an intervention (e.g. marketing campaign, new app feature) to the production which is open to all audience, soon. We would not be able to conduct experiment by then since there would be a limitation in time for designing the experiment (e.g. performing power analysis) prior to the release. In other situation, the experiment could not be conducted since simply because the team didn't have the resource to implement splitting function in production backend. Another problem that usually arose would be when the team observes something that suddenly changed in the past, and we would like to find out which action actually cause that difference to happen. Apparently, there are numerous actions or events that have occurred which all have the probability of causing that trend to change.

This is where Causal Impact comes in handy. In short, what we're trying to do is comparing the actual observed trend with the predicted trend that would have happened if the intervention did not take place (i.e. counterfactual or synthetic control), and measure the difference between both. The concept is quite simple and intuitive, but the implementation is quite complex. Unlike experimental data, we will only observe one outcome of the population at the same period of time since they are all always receiving the similar treatment for the same period of time. That is why we need a prediction model to estimate the potential outcome that would have happened if the treatment did not change, or if different kind of treatment was implemented instead. Having a point estimate is not enough, since we need to estimate all the missing potential outcomes.

The most powerful ingredient we can use to predict what could have happened is other time series that are related to our outcome variable (i.e. covariates) but we assume would not be impacted by the treatment / intervention. This most powerful ingredient is also the hardest component to get since we need to have enough understanding of the overall metrics (having a domain-specific knowledge will be really useful in this case).

I won't go deeper to the technical side, but what the causal impact model does is basically:

1. Draw posterior sample from the graphical model (from the moment intervention happened and onwards i.e. post-period)
2. Build a model based on pre-period (e.g. model B)
3. Use model B to predict what should have happened in post-period
4. Draw samples from model B
5. Get a posterior distribution of our causal effect

Google has an open source package in R which was built based on Bayesian structural time-series model: CausalImpact. There's also a simple tutorial there which is using one covariate for predicting the outcome. Again, the actual problem in real world will require you to have a handful of covariates to get better predictor of the causal effect.