A Reluctant Ombudsman

On Travel

noreply@blogger.com (Daniel D'Mello) — Wed, 13 Jul 2016 15:57:00 +0530

Why do people travel? I was reading Karl Pilkington's two books titled 'An idiot Abroad', detailing his travels to the modern seven wonders of the world, and a few other 'must see before you die' places, and his reluctance to be impressed by what he sees. As is often the case with being presented with a contrarian perspective, it made me rethink my own views about travel.

I actually like Karl Pilkington immensely. I think he's your everyday regular happy guy (even if he doesn't look it) and leads a happy existence without any wants. He's happy with what he has, and therefore doesn't feel a yearning for something he doesn't want, like travelling the world to see the Great Wall of China or gorillas or whales or swim with sharks or to skydive. He's just happy to wake up in his own bed everyday, go to work, go to the pub for a pint and then go to bed.

To a Western observer, this would be perceived as narrow or small-mindedness. In the enlightened West, you can't appear to turn down an opportunity at international travel, the chance to expand your horizons, broaden your mind and learn from other cultures. You're supposed to put aside your city comforts and challenge your perceptions of life and society by immersing yourself in different cultures and experiences as this might give you a different perspective on your own life and make you a better more rounded human being [insert mandatory joke about Pilkington's head].

While I empathise with the open-mindedness of this narrative, I also see the danger of it turning into an overbearing patronising one, that turns the world-weary traveller into a self-entitled preacher who looks down at anyone not as well-travelled as himself/herself. "You haven't seen the Northern lights? You haven't spent a night in a treehouse in an Amazonian village? You haven't accidentally slept with a ladyboy in Bangkok? You're missing out on so much. You can't be as happy as I am. You're not as complete as I am."

I wonder if this narrative is something we need to recognise. This narrative that tells us that we aren't happy unless we do x,y,z. I don't buy into it. Sure, you can learn a lot from international travel, but how healthy is this action when it comes from a personal yearning brought about by comparing yourself to other people and their lives? At what point can you truly travel the world as an individual, and not because society tells you that you can't be a fulfilled individual until you do it. If backpacking the world is the same as following a trend, then how different is climbing Mount Kilimanjaro from buying a new smartphone?

What then are more righteous reasons to travel? Curiosity? Boredom? Excess time or money? I actually find these better reasons to travel, though some people today would frown upon anything that doesn't involve an immersive experience for purposes of enlightenment. You know who I mean. People who classify themselves as travellers rather than tourists.

---------------------------------------------------------------------------------------------------

It's interesting to look at these two forms of travel. The tourist route is more of a work hard, play hard phenomenon. We lead these stressful rat races that we tell ourselves is normal, and then go on vacation to 'de-clutter' or 'de-stress'. Is travel then our means to escaping stress, like pressure being released from a valve? Is it a by produt of living a certain type of existence? Well, not necessarily. You can live a relatively stress-free life w.r.t work and still want to travel occasionally. What would make you want to travel? Maybe it's still peer pressure, curiosity, a desire for more excitement in your life. Whatever the reason, it's an expectation that needs to be fulfilled, just like long term/immersive travel/backpacking, but with a different or missing snob factor. The difference between these two forms of travel has been presented in documentaries like 'A Map for Saturday'.

Personally, I'm against being judgemental about any particular kind of phenomena. I find it more interesting to observe why a phenomena exists and how it came to be. Looking at the process, the mechanism, the underlying factors can tell us how society functions.

One aspect is money. I think anyone would travel if they had an excess of money. I think that people of means are merely doing what any one would with a shortage of time, given work constraints and an excess of money, i.e make the most of your vacation time. I understand why people do this. I don't completely understand why people backpack. If we are truly satisfied with our lives, we wouldn't want to change them. People who backpack i.e green travellers as opposed to grey travellers, could be curious about the world, afraid they're missing out, or maybe they're looking to find themselves, or a feeling of belonging. Which again comes down to expectations.

I suppose people are different in terms of what they want from life. Some people like Karl Pilkington are happy with what they have. Others just want more. Maybe our expectations are linked to our upbringing, circumstances and personality.

----------------------------------------------------------------------------------------------------

Looking at the exact opposite of this viewpoint, I am reminded of certain characters from the Alexander McCall Smith series I've been reading - 'The No.1 Ladies Detective Agency'. Set in Botswana, the lead character and her friends have no desire to leave home. That is where their happiness lies. They have a connection their their land and the lives connected to it, and even though they have problems, they are generally happy where they are.

You tend to see this a lot in certain societies. Again, I'm not saying that village or small town life is better than a life spent learning and travelling. It's not like village life in India is ideal, even accounting for only rich villages. Being tied to a place culturally can give you a strong sense of identity, but it can also lead to narrow-mindedness, suspicion, and in-group out-group mentality. I'm certainly not a fan of that. But to assume that the opposite of small town insularity is travel is ridiculous. You can spend your life under house arrest and still be a wise person. You probably won't have experienced as much as the international nomad, but you won't necessarily be the most insular person around, just as the nomad won't be the most broad-minded person around.

Maybe the best way forward is to not define yourself and your happiness in terms of other people's expectations. Easier said than done I imagine. You will always be influenced by the decisions of others. Cultural evolution theories tell us that this makes our lives easier.

-----------------------------------------------------------------------------------

Interesting also is the media depiction of travel. Western society tends to depict travel from a highly individualistic point of view, in keeping with the self-actualisation narrative that they love. "You are special, if you are unhappy in your current situation, it's because you are misaligned with your place in the universe, and you need to find yourself and your place in the universe in order to be happy".

You can see this in films like 'The Art of travel' where a high school grad decides to embark on a jungle adventure after his carefully laid plans go astray, helping him find himself in the process (or rather a new self he didn't know he had). Or in road trip films like 'Paper Towns', where the protagonist realises that life is more than plans and chasing dreams and should be more 'live in the moment', or 'The Secret Life of Walter Mitty, where the protagonist experiences something he missed out on in his youth that now 'completes him' and 'makes him whole again'. I wasn't impressed by the Walter Mitty film. It was beautifully shot, and I get the appeal that a certain section of society have for it, and that some people who feel like they missed out on travel in their youth might identify with the main character. A lot of these films are films that I might have identified with 10 years ago, but not now.

I'm not negatively criticising this narrative. In fact I don't have much to compare it to since I'm not aware of how other cultures represent travel in their media.

On Art

noreply@blogger.com (Daniel D'Mello) — Thu, 7 Apr 2016 20:19:00 +0530

A friend and I visited the Tate Modern on the weekend. The last time I was there, London was hosting the Olympic games. My visit then was brief, just a short walk around and a quick photo, like most first-time tourists in the city. This time we went specifically to see the Alexander Calder exhibit. The American sculptor worked mostly with wire, constructing familiar faces and figures and eventually more abstract forms.

-------------------------------------------------------------------------------

I don't often go to galleries and art exhibitions, which is peculiar given that one of the first things I do in a new city is see the museums and galleries. I suppose it's just something I do to tick things off of my list, to make sure I'm using my time well and not missing out on anything society deems it appropriate one experience in a new city i.e the typical tourist's attitude.

I also think it's partly due to my minimalist approach to lifelong learning. I have a fixed set of interests, like learning, behaviour, psychology, data science, etc., and I tend to surround myself with stimuli that pique my interests in these areas. The fine arts, though beautiful, have never really inspired in me any thought or action whereby I improve in any of my interest areas and so to save time I minimise the time I spend on what isn't of obvious utility to me.

Perhaps this is a failing on my part. I suppose there are narratives in the arts that reflect on the sciences and philosophy that maybe enable us to interpret our interests from a newer richer more useful point of view. Maybe I just haven't discovered this yet. In all fairness, I do try to spend as much time as I can when I'm at a gallery actually trying to reflect on and interpret what I see in the context of my own interests.

The problem is, and I have noticed this when I was at the National Portrait Gallery in both London and Edinburgh, is that there are no obvious stories being told or narratives being fed into by the pictures. Yes, there's a bunch of pretty pictures arranged chronologically or by artist on the walls, but after a while they all tend to blend together into a pastiche. I find that I'm not interested in something that holds no value for me because it doesn't seem to rearrange what I already know in a new way that inspires me to act or think differently.

Perhaps art would make more sense if, like science, exhibitions focused on modelling the work of one or more artists in terms of specific narratives that make sense to the viewer, rather than treating the viewer to a boring history lesson. A case in point would be the recent Big Bang Data exhibition at Somerset House. I could tell the exhibit designers had a lot of valuable raw content to work with, and they chose to design exhibits in order to fit a number of narratives, though personally I found it a bit left-leaning. Nevertheless, it was interesting.

-----------------------------------------------------------------------------

Which is why I was glad I visited the Alexander Calder exhibit. Unlike my previous underwhelming forays into the world of art, I was able to see not just art itself but expressions of love. Here was a man who truly let himself be inspired by assorted phenomena, many of which are of interest to me, and worked them into what was then a novel medium. This is way more interesting and relevant to me than the evolution of brushstrokes across canvas over the last 500 years in Europe. I'm fascinated by the thought process connoted by humans through physical expressions of things they are interested in, and three dimensional physical objects seem to appropriate this for me in a way that paint doesn't.

Among other things that struck me is how artists tend to come from families with other artists. I wonder if there's room here for an experimental study. If this is true, what moderates the phenomena of a artist emerging from a non-artistic family, apart from genes and interest in the arts? Is it familial support, money, independence, society, culture, stability?

Another thought I had was how frequently artists tend to be inspired by each other. Calder for example was inspired by Piet Mondrian. I'm sure there are other examples.

Calder seems like a guy who loved to play with objects, first wire to denote actual objects, and use them in performances, and later to denote static and then moving abstract images, some of them scientific in nature. I particularly liked this because I was able to draw parallels between Calder and Jan Švankmajer, who worked with claymation and puppets to tell beautiful stories.

The use of art to reflect scientific phenomena is a separate field in itself, there are so many examples of scientists, programmers, designers, artists and even landscape designers working on science-inspired projects, and these appeal to the geek in me.

-----------------------------------------------------------------------------------

We also saw the Performing for the Camera exhibit while we were there. Also very thought provoking, I can still remember that photo of Ai Weiwei breaking a two thousand year old vase, to protest government control. I get it. If your country is corrupt to you, then the objects it uses to symbolise it's apparent greatness mean nothing to you, and a great way to protest this is to destroy that object, similar to flag burning.

I doubt that many people in a country like India would understand an act like that. Burning the Indian flag would be seen as tantamount to treason or sedition, on the basis that the ideals that the flag represents are indisputable, which is of course false. Nothing is beyond protest. And any attempt to push back and prevent any disagreement with the establishment will only concentrate the opposition.

You see this happening consistently across history. The status quo is disrupted by an outside group, the two groups struggle until there's a new order which becomes the new establishment which is then challenged and so on. I think countries would benefit from taking the long view.of things, seeing as how everything that is happening now has happened before. A better understanding of power structures and leverage would actually serve to help reduce conflict times and speed up progress.

How to convince an atheist of God's existence

noreply@blogger.com (Daniel D'Mello) — Wed, 23 Mar 2016 05:25:00 +0530

How do you convince a non-believer that God is real? Fight them on their terms. Atheists like rationality, experiments, data and hypothesis testing. So devise a study that has predictive validity. It's that simple.

Make a specific verifiable prediction about the future whose outcome is not subject to vague or multiple interpretations apart from a religious one. A specific prediction that is so unlikely to occur except in the event of supernatural intervention. And if it were to occur then the only possible inference would be the supernatural. This is your hypothesis. Then devise an experiment to test this hypothesis, with controls.

For example, here's one hypothesis to prove that God is real and he answers prayers - over time t, a clear majority of n number of people suffering from a terminal illness who pray to be healed, will be healed from that illness. As a control, we can compare the results from this study to those from a group of n number of people with a terminal illness who do not pray to be healed. If the death rate is significantly lower in group A, then prayer works and God is real, if it is more or less the same in both groups then prayer has no effect and God is less likely to be real.

Also, the number of people who pray to be healed and die, or the number that don't pray to be healed and live should be so minuscule as to be negligible. In other words, for us to know that prayer works, the majority in both praying and non-praying groups should be true positives and true negatives respectively. And the number of false positives and false negatives should be tiny.

This is a good hypothesis because it separates a real effect from what would be chance or randomness but which might be confused for something else by believers.

One person being healed after prayer is not valid or reliable evidence in itself. It could be due to a number of factors, some of which could be as yet unexplained. A true effect would be visible over a group of people over time under similar conditions. And even if this happened, if a large number of people got better repeatedly after praying, you would still need to compare that effect to a similarly large group of people who got better repeatedly over the same time period without praying. By controlling for this one variable, you would ensure that prayer alone and not other confounding factors were at work.

No rationalist will every accept the results of a study as proof of course. You never 'prove' anything in science. You merely suggest that something is more or less likely. You goal here is to conduct an experiment that demonstrates that obtaining the results you obtained by chance alone would be so unlikely as to be as good as accepting that God exists (or perhaps an alien or the Matrix posing as God). This is as close that any scientist will come to believing in God.

I have yet to come across any data that shows that prayer works. All the evidence I am usually shown is no different from statistical noise (false positives or negatives) rather than true effects. But the experimental design is robust and I'm willing to change my beliefs and believe in God if that's what the model reflects. But we all know that's never going to happen.

---------------------------------------------------------------------------------------------

Counter claims like God shouldn't be tested, God chooses not to be tested, God does what he will, you can't understand the mind of God, God has a larger plan, etc. to explain why prayer doesn't always work are all invalid and pointless as they don't support God's existence. They merely offer data that is no different from random chance or statistical noise (if not worse), making you question why you would choose to believe in God to begin with. If all the evidence for God is so inconsistent as to be no different from evidence for no God, then why hold to such a theory?

And the argument that you're trying to measure something that isn't measurable or isn't supposed to be measurable is of course rubbish. That's the point of doing the measuring. If you can't measure an effect, then you have no reason to believe in it. And if you still believe in it, on what basis?

Of course, a true believer doesn't trouble himself or herself with such trivial concepts like evidence. The best way to Know is to surrender yourself to the unknowable, to have the humility to know that you will never understand what you were never meant to understand, to take a leap into darkness, which is a good way to delude yourself into believing anything, true or false, and which also comprises logical fallacies like circular reasoning and false premise reasoning by first assuming something is true in order to believe in it.

While this may bring comfort to people it is still the equivalent of a logical fallacy, and if something is the equivalent of a logical fallacy, it is as likely to be true as untrue. Which is why I default to not believing and people who prefer comfort default to believing. Two different frameworks for making sense of the world. Ironically, it's the scientific one that actually does more to prove the existence of the supernatural. Which is why we build verifiable models to make sense of the world. Religious models are just lazy. They explain too much by being deliberately vague. Great for self-delusional comfort, not a good way to derive insight.

One ball at a time

noreply@blogger.com (Daniel D'Mello) — Sun, 28 Feb 2016 21:39:00 +0530

I recently read Andrew Flintoff's biography - Second Innings: My Sporting Life - and was quite taken with his descriptions of the ups and downs in his playing performance over the years. Yes, there was alcohol, intrinsically linked to it all, but there was also his own psychological state, choppy and uneven. Reading about his mind made me contrast his mental make-up with those of players in the Australian team, who seem more naturally aggressive in the most nonchalant way, almost childlike in the way they approach life.

I don't know much about sports psychology, but It seems to me that you're at your best when you're simply playing and not thinking too much about playing. Flintoff was at his best when he was on a mood high, happy with his place in the world, playing without care, visualising himself as a giant, fearless, untouchable and taking it one ball at a time. I think it's an attitude thing. It might be better for your performance to stop thinking about your performance, and just live in the moment, one ball at a time.

Perhaps that's why the Australians are so good at cricket. That and trust. you've got to have good camaraderie with your team mates, coach and management to put you in a good mood and clear your mind. But other that that, there's an element of not caring about the outcome of a match that I think helps. Because really, you have no control over much, less so the outcome of a sports match, and the sooner you accept that, the better off you will be. Because then no matter how good or bad the match outcome, you see no reason to blame yourself, to doubt yourself. You did your best, everything else was out of your control, better luck next time.

And I suppose the ability to see things in the larger perspective helps - it's just a match, there are people on this planet with real problems and bigger issues, you're just swinging a bat or throwing a ball. Just do the best you can do at that moment and let everything else sort itself out. If you really suck, they'll replace you with someone else. That's their problem, not yours.

I wonder if extending this attitude to business helps. Maybe the most successful business professionals are those who don't act one way or another but simply live out their natural mental states at work. Focus on the job at hand, let everything else sort itself. Whether the organisation lives or dies is not your immediate concern. Leave that to someone else. Do a good job. Be the best you can be. How your work fits into the larger scheme of things is a useful way to think if it's part of your job, if it will help you get better at what you do, but if it's only going to fill you with self doubt and stress, then what's the point? I guess that works if you're a specialist and not the owner of a company, because then you're going to need a slightly different attitude to your work. Or maybe not. Perhaps even owners just need to tackle one challenge at a time.

On Hiking Differences

noreply@blogger.com (Daniel D'Mello) — Sun, 7 Feb 2016 07:42:00 +0530

The biggest difference between hiking in the UK and South/West India is the investment in equipment.

In Mumbai, and I suppose Pune, Chennai, Bangalore or anywhere in South India, you can afford to be a minimalist when you hike i.e you don't need a lot of stuff. All I used to wear on a day trip from Mumbai was a t-shirt, non-denim trousers, sneakers and a light raincoat. I'd swap the trousers for shorts if it was a shorter hike, involved a beach, flat open ground, or involved wading through water. I'd stick with trousers if it involved forests, shrubbery, thorns and mosquitoes. The fact that most of my hikes took place in the monsoons didn't matter. It was still humid enough to warrant the bare essentials. I did carry a wool pullover on overnight trips just in case it got cold. It was the only time I ever used the pullover normally stashed away in the back of my wardrobe in Mumbai.

I usually carried a light raincoat. Not among the most durable of apparel, it did its job, which was to keep the bulk of the rain off my body and cotton clothing till the end of my hike. I know guys who hiked shirtless. In a humid monsoon hike, maybe polyester shorts and shoes are all you need. I wore a simple hat to keep the sun out of my eyes and the rain off my glasses. Most people didn't. They found it too hot, irritating or distracting.

My shoes were initially everyday sneakers. Yeah they weren't the best for rough hiking, but they were great for most of my hikes that involved flat trails. I switched to Woodlands, which helped with longevity. Again, not a priority for casual hikers who stick to flip-flops or sandals. I never considered wearing gaiters. No idea if you can even buy them in India. We just considered water in our shoes a normal unavoidable thing. Gaiters can help keep your shoes dry to an extent, but not when you're shin deep in a flowing river. My socks were normal cotton ones. I never needed insulating ones. There was no cold to protect against. In hindsight I think wearing thick or double pairs of cotton socks would have meant less damage to my toes.

Contrast this with the UK, where people usually wear professional branded light stretchable hiking trousers that wick rain away. And waterproof overalls over wear over your trousers in heavy rain. And hiking trousers with thick lining on the inside in case it's a winter hike. Or perhaps just thermal on the inside. Or maybe lycra running pants. As long as you have a base layer. The more expensive the better the quality. Depending on your budget, you can buy anything from £5 thermals that are 50% synthetic, to £25 thermals that are 100% synthetic. And professional hiking T-shirts that wick sweat away. Short sleeves in summer and long sleeves in winter. With probably a base layer underneath in winter. With a mid layer and jacket on top. The weather dictates what you wear.

You always carry a rain jacket, preferably one made of weather-proof Gore-Tex for toughness. Or one that's simply weather resistant. Cheaper, but shorter-lasting. You could wear a mid layer like a fleece jumper if it's colder. A medium or heavy fleece in cold weather, or a medium fleece with a jacket if it's snowing/raining. A micro-fleece in autumn, or a jacket on top if it's snowing/raining. Or just a jacket with fleece lining on the inside to combine the best of both worlds, unless it gets warm and you'll have to either keep it on a sweat, or take it off and freeze. This is why layers are useful. Thermal monkey caps or hats that protect your ears against the cold and wind are common. And so are wide brimmed hats that protect against the sun. And hiking sticks to keep your balance and take the pressure off your knees on downhill climbs.

And then there's the shoes. You get weather-resistant hiking shoes (Gore-tex again). And professional thick hiking socks. I usually buy my shoes one or two sizes too large and then wear a couple of thick hiking socks to protect my toes and keep my toenails. The cushioning helps. If you don't hike often, you can always reuse your running or gym equipment. I've seen a lot of people show up for hikes in tights, running/sports jackets and running shoes. I suppose this is OK for day hikes on easy ground. Running/sports clothing tends to be fragile as it's made for quiet straightforward runs along city streets that only involve sweat, cold and light rain. On a challenging hiking trail involving thorns, rocks, stretching, mud, sleet and hours of continuous use, they'd fall apart.

And then there's the brands. In India, we mostly wore what we had lying around. My total annual clothing budget for hikes was zero. In the UK, my hiking trousers are from Craghoppers, my fleece from Pierre Cardin, my jackets from many assorted places, my hat and gaiters from Karrimor, my shoes from Crivit, my socks from Gelert. It will get worse if I do winter hikes. Other common brands are Merrell, Patagonia, Berghaus, Rab, The North Face, Regatta, Marmot, Paramo and Hi-tec. And then there's the walking poles, head torches and energy bars. You could spend hundreds of pounds a year on equipment.

On Null Hypothesis Significance Testing, P values and the Scientific Method

noreply@blogger.com (Daniel D'Mello) — Sun, 3 Jan 2016 19:20:00 +0530

Hypothesis Testing

Hypothesis testing is essential in science to determine the presence of an effect. A technique commonly used is NHST, which tests if the data points in your alternative distribution are representative of the normal distribution i.e if your data distribution is different from what would be considered 'normal', and assigning a p value to your data. If the Mean in your data sample is different from the one in the normal distribution, this might tell you that your data is not simply a random sample but that an effect (your variable) is present.

We conduct Hypothesis Testing by comparing our alternative hypothesis against a null hypothesis. You either reject or fail to reject the null hypothesis (double negatives can be used in statistics - not implausible, failed to reject, etc.).

Failing to reject the null hypothesis - This does not mean that the null hypothesis is true, only that this sample does not show that the alternative is true. Not rejecting a position like the null hypothesis does not mean that we're saying it is correct.

Rejecting the null hypothesis - This does not mean that the null hypothesis is false/not true. Neither does it mean that the alternative is true. It just means that this sample shows that the data is different from the null. Another sample might not.

These two points above are important to understand because when we look for effects in data, particularly noisy data which might be influenced by a lot of factors, you cannot simply reduce the act of spotting an effect to rejecting or failing to reject a null hypothesis. This is because the null hypothesis is almost always false.

When you sample from a population, it will be a coincidence indeed if you get the exact same means in both your experimental and control samples. I think the more important question to ask is how much of an effect is present and under what conditions will it vary. Statistics is no substitute for thinking. You need to decide what an important effect is.

Other points to remember -

- If you have a research question, circle around the problem, address it in different ways. Don't frame it in one specific manner and pin your conclusions on a null hypothesis to be tested.

- Hypothesis testing does not have to be applied to all questions. You can have one-off events worth studying that do not need falsification.

- It's OK to conceive your hypothesis after you have conducted research but it should be before you have analysed data statistically (more on this later).

- Hypothesis tests are always about population parameters, never about sample statistics. We always use the sample data to hypothesise about the population mean, not the sample mean.

- Hypothesis testing and significance testing are different things. Hypothesis testing or Null Hypothesis testing is about rejecting or failing to reject a null hypothesis, Significance testing is about assigning a p value. We commonly use these two together in a hybrid called NHST, which is controversial.

Null Hypothesis Significance Testing (NHST) and P values

In order to conduct a hypothesis test, we usually assign a significance value, a threshold on which we decide whether to reject or fail to reject the null hypothesis. This is how the NHST methodology works, but it has drawbacks, like a dependance on the p value. A p value is supposed to quantify the strength of evidence against the null value. It tells you how unusual the occurrence would be if it was due to chance.

The p value is the probability of observing a sample statistic like the mean being at least as extreme/favourable as it is in this sample, given our assumptions of the population mean.

p value = P(sample mean being as extreme | assumption about population mean)

It is simply the probability distribution on a normal normalised distribution like a Z score table (you can find it using the pnorm function in R). For example if you test two groups of people and group A gets 5 and group B gets 7 and you want to see if their scores are significantly different from each other, you subtract the differences and get 2 and then decide if this is significantly different from your null value, whatever it is (probably 0), given a certain standard error (Remember that all statistics is essentially a test statistic divided by the error in that statistic).

One way to do this is to be so immersed in your subject matter, be a complete expert at it and have full subjective contextual knowledge that you know subjectively if a difference of 2 really matters, if it really translates to real world significance. Remember that real world and statistical significance are two different things.

In statistical significance, you would run your test statistic against a normalised distribution, assuming it follows one, and your data might just be deemed significant if you get a low p value. The low p value is supposed to tell you that the probability of getting this difference of 2 is low i.e on the lower end of one end of the normal distribution, given a null default.

There are a few drawbacks to using p values as indications of significance. This paper shows us the harmful effects of using NHST and confusing statistical significance with real life significance but I've included my own notes below.

- Significance testing tells you more about the quality of your study (variation and sample size) than about your effect size which is more important. Andy Field has written a very easy-to-follow chapter on this topic.

- As I said before, p values are the probability of observing what you observed given a null default, but the default is never null. The null hypothesis might always be false since two groups rarely have the same mean. How then do you make sense of how probable your data is?

- The p value is conditional on the null hypothesis. It is not a statement about underlying reality. Even if it is accurate, the p value is a statement about data when the null is true, it cannot be a statement about data when the null is false.

- A p value is not the probability of the null hypothesis being true or false. The p value is the probability of extreme data conditional on a null hypothesis.

- It is not the probability of a hypothesis conditional on the data. P values tell us about our data based on assumptions of no effect, but we want a statement of hypotheses based on our data. To infer latter from a p value is to commit the logical fallacy of inverting conditionals.

- P values do not tell you if the result you obtained was due to chance, they tell you if the result was consistent with being due to chance.

- p values do not tell you the probability of false positives. The sig level (not the p value) is the probability of the type I error rate i.e P(Type 1 error) or P(reject | H0 is true).

- This paper does a good job of expanding on my points above, listing a lot of the common misconceptions about p values and NHST. Highly recommended.

- If you're studying a non-stable process that spits out random values, p values are not meaningful b/c they are path dependent. In these cases, the p value isn't meaningful b/c it is a summary of data that has not happened, under assumptions that further data will follow a certain distribution.

- People use 0.05 as a significance level, but need to remember that hypothesis tests are designed to call a set of data sig. 5% of the time, even when the null is true.

- Many studies show that you have a a very good chance of getting a significant result that isn't really significant with a significance level of 0.05 (about 30% of the time). This paper in particular does a good job of explaining the high false discovery rate using a significance level of 0.05 and compares it to the screening problem, and this article summarises the points well. You can use a lower level like 0.001, but it really is up to you to decide what is statistically significant.

The Scientific Method

All of this tells me that it is best, when tackling a solution to go back to the philosophical foundations of why we do things.

Note that you only create a theory or hypothesis after you have evidence. Theories have to be based on evidence, preferably good data-driven evidence. You can't first make up a theory and then look for evidence to confirm or falsify your theory. This is how superstitions and pseudoscience are created. A deliberately vague theory will never be confirmed or falsified, only made to look unlikely. While quantifying how likely or unlikely the existence of an effect is, is the point of science, doing so is a waste of everyone's time if the effect was made up to begin with, so don't do this.

If you see something weird you can't explain, you don't automatically give it a name. That's merely classifying a phenomena, putting it in a box that represents what you already know of the universe, which is incomplete. And your classification system or model or framework could be wrong. You need to do more. It is best to sit on the fence, admit your ignorance, and keep exploring, digging and asking questions of your phenomena, all the while building better and better models to explain it and make predictions. This is preferable to classifying your phenomena in terms of some-pre existing narrative that fits your own socio-cultural context, which would be a failure of critical reasoning.

I see this all the time. Once people identify with a narrative, everything they see will serve to strengthen that narrative. Supporters of a political party do not support that party because the evidence led them to support that party, they do so because of other reasons, like values that they identify with. But once the decision is made, evidence doesn't matter. We are slaves to narratives. Everything that follows is confirmation bias.

We use models because of their usefulness, not because they are correct. It seems to me that the best way to tackle a scientific question or puzzle is to first do exploratory research, just lots of multiple comparisons, or A-B testing, and obviously we wouldn't use p values here. We look at our exploratory data, at possible trends we see and that might or might not be true, that might reflect some underlying connections, and then create hypotheses based on what we've found in the data.

Here is where we switch from exploratory to confirmatory research. To confirm or falsify our hypotheses, we need to run experiments, which can involve hypothesis testing. And we have to gather new data for this. We cannot use the same data set for both exploratory and confirmatory research as that would be cheating ourselves and would not be scientific.

We pre-register our experiments so we can't change our minds later and claim we were always looking for what we ended up finding. This is called the garden of forking paths or researcher degrees of freedom or p hacking - You can only test 1 hypothesis, not 20 and then report only 1. Or drop one condition so you get a sig. p value of < .05.

There are really millions of variables that can correlate significantly with each other. Which is why we get significant correlations when we generate hundreds of 10 number strings of random numbers and then compare two strings. When you compare enough variables, you will find significant results. This is noise. This is just how large data works, or data without theory, or data with a theory that is ad hoc or made up and not evidence based. This is how superstition works. You need to look beyond this, to see if any of these correlations or effects are consistent and not merely noise.

So we conduct our confirmatory research, get our results, and then replicate to see if the results hold. Replication ensures that we confirm that the effect is real and wasn't just a coincidence. Also, keep in mind that if your hypothesis was based on a solid non-noisy phenomena or theory that that you had good reason to believe existed or was true, then replication should merely help ascertain this one way or another and not be a threat to you. It should all be part of the process of good science. If your effect was made up to begin with, or was noisy, then no amount of replication is going to help falsify something that never should have been investigated in the first place. in this sense, the original experiment bears no special status over and above the replication. They both need to be treated the same.

----------------------------------------------------------------------------------------

This then is 3 different experiments that we have conducted to find one effect. And where do p values come in? I think you can use them for confirmatory research, but only to tell you about your sample data distribution, about the probability that the data is consistent with chance, under repeated attempts. But you cannot use p values to tell you about your hypotheses. From what we've seen, p values cannot do that. They were not set up for that purpose and they don't work that way. You should be able to tell what a truly significant result is in your study without p values, or by looking at other statistics. Or maybe using Bayesian statistics.

On Happiness

noreply@blogger.com (Daniel D'Mello) — Sun, 3 Jan 2016 02:43:00 +0530

I've been thinking about happiness recently, which is probably something that someone who is truly happy wouldn't do. Happy people don't think about or look for happiness. They merely live out their happy lives as normal. But over-thinking things is part of who I am, and it brings me an extreme sense of satisfaction, which I suppose is different to happiness but still important.

I meet a lot of expats in London. International working professionals here on a contract. They all come here for a change, to lead a better life, to make more money, to travel and see new places, or other reasons that they claim brings happiness. And I wonder how many of them are happy. Whether this is a useful question to ask is something I'll get to later. But lets say it is. Lets say happiness is important. Do people who move here for work end up happier than they were in their own countries? I'm not sure. A lot of them feel like they're merely chasing happiness, like they're still searching for something that they'll never find, or that they've only found temporarily until another happiness goal catches their fancy. I'm not sure.

There's this TED talk that says that happiness is the mostly the quality of our relationships with other people, and I'm inclined to agree with this from the point of view of my own personal context. I personally derive a lot of happiness from good close personal relationships and shared experiences with family and friends, though I also think that other factors help - like having low expectations about certain things, having a pragmatic view about bad things that happen to you, having a positive attitude towards everything, and not tying your ambitions and career goals to happiness. Work for money, create for love, right?

This other talk separates happiness into synthetic and real. Synthetic happiness comes from doing what you are told will bring you happiness, accepting things you cannot change, and rationalising bad things as normal and happy. Also, people like things more when they think they're going to lose them. It defines real happiness as when we get whatever we want, which is something I don't get because we never get what we want and will constantly be striving from one happiness goal to another i.e one temporary island of happiness to another temporary island of happiness. It could just be semantics, but this isn't real happiness to me, this is just temporary contentment. But I guess this is happiness to a lot of people in the western world, who feel like they need to be in control of every aspect of their lives, and that control brings happiness. I take the other view, which is that since so much is out of your control, you can only be happy by letting go of it all and just do things you enjoy without hurting people, and take everything else in your stride without imagining that the universe is conspiring against you. Which is where the synthetic happiness come in.

Then there's 'The Geography of Bliss' by Eric Weiner. A somewhat humorous look at why people in some countries are generally happier than others. Some of Weiner's book is of course typical western narrative tropes and hyperbole - Columbus, China's greed is bad, etc., but i picked up a lot of interesting points. Weiner visits the happiest countries on Earth to find out what makes them happy, while not confusing correlation/association with causation. Just because happy nations are characterised by certain factors doesn't mean these are causal factors, it could be the other way around.

The happy countries -

- The Dutch have things taken care of, and have permissive attitudes towards sex, drugs, etc.

- The Swiss are less tolerant than the Dutch, they have rules, boredom and nature. They are not ecstatic joyful, but content. They also have cleanliness, punctuality, things taken care of, they don't provoke envy in others, but suppress envy by hiding their wealth. They are surrounded by beauty and nature. They trust their neighbours, and having a sense of history and where they're from. They have fewer choices.

- The Bhutanese don't have unrealistic expectations. They don't try to be happy or try to achieve it. They don't talk about or analyse it. They don't ask themselves if they will cease to be so. Ignorance is bliss. There is also a lot of death, which gives you a different perspective on life. You develop a new way of seeing things after living with it. They are poor, but that doesn't matter. Money is only a means to an end. It is trust in people and institutions. Material wealth doesn't become so important.

- The Qataris leave everything to God. Maybe happiness come from beliefs, not necessarily religious beliefs. They belong to one tribe with many rules, that allows you to have no rules outside it because you just won a lottery and can do anything with the money. You are happy as long as you are a high ranking member of this tribe. You don't need ambition or high expectations. The money takes care of everything. If this culture-less life is to your liking, you are happy. But money isn't everything - it has diminsihing returns. You will always crave somethign else.

- The Icelanders are naive. They are free to try and to fail. They have a conection to their language. They are a small country, feel kinship to each other, protective of their well-being. Enjoy writing. Not affected by SAD. Have multiple identities, no envy of others. Suppress envy by sharing everything with others. A sense of self actualisation and the freedom to do what you want. they are free to share ideas without copyright. Self-delusion might be good - there's no one to tell you not to do somethign or express yourself. They constantly fail and create rubbish, but are happy doing so.

- The Thais have mai pen lai (never mind), jai yen (let it go), sanuk (fun). They have fun at work instead of the American work hard, play hard mentality. Their fun is interspersed throughout the day rather than regimented and taken too seriously. They don't take things too seriously. They don't think about things like happiness to much. Ignorance is bliss? They smile a lot.

The unhappy countries -

- The Moldovans have a lot of envy, are relatively poor compared to their European neighbours - poverty breeds envy of other's riches - there's also lack of trust - if something goes wrong, it is not their responsibility to fix. There's a feeling of powerlessness, helplessness.

The somewhat happy countries -

- The British believe in muddling through, getting by. They are reserved, not tactless, are afraid of offending people, don't hug, are a country of grumps. Does culture impede happiness? I don't think it's that simple. Having lived in England and Scotland, I think people here are definitely happy, they just don't show it (btw, don't ever introduce yourself right away in an English pub - rookie mistake). But I'm not sure why they would rank lower than the other countries.

- The Indians are a mixed lot. The ones who are happy believe that life in an act, and don't take it too seriously. New tech cities are both the problem and the solution. People have long long work hours, poor work life balance, and then special workshops and ashrams to fix them. Calcutta's poorer are happier than America's poor - stronger family ties? (Btw, flattery can get you an interview in India, and much else). He says nothing about unhappy people in India. I guess it could be a lack on trust in your neighbour and public institutions. All the happiest people I know in India derive happiness from relationships in their communities, but not necessarily within communities. Indian diversity can be comforting, but I think people's biases and ingroup-outgroup mentality combined with their narrow-mindedness about culture can serve to increase create distrust and hate.

- The Americans are constantly searching for happiness. Their unhappiness could come from unrealistic expectations. Self help books teach them to look inwards not outwards towards relationships that really matter. Maybe you nee to commit to a place or people to be happy, you can't always have one foot out the door.

What happiness isn't -

To quote the book, "Happiness is not feeling like you need to be somewhere else or doing something else." But I think that's your other goals, which are fleeting and constantly changing. I think it's fine to have them, we all have career and self-fulfilment goals and wants, and striving to accomplish them is fine, but our success or failure in said exercise shouldn't make a difference to our happiness, if in fact happiness is more important.

It's not about ambition or success. Failure might happen despite your best laid plans, and while success can bring you satisfaction, I feel it's the journey, the striving for success that brings you happiness.

Knowledge doesn't necessarily make you happier, though it has other obvious advantages. So is ignorance bliss? Not necessarily, in my opinion. It's not about knowledge vs ignorance w.r.t happiness. Neither is a factor, your happiness depends on other things.

It isn't about money or material wealth. Money helps, but just a little, it doesn't guarantee lasting happiness. Law of diminishing returns.

What happiness is -

We constantly try to synthesise happiness, we think it is something to be found. Perhaps it is more a thing to be created, or a state to be evolved into. To my understanding, it is having close personal healthy relationships with friends and family, living in a society with a lot of trust, and no envy, uncertainty or fear, and finally, having a pragmatic outlook on life, understanding that events are unpredictable, but having something to look forward to and doing your best anyway, about having a sense of not-wanting. Living among a homogenous society with reliable public institutions and like-minded people also helps. The closer knit the community the happier you are, as long as you subscribe to the cultural mores of that community. Tough luck if you don't. Perhaps that's why people move away. To me, certain environmental conditions also matter, like living in clean cool quiet surroundings with access to good food and being intellectually stimulated.

The Science of Everyday Thinking on EdX

noreply@blogger.com (Daniel D'Mello) — Mon, 21 Dec 2015 01:01:00 +0530

I had the pleasure of completing 'The Science of Everyday Thinking' on EdX recently. The course deals with a lot of stuff i've been thinking about for the past few years, so I noted a lot of my thoughts.

Illusions

The course begins by stressing that it is really difficult to put yourself in the shoes of others. We over estimate the abilities of others to know what we know. An example of this is when we tap out a song on a table. We expect 25% of people to guess the song correctly but in reality only 2.5% do.

We're great at pattern recognition, maybe even too good at it. Things float to the top of our minds that match our expectations, so we see real effects in noisy data, for example - a face on toast. We sharpen things to what we expect to see - the 'expectancy effect' - and level those that we don't.

The course also stresses on how faulty memory can be. Memory is not like a video camera. Every time we remember something we reconstruct past events in our mind. I have had personal experience with this when helping one of my classmates at uni with false memory experiments. It was interesting to see how people really believed that they had seen something when they hadn't. I do this to, which is why I now write down certain events immediately after they happen so I don't get sequences of events mixed up.

We exhibit Naive Realism - we think the world is as we perceive it to be. This is wrong.

We exhibit fundamental cognitive error - we tend to underestimate the contribution of our beliefs and theories to observation and judgement, and fail to realise how many other ways that they could have been interpreted.

Know Yourself

Planning fallacy - we are terrible at planning or judgement-making or self-assessment. Examples are driving, attractiveness & morals. Even though we fall on a bell curve for some of these, and 50% of the population falls below the median, we are incapable of accepting that we could be in the bottom half. Statistically speaking we all have to be under 50% at some point, but we will never admit it.

I've seen this first hand when planning my own goals. Many a time, I've planned out a journey assuming I'd be ready by a certain time only to find I've taken longer to get ready. I overestimate my own ability to be ready in time. It's the same with my learning goals. I keep subscribing to the belief that I am a super-fast learner and can do multiple courses at once, and I always end up struggling with too many things on my plate. I've learned to cut back and take things slower. No one can be great at everything. I've also seen this when proof-reading for foreign students au University. Students would be incredulous at the number of mistakes I found in their writing and the amount of re-writing that was required. They thought their grammar was decent, when it wasn't. Their unrealistic expectations were tied to incorrect evaluations of their own abilities.

The false-consensus effect - we overestimate the extent to which our beliefs are typical of those of others. We believe that other people generally think like us. Important to be reminded that this is not the case.

People don't even know what makes them happy. The true reasons people are happy are usually different from the reasons they provide. I need to do a separate post of happiness as I'm currently researching this.

Job interviews are usually bad because of confirmation bias - interviewers see what they expect to see. They make up their mind about a candidate soon after they meet them and then only ask questions that confirm their beliefs. Structured interviews, where every candidate is asked the same question, are better.

People tend to exaggerate the long term emotion effects that events have on us. In reality, emotional trauma can have bad effects on us but for the most part we tend to over-emphasise their effects.

People have a strong 'order effect' when selecting from an identical pool - they mostly pick what's on the right. And then they don't believe the reason why - which shows that we don't know ourselves well. We don't even know why we make certain choices.

Intuition and Rationality

Kahneman differentiates between System 1 and system 2 thinking i.e intuition and rational thought.

The Anchoring Effect is powerful - but be careful of noise in the data.

The Representativeness Heuristic - the frequency or likelihood of an event by the extent to which it resembles the typical case.

But from a practical point of view, do be careful of thinking too statistically - in the Rudy the farmer example, where there are far more farmers than lawyers, statistically it would make sense to pick farmer as the option but a bit more context would propbably point towards one of the other options like lawyer.

Learning

I really enjoyed this part of the course as I could take away more from this part than any other. Keys to learning better are to -

Distribute practice over time - spacing helps.

Set calendar reminders.

Use Retrieval practice - instead of merely re-reading material, cover and try to recall it.

Learn by doing - practice and discuss the content.

Vary the settings in which learning takes place.

Relate learning to your everyday experiences.

An important thing to remember is to not mistake fluency with learning. If you're finding a new topic too easy, you're probably not learning it well enough. You only think you understand it.

Experiments

Beware the Gamblers Fallacy.

Apple's shuffle feature - people don't understand how randomisation works, Apple had to make their product less random so people would perceive it as being more random even though it wasn't.

Finding Things Out

Many phenomena are simply examples of Regression towards the Mean - things balance out. This is more apparent when there is more noise in measurement.

Also, Post hoc ergo propter hoc - we assume a causes b because b followed a. It's kind of like those other common biases that make us believe in superstitions, like correlation is not causation, or false premise reasoning, or circular reasoning.

Experiments show that for most competencies, there is no diff between large and small class sizes.

Six leads to opinion change -

What do you really believe anyway?

How well based is your belief?

How good is the evidence?

Does the evidence really contradict what you believe?

What would be enough to change your mind?

Is it worth finding out about?

Extraordinary Claims

There are multiple ways you can interpret things.

Question your intuitions and be willing to give them up.

People tend to accept information that is consistent with their pre-existing beliefs at face value, but critically scrutinise information that contradicts their beliefs.

Health Claims

Pseudo-scientists tend to make ambiguous statements that you can contort to your expectations.

The Placebo Effect can be a false positive response, but most are Regression to the Mean. People seek help when they are sickest.

The Availability Heuristic - if a treatment turned out negative, you would never hear about it.

Like cures like - a diluted part of the disease can cure the disease - is a common false belief.

Natural is not necessarily better - arsenic is not good for you, indoor plumbing is.

Clustered disease is possibly the availability heuristic. You're confusing normal randomness and noise for an actual effect. You need to create and test a hypothesis to determine if a true effect like cancer clusters exist in a population.

Always ask - what about the other 3 cells? Given that you can have true positives, true negatives, false positives and false negatives, always look at the costs and benefits of the two ways that you can be wrong.

Applied Claims

For example - facilitated communication, forensic science, conspiracy theories, gun laws, gay marriage, asylum seekers.

The Expectancy Effect affects interpretation of forensic evidence like DNA. Experts who expect or desire to see something see the evidence in ways that are consistent with what they want to see - this is in part helpful, but can be disastrous.

People tend to focus exclusively on what they consider to be the evidence.

Belief in conspiracy theories is mostly cherry picking information.

False consensus effect - everyone thinks that everyone agrees with them.

Exploiting the Situation

There is not much correlation between personality and cheating, it is more about the situation. Certain situations can encourage honesty.

Social conformity, the bystander effect, attribution error.

We assume that the way we see the World is the only way to see the world and anyone else that sees it differently is wrong and we attribute it to their education, personal biases, propaganda, lower intelligence.

Milgram experiment - authority factor, diffusion of responsibility factor, channel factor (increase in shocks in incremental steps), no clear exit.

Nudging changes the channel factors to induce behavioural change.

Putting it all together

Be aware of your intuitions.

Have a healthy skepticism.

Simulate your future desirable performance in the present.

Test hypotheses.

Pick a few areas where you want to change what you're doing w.r.t thinking and personal biases, and focus on those.

Just because something is portrayed confidently doesn't mean it's true.

Read.

----------------------------------------------------------------------------------------------

I really enjoyed the course. I initially felt that the instructors spent way too much time on discussing personal biases and our inability to be objective and accurate with our perceptions and beliefs, and that they were repeating these points through the first half of the course, but I see now how useful and essential this was. Indeed, only good can come from these constant reminders.

Throughout the course, I was reminded of the biases people use to justify their superstitions and irrational beliefs, and why they won't change their minds even after being presented with evidence. For some reason or another, people will believe what they want to believe, and then pick and choose evidence to confirm that belief. They will see patterns where there are none because that is what they would expect of that belief. It helps if the belief is vague to begin with. This makes it easier to confuse noise for a true effect. They will assume that everyone should think this way. They will not understand that everything they see and interpret this way can be interpreted in many different ways by different people. They will not accept that their beliefs are a result of critical reasoning flaws or cognitive biases, nor be willing to test and verify their beliefs experimentally.

Learning by Context

noreply@blogger.com (Daniel D'Mello) — Wed, 22 Apr 2015 10:58:00 +0530

Education is not the simple direct process people think it is.

When you give a kid 10 encyclopaedias, he/she doesn't learn them all by heart. People, including children learn and remember by context. By memorable information.

I had over a dozen encyclopaedias and general knowledge books reference books when I was younger. I simply used look at the pictures and read a little of the text if the pictures looked interesting enough. That's a lesson for learning designers. Focus on visual information and cues as much as possible. As little text as possible.

But that's not all. You still have 500 pages in your book full of information that the reader really has no reason to be interested in. Which is why that information has to be put in context that's interesting to the reader. My books were full of information about dinosaurs, but I had no reason to remember this information until I began playing Top Trumps card games about dinosaurs. It was the same with animals, cars, bikes and football. Playing these card games gave me a reference point for these topics. Browsing through my humongous books now became a slightly more productive exercise, as I'd stop to read and learn a little more about dinosaurs, animals cars or bikes when a picture caught my eye.

This is why children need to be exposed to as many specific stimuli as possible. Let them develop their own likes and ideas. Then back these up with a lot of easy reference material. Both of these are essential for building knowledge. Just one is not enough. A person is more apt to learn about Ancient Greek mythology when they have watched a film or cartoon about it and then a book or Wikipedia. Wikipedia itself would be next to useless. Because there's no motivating factor in studying a lot of information. The film provides context, and reference points that the book builds on.

Interesting Links

noreply@blogger.com (Daniel D'Mello) — Mon, 13 Apr 2015 21:36:00 +0530

Help Net Neutrality in India

Who Does Peer Review?

Below the Earth's Surface

The Seinfeld Situation

Polandball

People are Animals Too

The CIA's Animal Spies

Best WWE Oversell

Research Methods is Exciting

Funny author stories

What is the most George Costanza-esque reason you broke up with somebody?

An introduction to Homeopathy

The man who destroyed America's Ego

Brain training is bunk

How one Japanese cartoon spawned a genre

I miss this guy's blog

Making Sense of the World

noreply@blogger.com (Daniel D'Mello) — Tue, 7 Apr 2015 13:51:00 +0530

Imagine the following three possible frameworks or models used to make sense of the universe, this world, and all behaviour and actions.

One

This universe was created by a race of aliens from another universe or dimension. Perhaps they exist outside of space and time (whatever that means). They either created or monitor all activity on earth. Perhaps they have ways of understanding and following human desires, or perhaps they interfere in human activity, according to their whims and fancies, or maybe not.

Two

This universe is a function of the Matrix. It is an artificial construct, a virtual reality built by citizens of the future, human or robot, as a giant experiment or project of value. As such, our existence is really just our consciousness responding to whatever 'they' want us to see. Our bodies are either plugged into machines somewhere in the future, or we only exist in digital form.

Three

This universe was created by the Abrahamic God. He exists outside or space and time. He hears and sees everything. He chooses to answer or ignore prayer. He has a plan for everyone. His ways are mysterious.

----------------------------------------------------------------------------------------------

These are your three frameworks. I made up two of them, with inspiration from the media. The third is a commonly espoused belief system. Here's my question - what makes the third model any less ridiculous than the first two?

Granted, my description of all three were brief. But you are free to build on all three. If your goal in choosing a belief system, framework or world view is 'does this make sense?' or 'does this prove useful?' or 'does this explain everything in this world?', then all three frameworks are equally useful.

Take any physical law or phenomenon like gravity or electromagnetism. Take any biochemical unit or process like cell or an organ system. Take any aspect of human or animal behaviour like jealousy or altruism. Any of the three frameworks could be used to explain these facets of our world. They can all be used to explain economic productivity, evolution, tsunamis and genocide.

All the three models are somewhat vague, and that proves advantageous. Being vague means the model explains more variation. The more specific the model is, the easier it is to undermine. You could explain away any bizarre phenomenon into the variation that the model accommodates. And since it accommodates everything in its vagueness, the model is never wrong.

Do you see the problem here? All three models cannot be correct. And these are just three. The number of models you could invent to explain variation in what you observe is infinite. How do you tell right from wrong?

'All models are wrong, some are useful'. We use models that useful to us. We don't really care how right they are. We use a model because we find it useful. Because it makes sense to us in our specific context. But if all three models are equally useful, then why prefer one over the other? Is it because only one is the product of historical thought and cultural evolution, and the other two are more recent and more clearly 'fake'?

We think prayer works because it works for us, and that's enough. We don't seem to care that historical data shows us that affliction and death rates for polio, smallpox and cancer have no relationship with prayer. That irrespective of how holy you were or how hard you prayed, if you had cancer in 1900, you would likely die. That cancer survival rates have more to do with the invention of surgery, radiation and chemotherapy than anything else.

Perhaps it is best to go beyond retrospective usefulness in picking the right model, seeing as how any speculative thought can account for all variation in observable phenomena. Perhaps we should stop asking 'how much variation does this model explain?' and instead ask 'how much predictive validity does this model have?'

Any made up theory can explain what you see around you. It doesn't matter if the model involves God, aliens, AI, time travelling robots, space monkeys, or a new scientific theory. All these models or frameworks can seem equally 'valid' or 'right' in that they have an explanation for everything, answers to all your questions.

To decide which model is right, or to create a better model, it seem much more intuitive to base that model only on the evidence you have, incomplete as it is, and then test it and continually modify it by making predictions, admitting all along the way that your model will always be imperfect and a work in progress.

p.s.

In 50 years, when we do develop a vaccine or cure for AIDS, the same people who call AIDS a punishment from God will be thanking God for answering their prayers and curing people. It's easy to validate any model using retrospective post-hoc rationalisation, especially if your model was vague to begin with, you never tested it by making specific predictions, and you're making it up as you go along, avoiding any attempt at testing your beliefs and instead picking and choosing facts that seem to fit into your pre-existing framework and ignoring everything else or considering it a test of your faith.

On Poverty

noreply@blogger.com (Daniel D'Mello) — Thu, 2 Apr 2015 22:48:00 +0530

A lot of us don't realise this but it is really difficult to get out of poverty, even if you really want to.

There are a lot of economically disadvantaged people out there who are smart and hardworking and have the means to remove themselves from poverty. And they do so very slowly. Because it's not an easy process. Let's say you work a double shift for six months and save up a bit of money, and then suddenly a family member gets sick, and of course they don't have health insurance because you can't afford the monthly payments. So all your savings are blown away. Or imagine you've been saving for six months and then just when you're going to use that money to get to the next stage, start a business, invest in something that will grow your money for you, you meet with an accident, or you're robbed, or you need to buy a new fridge or washing machine. Your savings are gone along with your plans. You're back to square one.

Because something bad always happens. This is what it's like to be poor. It's always one step forward, one step back, and so on. The rest of us don't experience this because we're rich, relatively speaking. We have a cushion. We have money in the bank that we can use to buy a new AC, fridge or washing machine. We always have enough money for healthcare. We have relatives we can borrow money from, networks of friends and ex-colleagues we can use to find a new job. We have cushions. The poor don't. These cushions serve to keep us from stumbling, they keep our careers moving forward and not grinding to a halt every time something bad happens. This is why I don't get how people look down on the poor as if it's their fault, like they are lazy. They aren't. No one likes being poor. They're working to get out of it. You just can't see it because of your privilege.

So how do you get rid of poverty? You could increase wages. Imagine a janitor in Sweden. He gets a minimum wage that's enough to afford a home. He's not rich, but he makes ends meet. Same with the UK or US. Now imagine a janitor in Mumbai. His wage, even if above minimum, would be nowhere near enough to rent a flat. So he lives in a slum. He saves more that way. Could the government enforce a minimum wage that's high enough so everyone can afford proper housing and not live in a slum? Sure, but employers would pass that burden back to customers. We would eventually pay more for items, and would want higher pay ourselves to cover the difference. Which isn't a bad thing. We would all earn more, and pay more more some things. For people doing menial work, their savings would be low but their living conditions would be decent. The rest of us would have higher pay and higher expenditure and our savings would be proportional. More importantly, we would all be living in a country with a higher standard of living, and no slums.

Or we could just leave it all up to market forces. The problem with this is that in a country with fewer opportunities, and less competition, employers can pay as little as they want, if they know there aren't any alternatives for you. They can always claim that people are free not to work for them if they find the salaries too low. This might be fair to the employer, but not to workers, because they live in a country with few opportunities by default, so they really have no where else to go to, and they can't all start their own businesses overnight because they are mostly disadvantaged to begin with. So they settle with being exploited because some pay is better than no pay.

That's how rich people like Donald Trump end up legally getting even richer by building large buildings in the Middle East using voluntary 'slave labour', people who are too poor to do anything else and who aren't even allowed to keep their passports. Is it their fault their country didn't give them enough opportunity? Is it their own fault they were not smart enough to get rich on their own?

If you want to live in a developed country you need to remove absolute poverty. Relative poverty will always exist in a capitalist system, and that's OK as long as inequalities don't create further absolute poverty or lead to monopolies that create status quo institutions that can lead to exploitation. You could remove poverty by raising the minimum wage, ensuring that everyone has a liveable income. This by itself will only do some good. In Indian cities like Mumbai, it will enable to people to live in better places, or let them grow their savings.

The government could just subsidise education completely of course. It already does that to a large extent. But that won't cure poverty on its own. If you waved a magic want and gave every Indian a PhD tomorrow, they still wouldn't have the ecosystem to use their skills. There would still be massive unemployment. You can't stop at education. You also need an environment that demands new skills, that serves as a market for these skills, so people can exchange their skills for money.

They would also need a market that enables them to finance themselves and create their products easily. You could reduce bureaucratic procedures and other red tape involved in growing businesses, and incentivise patents and loans, to encourage self-employment and innovation. In the long term, this would create more jobs, and in turn serve as a motivator for people to up skill themselves, which would get them higher salaries, and better lifestyles.

On International Politics and World Peace

noreply@blogger.com (Daniel D'Mello) — Wed, 4 Feb 2015 16:56:00 +0530

Back in the 90's when I was in school and the US imposed sanctions on India for its nuclear tests, we cried hypocrisy. How could the US punish India for building nuclear weapons when the US also stockpiles them? I've come a long way since then, but only after a lot of self-education, education which I unfortunately didn't receive in school.

Self interest and game theory

The thing about international politics and diplomacy is, no political action is a result of principles, other than those associated with self-interest. This is a historical fact. Countries do not build alliances or rivalries based on principles, they do so based on what maximises their own self-interest.

This can be modelled using game theory. Draw a checker box. Put one player on the x axis, and the other one on the y axis. On each player's axis, list the name of the interaction i.e. cooperating, trading, going to war, etc., with the the player. Now in each box on the checkerboard, enter the values of one players's action given the other players action. These could be positive, negative or zero values. For example, the value of player 1 going to war while player 2 is at peace might give player one a high payoff and player 2 a negative payoff. Whereas the payoff to the players if they both decide to trade with each other could result in each of them getting an even higher payoff compared to the payoff that one got by attacking the other. In this way, countries pick the box with the highest value for themselves. The cost-benefit equation is of course more complicated than this, as any political scientist or economist will tell you. Like a giant live chess board of life, each country has to look at the best way to maximise its own interests in real time. The main difference is that chess is a zero sum game, whereas in politics more than one country can win.

This is the real driver behind political policy and action. When you look at history afresh after having learned this, you find no reason to use infantile terms and phrases like "we're friends with this country", "these countries have always been friendly", "these countries are enemies", etc. Another thing you feel no need to do is to cry hypocrisy at the actions of other countries - "this country is being hypocritical".

When the US imposes sanctions on other countries for conducting nuclear tests when they themselves own an arsenal of nuclear weapons, this might be technical hypocrisy, but that's missing the point. The US does it because it's in their interest to do so. When you have nuclear weapons, it makes sense to prevent countries that are not aligned to you from obtaining the same weapons. It's just good strategy. Countries are only allies because it's in their mutual benefit to be allies. Because it pays to be allies more than it does to be something else. Here's another example - when the US commits to religious tolerance but backs Pakistan and Saudi Arabia with military aid. This isn't hypocrisy in a political sense. You say what you need to for votes, to define your value system (all countries run on value systems) but your actions have to be concurrent with good strategy, with game theory. In the end, self-interest wins out.

How countries evolved

If every country had to follow principle-based politics, the world would be a better place, as long as every country agreed to follow the same principles. But they don't. Countries that exist today didn't exist 10000 years ago. We started off as nomads and hunter gatherers. Over time, different groups of people came together because division of labour made sense. These different groups had different principles, but they all focused on maximising their payoffs, whatever they were. Groups with different resources decided to trade with others as they both needed resources that the other had. Some developed a trusting relationship based on reciprocal cooperation, they didn't need to safeguard themselves against each other militarily.

But what if there's a change of leadership and policy? Now one civilisation grows stronger, and begins to conquer more land and people. The other civilisation has a choice, do they ignore the first one, join forces with it, let themselves be conquered and assimilated, or turn into conquerers themselves to avoid being taken over? Also, military might isn't the only tool to use in your defence. There's also religion, which makes cultural assimilation easier, and economic conquest, where countries subjugate each other economically. Trusting your neighbour explicitly means giving them a chance to exploit you, and you won't do that if you think there's a chance that they will. Replay this scene for different groups across thousands of years, and through numerous conflict and death, we have the world today, fragmented groups with different value systems and ideals, terrified of being exploited or losing out. Hence, there's not much trust.

Self interest and trust

Which is not to say that trust doesn't exist. It does, but it comes about when it's in the players' self interest. The US and Canada can have a porous border because there's very little risk of a war breaking out between them because they have been at peace for so long and have reached numerous lucrative trade agreements in the process. If this peace was broken by say, America invading Canada and taking even some of their land, it would hurt trade, and Canada might align itself with an enemy of USA. It pays to keep your neighbours as allies to act as buffers between your other enemies. What we call international trust is ultimately all about money and security. Self-interest wins.

Here's another example - the Nordic countries being at peace. Denmark, Finland, Sweden and Norway all have porous borders, as do other European countries. The likelihood of a country misusing this trust and upsetting the status quo is low because the consequences would be dire. Border security would be strengthened, relations would sour, money would be lost, everyone would suffer. The payoffs from committing such an action wouldn't justify the costs. This is why the Nordic countries are at peace. They weren't always at peace. A long time ago they were at each others throats. But back then, it paid to conquer and kill each other more than it did to trade and cooperate freely. This also explains why Russia recently annexed part of Ukraine. The payoff (in terms of access to resources and trade routes) exceeded the cost (meagre threats from NATO?). Self-interests wins.

The situation today

Look at the two major power blocks today - the US and China. It used to be the US and the USSR. Preceding World War 2, the US was just another country gaining affluence through trade and innovation. Following World War 2, it emerged as a dominant superpower and its alliances with a number of European partners was sealed. But competition emerged with the USSR, whose economic and social polices rivalled that of the US. Here we have a case where countries have an internal value system linked to their economic systems, so they become economic competitors to protect their social systems. The US was terrified of communism, and the USSR was intent on spreading it. So they both embarked on policies of expansion. The USSR annexed and funded countries that embraced communism, crushing any opposition. The US did the same, backing fascist murderous dictators worldwide as long as they rejected communism. The devastation this wrought was immense in terms of human life. But it was in both countries' interests not to stop their activities, because then the enemy would have an upper hand.

You see this mirrored today with the US and China, with China funding infrastructure projects in a number of Asian and African countries in exchange for political support, while the US can only count on its bases like South Korea and Japan for leverage, in addition to NATO. China knows North Korea is a powder keg but continues to maintain friednyl relations with them because they can use North Korea as leverage against the US if they need to. It pays to keep them close as an ally. Which is why the US has a military presence in Taiwan and the Pacific. Self-interest always wins.

You also see this mirrored today with nuclear weapons. No world leader truly believes that these weapons are good, but they can't help keeping an arsenal as long as their enemies have them too. It's only the smaller countries with no ambitions of power that don't need nuclear weapons, but they are either aligned to a power block (like Bhutan), or are not threatened by one (like Oman).

Books like Isaac Asimov's Foundation series really open your eyes to these sorts of situations and decisions. You begin to see beyond the values that you were raised on, the values that countries should be run on, and see the world for what it really is, a blank slate ready for exploitation by power-hungry people ready to exploit anything.

Which is not to say that values don't matter. Of course they do. But they keep changing, and we need to be mindful of this, and ensure they change in a way that's best for all of humanity (if we are to take a humanistic approach towards existence). The Mesopotamian civilisation used slavery because it made sense to do so, values be damned. This doesn't make it right, but it made economic sense at the time, and later for thousands for years, until we decided that slavery was wrong. This didn't happen overnight. It took time. it's the same for universal suffrage, or homosexuality. Values change. But change takes time.

Attaining world peace

So how do you reduce international conflict and attain world peace? Again, you can use game theory to figure this out. Prevent countries from warring with each other by making it too costly (relatively speaking) to do so. People will always strive for power and self-interest. You can't take this urge away. You can only develop a political ecosystem in which acting upon the urge is too costly given more attractive alternatives. An ecosystem is which countries are incentivised to trade and cooperate peacefully with each other.

One way to do this is to build trust between all the different countries that currently exist. One way to do this is by removing any perceived threat between two countries and increasing trade opportunities. And you do this by economically developing every country equally. Remove economic gaps, invest in education and healthcare, make all countries economically powerful so they can serve as trading partners with each other. This serves as a status quo, a deterrent to attacking each other. Over time, this becomes trust.

Of course, this should work better if the countries have similar value systems, as differing value systems pose a threat. For example, the world's number one economy - China - is communist, while the US-NATO power block isn't. Both blocks trade profitably with each other, but mistrust exists. A common value system would probably remove this. Remember that power blocks are only formed as reactions against perceived threats from other blocks. The other way to attain world peace is of course to ensure that there is only one power block in existence - yours.

p.s.

This Crash Course World History series is extremely informative w.r.t observing patterns in group behaviour across human history. By watching a concise approximation of human cooperation and conflict across time, you begin to observe patterns in group behaviour. Watch if you have 20 hours to spare.

World history Part 1 - http://www.youtube.com/playlist?list=PLBDA2E52FB1EF80C9

World history Part 2 - http://www.youtube.com/playlist?list=PL8dPuuaLjXtNjasccl-WajpONGX3zoY4M

Movie reviews - Coraline, ParaNorman & The Boxtrolls

noreply@blogger.com (Daniel D'Mello) — Sun, 25 Jan 2015 19:48:00 +0530

I have been catching up on the films from Laika studio recently. Even disregarding all other factors with which we judge a film, the body of work they have produced with respect to only stop-motion animation quality is astounding.

Coraline (2009) is their best work yet. A visual masterpiece. And incredibly creepy. Worth spending money on for the visuals alone. The story was above par too. Most films have a typical plot line where a character is shown to desire something, then has obstacles put in the way of that desire, and spends the film overcoming these obstacles to achieve a resolution. This films takes a slightly different view, where like 'Alice in Wonderland', we see the main character change goals midway through the film, which is when the real source of conflict is revealed. Of course, they film makers did have good source material to work from (a Neil Gaiman story). This might also explain why the film was one of the creepiest I've ever seen. I was also surprised by how well crafted the film was in terms of pacing. The film wasn't long, and yet seems evenly paced throughout.

ParaNorman (2012), a dramedy, was a whole lot of fun to watch. The story of acceptance is a little predictable (but still reasonably engrossing), and the animation wasn't as great as 'Coraline' (maybe that comes from setting your film in a typical suburban setting), but it was still good. The supporting characters are colourful without being annoying, and they get the comedy right. The lead character's personality and development are both above par. For that matter, so is the script. The lead character's thoughts and conversations with others are perhaps the best thing about this film. The best part of the movie is of course the final confrontation with the little girl. A work of art on all levels, it's worth watching the entire film for that scene alone.

The Boxtrolls (2014) is a visual masterpiece. The set design itself is better than 'Coraline'. Again, watch it for the visuals alone. I liked the story, but felt it was aimed at younger audiences. The story felt old, like it has been done before, so it felt a little predictable and low key for me. Nothing about the plot really stood out. The Boxtrolls and ParaNorman are almost like inverted versions of each other, with the Boxtrolls being the better visual experience, and ParaNorman having a fuller, more emotional story to tell, with characters you really relate to. But this shouldn't take away how awesome the film is. Each individual frame is so well crafted it's tough not to admire the film.

On Beliefs, Assumptions and my World View

noreply@blogger.com (Daniel D'Mello) — Wed, 17 Dec 2014 01:01:00 +0530

Beliefs are not equal.

I had a discussion with a gentleman recently who refused to accept that someone could have no beliefs. This because I told him I had no beliefs. I personally don't like to use the word belief. I would rather refer to any position I hold as a model or approximation of the truth. These models are in turn built on assumptions of properties about this world that could change. It is easy to dismiss the difference between approximations and beliefs as mere semantics, but do keep in mind that semantics is the first thing you learn in a Philosophy 101 course, and ensures that everyone begins a discussion on the same page, instead of ending up talking about different things while referring to the same term.

Anyway, I see all views, opinions and theories as merely models that are built on assumptions. Nothing is based completely on evidence of course, as even the most basic evidence requires assumptions of the properties that the evidence is based on. For example, the colour red is not really seen by everyone in the exact same way. Our vision ensures that we all see the colour slightly differently, even if this difference is practically negligible. We still call it red though. This is an approximation. A generalisation. But there's more. We assume that the colour red, like other colours, exists as waves made up of photons. We don't know much about how light exists as energy, but we have created models that explain and exploit its properties to a degree that is useful to us.

Of course, none of this may be real. We may all be plugged into the Matrix. This could all be a dream. The colours may not exist. This universe may not exist. The properties of this physical world that we think we know about may only be a function of a dream world we inhabit and not part of whatever is really out there. But we don't know for sure if this world is fake, and so we act under the assumption that this universe and all the properties in it are real. Because this is the only practical way to live if our goal is comfort and happiness. We don't know if we exist in the way we experience. But it is best to assume that we do.

So in this sense everything is an assumption. But that doesn't mean that all assumptions are equal. There is a hierarchy. If there weren't then any view or model we created, no matter how crazy, could all be equally plausible. So what we do to maintain order in our world is assume that certain things are probably real like our universe and our existence. We then build the rest of our models on top of these basic assumptions.

Now we need to be really careful about how we construct these models, because a lot of them are based on questions that involve incomplete definitions and subjectivity. For example, do we have free will? Luckily, a lot of our models are objective, and built on physical laws whose properties we can approximate quite well. We use mathematical operations to build bridges. Mathematical identities themselves like Pythagoras theorem are perfect and exist for themselves with no exception, at least under the assumptions of the mathematical laws of this universe. We don't know why these identities exist, but we know that they do and how to exploit them. This is not an excuse for a belief in the supernatural. That is simply uncalled for given the evidence. We simply do not know why identities exist. That is all. Any models explaining why will need additional evidence.

You could of course make up your own inductive proof for a supernatural entity that exists outside the laws of this universe and space and time and matter but you would eventually have to face the fact that the properties of this proof are made up by you. i.e the proof works by induction, just like mathematical proofs work, because you assume all the properties needed for it to work, like we do in math. You don't know if these properties are real, you just assume that they are. A logically valid argument will still lead to a false conclusion if its premise is false. If your assumptions are unfounded, then no matter how good your argument, the conclusion you reach will still be only as good as your assumptions. This is why models for God's existence are both perfect and probably wrong.

Moving on, when you build a model, you identify a pattern and make predictions based on evidence. Sometimes, you use other people's models. You act on expectations that another from another model that you know very little about. For example, when you get sick and pop a pill. You don't know anything about what you're consuming. But you take it anyway expecting to get better. Is this a belief?

You could call it a belief, yes. Like the belief that the sun will rise tomorrow morning, given a normal solar system, or the belief that you will be able to walk or talk tomorrow, given no major changes to your body. You could call these beliefs, and they are all based on assumptions. But are they the same as religious beliefs? No, of course not. Because unlike religious beliefs, all these beliefs are verifiable. You cannot know for sure if a pill will cure you, but you know that you can look up the details of the pill if you wanted to. You can examine the skies or your body for patterns if you want to confirm the expectations you have for your model. In other words, these models are verifiable. Not a 100% verifiable of course. Pills do not always work. Solar systems and human bodies do not always work the way we expect them do. Errors abound. Things unaccounted for. The model is updated with new data. This is how critical reasoning works. Religious models are different. They rely, as I have said, on assumptions that are unverifiable. They might lead to useful but false conclusions. Religious belief may be useful, but it is also unverifiable.

So now we have not just beliefs, but levels of belief. There are verifiable and unverifiable beliefs. This allows for some degree of subjectivity, as what is verifiable depends on how good the evidence is, and all evidence comes down to further assumptions, which always comes down to our assumptions about this universe and our existence. But we can say for sure that some beliefs are more verifiable than others, because some evidence is better than others, assuming the basic laws governing this universe. Evidence that holds up to falsifiability and has predictive value will always be better than anecdotal evidence that relies false premise reasoning and confirmation bias. This is not to say that unverifiable assumptions are wrong. This is impossible to tell, but that's the problem.

We are mostly concerned with truth or falsity of assumptions based on the evidence we have. But since we can only examine the evidence in light of what we know about the universe, and since this is itself a series of assumptions that do not take into account what we don't know, then of course anything we postulate about God or a supernatural being could be true. Not probable, but possible. I wouldn't say that assumptions based on rules outside this universe are something we shouldn't bother to think about. But we definitely are limited in the ways we can verify them, given that all our means of verification exist only in this universe.

So yes, perhaps I do have beliefs. I suppose I do live my life along expectations of how the world should work even though I don't always understand why it works this way. These could be called beliefs. And they are certainly different from supernatural beliefs. My beliefs are based on assumptions that are verifiable, at least to a certain extent. I think is a more practical way to live for the moment, compared to holding beliefs that are unverifiable, because at least I can explain why I hold a belief. I can justify my beliefs with evidence. What about you?

Stats Blogs I Follow

noreply@blogger.com (Daniel D'Mello) — Sun, 14 Dec 2014 04:41:00 +0530

These are the stats blogs that make me better at what I do.

For stats literacy - http://www.statschat.org.nz

This is the main one for serious statisticians. Andrew Gelman, a statistics professor and Bayesian statistician and programmer, critiques poor statistical practices. Very informative - http://andrewgelman.com

For some advanced talk and a lot of useful links - http://simplystatistics.org

Deborah Mayo writes about philosophy of statistics - http://errorstatistics.com

Great learning resource for advanced statistical concepts - http://www.mii.ucla.edu/causality/

You might learn a few things from Daniel Lakens' blog - http://daniellakens.blogspot.in

A nice revision of important concepts with comics - http://statistically-funny.blogspot.in

More on probability theory - http://underpoint05.wordpress.com

Education without Innovation

noreply@blogger.com (Daniel D'Mello) — Sat, 13 Dec 2014 03:54:00 +0530

Most complaints about education systems revolve around them being mostly theory without any practical application. This is a problem because practical application like research methodology & computer lessons are a large part of what you need to go from being a theorist to a practitioner. It is no use studying concepts if you can't use them. But there is another problem I have with the system and it is lack of innovation.

Students are great at learning theoretical concepts. They are great at regurgitating what they are taught in the form of an essay. Sure this is a form of learning. But it is not innovative. When you gain knowledge being taught to you, you grow to the level of the person teaching you, but you don't necessarily exceed this level. This is why learning itself is useless for humanity without innovation. To truly make a change you need to go beyond what you are taught. You cannot simply learn concepts in a vacuum. You have to combine them to come up with new concepts. This is how new things are created.

It is the same for practical lessons, which may suffer from the same problem. I can put students through computer classes, but it will not mean much if they just recreate what I can, unless you want no new development. The best way forward is to teach your students the basic concepts with practical application, but to connect these lessons with existing questions, theories or ideas that they already have. This makes their learning context dependent, and motivates them to go beyond their lessons, to use what they have learned to create something new.

---------------------------------------------------------------------------------------------------------------

To see this in a broader historical context, countries that invested heavily in scientific innovation have always also been quick to reduce poverty and grow economically following innovation. Innovation makes you rich.

For example, Britain once had a lot of poverty. They were able to grow as a nation and coloniser and reduce their poverty because they innovated. This does go hand in hand with how much poverty you have of course. Britain had some labour, but not a lot i.e their labour was expensive because they were few, so they were forced to innovate, to find ways to mechanise processes that didn't require labour. They invented the steam engine, among other things, which meant that more resources could be processed, and which made the means of processing them even cheaper. This also meant that they could now do things quicker and cheaper than other more labour intensive countries could.

Compare this with India, where everything was done by hand. It still is in many villages, because it is still economical to do so. The low cost of living and availability of cheap labour acts as a deterrent to invent in technology. This is of course fine if you aren't competing with other countries and if those other countries are peaceful. But this isn't the case. Exploiters gotta exploit. Britain had superior technology because they innovated because they were under pressure to do so. India had cheap labour so there was no pressure to innovate and so didn't have superior technology. It was the same with a lot of African and Asian countries where labour was cheap. No investment in technology. No incentive to innovate. And of course the countries with superior technology ended up colonising the countries without any.

Bouncing back

You also see this with countries like Japan and Germany. Germany of course had a history of scientific development. But Japan didn't. It is interesting to see how these two countries managed to become economic powerhouses and developed countries despite losing world wars. Germany invested heavily in industries prior to both world war one and world war two. Even though they lost the wars, they still had the brains, the skilled technicians to build their economy, to continue creating, processing and selling products and services that other countries needed, which kept the money coming in, which meant they could continue to invest internally, in infrastructure, healthcare, education, and yes, in science and technology, to keep that loop going.

It was the same with Japan. A country with immense poverty before world war two, they invested heavily in technology and innovation. They knew they were decades behind other countries in scientific development because of their isolationist policy. Political ambition and conquest drove their industrialists and businessmen to invest in technology, to send their best people abroad for training, to bring back, adopt, copy or recreate whatever they could, to bridge that gap between themselves and the west. Which they finally did. In a very short time frame to boot. Sure this was partly driven by war, but following their loss, which included recovering from two atomic bombs, they still had the scientific knowhow to become the number one economy in Asia. Because they had invested in technology like no other country had. So even though they lost, they were still number one in Asia in science and technology.

History shows us that winning or losing wars doesn't matter as long as you own superior technology and a workforce that knows how to use it. You might occasionally grow overambitious, make dumb decisions like invading another country, and getting your ass kicked and pride hurt, but as long you still own superior technology, you will always bounce back quickly.

Owning the future

China and India were happy being agrarian societies, while Japan correctly ascertained that if you wanted to be a world leader, you had to own the technologies that no on else had, because this gave you an advantage. You had to have products and services that made you more powerful, because you were able to do things better than any other country (like build better factories that built better cars, faster planes, etc.). This not only gives you a military advantage, but also something to sell to other countries for a very high value.

Having better weapons not only gives you a military advantage, but it also creates a new market for exports. Having more money go into medical research means a better healthcare industry which means better trained doctors and hospitals with more advanced tools and techniques, which they can export. It also means better pharmaceuticals, which can be licensed or manufactured abroad. Again, the foreign countries that lack innovation only get to do outsourced blue collar work, not highly paying work. R&D stays at home. No country that owns technology is going to sell it. This has changed to some degree in recent years, with companies becoming more global, and R&D happening worldwide, which is an interesting change. It flattens the playing field somewhat.

But it's still shocking that people ruling countries today still act like they don't get the fact that for innovation to truly benefit you, you have to partake in it, so you end up owning the technology that results from it. When you look back at the recent history of India, it is shocking that there has been no efforts at home grown anything. If all you do is import foreign technology, you aren't owning the technology, you're simply renting it, or buying an end product of that technology, which is easily outdated. When India buys weaponry from Russia, Israel or France, it's buying old technology, perhaps even second hand products. Even if it is better than what its competitors have, it is still no comparison to having your own state of the art military industrial complex, like the US, Russia or France have.

This doesn't only go for weaponry, but also for public infrastructure like trains. Why does India have to go to France, China or Japan to build a Metro or Monorail? Because it doesn't have the technology to do it internally. It has to contract the design work out to foreign firms, and then use local labour to build them. This despite the fact that monorail technology is over a hundred years old. This shows you how backward India is, how lazy it has been at innovating. It isn't like the incentives weren't there. They were, just as they were there for Japan. I don't mean war, but the incentive of not being left behind, of wanting the best for your people.

Indian leaders simply do not have this vision. If they did, they would invest more in education and research. Without these, you're always going to be second best. You're always going to be left behind. And your country might always be exploited, particularly in terms of trade. Crops don't fetch the same prices that advanced technology does. A lack of innovation means that you're constantly dependent on other countries and their aggressive policies for products and services. To be the best you can't keep chasing the best, you have to outrun them. To chase is to lose. If in ten years you aim to be where the US is today, say n years ahead, then you're still going to be n years behind the US ten years from now. Your goal should be to grow at a faster rate than your competitors if you want to catch up with them.

Reducing poverty

This is also how you get rid of poverty. Yes, low cost labour intensive production provides jobs to everyone, but it also sustains poverty because it doesn't really help the economy. In 50 years, when other countries have moved on to other technologies and you're still using a low cost labour intensive system to make things by hand, your economy will be in bad shape. Your workers might have jobs, but their pay will be low, because their work is simple and there's many of them. They might have just enough to cover food and basic living expenses, but no money to spend on more expensive goods and services, which means low purchasing power and a smaller market for expensive goods, which hurts the economy. Whereas other countries that abandoned labour intensive production ended up with a highly skilled workforce that are highly paid because labour is now expensive because their skills are valued, and they can now buy expensive stuff, which creates a market for more expensive items, which in turn drives the economy. This is pretty much a comparison of socialist India and capitalist USA in the 80s.

This isn't a bad thing if Indians don't care about foreign products. But they do. They care about a better quality of life. However, because their economy is in bad shape, there's no money for the government to invest in infrastructure. This is partly due to subsidies, but those subsidies wouldn't exist or matter if your people were richer, which they would be if they had higher order skills that they could sell for more money. If you invest in innovation, say in factory production, there might be some job loss, but in the long run, you will need a highly skilled workforce to manage these new processes. You could have a million people harvest cotton by hand, or you could have machines do it, and have those million people do more specialised, highly paying work, like overseeing the machines, maintaining them, working towards business strategy, doing logistics, HR, marketing, PR, sales, client relationship management, IT.

Through innovation, investing in scientific development and developing new and better ways to do things, you make life better for your workforce. Instead of earning a pittance doing low value work, they're now earning a lot doing high value work. This is how countries and economies grow. Innovation makes you rich. It's a costly investment, but the returns justify the costs. You see this as a historical pattern when you look at present day industrial nations that used to be agrarian - China, USA, Japan. All you need are leaders who can see and learn from history.

On Anecdotal Evidence

noreply@blogger.com (Daniel D'Mello) — Fri, 14 Nov 2014 16:21:00 +0530

Too many people rely on anecdotal evidence (personal experience or cherry picked examples) to assess if something is true, and I don't like it.

To me, everything is a model. All the ways in which we view the world, or our explanations for various phenomena like behaviour, are merely models. The techniques we use to estimate weather patterns are models. The techniques we use to estimate group dynamics are models. All estimates are models. There are a number of ways to consciously build models. You could use anecdotal evidence. You could also use critical reasoning.

There's a famous phrase that goes, "all models are wrong, some are useful". I like this because it feeds into what scientists do. Science is not about finding the 'truth'. It can be about the pursuit of the truth, but the truth might never be known. Therefore, all you do is continue to build better approximations of the truth, or better models to explain and predict phenomena, for both academic and practical purposes. This is what science does. Science is essentially a mix of critical reasoning and research techniques combined with domain knowledge. The sciences - Biology, Psychology, Chemistry, Physics - are merely fields of knowledge, domains that revolve around certain interest areas. Of course there is overlap. But these are not sciences because they encompass domains of study. That's half of it. They are sciences because they use critical reasoning techniques to investigate and build models that approximate the truth.

Where does anecdotal evidence come in? Anecdotal evidence is a first step towards building a model, but not evidence for the model. Anecdotal evidence is the presence of something interesting that requires further study. You see a ghostly white figure at night. You have no idea what you are looking at. You investigate, you make a hypothesis and attempt to verify it. Things can get a bit shaky if you skip the investigation and rush to make a claim, because anecdotal evidence could be due to a number of causes, not just the one you have in mind. False positives abound. This is why it is important to treat anecdotal evidence as a first step only. It would be disastrous to claim something as fact based on personal observation, and then find out that your claim is wrong because you didn't properly investigate the matter.

Let's take some examples. The claim that God is real. There are various types of evidence for this claim. One is prayer, a type of anecdotal evidence. I pray for something, something happens, therefore God is real. Anecdotal evidence like prayer cannot be evidence for the existence of God till it is verified. For every anecdotal claim of prayer working, there could be another for it being useless. To verify if prayer works, you would have to experimentally demonstrate its effectiveness. This is called falsifiability. Note that this is neither proving nor disproving the existence of God. This is not the question at hand for the scientist. It might be the question at hand for the person claiming God's existence and using prayer as an example, but for the scientist the investigation only concerns the effectiveness of prayer. A scientist who demonstrates that prayer is useless is not proving or disproving the existence of God. He or she is merely verifying a specific claim. This is important to remember. Science is not always concerned with the big questions. It is merely a tool to verify claims or existing models. After all, prayer is a model of how the world works. A scientist can spend his or her entire life falsifying such claims. This would get us nowhere if the claims were spurious to begin with. This is why anecdotal evidence should not be used to claim something. Because there are more reliable ways to build models.

[This is why proving or disproving the existence of God is a futile activity. No one knows exactly why the concept of God came about. We have theories. But nothing that seems to be founded in verifiable evidence. There is a lot of anecdotal evidence, but upon verification, a lot of it does not hold up to scrutiny. This is not to say that any of the thousands of Gods do not exist, or that people are wrong in believing in them. Science cannot falsify something that was made up to begin with, or is currently too difficult to verify. It can only analyse the evidence and show over time how improbable something is, using existing methods. True falsifiability is impossible. Which is why we will never be able to disprove the existence of the Loch Ness monster either.]

Here's another example. Psychometric tests like MBTI. HR professionals love them. But the data from meta analyses picks holes in the test's reliability and validity. But HR professionals who have used these tests swear by them. One person I spoke to even compared it to the accuracy of a horoscope while praising it (I doubt he was trying to be ironic). This kind of reliance on anecdotal evidence to back something, is used as a model by a lot of people, just like people use prayer as a model. Why do they use it when there are scientific techniques that discredit these models? I have no idea. Maybe people are ignorant. Maybe they find it easier to act on someone else's recommendation or 'try it yourself first' advice rather than doing personal research. Maybe they think that discrediting one model will mean discrediting a larger model that they have more of an emotional investment it. Maybe they already choose to believe in something to make themselves feel better. Maybe creating a faulty but useful model works for them. Maybe the model's degree of usefulness wins over the fact that it is wrong.

Which is interesting because of what I said earlier - all models are wrong, some are useful. Let's say human sacrifice to appease the weather Gods is supported by anecdotal evidence i.e. a group of people practice human sacrifice and choose to notice only when the weather changes for the better, convincing themselves of a correlation between the two. They of course ignore instances when sacrifice does not affect the weather, attributing it to human fault or God being angry with them, or it all being a part of God's larger plan. Now let's say hypothetically that this model/belief is the only thing keeping this society stable.

Note that science isn't always concerned whether the effect is real or not, or if belief in it should continue. Yes, assumptions are faulty. Correlations abound in large amounts of data. They're a function of statistical noise. Experimentation should verify the probability of the correlation. But even if it finds that the correlation/belief/model of human sacrifice for better weather is wrong, it doesn't erase the fact that it is useful. Now replace human sacrifice with belief in God, or MBTI. These models might work in certain contexts. Belief in God helps people in certain contexts. Belief in aliens might just help society. I have no idea. MBTI might be useful in certain contexts. Neither of these models might be correct, but they can be useful. If MBTI works for you, then great, use it. But that doesn't mean it does what it claims to do, which is why you wouldn't be right in recommending it to me. Which is why people need to look at the evidence to verify if a model is good for them, and not rely solely on anecdotal evidence, or else risk disappointment.

In summary,

1. Anecdotal evidence can be a good first step to further research.

2. If you notice something interesting, collect data, find patterns, make a hypothesis and verify it. Then make a claim.

3. Your claim is your model. It can only be built on the elements in point 2.

4. Anecdotal evidence by itself cannot be used to build models.

5. If someone builds a model that approximates what they think is the truth, question their assumptions and verify the evidence.

6. If their model is build on anecdotal evidence (personal or cherry picked examples), reject the model for being incomplete.

7. Their model is not necessarily completely wrong but it is pointless to consider something correct if it hasn't been verified, even if it is useful.

8. A model's usefulness does not necessarily reflect its correctness.

On Science Journalism

noreply@blogger.com (Daniel D'Mello) — Mon, 4 Aug 2014 21:09:00 +0530

Few people do science journalism right, and so few comprehend the difference.

I came across this recently. I clicked the link, which took me to an article describing a paper that I downloaded and read. The paper itself was OK as far as social science papers go, but as usual, elements within the media that don't know any better jumped on it.

The paper concludes "that individuals with an East German family background cheat significantly more on an abstract task than those with a West German family background." It also concludes that "The longer individuals were exposed to socialism, the more likely they were to cheat on our task."

The points I would like to raise are below.

The first point is with the statistical inference used in the study. The researchers note that both groups cheated, but those with East German backgrounds cheated more than those with West German ones to a degree that was statistically significant. I won't go into detail about p value hacking, confidence intervals, effect sizes and power here, but suffice it to say that a statistically significant result does not reflect an actual real life effect. This is just a function of probability. Neither group of people may have cheated. But the statistical techniques picked up on variation that the researchers deemed significant. We do not know if this significant difference represents cheating in real life, or if it would hold if the study were to be repeated.

Even if the effect (cheating) were present, there is no way that you can automatically extrapolate the results of a game to a judgement of people's moral attitudes in general. This is because morality is complex. The fact that some people might use the opportunity to cheat if given the opportunity to do so in a game of dice does not necessarily reflect their attitudes in general, or choices in other situations. The researchers use terms like 'value system', but do not define what this encompasses. What constructs and concepts make up a value system? Is it objective or context dependent?

You might download a film illegally, but this does not make you a thief in general. Your choice to download a film at that point in time is a function of the cost-benefit equation to you and the social context of your choice, how many other people are doing so, your perception of the 'rightness' or 'wrongness' of your act, etc. It does not necessarily reflect you attitudes or preferences in other contexts.

Even if you could extrapolate the effect observed in the game to life at large, you must remember that Correlation is not Causation. The fact that those with an East German background cheated more does not indicate that their background is what caused them to cheat. No one is denying that economic systems can change behaviours and attitudes of people, and it is worth studying, but you cannot jump to conclusions. You need to remove all the confounding variables, false positives and other possible causes. You do this by making as many comparisons as possible. Did the researchers do this? Not completely.

East Germany was not merely socialist but thrived on a culture of fear and repression, with secret police spying on citizens. There are social and cultural factors that could have lead to people developing a habit of cheating and might have had nothing to do with the economic system. The researchers have identified two of these - economic scarcity and social comparison - but were not able to verify them using their methods.

Also, the paper fails to mention if the researchers took into account the fact that former East Germans have been living in a new economic system for 23 years (1990-2013) and how this might have changed their preferences/choices/attitudes to the extent that it makes the effect of their background meaningless to the study.

Even if all the points above are wrong and the researchers' assumptions thus far are correct, the inference that people exposed to socialism cheat more would still be incorrect, as socialist economic systems themselves are very different from country to country. East Germany was a comparatively impoverished socialist country compared to the scandinavian countries for example, which also had elements of socialist governance. One could argue that the scandinavian countries were never truly socialist, but that's missing the point. If the authors are talking about one specific kind of socialism they need to be clear about this. If they are referring to socialism in general, then they need to test population samples in other formerly socialist and presently socialist countries, and control for cultural and other differences, before they can make such an inference. They have not done this either.

--------------------------------------------------------------------------------------

All in all, this is not a bad paper, compared to others I have read. The researchers are quite honest about most of their limitations. However, no one else seems to care. The original article that linked to this paper merely reiterated the findings as if they were correct, without taking into account the researcher's alternative explanations. This is bad journalism.

There are so many papers being published every month in various journals. Sometimes, the journals themselves are shady, and publish poor research for a fee. Researchers are under pressure to publish as this is what determines their reputation and pay in academia. So they tend to fudge data or manipulate it in dishonest ways to get positive results. Journals have a publishing bias towards positive results. And the journalists who write about the papers that are published in journals are usually under tight deadlines too. They cut corners. They trivialise, generalise and indulge in simplification. They have a poor understanding of scientific domains, empiricism, and critical reasoning. Most don't bother critiquing the papers they report on.

I have read so many bad science articles in the past 3 years that I have had to whittle down my RSS feeds to the extent that I only follow a few news feeds, scientists and professional science writers. On Twitter I am even stricter. I do not follow any pop science accounts, only professional researchers, people who will either share original research, or go the extra mile and critique people's research rather than blindly sharing links they come across. The best science communicators out there do not even bother writing about the latest developments in Psychology, given the faults in the field, the shakiness of results - the p value hacking, selective sampling, failure to replicate, false assumptions, etc.

So it's always sad when an individual with a lot of followers shares a bad article. I am not writing this to be mean or to hurt anyone's feelings or discourage anyone's work. I'm just saying that there is a clear demarcation between good science writing and poor science writing. I do not expect every journalist out there to be able to critique a paper (though it would help) but I do expect even a beginner to know the difference between a balanced and a biased article.

When a journalist with impressive credentials chooses to share a link to a clearly biased article that, to push an agenda, deliberately ignores the limitations in a paper that any 2nd year undergraduate student at a middle ranked Psychology department in the UK would notice, you question that person's credentials.

This is not an isolated case. There are other people on Twitter with a massive reach who also tend to share terrible links. I am sure they are lovely human beings who want the best for humanity and are smarter and more accomplished than me in many ways, but they still share terribly written pop science articles that distort a subject just because they have a catchy title or byline.

So here is some advice when you come across a piece of science journalism -

When something sounds too good to be true, it usually is not true. In the social sciences, discoveries are few and far between, so an article that claims to have discovered a major effect must be met with skepticism.
If an article claims something that you're sceptical about, read the paper and double check if those claims are true.
If you cannot critique the paper or do not have the time, do not share the article. Wait for someone else to critique it.
Follow professional science communicators who know how to critique scientific discoveries, and not mass produced pop science junk. The pros know how to write a balanced piece. Pop news channels just want to grab eyeballs and don't care about accuracy.

Proximate and Ultimate Explanations

noreply@blogger.com (Daniel D'Mello) — Fri, 13 Jun 2014 12:23:00 +0530

Proximate and ultimate explanations are among the first terms that you learn about when studying ethology. These terms are used in other contexts, in addition to the study of behaviour, where they mean slightly different things, so it's important to understand them and not get them confused.

Why does an animals behave in a certain way? An animal's behaviour can be explained in proximate and ultimate terms. Proximate explanations deal with the 'how' of a behaviour i.e the underlying or mechanistic reasons behind a behaviour. Ultimate explanations deal with the 'why' of behaviour i.e the usefulness of this behaviour to the creature and how it came to acquire it.

--------------------------------------------------------------------------------------------

Here's an example - birds singing in spring.

Proximate questions - How do birds manage to sing in spring?
Proximate answers - Daylight induces changes in hormones which make them sing. They learned to sing when young.

Ultimate questions - Why do birds sing in spring?
Ultimate reasons - For mating/reproductive value. The vocal chords of distant relatives and extinct birds indicate that this trait evolved concurrently with overall fitness.

---------------------------------------------------------------------------------------------

It is important to realise that proximate and ultimate reasons are both explanations for the same behaviour/phenomenon but from different perspectives. You could say that proximate explanations provide the reasons underlying behaviour (what is it due to?) while ultimate explanations look at the bigger picture (what is it for?)

Proximate behaviours usually provide mechanistic reasons for behaviour or describe the 'triggers' behind behaviour. You can think of them as being the result of things occurring in the animal's body (e.g. hormones, nervous system, genes, age) or immediate environment. Proximate explanations can be further divided into mechanistic (causation) and ontogenetic (developmental).

Mechanistic explanations (how does it work? how was it caused? what caused it?) usually deal with processes within the body that follow simple rules, like neurons and the nervous system, hormones, pheromones and other bio-chemical processes.
Ontogenetic explanations (how did it develop?) cover behaviour from the nature-nurture or gene-environment angle. These explanations build on mechanistic processes, and relate them to what's going on with the individuals environment, like learning or other aspects leading to behavioural development.

Ultimate explanations usually describe the function of a behaviour in terms of evolutionary history and function. Ultimate explanations deal with evolutionary benefits of a particular behaviour. These can be further divided into phylogenetic (evolutionary history) and adaptive (functional) explanations.

Phylogenetic explanations (how did it evolve?) deal with why this behaviour might have evolved over successive generations instead of being lost. We look at the evolutionary history of the creature to see how natural selection worked on this trait.
Functional explanations (what is it for? what purpose does it serve?) deal with the benefit the behaviour confers to the individual in terms of its current environment. It is important to remember that an individual can have a current trait that is adaptive without it being an adaptation.

---------------------------------------------------------------------------------------------
Here's another example - Honeybees swarming (splitting up and building new colonies elsewhere).

Proximate questions - How to honeybees manage to swarm? What factors lead to swarming?
Proximate answers - Because of the way their central nervous system responds to other bees doing the waggle dance. Or because this behaviour is triggered by colony size, brood comb congestion, worker age, or the queen having reached her maximum egg laying rate.

Ultimate questions - Why do honeybees swarm?
Ultimate reasons - For reproduction, survival, more food resources.
-----------------------------------------------------------------------------------------------
And one more - birds building nests.

Proximate questions - How does a bird know how to build a nest?
Proximate answers - It could be a genetically programmed or learned behaviour.

Ultimate questions - Why does a bird build a nest?
Ultimate answers - Because a nest assists in mating and so improves reproductive success, which means genes are passed on to the next generation.
-------------------------------------------------------------------------------------------------

On MOOCs and Online learning

noreply@blogger.com (Daniel D'Mello) — Sun, 27 Apr 2014 19:01:00 +0530

I think MOOCs are important and useful. I just think that a lot of them aren't following instructional design principles and enabling learners in the way that they should be. It's a great medium to change the world, but it's being run by computer scientists and businessmen with minimal input from learning designers, and this needs to change.

A note of advice, don't take more than one MOOC at a time. The first time I discovered and registered for MOOCs was 2012. I registered for four, but then realised I couldn't follow all the courses. Even after reducing the number to one, I couldn't cope with both the MOOC and my studies. I tried taking more MOOCs when my schedule cleared up in 2013. But again, I registered for too many. I finally completed two courses simultaneously from March to May, but the workload was so high that I decided to stick to one course at a time in future. And the only reason I was able to manage two was because one was really easy.

I learned two lessons here. One, a MOOC is a full time course of study. It's equivalent to one college level module, and a heavy one at that. It requires daily participation on your part, and is certainly not a 'one day a week' thing. You don't just watch a few videos and take a quiz, you need to do a lot more for the course to be effective. There's a lot of reading to do if it's a knowledge based course. And a lot of practice if it's a skill based course.

There's also a lot of knowledge sharing in online groups. And you need to budget your time accordingly. Granted, you don't always know exactly how much time you'll need at the start. Which is why it's a good idea to audit courses when you're not sure. And just drop out if it's too much for you. I tend to do this a lot, especially when the subject area is completely new to me. I've dropped out of around 6 courses for every one I've completed.

The second lesson is about the design aspect of MOOCs. Understanding a concept takes time. Learning comes from reflection, practise, application and knowledge exchange. You do not learn something by watching four 15-minute videos of the topic each week and then taking a quiz about it. The true test of learning comes not from summarising what you've learnt but applying what you've learnt to a new context. That's the real challenge. Do MOOCs meet this?

I say no. Most MOOCs consist of mainly videos and reading materials. Videos are at best an overview, an introduction. They cannot be the entirety of the course material. You should ideally watch a video, and then do a lot of follow up reading (the best MOOCs have their own textbooks), note taking, introspection, sharing ideas with others, summarising your conclusions in the form of essays, and a lot of follow up exercises involving applying your ideas to novel situations. This is how learning takes place.

And this is the problem with MOOCs. They're mostly just videos and quizzes, and they should be more. A course with just videos and quizzes and maybe a few assignments can never completely teach a complex subject to the extent that you begin using its ideas as a practitioner. Secondly, this type of course encourages sole study without group interaction, which is not preferable. Third, it fools you into thinking you're now an expert on a subject because you got a good score on a multiple choice quiz on the subject every week for eight weeks.

Multiple choice quizzes are generally not the best learning facilitation tool, given the amount of guesswork taking place. I understand that you can't have teacher graded essays or exercises in a class of 15,000 students, but peer review should definitely be an option.

Real learning takes place through reflection and practice, which requires time. One of the better courses I've taken was on Psychology and had it's own free online text book that was required reading for the course, and was comprehensive in the materials it covered. However, I would have liked more essays and exercises to cover the practical aspect.

Another good one (on mathematics) had loads of exercises that needed to be discussed in the group forums. Group learning is a good thing, and one of the main advantages of online learning. You have so many more classmates to share ideas with and learn from, and you do this on your own time. When a course only revolves around material presented through videos, without any other reference material or exercises, the course forums turn into a wasted opportunity, as you're only discussing topics covered in the videos, which is not extensive anyway.

MOOCs will never replace college education or be taken seriously as a means of education if they don't have their students use more reference material and application based exercises, which encourages reflection and knowledge sharing, and better forms of evaluation.

7 ways to study in the UK for cheap (what you won’t read elsewhere)

noreply@blogger.com (Daniel D'Mello) — Wed, 23 Apr 2014 22:51:00 +0530

So you’ve been admitted into a university in the UK. You’re fees are probably 3 times higher than those of all your local classmates. Let’s crunch some numbers. Average Masters programmes in the UK cost at least 12,000 GBP for international students, and rising every year. Living expenses are approximately 6,000 GBP on average. So how do you ensure you spend a year in the UK without burning a 18,000 pound hole in your (or your parents’) pockets? Here are some useful tips.

1. Find a part time job

If you want to make enough money to offset as much of your living expenses as is humanely possible, you need to find a part time job ASAP (forget about your fees, no job will ever pay you enough to cover that, apart from one where you sell weed). A part time job ensures that you earn steady income on a weekly basis to cover your rent, food, travel and other expenses.

As a foreign student, you will be allowed to work 20 hours a week, and if you do work at this maximum capacity from day one, at a minimum wage of around 6 pounds per hour, you should earn 6400 pounds over twelve months, enough to cover your living expenses. In reality, it might take you a couple of weeks to find a part time job, when you do find one you might not be get enough to fill your 20 hour capacity, and you will be taking breaks from work during exam season or are busy with other aspects of your course, so you will probably make less than 6400 pounds.

Where do you find a part time job? Look at your university website for vacancies. Do they have a student union? Or a career center? Contact these groups to see if they know of any vacancies. Follow them on Facebook and Twitter to keep up to date on vacancies. Your university will probably have shops, restaurants & cafes on campus. Contact them to see if they need any staff. Do this weeks before you arrive, or there might not be any vacancies left by then. If you do see a job vacancy online, apply immediately. There might be hundreds of applicants, and vacancies are filled on a first come first serve basis.

If you don't find anything, look around for jobs as soon as you arrive. Talk a walk around the town or city you are staying in during your first week. Drop your CV in at all the coffee shops and fast food joints so they know you’re looking for a job. A good thing about the UK is that there are loads of Indian restaurants everywhere. And Indian restaurants in the UK tend to hire Indian students. Make a round of all such restaurants in your area to see if they need any help. They're always on the lookout for waiters and waitresses but don't bother advertising and usually recruit through word of mouth.

Extra work is usually available during the Christmas and Easter breaks. These vacancies are usually temporary in nature, lasting for 2-4 weeks. Additionally, your university itself should have internal vacancies that open up during the course of the academic year. If you're good at something technical, look for part time teaching jobs where you can teach undergrads for a semester. The pay is really good.

Please do note that finding work during your course should not take precedence over your academics. You have spent a lot of money to come to a foreign country to study, and you shouldn’t risk sacrificing this for immediate economic gain, even if this is what your employer wants.

2. Don’t stay on campus. Find private accommodation.

Campus accommodation in the UK is comparatively more expensive, and can increase your rent by 30%. Private accommodation by contrast is usually around 800-1000 pounds lower.

Also, staying on campus means you will probably be required to commute to your town or city centre to stock up on groceries every week. This is inconvenient for two reasons. One, you might not always have room in your fridge or freezer for a week’s worth of food, so you might have to make more than one trip. Two, the money spent on the commute is going to add up. Think two pounds every week for a return bus ticket, for the minimum 10 month (45 week) duration of your course. That's 90 pounds just for the shopping commute. With private accommodation you could try to get a place closer to a supermarket, and walk instead. You'd save 90 pounds. And you wouldn't have to worry about making multiple trips or kitchen space.

Find a cheap place to stay preferably before you get to the UK. Post queries on your university Facebook pages and other online forums which students frequent, asking if anyone needs a roommate. Check gumtree.com. Contact former students, particularly Indian ones, to ask if they know of a cheap place to stay, or can recommend a good landlord.

3. Shop smart

Shops on campus can be expensive. Do your shopping at one of the larger supermarkets, like Co-op, Tesco, Aldi or Lidl. Also, constantly be on the lookout for good deals. Larger supermarket chains tend to mark items down by 25% a day before they expire. Avoid tiny neighbourhood convenience grocery stores. They usually mark items up by 10%.

4. Track your expenses

Set a weekly spending limit and don't cross this figure, no matter what. If you do, make up the difference by spending less the following week. Make a note of your expenditure so you know if you're nearing the limit. Record what you spend on most and try to reduce this.

5. Take part in experiments

Universities in the UK have Health and Psychology departments whose students conduct experiments for which they require human volunteers. These experiments can last from 15 minutes to weeks, and usually pay around 5 pounds an hour. Drop by the offices of these departments around dissertation time, or keep an eye out for notices requesting volunteers. Some of the experiments can be fun, and you usually get to know a little bit more about yourself.

6. Proofread

A lot of students on campus come from countries where English is not a first language, and aren't very comfortable writing long essays in English. If your own English writing skills are good, you can offer your services as a proofreader. Put up notices around campus advertising your services as a proofreader, or get the word around through your friends. Professional proofreading services charge hundreds of pounds to proofread essays, so you should be able to get work by charging less. Even a fee of 50 pounds would be a bargain for students looking to improve their dissertations.

7. Don’t smoke

Cigarettes are expensive in the UK. A pack can cost around 7 pounds. That’s enough for a meal at a restaurant. Do yourself a favour a try to kick your smoking habit before travelling abroad. Or fill your suitcase with about 200 packs of ‘Goldphlake’. How you’d get that through customs is another problem, though.

The Best Time of Year to Visit India

noreply@blogger.com (Daniel D'Mello) — Tue, 15 Apr 2014 17:02:00 +0530

What do you feel like doing or seeing in India?

All the best beaches, across Gujarat, Maharashtra, Goa, Karnataka & Kerala are in the south, with a decent night life, temples, hill stations and historical towns thrown in. The best time to visit the south is December-February.

This is also the best time to visit central India, Maharashtra and Madhya Pradesh, to see some of the World Heritage sites. It's either hot or rainy in these places during the rest of the year, though these places do contain the best national parks in the country and the best time of year to see these (and tigers) is April-May because this is the only time of year that your view of animals isn't obstructed by tall grass, it being the dry season, and the animals all tend to gather at watering holes, making them easier to spot.

If you feel like medieval palaces, Mughal architecture, the Taj mahal, sand dunes, forts, etc, you will need to go north i.e Delhi, Rajasthan & Punjab, and again the best time to see these is either Nov, before it gets too cold and hazy, or Feb. Best to avoid Dec and Jan.

This holds true for the N.E as well. It's a great place to visit for hiking, root bridges, caving, food and national parks, and the best time is either Nov or February-Mar. Avoid Dec-Jan because of the long holiday season where everything is shut, and because of the cold.

If you feel like visiting the Himalayas or some of the other northern mountain ranges for the views or to go hiking, you'll want to visit Ladakh, Himachal and Uttrakhand, which are best visited in the warmer months, May-Jun.

So there you have it. May-Jun for the mountains and national parks. Nov-Feb for the rest of the country.

Working with Data Survival Toolkit

noreply@blogger.com (Daniel D'Mello) — Wed, 9 Apr 2014 16:46:00 +0530

It's important to know certain fundamental things about data to be a good manager or researcher. Here's a brief list. These are the basic ideas that make up a survival toolkit for data collection and interpretation, be it in business, marketing, psychology or anthropology. Become an expert at these concepts and you will be better placed to interpret reports, work with data or design a study, than most people on the planet.

Research Methods and Data Collection

Treatment and control groups.
Variables - numerical (continuous, discrete, ratio, interval) and categorical (nominal, ordinal, binary).
Independent and dependent variables.
Correlation does not imply causation.
Anecdotal evidence.
Populations and samples.
Observational (correlational) and experimental methods.
Sampling strategies.
Controlling, randomisation, replication, blocking.
Double-blind.
Placebos.
Measurement error.
Reliability and validity.
Confounding variables.
Between and within groups.
Variation.

Data Analysis

Data matrices
Frequency distribution
Graphs and plots
Scatterplots
histograms
Mean, median, mode
Skew
Standard deviation
Box plots
Quartiles
Outliers
Row and column properties
Bar plots
Pie charts
Hypothesis testing
Confidence intervals
Standard error
Probability
Inference
Effect sizes
Power
Correlation
Regression
Analysis of variance

And how do you become good at these concepts? Pick up a good book, join a course, and practice what you learn. Or call me, I do workshops :-)

Movie Review - 'Frozen'

noreply@blogger.com (Daniel D'Mello) — Tue, 1 Apr 2014 19:33:00 +0530

The opening credits are really classy. Like they're preparing you for an epic. The film begins with some guys singing about ice. I know the film's not about them but I love the theme setting going on. Like the intro during 'The Prince of Egypt'. It tells you what the film is going to be about.

First Act

It then moves to these two sisters - Elsa and Anna - playing. Elsa can use magic, and she accidentally hurts Anna. I love the tone of this scene. Everything just works, the characters, the music, etc. And it gets more powerful on repeat viewing, as you know that this one scene sets off so much tragedy.

These trolls save Anna, removing her memories of Elsa's magic. I'm not sure why they do this, except to move the story forward. It seems like a pretty big deal to remove someone's memories. Tragic even. They also warn Elsa about her magic being dangerous, fear being her enemy and how she must learn to control her powers. The parents then decide to separate the girls, thereby ensuring a fearful Elsa, which is exactly what the trolls warn them against (*sigh*). Double tragedy. But a good setup for the film.

The following exposition through the song 'Do you want to build a Snowman?' sets an even sadder mood. It's like this film is a celebration of tragedy. The girls grow up apart. Anna wants her sister back. And Elsa is living in fear. The parents really screwed things up, with help from the trolls. And then they die.

Though the song itself is good, it leaves a lot of open questions. What did Elsa do during her time alone in her room? Were the sisters separated completely during all these years? Did they not see or talk to each other at all? How does that affect their relationship? Do they even have one by the end of this song? The song gives us nothing. No interaction between the sisters. A real letdown after that good setup. Also, by the end of the song, we observe what Anna wants, but not what Elsa wants. This we have to infer.

Then we see Anna today. There's a party. She's excitable. And wants more from life, including true love. Nothing about her relationship with her sister, which is weird. Is she over her sister shutting her out? And wait, was Anna shut up in the castle as well? For 13 years? Why? She doesn't have any powers. What did all these years of isolation do to her? Where are they going with this true love thing? This song 'For the first Time in Forever' changes the tone of the film for me, makes it feel a bit kiddish, more like 'Tangled' for some reason. But hey, that's the demographic they're going for. "We interrupt this seriousness to bring you a dreamy princess song about true love". Still, what's the real story here? Again, we observe what Anna wants, but still nothing about what Elsa wants w.r.t her sister. More unanswered questions.

Then Anna meets Hans. Elsa is crowned and continues to be nervous. We get a scene with the sisters talking (at last!) and it's delightful (Elsa reaches out to Anna, who reciprocates, and they share a few laughs) but is quick and doesn't touch on the past, so we still have questions about their isolation and current relationship. Did the girls really not see each other all these years? Is this the first time they're meeting and talking since they were children? We've not seen them interact since the start of the film, when they were toddlers. Do they even have a relationship? We need more context, Disney. Is this really a film about a sisterly bond with minimal interaction between the two sisters?

Then Elsa shuts Anna out again, and we see both the sisters are unhappy at this. Then a song with Hans. More 'true love'. Where is this going? The song is catchy, but the proposal seems silly. Then there's the argument between the sisters about marriage. We finally see Anna lash out. So the love angle was just a cue for the argument? So the story is really about the sisters? OK, but the love angle with Hans seems a bit forced. And the story still feels a bit shallow and one-sided. We have to assume we understand Anna's agony, despite not seeing any evidence of an actual sisterly bond at risk, and we still don't know what Elsa wants. We can observe she's sad, and we can infer she wants normality, but we don't see her do anything about it. Does she not have any goals at all?

Still, I love everything in this scene. There's finally some conflict to sink your teeth into, though it's short lived. Elsa accidentally reveals her powers and flees in fear & shame, accidentally freezing her kingdom in the process. Anna follows to fix things with her sister and the kingdom, leaving Hans in charge (lol wut?). OK so the story is definitely about the sisters, but what's going through Anna's mind right now? She's just seen her sister's powers for the 'first time'. She must realise this is why she was shut out all these years, but we're never shown her realisation of this or what this now means to their relationship.

Apparently it doesn't make a difference to Anna wanting her sister back. OK. Not much character development here. We weren't even sure if the sisters had an actual relationship, and now we have to suddenly assume that Anna's unconditional love for her sister is normal, without having seen them bond at all in the last 13 years? Bit of a stretch. Also, Anna doesn't think her sister is dangerous or would hurt her, which is kind of silly seeing as how she almost just did, and also that Anna seems so confident about this given that they've spent 13 years apart and don't really know each other. What does Anna know that we don't. Where is this confidence coming from? it would have been better for the character to show her as just a little bit fearful of her sister and doubtful of her own safety before rushing off after her sister.

Transition

Then we switch to Elsa, and there's the big beautiful song where Elsa comes into her own. It begins with sadness as she realises her loneliness, and that she's failed to hide or control her powers, and failed to be what 'they' wanted her to be. And with her secret out, there's nothing she can do now but flaunt it? She lets her magic lose, first tentatively, then with relief and confidence, creating a new home.

She's happy. And it's more than happiness. It's self-realisation, finding her spot in the world. Embracing who she is, something she couldn't do earlier. She's free. As beautiful as this moment it, you've got to wonder why this transformation is happening? Why is she using her powers, and why the happiness? Who exactly is Elsa? A daughter, sister, queen, ice queen?

Till now, we've seen Elsa as an obedient daughter, reluctant monarch, distant sister, an individual trying to be responsible, shutting people out, hiding her powers and trying to control them so as not to hurt people, and in the process turning into a fearful recluse. But now that her secret's out and she has fled in fear, and she's far away from people, why does she use her powers anyway? How do they help her?

Here's where we have to infer that her powers have been tied in to her identity more than we thought. Elsa's powers are not an irrelevant accessory. She is her powers. And when she was made to suppress her them, she was not suppressing an extra ability or a minor talent, but a core part of who she is. She was made to be afraid to be herself. And now that her secret's out and she's away from everyone, there's nothing to fear or suppress, she can relax and be herself. By using her powers now, she's not doing anything unusual, she's simply being herself for the first time in her life since she was a child. This is the only possible explanation for why she would choose to embrace her powers as her identity at this point.

This is a beautiful transformation, but it leaves the viewers with a lot to infer without any context. We've only seen Elsa use her powers freely once before, as a child, when she presumably considered it little more than a play thing. We're not given to understand that she thought of her powers as anything more, or that the way she perceived her powers changed over the years. And when we see her using her powers now, 13 years later, the film expects us to just assume, in a second, that her powers were tied to her identity all along, or became that way at some point in her adult life? This is a lot to ask of the viewers.

Also, why is Elsa so happy? Granted her earlier self was part facade, but not all of it. She still has a kingdom, a sister. She's still leaving them behind out of fear, not out of self-realisation. Shouldn't that reflect in the film? Nope. We just see happiness. Apparently being happy at her newfound freedom overrides the loss of family. Does this mean Elsa's previous life didn't mean all that much to her? What does she think of her relationship with Anna at this point? Or do we infer that this just a bitter happiness? Or that Elsa is lying to herself? So many questions.

After waiting an entire act to get to know Elsa better and find out her plans w.r.t her sister, the audience is left wanting as the film takes us in a completely new direction, showing us a side of Elsa we never knew mattered, and ignoring the one pertaining to the plot i.e. the sisters' relationship. What is going on with the story?

Nevertheless, this is the first we get to see of Elsa as an independent character, and not a character portrayed solely in the context of her sister. Also, this is the first real character development we get to see in her. Till now it's only been about Anna pining for her sister and then being excited at the party and pining for love. And here's where you really realise how much you like Elsa. She's way more interesting than her sister.

At this moment, you're happy that she's finally getting her moment of happiness after years of tragedy, sadness and fear. You're rooting for her, you want to see her happy, you're enjoying the transformation, you're emotionally invested in it, in her, even if it makes little sense and doesn't work for the plot. Also, this is a huge (and risky) transformation in a character that has played second fiddle till now, and opens up more questions, but you don't care, you're enjoying every moment of it. That moment that Elsa smiles for the first time and then later when she lets down her hair, that's when you fall in love with her. We're given something juicy at last, and hopefully it will pay off.

Second Act

And we're back to Anna, who's extremely good natured about the whole thing about her sister running away, and keeps playing the one-dimensional selfless sister, even blaming herself for Elsa freaking out. She teams up with Kristoff (a really likeable character) and meets Olaf (who's funny despite the irritating trailer but has a pointless song) and then finds her sister.

Anna is just a selfless lover who's determined to find Elsa and doesn't want her to be alone, and claims that she is not in the least bit afraid of her sister. Again, this is strange given that we don't know how close the sisters are exactly. We simply have to take Anna's claim at face value, and assume that the fact that she wants her sister back and isn't afraid of her even after she's grown apart from her and discovered her powers, means that the events of the past night haven't really had much of an impact on her. Again, there's no character development with Anna. The scene where she hesitates before knocking on Elsa's door is beautifully thought out. "That's a first." The continuing use of doors as metaphors is good, but this moment seems to glide by too quickly. Why did Anna hesitate after all this confidence? I'd have liked to see more moments like this.

We finally see Elsa again (it has been over 20 minutes since we last saw the film's most interesting character). Her fear is still present, despite her transformation. A fear of letting people close because she might hurt them. Anna, despite seeing her new transformed sister, blindly and happily accepts Elsa's change and even apologises for setting it off (seriously?).

I expected a lot more conflict during this meeting. I was hoping the sisters would discuss the past, Elsa talking about who she really is, describing her new found identity, the years of repression, sharing her side of the story she's been hiding all these years, letting Anna know how much more difficult the isolation was for her, and telling her about her wiped memories. And probably have Anna show more shock and awe at Elsa's change, and get angry at Elsa's lack of trust in her despite her unconditional love, all the unnecessary secrecy, and having to bury their parents alone. But no, we get nothing. The sister don't talk discuss the past, Anna remains the same.

The only change we see is in Elsa, where despite her freedom, she's not truly happy and is still living in fear. And we finally see her become aware of her frozen kingdom and her inability once again to control her powers. OK, that's something. But it seems we're back to square one. Anna is still selfless and wants her sister back, and Elsa is still afraid and has no idea how to control her powers. Pretty much the same thing we've been seeing for the whole film.

So it seems Anna's character arc is really simple, and Elsa's major transformation in 'Let it Go' wasn't really that relevant. All that self-empowerment turns out to be a damp squib, since it counts for nothing. Elsa achieved some happiness at not hiding her powers anymore, but it doesn't really move the story forward since she's still afraid and can't unfreeze her kingdom. Kind of an anticlimactic let down after 'Let it Go'. Would the film have worked better without it? Plus, the whole self-empowerment thing in 'Let it Go' diverged from the storyline to begin with, Elsa's character arc so far seems to be only about learning to control her powers, when it should also have involved her relationship with her sister at some point.

Anyway, Elsa throws Anna out of her castle when she refuses to leave, accidentally fatally injuring her. This would have been a perfect time to let the viewers know what the point of the film is. We know by now that Anna is uni-dimensional, so the rest of the plot has to be about Anna's journey continuing till she gets her sister back, and Elsa learning to control her powers, but the story diverges from this point onwards.

Anna meets the trolls, who tell her how to get cured. There's an irritating unnecessary song. Neither the trolls nor Kristoff tell Anna about her past and her memories being wiped, which is weird as that would let Anna know why her sister wants to be alone, and maybe make Anna more fearful of her sister, and their eventual reunion tougher to achieve. And the focus now shifts away from her relationship with her sister to her love triangle (this story is jinxed). In the meantime, Elsa is captured and brought back home, but she's still scared and wants to run away (surprise!).

Third Act

There's this sudden plot twist with Hans, which I'd normally love given that we get so few animated films with sudden twists. But in this case it just seems to further muddle the plot. So what if Hans is evil? I don't really see a point to it, except to teach Anna a lesson about love, which seems to be more of a minor plot point. We know that the real story is Anna-Elsa (roughly), and Anna is one-dimensional, so the Hans romance angle was always going to be an unnecessary distraction whether he was evil or not. Anna learning a lesson about love isn't going to change the equation she has with her sister. One thing it does do is remove that one distracting notion we all had in our heads about Anna resolving this love triangle, as the story now leaves us with only Kristoff.

I love this tender scene between Anna and Olaf. It's a good way to remind the viewers what love is. Anna then realises it's Kristoff who loves her and is going to save her, which is by far the most pointless arc in this film. The filmmakers can't expect an audience to buy into love between Anna and Kristoff in the one and a half days that they've been together as friends, when the film has already mocked the true love between Anna and Hans happening in the less than one day that they've been together.

I get that Anna might not love Kristoff back at this point, but still want to kiss him if there's a chance it might save her life, but I don't get why she suddenly ignores Elsa's safety. Anna put her quest to get Elsa back on hold to come back to be saved by Hans. When that doesn't happen, and Hans reveals he plans to kill Elsa, Anna is concerned for his sister. Then when Olaf rescues Anna, she suddenly forgets her sister and focuses only on herself. I fail to see why. Anna has been mostly selfless till now. She might have wanted to let Kristoff heal her before trying to stop Hans, but that's only if she planned another quest into the mountains, in which case she could have at least alerted someone to Hans' plan. Also, what is Anna thinking as she's trying to escape and the castle begins to freeze over? For all she knows, Elsa is still in the mountains. She must be concerned about the sudden storm.

Anyway, Elsa has run away again (yup) and we finally hear some concern for her sister. Finally! We've had to wait till the end of the film for this revelation that belonged at the beginning. It doesn't mean much now. Too little too late. Where was Elsa's concern for her sister in the previous two acts? Where was it when she threw Anna out of her castle Moving on, in the process of finding Kristoff, Anna sees Elsa in danger, wherein she sacrifices herself for Elsa, thereby saving her own life as well. I get the 'self sacrifice as the ultimate act of true love' part, I just don't know why they had to create a love triangle to make it happen. Surely there could have been less confusing ways?

Also, the self sacrifice part seems kind of tacked on, like the film makers knew they wanted this scene in the film and worked the rest of the film around it. It seems to just happen. It works for Elsa, since she has no goals and is just a victim, but not for Anna, since she's one-dimensional and this decision (her sacrifice) doesn't reflect anything she learned about her sister or true love during her journey.

Anyway, Elsa is so grateful and relieved to have her sister back, she doesn't seem to be afraid of hurting her anymore, her fear seems to have disappeared, but for some reason she's happy. I don't get this. Elsa's a victim. Like a concentration camp survivor. Her freedom from fear after 13 years of isolation should leave her confused and uncertain, not happy, just as freeing a prisoner after years of isolation won't have them jumping for joy. To buy into Elsa's happiness, you have to assume this is what she wanted all along, but we never observe this in the film. We only observe fear. We can infer she wanted normality and a relationship with her sister, but she never expressed a desire to do anything about this, no goals. Therefore, her happiness just seems tacked on, like a kind of reward for no goals.

Also, Elsa is still not sure how to fix things, until Olaf reminds her it was 'love' all along. And that allows her to suddenly manipulate snow into disappearing and return summer to her kingdom. Then she's suddenly comfortable using her powers in public again. And the film ends. A quick emotionally unsatisfying cop out ending. Elsa's final transformation does't feel explained. How do you go from 13 years of suppression to 2 days of using your powers to suddenly learning to reverse their effects? Elsa's story doesn't seem coherent.

----------------------------------------------------------------------------------------------

On Elsa

Here are the problems I have with Elsa's depiction in the film.

First, Elsa is ignored despite being the more interesting character.

The best parts of the film involve Elsa - Elsa playing with Anna, talking to the troll, shutting people out in 'Do You Want to Build a Snowman?', being a nervous wreck during her coronation, talking to Anna, fleeing, singing, transforming, being scared again, struggling, fighting and running. On viewing the film again, I found I only wanted to watch it for Elsa. Both her character and journey are more interesting than Anna's.

Elsa is this beautiful, powerful, troubled, fearful, damaged, refined, tragic character, someone you can't help falling in love with. I'm not sure why I love Elsa so much. Maybe it's because of the qualities above, or that she could have worked so well in the story and yet doesn't, or maybe it's because she's the mature counter to Anna's clumsy self. Trouble is, Elsa is never really given much screen time compared to Anna.

Elsa spends most of act one being scared. We're then shown part of Elsa's change about 1/3rd into the film during 'Let it Go', at which point she has a sense of self-empowerment and you empathise most with her, but everything from then on is mostly Anna's love triangle with Elsa relegated to the background. At this point you wonder why she's ignored for so long. When she finally reappears it's only to have her get scared again, get captured, and give up, until she's healed by Anna.

There's no relevant internal change on Elsa's part, nothing remarkable in the film w.r.t self healing. All her actions only serve to propel Anna's journey forward. This is fine for a supporting protagonist, but it doesn't work since Anna's story isn't as interesting, raising expectations from Elsa's storyline. Unfortunately, Elsa comes across as more of a non-villain antagonist, but so poorly presented that she ends up being more of a plot contrivance for Anna, rather than the complex protagonist that 'Let it Go' and the rest of the film try to make her out to be. And again, this would be fine if Anna's story is more interesting, but as I've said earlier, it isn't. Which is why Elsa should have been given more screen time.

Second, Elsa's character arc is not coherent.

Elsa was always a doomed character. Nothing she tried was ever going to work. She couldn't help herself. She needed Anna to save her. And she didn't know this. This is her true storyline. To try and fail in ignorance until Anna saved her.

On the contrary, what the first act of the film does is make Elsa responsible for finding a way to control her powers. Nothing is said w.r.t Anna's role in this. Then 'Let It Go' establishes a new powerful identity for Elsa as an individual, without Anna. This arc leads us to expect something even bigger from Elsa towards the end w.r.t healing, an even bigger transformation where she finally achieves control of her powers, inner peace, true happiness, a sense of comfort with herself, with help from her sister. So when the film then shifts to Anna's journey and her love triangle, reverting Elsa to her fearful state and then having Anna heal her of her fear in a second, without explanation, that ending is just too quick given what we've seen and come to expect from Elsa, making her character arc seem like a sham, an emotional roller coaster ride that's just a plot contrivance for Anna's character arc.

There's nothing wrong with having Elsa as a doomed character or having Anna heal her. What's wrong is playing up Elsa as the more interesting movie character and then ignoring her story arc for Anna's less interesting one, and having Anna heal Elsa too quickly, without any context.

This is partly the fault of the ending being too quick, leaving viewers with too many questions and a lot to infer about Elsa, but part of the fault also lies with the content and purpose of the song 'Let It Go' itself, and how it was played up within the story. It deviates from Elsa's original character arc as a supporting character, and gives her more personality than the previous parts of the film did, making the jump in character too large, and which the rest of the film doesn't pick up on, making the jump a waste. More importantly, the song deviates from relationship building between the sisters, and takes us in a new direction by introducing a new angle to Elsa. This puts a lot of pressure on the rest of the film to tie up these loose ends, which it fails to do at the end.

Third, Elsa's ending leaves you with a lot to infer.

The film ends with Elsa not being scared anymore, and learning to control her powers. She's filled with happiness and relief after her sister comes back to life. We presume that this feeling replaces or overrides her fear of hurting her sister, as she's not afraid to touch her anymore. Additionally, her knowledge of love thawing what's frozen allows her to reverse or manipulate her powers to remove winter somehow, and presumably, with her fear of hurting her sister replaced, she can now wield snow and ice without fear of hurting anyone anymore, and is fine using her magic again.

How exactly is Elsa different now compared to at the beginning of the film? She isn't. She's back to square one. Both girls are. Anna has her sister back. And Elsa not only has her sister back can also use her magic again without fear.

Here's when you infer that the real villain in the film was fear all along, and that all Elsa needed was to not be fearful. She never needed to control her powers by suppressing them. She simply needed to use them normally, which she was never given a chance to do because of the accident that made her parents make her suppress her powers in the mistaken hope that this would control them. The accident was just a one off event, but her parents made her fear it, made her believe that that was a sign that her powers were out of control and needed to be controlled and suppressed, which was wrong. She has been doing the wrong thing for 13 years. Living in unnecessary fear for 13 years because of her parents. A horrific thought.

You have to infer all of the above in hindsight, that Elsa never needed to do anything special or learn of any technique or trick to control her powers. Her powers were fine as they were, but she needed to let go of her fear and replace it with love instead. A pretty complex inference to be made by the viewers at the end of the film.

I mean, the trolls themselves ask Elsa to find a way to control her powers at the start, hinting at the dangers that await her if she doesn't. This and the fact that her story involves unfreezing her kingdom leaves us to think that Elsa needs to do something different to control her powers than she did as a child. We think the film will involve her discovering a way to control her powers at some point. To end the film by saying that this was all wrong, she never needed to find a way to control her powers, she was fine all along, the only thing she needed was to not be fearful by using love, or accepting someone else's love, is a bit of a cop out, leaving the viewers underwhelmed, since we've been expecting something else. Your reaction at the end when Elsa ends winter is "wait, that's it?, that's all she had to do to control her powers, she couldn't have figured that out for herself in 13 years?" Why didn't the trolls just tell her? Why did they mislead by asking her to control her powers instead of just being herself and using love?

Fourth, Elsa's ending leaves a lot of open questions.

Elsa suppresses her powers for 13 years because she's scared of hurting people, undergoing who knows what kind of psychological damage, then suddenly transforms into an ice queen for 2 days, where her identity is inexplicably shown to be defined by her powers (though her fear is still present), and then suddenly lets go of her fear in a second with 'love'.

Elsa's fear disappearing doesn't feel explained. How do you undo 13 years of fear in a second? Is this only a temporary phase? Will she be afraid again? What if she accidentally hurts someone again, like she did when she was little? Another trip to the trolls to get that person healed? It's not like she has the power to heal people.

All we get to see is Elsa is not scared anymore and the film ends. I think it's all too convenient. We're supposed to take Elsa's healing for granted. A little more explanation would have been great instead of making the audience infer or guess at what might have happened in her mind, how exactly love replaced fear.

Fifth, Elsa's ending isn't as impressive as 'Let it Go', leaving the film imbalanced.

Elsa's power ballad is simply a work of art, something I keep going back to. The best bit of the song is at the start when her other glove comes off, and her loneliness gives way to her initial joy at being free to be herself. It's even better than her full transformation towards the end of the song because there's just so much realisation on her part at that initial point w.r.t her freedom. And this is the first time since her childhood that we see her happy with her powers. A powerful moment. Her transformation works emotionally for the audience because it's so out of place, so out of character within the context of what we know about her and the story so far. It's a leap of character development. That's what hooks us to this one scene, and makes every other scene inadequate and wanting.

Sure the song placement is wrong. Anyone who flees in fear after years of isolation isn't going to celebrate a new type of loneliness. Elsa might not be repressed anymore alone in her ice castle, but she's still afraid. Physically, she's free, but in her mind, she's still in a cage. So the best part of Elsa in the film is actually one of her worst parts, where she's still living in fear, but thinks she's free. But this isn't a bad thing as long as what follows in the film manages to reconcile this contradiction, and do better than the song.

Sadly, nothing does. Nothing following 'Let it Go' comes close to matching it in entertainment value, or in logically progressing Elsa's story. This was the high point of the film, and everything else is downhill from here. This wasn't the end of Elsa's character arc. You would expect the actual end to be better, more satisfying. Which it isn't. This is one of the reasons the film doesn't work for me.

Sixth, Elsa doesn't get closure.

Elsa's realisation of her final transformation where she lets go of her fear, and of the implications i.e. her parents were wrong, the guilt, all those years of unnecessarily shutting people out and living in fear, find no mention at the end of the film. Also, there must be a lot of stuff she needs to talk about with her sister. Apologise. Tell her about her wiped memories maybe.

As a kid, Anna loses a sister. But Elsa not only loses a sister but also knows why, and has to keep this a secret for 13 years. I imagine this is even more frustrating and tragic. The only people who share her secret are her parents, and they're really her only emotional support for 10 years. When they die, we get to see Anna sing about being alone, but we have to infer that Elsa's situation is now even more difficult, lonely and tragic, because she's lost her only crutch. Which is what makes her transformation in 'Let it Go' so satisfying to watch. It's been Anna's journey till now, but we finally get to see a bit of who Elsa is. And when that transformation doesn't work either, and when Elsa finally learns not to be afraid after all the running away and shutting people out, imagine the kind of emotional baggage she has left over. We really need some of that to be portrayed.

Seventh, the sister's reunion at the end (where they hug and then skate) seems incomplete and one-sided.

We're shown Anna's longing for her sister. That's her want. That's why she goes on an epic journey. To get her sister back. The journey is her need. This is Anna's goal. This is how she attains emotional fulfilment and learns about love and saves a kingdom in the process. This is clear. But Elsa's arc seems to be more about controlling her powers rather than being reunited with her sister. We're never explicitly shown a direct account of Elsa's longing for her sister that can be translated into a goal.

The sisters were great friends at the start of the film, but they grew apart, and Elsa goes on to want different things, and the film never reconciles this gap. Elsa's situation during 'Do You Want to Build a Snowman?' is more about sadness, during her coronation it's about control, sadness and frustration, during and after 'Let It Go' it's about being free and embracing her inner self, and the scenes after this all involve fear and sadness. The viewer could infer that that Elsa's sadness is due to an unfulfilled desire to have a normal relationship with her sister, but this doesn't translate to an overt actionable desire for a normal relationship with her sister.

We can infer clearly that Elsa wants to control her powers and not be afraid, but we aren't shown her need i.e. how that might happen, or what she wants to change exactly w.r.t this or reuniting with her sister. Put simply, Elsa doesn't have any goals, she's just a victim. So when Anna's arc crosses over with Elsa's, and Anna 'heals' Elsa, we're just supposed to accept it, without having anticipated it. This works for Anna's story arc (in retrospect we understand that Anna's only power is love, and that this was the answer to Elsa's question to her at the ice palace - 'what power do you have to stop me?') but it doesn't work for Elsa's arc, because we have no idea if this is what she needs. We're just shown that it works.

Which is why the film's end i.e. Elsa being reunited with her sister, is incomplete, because we can't appreciate her happiness at being reunited with her sister because she has spent most of the film pushing her sister away. We're constantly shown Elsa being afraid and running away, we aren't really shown what might help her, or what her desires or motivations are that might lead to a resolution. Sure, Elsa needed love, but we only see that in hindsight. There was no context, no foreshadowing indicating that this is what she needed all along. So when she's finally happy at being reunited with her sister, isn't she really just happy about being able to control her powers at last? Shouldn't Elsa's reunion with Anna leave Elsa uncomfortable and guilty?

The bond between sisters is more one-sided, and playing it up at the end of the film doesn't really mean much for the viewers. It feels empty. The story is more about Anna, who gets what she wants, and Elsa, who get what she needs, which turn out to be two different things. The reunion at the end feels inadequate, like it's missing Elsa's thoughts and emotions.

--------------------------------------------------------------------------------------------------

On Anna

I understand that the director Jennifer Lee wanted the film to be told through Anna's journey. She called it 'Shawshaky' at one point. Where Elsa and Anna are both protagonists but you're watching Elsa's story through Anna's journey. Or that Elsa is the one driving the protagonists's (Anna's) actions.

Ms. Lee says at one point, "Yeah and we knew her journey, we knew that Anna was an ordinary girl that’s got love as her only “superpower”, really. And that her journey is going to go from not understanding love (because like all of us at eighteen, growing up, we don’t), to mature love and the ultimate kind of understanding which is, you know, the sacrifice you’re willing to do for love." Which is all fine, but here are some problems I have with this portrayal of Anna in the film.

First, Anna is just not that interesting compared to Elsa, intrinsically. She's perky and optimistic, but we've seen this character in other films. She does mellow down towards the third act, but that's more due to her being struck by Elsa than any internal change, and she's back to her normal upbeat self at the end of the film.

On subsequent watching I found myself skipping or fast forwarding through most of the scenes that involved Anna talking or singing. The only scenes with Anna that I felt were worth watching again were the ones with Elsa in them, or the main one with Olaf, or her self sacrifice one. This is just a personal preference though.

Second, Anna's journey isn't as interesting as Elsa's. Elsa's story is more interesting, because her conflict is internal - coping with fear. Anna's conflict is mostly external - her relationship with Elsa - which is less interesting. Sure Anna resolves her conflict herself, unlike Elsa, but she doesn't have to change to do so, so it's less interesting (more on this later).

Anna, despite being likeable, having this heroic quest to suffer through, achieving her goals, getting her sister back and saving her from herself, understanding what love means, and saving her home and kingdom, takes a backseat compared to what's happening with Elsa. This could be partly due to the fact that Anna doesn't have a very interesting or unique personality, as mentioned before, but it's also partly due, I suspect to the fact that Elsa's story and character development are more interesting.

Third, Anna doesn't change much, or in any relevant way. Yes, she's fun to watch, and goes on this incredible journey and learns about love and changes everything, but she doesn't exhibit much personal growth or character development relevant to the story.

We see Anna growing from not understanding love to understanding that love is putting others before yourself. In the ballroom, Elsa asks Anna what she knows of true love. In the ice palace she asks her what power she has over her. These are fantastic clues telling us how Anna's journey will end (Anna herself doesn't know the answers to these questions but her eventual actions reflect an understanding of true love). Anna does change in this respect. She learns about love. The problem is, none of this is relevant to her journey.

There's nothing in the film that indicates that Anna wouldn't sacrifice herself for her sister if her quest hadn't happened. Indeed, Anna seems constantly unwavering in her love for her sister throughout the film. So the love triangle and having Anna learn about true love seem like more of a plot of convenience to make it seem like her choice to sacrifice herself for her sister is based on what she learned during her journey rather than her own intrinsic qualities.

Now this wouldn't be such a bad thing in itself. Yes, it's always nice to have your main character undergo internal change, but lots of great films work without this. Like 'Wall-E'. There's no character development or personal growth. The film is about a goal, and the hurdle-filled journey the lead character goes through to reach that goal. And it's a great film. 'Frozen' would have been the same if the film was just about Anna. But like it or not, Elsa steals the show. Elsa's character and journey are more interesting, making Anna's seem bland in comparison. In view of this, it would have been better to give Anna a better personality, or relevant character development.

-----------------------------------------------------------------------------------------

On the Plot

The film fails most in not creating a simple straightforward plot. The big epiphany of Anna's sacrifice doesn't feel earned. We're supposed to assume the girls love for each other hasn't changed in 13 years, without being shown this at any point of time in the film. We're shown Anna's want w.r.t this, but not Elsa's. When the payoff/resolution happens, we feel nothing, no emotions, except on introspection and repetitive viewing, so the resolution feels a bit superficial. You see that the sisters are happy, but you don't feel happy, because the whole relationship seems to be underdeveloped and one-sided.

So essentially the film's final resolution with the self-sacrifice bit involves one protagonist (Anna) taking a decision that more or less doesn't reflect much growth and is more of a 'right place at the right time thing', cueing a change in the second protagonist (Elsa), who isn't really a protagonist, after a lot of misdirection, and this change in her (her loss of fear), though internal, is not because she wanted or expected it, and no one knew that's what would help her anyway. It just happens that way. And the audience is pretty much once step behind during this final resolution and are left underwhelmed.

Disney just didn't manage to pull it off. Almost, but not quite. They created a troubled character and then decided to tell a story about her less interesting sister instead. It's a film about sisters without any actual relationship building between the two sisters, leaving you with an incomplete story, making it a mediocre product overall.

The characters are extremely likeable. Anna is adorable. Elsa is captivating. The animation with snow and ice is beautiful. The humorous sidekick is good, despite a misleading trailer. The music is fun. But the plot just doesn't work. The viewers are always one step behind. The story is incomplete and meanders. There's at least one unnecessary song. You don't really know why things are happening with these characters you love, specially at the end of the second act and most of the third act up to the final resolution. And the payoff doesn't feel earned. When the sisters finally find happiness, you're not sure what to feel.

They might have improved the film by making Elsa less interesting, continuing to portray her as a sad brooding scared person rather than have her achieve a false sense of self-emowerment at the end of the first act. Of course, this would have meant changing 'Let It Go', which is why we all love Elsa to begin with. Or they could have broadened her character arc, giving her more background context for her actions. Or made Anna a more complex character, with more anger perhaps.

I can forgive everything else, the trolls, the misdirection, everything, for just a little more Elsa in the film, for just a couple of scenes during the first act (during 'Do You Want to Build a Snowman?' and Elsa's coronation) and second act (a scene in her ice palace after having thrown her sister out) reflecting her desire for a normal relationship with her sister, and also at the end of the film, during the final resolution, reflecting an understanding of how she has come to control her powers and the implications of this. This would have given Elsa enough context to make the final payoff and reunion seem worthwhile.

[this book, part of the story canon, gives you more context on the sisters' relationship. it would have been great to have more of this in the film. I understand the first act was already heavy, but more context would have only made the film better.]

----------------------------------------------------------------------------------------------------

Why Care Anyway?

Most poor films are poor for good reason. Nothing tends to resonate with you. You don't care about the story or characters. 'Frozen' is different. It may be a mediocre film on grounds of a poor plot and character development. But it works thematically and emotionally, and how! It manages to somehow successfully play with mature themes and extremely likeable characters that hook you, reel you in, and leave you wanting more.

Most people, me included, don't mind watching a film without much character development as long as we get likeable characters and a plot with a resolution. We just love sinking our teeth into a character, even a simple one, as the film takes them on this torturous journey. We love this kind of emotional fulfilment. This is exactly what most action movies do. Think back to all the Marvel movies you've seen. Characters don't really develop. They simply blow stuff up and achieve a resolution. But 'Frozen' doesn't even do this. It doesn't even have a real plot or satisfying resolution.

What 'Frozen' does instead is capture and transfix us with these incredible themes, characters, moments, music and silences, to the extent that we don't initially care about the story or flaws anymore. They seem minuscule compared to this new world we are now a part of. And that could be why we look past its flaws and still enjoy the film.

Right from the background score during the opening credits (setting the mood for the film), the opening song with the ice harvesters, the scene with the girls playing (notice the silence), the trolls and the introduction of fear, the serious narration over events as the king speaks, the sad exposition by song, the initial conflict, Let it Go, the darkness, the beauty, the animation, Elsa's sheer presence, Kristen Bell and Josh Gad's extremely likeable voice overs, the facial expressions, Olaf's tender conversation about love with Anna, the powerful self-sacrifice scene on the frozen fjord, and again the use of silence throughout the film, all these elements simply work at an emotional level and work together to do something few animated films do - they make you feel. Feel for characters and themes so much that you forget the film doesn't have much of a plot, and leave you wanting more. One could say 'Frozen' is not so much a film as a collection of likeable characters and beautiful powerful moving moments with a plot loosely thrown in to connect them.

Is this why Frozen transfixes us so much? I would still say 'no', not exclusively, there's more. Consider other recent well-made and well-written films like 'Spirited Away' or 'Wall-E'. Not only do I respect them technically but I 'feel' them as well. They fill me with awe and wonder. I recognise them for the epics they are. They all have the same elements that I enjoy in 'Frozen', the same mature themes, silences, music and likeable characters. Yet they don't transfix me in the same way that 'Frozen' does. What's so different about 'Frozen'? The only difference I see is the lack of a plot and character development. Maybe that's it. Maybe it's these flaws that make this film move us so much.

The film keeps torturing the audience with questions about the plot and by Elsa, but never answers them, leaving the audience to infer the answers for themselves. This process of filling in the gaps coupled with the mature themes and likeable characters leads to you having a more personal connection with the film than if it were simply a good film.

You're not sure why Anna's memories are erased, but you go along with the plot, you don't know what kind of relationship the sisters have after 13 years apart, so you skip over the gap, but it's still there in the back of your mind. It's the same with the sister's relationship from Elsa's viewpoint, which is non existent, and from Anna's view point, which is incomplete. Ditto with Elsa's inconsistent character development, the unnecessary misdirection with the love triangle, and the quick one-sided resolution. All of these are inconsistencies that stick in your mind to some extent, but the most important one, IMO, is the lack of context for Elsa.

Elsa's story is just so incomplete, it hurts. More so because Elsa as a character is simply mesmerising. Anna, Kristoff, Hans, Olaf and Sven are merely forgettable comic relief compared to her, they're your standard Disney animated cardboard cutout characters. Spunky, optimistic, extroverted, grounded, evil, confident, funny, innocent, overly anthropomorphic. We've seen them all before. They're stereotypes.

Elsa is different. They broke the mould with her. Elsa's wants have nothing to do with her relationship with her sister, which is what the film is about. Elsa's only want is to not be afraid. This is incredibly dark, mature and tragic, making her a unique character like no other. Combine this damaged part of her character with her other refined qualities, and we're hooked. So when the film then proceeds to give us this tantalising transformation where she turns into an ice-queen, we're more than hooked, we're part of her self-realisation. We are Elsa. Of course, this character change can't sustain itself without support from the story, which never happens and so the ending sucks, leading to you feeling empty after watching the film. A lack of closure. Disappointment in the story. But you still worship the character. You want to see more of her.

So what the film does is it gives us this character, Elsa, who is just so far ahead of all the other characters in terms of how interesting she is, lets us fall in love with her, and then never really give us a satisfying story about her, leading to an even greater desire to see more of her. Do you see my point? Do you see how ironic this is? The same reason why Elsa doesn't work well for the story is the same reason she works so well for the audience.

Do you see how crazy this is, how unintentionally brilliantly exploitative this is? You want to see more of Elsa, more exposition from her. You expect more about her tragic past, her relationship, her guilt. And the film builds this up by dangling Elsa in front of you every now and then without really giving you enough of her. It's like they're practising 'less is more' with Elsa. Which actually works. The audience is left wanting more of her, expecting more from her, right till the end, which turns out to be disappointing, and even after the film ends.

You let all these inconsistencies and gaps gather at the back of your mind as you watch the film, as the characters grow on you, as you hope for a final resolution which never really comes. The ending is infuriating in it's inadequacy, but the inconsistencies are still there. And you care about them because the film's elements are so powerful, and you identify with the characters. So you can't wish away the gaps. You're aware the film has flaws and open questions, and you need to reconcile them. And as you try to do this, you admit that the film was powerful enough to makes the gaps stick to begin with. They're embedded in your mind, they're a permanent part of your psyche. You try to fill in the gaps yourself, by inferring what you must, in order to regain your peace of mind, leading you to form a connection with this film like no other.

Other animated epics are compelling emotionally fulfilling perfectly crafted stories about likeable characters. Which is why we praise them and enjoy watching them. 'Frozen' on the other hand is a compelling emotionally unfulfilling incomplete story about likeable characters, leaving you wanting more. That makes 'Frozen' a worse film technically, but might also make you connect with it and the characters and in the long run become a bigger fan of the franchise in a way no other film can.