Hernan Amiune Blog

The Mentor's Dilemma

2012-05-22 15:01:06.037664

As a teacher and entrepreneur, students and friends often ask me if they should pursuit an idea that I know it's very hard to achieve. In this situation I always have the following dilemma: Should I encourage them to pursuit the idea without warning about how hard it will be, with the risk that they fail, get demotivated and never try it again? OR Should I warn them about how hard it will be with the risk that they won't even try something that maybe they would have achieved? After watching <a target="_blank" href="http://www.youtube.com/watch?v=AY4ajbu_G3k&t=14m3s">this video of Andrew Ng</a> I realized I was not the only one with this dilemma.

The things I don't want to trade off

2012-05-22 14:28:27.371563

With time you gain experience and improve yourself but like everything in life there is always a trade off like the <a href="http://en.wikipedia.org/wiki/No_free_lunch_theorem">no free lunch theorem states</a> These are the things I don't want to trade off: While I gain patience to wait for things that take time I don't want to lose my impatience to do things as fast as I can. While I gain knowledge about how hard things are I don't want to lose my innocence to try things that seem impossible. While I know more about the world I don't want to lose my eager to know even more.

A/B Testing, Hill Climbing and Simulated Annealing

2012-03-04 23:35:27.711371

Some well known entrepreneurs like <a href="http://blog.asmartbear.com/local-minimum.html">Jason Cohen</a>, <a href="http://www.codinghorror.com/blog/2010/07/groundhog-day-or-the-problem-with-ab-testing.html">Jeff Atwood</a>, <a href="http://www.smashingmagazine.com/2010/08/26/in-defense-of-a-b-testing/">Paras Chopra</a> and <a href="http://www.seomoz.org/blog/dont-fall-into-the-trap-of-ab-testing-minutiae">Rand Fishkin</a> have been discussing about A/B testing and local maxima and while each of these posts are interesting to read and each one have good points, they seem to miss the fact that this is a very well known problem in computer science and there are some really interesting mathematic facts that I would like to highlight: <ol style="padding-left:20px;"> <li>A local maximum is better than a local non-maximum</li> <li>The search space is exponential, so if you want to find the global maximum of n A/B tests you will need to perform 2n tests</li> <li>To avoid doing exhaustive search and also avoid getting stuck in a local maximum there are two well known algorithms that can help, these are <a href="http://en.wikipedia.org/wiki/Hill_climbing">hill climbing</a> with restart and <a href="http://en.wikipedia.org/wiki/Simulated_annealing">simulated annealing</a>.</li> <li>If you have an algorithm to find the global maximum in polynomial time you probably have solved the P=NP problem.</li> <li>In the case of A/B testing in web pages if you consider every possible different web design that you can imagine as a test then you can’t prove that a global maximum exists because the number of possibilities is infinite and the function is not continuous.</li> </ol> This is the theory and now here is how to take advantage of this theory Your search space is the number of different web designs you can and want to implement and not the infinite number of imaginary web designs. Suppose you have a webpage with a Text1 an image of a Man and a Red button and you want to try a Text2 an Image of a Woman and a Green button. Then you have 23 = 8 possibilities. If you can try the 8 possibilities then you will find the global maximum for this search space. These are the 8 combinations with their respective values Text1,Man,Red = 0.4 Text1,Man,Green = 0.5 Text1,Woman,Red = 0.5 Text1,Woman,Green = 0.9 Text2,Man,Red = 0.3 Text2,Man,Green 0.4 Text2,Woman,Red = 0.8 Text2,Woman,Green = 1 If you don’t have time or resources to test them all then try hill climbing, you maybe end in local maximum but that’s better than nothing. An example of hill climbing will be You test Text1,Man,Red vs Text2,Man,Red and you get Text1 > Text2 You test Text1,Man,Red vs Text1,Woman,Red and you get Woman > Man You test Text1,Woman,Red vs Text1,Woman,Green and you get Green > Red So you end with Text1,Women,Green You can also use the simulated annealing algorithm and sometimes stay with changes that perform slightly worst in order to try to find a global maximum especially if you have a good feeling about those changes or if they improve speed or other aspects. Conclusion Test as much as you can. If you can't do exhaustive search, use known search algorithms like hill climbing or simulated annealing.

How much is an idea worth? Between $100 and $1000 is what I got.

2012-02-17 16:22:17.913995

If you are an engineer or a programmer I’m sure people come to you with “awesome” ideas all the time, and they probably are protective of their ideas as if they have a brick of rhodium in their hands. Now if you have some experience trying to convert an idea into a profitable business then you know that ideas aren't worth very much. And a lot <a href="http://sivers.org/multiply">has been said</a> <a href="http://www.codinghorror.com/blog/2010/01/cultivate-teams-not-ideas.html">about this</a>. Here, as I always try in my posts, I would like to give you another point of view, approaching this issue in a more mathematical way of reasoning. So let’s use something similar to the beautiful logic tool of <a href="http://en.wikipedia.org/wiki/Proof_by_contradiction">proof by contradiction</a>. Let’s think that you have a great idea. Let’s suppose you have an idea for a search algorithm that is ten times better than Google’s algorithm and let’s suppose that nobody will have a similar idea and nobody will be able to copy it or steal it. Now you need to think if it is possible to implement it. Is it theoretically, technically and economically possible and you have enough money to implement it? Let’s suppose the answer is yes. Now you need to make people use it. Do you have the money to advertise it and attract users? Let’s suppose the answer is yes. Now you need to earn money each time people use it. You can charge them or put advertising. Let’s suppose here that you copy the advertising model of Google. Now you need to make sure that your revenue is greater than the operating cost and taxes. Let’s suppose you can achieve this. Now you are rich! Like you can see its not impossible but there are a lot of suppose between the idea and the “you are rich” and each suppose seems to be pretty hard. In general, and theoretically, the best idea would be something that only you can do (or you can patent), that is very simple for you to do it and that you can charge other people a lot. An important thing to consider is that there are a lot of people in the world and the way all think is pretty similar so probably there are a lot of people thinking about something that is very similar to your idea. In fact I’m sure that this already happened to you. You have an idea then you search on the internet and there is someone already trying to do the same thing. It probably also happened to you when studying science, you read a couple of theorems in a book and you came up with a new one just to realize that it is in the next page of the book. Beside all of this I really like to think about new ideas. I have a lot of ideas written down and I usually sell them or give them for free if I think that I can’t implement them or I can’t get enough money or I can’t achieve any of the issues I mentioned before. Lately I’ve been participating in idea generation contests on TopCoder where you submit your idea and if the client chooses it then you get paid. I got paid between $100 and $1000 per idea and I’ve been quite successful there, some of my ideas where selected by companies like Alcatel-Lucent, PayPal, AOL, Truveo, Ribbit, BestBuy, TopCoder and others. It’s not a lot of money but I think it’s fair enough for just an idea. While what I have stated before is kind of disappointing about ideas this shouldn’t discourage you to pursuit your idea, it should motivate you because making that idea a reality is what it worth a lot. In fact besides that I have sold a lot of ideas I’ve been working for more than three years now on my own startup which is now profitable and growing. Maybe it’s not the coolest or innovative idea but it’s the one that I thought it was achievable considering where I am (Latin America), the money I got (almost nothing) and a lot of other aspects. And I have to say that I’m pretty happy with the choices I’ve made.

Learning isn’t fun. It was never fun. And it will never be fun.

2012-01-27 16:02:24.400778

If you agree with the following definition of learning: Learning is the process of failing repeatedly in the pursuit of achieving something until you finally succeed. Then learning isn’t fun by definition. Let’s prove it: <ul> <li>If you don’t like failing then obviously it’s not fun because failing is part of the process of learning.</li> <li>If you do like failing then you won’t learn anything because you are already happy with the result of failing so you don’t have much incentive to succeed.</li> </ul> But trying to make learning fun is something that everybody is trying to do these days so probably they mean something different than fun. Usually this new wave of people proposing fun learning put games as an example. So let’s create a couple more definitions in order to understand where this new wave goes: Benefit: is the benefit or reward that a student gets when the student achieves the task that it is supposed to learn. Examples: if she is learning to play guitar then the reward can be being able to play a song that she likes or becoming a famous guitarist. Cost: is the cost of failing in the learning process. This can mean the cost of mental and physical effort. It can mean the cost of spending a lot of time. It can mean the cost of humiliation in front of other people. Etcetera. Every time a person wants to learn something the person weights the Benefit versus the Cost of learning. If the benefit is zero or the cost is infinite then nobody will want to learn that. An interesting property of learning is that the higher the cost the faster people learn because people don’t want to make the same costly mistake again, but if the cost is higher than certain threshold the person will quit learning. Also interesting is that both the Benefit and the Cost of learning a specific activity are assigned by each person independently and it also varies in function of time for the same person. For example getting hurt can have a very low associated cost for one person and very high for another. In games, especially video games, in order to make people play more the Cost is reduced to nearly zero, you usually play alone or in anonymous mode so there is almost no chance of other people mocking you, the physical and mental effort is also minimal, you only need to make a couple of clicks or press a couple of buttons in order to construct an entire city or to make your character play golf even better than Tiger Woods. With the rewards, game creators try to trick the mind to make rewards look as if they were bigger, like telling you that you are the best of the world. While this doesn't fool many people, the ratio Reward/Cost is still very engaging due the low Cost. The problem when you want to translate this technique to the learning world is that the Costs are much much higher than in games. Learning to play an instrument, or learning to write a computer program or getting in shape take at least between 6 and 12 month and a lot of physical and mental effort in comparison with learning to play a video game which actually takes a couple of clicks and less than a day. While almost everything I wrote before is not very positive about game mechanics I think there are some good things in this new wave of gamification or game mechanization. A lot of people are trying to break hard and complex things to learn into smaller parts and give rewards for small progress which helps people to measure progress, keeps them motivated and reduce the number of people that quits.

Understanding

2011-11-07 12:44:51.798042

Here I just want to share one of the greatest quotes I've heard from one of my teachers: "At first you probably won't understand this but then you will do a lot of exercises until suddenly one day you start thinking you understand it." .- Nicolas Wolovick He obviously said this as a joke, partly at least, but what he said help me a lot with the way I try to understand things. Now I can deal much better with not understanding some parts of some complex theories. I leave the interpretation of this quote to the reader. But <a href="http://hernan.amiune.com/blog/entry/to-be-good-at-math-you-dont-need-to-understand-everything">here is a hint of my interpretation</a>.

Breaking the The Zen of Python

2011-08-26 23:50:37.795992

I think the <a href="http://www.python.org/dev/peps/pep-0020/">Zen of Python</a> is great. It's a list of good practices that will make your code less error prone and easier to understand for you and other programmers. But if you are trying to create a programming language for a machine with artificial intelligence you will likely want to break several of the principles of the Zen of Python. Why is that? Because the most "intelligent" machine that we know today is the human. Humans are "programmed" using natural languages like English. And natural languages break a lot of the principles of the Zen of Python. These are at least some of the principles that you would like to break: <ul> <li>Simple is better than complex: This is subjective, simple for one person can mean complex for another person. And more important, <a href="">the world is complex as Vladimir Vapnik said</a></li> <li>Errors should never pass silently: People give orders and want results, no excuses about why the order was not executed. <a href="http://www.html5rocks.com/en/tutorials/internals/howbrowserswork/#Browsers_error_tolerance">You want as much error tolerance as you can get.</a></li> <li>In the face of ambiguity, refuse the temptation to guess: Ambiguity is not that bad because it can lead to creativity.</li> <li>Explicit is better than implicit: <a href="http://www.youtube.com/watch?v=3-son3EJTrU">Not always</a></li> </ul> For current programmers this may sound like hell, why would you want a programming language that doesn't always do what you asked to do? But that's the way we talk with other people. Every time you ask something to a person you expect some reaction from that person but you are never 100% sure what the reaction of that person will be. These kind of "fuzzy programming languages" will not replace current programming languages but they will allow to create new and interesting applications.

To Be Good at Math You Don't Need to Understand Everything

2011-08-14 19:23:11.842012

I always thought that when you study something you need to understand absolutely everything, specially in Math and Physics. One of my high school teachers told me once that you really understand something when you are able to explain that to someone who doesn't know anything about it. First I thought he was right and I wanted to learn absolutely everything and it worked great for the basic stuff. Then when I was studying more complex things I realized that it was impossible to keep doing this. Finally I realized that my teacher's statement is partially wrong and what it is worst is that it will slow down your learning and creativity process. I realized this after learning a lot of math, logic, formal languages and how definitions and theorems work. <a href="http://www.youtube.com/watch?v=wMFPe-DwULM">Here is a Richard Feynman's video that points out the problem much better</a>: <div style="text-align:center;"> <iframe width="425" height="349" src="http://www.youtube.com/embed/wMFPe-DwULM" frameborder="0" allowfullscreen></iframe> </div> A Definition is something that you accept without questioning, maybe there are good reasons for that definition or maybe not but you don't care, the only thing that you need to know is that it can be useful for something later. So every time you find the word definition there is nothing to understand. A Theorem is something that you must prove by using only the definitions that you have accepted. So every time you see the word theorem there is something important that you need to understand. But (and this is the important thing) once a theorem is proved then you can take it as a definition. So you can put your effort in trying to find a new theorem instead of trying to understand an old one that was already proved. Of course, leaning the proof of old theorems will help you to find new ones but what I really want to point out is that there is a trade-off between where you want to start and how far you want to go, there are things that you won't need to understand in order to create new and useful things. For example you take that 2+3=5 is a definition and this allows to use it to make really fast calculations when you are shopping for something so you get the correct change and don't lose money. But 2+3=5 can be actually a theorem and you will need to prove it and this will take you probably a couple of hours instead of a second. Another good example is to take math as a tool, in the same way as a hammer is a tool, if you want to build a house and you just want a hammer to hit a nail then there is no need to know how the electromagnetic forces of the atoms of the hammer work in order to make the nail go inside the wood. My approach now is: learn or memorize something and try to apply that to create something new or more complex, and only try to understand it if you really need it to do one of those two things. Today knowledge is so vast that you must specialize in order to came up with new things.

The Future of Education on the Web and The Curse of Multiple Choice Tests

2011-07-29 16:34:07.675739

Is great to see new startups doing great things for education, it seems that finally something useful is emerging for students who want to learn something on the web. I just want to warn about something I realized when I was doing my final work about online education at my university: Multiple Choice Tests. Multiple Choice Tests are bad because of the same reason they are good. Let me explain this, Multiple Choice Tests were created to be easily checked for correctness, this makes possible that anyone with the answers can correct these tests even when they don’t know anything about the subject of the test. This “advantage” of Multiple Choice Tests makes them perfect to be processed by computers. So when you are developing an eLearning program implementing Multiple Choice Tests is great because is really easy to implement. But is it also good for helping students to learn? The answer is no. They were conceived to be easy to correct but not to be a great way of teaching. They are great for the developers who are programming the eLearning site but not for the students. A good method for testing and allow learning at the same time will be one where the student has more feedback about the solution she entered. Where the student can see intermediate results of her solution, where the student can modify some parts of her solution and then be able to see the different outcome. In conclusion something where the student can learn from mistakes. Natural Language Processing and Artificial Intelligence would be a perfect solution but that is not possible now and won't be possible in the near future. The solution now are Programming Languages or Formal Languages in general. If students can write the solution to an exercise using a formal language this can be parsed and executed step by step giving the students a lot more feedback about their solutions and will allow much more interaction. Take for example a problem were you need to find a function as the solution. Instead of giving a multiple choice, the computer can parse the function typed by the student and show a graph and a lot of real time information about what is she doing that will help her to understand much better what is she doing. This is just one example, if instead of using just math we add loops and conditional statements things can become much more interesting. In my opinion, allowing students to execute their solutions step by step and getting feedback on each step helps them to improve their knowledge. This can be applied to several subjects like Math, Physics, Chemistry, Computer Science and Economy but can’t be applied to other subjects which don’t use Formal Languages. For those subjects, sadly, Multiple Choice Tests seems to be the best option so far if you want to create automated computer tests.

Memorizing, Understanding and Minimum Description Length

2011-05-26 23:28:55.963408

When I was at school I heard teachers say to students the phrase “you must not memorize, you must understand”. But I never really understood what they meant by that until I studied Machine Learning. Now I think that I have understood what they really meant but I have also realized that memorizing and understanding are complementary things. Let me explain this: In Machine Learning to teach a machine something you show it several examples and then the machine must learn from those examples and predict the outcome of new examples. When the machine can only predict the provided examples we say the machine is overfitted and when it can predict correctly the outcome of new examples we say it can generalize. Let’s see an example, we want to teach a machine the function f(x) = 2x+1. So we show some examples to the machine like x = 1 => f(x) = 3 x = 0 => f(x) = 1 x = 2 => f(x) = 5 Then we ask the machine what is the value of f(x) if x is equal to {1,2,3,4,5,…} if the machine only guess the correct answer for x = {0,1,2} it is overfitted and if it can predict correctly f(x) for other values of x then we say the machine can generalize. Notice that generalization can be measured by the percentage of the overall new examples that a machine can guess correctly where 0% of correctness correspond to the maximum overfitting. Now we can make an analogy by saying that by Memorizing teachers refer to Overfitting and by Understanding they refer to some degree of Generalization greater than 0. My biggest concern about teachers saying to students that they must not memorize instead of saying that they need to learn to generalize is that Memorizing is a fundamental and mandatory part of the process of learning, understanding and generalizing. Let’s do an interesting experiment. Read carefully and try to remember the next two sequences: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, … 1, 19, 17, 7, 3, 6, 21, 4, … Interesting right? You can remember the longest one but you can’t remember the last and shortest one. This is because you compressed the first one by using things that you already know, things that you memorized before and saved in your brain. You probably compressed the first sequence to one of the following patterns: x0=1 xn=xn-1+1 or “count from 1 to 16” or for(i=1 to 16) print i; or you just pointed to the part of your brain where that sequence is saved. The second sequence is given by the recurrence equation x0 = 1 xn = (51*xn-1+37)%23 but like you don’t have nothing like this saved in your brain you need to remember the whole sequence without compressing it. We can say that you “Understood” the first sequence but you didn’t understand the second one. There are two amazing state of the art theories that study the best way of generalizing without overfitting even with a small amount of data: Vapnik’s Statistical Learning Theory: this is a great theory about how to find the best function that separates two classes of data points in the hyperspace and it has a great practical implementation called Support Vector Machines. Rissanen’s Minimum Description Length Theory: this is a really beautiful theory that relates compression with learning algorithms. Conclusion: if you see that your students are “memorizing” too much, a good solution would be to give them more examples that they can relate with something they already know.