We all have been in such systems for so long that we take it for granted. How else is anyone to learn? Maybe some can learn differently, but most can’t because they are unmotivated and lazy, they lack the critical skills to differentiate right from wrong on their own and they can’t assess their own level of expertise. At least, that is what I’m told, but I think it is unfair.

To me, this is like saying that we have to keep long-time prisoners in jail because they do not know how to organize themselves when given their freedom.

Indeed, if students who went through years of schooling cannot learn on their own, if they cannot assess their own progress, and if they generally cannot organize themselves without supervision, we have to wonder whether schools bear part of the blame. And I think they do: we enroll students in supervised and regimented systems where they are constantly told what to do, constantly tested by others and where they have to follow rigid rules as to what they should learn. It is no surprise that many students cannot work on their own when they leave school.

There are a few broken individuals who never really became adults. They have to be kept in check all the time because they could not survive on their own. But if these constituted the essential part of the human race, we would have gone extinct a long time ago. Our ancestors, not long ago, had to survive in small bands hunting small animals and grabbing whatever they could eat. They had to be incredibly resilient because human beings spread throughout the globe like no other animal species.

To put it bluntly, most people lack autonomy, they can’t be entrepreneurs, precisely because we have carefully beaten it out of them. I have two young kids and they are crazy. One of them is building a castle out of paper in his room. The project is huge and complicated and has worked on it for days, on his own, without anyone telling him what to do. He made mistakes (which he explained to me) and he had to fix them. How often do schools let students embark on self-directed projects? Almost never.

My sons are not exceptional. Like other kids their age, they behave in unconventional ways, trying crazy things on their own, having crazy thoughts on their own. Eventually, with enough schooling, they will settle down and do as they are told in a more reliable manner. They will become very good at following directions.

How good will they be at emulating someone like Steve Jobs, who repeatedly broke all rules? I fear for them that their sense of initiative and wonder will be killed by the time they finish their schooling. (Thankfully, I am a crazy dad with crazy ideas, so maybe I will mitigate the damage.)

Hence, as a teacher, I reject the industrial model as much as I can. I believe that, in an ideal world, we would not need any teaching at all. There is hardly anything you can’t learn through an apprenticeship. For example, if you just helped out Linus Torvalds for a couple of years, you could become an expert programmer. In fact, I suspect you would fare much better than if you just took programming classes.

The problem with apprenticeship is that it scales poorly. How much patience will Linus Torvalds will have for kids who hardly know anything about computers? How many could he coach? Would he want to have kids over at his house while he is coding?

We still use the apprenticeship model in graduate school. But to accommodate most students, I still haven’t thought of a better model than setting up classes. But should the classes be organized like factories with the teacher acting as a middle-manager while students act as factory employees, executing tasks one after the other while we assess and time them? I think not. My teaching philosophy is simple: challenge the student, set him in motion, and provide a model. I try to be as far from the industrial model as I can, while remaining within the accepted boundaries of my job. I have two rules when it comes to teaching:

• Focus on open-ended assignments and exams. Many professors are frustrated that students come in only for the grades. Probably because they focus on nice lectures and then prepare hastily some assignments. Turn this problem on its head! Focus on the assignments. If your students are not very autonomous — and they rarely are — give several long and challenging assignments (at least 4 or 5 a term). Do make sure however that they know where to get the information they need. Provide solved problems to help the weaker students.

  However, keep the assignments open ended. We all like to grade multiple choice questions, but they are a pedagogical atrocity. In life, there is rarely one best answer: assignments should reflect that. In some of my classes I use “programming challenges”: I make up some difficult problem and ask the students to find the best possible solution. Often times, there is no single idea solution, but multiple possibilities, all with different trade-offs. Quite often the students ask me to be more precise: I refuse. I tell my students to justify their answer. Over the years, I have been repeatedly impressed by the ingenuity of my students. Many of them are obviously smarter than I am.

  What about lecture and lecture notes? They are secondary. In most fields, the content, the information, is already out there. It has been organized several times over by very smart people. Books have been written on most topics. There is a growing set of great talks available on YouTube, Google Video and elsewhere. Your students do not need you to rehash the same content they can find elsewhere, sometimes in better form. Stop lecturing already! Just link to what is out there and encourage your students to find more using a search engine. Only produce content when you really cannot find the equivalent elsewhere. Please link to material beyond the grasp of most of your students: they need to know the limit of their knowledge.

  The famous software engineering guru Fred Brooks agrees with me:

  The primary job of the teacher is to make learning happen; that is a design task. Most of us learned most of what we know by what we did, not by what we heard or read. A corollary is that the careful designing of exercises, assignments, projects, even quizzes, makes more difference than the construction of lectures.

  For my years as a student, I hardly remember the lectures. They were overwhelmingly boring. And I soon learned that even if a teacher was remarkably able and he could give me the impression that I understood everything… this impression was quickly falsified when I tried to work the material on my own.

• Be an authentic role model. Knowing that someone ordinary, like your professor, has become a master of the course material means that you, the very-smart-student, can do the same. That’s the power of emulation.

  When Sebastian Thrun gave his open AI class at Stanford, tens of thousands of students enrolled. Sure enough, the Stanford badge played a role in the popularity of the course, but ultimately, it is Thrun himself, as a role model, that matters. He has now left Stanford to create his own independent organization (Udacity). Thrun must be confident about his success since he left his tenured position at Stanford, reportedly because he cannot stand the regular (industrial-style) teaching required at Stanford. One upcoming course is “programming a robotic car”. I have no idea how good the course will be, but it will be motivating for students to attend the class of the world’s top expert in the field of robotic car.

  The status of the teacher as an expert has always been important. However, the ability of people like Thrun to reach thousands of people every year through his teaching means that there is less of a market for teachers who aren’t impressive AI researchers.

Unfortunately, as long as I teach within a university, there are a few things I am stuck with:

• Deadlines: Some students are able to go through the material of a class in 4 weeks. Others would need 16 months. Alas, universities have settled on a fixed number of weeks that everyone must follow. If you complete the course faster, you’ll still have to wait till the end of the term to get credit. If you need more time, you will have to make special arrangements. Of course, schools follow the factory model: we can’t have workers come in and finish whenever they want. But outside an industrial setting, I think that deadlines are counterproductive. If I take a class in computing theory and end up proving that P is equal to NP, but I end up my paper a few weeks after the end of the course, I will still fail. Meanwhile, the good student who followed the rules but showed a total lack of initiative and original thinking will go home with a great grade. What do we reward and what do we punish?
• Grades: Grades are a very serious matter in schools. Denis Rancourt, a top-notch tenured physicist at the University of Ottawa, was fired after refusing to grade his students. (He would give A+s to everyone.) Grades are effectively the quality control mechanism of schools, where students are the product. Somehow, we have totally integrated the idea that we could sum up an individual by a handful of letters. It sure makes managing people convenient! It all fits nicely in a spreadsheet. Of course, students have adapted by cheating. Schools have reacted by making cheating harder. But I cheated all the way through my undergraduate studies getting almost perfect score in all classes. How? I discovered a little trick: at the University of Toronto, all past year exams were available at the library. If you took time to study them, you soon found out that, at least in the hard sciences, a given professor would always use the same set of 10 to 20 questions, year after year. So all you had to do was to go to the library, study the questions, prepare them, and voilà! An easy A. But it is all rather pointless. In theory, grades are used by employers to select the best students, but serious employers don’t do this. We use grades to select the best candidates for graduate school, but I doubt there is a good correlation between grades as an undergraduate and research ability. I know two top-notch researchers who have admitted getting poor grades as undergraduates. For years, I have served on a government committee that awards post-doctoral fellowships: I am amazed at how poor the undergraduate grades are at predicting how well someone might do during his Ph.D. Conversely, I have seen many graduate students who had nearly perfect scores throughout their undergraduate studies who are totally unable to show even just a bit of initiative. They do well as long as you always give them precise directions.

Credit: Thanks to Michiel van de Panne for the reference to Brooks’ quote.

Further reading: Making universities obsolete by Matt Welsh, an interesting fellow who left his tenured position at Harvard to go work in industry.

Disclaimer: Many people are better and more sophisticated teachers than I am. And the industrial model does work remarkably well in some settings. Yet I think that they the skills it fails to favor are increasingly important. We have to stop training people for factory jobs that are never coming back.

• There is a pre-existing problem P.
• There are few relatively simple but effective solution to problem P. Among them is solution X.
• We came up with a new solution X+ which is a clever variation on X. It looks good on paper.
• We ran some experiments and tweaked our results until X+ looked good. We found a clever way to avoid comparing X+ and X directly and fairly, as it might then become obvious that the gains are small, or even negative! We would gladly report negative results, but then our paper could not be published.

It is a very convenient story for reviewers: the story is simple and easy to assess superficially. The problem is that sometimes, especially if the authors are famous and the idea is compelling, the results will spread. People will adopt X+ and cite it in their work. And the more they cite it, the more enticing it is to use X+ as every citation becomes further validation for X+. And why bother with algorithm X given that it is older and X+ is the state-of-the-art?

Occasionally, someone might try both X and X+, and they may report results showing that the gains due to X+ are small, or negative. But they have no incentive to make a big deal of it because they are trying to propose yet another better algorithm (X++).

This process is called citogenesis. It is what happens when the truth is determined solely by the literature, not by independent experiments. Everyone assumes, implicitly, that X+ is better than X. They beauty of it is that you do not even need for anyone to have claimed so. You simply need to say that X+ is currently considered the best technique.

Some claim that science is self-correcting. People will stop using X+ or someone will try to make a name for himself by proving that X+ is no better and maybe worse than X. But in a business of science driven by publications, it is not clear why it should happen. Publishing that X+ is no better than X is an unimpressive negative result and those are rarely presented in prestigious venues.

John Regehr made a similar point about our inability to address mistakes in the literature:

in many cases an honest retrospective would need to be a bit brutal, for example to indicate which papers really just were not good ideas (of course some of these will have won “best paper” awards). In the old days, these retrospectives would have required a venue willing to publish them, (…), but today they could be uploaded to arXiv. I would totally read and cite these papers if they existed (…)

But there is hope! If problem P is a real problem, for example, a problem that engineers are trying to solve, then you can get actual and reliable validation. Good software engineers do not trust research papers: they run experiments. Is this algorithm faster, really? They verify.

We can actually see this effect. Talk to any Computer Scientist and he will tell you of clever algorithms that have never been adopted by the industry. Most often, there is an implication that industry is backward and that it should pay more attention to academic results. However, I suspect that in a lot of cases, the engineers have voted against X+ and in favor of X after assessing them, fairly and directly. That is what you do when you are working on real problems and really need good results.

Credit: This blog post was inspired by a comment made by Phil Jones on Google+.

Over time, I have learned that even if the reviewer is wrong, spending time to careful respond can be tremendously useful. If you are 100% correct, then you get to build up your confidence and can later answer similar criticism hastily. Very often, however, you did not do everything perfectly. Maybe your arguments and data are correct, but you might have presented them better.

There are specific strategies to deal with harsh reviews:

• Expose yourself regularly to criticism from total strangers. In my experience, if you rarely publish, you are more likely to have difficulty dealing with criticism. I have been called an idiot, I have had to deal with overly aggressive people and I have been ridiculed on occasion. Of course, I occasionally get depressed after receiving harsh criticism, especially if I thought I had produced great work and feel unappreciated, but I am typically able to recover mentally in minutes or, at least, hours. Part of it is just habit: my brain has learned that harsh criticism does not necessarily signify upcoming pain.
• It is critically important to distinguish yourself from your work. If someone repeatedly produces inferior work, his reputation will suffer. However, everyone (even Nobel prize winners) gets it wrong from time to time. It is important to keep in mind that most reviewers do not care that much about you. In fact, they often quickly forget about you while you ruminate over their review.
• The best way to address criticism is to take it one comment at a time. If someone finds ten different flaws in your work, don’t look at it as one message: break it into ten components and address each one separately. This approach scales up linearly: it just take ten times longer to address 10 flaws than one. Brian Martin describes it well:
  

  I’ve found a way to make the revision process easier. I don’t reread my text, because that just cements my previous approach. Instead, I go through the recommendations of the referees and the editor one by one, making changes. After I finish all those changes, large and small, I print out the whole article and read through it, fixing up expression and making it flow.

  Tackling recommendations one by one is important psychologically. Looking at a list of criticisms, sometimes pages of them, can be demoralizing; the task seems too big. Focusing on a single point is easier. Once it’s done, you can check it off and proceed to the next point, either immediately or tomorrow.

  Sometimes responding to a point requires additional work, such as obtaining and reading some new theory or doing some new calculation. It’s helpful to write down every step that’s required – for example, (1) order Smith’s book, (2) read the theory section, (3) write a one-paragraph summary – and tackle them one by one.

The directory of open access journals gives a list of over 300 open access journals in Computer Science. Thus, finding an adequate open access journal where you can submit your work is relatively easy.

However, there are a few sore points.

1. Indexing of open access Computer Science journal is generally weak

A journal needs to be indexed so that your fellow researchers can find out about your work. Most open access journals will be indexed by Google Scholar, but other indexes are important in Computer Science such as DBLP and the ACM Digital Library. Scopus is also often used by hiring and promotion committees. (Scopus is run by Elsevier.)

As I review the open access journals in Computer Science, I find that indexing is often a sore point. The next table shows that the ACM Digital Library does a poor job at indexing open access journals. In fact, I could find only two open access journals indexed by ACM. It cannot be explained by the prestige of the respective journals: some of these open access journals that ACM fails to index are just as good or better than others it indexes. And, of course, no ACM publication is open access. Quite clearly, ACM is doing little to help open access.

Elsevier and Springer allow authors of papers in some regular journals to make them available under an open access format in exchange for a one-time fee. Their journals are typically well indexed so they may offer good alternatives.

2. Many open access Computer Science journals require complete copyright transfer

To publish an article, a journal does not require complete copyright ownership. The only valid justification for requiring that the author gives away his copyright is to restrict access. When reviewing open access journals in Computer Science, I see that several of them inexplicably require complete copyright transfer:

Conclusion There is still much room for progress.

Elsevier has committed too many sins to give an exhaustive list: they have created fake academic journals so that pharmaceutical corporations could claim that certain facts appeared in a journal, they have sponsored evil regulations, and they have restrictive views on what constitutes fair use. Unbelievably, they were also involved in arms trade. They probably have the devil on their board of directors.

The boycott is currently lead by a famous mathematician, Timothy Gowers. Gowers accuses Elsevier of charging exorbitant prices for its journals.

Focusing solely on database-related journals, I decided to look at how much journals charge per article.

Observations:

• The price distribution appears almost random. I can see no relation between prestige or paper length and prices.
• Elsevier is hardly alone at charging high prices for papers. Wiley and Springer are just as expensive. Of course, it is possible that Elsevier ends up charging more through deals and bundling.
• ACM is very inexpensive on a per-article basis. However, ACM often asks the authors to pay page charges whereas Elsevier rarely does in my experience.
• Though SIGMOD Record is limited to short contributions, its price is unbeatable. And it has no page charge. Moreover, it is generally a well regarded publication venue among database researchers.

My take: The evidence is strong that high-quality inexpensive journals are possible. Current journals are up to an order of magnitude too expensive. However, Elsevier is selling what we want to buy: prestigious journals that people outside the best schools cannot afford. Just like middle-income Americans get into debt to keep up with the top 1%, colleges increase their library budgets to keep up with Stanford and Harvard.

The solution to overpriced journals is to reduce library purchasing power. Most colleges do not have the infinite budgets Harvard and Stanford have, and they should not act like they do. In fact, if we could reduce the purchasing power of most libraries to zero, then researchers and students would be forced to pay $20 or more per article. You can be quite certain that they would mostly read the cheaper (and more competitive) journals. And Stanford researchers want to be cited by the researchers from the lesser institutions so they would also migrate away from overpriced journals. Reduced budgets would still allow publishers like Elsevier to make generous profits, but they would only profit by offering great services at an affordable price.

Disclaimer: I am currently reviewing a paper for Pattern Recognition (an Elsevier journal), and I recently published in Discrete Applied Mathematics (another Elsevier journal).

Update: Though you can get articles from SIGMOD Record for free if you to the SIGMOD Record home page, ACM sells them through its Digital Library for over $10 a piece.

Typically, Hash tables have the property that looking up or storing a value associated with a key requires constant time. If you use user identifiers to retrieve names and phone numbers, you can scale up to millions and millions of users without performance penalty. However, the worst case complexity of a hash table is linear: it may need to go through most values each time you want to look up a key. Thankfully, the worst case is typically improbable: it only happens when too many objects hash to the same value. In practice, hash functions are chosen so as to spread hash values uniformly (pseudo-randomly).

Most programming languages like Java or C++ use deterministic hash functions. This means that given a string, it will always hash to the same integer, for all Java software in the whole world. And overall, deterministic hashing works quite well. Unfortunately, deterministic hashing is insecure. If your are building a web application, and hackers know which hash function you are using, they can create a denial-of-service attack and bring down your application. The gist of it is not complicated: it suffices to ensure that the hash tables fall back on their worst case performance.

This is very serious: it means that if you rely on the default hash functions of your programming language (e.g., String.hashCode in Java), your application could be at risk. On this issue, Alexander Klink and Julian Wälde issued a well written security advisory.

The fix is relatively simple: programming languages need to adopt random hashing. In random hashing, every time the software is initialized, a new hash function is picked, at random. This does not make attacks impossible, but it makes them much more difficult.

The problem is not novel. In 2003, Crosby and Wallach raised the issue and many responsible vendors fixed their products. Alas, the only programming languages to adopt random hashing were Ruby and Perl. Others are more reluctant.

So, how easy is it to hack the hash functions in, say, Java? Java uses an iterated hash function. At each iteration, iterated hash functions compute a new hash value from the preceding hash value and the next character. Strings in Java are hashed using the function
F(y,c) = 31 y + c.
where y is the previous hash value and c is the current character value. Thus, the hash value of a string made of the characters 65, 66 (corresponding to “AB” in ASCII) is 31 times 65 + 66 which is 2081.

Why does Java uses the number 31? The choice is somewhat arbitrary (and 31 might fail to be ideal) but because it is an odd number, the compression function F is permuting which helps distribute more uniformly the hash values.

It is fairly hard to construct reasonable strings that collide over 32 bits in Java. However, a modest hash table will use only the first few bits of the hash values. Let us consider only the first 16 bits. It is not difficult to check that the strings “Ace”,”BDe”,”AdF” and “BEF” all have the same hash value in Java.

Of course, having 4 strings colliding will not disrupt hash tables. But because the hash function is iterated, we can multiply the number of collisions. Indeed, any two same-length sequences of these four colliding strings will also collide. This means that you can construct 16 strings of length 6 all colliding (“AceAce”,”AceBDe”,”AceAdF”, “AceBEF”, “BDeAce”,”BDeBDe”,”BDeAdF”, “BDeBEF”, “AdFAce”,”AdFBDe”,”AdFAdF”, “AdFBEF”, “BEFAce”,”BEFBDe”,”BEFAdF”, and “BEFBEF”). You can keep going to 64 strings of length 9. And so on.

How badly does this impact the performance of a hash table? I tried inserting all the colliding strings in a Java Hashtable container. For comparison, I also inserted randomly chosen strings into either a Hashtable or a TreeMap (tree structure). The net result is that what should be a tiny cost (0.006 s) becomes a massive cost (30s). A server able to process thousands of queries per second might quickly become bogged down trying to process a couple of queries per second.

For these tests, I am using a MacBook Air with a 1.8 GHz Intel Core i7 running Java 6. My code is available.

Why aren’t programming languages adopting random hashing? A potential issue is that language designers like determinism. They much prefer reproducible bugs. Nevertheless, any expert programmer should be aware of this problem.

Further reading:

• Scott A. Crosby and Dan S. Wallach, Denial of Service via Algorithmic Complexity Attacks, Usenix Security’03.
• Daniel Lemire, The universality of iterated hashing over variable-length strings, to appear in Discrete Applied Mathematics.

Update: I initially reported that Ruby was the only language to adopt random hashing. In fact, Perl adopted random hashing with version 5.8.1. In version 5.8.2, Perl adopted an hybrid that switches between a deterministic and a random hash function when needed. (Thanks to Mike Giroux for the pointer.)

Open access is the idea that scholarship should be accessible to all. Many believe that we should require publicly funded researchers to make their work available to the public. That is, if some professor discovers a new algorithm or a new remedy while on a government grant, you should be able to download and read the paper freely.

To non-scientists, open access may sound like a socialist utopia. Why would anyone give away carefully curated content for free? The problem is that the content is overwhelmingly produced by scientists who have no share of the profit made by the publishers. These scientists are often funded directly or indirectly by the government. Journal editors are typically not paid. Reviewers are almost never paid. In fact, the opposite is true. Over the years, I have given thousands of dollars in page charges or conference registration to publishers. For example, several ACM journals request $60 per page to the authors (so that publishing 30 pages costs $1800). That is right: as a scientist, you are often asked to pay to get your worked published. Thankfully, most of these fees are paid by research grants, which often come from the government.

Open access is a problem for publishers however. In the current system, the publisher has a monopoly on the journals it has published over the years. This means that as long as researchers need access to these journals, the publisher can charge millions for access. Open access kills this monopoly. Certainly publishers can increase their profits by increase the page charges, but authors can also take their papers to other, more reasonable journals.

Nevertheless, I could never get excited about open access. I find it annoying that I cannot download papers freely, but astrophysicists have already solved this problem without any government intervention or lawsuit. Indeed, nearly all recent astrophysics papers are on arXiv. What matters is the culture: physicists care about being read, they love the web. In this sense, open access is a short-sighted fight.

Thus, a much more significant vision is Nielsen’s open science. Michael Nielsen is arguing for a culture shift in science: from a science obsessed with individual performance (and publications) to a science culture resembling more that of open source software or wikipedia.

I fear however that despite all the (well deserved) press that Michael Nielsen’s latest book has been getting, too few people understand the importance of this shift. It is not about becoming hippies. It is not a socialist utopia. On the contrary, the system we have right now is akin to an highly regulated industry. All power is in the hands of the government and a few large organizations (universities, publishers) working in tandem. The barrier to entry is maintained artificially high. Open science is really about creating “open markets” with freer exchanges. It has the potential to boost our collective productivity by orders of magnitude through the removal of unneeded friction.

Meanwhile, American corporations are concerned with copyright violations on the Internet. Thus, they are pushing a bill, the Stop Online Piracy Act (SOPA) which would allow the government to shut down web site that is suspected of violating copyright. Using SOPA, a publisher could have a repository of research papers shut down. While at it, the publishers are also promoting a bill, the Research Works Act which would make it illegal for government agencies to require open access from publicly funded researchers.

If you are one of the thousands of members of the Association for Computing Machinery (ACM) or the Institute of Electrical and Electronics Engineers (IEEE), then you are indirectly supporting this new legislation. Indeed, the ACM and IEEE are members of the Association of American Publishers (AAP). The AAP is a lobbyist for both proposals.

And we finally get a hint at why it is so hard it is to open up science: the business of science has become intertwined with businesses like the publishing business. ACM has to speak both as an association of computing professionals, and as a publishing house.

What should be a critical support service, the publication of results, ends up driving much of our culture. The journals become the science. The medium becomes the message.

In effect, we have too much organizational scarring tissue in science. It could be that we need to reboot the system. As a starting point, we should collectively recognize the problem. Repeat after me: scholarship is not a publishing business.

Further reading:

• Is Open Access publishing the solution? Really?
  
• ACM’s role in public policy
  
• Time for scholarly publishers to disavow the AAP

Update:

The ACM charges the authors of any conference for the publication of proceedings. However, they do not require payment for publishing in their journals: instead they request page charges.

The rationalization for patents usually goes like so: developing new drugs is extremely expensive, only large well-funded corporations can do it, and they won’t innovate unless we grant them a generous monopoly on their discovery.

Granting monopolies, even temporary ones, is expensive. We need to be sure that the gains out-weight the costs. In this case, the rationalization offered by the industry does not stand up to scrutiny:

• The U.S. and the U.K. have always had strong patent laws protecting chemicals and drugs. Meanwhile, continental Europe had much weaker patent protection. Until recently, you could not patent a drug or a chemical in Germany (1967), Switzerland (1977) and Italy (1978). Where did the pharmaceutical industry thrive before the 1960s? In Germany, Switzerland and Italy. Though Italy was the fifth produce of drugs in the 1970s, its industry is now practically disappearing.
• Without drug patents, India became the fourth largest pharmaceutical producer. It was forced to introduce drug patents in 2005. Haley and Haley (2011) analyzed the effect of the introduction of patents in India:

  (…) the rate of growth in innovation, as measured by investments in R&D, has fallen in India’s product-patent regime. (…) This study indicates that product-patent regimes do not necessarily generate greater rates of innovation than process-patent regimes, and may reduce innovation.

• Because companies cannot compete over the efficient production of drugs, they have a strong incentive to produce new drugs. People assume that this means more innovation. However, there is a much cheaper and safer path for drug companies: produce a variation on an existing drug. In this manner, you can go after the competition (by producing variations on their drugs) or artificially extend the life of your own patents (by producing endless variations on your own drugs). And indeed, we can verify that most drug patents concern me-too drugs. In effect, it is more profitable for drug companies to consider themselves as patent portfolios rather than R&D firms.

You might argue: aren’t you happy that the pharmaceutical industry has almost cured a disease like AIDS? I would, except that the pharmaceutical companies did no such thing. It took an academic researcher to figure out that a cocktail of drugs was the best option to treat AIDS. In fact, the majority of the funding in biomedical research comes from the government and private non-pharmaceutical funds. I have to ask: if the private sector with generous patent protections is so good at innovating, why do governments and private foundations bother to fund medical research?

We don’t have worldwide pharmaceutical patents because they benefit most of us. We have them because they benefit the richest 0.1% at the expense of all others.

Further reading:

• Michele Boldrin and David K. Levine, Against Intellectual Monopoly, Cambridge University Press, 2010.
• George T. Haley, Usha C.V. Haley, The effects of patent-law changes on innovation: The case of India’s pharmaceutical industry, Technological Forecasting and Social Change, 2011.

Other work is more like gold prospecting. Almost all freely available open source is never used. Almost all research papers are useless. Most books find very few readers. Most blog posts are written in vain.

With gold prospecting, the results are nearly as certain as with construction jobs. Programmers know that their current project won’t compete against Linux. Mathematicians know that their research paper will be read, at most, by handful of people. Writers know that their next book won’t sell more than a few hundred copies.

However, gold prospectors rely on probabilities. Even if your chance of success is only 1%, you are guaranteed (within 1%) to succeed if you try 500 times. If your chance of success is a meager 0.1%, then try 5000 times.

That is why so many open-source programmers, scientists and writers thrive despite the odds: they simply keep on trying. It is only a risky business if you cannot try multiple times. You might be tempted to object that writers don’t try to market their books 500 times. But some do! J. A. Konrath collected nearly 500 rejections from publishers. In a recent blog post, he provides a list of comments by major publishers who all rejected his best-seller The List. Though no publisher would take his book, he is now making $5000 a day from the sales of this self-published book alone. He won because kept trying long enough. I am convinced that most gold prospectors eventually succeed, if they try long enough.

Shortly after completing my Ph.D. I decided to become an entrepreneur. I did not have contracts lined up. I did not have a business idea. I did not have investors. I did not have business experience. Yet I never ran out of money or work despite the apparent risks. Some projects were failures and I did lose time and money. Other projects were rewarding beyond my expectations. It averages out over time.

Most prospectors just put their best work out and hope for the best. Sometimes they are lucky and the work is appreciated. Yet it is very difficult to predict success ahead of time. Some of the major research papers, including some work that lead to Nobel prizes, were initially rejected. In fact, the number of Nobel-winning research papers that were initially rejected is astounding (Campanario).

Though my arguments appear reasonable enough, consider how hard it is to apply the gold prospector model in real life. Young researchers are often judged by the work they did during the Ph.D.: you only have a few short years that make or break the rest of your career. We are quick to judge a writer based on the market success of his books: it is difficult for us to imagine that some could go from a nobody to a wealthy writer like J. A. Konrath in a couple of years.

We tend to be overconfident in our assessment of gold prospectors. A much easier, and more meaningful, distinction is between gold prospectors and construction workers. Some people are simply not interested in finding gold. They would rather just build houses. Among college professors, you have those who are eager to start yet another research projects (gold prospectors) and those who do research when it is a requirement to keep their job (construction workers). Among programmers, you have those who start their own projects, and those who will never touch code again if they can get a management job. Both types can excel at their jobs, but the spirit is different. Prospectors accept failures as part of their business, whereas construction workers would rather avoid failures entirely.

Reference: Juan Miguel Campanario, Rejecting Nobel class articles and resisting Nobel class discoveries.

Source: This post was inspired by a Facebook comment made by Venkatesh Rao and by a face-to-face discussion with Philippe Beaudoin.

• Innovating without permission
• Not even eventually consistent

February:

• Taking scientific publishing to the next level
• Ten things Computer Science tells us about bureaucrats

March:

• Know the biases of your operating system
• Breaking news: HTML+CSS is Turing complete
  

April:

• How information technology is really built

May:

• Automation will make you obsolete, no matter who you are
  
• The perils of filter-then-publish
  
• Is Wikipedia anti-intellectual?
  

June:

• Why I still program
  
• The language interpreters are the new machines
  

July:

• The myth of the unavoidable specialization
  

August:

• The Web is killing database systems

September:

• Science is self-regulatory… really?
  
• Emerging knowledge is a private business
  
• You think that users are faceless objects? You are obsolete!
  

October:

• Why aren’t we getting richer? The scarring tissue theory
  
• It is not where you work, but who you work with
  
• How database design fails us, and what to do about it
  

November:

• Are Relational Databases good for anything anymore?
  

December:

• Dealing with harsh criticism
  

Document-oriented databases such as MongoDB are not so simple. There is room for optimization. Using short names for attributes is better.

For example, in going from JSON tuples of the form

{date_achat:'1999-06-30',article:'Echasses',
quantite:1,prix:2800}

to these tuples where one attribute has a longer name

{date_achat:'1999-06-30',articlefromoutstore:'Echasses',
quantite:1,prix:2800}


you increase the space utilization per tuple by 12 bytes (from 105 to 117 bytes per tuple).

The converse is true. Using shorter names is better:

{d:'1999-06-30',a:'Echasses',q:1,p:2800}


The space utilization per tuple goes down to 80 bytes (from 105 bytes). This is a saving of over 20%.

It is tempting to do away with the attribute names entirely and save the data as array:

['1999-06-30','Echasses',1,2800]

Yet the space utilization remains at 80 bytes because the binary format used by MongoDB (BSON) does not store arrays concisely.

Should we worry about this issue? We live in an era of abundant storage and memory. MongoDB pre-allocates the storage to avoid disk fragmentation. Even the tiniest collection will use 128 MB, and larger collections are stored in 2 GB files: MongoDB is unafraid to waste nearly 2 GB or more. In fact, we might say that it is precisely because we live in such abundance that we can afford to use document-oriented databases. However, engineers still face problems with space utilization. Hence, it is useful to be aware of the effect that the names you choose will have, especially if you come from a relational database context where name length is irrelevant.

This is a perfect example of how to apply mental judo when faced with criticism: turn the negative energy which is opposing you, into positive energy that propels you.

I recently read Graeber’s Debt: the first 5000 years. I wrote about the book on this blog a few years ago. This is a brilliant, highly original book by an author who is leaving his mark in history. I wrote an excellent review (5 stars) on Amazon, and I recommend that everyone reads this challenging book!

Yet the book is flawed. Venkatesh Rao summed the problem well, though maybe too harshly:

Debt is not one big story spanning 5000 years, but more like a collection of 5000 little stories and arguments thrown together, with a bigger narrative almost slapped on as an afterthought.

How did Graeber react when Rao criticized him online? He blamed Rao for missing the boat:

Sad really. A book is only as good as its readers.

This is not the best response. So, how should you react?

1. Don’t be blinded by the negative. To be efficient critics of your work, people have to study it. In the process, they often identify strengths. Pay close attention to your critics. You will often find out that they are not trashing your work entirely. Consider the Scott Adams’ story: Dilbert is funny only while at the office. Ah! This means that Dilbert is funny while at the office? In the Graeber story,  Rao wrote: “the book provides a lot of astounding value”. See how Graeber paid no attention to this part?
2. Be constructive. Instead of opposing your critics, work with them. They will often identify real weaknesses in your work.  You may not be able to fix these weaknesses (e.g., you cannot re-edit your book), but you should at least be able to take them into account in the future. An answer that I give often is “I’m aware of this problem and I plan to write about it in the future”. Scott Adams’ story is the perfect illustration of this principle: instead of fighting his critics, he improved his work accordingly.

• Recent processors (e.g., Intel i7) can multiply and add a 32-bit number in a single CPU cycle or less. For each component in the arrays, we need to execute one multiplication and one addition. The latency of the multiplication is at least 3 cycles, meaning that we require 3 cycles to complete a multiplication. Similarly, additions require at least one cycle. Yet processors make aggressive use of pipe-lining: they execute several multiplications simultaneously so that they can produce one result every cycle.
• If you work with 64-bit integers and use some recent GNU GCC compilers (e.g., 4.5), you should disable Streaming SIMD Extensions (SSE) for better speed. In theory, the SSE instructions are ideally suited to the computation of scalar products. Yet the throughput with 64-bit integers goes from 1.3 cycle per multiplication with SSE2 disabled to 3.4 cycles per multiplication with SSE2. There is an optimization bug in the otherwise excellent GNU GCC compiler. Update: According to the numbers provided by John Regehr, this problem also affects some Intel compilers.
• When using SSE, 64-bit floating point numbers may be faster than 32-bit floating numbers. Years ago I was told to avoid 64-bit floating point numbers for performance reasons. It is not automatically good advice on all compilers especially if you require standards compliance.

For my tests, I initially used the flags “-funroll-loops -O3″ on a recent Intel i7-2600 with the GNU GCC compiler version 4.5. In each instance, I have tested with and without manual loop-unrolling and I only report the best score (in cycles per multiplication). The C code is available.

Upgrading to GCC 4.6.2 and replacing -O3 by the new -Ofast flag changes the results quite a bit at the expense of reliability (-Ofast disregards standards compliance).

Further reading: See my previous post on this topic Fast computation of scalar products, and some lessons in optimization

As a database researcher, I was “brought up” to think of databases as the ideal tool for data management. There was simply no good reason for anyone with data to manage to use something else. It was with dismay, then, that little by little I came to realize that the importance of databases for data management is diminishing tremendously out there in the real world. The advent of NoSQL was a clear call for anyone paying attention (although in theory NoSQL does not necessarily mean no-relational, in practice it does). The fact that scientists are using anything but (relational) databases for e-science is also another sign that the Apocalypse is among us (just kidding).

It seems clear to me that it must be admitted that there are situations in which a (relational) database is simply not the appropriate tool. This immediately brings forth a question: When is a (relational) database the right tool? My idea to attack this question was to look at the problems people are trying to solve, the processes they use to solve them, and to classify the data those processes deal with along relevant dimensions. Of course, what is relevant is in the eye of the beholder; but there are some characteristics that have traditionally been challenging: dealing with very large data sets, and dealing with irregular data. So I analyzed processes and their data based on those two dimensions.

• data size: to classify data by size, I used an entirely pragmatic measure. Roughly, if a data set fits into memory, it is small (up to 2GB or so in today’s computers); if it fits in a single disk (or, to be exact, a disk array with a single controller), complexity is medium (up to 1TB or so); if it needs several disks/parallel/distributed systems, it’s large (note that this last category includes perhaps too much, since it goes from a few TB to PTs and beyond). The motivation for this division is clear: algorithms that can assume that all data fits in memory are ‘qualitatively’ different from those that need disk access. And adapting algorithms for parallel/distributed environments presents non-trivial challenges. So yes, these categories are very much dependent on current architecture and technology, and may have to be revised once a great solid state device is developed. But until a revolution comes, that will just mean changing the exact ‘boundaries’. And I think the division is very relevant for people using the processes, especially for those that require interactive (or near interactive) response time.
• data complexity: intuitively, this has to do with the data structure or organization, but admittedly, it is a much harder nut to crack then the previous one. After some thinking, I decided to subdivide the problem along 3 ‘sub-dimensions’:

• Conceptual complexity: Intuitively, this is the number of elements (entities, relationships) in a conceptual model of the process’ data. This is clearly a bit vague, since one can build several conceptual models of the same data, but it is not important here which entities one “sees” in the data, as to ‘how many’. Of course, both issues are related, but still there is an intuitive sense in which this matters. So, if a conceptual model would identify 1 main entity, everything else being attributes or entities “strongly connected” to it, then that data would be considered simple -the idea is that you could keep all data in 1 file, with minimal redundancy. If you can identify several entities, then the complexity is medium. How many? There is no clear-cut number here. One possibility is to say that it’s not too much for one person to comprehend “at once”; based on the famous finding of “the magical number seven, plus minus two”, but taking into account that one can get helps from ‘external memory devices’ (i.e. pencil and paper), we could set an upper limit of 10 to 20 entities. More than that, and we consider the conceptual complexity large.
• regularity: relational databases assume that you can define a schema for your data before-hand. However, this is not always possible. There has been much research to deal with somewhat irregular (semistructured) data, but here I make another distinction that I consider more relevant to today’s processes: whether the schema is ‘close vocabulary’ (i.e. all possible entities/attributes/relations can be enumerated once and for all) or ‘open’. Schemae (both relational and object-oriented), taxonomies, ontologies, even DTDs and XML Schema-compliant data are closed. Of course, within those there are distinctions. When the schema comes first (relational and object-oriented) I consider that regularity is high (and complexity low); when schema and data may be decoupled (as in semistructured data), I consider that regularity is medium (and so is complexity). On the open vocabulary front, regularity is very low (and complexity is very high). Note that key-value stores are considered ‘open’, since in many of them the value is opaque (i.e. it could be anything). This is not really a schema, it’s just a convenient way to distribute data (keys are made up and used for hashing or sorting; they are not inherent to the data).
• schema rate of change: even if one has a schema, or at least a closed vocabulary, one may not be able to take full advantage of this fact if the schema keeps on changing. Relational databases implicitly assume that once a schema is created, there is going to be little to no change. At the other extreme, in some modern processes nothing is assumed about how data evolves, and there is complete freedom: two objects in a collection may have completely different attributes. Also, an object may change to the point that it has nothing in common with the original (except the key, of course). If there is no change, or very infrequent one, I consider complexity low; if there is some change allowed, but within certain limits, I consider it medium; if any type of change is allowed, I consider it high.

Thus, we can analyze a process by looking at the data it consumes and/or generates, to decide what kind of system better supports it. The funny thing is, under this analysis, RDBMS seem only adequate when size and complexity are medium -in particular, data size is medium, conceptual simplicity can be handled, but regularity must be high and schema rate of change must be low. For processes with other characteristics, a RDBMS may not be well suited. Some people will argue that most RDBMS nowadays can handle very large data and irregular data -since commercial systems come with ‘cluster’-based extensions, as well as extensions to handle XML. But many of the problems facing large datasets are not solved by throwing more hardware at them. For high availability, replication is needed; and this bring issues for transaction support. For complex data analysis, many times the approaches required are not supported (or not supported well) by SQL -hence, just being able to store the data is not enough. So for many processes, using a distributed RDBMS or a cluster-based RDBMS will still not do. As for extensions to handle object, XML, and even text, the problem is that each one of these extensions was basically a compromise that yielded unwieldy systems, lost much of the simplicity of the original relational model, and in exchange gave clumsy tools.

What happens in other situations? When your data is low in size and low in complexity, some processes make do with files and some domain-specific programs. This seems to be the case with much e-science. The overhead that a database brings is just no worth it. As for very large data, this seems to be the niche of the all these new shiny NoSQL systems.

So, all in all, it seems that RDBMS are being relegated to a very narrow niche. To be sure, it is still a very profitable one, so there is no big market pressure on database companies (yet, although some of them are already responding, like Oracle with its NoSQL database). One can argue that RDBMS have been ‘under threat’ before, and have reacted to it -but as a result,  RDBMS seems more bloated than ever to some users, while still failing to satisfy their information needs. Maybe it’s time to rethink the whole thing from scratch?

In Debt: the first 5000 years, David Graeber teaches us that we have the story backward. History followed a pattern that looked more like this:

1. We started out living in Smurf villages. The Smurfs are little blue fellows who have no use for money. Some contribute more, some contribute less, but nobody is counting. Typically, the more a Smurf contributes, the higher his status. Those who contribute little can be ridiculed or harassed, but everyone gets to eat as long as there is enough food.  When there is a major task, everyone joins up. This is communism and it requires a high level of trust. In short, you must have no incentive to hoard commodities. You may own your own food, but it is understood that anyone from the village who needs it can come and take it from you.
2. Some form of symbolic currency (such as gold or silver) emerges but it is mostly for ceremonial purposes. For example, you may give some amount of silver to a family so that you can wed their daughter. However, you are not “buying” the daughter. In any case, nobody will sell you bread in exchange for silver.
3. There is always some barter, but mostly between strangers.
4. As trade expands, it is no longer possible to rely solely on communism and barter. So credit appears. You give a merchant a pile of wood that he will trade in for food in the next village. Because you don’t entirely trust the merchant, you keep a record of the what you give him. Maybe you want to also share in the profit so you ask for interests. This record you have created is new “money”.  You can trade this record for other things while the merchant is away. Of course, the value of these records depend on your reputation. Kings and priests have a better chance of trading such records because they are better known. These records may refer to some units of a precious metal such as gold, but no precious metal is ever exchanged in practice.
5. Consumer debt has always been a problem. When people face too much debt, they tend to either rebel or exit the system. Some people might join nomads who later regroup and attack the cities. Eventually, governments have to defend the indebted population or face collapse.
6. When raising a large army, you need to pay them. How? More precisely, how do you get the conquered population to trade with your soldiers? Well, you have typically just acquired a large quantity of gold by confiscating jewels and such. So you create a currency out of it which you hand over to your soldiers. Then you ask the conquered population to give you tribute (or pay taxes) in the form of this new currency. People are then forced to trade with the soldiers.

My thoughts:

• Money came from credit. Currencies came after.
• Gold-backed currencies and corresponding taxes were invented to sustain military might.
• The American dollar is backed by American military might.
• Consumer debt and usury are closely related to slavery and correlated with the fall of empires.
• It is likely that we are genetically geared toward the Smurf-village model, that is, communism. Non-ceremonial barter was probably dangerous and uncommon. It is probably a myth that we are jungle animals who try to maximize the profit from every trade. It is maybe not surprising that people have so much difficulty with debt.

Credit: I got this book free from William Tozier. As far as I can tell, William does not benefit from the sales of this book. Rather, I believe that he wants to promote the debate on these issues.