Unless you live under a rock, you have heard of the Hunger Games trilogy by Suzanne Collins.
They recently made a decent movie out of the first book. Like in What came after, the books describe a near-future dystopia where war and oppression have reduced humanity to few towns supporting a relatively wealthy capital. What I found interesting in these novels, is how the main character (Katniss) is an anarchist. That is, she cares for those she love (her tribe), but she is rather immune to big ideas and propaganda. This becomes clearer as the story progresses, and many people have hated the ending for this reason. The trilogy is reasonably priced: you can get all of it for under $20.

In the Trilisk Ruins, Michael McCloskey describes a far future universe where human beings have encountered alien ruins on diverse planets.
These ruins have obvious commercial values: alien artifacts are immensely valuable. Meanwhile, the government has restricted access to these ruins to its own military. The main character is a xenoarchaeologist who is frustrated by the lack of access to these new findings. She decides to embark with a bunch of pirates/mercenaries who hope to visit new alien ruins before the military can get their hands on them. The novel touches on a common theme in scifi: it is unwise to put your military in charge of first contact with aliens. You can get the e-book for $2.99.

The Galactic Mage by John Daulton is a twisted, but fun story. We have human beings from Earth who were in contact with another human civilization across the galaxy. This civilization was apparently wiped out after it sent a warning. A fleet of ships from Earth is assembled to go investigate. Meanwhile, on a remote planet, a bona fide mage has decided to go explore space. He does so by teleporting himself (and his tower) in space. At first, the premise seems unbelievable, and it is, but it is fascinating to see how a mage might explore space. Without any scientific background, he is faced with several challenges such as unimaginable distances. Unfortunately, the novel never quite feels complete: many issues are left unresolved. Nevertheless, it is an entertaining book well suited for teenagers or people looking for a fun little book. You can grab it for $2.99.

Finally, the Empire of the Gods by David Stag is a well written space adventure. It describes a universe where most people live in misery under an oppressive and all-knowing government. The leaders derive their power from mysterious rods that seem to give them the powers of gods. I found it very entertaining even though there are obvious flaws. For example, as in the Galactic Mage, we are supposed to believe that the universe is populated by human beings. Thus, as scifi, this is a so-so book, but thankfully, the novel works well as a fantasy. The writing is solid. The e-book is only $0.99.

Disclaimer: I got Empire of the Gods for free from the author.

So we need a system whereas perfect strangers can make deals at a very small transaction cost. For this purpose, people use punk money:

• You publicly promise a favor in exchange for a service, you may stipulate the terms.
• The Web records your transaction.
• Your public reputation guarantees the transaction.

Six months ago, I needed a particular piece of software. In exchange for the code, I promised to promote a web site on the Google+ social network. Joshua Grochow won the contract and I owe him.

Punk money should also be able to solve more systemic problems. For example, it is often hard to find good reviewers for research papers. To solve this problem, we create an intermediary between the authors and the reviewers (e.g., a conference or a journal). This intermediary is often supported by a larger organization (e.g., a publisher) seeking financial gain.

As an alternative, I published the following open contract in the spirit of punk money:

1.  Write a research paper in my general area of expertise.
2. Send me the paper.
3. I will read it in a reasonable delay (not 3 freaking months), or tell you that I’m too busy. (If your paper is really bad, I might also ignore you, politely.)
4. I will give you feedback.
5. At some point, I might feel that the paper is quite good, and then I will publicly say so, putting my reputation on the line.

In exchange, you have to mention me in the acknowledgements.

So far one researcher took me up on this offer and I reviewed his paper (privately) within a few weeks. If many researchers adopted a similar open contract, we could create a workable lightweight alternative for scientific publishing. It will not replace journals or conferences, but it is also nearly free compared to the current system.

The famous mathematician Doron Zeilberger has used this punk approach to validate some of his research papers. Indeed, he has a set of papers that only appear on his web site. He validates some of them by asking other mathematicians to review his work. The net result is that you can probably trust these papers, after all some established mathematicians were willing to vouch for the work. Journals cannot offer anything better. In exchange, he acknowledges the other researchers. The transaction happens without an intermediary.

Of course, you could apply the same type of contracts to any type of publication. Perhaps you are willing to review books or novels, and promote them in exchange for some favor. Perhaps you are willing to provide code reviews for open source projects. And so on.

The concept is entirely general. Maybe there is an annoying bug in your favorite open source web browser, and while you cannot fix it yourself, you would be willing to put a bounty. What could it be? Maybe you are willing to have a pizza delivered to the house of the programmer who provides the fix. Or maybe you will post a poem in honor of whoever fixed the problem.

Punk money has three major characteristics:

• There is no real intermediary other than the Web. Or rather, the intermediary is light and easily replaceable.
• There is a written record.
• It is an explicit credit system, not free labor.

In contrast, a site like Stack Overflow allows you to ask a question for free. People who do the hard work of providing a detailed answer get clown money (e.g., reputation points). The intermediary (the owner of Stack Overflow) is not easily replaceable: it is an essential component of the system. Such specialized sites work amazingly well for specific problems, but punk money has far broader potential.

Note: I think there should be a Wikipedia article on punk money. It should trace back the origin of the term and provide sufficient context so that it stands a chance of meeting Wikipedia standards. I am too lazy too do it, but if you do it I will update this blog post with a link to the new entry and, if you wish, a note crediting your effort.

Further reading: See punkmoney.org for a related tool.

Update regarding taxation: The trades I have in mind are already happening without any fiscal consideration. I already review research papers, that’s a service I render. I also benefit from this service when I submit a research paper. I also support open source software (including handling bug reports) while benefiting from the open source software services of others (e.g., Linux). Yet there is no taxation involved.

Update regarding the word “currency”: What I describe is a credit system, not a currency. I used the word currency in my title because it sounded good.

Nevertheless, one thing that has become absolutely clear to me is that computer scientists do not know about significant digits.

When you write that the test took 304.03 s, you are telling me that the 0.03 s is somehow significant (otherwise, why tell me about it?). Yet it is almost certainly insignificant.

In computer science, you should almost never use more than two significant digits. So 304.03 s is indistinguishable from 300 s. And 33.14 MB is the same thing as 33 MB.

Why does it matter?

• Cutting down numbers to their significant digits simplifies the exposition. It is simpler to say that it took 300 s than to say that it took 304.03 s.
• Numbers expressed without significant digits often lie. Running your program does not take 304.03 s. Maybe it did this one time, but if you run it again, you will get a different number.

Please learn to express your experimental results using as few digits as you can.

• But we still need to print the articles on paper! At least in Computer Science, almost everyone has adopted electronic media. It is cheaper and more convenient. I carry thousands of research papers on my laptop: I would require a part-time archivist to get the same result with paper. And 99% of all references are a mouse click away. Given a research paper, I can quickly search through it for interesting terms. It is true that paper versions can sometimes be handy. However, we have this marvelous technology called the personal printer. You can get one for $100. And these printers are connected to computers smart enough to print just the pages you need. You need to review the proof of a theorem on paper? Just print out the proof, specifically. Most people who can afford access to printed journals can afford a printer and the printing costs.
• Reviewers prefer to review short papers. It can be more difficult to review a short paper than a long paper. I speak from experience. For example, I am currently reviewing papers for ACM RecSys where we have two tracks: short and long papers. It takes me just as long to review short papers. Indeed, reading the text itself is not the bottleneck. What takes the bulk of my time?
  • Checking the literature is time consuming. I often ask myself: did they really advance the state-of-the-art? Other times, I want to check how the submitted manuscript differ from previous work from the same authors.
  • Reviewing the methodology or the mathematical proof also takes me a long time, especially when the authors have omitted details.

  If the authors expand unnecessarily on uninteresting aspects of their work, or spend much time reviewing elementary facts, it does not slow me down much because I can easily skip it, as long as the work is well structured. In fact, I find that I can get the gist of an entire Ph.D. thesis, if it is well written, faster than I can understand some short research papers. To summarize: the number of pages is not the primary factor determining how long it takes to review a paper. The problem is not that papers are too long, rather it is that they are often written too poorly.

• We want to entice authors to be concise. Everything else being equal, a concise text will be better written and easier to read than its longer counterpart. However, everything else is not equal. For example, Venkatesh Rao’s brief History of the Corporation is a blog post containing 7000 words. It is an order of magnitude larger than most blog posts. Aren’t Internet users supposed to suffer from attention deficit? Surely, nobody has time for such a long blog post? Yet it has become a classic. It has been extensively covered by various Internet news sites and forums, cited thousands of times. This is no excuse to use long and complicated sentences or to repeat yourself: Rao is an expert writer even though he writes long blog posts. So, while it is true that we have little tolerance for boring ramblings, what matters is less the length of the text, and more how interesting it is.

Thankfully, page limits are going away, slowly. Adam Marcus sent me a link to the UIST call for papers where they are openly flexible regarding page limits:

While we will review papers longer than 10 pages, the contributions must warrant the extra length.

Similarly, John Regehr sent me link to the OOPSLA call for papers:

The length of a submitted paper should not be a point of concern for authors. Authors should focus instead on addressing the criteria mentioned above, whether it takes 5 pages or 15 pages. It is, however, the responsibility of the authors to keep the reviewers interested and motivated to read the paper. Reviewers are under no obligation to read all or even a substantial portion of a paper if they do not find the initial part of the paper interesting.

Further reading: Stephen King made a killing his novel The Stand. Yet it spans nearly 1500 pages. Rao wrote several posts on why he shouldn’t be expected to use few words: Seeking Density in the Gonzo Theater and Just Add Water.

Update: According to an anonymous reader, copy editing is often charged by the number of pages. So it can cost twice as much to publish a paper twice as long, even if you only publish it electronically.

A fascinating example is the French language itself. I long had this image of the French revolution as the French people, that is, the people who spoke French, rising up. But during the revolution in 1789, only half the population of France spoke some French. The state of France created the French language we know today. It was an act of social engineering to ensure that there would be a united French people.

A widespread instance of this strategy is political correctness. Apparently, it is racist to say that Martin Luther King was black. We don’t have firemen anymore, have you noticed? We have firefighters.

The term climate change is another fascinating example. Prior to 2003, we talked about global warming. It changed when Frank Luntz, a political consultant, convinced the American president to force people to talk about changes instead of warming, because it feels less threatening.

Another example is “intellectual property”. If “intellectual property” is bona fide property, then you should be able to steal it. Can you? The Supreme Court of the United States thinks you can’t steal intellectual property the same way you can steal cars:

(…) interference with copyright does not easily equate with theft, conversion, or fraud. The infringer of a copyright does not assume physical control over the copyright nor wholly deprive its owner of its use.

Yet even judges get confused. Recently a programmer from Goldman Sachs who copied and shared secret software was acquitted of theft charges. Yet, in its verdict, the court writes that he stole purely intangible property embodied in a purely intangible format. He cannot be convicted of theft, so why use the word in the first place? The intellectual property lobby goes even further when it talks about piracy. (Thankfully, they haven’t yet prosecuted someone for actual piracy.) Effectively, they have changed the language, they have gotten us to attribute new meaning to existing words, to associate piracy and theft to the infringement of exclusivity rights.

Scientists often play the same games. For example, to make something sound serious, just append engineering to it: knowledge engineering, software engineering, data engineering.

Experience has taught me to be suspicious of people who spends too much effort redefining words. They are probably not out to help you think clearly.

Credit: Thanks to Marc Couture for the legal reference and an inspiring discussion.

Related video: Too Much Copyright

Further information: Euphemistic Language by George Carlin, Words That Work by Frank I. Luntz, How Not To Say What You Mean by R. W. Holder

Update: I do realize that global warming and climate change refer to different concepts from a scientific point of view. But what people worry about is not so much the change, as change is unavoidable. Rather, we worry about the warming… don’t we? Or are some people really set on preventing any kind of climate change?

However, before you pack the integers, you might need to scan them to determine the number of bits needed (N). Unfortunately, it is a relatively expensive process.

Given a positive integer x, we seek the smallest integer N such that the integer x is less than 2^N. The value N is often called the integer logarithm of x.

There are several clever techniques to compute the integer logarithm using portable C code. Yet you can do better using processor-specific instructions. The GNU GCC compiler makes this easy with a special function that counts the number of leading zeros for 32-bit integers (__builtin_clz). Even so, it is relatively slow.

Thankfully, you can avoid computing the integer logarithm of each integer by a simple test involving a right shift:

if((x>>b) !=0)
b = integer_logarithm(x);

With proper loop unrolling, this is nearly as fast as bit packing.

Update: Preston Bannister correctly points out that you can do much better. Simply compute the logical or between all integers and then compute the integer logarithm of the result. It is much, much faster.

To experiment with this problem, I wrote a small program which finds the maximum integer logarithm of a large array of random integers. It then packs the integers using this logarithm.

• I find that I can pack between 1 billion and 2 billions integers per second.
• I compute the maximum integer logarithm at a rate of 3 billion integers per second.

When plotting the speeds as functions of the actual maximum integer logarithm, we see that the computation of the logarithm is not sensitive to the value of the actual logarithm, except for the approach based on the __builtin_clz function which is slower when the logarithm is less than 8.

In my tests, I used the GNU GCC 4.6.2 compiler on an Intel core i7 processor. My code is freely available.

Conclusion When packing an array of integers, finding the maximum logarithm can take anywhere from 1/4 to 1/3 of the running time. However, brute force techniques that compute the integer logarithm of every integer are much slower.

How did it go?
Today, not only do I now own two sailboats, I’m building a third one. Of course, they are radio-controlled boats, about 4 feet tall and 2 feet long. I find that I really like to design and build these little boats.

Did I fulfill my dream? I am sure the younger self would be unimpressed by the small boats I own, but he would be floored by the two young boys who accompany me when I test them. My dream of owning a sailboat became irrelevant. Today, in 2012, if I wanted to go on a sailboat trip, I would probably rent one. And what would matter most to me is the look on the faces of my boys.

I find that this irrelevance of my earlier dreams is a common pattern throughout my life. My younger self was dreaming about having things and being someone. He thought this would bring happiness. He was wrong.

Today, I focus on doing things. I do not own very much and I am not someone important. But I do fun things. I write this blog, I write research papers, I build boats, I publish software, I play video games with my boys… These are the things that matter.

Note: I am grateful to Maxime Larocque for keeping his old high school pictures and posting them on Facebook.

Why is that a problem?

• We end up working with simplistic models. If we consider the main table generated by dbgen, out of 17 columns, 7 have uniform distributions. This almost never happens with real data. Similarly, we often end up with attributes that are perfectly statistically independent. I believe that the randomly generated data is not be representative of the real data you find in real businesses.
• Because so many people use essentially the same data generators, we risk having solutions that are optimized for this particular synthetic data. This allows researchers to essentially cheat, sometimes unknowingly.

However, finding suitably large real data sets is difficult. My own research focus is on data warehousing. Most businesses are unwilling to share the data in their data warehouses. Even if they were willing to do so, sharing very large files is inconvenient.

Here are three moderately large data sets that I have used in my research:

• I found a table of wikileaks-related metadata on Google Fusion. By transforming it into a relational table, I was able to create a table with over a million rows. It is tiny by data warehousing standards, but the data is easily accessible.
• I like the Canadian Census from 1880. You have data about 4 million Canadians. The data is publicly available and convenient. See my paper in Information Sciences from 2011 for details on how we retrieved and processed it.
• There is weather data set that we have used repeatedly. It is relatively large (9 GB) and freely available as the Edited synoptic cloud reports from ships and land stations over the globe. You can retrieve all of the data files from the ftp directory and aggregate them in a single table.

Can we do better? I asked on the Internet for large tabular data sets (greater than 20 GB) that can be considered representative of business use. I have not found anything that match my needs yet, but I am investigating further one data set that I consider especially promising:

• James Long suggested I have a look at the US Census from 2000. I am currently downloading some of their data. I am unsure how big a table I can construct, but there are gigabytes of freely available data.

I also received other excellent proposals that I am not pursuing for the time being:

• Jared Webb suggested I grab one of the large health data sets. I ended up getting the SEER data set from National Cancer Institute, as recommended by Jared. I got a 210 MB zip file filled with relatively small flat files. The data is freely accessible, but you must fax a signed form. It is not nearly large enough for my needs.
• Howard C. Shaw III suggested the Eron data set. Unfortunately, the Enron email data set is primarily made of unstructured (text) data. It does not suit my purposes.
• Howard and Neil Conway also pointed out that I should look at the Amazon public data sets. These are moderately large data sets that Amazon makes available to its web services customers. In particular, Tim Goh suggested I look at the Freebase data dump. Unfortunately, I am not an Amazon customer and I am uneasy about basing my research on data that is only available through an Amazon subscription. Thankfully, many of the Amazon public data sets are available elsewhere. For example, Nicolas Torzec told me that freebase is making available its database dumps from their site. He says that the full dump is 4GB compressed and that it contains about 50M objects.
• Tim Goh reminded me of the Google Book n-gram data set. (Update: this is different from the Google n-gram data set which is not freely available.) It is not typical data warehousing data however.
• Kristina Chodorow pointed me to a page on quora.com containing a long list of publicly available data sets. There is a wealth of information there, but I find that a lot of times, access is difficult, or the data is unstructured or very specialized (e.g., web search).
• Jeff Green suggested I look at the NHS public data sets. I spent some time on their site and could only find small data sets.
• Aaron Newton, Michael Ekstrand and Nicolás Della Penna suggested a snapshot of the Wikipedia database. Unfortunately, again, much of the data is unstructured.
• Brian McFee pointed me to the Million Song dataset. From what I could tell, you have detailed data about a million songs. It is primarily geared toward music information retrieval.
• Matt Kevins pointed to the Google public data sets. Alas, I could not find out how to download the data sets and I am not sure how large they are.
• Aleks Scholz pointed me to the all-sky data set. It is a large, freely available, astronomy data set. Most of the data is made of floating-point numbers so it does not fit my immediate needs, but it looks very interesting.
• Israel Herraiz suggested I look at the Sourcerer project. It looks like a large collection of Java source code. This could support great research projects, but it is not a good match for what I want to do right now.
• Axel Knauf suggested web crawling data. Probably not a good fit for my current research though it can be very valuable if you work on web information retrieval.
• Peter Boothe asked whether I was interested in BGP routing data count. It looks like network data. I haven’t looked at it too much. Could be interesting, but it is probably too specialized.
• Tracy Harms asked about the Netflix data set. Unfortunately, this data set is no longer publicly available and it was only 2 GB. I used it in my 2010 Data & Knowledge Engineering paper on bitmap indexes.
• Aaron J Elmore pointed me to oltpbenchmark.com for an online transaction processing benchmark framework. My research is not primarily on transactions (OLTP), but this is a very interesting project. They have collected data sets and corresponding workloads. In particular, they link to Wikipedia access statistics. It could be very important if you are designing back-end systems for web applications.
• Òscar Celma points to a Twitter social graph which occupies several gigabytes.

(If you answered my queries and I have not included you, I am sorry.)

Yet we are trained to hold copyright as a natural right. People who infringe on copyright are labelled as pirates, thieves. We are told that they literally steal from hard-working creators.

Most people learn about copyright in schools. Often the local librarian will lecture students on the evil of photocopying a book. However, as is often the case, schools fail us. We have good reasons to be critical of copyright and we should question the myths that are often reiterated about copyright.

First myth: Copyright is meant primarily to protect authors.

This is a lie.

• State-enforced copyright came about with the Statute of Anne in 1710. It was the result lobby of a group English publishers who sought to regain their monopoly on publishing. Handing out the initial copyright to the authors was a political gesture: the goal has always been to get authors to hand over the copyright to the publisher, effectively giving the publisher a monopoly.
• In most countries, copyright hold for 70 years after the death of the author. Such a long-term copyright cannot possibly be meant to protect authors.

Second myth: Copyright protects the little guy.

Most of the revenue due to copyright go to wealthy individuals and corporations. Meanwhile, most people who rely on their copyright for a living (writers, musicians, and so on) have low incomes.

Third myth: Without copyright, there could be no innovation.

Some of the most innovative domains are virtually free from copyright:

• The fashion industry is effectively copyright-free. Anyone can come up with a new design for a dress. If the design is successful, it will be copied and it is unpractical to try to enforce copyright. Thus, fashion designers must constantly out-innovate the competition.
• Researchers freely hand over the copyright to publishers in exchange for nothing. Researchers are driven to invent and innovate because their remuneration and social status depends on their reputation. If anything, copyright on research work slows down progress.

Fourth myth: We know that copyright makes us collectively better off.

The evidence points in the opposite direction. Germany had weak copyright laws up until the Copyright Act of 1901. Yet, maybe because of these weak laws, it became a literary and scientific power:

(…), only 1,000 new works appeared annually in England at that time — 10 times fewer than in Germany — and this was not without consequences. Höffner believes it was the chronically weak book market that caused England, the colonial power, to fritter away its head start within the span of a century, while the underdeveloped agrarian state of Germany caught up rapidly, becoming an equally developed industrial nation by 1900. (No Copyright Law The Real Reason for Germany’s Industrial Expansion? by Frank Thadeusz)

Your dentist probably does not have access to the latest research papers in dentistry: subscribing to a single scientific journal can cost thousands of dollars a year. Is it any surprise if the general public is poorly informed when copyright is used to keep them away from the best science, leaving them only generic news content and blogs?

Even if you don’t care about science, you should be concerned with the cost of copyright. For example, have you seen the latest Star War movies? They are awful. But that is all we are going to get for at least another 70 years because George Lucas has a monopoly on Star Wars. Without copyright, or with more limited copyright, we would have had several creators competing to build better Star War movies.

Fifth myth: Without copyright, authors would not get paid.

Authors do not have to get paid. Scientists and many authors actually pay to be published. Some authors publish for the indirect benefits of their publications, such as an improved reputation.

However, when authors do get paid, a natural model is patronage. That is the model used by most scientists.

“But, Daniel, you are delusional! Not every writer can find a patron.” Am I? I have funded several book projects myself. For example, a lady called Kio Stark got $38,928 from us to write a handbook on alternatives to schooling.

Several authors get funded on kickstarter:

• Rich Burlew received $1,254,120 to get back in print an old comic book.
• Dennis McKenna received $85,750 to write a memoir.
• Cory Silverberg received $65,516 to write a book on where babies come from.

In fact, if you think about it for a minute, whenever you buy a book or a movie, you are being a patron to this project. So all work is the result of patronage. Cory Doctorow makes all his novels available for free from his web site. He happens to be one of the most successful writer of his generation. You can be confident that he is doing well financially. It works for him because people are willing to support him: his paying readers are his patrons.

I should add that whenever you follow a link to Amazon.com from my site, and purchase something, I get a percentage of the transaction. On a good day, I can make $5 with my blog. I could also add ads and make a few hundred dollars a month. You do not need copyright laws to make some money off a blog: your readers can act as your patrons.

My position: I see no justification for copyright. I am effectively a writer: I write lecture notes, research articles and blog posts. I get paid without relying on copyright. Instead, I have patrons: funding agencies, students, and blog readers. But if we insist on having copyright, it should at least be limited to a short term (say 5 years or less).

Further reading: Defining Property by Paul Graham and What We Know, What We Don’t Know, and What Policy-makers Would Like Us to Know About the Economics of Copyright by Ruth Towse. You may also enjoy my blog post Do we need patents?

Appendix

American authors have always enjoyed the protection of copyright laws in the USA. Prior to the adoption of the US constitution, authors in the US were subject to the Statute of Anne. However, the work of foreign authors in the USA were considered to be in the public domain up until the end of the 19^th century. One might think, since copyright is supposedly good for authors, that foreign authors would be penalized by this lack of copyright. It seems that there were not. Dickens made a fortune in the USA despite the lack of copyright. Foreign authors could sell their “authorization” and they would frequently negotiate advances in excess of what they could get in Europe. (References: Khan, Does copyright piracy pay and Plant, The economic aspects of copyright in books)

What about contemporary examples? Indian intellectual property enforcement has been historically weak. You can readily find copies of American movies made in India, and American studios do not bother suing: Indian courts place a high bar on infringement. Derivative works (such as making a movie or a book out of an existing work) are often not found to be infringing copyright in India whereas they would elsewhere. So how does India fare culturally? Well, outside of America, Indian movie production is unsurpassed (hence the term Bollywood).

Similarly, Japan, Korea and Taiwan have maintained weak intellectual property regimes. It is believed that this was a key factor to explain their economic growth during the second half of the XX^th century. (Kumar, Intellectual Property Rights, Technology and Economic Development: Experiences of Asian Countries)

A better measure is the number of citations a researcher has received. Google Scholar profiles display the citation record of researchers prominently. It is a slightly more robust measure, but it is still silly because 90% of citations are shallow: most authors haven’t even read the paper they are citing. We tend to cite famous authors and famous venues in the hope that some of the prestige will get reflected.

But why stop there? We have the technology to measure the usage made of a cited paper. Some citations are more significant: for example it can be an extension of the cited paper. Machine learning techniques can measure the impact of your papers based on how much following papers build on your results. Why isn’t it done?

People object that defining a metric based on machine learning is troublesome. However, we rely daily on spam filters, search engines and recommender systems that we do not fully understand. Measures that are beyond our ability to compute by hand have repeatedly proven useful. Moreover, identifying important citations can have other applications:

• Google Scholar says that I am cited about 160 times a year. On average, a paper citing me comes out every two days. What does it mean? I don’t know. I would be interested in identifying quickly which papers make non-trivial use of my ideas. I am sure many researchers would be interested too!
• I sometimes stumble on older highly cited papers. I want to quickly identify the significant follow-up papers. Yet I am often faced with a sea of barely relevant papers that merely cited the reference in passing. It would be tremendously useful for me to know which papers have cited the reference meaningfully.

Hence, I surveyed the machine learning literature on classifying citations. I found high quality work, but I feel it is an under-appreciated problem. So I got in touch with Peter Turney and Andre Vellino and we decided to promote this problem further.

Our first step is to collect a data set of papers together with their most important references. We believe that the best experts to determine what are the crucial references are the authors themselves!

So, if you are a published researcher, we ask you to contribute by filling out our short online form. On this form, you will be asked for your name and a few papers together with an identification of the crucial references for each paper. The form can take less than 30 seconds to fill out.

In exchange, we will publish the data we collect under the ODC Public Domain Dedication and Licence. If you leave us your email, we will even tell you when the data is publicly available. Such a public high-quality data set should entice a few researchers to write papers. And, of course, I might contribute to such a paper myself.

My long-term goal is simple: I hope that in a couple of years, Google Scholar will differentiate between citations and “meaningful” citations.

Now go fill out the form!

Note: I have an earlier version of this post on Google+ with several insightful comments.

Further reading: Building a Better Citation Index by Andre Vellino

Programmers nearly never store integers in this manner despite the obvious compression benefits. Indeed, bit packing and unpacking is expensive. How expensive? Intuitively, you might think that recovering 32-bit integers from a stream of packed integers must be at least as expensive as copying the 32-bit integers, and possibly much more expensive. If that is your intuition, then you might be wrong. It can be cheaper to recover 32-bit integers from packed 4-bit integers because you only need to load one 32-bit word to unpack 8 integers.

Clearly, packing integers in units of 17 bits is not especially convenient. Indeed, 17 and 32 are coprime. We expect that it would be much faster to pack and unpack integers in units of 4, 8 or 16 bits, than in units of 17 bits. Indeed it is but the difference is maybe not as large as you might think.

I have implemented efficient packing and unpacking routines in C++. To simplify the implementation, we pack and unpack integers in sets of 32 numbers. I have optimized the code using the GNU GCC 4.6.2 compiler.

On my macbook air (Intel core i7), I get that the unpacking speed is not very sensitive to the specific number of bits: generally, the smaller the bit width, the faster the unpacking. The packing speed is much faster when the bit width is 8 or 16. Even so, the difference is only by a factor of two or so. The results are presented in the next figure. On the y axis, you have the time (smaller is better). On the the x axis, we have the number of bits we packed to. For example, when bit is 1, we pack 32 integers into a single 32-bit word. When the number of bits is set to 32 bits, we have a regular copy.

I also provide the raw numbers behind the figure in the next table.

Conclusion: Bit packing and unpacking can be quite fast. In particular, it can be cheaper to unpack integers from a small number of bits to 32-bit integers than to copy the same 32-bit integers. Exact results will vary depending on your compiler and CPU.

Note: Strictly speaking my implementation packs the first bits of each integer: it is not assumed that the integers are between 0 and 2^bit. By adding this assumption, you can improve the packing speed somewhat (at least when the number of bits is not 8 or 16).

I am not shy: I can talk in front of 200 people without thinking twice about it. I don’t lack confidence. I have a large ego—too large some would say. (I got my wife to read this post and she particularly agrees with this last sentence.)

But my social interactions have high transaction cost: it takes me time and energy just to start chatting with someone. If I have to chat with dozens of people in a day, I end up exhausted. I can’t pretend to be your friend on the fly. My brain does not work that way.

I love how it is progressively becoming “ok” to be an introvert:

• Carl King wrote a beautiful essay: 10 Myths About Introverts. The last myth is the most important: “Introverts can fix themselves and become Extroverts”. Gays cannot become straight. Blacks cannot become white. I cannot become an extrovert. King’s essay impressed me so much that I bought his book So, You’re A Creative Genius: a great read if you are both a creative person and an introvert.
• Susan Cain gave a great talk based on her book: Quiet—The Power of Introverts in a World That Can’t Stop Talking. A core message of her talk is that we should recognize our bias against introverts. Not everyone works best in groups. And that’s ok. Schools and employers need to stop their attempts to fit introverts in the extrovert mold.

Further reading: Venkatesh Rao (another introvert) penned an intriguing analysis of introversion.

Note: This is an expanded version of a Google+ post.

Borjas and Doran examined the problem. Their starting point was the realization that the expertise of Soviet mathematicians differed from the expertise of American mathematicians. For historical reasons, these two communities did different mathematics. So we can examine side-by-side areas of mathematics impacted by the arrival of the new mathematicians and areas unaffected. Their verdict is that the influx of Soviet mathematicians was unhelpful to academic production:

It is (…) difficult to find convincing quantitative evidence that there was an improvement in the overall “output” of either the pre-­‐existing American workforce or of that community combined with the Soviet émigrés.

What happened? Basically, the number of research jobs, including tenure-track positions, is mostly independent from the supply of Ph.D.s: Harvard will not hire more professors next year even if all of the Ph.D.s in the world move to Boston. The number of new hires depends of factors such as government funding for research and the number of undergraduate students in research universities. So the new researchers just took the jobs that would have gone to other researchers. Young American researchers had to drop out of research for lack of a position.

This study offers an important policy lesson. Training more Ph.D.s in some targeted areas might fail to improve research output in these areas. In this instance, supply-side economics fails. It might be preferable to create new research jobs instead and attract the Ph.D.s with better salaries.

For example, imagine that the government wants to help cancer research. Providing more scholarships to graduate students either directly, or through grants to professors, sounds sensible. However, it might not have the desired effect at all, according to this study. It would be preferable to use the money to create more research jobs pertaining to cancer research.

Credit: Thanks to Larry Larbear for the reference to this study.

Let us look at a concrete example. I am given this array:
0,5,1,4,5,1,10,4.
I want to create a new array with these values:
0,5,1,40,5,1,100,4.

Most programmers would follow the following algorithm:

1. Copy the array first;
2. Iterate over the list of positions and updated values ([3,40], [6,100]) and correct the values.

If there are very few values that need to be updated compared to the size of the array, this approach is probably optimal. The copy of the array itself is very efficient because it is entirely vectorizable: the processor does not need to copy values one at a time, it can copy two or four at a time.

But what if 10%, 20% or even 30% of the values need to be updated after the copy? Then storing the list of positions can become wasteful. For my toy problem, I have two positions to record (3 and 6): it can use 64 bits when using 32-bit integers. If I want to be more efficient, I can use 8-bit integers, thus using a total of 16 bits. (Most modern computers favor 32-bit integers, and it is generally not computationally efficient to use integers with anything other than 8, 16, 32 or 64 bits.)

A more memory-conscious approach is to use a bitmap. That is, I store the following value using a binary notation:
00010010
I put a 1 at the third and sixth position and elsewhere a 0. This makes up the integer 72. In this manner, I never need more than one bit per value. In this case, I use only 8 bits in total, a saving of 50% compared to the alternative where I store each position using an integer.

We need a different implementation however, one where you check the bits of the bitmap before copying.

• for every position in the array
  1. if the bitmap value is 0 then copy the value from the source array;
  2. if the bitmap value is 1 then copy the next available updated value.

This new algorithm looks inefficient. There is a lot of branching inside a tight loop. Yet the bitmap approach can be faster when the density of updates is high enough (>2%), as the next table shows.

My C++ code is online. I used GNU GCC 4.6.2 with only the -Ofast flag. Hardware-wise, I am using a recent MacBook Air with an Intel Core i7. (I stress that using GCC 4.6 is important. Older compilers might give different results. Also, the Core i7 is a processors with aggressive pipelining: cheaper processors might give different results.)

As you can see, the bitmap approach is optimal: a copy with updated values indicated by a bitmap is just as fast as a simple copy (within 10%).

(I updated the numbers on Feb. 23rd 2012. Originally, my code processed the bitmaps 32 bits at a time. I found that it was much faster to process them 8 bits at a time, probably because it allows better loop unrolling. I updated the code again on Feb. 27th 2012 after a bug report by Martin Trenkmann, the numbers were slightly updated as well.)

Conclusion Indicating exceptions using a bitmap can save memory without any penalty to the running time.

Code: Source code posted on my blog is available from a github repository.

Mathematically, frame-of-reference and delta coding use the same principle: we apply an invertible transformation that maps a set of (relatively) large integers to mostly smaller integers. (This is a common pattern when compressing data).

Suppose that you wish to compress a sequence of (non-negative) integers. Consider the following sequence:

107,108,110,115,120,125,132,132,131,135.

We could store these 10 numbers as 8-bit integers using 80 bits in total. For example, we have that 135 is 10000111 in binary notation.

The frame-of-reference approach begins by computing the range and minimum of the array. We see that the numbers range from 107 and 135. Thus, instead of coding the original sequence, we can subtract 107 from each value and code this difference instead:

0, 1, 3, 8, 13, 18, 25, 25, 24, 28.

We can code each offset value using no more than 5 bits. For example, 28 is 11100 in binary notation. Of course, we still need to store the minimum value (107) using 8 bits, and we need at least 3 bits to record the fact that only 5 bits per value are used. Nevertheless, the total (8+3+9*5=45) is much less than the original 80 bits. In actual compression software, you would decompose the data into blocks that are maybe larger than 10 values (say 16, 128 or 2048 values). The overhead of storing the minimal value would be small. Moreover, there are computational side benefits to this format: if we seek the value 1000, we know it cannot be in the block if its minimum is 107 and we use only 5 bits to store the offset from 107.

Frame-of-reference works when the range of values in each block is relatively small. We can sometimes get better compression if the difference between the values is small. In this case, it is useful to look at the differences between successive values (e.g., 108-107=1, 110-108=2, 115-110=5):

1,2,5,5,5,7,0,-1,4.

Given this set of differences and the initial value (107), we can reconstruct the original sequence. Delta coding is the compression strategy where we store these differences instead of the original values. Some people like to think of delta coding as a predictive scheme: you constantly predict that the next value will be like the previous one, and you just code the difference between your prediction and the observed value.

In binary, the values 1,2,5,7 and 4 can be written as 001, 010, 101, 111, 100. If we did not have a negative value (-1), we could store these differences using only 3 bits per value. The negative value comes from the fact that our values are not entirely sorted (just locally so). However, as we shall see, this single negative value will cause us some trouble. How do we code the -1?

• The original values are 8-bit values. This means that -1 and 256-1 are the same numbers (modulo 256). That is 25+255 modulo 256 is 24. In effect, we compute differences in an integer ring. The differences become 1,2,5,5,5,7,0,255,4. Computing the modulo with a power of two is fast because computers use the binary format natively.
• If you know the value that was predicted (25 in our case). You know that the range of differences goes from -25 to 230. Thus for differences x between -25 and 25, we store them as 2x if it is positive and as -2x-1 if it is negative. Otherwise, we store it as x+25. One problem with this approach is that it may require much branching: the processor has to constantly check conditions before proceeding further. There may be a substantial penalty to pay when using modern superscalar processors.
• We can replace subtractions by bitwise exclusive or (xor) operations. It bypasses the issue entirely because xoring integers never generates negative values. The successive xor values are 7,2,29,11,5,249,0,7,4. A benefit of the xor operation is that it is symmetric: x xor y is y xor x. This means that inverting the order of the original list, we would simply invert the order of the list of differences. Obviously, computing the xor is quite fast.

Once we have the list of differences as non-negative numbers, we can then try to store them by using as few bits as possible. Unfortunately, in our case, we could to the conclusion that we need 8 bits to store the differences. We remarked however that for all but one value, 3 bits per difference would suffice.

So a sensible solution is to code the first 3 bits of each differences: 001, 010, 101, 101, 101, 111, 000, 111, 100. And then we add a pointer to the second last difference to indicate that we are missing 5 bits (11111). The cost of coding this exception is about 13 bits. So the total storage cost would be (8+3+9*3+13=51). In this case, frame-of-reference is preferable to delta coding, but both are preferable to the original 8-bit coding which used 80 bits.

There are many possible variations. For example, you can also use exception technique with the frame-of-reference approach when almost all values fit in a range of values, except for a few.

Further reading: the document SZIP Compression in HDF Products and the corresponding CCSDS 120.0-G-2 data compression standard describe the application of delta coding for scientific data. Michael Dipperstein’s page provides a nice overview of generic compression techniques. The specific exception technique I described is from the NewPFD scheme first described in:

H. Yan, S. Ding, T. Suel, Inverted index compression and query processing with optimized document ordering, in: WWW ’09, 2009.