Anyway, apparently the idea now is:

1. Pick up the nearest book.
2. Open to page 123.
3. Find the fifth sentence.
4. Post the next three sentences.
5. Tag five people and post a comment here once you post it to your blog so I can come see!

Now, the closest book means I have to dig through the piles of paper on my desk.  I am assuming closest is in Euclidean distance and not in time it takes to get at it.  On my desk I have a few books that I have had here for ages and never use — a few manuals that I never look at because Googling is much easier, and a dictionary that I haven’t used since I discovered Webster online.  So, the closest book, once I made it through piles of papers I should have read but never got around to, is: Oxford Advanced Learner’s Dictionary.  It was a gift from my sister — I have the big, red, massive Webster myself, but I got this from here when she finished her studies and decided she would never need it again…

In my version — which doesn’t look like the picture at Amazon I linked to above — page 123 is bole to bona fide.

Sentence five is:

bolero n(pl~s) 1 (music for a) type of Spanish dance.

The next three are:

2 woman’s short jacket with no front fastening. boll n seed-case of the cotton pants or flax. bool-weevil  n destructive insect whose larvae eat cotton bolls. bollard n 1 short thick post on a quay or ship’s deck, to which a ship’s mooring ropes are tied. 2 short post on a kerb or fraffic island.

Wasn’t that a senseless exercise?

Now I need to pick five others to tag.  I’ll stick to people AU people who probably knows Widmann as well:  Peter JC, Peter Ahé, Kristian Høgsberg, Søren Sandmann and Michael Westergaard.

Now, where exactly am I supposed to leave a comment, Widmann?

I continue to be baffled about how information technology has changed the world.

When I started studying computer science, in 1995 (which is centuries ago in Internet years), the web was in the process of replacing Gopher and the Internet was not really something ordinary people knew about or used. Not in Denmark, at least. That changed fast.

Two-three years later, everyone had an Internet connection at home, and a little bit later, when Google showed up on the scene replacing AltaVista, Yahoo etc., the Internet replaced the encyclopaedia as the place you would find information (they may not like me saying this, but Google really did blow the competitors out of the water). I cannot really imagine living without the Internet and search engines any more. When I am not connected, I feel almost handicapped. In my work, I need to look up stuff all the time, and going through books and manuals is so much harder than just googling.

Anyway, that was a “local” change to life, and I didn’t really notice how the world got smaller through the same technology until a few years later. I was working on a project together with a guy in Australia, and one day it struck me how amazing it was that I could send an email in the evening about my progress so far, and while I slept he would work on the project and in the morning I would have his update. A letter would take days, but we were working almost interactively together on opposite sides of the world!

A little later, I was myself working in Australia, first at the University of Adelaide, a bit later at the University of South Australia — both times for shorter stays of a few months duration — and using instant messaging I could work interactively with people at home, and talk to friends. With my cellphone, I could talk to my friends while I was walking through a park in Adelaide and they were getting ready for bed in Denmark. It was a bit expensive, but it was possible.

Today, I frequently communicate over the ‘net with people from the North America, Europe, Australia and Asia. Very rarely South America, although I used to when I was working in a different field, and never really Africa for some reason, although I do get the occasional email from South Africa. If I want to, I can call people up all over the world, and as the price goes down (or Internet telephony takes over) I will be able to talk to people everywhere, from anywhere, over my cellphone.

Hell, I’m posting this sitting in my garden, connected over a wireless connection. I wouldn’t have believed this possible ten years ago.

There’s some drawbacks to this as well, of course, such as Jotun Hein never really understanding time zones and calling in the middle of the night from New Zealand to discuss a project… It happened a couple of times while Jotun was on sabbatical there. Mike Steel told me at a bbq that he had the same problem when Jotun calls him from the UK. But all in all, it is a great thing, in my opinion.

Information technology has made the world into a global village. It is said a lot, but in some sense it is true, and if you stop and think about it, it really is amazing.

Well, not the entire world has joined the village. The majority of people are not connected. I don’t know the numbers, but I wouldn’t think a billion people are connected (and we have just passed 6,666,666 billion people worldwide).

The program didn’t work for me. On the example data provided with the code, it complained about invalid input. Right! Naturally, I expected that Garrett had given me test data that was in some way outdated. It happens quie often that the test data isn’t updated when new features are added to a program, and that test examples no longer function the way they should. I make that mistake a lot, anyway.

So I got hold of Garrett, and yes the test example was incorrect and he sent me an updated test. It didn’t work either. We emailed back and forth a bit, and this time it was a bit of a puzzle. Everything worked fine at Garrett’s end, but the program just wouldn’t eat the input on my computer. I didn’t have the time to figure out why, yesterday, as I had a plane to catch, but I had another look at it today.

I first grep’ed for the error message I got: “CV frequencies must be between 0 and 0.50″. That line occurred a few places in the code, so I annotated the code with some output before each and that pinpointed this test as the problem:

if ((allelefreq[count] < epsilon[count]) || (allelefreq[count] > (0.50-epsilon[count]))) {
   printf("CV frequencies must be between 0 and 0.50\n");
   exit(1);
 }

Ok, so on Garrett’s machine, the test was false and on mine it was true, but the input files were the same. First I tested if the input was read in correctly (Garrett’s IO code is a bit of a mess, excuse me for saying it). allelefreq[count] was supposed to be 0.4 — and it was — and epsilon[count] should be 0.1 — and it was.

That’s when I spotted the problem. Do you see it?

Clearly 0.4 ≥ 0.1 and 0.4 ≤ 0.5-0.1 so the test should evaluate to false, but it doesn’t.

The thing is, floating point numbers aren’t real numbers (as you probably know). And for one thing, 0.1 cannot be represented in a finite number of bits (as a binary number) so it is approximated in the computer. So 0.5-0.1 is not 0.4 (although they could be represented by the same bit-patter as 0.4 is represented as). Whether 0.4 ≤ 0.5-0.1 or 0.4 > 0.5-0.1 on the computer depends on the way the numbers are represented.

Garrett was running his program on a 64 bit architecture (where the numbers were represented using 128 bits) and I on a 32 bit architecture (with 64 bits per number). That made all the difference.

Fun, eh?

Life as a postdoc

The postdoc (and grad student, for that matter) perspective is — and I recognize it both from my own experience and my colleagues — that you come up with most of the ideas, do all the work, and the PI just takes the credit. It is a bit exaggerated, but the point is that the vast majority of the time spend on a project is time spent by a postdoc or grad student. The advisor reads the occasional draft paper, comment on the research a bit here and there, but doesn’t really put in the hours.

I think this is true. I think this is absolutely true (and on this point I think I disagree with the post I link to above… or maybe not when I elaborate on the point…)

Before I elaborate, though, I should probably explain my own situation, so you know where this perspective comes from.

I am somewhere in between being postdoc or PI. The last couple of years I have been working on my own grants and not really been associated with any particular group, so I haven’t been a postdoc as such. Although I’m currently officially an associate professor, I am not associated to anyone, and I haven’t got tenure yet so I cannot start my own group. With the current rules in Aarhus, I cannot have grad students associated either. Instead, I have associated myself to different other groups, for individual projects, and when doing so take on roles as either “postdoc” or “advisor”, depending on the project and who else is involved.

Anyway, the situation is that I was relatively recently a postdoc — and remember how I felt about academic life then — and now I have experienced both the advisor and the postdoc world (but so far have never really been responsible for my own group, so my perspective might change later on).

So, as I said, I think it is true that on any one project, the postdoc will be doing the vast majority of the work, at least measured in hours. This isn’t really surprising. The postdoc can focus on one project (or a few projects), while the PI needs to manage many. Of course the postdoc will contribute the most time — but who contributes the most research?

Division of labour

This is a trickier question… first of all because it is not easy to define nor measure “research”. The best measure we have is the publications, but within a project you cannot use it to measure the output of the first author vs the last author.

The PI will be more experienced (or at least should be, but I’ll come back to that) so he will have a better idea about how to approach a problem, he will be able to spot dead-ends long before the postdoc, he will know the literature better, etc. This is all experienced he gained when he was the one doing the grunt work, and now he should use it to teach the postdoc. A PI hour might simply be worth more than a postdoc our because of the extra experience.

That is not the whole story, though. There is also what I would call a trade-off between “detail work” and “big picture”.

Most research is fiddling with details, trying to make epsilon improvements upon previous work. We all dream of a great breakthrough, but day to day research is about epsilon improvements, and that means fiddling with details. It requires a lot of work to fully understand the details of any kind of problem, and it is very easy to get lost in details and lose the big picture. How does the current work fit into the larger problems in a field, how does it related to previous work, and what will be the consequences of whatever you are trying to prove or solve? Someone needs to figure out the details, but someone also needs to keep the big picture in mind!

I’m not sure you can do both at the same time. At least I find it difficult to do both at the same time.

The postdoc deals with the details, and is probably the only one who fully understands the problem being worked on. The PI keeps the big picture in mind, trying to relate the current problem to the rest of the field.

The PI will spend less time on the particular problem — contribute fewer hours to the project — but at the same time he probably needs to keep up with a broader part of the literature and keep track of related projects to improve on this project. It doesn’t feel like he is contributing to the specific problem when he is just reading papers about other peoples work, but the hours he put in there helps him keep an eye on the big picture instead of getting caught in the details.

One of the perks is, of course, that keeping up with the literature pays off for more than one project at a time, so you get more publications that way. As I said, it is one of the perks of that side to research.

Experience matters (if you have it)

One thing is the “details” / “big picture” division, but another is experience. The postdoc will be less experienced in doing research. The whole package, including doing the actual research, writing the papers — just as important as doing the experiments and analysing data, really — applying for grants, etc. This is stuff you learn from experience, and is a lot easier to learn with the assistance of someone who already has the experience. That would be the PI.

The postdoc is still learning. He is being taught (or should be taught) by the PI. It might not feel like a student/teacher relationship, because it is very different from the typical teaching at a university, but it is such a relationship. It is just more of a master/apprentice relationship. The apprentice starts out doing only the grunt work, but gradually does more and more of the complicated work under less and less supervision, until he has mastered the discipline.

Experience is mainly a way to avoid dead ends, in my experience (no pun intended). When you have worked with something for a long time, you have a good intuition for what might work and what will not, so you avoid a lot of wrong turns. The experience of the PI will help the postdoc avoid a lot of pitfalls (probably without ever realising that there was a potential pitfall). There should be room for making mistakes — that is how we learn — but by supervising the postdoc, the PI can help to avoid the worst mistakes. By supervising less and less, leaving room for some mistakes but not until the postdoc is ready to deal with them, the PI allows the postdoc to gain the experience while still being a bit sheltered.

When supervising, it is actually harder to allow someone to make a mistake than it is to spot a mistake and avoid it, but sometimes mistakes are necessary to learn, so they should never be totally avoided… but that is a topic for another day…

Will the PI always be more experienced? Many paragraphs ago I promised to get back to this point. In most ways I would say yes. Writing papers, applying for grants, that stuff he should be more experienced. Otherwise he wouldn’t be the PI, really. But there is a limit to that experience and that is also important to keep in mind.

There’s two scenarios where the postdoc will be more experienced than the PI. If the project he is working on is outside the PIs previous experience, or when the detail-work is important but completely lost to the PI.

Let’s take the first scenario first. The PI gets an idea. “It could be fun to figure out if …” and although it is not really related to anything he has done before, he manages to get a grant for doing research on the project. He has the grant but not the experience. So he hires a postdoc to work on the problem, but cannot really supervise the science! (This is not as made up as it might sound; I have seen it happen a couple of times…)

The second scenario is probably more common. Here the problem is that by focusing on “the big picture” for so long, the PI has forgotten the details and ignores that that is where the problems really are. The big picture might tell you what the important questions are, but the answers are found in the details. People forget this too often, in my opinion. They ask an interesting question and then suggest “maybe you can solve it using an MCMC or something…” or “maybe you can solve it using dynamic programming…” but that is not a helpful suggestion at all! It is a complete disregard for the details, and the details are important (damn it)!

You loose the feeling for the details if you keep away from the for too long, and when that happens you cannot teach others how to deal with them any more.

Of course, a PI will never be as much into the details as the postdoc working with them every day, but luckily most problems are somewhat similar so what works in one setting might work in another so some experience is transferable. This is why experience can be substituted for hard work in many cases.

Sometimes the experience just isn’t there (for either of the two reasons I mentioned), and in that case the PI isn’t pulling his own weight on the project. A few general remarks to a draft and completely generic comments to solutions (”have you considered an ABC solution?”) are of little help. In such cases, I think a postdoc would be right in thinking that the PI just gets the credit but doesn’t do the work. Fortunately, it is not something that happens that often, and most PIs would recognise that they are not actually capable of helping with the problems and try to improve on it by getting to know the details a bit better.

Life as a PI

Being in transit from postdoc to PI I’ve noticed a few things that I never really thought about before. New problems when you are no longer a postdoc in a group. I’ve probably only noticed half of the new problems, since I have yet to experience running my own group.

Anyway, the most important difference, the way I see it, is related to the “big picture”/”details” thing. I no longer have time to focus on just a single problem for days or weeks. There is always a lot of different projects that requires my attention, and that means that I cannot put in the time really necessary to solve detailed problems. It is something I really miss, being able to completely focus on a problem for a couple of weeks, until I have it nailed. If I want to do that now, I have to do it in my vacation.

Another thing is the time it takes to come up with those few suggestions to drafts and projects. I would have guessed that you would spend half an hour before a meeting and that would be the time you would put into it when supervising a project. Maybe some people can do that, but I find that to really think about it I need to spend hours. Checking the literature, doing the math to check out if it works out, looking over the data with my own simple methods to get a feeling for it. It is not enough to get the feeling for the details that is really needed — as I said, there simply isn’t the time for it any more — but it takes a lot longer than I would have thought.

I’ve already begun to think of the postdoc days as the good old days, but I fully remember being envious of the PIs at the time. Over the coming years, if I get tenure, I will get more and more administrative and teaching tasks and I might end up missing the postdoc time even more.

I’ve been in Oxford and London the last few days — last weekend in Oxford and this week in London — visiting Jotun Hein in Oxford and David Balding at Imperial College.

The Oxford visit was more social than work. Over the weekend, it couldn’t really be more, but it was nice meeting the people there (who Jotun had working weekends, of course).

The visit to Imperial was for making progress on a paper I am writing with David. We didn’t make as much progress on it as I’d have liked — the time was simply too short — but we have a plan for how to progress from here, so that is fine. Meeting the group here and chatting with people has been very stimulating.

I’ve never really been much for going to conferences.  Sometimes they inspire me to some work, but usually I am just bored through most of it.  I’m just not good at sitting quietly and listening to talks.  Visiting people and chat with them, on the other hand, I find very inspiring.  Plus, you really learn a lot from talking with people that they would never mention in a formal presentation.

Anyway, now I’m soon on my way home.  Teaching awaits me when I am back, so until the end of term I won’t be able to go travelling again. Over summer I probably will.  Most likely to DeCODE.

Since constructing an alignment is really a statistical inference problem, we shouldn’t pretend that we can infer it flawlessly and completely ignore the uncertainty in the inference. Especially since doing so can bias the downstream inference. See a list of previous posts where I discuss these problems to get an idea about the problem.

At the very least, we should take into account the uncertainty associated with alignments. In many cases, however, we are not really interested in the alignment in the first place. It is a nuisance parameter in our model, necessary for extracting whatever information we are really interested in.

For example, if we want to annotate a sequence and want to use related sequences for this annotation, we are not really interested in alignments but in the annotation. It is just that we need alignments to get the annotation.

Or do we? Couldn’t we just integrate / sum over all alignments, with some suitable prior, and get rid of the nuisance parameter that way?

At least in theory, you can, and in many cases you can in practise as well. I have, myself, a paper under review doing this, but some colleagues in Oxford beat me to it (despite them starting their project long after we had the results for our paper … we have been a bit slow on this thing):

Combining statistical alignment and phylogenetic footprinting to detect regulatory elements
Rahul Satija, Lior Pachter and Jotun Hein

Bioinformatics 2008 24(10):1236-1242; doi:10.1093/bioinformatics/btn104

Abstract

Motivation: Traditional alignment-based phylogenetic footprinting approaches make predictions on the basis of a single assumed alignment. The predictions are therefore highly sensitive to alignment errors or regions of alignment uncertainty. Alternatively, statistical alignment methods provide a framework for performing phylogenetic analyses by examining a distribution of alignments.

Results: We developed a novel algorithm for predicting functional elements by combining statistical alignment and phylogenetic footprinting (SAPF). SAPF simultaneously performs both alignment and annotation by combining phylogenetic footprinting techniques with an hidden Markov model (HMM) transducer-based multiple alignment model, and can analyze sequence data from multiple sequences. We assessed SAPF’s predictive performance on two simulated datasets and three well-annotated cis-regulatory modules from newly sequenced Drosophila genomes. The results demonstrate that removing the traditional dependence on a single alignment can significantly augment the predictive performance, especially when there is uncertainty in the alignment of functional regions.

The idea, quite simply, is this: You can express the alignment problem statistically as a hidden Markov model (HMM), and you can express the footprinting problem as a hidden Markov model, and it is rather straight forward to combine two hidden Markov models into a combined model (at least as long as you do not worry too much about the probabilities and only consider the topology). Running the combined HMM, you can look at the marginal states can consider them solutions to the two original problems. Again, there are some problems with interpreting the marginal states as runs of the marginal HMMs because of the structure imposed on them by the joint HMM, but in most cases it doesn’t matter much.

In this paper, they happily just combine the alignment HMM with the footprinting HMM and thereby get footprinting annotations while summing away the alignments as a nuisance parameter.

In my opinion, this is the way to go.

There is one complication, though, and that is the running time. The way the HMM is constructed here, the number of sequences in the data is a problem. This is a general problem, regardless of whether you consider statistical or “plain old” alignment: the running time to get optimal alignments (or sum over them all) grows exponential in the number of sequences.

The way out of this is sampling methods, and there is an obvious Gibbs sampler solution. Using the forward and backward algorithm from HMM theory, you can sample the alignment of two sequences with the correct probability. So, in a multiple sequence alignment, you fix all but one sequence, you can use this trick to sample the conditional alignment of one sequence at a time. That is all you need for a Gibbs sampler.

Unfortunately, the Gibbs approach is left for future work, but I guess we will see something soon …

A year ago, we had a big meeting and discussed, all of BiRC, for an entire afternoon how to update the pages. It really goes without saying that nothing could be decided in such a large group. Instead, around Christmas, Storm, our director, called a meeting inviting those interested in updating our web-pages and willing to put in the work to implement the changes. Requiring that people should back up their ideas with actual work reduced the group significantly. We were only three, myself, Enette and Storm.

I got the task of coming up with a draft of the research pages and present it to the rest of BiRC.

What is a research project and what goes on a research center’s homepage?

The first thing I had to decide was what I wanted on the page. The form of the content is usually easier to figure out once you know what the content is.

On a webpage describing the research going on at BiRC, I wanted it to be the actual research going on at BiRC. This sounds obvious, but I have seen many research institutions listing their research interests with lists I find completely unbelievable. You might be interested in a lot of different problems, but there is a limit to how many fields you can make an active contribution to. I find it dishonest if you claim to be doing research in field where, really, all you have done is publish a paper related to the field five years ago, and you occasionally read a paper about the topic. Too often, that is what I have noticed.

There is nothing wrong with focusing on a few research areas at a research institution. Sure, it is fun to do a few projects outside your main research area from time to time, but if people read the web pages to find out what you are doing at the institution, then they should find the area where you are spending the majority of your time, not something you think about every second year. If a post doc wants to come and work at BiRC, he should know what kind of work we actually do, not what kind of work we like to read about.

Anyway, this is turning into a bit of a rant, but that is the kind of thoughts I was having at the time.

So, I wanted the page to contain our active research areas, and I wanted to back up the “we are actively working on this stuff” claim we are implicitly making when describing research on our web pages.

How do you prove that you are active? You put up the money or the work! Ideally, you want to prove that you are doing research in an area, by showing that you are actively publishing in it. If you do not publish, are you really doing research there? If you are, is it any good? I wanted each project to be able to show at least two-three papers a year to be considered active. I made an exception if there was funding for a project, but it hadn’t really produced any results yet… I am still not sure this is a good idea, but my arm was twisted a bit to allow for this…

Designing the web pages

After these considerations, I started to design the pages. I made an overview page, where people can scan through all the research areas at BiRC very quickly, and with links to sub-pages — one for each research area — that can contain more information.
On the main page I wanted a short description of each research area and a list of the most recent papers published in it (at least three, is my initial choice, but more if there are more papers published within the last year). This works as a sort of sanity check for whether we are describing an active research area or not, and also tells the world what kind of papers we publish in each area.

Putting it together

This design I presented to BiRC and no one objected. Good. Then I asked people to send me the projects (description and publication lists) they wanted on the page. Deadline March 15, which would give me time to set up the sub-pages and such, and then the new page could be put on the on May 1.

Naturally, but March 16 I hadn’t received anything and by May 1 I had a few half-baked descriptions.  Deadlines are rarely taken that serious when they are internal deadlines like this.  I am pretty bad at meeting them myself, so I cannot really complain.

With a lot of threats I got the material I needed over the weekend — well, enough to make descriptions of the main research areas at BiRC — and I put the pages together.  I’m personally maintaining more of them than I had planned.  I really only wanted to be responsible for the Association Mapping page.  This is my own main area, and the one where I consider myself the local expert.  I ended up maintaining virtually all the pages where I am contributing anything.  I plan to push the responsibility to someone else when I get the chance, but for now this is how it is.