The commentary is certainly worth reading – if you happen to have Nature access – although it will be enraging for some. As others have already noted, Goldstein uses the commentary as a platform for propounding his views on the genetic basis of common diseases. However, as someone without a dog in the rare vs common variant fight, Goldstein’s insinuations here didn’t bother me too much. What did bug me is towards the end of the piece, when Goldstein brings out a tired claim about the potential dangers of prenatal screening:

One potential problem with this is that numerous genetic risk factors will have diverse and unexpected effects — sometimes causing disease, sometimes being harmless and sometimes perhaps being associated with behaviours or characteristics that society deems positive. [...] Wholesale elimination of variants associated with disease could end up influencing unexpected traits — increasing the vulnerability of populations to infectious diseases, for instance, or depleting people’s creativity.

This argument – that removing serious disease mutations from the population will inadvertently deprive society of artistic ability, sparkling conversational wit or some other allegedly positive trait – is all too common, and tends to be followed by:

1. People nodding their heads very seriously;
2. “Take that physicist,” the head-nodders say, “you know, the one in the wheelchair. If there was prenatal screening for whatever he has back when he was conceived he would never have thought of, you know, whatever it is he’s famous for.”
3. More nodding of heads and serious expressions, as though this were a profound and troubling argument against prenatal screening for diseases.

However, for this argument to provide a serious moral case against prenatal testing, two things must be true:

1. Disease mutations must in fact be significantly correlated with some positive trait; and
2. The benefits to the world of these positive traits must outweigh the suffering and death caused by genetic disease.

The first is known to be true only in the limited setting (noted by Goldstein) of resistance to infection – for instance, the mutation that causes sickle cell anaemia also protects against malaria – but this seems likely to be an exception rather than a rule. It’s also unclear that this constitutes a moral argument against prenatal screening: the way to reduce the suffering caused by malaria is to work towards eliminating Plasmodium falciparum, not to deliberately allow a disease-causing mutation to continue to wreak havoc. And in any case, sensible screening could identify embryos at risk of sickle cell anemia (which only affects people carrying two copies of the mutation) without eliminating individuals resistant to malaria (who need only carry one copy).

Also, remember that Goldstein is talking here about very rare disease mutations, often affecting a single extended family; these aren’t like the sickle cell allele, which shows evidence for having been increased in frequency in malaria-endemic regions through positive natural selection.

The idea of a yin-yang balance of mutational effects is aesthetically pleasing in its symmetry, explaining its visceral appeal to the head-nodders mentioned above, but there’s no evidence that it represents a general principle of nature, and no compelling biological or evolutionary reason to expect gene-breaking mutations as a class to produce compensatory boosts to human traits. There are many great physicists who are not, in fact, confined to a wheelchair; there are many great artists who live without the burden of clinical depression. If screening for amyotrophic lateral sclerosis had been available in 1942 Stephen Hawking would probably never have been born – instead, another human being would have entered the world, able to choose his path in life without the suffering of a degenerative disease; and many highly successful physicists would still be working away.

But nonetheless, let’s accept that there may be some correlation between some disease mutations and some positive human traits. The question then is: should this correlation prevent us from giving parents the option of choosing not to have children who carry a severe disease mutation? Is the Jewish community wrong, for instance, to encourage pre-conception carrier testing to reduce the incidence of Tay-Sachs disease, lest it turn out that (as some have controversially argued) the disease allele is in fact associated with some positive trait?

To his credit, Goldstein doesn’t actually make this argument: he says he is “supportive of the rights of parents to choose,” even in controversial cases such as the APOE variant associated with increased risk of late-onset Alzheimer’s disease. Still, his mention of this yin-yang argument as something that may have serious merit – something that we, as a society, must ensure we discuss at length while adopting suitably concerned expressions – is unfortunate.

Within the next few years we will have the tools to allow every parent in the industrialised world to make a fully informed decision about whether they wish to have a child who will suffer from a serious, untreatable disease. There will, indeed, be serious consequences as this technology becomes widely adopted (as it will inevitably be). For instance, parents who choose not to use this technology and have a severely disabled child as a result will have to live with an extra new burden of guilt, and likely face additional social ostracism. As the number of children affected with severe genetic diseases decreases there will likely be decreased funding available for the treatment and care of disease sufferers. As Goldstein correctly notes, prospective parents will face complex choices: for those undergoing IVF, for instance, a finite number of embryos to implant will often mean trading one uncertain disease risk off against another. And there will inevitably be a drift towards embryo selection based on non-disease traits, with unpredictable (although, I suspect, manageable) consequences.

These are all perfectly valid concerns that need to be addressed. But the idea that prenatal screening will rob society of some crucial positive trait is nothing more than a distraction, and we should save our serious expressions and worried head-nodding for more concrete concerns.

This morning personal genomics company 23andMe launched an initiative designed to shift the balance in favour of participation of one non-European minority both in personal genomics and genetic research. The Roots into the Future project will recruit 10,000 African-Americans by offering volunteers free genetic testing, and full access to the results of their tests. The recruits will also be given an opportunity to participate in the same types of genetic research the company performs on its existing customers: basically, answering survey questions that can then be linked with genetic data to explore potential associations.

This is an important announcement. Personal genomics has, since its inception, been predominantly a game played by white people. An illustration: recent numbers on the ethnic breakdown of 23andMe customers indicate that of the ~81,500 customers with self-reported ancestry in the company’s database a whopping 74.7% are primarily of European descent. African-Americans are particularly poorly represented in the customer base, comprising just 1.2% (compared to 12.6% of the total US population).

That’s not to say that the personal genomics companies haven’t already made efforts to broaden the genetic diversity of their customer base. 23andMe’s founders appeared on Oprah in November 2008, and the company has made a rather determined effort to avoid too much pallid skin on its front page. Here’s a screenshot from the company’s home page in May last year (courtesy of the Wayback Machine):

Still, that 75% European number is a troubling one. A genetic revolution in healthcare is approaching – or at least that is the hope – and those who have received an education in the SNPs and haplotypes of modern genomics will be better-placed to take advantage. Right now, one of the best ways to gain insight into modern genetics is to dig deep into your own DNA; it would be a shame if such opportunities were only taken up by a select few with defective pigmentation genes.

Sadly, as was noted in a recent Nature opinion piece by Carlos Bustamante and colleagues, the dominance of European genomes is found in academic research facilities as well as the storage warehouses of personal genomics companies. A 2009 review found that just 4% of the participants in published genome-wide association studies had non-European ancestry. That’s not only a tragedy from a social perspective – it also represents missed scientific opportunities.

African-American genomics presents both challenges and opportunities. Finding the genetic basis of disease in such admixed populations is more complicated than in relatively genetically homogeneous European cohorts, requiring extra care to avoid false positives due to population structure. The greater genetic diversity found in African genomes can also be poorly captured in some places by standard genotyping arrays. However, there are positives as well: the decreased linkage disequilibrium in African genomes means that analyses in this population can zoom in on the real causal variants more effectively than in European genomes, where the signal of association tends to be spread out over much larger regions.

23andMe is rapidly (and fairly successfully) rebranding itself as primarily a research company rather than a consumer genetics provider. By leveraging its customer database of both genetic and trait information (obtained from online surveys) the company has demonstrated that it can not only robustly replicate known gene-trait associations, but also discover completely novel associations. 23andMe recently published two completely new variants associated with Parkinson’s disease, following on the heels of its 2010 publication of seven new variants associated with non-disease traits.

23andMe’s ability to generate novel genetic associations will rely on continuing to explore niches largely neglected by mainstream academia. That means focusing both on traits considered “frivolous” (such as the ability to smell asparagus in urine), and on targeting populations that have not been well-recruited by other research groups. The company’s new focus on African-Americans will complement existing campaigns targeting specific diseases (such as Parkinson’s and sarcoma).

Does this herald a wave of additional campaigns by 23andMe targeted at other under-represented minorities? Maybe, maybe not – in an email, research director Joanna Mountain told me that the company wanted to focus on African-Americans for the moment, but would “continue to review the possibility of expanding this project to include other communities.” She also confirmed that the company would be using its standard v3 chip for these analyses, rather than including additional content designed to capture African haplotypes more effectively. This may be a mistake, but not an irreversible one – the company can always go back to the DNA samples of customers (at least those who consent for their sample to be stored) for further testing. Finally, she gave a sense of the research priorities here: the company’s “initial focus is on replicating results from previous genome wide association studies conducted in other populations, but we will also look for ways to improve ancestry interpretations of the data.”

Overall this seems like a commendable initiative. It seems likely that – assuming the company achieves its recruitment goals – they will be able to contribute substantially to our understanding of disease genetics in a politically and medically important non-European population.

Today, after a year in advance online limbo without ever progressing to the print edition of the journal, and a formal Expression of Concern last November, the paper was fully retracted. There’s solid coverage of the announcement at the Boston Globe (including quotes from my Genomes Unzipped colleague Jeff Barrett), Nature, and of course the superb Retraction Watch.

Here’s the retraction notice in full:

After online publication of our Report “Genetic signatures of exceptional longevity in humans” (1), we discovered that technical errors in the Illumina 610 array and an inadequate quality control protocol introduced false-positive single-nucleotide polymorphisms (SNPs) in our findings. An independent laboratory subsequently performed stringent quality control measures, ambiguous SNPs were then removed, and resultant genotype data were validated using an independent platform. We then reanalyzed the reduced data set using the same methodology as in the published paper. We feel the main scientific findings remain supported by the available data: (i) A model consisting of multiple specific SNPs accurately differentiates between centenarians and controls; (ii) genetic profiles cluster into specific signatures; and (iii) signatures are associated with ages of onset of specific age-related diseases and subjects with the oldest ages. However, the specific details of the new analysis change substantially from those originally published online to the point of becoming a new report. Therefore, we retract the original manuscript and will pursue alternative publication of the new findings.

In a statement quoted over at Retraction Watch, the journal makes it more clear how the retraction decision was actually reached:

Sebastiani and colleagues submitted the corrected data to Science in December 2010, where the work underwent careful peer-review. Although the authors remain confident about their findings, Science has concluded on the basis of peer-review that a paper built on the corrected data would not meet the journal’s standards for genome-wide association studies. One such standard, for example, is the inclusion of a reliable replication sample that shows comparable results to those in the initial experiments.

The authors have therefore agreed to retract their paper.

In other words, the authors were still willing to stand by their results, but the journal wasn’t.

Questions remain about how the study managed to pass through peer review in the first place – virtually every complex trait geneticist I spoke to was immediately, massively skeptical about the article’s findings from the moment of publication – but it appears that Science has conducted a thorough investigation of the authors’ amended manuscript and made an appropriate decision. It will be intriguing to see if, when and in what form the study’s authors manage to republish their results.

(The image at the top shows the Manhattan plot from the original paper, with each dot in the plot representing a different genetic variant, and the alternating bands of colour showing different chromosomes. The y axis indicates the strength of the association between that SNP and longevity. As I noted in my post last year, the plot is unusual for a genome-wide association study in that all of the highest-ranked SNPs are hanging out there by themselves, rather than being flanked by a column of other associated variants – a pattern characteristic of genotyping error rather than true association.)

New sequencing technology Ion Torrent has made a splash with a paper in today’s issue of Nature. There’s no question the high-impact publication is a massive boost for the young platform, now nestled within the embrace of the giant Life Technologies (who acquired the startup for a surprisingly large price last August) and bracing for the impending launch of its most serious competitor, Illumina’s MiSeq.

The paper jumps the new platform through the standard hoops: some basic kicking-the-wheels, a test run on three bacterial genomes (Vibrio fisheri, Escherichia coli, and Rhodopseudomanas palustris), and then the traditional main event: the sequencing of a complete human genome. The genome in question is that of Intel co-founder Gordon Moore, the eponymous originator of Moore’s Law. There’s some pleasing symmetry here: Moore’s Law is frequently cited in the context of the massive decline in the costs of DNA sequencing; in addition, the Ion Torrent technology is based on the same kind of semiconductor technology pioneered by Moore. Refreshingly, the paper refers to Moore by name, which is a pleasant change from the rather affected pseudo-anonymity of other published genomes (e.g. Patient Zero).

Anyway I’m not going to comment at all here on the technical and bacterial work, which I have no doubt will be covered in detail by my esteemed colleagues Keith Robison and Nick Loman. My main interest in this paper is what it tells us about the ability of Ion Torrent as a potential platform for large-scale sequencing of human genomes, and a rival to current sequencing market leader Illumina. I also want to spend some time berating the authors of the paper for a thoroughly misleading piece of statistical sleight-of-hand that makes their accuracy numbers sound far better than they actually are.

What did they do?

The company sequenced Moore’s genome using their technology to an average coverage of 10.6x. This just means that on average each base in the genome was covered by 10.6 separate Ion Torrent reads, albeit with substantial variation: some bases had lots more reads, and some had fewer. You can see the distribution of read counts per base (in red), compared with the ideal distribution (a Poisson distribution, in green) in Figure 4b of the paper – I’ve copied a thumbnail to the right. It’s clear that there are plenty of positions in the genome with substantially less than 10 reads.

Let’s be very clear about this up front: by modern standards, this is a poor-quality genome. An average coverage of 10x means that most positions in the genome will be covered by at least one read – 99.21%, in this case – but in many of those locations, the number of reads will be too low to have any chance of accurately calling a heterozygous SNP (a base change where both different versions are present, one on the maternal and one on the paternal chromosome). This isn’t a function of the raw data quality – it’s simply a statistical consequence of sampling error at small sample sizes, that can only be overcome by additional sequencing.

It’s also an extremely expensive genome: even at this low coverage the sequencing burned through around 1,000 Ion Torrent chips, and in an NY Times piece yesterday sequencing guru George Church estimated the total cost of this project at around $2 million. That would be substantially lower at today’s prices, but still north of $200,000 for a poor-quality genome compared to less than $5,000 for a high-quality sequence from Complete Genomics. The yield of the Ion platform (in terms of bases per dollar) is of course going up rapidly, but I think it’s important to emphasise that Ion Torrent is not yet a remotely competitive technology for affordable whole human genome sequencing.

So how accurate is the genome sequence, really?

The authors attempted to explicitly estimate their error rate by sequencing Moore’s genome a second time using an independent technology: in this case, Life Technologies’ SOLiD platform, to a total coverage of around 15x. (The higher depth of the SOLiD sequencing understates the far higher yield from that platform compared to Ion Torrent; for this paper the authors ran over 1,000 chips on the Ion Torrent, whereas the SOLiD coverage was presumably achieved in a single run.) 15x coverage isn’t much better than 10x, so the SOLiD sequence would be expected to be missing plenty of heterozygous sites as well.

So, the authors have two separate low-coverage genomes, both of which would be expected to be missing plenty of SNPs – that means we would expect to see plenty of sites that differ between the two sequences (reflecting changes that by chance were detected by one platform but missed by the other). Yet the paper appears to cite a “validation rate” for the SNPs called by the Ion Torrent that is implausibly high:

To confirm the accuracy of our analysis, we also sequenced the G. Moore genome using ABI SOLiD Sequencing⁴³ to 15-fold coverage and validated 99.95% of the heterozygous and 99.97% of the homozygous genotypes (Supplementary Tables 1 and 2). [my emphasis]

There’s absolutely no conceivable way that a comparison between a 10x genome sequence and a 15x genome sequence could possibly result in a “validation rate” of 99.95% for heterozygous sites, at least not for any reasonable definition of the term “validation rate”. It takes some digging in the supplementary data to figure out what’s going on here. This is the definition of the term in the legend of Table S2, where the metric is referred to as the “percent same genotype”:

In cases where both datasets call the same type of SNP (heterozygote or homozygous variant) the proportion for which the genotype call is the same

The only way I can parse that sensibly is as follows: for sites that are called as heterozygous in both the Ion Torrent and SOLiD data, the “validation rate” is the proportion where the same two alleles are present. In other words, non-validated sites would only be sites where both platforms called a heterozyous SNP, but one platform said it was an A/G SNP while the other said it was an A/C SNP.

This is a near-useless metric, and does not correspond to any meaningful definition of the term “validation rate”. It gives us no information about what we actually want to know about, the proportion of sites where a SNP is called by one platform but not by the other – those are simply excluded from the comparison entirely. This is simply a measure of the platform’s ability to call the correct non-reference base at sites that are genuinely polymorphic, something that would be extremely high for virtually any half-decent sequencing technology. The only useful thing this metric does is provide a percentage with lots of convincing nines in it, which I’m sure the investors love, but I’m seriously perplexed that it managed to sneak past the manuscript reviewers.

Let’s take a more sensible definition of the term “validated”: for instance, let’s say it’s the proportion of sites called as heterozygous by Ion Torrent that also show some evidence of variation in SOLiD (we’ll generously say that the variant can be either homozygous or heterozygous in the SOLiD calls). Using this more plausible definition, the validation rate for Ion Torrent SNPs is just 88.0% at homozygous sites and 84.4% at heterozygous sites.

Ion Torrent could no doubt argue that this calculation is unfair to them: in many (probably most) cases, a discrepancy between Ion Torrent and SOLiD will be due to SNPs that were missed by the SOLiD technology, and thus aren’t really errors made by Ion Torrent. This is absolutely true, and in response I say: so do a proper job of validating your variants. Being a part of Life Technologies, one might imagine, should give the chaps at Ion Torrent a decent amount of access to SOLiD machines, and one more run of Moore’s genome on a SOLiD 4 would have given a far cleaner genome sequence for comparison. LIFE might even have one or two of those old capillary sequencing machines around that they used to sell: just 100-200 targeted capillary reactions around sites discrepant between the Ion Torrent sequence and a high-quality SOLiD sequence would have given plenty of data for an accurate estimation of the platforms real false positive and false negative rates.

Lack of proper validation is even more of an issue for larger structural variants. Here the authors steer clear of attempting to discover new variants, focusing instead on figuring out whether Moore carries any of the known structural variants called by the 1000 Genomes pilot project (PDF). Of 7,565 large deletions and inversions found by 1000 Genomes, the authors find evidence for 3,413 of them in Moore’s genome. That seems like a surprisingly large proportion to me, and it’s unclear how many of these calls are real: the authors report the results of a simulation using random genomic regions to estimate that 99.94% of their called events are real, but this number is not particularly meaningful as true deletion breakpoints are not well-represented by random chunks of the genome. And here there is absolutely no experimental evidence brought to bear – for instance, as far as I can tell no attempt was made to see how many of these apparent deletions also showed support in the SOLiD data, and certainly no attempt to independently validate the variants using a simple PCR assay.

All in all, a disappointing showing. This clearly isn’t a great genome sequence – it simply can’t be at 10x coverage, no matter how good the raw accuracy is – but the authors haven’t done enough experimental work to get a good sense of how accurate it really is. That means there’s very little we can say about the utility of Ion Torrent for whole human genome sequencing, apart from the fact that it’s currently too expensive to be practical.

What does Moore’s genome tell us about him?

Not much. The authors make a fairly cursory attempt at genome interpretation, pulling annotations from 23andMe‘s database and OMIM, but their results aren’t particularly useful. That’s not a criticism, by the way: the point of this paper was demonstrating a sequencing technology, not a functional annotation pipeline. (Incidentally, 23andMe’s database was apparently used without any formal collaboration with the company, suggesting the researchers simply scraped the information from the company’s website: it’s intriguing to see one of the companies attacked by the FDA and Congress as “snake oil” being used as the go-to source for functional annotation.)

However, I note that the indefatigable Mike Cariaso has already run Moore’s genome through his interpretation pipeline Promethease – you can get the results here. It appears Moore has an increased risk of baldness (check), altered responses to various drugs, and a potentially highly elevated risk of age-related macular degeneration. However, nothing that he couldn’t have learnt from a 23andMe test, at less than 0.1% of the cost.

Where to next for Ion Torrent genomes?

This has been a pretty negative post, because I’ve focused solely on a section of the paper that – I’ll be frank – was done pretty badly. It’s not intended to be a critique of the Ion Torrent technology as a whole, and I’ll leave an evaluation of the technical merits of the platform to others who know it far better than I.

Still, I can’t help but wonder if Torrent made a mistake in including a human genome in this paper at all. I mean, I know it’s traditional, and sequencing Moore makes for some easy headlines, but the Torrent platform simply isn’t currently suited to whole-genome sequencing and won’t be until its yield improves substantially (there are clear signs in the paper that this is happening, albeit perhaps a little slower than we were promised). In sequencing a human genome with this early-stage, low-yield technology, Ion Torrent was forced into a dilemma of its own making: either spend an obscene amount of money to generate a high-quality sequence, or spend a simply lewd amount of cash to generate a crappy sequence. In the end they opted for the second approach, and I suspect they would have been better off simply leaving Moore’s genome out of the paper entirely.

In any case, I should emphasise that given the slow pace of publishing, this is a genome that was put together using the technology of maybe 12 months ago. There’s no question that Torrent technology has been improving over that time, and while it’s still not at the stage of competing with Illumina on cost right now, it’s certainly possible that this will be more viable in 12 months’ time. Hopefully the next genome sequence published using this technology comes complete with sufficient validation data to get a real impression of its quality.

Top image: The Ion Semiconductor Sequencing Chip. (Ion Torrent)

Typical consumer response to seeing their own genome sequence.

How valid are the criticisms? It’s hard to say; this is a press release about two promised conference presentations, and in neither case (as far as I can tell) has the actual research been published. That hasn’t stopped the release being trumpeted by media as a blow against the personal genomics industry: the Daily Mail is predictably hyperbolic, leading with “Call to ban mail-order DNA disease testing kits that claims to predict life-threatening illnesses”; the Guardian runs with the only marginally better “Genetics tests flawed and inaccurate, say Dutch scientists”. In the absence of the details of the analysis (I’ve emailed one of the quoted scientists to see if I can get hold of this) there’s not much we can do except sift through the press release – a depressing way to discuss science.

So, what’s the actual story here? Firstly, a presentation by Erasmus University’s Cecille Janssens will discuss alleged problems with the risk prediction algorithms used by personal genomics companies 23andMe and deCODEme. The researchers “simulated genotype data for 100,000 individuals based on established genotype frequencies and then used the formulas and risk data provided by the companies to obtain predicted risks for eight common multi-factorial diseases”. Here are the key claims:

“So individuals in the increased risk group may have a four-fold increased risk of disease, but they are still far more likely not to develop the disease at all. For T2D [type 2 diabetes], where the companies calculated the average risk at around 25%, 32% of those assigned to the increased risk group would actually develop T2D compared to 22% in rest of the study population. This difference in disease risk is too small to be of relevance”, said [Erasmus University] Professor [Cecille] Janssens.

Right – so current personal genomic profiles offer a pretty small but non-zero benefit in terms of risk prediction for common, complex diseases, with the worst outcome being for weakly heritable diseases like type 2 diabetes. No surprises there; the companies typically express their predictions in ways that communicate the magnitude of increased risk fairly effectively. It’s unclear from the press release how the companies should be conveying risk differently.

“deCODEme predicted risks higher than 100% for five out of the eight diseases”, Ms Kalf will say. “This in itself should be enough to raise considerable concern about the accuracy of these predictions – a risk can never be higher than 100%. In the case of AMD one in every 200 individuals in the group would have received a predicted risk of higher than 100%, suggesting that they would definitely develop the disease.”

That is indeed strange. In the Guardian piece, deCODE Genetics CEO Kari Stafansson appears to argue that the team has misused the company’s risk prediction algorithm:

“We never report a lifetime risk over 90%. This is not how we use these models.”

Either the team got the models wrong, or deCODEme has failed to adequately communicate exactly which algorithm they use, or there’s something fundamentally wrong with deCODEme’s risk prediction. Without more details from the authors it’s hard to know which of these is true, although given deCODE’s impressive academic pedigree in risk prediction I’m putting a pretty low prior on that final possibility.

“They only take genetic factors into account when predicting risks for consumers, whereas in most multi-factorial diseases other modifiable risk factors, such as diet, environment, exercise and smoking have a much stronger impact on disease risk”, said Professor Janssens.

This is a non-issue. Companies can’t include environmental factors in their risk prediction models because the interactions between genetic and environmental factors is unknown. In general, the companies do an appropriate job of informing consumers of the fact that variables other than genetics affect risk (23andMe, for instance, indicates the fraction of risk explained by genetics vs environment directly underneath each of its detailed disease risk predictions). Again, it’s unclear how the companies should do a better job here.

The second presentation described by the press release is substantially more annoying. This time it’s about attitudes towards personal genomics among clinicians, based on “a survey of a representative sample of clinical geneticists from 28 countries across Europe on their experience of and attitudes to DTC genetic testing”. In other words, the researchers asked a group of people who see themselves as the gatekeepers to genetic information whether someone else should be allowed to provide that information. The results are predictable:

“Clinical geneticists’ concerns with DTC genetic tests are mostly related to the fact that these tests usually lack clinical validity and utility. Moreover, these tests are usually carried out without the provision of genetic counselling. According to the experiences of clinical geneticists, patients often do not know how to interpret the results they receive and are often confused by them. However, almost all clinical geneticists feel that they have a duty to provide counselling if patients contact them after having purchased a DTC genetic test,” says Dr. Howard.

So clinical geneticists feel that poor ignorant consumers can’t properly interpret the results of whole genome scans. Whether that actually translates into genuine difficulty in interpretation, of course, is quite a different matter. On this topic I’ll see your press release and raise you this one from late last year, which reported on a survey of over 1,000 personal genomics customers presented at the American Society of Human Genetics 2010 meeting (and not yet published); those researchers asked respondents to examine actual risk reports from personal genomics companies, and found that around 95% of them interpreted the results correctly.

The release continues:

“A person who undergoes a genetic test has to be accompanied – explanations, physical aid, the right to choose whether to know or not – and this is not true in the case of direct access to such a test”, said one survey respondent.

“Genome-wide scans by companies are totally unacceptable, as they can derive sensitive information about medically relevant conditions and will also provide lots of information which is difficult to interpret, even for medical professionals”, said another respondent. Presenting the results of such tests directly to individuals is unacceptable, the majority of those surveyed said.

You read that correctly – this information is difficult to interpret “even for medical professionals”, yet clearly the answer here is to force everyone to access their own genetic information through those same confused professionals. This argument is just as ludicrous here as it was when the American Medical Association made it earlier this year.

Bear in mind that the clinicians interviewed here – clinical geneticists – would not be the doctors that most consumers of personal genomics products would be forced to consult with before they can look at their own DNA. Instead, we would be legislatively required to sift through our data in the presence of general practitioners with (typically) zero formal training in modern genomics or genetic disease risk prediction. This is an absurd outcome; yet the press release argues explicitly for this approach:

Currently only a few European countries, for example France and Switzerland, have legislation that states that genetic tests can only be accessed via individual medical supervision. “Although this model is sometimes criticised for being too paternalistic”, says Professor Borry, “in the absence of a good working pre-market control of genetic tests, it could be a useful way of responding to some of the concerns over DTC testing.”

So, take-home message here: clinicians believe that people should be forced to pay clinicians in order to access their own genetic information. What a surprise!

While it is important that personal genomics companies consult with medical professionals in devising their risk prediction algorithms and interfaces, regulation of genetic testing should not be based on the views of the traditional gatekeepers of genetic information. It should also not be based on unsupported ideas about the ability of consumers to interpret risk predictions, or on paternalistic views regarding the potential dangers of learning more about our own genomes without a clinician to hold our hands. And please, forcing consumers to access their results in the presence of someone who knows less than they do about modern genetics is not a viable working solution to the impending challenges of genomic medicine.

In the complete absence of any evidence for harm to consumers caused by the personal genomics industry, the appropriate working solution is simple: enforce existing regulations to control technical accuracy and punish false claims, continue working to push companies to provide their customers with access to all of the information required to make a sound decision, and continue to work to generate evidence about the best approaches for generating and communicating risk information to the public.

See also: outraged comments by Razib and Trey. Here’s an excellent quote from Razib:

I think one of the key issues is that these genetic professionals view DTC genomics in the same category as a pharmaceutical. I view DTC genomics as part of the same family of consumer and social goods as information technology. When viewed in the context of our current medical-industrial infrastructure it seems that on the margin the future opportunities to reduce morbidity through better lifestyle choices and more information via DTC genomics is a no brainer. If there was social science evidence that people who receive false positive results are regularly committing suicide then obviously my preference for loose regulation would need re-examination. That would be like someone jumping off the tenth floor of an office building after a “blue screen of death.”

But from what I’ve seen at places likes Genomes Unzipped and FuturePundit these dystopian visions of mass hysteria don’t end up to panning out. Until that point it seems that the best avenue toward improvement of this technology is to allow the trial and error process of innovation to continue.

Here’s my submission in full, with some links added for context:

To:  Molecular and Clinical Genetics Panel of the Medical Devices Advisory Committee

Docket:  FDA-2011-N-0066

Submitter:  Daniel MacArthur, Wellcome Trust Sanger Institute

The views presented here are my own, and do not represent those of my employer.

Introduction
These comments are being submitted to the Committee for consideration as part of their deliberations and recommendations to the Center for Devices and Radiological Health concerning direct-to-consumer (DTC) genetic tests.

I write this from two different perspectives: firstly, as a working geneticist, and secondly as a consumer of personal genomics products. In my professional role I am a researcher at the largest genomics research facility in Europe, where I examine methods for extracting useful functional information from human genome sequence data, working towards analytical approaches that can be used in clinical practice. I am also a long-term observer of the personal genomics industry and a customer of four different genetic testing companies.

From both perspectives I am deeply concerned about fostering innovation in the field of genetic testing and its integration into medicine. As a consumer, I am also concerned with ensuring that I and others continue to have direct access to accurate information about our own genomes.

The dangers of unnecessary regulation of the genetic testing industry
Over the next decade the integration of genomic technologies with healthcare will permit the development of a new, predictive, personalised model of medicine. However, numerous technical and economic challenges will need to be solved in order to take full advantage of this emerging paradigm. It is thus crucial that regulatory obstacles to innovation in the young field of genomic medicine are kept to the bare minimum, while still protecting consumers.

The DTC genetic testing industry is diverse, with a range of target markets, technologies, and marketing practices. While some companies in the field have engaged in deceptive marketing (and should be punished for doing so under existing consumer protection laws, as noted below), the most popular companies should be commended for their responsible delivery of complex genetic information to customers. The innovation driven by this industry will serve the field of genomic medicine well: companies operating in this space have developed intuitive interfaces for genetic risk information better than anything currently provided to patients in mainstream clinical practice, and one company (23andMe) has created a novel paradigm for participant-driven research into the genetics of complex traits that has already resulted in a peer-reviewed publication[1]. As the technology and science of genetic testing advances, encouraging the proliferation of companies willing to (responsibly) test various approaches to analysing and delivering genetic results would pay dividends for mainstream genomic medicine.

Critics of DTC genomics frequently cite consumer safety as a key concern justifying stringent regulation of the industry. In fact the studies performed so far on genetic test consumers, while still limited in scope, provide no evidence to support a strong protectionist stance: recipients of DTC genetic information seem to respond sensibly, showing little sign of unwarranted anxiety about their results[2]. Further research is needed in this area, but certainly there is currently no compelling justification for restricting access to these tests from a public health perspective.

Regulatory mechanisms already exist to govern the accuracy of the raw data provided by these companies as well as to punish false advertising and unethical marketing (such as the Clinical Laboratory Improvement Amendments and the Federal Trade Commission, respectively). These mechanisms should be strengthened where necessary and applied uniformly across the industry. In addition, it is crucial that consumers have access to accurate, transparent information about the tests provided; this would be facilitated by the creation of a registry of genetic tests that included accessible descriptions of the markers examined in each test and the level of evidence supporting the claims made. The National Institutes of Health’s proposed Genetic Test Registry (GTR) would be a natural location for such a registry. However, the GTR is currently planned as a voluntary database; either mechanisms should be put in place to make enrolment in the GTR mandatory for genetic testing companies, or substantial efforts should be made to establish the GTR as a reputable, intuitive source of information for consumers, thus encouraging participation from providers.

Enforcing these three basic principles (accuracy of raw data, responsible marketing, and transparency) would punish irresponsible companies without unfairly affecting companies providing accurate information to consumers. Imposing further obstacles on the DTC genetics industry will inhibit innovation in a crucial field, both by increasing the cost of introducing new technologies as well as discouraging new companies from entering the market. Such restrictions would also unnecessarily restrict the access of the public to information about their own genomes.

Some stakeholders, such as the American Medical Association (AMA), have called for health-related genetic interpretation to be provided only under the supervision of a medical professional. To mandate this would be a grave mistake. As the AMA itself admits in its submission to this Committee (FDA-2011-N-0066-0006.1), physicians are currently unprepared to provide interpretation of sophisticated genetic data to large numbers of patients. Forcing consumers to obtain their results through gate-keepers who are self-admittedly unqualified to sensibly interpret those results would represent another unjustified restriction of consumer access to genetic information, and almost certainly decrease the quality of the provided interpretation. DTC genetic testing companies should be encouraged to work with clinicians in developing their interpretations (as the responsible members of the industry already do), but clinicians should not be the mandated point of access for all health-related genetic test results.

It is worth noting that the nature of DTC genetic testing make it an international industry; both samples and electronic information can be shipped easily between jurisdictions. As such, excessive regulation of US companies offering this service will simply encourage the establishment of similar providers operating outside US regulatory control, which could nonetheless be accessed (legally or illegally) by US residents. Harsh regulations that pushed this industry off-shore would destroy the opportunity for the US to build a vibrant, responsible source of innovation in genetic testing on its own soil.

Summary
We are currently on the brink of a transformation of the practice of medicine. If provided with light-touch regulation to govern laboratory accuracy, discourage unethical marketing and encourage transparency, and otherwise allowed to develop freely, the DTC genetic testing industry will play an important role in developing and testing innovation that will help to fuel this transformation. Regulations that step beyond this careful approach run the risk of inhibiting both innovation and consumer access to their own genetic information. I thus urge the Committee to carefully consider the cost of restrictions on the DTC genetics industry – both in terms of innovation and infringing on consumer autonomy – in making its recommendations.

Sincerely,

Daniel MacArthur

Research Fellow, Wellcome Trust Sanger Institute, Hinxton, UK
Blogger, Genetic Future (www.wired.com/wiredscience/geneticfuture) and Genomes Unzipped (www.genomesunzipped.org)

[1] Eriksson, N., et al. Web-based, participant-driven studies yield novel genetic associations for common traits. PLoS Genetics 6: e1000993.

[2] Bloss, C.S., et al. 2011. Effect of direct-to-consumer genomewide profiling to assess disease risk. New England Journal of Medicine 364: 524-534.

While the shape of the FDA’s planned regulation is still very much unclear – if indeed there are any genuine plans beyond public grand-standing – the tone of the agency’s two-day public meeting in March (best summarised by Dan Vorhaus) gave us a hint: there’s a very real chance that the agency will attempt to force consumers to access “health-relevant” genetic interpretation through a clinician.

Such a plan would have consequences that go far beyond the stifling of the personal genomics industry: excessive regulation, including the locking away of all health-related genetic interpretation behind a medical establishment that admits it is unprepared to deal with it, would raise the prices of genetic test access, create unnecessary obstacles to people with an intellectual interest in their genetic information, reduce the educational impact of personal genomics, and – most importantly – hinder innovation in the crucial field of genome interpretation. Such an outcome would be a particularly tragic mistake given that there is currently no evidence to suggest that receiving genetic risk information poses risks to consumers.

But most readers know my views. The main purpose of this post is to encourage readers to make their own opinions known to the FDA. Thanks to Dan Vorhaus, the agency reopened its period of public comment at the end of March. That means there’s an opportunity here for those of us who believe in the importance of direct access to our own genetic information to directly tell the FDA what we think of plans to inhibit this access: but you only have until Monday to submit your comments.

Given the tight deadline, the best way to submit is online: go to http://www.regulations.gov/ and reference docket ID FDA-2011-N-0066. (Edit: Razib suggests this direct link to the submissions page.)

I’ve been far too quiet on this issue here over the last month due to various other commitments, but plan to finalise my submission over the weekend. If you’re seeking inspiration for your own submission, the posts by Jennifer Wagner and Charles Warden on their submissions are a good place to start. Over the last month, Razib Khan has done an excellent job of waving the genomic access flag, as well as highlighting the non-trivial benefits of large-scale personal genomics for DIY ancestry research. Personal genomics companies have also come out fighting: 23andMe has encouraged the public to make their views heard at the FDA, and Pathway Genomics has engaged in some high-level lobbying. The issue has also received coverage in the mainstream press, most recently in a Wall Street Journal op-ed by author Matt Ridley.

Note that (with some notable exceptions) no-one is arguing that the personal genomics industry should be entirely free of regulation; rather, we have consistently argued that what is needed is light-touch regulation to ensure the accuracy of raw results, to punish false claims and unethical advertising, and to promote full transparency from companies to allow consumers to make an informed choice. Heavy-handed action from the FDA will do more harm than good; but unless the FDA knows that there’s sufficient public support for personal genomics, they may well take that path – so let’s do our best to ensure there’s no doubt about that support.

Reid’s first post is certainly worth reading: it provides some insight into the extent to which Complete has dedicated itself to the single-minded focus of creating a streamlined, automated production facility for churning out genome sequences. Reid explains the basic thinking behind this strategy:

The sequencing industry is now undergoing a big change with the emergence of the first standard sequencing project, the human genome. Instead of sequencing different organisms or parts of organisms, researchers (and an emerging group of clinicians) need to sequence thousands of complete human genomes — exactly the same organism and bases — over and over again. The inputs and outputs are standardized, and the sequencing process is identical every time. These projects don’t need the flexibility of the past; they need extremely high reproducibility and high accuracy at extremely low cost.

These standardized, repetitive human genome sequencing projects can be done using a flexible sequencing technology, but there is a better method: the automated factory approach. Developing dedicated technology and a specialized work flow — inflexible, but optimized for human genome sequencing — results in higher quality due to better reproducibility through automation and lower costs due to economies of scale.

This approach seems to be paying off so far: Complete announced last week that it had shipped 600 complete genomes to customers in the first quarter of 2011, and investment analysts are responding very positively to the company’s progress. The company currently claims to have captured around 40% of the whole genome sequencing market; if it can maintain that position as the demand for clinical sequencing explodes over the next few years it will be doing very well indeed.

From my selfish perspective as someone interested in personal genomics, the advantage here is competition: the better Complete does, the tighter the arms race between it and primary competitor Illumina will become. That means increasingly lower prices for genome sequencing, which in turn means the date at which I can afford to get my own genome sequenced approaches at an ever-accelerating pace.

The APOE results are locked – meaning that before being able to view them, customers need to read a brief information page and click a button indicating they understand the implications. Here’s a screenshot:

Once you’ve agreed that you understand, your results appear. Mine are reassuring – I carry two “normal” (ε3) copies of APOE. (I actually already knew this with fairly high confidence thanks to some analysis done by Luke Jostins a few months ago, which we’ll be writing about over at Genomes Unzipped shortly.)

The up-front warning and detailed information provided by the company all seems perfectly reasonable to me: intuitive and thorough, without being patronising. You can view the full information provided to 23andMe consumers here.

Customers who bought their tests before the v3 launch will need to upgrade to the new chip to be able to access their APOE results. The previous versions of the chip apparently didn’t include one of the key markers (rs429358) because this is difficult to call accurately using the Illumina platform – I’m not sure what has been tweaked to get this marker working on the v3 chip, but I can only assume the company is very confident in its results. A bad APOE prediction would be, to put it mildly, a potential PR disaster for personal genomics (as deCODEme learnt in December 2009).

[Edited to fix inflated risk estimate for ε4/ε4 homozygotes.]

Rare diseases matter

There are thousands of rare genetic diseases, ranging from the widely-known (such as Huntington’s disease, an adult-onset brain disorder) to the obscure (such as fibrodysplasia ossificans progressiva, a disease affecting less than one in a million people, in which the patient’s muscles are slowly replaced with bone). While individually rare, these diseases collectively create a tremendous burden of suffering: childhood-onset single-gene disorders affect nearly four children in every thousand live births, and are responsible for more than 10% of paediatric hospital admissions.

Previously, finding the mutations that cause these rare diseases was a lengthy process, relying on a technique called linkage analysis. Firstly, DNA samples were collected from large families affected by the disease. Secondly, these samples were examined at thousands of highly variable sites across the genome, to look for markers that were always found in patients but not in their healthy family members. Finally, researchers needed to comb through the dozens or hundreds of genes close to these ‘linked’ markers to look for mutations that might be disease-causing.

This whole process takes time, money, and more than a little luck. In addition, for certain classes of genetic disease – those that have only been found in small families, or are caused by genetic changes that arise spontaneously in patients rather than being inherited from either parent – linkage analysis is impossible. That means that while this technique has successfully deciphered the genetic basis of thousands of rare genetic diseases, many remain unexplained.

Finding the underlying mutations for these diseases is of far more than just academic interest. For patients and their families a full genetic diagnosis can provide a sense of closure after years or decades of enduring medical tests with no clear answers. In some cases identifying the underlying gene can provide important clues about the underlying mechanism of the disease, and perhaps even point to potential therapies. However, we also shouldn’t underestimate the raw scientific value of these studies: every gene we link to a rare Mendelian disease increases our understanding of the ways genes work together to build a human being.

Towards a solution

Over the last few years, rapid advances in DNA sequencing technology have begun to provide a cost-effective alternative to linkage analysis. Rather than first looking for the regions of the genome that are linked to the disease, cheap sequencing offers a simple, brute force solution: look at all of the genes in a patient’s genome, see which ones contain a likely damaging mutation, and then investigate those genes to see which is most likely to cause the patient’s disease.

New sequencing technologies have resulted in a dramatic drop in the cost of reading a person’s DNA. However, it’s still expensive to sequence their entire genome – all six billion letters of it will currently set you back somewhere in the order of US$20,000 (£12,200). Fortunately, most rare diseases (around 80%, by some estimates) are caused by mutations found in a relatively small fraction of the genome: the pieces that code for proteins, known collectively as the exome.

These pieces of protein-coding sequence are scattered across the genome, but only make up less than 2% of its total length. Using an approach called sequence capture – in which tiny DNA probes are used to pull out the protein-coding regions in a patient’s DNA, letting the remainder wash away – it is possible to extract and sequence only these regions. The small size of an exome means it can be sequenced from a patient for just a few thousand pounds – in many cases, substantially less than the cost of a series of single-gene tests.

Over the last two years exome sequencing has been applied to hundreds of patients suffering from undiagnosed genetic diseases. The first public success story, reported in Nature in August 2009, showed that exome sequencing in a four-member family could be used to re-discover a previously known mutation causing a disease called Freeman-Sheldon syndrome. Later that year the same group used the technique to pin down a previously unknown mutation that caused the rare developmental disease Miller syndrome. Since then the technique has been responsible for a string of successful discoveries: diseases such as Kabuki syndrome, Fowler syndrome and Schinzel-Giedeon syndrome, for instance, have all been pinned down.

In some cases the information revealed by exome sequencing resulted in crucial changes to a patient’s clinical care: in one example, summarised by Luke Jostins at Genomes Unzipped, exome sequencing of a young boy with severe bowel inflammation revealed a defect in an important immune gene, suggesting that the boy’s condition could be treated with a bone marrow transplant. Within six weeks of the operation the patient was able to eat solid food, and five months afterwards the disease had not recurred.

However, such successes have not come without challenges. Identifying the mutations that cause each disease has been complicated by the fact that all of us carry many apparently ‘broken genes’ that don’t actually cause disease; filtering these out has often required looking for mutations found in multiple patients and not seen in their healthy family members. In addition, exome sequencing is expected to miss a fraction of disease-causing mutations: for instance, all of those that fall outside protein-coding regions, or that are present in regions that aren’t well-captured by current technologies.

What fraction of diseases will exome sequencing solve?

The string of success stories in high-profile journals is promising, but hasn’t enabled us to judge what fraction of diseases the technique simply doesn’t work in (in most cases, failures don’t make it into the academic literature). However, at a recent meeting I attended in Hinxton, UK, Dutch geneticist Han Brunner provided some hard numbers based on his group’s analysis of over 200 patient exomes representing 30 different diseases: of these 30 diseases, 15 resulted in the discovery of a novel disease-causing gene, 5 turned out to be caused by mutations in previously discovered genes, and the remaining 10 are yet to give up their secrets.

There are some caveats to bear in mind here. Firstly, Brunner did note at the meeting that these 30 diseases were the ones where he regarded the analysis as “completed” – and diseases where the exome approach has been more difficult are more likely to still be sitting in the “uncompleted” stack. Secondly, it seems likely that the first wave of exome sequencing targets will represent the lower-hanging fruit: diseases where there is a clear phenotype shared between multiple patients, and with larger families available for analysis. As we venture down into the more complex cases, where there are only a few patients available or where the disease definition is messier (making it more likely that several different genes will contain causal mutations) the success rate will inevitably take a hit.

There are some obvious reasons why exome sequencing can fail. In some cases the disease-causing mutation won’t be in the sequenced regions, either because it is not protein-coding, because it’s in a region that’s badly captured by current technologies, or because it’s simply been left of the exome capture chip. Brunner provided a cautionary example of the latter situation: their attempts to find the mutation underlying Kabuki syndrome were unsuccessful because – as it turns out – the underlying gene MLL2 wasn’t present on their early capture arrays (it was present on other arrays, resulting in a Nature Genetics paper for another group). These cases will shrink as both capture and sequencing technologies improve, and as capture arrays begin to include more genes as well as functional non-protein-coding elements.

Diseases with more complex genetic causes will also remain problematic. If mutations in multiple genes cause the same disease, either more sophisticated statistical approaches (and more patients) will be required to dissect them, or clinicians will need to come up with ways of teasing apart distinct syndromes that show very similar clinical symptoms. For many diseases it will no doubt prove impossible to arrive at a single “smoking gun” gene – but at the very least, the exome approach should provide a set of candidate genes that can be tackled by clinical and functional studies.

So, even with these caveats, Brunner’s numbers suggest that applying exome sequencing to as many rare disease patients as possible will uncover the genetic basis for a substantial fraction (hopefully the majority) of them, yielding a rich harvest of new disease genes in the process. That means that this technique will provide a long-awaited answer to many rare disease patients while also improving our understanding of the function of human genes.

As large-scale exome sequencing projects continue to scale up around the world, hundreds of rare diseases will be unravelled within the space of the next one to two years. Never before has a single technique promised to reveal so much about genetic disease in such a short space of time. For geneticists, and for rare disease patients, these are exciting times indeed.