If you wrote to the organising committee of a scientific conference saying that you have a theory that there is a person in everyone’s bum and if you massage it in the right way, you can cure many illnesses, do you think you would be invited to give a paper to the conference?

Perhaps if you were invited, you might justifiably conclude that the discipline that the conference is in is a pile of rubbish. Or, at least, that the organisation that is hosting the conference is made up of a bunch of quacks.

Well, one professor of medical education in the UK proposed exactly this as a topic for a paper to a “Complementary and Alternative Medicines” conference in Jerusalem. And, to his surprise, he was invited to give a paper.

His conclusion is very nicely expressed:

So called integrative medicine should not be used as a way of smuggling alternative practices into rational medicine by way of lowered standards of critical thinking. Failure to detect an obvious hoax is not an encouraging sign.

He recounts the story in the British Medical Journal‘s Christmas edition, which is full of stuff that’s meant to be entertaining. His article is indeed entertaining and worth a read.

Check out his original pitch in which he subtly alludes to butt-kissing (“gentle suction”):

Recently, as a result of my developmental studies on human embryos, I have discovered a new version of reflexology, which identifies a homunculus represented in the human body, over the area of the buttocks. The homunculus is inverted, such that the head is represented in the inferior position, the left buttock corresponds to the right hand side of the body, and the lateral aspect is represented medially. As with reflexology, the “map” responds to needling, as in acupuncture, and to gentle suction, such as cupping. In my studies, responses are stronger and of more therapeutic value than those of auricular or conventional reflexology. In some cases, the map can be used for diagnostic purposes.

The idea of pulling a hoax to point out the absurdity of disciplines has a great history, which this hoaxster is aware of. He (somewhat self-flatteringly) compares his little stunt to the Sokal hoax in which Alan Sokal famously revealed the absurdity of postmodernist cultural studies by submitting a ridiculous paper to a respected journal and having it published.

If you haven’t read about the Sokal hoax, do some googling. Sokal has also recently written a book about the hoax and other related things that is on my summer reading list…

[Via Cardiology Update]

McLachlan JC (2010). Integrative medicine and the point of credulity. BMJ (Clinical research ed.), 341 PMID: 21147748

Yeah — I didn’t think this was a belief that anyone held either. But apparently it’s Danish urban folklore that you can become drunk by submerging your feet in an alcoholic beverage. At least, that’s what three Danish researchers say in the British Medical Journal.

These three guys decided, in the spirit of early science, to test the putative myth on themselves.

They sat with their feet in a laundry tub full of vodka for three hours and measured their blood-alcohol levels every three hours.

Unsurprisingly, they found that there were no significant changes in blood alcohol levels and they provide the following sober conclusion.

Our results suggest that the transcutaneous intake of alcohol (vodka, 37.5% by volume) through feet is not possible. We therefore conclude that the Danish urban myth about being able to get drunk by submerging feet in strong alcoholic beverages is just that; a myth.

Christian Stevns Hansen, Louise Holmsgaard Færch, Peter Lommer Kristensen (2010). Testing the validity of the Danish urban myth that alcohol can be absorbed through feet: open labelled self experimental study The British Medical Journal : 10.1136/bmj.c6812

The ethics of conducting medical trials can be tricky. There is always the risk someone could be exposed to a drug that is dangerous, or they could miss out on something of potential benefit.

So these issues need debating in medical journals.

But doctors and medical researchers are not ethicists and this was, I think, demonstrated by an article recently in the prominent journal The New England Journal of Medicine.

So bad was an argument made by these experts, that it’s really worth thinking about for a minute.

» The issue

The debate was about whether or not it is ethical to conduct placebo controlled trials of drugs that treat osteoporosis. The trouble is that we now have very good drugs that significantly reduce the rates of fractures, and giving someone a placebo is denying them something that will do them a great deal of good.

This problem needs to be weighed against the fact that controlled trials are needed to develop newer, better drugs. So, you can see that there is an issue here that needs to be debated.

One of the central questions, is whether there are alternative ways of trialling new drugs, without using placebos. One such way is to compare new drugs to existing drugs: “head-to-head trials”.

But the practical problem with conducting head-to-head trials is that they need to be much larger in order to show statistically significant effects.

» The good argument

In one article in the journal, two doctors said that this practical concern doesn’t matter.

The challenges associated with head-to-head trials “should not be considered an ethical justification for administering placebo to some patients, which would result in potentially preventable fractures,” they said.

This seems to me to be a good argument. The underlying principle is that the alternative to a course of action being practically difficult, doesn’t make that action ethical. Just because doing head-to-head trials is difficult doesn’t mean that giving people placebos is ethically OK.

» The bad argument

But the guys arguing that placebo trials can still be ethical in osteoporosis expected this argument, and rebutted it with what I think is one of the worst ethical arguments I have come across in a medical journal.

Ultimately, any trial involving [an active] comparison group would, given its necessarily larger size, result in more fractures over the duration of the trial than would be observed in a placebo-controlled study and, thus, it would offer no ethical advantage over the latter.

It seems to me that they’re defending the principle that the primary ethical consideration is the total number of negative outcomes observed in a trial.

That’s what they’re saying, right? Read it again:

…would, given its necessarily larger size, result in more fractures over the duration of the trial than would be observed in a placebo-controlled study and, thus, it would offer no ethical advantage over the latter.

But that is an absurd principle to defend. On the basis of that principle, if we were to do a large trial where we gave patients the very best care we know of, it could be unethical because if it’s large enough, it will observe a lot of negative outcomes.

Indeed, it would be an argument against collecting data on real-world outcomes in what are known as “observational studies”. If you observe a million people, then there will be lots of negative outcomes. But that clearly doesn’t make observing them unethical.

I put this point to the lead author in an email. I received a very nice and thoughtful reply, but I don’t think any light was shed on the matter. (I’ll post the reply in the comments.)

» The relevant ethical principle

Giving a placebo to 100 people, and seeing 5 negative outcomes is worse than giving best treatment to 100,000 people and seeing a few more negative outcomes.

The relevant issue is not the number of negative outcomes, but the risk of negative outcomes being imposed on each patient. It’s about the intervention we’re making, and whether that is ethical.

So we need to make sure we’re not giving people placebos when they could benefit from an active treatment and we need to make sure that active treatments we are trialling are likely to be safe and effective. We can never be sure, but with these matters, it’s always about the risk — the probability of harm.

Stein CM, & Ray WA (2010). The ethics of placebo in studies with fracture end points in osteoporosis. The New England journal of medicine, 363 (14) PMID: 20879888

Rosen CJ, & Khosla S (2010). Placebo-controlled trials in osteoporosis–proceeding with caution. The New England journal of medicine, 363 (14) PMID: 20879887

It seems that hardly a month goes by without another shocking example of drug companies hiding, manipulating or lying about data in order to mislead consumers about the safety or effectiveness of their products.

The big cases that have garnered media coverage around the world recently were Vioxx and Avandia. But there’s constantly smaller, equally shocking stories of how pharma cherry-pick data to exaggerate how great their drugs are.

This week, the British Medical Journal (BMJ) published some amazing research showing that a reasonably common antidepressant — reboxetine (Edronax) — is actually completely ineffective and possibly harmful. And, what’s more, the only reason it appeared to be effective in earlier meta-analyses, was because Pfizer, the dug’s manufacturer, was hiding data that showed it was ineffective!

What’s more, the only reason we thought it was equally as safe as other common antidepressants (like Prozac) was that Pfizer were hiding the evidence that it was more harmful.

But, you might ask, how exaggerated were the claims of  effectiveness?

Well, the German researchers calculated this by comparing what the published data said about the effectiveness of reboxetine, with what the pooled published and unpublished data said. The published data exaggerated the effectiveness by 115%. That is, it said it was more than twice as effective than it actually is, compared to placebo. (That means twice as likely to result in remission than it really was.)

It’s bad enough that Pfizer chose not to publish data that wasn’t positive about its shiny pill. But the real kicker to the story is that when the researchers initially approached Pfizer asking them for access to the data, they refused. Only after a report was published stating that there was no reliable evidence of the drug’s effectiveness, did Pfizer agree to hand over the data.

When they turned over all the raw data, the researchers uncovered biases in the published reports resulting from cherry-picking and biases between what trials were reported, and what were not.

As two editors of the BMJ said in an accompanying editorial, this kind of behaviour cannot be allowed. Compulsory preregistration of trials goes some way to solving the problem — it means that we know when trials are conducted, and not published. But to stop the biased analyses of the data, we need the raw data made publicly available.

I can’t help being surprised by the response pharmaceutical companies make when these revelations are made. Rather than saying “Yep, it looks like the drugs aren’t effective. We’ll withdraw them immediately,” or even, “Well, let’s wait a bit longer and see what the evidence says,” they almost always reply by saying that the drugs work fine.

GSK said it after it was revealed that they had fiddled with the evidence behind rosiglitazone (Avandia), Merck said it after Voixx and Pfizer are saying it now.

Until pharmaceutical policy is changed around the world, it really is going to be a matter of another day, another drug company cover-up. They’ll just keep getting away with it.

Godlee F, & Loder E (2010). Missing clinical trial data: setting the record straight. BMJ (Clinical research ed.), 341 PMID: 20940217

Eyding, D., Lelgemann, M., Grouven, U., Harter, M., Kromp, M., Kaiser, T., Kerekes, M., Gerken, M., & Wieseler, B. (2010). Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials BMJ, 341 (oct12 1) DOI: 10.1136/bmj.c4737

Has anyone else noticed the crescendoing attack on reason waged through the news media recently? Here in Australia I think it’s been particularly bad.

Between the uncritical reporting of the “miracles” Mary MacKillop brought about allowing her to qualify as a saint, Paul the octopus predicting the outcomes of the football, and every and any animal predicting the outcome of the recent Australian federal election, it seems that quackery, mysticism and superstition have become the norm around here.

These things aren’t just harmless bits of fun. They normalise a way of seeing the world that is completely false. And dangerously false. Without critical thinking, and an understanding of the skeptical methods of science, we have no hope of solving climate change and other serious problems.

So it was a great relief this week to read in the Sydney Morning Herald a prominently placed story taking on all the bullshit surrounding the McKillop miracles. (Declaration of interest: I know the author of the story very well.)

Professor Del Mar said popular acceptance of MacKillop’s miracle was part of a wider problem of people not understanding scientific and mathematical methods, exemplified by newspapers printing horoscopes and people using alternative medicines that had little evidentiary support.

“These things betray a false thinking that is not limited to Mary MacKillop or religion,” Professor Del Mar said.

I’ve been meaning to write about all this stuff for a while but rather than saying more here, you should check out the SMH piece.

MacKillop cancer prayers ‘betray a false thinking’

So, thanks for sticking around. Although the blog hasn’t been updated for a while, I’ve been busy behind the scenes fixing some stuff up, upgrading some other stuff and redesigning the blog.

And with a new design comes some new resolutions! This blog is never going to be a super-productive one like some others out there, but I am going to try to do a few posts a week, and at least one every week.

But not just any posts… Over the next week stay tuned for something about doctors and medical researchers making terribly poor ethical arguments, another thing about whether scientific studies can be trusted, and maybe another post or two as well.

I hope you all like the new design (it’s a slightly modified Arras theme) and that you enjoy the content over the next little while.

Oh, and if anyone wants to lend me some css or php expertise, please contact me! I’m a complete newb when it comes to these things.

As Good, Bad, and Bogus blows out the lonesome candle on its birthday cake today, I thought I’d take this opportunity to reflect on what’s gone on here over the last 12 months.

But rather than rambling on about what’s happened, what I’ve done, what I haven’t done etc, I’m going to just draw up a few lists. Enjoy!

UPDATE: In a kind of slight irony, the very first post here, which was really a link to a post I wrote for the Guardian Science Blog, has been listed as a finalist for the 3QuarksDaily philosophy prize. Cross your fingers for me!

UPDATE: Actually, it was only a semi-finalist… And it didn’t win. Oh well!

Most popular posts

1. Mythbusting booze: Hair of the dog – hangover cure or excuse for alcoholism?
2. Actually, maybe economists did prove money can buy happiness…
3. BOGUS: Today Tonight on Fish Oil, Weight Loss and Holograms
4. Mythbusting Booze: Myths and realities of alcohol consumption
5. ‘Gravity doesn’t exist’, says philosophically naive scientist/journalist

My favourite posts

1. Why you REALLY can’t trust small studies: the small study effect
2. The problem with drug trials
3. ‘Gravity doesn’t exist’, says philosophically naive scientist/journalist
4. Climate change and philosophy of science: Does climate science aim at truth?
5. Economists (don’t) prove that money can buy you happiness… And news outlets prove they’re crap.

My favourite Good, Bad, and Bogus-style pieces I’ve published elsewhere

1. Is quantum mechanics messing with your memory? The Guardian Science Blog
2. More than one shade of darkness The Sydney Morning Herald
3. #Nuttsack sparks debate about science and society, Crikey
4. The science of Chinese whispers gone wrong, Crikey
5. BMJ punks journalists for a christmas laugh, On Line Opinion

Should randomised trials be the only type of evidence accepted for rolling out drug treatments?

If so, then two researchers wrote in the Lancet this week that that we face a problem:

The evidence we have might not be the evidence we need, and the evidence that we need may never become available.

They are writing in response to the publication of a trial of a new combination drug treatment for stomach cancer that seems to extend people’s lives by nearly 3 months.

According to these guys, there’s probably lots of effective combinations of current cheap drug treatments to treat cancer, but we’re never going to get the evidence to prove it.

The new trial must have been crazy expensive: It consisted of patients from 122 different institutions in 24 counties on 4 continents and, as the editorialists point out, it would never have been paid for had it not promised huge returns for the pharmaceutical company that funded the trial, Roche.

This raises a sticky issue. Drug companies like Roche produce treatments that save lives. But they don’t do so because they save lives. Rather, they produce the drugs because they can make the company money.

So what happens when these two motivations come apart — when drugs that could save lives don’t make drug companies money? That’s the problem that the Lancet editorialists are referring to.

If we presuppose that only randomised trials produce evidence of sufficient quality to support decisions about the allocation of scarce resources, there is a problem. There is a lot of evidence on the effects of adding expensive new drugs to conventional therapies, but little evidence for when older, less expensive interventions are combined.

And boy are these drugs expensive. In the same comment piece, the authors calculated that the cost of each year of life gained by this new treatment, is about $100,000 AUD ($85,000 USD).

The authors of the study argue that this should be rolled out as a “new standard option” in the treatment of gastric cancer — and that’s not surprising given that the manufacturer of the drug not only funded the trial, but was involved in the data analysis and editing of the report.

So how are we supposed to get evidence about cheap drugs that will help millions of people but not make money for drug companies?

I don’t know. The authors mention the model in physics where large projects that don’t have obvious practical benefits are funded by governments and research institutions. And they seem to imply that there might be a way of not relying solely on randomised trials — but they don’t say what that is.

Whatever the way forward, it seems crazy that in the mean time, we can only get evidence about drugs which, for the majority of people around the world, are prohibitively expensive when there’s almost certainly some great cheap alternatives right under our noses.

Munro, A., & Niblock, P. (2010). Cancer research in the global village The Lancet DOI: 10.1016/S0140-6736(10)61022-7

Bang, Y., Van Cutsem, E., Feyereislova, A., Chung, H., Shen, L., Sawaki, A., Lordick, F., Ohtsu, A., Omuro, Y., & Satoh, T. (2010). Trastuzumab in combination with chemotherapy versus chemotherapy alone for treatment of HER2-positive advanced gastric or gastro-oesophageal junction cancer (ToGA): a phase 3, open-label, randomised controlled trial The Lancet DOI: 10.1016/S0140-6736(10)61121-X

This week we got a quick and entertaining lesson in the way science sometimes works… or doesn’t work.

Just last week, someone asked me how to tell good science from bogus science. My first, not entirely adequate response, was that you should check whether it’s been published in a peer-reviewed journal.

There’s lots of reasons why that response is not completely adequate, and one of those reasons was highlighted recently by an article being published in Virology Journal about how Jesus cured a woman of the flu.

Within three weeks the article was retracted but the fact that it got published in the first place shows that peer-review does not qualify something as good science.

I won’t say much more about this, but instead direct you to some good discussions of the debacle.

- Ivan Oransky, editor of Reuters Health, has a good overview of the debacle on his new and noteworthy blog Retraction Watch.

- Bob O’Hara has a funny discussion of the original article in This Scientific Life, written before the retraction. The editor of the journal posts his apology and retraction in the comments to the post.

Is it possible that if you have schizophrenia, smoking marijuana will actually improve your cognitive performance?

Since this blog is often concerned with the relationship between science and its communication, something which has come up once or twice here is the way drug and alcohol research is reported in the media.

Very often, it is reported that marijuana use causes schizophrenia and psychosis. This is despite the fact that there is quite strong evidence that it does not do so.

We also see government campaigns presenting very misleading information about marijuana and its effects.  Consider this one stating that “Cannabis can leave you permanently out of it”.

So it was not surprising this week, when we did not see headlines such as “Marijuana makes schizophrenics smarter,” even though there was some interesting research possibly suggesting exactly that.

But what’s really interesting about this research is that it seems to suggest that not only does smoking marijuana make people with schizophrenia smarter, it seems to show that the more often you smoke, and the younger you start, the smarter you get. (Yes, there are also other interpretations of the data. Continue reading.)

The researchers did two things: a meta-analysis and an experimental study. They found 10 studies that looked at the cognitive performance of people with schizophrenia who smoked marijuana and found a remarkable homogeneity between the studies.

Together, these 11 studies (the 10 in the analysis and the 1 they performed) found that people who had schizophrenia, and had a history of smoking marijuana, had a better visual memory as well as better planning and reasoning than their non-using peers. For several domains, there were no differences between the groups but for no domain, were the non-using groups better than the using groups.

Additionally, they found that a higher frequency of smoking was associated with higher cognitive performance, as was earlier age of smoking onset. (In their own study, the association was only present for people who started smoking before they were 17.)

How could these findings possibly be explained?

Well, of course one option is that smoking marijuana causes the better performance. This is plausible since it is known that people with schizophrenia and psychosis tend to have poorer cognitive performance and so something like marijuana might perform what the authors call “a neuroprotective role”. That is, the cognitive deficits that become apparent in people with schizophrenia at around puberty might be avoided if they smoke marijuana before, or at, that age.

However, there is another explanation of the data suggested by the authors that turns on the question raised earlier: does marijuana cause psychosis? If it does cause psychosis, the researchers suggested that there might be a group of people who would not have developed schizophrenia if they had not smoked marijuana. And that group of people, as a group less prone to schizophrenia, might also be less cognitively impaired. So people who have a history of marijuana would be more likely be a group that is less prone to schizophrenia and therefore perhaps have better cognitive performance.

So should people who think they might be schizophrenic go out and smoke lots of dope? Clearly not. While it seems to me that the evidence stands against the hypothesis that pot causes schizophrenia, the matter is far from closed and a lot of researchers do think there is such a link. Moreover, this latest research is not based on an awful lot of data and really needs to be replicated in larger studies.

What’s the bet, however, that the government website “PermanentlyOutOfIt.com.au” won’t mention this new and interesting research? Much better, they think, to stick with skewed, misleading messages rather than provide believable, balanced evidence from which people can make informed decisions.

[Via me at Psychiatry Update]

Yücel M, Bora E, Lubman DI, Solowij N, Brewer WJ, Cotton SM, Conus P, Takagi MJ, Fornito A, Wood SJ, McGorry PD, & Pantelis C (2010). The Impact of Cannabis Use on Cognitive Functioning in Patients With Schizophrenia: A Meta-analysis of Existing Findings and New Data in a First-Episode Sample. Schizophrenia bulletin PMID: 20660494

Frisher, M., Crome, I., Martino, O., & Croft, P. (2009). Assessing the impact of cannabis use on trends in diagnosed schizophrenia in the United Kingdom from 1996 to 2005 Schizophrenia Research, 113 (2-3), 123-128 DOI: 10.1016/j.schres.2009.05.031