An analysis of Hahn’s critique of my homeopathy papers: YES, IDEOLOGY DOES SEEM TO PLAY A PART

Forgive me, if this post is long and a bit tedious, but I think it is important.

The claims continue that I am a dishonest falsifier of scientific data, because the renowned Prof R Hahn said so; this, for instance, is from a Tweet that appeared a few days ago:

False claims, Edzard Ernst is the worst. Says independent researcher prof Hahn in his blog. His study: https://www.ncbi.nlm.nih.gov/pubmed/24200828 
His blog (German translation) http://www.homeopathy.at/betruegerische-studien-um-homoeopathie-als-wirkungslos-darzustellen…

The source of this permanent flow of defamations is Hahn’s strange article which I have tried to explain several times before. As the matter continues to excite homeopaths around the world, I have decided to give it another go. The following section (in bold) is directly copied from Hahn’s infamous paper where he evaluated several systematic reviews of homeopathy.

_________________________________________________________________________

In 1998, he [Ernst] selected 5 studies using highly diluted remedies from the original 89 and concluded that homeopathy has no effect [5].

In 2000, Ernst and Pittler [6] sought to invalidate the statistically significant superiority of homeopathy over placebo in the 10 studies with the highest Jadad score. The odds ratio, as presented by Linde et al. in 1999 [3], was 2.00 (1.37–2.91). The new argument was that the Jadad score and odds ratio in favor of homeopathy seemed to follow a straight line (in fact, it is asymptotic at both ends). Hence, Ernst and Pittler [6] claimed that the highest Jadad scores should theoretically show zero effect. This reasoning argued that the assumed data are more correct than the real data.

Two years later, Ernst [7] summarized the systematic reviews of homeopathy published in the wake of Linde’s first metaanalysis [2]. To support the view that homeopathy lacks effect, Ernst cited his own publications from 1998 and 2000 [5, 6]. He also presented Linde’s 2 follow-up reports [3, 4] as being further evidence that homeopathy equals placebo. 

_________________________________________________________________________

And that’s it! Except for some snide remarks (copied below) in the discussion section of the article, this is all Hahn has to say about my publications on homeopathy; in other words, he selects 3 of my papers (references are copied below) and (without understanding them, as we will see) vaguely discusses them. In my view, that is remarkable in 3 ways:

  • firstly, there I have published about 100 more papers on homeopathy which Hahn ignores (even though he knows about them as we shall see below);
  • secondly, he does not explain why he selected those 3 and not any others;
  • thirdly, he totally misrepresents all the 3 articles that he has selected.

In the following, I will elaborate on the last point in more detail (anyone capable of running a Medline search and reading Hahn’s article can verify the other points). I will do this by repeating what Hahn states about each of the 3 papers (in bold print), and then explain what each article truly was about.

HERE WE GO

_________________________________________________________________________

FIRST ARTICLE

In 1998, he [Ernst] selected 5 studies using highly diluted remedies from the original 89 and concluded that homeopathy has no effect [5].

This paper [ref 5] was a re-analysis of the Linde Lancet meta-analysis (unfortunately, this paper is not available electronically, but I can send copies to interested parties). For this purpose, I excluded all the studies that did not

  • use homeopathy following the ‘like cures like’ assumption (arguably those studies are not trials of homeopathy at all),
  • use remedies which were not highly diluted and thus contained active molecules (nobody doubts that remedies with pharmacologically active substances can have effects),
  • that did not get the highest rating for methodological quality by Linde et al (flawed trials are known to produce false-positive results).

My methodology was (I think) reasonable, pre-determined and explained in full detail in the article. It left me with 5 placebo-controlled RCTs. A meta-analysis across these 5 trials showed no difference to placebo.

Hahn misrepresents this paper by firstly not explaining what methodology I applied, and secondly by stating that I ‘selected’ the 5 studies from a pool of 89 trials. Yet, I defined my inclusion criteria which were met by just 5 studies.

___________________________________________________________________________

SECOND ARTICLE

In 2000, Ernst and Pittler [6] sought to invalidate the statistically significant superiority of homeopathy over placebo in the 10 studies with the highest Jadad score. The odds ratio, as presented by Linde et al. in 1999 [3], was 2.00 (1.37–2.91). The new argument was that the Jadad score and odds ratio in favor of homeopathy seemed to follow a straight line (in fact, it is asymptotic at both ends). Hence, Ernst and Pittler [6] claimed that the highest Jadad scores should theoretically show zero effect. This reasoning argued that the assumed data are more correct than the real data.

The 1st thing to notice here is that Hahn alleges we had ‘sought to invalidate’. How can he know that? The fact is that we were simply trying to discover something new in the pool of data. The paper he refers to here has been discussed before on this blog. Here is what I stated:

This was a short ‘letter to the editor’ by Ernst and Pittler published in the J Clin Epidemiol commenting on the above-mentioned re-analysis by Linde et al which was published in the same journal. As its text is not available on-line, I re-type parts of it here:

In an interesting re-analysis of their meta-analysis of clinical trials of homeopathy, Linde et al conclude that there is no linear relationship between quality scores and study outcome. We have simply re-plotted their data and arrive at a different conclusion. There is an almost perfect correlation between the odds ratio and the Jadad score between the range of 1-4… [some technical explanations follow which I omit]…Linde et al can be seen as the ultimate epidemiological proof that homeopathy is, in fact, a placebo.

Again Hahn’s interpretation of our paper is incorrect and implies that he has not understood what we actually intended to do here.

_____________________________________________________________________________

THIRD ARTICLE

Two years later, Ernst [7] summarized the systematic reviews of homeopathy published in the wake of Linde’s first metaanalysis [2]. To support the view that homeopathy lacks effect, Ernst cited his own publications from 1998 and 2000 [5, 6]. He also presented Linde’s 2 follow-up reports [3, 4] as being further evidence that homeopathy equals placebo. 

Again, Hahn assumes my aim in publishing this paper (the only one of the 3 papers that is available as full text on-line): ‘to support the view that homeopathy lacks effect’. He does so despite the fact that the paper very clearly states my aim: ‘This article is an attempt to critically evaluate all such papers published since 1997 with a view to defining the clinical effectiveness of homeopathic medicines.‘ This discloses perhaps better than anything else that Hahn’s article is not evidence, but opinion-based and not objective but polemic.

Hahn then seems to resent that I included my own articles. Does he not know that, in a systematic review, one has to include ALL relevant papers? Hahn also seems to imply that I merely included a few papers in my systematic review. In fact, I included all the 17 that were available at the time. It might also be worth mentioning that numerous subsequent and independent analyses that employed similar methodologies as mine arrived at the same conclusions as my review.

_____________________________________________________________________________

Despite Hahn’s overtly misleading statements, he offers little real critique of my work. Certainly Hahn does not state that I made any major mistakes in the 3 papers he cites. For his more vitriolic comments, we need to look at the discussion section of his article where he states:

Ideology Plays a Part

Ernst [7] makes conclusions based on assumed data [6] when the true data are at hand [3]. Ernst [7] invalidates a study by Jonas et al. [18] that shows an odds ratio of 2.19 (1.55–3.11) in favor of homeopathy for rheumatic conditions, using the notion that there are not sufficient data for the treatment of any specific condition [6]. However, his review deals with the overall efficacy of homeopathy and not with specific conditions. Ernst [7] still adds this statistically significant result in favor of homeopathy over placebo to his list of arguments of why homeopathy does not work. Such argumentation must be reviewed carefully before being accepted by the reader.

After re-studying all this in detail, I get the impression that Hahn does not understand (or does not want to understand?) the research questions posed, nor the methodologies employed in my 3 articles. He is remarkably selective in choosing just 3 of my papers (his reference No 7 cites many more of my systematic reviews of homeopathy), and he seems to be determined to get the wrong end of the stick in order to defame me. How he can, based on his ‘analysis’ arrive at the conclusion that ” I have never encountered any scientific writer who is so clearly biased (biased) as this Edzard Ernst“, is totally beyond reason.

In one point, however, Hahn seems to be correct: IDEOLOGY PLAYS A PART (NOT IN MY BUT IN HIS EVALUATION).

_____________________________________________________________________________

REFERENCES AS CITED IN HAHN’S ARTICLE

5 Ernst E: Are highly dilute homeopathic remedies placebos? Perfusion 1998;11:291.

6 Ernst E, Pittler MH: Re-analysis of previous metaanalysis of clinical trials of homeopathy. J Clin Epidemiol 2000;53:1188.

7 Ernst E: A systematic review of systematic reviews of homeopathy. Br J Clin Pharmacol 2002;54:577–582.

______________________________________________________________________________

For more information about Hahn, please see two comments on my previous post (by Björn Geir who understands Hahn’s native language).

This is also where you can find the only comment by Hahn that I am aware of:
Robert Hahn on Saturday 17 September 2016 at 09:50

Somebody alerted me on this website. Dr. Ernst spends most of his effort to reply to my article in Forsch Komplemetmed 2013; 20: 376-381 by discussing who I might be as a person. I hoped to see more effort being put on scientific reasoning.

1. For the scientific part: my experience in scientific reasoning of quite long and extensive. I am the most widely published Swede in the area of anesthesia and intensive care ever. Those who doubt this can look “Hahn RG” on PubMed.

2. For the religious part that, in my mind, has nothing to do with this topic, is that my wife developed a spiritualistic ability in the mid 1990:s which I have explored in four books published in Swedish between 1997 and 2007. I became convinced that much of this is true, but not all. The books reflect interviews with my wife and what happened in our family during that time. Almost half of all Swedes believe in afterlife and in the existence of a spiritual world. Dr. Ernsts reasoning is typical of skeptics, namely that a person with a known religious belief in not to trust – i.e. a person cannot have two sides, a religious and a scientific. I do not agree with that, but the view has led to that almost no scientist dares to tell his religious beliefs to anyone (which Ernst enforces by his reasoning). Besides, I am not very religious person at all, although the years spent writing these books was quite an interesting period of my life. In particular the last book which involved past-life memories that I had been revived during self-hypnotims. I am interested in exploring many sorts of secrets, not only scientific. But all types of evidence must be judged according to its own rules and laws.

3. Why did I write about homeopathy? The reason is a campaign led by skeptics in some summers ago. Teenagers sat in Swedish television and expressed firmly that “there is not a single publication showing that homeopathy works – nothing!”. I wonder how these young boys could know that, and suspected that had simply been instructed to say so by older skeptics . I looked up the topic on PubMed and soon found some positive papers. Not difficult to find. Had they looked? Surely not. I was a frequent blogger at the time, and wrote three blogs summarizing meta-analyses asking the question whether homeopathy was superior to placebo (disregarding the underlying disease). The response for my readers was impressive and I was eventually urged to write it up in English, which I did. That is the background to my article. I have no other involvement in homeopathy.

4. Me and Dr Ernst. I came across his name when scanning articles about homeopathy, and decided to look a bit deeper into what he had written. The typical scenario was to publish meta-analyses but excluding almost all material, leaving very little (of just a scant part of the literature) to summarize. No wonder there were no significant differences. If there were still significant differences the material was typically considered by him to be still too small or too imprecise or whatever to make any conclusion. This was quite systematic, and I lost trust in Ernst´s writings. This was pure scientific reasoning and has nothing to do with religion or anything else.

// Robert Hahn

_________________________________________________________________________

Lastly, if you need more info about Hahn, you might also want to read this.

Read more here

How does this post make you feel?
  • Awesome (0)
  • Interesting (0)
  • Useful (0)
  • Boring (0)
  • Sucks (0)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>