Acupuncture Research: The Down and Dirty on the Double-Blind

Before we get into the nitty gritty, please know that this post will make more sense if you have read Is Acupuncture...Real Medicine. Read it, if only for the great story of your *calm* acupuncturist, Lauren, getting super pi**ed off and then...and then....doing something about it!Okay. Now for the down and dirty on the double-blind.

Efficacy vs. Effectiveness

In the world of research, it's important to distinguish between an effectiveness trial and an efficacy trial.Effectiveness trials compare two treatments under conditions in which they would be applied in typical, routine care. These trials don't really give a schnitz about the underlying mechanisms of how something works - they're simply designed to find out which treatment works better. The results of these trials are often applied to clinical decision-making (i.e. for a patient with acute low back pain, how does acupuncture compare to physical therapy).I like to compare the effectiveness trial to that certain type of Mainah' that comes to the clinic, plops down in the chair, and says "Don't care what you do, just make me feel bettah.'' They simply want to feel well enough to ride the sled on the lake for a couple hours without pain, and if you can do that, expect peppermint fudge on the holidays.The efficacy trial asks a more.... demanding question: Does treatment X perform better than placebo? The efficacy trial looks to understandwhether a treatment has effects beyond the "context" or "ritual" of the treatment (i.e. is the patient feeling better simply because a medical professional focused attention on their needs?). The goal is to figure out whether the treatment has a specific impact on disease, as opposed to nonspecific effects. Efficacy trials are often referred to as "explanatory trials."I liken the efficacy trial to the type of patient who comes in knowing nothing about acupuncture, and once they feel better after a treatment, proceed to go nuts with curiosity. Truly. It's a weekly occurrence. They come in for their 2nd treatment and say something like this: "My back pain is better, and I think it might be the acupuncture. Do YOU think it's the acupuncture?"Listen. I'm trying to write an objective piece here. But for the sake of transparency: I always believe it's the acupuncture.Over the next couple of weeks, that die-hard-curious patient does some research of their own, and they attempt to remove ALL POTENTIAL INTERFERING VARIABLES that could have influenced the lessening of their back pain. They want to know FOR SURE whether it was the Acupuncture Prahran services they had or the new multivitamin they started, the morning stretches, or the new pillow they bought. They stop the multi-vitamin. They stop their morning stretches. They put their therapeutic pillow away for a couple of nights.Then they watch, feel, and listen.The results?Let's just say that acupuncture has been around for over 3000 years.Time To Get Serious: Big Ol' Problems in the Field of Acupuncture ResearchIn November 2007, The Society for Acupuncture Research held an international symposium to mark the 10th anniversary of the 1997 NIH Consensus Development Conference on Acupuncture. The symposium reflected the considerable growth that had taken place in the field of acupuncture research; however, it also brought to light a wicked big issue. Acupuncture performed well in the effectiveness trials, but failed miserably in the efficacy trials.WHAT??!!!The results from the efficacy trials sunk the hearts of acupuncturists around the world, and it became the favorite argument for those of the...anti-acupuncture tendency. The irony of all of this? No one was discussing the glaring gaps in knowledge that affected the way these studies were carried out. What do I mean by that? Am I making excuses for acupuncture? Well, you be the judge. Read on.The research methodology used in much of the research was far from perfect; in fact, some of it was simply poorly designed. This was somewhat understandable, seeing as acupuncture did not (at that point) have history as a commonly-researched modality of medicine.The hardest thing to understand? Why were some people so rabidly attached to the results of poorly designed research.What do I mean by this? Below is the EPL (everyday people language) explanation of what went down.THE WICKED BIG ISSUE: A number of clinical trials reported that true acupuncture is superior to usual care (good results in effectiveness trials); however, true acupuncture did not significantly outperform sham acupuncture (poor results in efficacy trials).EPL: Though patients feel better when acupuncture is added to standard care, poking someone with real needles isn't much better than poking someone with toothpicks. In other words, is acupuncture nothing more than the placebo effect at work?Things to Consider Before You Get on Facebook and Make the Bold Claim that Acupuncture Isn't Medicine 1. How is acupuncture being defined? An acupuncture treatment is not a simple intervention, and involves point selection according to diagnosis, specific point location, specific needle depth, and specific levels of stimulation. In most of these studies, there was no differential diagnosis (which is essential to an acupuncture treatment, as well as integral to basic Traditional Chinese medicine theory). The biomedical equivalent? A study of how well surgery worked for low back pain, when some of the patients in the study had knee pain, and no low back pain.2. Sham needling is not inert. In many of these studies, it showed that the sham treatment can produce a benefit that is procedure-related. What does this mean? Well it means that the "sham" in this scenario is NOT A SHAM! Often times, in these research trials, the sham treatment consisted of toothpicks in a guide tube that were tapped out, like an acupuncture needle, but didn't penetrate the skin. However, it's theorized that this type of sham needling might induce a significant physiological response due to microtrauma to the skin. The long and short of it: The sham treatment must NOT mimic the physiologic effects of the real treatment.Note: Placebos/shams are easier to deal with in drug trials since there is usually an understanding of how the drug being tested is metabolized, which receptor it is targeted to, and how it is inactivated. In order to develop an appropriate sham for acupuncture research, we first need a clearer understanding of the underlying mechanism of acupuncture. And that's a whole other blog post...3. Including a sham treatment may reduce the level of effectiveness of the real treatment: In the informed-consent paperwork of a research trial, patients must be informed that there is a 50 percent chance that they will get the real treatment, and a 50 percent chance that they will get the fake treatment - which essentially reduces patient expectancy of receiving the real (effective) treatment from 100 percent to 50 percent.4. The practitioner in the sham treatment cannot be blinded. Which means... that this is not a double-blind trial! In a typical drug trial, the doctor does not know whether she is administering the placebo pill or the real thing. You can't do this with acupuncture. Obviously, the dude with the toothpicks in the guide tube knows he ain't doing the real deal...So, What Does This All Mean? What does this all mean, you ask? Should we stop doing research on acupuncture? Of course not! Embracing biomedicine is important to the health of Traditional Chinese Medicine. Because you know what's a horrible, no good, very bad idea? Deciding that a system of medicine that has been around for a long, long time (with a great amount of success under its belt) simply isn't worth your time because it doesn't happen to fit within your framework of understanding health.We should, however, encourage clinicians and scientists to recognize that there is more work to be done in order to effectively study acupuncture.Acupuncture treatments should be studied from the top down (as multicomponent whole-system interventions employing a pragmatic systems perspective), as well from the bottom up (as mechanistic studies that focus on understanding individual treatment components and how the effects of these components interact and translate into clinical outcomes).Feeling super dorky about all of this? Check this out. Also, want to learn a little more on the biomedical theories regarding acupuncture? Click here.And to wrap this up in everyday people language: Before you make any assumptions about acupuncture, demand higher quality research that accounts for the complexity inherent to this form of treatment.Or come in and try it yourself. Because lawd knows I'd love to poke ya'.Want to read more? Check out posts by Mel Hopper Koppelman, starting with How to Win an Argument with An Acupuncture Hater and then check out Acupuncture, Science Based Medicine, and UFOs where Mel takes on (and systematically slaughters, in my humble opinion) a writer for the website, Science-Based Medicine. Read the discussion and decide for yourself!