Posts Tagged Scientific method
If you don’t care about the discussion, just skip down to the boo-boo in bold.
In my recent post about most scientific studies being wrong, I began questioning the very basis for evidence-based medicine. How can we know what we know? When we do a large scale human study, we have to trust the results. We can’t run off to a lab and have them do it again.
So I thought a quick check into alternative medicine advances might be in order. As expected, most studies are small and underfunded compared to the massive studies produced by drug companies. And it is those massive drug studies that are in question for their ability to give us truthful answers about what will work. If they cannot be reproduced in a consistent manner, then what chance does alternative medicine have?
There is another avenue for medical knowledge, largely disregarded by current researchers but widely used by the public. It is the school of what works. One of the basics, something we are taught from the school yard, is that ice helps boo-boos. In more medical terms, cryotherapy is largely regarded at efficacious for the treatment of minor acute trauma. But is it true?
Do Boo-Boos Get Better With Ice?
According to the Cochrane Meta-Analysis entitled: Does Cryotherapy Improve Outcomes With Soft Tissue Injury? (free full article here) the researchers concluded: “no authors have assessed the efficacy of ice in the treatment of muscle contusions or strains.”
That’s right, mothers across America. You are applying ice to those boo-boos without a shred of scientific evidence that the ice is effective.
Yes, icing has been shown to be effective after: “ligament repairs and knee and hip replacements. The results of these studies cannot be generalized to muscle strains and contusions.”
But ice works? You’ve seen it work? Mere anecdotal evidence. Unreliable and prone to patient bias. The researchers tenatively regarded ice as possibly helpful for pain, but concluded that: “Many more high-quality studies are required to create evidence-based guidelines on the use of cryotherapy.”
So until they do that, don’t waste your hard-earned health care dollars on ice cube trays and washcloths.
Let’s all wait until definitive studies conclude that ice does indeed help with boo-boos. It may not happen anytime soon, because ice is not patentable (although you know they keep trying). So we may need to create the “Boo-Boo Foundation” to fund ice research. Get ready to march in “Stop the Boo-Boos” marches and send your dollars in. Who knows? In a few decades we might just be able to apply ice to those bumps with the knowledge that it actually works.
Or, if you’ve been following the discussion about studies, we might conclude that good-hearted researchers might want to spend a little less time in the lab and a little more time in the playground. If they banged themselves on the monkey bars, they might just ask for a little ice.
Some things, because they work consistently and well, do not have research. Giving a hug and a kiss are also tried-and-true, unscientific, aids for boo-boo relief.
- I feel like a five year old with a boo boo. Help me take care of my knee. (ask.metafilter.com)
- Ode to an Ice Pack (thefridaynightwine.wordpress.com)
- Being a Klutz (theascentblog.com)
- I Heart First Aid Kits (womanmdsguide.com)
- 13 handy uses for ice cubes (mnn.com)
- Lessons From A Broken Nose (mydisruption.wordpress.com)
- Panthers Smith bruises knee, held out of practice (wcnc.com)
- My Worst Day As A Youth Sports Coach (ashimmy.com)
- Salute to ice packs!!!! (theaveragebookenthusiast.wordpress.com)
- Will Muschamp clarifies Jeff Driskel’s shoulder injury (jacksonville.com)
As the number of scientific studies exponentially mount, surely we are advancing scientific inquiry at an ever increasing rate. But perhaps we are simply increasing the scientific “noise.”
Enter the Reproducibility Initiative, which will try to reproduce your findings for you by an independent lab. Yes, we’ve come to a point where you will need to pay to have the results you think you have confirmed by someone else.
Consider the NewsDaily’s article that “Bayer Healthcare reported that its scientists could not reproduce some 75 percent of published findings in cardiovascular disease, cancer and women’s health.” Or that “Amgen reported that when the company’s scientists tried to replicate 53 prominent studies in basic cancer biology, hoping to build on them for drug discovery, they were able to confirm the results of only six.”
How is that possible? Don’t we have scientists dedicated to publishing whatever results occur?
Anyone who remembers science class knows the answer. When you got the results you expected, you didn’t go over the equipment and the method with a fine toothed comb. You assumed you did the experiment right and turned it in. Only when you got wildly odd results that didn’t agree with what you were looking for in the slightest did you go back over your method and equipment to find the error. Even if you had the highest ethics, it would be perfectly possible to miss some error as long as the results fell into a “reasonable” outcome.
Having more people do the same testing can lead to better results, but if you are all testing in the same area and watching one another’s results, chances are good that you started looking for the same results in your experiments. A friendly classmate might even help you by telling you how to change your equipment to get a desired result. So more tests do not necessarily lead to more accurate results.
In this appetizing little mathematical jaunt, the author takes us down the reality that false positives are far more likely than finding the truth. Even before we add in publication bias, tenure track pressures, and financial incentives, it is just too easy to find the results you’re looking for. The author Ioannidis states: “manipulation could be done, for example, with serendipitous inclusion or exclusion of certain patients or controls, post hoc subgroup analyses, investigation of genetic contrasts that were not originally specified, changes in the disease or control definitions, and various combinations of selective or distorted reporting of the results. Commercially available “data mining” packages actually are proud of their ability to yield statistically significant results through data dredging.”
So just how many of us are taking drugs created for an illness, supported by studies created to support that drug’s ability to treat that illness, and prescribed by doctors who believe that the drug will effectively treat our illness despite all of our claims that the drug really isn’t working? Meanwhile we as patients want to have something that works for our illness, so we spend a lot of time giving the drug “time to work” when it really never does anything to help us.
As someone who works in the alternative healthcare field, I don’t know whether to laugh or cry. It feels like the rug just got pulled out from under all the work we’ve done to start bringing the field up to the standard of scientific inquiry. Suddenly what was clinically relevant information is in question, and the standard drugs that we’re trying to compare to the alternatives are also in question. How do we know what works?
Fortunately, I’ve got an ace up my sleeve. I’ve been working with ornery, independent minded patients for years who don’t mince words when things don’t work. So I’ve got an ongoing practice based on what is working in the field, using my patients as my resources. Maybe it is time for all doctors to use their patients, rather than the drug reps, as their resource for what really works.
- “No.. Not Yet”:Answering the Patient’s Request for “Off Label Use” of a Drug (bioethicsdiscussion.blogspot.com)
- Shoring Up the Mantra of Science: Take Nobody’s Word for It (reason.com)
- Why scientists seem to change their minds (kirstyevidence.wordpress.com)
- Good Scientist! You Get a Badge. (slate.com)
- R&D experts applaud new initiative to validate biomedical studies (fiercebiotech.com)
- Glaxo’s Experimental Drug Halves Asthma Attacks in Study – Bloomberg (bloomberg.com)
- Experimental cancer drug makes mice infertile without side effects, scientists claim male birth control discovery (cbsnews.com)
- Amgen Halts Pancreatic Cancer Trial After Drug Failure – Bloomberg (bloomberg.com)
- Study off Mass. coast finds noise harming whales (bostonherald.com)
- Trusting the science (scienceornot.net)