Research 102

First of all, I would like to thank those of you who haven't bothered to cancel your subscription to this blog. Secondly, I'd like to note that if I'm only going to post every 2.5 years, I completely don't blame you for canceling your subscription to this blog.

Welcome back! Feel free to read the last post again, just in case you have forgotten all the minute details of what I wrote back in 2010. (Sigh.)

Now, shall we continue?

I will remain true to my last post, where I said, "In fairness, maybe the study is just fine, and the results are true and accurate. How can you tell that? I’ll get into more details of that next time." I'd like to cover just a little bit of experimental theory, because if you're like me, it's been a long time since that high school class where you had to keep a composition lab notebook that got turned in once a week. In real life, I actually do still have a lab notebook. However, instead of the black and white cover with wide ruled pages, it's got a brown hardback cover, a control number, and I have to make scanned copies once every six months to go into a quality assurance archive.

When was the last time you heard a friend, colleague, or tv reporter say, "studies show that..." or something similar? Probably very, very recently. But how often do you hear that person cite the actual study ("a March 2011 study in the Journal of Agriculture and Food Chemistry led by Dr. Knows Something at the Institute of Human Health...")? Probably less often. And even less often than that, you go pull up that academic journal, read the article itself, and follow up on the references. Right?

So... how do you know whether the study was even real? Or done correctly? Or had valid results? Let’s be honest: academic journals are typically written in ways that make the authors sound smart. I’ve peer-reviewed and edited some of these articles, and I can tell you from an insider perspective that the bigger the words and the longer the equations you use, the less people will question you because they don’t want to look stupid by not knowing your terminology. These things aren’t written for your average Reader’s Digest subscriber. They’re written to be convincing. But even the RD subscriber can pick out a few things that will help you determine how seriously to take a study. We’ve already talked about the basis for research itself- now let’s look at experimental aspects.

When you're looking to conduct scientific research, how you go about setting up, conducting, and interpreting your experiment makes all the difference in the world. While this isn't a comprehensive list, a few things to keep in mind are sample size, sample population and controls, scenario realism, and result framing.

First of all, let’s consider sample size and population. Whether you’re looking at rats, humans, lima bean plants, or air conditioners, the number of items in your experiment makes a difference. Conducting an experiment on a single person isn’t likely to yield results worth anything. Just because I give a person a heart medication and they die of pancreatic cancer doesn’t mean that the medicine caused that. Maybe it did. Or maybe not. Just because a mother who drank coffee in pregnancy gives birth to a child with allergies doesn’t mean that all pregnant moms who drink coffee will then have allergy-prone offspring. On the other hand, if you track 1,000 patients with heart disease being given a heart medication, and 200 of them die of pancreatic cancer, there’s a much higher chance that you’ll be able to link the two. In short, if you’re reading up on a study and they only used two people, take the results with a grain of salt. If they used 70,000, that’s a point in the “good research” column.

But just having an absurdly large population size won’t get you good results. Sample population and controls are also critical. Remember how in math you had to keep solving for x? The “x” was the variable. If you got into more complicated math, you had more variables- x, y, z. In super-duper math, we ran out of regular letters and had to switch to Greek letters. At some point, it just becomes too much to keep track of and control. Well, same thing happens in experiments. The more variables you have, the less control you have over your experiment. Anything you can keep entirely constant is your control. Picture this: you want to find out whether watering lima bean plants with dilute acid kills them or makes them grow stronger. (Don’t ask me why- maybe you bought stock in a vinegar production technology and like limas?) You already know that testing 2 plants probably isn’t a large enough sample size. What if one was already diseased? So you get 100 lima bean plants. You know that they are all about the same age and from the same seed producer. You put them in identical container sizes, and fill the pots with soil from the same source. You put them all in a location with the same amount of light exposure, and where the temperature is kept at a constant 80 degrees. What you’ve just done is controlled a bunch of variables. Light, nutrients, temperature, container size, soil permeability, etc. are all factors that *could* have affected these plants. But by controlling your environment, and controlling your population to plants with similar genetic and age factors, you can be more sure that what you’re watering them with is what would be causing growth differences.

And how do you water them? Ideally, you’d want your “watering” scenario to be realistic. You’d want to know how often lima bean plants should be watered, and how much water they need. It’s not realistic to water a tiny plant with a gallon of water (or acid) a day. It’s also not realistic to never water them. In real life, maybe 0.5 cups every other day is just about right. So you set up your experiment where you water 25 of the plants with pure water, 25 plants with half water, half dilute acid, 25 plants with three-fourths dilute acid to one-fourth water, and 25 with the dilute acid. For control, you water all of them using the same batch of water and acid, and you measure out each 0.5 cup “serving” to each plant. Now you’ve got a reasonable population size, a controlled environment and population sample, and a realistic scenario. You should be able to feel fairly confident that differences you see among the groups of 25 are a result of what you’re watering them with.

By the by, feel free to steal this lima bean thing as your kid’s science fair project this year. You’re welcome.

Last but not least, how you frame your results makes a big difference in the “take-away” message, or conclusions, from your experiment. There’s a book called “How to Lie with Statistics” that was written in the 50’s (still completely valid and recommended reading!) whose title is self-explanatory. Let’s say- and I’m completely making this up- that the 25 plants that got pure water grew 5 inches each, the 50 plants with mixed water grew 10 inches each, and the plants with just acid died completely. If your experimental conclusion is that “watering plants with dilute acid kills them”… yes. That’s true in this case. BUT, you’ve left out the important point that the 50 plants that received slightly acidic water grew better than the water-alone ones! This might not seem like a huge deal, but stuff like this gets used as a scare tactic. For example, we hear all.the.time. about needing to reduce the sodium in our diets, right? Because a high-sodium diet can kill you! Well, yes, but a no-sodium diet can kill you too. A lot of life- and this should not surprise you- is about middle ground. The truth is often somewhere in between extremes. Make sure that the way results are framed makes sense, considers context, and isn’t just serving an agenda.

This is getting really long, so, if you’re still reading, bless you. But let me wrap up.

Have you ever heard of "anecdotal evidence?" Anecdotal evidence is someone's story, basically. Let's say that I know three people who ate the cafeteria's pasta salad. All three got sick. Anecdotally, evidence suggests that pasta salad from the cafeteria makes you sick. However, does that really prove that the pasta salad makes them sick? (Well… *I* wouldn’t eat it, but that’s just me…) As you know by now, that’s not a real experiment. You don’t know that those three people didn’t also drink contaminated water. Or get simultaneous exposure last week to the flu. The best solution here would probably be to send a sample of that pasta salad off to a lab and have it tested for bacteria. But in the absence of that, maybe just have the chicken instead, and don’t go write a book about how all pasta salad causes illness.

So next time you hear “studies show,” question it. See if sources are actually cited. See if the study meets the criteria of being independent and unbiased. Then look at how the experiment was run. Think about the sample size, how variables were controlled (and which weren’t controlled), and how the results are presented. You’ll be in a much better position to decide whether or not that study really DOES show what it claims to.

Class dismissed.

-Schientist

Research 101

Apparently when I start a project, I should pay a bit more attention to when I’m actually going to be able to dedicate time for it. I didn’t really mean to wait six months before writing anything of substance, but here we are. *whistles innocently *

It took me a while to figure out the subject I should write on first. It occurred to me that one of the most fundamental things about science is research. As we all know, everything on the internet is true, right? I’m forever reading articles and blogs that make loud and proud proclamations that such-and-such a product is going to give you cancer within two years, or that doing/not doing a certain thing is going to cause your children to have three arms, and of course there’s always my favorite, “the government is lying to you about this substance and secretly trying to kill off the population by not telling you how dangerous it is!” Not being a huge conspiracy theorist, I tend to think that perhaps some of those claims go a little bit too far. But how can a person decide for themselves? In my profession, sometimes it’s as easy as asking the expert in the office next door, but if you don’t have a convenient schientist friend to ask, here are a few guidelines that may come in handy.

Scientific research comes in two major phases: literature review and experimentation. The former is looking at what has already been done and extracting good information that can give you a comprehensive picture of a subject. The next step for researchers is experimentation, which (ideally) provides information that fills gaps in what is already known. It’s actually important to understand both phases, because while what you participate in at home is likely to be literature review (how many of us have labs in the basement? Or… a basement?) it’s also helpful to be able to determine if the literature you’re reviewing is true and accurate.

So let’s start with the literature review thing. Just a quick peek at the internet will tell any fool that there’s a plethora of information on just about everything you never wanted to know. Where to even begin? A few key factors to consider are: trust in the source, funding, and presentation of information.

Thanks to the interwebs, everyone has an opinion and a free and fast way to share it. But just like in real life, some sources are definitely better than others. If your kid brother in high school tells you that the brand of dishsoap you use every day is going to kill you in a week (while picking his nose), would you be more or less likely to believe him than if your family doctor said the same thing? The doctor definitely is better qualified to be giving that kind of information, so hopefully you’d be more likely to believe him or her. Similarly, an internet doctor you’ve never heard of with no verifiable credentials should probably not carry the same kind of weight in your decisions as established medical expert centers such as the Mayo Clinic, the National Institute of Health, The American Cancer Society, or MD Anderson. Such institutions have years of experience, qualified doctors, and are typically on the cutting edge of research. Do they make mistakes? Sure. But there are a lot of checks and balances in modern research so that mistakes are generally quickly discovered and refuted. If you’re not comfortable with a government-run or profit based institutions, there are plenty of non-profit groups that are also helpful sources. Look for large groups with a wide base of participants, qualified experts, and a variety of funding sources.

And that brings us to the next big thing: funding. Pay attention to who is paying for the research you’re looking at. Let’s say that the University of Nowhere conducts a study to see whether a certain newly marketed pesticide causes cancer in lab rats. You read the study and it seems that the little rodents came through the study healthy and happy, so you decide to can safely use it on your lawn. Cool. But then you notice that the study was funded by the manufacturer of that new pesticide. Would that make you think twice about the results? It should! There’s an obvious potential conflict of interest there. In fairness, maybe the study is just fine, and the results are true and accurate. How can you tell that? I’ll get into more details of that next time. But it’s definitely a good idea to see if the people paying for the study have a significant profit stake in the outcome. Again, it’s not necessarily damning, but it should definitely make you look a little closer at the results. The most safe funding sources are usually from groups that don’t have a dog in the fight, like the National Science Foundation, for example.

Another consideration is how the information is presented. Alarmism is a big deal in information transfer. Invoking fear and panic by use of loaded words, scary pictures, threats against loved ones, or threats against personal wellbeing is a common tactic for getting inside a person’s head. Interest groups often make use of alarmism to make their information seem more urgent or important than it might actually be. Once a person sees a threat to their well-being or the well-being of someone they love, it is difficult to persuade them otherwise. Are the consequences true? Perhaps so. Think about certain product recalls: If salmonella is discovered in a line of peanut butter, it is urgent and important that people do not eat that product, because deaths can result in susceptible populations. But for the most part, good information is usually presented in a clear, calm, and unbiased manner. Most valid studies aren’t going to try to bully you or scare you into following the recommendations that are made, even if they’re strong. Keep an eye out for “buzz-words,” overreaching statements (“millions of people unknowingly suffer from this daily!”), overly broad statements of consequence (“if you don’t do this you’ll die!”) and of course profit driven recommendations (“you must buy our product to fight off this evil!”)

Just remember that a lot of people out there have agendas. Even when agendas aren’t profit driven, there are some very well-meaning interest groups out there with bad information. Take a good look at the source of the information, how studies were paid for, and how the information was presented. It may not guarantee that you’re getting the whole truth and nothing but the truth, but it will sure help you sort through the giant world wide weeds in your search.

Class dismissed.


-Schientist

Schience as I know it.

Well hellloooooo. Nice of you to drop by!

My name is Miriam. I'm a research schientist, according to my business card. (Yes, I had to get them reprinted.) I work for a research and development institute doing geochemistry, environmental science, and public outreach. I've been a science nerd about as long as I can remember. One day I found that if you mix baking soda and vinegar in a closed container, you get a mini-explosion... and I was hooked. I took loads of science in middle and high school, was on the geeky science team, and then went off to college intending to double major in science and music. As it turns out, aural theory is often at the same timeslot as biology, and it just didn't work out. So I stuck with science as a career and music on the side, got a couple degrees, and started working along the way. I've been really lucky so far with my work and university study opportunities. I've done everything from environmental analyses (like pesticides, PCB's, petroleum contaminants, etc.) to biosensors to nanoparticles to groundwater testing and underground chemical environments. It's all very exciting. The best part of all this is when I get to do field studies. It puts my normal nerdiness right at home out in the wild.

I also work in public outreach, taking the technical-schmechnical and translating it for the non-nerd types; often known as the Angry Citizens with Concerns. This is my FAVORITE. Seriously. I do public meetings, create posters and information sheets and other learning tools, and do lots and lots of talking. I could do this every day.

You'd think I'd get enough of all that molecular business on the job, but I spend a lot of time doing extracurricular academic journal reading (I know, I know... I need to get out more) on nutrition, pharmacology, and fitness. Biochemistry was my undergraduate concentration, and I've really enjoyed taking that info and applying it to the daily putting-of-food-into-mouth. Put all that together and I'm a bit of a lifestyle nerd as well.

My friends know this stuff about me, and as a result I wind up getting asked a lot of questions. "Should I be worried about Teflon?" "Why can't we recycle nuclear fuel?" "Does it really make my electric bill go up if I leave stuff plugged in?" "What's the difference between sugar and Splenda?" "How much protein should I be eating?" "Why IS the sky blue?" The goal of this blog, therefore, is to share useful day to day stuff that has to do with questions people ask me, anti-panic notifications, household stuff, and maybe just some generic interesting science info once in a while. How's that for a thesis statement?

Aside from all that, though... I'm a Christian, an organizing fanatic, and an overzealous collector of clothes, cute purses, and sparkly eyeshadow. (No sense LOOKING like a dork, after all.) I also, of course, spend way too much time on the computer. I appreciate the indulgence of a few online and real-life friends who have encouraged me to start this little blog in the rapidly expanding blogosphere, and I hope you'll come back again! Next time I'll even publish something useful.