Emory Professor of Psychology Donna Maney and Yesenia Garcia-Sifuentes (26G), a student in Emory’s Graduate Program in Neuroscience, published a study on Nov. 2 which detailed misreporting of sex differences in biological sciences.
According to Maney, there has been a push for all clinical and preclinical research to include males and females over the past decade. She said this push has largely come from funding agencies like the National Institutes of Health (NIH).
"This is a step in the right direction because there is a long history of research being conducted on males only, particularly in neuroscience,” Maney said. “During this push, however, we are witnessing a kind of frenzy to find and report sex differences in literally everything we study. And it seems that not everyone looking for sex differences is trained on how to test for them."
Researchers that divide their samples by sex report sex differences most of the time, Maney said. However, the study found that when a sex-specific effect of a treatment was claimed, authors failed to actually use statistical tests to determine whether males and females responded differently. Results of treatments were not compared between the sexes 70% of the time, according to the study’s results.
“When authors failed to test for a sex-specific effect, they claimed one 88% of the time,” Maney said. “In contrast, when the authors did the appropriate statistical tests, they reported sex-specific effects only 63% of the time. Thus, sex-specific effects may be over-reported.”
Additionally, Maney said that they encountered strategies that could mask sex differences, “such as combining data from males and females without first testing for a difference.”
When scientific papers publish sex differences, only one out of five studies analyze data by sex, according to Maney. In addition, more than half fail to back up a claim of a sex difference with statistical evidence.
“This terrifies me because women would be particularly vulnerable,” Maney said. “It is already the case that women can’t be prescribed a full dose of some drugs in this country, despite strong evidence that small sex differences in clearance rates are not clinically meaningful.”
Maney said that it’s important to address this issue to ensure that “decisions that affect access to health care are firmly grounded in rigorous science.”
As more papers begin to publish sex differences, it’s important that the use the proper statistical analysis so that the data is not disproportionately represented, Maney argued. Sex-based treatments may actually cause more harm then good if the data is lacking, Maney said.
Studies like this reveal a scary reality about sex-based studies, Maney said. However, she said that she hopes her study can be a force of change.
“Our hope is that the study will raise awareness about how sex-specific effects are detected and reported, and that the current standard of evidence is lacking rigor,” Maney said.
