Dear KAG: 20210601 Open Thread

Cuppa,

Here is ‘Peer-Reviewed” studies for them and comments from medical journal editors.

How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data

ABSTRACT

….A pooled weighted average of 1.97% (N = 7, 95%CI: 0.86–4.45) of scientists admitted to have fabricated, falsified or modified data or results at least once –a serious form of misconduct by any standard– and up to 33.7% admitted other questionable research practices. In surveys asking about the behaviour of colleagues, admission rates were 14.12%< (N = 12, 95% CI: 9.91–19.72) for falsification, and up to 72% for other questionable research practices. Meta-regression showed that self reports surveys, surveys using the words “falsification” or “fabrication”, and mailed surveys yielded lower percentages of misconduct. When these factors were controlled for, misconduct was reported more frequently by medical/pharmacological researchers than others.

Considering that these surveys ask sensitive questions and have other limitations, it appears likely that this is a conservative estimate of the true prevalence of scientific misconduct.

Why Most Published Research Findings Are False

Abstract

Summary

There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias…..

Research Misconduct Identified by the US Food and Drug Administration:
Out of Sight, Out of Mind

Importance

Every year, the US Food and Drug Administration (FDA) inspects several hundred clinical sites performing biomedical research on human participants and occasionally finds evidence of substantial departures from good clinical practice and research misconduct. However, the FDA has no systematic method of communicating these findings to the scientific community, leaving open the possibility that research misconduct detected by a government agency goes unremarked in the peer-reviewed literature….

Objectives

To identify published clinical trials in which an FDA inspection found significant evidence of objectionable conditions or practices, to describe violations, and to determine whether the violations are mentioned in the peer-reviewed literature.

Design and Setting

Cross-sectional analysis of publicly available documents, dated from January 1, 1998, to September 30, 2013, describing FDA inspections of clinical trial sites in which significant evidence of objectionable conditions or practices was found.

Results: Fifty-seven published clinical trials were identified for which an FDA inspection of a trial site had found significant evidence of 1 or more of the following problems:

falsification or submission of false information, 22 trials (39%);

problems with adverse events reporting, 14 trials (25%);

protocol violations, 42 trials (74%);

inadequate or inaccurate recordkeeping, 35 trials (61%);

failure to protect the safety of patients and/or issues with oversight or informed consent, 30 trials (53%);

and violations not otherwise categorized, 20 trials (35%).

Only 3 of the 78 publications (4%) that resulted from trials in which the FDA found significant violations mentioned the objectionable conditions or practices found during the inspection. No corrections, retractions, expressions of concern, or other comments acknowledging the key issues identified by the inspection were subsequently published.

Conclusions and Relevance

When the FDA finds significant departures from good clinical practice, those findings are seldom reflected in the peer-reviewed literature, even when there is evidence of data fabrication or other forms of research misconduct.

(Blogs are now performing the important tasks of scrutinizing papers and conclusions often finding gross mistakes.)

Editors In Chief of World’s Most Prestigious Medical Journals: “Much of the Scientific Literature, Perhaps HALF, May Simply Be Untrue”

… “It Is Simply No Longer Possible To Believe Much of the Clinical Research That Is Published”
Posted on June 1, 2015 by WashingtonsBlog
Corruption Is Destroying Basic Science

“Lancet and the New England Journal of Medicine are the two most prestigious medical journals in the world.

It is therefore striking that their chief editors have both publicly written that corruption is undermining science.

The editor in chief of Lancet, Richard Horton, wrote:
*http://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736(15)60696-1.pdf

Much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness. As one participant put it, “poor methods get results”. The Academy of Medical Sciences, Medical Research Council, and Biotechnology and Biological Sciences Research Council have now put their reputational weight behind an investigation into these questionable research practices. The apparent endemicity [i.e. pervasiveness within the scientific culture] of bad research behaviour is alarming. In their quest for telling a compelling story, scientists too often sculpt data to fit their preferred theory of the world. Or they retrofit hypotheses to fit their data. Journal editors deserve their fair share of criticism too. We aid and abet the worst behaviours. Our acquiescence to the impact factor fuels an unhealthy competition to win a place in a select few journals. Our love of “significance” pollutes the literature with many a statistical fairy-tale….

Similarly, the editor in chief of the New England Journal of Medicine, Dr. Marcia Angell, wrote in 2009:

“We can no longer trust Drug company clinical trials”
(Original Article I quote below has vanished… imagine that!)

….It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine….

In fact, in October 2017, Angell went further – she published an article in the New York Review of books entitled, ‘Drug Companies and Doctors: A Story of Corruption.’ 

Drug Companies & Doctors: A Story of Corruption

In her must-read essay, Dr. Angell skewers drug companies, university medical departments, and medical groups which set the criteria for diagnosis and treatment as being rotten with corruption and conflicts of interest…

Of course this is the same thing that Dr Fang was saying more recently.