Alum is architect of influential science-watchdog site
After three years, Brandon Stell reveals role in establishing controversial website
The mastermind behind a formerly anonymously run website that serves as a crowd-sourced watchdog of peer-reviewed scientific journal articles turns out to be Brandon Stell, a 1997 graduate of the University of Colorado Boulder.
Stell recently disclosed that he is the architect of , the controversial online public forum where scientists can anonymously critique each other’s published research. The site’s users have exposed many errors in published works.
While a student at CU-Boulder, Brandon Stell joined discussions of published, peer-reviewed research. He noted a high frequency of errors or other problems in the work, and he thought it was a shame that such feedback could not be shared widely. Eventually, this experience helped him hatch the idea for what became PubPeer. Photo courtesy of Brandon Stell.
The idea for PubPeer originated when Stell was an undergraduate at CU-Boulder studying ecology, populations and organismal biology. Stell regularly attended meetings to discuss recently published research.
The discussion groups dissected papers, scrutinizing each experiment and conclusion. This in-depth review of peer-reviewed research always revealed a few flaws.
“I’ve never been to a journal club meeting that didn’t turn up one or two issues about a paper,” says Stell.
Stell enjoyed the occasionally brutal discussions, but he left the meetings disappointed. The insights, remaining questions and possible flaws regarding the research hung in the air with no resolution.
“I always thought that the authors could maybe clarify an issue if they had a voice at the journal club, but they didn’t. And, when issues were brought up in the meeting, no other scientists in that field ever heard about it,” says Stell.
“That is really important because scientists build off of each other’s work.”
We thought, if we could create this community where people are discussing science that will take the focus off of where science is published and put it on the content or the data that we’re publishing.”
Stell longed for a centralized, public forum for scientists to discuss and critique each other’s published research.
The idea that grew into PubPeer percolated for years. Throughout his graduate studies in neuroscience at the University of California Los Angeles and a post-doctoral fellowship in Paris, Stell ranted about this idea in hallways, perpetually asking colleagues why someone else hadn’t already launched such a forum.
He even contacted Google Scholar, a specialized search engine for academic and scholarly literature, hoping to encourage it to establish such a site. His email went unanswered.
He caught a break when Richard Smith came to work in his lab in Paris. Richard and his brother George, who provides technical expertise, helped Stell launch the site in 2012. (PubPeer has revealed the brothers’ names but not their professional affiliations.)
The site published anonymous critiques for an obvious reason: Criticism is rarely complimentary. And because peers in the same area of expertise review one’s research, any critique carries the risk of offending a colleague. This could have significant consequences for the critic’s career.
“If some of my colleagues’ papers were critiqued, people who were reviewing my grants and my papers and basically deciding my career, if their papers were being critiqued on the site and they knew that I was behind the site, they probably wouldn’t be very happy about that,” Stell explains.
“It would also put pressure on me to remove comments that people were unhappy with. So, we were anonymous, and because we were anonymous, it made it difficult to promote the site. You can’t really do that anonymously.”
Despite the anonymity and lack of promotion, the site gained popularity. In fact, once anonymous commenting was enabled, the site’s use more than tripled in less than two years (). One success of PubPeer is that it allows other scientists in a field, who are building on that published research, to learn about other researchers’ critiques and concerns.
“When there are problems that are noticed, then other scientists [in that field] are aware of itimmediately because there’s now this forum,” explains Stell.
Stell and colleagues saw that this was going to have broader implications.
“If scientists don’t know that there are problems with a research article, this has major cost,” says Stell.
“There’s an economic cost – we’re wasting taxpayer money trying to reproduce or build on [research] that can’t be built upon. It also has a human cost because graduate students and post-docs could be wasting their careers trying to build on something that can’t be built upon. If a student spends three years trying to build on something that’s not very solid, and in the end can’t do it, it makes it very difficult for their scientific career.”
With a system like PubPeer, as soon as there’s a problem with a paper, people who use the research as a foundation for future initiatives, including policymakers, health-care providers, and pharmaceutical companies know about it.
Additionally, PubPeer provides an avenue for focusing scientists’ attention on the fundamental issue: science.
“We thought, if we could create this community where people are discussing science that will take the focus off of where science is published and put it on the content or the data that we’re publishing,” explains Stell.
“I feel that scientists are on the edge of an interesting moment in science where we’re collectively questioning the role of journals.”
“Publication is not the end of the road,” according to Stell. “It’s not, ‘OK I’ve been through the review and publication process, the paper’s now in a coveted research journal, I can cash it in for jobs, or grant money.’”
“If the data aren’t solid, there’s a conversation that’s going to happen about that.”
Roni Dengler is a writer for CU’s Science Buffs blog and a Ph.D. candidate in the Molecular, Cellular and Developmental Biology Department.