Co-reviewing and ghostwriting by early career researchers in the peer review of manuscripts, bioRxiv, 2019-04-27
AbstractThe goal of this study is to shed light on the involvement of early career researchers (ECRs) during peer review of manuscripts for publication in journals. In particular, we sought to better understand how commonly ECRs contribute ideas andor text to peer review reports when they are not the invited reviewer (“co-review”), and how commonly ECRs do not receive named credit to the journal editorial staff for these scholarly efforts (“ghostwrite”). First, we evaluated 1,952 publications in the peer-reviewed literature generated by exhaustive search terms that combined synonyms of “early career researcher” and “peer review” and found no previous studies about ECRs ghostwriting peer review reports. We then surveyed 498 researchers about their experiences with, and opinions about, co-reviewing and ghostwriting as ECRs. Three quarters of those surveyed have co-reviewed and most find it to be a beneficial (95% agree) and ethical (73% agree) form of training in peer review. Co-reviewing is the second most commonly reported form of training in peer review besides receiving reviews on one’s own papers. Half of survey respondents have ghostwritten a peer review report, despite the 45ths majority opinion that ghostwriting is unethical. Survey respondents report that the three major barriers to including co-reviewer names on peer review reports are a lack of communication between PIs and ECRs; a false belief that co-authorship is for manuscripts but not peer review reports; and prohibitive journal policies that are out of alignment with current practice and opinions about best practice. We therefore propose recommendations for changing this status quo, to discourage unethical ghostwriting of peer review reports and encourage quality co-reviewing experiences as normal training in peer review.
biorxiv scientific-communication-and-education 100-200-users 2019Excess significance bias in repetitive transcranial magnetic stimulation literature for neuropsychiatric disorders, bioRxiv, 2019-04-23
ABSTRACTIntroductionRepetitive transcranial magnetic stimulation (rTMS) has been widely tested and promoted for use in multiple neuropsychiatric conditions, but as for many other medical devices, some gaps may exist in the literature and the evidence base for rTMS clinical efficacy remains under debate. We aimed to empirically test for an excess number of statistically significant results in the literature on rTMS therapeutic efficacy across a wide range of meta-analyses and to characterize the power of studies included in these meta-analyses.MethodsBased on power calculations, we computed the expected number of “positive” datasets for a medium effect-size (standardized mean difference, SMD=0.30) and compared it with the number of observed “positive” datasets. Sensitivity analyses considered small (SMD=0.20), modest (SMD=0.50), and large (SMD=0.80) effect sizes.Results14 meta-analyses with 228 datasets (110 for neurological disorders and 118 for psychiatric disorders) were assessed. For SMD=0.3, the number of observed “positive” studies (n=94) was larger than expected (n=35). We found evidence for an excess of significant findings overall (p<0.0001) and in 814 meta-analyses. Evidence for an excess of significant findings was also observed for SMD=0.5 for neurological disorders. 0 (0 %), 0 (0 %), 3 (1 %), and 53 (23 %) of the 228 datasets had power >0.80, respectively for SMDs of 0.30, 0.20, 0.50, and 0.80.ConclusionMost studies in the rTMS literature are underpowered. This results in fragmentation and waste of research efforts. The somewhat high frequency of “positive” results seems spurious and may reflect bias.Trial Registration PROSPERO 2017 CRD42017056694
biorxiv scientific-communication-and-education 0-100-users 2019Breaking barriers The effect of protected characteristics and their intersectionality on career transition in academics, bioRxiv, 2019-04-02
Lay summaryIn the past decade the scientific community has been trying to tackle the historical underrepresentation of women in science and the fact that gender can constitute a barrier to career success. However, other characteristics, such as being of an ethnic minority or coming from an under-privileged background, have received less attention. In this study we find that ethnicity and socioeconomic status impact detrimentally on career progression in early career scientists, despite the fact that gender is more likely to be reported as a barrier. Our data suggest we need to widen the discussion regarding diversity and equality in science to incorporate potential barriers to career success in addition to gender.AbstractThe academic disciplines of Science Technology Engineering and Mathematics (STEM) have long suffered from a lack of diversity. While in recent years there has been some progress in addressing the underrepresentation of women in STEM subjects, other protected characteristics have received less attention. In this study, we survey early career scientists in the fields of ecology, evolutionary biology, behaviour, and related disciplines. We (i) quantitatively examine the effect of protected characteristics and their intersectionality on career transition, and (ii) provide practical suggestions, based on the qualitative responses of those surveyed, for overcoming some of the barriers we identified. We found that socioeconomic background and ethnicity impacted negatively on the quantitative measures of career progression we examined. Respondents that were female, LGBT, and from a lower socioeconomic background were more likely to report having faced a barrier, and the most frequent barrier named was related to gender. Our results suggest that respondents may have felt more confident discussing the experiences they have had related to their gender, potentially because there is now widespread discourse on this subject. However, respondents were less likely to discuss barriers they have faced in relation to ethnicity and socioeconomic status, despite the fact that the data indicates these are more detrimental to career progression. This may reflect the fact that these characteristics have received less attention, and are therefore deemed more sensitive. We hope that this study will stimulate wider discussion, and help to inform strategies to address the underrepresentation of minority groups in STEM subjects.
biorxiv scientific-communication-and-education 200-500-users 2019PlotsOfDifferences – a web app for the quantitative comparison of unpaired data, bioRxiv, 2019-03-18
AbstractThe quantitative comparison of data acquired under different conditions is an important aspect of experimental science. The most widely used statistic for quantitative comparisons is the p-value. However, p-values suffer from several shortcomings. The most prominent shortcoming that is relevant for quantitative comparisons is that p-values fail to convey the magnitude of differences. The differences between conditions are best quantified by the determination of effect size. To democratize the calculation of effect size, we have developed a web-based tool. The tool uses bootstrapping to resample mean or median values for each of the conditions and these values are used to calculate the effect size and their compatibility interval. The web tool generates a graphical output, showing the bootstrap distribution of the difference next to the actual data for optimal interpretation. A tabular output with statistics and effect sizes is also generated and the table can be supplemented with p-values that are calculated with a randomization test. The app that we report here is dubbed PlotsOfDifferences and is available at <jatsext-link xmlnsxlink=httpwww.w3.org1999xlink ext-link-type=uri xlinkhref=httpshuygens.science.uva.nlPlotsOfDifferences>httpshuygens.science.uva.nlPlotsOfDifferences<jatsext-link><jatsfig id=ufig1 position=float fig-type=figure orientation=portrait><jatsgraphic xmlnsxlink=httpwww.w3.org1999xlink xlinkhref=578575_ufig1 position=float orientation=portrait >
biorxiv scientific-communication-and-education 200-500-users 2019The life of P.I. Transitions to Independence in Academia, bioRxiv, 2019-03-11
The data in this report summarises the responses gathered from 365 principal investigators of academic laboratories, who started their independent positions in the UK within the last 6 years up to 2018. We find that too many new investigators express frustration and poor optimism for the future. These data also reveal, that many of these individuals lack the support required to make a successful transition to independence and that simple measures could be put in place by both funders and universities in order to better support these early career researchers. We use these data to make both recommendations of good practice and for changes to policies that would make significant improvements to those currently finding independence challenging. We find that some new investigators face significant obstacles when building momentum and hiring a research team. In particular, access to PhD students. We also find some important areas such as starting salaries where significant gender differences persist, which cannot be explained by seniority. Our data also underlines the importance of support networks, within and outside the department, and the positive influence of good mentorship through this difficult career stage.
biorxiv scientific-communication-and-education 500+-users 2019Where do our graduates go? A toolkit for retrospective and ongoing career outcomes data collection for biomedical PhD students and postdoctoral scholars, bioRxiv, 2019-02-09
Universities are at long last undertaking efforts to collect and disseminate information about student career outcomes, after decades of calls to action. Organizations such as Rescuing Biomedical Research and Future of Research brought this issue to the forefront of graduate education, and the second Future of Biomedical Graduate and Postdoctoral Training conference (FOBGAPT2) featured the collection of career outcomes data in its final recommendations, published in this journal (Hitchcock et al., 2017). More recently, 26 institutions assembled as the Coalition for Next Generation Life Science, committing to ongoing collection and dissemination of career data for both graduate and postdoc alumni. A few individual institutions have shared snapshots of the data in peer-reviewed publications (Mathur et al., 2018; Silva, des Jarlais, Lindstaedt, Rotman, Watkins, 2016) and on websites. As more and more institutions take up this call to action, they will now be looking for tools, protocols, and best practices for ongoing career outcomes data collection, management, and dissemination. Here, we describe UCSF's experiences in conducting a retrospective study, and in institutionalizing a methodology for annual data collection and dissemination. We describe and share all tools we have developed, and we provide calculations of the time and resources required to accomplish both retrospective studies and annual updates. We also include broader recommendations for implementation at your own institutions, increasing the feasibility of this endeavor.
biorxiv scientific-communication-and-education 100-200-users 2019