Course Peer Review Research Metrics How to Evalaute Information

Can metrics supervene upon peer review in indicating the quality & impact of enquiry?

---

The metric tide is certainly rise.

James Wilsdon, Professor of scientific discipline and democracy at the Academy of Sussex

Mapping the quality and impact of scientific inquiry has been a concern of all major stakeholders of research and development. The ii near prominent methods of evaluating research are peer review and quantitative metrics. The peer review procedure, wherein experienced researchers critique a manuscript, has been held in high esteem and is considered to add together not bad value to the quality of enquiry. Quantitative analyses of published research such equally the touch on factor provide a metrics-based cess of inquiry. Thus, information technology is articulate that while peer review provides a qualitative cess, metrics provide quantitative indicators of bear on. Over the years, with the growing pressure for evaluation of public spending on higher instruction and research, evaluation metrics are being attached more than importance than ever before. Metrics play a peculiarly vital role in processes of enquiry assessment such equally the Research Excellence Framework (REF) where quantitative indicators are used in conjunction with peer review to appraise the impact of research.

To review the role metrics play in determining the quality and impact of inquiry, the Higher Education Funding Council for England (HEFCE), which drives REF, set up up a committee in 2014 with the intent of improving the means in which funders and policy makers apply quantitative indicators for the next inquiry assessment exercise. The Independent Review of the Role of Metrics in Research Assessment and Management released a study in July 2015 titled The Metric Tide, chaired past Professor James Wilsdon from the Science Policy Research Unit (SPRU), University of Sussex, and was supported past experts in scientometrics, research funding, research policy, publishing, academy management and administration. This report explores in detail the potential uses and limitations of enquiry metrics and indicators, and sheds light on some of import problems such as: 'Can metrics replace peer review?' and 'Can a balance be struck between peer review and metric-based alternatives?'

Scientometric information helps in indicating the progression of knowledge. All the same, the reason metrics such as the journal impact factor and h-indices accept garnered criticism is that they fail to accept into account the effectively qualitative threads of inquiry that cannot be quantified using metrics. About of the new tools of research cess – including alternative metrics and online platforms and tools – are in early on stages of evolution. Hence, the written report suggests that they should non be overly relied upon without a deeper understanding nearly the way the information is collected and analyzed. According to Wilsdon et al., "Testify of a robust relationship between newer metrics and research quality remains very limited, and more experimentation is needed." They farther add that, "The majority of those who submitted evidence, or engaged with the review in other ways, are sceptical virtually any moves to increase the role of metrics in research management."

To throw light on whether metrics tin supervene upon peer review, the report looked at the applicability of metrics within different research cultures, compared the peer review system with metric-based alternatives, and considered what balance might be struck between the ii. The peer review system is not free of pitfalls, merely "information technology is the to the lowest degree worst class of academic governance we have, and should remain the primary basis for assessing inquiry papers, proposals and individuals, and for national cess exercises like the REF," states the study. Bulk of the respondents that helped compile the report (26 responses, 13 of which were from learned societies supported peer review) were of the opinion that "metrics must non exist seen as a substitute for peer review" and should "go along to be the 'gold standard' for inquiry assessment." It was likewise suggested that metrics could support peer reviewers in "making nuanced judgements about research excellence." Thus, peer review continues to exist favored by experts across disciplines.

The evidence collected by the report suggests that the HEFCE'southward 2008 pilot projection to explore the potential for bibliometrics found that bibliometrics were not sufficiently robust at that point in fourth dimension to replace peer review in the REF. Hence, the report recommends that peer review should not be replaced by metrics in the next REF assessment.

1 of the main ideas The Metric Tide report propagates is that "Metrics should support, not supplant, skillful judgement." It introduces the idea of 'informed peer review' wherein specific bibliometric data would supplement peer review, depending on the goal and context of assessment, for an all-circular evaluation.

The dominant thought of the report is that the future of research evaluation would need to accept a more mature research system that is founded on a combination of advanced metrics and informed peer review. Wilsdon et al. sum this up as "Peradventure a new body of 'translational bibliometrics' literature to flesh out the concept of informed peer review will emerge from these initiatives."

For an overview of The Metric Tide report, click Responsible metrics can change the futurity of enquiry evaluation: The Metric Tide report

Be the offset to clap

for this article

Published on: Jul 23, 2015

edwardscarded.blogspot.com

Source: https://www.editage.com/insights/can-metrics-replace-peer-review-in-indicating-the-quality-impact-of-research

Share this post