26 August 2019 in Minerva
Notions of research quality are contextual in many respects: they vary between fields of research, between review contexts and between policy contexts. Yet, the role of these co-existing notions in research, and in research policy, is poorly understood. In this paper we offer a novel framework to study and understand research quality across three key dimensions. First, we distinguish between quality notions that originate in research fields (Field-type) and in research policy spaces (Space-type). Second, drawing on existing studies, we identify three attributes (often) considered important for ‘good research’: its originality/novelty, plausibility/reliability, and value or usefulness. Third, we identify five different sites where notions of research quality emerge, are contested and institutionalised: researchers themselves, knowledge communities, research organisations, funding agencies and national policy arenas. We argue that the framework helps us understand processes and mechanisms through which ‘good research’ is recognised as well as tensions arising from the co-existence of (potentially) conflicting quality notions.
29 June 2020 in Research Evaluation
The current range and volume of research evaluation-related literature is extensive and incorporates scholarly and policy/practice-related perspectives. This reflects academic and practical interest over many decades and trails the changing funding and reputational modalities for universities, namely increased selectivity applied to institutional research funding streams and the perceived importance of university rankings and other reputational devices. To make sense of this highly diverse body of literature, we undertake a critical review of over 350 works constituting, in our view, the ‘state-of-the-art’ on institutional performance-based research evaluation arrangements (PREAs). We focus on PREAs because they are becoming the predominant means world-wide to allocate research funds and accrue reputation for universities. We highlight the themes addressed in the literature and offer critical commentary on the balance of scholarly and policy/practice-related orientations. We then reflect on five limitations to the state-of-the-art and propose a new agenda, and a change of perspective, to progress this area of research in future studies.
3 October 2018 in Studies in Higher Education
From 2012 USA universities entered new partnerships with private sector companies including Silicon Valley start-up Coursera. Coursera spearheads a new broad online learning segment of the fast growing global ‘educational technology’ (EdTech) sector. They offered free ‘massive open online courses’ (MOOCs) for global, universal learner audiences. Since 2015 several USA universities and Coursera expanded into ‘post-MOOC’, paid, accredited online modules and full degrees. We frame these post-MOOC developments as shaped by dynamic EdTech/university relationships and argue universities have been actively, and willingly, re-shaping higher education with EdTech; they are not passive victims of a potentially disruptive global ‘MOOC phenomenon’. Our argument builds on interviews at six highly committed USA universities and at Coursera. These reveal rationales for post-MOOC developments related to: actions and attitudes of university actors; university resources; differing teaching subject areas; and exclusivity and longevity in relationships. We suggest that post-MOOC EdTech/university relationships are symbiotic, with three possible variants: commensal (neutral); mutualistic (positive); and parasitic (negative). We finally question whether current relationships may yet change from largely mutualistic to parasitic, given the apparent ambitions of Coursera and the wider global EdTech sector.
24 February 2016 in Minerva
11 September 2012 in Research Evaluation
We describe the development and testing of a framework to characterize researchers individually (a profile) and in aggregate (as types) at the first stage, baseline step of a controlled, two-stage study of public research funding agency impacts. Our framework characterizes researcher attitudes and attributes, and conditions and opportunities experienced, addressing: ‘demographic’ factors; researcher ‘approach’; and ‘standing’ (organizational career, knowledge community career, and local and national research environment aspects). This integrated demographic-approach-standing (DAS) framework is tested using a survey of 184 applicants to the inaugural, 2007 call for the European Research Council (ERC) Starting Grants (StG). Successful applicants are characterized to match-pair with a control group of quality-screened unsuccessful applicants. Given the inherent difficulty to identify in advance the ‘frontier’ researchers the ERC aims to fund with its StG, we characterize ‘frontier-potential’ factors that might lead researchers later to become regarded as frontier. We develop researcher types using several of our framework elements: researcher intellectual field mobility; novelty and risk-taking; independence; output productivity; and local research workplace standing. We find a variety of grantee types, but primarily not yet independent researchers whose ‘standing’ could be impacted upon by the early-career StG scheme. Lastly, we suggest impact pathways between types we could capture following a second stage survey, and discuss limitations to our framework including revising it to characterize better local research environment aspects.