14 March 2014 in Research Evaluation
Because SCImago Journal & Country Rank is increasingly considered for its use with evaluative purposes, it is necessary to have a qualitative and quantitative analysis from a theoretical and practical perspective. The aim of this article is the analysis of the advantages and limitations of SCImago Journal & Country Rank. Some weaknesses from the perspective of its use for research evaluation purposes are identified and detailed, such as the absence of construct definition, lack of coherence between statements of its developers and observable features in data, gaps in the coverage of journals and its consequences for comparative ends, issues related to the comparability of different citation networks, lack of a trustworthy ordinal position of journals in the output ranking, retrospective backups in the underlying data and its stability over time, methodological issues in quartile construction, comparatively low discriminative capacity of the indicator, issues related to parameter-fixing procedures and supposed manipulation resistance, degree of transparency and reproducibility of the results and errors in country affiliation of documents in Scopus and its consequences for the Country Ranks. It is concluded that, although the idea behind SCImago Journal Rank is laudable and the indicator is a sophisticated alternative to other quality indicators, SCImago Journal & Country Rank omits a large amount of information, putting into question its transparency, reliability, and suitability for evaluative purposes in its current form, although most of the identified problems can be solved and might be the object of future improvements.
21 September 2017 in Computer Assisted Language Learning
15 February 2016 in Scientometrics
11 December 2012 in Research Evaluation
Books are a key channel in the scientific communication processes for researchers in Social Sciences and Humanities. Although the best way of assessing the monographs would involve its individualized reading, this method is practically unfeasible both in terms of cost and time. An indicator which could be used is the prestige of the publisher of monographs and it is essential to take into account the academy’s opinion to determine it. This article reports a survey launched to 11,647 lecturers and Spanish researchers, with a response rate of 26%, in which the respondents were asked to point those quality indicators they consider the most relevant for books as well as their opinion on the quality of publishers. As the main results, there are reported the top-rated publishers, as well as the characteristics which, according to the respondents’ opinion, determine their quality; among these characteristics, stands out the fact of counting with an assessment system with external reviewers. Regarding the list of publishers, it is also explained the indicator which allows its ranking. It is also defended the use of the results in the form of categorization.
1 April 2017 in Research Evaluation
In the past decade, a number of initiatives have been taken to provide new sources of information on scholarly book publishing. Thomson Reuters (now Clarivate Analytics) has supplemented the Web of Science with a Book Citation Index (BCI), while Elsevier has extended Scopus to include books from a selection of scholarly publishers. More complete metadata on scholarly book publishing can be derived at the national level from non-commercial databases such as Current Research Information System in Norway and the VIRTA (Higher Education Achievement Register, Finland) publication information service, including the Finnish Publication Forum (JUFO) lists (Finland). The Spanish Scholarly Publishers Indicators provides survey-based information on the prestige, specialization profiles from metadata, and manuscript selection processes of national and international publishers that are particularly relevant for the social sciences and humanities (SSH). In the present work, the five information sources mentioned above are compared in a quantitative analysis identifying overlaps and uniqueness as well as differences in the degrees and profiles of coverage. In a second-stage analysis, the geographical origin of the university presses (UPs) is given a particular focus. We find that selection criteria strongly differ, ranging from a set of a priori criteria combined with expert-panel review in the case of commercial databases to in principle comprehensive coverage within a definition in the Nordic countries and an open survey methodology combined with metadata from the book industry database and questionnaires to publishers in Spain. Larger sets of distinct book publishers are found in the non-commercial databases, and greater geographical diversity is observable among the UPs in these information systems. While a more locally oriented set of publishers which are relevant to researchers in the SSH is present in non-commercial databases, the commercial databases seem to focus on highly selective procedures by which the coverage concentrates on prestigious international publishers, mainly based in the USA or UK and serving the natural sciences, engineering, and medicine.