“When You Use Social Media You Are Not Working”: Barriers for the Use of Metrics in Social Sciences

Authors: Lemke Steffen, Mehrazar Maryam, Mazarakis Athanasios, Peters Isabella

Abstract: The Social Sciences have long been struggling with quantitative forms of research assessment—insufficient coverage in prominent citation indices and overall lower citation counts than in STM subject areas have led to a widespread weariness regarding bibliometric evaluations among social scientists. Fueled by the rise of the social web, new hope is often placed on alternative metrics that measure the attention scholarly publications receive online, in particular on social media. But almost a decade after the coining of the term “altmetrics” for this new group of indicators, the uptake of the concept in the Social Sciences still seems to be low. Just like with traditional bibliometric indicators, one central problem hindering the applicability of altmetrics for the Social Sciences is the low coverage of social science publications on the respective data sources—which in the case of altmetrics are the various social media platforms on which interactions with scientific outputs can be measured. Another reason is that social scientists have strong opinions about the usefulness of metrics for research evaluation which may hinder broad acceptance of altmetrics too. We conducted qualitative interviews and online surveys with researchers to identify the concerns which inhibit the use of social media and the utilization of metrics for research evaluation in the Social Sciences. By analyzing the response data from the interviews in conjunction with the response data from the surveys, we identify the key concerns that inhibit social scientists from (1) applying social media for professional purposes and (2) making use of the wide array of metrics available. Our findings show that aspects of time consumption, privacy, dealing with information overload, and prevalent styles of communication are predominant concerns inhibiting Social Science researchers from using social media platforms for their work. Regarding indicators for research impact we identify a widespread lack of knowledge about existing metrics, their methodologies and meanings as a major hindrance for their uptake through social scientists. The results have implications for future developments of scholarly online tools and show that researchers could benefit considerably from additional formal training regarding the correct application and interpretation of metrics.V

Citation:   Lemke S, Mehrazar M, Mazarakis A and Peters I (2019) “When You Use Social Media You Are Not Working”: Barriers for the Use of Metrics in Social Sciences. Front. Res. Metr. Anal. 3:39. doi: 10.3389/frma.2018.00039



Source: Frontiers in Research Metrics and Analytics

Open access in ethics research: an analysis of open access availability and author self-archiving behaviour in light of journal copyright restrictions

Authors: Mikael Laakso, Andrea Polonioli

Abstract: The current state of open access to journal publications within research areas belonging to the humanities has received relatively little research attention. This study provides a detailed mapping of the bibliometric state of open access to journal publications among ethicists, taking into account not only open access publishing in journals directly, but also where and in what form ethicists make their journal articles available elsewhere on the web. As part of the study 297 ethicists affiliated with top-ranking philosophy departments were identified and their journal publication information for the years 2010–2015 were recorded (1682 unique articles). The journal articles were then queried for through Google Scholar in order to establish open access status (web locations, document versions) of each publication record. Publication records belonging to the 20 most frequently used journal outlets (subset of 597 unique articles) were put under closer inspection with regards to alignment with publisher copyright restrictions as well as measuring unused potential to share articles. The results show that slightly over half of recent journal publications are available to read for free. PhilPapers and academic social networks (Academia.edu and ResearchGate) were found to be key platforms for research dissemination in ethics research. The representation of institutional repositories as providers of access was found to be weak, receiving the second lowest frequency rating among the eight discrete web location categories. Further, the study reveals that ethicists are at the same time prone to copyright infringement and undersharing their scholarly work.

Citation: Laakso, M., & Polonioli, A. (2018) Open access in ethics research: an analysis of open access availability and author self-archiving behaviour in light of journal copyright restrictions. Scientometrics, 1-27. https://doi.org/10.1007/s11192-018-2751-5 


Source: Scientometrics

Reviewers are blinkered by bibliometrics

Authors: Paula Stephan, Reinhilde Veugelers& Jian Wang

Abstract: There is a disconnect between the research that reviewers purport to admire and the research that they actually support. As participants on multiple review panels and scientific councils, we have heard many lament researchers’ reluctance to take risks. Yet we’ve seen the same panels eschew risk and rely on bibliometric indicators for assessments, despite widespread agreement that they are imperfect measures1–6.

The review panels we observed last year were using bibliometrics in much the same way as they did before the 2015 Leiden Manifesto4, the 2012 San Francisco Declaration on Research Assessment, which Nature is signing, and similar exhortations against their use. After all, bibliometric measures offer a convenient way to help evaluate a large number of proposals and papers.

Although journal impact factors (JIFs) were developed to assess journals and say little about any individual paper, reviewers routinely justify their evaluations on the basis of where candidates have published. Panel members judge applicants by Google Scholar results and use citation counts to score proposals for new research. This practice prevails even at agencies such as the European Research Council (ERC), which instructs reviewers not to look up bibliometric measures.

As economists who study science and innovation, we see engrained processes working against cherished goals. Scientists we interview routinely say that they dare not propose bold projects for funding in part because of expectations that they will produce a steady stream of papers in journals with high impact scores. The situation may be worse than assumed. Our analysis of 15 years’ worth of citation data suggests that common bibliometric measures relying on short-term windows undervalue risky research7.

How can we move beyond declarations and wean reviewers off bibliometric indicators that bias decisions against bold work?

Citation: Paula Stephan, Reinhilde Veugelers& Jian Wang. Reviewers are blinkered by bibliometrics : Nature News & Comment. Nature 544, 411–412 (doi:10.1038/544411a


East African Social Sciences and Humanities Publishing: A Handmade Bibliometrics Approach

Author: Nora Schmidt

Abstract: For Eastern Africa, very little information about the SSH knowledge production can be found from a European perspective. Adequate indicators like information-rich bibliographic databases that cover journals and book publishers based in East Africa are lacking. This research in progress explores their indexing situation in detail, their development, which is closely connected to political history, their (non-)usage, and affiliations as well as career-stages of their authors. Furthermore, it also pays attention to SSH researchers based in East Africa who use other publication venues. Any bibliometric analysis in this field needs to rely on manual data collection, otherwise it would be heavily biased. This study lays out the foundation for citation analyses, qualitative research on the publications’ content and the self-description of East African scholars against the background of an academic environment that is often described as “international”.

Citation: Schmidt, Nora. (2016, October). East African Social Sciences and Humanities Publishing: A Handmade Bibliometrics Approach. Zenodo. http://doi.org/10.5281/zenodo.162217



The Impact of Gender on Citations: An Analysis of College & Research Libraries, Journal of Academic Librarianship, and Library Quarterly

We’re preceding this post with a short PSA. Today’s International Women’s Strike offers a good opportunity for those of us interested in LIS research and scholarly communication to reflect upon the persistent gender gap in our own field, both in professional practice and in our knowledge production practices. Though we comprise the majority of the profession, women are still paid less than their male peers and assume leadership positions at lower rates. We are also take on unpaid “service” work at higher rates–work like what we do here at The Idealis. That’s why we, too, are striking today, having scheduled this post a few weeks back.

In solidarity,
Stacy, Lily & Nicky
The Idealis

Author: Malin Håkanson

Abstract: Three scholarly core journals of library and information science (LIS) were analyzed with respect to gender of article authors and gender of authors cited in these articles. The share of female contributors to these journals has certainly increased during the studied period, 1980–2000. However, the results of the quantitative citation analysis show puzzling differences concerning female and male authors’ citation practice. There may be a gender bias in LIS publishing, even though female authors have become more numerous. Further studies are needed to uncover the influence of other variables, such as subject content of the articles.

Håkanson, M. (2005). The Impact of Gender on Citations: An Analysis of College & Research Libraries, Journal of Academic Librarianship, and Library Quarterly. College & Research Libraries, 66(4). doi:10.5860/crl.66.4.312



Scholarly Metrics Baseline: A Survey of Faculty Knowledge, Use, and Opinion about Scholarly Metrics

Authors: Dan DeSanto and Aaron Nichols

Abstract: This article presents the results of a faculty survey conducted at the University of Vermont during academic year 2014–2015. The survey asked faculty about: familiarity with scholarly metrics, metric-seeking habits, help-seeking habits, and the role of metrics in their department’s tenure and promotion process. The survey also gathered faculty opinions on how well scholarly metrics reflect the importance of scholarly work and how faculty feel about administrators gathering institutional scholarly metric information. Results point to the necessity of understanding the campus landscape of faculty knowledge, opinion, importance, and use of scholarly metrics before engaging faculty in further discussions about quantifying the impact of their scholarly work.

DeSanto D & Nichols A. (2017). Scholarly Metrics Baseline: A Survey of Faculty Knowledge, Use, and Opinion about Scholarly Metrics College & Research Libraries vol. 78 no. 2, pp 150-170 doi:10.5860/crl.78.2.150



La producción científica colombiana en SciELO: un análisis bibliométrico

Abstract: A bibliometric analysis of the Colombian scientific production indexed in SciELO Citation Index is presented. Some of the settled objectives are to determine the total production and the degree collaboration, to identify the most prolific Colombian universities as well as the main academic collaboration networks. The sample was integrated by 15302 documents published in twelve years.

A remarkable increase of the scientific production was found, with a global degree collaboration index of 0.75. It was found that the state universities are the most prolific ones, being the Universidad Nacional and the Universidad de Antioquia the first two producers. A growth of the international collaboration was detected, especially among Latin-American countries. Spain is the main academic collaborator of Colombia. It is shown a bias in pro of domestic journal publications. Health Sciences journals constitute the principal core of the Colombian scientific production.

Se presenta un análisis bibliométrico de la producción científica realizada en Colombia indexada en la base de datos SciELO Citation Index.
Algunos de los objetivos fueron determinar el volumen de la producción
y determinar el grado de colaboración, identificar las universidades colombianas más productivas y determinar cuáles son los pares
académicos en la investigación científica y tecnológica en Colombia. Se
analizaron 15 302 documentos publicados en doce años.

Se halló un notable incremento en la producción, con un grado de colaboración
global de 0,75. Las universidades públicas son las que más
publican, destacando la Universidad Nacional y la Universidad de Antioquia.
También se constató un aumento de la colaboración internacional,
especialmente con los países de la región latinoamericana. España
es el principal socio académico de Colombia.
Hay un sesgo en favor de las publicaciones realizadas en revistas nacionales.
Las Ciencias de la Salud se confirman como el mayor núcleo de la
producción científica en Colombia.

Maz-Machado, Alexander and Jiménez-Fanjul, Noelia Noemí and Villarraga-Rico, Miguel Ernesto. (2016). La producción científica colombiana en SciELO: un análisis bibliométrico. Revista Interamericana de Bibliotecología, vol. 39, n. 2, pp. 111-119.