“When You Use Social Media You Are Not Working”: Barriers for the Use of Metrics in Social Sciences

Authors: Lemke Steffen, Mehrazar Maryam, Mazarakis Athanasios, Peters Isabella

Abstract: The Social Sciences have long been struggling with quantitative forms of research assessment—insufficient coverage in prominent citation indices and overall lower citation counts than in STM subject areas have led to a widespread weariness regarding bibliometric evaluations among social scientists. Fueled by the rise of the social web, new hope is often placed on alternative metrics that measure the attention scholarly publications receive online, in particular on social media. But almost a decade after the coining of the term “altmetrics” for this new group of indicators, the uptake of the concept in the Social Sciences still seems to be low. Just like with traditional bibliometric indicators, one central problem hindering the applicability of altmetrics for the Social Sciences is the low coverage of social science publications on the respective data sources—which in the case of altmetrics are the various social media platforms on which interactions with scientific outputs can be measured. Another reason is that social scientists have strong opinions about the usefulness of metrics for research evaluation which may hinder broad acceptance of altmetrics too. We conducted qualitative interviews and online surveys with researchers to identify the concerns which inhibit the use of social media and the utilization of metrics for research evaluation in the Social Sciences. By analyzing the response data from the interviews in conjunction with the response data from the surveys, we identify the key concerns that inhibit social scientists from (1) applying social media for professional purposes and (2) making use of the wide array of metrics available. Our findings show that aspects of time consumption, privacy, dealing with information overload, and prevalent styles of communication are predominant concerns inhibiting Social Science researchers from using social media platforms for their work. Regarding indicators for research impact we identify a widespread lack of knowledge about existing metrics, their methodologies and meanings as a major hindrance for their uptake through social scientists. The results have implications for future developments of scholarly online tools and show that researchers could benefit considerably from additional formal training regarding the correct application and interpretation of metrics.V

Citation:   Lemke S, Mehrazar M, Mazarakis A and Peters I (2019) “When You Use Social Media You Are Not Working”: Barriers for the Use of Metrics in Social Sciences. Front. Res. Metr. Anal. 3:39. doi: 10.3389/frma.2018.00039

VIEW

 

Source: Frontiers in Research Metrics and Analytics

A multi-disciplinary perspective on emergent and future innovations in peer review

Authors: Jonathan P Tennant, Jonathan M Dugan, Daniel Graziotin, Damien C Jacques, François Waldner, Daniel Mietchen, Yehia Elkhatib, Lauren B Collister, Christina K Pikas, Tom Crick, Paola Masuzzo, Anthony Caravaggi, Devin R Berg, Kyle E Niemeyer, Tony Ross-Hellauer, Sara Mannheimer, Lillian Rigling, Daniel S Katz, Bastian Greshake Tzovaras, Josmel Pacheco-Mendoza, Nazeefa Fatima, Marta Poblet, Marios Isaakidis, Dasapta Erwin Irawan, Sébastien Renaut, Christopher R Madan, Lisa Matthias, Jesper Nørgaard Kjær, Daniel Paul O’Donnell, Cameron Neylon, Sarah Kearns, Manojkumar Selvaraju, Julien Colomb

Abstract: Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform and reduce the biases of existing models as much as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that could, at least partially, resolve many of the socio-technical issues associated with peer review, and potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

Citation: Tennant JP, Dugan JM, Graziotin D et al. A multi-disciplinary perspective on emergent and future innovations in peer review [version 3; referees: 2 approved]. F1000Research 2017, 6:1151
(https://doi.org/10.12688/f1000research.12037.3)

View

Source: F1000 Research

Can We Count on Social Media Metrics? First Insights into the Active Scholarly Use of Social Media


Authors: Maryam Mehrazar, Christoph Carl Kling, Steffen Lemke, Athanasios Mazarakis, Isabella Peters

Abstract: Measuring research impact is important for ranking publications in academic search engines and for research evaluation. Social media metrics or altmetrics measure the impact of scientific work based on social media activity. Altmetrics are complementary to traditional, citation-based metrics, e.g. allowing the assessment of new publications for which citations are not yet available. Despite the increasing importance of altmetrics, their characteristics are not well understood: Until now it has not been researched what kind of researchers are actively using which social media services and why – important questions for scientific impact prediction. Based on a survey among 3,430 scientists, we uncover previously unknown and significant differences between social media services: We identify services which attract young and experienced researchers, respectively, and detect differences in usage motivations. Our findings have direct implications for the future design of altmetrics for scientific impact prediction.

Citation: Mehrazar, M. et al., (2018). Can We Count on Social Media Metrics? First Insights into the Active Scholarly Use of Social Media. ArXiv. https://arxiv.org/abs/1804.02751.

View

Source: ArXiv

What do computer scientists tweet? Analyzing the link-sharing practice on Twitter


Authors: Marco Schmitt, Robert Jäschke

Abstract: Twitter communication has permeated every sphere of society. To highlight and share small pieces of information with possibly vast audiences or small circles of the interested has some value in almost any aspect of social life. But what is the value exactly for a scientific field? We perform a comprehensive study of computer scientists using Twitter and their tweeting behavior concerning the sharing of web links. Discerning the domains, hosts and individual web pages being tweeted and the differences between computer scientists and a Twitter sample enables us to look in depth at the Twitter-based information sharing practices of a scientific community. Additionally, we aim at providing a deeper understanding of the role and impact of altmetrics in computer science and give a glance at the publications mentioned on Twitter that are most relevant for the computer science community. Our results show a link sharing culture that concentrates more heavily on public and professional quality information than the Twitter sample does. The results also show a broad variety in linked sources and especially in linked publications with some publications clearly related to community-specific interests of computer scientists, while others with a strong relation to attention mechanisms in social media. This refers to the observation that Twitter is a hybrid form of social media between an information service and a social network service. Overall the computer scientists’ style of usage seems to be more on the information-oriented side and to some degree also on professional usage. Therefore, altmetrics are of considerable use in analyzing computer science.

Citation: Schmitt M, Jäschke R. (2017). What do computer scientists tweet? Analyzing the link-sharing practice on Twitter. PLOS ONE 12(6): e0179630. https://doi.org/10.1371/journal.pone.0179630

Data Availability: All data are available from the Zenodo repository (DOI: 10.5281/zenodo.580587)

View

Geographic variation in social media metrics: an analysis of Latin American journal articles


Author: Juan Pablo Alperin

Abstract:
Purpose: This study aims to contribute to the understanding of how the potential of altmetrics varies around the world by measuring the percentage of articles with non-zero metrics (coverage) for articles published from a developing region (Latin America).

Design/methodology/approach: This study uses article metadata from a prominent Latin American journal portal, SciELO, and combines it with altmetrics data from Altmetric.com and with data collected by author-written scripts. The study is primarily descriptive, focusing on coverage levels disaggregated by year, country, subject area, and language.

Findings: Coverage levels for most of the social media sources studied was zero or negligible. Only three metrics had coverage levels above 2%—Mendeley, Twitter, and Facebook. Of these, Twitter showed the most significant differences with previous studies. Mendeley coverage levels reach those found by previous studies, but it takes up to two years longer for articles to be saved in the reference manager. For the most recent year, coverage was less than half than what was found in previous studies. The coverage levels of Facebook appear similar (around 3%) to that of previous studies.

Research limitations/implications: The Altmetric.com data used for some of the analyses was collected for a six month period. For other analyses, Altmetric.com data was only available for a single country (Brazil).

Originality/value: The results of this study have implications for the altmetrics research community and for any stakeholders interested in using altmetrics for evaluation. It suggests the need of careful sample selection when wishing to make generalizable claims about altmetrics.

Citation: Juan Pablo Alperin, (2015) “Geographic variation in social media metrics: an analysis of Latin American journal articles”, Aslib Journal of Information Management, Vol. 67 Issue: 3, pp.289-304, doi: 10.1108/AJIM-12-2014-0176

View

“Facebook for Academics”: The Convergence of Self-Branding and Social Media Logic on Academia.edu


Authors: Brooke Erin Duffy, Jefferson D. Pooley

Abstract: Given widespread labor market precarity, contemporary workers—especially those in the media and creative industries—are increasingly called upon to brand themselves. Academics, we contend, are experiencing a parallel pressure to engage in self-promotional practices, particularly as universities become progressively more market-driven. Academia.edu, a paper-sharing social network that has been informally dubbed “Facebook for academics,” has grown rapidly by adopting many of the conventions of popular social media sites. This article argues that the astonishing uptake of Academia.edu both reflects and amplifies the self-branding imperatives that many academics experience. Drawing on Academia.edu’s corporate history, design decisions, and marketing communications, we analyze two overlapping facets of Academia.edu: (1) the site’s business model and (2) its social affordances. We contend that the company, like mainstream social networks, harnesses the content and immaterial labor of users under the guise of “sharing.” In addition, the site’s fixation on analytics reinforces a culture of incessant self-monitoring—one already encouraged by university policies to measure quantifiable impact. We conclude by identifying the stakes for academic life, when entrepreneurial and self-promotional demands brush up against the university’s knowledge-making ideals.

Citation: Brooke Erin Duffy and Jefferson D. Pooley. (January 2017). “Facebook for Academics”: The Convergence of Self-Branding and Social Media Logic on Academia.edu. Commons Open Repository Exchange. http://dx.doi.org/10.17613/M6CD2F

View

Academic Librarians as Knowledge Creators


Author: Donna Witek

Abstract: Despite support from national organizations like the Association of College and Research Libraries (ACRL) and the American Association of University Professors (AAUP), pursuing research and scholarship remains a challenge for academic librarians, even when the literature connects these activities to greater effectiveness in the practice of academic librarianship. This essay examines the history and present state of the questions of faculty status and tenure for librarians, and relates these questions to that of performing scholarly research and creating and disseminating new knowledge as an academic librarian. It then offers as a case study my experience identifying and pursuing a research agenda in collaboration with a faculty colleague in another department at my institution, with the goal of both sharing what has worked for one academic librarian (n=1) while also critiquing the system within which that success has occurred. The essay concludes with a list of creative strategies academic librarians can put into practice to become successful knowledge creators in the field of library and information science.

Witek D. (2014). Academic Librarians as Knowledge Creators. The Journal of Creative Library Practice.

View

Data Curation and the Arts: How Do Musicians Curate Their Data?


Authors: Amy S. Jackson, Jonathan Wheeler, Todd Quinn

 

Abstract: Professional musicians were surveyed to determine how personal, amateur recordings of performances are shared with students and colleagues. Sharing files on social media is common, with Facebook, YouTube, and Vimeo used most frequently. Although these are popular social media platforms, they do not have enhanced format support and robust metadata. Additionally, licensing terms for each platform differ, and may be not in the best interest of the musician. Although recordings are not traditionally considered data, data curation principles can be applied to these types of files, and the library is positioned to become an active participant in this process.

 

Citation: Jackson, A. S., Wheeler, J., & Quinn, T. (2016). Data Curation and the Arts: How Do Musicians Curate Their Data? Music Reference Services Quarterly, 19(3–4), 191–207. http://digitalrepository.unm.edu/ulls_fsp/115/

 

VIEW

 

Scholarly use of social media and altmetrics: a review of the literature


Authors: Sugimoto C, Work S, Larivière V, Haustein S

Abstract: Social media has become integrated into the fabric of the scholarly communication system in fundamental ways: principally through scholarly use of social media platforms and the promotion of new indicators on the basis of interactions with these platforms. Research and scholarship in this area has accelerated since the coining and subsequent advocacy for altmetrics — that is, research indicators based on social media activity. This review provides an extensive account of the state-of-the art in both scholarly use of social media and altmetrics. The review consists of two main parts: the first examines the use of social media in academia, examining the various functions these platforms have in the scholarly communication process and the factors that affect this use. The second part reviews empirical studies of altmetrics, discussing the various interpretations of altmetrics, data collection and methodological limitations, and differences according to platform. The review ends with a critical discussion of the implications of this transformation in the scholarly communication system.

Citation: Sugimoto C, Work S, Larivière V, Haustein S. (2016). Scholarly use of social media and altmetrics: a review of the literature. JASIST

View