Why don’t some publish PDFs written by marginalized scholars? Altmetrics.
Gerardo Muñoz’s graduate school professor did not post a PDF of the assigned author’s paper to Canvas, the classroom learning management system. The instructor explained that they read works by marginalized scholars and that these authors would benefit if students uploaded their work so that the resulting metrics could provide an indication of the impact of the scholarship. Muñoz thought that was remarkable, so he tweeted about it, and that Tweeter has since racked up nearly 50,000 likes and thousands of retweets.
Many replies to Muñoz’s tweet expressed support, including “BEAUTIFUL,” “Data is everything,” “Normalize THIS” and “Damn.” Others expressed concern about the professor’s politics, including some who wrote: “an amazing way to make sure your students don’t read said marginalized thinkers scholarship,” Where “Oh how I hate living in an algorithm driven society.”
At the heart of Muñoz’s Twitter storm are altmetrics, or alternative metrics, a relatively new measure of research impact that tracks how academic work is discussed, shared, read, and reused online. This may include the number of page views and downloads, shares and likes on social media, and mentions in online news, forums and policy documents.
Researchers have long relied on a variety of metrics to understand the relative value of their research. A journal’s impact factor, for example, is often considered to confer prestige on the journal’s authors. Similarly, an author’s h-index combines information about the author’s article count with the citation count to suggest a value. The authors also count citations as a stand-alone measure.
Proponents argue that altmetrics can complement traditional scholarly metrics in important ways, especially given the limitations of traditional scholarly metrics. Quotes, for example, often take years to accumulate. Journal impact factors do not provide granular information about individual articles and may rely on flawed statistical arguments, at least according to some.
In contrast, altmetrics arrive quickly, in real time, and provide insight into how research influences societal conversations, thinking, and behavior. These comments can be particularly meaningful for underrepresented and early-career researchers. But altmetrics aren’t a panacea, as they are susceptible to manipulation and don’t always measure quality.
“Downloads are important,” said Tia Madkins, an assistant professor at the University of Texas in the Department of Curriculum and Instruction at Austin. “We talk about supporting academics of color and what it means for people to know their work is valued… People don’t always cite your work, so some universities and colleges have become broader in their view of how your work is used. .”
Altmetrics helps tell a story
Because scholarship now exists in the media beyond printed materials, some scholars now list altmetrics on their resumes and grant applications, especially when their colleges or funders indicate that they value the information.
“It’s part of a larger conversation about the need to modernize promotion and tenure criteria, because right now – how shall I put it? – they’re based on certain metrics that may not be as informative as they are. have been sold or are used as proxies for something they don’t actually represent,” said Megan O’Donnell, a librarian at Iowa State University.
Admittedly, altmetrics and traditional metrics are sometimes correlated. For example, a 2018 study identified a strong positive correlation between communication of scientific research on social media that occurred within weeks of publication and citation rates that accrued over months and years. The researchers concluded that an early rise in the profile of one’s online work can predict an increase in future citations. In other study As of 2018, researchers found that early altmetrics that counted online mentions of research products, such as mentions in mainstream news, could predict the number of future citations.
Altmetrics Collection Tools
Altmetrics have emerged alongside the rise of open access research. Many university librarians now refer researchers to Metrics Toolkit, a free resource developed “to help scholars and reviewers understand and use citations, web metrics, and altmetrics in research evaluation.” The tool kit Editorial Committee is made up of experts versed in research impact measurement who serve for one year at a time.
The PLoS Impact Explorerfor example, provides real-time metrics on “which articles are getting the most buzz across social media sites, newspapers, and online referral managers.”
Free Altmetric Bookmarklet tool allows researchers to obtain metrics for an article they have published. Researchers can also embed code on web pages to get an “attention score” reported in the center of a colorful donut.”
Limits of Altmetrics
Altmetrics shares some limitations with traditional academic metrics. For example, a work may have a high number of engagements for negative reasons, such as clicks or downloads that follow the news of a fraudulent search. Additionally, alternative and traditional metrics may reflect implicit bias in a community.
But some limitations are unique to altmetrics. The broad term has no single definition, which means that altmetrics from different researchers can rarely be compared. Additionally, altmetrics can sometimes paint an incomplete picture of a researcher’s work. For example, when a PDF of a document is shared via technology, such as email or on a course learning management platform, those page views are not counted. The Muñoz graduate school professor sought to avoid this problem when she asked students to search and download the article on their own, skills that can foster students’ ability to use research tools. Solving one problem, however, can sometimes introduce another.
“For authors, providing a link is preferable [than providing a PDF] because then every downloaded and read visit is tracked,” O’Donnell said. “For students, direct access to the PDF ensures they are reading the right material.” A professor can provide a link to the article, assuming the library has a subscription to the journal. If the library does not have a subscription, instructors may have few options other than publishing PDFs, assuming they have obtained permission.
That said, altmetrics overcome some limitations of traditional metrics. At least Altmetrics helps to understand how many readers viewed the article, even if the picture is sometimes incomplete. The log impact factor, for example, does not provide this information at all.
Yet no metric, traditional or alternative, is perfect.
“Almost anything can be played,” O’Donnell said. “Automatic bots can roam and download anything,” inflating a tally. “There are also ways to play with traditional metrics.” Quote cards— groups of colleagues who work together to cite each other’s work — have helped researchers play with traditional metrics, O’Donnell said. At least altmetrics that sound too good to be true can often be traced, she explained. For example, a searcher who claims to have tens of thousands of engagements on a tweet should be able to produce the tweet easily. Also, downloading bots can be exposed.
Although altmetrics can help tell a story about the societal impact of a researcher or their work, few argue that they should be used in isolation.
“A more rigorous peer-reviewed citation remains the most reliable measure of impact and quality of work,” said Hui Zhang, digital services librarian at the University of Oregon. But Zhang is happy to see that altmetrics are “finally getting attention,” especially since they provide more opportunities to make the work of underrepresented and early-career researchers more visible.
“We’re in a time where people aren’t using our work the way they were even 10 years ago,” said Madkins of UT Austin. She is pleased that her institution is asking faculty members to include scholarship measures used by practitioners or other stakeholders in their tenure and promotion portfolios. “Knowing that people outside of academia are using my work is really valuable, because that’s what matters.”