THESE DAYS, scholarly metrics are a high-stakes game and a lot of effort is being directed toward finding relevant and credible methods to assess and present scholarship.
In LJ 1/15 (ow.ly/KQ4yY), we analyzed Journal Citation Reports (Thomson Reuters) in which Impact Factor, the most consistently accepted metric for scholarly assessment, has held sway since 1975 (see also “InCites Gets a Makeover,” ow.ly/KQ3Fn.—Ed.). We also evaluated two approaches for delivering article level alternatives—Plum Analytics (EBSCO) and Altmetric for Institutions (Altmetric)—and reviewed Cabell’s Directories of Publishing Opportunities (Cabell Publishing), in which directory-type information is combined with Cabell’s journal analysis and ranking system.
This time, we briefly revisit an updated Cabell’s and consider Impactstory, whose stated goal is “to build a new scholarly reward system that values and encourages web-native scholarship,” along with another Thomson Reuters product, Book Citation Index, which aims to turn the static book citation into a dynamic research tool.
Since several publishers now incorporate scholarly metrics into their own titles, we also take a look at the various approaches of BioMed Central (Springer), the Public Library of Science, the Nature Publishing Group (Macmillan), and Scopus (Elsevier).
BioMed Central Springer Science+Business Media; www.biomedcentral.com.
To request a free trial, email info@biomedcentral.com
content BioMed Central (BMC) displays an impact factor for journals tracked by Thomson Reuters for three years and includes a list of those titles tracked, as well as those that have been calculated with unofficial impact factors, since these are relatively new. According to the BMC website, Thomson Reuters is currently considering additional BMC journals for tracking.
Article metrics—access counts, highly accessed, and citation counts—are provided for all articles published by BMC. “Access counts” displays the number of times an article has been accessed either within the previous 30 days, the previous year, or all time. Currently, only article accesses on BMC web pages are counted and do not include access from indexing sites such as PMC, Medline, or Scopus.
Next, the “highly accessed” indicator identifies articles that are frequently viewed relative to their age and the journal in which they were published. Articles must be at least 14 days old in order to receive the designation, and once an article has qualified it will remain permanently flagged, allowing authors to include this designation when listing their publications.
The citation count is shown on the article metrics page. Links are provided to obtain citation lists and counts from PMC, Scopus, Google Scholar, and Web of Science. F1000 evaluation is also included; if an article has been evaluated by the F1000Prime faculty, a link to the evaluation is provided. Lastly, social media “share” buttons can be used to give out individual articles.
BMC displays the Altmetric.com score and “donut” on the article metrics pages of published articles. Each institutional member page also enables documents to be sorted by access count (most viewed) and altmetric.com score (trending articles).
Authors can view the latest access statistics for all materials they have published with BioMed Central by logging into the “My Manuscripts” page.
Book Citation Index Thomson Reuters; wokinfo.com/products_tools/multidisciplinary/bookcitationindex. To request a free trial, email general.info@thomsonreuters.com
content Situated within the Thomson Reuters Web of Science Core Collection, Book Citation Index (BCI) turns the formerly static book citation into a dynamic tool for launching a research project and determining the scholarly relevance of a given book. While it should be of considerable value to natural science researchers already accustomed to working with Web of Science, BCI offers social scientists and humanities scholars an excellent place to start their research.
Approximately 60,000 books are found in the BCI database. Social sciences and humanities titles comprise about 60 percent of this figure, while the remaining are texts on science, technology, engineering, and medicine. BCI adds nearly 16 million new cited references to the Web of Science Core Collection. Coverage begins with 2005, and Thomson Reuters’ aim is to add 10,000 books annually.
USABILITY At institutions with multiple Thomson Reuters products, BCI resides within the Web of Science Core Collection and is accessible via the “All Databases” pull-down option on the Web of Science homepage. Clicking on “More Settings” provides a view of all of the citation databases to which the user’s institution subscribes. BCI is divided into separate Science and Social Sciences & Humanities databases, but they may be searched simultaneously by selecting the boxes for both.
The initial view is of a single search box set up to run a basic search. Boolean operators (AND, OR, and NOT), as well as the proximity operator NEAR, may be included within the search box. Wildcards and truncation are also available. If the user opts to “Add Another Field” in order to construct a more advanced search strategy, the new search box is connected to the first one via a pull-down Boolean operator. In addition to searching by topic, users may specify searching in the title, author, author identifiers, group author, editor, publication name, DOI, or year published fields. One can also include timespan limits for the query—all years or a range of years.
Searching the phrase “critical race theory” produced a results list with 85 items from the Web of Science Core Collection. Results may be sorted by publication date, first author, relevance, source title, conference title, and times cited (which would be the most useful for determining impact).
Researchers have a long list of options when it comes to refining results, including broad Web of Science categories (such as law, ethnic studies, and political science), document type (such as “article” and “book chapter”), authors, book series titles, funding agencies, languages, countries/territories, and several others.
Examining the most cited item in the results list— M. Cole’s Critical Race Theory and Education: A Marxist Response—and specifically looking at the Citation Network area of the record, demonstrates the power of BCI’s approach. The user first sees that Cole’s book has been cited 34 times in the various Web of Sciences databases, and that full citations for all results can be readily accessed.
There is also the indication that Cole includes 440 cited references (from the Web of Science databases) in the bibliography and that all 440 of them may be immediately accessed. There’s a little quirkiness in the cited reference list here—namely, that many items read “Title: [not available]” if the item in the bibliography is a book. The book title does display below the author’s name, however, and clicking the links option to access the full text, in our case, would take us to our local Serials Solution utility, in which users can easily initiate a title search and locate the book if it’s in our collection. Awkward, but not fatal.
Next, users may view related records (i.e., any of the 15,000-plus records that share references with this record). That is followed by the option to view a citation map, which the help screen describes as “a graphical representation that shows the citation relationships (cited references and citing articles) between a paper and other papers.” Finally, there is the option to create a citation alert.
Returning to the results list, users may analyze the fruits of the search, which in this instance indicates that a fifth of the citations to Cole’s book are self-citations. The other analytical option is to run a citation report, which yields aggregate citation statistics for the complete set of search results that are being examined.
Thomson Reuters promotes BCI, in part, for the role it can play in assessing the output of one’s institution. After some experimentation, we searched “Union AND 12308” in the address field and received a list of 88 items, most of them book chapters, by Union College faculty members.
BCI also offers a “Cited Reference Search” option. In the context of book searching, this was most valuable for locating citations to some of the more popular science and social science titles such as Elizabeth Kolbert’s The Sixth Extinction (cited as 6 EXTINCTION EARTH and 6 EXTINCTION UNNATUR), Nate Silver’s The Signal and the Noise (cited as SIGNAL NOISE), or Chad Orzel’s How To Teach Physics to Your Dog (cited as TEACH PHYS YOUR DOG).
Search results can be easily printed, emailed, or exported to EndNote. Search strategies may be used to create alerts.
PRICING Generally speaking, the cost of Book Citation Index alone is roughly equivalent to that of the entire Web of Science Citation Connection, so anyone eyeing BCI would probably opt for the more encompassing collection.
VERDICT The conventional approach to database searching—identifying new and relevant sources to support a research project via author and keyword searching—gets a big boost from Thomson Reuters Book Citation Index, in which users can not only discover key titles of interest but readily access all the scholars who have cited a title since it was published; and just as effortlessly milk the author’s bibliography for every useful item drawn on in the original research.
The most relevant books on any given subject may no longer be the ones that your beloved art history professor can rattle off without batting an eye, but the ones that BCI indicates are cited repeatedly in the literature. For humanists and social scientists in particular, this takes the power of the cited reference in an all new direction.
Cabell’s; Cabell Publishing; www.cabells.com. To request a free trial, email info@cabells.com
Cabell’s Directories is scheduled to release an updated website in May 2015, with numerous additional features and functionalities. Although their launch date was still months out at the time we were concluding these reviews, we can tell you that the changes we were able to preview will be a noticeable improvement.
Cabell’s new site takes on a totally fresh look, with a new color palette and a more intuitive platform. A few highlights include added personal profile features, in which users can participate in rating their submission experiences, receive custom calls for papers alerts, mark journals and search results as favorites for quick access, take part in discussions, and customize their interactions as authors with photos, personal interests, etc.
Administrative features will include federated single sign-ins, sign-ins available via institutional IP addresses, traditional username/password login, administrator accounts for all subscriptions, and the options to track usage and manage accounts online and provide online renewals.
An interactive options bar will include tabs for comparing journals and tracking changes within journals such as Impact Factor (IF) and other contextual classifications. The classic view displays all of the journal’s current information in one place while the “Explore This Journal” option shows various tools and reports including metrics, contextual analysis, submission experience, contacts, and more. Choosing any of the tabs will display a different interface with further detail, based on the selection.
The journal title display will now provide the latest IF from Thomson Reuters and will contain the acceptance rate; a “Contextual Analysis Toolset,” which lists a summary of the analysis areas; the access model (open access, hybrid open access, delayed open access, etc.); and average time to publication.
The new site launch will also add directories in math and six disciplines of science: biological sciences, chemistry, oceanography, astronomy, geology, and physics.
Impactstory; Impactstory; impactstory.org.
To request a free trial, email team@impactstory.com
content Impactstory (LJ 11/15/15) is aimed at the scientific community and is designed to give researchers the means to investigate and share the outcome of not just their scholarly publications but all of their research products. This includes more contemporary evidence of scholarly engagement such as datasets, blog posts, and software tools. “By helping scientists tell data-driven stories about their impacts,” the Impactstory website argues, “we’re helping to build a new scholarly reward system that values and encourages web-native scholarship.”
Impactstory is an open-source, web-based scholarly analytics tool that is currently funded by the National Science Foundation and the Alfred P. Sloan Foundation.
USABILITY The no-frills homepage has links to an “about” page, FAQs, a sample profile, and the free trial sign-up along the left-hand side of the screen, plus an invitation to create a profile at the bottom of the main frame. The profile takes just a minute to complete, with the author asked to supply his or her name (as it appears on one’s academic publications), affiliation (with title), email address, areas of interest, and personal homepage.
The next step gives authors various ways to populate the profile they’ve just created. The first involves linking to one of six web-based scholarly resources and repositories—Figshare, GitHub, Google Scholar, ORCID, Publons, and SlideShare—in order to import one’s research. Users may also add items to the profile manually, or by sending an email to Impactstory whenever they publish something new. We selected the Google Scholar option, visited that site, and modestly executed an author search on “Bruce Connolly.” Items from the resulting list could be selected or removed, and then added to the profile as a batch.
Impactstory adds new articles to the profile automatically using a statistical authorship model. Duplicates are merged or deleted. Although it’s not recommended, authors may opt out of the automatic update process and request, instead, an email notification giving them the chance to review and confirm any updates. Authors may edit, add, or delete individual articles in their profile; there is also the option to merge duplicate records. Impactstory collects citations to an author’s publications and displays them automatically.
The resulting profile includes the author’s basic information, a picture (if the contributor chooses to add one), and a listing of publications, with the number of citations and the publication year. A frame on the right of the profile page displays citation indexes for all publications along with those since 2009. Clicking the citation indexes heading itself displays a bar graph of all the author’s citations year-by-year.
The “Cited by Number” option provides a link to Google Scholar, where articles citing an author’s publications are displayed. An author’s h-index (which aims to communicate both productivity and the measure of scholarly impact) and i10-index (indicating the number of an author’s publications that have been cited ten or more times) for all years and since 2009 are also displayed. There are links to the publications of any coauthors, as well.
Once the author makes the profile public (in Google Scholar, in this instance) it is searchable by the author’s name.
PRICING Subscriptions are priced at $10 per month and $60 per year, payable by credit card. Thirty-day free trials are available without having to enter credit card information.
Those wishing to purchase multiple subscriptions for everyone in a department or office may do so as a single credit card purchase. Whoever makes the original purchase will receive a coupon code, via email, redeemable for the number of subscriptions specified in the credit card purchase. These codes may then be distributed at the purchaser’s discretion.
Author profiles are deactivated after the trial period ends if the author doesn’t subscribe. Subscribers, however, retain full control over their data, even after ending their subscriptions, within the parameters delineated by the terms of service of Impactstory’s data providers.
VERDICT Scholars and researchers looking for a simple means of comprehending and communicating the value of the contributions they are making to their fields have an ally in Impactstory. Whether their work follows the conventional route of peer-reviewed journal articles or embraces a range of more contemporary, alternative scholarly outputs—articles downloaded and shared, blog posts referencing their research, or datasets manipulated—Impactstory can help them gauge their impact on the world of knowledge.
Nature Publishing Group Macmillan Publishers Limited; www.nature.com.
To request a free trial, email subscriptions@nature.com for personal and print subscriptions and institutions@us.nature.com for institutional site licenses
content Nature Publishing Group utilizes metric data for many of the journals published on their platform such as Nature, Nature Communications, and Scientific Reports. Article-Level Metrics (ALMs) are available for research articles published since 2011 and include frequency of updates, (metrics data that is updated hourly in most cases; in some instances, such as page views, the data is not available until 48 hours after an article is published); citations, a single number count for article citations from each service’s database (Web of Science, CrossRef, and Scopus) that is updated daily once they become available; and page views, a cumulative count of full-text article views that provides HTML views and PDF downloads.
These data reflect only usage on the nature.com journal platform, as tracked by WebTrends, and are only for articles published on or after January 1, 2012. The page views data are available 48 hours after online publication and are updated daily. Individual mainstream news sources, blog posts, and Google+ posts, provided by Altmetric, are updated hourly. The Altmetric score is calculated based on the online attention an article receives while Twitter demographics provide the number of tweets broken down by country of origin for the specific account. Geographic breakdown for the Twitter sources is also provided by Altmetric and is updated hourly.
Within some titles, users can view an interactive PDF of an article in ReadCube, a reference manager, citation tool, and enhanced PDF reader, which allows annotations, sharing full-text access, adding to a ReadCube Library, and more.
Public Library of Science; Public Library of Science; www.plos.org.
To request a free trial, email plos@plos.org
content We have selected a few examples of publisher metrics to illustrate the current trends. There are many partnerships with Altmetrics.com, including open-access models as well as subscription-based publishers. To name just a few, Highwire, Royal Society of Chemistry, Springer, Wiley, and the newcomer Paperity—an aggregator of open access journals—all display the Altmetric.com “donut.”
Public Library of Science (PLOS) was one of the earliest publishers to offer Article-Level Metrics (ALMs) on all articles. Online “article usage” data, as recorded at PLOS sites, is available for all PLOS journals and for all articles published, from mid-June 2005 to present. Newly published articles require 48 hours before data usage is displayed.
Usage data for articles from PubMed Central (PMC), a free full-text archive of biomedical and life sciences journal literature at the U.S. National Institutes of Health’s National Library of Medicine (NIH/NLM), was introduced in 2010, and PLOS started displaying this data in 2012. PubMed Central usage data is updated monthly. The total article usage data is an aggregate of PLOS and PMC usage, combined.
The full ALM data set, which is updated monthly as a .csv file, is freely available for every article published by PLOS. PLOS also makes its application programming interface (API) and accompanying documentation for ALMs freely available. Users can access the metrics tab on every article within PLOS, which displays a general breakdown as well as a detailed table form for each category.
The regularly updated data falls into the following categories: viewed, which includes page views and PDF downloads from both PLOS and PMC as well as original XML downloads from both sources; citations computed by third-party databases and search engines such as PMC, CrossRef, Scopus, Google Scholar, and Web of Science; social bookmarking from providers such as CiteULike and Mendeley; mentions on third-party blog aggregators and social networks including Facebook and Twitter and even feedback within the comments section of the PLOS website; and items recommended from F1000Prime.
F1000Prime is a service composed of nearly 5,000 faculty members: scientists and clinical researchers in all areas of biology and medicine who recommend the most important articles, rating them and providing short explanations for their selections. [Read an interview with founder Vitek Tracz at ow.ly/KQFLM.—Ed.] Each reviewer provides a numerical score to their qualitative evaluation. The metric is a sum of the scores from all of the reviews that the recommended article has received.
To find out “which articles are seeing the most buzz” from social networking sites and online newspapers, check out PLoS Impact Explorer, a mashup of altmetrics data from Altmetric.com with articles from PLOS.
Scopus; Elsevier B.V.; www.elsevier.com/online-tools/scopus.
To request a free trial, email usinfo@elsevier.com within North and Central America and nlinfo@elsevier.com within Europe, the Middle East, or Africa
content Scopus, an Elsevier product, has also incorporated Article-Level Metrics (ALMs) into its search results in collaboration with Altmetric.com, expanding the range of traditional metrics by including scientific blogs, social media sites (Twitter, Facebook, Pinterest, and Google+), social bookmarking sites (Mendeley, CiteULike), scientific publications such as Scientific American, mainstream media outlets such as the New York Times, and non-English-language publications such as Die Zeit and Le Monde.
The Scopus platform includes a demographics tab that displays the geographic breakdown of where the interest for a given article originates. By default, ALMs display within the sidebar of article and abstract pages and users can elect to disable these options.
For Scopus subscribers, the journal analyzer allows users to select up to ten journals for comparison. Results are uploaded into graphs, making it easy to see how journals are performing relative to one another during a specific time period (the default is from 1996 to the present). Source-Normalized Impact per Paper (SNIP) and SCImago Journal Rank (SJR) are integrated into the Journal Analyzer. SNIP makes comparing journals across disciplines possible and is especially beneficial when evaluating multidisciplinary journals. With SJR, the subject field, quality, and scholarly reputation of the journal has a direct impact on the value of a citation, so a citation from a source with a relatively high SJR is worth more than one from a source with a lower SJR. SNIP and SJR values can also be viewed per journal in the “Browse sources” tab.
Other Scopus metrics include Impact Factor, h-index, and Impact per Publication (IPP). The IPP measures the ratio of citations within a year to scholarly papers published in the previous three years, divided by the number of scholarly papers published in those same years. For a detailed description and comparison of metrics, visit the publisher’s website.
