Goals

The primary goal for this plan is to increase the quality and quantity of citations from WSU as part of the WSU Drive to 25 (D25) initiative. Citations are an academic coin of the realm. They mark both the new knowledge created by our faculty and the accumulated acknowledgement by their peers that their work is valuable. Citations are an important reflection of a research University’s core mission because they reveal the impact and influence of its research. This is especially the case for the land-grant university whose mission is to advance, extend, and apply knowledge to the state, nation, and world.

This plan details specific actions aimed at significantly increasing WSU’s performance on research citations for CY2018-2020, in preparation for meeting the ultimate 2030 goal. Measuring and increasing impact is multidimensional and multifaceted. Therefore, several actions in this plan are focused on supporting faculty to increase impact (see Recommendation 1), while other actions are focused on improving measurement of that impact (see Recommendation 2). In addition to being central to the D25, this metric also relates to WSU’s pursuit of membership in the Association of American Universities (AAU). The D25 citation metric is a traditional bibliometric defined as follows (www.aau.edu):

Citations: Thomson Reuters (Clarivate) InCites citations database provides an annually updated measure of both research volume and quality….the average of the institution’s citation data for the most recent three overlapping five-year increments.

Currently, WSU tracks the average H-index per faculty as part of the strategic planning process (see the 2016 Strategic Plan Progress Report). Based on this evidence, WSU is behind WSU’s strategic planning peers, but not as far behind compared with other AAU metrics. Additional data from National Taiwan University finds that WSU ranks at #67 in comparison to Michigan State University, ranked at #25. At the time of this report, WSU does not subscribe to the Clarviate InCites software package, so this report does not contain a ranking of WSU relative to our peers using the above metric.

A secondary goal of this plan is to recommend the consideration of additional approaches to assessing research impact beyond traditional metrics. Such “altmetrics” have gained increasing attention in academia and are also considered herein.

Based on the preceding, this action plan has the following subgoals:

• Determine the quantity and quality of publications at WSU compared to IEC peers, and suggest potential strategies for enhancing WSU’s published research output.
• Develop a short-term plan, to be implemented immediately, to increase citations (per the AAU metric) and other metrics and indicators of impact.
• Identify longer-term strategies, as well as associated infrastructure and research support needs, to further increase WSU’s citations (per AAU metric).
• Develop an approach to examine the role of altmetrics in assessing other dimensions of WSU’s research impact.
• Identify and define multiple citation metrics and their measurements.

Assumptions

There are four assumptions underlying this plan: First, we assume that there are only two approaches to assessing research impact that should be considered. The traditional approach, sometimes called bibliometrics, quantifies journal citations and is represented by the WSU metric and similar measures (e.g., Google Scholar). Bibliometrics are useful for the hard and social sciences, but generally inadequate for arts and humanities. Altmetrics have recently emerged that capture other forms of references to research, scholarship, and creative output (e.g., via blogs, LinkedIn, Twitter activity, media exposure, etc.). Altmetrics, in principle, could enable the University to measure items such as artistic engagement with increased rigor.

Second, we assume a need to determine an appropriate ranking comparison for this D25 metric. In contrast to many of the D25 metrics, which are tracked by the Center for Measuring University Report (CMUP), the citations metric is not captured by CMUP.

Third, compared to other goals that may have a revenue generating dimension, increased citations will be primarily, if not entirely, a cost center for WSU.

Finally, pursuit of the D25 citations goal will require combined and coordinated effort from WSU academic units, as well as administrative units.

Activities and outcomes

Recommendation1: Increase the quantity and quality of publications at WSU.

ACTION 1.1: Improve the “culture of support” for WSU’s researchers.

Working closely with the WSU Office of the Provost, Office of Research, and University Communications, the WSU Research Council (RC) should develop a plan to enhance the research support culture of WSU. Various ideas could be included in such a plan: recruitment and retention of high-quality PhD students and postdoctoral fellows, key players in many publications, through co-ops, mentorship, and stipends; balance teaching and research efforts across the faculty (e.g., allow faculty who are not engaged in research to contribute more to the teaching mission of WSU, thereby providing research-active faculty more time for research); leverage University-approved centers and institutes on campus to disseminate results; and support international collaboration. Any such plan must consider other D25 research metrics.

ACTION 1.2: Assess the importance of publication quality and quantity in pursuit of WSU’s research goals and in comparison to peers.

Charge the RC with defining whether WSU should pursue quality and quantity, or quality versus quantity, in its publications, and determine the level of consistency across units and colleges with the aim of improving research citations and enhancing WSU’s research reputation. Nuanced analysis may call for a deep dive into both the top-rated citations and the rankings in the medium and long tails, where less heralded researchers might be making outsized, singular, and ultimately flagship contributions. This should be an interdisciplinary effort that also considers the internal dimensions of citations, including but not limited to annual performance reviews (APR), research start-up funding, promotion and tenure, etc.

Recommendation 2: Develop a short-term plan (CY 2018) for immediate implementation to increase citations per the AAU metric.

Action 2.1: Standardize key faculty and publication data that can enhance citations.

Working with the WSU Libraries, Office of the Provost, University Communications, and the Office of Research, WSU should immediately encourage all faculty to obtain an ORCID ID, the unique digital identifier that helps ensure credit for work. This effort would include providing an optional field on the revised APR form for 2018 reviews and asking department chairs to urge faculty to obtain the ID. The University should ensure all published research has an associated digital object identifier (DOI) and other persistent work identifiers. Also, WSU needs to develop a checklist of key items associated with an upcoming publication (e.g., key standard information to include in the paper such as correct University affiliation) and distribute to faculty.

Action 2.2: Invest central resources to support dissemination of published research.

WSU Libraries has developed a research repository called the Research Exchange (https://research.libraries.wsu.edu/xmlui/) that supports dissemination of published research by WSU faculty. The potential of the repository is unmet due to low awareness and insufficient use by faculty. Central investment in the repository, particularly in faculty promotion and staff support, is immediately required. In addition to the repository, the Office of Research should identify a pool of funds to support fees required to make top publications open access. For example, there could be a competitive process by which faculty can simply apply for “open access funds” to increase the dissemination of their published research with the goal of getting WSU’s best research out to the world.

Action 2.3: Create a culture of research dissemination by developing a communication plan and support to enhance dissemination of faculty’s published research.

The WSU Office of the Provost, Colleges, Office of Research, and University Communications need to begin immediate work on creating a culture within WSU of research dissemination. In particular, faculty need to understand and internalize the idea that a researcher’s responsibility does not end once a paper is published.

In support of this effort, the RC should support the development of a strategic communications plan to increase research dissemination, including the following dimensions:

1. Create “standard operating procedures” that guide researchers through the dissemination and basic steps for publicizing fresh research (e.g., early notification to appropriate department, posting research to social media and WSU Research Exchange, submitting to ResearchGate, presenting at conferences, alerting and working with college and University communicators, etc.).
2. Use of ORCID IDs and DOIs in all external communications.
3. Creation of online FAQ and hosted brown-bag lunches to discuss the importance of disseminating research results.
4. Assist faculty in building an online presence, such as building up a profile of work they are doing and take advantage of University services designed to help facilitate online use.
5. Encourage researchers to upload publications into WSU’s Research Exchange repository.
6. Utilize various forms of “research social media” (e.g., ResearchGate); and subscribe to relevant publication communication services (e.g., PubMed for biological and life sciences faculty).

Implementation of any and all recommendations should consider copyright standards.

Recommendation 3: Identify longer-term strategies (CY2018-2020), as well as associated infrastructure and research support needs, to further increase WSU’s citations (per the AAU metric).

Action 3.1: Develop a process to better recognize faculty leading their field in terms of impactful research.

Charge the RC with identifying a University-wide criteria and mechanism to recognize faculty who are highly cited in their fields. Factors to consider by the RC should include: the number of authors on the paper; the nature of typical citations in the field; length of time associated with the research; the author’s role in the paper and its findings; and the paper’s relevance to WSU’s overall research reputation; as well as other factors that impact the overall level of citations of any particular paper.

Action 3.2: Determine the impact of pursuing increased citations in relation to other D25 goals and research goals of the institution.

WSU has multiple research-oriented D25 goals. The RC needs to carefully consider how the pursuit of increased citations may negatively or positively influence other research efforts, and vice versa. For example, do increased citations relate to R&D expenditures at WSU?

Action 3.3: Develop a comprehensive plan with appropriate resources to bring new research to the attention of the WSU community and external media sooner than later.

Charge University Communications and college/unit communications staff to develop a plan to identify research earlier in the publication process so that it can be publicized in a more timely fashion. Central to this will be allocating appropriate resources to support an “early alert system” for pending papers. Other aspects to this plan might grow organically from Action 2.3. The plan could also include: improved media contacts that go beyond research-specific mailing lists; training in how to tell one’s story outside of academic circles; more efficient use of data contained within Activity Insight; encouragement of faculty to communicate when a publication is accepted/available; working with local unit communications staff to determine and announce the top 10 research papers each month; and utilization of PubMed and other research-based social media to communicate recently published articles.

Recommendation 4: Identify and define multiple citation metrics and measurement.

Action 4.1: Select between one of three alternate approaches to obtaining citations data and defining our peers for this metric.

There are several sources of data to track WSU’s citations in comparison to its relevant peers. However we have identified three primary options moving forward: 1) the National Taiwan University Ranking, 2) internally generated data using Clarivate’s InCites, and 3) Google Scholar. The RC should recommend one or more of these three options based on input from the faculty as well as WSU’s Office of Research and Institutional Research. Central to this choice is consideration of the peer set to be used for the citation metric. Trials of the various options are already in progress.

Action 4.2: Define additional citation-based metrics (based on appropriate data) to assess the impact of WSU’s research.

Once a source of citation data is identified (see Action 4.1), it will be useful for the RC to examine other dimensions of this data to consider the impact of WSU’s faculty research, scholarship, and creative output. Examples include: identifying possible uses of these metrics for arts and humanities faculty; determining the “real impact” of our faculty’s publications (while taking into account things such as the number of authors and quality of journal); and analyzing citations in comparison to other research goals (e.g., citations per R&D expenditure, citations per doctoral student).

Recommendation 5: Develop WSU’s approach to the role of altmetrics in assessing other dimensions of research impact.

Action 5.1: Assess WSU’s approach to altmetrics during 2018.

Charge the RC to consider disciplinary differences when assessing the value of altmetrics (e.g., altmetrics may assess impact for arts and humanities more effectively than traditional metrics, but may also increase signal-to-noise ratio). The RC should rely upon expertise of WSU library faculty to ensure full consideration of different types of metrics (e.g., traditional bibliometric measures, performative metrics, and “research objects” metrics). The RC should consider the implementation of a trial use of altmetrics from commercial providers (e.g., Klout, Plum Analytics) for a significant period of time (at least 12 months) to gauge its value to WSU. University Communications should monitor altmetrics findings for news, features, and profiles that could, in turn, improve the altmetrics.

Metrics

The RC will develop specific metrics to support achievement of the overall citations goal. Examples for metrics to be considered by the RC include:

1. Number of WSU faculty with an ORCID ID.
2. Number of publications submitted annually to the WSU Research Exchange.
3. Number of WSU Publications for the year in Web of Science.
4. Once metric and source are identified, public university publication/citation ranking

Appendix

Introduction and background

The Drive to 25 (D25) is a presidential-based initiative to develop strategies and plans for Washington State University (WSU) to be recognized as one of the nation’s top 25 public research universities. More formally stated (www.wsu.edu/drive-to-25):

Washington State University will be recognized as one of the nation’s top 25 public research universities, preeminent in research and discovery, teaching, and engagement by 2030.

The Drive to 25 builds on the cornerstones of our institutional Strategic Plan and its two pivotal goals:

1. WSU will offer a transformative educational experience to undergraduate and graduate students
2. WSU will accelerate the development of a preeminent research portfolio

A series of 11 metrics have been developed to assess progress on the D25. The focus of this document is to layout the charge for a working group to develop a plan for increasing WSU’s performance on research citations (as measured by Clarivate InCites). This metric is one of four D25 metrics directly associated with WSU’s goal of gaining membership in the Association of American Universities (AAU).

The AAU citation metric is one of four Phase I Indicators that are used by AAU as “primary indicators of institutional breadth and quality in research and education.” The metric is defined as follows (www.aau.edu):

Citations: Thomson Reuters (Clarivate) InCites citations database provides an annually updated measure of both research volume and quality and will provide a valuable complement to [other] indicators listed above.

The average of the institution’s citation data for the most recent three overlapping five-year increments.

• InCitesTM, Thomson Reuters (2015). Web of Science^®. These data are reproduced under a license from Thomson Reuters. http://incites.thomsonreuters.com .

Compared to many of the other D25 metrics which are being tracked for comparative purposes by the Center for Measuring University Report (CMUP), the AAU citations metric is not captured by CMUP. Thus, for comparison purposes, we will use the peer group identified by WSU’s Institutional Effectiveness Council (IEC; https://strategicplan.wsu.edu/institutional-peers/).

Citations at WSU and beyond

Based on initial research, there are two approaches to capturing the impact of research, published scholarship, and creative activities from faculty. The first includes traditional metrics, sometime called bibliometrics, that quantify citations in various journals. Traditional metrics are represented by the AAU metric (Clarivate InCites) and other similar metrics such as Google Scholar. Bibliometrics are quite useful for the hard sciences, reasonably useful for the social sciences, and nearly inadequate for many in the arts and humanities. Recently, alternative metrics (so called “altmetrics”) have emerged that capture other forms of references to our research, scholarship and creative output (e.g., references in blogs, LinkedIn, Facebook likes, Twitter activity, etc.).

In terms of WSU’s strategic plan, WSU’s Institutional Effective Council (IEC) measures “Citations per faculty member & H Index” (see details below). This approach is a traditional form of bibliometrics but is different from the AAU metric.

Citations per faculty member H index Number of citations, number of faculty members, H-index, calendar year. In Web of Science, two years of “articles” associated with WSU under the Enhanced Search option are selected. Web of Science then generates a Citation Report which displays number of citations for each article and in total and also produces an H-index (Hirsch). For 2014, articles published in 2012 and 2013 were chosen. The Citation Report shows citations found from 2012 through the date of the query (there is no way to select the citation dates). The articles are then ranked in descending order of number of citations, with the article having the largest number of citations listed first. The H-index is then the ranking such that the number of citations and the ranking are equal. For 2014, the 42nd ranked article had 42 citations. It is not an average of individual h-indices, but a ranking value based on an evaluation of all papers with authors associated with WSU. The number of genomics (methods) papers with their high immediate citations numbers (due in part to the fast publishing cycles for those journals) are affecting the h-index ranking number for WSU. Peer information is developed the same way from Web of Science and faculty numbers are taken from IPEDS. WSU Strategic Plan, p. 76

From the 2016 report, WSU has improved citations per faculty (using the H index) but falls behind our peers.

 

Charge for Citations Working Group

The Citations Group will develop a plan outlining various action steps to significantly increase citations at WSU for CY 2018-20 based on the AAU metric and other appropriate bibliometrics. In addition to growing citations as captured by these bibliometrics, the working group will also make initial recommendations on how WSU should approach the emergence and importance of various altmetrics. Finally, the group will examine if WSU is producing sufficient numbers and quality of research that can generate impact as expected by a top 25 public research university.

Specifically, the group will focus on the following five goals:

• Determine the quantity and quality of publications at WSU compared to IEC peers, and suggest potential strategies for enhancing WSU’s published research output.
• Identify and define multiple citation metrics and their measurement.
• Develop short-term plan for immediate implementation to increase citations (per AAU metric).
• Identify longer-term strategies, as well as associated infrastructure and research support needs, to further increase WSU’s citations (per AAU metric).
• Develop approach to examine the role of altmetrics in assessing other dimensions of WSU’s research impact.

The D25 Citations Group will be a modest-sized, nimble working group that includes both faculty and staff from the academic colleges and key administrative functions. As with other working groups associated with the D25 metrics, the WSU Research Council (WSURC) will assist with implementing short-term action steps and developing specific action steps for longer term strategies including altmetrics. The WSURC is a perfect group to implement action steps associated with the goal of increasing citations at WSU, in support of the University’s D25 and strategic plan.

Structure and process

The D25 Citations Group is co-chaired by Christopher J. Keane and Dave Sprott. The Citations Group will include two broad work groups, the Cabinet and the Planning Team, plus an Integration Team.

• The D25 Citations Cabinet will oversee the broad strategic direction of the D25 Citations action plan and include senior leadership from the University. The initial five goals for the group listed above will be discussed and approved by the Cabinet. The Cabinet will approve the final action plan and present it to WSU leadership and the Research Council.
• The Planning and Integration Teams will include representatives from WSU colleges and key administrative units including Marketing and Communication, Information Technology Services, the Office of Research, and Institutional Research. The Planning Team will be charged with developing an action plan (5 pages 5 max, excluding appendices) that address the 5 goals outlined previously in the citations group charge. The Integration Team will guide discussions and draft the action plan.
• The WSU Research Council will support implementation of the action plan approved by the Citations Cabinet. In addition, staff from key administrative units (in particular, Marketing and Communication, Information Technology Services, the Office of Research, and Institutional Research) will aid as appropriate the WSURC when implementing the plan.

Membership of D25 citations group

Cabinet: Christopher Keane, David Sprott, Gil Picciotto (WSU Foundation), Keith Moo-Young (WSU Tri-Cities), Sasi Pillay (Information Technology Services), Phil Weiler (Marketing and Communications)

Planning and Integration Teams: Talea Anderson (Libraries), Tammy Barry (CLA, Psychology), Derek Brown (Office of Research), BJ Carlson (Carson College of Business), Scott Connors (Carson College of Business), James Downes (Institutional Research), Suzanne Fricke (Animal Health Sciences Veterinarian), Rita Fuchs Lockensgard (College of Veterinary Medicine), Jacob Leachman (Voiland College of Engineering and Architecture), Sarah Mahaffy (Office of Rsearch), Dorrie Main (CAHNRS), Sue McMurray (Carson College of Business), Sterling McPherson (College of Nursing, WSU Spokane), Squeak Meisel (CAS, Fine Arts), Brianna Murphy (Office of Reseach), Lorena O’English (Libraries), Andrew Perkins (Carson College of Business), Nathan Roberts (Carson College of Business), Kiara Seguine (Carson College of Business), Eric Sorensen (University Communications), Juming Tang (CAHNRS), Greg Yasinitsky (CAS, School of Music)