The Becker Medical Library Model for assessment of research impact is a framework for tracking diffusion of research outputs and activities to locate indicators that demonstrate evidence of biomedical research impact. It is intended to be used as a supplement to publication analysis. Using the Becker Model in tandem with publication analysis provides a more robust and comprehensive perspective of biomedical research impact. The Becker Model also includes guidance for quantifying and documenting research impact as well as resources for locating evidence of impact.
Could you share some of the background or challenges that enticed you to develop this model?
The project resulted from completing an ex post study in 2007 of the research outputs and activities of a large clinical trial. We went beyond citation analysis and located many examples of research outcomes that were not discernible using publication data. Citation analysis alone does not reveal whether research findings result in new diagnostic applications, a new standard of care, changes in health care policy, or improvement in public health. We discovered that diffusion of research outcomes transcends publication data and one must go beyond using publication data to provide a full narrative of meaningful health outcomes.
After the project was completed we had about approximately 100 examples of indicators of research impact including those not related to the study. We decided to create a listing of these examples for others to use and called this listing the Becker Model. Since then, we have added many more examples and we continue to do so, updating the model about every 6 months or so. We have also included examples of various research outputs and activities to make it easier for people to apply the model.
For a full description of the genesis of the Becker Model, see Sarli, C.C., Dubinsky, E.K., Holmes, K,L. “Beyond citation analysis: a model for assessment of research impact”. J Med Libr Assoc. 2010 Jan; 98(1):17-23.
Was the model built to be used more by researchers or by evaluators?
The Becker Model (and related information on the website) is intended for any audience as needed for their purposes with a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License assigned. We highly encourage anyone to use, modify, and/or adapt the model as they see fit as long as there is no commercial use, and we also ask that people notify us of their use so that we can better understand how the model is being applied. The Model has been used by researchers examining their own work, evaluators trying to better understand the impact of research efforts on the individual and group level, by agencies that wish to see the return on investment of funding awards, and librarians supporting their faculty and evaluation groups on campus as well as those librarians who are beginning to provide services and consultations in this area.
We welcome suggestions for new indicators of impact. It has been our experience that new indicators tend to reveal themselves organically during the process of applying the model to an individual or group. When the model was first launched in March 2009, there were approximately 100 indicators of impact examples. To date, there are over 350 examples and the list is updated at least twice a year. Our insider joke is that the Becker Model is in “perpetual beta.”
How do the different aspects of the model work together as indicators of quality?
We do not assign any type of quality measurement to the indicators as noted in the Becker Model. The indicators of research impact examples are simply examples of biomedical impact with no differentiation as to ranking or significance. The indicators are grouped under various stages, or pathways, based on the research cycle with some overlap between the stages. These pathways will vary based on the discipline, but there are some commonalities across all disciplines.
Is the model modular? Can parts of it be used or does it have to be used as a whole in order to capture quality in an accurate manner?
Users are welcome to use any part or all of the Becker Model for their purposes. The definition of quality is to the discretion of the user. It is also up to the user to assign any ranking or significance to specific indicators of research impact.
How do you balance between the quantitative and qualitative parts of the model? Are they given equal weight? Can an institution/researcher decide the appropriate weights for their evaluation?
Absolutely. Users are welcome to assign their own ranking system for specific indicators of research impact based on the outcomes that match their research or program goals. Some organizations may prefer to assign a specific indicator a higher level of significance over others. We recommend that any report using the Becker Model include both quantitative and qualitative indicators of research impact and include multiple examples of such. No single example or metric should be used to demonstrate research impact.
The model has been running since 2009: could you tell us about some of the successes you observed in its use?
We are just tickled by the response and feedback to the model. It has far surpassed our expectations. We find that the list of indicators of impact can be quite useful as a checklist for scholars and investigators as they review their project and prepare for grant progress reporting, tenure/promotion, or for departmental reports. The checklist helps jog memories and identify outcomes from their own research as well as ideas for using publication data to tell a story about their research. Other institutions and agencies have reported using the model for their evaluation projects and others have adapted it for different disciplines such as Anthropology, Archaeology, Nanotechnology, Agriculture, and others. But most of all, The Becker Model has been helpful as a means of engaging users to think about ways to report on research outcomes beyond publication data.
Can you see the model adapted to other disciplines? If yes, to which?
Yes, to a point. The Becker Model was developed with an emphasis on indicators of outcomes specific to biomedical research. However, some indicators based on publication data are universal among disciplines. A number of the Strategies for Assessing Research Impact are applicable to a wide variety of disciplines, as well.
10 Strategies for enhancing research impact
Consider these strategies for enhancing the visibility and impact of your research from the authors of the Becker Model. The strategies are divided into three categories: Preparing for Publication, Dissemination, and Keeping Track of Your Research. A full listing of the strategies can be found at https://becker.wustl.edu/impact-assessment/strategies.
Authors should use the same variation of their name consistently throughout their academic careers. If the name is a common name, consider adding your full middle name to distinguish it from other authors. Authors should also use a standardized institutional affiliation and address, using no abbreviations. Consistency enhances retrieval. See Establishing Your Author Name for more information.
Present preliminary research findings at a meeting or conference and consider making your figures available through FigShare and your presentation materials available in your institutional repository or on a sharing site such as SlideShare so that others may discover and share your materials post-event.
Consider the desired audience when choosing a journal for publication. Topic-specific journals or journals published by a specialized society may disseminate research results more efficiently to a desired audience than general science journals. More specialized journals, even with a potentially smaller readership, may offer an author broader dissemination of relevant research results to their peers in their specific field of research. For more information on selection of a journal for publication, see Preparing for Publication: Factors to Consider in Selecting a Journal for Publication.
Submit the manuscript to a digital subject repository such as arXiv or to your institutional repository.
Enrich your visibility through press releases and an established online presence. Issue press releases for significant findings and partner with the organizational media office to deliver findings to local media outlets. Set up a web site devoted to the research project and post manuscripts of publications, conference abstracts, and supplemental materials such as images, illustrations, slides, specimens, and progress reports on the site.
Share the research data generated by the research and deposit research data in appropriate repositories. One study, “Sharing detailed research data is associated with increased citation rate,” demonstrated a correlation between shared research data and increased citation impact. Consult data management guidelines for suggestions on organizing, managing, and sharing your data. The University of California Curation Center of the California Digital Library provides a comprehensive set of guidelines in their DMP Tool.
Leverage social media: start a blog devoted to the research project, communicate information about your research via Twitter, and contribute to a wiki in your area of work or research.
Keep your profile data up to date on social networking sites aimed at scientists, researchers and/or physicians and inquire about these tools at your institution or within your organization. Some highly adopted enterprise-level platforms providing verifiable data about scholars include VIVO, Profiles, and SciVal Experts. These institutional efforts leverage structured data about researchers to provide current and validated data which can be used to visualize your efforts and identify new resources and collaborators.
Register for an ORCID iD and curate your ORCID record with your scholarly contributions. ORCID identifiers provide you with a way to differentiate yourself and highlight your professional activities.
Become acquainted with how your work is being used in the online world via bookmarks and links to the article or data, conversations on Twitter and in blogs about the work, and various methods of sharing and storing content. Some great tools that provide this type of information for articles and individuals include Altmetric and ImpactStory.
For the model’s website please visit: https://becker.wustl.edu/impact-assessment/model
Contacts:
Cathy Sarli
Scholarly Communications Specialist
sarlic@wusm.wustl.edu
Kristi Holmes
Bioinformaticist
holmeskr@wusm.wustl.edu