From March 19th through the 21st, ETS’s Management of Electronic Resources and Services committee moderated an eForum on Assessing Electronic Reference Services. The eForum provided an opportunity for participants to share their experiences with assessing services and ask questions of experts and peers. Below can be found a summary of each day and lessons we learned.
The first day’s theme was projects. The discussion centered on what certain libraries are doing to assess their ereference offerings. Special emphasis was placed on how chat is being assessed whether it be through analyzing transcripts, the use of a rubric, or using surveys at the end of a transaction. Some libraries have begun to engage in proactive chatting by having vendors add chat widgets in the databases themselves.
The theme of the second day was methods. The highlight of this day was an hour of discussion with Dr. Marie Radford, Professor of Library and Information Science at Rutgers, about assessing chat reference. Dr. Radford gave an overview of VRS assessment including reasons to assess, setting assessment in the context of institutional goals, input measures and outputs, and methods. She also gave her insights on surveys, transcript analysis, and what’s on the horizon in VRS assessment research, which is using transcript analysis to aid in the understanding of building interpersonal relationships. Innovative ideas from participants included sending a survey at the end of a complex chat transaction as a way to get more specific assessment data, “Reference Batting Practice,” in which a reference team talked about sample transcripts as representing a “home run, single, double, triple, or an out,” and the very simple but effective “smiley face” survey.
The theme of the final day was impact, which could be positive impact on services or using assessment data to show the impact of the library. Dr. Kris M. Markman, Director of Digital Learning and UX at Harvard Library, joined the forum to talk about about closing the assessment loop and acting on your assessment results. Dr. Markman discussed how to maintain commitment to acting on assessment and the importance of ensuring you aren’t assessing to assess but rather with a goal in mind.
The eForum Archives are accessible through the ALA Sympa listserv platform, http://lists.ala.org/sympa. Upon logging in, search for “rusaetse-forum@lists.ala.org” and click on “Archives.” If you have suggestions for future programming for ETS MERS, please email committee chair Melissa Becher at mbecher@american.edu.
