Book Review:
Irwin, B., & Silk, K. (Eds.). (2017). Creating a Culture of Evaluation: Taking your
Library from Talk to Action. Toronto: OLA Press.
https://journal.lib.uoguelph.ca/index.php/perj/article/view/4073/4058#.Wn2t41UrLcs
“This book is a valuable resource for anyone grappling with library assessment. It
presents a compelling argument for libraries to adopt a system of outcome-based
evaluation in order to assess the social, cultural, economic, or educational impact of
their programs and services on their users and the communities they serve. More
importantly, the book gives readers the tools necessary to do it.
The twelve chapters are equally well written by a variety of academics and practitioners
in the field of assessment. The chapters flow logically, the first outlining the need for
evaluation, while subsequent chapters explore research methods, impact measures,
challenges, outcomes, and uses. Although the book focuses on public libraries, the
information is applicable to any type of library.
Bill Irwin’s chapter “The Cost of Doing Nothing” starts the book off, and so much of it
rings true for me. Libraries have always been good at keeping “counts” (e.g., gate
counts, circulation statistics, workshop attendance), but how are these statistics used,
and how valid are they? Irwin gives us a personal anecdote about patrons manipulating
gate counts and circulation statistics. At some point, you may need to justify the costs
of your library’s services or to demonstrate their value to your stakeholders, and you will
want to be ready. As an apt example, Irwin offers the plight of the Toronto Public
Library, which had to prove its worth during Rob Ford’s term as mayor.
Kimberly Silk follows with a truly great primer on research. She takes us through the
entire research process in a manner that is easy to understand. Plus, she provides the
reader with links to a variety of tools for literature reviews, surveys, and data
visualization, as well as sources of library research studies that will surely come in
handy for anyone launching a research project.
The reader is then introduced to the logic model as a measurement system for the
library sector. An appendix containing two articles written in 2013 by the chapter
authors outlines the basics of this model. The authors suggest reading these articles
first before returning to the chapter to learn how to successfully implement the logic model. Some readers will find these articles useful; however I found reading them a bit
challenging because of the small font size and dark background colour. Luckily, the
chapter stands well on its own.
Several chapters on impact assessment follow: how to design, measure, and
communicate the economic, social, cultural, and educational impact of your
organization. All have extremely valuable information, but I found Chapter 8,
“Meas“Measuring Economic Impact,” particularly interesting. The formulas that Kimberly Silk
provides in this chapter could prove lifesaving for anyone tasked with calculating the
economic benefit or value of their library’s services, collections, or operations.
The next section of the book discusses how organizations can make the shift to a
culture of evaluation. Chapter 10, “How Outcome Evaluation Can—and
should—Change Your Practice,” shows how outcome evaluation can positively impact
management practices and strategic planning. Chapter 11, “Barriers and Challenges to
New Systems of Evaluation: Organizational Culture,” is a no-holds-barred account of
the challenges faced when introducing new assessment methods. Irwin presents a fivestage
model designed to represent the levels of institutional progression through
awareness, acceptance, and understanding of new models of outcome-based
evaluation. “It moves from grudgingly accepting the need for evaluation to
enthusiastically embracing it.” (p. 222).
The final chapter, “Building Influence: Communicating Results and Value through
Advocacy,” shows how assessment can be used for library advocacy. Having a system
of evaluation in place means having data on hand that can be used to validate the worth
of a library’s programs or services when needed, such as when funding is threatened.
Stephen Abram’s example on pages 245 and 246 is a definite eye-opener. Imagine
being able to prove to the government that cancelling a $500,000 subsidy to run a
province-wide interlibrary loan service will mean a loss of $21.2 million in value to the
residents of that province.
Appendices provide a generous supply of helpful information, including an extensive list
of library impact studies in Canada and around the world, case studies, a glossary of
terms, and a substantial bibliography.
In conclusion, although this book focuses mainly on public libraries, the information can
be applied to any type of library. This is a must-read for anyone involved in library
assessment.
Colleen Neely
Assessment Projects Librarian
Carleton University Library
colleen.neely@carleton.ca”
https://journal.lib.uoguelph.ca/index.php/perj/article/view/4073/4058#.Wn2t41UrLcs