Ouch … that hurt! The impact of impact by Matt Holland

You know you are trying too hard when colleagues say, with a touch of irony, “you’re very interested in impact, aren’t you Matt!” Possibly that request for another case study sounded too desperate. Here are ten mini reflections from personal experience of the theory and practice of impact.

#1 Impact is not something else, it’s every day …

It can feel as if there is a cult of impact supported with cases studies, surveys, tales of money saved and going the extra mile. These things are good, of course. The risk is that by making impact special or different it elevates it above good regular customer service. So there is impact and then the other stuff we do. Impact is the outcome of delivering good customer service everyday. That can be hard to distill this into a research instrument but can be easy to lose sight of.

#2 Users, impact and case studies

Impact case studies aim to capture instances of good practice filtered through the minds of our users. We ask our users to tell us what impact using the library has had on their practice or research. In most instances this is evidence of a positive relationship with library users but it isn’t empirical evidence of impact. For that reason case studies don’t really tell us, librarians, very much. However, they do provide good promotional copy and they have value in explaining to external stakeholders how using a library could positively effect users behaviours. This is not to denigrate case studies, but it is good to be clear eyed about their purpose and strengths.

#3 Don’t ignore secondary effects …

The frustrating thing about impact is that some aspects can’t be easily measured. One of the dangers of operationalising impact measures, reducing impact to a series of research instruments, is that you don’t look beyond those measures once they are deployed. There is a case still for making the bigger arguments that can’t necessarily be backed up with pie charts. For example the additional support that health care libraries offer to those in higher education, increasing students engagement with the courses, raising confidence and expectations and maybe ultimately pass rates and professional effectiveness within the organisation.

#4 Campbell’s Law and other nonsense … ?

Campbells’ Law[1], paraphrased badly, states that once you start to measure something you distort it to the point where it becomes ineffective as a measure because people start to game the outcome. That is probably true of a numerical measure but it is also true of a qualitative measure. So we begin to see our services as a series of case study opportunities, possibly gearing our interactions with users with a case study in mind or that positive response to an Impact Survey. Too far fetched? You decide …

#5 Lean times and hard choices

Demonstrating impact makes an all important connection to front-line services. In lean times when corporate services are subject to cost saving reviews and re-organisations in favour of frontline services then this is a key link to make. It can also make common cause with other services in education, learning and development. There is a strong motivator to get this right. It’s also a strong motivator to cut through the wooly thinking. Does what we do add value, is it cost effective, does it reach the front line, is it impactful? There are no easy answers but the questions have increasing relevance.

#6 Impact is long term project

… and it’s complicated. Impact is the end of a process. It is the outcome of many components of our services.   A good marketing plan, good execution of your strategy, sound service design, effective communication with your users, access to resources, budgeting and cost control etc. Impact is where all this hits the user and they get something they think is of value and makes a difference to their professional practice. In a sense the impact bit should be easy. It is the culmination of all the pieces that make up a library and knowledge services. This may not help but if you are struggling with this a broader perspective can inspire you to think more positively about impact. It’s about the journey as well as the destination.

#7 Polls and elections

The difference between opinion polls and the (general) election, is the difference between what people say they do and what they actually do. Asking people what they believe about using the library and knowledge service is only half the story. It’s the opinion poll. However, the means by which we communicate with users, for example through social media, also provide us with metrics which tell us what our users actually do. That’s the election bit. Yes of course the analogy isn’t exact but it’s a good corrective to find a metric to check what you are told. Is the CAS useful? How many people actually clicked through from your CAS. Is this website / blog useful. How many people actually visited the page? Use library resources? How many people actually log on?

#8 Building in impact measurement

It’s easy to recognise a good model when you see one. It can be harder to put into practice. Although I struggle with applying this model and it’s hard to retrofit to an existing service, highly recommended is Markless and Streatfileds Evaluating the Impact of your library[2]. Even my semi successful attempts to do this have been useful in framing thinking about impact measurement. The key is to think carefully about the questions you want to ask and build in the impact measures as you design your service.

#9 Personal Impact

What’s your professional impact? Metrics, citation counts, research impacts and alt metrics are all measures of impact. It’s useful to curate your personal impact as well and it’s never been easier. Not everything that is counted counts of course, but social media gives you an insight into your own professional impact from Twitter, blogs, professional profiles and other online communications. It’s an interesting game to play. It’s probably a game being played by many of our users who have to demonstrate research impacts.

#10 Help is at hand …

For those working in the NHS in England can use the excellent survey toolkit accessible here provided by the Library and Knowledge Service Leads [LKSL]. Colleagues who are thinking about Impact for their LQAF assessment have access to the support and examples of good practice from peers.

Matt Holland, Librarian NWAS NHS Trust BA, DMS, Dip Lib, MA, MCLIP Mobile: 07747456736 E-Mail: Matt.Holland@nwas.nhs.uk Website: http://www.networks.nhs.uk/nhs-networks/nwas-library-and-information-service NWAS LKS is supported by NW Health Care Libraries Unit (HCLU)

[1] Wikipedia. Campbell’s Law. Available from: http://en.wikipedia.org/wiki/Campbell%27s_law [Accessed 04 May 2015].

[2] Markless, S. & Streatfield, D., 2012, Evaluating the impact of your library. 2nd ed. London: Facet.

This entry was posted in Top Tips and tagged , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s