At the last LIHNN (Libraries in Health North West Network) Trainers’ meeting we discussed evaluating training and measuring impact. Clinical Librarians are almost always involved in delivering training and we most definitely want to measure impact.
I offered to collate training evaluation questionnaires used around the LIHNN network so we could come up with a ‘question bank’ for library and knowledge trainers ‘dip into’. This would build on the Training Evaluation Survey http://www.lihnn.nhs.uk/images/Documents/LIHNN/Trainers/TES.pdf document that had been produced a number of years ago by the LIHNN Trainers group. At Pennine Care we have been using some of the surveys from this document, but want to build on it to gain more information about impact and to help us improve our sessions.
I had already begun to read around the topic and our training department has been discussing evaluation and impact too. Trainers often use the Kirkpatrick model to help them evaluate training on different levels.
“The four levels of Kirkpatrick’s evaluation model essentially measure:
- reaction of student – what they thought and felt about the training
- learning – the resulting increase in knowledge or capability
- behaviour – extent of behaviour and capability improvement and implementation/application
- results – the effects on the business or environment resulting from the trainee’s performance”
Read more about it at http://www.businessballs.com/kirkpatricklearningevaluationmodel.htm
Most organisations are good at level one: happy sheets, questions about facilities and immediate reaction to the training.
Level two is a bit trickier for information skills training as we don’t have formal tests, but we do observe and talk to our course participants and a good trainer will also throw questions back to the audience.
Then, for level three, we want to know if they have changed behaviour when they are back in the workplace, this is even trickier!
Level four looks at results and impact, I would argue that we can be good at finding out the impact of our training on an individual basis by collecting case studies, but when it comes to reporting organisational impact we may struggle.
I have just finished collating the surveys we use in LIHNN (the ones I received) and will send the list with comments to the next meeting.
In the meantime I am sure the trainers group will ponder the levels and have a discussion at the next meeting (I won’t be there), and possible set up a working group. I think the library and knowledge trainers, and their managers, need to be thinking about collecting more of the second, third and fourth levels of Kirkpatrick’s model. I have a few ideas and possible solutions which I will add to this blog soon!
Any comments on this post would be welcome.
Lucy Anderson BA (Hons) MCLIP
Knowledge Specialist: Outreach and Liaison
Pennine Care NHS Foundation Trust