Evaluating the effectiveness of CPD - Health Care

It is inevitable that some types of CPD will be more effective than others. However, given that we all have limited time and money to invest in CPD we have to work smartly to ensure that the CPD with which we engage is worthwhile.
It is obviously difficult to know in advance how valuable a CPD course or programme will be but, as mentioned elsewhere in this chapter, careful planning around whether, for example, it will meet your needs in terms of knowledge or skills deficits you are aware of, or if you will have the opportunity in your job role to deploy your learning, are imperative. Furthermore, where possible you should ask the provider of the CPD for feedback and case studies from participants on the course or programme and the employers of the participants. Obviously, in reality, most providers will showcase only the positive feedback, so where possible you should ask to speak directly to previous participants and their line managers.

Whether you speak to previous participants or have to rely on case studies or sound bites, there are frameworks that can help us dissect how valuable a given programme of work-relevant learning may be. For example, The Kirkpatrick Training Evaluation Model (Kirkpatrick, 1998; Kirkpatrick and Kirkpatrick, 1994) suggests that learning can be divided into four levels
Level 1: Reaction. The first level in Kirkpatrick’s model is called ‘Reaction’ and relates to whether students found the learning relevant to their needs and if they enjoyed the learning. This is normally assessed by a feedback sheet at the close of the learning event.
Level 2: Learning. The second level is called ‘Learning’ and considers how students have learnt and retained the knowledge, skills and attitudes covered by the CPD. This level is assessed properly with a pre- and post-course test, such as a written examination or an OSCE.
Level 3: Behaviour. The third level is called ‘Behaviour’ and is concerned with whether the accumulated knowledge and skills are deployed during day-to-day workplace activities, that is whether or not practice was changed as a consequence of the CPD. This element may require interaction with the student’s line manager and colleagues, perhaps through feedback during an appraisal or a 360° feedback exercise, to explore how the student’s approach and performance have changed and developed and whether this is directly linked to the CPD.
Level 4: Results. Finally, level four is called ‘Results’ and considers if the CPD has ultimately had an impact on key deliverables. For example, in healthcare settings these may include target measures such as increasing patient satisfaction scores, reducing levels of recorded post-operative pain, reducing so called ‘never events’, or increasing the use of cheaper (but equally effective) medical equipment. The ‘Results’ level can be challenging to measure, as these impacts may take time (often months or years) to filter through to the workplace and determining whether or not the CPD was a causal factor for the change can be difficult to measure.
Kirkpatrick’s model suggests that for work-relevant CPD to be deemed effective it should have a positive impact on each of the four levels. Therefore, when you see feedback from previous students you should carefully analyse what they say. For example, student A might have said ‘The course was fantastic, I had a great time and learnt a lot about new approaches to dementia care. I would recommend it’ while student B said ‘The course was challenging in a good way, I was quickly able to learn new ideas and change my approach to working with service users living with dementia – and the feedback from service users’ carers was fantastic’. Student A is perhaps reflecting that the CPD course s/he went on was effective for them at Kirkpatrick levels one and two, whereas Student B is implying that for them the learning was effective at levels one, two and three.

Time for reflection
Consider a recent learning activity you have undertaken. On which of Kirkpatrick’s levels would you consider it to be effective and why?


An outline practical approach to planning and undertaking a balanced CPD portfolio
Any meaningful approach to CPD, irrespective of your clinical discipline, is underpinned by adhering to a set of key principles:
Recording: Being meticulous in the way in which you keep a record of both your formal and informal learning.
Reflecting: The act of capturing your thoughts in writing, or via a voice recorder, can support your learning and help you consider how you are transferring the learning to practice.
Tracking: Look back every few months and consider what types of learning you have engaged with. Is it all formal or informal or is there a balance? Is it all specialist or generalist, or is there a balance? Is one project or CPD course dominating your efforts? Are you only engaging with topics you are comfortable with, or are you pushing out into new areas? Is your recording of CPD helping you to learn and document your learning, or is it inadequate? What is missing from your CPD mix?
Planning: Once you have worked out what is missing from your CPD mix you need to plan how to respond to the identified gaps. Are you using your workplace appraisal to consider your performance and how CPD to improve it? Are you seeking feedback from colleagues to gain insight on your CPD needs? Are you thinking sufficiently about your career development plans and how this may impact your CPD agenda?

Reviewing: Through the process of Recording, Reflecting, Tracking and Planning, you should be able to set out a clear examples of forward-looking CPD objectives. These objectives should be reviewed regularly by yourself, your line manager, mentors and other interested parties.
In terms of further building firm foundations to your approach to CPD, it is important to start out by really contemplating where you are in terms of your own development. It is often helpful to look at your current job specification, or if you are a student to look at role specifications for the types of jobs you would like to secure upon graduation. You may also want to reflect on your own CV and any recent reviews, such as your last performance appraisal, meeting with a mentor or a tutor. Consider four interrelated areas:

Job-specific skills and knowledge: The specific knowledge, skills and attitudes you have now that pertain to the job role you have now or aspire to acquire, and the gaps you perceive you have.
Capabilities: These are more generic competencies you may need in the workplace. For example the ability to be calm under pressure, lead new projects, manage a budget or listen to constructive feedback. Which key capabilities do you possess and which are you lacking? Also consider which of these competencies may be transferable into other roles or work settings
Work values: Give some thought to your values and what gives you satisfaction in your work. For example, are you driven by the challenge of being the best, leading the implementation of new approaches to work, acting as an advocate for service users, being the person in a team on whom others depend or feeling secure in your role. What drives you and leads to job satisfaction?
Interests: Also consider areas of life, both at work and beyond, that you find stimulating and rewarding. For example, do you enjoy organising events, reading, supporting others to bring the best out of them, being competitive and reaching targets, or helping people to learn?
From this described approach it can then be easier to sketch out an outline plan for the future. Clearly this plan may change, but writing down some aspirations can be helpful. For example:
In 100 words or less think about where you aspire to be in two, five and ten years’ time.
In 100 words or less capture the strengths that will help you to achieve your stated aspirations.
In a further 100 words consider the areas that require development which unless attended to will hold you back.
This will then enable you to start to set some clear CPD objectives that you can begin to develop and refine. From this, an action plan should start to emerge.


Conclusions
In summary, this chapter strives to get you thinking about your understanding of the role of CPD and its link to your current and future employability. CPD plays a significant role in ensuring that you provide safe and effective care to your patients, meet the demands of your employers, and develop a career that is rewarding. The learning in this chapter should enable you to think about how you plan, design, learn and apply your CPD to your job role.Potential interview questions in this area

Outline, with possible examples, the CPD you would like to undertake over the next 12 months. Why, in your view, will these lead to improved patient care?
How have you identified in which areas you require CPD?
Have you developed a CPD plan?
How would you ensure that the learning from CPD is transferred into your practice?
Are you aware of the different types of CPD your regulatory body require you to undertake?

References

Archer W and Davison J (2008) Graduate employability: The views of the employers. London: the Council for Industry and Higher Education (CIHE).
Blume B, Ford JK, Baldwin T and Huang J (2010) Transfer of training: A meta-analytic review. Journal of Management, 36:1065–1105.
Flanagan J, Baldwin S and Clarke D (2000) Work-based learning as a means of developing and assessing nursing competence. Journal of Clinical Nursing, 9:360–386.
Joyce P and Cowman S (2007) Continuing professional development: Investment or expectation? Journal of Nurse Management, 15:626–633.
Kirkpatrick DL (1998) Evaluating training programs: The four levels, 2nd edn. San Francisco, CA: Berrett-Koehler Publishers.
Kirkpatrick DL and Kirkpatrick, JD (1994) Evaluating training programs. Berrett-Koehler Publishers.

Further reading

Brekelmans G, Poell RF and Van Wijk K (2013) Factors influencing continuing professional development: A Delphi study among nursing experts. European Journal of Training and Development, 37:313–325.
Covell CL (2009) Outcomes achieved from organizational investment in nursing continuing professional development. Journal of Nursing Administration, 39:438–443.
Gibbs V (2011) An investigation into the challenges facing the future provision of continuing professional development for allied health professionals in a changing healthcare environment. Radiography, 17:152–157.
Haywood H, Pain H, Ryan S and Adams J (2012) Engagement with continuing professional development: Development of a service model. Journal of Allied Health, 41:83–89.