HEFCE Project
Researching and developing a framework for the evaluation of academic and support staff learning and development in Higher Education
Our project
With success from HEFCE’s Leadership Governance and Management fund, a partnership led by Newcastle Business School and Northumbria University worked in collaboration with Oxford Brookes University and University College London (UCL) to investigate how staff learning and development is evaluated in Higher Education Institutions and how it could be improved and enhanced.
What is it about?
The key output is a sector-specific toolkit which consists of a framework of flexible evaluation tools which can be used to seek the value of staff development initiatives at the individual, functional and corporate levels. The toolkit has been developed over a period of 18 months and has been informed by best practice across public and private sectors, supplemented through a review of best practice literature along with focus group work and pilots within the project partner institutions. Universities can choose which of the tools they would like to utilise and then download and customise them for their own events and programmes. Importantly, all of the resources are free for UK HEIs.
Demand for the Project
The recent downturn in the economic climate has placed an ever increasing focus and pressure (within and without HE) on ‘Return on Investment’ and ‘business impact’ (CIPD 2008) of staff learning and development. Many HR Departments are reporting or are expecting a decrease in investment for staff development activity as a result of general financial pressure within businesses and the economy.
Discussions with colleagues in HE indicated a need for a bespoke HE tool that evaluates beyond the reaction (how the delegate ‘rates’ the event) and the perceived outcomes (what delegate thinks they may have learned) for a measurable outcome (an ability to measure the effects on the individual and the business). Evaluation of the impact of the Rewarding and Development Staff monies (which were injected into the sector c 2002-2005 has further highlighted the need for an evaluation toolkit for HE as it would appear that evaluation was restricted to inputs (the amount of money spent, and on what it has been spent) rather than outputs (little or no formal measurement of increased knowledge, capability or behaviour improvement) or outcomes (the impact or effects on the business).
The above clearly highlighted the need for HR in HE to establish a range of tools which will support HE to explore methods for assessment of the value of staff learning and development to institutions, in order that senior managers (and HEFCE ref RDS monies or similar investment bodies) have clearer measures/evidence of value for money/cost effectiveness of resources allocated to staff development.
The Design Stage
The design and production of our toolkit has been driven through identification of best practice from inside and outside of the Higher Education sector (throughout the world) as well as gathering information on what Universities need to meet their needs.
We have worked our way through the following five stages:
Stage 1 saw the team produce a critical review of theoretical models and literature to identify best practice and strategies to overcome potential barriers to effective learning evaluation. The development of an ‘environmental scanning’ document (which explores current and future challenges in the higher education sector, and presents both SWOT and PESTLE analyses) was also undertaken.
Stage 2 began with pilots of data collection before we finalised our strategy. We undertook focus groups and interviews with diverse groups across the three institutions to find out about their experiences, perspective and requirements. In addition we analysed the documents and supporting structures for learning, development and evaluation in each University.
Stage 3 involved the analysis of all of the data that we had collected so that we could start to make sense of our findings and create the outline of our toolkit.
Stage 4 gave us the opportunity to link our findings to the literature and best practice that we had identified earlier in the project. At this stage our ideas for the toolkit really began to take shape.
Stage 5 was the final element of the project and this was where we developed all of the tools that come together to form our toolkit. The tools were piloted with real learners and facilitators as well as within our project team and feedback was incorporated into the final design. At the end of this stage we had completed our toolkit with 47 customisable and sector-specific tools and started to share our work with the wider community.
Introduction to the Toolkit
Over thirteen chapters we have created a total of 47 tools that can be used for free by UK Universities to evaluate different elements of their learning and development provision. The tools have been designed so that they can be adapted to meet the needs of individual Universities and the diverse programmes they offer.
Download a sample of the content from the toolkit.
All of the 13 chapters can be downloaded for free on this website, please contact Dr. Fiona Robson at f.robson@northumbria.ac.uk to receive the password which will unlock all of the files.
Contents and introductory chapter
Chapter 1: The case for evaluation (MS Word)
Chapter 1: The benefits of learning evaluation (MS Powerpoint)
Chapter 2: Where are you now? A self-evaluation tool for Universities (MS Word)
Chapter 3: Before identifying and attending learning & development interventions (MS Word)
Chapter 4: Standard skills/knowledge based interventions (MS Word)
Chapter 5: Evaluating teaching & learning and research-focused interventions (MS Word)
Chapter 6: Evaluating management and leadership interventions (MS Word)
Chapter 7: Evaluating senior management and strategic interventions (MS Word)
Chapter 8: Qualitative methods of obtaining evaluation data (MS Word)
Chapter 9: Calculating return-on investment (ROI) (MS Word)
Chapter 9: ROI calculation spreadsheet (MS Excel)
Chapter 10: The skilled facilitator (MS Word)
Chapter 11: Enhancing the evaluation process (MS Word)
Chapter 11: Enhanced evaluation (MS Powerpoint)
Chapter 12: Evaluating coaching interventions (MS Word)
Chapter 13: Resources for the HR team (MS Word)
Chapter 13: Completed evaluation form spreadsheet (MS Excel)
Supporting Resources
The Project Management Team
Professor Sharon Mavin, Dean of Newcastle Business School
Lesley Lee, Deputy Director of HR, Northumbria University
Dr. Fiona Robson, Senior Lecturer in HRM, Newcastle Business School
In collaboration with:
Louise McDonald, Northumbria University
Ben Dale, Northumbria University
Rosemary Davies, Oxford Brookes University
Richard Laughlin, University College London (UCL)
