Evaluating Digital Transformation with Kathrin Cresswell

In this episode, IFIC Chief Executive Niamh Lennox-Chhugani is joined by Professor Kathrin Cresswell, Professor of Digital Innovations in Health and Care at the Usher Institute, University of Edinburgh.

Kathrin is a social scientist with extensive experience evaluating large-scale digital transformation programmes, including the National Programme for IT, the Global Digital Exemplar Programme, and most recently the NHS AI Lab. Drawing on this work, she reflects on what formative evaluation can offer complex, digitally enabled change in health and care.

The conversation explores why impact evaluation alone is rarely enough in complex systems. Kathrin makes the case for formative and process evaluation that is embedded early, identifies emerging risks, and supports programmes to adapt in real time. Together, they discuss why some evaluations “sit on the shelf,” the tensions between independence and partnership, and the challenge of demonstrating impact when digital interventions can take years to stabilise.

Looking ahead, Kathrin argues for evaluation that is closer to practice — co-constructed with frontline teams, focused on learning, and continually asking whether an intervention is truly addressing the need it set out to solve.

Key insights from Kathrin Cresswell

On formative evaluation
“You become part of the intervention… you make it work.”

On the limits of traditional impact studies
“By then, you have no idea how it works… and you miss your chance to look at how you could make it work.”

On expectations of rapid impact
“Asking after two years whether a programme was successful… is absolutely crazy.”

On evaluations that lack real learning
“They’re the ones where the people who commission you want you to find something that they know in advance.”

On being involved early enough
“We always come in too late.”

On staying focused on purpose
“We need to keep coming back to what need this thing is meant to address.”