Measuring impact challenges

Nov 8, 2022

Notify me of new releases

Measuring impact or impact evaluation is a complicated and frail science. Having worked several years in healthcare, a lesson learned is to bring impact measures far closer and into your realm of control. In this way you save, time, budget and resources building your own evidence for impact, rather than chasing measures that are far from reach.

Key takeaways

I have worked on a lot of projects that have promised impact to improve health and wellbeing of the population, to create change through innovation. All of them promise so much and I have yet to see a successful, measurable impact outcome.

The difficulty is setting an impact goal when the outcome is out of your control. Health is influenced by many programs and conditions which makes it difficult to attribute change to any single one program, yet funders and purchases of programs or interventions want to see evidence of impact even though it’s an unrealistic measure to attain.

For example: a six-lesson swimming program to teach basic skills in the event of an emergency (which was evidence based and world class acknowledged). Had an impact assessment which showed nil to negligible skills gained result. Why? Because the average attendance was just one lesson .. it was impossible to yield a successful result!

This is a frailties of impact evaluation – when you look at the program in isolation of the performance of the product or program across its entire customer/user journey.

Impact managers are typically appointed to well-meaning social businesses without a real-world cause and effect correlation, or means to measure and demonstrate how it’s really going to impact social problems.

Over the years, I’ve refined a program called an Evaluation Framework.

One thing I learned is that if you create a framework of measuring, that measures the real time program and product performance in terms of: how many people; what they think about the program; and how well the program and business is surviving and thriving as a commercial outcome. Then you have a viable way for businesses to measure impact by measuring the areas in their control.

Evaluation Framework

By aligning with existing secondary research to make statements about benefits gained, there is no need to wait for longitudinal research or random control trials.

My recommendation, if you’re going to do impact monitoring and you’re going to spend money on it, start at the beginning of the relationship journey … don’t start at the end.

Links

Shared Value Project – Shared value is a framework designed to create business solutions to social and environmental problems. Put differently, it’s a means to deliver on your purpose, profitably.
Measuring Outcomes: STRENGTHENING NONPROFITS
Theory of Change.org – to be honest, I haven’t found a good book yet, what do you suggest?

Timestamps: