Measuring innovation: how logic models can help

May 2021  |  SPECIAL REPORT: BUSINESS STRATEGY & OPERATIONS

Financier Worldwide Magazine

May 2021 Issue

Innovation is notoriously hard to measure. Articulating it, managing it and monitoring it are challenging tasks in any industry. But logic models – a tool from the world of public policy – can provide clarity among the confusion. Here, we look at why measuring innovation can be so tricky, and how logic models can help.

Innovation projects are complex

Most innovation programmes are multi-faceted. Rather than focusing on a single metric, they include a range of different objectives. These will be interlinked in various ways – from improving customer experience, through to generating new business and enhancing cost efficiency.

Different parts of the organisation will have different aims. And to complicate things further, innovation projects usually encompass both short-term, incremental change and long-term, transformative disruption. Put simply, it is a lot to manage – and to measure.

Evaluating progress is vital

An innovation programme usually takes the form of a portfolio – one that includes several diverse initiatives that mature at different paces. That means a well-implemented system is necessary to evaluate performance.

After all, businesses need to: (i) identify successes and failures; (ii) determine when to halt policies that are not working and accelerate those that are; (iii) promote initiatives internally and incentivise teams to carry them out; and (iv) report overall progress to top-level management.

The benefits of a system that does all these things are obvious. But implementing such a system is far from easy.

Setting metrics is difficult

One of the biggest hurdles is measurement. It is a real challenge to establish a set of metrics that can provide big-picture clarity on overall progress, while also providing sufficient detail about each individual initiative.

The level of difficulty increases further when you need to monitor slow-burn, long-term innovations alongside short-term objectives that produce outcomes from day one.

An example: UK retailer

These difficulties are illustrated by the case of a major UK retailer. The company had a mix of innovation initiatives in progress, aiming to meet a number of high-level goals – like growing online sales and increasing revenue.

To evaluate performance, it was tracking a broad set of metrics. These included high-level indicators like customer satisfaction, and more specific metrics like the number of people signing up to a new app. So far, so good. But with this mix of measurements to manage, and with several parts of the business involved, it soon ran into problems.

The retailer’s high-level metrics were affected by multiple initiatives, so it was hard to define causal links. There was no clear connection between the overarching metrics and the more specific ones, and there was a gap in capturing intermediate results for long-term innovations.

What is more, while established metric types were suitable for incremental innovations, they were proving less effective for new products and services. The business had seen a 658 percent increase in use of its home delivery service, for example – but how could it tell how much this was down to the pandemic, and how much was down to innovations in its service?

Logic models: what are they?

It was clear that the retailer needed some clarity. To take a step back and apply some big-picture thinking, the perfect tool for them was logic models.

A logic model is a step-by-step pathway, outlining how to move from input to impact. It is considered a best-practice tool in measuring the outcomes of public policies and can be very useful in commercial contexts too.

The model breaks down evaluation into steps, from initial inputs, to the activities they generate, and then to the outputs, outcomes and impacts produced. The basic structure of a logic model looks like this: step 1: inputs – resources used (usually time or money); step 2: activities – initiatives carried out using the inputs; step 3: outputs – things that result from the activities (usually short term); step 4: outcomes – things that result from the outputs (usually medium term); and step 5: impacts – changes that ultimately result from the outcomes (usually long term).

Let us take the process of building a sales app as an example, in a very simplified format for illustration purposes. Step 1 – inputs – would be the money invested. The activities carried out would be the building and testing of the app, and the output would be the app itself. The medium-term outcome would be customers using the app to buy products. And the overall impact on the business would be new revenue generated.

It is a helpful way of outlining the causal link between each step, giving businesses a greater sense of control over the entire process.

Logic models: how do they help?

Logic models help to guide innovation programmes at all stages: implementation, monitoring and evaluation of outcomes.

Perhaps most importantly, they make measurement much more straightforward – especially as different stages of the model tend to suit certain metric types. Inputs and outputs are best tracked using directly measurable units: resources like money and hours for inputs, and key results for outputs. Outcomes are best measured as new or changed behaviours, observed from customers or employees. And impacts are best assessed by overall results for the business, usually revenue generation or cost reduction.

Logic models can also help identify unintended consequences in measurement – like the ‘substitution effect’, when an increase in online sales is explained by a fall in in-store sales.

They can even highlight existing behavioural barriers, where nudges may be needed, such as offering instant rewards to customers to overcome the effort barrier of downloading an app.

Get a clearer picture of innovation

Experience shows us that logic models work particularly well in managing and monitoring innovation. They allow organisations of all types to obtain a clearer picture, and greater control, over their often-complex innovation projects.

Logic models help to categorise objectives: short term and long term, overarching and granular. And they provide clearer direction on how to measure different objectives, replacing blurred boundaries with logical steps.

This means clearer reporting, communication and initiative-setting , from day one to the long-term future – all of which are outcomes that any organisation implementing change can get behind.

 

Paula Papp is associate director, Laura Petschnig is a manager and Kalina Kasprzyk is a senior consultant at Frontier Economics. Ms Papp can be contacted on +34 650 341 017 or by email: paula.papp@frontier-economics.com. Ms Petschnig can be contacted on +44 (0)20 7031 7025 or by email: laura.petschnig@frontier-economics.com. Ms Kasprzy can be contacted on +44 (0)20 7031 7197 or by email: kalina.kasprzyk@frontier-economics.com.

© Financier Worldwide

BY

Paula Papp, Laura Petschnig and Kalina Kasprzyk

Frontier Economics

©2001-2024 Financier Worldwide Ltd. All rights reserved. Any statements expressed on this website are understood to be general opinions and should not be relied upon as legal, financial or any other form of professional advice. Opinions expressed do not necessarily represent the views of the authors’ current or previous employers, or clients. The publisher, authors and authors' firms are not responsible for any loss third parties may suffer in connection with information or materials presented on this website, or use of any such information or materials by any third parties.