Measuring Adoption

Rob Wilson, Business Unit Director – Application Solutions

We all know it is important to measure return on investment (ROI) when implementing a new solution. Solution ‘x’ saved ‘y’ dollars by requiring ‘z’ fewer hours from employees ‘a’ and ‘b.’ But what about those projects where the cost savings are less quantifiable? How do you measure the success of a project, for example, whose sole purpose is to streamline communication? Or what about a project aimed at improving customer satisfaction? When there is not a direct impact on the bottom line, how will you know if your project succeeded? You measure the outcomes.

Before you can measure the outcomes, you need to identify them, and determine the appropriate metrics to measure. A recent software adoption course I attended suggested using a format such as this to position your initiatives:

While I appreciate the spirit of this exercise, it would be better if it were more objective. Eric Ries, author of The Lean Startup, suggests an approach that is based more on the scientific method. According to sciencebuddies.org, the steps of the scientific method are:

• Ask a question
• Do background research
• Construct a hypothesis
• Test your hypothesis by doing an experiment
• Analyze your data and draw a conclusion
• Communicate your results

A simpler, more scientific way of articulating your objectives and measuring their adoption may go like this:

We believe that our customers would appreciate a more convenient way to pay their bills.
We will test this by creating an online form for customers to pay their bills on our web site.
We will know we have succeeded if 25% or more of our customers have paid 3 bills online within a year of launching the feature.

How you come up with that final statement is a topic for another day. You will get better at analytics with experience. I will say that measuring adoption over time is key, so do not get too disappointed if early adoption is slow, or too elated if your early numbers are through the roof. Your metrics should have some factor of time involved when measuring adoption. This brings me to the next point: how do you present your findings? You should not need a degree in reading log files to measure adoption. A better approach would be to build your objectives and outcomes right into the application itself, and display them on an administrative dashboard. In this case, a chart showing the trending in online payments – particularly by repeat customers – would be most effective.

Let us consider another example. Suppose your objective is to improve communication of your company policies with your employees. Emailing a policy to your staff would be difficult to track. Emailing a link to a policy on your intranet would be a step up – if you do not mind parsing log files. Ideally, however, you would email a link to your policy where it sits on SharePoint or some other platform, and with a little custom code, SharePoint can log a list of everyone who has read each policy (even a particular version of a policy). Creating some quick views would allow you to view metrics and provide the tools you need to nudge holdouts on key policies.

In summary, if you want to measure the success of a software initiative, you should determine the desired outcome, decide how you will measure it, and agree upon which results will show you have succeeded. Restate your value hypothesis on a scorecard on an admin dashboard, and track the results over time. I trust you will find that a focus on adoption such as this will greatly improve your decision-making over time – including knowing what features are important to your audience and justifying what projects should be implemented. If you would like help getting started, contact us for a consultation on software adoption strategies.

Contact your Account Manager at Keller Schroeder for more information about our Applications Solutions Group  and how it might benefit your organization.