Comments on: Here’s How to Measure ROI in the Real World

By: Why E-Learning is So Effective » The Rapid eLearning Blog

Why E-Learning is So Effective » The Rapid eLearning Blog — Tue, 02 Feb 2010 08:32:44 +0000

[...] consuming whether it’s online or not. With elearning, each time the course is accessed your return on investment improves because you are dividing the fixed production costs by number of uses. You also have [...]

By: Inspire Action » The PowerPoint Perspective

Inspire Action » The PowerPoint Perspective — Fri, 21 Nov 2008 18:15:03 +0000

[...] should be when you put it online. Tom Kuhlmann, an online training expert talks about this in his blog on measuring the ROI for online learning. He explains that you need to align the course or presentation outcome to measurable goals. I [...]

By: tom

tom — Wed, 29 Oct 2008 10:37:02 +0000

I think there are some interesting insights in here. I appreciate what’s been added. I typically lean more on the practical business side of things. However Karyn’s post reminds me of a book I once read.

The essence of the book was that sometimes you do stuff not so much because it’s practical or measurable, but you do it for the perception of value. I had a friend who worked for a company that I thought was quite progressive in how they trained. They set him up with all of the resources. Put him in a lab with experienced SME. Gave him a mentor. Etc. It was a really well designed training process and he was up to speed very quickly.

However, he complained that they never “trained” him. I asked what he expected and he basically wanted to go into a room and have a more formal training program with lectures and elearning courses.

By: Jeffery Goldman

Jeffery Goldman — Sat, 25 Oct 2008 17:53:51 +0000

When it comes to ROI, I have found my saving grace is the more frequent use/availability of contact management systems, and referral/sales tracking systems. I create, and evaluate, sales and product knowledge e-learning programs. Using the staff who did not take the course (there are always those who “don’t get around to taking the e-learning”) as a control group, many correlations can be made when comparing the two groups. The fore-mentioned systems provides a lot of data to compare and use of a control group eliminates many of the extraneous factors, such as seasonality, economics, marketing campaigns, etc.

I feel it is important to note that prior to attempting ROI, one MUST first progress through all 4 levels of evaluation (reaction, learning, behavior, results). For example, you will save yourself a lot of aggravation if you identify if they are even learning anything prior to seeing if it is affecting behavior.

By: Making Change » Elearning ROI: Can we lead the way?

Making Change » Elearning ROI: Can we lead the way? — Thu, 23 Oct 2008 15:11:28 +0000

[...] to show that your elearning adds value? Tom Kuhlmann recently suggested these [...]

By: Paul McKelvey

Paul McKelvey — Thu, 23 Oct 2008 14:14:21 +0000

Measuring the business effect of training is a job security task in this economic climate. If you can’t tell how training is affecting business outcomes, then you need to figure out how to measure it. Are sales up? Support calls down? Support calls shorter? Is it cheaper to do something?
ROI is actually an abbreviation with multiple meanings. For a front line manager, the objective of the training (reducing call times, for example) is what ROI means. For a manager of managers, dollars saved or headcount reduced become more important. The key thing is to answer the question with the asker in mind.
Because every training course must have a business need, the results can be measured by studying how well the course met the business need. If there is no business need for a course, then there is no reason to do the course.
Can we say in absolute certainty that training caused the ROI? Of course not. Too many things affect employee performance simultaneously, including factors over which the business has no control. We can, however, infer the effect of training. If sales go up 50% immediately after a training course, the likelihood is that some, if not all, the increase is due to the course. Businesses do not operate on certainty. They operate on probability without having all the facts to make a completely reasoned decision. As people engaged in the business of training, we need to enter the reality of our customers.

By: Kevin Thorn

Kevin Thorn — Wed, 22 Oct 2008 14:35:52 +0000

Karyn nailed the idea of an intangible value. David also points out that in business unfortunately we don’t get to the “table” without some kind of measurable value.

In Tom’s first topic – Customer Satisfaction = Success you cannot go wrong.

An observation: We focus so much on results that often times the quality of training is second to the compliance of who completed it. We know results are driven by performance; performance is driven by behaviors; behaviors are driven by thoughts. Yet, the attention on ‘results’ gets more traction.

I looked at this some time ago and tested this theory by flipping it the other way around – Think, Behave, Perform, Result. Given the learner the tools and resources to ‘think’ about their task and how to do it better, then suggest a change in those thoughts to change their ‘behavior.’ The goal is they would walk away from the training and ‘perform’ beter. Thus, the ‘results’ will take care of itself.

Back to ROI and the Customer. Various surveys sent to customers that focused on particular outcomes on how the learner was to ‘behave’ and ‘perform’ proved successful. Happy customers spend more money!

By: links for 2008-10-21 | the markfr ditherings

links for 2008-10-21 | the markfr ditherings — Wed, 22 Oct 2008 01:05:01 +0000

[...] » Here’s How to Measure ROI in the Real World The Rapid eLearning Blog (tags: roi training performance consulting) [...]

By: Paul Morrall (Australia)

Paul Morrall (Australia) — Wed, 22 Oct 2008 00:32:06 +0000

Great article and follow up comments.
Having worked both sides of the table I can see how it is sometimes difficult to promote the value of a learning intervention. Jack is correct about using ROE rather than ROI when it comes to meeting expectations. However, sitting down with your client (internal/external) and getting a good brief will ultimately guide you in whether they are looking for a financial (direct/indirect) return on the project. I thinks will see less of the “nice” to have and more of the “need” to have learning programs in the future as every organization watches it’s dollar. I found asking the question “why” is a real good start as I often like to start with the end in mind.

By: David Harper

David Harper — Tue, 21 Oct 2008 22:11:08 +0000

Karyn’s example is interesting and touching. But, one, a family isn’t a business; the family doesn’t need to optimize ROI. The family is thankfully free to pursue many different undefined agendas. Two, she’s sort of made the indirect case. The investment in intangibles produce value, only it’s hard to quantify the value.

But more importantly, the philosophy reflected is a retreating strategy. HR/training/learning types need to earn a seat at the table by understanding the language of business and connecting their contribution to the company’s overall goals. The retreat into “this is subjective” or even “this is intangle, indirect, and long-term in payofff” will always be doomed because it can justify any expenditure.

Businesses exist to produce some sort of ROI. Period. (metaphorically, i mean, ROI is imprecise and has no precise financial definition. In many organizations it would be a confusing goal owing.). Only efforts that support the goal can be justified. Sometimes the issue is short-term/long-term; i.e., we make hard investments today in return for intangible and long-term returns. Okay, that makes it more difficult, but it’s no excuse for not trying to tie an investment to something that is eventually concrete. Otherwise, the business is actually forsaking its fiduciary to make such investments. ROI ultimately means, it was worth doing; lack of it ultimately means, not worth doing or we don’t sufficiently understand our investment.

David