Proving Training Value through Measurement

by Brian Roseman

In August of this year, NCI launched a Training Performance Measurement System on our website. What does that mean for you?

In a nutshell, you can now see the actual knowledge gained from attending NCI training. You can also see reviews of the classes from the actual attendees. It matters because you can now be sure that your training dollars are being wisely spent.

Let’s look a little more in-depth about this new system, and what it means for you.

“If you don’t measure, you’re just guessing”

You’ve heard that line from us here at NCI over the last two decades or so and know that they’re words we live by.  The concept of measuring performance is key to NCI’s modus operandi. So we’re sharing some of our data to prove the performance of our training.

In the training and education world, there’s a process known as the Kirkpatrick Four-Level Model of Training Evaluation. That process encourages organizations to evaluate training. It offers specific guidelines to measure performance.

That caught our eye. Here we are talking about performance as it relates to HVAC systems and companies. Now we have a process to measure our main product – how well we train those whom you entrust us to educate and inform. Nowhere else in the HVAC industry will you find such data, such proof-of-concept. This is the epitome of what NCI means when we talk about “Performance-Based Contracting.” To see exactly how this Training Performance Measurement System works, we have to start at the beginning: with the student in the class.

At the start of every training event, students take a brief test to see what knowledge they already have on the subject. After the two- or three-day class ends, students take another test.

This confirms their knowledge of a topic and whether they qualify for certification (if the training includes certification). It also allows you to compare where they stood when they walked in the door to where they stand after the NCI training. The comparison is called their “knowledge increase” — a statistic gathered, from years of training that is now visible at a glance on the website.

You can see a screenshot of the knowledge increase here:

The bar chart to the left in the graphic is generated by real data, from actual training attendees. Check back in a month or two and the chart will change, as we upload new data. The chart and numbers it displays aren’t images – they’re dynamically generated codes representing true pre- and post-training knowledge from attendees.

Representing Evolution and Change – Inside and Out

We’ve collected this data for years to make sure our training is actually educating the industry. We always want to make sure our instructors are:

• Successful instilling the information to the attendee
• Always on the front edge of the HVAC industry, teaching things that can’t be learned anywhere else
• Sharing the big picture of how Performance-Based Contracting can revolutionize your business.

This visualized data lets you know if you’re spending your resources effectively. You can trust that this data is not inflated or distorted in any way.

It may be surprising that Combustion Performance and Carbon Monoxide Safety has more than 50% knowledge improvement. In this area there is much to learn. Anyone who’s been through the class knows that it’s eye-opening, and there’s nothing quite like it! Or quite like Jim Davis for that matter…

But wait, there’s more!

Big data does seem to lack that ‘personal touch’, doesn’t it? Luckily, we’ve got that covered.

The other part of the Training Performance Measurement System is our rating/review component. After every training event, we ask for feedback on nine subjects from class attendees. These nine questions are then each assigned a 5-star rating.

The average from the nine questions creates our Average Class Attendee Rating. You can see this above the Average Knowledge Increase area. Again, this is all based on real data; none of the numbers are manipulated to make us look better.

To find out even more, just click on the View More Reviews button located below the star rating. A shadowbox will open and you’ll see the breakdown of each question, its rating, plus –when available — actual quotes from attendees about that specific training.

We’re still working on populating the data for the quotes.

We don’t guess

We’re kind of nerds for measuring, making sure that what we do can be proven. Performance-Based Contracting, when properly applied, makes for happier, more comfortable customers. It also makes for happier, more responsible employees. Then guess what? It makes for happy, successful owners and managers.

This new system is just sharing with you some of the data that, we believe, shows the value of NCI training.