Why Most Training, Coaching, and Assessments Fail to Create Measurable Change

The Activity-Outcome Gap Most organizations can report how many people attended a training session, how many hours of coaching were delivered, or how many employees completed an assessment. Far fewer can report whether any of it changed what people actually do at work. This is the activity-outcome gap: measuring activity is administratively easy; measuring outcome requires a different kind of design discipline that most programs never build in from the start. The gap is not a minor methodological footnote. It determines whether a training or coaching investment can be defended to a finance department, whether an assessment result can be trusted for a personnel decision, and whether a development program is actually developing anyone. Kirkpatrick's Four Levels, and Why Most Evaluation Never Leaves Level 1 Donald Kirkpatrick's four-level model, first proposed in 1959 and still the dominant framework in the training-evaluation field, distinguishes Reaction (did participants like it), Learning (did they acquire knowledge or skill), Behavior (did they apply it on the job), and Results (did it affect organizational outcomes). The model is simple to state and difficult to execute, because each level requires a fundamentally different measurement approach: satisfaction surveys for Level 1, knowledge tests for Level 2, direct or supervisor-rated observation for Level 3, and business-metric tracking for Level 4. Widely-cited industry benchmarks on training-evaluation practice show a consistent pattern: nearly all organizations measure Level 1 reaction, a minority measure Level 2 learning gains, and only a small fraction ever reach Level 3 behavior change or Level 4 business results. The practical consequence is that most organizations know how a training or coaching program was received, and almost nothing about whether it changed anything. The Transfer-of-Training Problem Timothy Baldwin and J. Kevin Ford's foundational 1988 review in Personnel Psychology named this the transfer problem: the gap between what is learned in a training or coaching context and what is actually applied back on the job. Their model identifies three categories of factors that determine transfer: trainee characteristics (ability, motivation, personality), training design (the extent to which content and practice conditions resemble the real work environment), and work environment (whether a manager, team, or system reinforces the new behavior once someone returns to their role). The third factor is the one most programs ignore. A well-designed coaching intervention or training course can produce genuine learning and still fail to change behavior, because nothing in the participant's actual working environment supports, rewards, or even permits the new behavior once the session ends. Why Reaction Scores Don't Predict Behavior Change The "happy sheet", a post-session satisfaction survey, is the most common measurement instrument in training and coaching, and one of the weakest predictors of anything beyond itself. Sitzmann and Ely's 2011 meta-analysis in Psychological Bulletin found that self-assessed learning correlates far more strongly with affective outcomes, motivation and satisfaction, than with actual cognitive learning gains. In other words, participants can report a training session as valuable and enjoyable while retaining and applying almost none of its content, and a satisfaction score will not reveal the difference. This matters because reaction data is cheap to collect and easy to report upward, which creates an incentive to treat it as a proxy for effectiveness. It is not one. When Feedback and Assessment Backfire A further complication is that feedback, the mechanism most training and coaching programs rely on to drive improvement, does not reliably improve performance. Kluger and DeNisi's 1996 meta-analysis in Psychological Bulletin, covering 607 effect sizes from 23,663 observations, found that feedback interventions improved performance on average, but that more than a third of them decreased it. Their feedback intervention theory attributes this to where feedback directs a person's attention: feedback that shifts focus to the task improves performance, while feedback that shifts focus to the self, praise, criticism, or comparison to others, can undermine it. The implication for assessment design is direct. An assessment that reports a raw score without behavioral, task-level context risks producing exactly the kind of self-focused feedback that Kluger and DeNisi's data associates with performance decline. The One-Time Snapshot Problem A separate failure mode sits inside the assessment tools themselves. Classical test theory, formalized in texts such as Nunnally and Bernstein's Psychometric Theory, treats every observed score as a true score plus measurement error. A single administration of any psychological instrument, however well validated, cannot distinguish a real change in the underlying trait from ordinary meas