Measuring Skills that Matter

Most everyone agrees that “soft skills” matter, even if they don’t agree on what to call them. Soft skills, social emotional skills, non-cognitive skills, even 21st century skills can all generally refer to the attributes you’d appreciate in a good co-worker – someone who is collaborative, creative, empathetic, able to give and receive feedback, and able to focus to get things done.

I would argue that most of the time we assume that these types of skills develop naturally in school – we assume that by virtue of going to class, completing tests and projects, and finishing school that students graduate with both content knowledge and the ability to work well with others, think creatively, and solve unencountered problems.

To a certain extent, that’s true, as University of Chicago research James Heckman has shown. In one paper, “Hard Evidence on Soft Skills” Heckman compared three groups of American adults – those who had graduated with a high school diploma but did not attend college, those who received a diploma as adults (known as a GED), and those who dropped out of high school without a diploma.

In Heckman’s sample, the high school-only and GED graduates ostensibly had the same cognitive skills – that is, they performed similarly on a standardized test (the Armed Forces Qualification Test). You might expect that due to their similar achievement levels, high school and GED graduates would perform similarly in the labor market – and you’d likely expect them both to perform better than those with no diploma.

But they don’t. Heckman found that GED graduates actually perform more like those who never obtained a high school diploma than those who did. And those high school graduates? They perform better than both groups.

He attributes these differences among groups  to non-cognitive skills and argues the following:

– Variations in standardized tests can be partly explained by non-cognitive factors. Children who show curiosity and persistence, for example, tend to have higher test scores – they’ll stick with a task like a test and be interested in how everything turns out. That curiosity and persistence tends to be rewarded later in the workplace and beyond.
– Standardized tests fail to fully account for the range of skills one needs to be successful in the workplace and other arenas. So policymakers and analysts who judge a system solely by achievement test scores may be misled by the numbers (e.g. GED recipients are counted as high school graduates but in the long run don’t perform like them).
– In sum: “Monitoring school progress and creating programs to enhance skills requires a broader framework of measurement. Interventions that promote beneficial changes in personality have an important place in a portfolio of public policies to foster human development.”

I would expect a fair number of teachers to be nodding their heads in agreement with these conclusions. But how those teachers approach their work isn’t always informed by them, partly because of the pressures created by policymakers in the systems in which those teachers operate.

The tricky part here is that even though we know non-cognitive skills matter, it can be quite difficult to determine how we know students are making progress on them – as well as what strategies are best used to develop them.

Here in New Zealand, standardized testing took a big hit with the end of National Standards; a challenge now lurks in thinking about how system performance can be assessed. In the U.S., districts in California and charter organizations are broadening their measures of success, even as the debate continues about how to do so.

As I continue to visit schools and dive into research, I’m looking forward to understanding more about how schools themselves and developing systems to measure what matters among their students.