Something that’s come up in a couple of the different sessions I’ve attended at the American Public Health Association annual meeting is the problem of inadequate definitions of success. It’s important to set targets and measure progress against them – and missing targets can be a signal that it’s time to revise the strategy. But if the targets are set without sufficient thought, a person or group can think they’re succeeding when they’re not really doing such a great job.

One example of this came up in the session “Getting from here to there: Promoting health and environmental justice through transportation policy.” Heidi Guenin of Upstream Public Health mentioned the experience of a jurisdiction that measured the success of its transportation by how quickly cars were able to travel – but they realized that single metric didn’t capture other goals of transportation policy, like ensuring that different populations could get where they needed to go safely, and that people had multiple different transportation options. Guenin mentioned the Center for Neighborhood Technology’s Walk Score as one of multiple metrics that can provide information about transportation options in an area.

In the session on “Under-reporting of workplace injuries and illnesses: What has changed and what is still needed to ensure accurate job injury/illness data,” Jim Frederick of the United Steelworkers focused on the problem of relying solely on rates of days or time lost to workplace injuries. As a the 2008 report Hidden Tragedy (prepared by majority staff of the House Committee on Education and Labor) made clear, many workplace injuries and illnesses never get reported. A workplace might have a low rate of lost-time injuries because the employer is committed to injury prevention, or because the employer threatens workers with termination if they report injuries. In a United Steelworkers survey, union members reported on the kinds of metrics used to track safety in their workplaces, and days and time lost to injury were among the most common. Metrics like training or the time it takes to fix hazards once they’re identified could be better indicators of a workplace where safety is a priority, but survey respondents reported that they were used much less frequently.

Metrics matter, and the same program could be considered a failure by some standards and a success by others. Every so often, it’s a good idea to make sure we’re measuring the right kind of success.