We don’t often make time to confront our own uncertainty. With everything a person has to do, or could be doing, there’s little reason or incentive to go looking for the limits of their own understanding — least of all in the area where they’re supposed to be the expert.
So, I’m always impressed by those who choose to take our Outcomes Assessment with their colleagues. They are stepping away from the familiar terms and metrics that define their work to reflect on the value of their work and their institution to their public. Chris Argyris coined the idea of single and double-loop learning. Single-loop questions ask whether we’re doing things correctly; double-loop questions ask if the thing we’re doing is correct. In other words: single loop = how; double loop = why. Most of the time, we gravitate to single-loop situations, trying to get better at what’s worked in the past and what we already know or think we know. Occasionally, we make space for the double loop and reflect on whether what’s worked in the past is still even relevant or the right choice to begin with. The Outcomes Assessment makes space for the double-loop for those who want to go there.
Douglas Hubbard has shown that everyone is overconfident in their estimates, and that overconfidence is especially pronounced among experts. So when the assessment asks participants to reflect on how frequently they believe visitors achieve the various outcomes their museum can support, inflated estimates aren’t a flaw in the respondents. They’re the human baseline, which is critical for future measures.
Interestingly, institutions that re-take the assessment a year later seem to express lower confidence that their museum provides these outcomes. Hubbard would likely say that’s exactly right — it means the staff are becoming better calibrated. When we’re honest with ourselves and each other about what we don’t know, we’re in a better position to get curious and learn.
Calibration, in Hubbard’s sense, requires feedback. You make an estimate, then you find out the answer, and the gap between the two is what trains your judgment. The assessment on its own doesn’t close that loop. Rather, it captures the estimate. Which is why the year-over-year shift is encouraging but incomplete.
The point of the survey isn’t to find out how wrong people are. It’s primarily meant to be a tool to foster conversations inside museums about that specific museum’s unique value or capabilities.
The assessment also asks participants to make difficult choices. Museums often invest in visitor surveys that promise standardized measures either annually or on an ongoing basis. These collect some demographic data and ask thematic questions or perhaps gather satisfaction scores. The promise is a comparison between the museum and others like it. What’s missing is any insistence that the museum make strategic choices. The Outcomes Assessment is rather opinionated — it says, “You should first reflect on the value of your institution and decide what’s important. Only then should you survey visitors or members about those specific outcomes.” That’s because to do otherwise means you’re hoping visitors will decide what matters for you, which they can’t do and shouldn’t be asked to do.
As of today, Outcomes Assessment participants can access their results through the MaP website. Everyone at your institution who submitted a survey sees the same thing — your institution’s data and a live view that updates as new data comes in. That shared view has been missing until now, and I’m hopeful it will help change the conversation from “What did we say in the survey?” to “What outcomes might we prioritize in our exhibits, programs, and communications?”
But participants naturally want to know whether what they believe is actually so. They want to understand how their institution is performing against the outcomes they’ve chosen to prioritize. That’s the visitor feedback half of the loop, and it’s the piece our participants themselves have been asking for. It’s what we’re building next.
If you’ve already taken the assessment, log in and visit your results. If you haven’t yet participated, you can learn more and request the assessment here — or just reply to this email with any questions.
Have a good weekend,