Don’t bring a data knife to a pet-idea gunfight

If an organization hasn’t determined what value it’s trying to provide to the communities it supports, it’s operating in a vacuum. Organizations struggle in a vacuum — there’s an understandable tendency to fill the void. We have trouble understanding the nature of the problem, though, because we have a mission, a vision, and values. We have a strategic plan. On paper, all the boxes are checked. So what is this strange absence we keep running into?

It’s strange because you may only experience the absence by its symptoms. For example, you feel it when people start leaning on things like the collection as a proxy for value. To be clear, an institution’s collection is valuable, but only potentially. Every word in the dictionary is valuable in that those words have the potential to produce a book that inspires or a poem that makes people see the world in a new way. But no writer thinks that using more words will somehow increase the likelihood that their book will inspire people or that their poem will transform how people see the world around them. Objects, like words, are outputs. Outputs don’t guarantee outcomes. We know it’s absurd for a writer to expect to sell more books or write a better essay by just using more words, yet we think it’s reasonable to expect a museum to increase attendance or help people see their world in a new way by increasing the number of objects on display.

If colleagues or leaders are reaching for an absence-shaped solution like “more access to more stuff”, one way to respond is with data. In a case like this, you might start with research on attention. “Museum fatigue” is well-documented — it refers to how visitors have a limited attention budget, and as the experience gets denser, a person’s ability to attend to any one object declines. Museum fatigue research dates back to 1916(!), resurfaced in the 1930s, and reappeared just over a decade ago.¹ Maybe we’re doomed to keep rediscovering attention research every few decades.

Research on access shows that broader access doesn’t translate into more engagement. In interviews at one museum, visitors welcomed open storage but also reported feeling overwhelmed.² And in one study on blockbuster shows, half the visitors saw only the special exhibition; just 22% made it through most of the museum.³ So, more doesn’t necessarily produce more engagement.

So the data is out there, and it isn’t useless. After all, it’s what convinced you. But what convinced you won’t necessarily work for everyone else. The problem is that data alone — no matter how well supported — doesn’t always win in moments like this. You can’t bring a data knife to a pet-idea gunfight because stakeholders feel no ownership of the data. They are attached to their solution, though.

The weapon you need in that moment is a different decision that your colleagues feel ownership over. If you have established together which outcomes are most important, you can point to that agreement. You can say, “I hear your idea. Previously, we decided to spend the rest of the year helping people achieve social outcomes. Let’s talk about how putting more objects on display helps us achieve that.”

Of course, the connection between # of objects on display and social outcomes is tenuous at best. But now the choice is no longer between some research that the person feels no attachment to and their smart idea — it’s between a choice they made two months ago and a choice they’re proposing today.

But you have to have had that conversation about outcomes. Otherwise, all you have is data, and, unfortunately, most of us are not data-driven, even if we like to believe we are. Most of the time, we’re driven by our emotions and egos. (I don’t think I’m an exception to this, btw.)

That’s not to say that you can’t test for yourself. For example, you might think that more objects on display will lead to an outcome your visitors value. Maybe you have a hunch that an overwhelming display of amazing art will help people feel a sense of awe or stimulate their senses in ways other approaches won’t. That’s a hypothesis. Treat it like one. What will you need to observe to know whether that’s true? What evidence will you look for? Outcomes-first thinking isn’t a jail — No one says you can’t challenge the existing body of research. It just raises the bar for you and your museum. The work graduates from being a task (put more objects on display) to a challenge (we’re testing whether more objects will produce this outcome for these people).

But if your strategy isn’t centered on outcomes — that is, the value you provide to your participants — then there’s really no defense against personal preferences dressed up as solutions. In the absence of a shared purpose, outputs will win by default. We’re working with some museums now that are having this conversation. Reply to today’s letter if you’d like to talk about bringing it to your museum.

Have a good weekend,

Kyle