Build the Habit Before You Build the Dashboard

Your institution has a lot of data — maybe more than anyone realizes. You already know that data doesn’t automatically translate into decisions. The burden is that the decision to use data is always a new one. It never becomes automatic or at least habitual.

Imagine living on a planet where seasonal changes happen daily — every single day is dramatically, unpredictably different. Friday, you need a sweater, scarf, and heavy coat; Saturday calls for shorts and a t-shirt; Sunday, you’re back in a parka. Part of you ends up resenting the clothes, even though each article made you happy when you bought it. Living on that weird planet could be fine, maybe even fun, if you had some way to call on the correct articles of clothing given the conditions. That’s the infrastructure problem behind the shift from outputs to outcomes.

Most museums collect data. Even if you don’t have dedicated staff, you distributed a questionnaire last year; You read stuff AAM puts out from their surveys. And, in the broadest sense of “data”, your staff are noticing things and forming opinions (for better or worse) all the time. The gap is in the connective tissue between what you’ve captured and the decisions you’re trying to make. It’s difficult to shift from basic engagement metrics (outputs) to understanding the value of your institution (outcomes). But part of what makes it hard is that we often mischaracterize the change as a measurement problem when the challenge is often more about missing infrastructure that makes new habits easier to adopt. You have smart staff. They can learn new skills. But skills won’t change the fact that you’re on a planet where the temperature swings from subzero to stifling in a matter of hours.

Last week, I wrote about one lightweight entry point: The Black Box, a facilitative intervention that nudges you and your colleagues to search for ‘what would have to be true’ for the decision on the table to lead to better results. The Black Box is handy — like living on that weird planet and having a little weather widget built into the wall of your wardrobe.

But the Black Box alone isn’t going to turn your institution into a place with a steady stream of evidence of your organization’s value and impact — no one intervention will do that. What it has going for it is its lightweight design. There’s no onerous approval process for pulling a box out of the recycling bin, painting it black, and sticking it in the middle of the conference room table. The barriers to doing so are more of a social-emotional nature — a voice in your head that asks, Will my colleagues be okay with something like this? and Is this going to threaten my standing within the group? Those are real obstacles that you may or may not be able to overcome, but they’re a different kind of risk that you personally can own without dependence on others’ approval. You either take the risk and invite your colleagues to try something new, or you don’t, but the choice is yours to make.

But infrastructure — the subject of today’s letter — requires a different kind of buy-in. It’s arguably less risky in the social sense because there’s nothing weird or even unfamiliar about investigating operational solutions to organizational challenges. But someone is always going to reflexively play the limited-resources card at the first hint of change, and that’s the Achilles heel of any infrastructure-based intervention, no matter how small.

So, I imagine some of you read last week’s post and thought, “It would be too weird for me to try this at my next meeting,” and some of you will read this week’s post and think, “I’ll never be able to get approval for this kind of system at my museum.” My hope is that:

1. You’ll notice that there is no one path or solution to begin effecting this change at your institution

2. In understanding that there are different paths forward, you can begin to run your own diagnosis — that is, you can begin to examine: where do I have the most agency to begin at my museum? Do I have more leverage at the facilitative level (Black Box style interventions) or do the circumstances make a systems-level play more viable?

Because each organization’s culture and each individual’s abilities are different, you’ll have to determine what the tolerances are and where you can establish a beachhead. A newsletter written to thousands of people can only take you so far — you’ll have to translate these ideas to your own context (though you can find a time if you’d like help identifying where to start).

I'll use MaP's work with the Western Development Museum in Saskatchewan as the running example today — partly because it’s a case I can disclose in detail, and partly because the principles are portable to any organization regardless of scale.

There are three degrees of evidence in the system, each one informs the next:

1. Footprints

2. Trails (Evidence)

3. Destinations (Outcomes)

Footprints

Footprints are the smallest unit of evidence you can have. I sometimes think of them as noticings — that is, they’re just things that you notice and capture as you go about your work, or they can be tiny units of knowledge (concepts and summaries, for those of you steeped in Progress-Space Research methodology). At the footprint level, you’re operating with few filters because you never know what footprint might become a valuable trail of evidence down the road. Examples:

• Someone stopped on their way out the door to say how much they appreciated a helpful staff member

• After a school visit, a teacher sends a one-line email: “One of my students who never participates spent ten minutes asking questions at the Shark and Ray Touch Tank”

• A participant in a listening session describes feeling vaguely guilty after taking their kids to a movie because they think, ‘were we really together or did we just trade one screen for another?’

The footprint capture system is ideally as easy as possible and isn’t singular — you’ll need multiple input paths. You have to overcome the second-guessing people have — “Is this good enough or worth noticing?” The answer is pretty much always yes, in part because your downstream system is designed to do the filtering for you (a topic for a future post).

Unlike Trails and Destinations (below), there’s no translative layer for Footprints because there’s never any reason for you to talk about a Footprint with an external stakeholder. You’ll absolutely want to talk about Destinations with funders, which is why it has an external-facing name — Outcomes*.* Internally, it’s helpful to use metaphors because they’re sticky and resonance supports adoption. A Footprint in a Trail of footprints leading to a Destination is memorable in a way that may be helpful to the users of that system, but confusing for consumers of it. Use the metaphor internally, knowing that your director of development will speak to Outcomes and the Evidence that supports them with external stakeholders, not destinations or trails.

MaP collects footprints every day. Sometimes a footprint comes from me making a note about something that caught my attention in a meeting; other times it’s more passive — like the custom system I built (the “Switchboard”) that passively logs that someone new subscribed to the newsletter after reading last week’s post. We never know when or if one of these footprints will add up to a trail, and that’s ok.

MaP uses a range of tools — some off-the-shelf, like Notion, and others that are custom-built, like the Switchboard — that support a small organization like ours processing dozens of signals a day. Your infrastructure will look different. Even a small museum can begin noticing hundreds of Footprints every day. You can start with a shared spreadsheet where front-desk staff logs visitor comments or establish a process for digitizing comment cards. One institution I know uses a shared Google Doc — staff can jump in and jot down anything a visitor says that surprises them. These are all imperfect solutions. You’ll feel the limitations quickly. You’ll wonder how this kind of data could ever connect to a ‘real’ study you commissioned two years ago on what makes your museum’s members tick. That’s normal. The important thing is to interpret the frustration as an invitation to experiment and expand rather than an excuse to give up before you’ve really begun.

The principle is tool-agnostic: capture with minimal filtering and plan for downstream processes to do the sorting. You’re just focused on new habits and a place for your Footprints to land.

For example, late last year, Joan Kanigan — CEO of the Western Development Museum in Saskatchewan and a participant in our Value Articulation Intensive (VAI) — shared John Falk’s well-being research with partners in the provincial government and returned to the group to report that a senior government official’s first question concerned the range of values in the data she had shared. They weren’t rejecting the data, but they were responding to it in a way we didn’t foresee. That produced two footprints: One for Joan, who learned something about the stakeholder, and one for John and me about the landscape that our participants are operating in. I noted it in our Footprints database along with a dozen other footprints (cohort gatherings always produce lots of little footprints or clues¹).

Trails (Evidence)

A collection of footprints leading in a particular direction is a Trail or, if you’re speaking to someone outside your museum, evidence. Examples:

• Six footprints showing six different people told the person at the help desk how they have to renew their membership in person because they couldn’t do it online is a trail of evidence about the online renewal experience.

• A heatmap of the ticketing page on your website showing 40 dead clicks on a piece of bolded text is a trail of evidence about some unmet expectation or intent on the ticketing page. We might not know what it means yet, but it’s a trail.

• Seven donors gave seven different reasons why they gave over $1,000 to the museum. Sometimes trails can be interesting in their incoherence — that is, inconsistency, or the absence of a path, in a dataset can be just as valuable as a clear trail.

Returning to our VAI example, Joan could have stopped after that one footprint. It would have been a single data point leading nowhere. But instead began refining her approach — planning to lead with a single, conservative figure rather than a range when meeting with a different stakeholder. Over the following weeks, her approach continued to evolve. By year’s end, she had developed a two-track strategy: economic data for some conversations, well-being framing without dollar translation for others. Each refinement was a footprint in a trail of evidence. We couldn’t know if that first footprint would lead anywhere, but the capture habit helped us when it later became clear that a trail would emerge.

Destinations (Outcome)

A destination (’outcome’ if you’re talking to someone external) is a result that matters to the person or group you serve. That last part is important. If it doesn’t matter to the intended group, it’s not an outcome because it didn’t provide value to anyone but your organization. Examples:

• 100 people asked for help finding a gallery or exhibit over x weeks (that’s 100 footprints creating a clear trail of evidence); We updated the museum’s wayfinding by doing y and the number dropped by half (outcome).

• 13 members described in interviews feeling a sense of safety when walking the grounds in the early morning and evening (trail); After framing the value of membership in terms of ‘a safe place to enjoy the nature in solitude’, more members cite feeling safe as something they appreciate and more visitors are joining (don’t forget: outcomes need to matter to the people you support but they also have to be valuable to your organization — It’s the intersection of the two that determines what you prioritize).

Back to our VAI example: It could have been that the evidence trail would run cold. As great as it is to see a program participant doing the work our program encouraged her to do, doing work in an executive program is not an outcome. You might argue it’s a net win for MaP, since it suggests our program design is actually helping these folks in conversations that matter, but we have to define success in terms of outcomes that matter to the people we support. A director participating in a program is just engagement, which is an output; A director realizing financial results for her institution is an outcome.²

Fortunately, this particular trail didn’t run cold. Last month, Joan maintained the Western Development Museum’s full $4.2M in provincial funding for the coming fiscal year — at a time when many Canadian provinces are tightening budgets. That outcome rests on a host of factors — things like her years of relationship-building within the ministry, the leadership and advocacy skills she has developed over a career, a staff whose work makes the institution worth investing in, as well as her participation in the VAI program. We can’t say the program was the sole cause of the outcome — that’s not how complex systems work. But what the infrastructure made possible was tracing MaP's contribution within that system. We helped shape the context for absorbing new research, testing approaches with real stakeholders, providing community connections, and deploying what worked when it mattered. Without the habit of capturing footprints and following trails, we wouldn’t have been able to see those contribution pathways at all — and neither would Joan. The framework doesn’t necessarily let you claim full credit for outcomes. It lets you see how you participated in producing them, which is what you actually need when a funder asks what your program does.

One institutional story is just that — one piece of evidence leading to one outcome. But the underlying principle — lightweight, continuous evidence capture as an equal peer to periodic formal evaluation — has support beyond our experience.³

Ultimately, this infrastructure shift is about building a habit of noticing and adopting a responsive approach to learning.⁴ “This isn’t how other organizations do things” is never a great argument — and it’s a particularly weak one at a time when most museums doing what most museums do aren’t exactly thriving. Compliance-driven reporting — reporting on what funders or other stakeholders ask for, such as attendance or participation metrics — may help keep your doors open today. It won’t tell you whether the work is producing results that matter to the people you serve.

There’s a difference between episodic and continuous learning. Episodic learning is the kind of project-based inquiry you’re already familiar with — a study that’s meant to reduce uncertainty around specific decisions. Continuous learning is what happens when you treat your staff’s observations as a valuable resource worth capturing. By collecting footprints, identifying trails, and finding outcomes, we admit that we don’t always know what kind of learning is needed, and that the people closest to the work are already noticing what matters.

Kyle

P.S. Joan is joining our second cohort of the Value Articulation Intensive as a program collaborator this spring. If you’re facing a high-stakes conversation — board, funders, government — where you need your institution’s impact to land clearly, that’s what the program is built for. Cohort 2 starts April 30. Learn more and register→