In advance of today’s Value Realization Collaborative Office Hours gathering, a participant asked, "What are the concrete alternatives to measuring/capturing qualitative data without using surveys?"
This question challenges us to reconsider how we truly understand the people we serve, so I thought I’d share a more extensive written response for those who won’t be in attendance today.
Why Qualitative Data Matters in the First Place
Quantitative data tells you what happened. Qualitative data reveals why it happened.
Consider this: You might learn that visitors are very likely to recommend your museum to friends (a healthy Net Promoter Score), but you still don't know what it is about the experience that would prompt them to make that recommendation.
Even more revealing: We can't be certain they would actually recommend the museum. We only know they tell us they would. (Net Promoter Scores are dubious at best, but that's for another day.)
The Problem with Traditional Surveys
Qualitative data helps us understand the behavioral roots and underlying motivations that drive people's choices. Without it, we're building mental models based on assumptions rather than understanding.
When we rely solely on surveys to gather qualitative insights, we encounter significant limitations:
People write succinctly because they often don't know the answer
They choose the most readily available response, not necessarily the most accurate one
Questions like "Why did you come to the museum today?" yield surface-level responses that rarely reveal deeper motivations
The Uncomfortable Truth About Meaningful Data
To collect better, more reliable qualitative data, we must step into uncomfortable territory ± speaking to someone directly about their thoughts, feelings, and decision-making process.
Our natural tendency in conversations with strangers is to stay at the surface level — preferences, opinions, small talk. Going deeper feels unnatural, even inappropriate. Yet the most valuable insights lie beneath these surface interactions.
This is where specific frameworks become essential — structured approaches that help overcome our natural resistance to these deeper conversations.
The Spectrum of Qualitative Methods
Qualitative data collection exists on a spectrum from highly structured to completely open-ended:
1. Structured Verbal Surveys
In this approach, the interviewer reads aloud predetermined questions. This can elicit deeper responses than written surveys, but you’re still limited by the presence of preset questions and vulnerable to social desirability bias. It’s true that people may alter responses to appear more helpful or socially acceptable anytime they’re talking with someone else, but less structured approaches (below) can account for desirability bias in their delivery and results.
2. Semi-Structured Interviews
Semi-structured interviews strike a balance between rigor and discovery. The researcher prepares a thoughtful topic list with flexible probing questions, creating a framework that ensures consistency while allowing the conversation to flow naturally. As the interview unfolds, a skilled interviewer adapts their approach based on the participant's responses, following promising threads while maintaining focus on the core research questions. This methodology creates space for unexpected insights that might never surface in more rigid formats, yet provides enough structure to ensure the data remains relevant to your institution's goals.
3. Deep Listening Sessions
Deep listening represents the most immersive approach to qualitative research, centered around a specific purpose or goal rather than predetermined questions. These sessions focus intensely on understanding the participant's mental models and decision-making processes — the why behind their actions and choices. Though more challenging to execute, deep listening yields the richest insights about underlying motivations and behaviors — the kind of understanding that can fundamentally reshape how your institution supports community goals.
Practical Next Steps
To gather meaningful qualitative data without surveys:
Start small – Practice deep listening with colleagues or friendly stakeholders before approaching strangers
Begin with structure – Use semi-structured approaches until you build confidence
Document thoroughly – Record (with permission) so that you’re able to revisit the resulting transcript in the future
Look for patterns – Individual stories matter, but the real insights come from identifying patterns across multiple conversations
While ideally institutions use mixed methods for a complete picture, meaningful qualitative data gathered through conversation will reveal insights that quantitative measures alone simply cannot.
Have questions about implementing these approaches? Join our next Office Hours gathering – leave a comment on this post (“add me”) and we'll add you to the RSVP list when the next session is published.