Interview with Andreas Duess, Founder and CEO of FishDog
Here’s how to mind the “say-do” gap in your marketing surveys by enlisting the help of synthetic research.
Traditional person-to-person survey techniques have the apparent advantage of yielding results that mirror reality better than technology-based alternatives. However, as Andreas Duess of FishDog explains, there are a range of factors that may lead to your “authentic” survey data being seriously slewed, as well as slow and expensive to gather. For these and other convincing reasons, you may want to consider the latest techniques in synthetic research.
You’ve worked with companies including Cisco, Sony, and Autonomy before building your own communications agency and, more recently, exploring new approaches to consumer research. Was there a particular experience that changed the way you think about how businesses understand people?
After eight years in technology marketing, 16 years running a marketing agency with a focus on the food and drink vertical will cure anyone of taking survey data at face value. Client after client presented us with the same pattern: Here was expensive and beautifully presented research that said consumers wanted healthier options and were prepared to pay a premium for these. Then, a year later, the sales data said they bought indulgence at the discounter.
The gap between claim and behaviour, the “say-do” gap, was where most marketing money was wasted.
Importantly, and often confusingly for the client, both data points were true. It’s just that people answered as the person they wanted to be, but then went out and shopped as the person they actually were.
That gap between claim and behaviour, the “say-do” gap, was where most marketing money was wasted. Once you’ve seen it in category after category, you stop asking whether people are telling the truth and start asking whether the question itself can ever get at the truth. That reframe eventually led me here.
One of the longstanding problems in consumer research is that what people say they will do and what they actually do can be very different. Why has that gap been so difficult for businesses to solve?
Because it isn’t a lying problem, it’s a structural one. A questionnaire, and a focus group even more so, is a social performance. People will answer aspirationally, they agree with the interviewer, they say what sounds good, what makes them likeable or look sophisticated. None of that is dishonesty, it’s just how humans behave when they’re being watched.
The industry then made it worse, with a race to the bottom. Panels reward completion, not honesty, and a meaningful share of respondents are professionals who take surveys for a living and have learned what answers keep them qualified.
The gap also survived because seeing it requires holding two datasets at once – what people said and what they did – and those usually live in different departments with different budgets. The claim was in the research report while the behaviour was in the till data, and those two were often owned by different people with little motivation to cooperate.
Synthetic research is attracting growing attention, but the term can mean different things to different people. What does it mean in practice, and what separates a credible synthetic consumer from an AI-generated fictional persona?
In practice it means synthetic populations: large sets of individual profiles built from real behavioural data, calibrated so the population matches the real one on the distributions that matter: census demographics, media habits, occupations, attitudes.
The dividing line is provenance and hard work. For example, we recruit populations from real data. We don’t invent personas. One of my core beliefs is that creation is where bias lives; recruitment is where representativeness lives.
If a vendor typed a prompt that says “you are a 34-year-old mother of two in Buffalo,” every bias in that sentence was baked in by whoever wrote it. If the profile was recruited from observed data and validated against published distributions, you can audit it.
So the question to ask any supplier is simple: where did this population come from, and what real-world benchmark has it been tested against?
The obvious sceptical response is: why ask an AI what consumers think when you could simply ask real consumers? What do you think that criticism gets right, and what does it miss?
What it gets right: scepticism is always the correct posture toward any new methodology, including ours. Nobody should take a synthetic result on faith, and I am the first one to admit and advocate for that. There are jobs where speaking to real people is irreplaceable.
What it misses is that the baseline it defends isn’t clean. Recruitment fraud is rampant. Participants get coached to fake personas to qualify for paid studies. Professional respondents condition themselves to panels. Social desirability distorts every answer given to a human interviewer. “Just ask real consumers” assumes the real consumers in the panel are real, candid, and representative. Often they are none of the three.
It also misses the economics. For most business decisions, the alternative to synthetic research isn’t a rigorous human study; it’s no research at all. Most decisions are made on gut feel because research was too slow and too expensive to reach them.
The search for faster and more useful consumer insight eventually led you to create FishDog. What did you believe was missing from the way companies were conducting research?
Speed that matches the decision. Research took six weeks and a five-figure budget, so it was rationed to the biggest bets. Everything else, the pricing call, the message choice, the concept kill, ran on opinion and hierarchy. The most senior voice in the room won.
I watched clients pay for excellent research that then arrived after the decision had already been made. The report then either confirmed or contradicted a choice that had been made, often funded, and nobody was willing to reopen it.
What was missing was research at the speed and cost of the decisions companies make every week. When a study takes days instead of weeks and costs a fraction of a panel study, research stops being a special event and becomes a working habit that’s embedded in every day.
That conviction produced the platform – synthetic populations built from real behavioural data. What we sell on top of it are two focused products.
- TestScreen shows video content, trailers, pilots, full cuts, to a synthetic audience built to a broadcaster’s exact specification and reads the response second by second. TestScreen serves studios, streamers, and related industries.
- ThesisLab convenes panels of synthetic experts, the practitioners inside an industry, so investors can pressure-test a thesis in hours rather than scheduling weeks of expert calls. ThesisLab is designed specifically for hedge funds and related categories.
Same machine underneath, two very different rooms making decisions on top of it.
Trust is crucial here. If a company is going to make a significant product, marketing, or investment decision based partly on synthetic research, what should give executives confidence in the results?
There’s only one way to do this properly: performance against outcomes that actually occurred. Everything else is marketing.
Here is how we hold ourselves to that standard. We run a standing calibration loop against Polymarket, a prediction market. A panel of 100 statistically representative US personas makes daily predictions on live market questions, and every call is scored against the real-world outcome across 18 dimensions: the size of the population involved, macro versus micro, business, politics, science, conflict. Prediction markets resolve, so there is no arguing with the scoreboard. We know exactly where the panel is strong – large population, high-stakes questions – and where it is weaker, and we can tell a client which kind of question they’re asking before they spend money on the answer.
The loop also catches our blind spots. Earlier this year, the panel knew nothing about a celebrity story that was dominating the news cycle. The miss exposed a gap in how our population ingests news, and we fixed the feed. That’s the test working as intended.
The second thing is honesty about limits. Every methodology has biases, including ours. Good buyers don’t ask for a method with no biases; they ask to understand them, the same way they understand the biases in their existing methods. A supplier who claims their method has no failure modes is telling you something important about themselves.
So: benchmark against published studies, rerun prior research, compare predictions to what happened. Any supplier, human or synthetic, who won’t submit to that test doesn’t deserve the budget.
Some of the most valuable research findings are the ones nobody expected. Can synthetic research genuinely uncover something surprising about consumers, or is there a risk that AI simply reflects patterns already contained in existing data?
The criticism is aimed at the right target but it misses it. A language model on its own absolutely does reflect its training data and it will create data in the mushy, people-pleasing middle. That’s exactly why we don’t sell a language model. Our population layer is built from current behavioural data, weather and news input, even OCEAN 5 personality data. It updates every four hours, so the system takes the pulse of the real world in real time rather than replaying the past.
Disagreement inside an audience is exactly what executives can act on; it’s where the real insight lives.
As for surprise: the surprises don’t live in the average, they live in the splits. When 60 percent of an audience warms to a character and 40 percent doesn’t trust him, and the split follows a line nobody expected, that’s the finding. Traditional research, and LLMs, often sand this away by reporting the mean. We will always lead with divergence, because Disagreement inside an audience is exactly what executives can act on; it’s where the real insight lives.
Does this ultimately need to be an either / or choice? Where do you see synthetic research becoming particularly powerful, and where would you still insist on speaking directly to real people?
It’s not either / or, and it should never be sold like this.
Synthetic research is at its most powerful where the decision moves faster than human fieldwork can. A network deciding which pilot to greenlight can now put every candidate in front of a TestScreen audience, not just the one it could afford to test with a recruited panel. An investor working a thesis can question a room of synthetic industry practitioners the same afternoon the question comes up. Those are the decisions that never got to research before, because the clock beat the fieldwork every time.
It’s also the best prequel a human study ever had. Run the synthetic study first to find out what the expensive human study should ask.
I’d still insist on real people where the stakes and the stakeholders demand it: final validation of a major bet, regulated categories where traditional methods are mandated, lived-experience depth in sensitive areas, and very small niche populations where the data to recruit from is thin. We tell clients this before they ask. A methodology that knows its limits is a methodology you can trust inside them.
One attraction of AI-driven research is speed. But could making consumer insight dramatically faster also create a new problem – encouraging companies to research and react constantly rather than stepping back and asking the right questions?
Yes, and that failure mode is very real. The danger is that you end up with dashboards everywhere, and decisions nowhere.
My test for any piece of research is simple: if it doesn’t change what you build, buy, or believe, it’s a waste of time and money.
My test for any piece of research is simple: if it doesn’t change what you build, buy, or believe, it’s a waste of time and money. Speed doesn’t exempt you from that test. If anything, it makes the test more important, because you can now produce empty performative research theatre at scale.
What speed genuinely buys you is a better chance of being right before the cost of being wrong becomes painful. We call it “being wrong in private”. You can test the assumption before the production starts or the investment is made, not after. That’s not research-and-react. That’s moving the research to the point where it can still change the outcome.
As these tools become more sophisticated, do you think competitive advantage will come from having better research technology, better data, or simply being better at knowing what questions to ask?
The technology commoditises first. Everyone already has access to the same extremely capable models, so that edge doesn’t exist anymore.
Data is more durable. Populations built from real behavioural streams, kept current, validated against outcomes, that’s an owned asset that competitors can’t prompt their way into.
But the lasting advantage is the questions and, more than that, the willingness to act on the answers. We’re entering an intelligence economy; intelligence itself stops being scarce, and judgement becomes the scarcity. When any company can get a credible answer in hours, the differentiator is which questions you think to ask, and whether your organisation does anything when the answer is unwelcome. Most “say-do” gaps live inside companies, not consumers. Firms say they’re customer-led, then do what the highest-paid person in the room already believed.
If we look five years ahead, what part of today’s consumer research industry do you think will have changed beyond recognition, and what will remain surprisingly human?
The panel economy changes beyond recognition. The machinery of recruiting strangers, paying incentives, and waiting weeks for fieldwork can’t survive contact with methods that are faster, cheaper, and increasingly as accurate. The fraud and professional-respondent problems accelerate that, because the panel economy’s core claim that “these are real, candid people”, is already eroding from the inside.
Research also shifts from project to habit: instead of three big studies a year, continuous testing woven into weekly decisions, with human fieldwork reserved for the questions that earn it.
What stays human: deciding what matters, interpreting findings inside a business context the data can’t see, and the courage to act on an answer nobody wanted. The industry’s best people were never valuable because they could field a survey. They were valuable because they knew which finding changed the decision. That skill appreciates.
Finally, imagine a CEO tells you, “We know our customers extremely well.” What is the one question you would ask to test whether they really do?
“When did a customer last change your mind?”
If the answer is a specific moment, a finding that killed a product, rewrote a strategy, or reversed a decision they were personally attached to, they probably do know their customers. Knowledge that never surprises you isn’t knowledge; it’s assumption wearing a lanyard.
If the answer is a pause, they know their own beliefs about their customers extremely well. That’s a totally different thing, and it’s usually an expensive assumption.
