Hidden Needs Analysis: Creating Breakthrough Products

An Interview with Keith Goffin

The clear identification of customer needs is a fundamental part of successful innovation and most organizations aim to develop a deep customer understanding. However, the methods that most companies use for their market research – such as surveys, focus groups, and visits to customers – are just not effective. Such traditional approaches cannot uncover hidden needs – the needs that customers have not even recognized themselves.

In this interview, Keith Goffin, Professor of Innovation and New Product Development at Cranfield School of Management in the UK, talks about his latest book Identifying Hidden Needs: Creating Breakthrough Products (Palgrave Macmillan November 2010), and discusses the best methods for identifying unmet and hidden needs and the ways to translate these into breakthrough products.

Q: Most managers dream about having a product like an i-Pod or i-Pad, something that will sweep the market. But the reality is different from that, isn’t it?

Keith Goffin: Yes, unfortunately reality is very different. It depends on the sector that you are in, but there was a US study done about a year ago which found that 90% of new food and beverage products failed – yes 90%! That means that 90% of the new products put on the shelves of supermarkets are withdrawn within three months.

In some other sectors, the rate of failure is not as high, but all of the research shows whether you in automotive, engineering, B2B or B2C, the rate of new product failure can be very high. So new product development is risky – and yes, it is great if you have the i-Pad, the i-Phone or the i-Pod, but it doesn’t happen like that all the time. It is not quite that easy.

Q: Now, some of the reasons for new products failing seems to be down to a lack of understanding of customer needs; can you expand on that?

Keith Goffin: Yes; a lot of our research is currently looking at how companies design their products and services. I should stress that everything we will talk about today applies equally to products and to services. Most organizations conduct traditional market research – they will do interviews, they will do surveys and they will use focus groups. In each of these methods they essentially ask direct questions of their customers: what features would you like in the product in the future? The problem with such direct questions is that they do not probe very deep. So, customers often struggle to answer; they struggle to articulate what they actually need. Another problem is that customers in focus groups put a gloss, a spin on what they say. They will explain how they use a product but this will not correspond to the way products are used in practice.

A US study done about a year ago found that 90% of new food and beverage products failed.

And so, very often, companies just scratch the surface with their market research. The problem is if you are trying to design a breakthrough product, you can’t just ask your customers what they would like because you won’t get a very good answer. You are likely to get ideas for incremental variations, on today’s product and which means that the new product is likely to fail.

Q: You have done some research on new ways of looking at customer needs; can you say some more about that?

Keith Goffin: There are a number of methods that are emerging, and these tend to come from the social sciences. Techniques from psychology and anthropology can help us to understand how people think, as opposed to what they tell us.

So for instance, in anthropology there is a central rule that you listen to what people say about their culture, but you observe afterwards and you look for the so-called dis-junctures, you look for the contradictions – they said this but they do that. The social sciences go much deeper than just listening to what people say. If you go and observe people using products and services – you will find out far, far more.

So these techniques, as I say, are from the social sciences. There are a number of them which are used in combination. And I should stress that there is no one single technique that will solve all of your market research issues; it is a number of tools and techniques that you should use in combination and each of them gives you another piece of the puzzle of understanding what the customer really wants.

Q: Do you need lots of training to put these things into practice?

Keith Goffin: You need to understand that your organization is going to have to invest in this. This is not the sort of thing that people will probably have studied before because these are new and emerging approaches. It does take time and effort to look for customers’ hidden needs. These are the needs that customers themselves can’t normally articulate – we have to pull them out. In the management jargon we talk about “known needs” – those are the needs the customers have for features on products and services. Then there are “unmet needs” – these are also known, but they currently aren’t served by products and services on the market. But the “hidden needs” are the sorts of things customers would really want. If you provided products that satisfy their hidden needs they get excited.

Developing the capability within your organization to identify hidden needs is really worth the effort.

Q: I would like to take it to the specific and I would like to pick one example that rather intrigued me from your book which was about walking boots; a boot, is a boot, is a boot – but maybe it isn’t. Tell me some more.

Keith Goffin: Clarks, the well known manufacturer of shoes, has been around for a couple of hundred years in the United Kingdom. When the company entered into the walking boots market a few years ago they recognized that they needed to understand the market. So they employed a group of anthropologists who identified different types of walkers in the Lake District to understand how walking boots were used.

Q: They were actually in there, in the Lake District saying what? Oh, why did you do that? Or what are you doing?

Keith Goffin: Clarks asked walkers if they could walk with them for a few minutes and discuss things with them. Clarks filmed videos because it enabled them to analyse things in a systematic way. They asked people what was important about boots? People said things like “when I take my boots off I want to be able to clean them easily”.

Clarks filmed people sitting on the back of the hatchback of their car, taking their boots off. This gave them really detailed information about existing boots and what needed to be improved. But a really interesting point about Clarks was that they didn’t just look at the way people use boots, they looked at the way they purchase boots. Clarks observed people purchasing boots in European sports shops. They found on average that there would be typically about ten pairs of boots for each size; and a typical customer will take only two or three pairs of boots off the shelf to try on. Now the problem is, if you are a manufacturer, you have only got a 20-30% chance of your boots being chosen to be tried on. Clarks asked people which boots they would take off the shelf. People said things like the colour, the design, the make, the brand – which are all well known factors. But Clarks found by videoing people and conducting detailed analysis, that a lot of people would squeeze the tongue of the boot to try and understand, before trying it on, whether it was comfortable. Nobody wants to try on ten pairs of boots as it takes too long. People squeeze the tongue to judge if the boots are comfortable and this was a real ‘aha’ moment for Clarks in understanding how people select boots. Clarks then designed their boot not only to be comfortable in walking, but to have a very soft and well designed tongue.

Clarks found by videoing people and conducting detailed analysis, that a lot of people would squeeze the tongue of the boot to try and understand, before trying it on, whether it was comfortable. This was a real ‘aha’ moment for Clarks.

Q: Can you give me another example from a business to business context?

Keith Goffin: We have used these techniques in a number of business to business – B 2 B – situations and, for example, we worked with Bosch manufacturing equipment.

Bosch produces production line equipment for pharmaceutical factories and this is the sort of equipment that will fill hundreds of syringes per minute. These are big pieces of production line equipment 30 metres long and costing €5-6 million. With Bosch we went much deeper than normal market research: we went to people on production lines and asked them about the problems with their existing equipment and what was stopping them reaching their production targets.

When people are interviewed in their normal working environment, in this case a production line in a pharmaceutical factory – they are much more able to describe what they need. For instance by saying things like if this part of the machine was only designed in another way it would be far better.

In working with Bosch we also used a technique from psychology called repertory grid, where you get people to compare and contrast products to help them articulate their needs. Interestingly Bosch were going to be the fourth company to enter the market, so they knew if they did not come up with a unique design, they weren’t going to succeed. So Bosch were very interested to take a new approach to market research and it worked out very well and has led to a very successful product.

Q: One of the things that I was interested in from the book is the idea this isn’t just a set of techniques, if you were to make an organization really focus on the customer and hidden needs and so on, you need to do quite a bit of work to get these approaches embedded in the organization.

Keith Goffin: Yes; that is one of the things that took us quite a while to realise. We did a lot of research around the techniques, but what we are doing increasingly is also looking at not just at the technical aspects, but also the political ones. And what we are finding is that there are a number of barriers within organizations to taking these different approaches. Sadly and ironically, we are finding that one of the biggest barriers to adopting new approaches to market research is the marketing function in a company.

We are finding often that companies with strong marketing departments are very reluctant to do things in a different way.

So one of the first barriers to adopting new techniques is that marketing people see it as a threat. It is contrary to what they have been doing up until now, they perceive it as too different.

Another barrier is that companies tend to innovate incrementally. Hidden needs techniques are designed to be radical, to develop breakthrough products. But if you haven’t got a management support you are probably not going to come up with a breakthrough product.

Although initially we looked at the technical side of hidden needs, we are now finding that it is important not to forget the political side and actually change the culture of a company to enable the adoption of hidden needs.

Q: So Keith, a final message for people looking at this idea for the first time; what would you like to leave people with?

Keith Goffin: I think the thing that is coming out most strongly for us at the moment is that some organizations are reluctant to put enough effort into understanding their customers.

I think that this is a real shame because if you are going to be successful as a company, understanding the customer is something that is fundamental. And it is something you shouldn’t outsource to an agency. You need the internal capability to really understand your customer and their hidden needs.

About the Author

Keith Goffin is Professor of Innovation and New Product Development at Cranfield School of Management, where he is also Director of the Innovation and Process Management Community, and the Centre for Innovative Products and Services (CIPS).

Keith worked for fourteen years with Hewlett Packard Medical Products Group in a number of management roles in new product development and marketing. He joined Cranfield School of Management in 1995 and his current research interests are innovation strategy, project-to-project learning in R&D, and enhanced methods for market research, so-called ‘hidden needs analysis’. He has published extensively, with over eighty articles in journals and magazines such as the Journal of Product Innovation Management, Research-Technology Management and the Financial Times. He regularly acts as a consultant on innovation management to organizations such as Agilent Technologies, BASF, BT, HSBC, Leyland-Trucks, Rank-Xerox, Sony and Unilever. His latest book (with Fred Lemke and Ursula Koners) Identifying Hidden Needs: Creating Breakthrough Products (Palgrave Macmillan November 2010) gives a detailed explanation of how to conduct market research in a new and effective way.



Please enter your comment!
Please enter your name here