In „How many interviews until validation“ we mentioned that no single type of interview is appropriate for all situations. We stated that the research we are doing is more open in the very beginning of a company, product, or bigger feature and will need to be closed and concrete towards the end. We classified the stages along the knowledge funnel. For a company to stay on top of the business, it is important to get through the knowledge funnel as often and quickly and deeply as possible. That is why we will stress the knowledge funnel again and abstract from the interview techniques and elevate over to the types of research that should be done in the different phases from problem discovery to the optimization of the recipe in the end.

Pure research vs. applied research

Thankfully, Erika Hall in her brilliant book “Just enough research” already did that, so we can happily point to her and explain how she differentiates the different types of research required. As the title of the book implies, her approach towards research is quite pragmatic. Research is systemic inquiry in general and is divided into pure research - as in science - and applied research – helping us understand and solve real world problems with an economic application. She makes a point of not mixing those up. The point we can take away is: What we need is applied research with the purpose to make our products better. That also means that we do not have to achieve scientific purity. That saves a lot of overhead, but is still valid for our purpose of gaining insights for our own environment only.

Along applied research we have 4 types of research at hand:

1 Generative or exploratory research

This is the type of research at the very beginning of the product cycle when we don’t even know what problem to solve for our customer. The purpose is to be able to formulate a problem to solve and be halfway sure that this is a relevant problem. We do this by going places, watching people, interviewing customers, experts and sometimes also random people, researching the Internet very broadly amongst other activities. In interviews we ask very broad and generic open question and would not even have tasks to solve for the interviewees. We want to understand their life, environment, behaviors, look out for hacks they engage in to solve their problems to be able to identify them.

Of course, here the customers do not tell us their problems, we need to synthesize them, detect and understand patterns and get our own pattern and thus detect a problem to be solved. We do not yet have a clue of the solution.

The goal and purpose of generative research is being able to formulate, which problem we solve (for whom?) and why this makes sense.

2 Descriptive and explanatory research

Starting with a problem statement already narrows the solution funnel. Happy us! We now have a concrete problem to solve, but there are still gazillion of more or less viable ways to solve it.

We now need to gain better and more detailed understanding of the problem context and the context of the people we solve the problem to not fail. We need to find the sweet spot of “What and How”.

We can now address are more specific range of people closer around our problem, select the experts in a more informed way and finally the questions we will ask are more informed also. We try to fill our own knowledge gaps of the solution environment by what other people know. This brings with it the challenge and chance to let go of our own assumptions (“people normally do …”) and be open to what our interview partners have to bring in. It is our role not to use the results as validation of what we think anyway, but better understand what will work and how and why from the source.

It is of particular interest in this phase to get an understanding of which solution pattern or conventions will work in this context and which not. Because remember: using conventions (“shopping cart”) has great power, but only when used in the right context. Whereas inventing new solutions is of great power in new contexts but often underwhelming in known contexts. (Will you be powerful enough to replace the “shopping cart” metaphor with one that is better understood or this your vanity at play when trying so?)

The goal of this phase is to have a good understanding of the context we provide a solution for, get the right solution ideas and thus come to informed solutions for the chosen problem stated in the generative research.

3 Evaluative research

Now for the first time, we can do research that actually tests our solution ideas. After having identified a problem, understanding it and its context and the people around it, we can now approach people with our ideas and try to find out which work best and how to twist and turn them.

Now this is the research that we see happening the most. They are basically early UX tests. But you really need to make sure that you know where you are and that do you not try to extrapolate insights for generative or explanatory research from what you do in these tests. This can happen by accident but is not the purpose of these tests and the accidental results would not be valid.

We see lots of people that think they are producing the right products because they do lots of evaluative research. Nope. To think so is dangerous. Your testees in evaluative research will answer to your very concrete questions, will try to fulfill tasks in your task based tests as well as possible. Here, they won’t tell you that they do not have a clue why your company is in the market at all and that the can not find a match between your product and your brand messages and potential vision.  Don’t try to read these insight into this type of research. It is plain wrong. But it is still by far the most research we see.

The good message about this type of research: You can and should do it frequently and it can even be integrated smoothly into your development flow! (That’s also our assumption why we see it so often.) But just being “researchy” by doing this all of the time will not help you to build the right product.

With this kind of research, the goal is to know which solutions will work and if our solutions are getting closer to solving the problem we chose. In other words: Did we do enough explanatory research to come up with the right solutions?

4 Casual research

Once your product is out of the door, you are in the good position to watch people use it and thus understand why things are happening. Why do people abandon this shopping flow all of the time, why don’t they like to sign a form in your product, why do they always find some products and others not etc.

This is casual research and should and can be standing order for your product team. It will lead to refining your recipe and getting closer to perfection. It is the most concrete and closed research we can do. You could actually say that A/B testing is another level of concreteness of this kind of research out in the field.

Finally, here is our usual chart of the knowledge funnel with the research types 1 to 4 mapped to it.

Types of user research along the Knowledge Funnel

What you need to do before going into research is clear: Make it clear to yourself

• What your level of required insight is,
• Define the right level of research and
• Be aware what kind of questions you can ask and
• And then, don’t mix it up!

Also realize, that while evaluative and even much more so casual research can smoothly be integrated into the dev workflow, it may be culturally challenging, hard or even conceptually impossible to integrate deep explanatory and much more so generative into your dev flow. I would even advise you to let researchers be researchers for this type of research and devs be devs.

Have fun!

Titelphoto: Mike & Amanda Knowles, licensed under the Creative Commons Attribution 2.0 Generic license.