Video: Consumer Testing & Corona (interview with Sven Henneberg)
In the last weeks, we have shared with you how we at isi adjusted our sensory and consumer research...
How isi succeeded in managing a nearly seamless transition from onsite Lab testing to in-home testing at the onset of the Corona Lockdown. Part 2: Consumer Testing.
Click here for Part 1: Sensory Profiling.
"What experiences have we gathered during the crisis? Foremost, that we can transfer practically all consumer tests from the laboratory directly to the testers' homes," explains Sven Henneberg, isi Senior Manager, who works at the isi facility in Munich. The corona crisis has proven itself to be a nurturing situation for new ideas. Nonetheless, several hurdles had to be overcome in order to transition from the laboratory to the living room.
Sven Henneberg tells how ever-challenging products - such as frozen vegetable burgers - made their way from the manufacturer to the dining table of the testers. The studies also indicate that the results between laboratory testing and testing in the living room are comparable. At the moment, most of the consumer tests are again being carried out in the laboratories on a regular basis. The isi team is applying the recent experience of the last weeks and months to future projects. Home tests were and are an important part of consumer research, independent of the corona crisis. However, the crisis has further highlighted their importance.
Sven, when did you decide to close the labs at isi?
As soon as the German government's directive came into effect on March 16, 2020, we stopped all our consumer tests in the laboratory. Since this step had been foreseeable, we were not caught completely off guard. We had had a wealth of experience with home use tests. We immediately worked out how we could move our customers' projects that had been planned for the laboratory into the homes of the test subjects.
Was it so urgent for your customers to continue the projects?
Yes. Some customers were tied to a strict timeline for the launch of a new product. Others, on the other hand, decided that they would rather wait until the crisis was over. In addition, there were also new customers who were looking for alternative testing options because the Corona restrictions limited their ability to carry it these tests out for themselves. As a result, these customers migrated from their own facilities to ours. So, we had our hands full. The first priority was to thoroughly advise the customers.
What is the difference between consumer tests in the laboratory and consumer tests at home?
They answer different questions. The test in the laboratory, the so-called Central Location Test (CLT), is designed to acquire knowledge about products under standardized conditions. For example, the chosen test participants compare the variations of a tomato ketchup and evaluate the color, consistency, taste along with many more. Usually, the test is carried out once over a limited period of time, for example one hour. The products are presented by our isi team in precisely measured portions. The results tell us and our customers, for example at the beginning of a new product development, in which direction the product should be developed in order to be well received by consumers. If, for example, the ketchup variant with more curry receives a more favorable reaction, then the production team will know in which direction the recipe should be developed.
The Home Use Test (HUT), on the other hand, normally would have been introduced at a later stage of product development. The three top product variants having already been uncovered, the question would still remain as to which of the variants is most favored in everyday use. The test person then uses the different ketchups over a longer period of time, for example several weeks, just like a normal product, e.g. on a burger or with a grilled sausage. The standardization of light, temperature and time is removed from the scenario. The results might have shown that the test person prefers the more intensive variant in the laboratory, but the less intensive one in the reality of everyday life. With these results, the manufacturer can then start finalizing their recipe.
In my opinion, it makes sense to test products at home. For example, it happens more often than you think that people eliminate the subtle differences between two product variants due to the way they prepare them. The classic example is the frozen pizza. Some people leave their pizzas in the oven so long that it is just as dark brown on the top as on the bottom. In this case the aroma of the tomato pieces no longer plays a role. Even if a manufacturer is planning to replace an old product with a new one, it makes sense to carry out a consumer test that takes into account home preparation.
During the Corona period did you carry out classic Home Use Tests for the projects already in progress or did you choose to introduce a laboratory test into the home instead?
We asked our test subjects to set up a kind of test lab - a Central Location Test - at home, usually in the living room. The participants also sent us photos of what their set up looked like, so we got a good impression of the environment in which the tests took place. The photos show how much attention the test persons put into setting up their test location. Throughout the whole Corona period, we have published daily newsflashes on LinkedIn and Facebook and sent a newsletter to interested people. In this way, we established transparent communication channels showing how our work adapted to the changing situation.
Did the test persons accept the change without further ado?
Yes, there are several reasons for this. On the one hand, they were at home a lot anyway due to the partial lockdown, possibly a bit bored and therefore happy to be able to do something from home. On the other hand, they receive an expense allowance, which makes such a test attractive. In the event that participants drop out during the test, we recruited 40% more people as a cushion than we otherwise would have done for a classical laboratory test.
Now to the subject of the products themselves. How did you get the normally finely measured product samples to the test subjects?
That was one of our biggest challenges. One of our first in-home tests concerned lemonade. The product was already filled into small blinded bottles. So, we only had to forward the bottles on. Chocolate, canned vegetables and cat food are also suitable in this way. However, it is more difficult with dairy products, ice cream, and frozen goods. But we found a solution in each and every case. For example, one of our employees in the laboratory often weighed, packed, and coded the products, then a parcel service delivered it and the test persons were asked to cool the product immediately upon acceptance of the product.
Were spoons and glasses then also sent along for the tasting?
No, the test subjects supplied that from their own kitchen. This, and the fact that we could not influence light, time, humidity, background noise and temperature, meant that we had to relinquish control to a certain extent. That was the price for flexibility. However, our analyses show that this has had no influence on the results.
With a Central Location Test, you instruct the test persons and can directly answer questions during the test. At home, the test subjects have to read the explanations meticulously. Does this lead to mistakes in the execution of the test?
We met this challenge in three ways: good communication, individual support and supervision. We worked and reworked the accompanying letters until the test instructions were clear, understandable, and fully adapted to the home environment. Then we assigned a direct isi contact person for every test person, who was responsible for answering questions by e-mail. Just as importantly, the database allows us to observe live, in real-time who is where in the process in order to ensure that the individual steps in the test are being carried out according to plan. Our experience has shown again and again that the compliance of the test subjects is not the problem. The critical point is rather the shipping of the products.
During the Corona period, parcel services were not always reliable. How did you make certain that the frozen veggie burgers didn’t arrive at the test subjects two weeks late and thawed out?
We were well aware of this problem. Together with our customer, we decided, for example in the case of the veggie burger, that an internal logistics service would take over the distribution of these products. This ensured the compliance with the cold chain and guaranteed the delivery to the front door, even across national borders. The test person then received the product at the door, kept it frozen up until the tasting, defrosted it before the test and finally fried it in the pan for just under ten minutes. These are the manual operations that would have normally been carried out by our laboratory team. We asked the test persons to send us photos of their test and had close contact with them throughout. The test ran daily for over two weeks and everything worked perfectly. In the initial phase we had toyed with other ideas for transport – everything from refrigerated batteries to dry ice. It is important to be open to new ideas and weigh all the options.
Surely this extra effort makes consumer tests more expensive?
In this case, yes. Nevertheless, we were able to remain within the budget by reducing the test questions somewhat in consultation with the customer. In other situations, however, an In Home Test can also be less expensive. It all depends on the product and the questions being asked. Home tests usually require more consulting work, because customers and test persons require more information.
A question that customers have probably asked you many times: How could you be sure that the results of the In Home Tests are just as reliable as those of the Central Location Tests?
We have so much experience in consumer research that we presumed that the results were comparable. Large differences in acceptance can certainly be ascertained equally well with both test methods. However, in a Home Use Test we need a larger number of cases, i.e. we have to question more test persons than in a CLT. In other words, with the same number of cases, we measure more precisely in a CLT, because we minimize the entanglement of unintended effects through use of controlled test conditions.
Have you also reached your limits somewhere?
In our practical work we have always been able to find solutions. Even if this sounds corny, but that has to do with the fact that we work well together as a team. It is the trust between us that has made the great flexibility possible in the first place. If someone had told me before Corona what new processes we would be able to set up in such a short time frame, I would hardly have believed it. I will take the experience with me into other projects. However, we have all worked hard and, to be honest, I am particularly looking forward to my summer vacation and a bit of rest and recuperation this year. We can now test in the laboratory again. As well, we also have a number of projects that have been moved from March back to the second half of the year. We have not had a break yet during the corona period.
Where will the holidays take you this year?
With the family to the Baltic Sea. There I can take in a deep breath of fresh sea air and just shut down for a while. Maybe I'll come up with one or two new research ideas. The Corona-induced restrictions have caused a lot of confusion, but I would like to take the good ideas and opportunities that have emerged from this into the future, both personally and professionally.
In this short video, Sven summarizes the insights isi gained during the Corona crisis regarding consumer testing.
Dr. Fabienne Hübener is a freelance science journalist specializing in the senses and sensory research. She has been writing for us since 2017 and also likes to accompany our team with the video camera in the lab and at conferences.
Aug 7, 2020
In the last weeks, we have shared with you how we at isi adjusted our sensory and consumer research...