Robbie Blom

October 31, 2024

7 minute read

Two Ways To Test What Customers Want

Color, size, price, brand, features, quality, service level... the list of product and service attributes that customers care about is long.

But how do you know which ones matter most? Are there pockets of customers who care about different things? Does your product mix need to change to better meet their needs?

There are quantitative ways to validate your intuition on these questions, and in this article we'll cover two of them.

Conjoint Analysis and Discrete Choice Analysis have some hand-wavey complexity around them, but we'll see that they basically come down to three things:

1. Asking customers about what they like
2. Designing the questions to extract as much information as possible
3. Making a guess about preferences

With well designed questions and guesses, we can learn a lot about customer preferences, market segments, price elasticities, and more.

"20 Questions" Approach To Uncovering Customer Preferences

I'm thinking of something, and you have 20 questions to guess what it is. You can ask me yes or no questions, and I'll answer truthfully. You can guess at any time, but if you guess wrong, you lose.

Ever play this game?

20 Questions is a fun way to pass time on a road trip, but it's also a pretty good guide for designing a survey experiment to uncover customer preferences.

Like 20 Questions, the key to a successful survey experiment is to ask questions that give you the most information for narrowing down the answer.

The problem, though, is that customers don't always know what they want until they see it. Pretty hard to play 20 Questions when your playing partner doesn't know the answer!

We can solve for this by putting example products in front of customers and then asking them to make choices. Through their choices, we can determine what they care about most. Like 20 Questions, we can be smart about the questions we ask so that they narrow down the answer as much as possible.

Asking Customers To Choose

From a customer perspective, Conjoint Analysis and Discrete Choice Analysis look pretty similar. They both present a choice for some version(s) of a product, and then the customer expresses a preference.

The difference is in how the choices are presented and how the data is analyzed. In this section, we'll talk about presentation. In the section about making a guess, we'll talk about analysis.

Choices In Conjoint Analysis

Conjoint Analysis most commonly asks respondents to express preference or purchase intent on a rating scale like the one below. Respondents do this one at a time for different versions of the product in the survey.

There are, however, a couple common variations where respondents are asked to rank or rate all product versions at once:

• Ranking: Respondents are asked to rank all product versions from most preferred to least preferred.
• Points Allocation: Respondents are given a fixed number of points to allocate across all product versions based on how much they like each one.

Whichever way it's organized, we obtain a numerical measure of preference: a 0-10 score for each product version, a rank order, or a share of points allocated to each product version.

Choices In Discrete Choice Analysis

Unlike Conjoint Analysis, Discrete Choice Analysis presents respondents with a choice between two or more product versions. Respondents then choose among the options which product version they would buy.

This is a more realistic scenario than Conjoint Analysis because, in the real world, customers choose among multiple products at once.

However it's also a more complex scenario because we don't have an obvious number to measure preference. We'll see how this works in the section about making a guess.

Asking The Right Questions

Asking the right questions in 20 Questions means asking questions that narrow down the answer as much as possible. The same is true in Conjoint Analysis and Discrete Choice Analysis.

When we have many product or service attributes to test, the number of possible combinations adds up very quickly. For example, if we have 4 attributes with 3 options for each, then we have 81 possible combinations. That's a lot of questions to ask!

We can be smarter.

Actually though, we'll let the computer be smarter for us. Figuring out which questions to ask is not an easy thing to do by hand, but we've figured out the math for computers to do it for us easily.

The math involved is called design of experiments. Conjoint Analysis most commonly uses a method called fractional factorial design, and Discrete Choice Analysis most commonly uses a method called D-efficient design.

Both methods are designed to ask the fewest number of questions to extract the most information. Luckily, we can use software to do the heavy lifting.

Making A Guess

We've asked the right questions, and now we have a bunch of response data. How do we turn that data into guesses about what customers want?

The answer is that we run a regression.

In Conjoint Analysis, the regression simply explains the relationship between the product attributes and the preference score in each survey question. We can describe this relationship in terms of numbers assigned to each attribute that together add up to the preference score. Some attributes will affect the score more than others, and this is the insight we're looking for.

In the bicycle example above, our regression relationship would conceptually look something like this:

$Preference Score = \hat{\beta_0} + \hat{\beta_1}Frame + \hat{\beta_2}Brakes + \hat{\beta_3}Derailleur + \hat{\beta_4}Price$

The beta coefficients describe how important each attribute is to the preference score. For example, if the coefficient next to Price is high, then we know that price is a big driver of preference.

Discrete Choice Analysis applies a similar approach, but uses a slightly more complex regression strategy. Instead of predicting preference score, Discrete Choice Analysis predicts the probability that a respondent will choose any given product version.

Similar to Conjoint Analysis, though, we still obtain a weight for each attribute. If we found that price was a big driver in Conjoint Analysis, then we would see the same thing in Discrete Choice Ananlysis so long as respondents make choices based on price when they take the survey.

So back to 20 Questions... how do we make our guess?

We simply look at the weights assigned to each product or service attribute. If the weights are higher for some attributes than others, then our guess is that customers care more about those attributes than any of the others.

What We Can Do With This

Knowing the breakdown of what customers care about is really powerful. And the fact that we can put a number on it opens up a lot of possibilities for exploring what-if scenarios.

The most common what-if scenarios revolve around product design, pricing, market segmentation, and market share:

• Product Design: Because we know how much more customers like one attribute over another, we can design more successful products.
• Pricing: When we include price as an attribute in the survey, we can determine estimates of price elasticities, willingness to pay, and optimal price points.
• Market Segmentation: Because multiple people take the survey, we can find market segments by clustering together who liked what at a respondent-by-respondent level.
• Market Share Simulations: If you include competitive products in the survey, then you can simulate market share responses under scenarios where you introduce new products or change prices.

These types of what-if scenarios can be a really useful tool to support your intuition about customer and market dynamics.

Conclusion

Conjoint Analysis and Discrete Choice Analysis are two powerful tools for understanding customer preferences. They both work by asking customers to make choices among different product versions, and then analyzing the data to determine what customers care about most.

With a well-designed survey and a few statistical tools, we can learn a lot about customer preferences simply by asking them to make choices. Armed with this information, we can make better decisions about product design, pricing, market segmentation, and market share.