Would you rather spend 8-12 months building something you find people do not want to pay for at Launch, or 3 months determining whether your idea is desirable (do people want this?), feasible (can this be built), and viable (can a company be built on this)?
In this episode, I sit down with David Altounian – entrepreneur, investor, and professor – to build a roadmap to figure out: Will customers pay for this?
David: Do you want me to tell you?
Moby: Yes, so, hey, David.
David: Hey, how are you, Moby?
Moby: Why are you tired today?
David: So yesterday was a sixteen and a half hour day. First part of the day was spent here on campus doing my academic role as professor for a retreat. And my second half was working at the counter of an HPB sandwich shop making ice cream sandwiches and baking cookies from five till midnight last night.
David: Yeah, so two totally different extremes.
Moby: Why were you doing that?
David: Well, I’m working with a company in town called Moojo that does ice cream sandwiches that’s looking to franchise and expand. And I started helping them put their plans together, liked it, and actually I become a partner there. And so I’ve been in the store to try to really understand the processes and be able to package this thing so that we can take it to other locations and be able to replicate that experience across the country. So the best way to do it is get in there and do it. Last night was a little more doing than I expected. But that’s what I was doing.
Moby: Five to twelve making ice cream sandwiches. So you kind of … and you mentioned before that no one else was there, so you had to figure everything out.
David: Yeah, so I when we started doing this I was spending time. I go in for a couple hours at a time and work with the chefs. So part of putting this together is you got to create standards of practice, so that you can replicate the stores in different locations. And they’ve got a lot of good documentation, because the company’s been around for two years, and really spent time trying to build this from the ground up to be done this way.
But I still needed to take that and translate that into a standard format that we could use. And also because I eventually expect to get students working on some of the packaging of this as classroom opportunities. And so I needed to get to at least understand where I have of it and a baseline that they could work from. So I’d be going in for two or three hours at a time and doing it.
And I was learning, but some would come through and it would be too much for me to handle. I go to the full time or say, “This one’s yours. I have no idea.” And then last night there was nobody there but three of us, and we were … all three was the main owner of the of the firm, and two others of us that really had no experience. So I would look around and someone asked for something really bizarre, and I’d be like, “I have no idea how to do this.” And I’d look around and everybody would just shrug, it’s like, “Okay, I guess I better figure this one out.” So I found I am not good at making milkshakes.
David: It is not my specialty. I am too impatient to sit there and grind. But it was a great experience too, because I think that you really … you can think about the process somewhat needs to happen. But man, until you’re there and you’re in your 4th hour, and you’re trying to do something the same way, you don’t realize how difficult it is, how important it is. I’ll give you a really great example, the scoops need to be fairly standard in size first of all for cost containment, second of all so the customers have a similar experience. If one ice cream sandwich is this big and two hours later or the person next to him gets one that this big, you have a problem.
When you’re in the beginning of the day and you start using the scoops and all that, it’s really easy to go, “Yeah, we’re a take time and make sure it’s right.” By the time you’re four or five hours and you’re tired, it’s like, “I don’t care.” You can’t do that, so you can look and understand here’s a problem, this is harder to maintain the quality, hard to do things, how do we fix this? So really going through it having that experience and seeing what people on the line deal with I think it’s really invaluable.
Moby: And it was packed, right?
David: And it was packed. I swear I thought this was a candid camera exercise. I was sure that like the general manager had put cameras in there and they filmed us. The way this came about was the crew wanted to do a team building, a water park day. And we agreed that we, the partners, would actually run the store so they could do that, which they thought was great and hilarious at the same time. But I think we held our own. I talked to them today and we left the store in good shape, no one got killed. For me the measure was no one got mad and walked out, so …
Moby: Yes, so that works.
David: Yeah, and the great thing about this business is when people come in they’re happy, because they’re buying ice cream sandwiches. And when they leave they’re happy because they had an ice cream sandwich. So it’s a really great experience.
Moby: So I mean, this is a very successful business and you have built successful businesses, you’ve been a partner of businesses and you’re professor of entrepreneurship at a university teaching this stuff. What do you say to people who come to you who have some fair understanding of the problem but say, “Okay, I think I want to build this. How do I figure out if people want it?”
David: I think it’s a great question. I think one of the mistakes that people that want to do a business or create something major is thinking that they’re actually the customer. You can make something that’s fantastic and great for a market of one – you. And so it’s important to really spend time to understand who are the customers. First off who are the immediate customers, and then who are the ancillary or long term customers? And then what do those customers really want? And what do they need? And trying to profile and understand them.
Understanding your customer doesn’t just mean what their needs are from a products standpoint. I consider it understanding the entire persona behind them. And you hear the term buyer persona, and a lot of times I’ll see buyer persona … and it’s a description of a type of a customer for that type of a product. But in my mind a buyer persona is much broader and understanding customers are much broader. It’s understanding their social economic situation, where they are, where they get information, where they hang out, what their likes and wants are. And so I think successful companies really do a good job of profiling and understanding their primary customers or immediate customers and their ancillary or secondary customers.
To put a fine point on it, understanding for example at Moojo’s, we need to understand who the people are coming in, and you can go to the top of the go that’s a family, that’s college student. But the family, the person in that family is going to … is the buyer … is going to get information and be encouraged to go to the store very differently from the college student. The influencers are different, the where they get information is different, their economic decision making and purchase power is different, and you have to understand all those. You can’t just put a graphic up showing a family and go, “This is for families.” You have to understand where they are, where they hang out, what they care about, who actually is the customer.
In the case of a family it’s the kid, and the buyer is the father or the mother. And in the case of college student the buyer and the customer are the same person. So before anything is done I always view the most important thing is really understanding who those customers are, and then trying to get a sense of the profile and the life of those customers.
Moby: So when you have an idea and you’re thinking about your customer, so for example Moojo is right in front of the University of Texas in Austin. And when you say you’re going to make ice cream sandwiches, right, do you think about the fact that, “Okay, I have …” Like what’s a better approach saying, “Okay, I have these five kinds of people – families, single people, college students, teens, who can buy ice cream, who will buy from this, this place, that’s when we’ll place our store at the university?” Or versus saying, “Okay, I’m going to make an ice cream sandwich store at the university and these are the people which will be there.”
David: Both of those are valid ways to do it. I think again the economics may drive some of the decision. If it’s the first one which is, “Here’s the type of buyer that I have. And so I’m going to place a location where the buyer is.” Is a very valid approach to go, for example if you see a sporting facilities, sporting goods shop, they tend to be around schools and places where parents are going to buy sporting goods. Putting in an industrial area doesn’t make any sense. On the other hand you can take a concept and say, “I think this concept is fairly generic. I don’t know what the specific customers are for here. So I’m going to go in and I’m going to test in that spot. And then figure out the persona of who’s really come in.” Starbucks is a great example of that or a coffee store.
You know everybody’s going to a coffee but you don’t necessarily know by area what the percentage of is really to come in. But I think it’s a safe bet, you can get to a baseline for a while and then build up. My personal pain is that’s a much more riskier approach than going off into the other one. But you see it happen. I think both can be successful.
In some cases the more you are on the edge with a new product, which means more experimental it is, or the more there’s unknowns where you can’t look at buyers from before. The more you have to do that kind thing and we talk about test marketing, that’s where you can test market put it in, find out who it’s responding, who is responding well and then build around that. So both of those are valid ways to do it, but they have different approaches to how do you maximize it.
Moby: So I could potentially say that for this product that I want to build there are three potential markets, I believe … and then this come to the part where I don’t think my idea is the best thing ever, and I kind of say that I have this assumption, not belief, that this market needs it more … one out of the three markets that I think needs it more, and then I go in and test it.
David: Yes, yeah. Well, and this is where … lean start up’s a very popular thing, and the idea of a true lean start up of going in and starting something and pivoting, pivoting, doesn’t really work for something like Moojo. But there is a version of this where if that there’s an area but you don’t really understand the demographics, you don’t know if they’re going to like the product, then you figure ways to test it inexpensively, whether that’s you do a pop up store, you do a relationship with an existing vendor or store that you can actually test your product in, and if it goes well then you decide to invest more.
Those are all valid ways to do it. But what you’ll see and you’ve seen this happen before is someone will have a concept, not know if it works, but I’m going to go ahead and invest and build out a whole new space, and that nobody shows up. Six months later they’re gone. Well, there probably was an interim step to test that market first. In my mind that’s a failure of market validation. If you go and you spend that kind of money you put in a product or a service or a business, and six months or a year later you’re out of there because there’s been not enough takers. That’s a market validation problem. And it’s a pretty good indication. You should spent less money and validated it more first.
Moby: And you’re going to be more successful at the end of it.
David: Yeah, absolutely. You put a lot of effort in to put these things in. I don’t care why you’re doing this. Even for software you put a lot of money and an effort to get a product to market. So instead of having the supposition in your mind of, “My view of what customers want is the right thing, so we’re going to build this whole product out and then deliver it.” If you can do small versions of it, it works. The word people like he uses experiments. We can talk about experimenting with lean start up for software, it’s very easy. I’m going to do a minimum viable product, put it out there, and then iterate off it. And once we get a model that works we’ll go off and accelerate it.
But you can do that kind of experimentation with even a retail business. Again, pop up stores. In our case we have pop vans, we have vans that go around and do events. And we can tell by different locations in the city which events are giving us the best revenue. There’s other outlier or there’s other factors that may drive it, for example if we’re in Central Austin and happens to be because we’re at a fair or a festival. Maybe the volume is higher because the fair and festival is not there. But if you’ve done it a few times you’ll get an indication of how does that location compare compared to another location in the city.
And it doesn’t cost you anything. You drive a van there. It’s a few hours of someone’s time. And that is so much cheaper than going up and going to the headache of building a location and hoping that you’re going to be able to pivot or find the buyer persona that works. That’s crazy. Even if I can’t tell exactly in a location what those buyers are while I’m doing my experiments, I at least know there’s a baseline of business there that will support the business.
Moby: I really like those examples because I can see analogies everywhere, like in the physical space and online, because you said something like partner up with someone. I mean, if you have someone who has a 10,000 person e-mail list in your market, y ou can ask them, “Hey, can you put a blurb out and see if someone bites?”
Moby: Or you could go to … if you think your market is 25 to 40-year old tech people, then you go to a trade show and you spend 300 bucks on the booth and then you spend 200 bucks on promotion or whatever. You just talk to like 300 people in a day and figure out, “Is it something people want?”
David: I think that’s the right way to go do it. Yeah, and to that point I’m actually a huge fan of trying to go where the critical masses are. So here, think about trade shows a great example. If you know there’s a place …
Moby: How do you define a critical mass?
David: A group of people, enough that you have a sample size that’s meaningful. So I’ll go back to trade show and I’ll go back to Motion which was my business for Dell, we did this a lot. But Motion’s probably the better one. We knew we wanted to get the health care. We could have run around with our product and talk to doctor’s offices and hospital for a year and collected data. But we went to him which was later the large health care information management conference, largest one in the country. And in two days we could get to hundreds of health care professionals. And not only the users of the health care technology are walk around, but actually the buyers and decision makers.
So for $10,000 or $20,000, even if it was that much money for a week of time, we could collect as much data as we needed. The opportunity cost of being able to do that versus the actual time and dollar cost of going around for a year, or the risk cost of doing it for a year of onesie-twosies and not gaining that critical mass information, it’s so much more cost-effective to go to those events. And I feel the same way about the fairs and things like this. For this you can really get a sense when you’ve got a lot of people coming through what’s going on here, is this a valid market for us?
Moby: And that definition of critical mass that kind of goes back to what you said in the start which is know your customer.
Moby: So well, because otherwise you’ll think that you’re making a product and go to a general place where there’s a lot of different buyer personas and you might not get any hits or you might get too many … you don’t understand which buyer persona am I selling to.
David: Yeah, I figure that. Now let me give you a tip that I’ve learned painfully, and as you were saying this it really struck with me. You can actually waste your time in those critical mass environments if you don’t have some set questions that you want to validate or invalidate immediately. So I’ll give you an example from class. If I want to know how a class is really doing, how has it performing, how well they’re receiving the course, I could go talk to the class as a group. And you know what will happen? I’ll say, “Hey, how’s it going? What’s working? What’s not?” And what do you think is likely going to happen if I do that?
Moby: Ah, this is great. It’s a little hard but it’s great.
David: I’ll get that. That’s probably … well, I’ll get that from my positive students. Or I will get nothing, right? They’ll look at me like, “I’m not speaking up.” But if I go talk individually to a couple of them, tell me what’s working, what’s not. I just want to know. I really need to know from you. I’ll get maybe one or two things. Well, I think the homework is too hard. Or I just don’t think everybody’s participating.
And I take those and I go back to the classroom and I go, “I’d like to know. Is my homework too hard?” You’re going to get a different set of answers. People will actually respond to that specific question. Or, “Do I need to do something different to make sure everybody participates? Is something missing from my direction?” You’ll get feedback because they’re responding to something that resonates and something that’s specific as opposed to, “Tell me what you think.”
So what we try to do like when I would go to trade shows and we do focus groups there, we would do some individual interviews first of all and try to get a set of questions that we’re not sure about. Things like, “Are they really going to be willing to pay this kind of a price? Is the purchase …” Like for example with health care, “Does it really take six months to get approval for purchases?” Those are things we want to know. If you just asked, “Tell us about tablets and about hardware and about electronic medical record systems.” We would have heard about software, we would have heard about all kinds of things about how hard it is to get the doctors, which wouldn’t affected my business at all. What really would affect my business is what the single customer told me of, “It takes me six months to get an approval.” How long does it take you guys to get approval?
In a focus group when you’ve got critical mass you’ll get good answers, which is a much more direct targeted way to do this. And actually I use this even with my own personal stuff. I’ll asked people individually to just get a sense and try to go understand what the critical questions are that I need validation on. And then use this bigger critical mass group to try to assess out. And even if you’re not doing focus groups, if you went to a trade show and you’re walking and talking to 20 people one day, you can ask all 20 people the same questions. You will get a theme, as opposed to asking one person this week, going to the customer the next week, another customer. By the end of the 20 weeks you don’t know as much as what you did in the one day talking to 20 people. So I’m a big fan of where you get to critical mass doing that.
Moby: So the tactic is to start off by an understanding that you will get qualitative information?
Moby: And after talking to a few people then test the assumptions that you’ve built from that qualitative information and go towards quantified?
David: Yes, yeah. I think it’s a good way. Yeah, actually in research we call that mixed methods. Use the qualitative method to assess out the things you need to measure and test, and then you create the surveys to actually be able to do the quantitative analysis and do regressions to find out for your dependent variable which thing is the most impactful. But even to get the things to put on there you really need a qualitative assessment first.
But realize that both these, what I’m talking about, is still qualitative. But what you’re doing is you’re going from a single point input to mass input. You’re getting lots of data points together so that your sample size is much, much bigger and it’s much more unified together. Because even if you decided, let’s go back to our 20, my 20 data points spread over 20 weeks is not as efficient or even I think valid as the 20 in the same day, because over 20 weeks of getting one of the time, things are happening in their lives and what they’re doing.
But if you got to 20 people at the trade show they’re kind of in the same area. So all of the thing that may bias the input are kind of dead by that critical mass. If you go back to the fair example, if I went now as people over five weeks what they thought about ice cream sandwiches, you’re going to get depending on the weather of the day, how much money they have, are the kids driving them crazy. But if I’m at the festival and I’m serving and asking them as they’re walking around the festival, they’re all there having a good time. This is the same weather, it’s the same experience. They’ve got the same choices of alternatives. Even if I want to figure out a product compared to competition, why’d you go buy that one and not this one? You’ve got an environment that’s almost like a laboratory environment that you can use, so I’m a big fan of doing that and going where the critical mass is.
Moby: So find the communities of people who care about the things you do or have the problems that you think you want to solve?
David: Yeah, where they’re really focusing on what it is, that environment that you’re talking about. I mean, if you’re at a fair people are going to have fun. They’re going to ask about fun food. If you’re doing the health care product that we were, you have people there to buy health care products. You’re talking to them while they’re also thinking of other purchases. That does bias the decisions and their feedback, but it biases in a way that you want to know. They’re not going to tell you what you want to hear, they’re going to tell you based on their perceptions of what else they’re looking at. And I think those are really healthy ways to collect data.
Moby: What about the … Do you have a idea for a product or service regardless of what industry it is and you’re testing it out, how does a founder detach themselves from the idea and be able to take feedback especially when the feedback is wonderful when it’s like honest and saying, “Hey, this isn’t a great idea. You should pursue something else.”
David: Yeah. It’s a great question. It’s a really hard question. It took me a long, long time to learn how to deal with that, especially because I started my career in marketing and in product marketing where we defined specifications. So a lot of times we would do focus groups especially with professional focus group agencies, I would feel under attack where my idea didn’t get … the customer that doesn’t understand, let’s ask the question differently. And the professionals would say, “No, no, stop. Don’t bias the feedback. We want to hear this.”
What finally got me over this was when I realized that if they all agreed with the premises we’ve laid out, we wasted our time. That the real win is where you find those points of dissension, and that’s where the great stuff is. Because the stuff we put down, it’s what’s obvious. The stuff that we have is what we think. And if all you’re doing is telling me, “Yes, you’re right,” then all I’m getting is self-confirming validation. That doesn’t help me.
What I want to know is when I think this is what’s going on you say, “No, that doesn’t make sense.” Then you want to get to the next question – why, why? I want to know what’s behind that. But the mental thing that I had to switch was instead of rewarding myself for being right, I started mentally rewarding myself for asking the question a way that we found out where we were not right, that the wins for us were collecting how many places that they disagree with us, as opposed to how many places they agreed with us.
And it’s a really important mental switch because those places that they disagree with you are where you get the enlightenment. The places where they agree with you already, you just wasted time. What did you need to do this for? If everybody agreed I’ve wasted my money on this.
David: And so that … but that was, it probably took me because I’m a slow learner 10 years of doing this before I realized this is the case. Because here’s what happens, when you get more senior in those roles when you make those decisions and you start making decisions based on the self-confirming bias, the marketplace is not all self-confirming oriented. They are going to find your holes. And I prefer that I find the holes before the market finds the holes and shows that I was wrong.
And so once I started getting more senior and the people start looking going, “Wait, a second. How did you miss that?” It really did flip the switch for me to go, “Oh, you know what? I want to find as many negatives and as many counter positions as possible.” Because I may still decide to go with my original plan or my original feature or my original approach, but I can prepare, yes, I heard this, and here’s why I didn’t do it, or here’s why we made this different decision.
And you will be surprised how often that has actually come back and help us. When we said, “Well, why did you price it this way? Did you not know this would be a problem?” And go, “Actually we did. Here was the feedback we got. Here was the price feedback we heard. But here’s why we decided to stick with it anyways. It may not have been the decision but it was a because we didn’t know, it was because we had these other factors and it was an executive decision we made.” And usually that’s an, “Okay, let’s say that’s okay or let’s change the decision and move on.” It’s not like you missed it. And so the more words you see yourself the easier it is to plan and be able to plan to overcome them or to defend against them if they really do blast up and cause a problem.
Moby: That’s interesting. I saw an ad for a travel jacket two weeks ago, and as you were talking about all of this for some reason the analogy of testing that jacket came to mind. So I’m going to walk you through something. Point me out if this is crappy or if this is a methodology your people use, hypothetically if travel jackets are priced at $60 right now.
David: You mean the jackets with all the pockets for stuff?
Moby: Yes, yeah. And that’s where it’s going. And I use it and I think these suck. I could make a better one but it will cost me 120 bucks. I go to a place where there is a lot of travelers or my customers with a congregate and I start asking them questions. If I ask that question, do you want a better travel jacket? Their answer will be yes and it won’t be useful at all.
Moby: Instead if I could ask questions like, “Okay, how many back pockets are you in travel jacket? Is it cold? Is it warm? How often do you use it? Do you pack it somewhere? Do you think these features which I might have in my head, do you think these features are valuable and would you pay 150 for it?” If the answer is yes, I’m like, “Oh, my God.” Or the answer is no then I’m like, “Okay, maybe people in my market doesn’t want this.” Then I go down with 120. And that’s how I start thinking about a product.
David: Yeah, that’s one way … that’s definitely a valid approach . There’s another approach which goes in the same way but it starts the other way. It comes from the other direction. You have this jacket and you know that’s $60. It’s a version of conjoint analysis. How much more will you pay or how much less we paid for the feature coming out. Here’s this jacket for 160, it has ten pockets, it has a zipper and costs $60. Here’s a proposed jacket that here’s the differences, it’s got 20 pockets, would you pay $120? No. Would you pay $110? No. Would you pay $100? No. Would you pay 90? How many pockets? I got double the pockets, it’s got 20 pockets. Yeah, I’d probably pay 90.
First data point, you got a three dollars difference right there. Okay, now it’s got this removable lining, this one doesn’t, which means you can use in cold or hot. Would you pay $90 for this? Oh, absolutely. Would you pay $100 for this? And you could actually test against those two very quickly. And if you had 20 people you would start to get enough data points that you could collect. You could actually start doing real statistical analysis to set the price point and the features you needed.
And you can play … this is really what you do when people go, “What do you do for all that time with this research?” You can go back and do things like, “Okay, how many pockets? What are the variable pricing points and the feature points where we can get to the target price point? What do we have to have? What we have to justify? What resonated? What didn’t? Let’s start with the conjoint looking at pricing and features. Then let’s look at features and start adding and removing features and see how high we can get.” You take those two analysis together, even if you’re there for a week you’ll be surprised how clear the data is when you look at it.
You can look at that and go, “You know what? We have to at least 15 pockets. It has to be this price. We absolutely have to have the feature removable, because we can’t get enough differentiation without it.” You will really see that kind of feedback if you do what you said.
Moby: That’s a great way to do it, because you’re kind of narrowing down the attributes, so that might be …
David: Yeah, and you’re giving them something to compare again.
David: This is no different. You see this in advertising when you see one car versus another car, and they go, “Well, how much would you pay for it?” And even inside a brand, they’ll show you different models of the cars and then you’ll see the prices that are separate. Well, all of those are … to get to those things they do this kind of research. How much more would you pay for having turbo? How much more? Those are all done by conjoint analysis. And you really … that’s how you would do it.
Moby: That’s interesting. So one if I do that kind of analysis, one reason I see myself feeling is if I’ve dug 20 people and I’ve done this analysis about pockets and temperature and like zips or whatever. And if I come across with different answers, could it be … and nothing’s … there’s not trends or they might be a trend, could I be screwing up if I haven’t realized that out of 20 people, there are three different kind of customers who care about very different things. And now maybe I’m seeing a trend that doesn’t really exist in the sub-market that I want to serve, because I didn’t understand my sub-market.
David: Yeah, definitely that can be an issue. So usually if you see this and see that the factors are fairly even across, there’s not anything that stands out. To me easily what I look for is one of two things. First of all is my sample set so different that there’s nothing standing out, like again I’ll go back to the ice cream example, if I talk to the father, I talk to a college student, I talk to the bike messenger, I talk to all these people and they have nothing in common. To expect that you’re going to see a clear trend is unrealistic. So first what this tells is my sample … the population we looked at and sample with the population is so diverse that the there’s not a consistent group where I’m talking to. That’s the first thing.
The second piece that I would say is if you’re not seeing some differences, it means you have him … my concern would be you have too many things you’re asking, with not enough variation between the choices too. So you may have asked about pockets, about liner, about price, and maybe for that size sample set you should really only focus on pockets and ask ten questions about pockets to see the differences in the views of pockets, as opposed to you that’s one question on pockets, one question on liners.
So it’s usually one of two things, either your population is wrong or not wrong but it’s too diverse. Or your questions are too diverse, you’re asking too many things that people give you high level answers without enough striation to really see trends between them. So for example if let’s say we decided we were to go interview 50 parents that came into the store, and we’re going to say parents, right? But we ask them about ice cream, we asked them about sodas, we asked them about the cookies, we asked them about their purchase experience, their wait time. You may get to see some things. But I can’t make a cookie decision based on that or a sandwich decision, because I’ve asked them too many questions that are not related.
So if I really want to ask them about product preferences I would want to spend that time and ask 20 questions or 10 questions about products, what kind of cookies? Is vegan important? How important is it? How much ice cream, do you want more ice cream, less ice cream? Ask those questions. If you had 50 parents you’re going to say, “Is price for the cookie important? What’s the maximum price? What’s the minimum price? What minimum price? Do you think this must be crap food?” You will see enough variation that you can make some decisions. But again if you go back at that same 50 customer audience, if I asked about the whole experience, you’re not likely to get it enough information to make a good product decision.
Moby: And that goes back to what you were saying about understanding the entire process of the customer need, like if you’re starting a business what’s your buying cycle, how many managers have to make that decision to buy, how much does it costs, what’s their budget like, what’s the allocation for new software, are they technology advance?
David: Right. Well, and here’s the other reason that you want to know that. You want to be able to characterize these people well enough so that when you get outliers in your research, you cannot understand did they … Did that outlier fit into the real population? Or was there something different? If they’re real, okay, why are they real? If not, they really … This wasn’t a parent, this actually was a college student. Throw it out. Don’t let it influence your decision, because you realize because you know the customers or the sample you’re looking at, throw it, because it’s actually an outlier. It’s not really representative of an outlier’s data, it’s an outlier of the sample. Does that makes sense?
Moby: Yeah, it does.
David: When I do this stuff I really try to understand the audience, the questions I’m trying to answer, and really who it is I’m talking to who I’m trying to understand. So … and it sounds like it’s a lot, it really isn’t. It sounds way harder. It’s like if I’m going to ask about ice cream and I’m ask college students, I’m going to pay attention, “Does this college student really represent what I want to ask? Do I really know that college student?” And that’s where I spend my time.
And it may be that I look at you when you come in, “Yes, I’m a college student.” You have to do a survey. And I go, “You don’t look like a college student.” I’m going to ask a lot of questions because not because I don’t believe you’re not a college student, because I want to be able to characterize you different. You may represent a different type of college student I didn’t know about. It doesn’t have to do with my cookie question or whatever, but that’s really meaningful for me to do understand my customers.
I can’t tell you how many times I’ve gone, “Well, I know this is what you say doctors are, but did you know I’ve talked to people that do this and they fit our criteria of doctors, and they’re not anything like what we’ve interviewed. There’s a whole thing over here where they have different motivators and interest. We’ve got to look at them too.” And that’s a really important thing. But it had nothing to with the product, it’s because I understand by the time I’ve done that customers set.
Moby: So hypothetically, again for some reason I’m on a roll with analogies. It’s Friday, I don’t know.
Moby: Talking about like maybe finding new customers. If you do a pop up store with ice cream sandwiches and you go in front of park. And parents buy ice cream sandwiches, and you’re like, “Oh, it’s a great market.” And for some reason you discover … and you put a pop store somewhere else and there’s like five drunk college kids who walk up to you at 1:00AM, for some reason you have a pop up store at 1:00AM, or you’re bored or whatever. And they buy, and then 10 more people come and 15 more people come. And that’s a new market for you. Is that kind of what it is?
David: I don’t know if I’d consider it a new market. I consider understanding that there are drunk college students that actually like eating ice cream sandwiches at night, and that would be something I would go test. Because that absolutely could be, and not to give any shot outs to competition, but there’s actually a chain called Insomnia Cookies that has figured exactly that out. And there’s 150-ish stores around colleges. And their things are like the six-pack a cookie. They’re all named after giving cookies for college students at night that have the munchies.
So and that’s a whole thread, a whole market thing they’ve figured out. So yeah, that is absolute powerful to know. You would go, “Oh, yeah, that’s the segment.” You go, “There’s a new segment of the customer set that I want to go validate.” Because if I had five showing up and then I had another ten showing up, that tells me there’s something happening there. If it was five onetime and I never saw that again, I’d go, “Yeah, that was an outlier.” But if I start seeing a few of them, it’s like there’s something here, because we’re getting this activity without any work. Is that a real segment? If so, with a little bit of work make this a very successful business.
Moby: Interesting. What would you say … Or so if I come in with an idea right now and I say, “Okay, this is my product. This is my customer.” Because I’ve worked in this place 25 years and I think I know their customer. But it’s kind of a new technology and I want to just … I’m going to put my money in right now, and I don’t want to do the customer validation. How would you convince me to do this? Take a breath, take maybe three months and research.
David: Well, first I tell you, you can’t have any of my money until you show me some data. Let me do it on your question because I think it’s a good thing. Let’s say you’ve been doing this … let’s stay with your coats, just so we can have a real one. So let’s say you’ve worked in a store that deals with … in a department store for 25 years. You sold tons of these travel jackets, you know what your idea is, and you think you know the travelers. My first thing if I were you is I would go talk to other people that have sold travel jackets for a while and go, “I believe this is the customer for this. Do you guys think there’s anybody else I’m missing?”
And get a couple points, they may go, “No, you got it all. Okay, good luck.” Then I’d go, “Wait, wait, wait. Have you thought about these people? Because I’ve sold these to these.” And you collect other feedback. And the goal would be before I even go do my first data collection, go talk to other people that have that kind of experience and go, “Do you think I’m missing any segments here? Who else can we go after?” And once you identify the two and three, you can’t ten, let’s say the two or three segments you want to go after. Then you go do your research making sure that you’re talking to the three sets.
One maybe, one you think is totally dumb. But if you heard from somebody that I think that’s a real segment, go test it and prove that you’re … that it’s dumb. Or convince yourself they were right, and God, how did I miss that? Those are all for me the most fun with, it’s like, “Wow, I never even knew that one existed.” That’s how I would do it if I were you.
And here’s the other thing to be careful about though with this that I think you started to touch on, and goes back to the qualitative and the quantitative. If someone comes in me and said, “I’ve talked to the customers and I really understand this product and this segment, and this is the number.” My first thing I try to do is test the figure if it was anecdotal feedback. How many customers you talked to? 20. Okay, how did you talk to them? Well, I went around and just interviewed them. Okay, did you give them a survey? Did you give them anything to have some consistency of what you asked? No, but I’ve interviewed them all.
My confidence factor starts to go down. What I want to know is that you used this qualitative research to actually get to a set that’s representative. It maybe that you still did all qualitatively, but if you told me, “Look, I’ve talked to a hundred customers over the last two months using generally the same sort of questions and I feel pretty confident.” My confidence starts to go back up. I’d love to see that you had a qualitative study or a conjoint analysis on it, super.
But if you told me that you’ve done 150 over three months and this is consistently what you’re hearing, it starts to go up. I want to see not just you I’ve talked to four customers and this is what they say – my uncle Harold, two of his friends, people we know. That’s anecdotal feedback. I want to be able to take the feedback you give me. If you told 150 and I don’t believe you totally, I would actually be able to go after that and figure out what you said, and create a survey and go test against it and replicate it with a quantitative analysis. That’s to me the right way to go do it. The interesting way to do it.
Moby: So what’s that last jump off the cliff when … because there’s the infinite cost to infinitely perfect information. What’s that jump where you can say, “Okay, I know enough at this point.”
David: I think the split between when do I go to a true quantitative analysis or not is dependent on your needs for outside support, but I don’t say just funding. Funding is a big one. You may need to go do that because you really want to raise money and your investors are going to want to know more than your anecdote. If you can say, “I’ve talked to 150 customers, and I ran a survey, here’s my qualitative and quantitative feedback,” and they support each other. That’s an external driver that’s going to do that.
It could be that your employees may need you or people and your founder if you’re not a senior person, you need to convince them that you’re right. The quantitative helps. Like for example for me at Dell I needed to do quantitative, because even though I was the head of worldwide marketing for notebooks for example, I couldn’t say, “I’m the head of worldwide notebooks,” you do that or else I had peers all over the world that also were the head of notebooks for their region.
I had the product group but I could just tell them, “I’ve done the research. I’ve done the analysis.” They go … and they’re all marketing people, they’re smart, they’ll go, “Show me the data.” So as an influencer you may need the quantitative to support your qualitative work. It isn’t looking for the perfect data. It’s figuring out what the levers are you’re trying to pull. And then for me it’s when you have to influence, get external factors influence.
For Motion it was my supplier. We were asking them to do … my partner and suppliers to these products, we were asking them to do stuff that never been done. And they said, “Show us that when we do this that there’s a market for these products.” It wasn’t they didn’t believe us, but they needed their own validation, so they could go back to their management and go, “Look, they’ve done the research. It’s worth us taking the risk to building these products.” So to me it’s important to have those tools to be able to also get people on board.
Moby: Where you can convince other people maybe it’s time to jump.
David: Right. Whether it’s employees, suppliers, investors or whatever, and you could do it with confidence, I mean, that’s the other part is … What I find for me is the more quantitative validation I get, the more I’m much more comfortable about taking a hard position of, “Look, I did the research. I’ve talked to the customers. I understand them well. And here’s the quantitative data.” And be careful if you ever do this that you don’t go to the same customers, because otherwise it’s just replicating it in a different format. It’s not as valid.
Moby: This is fantastic.
David: Honestly I think most business and product success start from people understanding how to do this. And you don’t have to do it hardcore, expensive. You literally can’t interview people on your own, do a Qualtrics or a SurveyMonkey survey, get friends to get your email addresses and all that, and you can do this very cost effectively and collect data and do this. This does not have to be an expensive or really time consuming thing. It needs to be … If you want to do this right, it has to become part of your DNA. To me the harder the conversation, the riskier the question is the more I want to get some research behind it. That’s just been my experience.
Moby: Cool. This is like … I love the fact that we jumped between analogies all the time – ice cream, health care, and jackets.
David: Yes, they work. That’s the way it should be. It should … these are applicable all over.
Moby: That’s true, because every like business is at risk getting too many data to support your decisions, especially when you’re trying something new.
David: Yeah, the more unknown it is, the more you got to do the work of research. That’s not even just product market fit, that’s validation of a need. So those are even harder to do. So all right?
Moby: Thank you so much, David.
David: I’m happy to do it. See what we’ll back up.