innovation – Justkul

Steve Jobs and Market Research in Contexts of Uncertainty

Posted on January 11, 2014January 14, 2025 by John Hannon

According to some, market research is pretty useless in contexts of uncertainty. These contexts include disrupted markets, the introduction of radical new innovations, or the launch of new products and services. Steve Jobs explained this view in an interview:
“The problem is that market research can tell you what your customers think of something you show them, or it can tell you what your customers want as an incremental improvement on what you have, but very rarely can your customers predict something that they don’t even quite know they want yet. As an example, no market research could have led to the development of the Macintosh or the personal computer in the first place. So there are these sorts of non-incremental jumps that need to take place where it is very difficult for market research to really contribute much in the early phases of the thinking about, you know, what those should be. However, once you have made that jump, possibly before the product is on the market or even after, it’s a great time to go check your instincts with the marketplace and verify that you’re on the right track.” (PBS/Nova Interview,1990)
Steve Jobs could recognize the usefulness of market research, but he suggests it is not useful when the market is going through these non-incremental jumps. A similar mindset is found in an even stronger form in certain parts of the present-day startup community: it is often thought that it’s better to just bring a product to market and see what happens, rather than rely on misleading market research that is incapable of envisioning the future.

There is no doubt that many companies succeed by adopting approaches that do not involve much market research. However, this in itself is not a sound proof for ruling out the usefulness of market research, because we have no idea how many failed startups would have succeeded if they had paid more attention to the findings of market research. Instead of being based on a sound empirical argument, the plausibility of this argument seems to rest, in part, on a number of assumptions about what market research is, and how it works.
Standard Market Research and the Principle of Continuity
When Steve Jobs spoke about market research, he was likely speaking about a particular kind of research. These are the surveys asking you to rate services on a scale of 1 to 10, asking you whether you prefer red cars or blue cars, or giving you complicated conjoint trees to determine if it is more important for you to get more product or lower prices. All of these studies depend implicitly on a principle of continuity: that your present opinions can be used to reliably predict your future actions.
As Jobs appears to recognize, this principle can break down in disrupted markets. If we look at this principle in more detail, it typically implies at least four sub-principles, all of which can fail when one is conducting research:
1.   The overall market will be the same in the future as it is now. This principle is reasonable in many situations, but Jobs proved it false when his teams introduced the Macintosh, the iPod, the iPhone and the iPad, each of which changed the markets in fundamental ways. Furthermore, it fails all the time in ordinary business practice: financial bubbles burst, companies that are on the top of the world go bankrupt a few years later, entire industries disappear.
2.   A respondent’s beliefs will be the same in the future. Closely aligned with the preceding principle, a respondent’s views can change even if the other components of the market do not. Someone might be against sharing credit card numbers or personal information online when they take a survey, but come to accept the practice a few months later.
3.   Respondents are accurately conveying their present beliefs when they take a survey. It’s common knowledge that people sometimes lie on surveys, or under-report activities they are ashamed of. However, even when people are trying to accurately convey their present beliefs, they may fail to do so. In particular, we tend to think of our personalities as uniform over time and place, but many of us adopt very different personas when we spend time with our family, or with our friends, or with our colleagues, or when we are on our own. A view we reveal on a survey may not reflect the view we act on later while partying with friends.
4.   What someone says accurately reflects what they will do. This belief is questioned frequently today as big data enables us to actually track the difference between saying and doing. In certain cases, it may turn out that basing predictions on past actions rather than opinions produces more accurate results.
In markets that remain constant over a long period of time, it may be unnecessary to question any of these principles. For instance, a mere correlation between what people say and an increase in sales might be all that is needed for an incremental gain, and no one may care to think any more deeply about why it’s the case.
Market Research in Uncertain Times
However, as soon as contexts arise when the principle of continuity does not hold, companies and whole markets can be led badly astray if they merely follow past correlations. In these situations, as Jobs notes, respondents may very well not know what they will want.
Yet, even in the most disrupted markets, where there are many discontinuities at the surface, there may be underlying factors that do not change, and therefore at deeper levels the principle of continuity can still hold, and research still has something to say.
In these cases, instead of mechanically asking respondents, “How likely is it that you would buy this new product?,” one should begin by thinking hard about the context of purchasing and using the product and ask, “What things have to be true if people are going to buy this product?” Moving from the general question to a hypothesis about its components in this way does at least four things:

First, it forces us to articulate and address the most important assumptions we are making. Often, we are implicitly assuming things that will prove to be false. Identifying the assumptions before conducting research allows us to explicitly test them.
Second, once we have identified a broader range of assumptions, we may be led to reconceive the range of possibilities we need to take into account in our questions and in the subsequent analysis. For instance, we may determine that a particular assumption does not hold in general and thereby identify new segments. These different segments may have different needs and decision-criteria.
Third, it can lead us to identify important outliers that do not necessarily conform to our expectations. If we think that a factor is essential, and it turns out to be false for a small number of respondents, this may be a cue to talk further with those individuals. Perhaps they are anticipating the next big trend in the market.
Fourth, it makes us think more deeply about the enterprise we are undertaking. Part of this comes from the fact that good research leads us to ask “Why?” questions that we may not have even thought about before.

Analyzing decisions into components makes it possible to find continuities in conditions of uncertainty. Moreover, when we have the whole framework of assumptions and possibilities available, it then becomes possible to track underlying factors that do change through proxies and analogical trend analyses.

Implementing this approach well requires a different mindset and a broader range of skills than most standard market research companies offer. Identifying assumptions requires being able to break things down into their components like an engineer or analytical philosopher. Exploring possibilities and outliers requires a certain openness to possibility characteristic of design-thinking approaches. Thinking more deeply may require additional skills such as big-data analysis, statistics, expert interviews, or ethnographic studies. Yet, all of these approaches can be incorporated into a research strategy.
Many skills may be required, but great consultants and entrepreneurs often bring together many of these research skills. I’m thinking of when Orit Gadiesh talked to customers and metallurgists when she introduced a Bain client to continuous casting techniques in steel manufacturing. Or, when survey data led Howard Moskowitz to discover that there was no perfect recipe for Prego tomato sauce. Steve Jobs, especially, shows the passion of a good researcher. This is quite apparent in his interviews in which he enthusiastically discusses quantifying data, visiting 80 automated manufacturers in Japan to understand automation, or revising products after observing people using them. People can bring about the future through research, albeit of a non-standard type.
This much may be obvious. However, what might not be obvious is that even the most humble research tools can benefit from this approach. An online survey constructed to test hypotheses about decision factors can tell you not only why your customers make the decisions they do, but also can help you make better predictions in uncertain contexts.
Follow Jobs’ Advice: Ask the Right Questions
In another interview five years later, Steve Jobs was asked about his transition from being a hobbyist to being the executive of a multimillion dollar company. “How do you learn to run a company?,” he was asked. He observed, “You know, throughout the years in business I found something, which is, I always asked why you do things. And the answer you invariably get is that it’s just the way it’s done. Nobody knows why they do what they do. Nobody thinks about things very deeply in business, that’s what I found.” (Steve Jobs, The Lost Interview, 1995)
It turns out Steve Jobs didn’t like market research when introducing non-incremental changes, because he was a great researcher himself. According to his own account, the secret to his success in business was asking the right questions and not settling for status quo responses. Unless you have the luxury of being in an industry without innovation or disruption, you and your research company shouldn’t settle for the status quo either.
By @jfhannon, CEO at Justkul Inc., a research firm focused on the needs of strategy and private equity.

Key to Innovation: Hire a Philosophy Major

Posted on September 20, 2013November 30, 2015 by John Hannon

In a recent article in the New York Times (September 13, 2013), Susan Dominus described how the Office of Personal and Career Development at Wake Forest University is transforming the liberal arts by emphasizing the importance of job placement. Although the discipline of philosophy was only mentioned off-hand at the end of the article, the New York Times decided to entitle the article, “How to Get a Job With a Philosophy Degree.” I am not sure exactly why the New York Times decided to give this article this particular title, but given the rhetorical build up at the end, one surmises that it is because philosophy is thought to be obviously one of the most impractical subjects one can study. And when Andy Chan, who is quoted throughout the article, tells parents to “hold their tongue” if their child expresses a desire to be a philosophy major, because they may “decide they love math anyway,” and that he will help turn “academic risks into résumé-ready experiences,” it sounds like Chan agrees. Philosophy is a risk, but a good college placement office will teach your child to be successful anyway.

I have sympathy with the idea that there is something “impractical” about philosophy. The discipline has a long tradition of taking pride in this fact, and Plato famously argued that one would have to use force to get a philosopher to become involved in the common affairs of the world. In addition, many of the problems that philosophers think about have very little direct impact on day-to-day business. However, many of the problems they think about do have direct impact, especially in the ambiguous area of contemporary ethics. This is one of the reasons why Google has hired a philosopher.

However, the case for hiring a philosophy major is much stronger than this. Ms. Dominus’ article quotes the CEO Alan Naumann saying that he hopes graduates have “analytical skills.” It is difficult to find a subject area that provides better analytical skills than philosophy. Students intending to be philosophy majors consistently score among the top verbal, quantitative and analytical writing scores of the GRE, and in many years they score higher than all other majors. Philosophy majors do better on LSATs than any other majors, and easily outperform business students on GMATs. This should not be in the least surprising: contemporary analytical philosophy teaches logical reasoning, and philosophy majors have to deal with some of the most difficult texts and issues human beings have ever come up with, whether it be the foundations of morality, Kant’s transcendental dialectic, Aristotle’s metaphysics, or defining concepts like rotation in relativistic physics.

In the age of business innovation, however, the argument for hiring a philosophy major is even stronger than this. Philosophy has been many things, but since the time of Socrates, the art of questioning assumptions, asking, “Why?,” and drawing out the implications of alternative assumptions are the most fundamental activities of the discipline. Businesses now call this activity, “thinking outside the box,” but in the ancient world it was called dialectic, and it was the very core of the discipline. Even today, it is not an exaggeration to say that philosophy is the art of innovation, because it is the discipline most focused on questioning the status quo. If a company is not good at developing innovations, or is facing a situation in which existing frameworks are failing, then they may seriously want to add a philosopher to their payrolls. In addition, many philosophy majors have been extremely successful in business: George Soros, Reid Hoffman, Don Brownstein, Carly Fiorina, Larry Sanger, and Patrick Byrne among others.

In my own experience in business consulting, I find myself constantly drawing on the skills I acquired as a philosophy major. My experience learning philosophy is what enables me to reframe client challenges at the very beginning of a project, to take difficult problems and reformulate them into simple examples, to seek out the underlying reason for a correlation, or to identify analogous situations to a problem at hand. Talking to a philosophy major who is a successful consultant at one of the largest consulting firms, he said that he often finds himself asking the question “Why?,” when no one else in the room is. I find that it is this move from the how to the why that enables me to avoid errors others are prone to make, or to obtain a fresh perspective on an ambiguous problem. Having been a philosophy major I can ask these questions and explore the underlying logic of ambiguous and complex situations with more confidence than other more technically-focused majors. (Why-questions are also important for leadership too, see Simon Sinek’s Ted talk.)

Of course, not all philosophy majors will be at home in a business context. The very same skepticism that allows great innovations to happen also can lead philosophers to challenge the underlying assumptions of a business culture, capitalism in general, or to choose to pursue a life of the mind. In addition, contemporary business is often driven by mathematical models that some philosophers are ill-equipped to deal with. But philosophers do come in all stripes: in my own sub-discipline of philosophy of science many philosophers are experienced mathematicians, physicists, psychiatrists and biologists and quite comfortable with statistical concepts. Because these individuals have both mastered a science and philosophy, they can both do the math and understand deeply why the math works. It is curious that this subdiscipline is not on the radar of most corporations.

According to many business leaders today, they want employees with analytical skills who can questions assumptions and develop entirely new approaches for the 21st century. People who not only can handle complex or ambiguous problems, but also truly thrive in such environments. Yet, at the same time, HR departments in these very same companies routinely select for mere technical proficiency or for people who have only proven they can operate well within already existing frameworks. It’s about time that companies start realizing that in an age driven by innovation, hiring a philosophy major is not a risk, but not hiring one may be.

By @jfhannon, CEO at Justkul Inc., a research firm focused on the needs of strategy and private equity.

Image by Ty Fagan, courtesy of Katrina Sifferd.
See also the additional charts here.

61 Rules for a Culture of Innovation

Posted on September 8, 2013November 30, 2015 by John Hannon

How do you structure a corporation for innovation? There are many answers to this question, which vary depending on the size and purpose of the organization. However, if any company can claim to be a model for innovation in the past decade it has been Google. Not only has the company brought innovative products to market such as Gmail, Google Maps and the Android operating system, but also its open business model and culture challenge many assumptions of a more traditional, 20th century corporation. One person responsible for much of this innovative business model and culture is Jonathan Rosenberg. Jonathan was VP of Product Development at Google, and was responsible for much of the culture that yielded these innovative products. (He still is a Google employee.) In 2010 Jonathan presented a talk to his alma mater about the lessons he has learned as a manager. In the talk he outlined 61 rules for success. What you realize reading the list is that even though only six of the rules are explicitly focused on innovation, many of the others are important catalysts to the innovation process and helped make Google the effective company it is. The talk is useful, and has become a key reference point for founding my own corporation, Justkul Inc. Because an online video is not the easiest format to access individual rules, I decided to make a list of the rules he outlines in that video. In keeping with Jonathan’s emphasis on open innovation, I thought it would be useful to share the list here. Keep in mind that what follows is my version of Jonathan’s rules, and I make no claims that Jonathan would endorse this outline, or that I have captured every one of his rules correctly. You’ll have to watch his talk to decide. The list below is also no substitute for the actual talk, which supplements the rules with very useful and entertaining commentary and stories. I include some commentary of my own in italics in the post. If you would like to comment further, feel free to do so. I’d be particularly interested in any experiences you’ve had following or not following these rules, and what additional rules have been important for running your own innovative corporation. Communication, 9 Rules

Over-communicate in all ways all the time.

This is true, provided that actual communication is happening. Never take a lot of time in order to say very little.

Openly share everything with your colleagues.

Some caveats are obviously necessary here. The point is to share things that are actually useful for colleagues. Sharing such information not only gives a company a competitive advantage, but can enhance the connections between team members. This last fact is important, because if it is implemented incorrectly and does not foster these team connections, it can lead to a stifling, ineffective organization.

“Repetition does not spoil the prayer.”
Each word matters. Remember Blaise Pascal’s dictum: “If I had more time, I would have written a shorter letter.”
Great leaders are great teachers, and great teachers are great storytellers. Narrative is how we learn.
As leaders you learn more by listening than talking. “When you listen, you learn how things work as opposed to how you think they work.”
If you must talk, ask questions. People learn more from your questions than your answers.
If you actually know the answer in a business situation, stop listening and by all means talk. However, show what you mean through data.
Strive to respond to emails instantly. A manager is nothing but an expensive router, be a good one. Always ask, “Who needs to know this?”

I know people who have had great success following the rule of responding to e-mails immediately, but it doesn’t necessarily work for everyone, especially people engaged in work that requires concentration. Also, as Sandra Bond Chapman has discussed in her Forbes Next Avenue post, there are real risks associated with the multitasking mindset that results if one is constantly responding to e-mails. Personally, I would much rather have people take advice from Blaise Pascal’s dictum above, and take a little extra time to ensure communication is shorter and more efficient. However, the question “Who needs to know this?” should always be asked. If some people may or may not need to know it, thenGrexit offers a nice app for that. Company Culture, 16 rules

Avoid hippos (“Highest paid person’s opinions”).
You shouldn’t be able to figure out the pecking order of org chart by looking at the product.
Help organizations crush bureaucracy in all its forms. Dying organizations foment it.
When you are trying to accomplish something ask for a winning strategy AND the tactics needed to win.

Our variant of this is whenever we come up with a good idea, we always try to also ask what are the first few steps to implement it. It is always useful to take an idea one step further than you need to at any given moment on the path to implementation.

People are more productive when they are crowded.

This is not always true. Susan Cain’s provides a fairly good defense of the need not be be crowded in her TED talk about introverts

Empower the smallest of teams.
Working from home is a malignant metastasizing cancer. Ban it.

Given her recent actions at Yahoo, you can tell Marissa Meyer worked for the same company as Jonathan from this rule. Marissa qualified her response in light of the backlash it generated, but Jonathan appears to hold an even stronger version of the position. There are good reasons to have people in the office, especially given the importance of serendipitous interactions for innovation and the importance of co-location for agile workflow. However, I think that time away from the office can be equally valuable, not just for work-life balance, but also because it can promote other useful serendipities.

Engineers and product managers add complexity. Marketing adds management layers. Sales adds coordinators. Manage this.
“Knights are knights and knaves are knaves.”
Once someone reveals himself to be a liar, he is a liar.

In my experience context can force even honest people to lie. Philosophy abounds with examples of these cases. Design and manage a company in such a way that you these contexts do not happen.

Trust, but verify.

My favorite version is the Persian proverb, “Trust in God, but tie your camel tight.”

Focus on values rather than costs. In business more revenue solves all your problem. Use the 80/20 rule to determine what to focus on.

Good advice in general contexts, but one should add the exception famously identified by Clayton Christensen: in contexts of disruptive innovation following the 80/20 rule can sometimes lead a company astray. Fortunately, Google has a great antidote for this: the fact that employees can spend 20% of their time working on projects of their own design.

Never suggest copying a competitor.
Hope is not a plan.
Success breeds the green-eyed monster. Take away its key weapon, surprise; fight it with its kryptonite, humility.
Do all reorganization in a day.

Hiring and Development, 14 Rules

Know how to interview well.
Great people make a great company, which in turn attracts great people.
Managers don’t hire people, hiring committees should hire people.
Promotions should be a peer review process.
Don’t hire specialists.

This resonates with me, because so many companies seem obsessed with specialists now. For a lot of companies the situation is even more extreme: it’s not just that a company wants a Python programmer, but now it has to be a Python programmer with five years’ worth of experience who also knows how to do statistics in R who lives in Houston, and has led a team for 3 years (not 2 or not 4), and so on. If you define the parameters stringently enough, soon there will be no one who could possibly fulfill them. Equally important, it prioritizes proxies that have little correlation with performance over actual performance. Basically, most hiring processes are broken, and the current reliance on algorithms and big data may even make it worse. See the excellent post by Nick Corcodilos

You cannot teach passion.
Urgency of the role isn’t sufficiently important to compromise quality of hiring.
Identify bad eggs.
Get rid of bad eggs.
Diversity is your best defense against myopia.
You can’t punt the management training program.
Life is not fair; don’t try to make it fair. Disproportionately reward risk-takers and performers.
If you are going to pay Alex Rodriguez $33 million a year, feed him the ball. Build around people who have the most impact.
The best way to get rid of bad eggs is penguin-pecking.

I’m not sure I agree with this point. I know this approach is common in many legal firms, but sometimes a transparent direct conversation has value too. Equally important is taking extra steps to ensure the process is as painless for everyone involved as possible. All the consulting firms know that ex-employees make for future clients, so it is worth ensuring that no bridges will be burned. I also like firms that take creative approaches to this, such as how ?WhatIf! Innovation apparently provided job placement and detailed letters of recommendations when employees were forced to leave Decision-Making, 6 rules

Decision making is about consensus, not unanimity.
There is no consensus without dissent.
If there is doubt about what to do, consider your customer’s perspective.
Choose your goals wisely.
None of us are as smart as all of us.
Where there is harmony, there is no innovation. Innovation comes from disagreement, not from harmony.

Fostering Innovation, 6 rules

Most companies manage creativity in order to manage risk. Don’t do this.
Innovation comes from creativity. Creativity cannot be managed. It can be allocated, budgeted, measured, tracked, encouraged, but it can’t be dictated.
Create a culture of yes, based on optimism and big thinking.
Never stop someone from moving forward with a good idea because you have a better one.
A leader’s job is not to prevent risk, but to build the capability to recover when failures occur. A good failure happens quickly and provides many lessons. A bad failure takes a long time and you don’t learn anything.
A good crisis is a terrible thing to waste.

Humility, 10 rules

Learn something new so you can remember how hard it is to learn. Teach something so you can learn.
Never stop learning.
Humility is correlated with age. Arrogance is inversely correlated with age.
You get personal leverage through empowerment, delegation and inspection.
Judgment comes from experience, and experience comes from errors. Publish post-mortems when something fails.
Smart people can smell hypocrisy. Culture is set from the top, and once set it cannot be changed.
Don’t burn bridges.
Always ask yourself, would you work for yourself?
Write a self-review and be critical about yourself. Do this every year.
Communicate, confess, comply.

Jonathan emphasizes at the end of the talk that people should make their own lists and not simply follow his rules verbatim. However, I think Jonathan’s list is a fantastic place to start. Let us know what you think. By @jfhannon, CEO at Justkul Inc., a research firm focused on the needs of strategy and private equity.

All Our Problems are Simple?

Posted on August 16, 2013November 30, 2015 by John Hannon

A while back, when I was talking with managers of an international manufacturing company it came up that I was a philosophy Ph.D. It struck the managers as an odd background for business, and they asked how I came to be involved in business consulting. I explained that philosophy is about questioning assumptions, and in my sub-specialty, philosophy of science, the focus is on evaluating many of the same methodologies that underlie business decision-making. It is an ideal vantage point for choosing between methodologies, and for coming up with new and novel solutions. While impressed, they both questioned the need for such a background by saying, “All our problems are simple.”

My jaw dropped when I heard that. I know many of the challenges facing that industry — there are pages of difficult challenges listed under the risk factors of their own company’s 10-K — and the thought that any of these are “simple” seemed absurd. I’m sure the CEO of their corporation wouldn’t describe all these challenges as “simple.” In these managers’ defense, it may well be that they were only delegated simple problems, and therefore had the luxury of not having to think about the difficult ones. However, the conversation made me wonder whether their response was indicative of a larger problem with corporate decision-making: is it possible that managers systematically ignore complex problems because they do not have the right mindset and methods for solving them?

Simple Strategies

In order to answer this question, we need to understand why there might be a bias towards simple problems and solutions. It is surely true that many business problems or organizational challenges can be solved using very simple strategies. I have heard from a number of experienced management consultants that when they walk into a company, it usually takes less than two days to diagnose what is wrong and determine what needs to be done. The next three months will be spent dealing with the complexity of implementing that solution, but coming up with the strategy itself requires no really novel thought or innovation, just the application of a standard framework. Some examples of simple strategies that have worked in the past include:

Our production costs are too high –> outsource operations to China and India.
Some of our businesses are in decaying industries that seem unlikely to recover –> sell off those businesses.
Our costs are too high –> lay off workers.
We need to pump life into our business –> hire an advertising company to launch a campaign.

Although the implementation can be complex, coming up with these strategies is extremely easy: after all, anyone who passes a standard consulting case interview has to come up with a well-presented version of such solutions in less than 20 minutes.

A Framework Approach to Solving Problems

Consulting case interviews are instructive ways of understanding much of contemporary business culture, because they present, in a highly concentrated form, what many corporate leaders would consider the ideal way of solving a business problem.

The case interview begins with very little initial detail, and it is up to the candidate to structure the nebulous problem, obtain the necessary information from the interviewer, and develop a strategy that solves the problem in about 20 minutes. The main tool to enable a consulting candidate to succeed is what I want to call the framework approach to business. By using predefined frameworks to structure their line of inquiry well, a candidate can quickly diagnose a nebulous client difficulty without missing any important details.

A case interview typically proceeds in six steps:

Restate the client problem, and determine whether there are other related problems and what a solution would look like both qualitatively and quantitatively. (2 min.)
A pause, in which the interviewee can breathe while coming up with a framework. (1 min.)
MECE (mutually exclusive and collectively exhaustive) presentation of a framework for approaching the client issue and a discussion of how the consultant is going to proceed. (2 min.)
Branch elimination to diagnose the problems the company is facing and/or to come up with a solution (ideally with quantitative support) (15 min.)
Once all the significant problems are identified a recommended solution is defended with quantitative support. (3 min.)
Summary: a restatement of the problem, a recommendation of a solution, and next steps. (1 min.)

The heart of this interview, and the part that a consultant candidate absolutely has to have down are steps (3) and (4) above. These steps also have to be performed with the precision of a Swiss watch. This is where the candidate shows she is familiar with business concepts, that she can think on her feet, and that she knows how to structure a nebulous issue. Although the framework itself should not seem rote or contrived, it is typically hypothesis-driven, and based on one of a few dozen ways of approaching an issue: SWOT analysis, Porter’s Five Forces, profit analysis, value chain analysis, Bruce Henderson’s growth share matrix, etc. (One of the nicest quick overviews of frameworks comes from Victor Cheng; a PDF can be found here.)

There are good reasons why consulting firms focus on these two steps in interviews. Besides the fact that it helps one systematically identify the important issues that are driving a case, it also makes the consultant look on top of the issues and in control of the situation. One of the worst situations consultants can be in is one in which a client catches them missing an obvious solution or making a claim that cannot be supported with the available evidence. This looks bad, and can lead to a whole project unraveling. Consultants will have difficulty justifying their high billing rates in such a situation.

However, a consultant candidate who adheres to a clear framework without jumping around can justify every step of her analysis. If you interrupt a very good case interview candidate in the middle of an interview, she should be able to not only say where she is in the process, but explain exactly why her last question was the right question to ask. An example:

“Previously we agreed that the problem was a decrease in profit. This is the result of either increasing costs, decreasing revenues or both. We determined that revenues were not decreasing, so we concluded that costs must be increasing. As my initial outline indicated, we have identified five factors that could lead to increased costs, and we are now exploring the hypothesis that the price of raw materials has increased. Hence, I’m asking for numbers that can indicate to me whether or not this accounts for the bulk of our decreasing profits.”

The answer looks well-structured, and it is difficult to find fault in the consultant’s analysis. This looks good to even a highly skeptical audience.

Yet, there are trade-offs of this highly-structured framework approach. For one thing, it is typically biased against creative strategies. There are opportunities to be creative with the application of a framework, of course, but if we understand creativity as the ability to come up with untried and untested solutions for solving a business problem, this kind of creativity typically requires an iterative, exploratory mindset that typically cannot be fit into a tight time frame. When I was discussing case interviews with experienced interviewers, most agreed that one should not try anything creative in a consulting interview: stick to the framework, and creative strategies should only be mentioned in the most speculative way at the very end. This means that in an entire 24 minutes, only about half a minute can be devoted to coming up with truly novel and untried strategies. Given this, it is no surprise that people at major consulting firms have complained to me about a lack of creativity in new consultants.

Creativity and Framework Development Contrast the framework approach above with fields that are centered around creativity and innovation. We all admire strokes of creative genius that tie together unconnected strands in the flash of a moment, but most processes for developing creative solutions don’t work this way. Instead, the processes are typically time-consuming, repetitive, iterative, imprecise and involve many false starts and wrong turns. Because inspiration is a significant factor in coming up with new solutions, it can be unpredictable, and positively hindered by too much structure. (There’s a reason why great ideas often come in unstructured parts of a day like showering, or the hours before falling asleep.) Hence, a creative approach can appear amateurish to a culture that is driven by efficient frameworks. At a recent talk, a design-thinking professor recalled overhearing a client on the phone exclaiming with irritation, “These people have no processes!”As a point of contrast, humanities students are used to approaching nebulous problems in a creative way. Consider how different the scenario would be if you interrupted a humanities Ph.D. student in the middle of writing a dissertation. He might not be able to explain where he is in the process, or even where he is headed. You might get something like the following:

“I had been reading an article on analogy in ancient biology, which emphasized the relationship between the word “analogon” in Aristotle and the words “ana logon” in Plato. It made me think that Aristotle’s use of the word might be a neologism quite different from its original meeting. So, I compiled all 1000 or so occurrences of the word in Aristotle to see if he used it consistently, but he didn’t. There was a combination of old and new uses. Yet, as I was looking through the occurrences it seemed we could classify them into distinct types that are used consistently within each of three distinct contexts. I’m not sure what I’m going to do with this, but it may be connected to an article I read on the division of sciences in antiquity. Perhaps the division of sciences article does have an impact on my own work after all?”

The context of writing a dissertation is very different from passing a case interview. Yet, I think the two answers quoted above illustrate some important differences between creativity and framework approaches. In the case interview framework above it’s not only that the goal is clear, but also the steps needed to reach the goal are clear. You first determine whether there is increased cost or reduced revenue, and in order to determine cost you determine the different components of cost. At each of these steps new challenges may arise, and the framework itself could morph into another framework, but nevertheless the overall structure is relatively clear. A consulting candidate would be faulted for jumping around unexpectedly or not sounding confident about what the next step in the analysis will be.

Contrast with the dissertation case above. In this case the goal is nebulous and unclear. Something was vaguely related to something else. The initial step of looking up over 1000 occurrences of a word in Greek texts is not the most efficient use of one’s time. The fact that the initial hypothesis was refuted makes that time seem even less well-spent. Finally, the connection with another article at the end didn’t produce a clear conclusion, but only a question for further exploration. However, adopting this less-directed approach can be essential for writing a good dissertation. The main goal of a dissertation, especially in philosophy, is to add new questions and novel insights to an academic discussion, and tasks like the tedious process of reviewing all the occurrences of a word can be important steps to those new insights. Such insights are seldom obtained through a simple and straightforward process.It’s this back and forth process with no clear goal that probably contributed to the “No processes!” comment above. Yet there is a process, it’s only happening at a higher level. By describing it as “higher” I do not mean to say it is better in all cases, only that it is the kind of process that was probably used to develop the framework the consultant candidate is using in the first place. Innovation precedes implementation, and both require different mindsets.

When Simple Strategies are Not Enough

In the beginning I mentioned how some managers had told me that all their problems were simple. In many industries this may still be partly true, but as more and more people become familiar with previously known frameworks, as freemium options disrupt the marketplace, as business becomes increasingly international, as new technologies and business models are developed almost daily, as the problems to solve become more global and complex, it is becoming increasingly necessary for companies and non-profits to do more than develop simple strategies. In a 21st century world, new, innovative and creative strategies are increasingly necessary.

Yet, there is a balance: too much creativity can also be a bad thing for most organizations, and there is significant value in the efficiency of a framework approach. Further, a good hypothesis-driven methodology, such as the kind that gets one past consulting interviews, should be in every employee’s toolkit, and often is necessary to bring a new creative idea to fruition. However, a more innovative approach should be prioritized in at least five different contexts:

In disrupted industries that require that fundamental assumptions of the industry be challenged.
In launching new products, services or business models in markets that do not yet exist.
In areas of philanthropy and education where the emphasis is long-term planning, the coordination of a vast number of interests or inputs, or on changing a deeply-held mindset or belief.
In any discipline that is focused on creating new paradigms or new ways of looking at the world.
In ethnographic studies, trendspotting, market research or long-term forecasting when the goal is to gain insights rather than merely confirm something with a statistically valid sample size.

In these areas it can be useful to supplement standard business models with very different approaches and skill sets. In some cases this may even require changing internal structures or the culture of a firm, or hiring a philosopher, in other cases it may require outsourcing some processes to a third party. (We recommend Justkul Inc., but admit to being biased here!)

Not all problems are simple. However it is done, businesses, corporations and philanthropic organizations will increasingly need to embrace both creativity and framework approaches to succeed.

By @jfhannon, CEO at Justkul Inc., a research firm focused on the needs of strategy and private equity.

Failing for Success: The Importance of Low-Fidelity Prototypes

Posted on March 4, 2013November 30, 2015 by John Hannon

Many of the strategic issues we’ve been thinking about at Justkul Inc. revolve around the intersection between design-thinking and corporate strategy. Although we believe both can be compatible, they are often in tension with one another. One of our hopes is to bring both fields closer together.

One concept about which many design thinkers are seemingly at odds with corporate strategists has to do with failure. Business strategy often ruthlessly avoids failure. Yet, in many cases, “failure” can be good and should be strategically incorporated into business processes. I say “strategically,” because not all failures are the same, and recognizing this point can have a profound impact on innovation.

To understand the importance of this point, it is worth keeping in mind how large corporations succeed. Corporations do this primarily by developing reliable processes that can be replicated through time while avoiding failure. Some industries, such as the automotive industry and manufacturing aim at six sigma accuracy, which is typically defined as less than 3.4 mistakes per million products. When you consider that individual products can have hundreds if not billions of components (think computer chips), to even come close to achieving this level of accuracy is an amazing accomplishment, and requires not only technological innovation, but also corporate cultures that are committed to the goal of avoiding failure.

Yet, even though many mature companies aim for that level of accuracy in some processes, it can be detrimental to other processes. For instance, any firm that is developing a new innovative product should generally not aim for perfection in the beginning of the product’s development, but should rather aim for rough prototypes that test the fundamental ideas first. This is because the process of developing a revolutionary new product often requires a great deal of failure. At an opposite extreme to six sigma accuracy consider the famous 5,127 prototypes that James Dyson made before he finally hit upon his famous vacuum. If you want to look at this design process using the same scale as a six sigma process, instead of 3.4 failures per million, that would be a failure rate of about 999,800 per million, or 294,000 times an acceptable six sigma rate! No large corporation would survive if it had that kind of failure rate in one of its key replicable processes.

However, Dyson’s “failures” were of a very different sort than the failures that prevent a company from achieving six sigma accuracy. They may have been failures so far as they failed to manifest the final successful product, but successes so far as they were important steps along the way. Thomas Edison encapsulated this thought well: “If I find 10,000 ways something won’t work, I haven’t failed. I am not discouraged, because every wrong attempt discarded is another step forward.” Dyson and Edison’s “failures” reflect the very nature of the process of innovation.

Most companies have come to recognize this truth in some form, and the phrase, “fail early, fail fast, and fail often,” has become a mantra for R&D. Consequently, management is usually willing to tolerate a certain degree of “failure” from their R&D departments, and to understand that the same metrics cannot be applied there as in other parts of the corporation. However, being tolerant of failure often doesn’t go far enough. There are circumstances in which failing to produce a perfect, compelling product should not only be tolerated, but encouraged.

Aiming for Imperfection

Imperfection should not only be tolerated, but encouraged in the early stages of product development. In these circumstances, striving to produce too perfect a prototype can lead to significant failures later on.

To see this, note that any prototype of a product will succeed in some ways and fail in others. In one extreme let’s imagine the 5,126th prototype of Dyson’s vacuum cleaner. I have no idea what the product looked like, but given the late stage in the development, I imagine that it was very close to the final product. It had high fidelity in relation to the final product. When Dyson was evaluating such a product, most of the details were likely working satisfactorily, but there may have been one or two things that were off: perhaps the color or the texture of a surface material, perhaps there was a defect in the motor that needed to be fixed. But I hazard to guess that at this stage of development 99% of the product was in its final form and functioning correctly.

Again, I have no idea precisely what Dyson prototype #1 looked like, but given the nature of innovation, I would be inclined to imagine it was very different from the final product. It had low fidelity in relation to the final product. It probably did not match the final product in shape, color, size or even function. It might have even been made out of cardboard or drawn on paper.

The level of fidelity of a prototype is important because it is directly related to the type of feedback one will receive. If a potential purchaser of a Dyson vacuum cleaner were shown prototype 5,126, that person might decide to buy it on the basis of its color, regardless of what functional innovations it incorporated. The product just looked nice. In contrast, a viewer of prototype #1, whatever that looked like, would probably not be distracted by the color. In fact, it would be pointless to ask about color at that stage, because color preferences will likely change as other important factors changed.

If the goal is to receive feedback on fundamental concepts, then there is no need to produce a finished model. In fact, the finished model with all its polish and beauty can be a distraction from the intended evaluation. Hence, for a very interesting reason, one should not aim to produce perfect and beautiful prototypes early on in a design process: they can be distractions from the basic innovations that one wants to evaluate.

Not just a Concept for R & D

The same principle holds in areas far removed from product development, and this concept can be carried over into all aspects of business and life. For instance, the broad outlines of a new business strategy might be evaluated incorrectly if it is initially packaged too well. A hiring process might not be optimized if a job candidate is expected to perform too well on an initial interview. An initial brainstorming session is often less productive if there is an expectation for too refined a result. And as I will explain in a future post, one will learn things more slowly if one doesn’t provide adequate space for failure.

Everyone wants to have their ideas succeed. Corporate culture only reinforces this inclination whether you are in a six sigma operation or a less than one sigma one. The fact that beautiful prototypes can succeed more often than ugly ones, that quantitatively validated ideas can succeed more than those that are more nebulous, encourages us to adopt certain strategies. But when it comes to early stages of a product’s or an idea’s development, this in itself can be a mistake. Fail early not only because it is better to get such failures out of the way in the beginning, but also because it may be the only way to ensure products will succeed in the end.

By @jfhannon, CEO at Justkul Inc., a research firm focused on the needs of strategy and private equity. This post and the concept of fidelity were inspired by an excellent talk I attended by Anijo Matthew (@anijomathew) on 2/13/2013.