In April 1985, the Coca-Cola Company launched what became known unofficially as ‘New Coke’. Its sweeter formula was intended to draw market share away from Coca-Cola’s primary competitor PepsiCo. But the product’s introduction backfired spectacularly and the company reintroduced the original formula within three months. Despite a rebrand during the 1990s, the new formula was eventually discontinued in July 2002. What had gone wrong? How could the company have been so out of step with its customers?

In assessing market readiness, Coca-Cola had conducted thousands of blind taste tests, facilitated focus groups and conducted quantitative surveys. These traditional research methods all revealed a preference for New Coke over the classic recipe. The researchers were convinced that they had a strong evidence base for the superiority of the new product. But they had not accounted for the emotional attachment consumers had with the Coke brand or the cultural meaning carried by the classic formula.

The professional identity of the researchers rested on their dedication to scientific measurement and their methodological rigour, but this meant they overlooked the relevance of what could not be measured, including consumer beliefs and feelings. By focusing on which product tasted better rather than asking test participants how they would feel if classic Coke was replaced with the new formula, the researchers inadvertently skewed the results.

Blind spots

The Coca-Cola researchers’ blind spots correlated with psychologist Carl Jung’s concept of ‘the shadow'. This contains those aspects of ourselves that we reject, suppress or fail to acknowledge – which serves as a basis for my new book Shadows at Work. Jung demonstrated how these qualities retreat into the unconscious, continuing to shape our behaviour as individuals and as communities. Even without our conscious awareness, shadows inform how we think, the choices we make and our actions, as well as the impact our behaviour has on other people.

In this way, alternative approaches, research methodologies, perspectives and interpretations were pushed into the shadows by the Coca-Cola researchers. The approach failed to allow for methodological flexibility and the focus on data drew attention away from the cultural insights that emotionally intelligent interaction with test participants would have afforded them. Science and numbers won out over the messiness of human relationships.

Decades after Jung first introduced the concept, the work of political scientist Philip Tetlock has validated the psychologist’s insights, demonstrating how professional identity informs what people can and cannot see. Tetlock contrasts those who have curious, intuitive and generalist tendencies (foxes) with hyperspecialists who dedicate themselves to ‘one big thing’ (hedgehogs). Hedgehogs are inclined to suppress doubt and uncertainty, ignoring evidence that doesn’t conform to their preferred approach or that challenges their expertise.

In forming their professional identities, hedgehogs like the Coca-Cola researchers create potentially catastrophic blind spots. This is a recurrent issue in market and social research. During the lead up to the Brexit vote, for example, pollsters relied on a familiar phone-based approach, ignoring other ways that voters could be engaged. Their attachment to an established model blinded them to change and distorted their findings.

However, when people are willing to adopt the more holistic approach of the fox, acknowledging and working with what previously has been pushed into the shadows, they often discover valuable sources of insight that have been hidden in plain sight.

Shadow work

Of course, researchers must maintain their professional rigour, but it is still possible to accommodate fox-like behaviours. Tetlock’s study of people who consistently outperform political experts with their predictions (superforecasters) points the way for an approach to research that engages with our acknowledged capabilities but also with those that have been pushed into the shadows. Preferences and assumptions should always be questioned, biases challenged, and room granted to alternative methods and perspectives.

The Coca-Cola researchers, for example, could have been more open to emotional responses from customers, not only during blind tests but in focus groups where they could have posed explicit questions about replacing the classic version of Coke. They also could have benefited from inviting outside perspectives and allowing for cultural insights to be incorporated with their scientific findings.

Rather than allowing our shadows to have unconscious influence over us, they can be brought into the light and integrated with established practices. Shadow work for researchers, therefore, means acknowledging theoretical and methodological attachments and seeking data and insights that break us out of our professional comfort zones.

In this way, we can become more comfortable with contradictory findings, acknowledging that in an increasingly complex world, being able to say ‘I don’t know’ can be more valuable than a strongly held conviction that blinkers alternative possibilities.

Steven D’Souza is an educator, speaker, executive coach, author, and senior partner at Korn Ferry. His book, Shadows at Work: Harness your dark side and unlock your full potential, is now published

References

• D’Souza, S. ( 2025 ). Shadows at work: Harness your dark side and unlock your leadership potential. LID.
• Hays, C. ( 2004 ). Pop: Truth and power at the Coca-Cola company. Hutchinson.
• Tetlock, P. E. ( 2017 ). Expert political judgment: How good is it? How can we know? Princeton University Press. Second edition.
• Tetlock, P. E. & Gardner, D. ( 2015 ). Superforecasting: The art and science of prediction. Random House.