494. Why Do Most Ideas Fail to Scale?

Kmart's BlueLight Special was a successful in-store promotion that revolutionized retail innovation and significantly impacted sales. The BlueLight Special was an effective way to clear out unwanted inventory while increasing customer engagement and spending. This promotion was such a phenomenon that it caught the attention of competitors like Walmart. However, Kmart's decision to centralize the BlueLight Special ultimately led to a decline in its effectiveness. The lesson learned is that successful promotions should not be brought under rigid centralization but should be allowed to adapt and evolve in different markets.

John List's book, The Voltage Effect, suggests that all scaling failures stem from a failure of common sense, and that there is no single quality that distinguishes ideas that have the potential to succeed at scale from those that don’t. Instead, List argues for a 'common sense checklist' to identify potential issues. He emphasizes that using incentives that can scale is paramount, whether you're in healthcare, education, or business. Learning to avoid false positives, assessing whether success is due to the chef or ingredients, and knowing when to quit are also essential. List proposes moving towards producing policy-based evidence, rather than relying solely on evidence-based policy.

John List calls for a revolution in research methodology, urging researchers, policymakers, and businesspeople to take into account the constraints that come with scaling up an idea. He gives an example of the Chicago Heights Early Childhood Center, where he deliberately did not hire the best teachers to see if the program would still work with below-average teachers, which is necessary for scaling up. List argues that researchers should move beyond efficacy tests and also test programs for scalability, especially with regards to budget and workforce. Otherwise, promising research may not be effective when scaled up. List's book, The Voltage Effect, explores the concept of 'voltage', which refers to the power of an idea to create meaningful change.

Voltage drops can cause a great electric charge of potential to dissipate and harm businesses, resulting in dashed hopes, squandered money, hard work, and time. Evidence-based policies should be implemented at scale, especially in projects involving human-centric programs. Fidelity is of utmost importance in scaling programs, and testing them prior to that. Furthermore, humans are not scalable as they tend to undo even the best presets and features, making smart technology less efficient than what it is designed to be. By prioritizing evidence-based policies and not assuming a rational audience, businesses and programs may better navigate such challenges and minimize the impact of voltage drops.

John List's unique background and experience have made him a successful economist and valuable asset to both academia and private firms. Unlike many of his peers, List did not attend top-tier schools and instead grew up with working-class parents, giving him a different perspective on people and markets. His lived experiences, combined with his original research, have allowed him to be successful in the field of economics and bring a new approach to experimentation. List's success has also made him a sought-after consultant, with private firms eager to profit from the hidden secrets in academic research. His ability to infuse economics into all aspects of a company has made him a valuable asset to firms seeking to grow and innovate.

John List, a renowned economist and seasoned entrepreneur, suggests that scaling up a business or institution requires understanding power dynamics and fostering diversity. When leaders become the power holders, they may not be fully aware of the power imbalances they create within their organizations, especially for people from marginalized communities. Building a diverse team from the early stages of growth can address this issue, while also bringing in a variety of perspectives that can enhance the organization's performance. Diversity can stem from observable characteristics such as gender and race and unobservable characteristics like socioeconomic background. By prioritizing diversity and awareness of power dynamics, leaders can create a more inclusive and successful workplace culture that can withstand the pressures of scaling.

Embracing diversity within organizations can bring about a wealth of new ideas and solutions to problems, ultimately leading to increased productivity. However, it is important to be mindful of the potential for voltage drops when scaling up ideas or products. It is crucial to test on a small scale before rolling out to the entire group. The success of the polio vaccination is a prime example of an idea that scaled up well. On the other hand, the D.A.R.E. program serves as a reminder that even seemingly great ideas can fail. Thus, careful testing and consideration must be taken before expanding any idea or product.

The D.A.R.E. program, famously endorsed by Nancy Reagan, failed to effectively deter drug use in America due to a common problem of scaling failures known as false positives. False positives arise for many reasons, such as bad measurement or wishful thinking, and can be particularly problematic in social sciences where it's difficult to establish cause and effect. The failure of the D.A.R.E. program highlights the challenges of scaling policy initiatives based on limited evidence. The case also raises questions about the effectiveness of mass-media campaigns against drug use, such as the Partnership for a Drug-Free America's 'This is your brain on drugs' ad, which may have backfired by normalizing drug use. Scaling policies must be rigorously tested and evaluated before implementation to prevent wastage of resources and unintended consequences.

Data manipulation and academic shenanigans can create a black eye for science, including economics. To prevent future occurrences, quick replication and partnerships with trusted organizations must be established. Journals are now demanding upfront data for review, and safeguards like using Benford's Law can detect potentially fabricated data. The social sciences are in the early stages of data generation, and the need for accuracy is vital, especially as science can have a significant impact on society, as seen with Covid vaccinations. By implementing transparency and security measures, we can improve the reliability of research in social sciences, particularly economics.

Incentives that can scale are crucial for successful implementation of initiatives. The Voltage Effect book by John List provides an example of using incentives to increase tax compliance in the Dominican Republic which brought in an extra $100 million in tax revenue. The messaging campaign aimed to tip the balance of the benefit-cost analysis for potential tax evaders by reminding them of the jail sentences or public exposure for tax evasion. The message about jail time was found to be more effective than the one about public exposure. However, if the government had to put all the tax evaders in prison, finding enough prison capacity would be a scaling problem. Thus, using scalable and effective incentives is key to maximize the impact of initiatives.

To successfully scale a business, decision-making should be based on marginal thinking, rather than average analysis. This means looking at the impact of the next dollar spent, rather than analyzing an entire pool of spending. It's important to consider the network externalities of a product or service when growing it. While growth can provide benefits, organizations must be mindful of the potential negative consequences that come with scaling. Companies like Facebook are examples of both scaling success stories and cautionary tales due to their vast reach and potential impact on society.

While U.B.I. pilot programs have shown success, it's important to consider the potential spillover effects on the local labor market and community when scaling up. When considering quitting a job or passion, it's crucial to think about the opportunities being given up and whether they are worth sacrificing for the sake of staying in a situation that may have gone wrong. Optimal quitting is not a one-size-fits-all solution and requires careful consideration on a case-by-case basis.

Periodically evaluating your current situation and comparing it with others in your peer group can help you determine whether to continue or quit a particular activity. This is especially important in competitive markets where only some people will succeed and many others will fail. To be successful, we need to have a comparative advantage in what we do. Additionally, we should not be afraid to admit when we are not good enough at something and be open to changing course. The secret to high-voltage scaling is understanding when to quit and not clinging to unrealistic dreams. By doing regular comparison shopping, we can be more serious about changing the world.

John List, an economist and frequent guest on the show, acknowledges the criticism he receives for pointing out the impracticality of research ideas that look great on paper. He also discusses the negative association with his name due to a famous serial killer with the same name. Despite this, List continues to share his research findings in hopes of making a positive impact. His new book, The Voltage Effect, explores issues related to scaling and policy-making. List's wife, pediatric surgeon Dana Suskind, is also publishing a book called Parent Nation, which explores unlocking a child's potential. In conclusion, List and Suskind are dedicated to advancing research that can have a positive impact on society.