From Spontaneous Order to Ordered Spontaneity
Liberalism — the great social innovation of rule by rules rather than rulers — has found itself under attack in recent years. This might have come as a shock to Americans back in 1989. At the time, Mikhail Gorbachev was the leader of the Soviet Union. The Cold War was winding down. And Francis Fukuyama had just published a landmark article, "The End of History?" which took the world by storm.
Fukuyama argued not that history will literally end or that major conflicts and transformative developments will cease to happen. Rather, he contended that economic and political liberalism had defeated all rival models for social organization, and that no new rivals were in prospect. "The triumph of the West, of the Western idea, is evident...in the total exhaustion of viable systematic alternatives to Western liberalism," he wrote. Marxism? Failed. Fascism? Failed. Theocracy? Failed. None could produce the miracles of social organization, the blessings of freedom, and the relief from warfare that liberalism had provided. None could show a plausible pathway to social success, even in theory.
But developments since the fall of the Berlin Wall suggest that liberalism may have triumphed over one set of challenges only to be ambushed by another — one with which classical-liberal thinking has failed to keep pace.
This essay presents the story of how my thinking about liberalism has evolved over the past 30 years. By bringing my own intellectual journey to the foreground, I hope to capture something about liberalism's path since the Reagan revolution. More importantly, I hope to acknowledge a tragic oversight in my thinking about modern democracy, capitalism, and science — an oversight that I was not alone in committing. I assumed that liberal social systems are largely self-ordering. It turns out they are anything but.
THREATS FROM WITHIN
In 1989, when Fukuyama published his watershed essay, I was working as a journalist for National Journal, an elite policy magazine in Washington, D.C., where I wrote about Congress, the economy, and (my personal favorite) agriculture.
I immediately saw truth in Fukuyama's thesis. I still do. Organizing a dynamic global economy — processing untold amounts of information, inventing countless new technologies and business models, delivering vast quantities of goods — is something only economic liberalism can achieve. Other systems are incapable of gathering the requisite information, motivating entrepreneurs, organizing resources, coordinating networks, and more. Whatever their flaws, liberal democracies exhibit similar virtues. Their regular elections, checks and balances, distributed decision-making, guarantees of minority rights, and incentives to compromise enable liberal democracies to govern large, diverse societies without oppressing them or collapsing into chaos and war. No other political system can make that claim.
I also came to see that Fukuyama's dynamic duo — capitalism and democracy — was incomplete. A third branch of liberalism existed, a branch even more important and successful than the first two. I called it "liberal science": our distributed, rules-based, depersonalized social network for making knowledge. This network includes the hard sciences, of course, but I added the modifier "liberal" to broaden the concept beyond those disciplines and include the whole truth-seeking community, including the social sciences, the humanities, even journalism.
Like its political and economic siblings, liberal science possesses the capacity for astonishing feats of social organization. It produces knowledge at a prodigious and accelerating pace while also foreclosing the use of war and coercion to impose someone's idea of truth on everyone else. If anything, liberal science has achieved more in the realm of knowledge than liberal democracy and capitalism achieved in the political and economic realms.
But even in 1989, dark clouds had gathered. One came in the form of new doctrines arising in academia. Critical theorists argued that bad ideas, even bad words themselves, are socially oppressive and emotionally harmful. Post-modernists argued that all knowledge claims, including scientific claims, are mere impositions of power. Speech codes could thus be justified as preventing "verbal violence," or "words that wound"; censorship could be justified as leveling the playing field for historically oppressed communities.
I called these academic doctrines the "humanitarian" and "egalitarian" threats to free inquiry. The humanitarian model turns liberal science into a human-rights violation, thus precluding the progress of knowledge. Meanwhile, the egalitarian model insists on favoring supposedly marginalized claims or claimants, which requires someone to regulate speech and inquiry. This opens the door to authoritarianism.
I also worried about liberal democracy. Fukuyama may have been right about the failure of challenges from without, but what about rot from within? Thanks to my reporting on agricultural subsidies, I saw a problem. Agricultural programs designed in the 1930s and '50s — including subsidies for commodities like honey, mohair, and wool — were chugging along, even as their original rationales had long since expired. The federal wool program, to take one example, was launched to ensure the availability of wool for military uniforms. But the military had switched to polyester by the 1960s. Do you think the subsidy went away?
As a D.C.-based farm lobbyist once told me with a rueful smile, "there's nothing more permanent in this town than a temporary program."
A light bulb went on. I quit my job at National Journal to write Kindly Inquisitors, a book published in 1993. This work immersed me in the thinking of Karl Popper, the 20th century's greatest philosopher of science. Popper noted that the scientific method is neither experimental procedure nor logical method, but institutionalized trial and error. Knowledge can be advanced and accumulated by floating hypotheses, subjecting them to criticism, seeing which survive, and then floating and testing some more. The method of trial and error scales massively and obviates the need for centralized control, instead allowing a worldwide scientific community to fill in what philosopher Susan Haack likens to a transcendent crossword puzzle on which, every day, millions of investigators add new letters and change old ones.
Popper's thesis was elegant and powerful, and it drew upon the grandfather of all trial-and-error systems: biological evolution. Science, Popper explained, is an ecosystem in which knowledge evolves as an emergent property of competition and falsification. Like natural evolution, science is organic, directional (though never final), and spontaneously self-ordering.
Liberal democracies, too, draw their strength from something resembling biological evolution. Like evolution, they rely on trial and error. And instead of reaching, or attempting to reach, fixed states of perfection — final justice, final government, final policies, or any other finalities envisioned by French and Marxist revolutionaries — they self-correct as politicians, voters, and institutions adapt to changing times, coalitions, and public desires. American democracy may adjust imperfectly, but it has evolved much more effectively than sclerotic authoritarian systems like the Soviet Union.
All of this was true. Yet reflecting on the lobbyist's remark — that there's nothing more permanent than a temporary program — made me wonder: What if trial and error went awry in government?
The University of Maryland's Mancur Olson had already broached a related question in the context of economics. In two great books, The Logic of Collective Action and The Rise and Decline of Nations, he argued that under stable political orders, economies become clotted with cartels, rent-seekers, and subsidies. As "distributional coalitions" — or pressure groups — win and defend sinecures and monopolies, economies lose dynamism.
The same thing, I realized, happens to democratic governments. Because the benefits of government programs are concentrated but their costs are dispersed, their beneficiaries have much more incentive to defend them than anyone else does to abolish or reform them. This is why farm subsidies are almost never repealed. As legacy subsidies and programs accumulate, they monopolize resources and crowd out alternatives. Stuck with its first tries, government loses the ability to use trial and error to solve problems. It gradually becomes a calcified collection of mostly obsolete programs and benefits that are there because they are there.
Thus inspired by evolutionary thinking, I published Demosclerosis in addition to Kindly Inquisitors. Both adopted evolutionary models. Both emphasized the centrality of trial and error in government and science. Both warned of forces that could corrupt and calcify liberalism by disabling or politicizing the method of trial and error. Both saw the preeminent dangers to liberalism coming from interests and activists that impede competition and choice.
Those insights still stand. So what did I miss?
DISINTERMEDIATION
I use the word "liberalism" to mean not modern left-wing progressivism, but Enlightenment-style classical liberalism. That older liberalism embraces the following doctrines, among others: that people are born free and equal, that laws and rules should be impersonal, that minority rights should be protected, that coercion and violence should generally be off-limits, and that free speech and toleration are fundamental rather than instrumental values. This kind of liberalism is a broad church with many branches. In the modern era, however, a predominant style emerged that rallies to the flag of what might be called "disintermediation."
Beginning in the 1960s (but drawing on older roots, including the writings of Jean-Jacques Rousseau), Western liberals came to distrust and condemn any institution or authority that stood between individuals. Anything that restricted or even guided individual choice became suspect.
This distrust was understandable, and to no small extent justified. After all, for decades — forever, really — institutions had enforced pernicious attitudes like racism, sexism, and homophobia; intermediaries had engaged in cronyism and self-dealing; authority had abused power and enforced conformity. As Yuval Levin notes in his important book A Time to Build, our society needed a corrective push toward individualism and liberation. Yet as Levin also observes, the correction went too far. Americans developed an outright hostility to intermediation and institutions.
Institutions like party organizations, unions, media, and universities had been so strong for so long that it became easy to take them for granted. We forgot why they were there in the first place. Did we really need political parties? They were dinosaurs, after all — remnants of a less enlightened age in which party apparatchiks selected presidential nominees in smoke-filled rooms. In 1968, Hubert Humphrey won the Democratic presidential nomination without entering a single primary. Dismayed reformers swept away party insiders' gatekeeping function, implementing a direct plebiscite. So obvious was the principle that the people should choose party nominees that Republicans hastily followed the Democrats' example. What could go wrong?
For Democrats, disaster hit promptly with the nomination and subsequent rout of presidential candidate George McGovern. After that, party insiders temporarily wrested back some of their gatekeeping power through the informal system of influence known as the "invisible primary." But for Republicans, things took longer to fall apart. Finally, in 2016, Donald Trump — a demagogic renegade who is not in any meaningful sense a Republican — swept past all barriers to capture the party and the presidency. That same year, Democrats only narrowly averted a takeover by a far-left insurgent who is not in any meaningful sense a Democrat.
Today, both party establishments, but especially the GOP, are largely spectators in the process of choosing their own party's candidates — a condition unheard of in other major democracies. Because so many primary races are decided by small numbers of voters who tend toward extremism, the system elevates politicians who are not only less experienced, less capable, and less responsible than the ones chosen in smoke-filled rooms, but often less representative of the electorate as well. The role of formal parties is reduced to jawboning, sponsoring direct mail, staging debates, and serving as vehicles for whichever candidates and factions seize the steering wheel.
A parallel process unfolded in the epistemic realm. At the turn of the millennium, the digital revolution brought forth a blaze of disintermediation, which, at least initially, looked like all upside. Unchained from the gatekeepers of Big Media, ordinary citizens would have all the world's information at their fingertips. Everyone could encounter everyone in unfiltered, unfettered peer-to-peer conversations. A thousand flowers would bloom in an Eden of interconnectedness.
Of course, instead of a thousand flowers, we got a handful of online giants with more market dominance than the television networks had ever enjoyed. Instead of a social-media culture of respectful criticism and structured argument, we got performative outrage, canceling, and trolling; we got bots and algorithms fine-tuned for polarization and radicalization. It became impossible to be sure if an entity we were interacting with was even human. The digital arena was not a marketplace in which better ideas reliably bested the competition; it was not even epistemically neutral. Instead, its ad-based model actively favored misinformation and tribalism, because misinformation and tribalism attract eyeballs, and eyeballs make money.
Some people said that's just fine: The consumer knows best what the consumer wants. In an era of political and epistemic consumerism, intermediation and institutionalism drew indifference, skepticism, and outright hostility from all three of America's major political and cultural movements. Progressives saw intermediaries as corrupt obstacles to participation, self-expression, and emancipation. Conservatives dismissed intermediaries as captives of special interests and the cultural left. Libertarians damned intermediaries as cartels that distorted competition and restricted choice.
And so, in many ways and for many reasons, America set about disintermediating wherever possible. Mediating institutions like the parties, mainstream media outlets, unions, and civic groups emerged damaged or dysfunctional. Even religion became consumeristic — a shopper's paradise.
To be clear, some changes were necessary. Americans did some valuable disintermediating. But in the end, we forgot the reasons we needed intermediation in the first place.
So let's ask the question: Why intermediate? Why aren't consumerism, competition, and choice enough?
In a word, the answer is: humans.
THE TEMPTATIONS OF SOCIOPATHY
Plato warns in the Republic that an amoral person with power and insufficient accountability will use that power to oppress others. Thomas Hobbes argues in Leviathan that in an unstructured, unconstrained environment, humans war against each other, making life "nasty, brutish, and short." Adam Smith understood that greed and self-interest are foundational motivators (though not the only ones). America's founders knew that civic virtue cannot be taken for granted, and that popular impulses must be cooled, channeled, and restrained.
Those luminaries all understood that the temptations of sociopathy — of self-interested and anti-social behavior — lurk within all of us. They understood that tribalism, domination, and violence issue from deep-seated human impulses. They understood that cognition is susceptible to all manner of biases and deceptions, against which intelligence and good intentions provide no surety.
Guided by these insights, those thinkers pursued their own versions of the same task: engineering social environments that would deter sociopathy, incentivize cooperation, and force people to check their beliefs and biases against those of others. To paraphrase social psychologist Jonathan Haidt, liberal democracies only work if the social and legal settings are right — and such settings are as numerous as they are complicated.
Who does the work of calibrating and maintaining those settings? All of us, to some extent — but especially institutions and intermediaries. Consider the work that party organizations do when they function as intended. They block sociopathy, for instance, by blocking renegade presidential runs — as they did with Henry Ford in the 1920s and George Wallace in 1976. They organize collective effort by brokering compromises and building coalitions among diverse interests and factions — something individual voters cannot do at the voting booth. They teach values by acculturating youths and immigrants in the ways of our politics (one writer characterizes the old party machines as schools for politics). They recruit and train competent politicians, tap promising candidates, clear pathways for them, and promote them through a succession of offices.
Contrary to popular belief, party organizations did not impede individuals, but rather effectuated them. They organized the machines and networks through which rank-and-file partisans could participate between elections. Party members could become ward heelers, march in torchlight parades, or volunteer in the local Democratic or Republican club. Even on bad knees, my uncle religiously attended meetings at his local Democratic club into his old age. Although he was only a retired garment worker, a member of Congress and several New York City party dignitaries came to his funeral.
Of course, the party machines of yore could be abusive. They needed reform. But there was a baby in there with the bathwater.
Today, those energetic and empowered political parties survive only in history textbooks. Participation in politics for most people amounts to voting once every couple of years and maybe donating money. The parties, meanwhile, primarily function as brands, not institutions. A few people may volunteer for a candidate or get involved with a partisan cause, but the large majority tune in to politics only on Election Day, if at all. This is a thin kind of political life.
Twitter is no substitute for face-to-face engagement with others through intermediaries — human intermediaries. Algorithmic intermediation, through which we participate by clicking on links curated by software, is woefully insufficient.
We think of intermediaries as disempowering. But if we operate as consumeristic individuals in a bureaucratized society without healthy intermediaries like parties and unions and churches, what exactly can we do to influence our larger society? As it turns out, not much. And people are not happy with that answer.
I understood in the 1990s that liberalism depends on complicated rules, but I supposed that once those rules had taken root, they would endure untended absent some catastrophe. By the turn of the century, it became clear that this was not the case.
In 2000, Robert Putnam published his influential book Bowling Alone. He found a surprising and sharp decline in many forms of face-to-face participation. Membership and volunteering had declined in everything from religious groups and labor unions to parent-teacher associations and fraternal organizations. Other scholarship documented declines in social capital — the trust-based interconnections that protect and define us. Social capital was not just declining, but bifurcating. As big-tent intermediaries like parties withered, professionals with advanced degrees amassed their own social resources and hoarded them, leaving working-class high-school graduates behind to face disintegrating schools in declining neighborhoods.
At the same time, polls provided witness to the public's deteriorating trust in institutions. Fukuyama himself, in his 1996 book Trust and other writings, noticed that the United States, or at least a significant chunk of it, was starting to look less like a high-trust, low-friction society akin to Scandinavia and more like the low-trust, high-friction societies of the developing world.
A wake-up call came in 2003 from an unexpected quarter. We all recall what was supposed to happen when the United States invaded Iraq. Saddam Hussein had suppressed civil society, but Iraq was a middle-income country with an educated populace, a largely secularized culture, and a well-developed bureaucracy. With Hussein's foot off its windpipe, Iraqi civil society would re-inflate. Setbacks would occur, of course, but Iraq seemed a promising platform for democratic transformation. With America's friendly nudge, liberation would blossom into liberalism.
Instead, Hobbesian chaos consumed postwar Iraq. In June of 2003, a Washington Post article described how U.S. troops had "cleared garbage from a field in Fallujah, resurfaced it with dirt, and put up goal posts to create an instant soccer field." The next day, troops returned to find the goal posts stolen, the fresh dirt scraped away. One bewildered U.S. soldier remarked, "[w]hat kind of people loot dirt?"
Elsewhere, the hopeful rush of democratization in the 1990s gave way to what Larry Diamond calls the "democratic recession." This letdown recalled the way in which the euphoria of the early years of Boris Yeltsin's presidency — when Russia seemed destined to join the community of democratic nations — was followed by the authoritarianism, corruption, and cynicism of Vladimir Putin's regime.
In 2016, we witnessed a final, decisive, and devastating shock. What was left of the Republican Party organization collapsed as a sociopathic demagogue seized control, first of the party and then the presidency. Casting aside norms left and right, he launched a systematic campaign of Russian-style disinformation against his own country, subverted the administration of justice, and disregarded his oath. Had he ultimately gotten his way, we would not have had a democratic transition of power. In other words, we would have lost our democracy. Yet instead of being damned by Republicans as a villain, he retained his grip on the party and its base.
So began a new journey for me. I had taken liberal institutions and norms so much for granted that, in some sense, I had stopped even seeing them — at least until they failed. I had come to regard them as natural background conditions: Liberalism would flourish simply by being allowed. The developments of the 2000s convinced me that I had made a fundamental mistake. It was an error James Madison would not have committed.
SHORING UP LIBERALISM
Thus chastened, I endeavored to see, understand, and defend the sometimes submerged, often disparaged institutions and social norms upon which liberalism stands.
One strand of that work is political realism, which holds that democracy depends on political machines and parties — even on hacks and careerists — to do what the political scientist James Q. Wilson called "assembling power" in the formal government. Generations of reformers had handicapped political middlemen in favor of direct primaries, small donors, and open meetings. The result was exactly what had been foreseen by realist political thinkers from Niccolò Machiavelli to Daniel Patrick Moynihan, Edward Banfield, Nelson Polsby, and Wilson himself: chaos.
Readers of my work on political realism will encounter a frame of mind alien to the political-reform community but robustly represented among academic political scientists. This outlook regards with skepticism today's chicken-soup prescriptions for political reform: ever more direct participation, ever more individual choice, and ever more transparency; in short, ever less intermediation.
Some of those prescriptions might be good ideas in some contexts, some might not be. But none of them can replace the political organizations and professionals who can balance competing interests, broker complex compromises, organize coalitions, coordinate messaging, recruit competent candidates, reward team players, marginalize renegades, and plan for the future. Indeed, nothing will work well in our politics unless we reestablish the founders' hybrid system in which the public and professionals work in tandem, each doing what the other cannot. To reform with an exclusive focus on choice and participation is akin to adding more airline flights while firing all the air-traffic controllers.
My book The Constitution of Knowledge, based on an essay published in these pages, embodies a second strand of my work toward re-intermediation. It does not repudiate Kindly Inquisitors, but adds a crucial piece of the puzzle that the earlier book missed. Following John Stuart Mill and Popper, Kindly Inquisitors assumed that knowledge emerges organically from a process of critical exchange, much as species emerge organically from evolutionary competition — all true and good. But Mill and Popper harbored no particular interest in institutions. Instead, they imagined something like a peer-to-peer network of individuals exchanging ideas: I present an idea, you debunk it, someone else weighs in, and so on.
We know now, however, that simply throwing people into unorganized contestation produces not constructive criticism and debate, but anger and outrage, sectarianism and schism, ad hominem attacks and warring tribal loyalties. The world of unstructured interpersonal exchange looks a lot like Twitter — an epistemic version of Hobbes's state of nature.
The magic of liberal science lies not simply in freedom of inquiry and open public conversation, but in the particular structure imposed on the conversation and accepted by the participants. In much the same way that the U.S. Constitution imposes institutional protocols and disciplines on democratic decision-making, liberal science imposes a host of similar guardrails on the search for truth. Collectively, those institutions, protocols, and disciplines compose the "Constitution of Knowledge."
My latest book seeks to make that implicit constitution visible and to unapologetically defend it. It is a first step toward what I call "Madisonian epistemology," which holds that free speech, while necessary, is not sufficient. Objective knowledge — humanity's greatest product — does not emerge automatically or organically from public conversation. It is instead the product of a built regime of interlocking rules and institutions: rules and institutions set up, like Madison's Constitution, to channel disagreement into cooperation, replace coercion with compromise, exchange violence for persuasion, and protect against both tyranny and chaos.
TILTING TOWARD MADISON
My own sometimes painful discovery of the limits of liberal self-organization reflects the deeper question of our regime: Can we move beyond overreliance on freedom, choice, and competition — important as they are — to rediscover and rebuild the institutional intermediaries and rules that structure liberal life? Can we thereby ensure liberalism's viability in the face of new and unforeseen challenges?
By "we," in this instance, I mean classical liberals. They have made brilliant contributions to our understanding and appreciation of freedom. But partly as a result, they find themselves lagging when it comes to understanding and explaining the value of institutional mediation. Too often, they see freedom and intermediation as mutually antagonistic. In fact, the opposite is true.
My plea to classical liberals is this: Think less about spontaneous order and more about ordered spontaneity. Think about institutions less as constricting individuals and more as empowering them. Think about career politicians less as obstacles to democratic decision-making and more as partners in the complex process of governing. Think less about dismantling intermediaries and more about rebuilding them.
In those ways and others, we should tilt away from Mill and toward Madison. Both are liberal touchstones, but neither suffices by himself. They need each other — and we liberals need them both.
