THE CYBERNATION REVOLUTION
The 1964 TRIPLE THREAT MEMORANDUM is a cautionary tale in forecasting the future, but this time really could be different. In this week’s System Check, we go back in time to try to see the future.
Free Systems is very focused on the emerging politics of AI, and especially the risk that AI will concentrate economic and political power. Because I find these questions to be so profound, I have to sometimes remind myself how hard it is to actually predict the future. I don’t want to make the same mistake so many academics have made before, and obsess over a problem that turns out not to be the real problem (remember The Population Bomb? Stanford is still wearing that embarrassment.)
What’s old is new
Here’s a good cautionary tale. In 1964, a group of important Americans, including Nobel Prize winning economist Gunnar Myrdal and Nobel Prize winning chemist Linus Pauling, sent President Johnson their TRIPLE THREAT MEMORANDUM. The letter expressed their “foreboding about the nation’s future.” They declared an urgent need for “public measures that move radically beyond any steps now proposed or contemplated.”
What were they so worried about? Something they called THE CYBERNATION REVOLUTION.
They feared that no one would need to work due to automation, and that this would create a political crisis. They recommended dramatic policy action “to develop ways to smooth the transition from a society in which the norm is full employment within an economic system based on scarcity, to one in which the norm will be either non-employment, in the traditional sense of productive work, or employment on the great variety of socially valuable but ‘non-productive’ tasks made possible by an economy of abundance; to bring about the conditions in which men and women no longer needed to produce goods and services may find their way to a variety of self-fulfilling and socially useful occupations.”
Sound familiar???? It’s almost eerie. It is the exact same conversation the labs are having about AI today.
Spoiler alert: the experts were SUPER wrong in 1964. There was no urgent job displacement, and no need to pursue dramatic policies to forestall it or help Americans to adapt to it.
…but this time really does feel different
Here’s the other part of the problem, though: what’s going on now really does feel different, though, doesn’t it? If you just use AI chatbots to help you write, I can definitely see how you’re underwhelmed and dubious on the whole thing. But if you use coding agents, it’s hard to escape the feeling that something super profound is changing.
Here’s a good recent example:
This piece is insane! Just for fun, Rohit spun up an entire new simulation of evolution. Reading it gave me a profound sense of awe—just a year ago, it would have been totally unthinkable that someone could casually drop a blog post like this. It’s extraordinary!
The sense that we are living through crisis is not new
Whichever way AI heads, it’s always good to remember that the world is constantly facing immense crises. I was reminded of this yesterday. In helping my dad to clean out his office, we found this remarkable letter from Herbert Hoover to my grandfather in October, 1941.
At the time, my grandfather was a professor at Stanford, and he was part of a group of faculty who had signed an open letter entitled “Dynamic Defense.” Here’s an excerpt:
Hoover, who at the time led the Hoover Institution, wrote to my grandfather and they exchanged several letters—unfortunately I don’t have the full set, but I found this letter which concluded the back and forth. From context, it seems that my grandfather accused Hoover of not taking the threat of isolationism and the rise of fascism seriously enough, which produced this final rejoinder.
Again, it’s a good reminder that this is far from the first time we’ve felt like we’re on the edge of something enormous and consequential. (Another spoiler alert: my grandfather clearly turned out to be right, given what would happen that very December.)
Tweet of the week
I talk to a lot of people in Silicon Valley about how we’ll keep the most essential, most intellectual parts of humanity alive in a rapidly transforming world. One of the themes that resonates the most with me—and is suffused throughout this post—is a return to ancient wisdom. I find myself craving old books, yellowed old journal articles from the mid 20th century, and histories of ancient times.
Question of the week
I’m building out a “Free Systems Library” of classic books and papers that capture our philosophy. Condorcet, Montesquieu, Madison, Paine, obviously, but more modern stuff too. What titles should I include??
I held my comment on your previous post, because I didn't really want to get into it. But it sounds like you want to get into it :)
I'm quite skeptical of the claim that we'll get to a post-human-work economy any time soon (any time in my life time). But while my predictions of hype not meeting reality have been pretty good so far, I agree the circumstances around this prediction are a little more murky. So, I have less confidence in this prediction than my previous ones.
But the core reason for my skepticism is the same as my past skepticism: Because this prediction is based entirely on speculation that the technology will continue to develop - and break through current ceilings - quickly and without fail. And historically, that's not a great bet, as the Cybernation Revolution mini-panic demonstrates.
One of the reasons for this is that both technology development and adoption take **TIME**. There is a very long tail to both of them.
But the main reason is that the capability to replace human work just isn't there yet. Not even close.
In fact, the proven technology we currently have (LLMs) literally can't get us there. We can't even overcome its inherent weaknesses (hallucinations, prompt injection, and other forms of misalignment) without handicapping the value the technology brings to the types of work it's already most useful for.
Which means we just don't have a technology - either already rolled out, or even in the pipeline - which will be able to replace the work of humans at 90% or more of knowledge work tasks.
Could we get there quickly? Well, anything's possible; but any prediction that we actually *will* is pure speculation. And though I'm not a political scholar, my limited knowledge suggests that making policy based on speculation is almost always a losing bet. In fact, it almost always seems to cause more problems than it solves.
So, to actually get to AGI (which is what's really necessary for the kind of displacement the AI cheerleaders are predicting), we need some new technology. And we don't even know what it is yet, let alone how fast we'll be able to develop and deploy it.
Hence my skepticism about yet another speculation-dependent prediction.
On the other side of the argument, there are some technologies in development (like symbolic AI) that might get us at least part of the way there. Which is why my skepticism is a little less confident this time around. But...speculation is still speculation until the technology is deployed in the real world.
My own personal prediction - which may affect how we view the urgency - is that we won't get to AGI until we create a system that's composed of multiple specialized machine intelligences working together. Because that's how human intelligence works.
The human brain isn't one big blob of intelligence that handles everything, but a system of 30 or more specialized regions, which ultimately work together to create what we consider real intelligence. I suspect we'll need something similar to reach AGI. LLMs will play a role (e.g. one specialized system). Symbolic AI will probably also play a role. But we'll likely need to develop specialized systems for several other types of tasks, and then to figure out how to coordinate them all well enough that they can act as a unit (at least as well as a mostly-sane human).
Thus, I expect AGI to take much more time than the cheerleaders are predicting. If my speculation is even partially correct, then rather than one big displacement, we may see a slow, rolling displacement of human work over a longer period of time, as new specialist technologies fill in the gaps in the AGI brain. That would make this prediction look quite a bit like the Cybernation Revolution. Because it would give us some time to adjust to each new incursion into the domain of human tasks that have previously resisted automation.
None of this means we shouldn't consider what society looks like when we have real AGI; we should. But it does suggest that we might want to question the urgency we place on dealing with the societal disruption of AGI until we at least have some small clue whether we'll even be able to get there - let alone how.