Gaming the Future: Technologies for Intelligent Voluntary Cooperation

2. OVERVIEW | What to Expect From This Game

Feb 14, 2022

Who are you?

You are the result of evolutionary games:

“Not one of your pertinent ancestors was squashed, devoured, drowned, starved, stranded, stuck fast, untimely wounded, or otherwise deflected from its life's quest of delivering a tiny charge of genetic material to the right partner at the right moment in order to perpetuate the only possible sequence of hereditary combinations that could result -- eventually, astoundingly, and all too briefly -- in you.”1

What awaits after a few more iterations of this game? This depends, in part, on your decisions.

You could make nihilistic outcomes more likely:

"In some remote corner of the universe, poured out and glittering in innumerable solar systems, there once was a star on which clever animals invented knowledge. That was the highest and most mendacious minute of "world history" — yet only a minute. After nature had drawn a few breaths the star grew cold, and the clever animals had to die. There have been eternities when it did not exist; and when it is done for again, nothing will have happened." 2

Or you could make fantastic futures more likely:

“Whether anyone else is out there or not, we are on our way. [...] Expansion will proceed, if we survive, because we are part of a living system and life tends to spread. Pioneers will move outward into worlds without end. Others will remain behind, building settled cultures throughout the oases of space [...] Where goals change and complexity rules, limits need not bind us. [...] New technologies will nurture new arts, and new arts will bring new standards. The world of brute matter offers room for great but limited growth. The world of mind and pattern, though, holds room for endless evolution and change. The possible seems room enough.”3

This book starts with high hopes for the future and the following conundrum: You’re in this game of civilization with players who have a diversity of values. Some values are complementary, some opposing, and some include the destruction of the playing field. As the game continues, rapid increase in technologies give everyone more power. If you hate civilization, your next move is easy. But if you love this game, and want to see it evolve over many iterations, you have a more complicated set of choices to make.

Civilization is an inherited game shaped by those before you. If you’re happy your ancestors didn’t lock you into a future, should you leave everything up to future players? If only that was an option. Any move you make will affect the choices that future players have available to them. You must play your turn, and therefore, you must choose among games to pass on to future generations. By choosing strategies of intelligent voluntary cooperation, you can set the game of civilization upon a path of rapidly increasing intelligence serving a diversity of goals.

Let’s look at a high level overview of what we later cover in depth.

Where To Start

We begin by showing that goals differ across players of civilization. Each of us has subjective best guesses about the world. We disagree about what future state to move to and about how to get there. Some want to grow a pristine garden; others want to explore new worlds; others want to drive the pursuit of knowledge. These differences will become more apparent as players evolve. Some future players may have artificial minds. What they want may be very alien to human players.

In light of this ignorance, relying on voluntary interactions across players is a good heuristic to serve different goals. A voluntary action only depends on a player’s internal logic, leaving them “free” to engage or not engage in interactions. We tend to only consent to moves from which we expect benefit rather than harm. Such moves, which make at least one player better off without making anyone worse off, are called Pareto-preferred. As a rule of thumb, voluntary interactions gradually move civilization into Pareto-preferred directions, i.e., directions that tend to be better for everyone by their own standards.

We show that this principle has a good historic track record. Human civilization is growing less violent over time and many things players care about, from health to education, are improving. Voluntarism enables cooperation but it does not, by itself, bring it about. It took thousands of years of institutional evolution to create complex systems of prices, property, and institutions that help players cooperate better. Instead of arguing about dividing the pie, they get better at growing it.

As the game continues, it becomes increasingly intelligent. One constraint on civilization’s intelligence is that each of its players plans mostly in ignorance of others’ plans. Institutions evolve to better coordinate across them by providing signals about what would be beneficial to do. Some players are humans, some are institutions themselves, and an increasing number will be software entities. Composed together by improving networks of voluntary cooperation, they increase the adaptive intelligence of civilization. In the pursuit of their highest values, they unlock new levels across the board.

What To Seek More Of

Approaching such futures requires progress in technologies of cooperation. Much of players' ability to benefit from each other is still limited. Perhaps the biggest problems arise when there is a state of the world that we would all prefer to jump to, but lack the coordination to do so. A look at how institutions evolved to deal with these factors shows how to further diminish them. With contracts, players can make binding commitments to particular future actions and cooperate for mutual benefit.

Countless cooperative constellations are possible across the 7 billion players of civilization. We’ve unlocked many of them but we can do even better. The internet secures the right to information, cryptocurrency grants monetary sovereignty, blockchain makes institutions incorruptible, and smart contracts democratize the right of contract. Cryptography-based commerce, cryptocommerce, can create a base for amplifying and democratizing cooperation.

Some coordination problems will remain tricky. Drafting mechanisms for 7 billion people to find each other, speed up the bargaining process, and enforce an arrangement is extraordinarily difficult. But it could unlock previously inconceivable layers of civilization. We won’t jump to this world tomorrow, but we can gradually grow into it.

What To Defend Against

As civilization evolves, players will unlock new capabilities. Biotechnology will deliver healthier lives, nanotechnology will provide wasteless manufacturing, and AI will accelerate unprecedented discoveries across the board. But the same technologies could be leveraged to cause unprecedented destruction. The economics of fighting wars could lead to pervasive robotic enforcement via lethal autonomous weapons and surveillance.

When guarding against the downsides of powerful technologies, players must resist the temptations of solutions that create more problems than they solve. Statist solutions that centralize the capacity for violence without checks and balances are such a danger. Checks on U.S. power, the world’s leading military player, are decreasing, while checks on its rising rival, China, are near absent. Both have access to weapons that could nuke the playing field of civilization.

Decentralized defense systems that allow for multipolar monitoring and cross-checking can make their own dangers more visible. Cryptography can make them more privacy-preserving. Such systems are hard to envision but will emerge from today’s game. So called “black boxes” already provide indelible records for internal surveillance of automated systems. Smartphone cameras already democratize surveillance, making human enforcement more accountable.

Any desirable future will rely on computer security at every level, from hardware, to operating systems, to software, all the way to the user interface. Computer security is essential to de-risking cooperation that increasingly takes place virtually. It is also essential for preventing automated weaponizable technologies that make mass-killing trivially easy. Fortunately, there are promising candidates to address the problem; instead of adding security to a system last, they prevent insecurities from the very start. The seL4 microkernel is an excellent example of a provably secure system.

To make computer security adoptable, we need a mixture of research and entrepreneurship to test it in the real world. The cryptocommerce ecosystem already serves as a test arena, where rogue actors compete to steal cryptocurrencies. It is hostile enough that insecure software dies quickly so that the ecosystem is populated by the survivors. This gives us a better chance at building a full secure software stack from the foundations to the user. Such play-tested systems can grow within, co-exist with, and eventually outcompete current insecurable software infrastructure.

What To Hope For

As civilization expands, it could act like a seed crystal dropped into a supersaturated solution, expanding its ordering principles in all directions. There is no law saying that the result will be the continually growing spontaneous order intelligence of civilization. It could be the outcome of a winner-takes-all arms race to expand first.

If human players upgrade their tools to cooperate with artificial players newly arriving on the scene, we have much to look forward to. But even if all goes well, the universe will eventually no longer be able to sustain computation of any sort, especially not the complex computation required for intelligence. Even if we create a game which makes everything up to now seem like an insignificant speck, it is all temporary. Nevertheless, insofar as the in-between is shaped by what current players value, it’s on us to shape what happens between now and then.

The game is on and the stakes are high. Let’s play.