Tuesday, October 10, 2017

From Personal to Prison Gangs: Enforcing Prosocial Behavior

Background: This is part of a short series on high-level principles relevant to political/social issues. The first post set up some ground rules for the general approach.

David Friedman has a fascinating upcoming book on alternative legal systems. One chapter focusses on prison law - not the nominal rules, but the rules enforced by prisoners themselves.

The unofficial legal system of California prisoners is particularly interesting because it underwent a phase change in the 1960’s.

Prior to the 1960’s, prisoners ran on a decentralized code of conduct - various unwritten rules roughly amounting to “mind your own business and don’t cheat anyone”. Prisoners who kept to the code were afforded some respect by their fellow inmates. Prisoners who violated the code were ostracized, making them fair game for the more predatory inmates. There was no formal enforcement; the code was essentially a reputation system.

In the 1960’s, that changed. During the code era, California’s total prison population was only about 5000, with about 1000 inmates in a typical prison. That’s quite a bit more than Dunbar’s number, but still low enough for a reputation system to work through second-order connections. By 1970, California’s prison population had ballooned past 25000; today it is over 170000. The number of prisons also grew, but not nearly as quickly as the population, and today’s prisoners frequently move across prisons anyway. In short, a decentralized reputation system became untenable. There were too many other inmates to keep track of.

As the reputation system collapsed, a new legal institution grew to fill the void: prison gangs. Under the gang system, each inmate is expected to affiliate with a gang (though most are not formal gang members). The gang will explain the rules, often in written form, and enforce them on their own affiliates. When conflict arises between affiliates of different gangs, the gang leaders negotiate settlement, with gang leaders enforcing punishments on their own affiliates. (Gang leaders are strongly motivated to avoid gang-level conflicts.) Rather than needing to track reputation of everyone individually, inmates need only pay attention to gangs at a group level.

Of course, inmates need some way to tell who is affiliated with each gang - thus the rise of racial segregation in prison. During the code era, prisoners tended to associate by race and culture, but there was no overt racial hostility and no hard rules against associating across race. But today’s prison gangs are highly racially segregated, making it easy to recognize the gang affiliation of individual inmates. They claim territory in prisons - showers or ball courts - and enforce their claims, resulting in hard racial segregation.

The change from a small, low-connection prison population to a large, high-connection population was the root cause. That change drove a transition from a decentralized, reputation-based system to prison gangs. This, in turn, involved two further transitions. First, a transition from decentralized, informal unwritten rules to formal written rules with centralized enforcement. Second, a transition from individual to group-level identity, in this case manifesting as racial segregation.

This is hardly unique to prisons. The pattern is universal among human institutions. In small groups, everybody knows everybody. Rules are informal, identity is individual. But as groups grow:
  • Rules become formal, written, and centrally enforced
  • Identity becomes group-based.

Consider companies. I work at a ten-person company. Everyone in the office knows everyone else by name, and everyone has some idea of what everyone else is working on. We have nominal job titles, but everybody works on whatever needs doing. Our performance review process is to occasionally raise the topic in weekly one-on-one meetings.

Go to a thousand or ten thousand person company, and job titles play a much stronger role in who does what. People don’t know everyone, so they identify others by department or role. They understand what a developer or a manager does, rather than understanding what John or Allan does. Identity becomes group-based. At the same time, hierarchy and bureaucracy are formalized.

The key parameter here is number of interactions between each pair of people. In small groups, each pair of people has many interactions, so people get to know each other. In large groups, there are many one-off interactions between strangers. Without past interactions to fall back on, people need other ways to figure out how to interact with each other. One solution is formal rules, which give guidance on interactions with anyone. Another solution is group-based identity - if I know how to interact with lawyers at work in general, then I don’t need to know each individual lawyer.

In this regard, prisons and companies are just microcosms of society in general.

At some point over the past couple hundred years, society underwent a transition similar to that of the California prison system.

In 1800, people were mostly farmers, living in small towns. The local population was within an order of magnitude of Dunbar’s number, and generally small enough to rely on reputation for day-to-day dealings.

Today, that is not the case [citation needed].

Just as in prisons and companies, we should expect this change to drive two kinds of transitions:
  • A transition from informal, decentralized rules to formal, written, centrally-enforced rules.
  • A transition from individual to group-level identity.
This can explain an awful lot of the ways in which society has changed over the past couple hundred years, as well as how specific social institutions evolve over time.

To take just a few examples…
  • Regulation. As people have more one-off interactions, reputation becomes less tenable, and we should expect formal regulation to grow. Conversely, regulations are routinely ignored among people who know each other.
  • Litigation. Again, with more one-off interactions, we should expect people to rely more on formal litigation and less on informal settlement. Conversely, people who interact frequently rarely sue each other - and when they do, it’s expected to mess up the relationship.
  • Professional licensing. Without reputation, people need some way to signal that they are safe to hire. We should expect licensing to increase as pairwise interactions decrease.
  • Credentialism. This is just a generalization of licensing. As reputation fails, we should expect people to rely more heavily on formal credentials - “you are your degree” and so forth.
  • Stereotyping. Without past interactions with a particular person, we should expect people to generalize based on superficially “similar” people. This could be anything from the usual culprits (race, ethnicity, age) to job roles (actuaries, lawyers) to consumption signals (iphone, converse, fancy suit).
  • Tribalism. From nationalism to sports fans to identity politics, an increasing prevalence of group-level identity means an increasing prevalence of tribal behavior. Based on this, I predict that social media outlets with more one-off or low-count interactions are characterized by more extreme tribalism.
  • Standards for impersonal interactions. “Professionalism” at work is a good example.

I’ve focussed mostly on negative examples here, but it’s not all bad - even some of these examples have upsides. When California’s prisons moved from an informal code to prison gangs, the homicide rate dropped like a rock; the gangs hate prison lockdowns, so they go to great lengths to prevent homicides. Of course, gangs have lots of downsides too. The point which generalizes is this: bodies with centralized power have their own incentives, and outcomes will be “good” to exactly the extent that the incentives of the centralized power align with everybody else’ incentives and desires.

Consider credentialism, for example. It’s not all bad - to the extent that we now hire based on degree rather than nepotism, it’s probably a step up. But on the other hand, colleges themselves have less than ideal incentives. Even setting aside colleges’ incentives, the whole credential system shoehorns people into one-size-fits-all solutions; a brilliant patent clerk would have a much more difficult time making a name in physics today than a hundred years ago.

Of course, all of these examples share one critical positive feature: they scale. That’s the whole reason things changed in the first place - we needed systems which could scale up beyond personal relationships and reputation.

This brings us to the takeaway: what should you do if you want to change these things? Perhaps you want a society with less credentialism, regulation, stereotyping, tribalism, etc. Maybe you like some of these things but not others. Regardless, surely there’s something somewhere on that list you’re less than happy about.

The first takeaway is that these are not primarily political issues. The changes were driven by technology and economics, which created a broader social graph with fewer repeated interactions. Political action is unlikely to reverse any of these changes; the equilibrium has shifted, and any policy change would be fighting gravity. Even if employers were outlawed from making hiring decisions based on college degree, they’d find some work-around which amounted to the same thing. Even if the entire federal register disappeared overnight, de-facto industry regulatory bodies would pop up. And so forth.

So if we want to e.g. reduce regulation, we should first focus on the underlying socioeconomic problem: fewer interactions. A world of Amazon and Walmart, where every consumer faces decisions between a million different products, is inevitably a world where consumers do not know producers very well. There’s just too many products and companies to keep track of the reputation of each. To reduce regulation, first focus on solving that problem, scalably. Think amazon reviews - it’s an imperfect system, but it’s far more flexible and efficient than formal regulation, and it scales.

Now for the real problem: online reviews are literally the only example I could come up with where technology offers a way to scale-up reputation-based systems, and maybe someday roll back centralized control structures or group identities. How can we solve these sorts of problems more generally? Please let me know if you have ideas.

Wednesday, October 4, 2017

IQ Scores: What are they good for?

I recently encountered two articles arguing against making too much of one’s own IQ score. Both of them mostly boil down to “IQ tests are a REALLY noisy measure of g, and on an individual level the noise is going to mask a lot of the signal”. This is totally 100% correct, and is probably responsible for most of the hand-wringing those two authors encounter.

But let’s cut past the noise: suppose you’ve taken an IQ test, and the SATs, and maybe throw in some other measures too. That’s all Bayesian evidence for your underlying g, so you can put them all together to get a hopefully-less-noisy estimate. The result tells you something about your intelligence relative to the rest of the population. What should you do with this information? In particular, if the number is lower than you’d like, what’s the right response?

There are multiple good answers to that question, most notably relative advantage - pick up an intro microeconomics text if you want to know more about that one. But in keeping with my usual policy of “don’t write things that somebody else already wrote”, I’ll focus on an answer which I haven’t seen used in this context before: strategic variance.

The Underdog Strategy
Suppose you’re in an oversimplified one-on-one basketball game with three rounds. Strictly speaking, your opponent is a better player than you: they average 24 points per round, while you average 22. But you have a trick up your sleeve: your opponent is a one-trick pony, while you have multiple play styles. One play style is conservative and consistent: in one round, you’ll score 22 points and your opponent will score 24, consistently. Your other play style is more aggressive, with more variance: in one round, your opponent will score 24, and you’ll score either 14 or 30, with a 50% chance for each.

For both play styles, your opponent averages 2 more points per round than you do. But you can still win more often than not. Here’s the strategy:
  • Round 1: play aggressive. You end up ahead by 6 points (50% chance) or behind by 10 (50%).
  • Round 2 & 3: If you’re ahead, play conservative; if you’re behind, play aggressive. If you wound up ahead in round 1, then you’ll play conservative for the next two rounds and win by 2. If you wound up behind in round 1, then you’ll play aggressive for 2 rounds and have a 25% chance of a comeback.
  • Put that all together, and your chance of winning is 62.5%.
Despite your opponent scoring more points on average regardless of strategy, you can still win more often than not!

The example is somewhat artificial, but the idea generalizes:
  • When you’re “ahead”, play conservative - avoid risk, minimize variance.
  • When you’re “behind”, play aggressive - take risk, maximize variance.
The principle generalizes easily to practically any game with some way of keeping score - virtually all sports, board games, card games, and so forth. (On a side note, it also applies to bacterial chemotaxis.) Let’s apply it to real life.

The Underdog Strategy in Real Life
Suppose my goal in life is to solve some major open problem in math/science - we’ll use the P-NP problem as an example. Then my own intelligence - g, IQ, whatever measure we’re using - is a very useful indicator of how far “ahead” or “behind” I’m starting.

Consider Terence Tao - presumably he’d be starting way “ahead” by this criteria. If he spent a year or two focussed entirely on P-NP, he’d probably be one of the top 5 smartest people ever to invest that much effort in the problem. There’s a reasonable chance that he could solve it by brute force of intellect - by being smarter than anyone else who’d tried. Maybe P-NP is straightforward for anyone who’s up-to-date with known circuit complexity lower bounds and has sufficiently high g, and the problem is just waiting for someone smart enough to come along and put the pieces together. There’s a realistic chance that Terence Tao could be the first person to come along who’s smart enough.

When you look at it like that, if Terence Tao decides to seriously tackle P-NP, then just spending a year pushing current approaches would be a very reasonable starting point for him. He’s starting out “ahead”, so a conservative low-risk strategy makes sense as a first thing to try.

But what if I want to solve P-NP?

I’m smart, but I wouldn’t be in the top 100 or probably even the top 1000 smartest people who’ve tackled P-NP. I will never be able to solve P-NP by brute force of intellect, by taking the standard approaches and throwing my own intelligence at them. I am not that smart.

I’d be starting “behind” in this game, my chance of winning is low a priori, so to maximize my chances I need to add variance. In this context, that means trying weird approaches. Investing effort in tools which may or may not be useful but are definitely different from what everyone else is doing, and applying those tools to the problem. On average, any particular random thing is less likely to be the key to P-NP than lower-bounding circuit complexity. But I, personally, am more likely to solve P-NP by applying some weird technique from probability or physics or economics or even biology, than by pushing already-popular strategies. (That doesn’t mean I shouldn’t get up to date with current research, just that I shouldn’t invest much effort pushing past the cutting edge in that particular direction.)

One last comment to wrap it up: strategic variance only applies to binary problems, where you either clear the bar or you don’t. If your goal in the oversimplified one-on-one basketball game is to improve your average score, then variance won’t help. If your goal in life is to maximize your expected earnings, then again, variance in and of itself will not help. On the other hand, if your goal in life is to become a billionaire, then strategic variance - i.e. risk taking - will help.

Generalizing further: as humans, we tend to invest less effort than we should in high-level life strategy. Things like estimating your own g/IQ are useful mainly to inform that strategy. The more strategic you are, the more value you can extract from that information. Strategic variance is just one class of strategies. Relative advantage is the main underlying strategy - e.g. in the oversimplified basketball game, you win by exploiting your relative advantage of being able to adjust your own variance. Study relative advantage, and pay attention to which tradeoffs are cheaper for you than for others.

Tuesday, September 26, 2017

Knowledge, Power and Politics

Benjamin Jesty
Our story begins in 1774, a year before the American Revolution, in Dorset county, England, with a dairy farmer named Benjamin Jesty.

A wave of smallpox was running across England that year. Jesty himself was in no danger - he had previously contracted cowpox. The cowpox was contracted by milking infected cows, and was well known among dairy farmers to convey immunity against smallpox.

Unfortunately, neither Jesty’s wife nor his two children had any such advantage. When smallpox began to pop up in Dorset, Jesty decided to take drastic action. He took his family to a nearby farm with a cowpox-infected cow, scratched their arms, and wiped pus from the infected cow on the scratches. Over the next few days, their arms grew somewhat inflamed and they suffered the mild symptoms of cowpox - but it quickly passed. As the wave of smallpox passed through the town, none of the three were infected. Throughout the rest of their lives, through multiple waves of smallpox, they were immune.

The same technique would be popularized twenty years later by Edward Jenner, marking the first vaccine and the beginning of modern medicine.

Power vs Knowledge
The same wave of smallpox which ran across England in 1774 also made its way across Europe. In May, it reached Louis XV, King of France. Despite the wealth of a major government and the talents of Europe’s most respected doctors, Louis XV died of smallpox on May 10, 1774.

Academics in the study of politics typically define “power” as the ability to influence or control the actions of other people - a slight generalization of “authority”. I want to discuss a somewhat more general notion of power - a kind of power which Benjamin Jesty possessed, and Louis XV lacked. I want to discuss power as the ability to influence or control the world around you, in whatever manner is relevant to you.

Louis XV had no shortage of authority. He had money, and his word was law. Louis XV could give orders, and they would be followed. But no orders he could give would conjure up the cure for smallpox. Jesty, on the other hand, had the means of a farmer and no authority to speak of. But Jesty had a power which Louis XV lacked - Jesty knew how to prevent smallpox. Jesty did not need money or authority; he had knowledge.

They say that knowledge is power, but the relationship is more fundamental than that. Knowledge is… the first power. The base power. The foundation upon which any other form of power must sit.

Without knowledge, any other kind of power is worthless. Without knowledge, no amount of money or authority could stop smallpox. With knowledge, it was trivial. For Jesty, immunizing his family against smallpox was a day’s errand. It was a thing-he-could-do in much the same way as walking into town to buy some salt.

What if we extend this same idea to politics?

Politics Today: Total Spending
I propose that, in the sense that I’ve been using the word, our political figureheads possess little-to-no power.

Let’s take a relatively simple problem: suppose you’re the Republican party, and you want to dramatically scale back government spending as a share of GDP. The GOP controls most state governments, both houses of congress, the presidency, and the supreme court. They’re the Louis XV of this story; they have the authority to do whatever they want. And paring down the scale of federal government has been a major plank in the conservative platform for decades. And yet… there doesn’t seem to be much paring underway.

Some people take the cynical view, that Republican figureheads don’t actually care about cutting spending. But a theory like that could explain any behavior - and besides, honest stupidity always has a higher prior than malevolence. Here’s my theory: Republicans lack the power to prevent government spending growth in much the same way that Louis XV lacked the power to prevent smallpox.

To see why, we’ll need to briefly dive into the numbers on government spending.

Let’s start with the main graph:

As the graph shows, pretty much all the growth in government spending over the past few decades has come from entitlements - primarily social security, medicare and medicaid. Everything else has been squeezed in an attempt to keep the budget in check as entitlement spending soars.

Within entitlements, growth is driven mainly by social security and medicare. The math is pretty straightforward: old people are expensive, the population is getting older very quickly, carry the two, spending goes up.

There is no path to stop the growth of government spending without cutting off old people. But this is counterintuitive to a lot of people on the right - the government didn’t used to spend so much, so why the hell must it spend so much now? Oh, we’re spending on healthcare? The problem must be that healthcare has become less efficient! We need to let the market do its job!

At some point, I will write a blog post entirely digging into data on healthcare costs. Suffice to say, efficiency is not the limiting factor. (For those on the left: no, price gouging by insurance/pharma/other companies is not the main driver either.) The main driver is people getting older, and people consuming lots of healthcare (e.g. high nursing home usage).

But if the Republican party is the Louis XV of this scenario, who’s the Benjamin Jesty?

Let’s view reduction of government spending as a knowledge problem. Imagine you want to make reduction of government spending a thing-you-could-do, in the same way that going to the store to buy some milk is a thing-you-could-do. What knowledge would make that possible?

It’s obvious once you see it: antiaging. If and when antiaging technology hits the market, medicare spending will evaporate. The expensive health problems of the elderly will disappear, and with them the growth of government spending.

Generalizing the Concept: A Different Approach to Politics
When smallpox died, it didn’t fight back.

If someone were to introduce cheap, credible antiaging technology to the market today, government spending would drop like insurance stocks during hurricane season. It would happen entirely automatically: demand for healthcare would fall, causing medicare claims to fall, causing spending to fall. Congress would be left arguing over what to do with their giant budget surplus.

This wouldn’t take some great social movement or political campaign. It wouldn’t require arguing or fighting or convincing or converting. It would just take knowledge, an idea and a little elbow grease. It could be done by a small team, or possibly even one person. Heck, it would probably even be profitable.

Technology and economics lead inexorably to an equilibrium. Try to change the equilibrium by politics, and you’ll be fighting gravity all the way. But if you change the technology, then gravity is on your side. Political movements take millions of people, but a technological change takes just a handful of researchers and/or entrepreneurs. The rest is gravity.

But there’s a trade-off: effecting social change through technology requires knowledge. It requires an in-depth understanding of the underlying causes of the equilibrium we wish to change. Usually, that means economics and game theory, plus a bunch of domain-specific research, plus whatever skills are needed to implement the new technology.

Political movements make it easy to try, but rare to succeed. A technological approach requires far more work and knowledge to even attempt, but offers a much shorter path to success.

Tuesday, August 8, 2017

Reverse Debate

One observation about Google’s diversity memo firestorm: neither side seems to be changing their minds. Ideally, at least one side would be able to learn something. Maybe have a public debate?

There are two universal problems with public debate. First, they’re more a measure of debate skills than statistical strength of evidence. Second, confirmation bias means they often just increase polarization. (Third, in today’s environment, anything said by either side will likely be twisted beyond recognition by opposition reporters.) The net effect is that public debate rarely changes anyone’s mind about anything.

This creates an interesting challenge: design a public debate format which actually makes people likely to change their minds. Ideally, people would be more likely to converge on an answer, and sides would become less polarized rather than more.

(I’m about to present my solution, so if you want to try this challenge yourself, pause reading now.)

My solution:
  • Side A must present the arguments of side B.
  • Side B judges whether their arguments have been presented accurately.
  • They then switch, back and forth, for as long as the debate allows.
If you’re familiar with the Ideological Turing Test, this is the same concept projected into a debate setting.

The format can vary somewhat depending on the scoring, but the simple pass/repeat rule is my favorite:
  • Side A presents the arguments of side B.
  • If side B is not satisfied with the presentation, then side A must try again.
  • This repeats until side A succeeds, at which point the two teams switch roles.
Of course, repetition would grow dull in a public debate setting. To avoid actually needing to repeat a lot, both sides would practice together exhaustively before the public “debate” - in all likelihood they’d converge on a common view before the spectacle even began, and the public debate would consist of each team trying to convince those on their own side!

One important question remains: how would anyone be motivated to participate in such a debate in the first place? The sort of people attracted to an honest, collaborative effort of this nature are unlikely to be those most in need of it.

In the context of the recent Google firestorm, motivation would have been simple: people’s jobs were on the line. The publisher of the memo had his job on the line already. On the other side, plenty of people threatened to quit if he wasn’t fired. So, invite one of those people to challenge the publisher in a metaphorical debate to the death! Once either participant finished the debate, successfully representing all of their opponent’s arguments, they’d be asked if they still thought their opponent should be fired. Both participants could potentially lose their jobs, if they both passed the debate and decided to fire each other.

Of course, we wouldn’t really hope/expect that to happen. Really, we’d hope that at least one side saw the light, and then turned around to preach to their own tribe. The point, after all, is to actually change people’s minds.

Thursday, July 20, 2017

Detecting Intelligence with an Unknown Objective

This is the second post in a series on theory for adaptive systems. The previous post argued that the lack of good adaptive systems theory is the main bottleneck to scientific progress today. The main goal for the next few posts is to lay out questions and problems, and to suggest possible approaches toward quantitative solutions.

Today’s question: How can we recognize adaptive systems in the wild?

To be more concrete: suppose I run a ridiculously huge Game of Life simulation with random initial conditions. What function can I run on the output in order to detect adaptive system behavior within the simulation? Specifically, we’re looking for subsystems of the Game of Life which:
  • Learn from their environment
  • Use what they learn to optimize for some objective
I see two major difficulties to this problem:
  1. We don’t know the system’s objective.
  2. We don’t know what defines the “system”.
I’ll focus on the first part for now; defining the “system” will be a running theme which I will revisit toward the end of these posts.

Example 1: Street Map
Imagine cars driving around on a street map which looks roughly like this:
Suppose two types of cars drive around this map. The first type wanders about, picking a random direction at each intersection until it reaches its destination. The second type knows what the map looks like, and takes the shortest path from its starting point to its destination. Looking at their paths as they drive, how could we tell the two apart? In particular, how could we tell the two apart without knowing the destination?

It would be tedious but straightforward to build an elaborate statistical test for this particular problem, but it wouldn’t generalize. Instead, I’ll point out an heuristic: the intelligent cars, the cars which take the shortest route, will almost always start by driving toward the center, and almost always finish by driving away from the center.

Why? Pick two points at random on the map. Look at the shortest path between them. A majority of the time, it will go through the center point. Even when it doesn’t, it almost always goes first toward the center, then away - it never gets closer to the center then further then closer again.

(For the mathematically inclined: you can prove this by looking at the map as a tree, picking a root, and viewing “distance to center” as the depth within the tree.)

Example 2: Shortest Paths
In the street map example, we can detect “intelligent” behavior by looking for cars which first go towards the center of the map. This behavior is statistical evidence that the car is following a relatively short path to some destination.

Can we generalize this? “Intelligent” cars only start by going toward the center because that’s the shortest path. Even on a more general map, we could look for statistical patterns among shortest paths. On a real-world road map, “shortest paths” over significant distances usually hop onto a highway for most of the drive. Even locally, there are more central and less central roads. Without diving into any statistics, it seems like we could take a typical road map and develop a statistical test to tell whether a car is following a short path between two points, without needing to know the car’s destination.

But what makes the short path “intelligent” at all? Why do we intuitively associate short paths with intelligent behavior, as opposed to wandering randomly around the map?

Example 3: Resource Acquisition
Let’s look at the problem from a different angle. One characteristic behavior of living creatures, from animals to bacteria, is a tendency to acquire resources.

In biology, the main types of resources acquired are energy and certain standard biochemicals. Each of these resources is stored - e.g. energy is stored as starch, fat, ATP, an electric potential difference, etc.

Why would adaptive systems in general want to acquire and store resources? Because it gives the system more options. A human who accumulates lots of currency has more options available than a human without any currency. A bacteria with a store of energy has more options than a bacteria without. Ultimately, those resources could be used in a variety of different ways in order to achieve the system’s objective.

Whether it’s a human taking a vacation or buying a car, a bacteria reproducing or growing, a pool of resources offer options suitable to many different situations. Intuitively, we expect adaptive systems to accumulate resources, because those resources will give the system many more options in the future.

Example 4: Time as a Resource
One universal resource is time. In this view, saving time is a special case of accumulating resources: time saved can be spent in a wide variety of ways, offering more options in the future.

This ties back to the shortest path example. We expect “intelligent” systems to take short paths in order to save time. They save time because time is a universal resource - time saved can almost always be “spent” on something else useful to the system’s goal.

In the street map example, we run into a more unusual resource: “centrality” in the road map. (Mathematically: height in the tree.) A more central location is closer to most points. By moving toward the center of the map, a car accumulates centrality. It can then cash in that centrality for time savings, converting one resource (centrality) into another (time).

A Little Formalization
We now have a handful of examples of intuitively “intelligent” behavior - short paths, energy and currency accumulation, saving time. These examples all amount to the same thing: accumulating some useful resource. Can we formalize this intuition somewhat? Can we generalize it further?

In AI theory, there’s a duality between constraint relaxation and heuristics. A constraint relaxation would be something like “what could the system do if it had more of resource X?”. The amount of X is constrained, and we “relax” that constraint to see if more X would be useful. That constraint relaxation has a corresponding heuristic: “accumulate X”. That heuristic is useful exactly when relaxing the constraint on X is useful.

All of our resource accumulation examples can be viewed as heuristics of that same form: “accumulate X”. Each of them has a corresponding constraint relaxation: “what could the system do if it had more of resource X?”.

In principle, any formal heuristic can be viewed as a resource. But the examples we addressed above seem more specific than heuristics in general. They share some common features beyond general formal heuristics:
  • Each resource is highly fungible. Energy, currency and time are easy to trade for a very wide range of other things, and other things are easy to trade back into energy, currency, and/or time.
  • Each resource can be stored efficiently. Cream is not a good resource for humans to accumulate; it spoils quickly.
  • Each resource is scarce. Bacteria need water, and they could accumulate water, but they’re usually surrounded by unlimited amounts of water anyway. No point storing it up.
In some ways, these are just criteria for what makes a good formal heuristic. In order for a formal heuristic to accelerate planning significantly, it needs to be scarce and storable and fungible. In order for something to be a good resource to accumulate, it should be a useful heuristic for planning problems.

Problems. Plural.

Remember where we started this post: we want to detect adaptive systems without necessarily knowing the systems’ objectives in advance. All the resources listed above make good heuristics not just for one problem, but for a wide variety of different problems. Why? What do they have in common, beyond generic formal heuristics?

The Ultimate Resource
Let’s go back to where formal heuristics come from: constraint relaxation. Intuitively, by accumulating resources, by following an heuristic, by relaxing a constraint, a system gives itself more options. That’s why it’s useful to have more energy, more currency, more time: the system can choose from among a wider variety of possible actions. The action space is larger.

This is the ultimate resource: accessible action space. The more possible actions available to an adaptive system, the better. A good resource to accumulate is, in general, one which dramatically expands the accessible action space. Fungibility, storability, and scarcity are all key criteria for something to significantly expand the action space.

Time to go back to the opening question: suppose I run a ridiculously huge Game of Life simulation with random initial conditions. What function can I run on the output in order to detect adaptive system behavior within the simulation?

This post has only addressed one tiny piece of that problem: an unknown objective. Later posts will focus more on information processing, learning, and defining the system. But already, we have a starting point.

We expect optimizing systems to accumulate resources. These resources will be fungible, storable, and scarce in the environment. The system will accumulate these resources in order to expand its action space.

So what might we look for in the Game of Life? Very different kinds of resources could be useful, depending on the scale and nature of the system. But we would certainly look for statistical anomalies - resources are scarce. Those anomalies should be persistent - resources can be stored. Finally, the extent of those anomalies should grow and shrink over time - resources are acquired and spent.

It’s not much, but it’s a starting point. Hopefully it suggests a flavor for what sort of things could be involved in research on the subject. Or better yet - hopefully it gives you ideas for better approaches to tackle the problem.

Next post will talk about how to extract an adaptive system’s internal probabilistic model.