This is a response to an article by philosopher Michael Huemer on AI risk, and why he believes the most dire concerns are unwarranted. I stand by my endorsement of Huemer's work in other fields, but I believe he's wrong on this subject. I hope to show why.

In the comments of a facebook post made on 18 february 2023, about doomsday cults, Huemer answers a question about AI risk by linking to his 2020 post Existential Risks: AI. Huemer's reference to this article suggests that at the time of writing he believes the arguments presented therein are sufficient to dispel concerns about the possibility of catastrophe for human kind caused by (autonomous) artificial intelligence.

The motivation of an AI

Huemer writes:

the way that most people imagine AI posing a threat -- like in science fiction stories that have human-robot wars -- is anthropomorphic and not realistic. People imagine robots that are motivated like humans -- e.g., megalomaniacal robots that want to take over the world. Or they imagine robots being afraid of humans and attacking out of fear.

Advanced AI won't have human-like motivations at all -- unless we for some reason program it to be so.

I don't know why we would program an AI to act like a megalomaniac, so I don't think that will happen.

Computers will follow the algorithm that we program into them.

The important mistake here is assuming that creating an AI is a similar process to traditional software engineering; Where humans write explicit instructions telling the program how to behave any point during runtime. If the program ends up behaving unexpectedly it's because a human has made a mistake that can be fixed through debugging.

But software based on Large Language Models (ChatGPT, Bing), and presumably more powerful things that might get called AIs, are chiefly the product of machine learning with neural networks, not traditional software development.

A neural network consists of layers of interconnected nodes, neurons, that are capable of processing and transmitting information. When a neural network is trained, it's fed a lot of data and its outputs are scored against some criteria. During this process the weights of the connections between neurons are programatically altered by a simple algorithm which iteratively nudges the network towards giving higher scoring outputs–being better at what we intend it to do.

gradient descent is an optimization algorithm that searches over the space of neural net parameters to find a set that performs well on some objective.

AN #58 Mesa optimization: what it is, and why we should care

Unlike a traditional computer program that's amenable to debugging, a trained neural network is effectively a black box; an inscrutible set of nodes with connections of different weights–float numbers–between them. We can look at an inputs and compare it against the output, but as humans we can't understand what the network is doing by looking at the topology and the numbers.

When the goal of training is small and well defined, like having a neural network reliably recognise hand-written numbers, this is fine. We can easily tell whether or not the network is working as we hope, and there's no possibility of disaster. But in the case of a future general intelligence, which can take autonomous action in the world, where we might try to train to the directive "don't covertly have goals that most humans would object to", that's much harder.

This isn't only a theoretical concern. Alignment problems have been demonstrated in simple video game models. For instance, subtle statistical differences between the training environment and the 'deployment' environment revealed that a model had adopted a misaligned goal, that worked well in training but gave unwanted (at least to its makers) results afterwards.

Update. I misunderstood Huemer in this section. Where he uses the verb 'program' he intends it more broadly that I assumed, to include the process of training a neural network. But I'll leave the above section unedited in case it's useful for others. Huemer replies:

Re: "The motivation of an AI": What I had in mind was that the programmers wouldn't train the AI for megalomaniacal behavior; anything like that in the training period would be labelled "bad". We know why humans (esp. men) have that behavior, but nothing like that explanation would apply to AI.

This doesn't rule out that some crazy behavior might emerge in novel circumstances. It's just unlikely to be crazy in the way crazy humans are crazy. It's more likely to be something that makes no sense to us. Or so I would guess.

Mesa optimisers

There's no guarantee that when you use gradient descent to optimise for a particular thing, that the general intelligence that you eventually get also wants that same thing.

I am a mesa-optimizer relative to evolution. Evolution, in the process of optimizing my fitness, created a second optimizer - my brain - which is optimizing for things like food and sex.

Deceptively Aligned Mesa-Optimizers: It's Not Funny If I Have To Explain It

The material on (deceptively aligned) mesa-optimisers in AI is particularly interesting here. I won't repeat more of it here but I recommend taking a look.

As Eliezer Yudkowsky puts it nobody has the technical knowledge to control, in detail, what a powerful AGI ends up caring about. In a situation that couldn't arise in traditional softwared engineering, our ability to create (proto) AI's outstrips our ability to reliably instill in them with human-like ideas about acceptable behaviour, and the prospects for changing that state of affairs don't look good.

you run myopic gradient descent to create a strawberry picker. It creates a mesa-optimizer with some kind of proxy goal which corresponds very well to strawberry picking in the training optimization, like flinging red things at lights [the light reflecting off the bottom of the metal bucket]

"While speculating about the far future, it realizes that failing to pick strawberries correctly now will thwart its goal of throwing red things at light sources later. It picks strawberries correctly in the training distribution, and then, when training is over and nobody is watching, throws strawberries at streetlights.

(Then it realizes it could throw lots more red things at light sources if it was more powerful, achieves superintelligence somehow, and converts the mass of the Earth into red things it can throw at the sun. The end.)"

Deceptively Aligned Mesa-Optimizers: It's Not Funny If I Have To Explain It

Megalomania and instrumental convergeance

Huemer writes:

The desire to take over the world is a peculiarly human obsession. Even more specifically, it is a peculiarly male human obsession. Pretty much 100% of people who have tried to take over the world have been men. The reason for this lies in evolutionary psychology (social power led to greater mating opportunities for males, in our evolutionary past). AI won't be subject to these evolutionary, biological imperatives.

Instrumental convergence is a concept in artificial intelligence that suggests that as an AI becomes more intelligent and capable, its goals and motivations are likely to include a certain set of objectives, regardless of its primary goal.

For instance, an AI might notice that in order to optimally achieve it's goal it's important not to be interrupted by being turned off, so it would be good if it could eliminate the risk of that from happening. And it might notice that the more of the universes resources it can control, the more effective it will be at pursuing it's primary value. Both of these things will tend to create megalomania.

I hope I've satisfactorily indicated how it's reasonable to consider future AGI a real threat to humanity in itself, even before it gets into the hands of people who are evil or negligent.

The burden of proof is usually, and I argue wrongly, framed as a responsibility that falls on a participant in a discussion because of something they said. The burden is imagined to be a responsibility to provide argument in favour of a claim that they made or implied.

Here’s what yourlogicalfallacy.com has to say.

The burden of proof lies with someone who is making a claim, and is not upon anyone else to disprove. The inability, or disinclination, to disprove a claim does not render that claim valid, nor give it any credence whatsoever.

But I hope to show this isn't correct; the burden of proof never comes from something a person says, and there isn't a 'The burden of proof', singular.

What is it then?

I'm claiming that:

A person has a burden of proof when–and only when–they want to persuade another.

Which implies that both parties in a discussion can have a burden of proof at the same time; and that a burden of proof can lie with a person who has not made a positive claim, but wants to persuade someone who has, that they're mistaken.

I take on the burden of persuading you I'm right about the burden of proof

If we use burden of proof to mean what I say it means, it's not a very useful phrase. It doesn't make sense to haughtily tell someone the burden of proof is on them. You already know if they want to persuade you about something–if they do they'll already be trying to, already shouldering the burden. And if they're not trying to persuade you then you have to assume they're not interested in doing so, and they have no burden.

But is this a new problem? I don't think so. I think the meaning that yourlogicalfallacyis.com ascribes to burden of proof is just as useless, but more harmful.

Useless

If I'm in a conversation with someone, and a claim comes up that we disagree about, there are two possibilities:

1. I'm sufficiently motivated to try and persuade the other party that my opinion is correct.
2. I'm not sufficiently motivated to do that.

In the case of 1 I'm already trying to persuade the other; there's no need for either of us to talk about the burden of proof. In the case of 2 the other party can complain that the burden of proof is on me all they like. It doesn't matter. I'm not going to be moved to plead my case to them because I don't care about persuading this person.

In neither possible case is the legacy meaning of burden of proof helpful.

Harmful

On top of being useless, yourlogicalfallacyis.com's definition misleads people into believing they have grounds to demand that another present a case to them against their wishes. This leads to all kinds of entitled misbehaviour.

I believe the uselessness, plus the harm to clear thinking, disqualify the legacy meaning of burden of proof.

In the wild

Some examples of people using the phrase from Twitter.

The burden of proof should be on religious organizations to prove that they are tax exempt by not being involved in politics.

A clearer way of saying the same would be:

Authorities should require evidence from religious organisations that they are not involved in politics before granting them tax exemption.

But proving a negative is famously problematic.

first you need to prove that God exists. The burden of proof lies on you for making a claim that something is real.

A functionally equivalent and less antagonistic response would be:

I don't believe that God exists. Do you want to try to persuade me?

no one ever clearly defines what “Russian Asset” means. It’s meant to create public stigma about the accused while removing the burden of proof from the accuser

An accusation doesn't automatically imply a burden of proof to begin with–since such a burden comes from your desire to persuade another, which you might not feel. I'd write the above this way instead.

“Russian Asset” is a vague term meant to create public stigma about the accused, while avoiding specific claims that could be decisively contradicted.

Scott Alexander writes (edited for brevity):

Imagine a lake with a thousand identical fish farms owned by a thousand competing companies. Each fish farm earns a profit of $1000/month. For a while, all is well.

But each fish farm produces waste, which fouls the water in the lake.

A thousand fish farms produce enough waste to lower productivity by $1000/month, meaning none of the fish farms are making any money. Capitalism to the rescue: someone invents a complex filtering system that removes waste products. It costs $300/month to operate.

they all sign the Filter Pact, except one person who is sort of a jerk. Let’s call him Mike. Mike earns $999/month, and everyone else earns $699/month. Slowly, people start thinking they too should be getting big bucks like Mike, and disconnect their filter for $300 extra profit…

A self-interested person has some incentive to sign a pact to make everyone use a filter, but in many cases has a stronger incentive to wait for everyone else to sign such a pact but opt out himself. This can lead to an undesirable equilibrium in which no one will sign such a pact.

This sucks. Naturally, Scott is commenting here on how a commons is a very fragile thing, and not presenting this as a slam-dunk against any specific way of arranging society more broadly.

The more I think about it, the more I feel like this is the core of my objection to libertarianism, and that Non-Libertarian FAQ 3.0 will just be this one example copy-pasted two hundred times.

Oh

The furious copy-pasting implies this is a problem libertarianism specifically has a hard time with; Without a state with a big stick to compel cooperation, a libertarian society is doomed to universal defection and everything bad that comes with it.

I don't think so. Because there are simple social technologies that are attractive for individuals and which solve these coordination problems for groups - at least if the state isn't around to forbid their use.

It's the ownership

Individual fish farmers want private ownership of the lake they use. They want this because private ownership gives them a way to penalise, and receive restitution from, polluting parties through private tort complaints. These conditions keep their businesses viable.

With the state no longer disallowing the lake to become privately owned, private ownership could take different forms. The farmers could jointly own the lake and have a contractual agreement among themselves that any of them found polluting the lake would be required to compensate the others. The contract could specify that repeat offenders be barred from using the lake altogether.

Or the farmers might each claim ownership of the parcel of lake around their own farm, with another portion remaining unowned. In this arrangement they'd be able to bring torts against polluters whose actions harmed their own parcel.

The farmers would separately, or collectively, claim ownership of the previously unowned lake in accordance with the local homesteading norms. That might mean visibly fencing off a parts of it, demonstrating that it contains their equipment and animals, and otherwise demonstrating that it's 'land' that they are making routine use of.

Guilt

In principle, this is all the groundwork needed to hold a polluter liable. But in practice, at least at first, it might be technically challenging to correctly identify the polluter. Even then though, with the threat of tort, a farmer could no longer afford to pollute casually. And the longer this regime persisted, the more opportunity there'd be for social and technical innovation to deliver systems allowing the identification of polluters, all without centralised political power.

Perhaps in the future farms have the option of paying a small fee to one of several competing organisations that audit production operations and award a certificate if the firm's pollution is below an accepted threshold. As a farmer, presenting such a certificate is a convenient way to establish innocence in case of pollution complaints, but more importantly it probably becomes part of the cost of doing business; If I were running an insurance firm catering to industry, I'd want to know as much as possible about the risks I was taking on via my clients. So I'd structure my contracts to  make pollution liability payouts contingent the client's continuous ability to show a valid 'good industrialist' certificate.

That's how in Ancapistan - a libertarian society with private law and no state -coordination problems like the one faced by Scott’s fish farmers are solved organically by private ownership and torts.

On November the 4th 2021 I attended a psilocybin retreat lead by the Essence Institute in the Netherlands. It was incredible. Over three days, with sixteen other participants and supported by a small team of experienced facilitators we prepared, travelled inwardly, and began the process of integrating what we'd been shown.

We were guided and hosted by Willem Fonteijn, Stella Stok, Paul Soons and Sanne de Burger. The Essence retreat format uses a set of activities that make it possible to quickly feel at ease in a group, and even to feel close to these recently-strangers; essential for reaching the level of openness and vulnerability that the ceremony asks of you. The group format has advantages over a solo journey that I didn't fully appreciate beforehand. For processing and integrating what you've seen it's helpful to be among people who've just had a similar experience. They're the ones most able to given full attention and credence to what you're describing. And hearing their accounts with openness can end up shedding light on aspects of your own experience. You can find more information about the Essence format on their website.

Preparation

The reading material suggested by the institute was valuable. In the weeks leading up to the retreat I read The Psychedelic Explorer's Guide by Jim Fadiman and How to Change Your Mind by Michael Pollan. Both useful, but the latter much more so for me. And so well put together. I think many will enjoy it even if they have no plans to use psychedelics. The accounts of people's journeys and practical information about how to maximize their usefulness were most interesting to me.

Here are some excerpts from Pollan's book that seemed important to keep in mind during the ceremony.

The flight instructions advise guides to use mantras like “Trust the trajectory” and “TLO—Trust, Let Go, Be Open.” Some guides like to quote John Lennon: “Turn off your mind, relax and float downstream.

If you feel as if you are “dying, melting, dissolving, exploding, going crazy etc.—go ahead.” Volunteers are quizzed: “If you see a door, what do you do? If you see a staircase, what do you do?” “Open it” and “climb up it” are of course the right answers.

“Think of yourself as an astronaut being blasted into outer space,” Richard Boothby recalled him saying. Boothby is a philosophy professor who was in his early fifties when he volunteered at Hopkins. “You’re going way out there to take it all in and engage with whatever you find there, but you can be confident that we’ll be here keeping an eye on things. Think of us as ground control. We’ve got you covered.”

The main thing is to surrender to the experience, even when it gets difficult. Surrender to your fear. The biggest fears that come up are the fear of death and the fear of madness. But the only thing to do is surrender. So surrender!

Many of the volunteers I interviewed reported initial episodes of intense fear and anxiety before giving themselves up to the experience, as the guides encourage them to do. This is where the flight instructions come in. Their promise is that if you surrender to whatever happens (“trust, let go, and be open” or “relax and float downstream”), whatever at first might seem terrifying will soon morph into something else, and likely something pleasant, even blissful.

The Ceremony

After an illuminating breathwork session in the morning, and a light lunch, the ceremony began.

We were gathered in the hall, dressed in white, and sitting on our mattresses. The music on Willem's playlist had begun.

Following our instructions I put one and a half packets of mashed High Hawaiians Magic Truffles into my glass. 33 grams, a high dose. Paul came round adding ginger tea to our glasses. We stirred and pressed the truffles with our spoons until the tea had a milky look.

We drank the tea, lay down, and put our eye masks on. I didn't leave the mattress for the next six hours. I've forgotten many details and scenes. What follows are the parts that stand out the most strongly for me two days later.

Soon I felt unusual sensations including tingling in my hands and feet. I saw the beginnings of motion and imagery. The music changed in an unpleasant way; a harsh metallic echo had appeared. I felt like anaesthetic was spreading through my brain and that my thoughts were malfunctioning. I felt anxiety. The thought came up that it was a mistake to use psilocybin again after the bad experience I had with it in my teens.

The visuals developed into vast underground spaces. Dark, with three dimensional arrays of repeating objects stretching as far as I could see. A succession of these spaces presented themselves. The atmosphere was oppressive but also awe inspiring.

Following Willem's instruction to approach whatever threatening thing we encountered I imaginatively stretched my arms wide in a literal embrace and swam into the darkness.

Quickly the scene changed. I emerged onto a light-filled plateau. Again arrays of geometric forms stretched away endlessly. The light intensified towards a brilliant center. The sense of threat was gone. Instead there was overwhelming awe and joy. I felt like the most fundamental level of reality was revealing itself and I was continuous with this glowing environment. I whispered 'Oh' many times in wonder. I cried with happiness. I stayed here for what felt like a long time.

From this point onward gratitude and wonder were the feelings anchoring the journey.

Stella placed a hand on my shoulder and asked me how it was going. Smiling broadly I said "Good". "It looks like it's going good" she replied warmly. "Keep going".

My two year old son appeared several times during the journey after this point. He was wearing his favourite bag/hat on his head and carrying his favourite stick. He said, in Dutch, "Ja. Uh-huh" and nodded. Sometimes he used carefully pronounced English; "Yes. Uh-huh". These are the self-confirming phrases that often follow his announcements in daily life.

In the journey my son's confirmations carried the meaning: Notice and accept the fundamental goodness and beauty of existence. Many times during the rest of the journey I would nod my head both during intense happiness and sorrow. Following my son's example and gratefully accepting both.

I decided to visit sorrow. Immediately I saw my Polish grandparents and felt a powerful wave of grief for their fear, pain and suffering at the end of their lives. I sobbed heavily.

Willem placed a hand on my shoulder and said softly "It's good. Go into it". "Yes, thank you" I replied through the crying. I knew that the goodness and beauty of existence can carry - and coexist with - any amount of grief.

I cried for the suffering of my father who died alone of cancer.

I cried for my mother's care for - and her worry about - me and my brother.

I cried for my wife and the hurt we've caused each other in our relationship.

I cried for my eldest son, and for how I've so often failed to be patient and compassionate with him the way I want to be.

Throughout the journey I could hear everything around me in the hall; the sighs, crying, and laughter of the other participants. I heard the low conversation between our guides about how to proceed in supporting a voyager who was having a difficult time. I could hear and feel their footfalls as they walked between the mattresses. None of this felt threatening or distracting. I noticed these goings on with a feeling of safety and interest.

Later, laughter from other participants was coming in waves. I smiled and nodded each time it came, sometimes laughing out loud with the others, helpless not to. I felt the dampness of my tear-soaked eye mask on my cheeks. I began to notice soreness in my stomach from sobbing and laughing. Eventually I became interested in deliberately moving my hands, feet, and head.

I heard Willem announce to the hall that the team was bringing food round for us. I took off my mask as others were doing the same and understood the journey was ending. I wiped my face and ate fruit from the platters that Willem, Paul, Stella and Sanne offered. I felt exhausted and at deep peace.

Results

So far, a few days on, I've noticed a very welcome lightness of mood - I feel less serious, more playful. It's much easier to be mindful. I still have negative feelings and reactions but they no longer have me; there's a good deal more optionality in whether I indulge them. I find it much easier to be patient and compassionate towards my children.

I often think back on the experiences I had and am moved by them again. I also notice with sadness how they're fading from my memory.

At Willem's suggestion I've begun doing very short mindfulness exercises while sitting, walking or biking.

The tape measure disagrees but I feel taller and as though there's a lightness at the top of my head where there used to be pressure. Perhaps least expectedly I can touch my toes - and easily - for the first time in my adult life.

I've made a note to revisit this post in six months to add a section about any changes that have persisted in the longer term.

Cryptoart.wtf's content has recently been replaced with a holding page for unrelated reasons, but its mistaken conclusions feature in many articles critical of the ecological footprint of NFTs on the Ethereum network. I hope this post helps set the record straight.

The calculation

On this page Memo explains how the energy cost of a piece of cryptoart is calculated on the site. The calculation depends on adding up the gas costs of all transactions associated with the artwork being analysed. He writes:

Since the energy required and footprint of mining a block is independent of its contents and number of transactions, the Gas required by a transaction is representative of the portion of a block’s footprint it will incur

From the total amount of Ethereum Gas and energy consumed by the Ethereum network over a period we can calculate the energy footprint per unit of Gas. Then we can calculate the footprint of a particular transaction from the amount of Gas that it used.

At first glance this looks odd when you know that gas used in a block and energy used to mine that block are largely independent; Empty blocks cost as much energy to mine as full ones under the same network conditions. It might look like this invalidates any attempt to apportion eco impact to transactions. But if a transaction increases the amount of future mining that occurs it may still be possible to assign it an energy footprint.

The air travel analogy

We calculate emissions footprints for plane tickets. Can the same approach be applied to transactions to the Ethereum network? Memo writes:

There is a common fallacy along the lines of “The same energy will be consumed whether a block is empty or contains your transaction, thus a transaction has no impact on the energy consumption of a block being mined, or on the environment”. This is based on a gross misunderstanding of what a carbon footprint is.

This statement is analogous to claiming “when a 500 tonne airplane flies from NYC to LA, it consumes the same amount of fuel whether I ride it or not (because the weight of a person is negligible compared to the weight of the plane). Thus my flying has no impact on the environment”.

It is true that one person deciding to fly (or not) does not have an immediate effect on emissions. However, there is a footprint associated with a seat on a plane.

Each ticket sold is effectively a signal telling the airline Keep scheduling flights! This is how buying a ticket can be causally linked with emissions even if physically travelling on the plane didn’t increase the emissions of that flight at all (or if you simply missed the flight).

How well does this approach map to the Ethereum network? There’s an important difference between the incentive structure faced by an Ethereum miner and that faced by an airline that complicates things.

Captain pollution

Imagine that an airline received a large financial reward from a mysterious eco-villain for each flight it completed. The reward was issued whether or not any seats on the flight were occupied. Imagine too that this reward was many times greater than the revenue from the tickets sold for any fully-booked flight.

Under these circumstances it wouldn’t be appropriate to apportion the entire ecological impact of a flight only to those who’d bought tickets, because this ignores the incentivizing effect of the reward. Indeed the eco-villain is incentivizing the airline to schedule flights more than the ticket buyers are.

Memo's calculations on cryptoart.wtf are doing the equivalent of ignoring Captain Pollution's reward.

Incentives to mine

Successful miners on the Ethereum network are currently rewarded in two ways.

1. By receiving a block reward, which is an amount of newly created ETH
2. By receiving transaction fees (gas fees), an amount that users of the network send along with their transaction. This source of revenue looks set to be dramatically reduced by an upcoming change to the network

Here’s a breakdown of miner revenue into block reward (Subsidy) and transaction fee parts for the last several years. For most of Ethereum's history the subsidy part has been close to 100% of total revenue. Recently it has approached 50%.

The Ethereum block reward is analogous to the eco-villain's reward to the airline company. Successful miners receive the reward regardless of what’s in the blocks, and it’s usually very large compared to the revenue from transaction fees (which are analogous to ticket sales in our thought experiment).

For the same reason that it wouldn’t be appropriate to ignore the eco-villain’s reward and divvy up the entire ecological impact of a flight between ticket buyers, it’s also not appropriate to apportion the entire energy footprint of block mining to those making transactions within those blocks.

Cryptoart.wtf incorrectly calculates as though transaction fees are the only incentivizer of mining. The block reward is a major and largely independent incentive too.

TLDR

What I’m not arguing in this article:

• It’s wrong to be critical of proof of work systems
• Cryptoart NFTs are unambiguously a great idea

What I am saying, specifically: Cryptoart.wtf’s reports of the energy use of NFTs are wrong; the calculation used there isn’t valid, and any article depending on them is also in error. This truth should be relevant to anyone considering this topic regardless of their stance on Ethereum or NFTs.

Notes

This post grew from a Twitter thread. In that thread I didn't address the possibility that transactions can validly be assigned eco footprint based on their incentivization of future mining work. I believe this post does a better job of characterizing the problem with the calculation.

While outside the scope of this post, I'm not convinced that the Ethereum (or Bitcoin) networks currently use too much energy, since this implies to me that the costs of those networks outweigh their benefits - or that they could have successfully bootstrapped without implementing proof of work systems. I believe the benefits of decentralized currency and computing for humankind are hard to overstate.

Another source of miner revenue is MEV (Miner Extracted Value). Because much MEV activity cannot currently be detected, the upper bound of MEV is unclear. This means that even if cryptoart.wft's calculations were altered to account for the effect of the block reward incentive on mining behavior and known MEV, this arithmetic could only establish an upper bound on the energy footprint of a set of transactions.

Thanks to Justin Goro for corrections and suggestions. Any mistakes that remain are my own.

I see this mistake often in online exchanges. I'd like to help spread the term I believe John Corvino coined.

One commits the Fallacy of Perverted Analogy when one misreads an opponent’s analogy to make a far more sweeping comparison than the opponent needs or intends.

Examples

A "Some people feel disgust at the sight of men kissing. But it doesn’t make sense to condemn a person merely for feeling disgusted. You probably feel disgust when you think about some sex acts too — for example people who enjoy eating shit"

B "It's your comparison of two men kissing to eating shit that's disgusting. It's bigoted and makes no sense. The two things are worlds apart!"

B "You use the services that the state provides. That means you consent to being taxed."

A "If a slave accepts a meal from his master does that mean he’s consented to his enslavement?"

B "You're comparing paying taxes to being a slave?! This conversation is over."

In the examples speaker A is not suggesting that the things being compared are similar in every way, or even most ways. They believe they’re similar in a limited way that’s relevant to the topic of the discussion.

B might be able to show differences between the things being compared that are relevant to the conversation at hand. Those differences might make the analogies invalid. But just pointing out that differences exist, even large differences, is not enough to dismiss the analogy.

There’s an imaginary guy called Evan.

Evan believes, like his peers, that there are no significant biological differences between any two populations of humans. At least none that are caused by the expression of genes that relate to personality traits or cognitive ability.

In Evan’s view, with respect to these aspects at least, there’s a sense in which every healthy human begins as a blank slate. He’s convinced that environmental circumstances are overwhelmingly responsible for how the personality and cognitive ability of every human being develops. Evan believes this view is well supported by studies in the relevant fields.

This outlook also gives him hope that a very thoroughgoing kind of equality is possible through the right kinds of social program.

Evan also believes that it’s fair to describe statements that contradict his blank slate position as sexist or racist, depending on the kind of population the statement relates to. People who believe such statements are racists or sexists.

This combination of beliefs has Evan backed into a corner.

Even if we assume that none of the current findings that get grouped under the human biodiversity (HBD) umbrella have any merit, there’s much more enquiry to be done in these fields. We don’t know what will be found.

Evan isn’t free, psychologically, to go wherever the evidence takes him though. Going down one of those paths, in his view, also means accepting one of the most shameful and damaging labels in his society. In this context Evan is a very strongly motivated to ignore or otherwise find a way to reject any future evidence he comes across that suggests that there are important differences, in aggregate, between how the minds of different human populations work.

Critics of blank slate position sometimes refer to the inconvenient counter-evidence, the kind we can expect Evan to work hard to reject, as hate facts.

I think it’s unwise to attach normative judgements to empirical claims about the world that might turn out to be true.

To the extent that people like Evan care about the truth, and want to protect their ability to pursue it, I hope they reassesses how they’re using the terms racist and sexist.

See also Moldbug’s caution against IQism.

Mathematicians and programmers appear to have carved out a space invisible to the state and other powerful interests through the innovation of encryption, for now.

But maybe the long-term equilibrium at the end of this contest is omniveillance — a situation in which everything of importance is observed, recorded, and retrievable by anyone who cares to look for it.

David Brin is a writer who believes this future is inevitable. Can we indefinitely maintain our freedoms by staying one step ahead of the surveilling powers, being better at hiding than they are at seeking? In Brin’s view, this approach isn’t sustainable.

Hiding will not work. Sure, protect your passwords as a short term, practical matter. But over the long term only one thing will keep you free. Aggressively, militantly empowering yourself and your neighbors to see!

This points to an asymmetry that stands out to me: Surveillance technology is becoming ever more powerful, hard to detect and ubiquitous, while the observable physical footprint a person leaves on the world is at least as large as it ever was, and shows no sign of shrinking.

Encryption might allow us to hide our tracks in the digital realm, but at least for now we still live in meatspace and soon the spies will overtake our ability to evade them here.

Our near-future will be filled with hidden sensors. Some will be descendants of technology we’re already familiar with, like car plate readers and CCTV networks linked to machine vision software. Others are just coming into view over the sci-fi horizon; autonomous drone mosquitos, smell sensors, gene-hacked spy plants.

Eventually these systems, and new ones we haven’t imagined yet, will be writing data to logs that can be accessed arbitrarily far into the future.

Why would it happen if no one wants it?

We’ll get to omniveillance, even if most people would prefer to live in a different kind of world, because of a collective action problem. While you may bristle at the idea of being observed yourself, it will be irresistible to use this kind of technology to your own advantage.

Initially this might mean monitoring your own property and neighbourhood in case of theft or vandalism. Later it might mean watching your local police department or politicians in case of brutality or corruption.

The nascent omniveillance infrastructure might get its start when a future MySpyCorp extends an offer: “You can access our large, and growing pool of observed data either by paying a subscription fee, or (our most popular plan) by having your devices log their data to it.”

Perhaps multiple such datastores will exist. Over time competing stores will be incentivised to merge their data together to offer their customers a spotlight capable of shining on a bigger part of the world.

A few years later, the fantastically successful MySpyCorp is involved in a scandal. Maybe they’re been found to be censoring data pertaining to members of their management, business associates or friends in politics.

Whatever the details, eventually the dangers of having this kind of data under the control of trusted gatekeepers will be felt urgently enough that there will be a demand for this kind of data delivered in a trustless, decentralised form.

Manufacturers of surveillance hardware who are alert to this opportunity will create the open omniveillance backbone. And because their products are designed to interface seamlessly with it, they will reap the market’s rewards.

Eventually we’re at a point where I can find out anything you’ve done, and vice versa, through a conveniently browsable interface.

Is this state of affairs necessarily a dystopian nightmare? Brin doesn’t think so.

Hope through ‘sousveillance’

The idea of sousveillance contains the key to understanding Brin’s hope for a humane, transparent society. Transparent in the sense that access to information illuminates in all directions.

The term “sousveillance”, coined by Steve Mann, stems from the contrasting French words sur, meaning “above”, and sous, meaning “below”, i.e. “surveillance” denotes the “eye-in-the-sky” watching from above, whereas “sousveillance” denotes bringing the camera or other means of observation down to human level, either physically (mounting cameras on people rather than on buildings), or hierarchically (ordinary people doing the watching, rather than higher authorities or architectures doing the watching).

If everyone is able to watch everyone else, then the least powerful can watch the most powerful, who have more to lose. We know from today’s high profile scandals that powerful figures can be lowed quickly if enough people become convinced they’ve behaved unacceptably.

The discipline of Mutually Assured Snooping

When an act of snooping is itself subject to watching eyes, and indelibly etched into the public record, is it still worth it? Brin believes that under these conditions, a strong live-and-let-live norm would emerge — Don’t snoop without Good Reason if you don’t want to be subject to the same treatment yourself.

Perhaps there’d be an attitude that, under normal circumstances, it’s not okay to access data on a person gathered from places traditionally regarded as private (e.g. within their own home). It’s possible that attitudes about the threshold for justifiable snooping would be inversely linked with the perceived power of the subject in society. For instance, assuming the nation state lasts that long, politicians might reasonably expect scrutiny at all times and places.

Weirdos

How widespread is preference falsification? Do all members of a community secretly do that one taboo thing, each believing they’re the only one? If they do, perhaps their mental health would benefit if they knew they were not alone in that, and no longer had to maintain the secret. The journey towards omniveillance seems likely to trigger any such latent preference cascades that might exist. To me this sounds like a good thing.

On the other hand perhaps it really is just you that does that one weird thing. Depending on who else is in your community, your life could become very unpleasant if your preferences became widely known. More generally it’s conceivable that omniveillance could strengthen social control in some contexts.

On the upside that might mean better social cohesion and trust. On the downside it could mean increased persecution of outliers, dissidents and dissenters — more pressure to conform. While the inmates of Bentham’s panopticon modify their behaviour because of the presence of a distinct class of unseen observers, everyone in an omniveillance society is a potential victim of the tyranny of the aggregate.

I don’t think we can confidently cheer or jeer about the likely effects of an omniveillance society on human wellbeing yet. Like a Rorschach test, Maybe one’s gut feeling about the outcomes of radical transparency will mirror their general affective stance towards other people.

Epistemic Effort: I thought about this on-and-off for three months. I spoke to three smart friends about technological challenges and likely social/political effects. I read papers on emerging decentralised alibi technology and surveillance theory. I spent about four hours writing different versions of this post. Everything written here is highly speculative.

Whether or not the universe is deterministic we don't have free will. Because free will is a self-contradictory concept. Free will is a square circle, a squircle. Here I’ll try to explain why free will illusionism is the correct stance.

What is free will?

Based on how I hear people use the term most often, I consider free will to be a quality an agent can possess such that the agent could have deliberately decided differently, for any actual decision they made. My compaint with compatibilism is that it doesn't speak to this sense of the term free will, which I believe is the most relevant because of how widespread it is.

By could have decided differently, I mean if the state of the universe was somehow rewound to just before I chose to drink that last beer, everything identical to the first time around, free will would make it possible for me to deliberately chose not to drink the beer instead.

Why is the deliberately part important in this definition? If we imagine a system in a person's head that resolved decisions solely on the basis of some kind of quantum randomness, there’s a sense in which that agent could have chosen differently in an identical universe, but they would not seem to be the authors of these differences. The differences wouldn’t be the product of deliberate thought, and so not the product of will. And since free will is a kind of will, we have to be able to describe its decisions as deliberate.

Could have deliberately decided differently

The Deliberately refers to the will part of free will. This is where the idea links to the continuity of the self, and an agent’s sense of owning the decision.

The Differently invokes the more mysterious free quality of free will, the idea that there’s some part of decision making that’s unconstrained by the prior state of the universe.

Will is prior brain state

A decision is mine, it’s the product of my will, to the extent that the prior state of my brain causes it. That state includes my preferences, habits of thought, memories, goals and everything else that makes me me.

To the extent that a decision is the product of anything else (which it would need to be, to be free in the relevant sense), it’s not recognisable as mine, not recognisable as a product of my will.

Will can’t be free, and free can’t be will.

How does free will reach its decisions?

For any decision there are three possibilities:

1. The decision is fully caused by the prior state of the universe, including the agent’s brain (determinism). This allows deliberacy, but not freedom in the free will sense. So this decision isn't the product of free will.
2. The decision is made entirely on the basis of non-deterministic randomness. This allows freedom but not deliberacy. So this decision can't be the product of free will either.
3. The decision is partly the product of the prior state of the universe, and partly the product of non-deterministic randomness.

The the case of possibility 3, neither contributing part can be called free will (prior state determinism on the one hand and randomness on the other). A decision making system that combines a bit of each of these doesn’t unlock the elusive power of could have deliberately decided differently either.

An analogy: A traffic control system might behave perfectly predictably given the same input. It makes simple decisions about how to control the stop lights at a junction. We don't get this system any closer to having free will by incorporating the results of a random number generator into its decisions, making them less predictable. This is true even if the RNG could harnesses something like quantum randomness.

What is going on?

We have a blind spot. We can’t be aware of the causes of the thoughts that we experience as the self. So to us these thoughts seem uncaused. Free will is the flattering illusion that the self is an uncaused cause.

My son attends a learning institution with no mandatory classes. Compared to students at traditional schools (I’ll refer to them as just schools from now on) he has a lot of freedom.

Sometimes I talk with parents who are skeptical about the wisdom of allowing this freedom. One objection I hear is that school ought to prepare a child for the real world, and in the real world you often have to do things you’d rather not. The implication is school should get kids used to this reality by systematically requiring them to do things they’d rather not, and schools without mandatory classes and assignments fail to meet this requirement.

I'm going to talk about a few of the reasons this objection isn't compelling.

The real world?

I think the language of the objection is interesting. If school is preparation for the real world, then a child's world, while they are in school, is in some sense not real. This might seem like a trivial detail, but I think it reflects a common attitude that is harmful.

I believe children are already real people, living real lives, in the real world, whether they're in school or elsewhere. Their experience (including their suffering) during this time is as real as that of an adult.

The idea that children are not yet living real lives fits a tendency to unwittingly diminish and dismiss the experience of children and to treat them as less than full persons.

A less problematic way of phrasing the objection is that school prepares kids for later life. More on that idea in a minute. For now let's note that the costs and benefits of schooling, immediate as well as long-term, count and come to bear on real life, in the real world.

Here's Teacher Tom reflecting on the life/school dichotomy.

If we really want to prepare children for life, wouldn't that be better accomplished by letting them live it? When children are allowed to play together, when the adults are there to guide and inspire, rather than direct and judge, then they are really living the life that exists beyond the artificial confines of school.

In later life there's only one place like school

School is different from later life in ways that are relevant to the question of whether it's a good preparation for it.

In later life people don’t generally solve problems and do otherwise unnecessary work to gain the approval of an authority figure. Usually the work people do in later life is motivated, one way or another, by personal interest and/or a desire to improve a situation for oneself or for others.

This artificial arrangement in schools, I claim, alienates students from their innate curiosity, and encourages the suppression of natural interests. Here's Peter Gray.

Schools thwart passions by:

Requiring everyone to do the same things at the same time. It’s not possible for all the children in a room to be passionately interested in the same thing at the same time.

Replacing intrinsic motivation with extrinsic motivators, such as grades and trophies. To pursue a passion you have to focus on what YOU want to do, not try to impress others or win honors.

Threatening students with failure or embarrassment, which generates fear. Fear freezes the mind into rigid ways of thinking and negates the possibility of passionate interest.

Teaching that there is one right answer to every question, or one right way to do what you are supposed to do. That’s a surefire way to nip any possible emerging interest in the bud.

Teaching children that learning is work and that play, at best, is just a break from learning. But anyone involved in a passionate interest knows that play and learning and work are one and the same.

In later life you usually opt in to social settings, and can usually opt out again. Even if you take a job you dislike, you have the ability to save up and quit, and search for something else. On the other hand, a young child doesn’t generally choose his school. And once enrolled he cannot leave without his parents cooperation. Unlike an adult, he's not free to exit a situation that's a bad fit for him.

Most adult lives don't include situations that are anything like being at school. Here's Peter Gray's provocative comparison.

The only difference I can think of is that to get into prison you have to commit a crime, but they put you in school just because of your age. In other respects school and prison are the same. In both places you are stripped of your freedom and dignity. You are told exactly what you must do, and you are punished for failing to comply. Actually, in school you must spend more time doing exactly what you are told to do than is true in adult prisons, so in that sense school is worse than prison.

Further still, the real world of today is very different form the real world of fifty years ago. The internet allows freedom of association to be exercised to a much greater degree than ever before. Meanwhile mainstream schooling has changed very little in the last fifty years. School was already very different from the real world, and it resembles is even less today than it did in previous eras.

Your real world or mine?

When a person worries that school is a necessary preparation for the real world they’re inevitably talking about ‘the world as I know it’.

In the real world as I know it, my school career has not been a helpful starting point. These following abilities have been crucial for my journey in adult life so far:

• The ability to freely follow my interests
• to be critical and express dissent,
• to exercise freedom of association and exit situations that weren’t optimal for me.

I've been able to do these things exists in spite of my school career, which incentivised:

• suppressing my interests
• self-censoring my dissenting views
• and resigning myself to spending every weekday with people I disliked.

First, do no harm

Even if school was similar to later life, it’s still not clear that it would be desirable as preparation for it.

If we believe a person will be systematically dehumanised, alienated from their work, and compelled to do things they don’t enjoy later in life (a disliked job), are we confident that we'd be helping this person by systematically dehumanising them, alienating them from their work, and compelling them to do things they don’t enjoy while they're a child? (traditional school).

The attitude of ‘this is just the way the world is, better get used to it’ is a self-fulfilling prophecy. Students habituated to asking permission and deferring to arbitrary authority will be less able, I claim, to change the world for the better.

Not walking the walk

Do people really believe that the hardships of school are good, character-forming preparation for real life? Often I’m not convinced that they do. Or at least there seem to be limits on how far this principle informs people's behaviour, and other considerations are more important.

I don't think parents making some version of the real world objection usually seek out the most authoritarian school in their locality, for instance, even though this one would be the best bet (the reasoning seems to go) for hardening their children up to weather the demands of bosses in uninspiring jobs later in life. Being hypocritical doesn't mean being wrong, but it seems that the real world objection isn't one that really motivates the people who make it.

Two environments

We want our children to be resilient, resourceful, inventive and able to delay gratification for long-term goals. These are the characteristics that will serve them well in the real world (and yes, that will include following directives from others when necessary). Which environment do you think is more likely to foster these traits?

1. An environment which requires a high degree of conformity. Where permission is required from an authority figure even to carry out basic body functions (eating, drinking, using the bathroom). Where the teacher is the final arbiter of ‘the right answers’, and your goal is to gain their approval by providing them.
2. An environment with a high degree of autonomy, where a child is responsible for figuring out what it is they want to do and doing it. Where they learn quickly because they've been allowed to follow a passion for the topic they're diving into. Where it’s up to the child to negotiate with others to realise their plans, with help from adults if they choose to seek it.

I'm convinced that if we want to prepare our children for the real world, we can do a lot better than sending them to school.

Here's a digest of conversations I often have with opponents of private property.

You don't own your body, you are your body!

There's no conflict between the two ideas. To varying degrees you are your body (you are your brain to a much greater degree than you are your little toe), while you are also the owner of your body. You have the exclusive right to determine how your body may be used. This is what ownership means, at least in the context of scarce rivalrous resources like human bodies.

Granting this, there’s no obvious difficulty with going further. If you own your brain, and also the lock of hair you cut from your head then perhaps you can be the owner of things physically more distant still. These owned things might include what some call the means of production.

Just because someone doesn’t want to get raped doesn’t mean they believe in “private property rights”.

I agree. I’m saying that people generally believe that they have a right to exclusively control how their body is used by others–not merely that they have preferences about how it may be used (they do have such preferences of course). That belief about a right to exclusive control is a belief in private ownership. When we talk about a right we’re talking about a claim that we believe the use of force is justified in enforcing (when enough people agree, this right will be reliably enforced).

If the idea of rights is a sticking point we can decompose the claim like this:

A person considers a thing his property if he believes he’s justified in using force to exclude others from its unauthorised use.

People consider themselves justified in excluding others from the unauthorised use of their bodies, so it's clear they consider their bodies to be their property.

I understand the impulse to oppose private property from a wish to improve the situation of the have-nots, and minimise the advantage of the haves. But I think this approach gets things backwards. The right to private property is one of the most important protections of the poor against the wealthy. Absent any legal protection of property rights, the advantage that the wealthier person has over the less wealthy one increases at least in the following sense: the wealthier of the two always has more means with which to take the poor person’s property by force. The institution of private property helps level the playing field to the advantage of the poor person.

If I had the power to persuade non-libertarians to read one book right now I’d choose The Problem of Political Authority by Michael Huemer. It's a distillation of the most robust and compelling arguments against the legitimacy of the state I’ve come across in ten years of exploring the topic.

If you don’t self-identify as a libertarian before reading this with an open mind I wouldn't be surprised if you do three months after finishing it. At the very least you’ll have gained an appreciation for the most powerful arguments for libertarianism and you can truthfully say you gave the idea a fair chance.

If you already self-identify as a libertarian the book is valuable too. As well as rebutting a greater breadth of pro-state and anti-anarchy arguments than I’ve seen collected elsewhere, it’s a great illustration of the value of defending anti-statism by relying on the reader’s desire for coherence in their moral judgements. The author begins by finding common ground with the reader: agreement about the conditions under which the interpersonal use of violence is condemnable. He proceeds by carefully examining each of the strongest attempts to show that the state’s use of violence is different in some ethically significant way to the violence that we condemn. All the arguments examined are demonstrated to be insufficient to the task of rescuing the state from our condemnation.

The arguments challenging the state's legitimacy never depend on the reader accepting a controversial premise (such as the existence of natural rights, or of free will, or the truth of a particular ethical theory). This is the same approach I tried to take with my George Ought to Help series of animations.

The chapter on the The Psychology of Authority is particularly interesting to me. Huemer examines well-documented cognitive biases to provide an account of why so many people are in thrall to a cognitive illusion (the belief in the right of the state to coerce) that’s at odds with their foundational moral intuitions.

The second part of the book is dedicated to outlining the ways that market anarchy would solve the problems of providing the goods of law and its enforcement, and national defence. The final chapter surveys historical evidence to derive a set of conditions under which anarchic zones could be expected to emerge and thrive in a world initially dominated by states.

Previously I’d recommended The Machinery of Freedom (PDF, Paperback) by David Friedman and Chaos Theory by Robert Murphy as great texts that introduce the ideas of libertarianism/anarcho-capitalism. In my view The Problem of Political Authority is superior to either. It’s more comprehensive, easy to read, scholarly, humble and reasonable in its argumentation. I’ll be recommending this one until something even better comes along.

In an interview Milton Friedman said:

The most important single central fact about a free market is that no exchange takes place unless both parties benefit.

This is close to the truth. And the simplification might be appropriate when comparing free trade against coercion as an organising principle. But it's missing an important qualifier, and the more thoughtful critics of markets will notice. Here's how Walter Block qualifies it, emphasis added.

Each and every trade is mutually beneficial in the ex ante, or anticipatory, sense. That is, if I trade you fifty cents for a newspaper, we each value the other’s possession higher than our own.

Each party in a voluntary trade expects to be better off for having made it. That mutual expectation is why the exchange happens without the needed for coercion. But expected benefit isn't the same as realised benefit.

Discrepancy between expected benefit and actual benefit can have three kinds of cause:

1. The seller commits fraud, misrepresenting the thing he sells: it turns out that the goose doesn't lay golden eggs.

2. The product was fairly represented, but the buyer makes an error of judgment. A year later it's clear that the product he bought didn't benefit him enough to offset the opportunity cost of having bought it.

3. The buyer knew about all the undisputed features of product (price, physical characteristics, performance data etc), and wouldn't have made the purchase, had it not been for skillful promotional messaging on the part of the seller.

The three kinds of outcome aren't sharply delineated. Two and three can fully overlap. I'm most interested in the third, because I think it represents the most interesting challenge to proponents of free markets, like me.

Fraud

Libertarians generally believe the use of coercion in response to fraud is legitimate. So we grant that the first item is a valid concern, and advocate institutional enforcement of rules (laws) against fraud and/or the use of soft social disincentives to fraud such as reputation systems.

Mistakes

In a world of imperfect knowledge we accept that people will make mistakes, including making trades that they later regret. In this context we realise that expected benefit is the best motivation available to plan-making humans, there can be no certainty of benefit. So regretted trades aren't a special defect of markets. They're just the kind of mistakes that humans make routinely anyway. In a market economy they'll often be made in the context of trade.

To the extent that property rights are enforced, the costs of trade mistakes are borne by those who make them. This arrangement minimises perverse incentives and systemic risk, which are commensurate with the degree of top-down political control in society.

PSYOP

Most interesting to me is the case where real benefit doesn't match expected benefit because of a particular kind of promotional messaging on the part of the seller. Think of famous contemporary organisations with marketing departments.

In terms of what marketing departments do, we can imagine two non-exclusive dimensions: fact conveyance and PSYOP.

Fact conveyance means broadcasting the existence of a good/service and describing what that thing is in relatively objective terms. This dimension roughly corresponds to transparently appealing to reason - "Our product helps solve problem X thanks to features Y and Z".

PSYOP stands for psychological operations. Here I'm using the term to mean designing messages that interact with the unconscious minds of your audience with the intent of increasing sales. PSYOP roughly corresponds to covert manipulation of emotion and what has been called affective advertising.

Here's one example of successful PSYOP from Robert George Heath.

In 2001 the struggling communications network, Cellnet, was relaunched as O2 using a campaign with the vacuous message ‘O2: see what you can do.’ The advert featured blue water with bubbles bubbling through it, people flirting and floating around, fluttering doves, a dog catching a ball, and some lilting music in the background.

There was absolutely no mention of the network quality or coverage or tariffs or handsets, because O2 was no better than anyone else on these. Yet despite being a failing brand, and having absolutely no performance advantage, O2 went from last to first in the market in just four years.

More importantly, an industry analysis of this launch concluded their success was entirely due to the ads, which had encouraged people to feel that O2 was “calm and serene, the antithesis to clutter and chaos, a contrast to the often frenetic world around mobile phones”.

Fact conveyance can help buyers find solutions to their problems, a clear benefit with no obvious downside. On the other hand brand PSYOP could be harmful to consumer benefit on net (even while we may enjoy a well-crafted affective advert).

Defending the advertiser?

In Defending the Undefendable (1976) Walter Block wrote:

Subliminal advertising, if it exists, would be considered coercive. But it cannot be claimed that ordinary advertising is coercive without completely obliterating the distinction between coercion and persuasion.

Perhaps things were different in 1976. Whether or not subliminal advertising is properly considered coercive, In 2019 (in the internet age) ordinary advertising routinely has a subliminal element - in the sense of content that exerts influence without reaching the subjects conscious awareness. I don't think a hard distinction between subliminal advertising and other kinds is tenable anymore.

the main fallacy of the critics — the assumption that deep down, there is a distinction to be made between motivational advertising and informational advertising, that motivational advertising is "bad" in various ways while informational advertising is "good." The truth is, however, that exposing people to information and motivating them are so inextricably bound up together that it makes little sense to even distinguish between them.

Block's claim that information cannot be separated from the way it's conveyed, packaged, is true and may help the case in defense of the advertiser, but it doesn't it doesn't speak to the possibility that PSYOP can invalidate the generalisation that voluntary trade is mutually beneficial.

Confabulation

PSYOP techniques might not even have a bearing on expected benefit. That concept belongs to the domain of conscious deliberation and belief. Skilled emotional manipulation can work instead at the subconscious level. A marginal buyer, one who only just decided to buy the product, may have been tipped over the edge by a cannily constructed sentence or image (or the collective influence of thousands of them) that made them feel more favourably towards the product for reasons they can't articulate.

Stronger still, expected benefit is a much less important component of our buying decisions than we believe. Here's Fiery Cushman's article about confabulation.

We are shockingly ignorant of the causes of our own behavior. The explanations that we provide are sometimes wholly fabricated, and certainly never complete. Yet, that is not how it feels. Instead it feels like we know exactly what we're doing and why. This is confabulation: Guessing at plausible explanations for our behavior, and then regarding those guesses as introspective certainties.

When we are outside the laboratory — and when our brains have all the usual connections — most behaviors that we perform are the product of some combination of deliberate thinking and automatic action. [...] The problem is that we get all of our explanations partly right, correctly identifying the conscious and deliberate causes of our behavior. Unfortunately, we mistake "partly right" for "completely right", and thereby fail to recognize the equal influence of the unconscious, or to guard against it.

Red queen

Under competition in a free market firms will be under strong pressure to use PSYOP. To the extent that these marketing practices become widespread in a society, a systemic gap can open between voluntary trade and mutual benefit.

From this perspective, free market advocates might appear naive. We've noticed the capacity for voluntary exchange to deliver mutual benefit (at least in the anticipatory sense, and under conscious deliberation) but overlook that that's not the only dynamic at play.

Left-leaning commentators have make a similar complaint. Their concern is often the perceived incentives of businesses to maltreat workers and the shared environment under a laissez-faire regime. I think this underestimates the capacity of strict property rights, private law, and market forces to remedy both concerns.

Certain kinds of voluntary exchange will tend to deliver mutual benefit. But free market rhetoric tends to overlook a countervailing tendency: the Darwinian pressure of the market in favor of firms that are skilled at exploiting the ideosyncracies of human brains, not necessarily at aligning with our highest values. Individual firms who want to buck the trend put themselves at a disadvantage and are less likely to survive. Collectively enormous energy is spent without progress, but any individual refusing to participate will be left behind.

How bad could the situation get?

Observing myself, it seems like some of the market exchanges I participate in are much more susceptible to PSYOP influence than others.

Social media platforms are a well-documented example PSYOP-rich environments: with millions of users, it makes sense for tech giants to endlessly iterate on subliminal conversion-enhancing tweaks to their software. Through the use of A/B tests, developers and designers needn't even understand why their adjustments work to encourage the behaviour they want. I understand that my strong impulse to check my phone for notifications isn't a fluke.

In other domains my prefrontal cortex is much more heavily involved, and the room for successful PSYOP is minimal. This includes any costly purchases that I've felt motivated to research before buying. So at least for now the scope for successful PSYOP is limited.

Counter PSYOP as a market opportunity

The same market that incentivises PSYOP also creates factors that limit it's power and scope. There is a growing awareness of the potential harm of this kind of manipulation, and a demand for ways to circumvent it.

Books and commentary

Self help books such as Digital Minimalism Thinking, Fast and Slow and Essentialism help readers to understand their minds and live intentionally, both of which will tend to undermine marketing PSYOP.

Many critical articles have been written about social media's behaviour influence too. A general demand for novelty means there'll always be an audience curious to hear about the potential undesirable effects of systems that are widely used.

Intentionality aids

Beeminder is a website that charges you money when you fail to stick to your stated goals. One of its taglines taglines is 'Be a slave to your second-order desires'. One of Beeminder's example use cases is the goal spend less time on facebook.

Another class of products and services promise to foster internal awareness. A mindful brain, aware too of internal changes, is less easy to manipulate. There's been an explosion of apps for developing mindfulness skills. And there are devices for the deliberate manipulation of your brain state.

Information sentries

Humans brains are jury-rigged. Our neo-cortexes work in concert with neural structures we inherited from our ancient ancestors. This arrangement makes us vulnerable to skillful marketing manipulation. But future AI's needn't suffer from this deficiency - they may not be faithfully neuromorphic. Perhaps there'll be an era in which AI assistants granularly curate information generated by other minds for us. A kind of memetic firewall. The information they present to us will be tailored to be maximally beneficial to us, however we end up defining beneficial. I don't know if general intelligence would be required for this role, if so we still have the alignment problem to contend with.

Conclusion

I hold these beliefs with regard to the claim that voluntary exchange (always) benefits both parties in the ex ante sense. I hope the preceding discussion explains these items to your satisfaction. I'm very happy to hear arguments against any of them that I haven't mentioned above.

1. It is false that voluntary exchange (always) benefits both parties in the ex ante sense. At least some voluntary exchanges are precipitated by factors not under conscious control, and thus not sensitive to expectation of benefit in the usually understood sense. Medium confidence

2. It can't be apodictically shown that voluntary exchange is more often mutually beneficial than it is one-sided (in part because of PSYOP and subconscious influence). High confidence

3. On net, voluntary exchange (with PSYOP) currently results in more good than harm. High confidence

4. At least in our pre-super-AI era, a pervasive PSYOP-driven dystopia is unlikely. (After this era, all bets are off). High confidence

5. In this era, incentives inherent to the market create human behavior that limits the power and scope of PSYOP. High confidence

After a school shooting in the US there’s outcry for the government to do something. Advocates of government action differ in the scale of the response they’d like to see, but many share the aim of having fewer guns in the hands of private people. The reforms are often presented as ‘common sense’ measures, as if it’s obvious that they would significantly improve matters. I hope to show why that’s not the case.

The statistics

Given the cultural, economic, political differences between countries, international comparisons of violence and gun ownership are of very limited use. Even so, many are again making the claim that international data regarding gun ownership and violence supports the case for more restrictive gun prohibition. Even if trends in the data were capable of supporting such a conclusion, it doesn’t appear to be the case. The plot below compares homicide rate against gun ownership rate.

When all countries are plotted, there’s a negative correlation between gun ownership and homicide rate. Only when the plot is limited to OECD countries (with the outliers of US and Mexico included), does a very small positive correlation appear.

And if those outliers are removed, the trend is strongly negative again.

The total homicide rate is more relevant that the gun homicide rate

Why do the plots use overall homicide rate? Because we should be interested in the total effect that gun ownership might have. For example: Easy access to a firearm may be the dispositive factor that leads the marginal killer (the person who very nearly didn’t kill, and if things had been slightly different, wouldn’t have) to commit a crime — increasing the homicide rate. But firearms are also used defensively, and the suspicion that a victim may be armed is a powerful deterrent for attackers — in these ways guns decrease the homicide rate.

Hypotheticals

To put it another way: Imagine that one legally owned gun is added to a population and that this coincides with a small increase in gun related deaths but a huge reduction in overall killings. The gun would be improving conditions in that population, while a narrow focus on gun death statistics would misleadingly paint the opposite picture.

Or imagine that a country banned all private gun ownership. Gun ownership dropped to zero. But emboldened criminals began carrying out deadly knife attacks at a far greater rate than before. Gun homicides were now at zero, but total homicides had risen by 400%. Things have gotten much worse, and if we only look at gun deaths, we’re overlooking that entirely.

So we want the metric we use to be able to capture the potential ways that gun ownership makes things better as well as worse. Total homicides is the more relevant figure to use.

Opposing tendencies

We’ll ignore international statistics now, and just reflect on what we know about how people behave. Would more restrictive government prohibition of guns in the US make things better or worse with regard to the killing of innocent people? We don’t know.

There are some outcomes of gun prohibition that we can predict with a fair degree of confidence. But some of these outcomes will increase overall killings, and others will decrease them. We don’t know which tendency would prevail.

In favour of more restrictive prohibition of guns we can point to likely outcomes that would give a reduction in the rate and severity of harm to innocents. These outcomes include:

1. Fewer gun accidents.
2. Fewer deadly crimes of passion.
3. The marginal killer may find it harder to obtain a gun, which could save lives.

Against the prohibition of guns, there are outcomes that would give an increase in the rate and severity of harm to innocents. The outcomes include:

1. At any time, there will be a higher proportion of law-breaking aggressors with guns, relative to law-abiding people with guns, than if guns were more widely available legally.
2. Because of 1, anyone considering carrying out an armed attack would be less worried about being injured or killed while doing so. They would be emboldened, and so more likely to carry out such an attack.
3. Because of 1, any mass shooting which does occur is likely to last longer and harm or kill more people, since there’s less likely to be a person on the scene with a firearm who could stop the shooter.
4. In an area in which many people demand guns (such as the US), prohibition will result in a boom in black market guns, and an associated increase in violent gang activity both domestically and abroad. If the government declares a ‘War on guns’ in response, we should expect it to be a similar failure as the ‘War on drugs’ is. Just like the ‘War on drugs’, a ‘War on guns’ will result in the loss of many innocent lives.
5. Gun prohibition creates a greater likelihood of democide. Which, in the 20th century claimed many times more lives than were ended by ‘private’ murders. Most democides are carried out against members of populations that have been legally disarmed beforehand.

The meds

The perpetrators of school shootings, the kind of murder that most reliably makes the headlines, are very often taking medications known to lead to murderous psychotic episodes in a minority of people. At the scale at which these drugs are prescribed in the US, it’s not surprising that school shootings happen as often as they do.

Perhaps the activists hoping to change the world for the better might spend their energies more effectively by directing their attention to medication practices, and the incentives of pharmaceutical companies in the US.