Alkahest wrote:

I'm more specifically thinking about the philosophy known as abolitionism (http://en.wikipedia.org/wiki/Abolitionism_%28bioethics%29), which advocates making all feeling beings happier (using SCIENCE!) and the idea of "moral enhancement" (http://philosophynow.org/issues/91/Moral_Enhancement), which advocates making humans nicer (using even more SCIENCE!). If one were to try to find a place for these ideas in the world of Eclipse Phase, where would it be? How could one include such themes in this game of ideas?

Abolitionism seem to be implemented to a large degree already. From Splicers and up, morphs are engineered for physical and mental health. People don't feel bad just because they're born grumpy, or prone to depression, or with a weak immune system and poor health. As for the more extreme versions, maximizing happiness is a very dangerous goal. Would you want to become a wirehead (wires in your brain stimulate your pleasure center, so you experience eternal bliss)? Most people wouldn't. Also, while people say they want happiness, they very often do quite little to actually become happy. Having children is a common example of something that all research shows make you significantly less happy, but we still do it. Ambition and working a lot is also a source for major unhappiness, and the rewards even if you succeed tend to do very little for your happiness. I've done both with my eyes open and I honestly don't value happiness that much. Would we really want to maximize happiness when it would mean we stopped reproducing and striving hard for things? Such a society would fail. Would I choose that individually, strip out my drives for other things than happiness? Hell no, I don't want to be a childless loser. If anything, I'd consider rewriting my brain so I'd also enjoy happiness at taking care of my children and working hard (imagine if you actually liked being woken at 3AM to comfort your child, and pulling an all-nighter at work was as fun as playing computer games). But from the descriptions of EP morphs, I suspect they're somewhat like that already - and note that from the outside, you'd observe the opposite of what you'd expect from an abolitionist society, as you'd see people doing MORE things that you regard as causing unhappiness! Moral enhancement is at best a completely silly idea, at worst utterly terrifying. We don't individually want to improve our morals (which I assume goes beyond someone just getting a fix allowing them to resist the desire to shoplift because they know it will mess up their life). If I think being tolerant towards religion is a immoral, why would I change that? If a fundamentalist thinks gay marriage is wrong, why would he change that? If I believe that economic growth is more important both for me and for the world's desperatedly poor than slowing global warming, why would I change that? Would I willingly give up my ability for deception? The answer to all these questions are of course "no". I'd love for a lot of other people to modify their morals though. Imagine a world without socialism, religion, and environmentalism, how much better off everyone would be. As a tool for oppression it is certainly tempting - and unsurprisingly, the idea of moral enhancement seem to come with left wing ideas, though of course any oppressive regime or fanatic would love it too. As a libertarian, I certainly can't be anything but horrified at the idea. It's the ultimate weapon, and should be treated as such. It is a good alternative to other horrifying things like life imprisonment of criminals and dropping bombs on Taleban soldiers, but if something like that is ever used, it is important that we retain the same degree of horror at its use as when we kill people. There's a risk that its invisible effects could make us tolerant of its use and allow us to go on a slippery slope, and we need to be aware of that. In certain circles, I could imagine moral enhancements being used as a form of moral signalling though. You could prove that you were a devoted and uncorruptable patriot, or that you actually did put animal rights equal to humans, or whatever. Peter Watts' Rifter trilogy had extensive use of such modifications. The people charged with the really tough decisions (like firebombing an area and killing everyone in it to prevent the spread of virulent disease) had their brain chemistry modified so they were compelled to act in the way that maximized population utility, so they could be allowed to act quickly without oversight. The moral tweaks they give spies were quite a bit more sinister...

Arenamontanus wrote:

I did a brief write-up of the abolitionists at http://www.aleph.se/EclipsePhase/Pleasure.pdf I think abolitionism as an individual philosophy can be found nearly anywhere in the non-conservative parts of the solar system (it is likely not too well received among Ultimates or Jovians).

I'm not sure it goes down that well in other parts of the system either. The common desire for enjoying life and experiencing happiness is not abolitionism. Abolitionists take it to the next level. Abolitionists won't do common things like have children for example (unless they fix their brains to actually enjoy all the work, obligation and worry that comes with children). Abolitionists will get the implants you mention in your write-up, which all have effects that most would find detrimental. They're likely to end up as junkies or moochers, or the equivalent, aren't they? Even in the tolerant autonomist habs they're unlikely to like people who don't pull their own weight. Extropian and inner system habs they'll tend to become poor. Perhaps the real abolitionists are inner system retired wealthy people :)

Quote:

Moral enhancement is intriguing. This could be something both the PC and some autonomist groups pursue: the PC have culture experimentation projects and worry about social cohesion, and autonomists really need to influence transhuman nature to make their societies scale up better. Plenty of tech around to implement it, from psychosurgery to drugs to moral enhancement software in muses.

But why would individuals use it? Peer pressure? Unless you have eusocial morals to begin with, would you really want to do that to yourself?

Smokeskin wrote:

Abolitionists will get the implants you mention in your write-up, which all have effects that most would find detrimental. They're likely to end up as junkies or moochers, or the equivalent, aren't they?

The whole point of abolitionism is to get rid of pain and maximize pleasure *without failing at life*. The implants I mention are clearly not quite right for the job, but I would expect abolitionist-minded researchers working on fixing that. Remember that pain control is already fairly well in hand in EP: people can live pain-free lives with great success, so a fraction of the whole program is already done. Fixing suffering is a deeper problem that still needs a fix, although EP medicine is clearly fairly good at it. I can totally see abolitionists working on very serious projects in Cognite, finding their ideology compatible with both corporate profit and helping suffering people. It would be fun to do a game with abolitionist exhumans. They are junkies, but they get their fix from the inside, so they can be highly active in the world - their motivations are about different kinds and intensities of pleasure. Every action is a delight, even failure is an amusing challenge to overcome ("Oh, I lost another life! It is so bothersome to have to respawn in this game"). Hmm, maybe they are trying to acquire as much sensory information as possible to run through their illegal hedonium mainframe hidden on some asteroid. So now they try to rob or infiltrate XP repositories... or steal infugee egos whose memories can be experienced (don't worry, the egos get to live in the mainframe too - in absolute bliss). "We are not selfish and we are not lazy. You are free and welcome to join us, but you need to work at learning how to control a turbo-charged motivation system. I spent a year realtime in accelerated simspaces figuring out how to function as an agent: every time I failed we froze the fork and rolled back the ego a bit. There are literally tens of thousands of me trapped in bliss in that server, and I plan to ensure they continue to enjoy forever once we spread hedonium across the universe. Does that horrify you... or does it *excite* you?"

I think we should distinguish between Rational enhancements: Enhancements that improves your self discipline, reduces your cognitive biases, improves your ability model the effects of your actions, etc. These don't modify your moral compass or utility function, only your ability to act according to it rather than lead your personal weaknesses lead you astray. It basically increases your ability to act rationally. Moral enhancements: Modifications to your moral compass and utility function. With this, your ideas of good and bad change. You used to want to fly to the Maldives for vacation, now you feel better about reducing your carbon footprint. Happiness enhancements: This should perhaps be a subgroup of the rational enhancements, but it is still slightly different. This simply aligns your happiness response to your utility function. If you want to be a special forces soldier, you'll make it through the grueling exercises not through sheer willpower but because you have exceptional willpower, but because you're enjoying every second of it. While this one seems to let everyone be an abolitionist for free, or let abolitionists act like everyone else, you need to be careful with this one as it seems extremely susceptible to unintended consequences! Actively enjoying the stress of special forces training could be detrimental to your health. If we muddle them up, we're not talking about the same thing. I've certainly railed against moral enhancements, only to be met with the wonders of rational enhancements which I'm all for. We're inadverdently going to make straw men if we don't define our words.

Alkahest wrote:

As for other people pursuing objectives that do not make them happier, well, humans are well-known to not be very rational creatures.

It is a mistake to think that the pursuit of happiness is rational. We value a great many things that are othogonal or even opposed to happiness, things like children, ambition, pride, duty, responsibility. Happiness is only one part of your average human utility function (and I'm not talking about the revealed preference utility function which is tainted by many irrational actions). As I see it, the difference between abolitionists and everyone else is that the abolitionists value happiness much more than everything else. There are actual differences between the utility functions of humans and abolitionists, and is not just a matter of lacking the right rational enhancements.

If we're talking revealed preference utility functions, I completely agree. That's pretty much just defined as the function that's maximized by what we actually do, and what I called rational enhancement will change our actions. But let us assume that a perfect rational enhancement exists. With it you would remove all your biases, you'd be able to model your future mental states and sum their utility over time with correct discounting, you had perfect self discipline and the ability to set up mental commitment devices, etc. This is a very different human being for sure, but it would have a utility function left still that contained the essence of your moral compass (among other things). Something that changed that essential utility function would be a moral enhancement. I don't doubt at all that my utility function would change with perfect rational enhancement. Of course some of the things I think I want are only because of my muddled thinking, limited mental capacity and biases, and with clarity I'd want other things. But clarity wouldn't change everything. If that was the case, there'd be only two options left: 1) Perfect rationality came with its own utility function 2) Perfect rationality is only possible without any utility function Option 2 seems non-sensical (since imo the definition of perfect rationality is to be able to maximize the expected value of your utility function). Option 1 would solve a question in Friendly AI research, where some people believe that any sufficiently advanced agent would be cooperative, compassionate, and so on - a sentiment I don't share. To give a concrete example. The moral enhancement link given in the OP mentioned global warming. I'm personally a humanist (and I expect myself to become a "sentienist" and include other sentient entities once such are discovered or constructed, provided insurmountable conflicts of interests don't arise, and of course our understanding of sentience could undergo a revolution). I only place value on humans, and that means I only value nature and the biosphere to the extent that it affects humans. Contrast this to the more common environmentalist view where other things than humans are valued greatly. For example spending trillions of dollars on slowing global warming as we're doing now is obviously inhumane. The cost and resulting lack of economic growth means millions of people remain stuck in deep poverty with increased mortality and much reduced quality of life, and the scientific predictions of the difference in consequences from our current mitigation efforts does in no way justify all this misery. If you value the environmently greatly compared to humans, current policies make sense though. Another example would be animal right's activist, some of which believe that animal testing in order to develop life saving drugs for humans is immoral. These are fundemental moral issues that are independent of biases. Of course, some people would if they got a perfect rational enhancement discover that they really were humanists and their support for efforts to reduce global warming were the result of a bias, while others would discover that they actually cared about the environment and large parts of their consumption was only due to personal weakness and hedonism. But I don't know of a convincing argument for why to value humans over nature (even if I could point to some brain circuitry and say "there, right there, you're supposed to think like me" that wouldn't be a convincing argument, would it?). It seems to follow that there are utility functions that exist outside of our rational ability. And I fail to see why anyone would change their utility function. Rationally, you would at any point in time make decisions that maximized your current utility function. By changing your utility function, you would in the future make decisions that maximized the new utility function rather than your current utility function, and this would of course lead to lower expected value according to your current utility function. Ergo changing your utility function reduces the expected value, so no rational agent would do that. I actually believe that's a formal argument for why no rational agent should accept a moral enhancement, except for reasons of external pressure. For example, if my current utility function made it impossible for me to get a job and this was important to me according to my current utility function, I could be in a position where the expected value according to my current utility function would be higher when evaluated on the actions I'd take to optimize the expected value of the new function. Let F be my current utility function and x the future actions I'd take to optimize the expected value of F. Let F* be the future utility function and x* the future actions that optimized its expected value. It follows that iff F(x*) > F(x), I should change my utility function to F*. This could only be the case if x* is impossible while my utility function is F, since x is the optimal actions under F. And if you care about the integrity of your utility function (and I believe it's at the heart of what we are as individuals), that should make you even more worried about moral enhancements. It isn't just oppression that's a risk, external agents or market forces could simply incentivize you to change it.

Some very good points Alkahest. I'll need to spend some time pondering them and also on actual work and family interaction, so it might be a few days before I reply, but this page is added to my reading list so I'll get back here even if there isn't any new activity. One thought I'll leave you with though: perhaps our utility functions have or should have a negative term for strict utility function conservation over time, so we'd maximize our expected value by sometimes modifying our function? It seems highly maladaptive to have your utility function locked in like you describe (you're absolutely right that this is the logical consequence). A 2-order system, with a higher-level function that evaluates and updates your utility function, doesn't seem to solve problem as it just reappears at the higher level.