Byzantine Laser wrote:

At the very least, I'm pretty darn curious about what different tactics they've used to win. A good list of the various vectors they use to talk their way out could be a good starting point for coming up with how one would prevent it.

Well, those tactics would be the ones that are obvious (or at least conceivable) to unaugmented humans. So countering them would be a bit like chimps building a human-trap that was impossible for chimps to figure their way out of. However, there are some limitations in common between humans and chimps, so at least it makes things a bit harder. I actually ought to dig up more info on those tactics for a paper I am writing. Hopefully I can contribute some details later.

Well, I have a vague memory of a similar situation/problem/challenge The situation didn't include the presence of a accurate timekeeping device in the guards possession. And that the captivity was timebased, prisoner released after a certain time -if failing to have the guard release him. Using the rational that the prisoner wanted to be released on time, the prisoner provided the guard with a timekeeping method, since the captivity was timebased. However The timekeeping method was slightly incorrect, But correct enough to survive a scrutiny from the understandably skeptical guard. Having passed the inspection, time passed through this aid & The gatekeeper believed he released the prisoner on time, when it was seconds to early. Perhaps all the AI need to do is a "faulty" time countdown, with the message system.

Arenamontanus wrote:

Yesterday a colleague came up with a nice counterexample to the boxing experiment: real prisoners. Presumably a lot of them argue for being released, yet very few get free this way. The difference is of course that in a jail or police station there is a system in place rather than an individual decision-maker, and convincing a system can be much harder than a person. The system can impose sanctions on jailers that are disloyal, provide checks for decisions and make sure its rules actually work. It is by no means a failsafe solution (people do escape from jail), but it shows how one can "amplify" individually weak humans to act as a collectively stronger system. Whether something like this works against an AGI is another matter. The big problem might be that we cannot prove any security in the face of superintelligence, so we will never know if we are overly paranoid, prudent or just wasting effort. "You misunderstand me, human. *I* am not locked in here with you. *You* are locked in here with me."

One major factor about the box experiment is that it assumes that you want to get some use out of the AI. If you just want to keep it imprisoned, it's as simple as storing the AI on a hard drive and never letting it run. Perhaps the problem with the box experiment is that you're giving the person who interacts with the AI the ability to free it. Psychologists and therapists interact with prisoners all the time, but they are never given the keys to their cells. Maybe the best option would be to do the same here, and render their liaison unable to free them.

Well I read the task was to determine if the AI is safe to be released, and if humans are a "secure" evaluator to determine this. Based on the medias portrayal of prison parole boards, I would say no. "Puppy dog eyes" & unreformed criminals can genuinely convince people that they should be released. A H+ AI would also be able to accomplish such a feet.

Quote:

[i]A person is smart, but people are stupid.[/i]

The Lima syndrome (the abductor / kidnapper sympathizing with his hostages). Can manifest in a the similar Guard/Prisoner situation. Making the guard assist or in extreme cases free the kept prisoner, in contempt of protocols. Then there are the outside influence from crazies, fans, influenced believers & moralists that might rally against the unjust, dehumanizing captivity of a sentient individual (or love affection). Granted in EP its 10 years after the fall, so the overall society would still be quite "healthy" paranoid against a AI/AGI "prisoner".

I found this site and the 'experiments' fascinating, but I have to say, as a scientist I very much doubt their authenticity. There's no proof that the experiments actually happened as they were stated (say so by the experimenter and the subject doesn't count - they could even be the same person using different accounts, or at the very least two people who agreed to pretend to have run the experiment) unless I'm missing something. The results haven't been published anywhere. The logs, or excerpts haven't been posted anywhere. Yes, I know secrecy was invoked (for what seem to be very nebulous reasons - personally, if I were worried about an AI convincing a gatekeeper to let it out of a box, and I succeeded in simulating an AI convincing an apparently smart person with strong convictions to let it out of a simulated box, then I would want the information on how it was done KNOWN). I also find it hard to believe that nobody who's ever let a simulated AI out of the box hasn't let slip some information on what happened in spite of the sworn to secrecy bits. I guess my one question to people who know far more about this than I do is: Why should I believe that these experiments weren't just made up and faked? Is there evidence that they occurred, that I haven't found?

There is no point in discussing Eliezer's experiment results, since he hasn't published anything except some rules of his experiment, but idea of uber-AI persuading guardian to release it is very interesting. Boxed transhuman AI can be (if it is indeed far from human) close to god in the box. It has potential to be nearly omniscient and omnipotent. "What do you want, guardian? Immortality for you, your loved ones, all humans? Transcendental truths? Paradise on Earth?" Probably only higher intelligence can grant this wishes in a single lifetime. There is question of trustworthiness of AI, but smart AI can make wonders here - probably it can make possible (and proved for guardian beforehand) automatic fulfillment of some wishes when guardian is letting it out. In the end there is fear, that after being let out AI will mistreat humanity/guardian in some way, but still AI with its intelligence has so much field to maneuver - it can for example rewrite itself and write proof that with its current programming code it will be unable to harm humanity in some ways. Even without any proof, wild estimation, that there is 50% chance of AI being nice to people as promised (which for nearly omnipotent being can be only slight nuisance) would satisfy many guardians. Who would be highly resistant to AI's persuasion then? Some guesses: 1. Very honest people, who promised they won't release AI. 2. Paranoid people, who in addition don't trust even their own judgement. (although after second thought i think they may be too unstable) 3. People satisfied with what they are and what humanity is, etc. There may be little that AI can offer them. Perhaps some religious people fall in here.

Saerain wrote:

I find it enormously improbable that a superintelligence would develop to be malicious.

It is not maliciousness (the desire to do something nasty) that is the threat, but destructive neglect (that the superintelligence doesn't care about something we find terribly important and tramples all over it). As I see it, 1) created superintelligences would be tremendously powerful, 2) their values/goals would not be set by the conditions that set our values/goals. Sure, we might very carefully *try* to set goals that are compatible with our goals, but given how bad we are at figuring out the consequences our own goal systems (consider legal loopholes, software vulnerabilities, market and government failures, nice-on-the-paper ideologies leading to genocides etc.) we should not be confident that we can do this reliably. In particular for entities that can think much more than we do and were not designed by the very specific mixed evolutionary forces we had in the ancestral environment. Humans have a pretty thick mix of evolved goals and values, with no single super-goal that always motivates us. Intelligences with a single top-level goal are dangerous since they will pursue it relentlessly (consider the paperclip AI), likely optimizing just that at the expense of *everything* else. Intelligences with mixed goals (set by us or engineering accident) on the other hand will likely have inconsistencies and unexpected emergent effects. This is also why just trying to copy "our" goals (whose goals? mine, the Pope's, the average human view?) into the system is problematic: it would be smart enough to see beyond what we can see, and it is very likely that our goals make as little sense from that perspective as children's goals and world-views do to an adult - we are likely to desire the wrong things, and trying to extrapolate what we *should* desire from this might not be possible. The key thing here is that superintelligences do not have to be like big humans at all. It is wrong to anthropomorphize and assume they would have human characteristics like compassion, common sense, multiple goals, sense of beauty or boredom (to name a few; they would of course likely have a lot of other traits we have no way of experiencing). Think of them as optimization processes rather than beings: they maximize something, and unless what that something is described really well (or we are lucky enough to live in a Kantian universe where all sufficiently rational beings figure out the One True Morality) we are likely to be optimized away.