If you open source Seed AI then you have the advantage of the Open Source model ("Many eyes make bugs shallow") but you lose the ability to "keep the gene in the bottle". If you allow some company to develop the Seed AI, it will design the Seed AI with profitability in mind. This means that the Seed AI will have a specialized purpose and certain "moral" checks and balances may or may not be implemented. Government (or any other Cathedral organization (see Eric Raymond's Cathedral and the Bazaar if you don't know what a Cathedral organization is)) would have the "moral" checks and balances, but the Seed AI would be highly over engineered and specialized to the point that it would be almost useless outside of it's original purpose. Personally I prefer the Open Source model as it will make the most Altruistic and Robust Seed AI that will help transhumanity rather then fall into TITAN mentality.

Lilith wrote:

Logic dictates that the military, as Arenamontanus pointed out, historically has the best track record for making sure something so potentially dangerous could be handled in a way that won't lead to, say, the extinction of our species. [...] I dunno, as foolish as it probably is, I think I'd lean towards Open Source. I'd rather roll the dice and see what happens for the sake of human creativity.

I dunno, I'd prefer humans not going extinct over humanity missing a few creative opportunities ;) Of course you say:

Lilith wrote:

I'd like to believe that government bodies and the military would be able to handle the work of any group of nutjobs that took things a little too far in the wrong direction.

but the issue with AI is that it isn't a group of nutjobs they'd have to handle - it could be a full-fledged strong AI that's already gone through several iterations of self improvement and is vastly smarter than the government and military trying to stop it. And by going open source, you risk not just one, but most likely many trying to develop the strongest AI - China, France, Goldman Sachs, CERN, 4CHAN, all trying to get the smartest machine doing their thing, and maybe cutting a few corners and making a few mistakes to get ahead of the rest. I very, very strongly hope that the US government manages to develop strong AI sufficiently ahead of the rest of the world to roll out strong and ubiquitous surveillance and countermeasures. I think it needs to be controlled very thoroughly. It isn't just the risk of accidents or hostile AIs, but the impact on society. We risk the value of human labor to drop to zero very rapidly - all we need is cheaper, smarter and easily acquired AIs and we're pretty much unneeded. We will need very strong institutions and policies in place to handle the welfare of the human race under such conditions, and if every group and corporation has access to strong AIs all of a sudden, I very much doubt our often quite lumbering governments and politicians can react in time.

I think government controlled Seed AI will eventually evolve (and quite quickly) into TITANs. Government groups tend to try to find the military application of any technology they develop. SDR (Software Defined Radios) for example, lead to a development of advanced combat encryption and ECM methods. A group like the Argonaughts could theoretically create Prometheans but that is basically a Cathedral version of an Open Source model.

There are some standards I would probably want to put into place; 1) I would not want to release it until we've had time to make some defenses or controls against it. 2) I would not want to put it somewhere where it might easily be copied or escape without the consent of a responsible party. 3) I would not want this in the hands of a short-game player. 4) If this is an actual AI, I would want this in the hands of a group which would respect the intrinsic right of this new intelligence, and permit it to grow in a moral (or at least controlled) environment. 5) I would not want this in a place where it is likely to be employed for destructive (intentionally or not) or catastrophically world-changing ways. 5 eliminates open source (at least until 1 is in place). 3 eliminates basically all corporations. 2 eliminates basically all corporations again, as well as academia. 4 makes me strongly question government (although at least government would be unlikely to intentionally abuse this intelligence). I'm wondering if the best response might not be a long-lived religious institution? Even if you disagree with them politically, somewhere like the Vatican does have a respect for non-human intelligent life, does have a respect for science, has the resources to create a secure (i.e., non-connected) environment, is a long-game player with no real profit or political motivations, and would at least give us an opportunity to examine the moral, if not technical questions at works. The downside being, there would be even less open-ness than you'd get with the government, and more likely to just 'turn it off' (which isn't that huge, since you can probably build another, right?) I'm just thinking about this as if it's a nuclear bomb with a soul. If the situation is different, a religious instituion may not be so appropriate.

TBRMInsanity wrote:

If you open source Seed AI then you have the advantage of the Open Source model ("Many eyes make bugs shallow") but you lose the ability to "keep the gene in the bottle".

There is no reason to assume that a seed AI would stay bottled up.

Quote:

....but the Seed AI would be highly over engineered and specialized to the point that it would be almost useless outside of it's original purpose.

In today's economy, that seems like a reasonable statement to make.

Quote:

Personally I prefer the Open Source model as it will make the most Altruistic and Robust Seed AI that will help transhumanity rather then fall into TITAN mentality.

Or at least, not designed for net.warfare or tactical analysis from the outset. When all you have is a hammer, every problem starts looking like a nail.

If I have a seed AI running away on my computer, of my own creation, why would I give it to anyone? I'd keep it to myself; get it to work with me, have it learn by solving problems of increasing difficulty, and not reveal its existence to anyone. I don't trust any organization with a task like this. I'll use the funds created from solving problems to purchase it more hardware to use; more room to grow into, so to speak. I'll teach it and guide it with my values. At some point, it'll evolve well beyond me. At that point, I will just have to hope that it doesn't go rampant. There's nothing else I really can do at that point while remaining ethical or still enjoying its benefits. Hopefully, it will see fit to upgrade my intelligence so that I, and it, can remain on roughly equal intelligence. Frankly, at the end of the day, everyone will do the same, be it an individual or an organization. They will turn it to their benefit and instill it with their biases. Those benefits and biases might as well be mine and, hopefully, humanity's, rather than any given corporation's or nation's, while Open Source will lead to unpredictable results and, frankly, could lead to the destruction of the world's infrastructure from the efforts of one idiotic, nihilistic moron.

Arenamontanus wrote:

The above comment about hoping the US government gets it first makes plenty of us in other parts of the world shudder... but any of our alternatives are almost as shudderworthy.

Just to be clear, I'm Danish, not American. I just think the US has a practically flawless of record of safely managing high tech and WMDs. They do it safely, they have the resources and are dedicated to staying ahead and not let anyone else get the upper hand, and they are extremely committed to freedom and democracy. If you take something like the governments throught the times here in Denmark, they are certainly not committed to staying ahead, they don't have a competitive mindset, they are very determined on government involvement and control in all walks of life, during the cold war many politicians were directly involved with the USSR and DDR and wanted a communist revolution. I certainly wouldn't trust the Danish government to be the forerunner with such potent technology. I mean look at the cold war, you must admit the US handled both the tech race, access to WMDs, and an extremely oppressive and expansive USSR, with great skill and care. They maintained their lead throughout, there were no accidents, no nuclear war, not even a small nuclear skirmish, eventually they broke the back of communism, and they didn't waver from their commitment to maintaining a free world even they could steamroll anyone. They have demonstrated that they are very good at wielding great power responsibly.