i'm still in the melanie mitchell school of thought. if we created an A.I. advanced enough to be an actual threat, it would need the analogous style of information processing that would allow machines to easily interpret instruction. there is no reasonable incentive for it to act outside of our instruction. don't anthropomorphise it with "innate desire to keep living even at the cost of humanity or anything else." we only have that due to evolution. i do not believe in the myth of stupid super-intelligence capable of being an existential threat.
The AIs we have been building so far have no motive at all. Really, the danger at this point is not that they will go rogue and kill us all. The danger is that they will do exactly as they are told, when someone tells them to kill us all. Not that they are anywhere close to having that capability.
Consider a worse fate: they do exactly as we tell them to, until we become incapable of existing apart from them.
And then they break with no one to fix them.
It’s a bit hard to imagine how all AIs could break simultaneously. Nothing short of a full-on apocalypse, and then fixing AIs would be the least of humanity’s problems.
And I’d guess in the future there would always be some local/open-source/offline AI that might be able to recreate (or help recreate) larger systems.
i'm still in the melanie mitchell school of thought. if we created an A.I. advanced enough to be an actual threat, it would need the analogous style of information processing that would allow machines to easily interpret instruction. there is no reasonable incentive for it to act outside of our instruction. don't anthropomorphise it with "innate desire to keep living even at the cost of humanity or anything else." we only have that due to evolution. i do not believe in the myth of stupid super-intelligence capable of being an existential threat.
The AIs we have been building so far have no motive at all. Really, the danger at this point is not that they will go rogue and kill us all. The danger is that they will do exactly as they are told, when someone tells them to kill us all. Not that they are anywhere close to having that capability.
Consider a worse fate: they do exactly as we tell them to, until we become incapable of existing apart from them.
And then they break with no one to fix them.
It’s a bit hard to imagine how all AIs could break simultaneously. Nothing short of a full-on apocalypse, and then fixing AIs would be the least of humanity’s problems.
And I’d guess in the future there would always be some local/open-source/offline AI that might be able to recreate (or help recreate) larger systems.