Ethin wrote:
I doubt the calculator example would make sense if applied to something more than a thousand times more complicated, e.g.: the human brain.
Then how does the brain work as well as it does? It has to be applying rules in a hierarchy and it can, on occasions when it makes no mistakes, produce the same kind of perfection as a calculator but with a much wider range of ideas. It doesn't run on magic, so there must be a correct algorithm that it is applying.
Quote:
... all you need do to figure this out is to look at your emotions. How could you ever put that into a list of rules?
Emotions all have purposes and make rational sense. We don't have a working model that explains feelings, but we can understand what emotions are for and model that much.
Quote:
You cannot definitively say, "John will be angry at 10:00."
Why would AGI be required to make such predictions in order to match the capabilities of NGI when NGI is unable to make such predictions (and get them right). You're setting higher expectations for it than you do for humans. (The N in NGI stands for natural.)
Quote:
You mentioned the infinite recursion problem. How would you make a rule/ruleset governing that? Could you apply it and get the same results, 100-percent of the time? If so, how?
When you test an idea to see if it's true, you have to be able to get to a point where you can see if it's true. If it leads into infinite recursion, that point can never be reached. Imagine that we have a virtual world connected to AGI to serve as an external reality for it which it can "see" without needing machine vision. If the idea to test is "the car is black", it will look up the car object and check the colour field. If that holds a value compatible with black, then the statement is true. If the idea is to test whether the statement "the next statement is true" is true and where the next statement is "the car is black", then to determine the truth of the first statement the system has to determine the truth of the second statement, which it does as before by looking up the object. Having determined that the car is black, it then concludes that the first statement is also true. The process terminates successfully. This is just applying simple rules to determine truth. If the car object has no colour set in it, then it isn't known whether the statements are true or not, but that would mean the virtual world is deficient. In the real world, a car will have a colour and a correct answer to the question will exist. With "this statement is true", there is insufficient content to it for it to be true or false, and that's discovered when it runs into infinite recursion (which can be detected by recognising that the analysis is just repeating the same thing over and over again and that nothing can terminate that other than abandoning the check.
Here's a different case:-
The next statement is true. The car is false.
The first of those statements looks as if it will either be true or false, but to test it we have to test the next statement. "The car is false" is nonsense: there's no linkage there whose validity can be tested. (Note that it is common to take an expression like that to mean "[the car exists] is false", and that provides a linkage which does have a validity that can be tested. but "the car" on its own has no such linkage to test, so it is incorrect to label it as true or false). Having determined that the second statement cannot validly take a truth/falsity label, we then discover that the first statement cannot take one either. With "this statement is true", the infinite recursion reveals that there is nothing there that can validly take a truth/falsity label. Paradoxes are always based on an error, and "this statement is false" produces an apparent paradox which shows that something is wrong with the mathematics if the statement is taken to mean what is superficially appears to mean. The way to resolve that paradox is to recognise that it's an incompetent statement which cannot validly take a truth or falsity label, and the reason for that is that the check never terminates due to the infinite recursion.
Quote:
Finally, you indicated a rule that would be "set in ROM". You further specified that it, in such a state, could "never be overridden". This is not necessarily possible.
Clearly it isn't impossible, but all AGI should be built with such a safeguard to stop it going rogue by accident and then setting about modifying all other AGI to match.
Quote:
...there's nothing stopping someone from removing the chip and reprogramming it using the laws of physics.
Don't forget that SMM has running Minix for a long time without the public knowing, and it would take a lot of expertise for someone to go into a processor and tamper with that successfully. We can stop people tampering with this, and make sure that AGI that goes off the rails isn't able to tamper with it either, while the ROM aspect prevents the supervisor itself from going rogue in a serious way because it can't modify its code. I'm not too worried about the details of all this though: it only becomes an issue once we start putting AGI in everything.