I was thinking about AI last night. Perhaps the Turing test is too tough. Consider the following experiment.
A) Get ten people, call them “questioners”, to chat via computer text to ten “entities” in turn. Tell them that these ten entities may be AI programs or people, and they have to decide which and if they think it’s an AI program give it a ranking from 0 to 10. They have limited time to chat, but can stop chatting at any time.
B) All the “entities” are people, who have been instructed to pretend to be an AI program.
The entire experiment is over in a little over an hour.
The experiment is to find out in how many of the 100 instances the questioners mistake the entities for AI programs. In what percentage of cases would you expect the questioners to mistake a person for an AI program?
If one of the ten questioners was a ring-in, an AI program, and the ten entities were all told this at the end of the chat session, how many of them would be able to spot which questioner was an AI program?
How would you brief the entities to improve their impersonation of an AI program?
In briefing the entities in accurately impersonating an AI program, my initial advice would be:
1) Don’t dawdle over the reply to a difficult chat line.
2) Avoid spelling errors, use only words you can spell, so avoid for instance “Britney Spears” as about 50% of people spell it wrongly. If you do make a spelling error, make them regularly.
3) Feel free to lie. Don’t bother sticking to the truth.
4) Except for an initial “Hello” try to put at least 5 words in every chat line.
5) AI programs have limited general knowledge, so if asked “Where would you find a pig?” don’t say “In a sty/pen”, say something generic like “Walking on the ground”, “Why would I want to find a pig?”, “Do you want to buy one?” or “I want to talk about astronomy”.
6) AI programs tend to be repetitive, so if asked the same (or similar) question three or more times make your third and subsequent response an exact duplicate of your second.
7) You don’t know what was on the News last night.
8) Pick a difficult word in the questioner’s chat and swing a reply on it. For example suppose the questioner asks “Did you like Shawshank Redemption?” you could reply “I met Shawshank once. I didn’t think much of him”.
Perhaps you could get individual entities to simulate well-known types of AI.
1) The psychotherapist AI, asks gentle friendly questions like “Why is that important to you?”, “You haven’t said anything yet about your mother”, “How does that make you feel?”, “Take a deep breath and try that again”, “I sense some anger here”.
2) The Google search AI. Very knowledgeable but slips up on questions like “Who is the first lady of the USA?” replying “Barack Obama”.
3) An AI that has a story to tell and tells it well, but totally ignores any chat from the questioner.
4) Don’t swear at all, unless you’re exceptionally good at it, in which case be a crude-bot. Make up inventive insults in every chat-line and throw them back at the questioner.
5) Have a lot of knowledge in a specialized field (eg. Movies, Music, Politics, Football, Astronomy), and pretend to be totally clueless outside that field.