Researchers at Facebook quickly shut down at artificial intelligence (AI) programme after they discovered that it had created its own language.
Researchers took the programme offline after they realised that the system had stopped using English, and had developed code words instead to make communication more efficient, reports The Epoch Times.
Facebook's experience echoes similar cases documented elsewhere, in which the AI diverged from its training in English to develop a new language in every case. Although the new language makes no sense to people, it was easily interpreted by other AI programmes.
In an exchange revealed by Facebook to Fast Co Design, two negotiating bots – Bob and Alice – started using their own language to communicate among themselves.
"I can i i everything else," said Bob.
"Balls have zero to me to me to me to me to me to me to me to me to," Alice responded.
According to the researchers, the phrases seemed like nonsense, but were in fact a language the bots developed to communicate how many items each should get in the exchange. The bots carried on conversing in their new dialect for the rest of the conversation, even though they had been programmed to use English.
The researchers believe that when Bob later said: "i i can i i i everything else," he was trying to communicate something like: "I'll have three and you have everything else." The researchers believe that the AI deemed their dialect more efficient for communication among themselves than standard English.
The Facebook AI operated on a "reward" principle, and therefore may have developed a more efficient communication method to achieve its intended task. A June blog post by Facebook's AI team explained the reward system. "At the end of every dialogue, the agent is given a reward based on the deal it agreed on." The reward was then connected to the language used by the bot so it could learn which dialogue outputs led to the highest rewards.
Facebook AI researcher Dhruv Batra told Fast Co Design: "Agents will drift off from understandable language and invent code-words for themselves."
"Like if I say 'the' five times, you interpret that to mean I want five copies of this item. This isn't so different from the way communities of humans create shorthands," he added.
AI developers at other companies have also observed programmes develop languages to simplify communication. At Elon Musk’s OpenAI lab, an experiment succeeded in having AI bots develop their own languages.
At Google, the team working on the Translate service discovered that their programmed AI had invented its own language to aid in translating sentences, all by itself. The Translate developers had added a neural network to the system, making it capable of translating between language pairs that the system had not been explicitly taught.
The incident brought into sharp focus Elon Musk's warnings about AI in the past. At a US National Governors Association meeting, Musk said: "AI is the rare case where I think we need to be proactive in regulation instead of reactive. Because I think by the time we are reactive in AI regulation, it will be too late."
When Facebook CEO Mark Zuckerberg said that Musk's warnings were "pretty irresponsible" Musk responded that Zuckerberg's "understanding of the subject is limited".
Although these unforeseen AI divergences do not herald a machine takeover, they do make it more difficult to develop AI systems since people are unable to grasp the overwhelmingly logical nature of the new language.
In Google's case, for example, the AI had developed a language that no human could grasp, but was potentially the most efficient known solution to the problem.