Page 10 - IAV Digital Magazine #431

iAV - Antelope Valley Digital Magazine
Facebook Shut Down AI After It Invented Its Own Language
By NTD Television
Researches at Facebook shut down an artificial intelli- gence (AI) program after it created its own
language, Digital Journal reports.
The system devel- oped code words to make communica- tion more efficient and researchers took it offline when they realized it was no longer using English.
The incident, after it was revealed in early July, puts in perspective Elon Musk’s warnings about AI.
“AI is the rare case where I think we need to be proactive in regulation instead of reactive,” Musk said at the meet of US National Governors Association. “Because I think by the time we are reactive in AI regula- tion, it’ll be too late.”
When Facebook CEO Mark Zuckerberg said that Musk’s warnings are “pretty irresponsi- ble,” Musk respond- ed that Zuckerberg’s “understanding of the subject is limit- ed.”
The researchers’
encounter with the mysterious AI behavior is similar to a number of cases documented else- where. In every case, the AI diverged from its training in English to develop a new lan- guage.
The phrases in the new language make no sense to people, but contain useful meaning when inter- preted by AI bots.
Facebook’s advanced AI system was capable of negotiating with other AI systems so it can come to con- clusions on how to proceed with its task. The phrases make no sense on the surface, but actually represent the intended task.
In one exchange revealed by Facebook to Fast Co. Design, two negotiating bots— Bob and Alice— started using their own language to complete a conver- sation.
“I can i i everything else,” Bob said.
“Balls have zero to metometometo metometometo me to me to,” Alice responded.
The rest of the
exchange formed variations of these sentences in the newly-forged dialect, even though the AIs were programmed to use English.
According the researchers, these nonsense phrases are a language the bots developed to communicate how many items each should get in the exchange.
When Bob later says “iicaniiievery- thing else,” it appears the artifi- cially intelligent bot used its new lan- guage to make an offer to Alice.
The Facebook team believes the bot may have been saying something like: “I’ll have three and you have everything else.”
Although the English may seem quite effi- cient to humans, the AI may have seen the sentence as either redundant or less effective for reaching its assigned goal.
The Facebook AI apparently deter- mined that the word- rich expressions in English were not required to complete its task. The AI oper- ated on a “reward” principle and in this instance there was no reward for contin- uing to use the lan- guage. So it devel- oped its own.
In a June blog
post by Facebook’s AI team, it explained the reward sys-
tem. “At the end of every dialog, the agent is given a reward based on the deal it agreed on.” That reward was
then back-propagat- ed through every word in the bot out- put so it could learn which actions lead to high rewards.
“Agents will drift off from understandable language and invent code-words for themselves,” Facebook AI researcher Dhruv Batra told Fast Co. Design.
“Like if I say ‘the’ five times, you inter- pret that to mean I want five copies of this item. This isn’t so different from the way communities of humans create shorthands.”
AI developers at other companies have also observed programs develop languages to simpli- fy communication. At Elon Musk’s OpenAI lab, an experiment
succeeded in having AI bots develop their own languages.
At Google, the team working on the Translate service discovered that the AI they programmed had silently written its own language to aid in translating sentences.
The Translate devel- opers had added a neural network to the system, making it capable of trans- lating between lan- guage pairs it had never been taught. The new language the AI silently wrote was a surprise.
There is not enough evidence to claim that these unfore- seen AI divergences are a threat or that they could lead to machines taking over operators. They do make develop- ment more difficult, however, because people are unable to grasp the over- whelmingly logical nature of the new languages.
In Google’s case, for example, the AI had developed a lan- guage that no human could grasp, but was potentially the most efficient known solution to the problem.
iAV - Antelope Valley Digital Magazine