Facebook engineers have developed a new method to help them identify and prevent harmful behaviors, such as users who spread spam, cheat others, or buy and sell weapons and drugs. They can now simulate the actions of bad actors using AI-powered bots by letting them lose on a parallel version of Facebook. The researchers can then study the behavior of the bots in the simulation and experiment with new ways to stop them.
The simulator is known as WW, it is pronounced “Dub Dub” and is based on the actual Facebook code base. The company released a document on WW (named after the simulator is a truncated version of WWW, the global network) earlier this year, but shared more information about the work at a recent roundtable.
The investigation is being led by Facebook engineer Mark Harman and the company’s artificial intelligence department in London. Speaking to journalists, Harman said WW was an enormously flexible tool that could be used to limit a wide range of harmful behavior on the site, and he set the example of using simulation to develop new defenses against scammers.
In real life, scam artists often start their work by hovering around user friendlies to find potential brands. To model this behavior in WW, Facebook engineers created a group of “innocent” bots to act as targets and trained several “bad” bots who scoured the net to try to find them. Then, the engineers tried different ways to stop the faulty bots, introducing various restrictions, such as limiting the number of private messages and posts that the bots could send every minute, to see how this affected their behavior.
Harman compares the work to that of urban planners trying to slow down on busy roads. In that case, engineers model traffic flows on simulators and then experiment with introducing elements like speed bumps on certain streets to see what effect they have. WW simulation allows Facebook to do the same but with Facebook users.
“We apply ‘speed reductions’ to the actions and observations our bots can take, and we so quickly explore possible changes we could make to products to inhibit harmful behavior without harming normal behavior,” says Harman. “We can scale this up to tens or hundreds of thousands of bots and therefore, in parallel, search for many, many different possible […] restriction vectors “.
Simulating the behavior you want to study is a fairly common practice in machine learning, but the WW project is notable because the simulation is based on the actual version of Facebook. Facebook calls its approach “web-based simulation.”
“Unlike a traditional simulation, where everything is simulated, in web-based simulation, the actions and observations are carried out through the actual infrastructure, making them much more realistic,” says Harman.
However, he emphasized that despite this use of real infrastructure, bots cannot interact with users in any way. “They can’t really, by construction, interact with anything other than other bots,” he says.
Remarkably, simulation is not a visual copy of Facebook. Don’t imagine scientists studying bot behavior in the same way that you might see people interact with each other in a Facebook group. WW does not produce results through the Facebook GUI, but records all interactions as numerical data. Think of it as the difference between watching a soccer game (real Facebook) and just reading the game statistics (WW).
At this time, WW is also in the investigation stages, and none of the simulations the company has conducted with bots have produced any real-life Facebook changes. Harman says his group is still running tests to verify that the simulations match real-life behaviors with enough fidelity to justify real-life changes. But he believes the work will result in modifications to the Facebook code by the end of the year.
Certainly, there are also limitations for the simulator. WW cannot model user intent, for example, nor can it simulate complex behaviors. Facebook says bots search, make friend requests, leave comments, post, and send messages, but the actual content of these actions (like the content of a conversation) is not simulated.
Harman says that WW’s power, however, is its ability to operate on a large scale. It allows Facebook to run thousands of simulations to verify all kinds of minor changes to the site without affecting users, and from there, it finds new behavior patterns. “I think the statistical power that comes from big data is still not fully appreciated,” he says.
One of the most interesting aspects of the job is WW’s potential to uncover new weaknesses in Facebook’s architecture through bot actions. Bots can be trained in various ways. Sometimes they are given explicit instructions on how to act; sometimes they are asked to imitate real life behavior; and sometimes they are only given certain goals and left to decide their own actions. It is in the latter scenario (a method known as unsupervised machine learning) that unexpected behaviors can occur, as bots find ways to achieve their goal that engineers did not predict.
“Right now, the main focus is training the bots to mimic things that we know are happening on the platform. But in theory and in practice, bots can do things that I do not have seen before, “says Harman.” That’s really something we want, because ultimately we want to get ahead of bad behavior rather than continually playing to catch up. ”
Harman says the group has already seen some unexpected behavior from the bots, but declined to share any details. He said he did not want to give hints to scammers.
