Artificial intelligence has beaten three of the world’s best human drivers on three different tracks in the racing video game Gran Truism Sport, by gaining ground at the most difficult parts of a track.
The AI, named GT Soph, was able to execute tactical moves such as using an opponent’s slipstream to boost itself forwards and block its opponents from passing.
Peter Burman at Sony AI in New York and his colleagues trained the system using deep reinforcement learning, a type of machine learning that uses rewards and penalties to teach the AI’s neural network how to win. During training, GT Soph, which was running on a separate computer, played the game on up to 20 PlayStation 4 consoles simultaneously.
The team gave the AI the ability to accelerate, brake and steer, along with real-time information on the position of the cars in the game, including its own, and a map of the next 6 seconds of the track, which meant sight of a longer distance ahead when the AI travelled faster. The researchers note this information isn’t available to human players, giving the AI a slight advantage, but say that people have other advantages, such as the ability to change gears manually, which the AI didn’t have.
Burman and his team rewarded GT Soph for staying on the course and driving faster, and penalized the AI for veering off course or slowing down. Within a couple of days, the AI learned to complete the tracks faster than 95 per cent of human players, compared with rankings from online leader boards.
Over around nine more days of training, GT Soph shaved tenths of a second off its lap times over a total of 45,000 driving hours to finish faster than any human player on the leader boards.
However, GT Soph hit a stumbling block. The AI initially avoided overtaking fast opponents in order to minimize the risk of collision and maximize rewards. To overcome this overly safe behavior, researchers rewarded the AI for passing opponents and penalized it for being overtaken.
By joining forces with a competitive GT player, the team focused on a small set of the most difficult parts of each track, so that GT Soph could quickly learn to excel in those areas. The researchers then challenged four of the world’s best GT players to compete against four copies of GT Soph in a team race, so eight cars were on the track in all. The AI won 104 to 52, with points calculated according to racers’ final positions.
“The results suggest that it could be possible for game developers to use deep reinforcement learning to design and test their games, and to produce interesting opponents and teammates for human players,” says Igor Babushka at OpenAI in San Francisco.
The findings will have little impact on improving autonomous vehicles, which must sense the environment themselves and navigate more variable conditions.
“While GT Soph managed to achieve impressive results, it’s very much dependent on fine-tuned parameters and features specific to the game,” says Georgios Yannakakis at the Institute of Digital Games in Malta.
You must be logged in to post a comment.