A new, experimental version of the Gran Turismo Sophy AI system was just revealed at the GT World Series finals in Amsterdam, and we had the chance to sit down with the Sony AI research team to learn more about what’s new and how it works.
Unlike traditional racing game AI that follows predetermined paths with fixed behavior rules, Sophy uses advanced machine learning to dynamically adapt its driving strategy, giving it more unpredictable, human-like decision making capabilities and skills that allow it to genuinely challenge even professional drivers.
Until now, players have only been able to race against Sophy, but this new version allows players to actually control and manage Sophy’s behavior, as if they were playing the role of a race engineer.
This is an interesting development in the evolution of Sophy, and to talk more about it, GTPlanet had the chance to speak with Dr. Kaushik Subramanian and Dr. Patrick MacAlpine, two Sony AI researchers involved in Sophy’s development. Watch the full interview below, or keep reading for the highlights.
Table of Contents
- Enabling Player Control
- Slowing Sophy Down
- Sophy’s Emoticons
- Longer Races
- AI Driver Fatigue?
- AI Race Strategy?
- Sophy’s Future
Enabling Player Control
For an AI agent designed to drive by itself, suddenly adding the dynamic element of player control represents a fairly significant shift in thinking around how Sophy actually works.
“To allow for player control, we had to really innovate and update different parts of our training process,” Dr. MacAlpine explained. “For instance, we needed to add input features for tire wear and fuel consumption so that GT Sophy would be really sensitive to them. We also added reward functions so that Sophy received penalties based on certain levels of tire wear and how fuel is consumed. We also made some updates to our reward functions to enable the control of aggression and overtaking behavior.”
“One particular challenge that came up when training for this exhibition is that typically, tire wear is not really accumulated evenly across all four tires,” he continued. “For instance, sometimes there may be more tire wear on one side of the car compared to the other. We wanted Sophy to learn to robustly control the car under all driving conditions and dynamics during a race.
“To account for this variance in tire wear we needed to sort-of extend the driving scenarios that we used for training […] to really cover any potential patterns of tire wear we thought that might be encountered during a race. This is really important to make sure that the agent Sophy is equally performant and robust at different control settings […] as they may also potentially require different styles of driving.”
Slowing Sophy Down
When Sophy made its first debut, it was clear it could race — and win — against some of the best Gran Turismo players in the world. The challenge then became figuring out how to slow it down so the rest of us could still enjoy racing with it.
Various methods have been used since Sophy was released to the public in Gran Turismo 7 updates, but the dynamic nature of performance introduced by player control meant that more sophisticated techniques would need to be employed for the latest version of the agent.
“During the exhibition, human players were able to change the pace of Sophy across five different settings. Those pace settings were mapped to different levels of tire wear that Sophy was trying to conserve, which resulted in faster and and slower lap times,” Dr. MacAlpine said.
“In addition to slowing down Sophy by controlling tire wear or tire slip, we’ve also found that the pace can be adjusted by controlling how much Sophy tries to conserve fuel as well.”
Sophy’s Emoticons
Since it was first launched to the public, emoticons appear above Sophy-controlled cars to give the agent a sense of personality. Angel Inostroza, who had a chance to control Sophy during the latest demonstration, mentioned the emoticons take on a new sense of importance when considering what commands to send to it.
The emoticons, however, are actually the work of Polyphony Digital and are not direct outputs of the agent itself.
“They are set in a way where it relies on specific racing situations,” Dr. Subramanian explained. “The main purpose for that is to allow players to engage in interesting ways with the agent, and maybe interpret what the AI is doing. […] They are helpful to let the other players know what kind of aggressive mode [the agent is in] or how close or far the agent feels like it is to other cars around it.”
Longer Races
Members of the GTPlanet Forums were quick to notice the latest exhibition race was considerably longer than the current Sophy races available in Gran Turismo 7. Is that a sign of things to come?
“For longer races, elements of race strategy — in particular managing fuel and tire consumption — certainly become more important,” Dr. MacAlpine replied. “The capabilities for managing these elements demonstrated during the exhibition are an important step for Sophy’s potential to be used for longer races in the future.
“Our hope is to continue to extend GT Sophy’s capabilities in different ways, including making it proficient in longer races. However, there are several research challenges to address and we’re learning a lot from explorations like this exhibition.”
Teaching the agent to deal with the dynamics of tire wear and fuel consumption are the clear challenge.
“For the exhibition, GT Sophy is trained with both fuel consumption and tire wear turned on,” he continued. “It’s experiencing changing amounts of fuel remaining and tire wear distributions during training. Because of this, Sophy is able to learn to respond to changing driving characteristics in real time.
“By including these new training scenarios, this allows us to make sure the agent understands how to respond to these different contexts that it may encounter.”
AI Driver Fatigue?
In the classic B-Spec modes of Gran Turismo 5 and 6, players’ computer-controlled drivers had to be carefully managed: they would become “fatigued” if they were instructed to drive too fast for too long. Is this something we could see again?
Although not currently implemented in this latest demonstration version of Sophy, the researchers have considered it.
“The idea of mistakes, or performing sub-optimally, is something we’ve thought about,” Dr. Subramanian shared. “That’s definitely something that we have discussed, but there’s a lot of interesting challenges for us to work on before we get to that point.”
AI Race Strategy?
With the ability to adapt to different levels of tire wear and fuel consumption through longer races, could Sophy start to make its own AI-fueled decisions on overall race strategy?
“How to strategically manage tire usage and fuel consumption is something we’ve certainly been thinking about,” Dr. MacAlpine revealed. “The new capabilities displayed by GT Sophy during the exhibition at this World Finals demonstrate some of our exploration and experimentation on this topic.
“We’re still thinking about how to incorporate race strategy and collecting feedback from the exhibition so that we can continue to test and refine these capabilities for potential future use, and also identify experiences that will really bring new levels of engagement for the player to the game.”
It appears, however, that Sophy continues learning despite the various challenges the researchers keep throwing at it.
“What is particularly interesting for us is what the AI is able to learn under these conditions,” Dr. MacAlpine continued. “For instance, we can ask questions like ‘How many laps would the AI be able to go with a certain level of tire wear?’ or ‘How long would the fuel last if the car is driven at at the fastest pace?’ and in some cases these answers can be quite surprising to us. It turns out Sophy is able to really conserve fuel and tire wear at times, and go a lot farther than than maybe we had thought.”
“And, the fact that we can have an AI test some of this for us,” Dr. Subramanian added. “We can set a specific pace that we want for the car, or specific tire wear conditions, or specific fuel consumption levels, and run it for a few laps and see what happens. This allows us to test different strategies and see how they play out in competitive races.”
“Having this ability allows us to ask some very interesting questions,” he continued. “As [Dr. MacAlpine] said, sometimes the answers are surprising. We might have had some expectations, ‘Oh I think the AI can do X laps,’ but it turns out it can do maybe one more. Or, maybe it doesn’t go that far. Things like that help us set our expectations with regard to what the AI can do under different conditions.”
Sophy’s Future
For now, no specific information can be shared regarding when — or even if — this latest version will be coming to Gran Turismo 7.
However, it’s clear GT Sophy continues to be a strong area of research for the Sony AI team. We’ll keep a close eye on the project to see what comes next — stay tuned!
See more articles on Gran Turismo Sophy and Sony AI.
