As the first significant public release of technology from Sony AI, which was formed in 2020, the upcoming Gran Turismo Sophy Race Together mode for Gran Turismo 7 will be many players’ first exposure to complex AI technology that could have a lasting impact on many of Sony Interactive Entertainment’s games. As such, if the AI is rude, unfair, and unbeatable for many players, GT Sophy could be seen as yet another warning sign that AI isn’t ready for primetime yet. That’s why ensuring GT Sophy has proper sports etiquette is a priority for its developers.
“In racing, you need to make sure that you’re driving aggressively because you do want to win, but at the same time, you want to drive fairly,” Sony AI COO Michael Spranger tells Digital Trends. “You can’t bump into other cars in order to gain an advantage and need to make sure that everybody can fairly compete. That’s an interesting area because these rules are quite imprecise; there’s a rulebook, but it needs interpretation from judges. Getting this right — driving aggressively and competitively versus driving fairly — is one of the big challenges of GT Sophy.”
Sony AI, Sony Interactive Entertainment, and Polyphony Digital have confirmed that players will be able to race GT Sophy in a limited-time Gran Turismo 7 mode starting on February 21. To get to that point, its developers did a lot of work refining not only GT Sophy’s skill via reinforcement learning but its etiquette toward a nebulous set of courtesy-driven rules as well. Ahead of its addition to Gran Turismo 7, Digital Trends spoke to Sony AI COO Michael Spranger and GT Sophy Project Lead Peter Wurman to learn more about their ethical approach to GT Sophy’s sportsmanlike conduct, and what future uses they see for this technology in gaming.
The following interview has been edited for brevity and clarity.
What made Gran Turismo the most appealing option to develop an advanced AI for first?
Michael Spranger: It was because it was possible and because Gran Turismo is a really exciting game. Gran Turismo has been part of PlayStation from the very beginning. It’s an iconic game that provides this specific challenge area of physical realism, strategy, decision-making, and sports etiquette. Also, it has a big audience of people actually following the game, and that means that any technology we develop in the game can have a potentially meaningful impact on millions of players. I also love the game. These things combined made us choose Gran Turismo as one of the first targets for gaming AI projects.
How does reinforcement learning allow this AI to take more risks or drive more impressively than your standard game AI? Is it just trying to drive the most precise racing line?
Peter Wurman: There are different levels of skill. The baseline is driving faster on the track, so reinforcement learning works by letting the AI agent explore what effect its actions have on the world and then try to maximize its reward. By giving it positive signals for going faster on the track, it learns to drive around the track very fast. It learns to press on the accelerator and break into corners so it doesn’t crash, which works great for training it for a time-trial scenario.
To actually make it a competitive racer, we had to give it other reward signals for passing cars that are in front of it and not preventing cars behind it from passing it. We also gave it penalties for colliding or bumping into other cars. This combination of rewards and penalties took some effort to get right, but once we did training with the right population of opponents, it learned to be a very effective racer.
What were the biggest learnings from the first couple of Race Together events that you held with top Gran Turismo Sport players?
Peter Wurman: If you’ve followed along with the project, you’ll know there were two races. The first we did well in, but we didn’t win the Team Score. Some of the interesting things we learned from that had to do with finding the balance of assertiveness versus being a good sport. We found out two weeks beforehand that the agent we had trained was too aggressive and wouldn’t be allowed to race, so we had to scramble and adjust things. You can’t have one AI agent that was really good at driving fast and another that thought about the tactical aspects of passing; instead, we had to incorporate them together into one coherent agent that was doing both at the same time. Those are the things we saw after the July 2021 race, so we felt much more confident going into the October 2021 race.
Michael Spranger: There was this thing where we were told it was too aggressive, but then on the first exhibition race day, we actually lost the race because were not aggressive enough and timid. The problem of how you balance those things was really apparent across the development of the project.
What changes had to be made to GT Sophy when you made the decision to add it to Gran Turismo 7 and not just train it in a test environment?
Peter Wurman: The biggest is that in all the exhibitions, GT Sophy ran on a separate computer and talked through a networking connection to the PlayStation console. Here, GT Sophy is embedded in the game. Making it work on a PlayStation with the resources that were available to us after all of the rendering and other stuff that’s going on was a bit of engineering work.
The other thing is that Polyphony Digital really wanted the agent to be an extremely good example of racing style and racing skill. We put in some changes to encourage GT Sophy to stay within the lines more. If you’ve watched any of the exhibitions, you’ll see that it was really good at using all of the track, sometimes putting two tires in the grass. That’s legal, but not exactly what you could do in a real race car. We made some changes to keep GT Sophy between the lines more. It’s still very fast but a little cleaner.
We’re mostly focused on trying to adhere to the rules of sportsmanship in automotive racing.
What are the main things the Sony AI team is hoping to learn from adding Gran Turismo Sophy Race Together to Gran Turismo 7?
Peter Wurman: The main one is that we hope people enjoy playing against it, feel like they’re really challenged at all levels of skill, and are learning to become better-simulated racecar drivers.
Michael Spranger: We also want to see how people interact with the technology. Right now, it’s a handful of people who have experienced the technology, so we’re curious to see how people are going to use this to learn for themselves, gain new skills, and have new experiences. I think that’s at the core of the project in many ways. We have the technology breakthrough, and I think that’s really important, but I think we’re unlocking a brand new stage for people and in Gran Turismo. In many ways, the built-in AI has a quite narrow band of performance, and you can surpass that once you become an intermediate driver on each difficulty.
Even with the relatively few track-car combinations that we unveiled here, from what I can do to what some of the top players can do, everybody can find a really interesting match. We hope to see more of that and see how people learn from that. It’s also very difficult for some players to find good opponents, and for the first time, you can have repeated interactions with a worthy opponent without having to call up all your friends and have them all come together. It’s going to be something where people can make a lot of progress in getting better at the game because they can have these tailored experiences to some extent.
On the ethical side of things, can you explain the process of determining what constitutes sportsmanlike conduct for GT Sophy so it isn’t harassing certain players?
Peter Wurman: Well, it can’t talk to you, so there are a lot of bad things GT Sophy can’t do because that’s not part of the game. We’re mostly focused on trying to adhere to the rules of sportsmanship in automotive racing; now, those are pretty nebulous. You can’t block a legitimate passing opportunity and things like that. There are a bunch of fuzzy words in these rules that were hard to encode and program, so we had to approximate it with reward signals and penalties if it did things like bump another car.
Then, we had to give it to Polyphony Digital and test drivers and let them give us feedback because racing against other AI agents doesn’t really show us if it’s ready to be racing against the variety of human racing styles. That took some effort and iteration to get that right.
Michael Spranger: For this release, I think it’s fair to say that it is competitive. It is going to try and overtake players, but it’s not going to divebomb them. It’s competitive but polite. It’s definitely putting pressure on you; you’re going to see it in the rearview mirrors trying to overtake you. At the same time, it’s going to try and give you some space so you can race competitively but fairly.
Peter Wurman: But if you make a mess, it’ll take advantage.
Michael Spranger: That’s right, instantly!
Where do you see GT Sophy and AI technology in Sony games going over the next five years?
Michael Spranger: Obviously, we can’t announce anything specifically, but in general, there are multiple layers to it. There are still challenges and things we can do within Gran Turismo to unlock new experiences, and I don’t think we’re finished with that. There’s real excitement about pushing the technology further. Then, from the Sony AI perspective of corporate R&D and strategic initiatives, we’re really excited about what this technology can bring to gaming in general.
Other games will give us a different challenge to overcome and will further the capabilities of the technology. It’s an emerging technology that has had some successes in the past, but I do think what we’re doing here is quite distinct in the sense that we do the scientific breakthroughs but then we also deploy it and make sure that people get their hands on the AI technology. That is going to push the technology forward and unlock greater experiences.
Interaction is at the core of PlayStation, it’s not an error that it’s called Sony Interactive Entertainment. It’s really about what your interaction with the things that you find in the game is, and I do feel that this technology has the potential to really unlock the next generation of games, and we’re really excited about that.
