Gran Turismo Sophy: Sony AI x Polyphony Digital

[Griffith500]

The AI most likely won't have any kind of learning capability of its own when it makes the game. PD could harvest replay data to keep updating things in the background and issue patches to the AI's knowledgebase. Maybe they could also utilise "the cloud" to make it almost concurrent.

It'll be a completed, fixed, trained model that will easily run on any hardware the same as any other procedural AI model does. It won't need to be a learning AI to appear adaptive to the conditions. I don't believe the AI was learning during the races themselves, but the experience was folded in between races to amuse the human racers and test the researcher's approach.

The reasons for the hardcore hardware in the interface are twofold:
1. It's a pure research prototype - no optimisation, no integration; it's all hacks.
2. It's reproducing and processing a video stream, not just pure game data (because of prior research for self driving applications).

Also it's easier just to use a computer you have to hand, even if it is overkill, rather than design and fab and program a bespoke ASIC, just for the purposes of the demonstration. A bit like using a whole computer for your NAS instead of a bespoke compact package, even if Sony are paying.

Sometimes Sophy made very sharp and very discrete wheel movements, looks very unrealistic and arcade-like. Sophy's speed of turning wheel is too fast imo. People arms have inertia. And at first we (people) need to see result of our sudden sharp wheel movement, and only after that we can make correction of the turning to the same or opposite side. Constant 10 Hz for Sophy is a very fast brain, need to correct.
And Sophy needs some additional kilograms in car weight and less grip tyres to make his/her/its actions smoother

Ironically the erratic steering is because 10Hz is too slow. They should use a delay instead, or mixed rates of control application and information feedback, probably both. Simple limits on input speeds would make it look better, but it won't be as competitive on track.

In any case, they would need to start the learning from scratch, but they've probably done that hundreds of times already. They have time to tweak all of those things to make it look more natural. And they can decide whether to keep it visually based or translate to pure game data so it can be used for opponents without the need for external hardware.

 

[Dave A]

The reason is the hardware that it's currently running on. Yes, the backbone of that hardware is for the "learning" part of the AI (the trainer), but even the hardware that's handling the interface between the PS4s and the trainer is non-trivial.

I've said this before, but 2 cores and 3.3GB RAM to interface with one AI is significant. That's just the part running the PS4 control and forwarding data to the trainer. It's going to take improving that by a couple of orders of magnitude in order to get a small handful of AI running alongside the actual game.

That's a really, really good reason to believe that it won't be possible on PS4. I get that people want to be optimistic, but there's plenty to justify why we're unlikely to be seeing this in public games any time soon. Saying that there's no reason to think that it won't be possible on PS4 is just ignoring the actual facts of the paper.

Again, all true, but like you said in the post of yours that you quoted it's unlikely that it's been optomised and possible that it doesn't even need all that power.

Of course, it may not be possible to run on a PS4, but'then Kaz has stated he hopes to bring it to GT7 in the future. Of course, we also know not everything he says he hopes to do happens, and we know that doesn't automatically mean it will come to the PS4 version even if it does reach the PS5 version of GT7.

If we assume Kaz made the statement about GT7 because he knows it is possible at the very least, in theory, on PS5, is it a huge leap to say it'll be possible (in theory) on PS4 also? Not guarenteed but I'd say it should be possible, very possible in fact. I accept, saying "no reason.." wasn't accurate, but the intent was as a figure of speech rather than statement of fact. i wouldn't place too much stock on what the development build is running on at the moment.

Time will tell, any GT7 inclusion is speculation at this stage. Even if it does make it into GT7 we don't know what form, it could be as simple as 1 v 1 special races. We'll have to see, but it's promising for the future. Let's just hope they stick with it unlike the point to point tracks from GT5 that were a great idea that needed more time and effort to refine.

 

[Boxster]

Again, all true, but like you said in the post of yours that you quoted it's unlikely that it's been optomised and possible that it doesn't even need all that power.

Of course, it may not be possible to run on a PS4, but'then Kaz has stated he hopes to bring it to GT7 in the future. Of course, we also know not everything he says he hopes to do happens, and we know that doesn't automatically mean it will come to the PS4 version even if it does reach the PS5 version of GT7.

If we assume Kaz made the statement about GT7 because he knows it is possible at the very least, in theory, on PS5, is it a huge leap to say it'll be possible (in theory) on PS4 also? Not guarenteed but I'd say it should be possible, very possible in fact. I accept, saying "no reason.." wasn't accurate, but the intent was as a figure of speech rather than statement of fact. i wouldn't place too much stock on what the development build is running on at the moment.

Time will tell, any GT7 inclusion is speculation at this stage. Even if it does make it into GT7 we don't know what form, it could be as simple as 1 v 1 special races. We'll have to see, but it's promising for the future. Let's just hope they stick with it unlike the point to point tracks from GT5 that were a great idea that needed more time and effort to refine.

As I have said earlier in this thread, if PD and Sony were not able to implement some sort of improved AI into GT7, it would be very poor PR to introduce Sophy with some brouhaha now.
So, although anything is possible, I would expect the GT7 AI to be improved at launch and to incorporate some form of Sophy within 2 years or so.

 

[Griffith500]

A simple stopgap is to use this deep learning to fine tune the pre existing AI algorithm. That is, have it run laps whilst guessing at all the parameters in the existing model until it gets competitive.

They can learn so much about the inputs to their model by rewarding the learning AI for different outcomes. E.g. no contact, maximum single lap pace, tyre conservation separately from fuel conservation (and both together), not being overtaken, staying off the racing line etc etc etc

I agree that any lack of improvement will land a little awkwardly now.

 

[GTDegu]

If the AI gets penalised during a race does it learn from it not to do it again?

 

[Magog]

If the AI gets penalised during a race does it learn from it not to do it again?

Yes, that's one of the learning mechanisms.

 

[GETEDA]

I would like to meet Sophy in online races 👍🏻

 

[SebastianL]

I would like to meet Sophy in online races 👍🏻

Same, I hope a long lasting and fruitful relationship will evolve from it 😂🤣😂

 

[GETEDA]

@SebastianL

She is probably quiet, so we avoid arguments 😂

 

[Dave A]

@SebastianL

She is probably quiet, so we avoid arguments 😂

I think Sophy is a closet punter, waits untill you aren't looking.

 

[SebastianL]

@SebastianL

She is probably quiet, so we avoid arguments 😂

the perfect gf, she does not speak

I think Sophy is a closet punter, waits untill you aren't looking.

you must have missed the tuscan speed 6 and cerbera speed 12 in the previous games; I thought I was the only one driving them I hope GT7 will get much more cars, like GT4

 

[tofuhero]

Ironically the erratic steering is because 10Hz is too slow. They should use a delay instead, or mixed rates of control application and information feedback, probably both. Simple limits on input speeds would make it look better, but it won't be as competitive on track.

I'm not sure about that; while 10Hz seems slow to us, Sophy doesn't have delayed reaction times like we do(*). Sophy does need to be smoother with its inputs, that's down to the way her steering decisions are input into the system (i.e. turn right 15 degrees immediately vs. turn right 15 degrees with a quadratic timing curve animation over 1 second) and not how much information Sophy has. There are also programmatic ways to combine animations additively so that you can have smooth-looking steering even if the inputs are erratic.

* Put another way, Sophy gets information every 0.1667 seconds and can respond immediately. Our visual perception is about 0.08, and reaction time varies widely. Apparently the average F1 driver has a reaction time of 0.2 seconds, and the average driver ranges from 0.4 to a full second in reaction time. So even though Sophy gets information updates at half the frequency we do, she can react to those updates immediately. So if the F1 driver receives two perceptual updates from their eyes in 0.1667 seconds and Sophy only gets one of those in that time, Sophy can still respond faster than the F1 driver. Given that the Sophy paper in Nature said that increasing the data rate did not improve her driving, I would say this kind of bears out. (if my math or info is wrong here, mea culpa)

 

[Imari]

the perfect gf, she does not speak

Ugh. Gross. Just buy a RealDoll and spare the rest of humanity.

 

[05XR8]

Say I’m driving my tuned Beetle and Sophy is using a Lancia, think there’d be some chemistry between the cars?

 

[Griffith500]

I'm not sure about that; while 10Hz seems slow to us, Sophy doesn't have delayed reaction times like we do(*). Sophy does need to be smoother with its inputs, that's down to the way her steering decisions are input into the system (i.e. turn right 15 degrees immediately vs. turn right 15 degrees with a quadratic timing curve animation over 1 second) and not how much information Sophy has. There are also programmatic ways to combine animations additively so that you can have smooth-looking steering even if the inputs are erratic.

* Put another way, Sophy gets information every 0.1667 seconds and can respond immediately. Our visual perception is about 0.08, and reaction time varies widely. Apparently the average F1 driver has a reaction time of 0.2 seconds, and the average driver ranges from 0.4 to a full second in reaction time. So even though Sophy gets information updates at half the frequency we do, she can react to those updates immediately. So if the F1 driver receives two perceptual updates from their eyes in 0.1667 seconds and Sophy only gets one of those in that time, Sophy can still respond faster than the F1 driver. Given that the Sophy paper in Nature said that increasing the data rate did not improve her driving, I would say this kind of bears out. (if my math or info is wrong here, mea culpa)

Well that's exactly my point. It should be delayed, not artificially stepped. Sophy gets new information every tenth of a second: 100 ms is 10 Hz.

My suggestion is to update Sophy at physics rate but delay the stream by 200 ms. Sophy will exhibit smooth control but will have learnt to anticipate corners and the like and won't be able to respond any faster than the fastest humans.

I mentioned limiting Sophy's input speed too, repeating myself from a previous post even.

Hiding the erratic steering won't hide the erratic response of the physics engine, so that will still look uncanny.


The fact that a computer AI cannot increase its pace beyond 10 Hz control inputs says nothing about how it appears in practice, since that's what we're complaining about. It also says nothing about human performance, or the performance of vehicles in general. As an example, modern ABS systems operate in excess of 10 Hz, so there must be some advantage to that finer monitoring and control of the physical machinery.


In my mind, the 10Hz figure is clearly a limitation of how the information is processed in the overall learning and control model. It wouldn't surprise me if their overall scheme (game to "Sophy" back to game) has an end-to-end latency of 100 ms.

This has been a blind spot in similar kinds of research in the past. Information delay is information delay, whether that be by slow polling rates or long latencies, the effect is the same. But one looks and feels better than the other.


Then again, given the video basis of the machine learning and the self driving car motivation for the research, delaying the information stream requires a relatively large video buffer. There's also no motivation to reduce the performance in that instance. A model based on game data and for entertainment use doesn't have either limitation. So it is what it is, a research prototype.

 

[Normandy12]

I will have the chance to speak with Kazunori Yamauchi and the Sony AI team about Gran Turismo Sophy in the coming days — let me know if you guys have any specific questions about Sophy that you'd like answered. 👍

Watching the videos of Sophy, it appears that the A.I. drives like the opponent it is competing against. Does the A.I., Sophy, actually use the players driving inputs and then try to improve upon those inputs, using the cloud computations, to improve lap times?
I'm curious about this, as to why Sophy would drive in the fashion it does against different people, for example, it appeared to not drive cleanly all the time and approved on that behavior. So, is it using the players lap times and improving on that performance?

 

[Barra333]

Edit: I'm an idiot. Old news

 

[ShiftingGears]

I hope Jordan asked Yamauchi whether we can add truck nuts to the Ford Raptor

 

[happycorey]

I hope Jordan asked Yamauchi whether we can add truck nuts to the Ford Raptor

Thank you for this

 

[kjeldsen]

Two important questions and answers here:



Yes Sophy will be a GT7 patch (confirmed once again)
About difficulty, "GT Sophy will always understand the surroundings, including the level of the drivers.." so a little bit vague but at the same time very interesting..

 

[Nebuc72]

About difficulty, "GT Sophy will always understand the surroundings, including the level of the drivers.." so a little bit vague but at the same time very interesting..

I hope this isn't referring to any type of rubber banding hahaha - that's ruined enough GT AI to last me a lifetime. Not sure what else it could mean aside from "it's slower if you are slow, and quicker if you are quick."

In which case, I'm hoping its more of an adaptive difficulty level that constantly keeps things challenging rather than a "if you are ahead, it gets faster, if you are behind, it slows down"

 

[Pete_J_C]

who else heard the part during the Q&A, in response to the question (paraphrase): "how will you make sophy compete at a lower level against average drivers?" and then his response was: "use a sophy that has been on the track for less time and therefore learned less", which is.. great! but then he said: "another idea is to give sophy a slightly slower car".. with kaz enthusiastically nodding, and I nearly slam my face into the keyboard

 

[GETEDA]

Would be great to see the "Event Synthesizer" (GT2) again but using the Sophy AI.

 

[Silver Arrows]

Yes Sophy will be a GT7 patch (confirmed once again)

And again, I'd like to see something a lot more concrete then simply saying that it will be a GT7 patch, especially in the form that they're showing Sophy to be now. Track Creator's ghosts loom large over this project, to be honest.

In which case, I'm hoping its more of an adaptive difficulty level that constantly keeps things challenging rather than a "if you are ahead, it gets faster, if you are behind, it slows down"

I agree. I think a lot of people treat 'adaptive difficulty' as rubber banding, when it really isn't. It allows games to adjust based on use cases, and it does feel much fairer then rubberbanding either which way. I know that for MLB The Show, you can apply adaptive difficulty for hitting and pitching, and it feels a lot more natural and fits well for cold streaks at the plate, getting shelled on the mound, and then adjust when you're on a hot streak for hitting.

But...

"another idea is to give sophy a slightly slower car".. with kaz enthusiastically nodding, and I nearly slam my face into the keyboard

Knowing Polyphony's habits with regards to AI (Which will stay true unless further evidence to the contrary crops up) this feels like the more likely outcome.

 

[kjeldsen]

And again, I'd like to see something a lot more concrete then simply saying that it will be a GT7 patch, especially in the form that they're showing Sophy to be now. Track Creator's ghosts loom large over this project, to be honest.


I agree. I think a lot of people treat 'adaptive difficulty' as rubber banding, when it really isn't. It allows games to adjust based on use cases, and it does feel much fairer then rubberbanding either which way. I know that for MLB The Show, you can apply adaptive difficulty for hitting and pitching, and it feels a lot more natural and fits well for cold streaks at the plate, getting shelled on the mound, and then adjust when you're on a hot streak for hitting.

But...


Knowing Polyphony's habits with regards to AI (Which will stay true unless further evidence to the contrary crops up) this feels like the more likely outcome.

I don't recall Polyphony promising and promoting a feature this much and then not following through. So even if you have a very valid point this time I am optimistic but I don't expect anything this year, the answers are simple to vague to make me think otherwise.

 

[happycorey]

Kinda strange we've waiting all week for new information but this dropped, with what seemed like a Q&A session from the debut on the 9th. But GTP article hasn't drop around the same time?

 

[Imari]

who else heard the part during the Q&A, in response to the question (paraphrase): "how will you make sophy compete at a lower level against average drivers?" and then his response was: "use a sophy that has been on the track for less time and therefore learned less", which is.. great! but then he said: "another idea is to give sophy a slightly slower car".. with kaz enthusiastically nodding, and I nearly slam my face into the keyboard

I'm not sure that a Sophy that has learned less is a great solution either. It'll be slow around the lap, but because it hasn't learned to control the car properly yet. An AI that is slow because it weaves all over the track and misses every apex isn't what you want to put beginners up against, it's hard to drive against cleanly and it teaches them all the wrong lessons.

I don't recall Polyphony promising and promoting a feature this much and then not following through. So even if you have a very valid point this time I am optimistic but I don't expect anything this year, the answers are simple to vague to make me think otherwise.

The one that's most likely to be compared is the GT6 track maker, which definitely came out but the goalposts shifted substantially over the years as they developed it and figured out that what they had originally been talking about wasn't really feasible. We'll almost certainly get something, but it doesn't mean that it will necessarily bear much resemblance to the Sophy that we see now.

 

[EnzoRacer]

Yes Sophy will be a GT7 patch (confirmed once again)

May be...may be. Some Vision GT "confirmed" GT6 cars were never released.

UPD. And Kaz said (timestamp 21:24) "We want to deliver GT Sophy to all players through a future GT7 update". They just want it. So imo it's not confirmation, it's just a wish.

 

[ShiftingGears]

So to summarise, they want to add it in a future update however they dont yet have an idea how to scale it so its less than superhuman.

 

[kjeldsen]

May be...may be. Some Vision GT "confirmed" GT6 cars were never released.

UPD. And Kaz said (timestamp 21:24) "We want to deliver GT Sophy to all players through a future GT7 update". They just want it. So imo it's not confirmation, it's just a wish.

I will quote myself:
"I don't recall Polyphony promising and promoting a feature this much and then not following through. So even if you have a very valid point this time I am optimistic but I don't expect anything this year, the answers are simple to vague to make me think otherwise."

25 people from SONY AI worked on Sophy, several press events and lots of PR all around the world on social media platforms. It will be a huge defeat if it never makes it into GT7