GT7 is compatible with motion rig ?
- Thread starter poumpoum
- 686 comments
- 191,604 views
- 50
- Germany
I plotted a kart lap on a round course with the data of unknown semantics 0x94, 0x98, 0x9c, 0xA0:From what I've seen, two of the three first ones are very small values, hovering around 0. The second of the three (0x98) seems to hold just below 1, at 0.99... and don't change much. The fourth value (0xA0) varies a lot, both positive and negative, and may well go towards a value (like say 50) on braking, but remains at that value after stopping...
Can't really make head or tail of these (0x94-0xA0)... Possibly if someone could graph them along with a replay, it would be possible to make something of them.
Either way it's fun to dig in to some detective work
See circle_kart.txt for the actual data. It is a HTML file, so please rename to circle_kart.html.
Here is the Youtube video with the lap as a replay. The plot starts when the timer is clocking the lap.
I made some odd moves at the end. It shows at 0x98 and 0x9c. I made the same plot in Spa, where I had a peak for 0x94 and 0x98 in Eau Rouge. Maybe it is related to downforce?
You can hover over the data points to get an index. The index is also used in the lap diagram on the top right. So you can make out the position.
Hope you can make something out of it. I could not.
Last edited:
- 2,228
- Bielefeld
- tarnheld
This is a description of the plane of the road below the car tires, in a fancy mathematical dressing. See this post for further info. The distance of the car position to this plane is the ride height. I guess it might come in handy to calculate some other stuff, maybe in relation to the suspension or for a motion rig, but for now it's rather uninteresting for racing dashboard usage. You could calculate the current road inclination or banking from this plane, or you could determine if the car is flipped or doing a ski stunt.
- 131
- Oslo
So I checked the constant packet time of 1/60 and that's why I moved to packet arrival time as some of the laps were half a second off. I also couldn't really make it any more precise by just replacing the logic with the time progression, but, well two insights have come from this:
1. When pausing the game and using the +1 lap / -1 lap sliders on the replay to position the car at the beginning of the lap. The car's positional x/y/z ALWAYS matches the first packet I logged for the new lap, i.e. the first packet of a new lap is ALWAYS when the car crosses the timing line, regardless of how long it has been since the last packet.
2. Time progression is not always linear. This is likely expected at the beginning of a lap from (1), but I am investigating more.
Last edited:
- 131
- Oslo
A followup. 3 columns:
Col 1: Lap time in the packet data
Col 2: Lap time calculated from time progression
Col 3: Lap time calculated from packet arrival times.
(lap 1 is off due to the timer starting way before the car reaches the s/f line to go from lap 0 -> lap 1).
I can see that the lap times are not too dissimilar so I think I will use them in place of capturing packet timings to avoid that extra source of computation and storage for no real discernable benefit.
Actual time progression fluctuations (I worked out a 5x time multiplier, hence the division):
Over the course of the race, each lap packet time means are very accurate to 10us. I wonder if we can use that as a constant going forward for each packet timing...
Nope. Not any more accurate. I am so confused as to where all the time goes on some laps:
Every method is nearly 20ms short which is nearly always more than 1 packet. I can only think of 2 possibilities:
1. the last packet describing the end of the circuit is discarded when you cross the start/finish line to start the next lap so the last packet timing needs calculating
2. I pointed out earlier that lap times displayed in replay (at least from earlier patches) do not match lap times in the log. It's been a while since I raced on GT7 so I am wondering if part of these differences can be chalked up to (undocumented or known) physics changes between patches and the game playing "catchup" in the replay when things go awry (hence why older patches don't display laptimes in the replay) and the data isn't correct.
I'll do a race and see what happens.
EDIT: Just did a 5 lap "race" around Tsukuba with bots on the controller:
Those are looking much more stable packet times.
Col 1: Lap time in the packet data
Col 2: Lap time calculated from time progression
Col 3: Lap time calculated from packet arrival times.
(lap 1 is off due to the timer starting way before the car reaches the s/f line to go from lap 0 -> lap 1).
I can see that the lap times are not too dissimilar so I think I will use them in place of capturing packet timings to avoid that extra source of computation and storage for no real discernable benefit.
Actual time progression fluctuations (I worked out a 5x time multiplier, hence the division):
Over the course of the race, each lap packet time means are very accurate to 10us. I wonder if we can use that as a constant going forward for each packet timing...
Nope. Not any more accurate. I am so confused as to where all the time goes on some laps:
Every method is nearly 20ms short which is nearly always more than 1 packet. I can only think of 2 possibilities:
1. the last packet describing the end of the circuit is discarded when you cross the start/finish line to start the next lap so the last packet timing needs calculating
2. I pointed out earlier that lap times displayed in replay (at least from earlier patches) do not match lap times in the log. It's been a while since I raced on GT7 so I am wondering if part of these differences can be chalked up to (undocumented or known) physics changes between patches and the game playing "catchup" in the replay when things go awry (hence why older patches don't display laptimes in the replay) and the data isn't correct.
I'll do a race and see what happens.
EDIT: Just did a 5 lap "race" around Tsukuba with bots on the controller:
Those are looking much more stable packet times.
Last edited:
- 131
- Oslo
Lap time from difference in time progression: 2:00.239
Lap time from differences in packets arriving at my PC : 2:00.236
Lap time from deriving time from distance between two points using pythagoras and packet meters-per-second values: 2:00.237
Lap time from taking packets at a constant 1/60th of a second: 2:00.133
Lap time from wattching replay: 2:00.266
Lap time from packet data: 2:00.255
So I am stumped as to what is going on.
Lap time from differences in packets arriving at my PC : 2:00.236
Lap time from deriving time from distance between two points using pythagoras and packet meters-per-second values: 2:00.237
Lap time from taking packets at a constant 1/60th of a second: 2:00.133
Lap time from wattching replay: 2:00.266
Lap time from packet data: 2:00.255
So I am stumped as to what is going on.
- 131
- Oslo
Ok, so I decided to move on from trying to get approximate lap times from the packets and just assume that the data is at least consistent inside the packet data, so I decided to knock up a nice Dash dashboard to analyse the data. I'm not good at styling so ignore that for now, but the data is at least usable.
- Laps can be selected via dropdown or boxed on the lap time chart.
- hovering over the speed graph will highlight the point on the lap chart where the hover is.
- You can box or lasso parts of the lap map zoom in and focus on that area.
- Files can just be dropped in to a folder to be loaded.
- 2,228
- Bielefeld
- tarnheld
This bugged me as well and i could reproduce your problems as well. Spent a considerable amount of time on this, and long-story-short: the PD packet timing madness is cracked! I can reproduce the current lap time for every packet as it is displayed in GT7 without relying on the day time progression, i just need the packet counter, the lap number and a very crucial ingredient: the last lap time. So bear with me as i try to unravel the wondrous puzzle PD has presented to us...
So first off, the assumption that the packets are measured every 1/60 seconds is correct for almost all packets, but not for one very special packet -- the packet you will get that has a new lap number in it. This packet also contains the last lap time. This packet is measured at a different time on the timeline and you will get this point through a calculation that is pretty involved. So let me set the scene of the packet train and it's timing, below you see each packet with it's packet number, the time delta it represents, and the total elapsed time we can calculate from the packet deltas. You also see that only the packets where the lap increases have a different time delta:
| packet number | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | ... |
| packet lap | 0 | 1 | 1 | 1 | 1 | 2 | 2 | 2 | 2 | 3 | 3 | ... |
| packet time delta | 1/60 | A | 1/60 | 1/60 | 1/60 | B | 1/60 | 1/60 | 1/60 | C | 1/60 | ... |
| elapsed time | 1/60 | 1/60 +A | 2/60 +A | 3/60 +A | 4/60 +A | 4/60 +A+B | 5/60 +A+B | 6/60 +A+B | 7/60 +A+B | 7/60 +A+B+C | 8/60 +A+B+C | ... |
| lap time on arrival of packet | 0 | 0 | 1/60-A | 2/60-A | 3/60-A | 3/60-A+B | 1/60-A-B | 2/60-A-B | 3/60-A-B | 3/60-A-B+C | 1/60-A-B-C |
You see that we have to accumulate all the special packet time deltas to get an accurate elapsed time. One more specialty to be aware of: the packets arrive also when the game/replay is paused. You can determine if a packet is paused from the flags. A paused packet has effectively a zero time delta.
So how do we get to the special packet time deltas? It is also right there in the packet table: each lap time is the sum of all normal packets plus the time of one extra packet, and we get the lap time of the last lap in every packet. So get can get the time delta by summing up all normal packet time deltas and subtracting from that the last lap time. As a generous gitft, PD made the time delta of the first special packet zero.
With these ingredients we can build a lap time counter that will show the exact same time that PD shows in the game. Here is a very rough demo, you should start it before starting a race or TT as it needs to receive all packets or all laps, but it will print out the same current lap time as in the game:
Python:
import socket
import sys
import struct
import math
import datetime
# pip3 install salsa20
from salsa20 import Salsa20_xor
#https://github.com/Nenkai/PDTools/blob/master/SimulatorInterface/SimulatorInterface.cs
SendDelaySeconds = 10
ReceivePort = 33740
SendPort = 33739
port = ReceivePort
if len(sys.argv) == 2:
# Get "IP address of Server" and also the "port number" from
ip = sys.argv[1]
else:
print("Run like : python3 pd-lapcounter-demo.py <playstation-ip>")
exit(1)
# Create a UDP socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# Bind the socket to the port
server_address = ('0.0.0.0', port)
s.bind(server_address)
s.settimeout(10)
#https://github.com/Nenkai/PDTools/blob/master/PDTools.Crypto/SimulationInterface/SimulatorInterfaceCryptorGT7.cs
def salsa20_dec(dat):
KEY = b'Simulator Interface Packet GT7 ver 0.0'
oiv = dat[0x40:0x44]
iv1 = int.from_bytes(oiv, byteorder='little') # Seed IV is always located there
iv2 = iv1 ^ 0xDEADBEAF #// Notice DEADBEAF, not DEADBEEF
IV = bytearray()
IV.extend(iv2.to_bytes(4, 'little'))
IV.extend(iv1.to_bytes(4, 'little'))
ddata = Salsa20_xor(dat, bytes(IV), KEY[0:32])
#check magic number
magic = int.from_bytes(ddata[0:4], byteorder='little')
if magic != 0x47375330:
return bytearray(b'')
return ddata
def send_hb(s):
#send HB
send_data = 'A'
s.sendto(send_data.encode('utf-8'), (ip, SendPort))
print('send heartbeat')
class LapCounter:
def __init__(self):
self.lap = -1
self.paused = -1
self.tick = 0
self.pstart_tick = 0
self.lstart_tick = 0
self.lstart_ms = 0
self.paused_ticks = 0
self.last_lap_ms = 0
self.special_packet_time = 0
def update(self,lap,paused,tick,last_lap_ms):
if lap == 0: # we have not started a lap or have reset
self.special_packet_time = 0
if lap != self.lap: # we have entered a new lap
if self.lap != 0:
normal_laptime = self.lapticks()*1000.0/60.0
self.special_packet_time += last_lap_ms - self.lapticks()*1000.0/60.0
self.lstart_tick = self.tick
self.paused_ticks = 0
if paused != self.paused: # paused has changed
if paused: # we have switched to paused
self.pstart_tick = self.tick
else: # we have switched to not paused
self.paused_ticks += tick - self.pstart_tick
self.paused = paused
self.lap = lap
self.tick = tick
self.last_lap_ms = last_lap_ms
def pausedticks(self):
if not self.paused:
return self.paused_ticks
else:
return self.paused_ticks + (self.tick - self.pstart_tick)
def lapticks(self):
if self.lap == 0:
return 0
else:
return self.tick - self.lstart_tick - self.pausedticks()
def laptime(self):
laptime = (self.lapticks() * 1./60.) - (self.special_packet_time/1000.)
return round(laptime,3)
send_hb(s)
print("Ctrl+C to exit the program")
pknt = 0
lapcounter = LapCounter()
while True:
try:
data, address = s.recvfrom(4096)
pknt = pknt + 1
print("received: %d bytes" % len(data))
ddata = salsa20_dec(data)
if len(ddata) > 0:
tick = struct.unpack_from('i',ddata,0x70)
current_lap,laps_remaining = struct.unpack_from('hh',ddata,0x74)
best_lap_ms,last_lap_ms = struct.unpack_from('ii',ddata,0x78)
rpm_min,rpm_max,v_max,flags = struct.unpack_from('hhhh',ddata,0x88)
paused = flags&2 # second bit in flags is paused
lapcounter.update(current_lap,paused,tick[0],last_lap_ms)
print ("lapcounter.laptime:",lapcounter.laptime())
if pknt > 900:
send_hb(s)
pknt = 0
except Exception as e:
print(e)
send_hb(s)
pknt = 0
pass
Last edited:
- 16
- Germany
- jajaballard
- 131
- Oslo
I KNEW IT. I should have guessed this from the fact that I could observe that the start of each new lap was ALWAYS on the start/finish line exactly, which made no sense from the perspective that EVERY packet was logged at equal intervals, and the distances made sense so the time should have. An accumulating delta for each lap was the only thing I didn't think of. I was playing with lap packet offsets and all sorts of nonsense, but as of now I can hopefully put this last one to bed, which then means...
I can add this, tidy up the code and then I'll put it on GitHub. Note as this is a Dash app, it will require Python and a bunch of libraries installed to get working. I'll see if there's an easy way (py2exe?) to make an executable out of it, but it will be large.
Last edited:
- 131
- Oslo
One snag. This code works well for when lap 1 starts at the start/finish line. If you do, for example online dailies, or FIA races with rolling starts then cars at the rear, when the start counts down to 0 has the lap 1 timer start on lap 0 which throws off the calculation because lap 1 will not have a lap time of 0 at the start of the lap, but some arbitrary time, t that needs adding on to the lap time. I am trying to see if there are ways of identifying the first packet in lap 0 where the player takes control, and thus the clock starts.
You can see lap 1 has a 4.7s difference for Brands Hatch which nearly matches the replay:
You can see lap 1 has a 4.7s difference for Brands Hatch which nearly matches the replay:
- 63
- Australia
- gti_sdn
check out this post:
GT7 is compatible with motion rig ?
Thank you very much! A-Maze-Ing stuff. Keep up the great work everyone, what a community we have here at GTP ❤️ Jerome
www.gtplanet.net
They have created a proxy between GT7 and SimRacingStudio, it might get you going with Wind simulation.
- 126
- Sweden
- Bornhall
That's quite a discrepancy indeed... Makes you wonder how feasible it is to use on replays. I assume that the live telemetry surely must be the "holy grail" so to speak, with the most accurate data. One would think they may be doing some kind of data culling and/or compression for the replay though, so you might lose some accuracy (sort of like artifacts in JPEG compression) – BUT, as you say, that's quite a noticeable difference, so... Weird.
How does lap times compare from live versus replay? I notice rpm and such also differs, so one wonders what values remain the same from live to replay.
Edit: Curious, if you overlay graphs of the speeds over time, do they line up linearly?
Last edited:
- 131
- Oslo
Yeah, with all the hassle I was having with getting proper lap times and stuff, I kind of went down the route of "Is the replay just approximating values to make it look correct?"
I ran it through my graphs (ignore that it says lap 1 and lap 2, those are just labels I had to uniquely identify the data). That speed discrepancy definitely shows up though:
I decided to look at the data before that point:
It's pretty much identical, but after that weird differential, it starts to drift... You can definitely see it on the throttle chart in the first pic.
Both data sources for this single lap are the same length:
So yeah.
Last edited:
- 1,394
- Tennessee
- tnfedexman
Y'all are way above my paygrade. I just enter races using a DS4 with a maxed out PP and buy cars. I used to use a wheel and headset. My wife made fun of me. It got uncomfortable. Manhood was questioned. I usually race when everyone in the house is asleep. No sound on.
Hello !
I thought I had made some progress , but looks like I'm in a dead-end now. I focus on motion rig with roll/pitch/yaw/heave/sway/surge like @ashupp and used code from @Bornhall and @gt7coder for the packet structure, and of course based on @Nenkai work. I generated CSV from the decoded packet and tried to make some graphs to understand and figure values I needed. But I'm lost now.
What puzzles me is that for exemple I see positive and negative values for all velocity axes whereas I have always moved forward so I expected at least one of the three values to be always positive.
I was also unable to recreate the track aspect as some of you do, by plotting X,Y on a graph. This makes me think I have an issue with my code.... but values seem to move nicely without jumps.
Do you guys have the same behavior with the raw data ? Any idea of what I have missed ?
Packet format I used is this one, and seems accurate as values are moving smoothly withtou errors.
The full code is here just in case: https://github.com/vthinsel/GT7Proxy
EDIT: I think I found the issue for the map: the Z axis is not what I used to play with other games such as PCars, but once axis are swapped, the map gets displayed fine. But I'm still puzzled about velocity axes behavior. Orientation looks almost fine (Y axis being used for yaw)
I thought I had made some progress , but looks like I'm in a dead-end now. I focus on motion rig with roll/pitch/yaw/heave/sway/surge like @ashupp and used code from @Bornhall and @gt7coder for the packet structure, and of course based on @Nenkai work. I generated CSV from the decoded packet and tried to make some graphs to understand and figure values I needed. But I'm lost now.
What puzzles me is that for exemple I see positive and negative values for all velocity axes whereas I have always moved forward so I expected at least one of the three values to be always positive.
I was also unable to recreate the track aspect as some of you do, by plotting X,Y on a graph. This makes me think I have an issue with my code.... but values seem to move nicely without jumps.
Do you guys have the same behavior with the raw data ? Any idea of what I have missed ?
Packet format I used is this one, and seems accurate as values are moving smoothly withtou errors.
class GTDataPacket:
def __init__(self, data):
## Format string that allows unpack to process the data bytestream:
gt_format = '<ifffffffffffffffccccfffffffffffihhiiihhhhhhBBBcffffffffffffffffffffffffffffffffffffi'
(self.magic, #int32
self.position_x, #single
self.position_y, #single
self.position_z, #single
self.velocity_x, #single
self.velocity_y, #single
self.velocity_z, #single
self.rotation_x, #single
self.rotation_y, #single
self.rotation_z, #single
self.northorientation, #single
self.angularvelocity_x, #single
self.angularvelocity_y, #single
self.angularvelocity_z, #single
self.body_height, #single
self.rpm, #single
self.iv1, #char
self.iv2, #char
self.iv3, #char
self.iv4, #char
self.fuel_level, #single
self.fuel_capacity, #single
self.speed, #single
self.boost, #single
self.oil_pressure_bar, #single
self.water_temperature, #single
self.oil_temperature, #single
self.tire_temp_FL, #single
self.tire_temp_FR, #single
self.tire_temp_RL, #single
self.tire_temp_RR, #single
self.pkt_id, #int32
self.current_lap, #int16
self.total_laps, #int16
self.best_lap_time, #int32
self.last_lap_time, #int32
self.day_progression_ms, #int32
self.pre_race_start_position, #int16
self.pre_race_num_cars, #int16
self.min_alert_rpm, #int16
self.max_alert_rpm, #int16
self.calculated_max_speed, #int16
self.flags, #int16
self.suggestedgear_gear, #byte
self.throttle, #byte
self.brake, #byte
self.padding_byte1, #byte
self.road_plane_x, #single
self.road_plane_y, #single
self.road_plane_z, #single
self.road_plane_dist, #single
self.tire_rps_FL, #single
self.tire_rps_FR, #single
self.tire_rps_RL, #single
self.tire_rps_RR, #single
self.tire_radius_FL, #single
self.tire_radius_FR, #single
self.tire_radius_RL, #single
self.tire_radius_RR, #single
self.susp_height_FL, #single
self.susp_height_FR, #single
self.susp_height_RL, #single
self.susp_height_RR, #single
self.unknown_single1, #byte
self.unknown_single2, #byte
self.unknown_single3, #byte
self.unknown_single4, #byte
self.unknown_single5, #byte
self.unknown_single6, #byte
self.unknown_single7, #byte
self.unknown_single8, #byte
self.clutch_pedal, #single
self.clutch_engagement, #single
self.rpm_clutch_gearbox, #single
self.transmission_top_speed, #single
self.gear_ratio1, #single
self.gear_ratio2, #single
self.gear_ratio3, #single
self.gear_ratio4, #single
self.gear_ratio5, #single
self.gear_ratio6, #single
self.gear_ratio7, #single
self.gear_ratio8, #single
self.car_code, #int32
) = unpack(gt_format, data)
The full code is here just in case: https://github.com/vthinsel/GT7Proxy
EDIT: I think I found the issue for the map: the Z axis is not what I used to play with other games such as PCars, but once axis are swapped, the map gets displayed fine. But I'm still puzzled about velocity axes behavior. Orientation looks almost fine (Y axis being used for yaw)
Attachments
Last edited:
- 131
- Oslo
Saw some funny stuff near the end of the lap...
That red line (replay) definitely doesn't look to be err... On the money. I would hazard a guess the replay file has either a lower resolution or the replay has some other kind of averaging going on where it doesn't care what data is being output as long as it's approximate over the course of a lap and plays to the screen how it should.
It explains why I didn't think about it before. Earlier replays I saw the max throttle bouncing between 254 and 255 and just dismissed it as I had to clean the throttle pedal on my CSL Elite due to the potentiomenter being a bit bouncy at low values so just assumed this was the beginning of it.
I have also thought that could this be some weird artifact where the live data is actually being smoothed, and the replay has raw data and this sawtooth effect is the vibration from the trigger controls, but I don't see under braking and I don't see it on all corners, so I dunno.
I think for purposes going forward I am just going to ignore replay files when it comes to any kind of analysis or development because it's just... weird.
That red line (replay) definitely doesn't look to be err... On the money. I would hazard a guess the replay file has either a lower resolution or the replay has some other kind of averaging going on where it doesn't care what data is being output as long as it's approximate over the course of a lap and plays to the screen how it should.
It explains why I didn't think about it before. Earlier replays I saw the max throttle bouncing between 254 and 255 and just dismissed it as I had to clean the throttle pedal on my CSL Elite due to the potentiomenter being a bit bouncy at low values so just assumed this was the beginning of it.
I have also thought that could this be some weird artifact where the live data is actually being smoothed, and the replay has raw data and this sawtooth effect is the vibration from the trigger controls, but I don't see under braking and I don't see it on all corners, so I dunno.
I think for purposes going forward I am just going to ignore replay files when it comes to any kind of analysis or development because it's just... weird.
Last edited:
- 131
- Oslo
There is a field in the data affectionately known as "tick" which increments with each packet. Usually these are stamps that equate to things like "number of frames rendered" or "amount of seconds past since the PS was last rebooted" or something along those lines.
I checked the data and there are no missing numbers in the ticks, meaning that all the data that was sent, I have, so in this case it's probably not FPS drops, and it's not lag because it's being sent from my PS to my PC which is on the same switch, and I am measuring packet times.
- 126
- Sweden
- Bornhall
Another thing though, is the data points always the same (error-wise) if you run the same replay twice (or more)?
I'm thinking if there is some kind of read buffering that for some reason lowers the priority of the thread that spurts out the telemetry, and it somehow knows to "catch up" to itself... Could possibly be while the replay is stored as well, and if so, the values from the replay should be reproducible every time.
I've also been pondering if it would be a good idea to average out like the last 10 ticks of data for some values like speed, rpm and such. Or maybe even weighting "old" values lower than the latest tick values, giving maybe a more up to date current value. Reason for me thinking about this is because I'm not sure it's really necessary to (post-race) evaluate telemetry in such high resolution as 1/60th. Mostly this is why I haven't implemented any kind of logging myself.
- 131
- Oslo
Ok, so after some experimentation, this is what I have found:
i did a quick 5 lap Sunday Cup race around Northern Isle Speedway and recorded 5 sets of data:
I had to do a lot of double-checking as the data in replay 2 didn't match replay 3 for some reason as it was off-by-one packet.
i did a quick 5 lap Sunday Cup race around Northern Isle Speedway and recorded 5 sets of data:
- Data being recorded live as I was driving
- Data being recorded from the replay that is shown after the race and credits, but before you go back to the screen to save the replay.
- A replay ran from the main post-race screen where you can save the replay, or exit, or race again.
- A replay of the race loaded from "Showcase" after saving the replay.
- Another run of the same replay after exiting and reloading.
- The live data doesn't match ANY of the replays, it is different in the few instances I showed earlier in that the speeds are off by a few kph, so the rest of the data is. Ironically enough, tick 0 on a lap is identical, it just drifts from there.
- Replays in points 2-5 are all identical data.
I had to do a lot of double-checking as the data in replay 2 didn't match replay 3 for some reason as it was off-by-one packet.
