At the end of the day, I think it's up to the tuners to quantify their lap times NOT against the standard car, but against other tunes of the same class that they've done.
Ideally, a tuner should run their car in free run mode for a period of laps, non stop to work out an average speed of the car over that distance. They should then post the FREE RUN LAP AVERAGE and then post the FREE RUN FASTEST LAP.
Now those times alone are nothing to go by if you don't have other times to benchmark them against and seeing as online and free run cannot be compared, the tuner should also benchmark these average and fastest lap times against OTHER tunes that they've done.
Example:
LEXUS IS-F
Average Lap Time: 2'22.579
Fastest Lap Time: 2'20.998
BMW M3 V8 Coupe
Average Lap Time: 2'23.014
Fastest Lap Time: 2'20.755
The above is an example of how one tune relates to another (in the SAME environment), so you could take one look at that and say the BMW is a faster car when it's on the very edge, but on average, it's a harder and slower car to drive than the Lexus.
As tuners add tunes to their garage, their collated data begins to grow and racers can get an idea of what one car is capable of versus another.
At the moment, there's too many generic statements about how fast a car is. I want to know if you've done the backup running. the problem then is, tuners then fudge the figures to either make themselves look like they were on the ragged edge of testing or to boost the popularity of their tune versus other garages.
Disagree with a fair chunk of your post mate, I do this as a free service to people, not to sell a product and make money from it and I also have a real job.
If your going to accuse the tuners here of fudging figures then:
a) Dont come to my garage asking for a tune
b) You are casting real doubt over the ability of the garages to have any credability and integrity for what they do.
I have yet to see any indication of this so not sure why you would even raise this.
What we are trying to achieve here, is a measure of the improvement a tune has made.. straight forward and simple. Theoretically this value should hold true for a wide cross section of drivers who take this car and test it against a base tune. That measure of gain we are talking about here is your proof gain. However if you are willing to accept we fudge figures, then there is nothing we can put in print that would hold any value right ? I cant be assed taking photos of everything I do so you as a driver have 3 choices:
a) Dont use my tune
b) Trust the feedback and reviews left by other people
c) Take a tune and make your own decision
My job is to take a car and improve it... thats all. Im not here to compare cars and tunes against each other, thats your job as the driver if thats the data you want, so please be my guest and take these cars out and do those tests and let us all know the results. Id be interested to see how you go, but what I think we will see is that some cars will suit some peoples driving styles moreso than others, and some cars will always be faster than others.
Whilst I have some tunes that are published, the majority of the work I have done here has been tailored to individual needs where I will happily work with someone on a tune and problem they have with a car to get a setup that suits their driving style. You cant measure that and I dont publish those tunes because people want this setup to help get an advantage when racing.
Look forward to seeing your results.