I would say for GT1 it's the Normal and Hyperspeed events for me.
GT2 had the Laguna Seca Viper test and the Historic car cup. (That damn GT40
)
GT3 had a pick and mix of tough tests and races,
Special mention goes to the Skyline at Trail Mountain test for really grinding my gears.
2hr enduros, one with stock cars, a whole championship with long races in stock cars, the Yaris/Vitz series at professional level and the FGT world championship.
GT4: let me get the obvious candidates out the way, driving missions 23 and 34. The Opel Speedster cup is one and the laps at the Green Hell in the 190e mercs is another.
GT5 has the most obvious candidate for this, the Vettel X events.
The Alfa at Eiger and the Lamborghini drive at night is another good set of choices and the RX7 seasonal at Tsukuba and the Historic cup get an honourable mention.
GT5P had the F40 at Suzuka one lap magic.
GT6 brought us the Goodwood Festival time trial in the prototype X2010 and as of the 30th of May, the time attacks in the Lotus 97T.
To some and myself, these tests and races took our abilities and our patience and even our sanity to the limit.