Hard choice between GT1 and GT3, but I have to go for GT1. The first one had the biggest impact on me. I'm kinda surprised that noone has placed GT1 on the first spot. With GT1 the GT franchise started, there was no racegame that even came close. It was so different than other racegames, with alot of cars and tracks, more realistic physics, license tests, tuning your car, etc. It hit like a bomb, I remember everyone at school talking about it, because it was totally new.
It's a bit like with GTA. Sure, Vice City was bigger and had motorcycles and San Andreas was much bigger with many more features, but they didn't had the same impact for me as GTA 3 had. For the first time we could play GTA in 3D, with a open 3D city, which was quite something back then.
So although the successors have more features and are bigger with better graphics and stuff, for me it's about which one had the biggest impact, and that's GT1. Best racegame ever imo.