I've always thought it was apropriate that they called GT "The Real Driving Simulator" and not "The Real Racing Simulator" because the physics and the realistic driving is what they put the most effort into and to be honest, the racing isn't really that great. The AI barely qualifies as such, just following the one line with no regard for the player whatsoever. The AI cars are literally glued to the road too, even if you totally ram them at high speed going into a corner, somehow they still manage to stay on the road. Lapping backmarkers is likely to get you rammed from behind under braking, assuming you don't get pushed off the side of the track lapping them in the first place. Only 6 opponents on tracks like the Nurburgring and Le Sarthe, on any track even? Doesn't sound like a really great race to me.
Of course the real culprit for a lot GT4's failings, (not all of them(!), but certainly the unrace-ability of the higher-poly'd cars and perhaps the AI's utter stupidity also), is the lack of grunt on the PS1 and PS2, and so be it I guess - there's only so much the hardware can do. It just makes me sad that GT4 will never be ported to another platform (coughPCcough), because as awesome as it is, it has so many little issues here and there that really need to be addressed to make it the over-the-top-spectacular racing game that it ought to be. The hardware limitations would be gone so every issue that's due to that would could fixed overnight probably. And a PC port would allow for all the other little annoyances to be fixed too. *sigh/shrug*
Sorry if I've wandered a bit off topic, GT4 is totally awesome, but it's got so many issues it's depressing. edit: Anyway, the point was that GT, up to GT4 at least (I don't have a PS3 so can't comment wrt post-GT4) has always been more about the driving than the racing. So to answer the question "what the point of these cars you can't race?", they're there so you can drive them, because driving (fast, admittedly) is what the game's really all about.