Basically if you're solo or against one opponent, you get the top down and an animated driver. The PS2 can't animate a driver and 5 other cars. It's just too many polygons.
You say, "well, this game does it!" But that game's cars look like poo next to GT4's cars, don't they? If you give up polygons, you can get more things. If you want really nice looking things, you can't have as many of them.
Until 5 on 3, that is.
(That's GT5 on PS3, for those who thought I was talking hockey for some reason.)