From what I've seen, in Forza cars look "plastified" to me, don't know if it's the lighting engine, if it's the textures, but they look kind of weird to me. Someone with PCars will be able to compare both games, but from what I've heard, they look good at the same level, that maybe one looks better in some things and the other in others. So yeah, the only game that clearly beats GTS and all it's competitors is Driveclub, the rest is pretty much even between them.
But then, if it was only focused on racing, a great percentage of the players wouldn't bother to buy it. A great part of the GT fanbase comprises casual players, who don't have time to invest in the game so they can be "professional" sim racers. Then, you have those that don't have enough money to buy a wheel. Some of them play these games just because they love cars, to drive some of their dream cars, etc. The moment you scrap road cars from a GT game, you loose a huge part of the community.