I don't know anyone in real life who has such an interest in driving games as I.
LAN battles, 24 hour races in (drunken) stints, Car trading, i could do none of this in GT4.
This is quite a significant factor in my tendency to prefer the offline play. But the offline in GT5P is very limited. In GT5, if its single payer is anything like the depth of GT4 GT mode there is a chance i will never bother with the online at all.
So no, i don't think GT is as popular as we think it is. I know people who have bought the previous games, and 'meh' sums up their opinion of it.
The outstanding sales figures are not representative of the players who enjoy this game thoroughly.
Sales arn't the problem. Some people buy the game but just don't 'get' it, or just don't 'get' driving games in general.
EDIT:
The fact the majority of my mates are predominantly PC games is a factor... And they have Xbox 360s aswell for some reason, presumably because they were 'cheaper' than PS3.