I have both games and enjoy both, but GT provides something special imo.
The graphics in FM4 are still FM3-like but with shinier cars, (IBL does help) but the game just seems like it caters to a younger audience, or more user friendly, idk yet
With limited steering wheel options, permanent assists, lack of different weather conditions, easy difficulty, FM does have it's issues (for me)
But, rivals mode, livery editor, and autovista are cool
GT5 is still the game to offer a true driving experience. FFB is more precise and stronger, levels (tracks) are more diverse and varied (snow, wet, night, sand, etc). The little things like horns, wipers, lights, illuminated dash, interior views (l/r, and rear) really add more to the car and to the immersive experience. The physics presented in 2.0 feel tight and responsive...
Remember, these are just my impressions and time will tell which game will reign supreme, but I'm still going to enjoy both as they are both
fun, imo