Didn't read any of the posts, just going to state my opinion and be on my way...
I'm a bit GT biased, I'm sure, because I've been a fan since day 1. Only owned Forza 4 for 6 months.
Long story short, I got rid of Forza, it just didn't do it for me.
In some (many?) ways, Forza is superior, but when you're in a Ferrari blasting around the Nurburgring in Forza, the experience absolutely pales in comparison to GT5. Doesn't even come close.
One of the things I liked about Forza is that the cars can feel more "alive". For example, when at a stop, you can rev your engine and the whole car shakes and moves- in a realistic way too. In GT5 nothing happens.
The best car sounds in GT5 I think are better than the best of Forza's, but Forza's sounds are far better on average. Forza's engine and exhaust sounds are pretty good, but they seem to all sound the same after a while. They also don't sound very realistic, like they're tweaked to be more intense and dramatic than IRL.
I also really like the Autovista feature in Forza. I can't believe GT doesn't have something similar. What's the point of owning some beautiful cars if you can't get close to them and check them out? If PD doesn't implement a similar feature in GT6 they will have really dropped the ball.
I would say that small (like 900kg and under) cars in Forza are much funner to drive and feel more realistic than in GT5. When you take them to the limit, there's a sense of drama, even danger. In GT5 they are rock solid and stable, which is not at all accurate. I think the same is true for old, classic cars.
But at the end of the day, GT5 takes the cake. Superior graphics, superior high-speed physics, and other minor details make it a more immersive experience, and push it (just barely) into the simulator category. Forza to me is still an arcade game, although a very good one.