You could have the world's best physics engine pushing sliding cubes around a black and white map with no textures. It would suck.
A cube on a texture-less world is an extreme way of putting it, but you're completely correct.
The way I put it is this: you can have the best physics system in the universe but if what is getting rendered to the screen does a rubbish job of reacting to said physics - such as not seeing stiffly-sprung cars bouncing over bumps - then the physics won't "feel" anywhere near as good. Imagine if you're in helmetcam and neither the track, visible portion of the vehicle or the camera itself move up/down/left/right/forwards/backwards; you're going to have a damn hard time picking what the suspension is doing, the extent of weight transfer under acceleration/braking etc.
In a real car you can feel what your vehicle is doing through vibrations in your seat, pedals, wheel, gear shifter and so on in conjunction with what you physically see happening around you. Short of sitting on a giant metal spike and connecting it to the force feedback motors in a wheel or gamepad a game is already short one medium of response (what comes up through your seat), and in most cases can't provide feedback through pedals or shifter either. When you take those out, all the game is able to offer is feedback through the wheel - force - and on the screen - visual - so they have to be as accurate and as high-fidelity as possible.
It's important to make the distinction between "good" graphics and graphics that convey exactly what physics systems do. "Good" graphics would be something like Crysis 3. That's not the sort of graphics that are important in a racing simulator. What you want is to be able to see tiny, small, very detailed changes in the environment - if the physics system determines that a wheel+tyre needs to lift up a few centimetres in an oscillatory manner when you drive over a rumble strip
then you should be able to see that happen. If you can't see that happen (ignore the fact you can't actually see this happen while driving unless in an open-wheeler; replays/trackside cams bare all) then how will you know that the physics system simulated it? Obviously having "good" graphics helps to an extent; it's easier to see what the physics system is doing when you have circular wheels as opposed to octagonal ones, but there is a point where the quality of textures, number of polygons etc become irrelevant...
Record a clip of a sim that has passable fidelity in its graphics, do the same with one that has much higher fidelity (that is you can actually see wheels and whatnot reacting to the environment), show the resulting footage to someone and ask them to tell you which sim they feel has better physics. Easy, simple test with an obvious result.
rFactor 2 has a tyre deformation model. rFactor 1 does not. Guess which game feels better to drive in a t-bar cam open-wheeler?
). The point is SMS is relying on the crowd that actually buys into this product to direct it and thus make a "perfect" formula for future pcars or other sims. To sit there and just be negative for no reason other than to be negative, is squandering a chance that we haven't really had. 
