Who said they wanted damage because other games have it? It certainly wasn't me. And if what over games have is good, then there is no reason to not want it.
I race clean, I even avoid ghosted cars online. But damage has a chance/risk element to it, so even if you drive clean and well, you can still be effected. Adding that element and the additional pressure that damage brings into a race would increase the fun because racing would feel more authentic.
You have a point saying that a simulator doesn't have to simulate everything, but it had better get the important bits. I'm pretty lenient on damage in GT because it proclaims itself as a driving simulator. A true racing simulator needs damage, because damage is part of racing. That's why I think that GT does not need damage; but since GT focuses on racing, damage would still be a good thing.
In all honesty, I don't care much for dynamic weather. In terms of value, it's probably as good an investment as damage. It would be a nice feature to have, but I don't see why you'd want it over damage.
I don't know about you, but if GT has damage I will have it on from the moment I start playing. I don't plan on tossing it aside in hours. Besides, your last paragraph to apply to just about any feature, so it's irrelevant to the damage debate.