I played most NFS games from the first game through 2010's Hot Pursuit reboot. High points along the way, for me, were Hot Pursuit 2, Underground 2, and the original Most Wanted from 2005. I decided to give Payback a shot when I saw it as the free game this month. Since there were several titles I missed this decade, I can't say it's the worst NFS of all time, but it's definitely my least favorite among the ones I have played.
As already mentioned by several people, the speed card upgrade system is bad. Like Eunos, I'm not actually finding it a big hindrance to progression through the story, since you get so many of them, but it's still a pretty dumb way to do things. Just let me spend in-game money on the parts I want to upgrade. Why make it any more complicated than that?
Assessing the car handling in any game is pretty subjective, but I find Payback's to be particularly awful. Forcing a skid with the absurdly over-powered handbrake somehow has zero effect on your speed, while using the regular brake and trying to maintain some semblance of a racing line causes you to slow waaaaaaaaaaay down. Yes, yes, it's not a simulator, I get that. But I look at something like Forza Horizon as a good example of "arcade" car handling; It's unrealistic and forgiving of ham-handed driving, but it still acts intuitively. Payback does not.
As mentioned by GTBT100, the off-road behavior of the cars is exceptionally bad. There's no sense of varying levels of grip, no predictability at all to what line will or will not slow your car down. If I try and stick to the nominal "road," AI cars blast by me by cutting corners, but when I try and do the same, it seems I have about a 50-50 shot of being severely slowed down. The races become trial-and-error runs to figure out in which spots the game has decided it is beneficial to leave the course.
The sense of speed in this game is off, too. Acceleration feels fairly sluggish on most cars, and there's never really a sense of going all that fast, unless you're going 200+ mph, at which point you're suddenly piloting a starship through warp space. There's no linearity to it all.
Since most of the games I've given a miss to over the years have all gotten reviews ranging from mediocre to abysmal without any bright spots, it seems to me the series has lost it's way. In my opinion, for grab-and-go arcade racing fun, 2010 Hot Pursuit and Burnout Paradise are clearly superior. If you want to layer on a jam-packed open world, some deep customization, and a dose of realism to the handling, go Forza Horizon 3/4. Payback doesn't do anything nearly as well as any of those games.