It is difficult. I have spent about an hour or two on Cataluyna Cicruit Experience stage 3 alone and have just cracked it now. Go figure I do the full lap on my 1.5st attempt.
If you want to get gold in everything, you can do it, it just won't come overnight. The first time I attempted Gold in all the license tests back in GT2 days, it took me more than a month. Now I think they are too easy.
It isn't controller settings, or car settings. You set the controls up for how you would like, whatever makes you the most comfortable. If you aren't getting the times, it is because you are doing something wrong. You need to analyse what you are doing, where you may be doing it wrong and adjust, or try different things and see how you go against your ghost.
I find a lot of time is made up on corner entry. Not late braking, but braking early enough that the car is setup for the best exit possible. Corner exit is all about how the car was setup before the apex, and it isn't always that obvious when you are hard on the brakes thinking you are maximizing your speed into the corner, where it is actually costing twice as much time on corner exit.
For reference, my controller settings are.
Dual Sense
Controller Steering Sensitivity: 7
Vibration: Default
Buttons: Default
I use triggers for brake/accel, and I never use both brake and accel at same time. As I right foot brake on a wheel, I replicate that with the controller.
I do use stick extensions, kontrol freeks for a bit more control on steering.
Assist Settings:
Everything 0, disabled, incl ABS.
Edit: Also I find if I can't beat a task, taking a break, coming back an hour later or the next day helps a lot. Breaking a controller never helps, as satisfying as it may be in the moment.