Physically, anyone with normal working hands and feet can place the same inputs into the pedals and wheel. The biggest problem is the mind. Some people have habits that they dont even notice themselves which slow them down, while others just cant quite figure out on their own the absolute fastest way around the track.
Most fast drivers can figure out the "consistently quick" way to get around the track, which is what is used in races most of the time. Then there are the guys who go beyond that to find the absolute fastest lap possible from experimenting different things throughout the track. They already know the fast line, they are now looking for extremities in each corner.
If you can put aside your bad habits, then figure out the fastest way possible around the track, the next step is then executing it. If you can figure everything out, and execute it, then you will probably be fine.
The thing to think about also, is just because you are slower in a time trial, it doesn't mean you would not give those drivers a competitive race, after all, you are competing for fractions of a second, which could be a car length or two.