The problem we have, is that you're already a couple of levels more complex in your argument than the typical voter probably is. Too many foreigners = EU's fault = vote Leave = Less foreigners. The reality as
@Touring Mars points out is very different on a number of levels. As I see it (and I might be wrong) the problem it will cause with 'illegal immigration' is in letting people in in the first place. Stopping someone at a border is probably much easier and more effective than trying to find and remove them once they've been here for weeks. I'd also suggest that the element of society most voters want to remove, is those that can and will fly under the radar given the chance.