I don't think Callmedave gives a fuk either way. He is just a fairly wealthy puppet for the proper, really wealthy money men backing him. He will say, and do, whatever they tell him. If they told him to, he would happily shag a pig live on TV.
You're right, the only big weapon the "outers" have in their armoury is the pommy Englander's fear of Jonny Foreigner. The "inners" will just continually, over and over, and over tell folk that leaving the EU will have virtually no impact on immigration. It will eventually become so obviously true that most folk will wonder why they are having a referendum at all.