Conservatives Panic as UK Voters Turn to Nationalism

David Cameron faces humiliation at the hands of the nationalist rivals he once dismissed as "a bunch of fruitcakes and loonies and closet racists". Opinion polls suggest the British prime minister's Conservative party is heading for a landslide defeat by the UK Independence Party (Ukip) in the forthcoming by-election in the seaside town of Clacton, east of London.

No one in the Conservative high command saw this disaster coming. It was caused by the unexpected defection of Douglas Carswell, the stern and devoutly Eurosceptic MP for Clacton, to join Ukip. Carswell has resigned his seat and is now campaigning hard against the Conservatives as the Ukip candidate in the resulting by-election.

Cameron believed he had done enough to parry the threat posed by Ukip. The prime minister thought that by promising – if re-elected at the general election in May 2015 – to renegotiate Britain's membership of the European Union, and to let voters accept or reject that deal in a referendum, he would be able to profile himself as the one true champion of national independence.

Carswell has driven a coach and horses through that calculation. He says he does not trust the "clique" round Cameron who run the country from a "sofa" inside the prime minister's residence at 10 Downing Street: "The problem is that many of those at the top of the Conservative party are simply not on our side."

It is an intensely populist position. Carswell sees himself as a champion of the British people as they seek to overthrow the decadent, out-of-touch elite which has allowed immigration to get out of control and is handing power on a plate to Brussels. Such sentiments go down particularly well in the coastal resorts of eastern England, which contain above-average numbers of retired, working-class voters, who lament the way their country has been changed by mass immigration. Clacton is just such a town, situated as it is in Essex, a county containing many ex-Londoners, who moved out of the British capital because they dislike how multi-racial it has become.

Although Carswell asserts that Ukip is "not an angry backlash against the modern world", and welcomes the contribution made by immigrants, he is also insistent that Britain must regain control of her own borders. As a Conservative MP, he became very popular by calling for tough immigration controls. He now has a good chance of repeating that success as the town's Ukip MP.

In May this year, Ukip on 27.5% topped the poll in the European elections, enabling it to win 24 of Britain's 73 seats in the European Parliament. But the general expectation was that its vote would fall back so sharply by next May's elections to the British Parliament that it might well not win any seats at all.

Westminster's first-past-the-post voting system makes it exceptionally difficult for new parties to gain a foothold. But victory in Clacton would greatly increase Ukip's chances of doing so. If Carswell does manage to become the party's first MP to win election to Westminster, he may well embolden other Conservative MPs to follow his example – a nightmarish prospect for Cameron, who could find that, between now and the general election next May, he is shot in the back at frequent intervals by his own side.

The Conservatives had intended to spend the next eight months ramming home the message that they have rescued the British economy. But it is unlikely to carry conviction if some of Cameron's own MPs have so little faith in him that they decide to join another party. The chances of the official opposition, the Labour party, returning to power in 2015 suddenly look very much better. For although Labour, under the leadership of Ed Miliband, presents an uninspiring spectacle, it has at least managed to remain united.

It will not come as much comfort to Cameron that versions of his problem can be seen across the western world. Almost wherever one looks, the prospects of mainstream conservative parties are being clouded by rivals, which claim to express with greater force and sincerity the concerns of nationalistic voters.

In Germany, a Eurosceptic party, Alternative für Deutschland, has just taken a higher-than-expected 10% of the vote in regional elections in Saxony, and has emerged as a possible coalition partner for the Christian Democrats. In France, the Front National, led by Marine Le Pen, took a quarter of the vote in the European elections and has realistic hopes of reaching the second round of the presidential elections in 2017.

How do conventional parties avoid being outbid by their more nationalistic rivals? In France, some of François Hollande's socialists have themselves struck a nationalistic note, and have called for French policy not to be subordinated to German demands for greater austerity. But these cries of defiance helped precipitate the recent collapse of the French government. In Germany, Angela Merkel insists she will not do deals with AfD, but it is perfectly possible that whoever succeeds her will decide such deals make sense.

In Britain, it is quite inconceivable that Cameron will do a deal with Ukip, or that Ukip would be prepared to enter into an alliance with a leader who has been so rude about its members. Ukip will instead try to foment divisions within Conservative ranks, not just by encouraging further defections, but by indicating that it might be willing to form an alliance with a different leader, such as Boris Johnson, London's mayor. For Cameron, maintaining any kind of cohesion is now much more difficult: he now finds himself fighting on two fronts, against Labour and against Ukip.