Gypsies are gathering in the South of France to celebrate their saints

Drive south beyond lyrical Arles and the lines of skinny poplars and clusters of villages, peopled largely by aging boules players, will soon give way to a green, marshy wetland where rice fields run into freshwater marshes and herds of white ponies scatter among black bulls and pink flamingos. This is the Camargue, a tough and uncompromising landscape unlike any other part of France, covering over 80,000 hectares of the Rhône delta, forming Europe's largest river delta.

It's a mesmerising place, where the mistral, that harsh, north-easterly wind, blows almost relentlessly, even in the summer, when the Camargue crackles under the hottest sun. Mosquitoes love its brine lagoons and marshes, which make up a third of its landscape, so do the 400 species of birds which flock here, most famous of which are the flamingos. There's something dazzling, almost mind-boggling, about the site of these glamorous birds, les flamants roses, flocking in their hundreds to the brine ponds and picking their way among the sea lavender, tamarisks and reeds.

But more than the verdant wildlife, it's the white horses, right, or chevaux Camarguais, who are the passport right to its beating heart.

It's appropriate, too, that such a hypnotic, haunting landscape should be held in such deep affection by gypsies, who make their annual pilgrimage from all over Europe in the week leading up to 24 May to celebrate the festival of their patron saint, Sarah-la-Kali, or the Black Madonna; ever the outcasts, Sarah has not – yet – been recognised by the Vatican.

The celebrations focus on the Camargue's largest town, Les-Saintes-Maries-de-la-Mer, on the coast. The Marys who give this dazzling white town its name are Mary Magdalene, Mary Salome and Mary Jacobe, who were said to have sailed here from Alexandria following the Crucifixion with Joseph of Arimathea and their Egyptian servant girl, Sarah. Legends swirl around who exactly Sarah was, and why the gypsies hold her in such deep reverence, but one version suggests the Marys used her cloak to walk to safety on the shore in a raging storm.

Whatever the reason, today the pilgrimage is ostensibly a religious festival, but the scores of musicians who throng the streets are a vivid reminder that the gypsies also welcome this as a chance to converge together to celebrate what it means to be a gypsy. For a week during the festival, the town breathes gypsy spirit, their caravans parked higgledy-piggledy on verges, with lines of washing strung between them. Dogs, chickens and children cluster around the caravans, and there are horses tethered there, too.

The Church of the Saints in the middle of this small town is a focal point that throngs with people when the statues of the Marys, usually stored in the upper chapel, are brought into the main church, alongside Sarah, who lies in the crypt. It's not easy to get through the crowds of gypsies but anyone with the energy to fight their way in will be rewarded by the moving sight of gypsy babies being lifted to touch the relics as they're lowered into the church in flickering candlelight.

A sort of tension pulsates through the town among the fairground rides in the days leading up to the 24th, when Sarah is carried in a strange, slow, rambling procession through the town to the windswept beach. Here, muscular gypsy men on white horses wearing black hats carry the effigy to the beach, where four gypsies will bear her into the water. They are followed by a colourful throng of women in colourful dresses and men, some dressed in incongruous pin-striped suits, who wade knee, chest, even shoulder deep into the water. Little girls with dark eyes and long locks swim through the sea in their flamenco dresses, and there's a sense of cultures, races, religions colliding as gypsies walk beside some of the hundreds of tourists who gather to watch this extraordinary spectacle, as well as hippies and horse-drawn travellers from as far away as England.

After Sarah is returned to her crypt, the crowds disperse through the streets where musicians cram every corner and Balkan bass melodies are played outside every pizza restaurant. The Gitans, who originate in this part of France but have largely settled in Spain, are rightly famed for their dancing skills as they originally took flamenco to Spain.

The ceremony is repeated a second time in the days after the 24th, when the Marys are also processed into the sea. On the third day, there's a further ceremony for the Marquis de Baroncelli, born in Aix in 1869 but who settled in the Camargue. He was a passionate champion of gypsy rights, including that to worship Sarah.

After the heaving intensity of the gypsy festival, it's a relief to escape out to the salt flats and reed beds beyond Salin de Giraud to the huge, mostly empty beach at Piemanson, or take a tour to Aigues-Mortes, built by Louis IX as a port for his Crusaders, in the north-western tip of the region. Here the streets are lined wih ice-cream stalls, with their palette of candy-coloured sweets, and clothes shops slung with the white linens favoured by holiday-makers.

There's a picturesque drive back into the Camargue through the vineyards around Bac du Sauvage, to cross the Petit Rhone by chain ferry and wind on through Cabanes de Cambon to the ornithological park, which is the best place in the area for watching the flamingos.

During the spring and into the summer the courses Camarguais, or blood-free bull-fighting, takes places in dusty arenas across the area. Here, local boys in track suits, the raseteurs, pluck rosettes from the horns of bulls as they charge around the arenas. Equine life is everywhere in the Camargue, with lines of écoles d'equitation (equestrian schools) advertised by the roads, where tourists ride in big Western saddles on white ponies across the salt plains and through meadows lined with vetch and clover.

When the gypsies have dispersed, the Camargue returns once again to the guardians, always picturesque in their pale Provençal printed shirts and narrow-brimmed hats, who will look after the land, the bulls and the horses, until the gypsies return again at the same time next year.