But Mr Houellebecq failed to convince the court to ban the film because he had signed a release with director Stefan Ruitenbeek and the Dutch Keeping it Real Art Critics (Kirac) collective after the scene was shot in Paris.

"I was tired, the day had been long and the necessary wine had already been drunk," Houellebecq – considered by many to be France's greatest living novelist – said. "I don't have the ambition to become a porn star at my age."

Qianyum Lysis Li, the writer's third wife, told the director over dinner last year that her husband wanted to "make a porn film to counter his gloom".

The men then swapped emails discussing plans to make "a work of art, in which the distinction between fiction and reality takes shape as a game with the rancorous paranoia of your enemies". The director filmed the author having sex with Jini van Rooijen, an actress and philosophy student.

The author is famous for his provocative works which have dealt with themes including sexuality and religion. The only restriction in the contract he signed with the director was that the writer's face and genitals could not be captured in the same shot.

Houellebecq said the contract had put him at Ruitenbeek's "mercy". The judge found that while it was "far from balanced" and gave the Dutch director extensive rights, it was not illegal.

Houellebecq was ordered to pay €1,400 in costs after the court in Amsterdam decided there was not enough evidence to show that his judgment had been impaired by the wine or by depression.

"It is incomprehensible why Houellebecq participated in the recordings if he found the agreement really problematic," the judge said.

Relations between the author and director soured soon after the shoot and deteriorated further after a trailer for Kirac 27 was released in January, showing a shirtless Houellebecq kissing and fondling a woman on a bed.

In the trailer, director Ruitenbeek said Houellebecq contacted him after his honeymoon in Morocco was cancelled.

He claimed he told him he was depressed because his wife had spent a month booking prostitutes for the trip, which was not going ahead.

"I told him I knew plenty of girls in Amsterdam who would have sex with a famous writer out of curiosity, and that I would arrange the hotel for him if I had permission to film everything," the director said in the clip.

Houellebecq said the trailer had damaged his and his wife’s reputation. He was also reportedly furious after Ruitenbeek told the Vice website that Houellebecq was "really good in bed".

"Mr Houellebecq is seriously considering an urgent appeal," his lawyer said after the verdict, which followed a similar defeat in the French courts.

Houellebecq was tipped to win the Nobel Prize for literature last year, but it went to Annie Ernaux, who called his ideas "completely reactionary and anti-feminist".