Sarkozy and Bruni-Sarkozy "cannot accept that private conversations have been recorded and published without their consent", the couple's lawyers Thierry Herzog and
Richard Malka said on Thursday when they announced that the couple would take legal action alleging breach of privacy.
The charges carry penalties of a year in prison and a fine of up to 45,000 euros.
His lawyer says he will take legal action over their alleged theft and against the press for receiving stolen objects.
His son, Georges, who is no longer on friendly terms with his father, has denied speculation that he is the source of the leaks.
Buisson, who began his career as a journalist on the far-right magazine Minute, is a long-time advocate of a rapprochement between the mainstream right and the Front National and other hardline groups.
He became an adviser to Sarkozy in 2005 and was appointed head of France's history TV channel, Histoire, in 2007.
Having succeeded in persuading Sarkozy to set up a ministry of national identity and immigration, during the 2012 election campaign he was the principal proponent of a shift to the right in a bid to stop a flight of voters to Marine Le Pen's Front National.
He is said to idolise Sarkozy, who is now reported to be furious with him, and to have made the tapes in order to write a book about his time with the president.