While acknowledging the Indian authorities' efforts to address the problems facing the 3-14 October event, Fennell said that "a lot of work" still needs to be done.
"What is clear is that the massive work that is being done now ought to have been done before," he said.
The complaints from competing teams about filthy accommodation and poor construction have damaged India's reputation, Fennell suggested.
"People are questioning whether we should have come to India."
Nonetheless, he confirmed that all 71 teams would take part in the New Delhi Games.
Several individual athletes have pulled out of the event over concerns about security threats or the risk of catching dengue fever, including British Olympic 400m gold medallist Christine Ohuruogu and world triple jump champion Phillips Idowu, also of Britain.
"Thousands of people are working overtime virtually round the clock," reports RFI's correspondent in New Delhi, Vikram Roy.
But the mood among Indians remains "gloomy", he says.
"When India said it was going to be the costliest ever Commonwealth Games, Indians had a sense of pride because of the legacy infrastructure that was coming up - the metro services, the beautiful roads, the swanky flyovers... And now suddenly every looks like a bit of a bad joke."
India has spent an estimated 7.4 billion euros preparing for the event, making these the most expensive Games to date.