He made nine changes to the side that drew 2-2 with Chelsea in the Premier League on 3 January.
"We have many players on the edge and we play again on Wednesday," Wenger said. "When you don't win the selection is always questioned. I believe we had an experienced team - eight or nine internationals. It would be easy to think that was the problem.
"When you're against a team like that they're very difficult to beat. Defensively you cannot afford the mistakes we made."
US international Eric Lichaj opened the scoring for the home side with a header midway through the first half but Per Mertesacker equalised for the visitors soon after.
Lichaj bagged his brace just before half-time with a spectacular volley from the edge of the penalty.
Wenger, serving a touchline ban due to his comments over officiating in Premier League matches, was forced to watch from the stands as Rob Holding conceded a penalty that Ben Brereton converted in the 64th minute to stretch the home side's lead to 3-1.
But there was hope for the north Londoners. Danny Welbeck reduced the deficit with 11 minutes remaining. But the comeback could not be completed.
Kieran Dowell netted from the penalty spot to restore the two goal advantage.
Gary Brazil, Nottingham Forest's caretaker manager, said the result should serve to motivate the players and the club. "This football club should be playing against the Arsenals of this world, and they should be beating them," he added. "These fans deserve to be watching Premier League teams and that's what we've got to drive towards."