Strachan was unveiled as national team coach earlier this month after a poor start to the World Cup qualifying campaign cost Craig Levein his job in November.
The Scots sit bottom of Group A with just two points from four games and little hope of booking their place at Brazil in 2014.
But, like Strachan, Caldwell is not willing to write off Scotland's chances just yet.
He said: "When there are as many points to play for, there is always a chance.
"It's obviously a long shot. We're bottom of the group at the moment so we would need to win the next three games to put ourselves into next season with a real opportunity.
"To do that is going to be very difficult but, knowing the manager, he will come in trying to qualify and so will all the players.
"We won't give it up until it's mathematically impossible."
The Wigan defender believes securing a morale-boosting first win of the campaign could make all the difference.
He added: "Confidence is a huge thing. If we can win games it gives you confidence to kick on and improve.
"Results can affect that and you can see that the team's performances have been a bit edgy of late.
"We've been trying to win games but we've not really had the confidence to go and do that.
"Hopefully we can get off to a good start, get a bit of momentum going, get the fans behind us and kick on from there."
Caldwell worked under Strachan during their time together at Celtic and is keen to see if an international role will result in any changes to his managerial style.
He said: "It will be interesting to see if he changes or how he changes.
"He's an experienced manager, he's managed at a lot of clubs and he has managed Celtic, which is a huge club in Scotland.
"He understands the Scottish media, the supporters and the people so I think he's got great experience for the job and I think he's ready for the job."