Winners in 1976, the south coast side last made the final in 2003 and ensured their place in the fifth round with a 2-0 victory against Yeovil on Saturday.
Guly do Prado and teenager Sam Gallagher netted the goals to secure progression in a competition which has taken heightened importance for Saints given they are all but certain to finish mid-table in the league.
That increased focus was highlighted by strong line-ups in the matches against Yeovil and third-round opponents Burnley, with the strength in depth available making veteran goalkeeper Davis confident of a cup run.
''I really believe that we've got the capabilities of going all the way,'' he said.
''Whether we get that bit of Lady Luck on the way remains to be seen, but what I am pleased with is that lads who don't play in the Premier League every week when they get their opportunity they step up and play well.
''That's great news for the squad as well that whoever puts the shirt on plays to that level, and that's what we're looking to do to keep the competition going, and you never know where it might take us.''
A cup run would certainly put a smile back on Southampton fans' faces after a wonderful start to the season petered out and 2014 began turbulently.
The Italy international allegedly head-butted and punched Fonte, making a January exit look increasingly likely.
Osvaldo's departure would see 18-year-old Gallagher bump up the pecking order - and deservedly so given his performance against Yeovil.
Brought on as a second-half substitute, the 6ft 4in forward proved a nuisance and scored an impressive first senior goal to wrap up victory.
''I was very pleased for him,'' Davis said. ''Sam's impressed me since he's come into the first team squad and he always seems to squeeze one past me in training which I've reminded him to cut that back a little bit.
''He took his goal very well and not just his goal, his all-round game as well.
''He was good when he came on and it was good for him, and had I not been at the other end of the field I might have gone up to the middle of the field and shook his hand.
''We've got a manager that wants these guys to come through and play, and be involved.
''Secondly, they're involved in training every day and when you step into the first team it can be a bit of an eye opener for some of the youngsters to step up to the pace as quick as they can. Sam and Harry Reed have certainly done that and got their rewards again.''
Davis and Co's attentions are now on Tuesday's return to Premier League action against Arsenal, while Yeovil face high-flying Derby in the Championship - a match made less daunting for the relegation-threatened Glovers after an impressive display at St Mary's.
''We're a young side,'' midfielder Joe Ralls, on loan from Cardiff, said.
''It is brilliant for all of us boys to test ourselves against quality players, some big names out there and lads that are being to other places.
''We knew it was going to be a tough game but we wanted to come and show no fear. Have a really good go.
''We made a good account of ourselves, especially in the first half.''