Saints have refused to accept the refereeing chiefs' decision to rule out action against Mark Clattenburg after he was accused of verbally abusing midfielder Adam Lallana.
Southampton are considering an appeal to the FA over the ruling by the Professional Game Match Officials Limited (PGMOL) that Clattenburg has no case to answer for his conduct in the December 29 Premier League 2-1 defeat to Everton.
Southampton wrote to the referees' governing body on New Year's Eve, to formalise their complaint against Clattenburg. But after a review the PGMOL have ruled there is no case to answer.
In a letter seen by the Press Association, Southampton claimed Clattenburg told Lallana: "You are very different now, since you've played for England - you never used to be like this." Now Southampton have pledged not to let the matter lie.
A club spokesman said: "An official insulting any player, no matter his intentions, is clearly not acceptable behaviour. For this reason we do not accept the verdict of the PGMOL in relation to this case, and do not consider the matter to be over.
"Under the circumstances we do not feel it appropriate for Mr Clattenburg to officiate in any of our matches until this matter is properly resolved."