The altercation that broke out between Cam Newton and opposing coaches at a youth football tournament was the result of trash-talking gone too far ... so says two men involved in the scrap, who claim things only got physical after the ex-NFL quarterback initiated contact.

TJ and Steph Brown -- coaches for Top Shelf Performance, where Cam used to help out -- shared their side of the scrap on Monday ... and they insist the former Panthers superstar is to blame for the ugly scene.

"It just should never have gotten to that," TJ told Fox News Digital ... adding Cam had been jawing at him throughout the weekend and demanded they place a wager on their game.

"Cam jumps out of his car and yells, 'Hey, all that sorry s*** ain't gonna work today. What we betting today? What we betting?'" TJ continued. "I'm like, 'I ain't betting you, bro. I already beat you three, four times. You got to beat me first.' He like, ‘What we betting? I got plenty of money. I ain't ever running out of money.' That’s when I told him, 'That's your problem. All you care about is money. You think money is everything. You can't buy me.' And I walk off."

TJ claims just three minutes into their matchup, Cam continued to taunt his old colleagues ... saying, "'Hey, I know that play. That's my stuff. I taught y'all everything y'all know. I'm y'all daddy. I made y'all. I’m the reason for everything y'all got going on.'"

When the coaches confronted the former NFL MVP, TJ said Cam grabbed Steph ... which led him to react by throwing a punch.

"He's just got to understand we're people," he added. "Nobody's just going to let you go around disrespecting and talking crazy to them. I know how to brush it off, but not everybody's like that."

Worth noting -- Cam's C1N Red team was still able to end up with the victory on Sunday ... with Newton congratulating his squad on the 15U championship.