The MP for North East Somerset caused controversy amongst students who raised questions of whether to ban the politician from giving his speech.
A video was posted on social media showing Mr Rees-Mogg involved in the fracas as a punch is thrown.
It is unclear whether the punch was in the direction of the Brexiteer, but he appears to step in front of an antagonist.
TWITTER/CHLOEKAYEX
Mr Rees-Mogg can then be seen pushing the man in sunglasses and hat away, seemingly trying to ease the tension in the room.
Chloe Kaye, who posted the video on Twitter, wrote: “A huge amount of (physical) violence a Jacob Rees-Mogg speech in UWE Bristol.”
Speaking later to Express.co.uk, the politics student said: "I went into the talk to hear about Jacob Rees-Mogg and suddenly as soon as he comes in about six masked individuals run in screaming, 'bigot, racist, sexist'.
"They're screaming, absolutely no university security to be seen. Jacob Rees-Mogg screams, 'I believe in free speech,' so he runs up to them and actually wants to start talking to them."
Ms Kaye adds: "The crowd got angry, there was some pushes and a bit of violence, but no one was hurt."
Jacob Rees-Mogg frustrated with the interruption then proceeded to continue with his speech despite the lack of security to intervene, according to the student.
"He was actually trying to hold the peace," Ms Kaye said. "He was trying to get people, he wanted to talk to them.
"He was very calm, he wasn't at all angry or with a tone similar to what they were saying.
"He came down and said, 'You know I believe in free speech' and began to talk."
FACEBOOK/PETER THOMPSON
In another social media video of the incident, the man in sunglasses can be seen talking calmly to Mr Rees-Mogg.
The man said: You're not worth debating.
"Your views are not worth debating."
The scuffle appears to start as he is then grabbed from behind and pulled away.
A third video shows Mr Rees-Mogg remonstrating with a group of masked men.
It is unknown in which order the events occurred.
Mr Rees-Mogg was scheduled to talk Bristol University's Politics and International Relations Society.
The Conservative MP after the event said: "They shouted at me but they weren’t going to hit me. They didn’t want to talk politics, they just wanted to stop the event.”
“They shouted no platform and other insults, but I’m of the sticks and stones school of thought,” he told The Telegraph.
“I wanted to stop anyone being hit because the whole thing would have degenerated. I didn’t think anyone was going to hit me so I felt quite safe intervening. I spoke afterwards, I was there for ages. There were lots of questions from the audience.”
He added he has had “worse confrontations with The Guardian” and was “absolutely fine” after the event.
