This is probably the crux of it
My sophomore year one of my roommates was a football player who had trouble with frats... let's call him "Sam"(that wasnt his real name, but just to conceal identity)
One night Sam was in the student parking lot and a big F250 truck full of drunk frat guys backed into Sam's car and completely smashed the rear of Sam's car like a soda can. Of course the frat guys drive off, but Sam was able to get the plate number, called the cops, the frat guys were arrested in the parking lot by the bookstore. Of course the frat guys and the entire frat were pissed at Sam and harrassed him non stop over it the next few weeks for snitching.
One night they were harrassing Sam over social media so Sam called them cowards and told one of them to come to our dorm to say it to his face. Of course instead of that one guy coming, about a dozen of them showed up. 12 on 1, yeah that's a fair fight. Sam just took the high road and called the RA who got campus police to escort them out.
Things settled down for a while but a few months later Sam got invited by a sorority girl to their date party mixer downtown that was co-hosted by the sorority and another frat. The frat that hosted it wasn't the same one who had the problem with Sam, but they still kicked him out of the party, because the girl who took Sam to the party was being pursued by one of the frat guys in the frat who was co-hosting the party and he had sour grapes. Of course the sorority sided with the frat, and therefore the Sorority Girl who invited Sam fell in line too, of course you always have to side with your frat/sorority or else. Basically all threw Sam under the bus. He had to call me to give him a ride home since they wouldn't let him take the bus back to campus. He was really bitter about that but again took the high road.
Things finally came to a point at a baseball game in the Spring. One of the frat guys from the truck incident and his dad began to jaw at Sam on the sidewalk area behind the stadium 3rd base line. Sam decided to stick up for himself and began to jaw back with the dad and frat guy telling him to leave him alone. The dad grabbed Sam by the collar to get in his face and Sam delivered a punch right into the dad's cheek. Instead of fighting back the Dad called one of the assistant AD's to eject Sam from the premises. Apparently this dad was a big donor and said he has so much influence he just needed to make one phone call to get Sam kicked out of TCU. Sam never got kicked out but it did worry him for a while. But the police weren't called because technically the Dad was the aggressor. Those frat guys never messed with Sam again as far as I was concerned.
Long story short while taking the high road is usually the best route to go sometimes a good punch to the face is the best way to put an end to shenanigans for good. Because sometimes fighting back is the right message to send to tell people to knock it off.
Not all frats are bad. Not all frat guys are bad. There are good frats and bad frats. Within frats, good or bad, there are good guys and bad guys. Judge the individual and not the group. My room mates my junior/senior year were frat guys and they were all good people.
It's just a bunch of sexually charged 18-22 year old guys who are in competition for the women on campus. That's the crux of it. The frat guys think they run the campus and are the most alpha men on campus, and think that by not letting athletes into their party they are showing themselves as "above" the athletes and more important, in an attempt to impress the females of campus. Hence why the football guys took over Old Rips when I was a student - it was their place they could go and party without worrying whether or not they were welcome.