Freedom of speech literally is freedom from consequence. The government cannot prosecute you for your speech, that's the whole point. Other private citizens can choose not to associate with you or your business, private business can refuse to serve you (banning in a game over hate speech for ex).
Shoving off all the people deemed bigots into remote corners and not letting them speak sounds great in theory. But you just force them into echo chambers where they will get no exposure to opposing viewpoints or maybe meet someone who will actually have them change their mind. They are much more likely to become radicalised in such situations.
Personally if a small private business does not want to serve someone, that's their right imo. Even if their reason for doing so is utterly stupid. Refusing to bake a cake for a gay couple for example. If they are going to refuse you service based on your sexuality then take your money elsewhere and make it known to other lgbt they should too. Forcing them to bake a cake makes no sense to me, because you are paying them when they object to your existence. Why would you? On the same premise if someone wanted a business meant to serve exclusively lgbt, they should have the right to do so without being forced to serve straight + cis people. Each business is knowingly taking a loss and customers have no shortage of other options to choose from. Only exclusions to such things should be human rights such as healthcare. Refusing life saving treatment due to your religious views should indeed be criminal.
Right wing politics is on the rise because people are frankly tired of being told what their views have to be and what they HAVE to do. And if you don't conform well you must be a bigoted monster, it's what a very loud minority of people are yelling about and being pandered too and it's gotten old. I believe in tolerance, you do you and accept others doing their own thing too. You don't have to like it, but you shouldn't be trying to punish or attack them for w/e they are or do. I really dislike religion but I have to tolerate and have some basic respect for people that practice even when their views are totally opposed to my own. Demanding more is not tolerance, you are just trying to bully people into compliance.