Hot take but I feel abandoning duties should have even more severe penalties than they already do.

The point of the bonus for doing roulettes is exactly that - a bonus. It isn't an obligated gift for logging in, you don't deserve it just for doing it. By doing a roulette you agree to do whatever is handed to you AND are given a reward for doing so. If you don't want to do a a particular dungeon then you can queue normally, there's nothing stopping you.

My personal suggestion is that if you abandon a roulette duty within the first 15 minutes you should lose the ability to get the bonus for that particular rou for the rest of the day until the reset. Forfeit the dungeon? Forfeit the bonus. Simple as that.