Under the current system, forefeitting the bonus would be a good idea, but not an effective one. The system isn't flawless, and plenty of people know how to exploit it. Trolling, vote abandon spam, vote kicks for petty reasons. . . there are too many ways to play it against someone, fair or unfair. Penalizing soldiery would only drive people away from dungeons and roulettes once again, so the answer isn't necessarily to stack penalties.
