Tickbox allowing it to be disabled is a nice idea.
I'd have few more proposals:
- give an option so that only the person initiating the Ready Check is spammed with the results window
- if someone doesn't click Yes/No within 30s, don't mark them as X, make a third icon, let's say ?. This way we will know if someone purposefully clicked No because their cat just sat on the keyboard, or they didn't click anything because they're away from keyboard.
- allow us to choose a sound file to be played when Party Finder or Ready Check option pops up.