Honestly, when I run caravan, I let everyone know that if they don't get a spot for the run they should sit out for that run and disband from the party. When the next run comes along and they get a spot in I'll be sure to invite them back. There is no reason not to have all 8 players who are doing caravan not be in the same party. If you really wanted to get back at them though, just talk to the NPC or one of the chocobos in the caravan and keep it in dialogue. The caravan wont move forward until you let it go... so they'll have to invite you in if they want to complete the run.