I personally use Fraps to capture video and Handbrake for encoding. I agree that Youtube's encoding sucks but my videos haven't turned out too bad using this method.

Example: https://www.youtube.com/watch?v=hr312Yu0uZY