I have a friend in the exact same situation as you. The only way our large group of friends can play with him currently is to drop our 50+ hours of progress from phase 4 and early access and make new characters.