Okay it turns out the old keyboard I used with the PS4 doesn't pair well with the PS5. Figured it out.