Depends on your job and what you do with it. I use 3 1/2 bars for Bard in PVP because I have a lot of little tricks (which I forget to use half the time). I can get by with about 2 on White Mage in casual endgame.

One suggestion is to use your mouse buttons and your most easily accessible keys for your combat abilities. Inventory doesn't really need to be on a hotbar, it can just be bound to a key like "I".