Just use the manual filters which are activated by using L1 + Square, X, Triangle, and Circle. They are in the filters menu of your character config. You can customize each of the 4 filters individually so you don't have to micromanage in the config whenever you want to change jobs. You will probably want to disable the auto-filter (the one that switches if your weapon is out or not).

Alternatively, you can just keep the auto-filter on and click the left thumbstick while not targeting anything in order to draw your weapon. This will make the auto-filter change to the "Others" mode where you only target Enemies and NPCs.