Fishing - Veteran Trade Question

So I'm using the spearfishing gig in the Ruby Sea. Trying to catch the swordtip for some crafting. I start and catch a Little Dragonfish. I then use the Veteran Trade skill and get the message "little dragonfish are aware of your presence and have left the area". Next catch is a wentlefish but the next one is a little dragonfish.

Is this skill broken or am I doing something wrong?

Did you use a new time the Veteran Trade ?

it work like that :

Spot with 4 fishs (A B C D)
1st try : Catch fish A - don't want it anymore > Use Veteran
2nd and next try : you can get B C or D fish (but not A ELSE if you use Veteran on a new fish)

Veteran work only on 1 fish on each spot. If you use veteran on A on 1st and use it on C dish on 2nd try > On 3rd try you can get A fish (with B & D)

Ah gotcha tyvm