While there is nothing wrong in the rules sense, it's more of a courtesy thing to hold off attacking a hunt mark right away until more people come along.

So it's not wrong. It's just polite to wait and let others join in.

I've long since learned that if I don't make it in time to a hunt target, then I just shrug my shoulders and move on (it will respawn again after all).