I'd imagine people complain about this due entirely to ignorance. People are always saying how skillspeed is bad because it drains TP too quickly, but as you said that's only true in fights where you could drain your TP completely. If you're never running completely out of TP then you're just doing the same damage as before, only faster.