The whole thing with pugilism is the fact that, while punching isn't that strong, you can get in a lot of hits in quick succession. This is explained at the start of the pugilist questline.