When I was on the ps3, I noticed the same thing also. My mining was just slower. Pickaxing with someone side by side always showed them finishing off first and moving on.

On the Ps4 I am on par with pickaxing and I only notice the slowdowns when I get connection issues. (Ex: Netflix and Youtubes running at the same time I am playing)