Sounds a lot like you suffering from lagging spikes.
Try to use ethernet if possible.