I've never had this problem on ps4 and I used to garden A LOT. Was super easy for me, I would do what the other people have suggested. Or use mouse mode to make it easier for you?