I think out of the 100 or more runs i've done in Fractal and Nevereap the Ninja chest has only ever dropped once.I wasn't a ninja at the time of course.
It's RNG though like everyone has said.I bet there are other players who see this chest piece the whole time.And i secretly hate them for this XD