You're forgetting that relics are optional, it's not even necessary to do them for progression or MSQ. Hence, if you decide to do it, it's a decision you made, and not one the game forced on you.
That being said, the first time doing anima relics can be very daunting, and it certainly was for me. Just need a little patience on that step And start farming umbrites/sands/aether oils early!