When to start using materia if at all.

[LilimoLimomo]

I don't plan on doing extremes or savages but I will definitely do lvl 90 other endwalker stuff when I reach that point .

This means you'll never have to use materia; it's simply not necessary outside of the content you've stated you're uninterested in.