It's not worth worrying about until max level because you keep upgrading your gear before then.

You would probably get away without materia even in challenging content unless you were doing it in the first weeks, but it is recommended to use it for that at least because it is respectful to be doing your best and bringing your all.