There's a daily quest to get a Mysterious Map by doing Expert Roulette, so doing that every day cuts down significantly on how many seals/tomes you spend on Alexandrites.
FATEs also have a small chance to drop Alexandrites if you do them while equipped with your Animus (or Novus or Nexus). It's not a great primary source, but if you're just waiting on a queue or not doing anything else, might as well.
But yes, the materia part is a pain. For most stats, you can put up to 44 on the scroll. That's 11 level 1 materia, 11 level 2, 11 level 3, 11 level 4. And that's assuming every meld is a success, which they won't be. On the bright side, you can redo the stats one at a time later with more Alexandrite. So if you can't afford all the Det (e.g.) you want right away, you can put something else on and replace it with Det later.