The problem lies in how they keep living in there.

They need 'soul' from other shards/source. Keep them living is the same as agreeing to Ascians using people to returns their long lost comrades.

Simple as that