In order to understand when to use Clemency, you need to use your gut instincts.
  1. Judge if the healer seems to be competent. Are they casting damage spells? This is a sign of confidence; that healing you is such a breeze that they can get away with not healing you. If healing you is that easy for them, trust them.
  2. Have they, at any point, demonstrated keeping you alive under sustained pressure, like a huge pull with 20-30 mobs attacking you?
  3. Have they not struggled to heal you? Haven't had to cast Cure 5 times just to get you to full health? If they have, that's a sign they may be undergeared or inexperienced, and may not be able to save you at some stage.
Only actually use Clemency if the healer has failed the above criteria, your health is imminently approaching 5% and a heal not being casted now will be your death.

This is all assuming you are thinking of using Clemency on yourself. If another party member is near 5% health and devastating mechanics are incoming, use it immediately or use Cover if mid-combo. What may aid this decision is again the demonstrated competence of the healer and whether any party members are already dead.

If you're doing it right, you won't use Clemency at all in most dungeon runs, but will encounter plenty of situations over time where it prevents a wipe and turns the tide back in everyone's favor.

I've personally used it a lot during wall-to-wall pulls of Drowned City of Skalla, Ala Mhigo because it's somewhat tightly synced and hard on some healers and The Burn because the last boss wipes a good amount of parties.