In a way, flat multipliers are a diminishing returns by nature. Going from x1 to x1.1 is a gain of 10%, while going from x2 to x2.1 is a gain of 5% of your former dps. Even if you gained 0.1 in both case for the same amount of points.

Skill Speed and Spell speed are more efficient the more you put points in, in a way. Reducing 0.01 GCD from 2.50 is proportionnaly less powerful than reducing 0.01 from 2.00 GCD.
To take some obvious numbers, reducing 0.01 GCD from 0.02 GCD is an upgrade of +100% attack speed. While reducing 0.01 from 0.10 GCD is an increase of 10% attack speed.