Really, it depends on the duty, and then it depends on your party members.

If you land in a leveling dungeon with all four of you being level 60 and well-geared, even with the synced aspect of the duty, you can probably speed run it with the exception of a few dungeons (please do not pull the entire first room of AV; you will all end up pulverized no matter what, and also that large pull in Dzemael with the frogs can be a sticky situation for anyone--experienced or not).

If you load in and see that one or more people are not synced down, or are a few levels above the duty sync yet poorly geared, probably not a good idea to speed run, especially if the poorly geared individual(s) is/are your tank and/or healer. If there is ever a new person bonus, also probably not a good idea to just bulldoze through the duty either.

I don't encounter a lot of people that try to speed run the leveling/story dungeons (however, there are still those out there that do), but I particularly find it annoying when people want to steamroll through everything. Even if I am level 60 and only there for the tomes bonus the roulette gives me. Because usually, there is one person there trying to learn the dungeon, or trying to level up/learn an alt class/job, and speed-runs don't exactly serve as the best learning environment. But that's just my opinion.

(Obviously, I don't want to spend 45 minutes in a leveling dungeon either, but if you have people that are new, or leveling up a class/job, some courtesy and patience is always much appreciated by them, I'm sure.)