You might lose some DPS, but it will be a literally marginal amount, almost completely offset by the increased strength. A new body piece will give you additional defense and vitality, which is also useful and worth considering.

But, here's the big point: being casual is fine and all, but that very thing makes this topic strange. If you aren't really pining to do the new Extremes or the Second Coil, then there's no need for you to gear optimally. If you are actually going for them, then strict Crit/Det stacking will completely screw you over because you won't have enough Accuracy to tackle newer content.

If you're looking to tackle the Second Coil/Others:

Helm: Ignore Soldiery helm, High Allagan is better.
Chest: Get Soldiery Body. The Accuracy is essential.
Gloves: Excellent secondaries, but I think the Accuracy on the High Allagan is more desirable.
Waist: High Allagan, no question.
Legs: Soldiery.
Feet: Soldiery.
Neck: Soldiery.
Earrings: Soldiery.
Wrist: High Allagan.
Rings: Both, since there are only two choices anyway.

First purchase? Probably the Body.

Keep in mind that the set listed above is a matter of opinion and is based on my idea of what might be Best in Slot while focusing on Crit and Determination and ensuring that Accuracy is "capped" without food.

If you are not looking to tackle the new content:

The Earrings are probably your best way to actually get an upgrade in DPS right now.