Spectating definitely needs to be their for PvP seeing as it's a coliseum and allowing onlookers is a given requirement.
As for spectating in a dungeon, I think it kind of depends on what the purpose of it was. I'd be up for it being linkshell/free company restricted for the purpose of teaching, or for the spectators to do some video recording. It's a big thing currently to do video demonstrations and let's plays and this adds an feature that relates to that. By setting it up as a linkshell/FC only idea, it can be defined in that particular groups guidelines that they do allow spectating during raids and that joining them is subject to your agreeing, as is the case for joining any group. If you don't agree with their politics, you don't join them.
The biggest problem that may arise in this case, in my opinion, is a group gaining an unfair advantage over others because they have more eyes and minds available for a given content. With a free roam camera (even one that is restricted to the room that the party leader is in), it lets spectators provide feedback about things that are going on around the party. Said party maybe be focussing on the current battle, and some may miss certain things happening, but Spectator 1 sees it and informs the party. It shifts the balance. They aren't a natural part of the design of the event. With PvP this is somewhat different, simply because it is player versus player, and not player versus the system.
For public groups, it either needs to be a unanimous vote to have spectating or disallowed completely. The problem is, as others have pointed out, that some people may not feel comfortable with being watched, and similarly, may not be comfortable with the idea of maybe having to vote for this being enabled. And we all know how bad auto-translate has been up to now at communicating intentions. If a person doesn't understand the intention and is then required to make a decision which may or may not end up with them being removed from the party, then you end up with a negative situation because the party leader at the time really wants to be watched.
With the content finder, this isn't as much of an issue because you have make it clear through the system what the intentions of the party is. If shout groups aren't something that's going to be the norm and that the finder can be used as the better alternative (such that it makes it very clear what you want to achieve like desired party members, particular loot rules wherever that may apply), then it reduces the potential for problems.
Private group spectating, I support assuming it doesn't interfere with the design of the raids. How much is restricted should be monitored and adjusted as necessary. It's difficult to assess completely without first trying it out.
With public groups, however, a decision to implement it needs to take into consideration the whole population not just a select group of people even with the proposed features such as voting and not displaying the results. It needs to have a lot more data collected from players who come into the game at and following official launch.