I keep getting this too. The last time (this morning), I logged in and was just standing around Camp Dragonhead and got a random invite with no message. It could have been a FATE group looking to fill slots, but I didn't want to join one (I certainly never asked in /shout), so I declined and moved on.
I vaguely recall something happening like this in Warcraft. I accepted it, back then, and it was an extra annoying gold spammer who wanted to advertise to me in the most invasive way possible. So I would say, just decline and move on. Blacklist the person if they're really bothering you.