For me, I'm glad I started elsewhere, because now when I need to visit my home city, I'm quite happy to make the trip. Even if I do spend most of my time in Ul'dah. Adventure out, its fun that way. There is lots to explore, take the time and do it.
There's 5 main areas to check out. La Noscea, Black Shroud, and Thanalan include the 3 main cities. Each area contains content for ranks 1-50 and beyond.