Not all cutscenes can be watched again at the inn. This is especially the case for job/class cutscenes — only a handful of key scenes would be saved.

That's the most important reason why players shouldn't skip them. If you want only the text summary, that's available from the quest log, as Mikedizzy said.

For more comprehensive summaries of the story to date, there always the Lore Train at Gamer Escape.