Yep, there is really something weird about the timer.

I left Amdapor Keep at 8:50 a.m. (GMT+2), and recieved the 15 min. penalty, but I never opened the window to check.
I was standing in Old Gridania at this time. Well, I watched some youtube while waiting.
At 9:27 a.m. I opened the DF window, and it still said 15 min remaining. I kept it open until I saw it actually count down to 14 mins, at wich I decided to just enter my Inn room....
But when I opened up the DF window inside my inn room, the bloody timer had reset itself to 15 mins again.

It is now 9:45 a.m, and there is still 3 minutes remaining.
So yeah, its actually been a 1 hour lockout due to this.