So, here's the thing: the wall goes down after 2 minutes. The AFK timer is 2 minutes. That means that you only get kicked if:

1. You don't do anything after zoning in (no buffs, no mount, no nothing)
OR
2. You stay AFK after the wall drops

So, if you get kicked, yeah, you're AFK.

Edit: And, starting the timer AFTER the wall drops? Sorry, I don't need the first 2 minutes of the game to be with us down a person.