Have you try press and hold the PS button and choose Quit Game? That's how you quit the game on PS3.