"Loosing my religion" is a phrase in the South that means losing one’s temper or civility. It can also refer to having lewd thoughts about someone of the opposite gender (or in Michael Stipes case either gender).