I don't understand why you think you need a "reason?" It's more comfortable. If you're wearing a belt it restricts your movement, he may try to incorporate stretching. You're not getting your pants wrinkled, etc.
He's not going to say anything about the fact your took your pants off, there's zero reason to talk about it.