Sorry for being so late to the party. I have to tell you, what you have told us makes this sound very one-sided - his. Your husband seems to be very controlling and unwilling to entertain ideas that did not originate in his head. I'm glad you are seeking counseling on your own. My suggestion, like others', is to seek legal advice. While I get your feeling guilty for seeking sex outside of the relationship, you did so for a very good reason: he decided there would be no sex inside of the relationship. His refusal to discuss the issue, his waving his hand and telling you to have fun with "your boy," and his threats do not bode well for a happy relationship. Seems like you have to ask yourself what you get out of staying in this controlling, sexless, and apparently affectionless relationship? Sounds like your counselor is trying to steer you on a path for answering that question.