I think maybe you haven't seen the neighbors peek into the patio lap dance area because they don't do it every time, and because, as you've said, you haven't gotten lap dances there. The lap dances happen in the patio, in the back, and I've seen the neighbors looking over. If you haven't gotten lap dances, then you probably haven't been in the back yard/patio.
You've been twice, and I'm glad you haven't seen the mother and kids. It means they're probably not being exposed to the party downstairs much. But I've seen them. I can describe them. They've used the front door to return home.