I would imagine that in order for a "swinging lifestyle" to work both partners must be able compltetely satisfy each other together on a consistant basis. Only then, could you venture out for other partners, and NOT neglect each other. This is sooo not happening in your situation so I would not classify it as a swinging lifestyle., I mean you might feel it's you and he having this great swingers lifstyle but he appears to be taking it as an open marriage to do as he like. You both are not on the same page, so to speak. You both really need to get back to the basics of your relationship, meaning no sex with anyone but each other. You need to work on open and honest communication of what it working right and what is working wrong. You need to have a EXTENDED period of time where you and he are enjoying a healthy mutually satisfying sex life before you can contemplate introducing other partners into the mix.