This is basically a twist to the premise of Gintama scandal arc where Gintoki got smashed and everyone wanted to teach him a lesson by accusing him of having sex with them in a drunken stupor, Madao included. He then tries to 'take responsibility' by moving in with everyone separately in a hostel with none of the women and madao 'knowing' each other's affair.
By the time the prank blows up he realises that he didn't have sex with anyone besides Madao.
In my opinion gintama did it miles better. But this manga is a different altogether with a focus on romance.
I can only assume the guy is bi, as I see no reason why mc would pick the guy (supposedly somewhere elsewhere with his gf) instead of the girls naked all ontop of him if he were straight. Too out-of-the-way, and the magic didn't sound like it had any such effect.