His wives are female ninjas according to him. They probably aren't doing anything they don't want to even if they take customers. Drug the clients and let them think they were too drunk to remember the deed. Or the values are just that different because they're all ninjas and the females were supposedly used this way anyway, as honey pots for information gathering and assassinations. They just talked about how rich people are buying out prostitutes to marry them too, the thinking is a bit different depending on the people, place, and time period and circumstances.