Is Ryuu actually surprised that Madoka and Yuzu thought that he was treating Koharu like the wife he had wanted all along?
He seems to have no communication skills outside of talking a woman into sex.
If it weren't for Koharu, not only would Madoka have left him, but Yuzu would have left him as well.