He just doesn't want to do them.
When the water dragon contact him about the red dragon to complain about red going to a dragon brothel he said to red as why he wasn't told about it.
He went to a human brothel after he was aroused by his companions.He said it directly to them after he admitted to the women he got aroused by them.