This is not a harem manga at least not by the definition, romantic slice of life comedy sure but the protagonist has no feelings for anyone else but his wife and never actually does anything else with any of them either just random ecchi moments, I also don’t think it should be tagged as smut...
This one wasn’t for me, while a fan of the harem genre the fact he seemed to spend that time with prostitutes was a breaking factor, I much prefer any romance to be about love not paying someone for it for that reason I only read the first chapter and stopped