I think people aren't remembering what Sweet Mask is all about. He has no pity for any hero/fighter who isn't strong enough to do what they're supposed to. To him, those brainwashed super soldiers deserved the deaths they got for losing and becoming slaves. Sweet Mask's sense of "justice" is in a sense, hypocritical and hypersensed. He has a hyper sense of justice, that anything that hinders this justice should just be eradicated. And on top of that, he hates whatever isn't beautiful. Hence why the tongue/joker scars ticked him off. But it's hypocritical because he himself turns hideous as well when he's in fight mode.
Not just that, but to Sweet Mask, power is beauty. Those who do not have power are not beautiful. Those who lose, like those brainwashed super soldiers, were the opposite of beautiful. He'd probably have a uber hard on for our Saitama if he knew the power that Saitama possessed.