"The duke is so important that the prince sent an SS rank adventurer and his own brother to help out"
"clearly the duke's in a bad position here"
This is bad writing. The brother can mouth off all he wants, but the Duke would realise that he can't be alienated by the brothers, because he'd just fall into an enemy camp instead to get support from adventurers. The protagonist acting high and mighty is some rubbish stuff.