there is next to no reason for him to hide at all, a simple explanation would solve the problem and even if they doubting him, not like he clean the place alone, Shirogane can tell them what going on too.
the whole conflict in the first few chapters is rather weak, unlike other isekai with weak stat but OP grow rate, he already have maxed stat but he keep say it in a way people misunderstood him and treat him like trash. I also would rather him being someone from the military, that would fit him...