why would he freak out about the perception thing? its hella good for him and everyone! like that thing with the egg, thks to his perception its completely safe to eat
It's probably because he will interact with others in that world. He might buy some ingredients or other things from that fantasy world and it will turn out different in his skill world.
For example, imagine there is this life resurrection fruit in that fantasy world that looks like an apple... but when he tries to plant it in his world, it would only turn out to be a normal fruit. lol
... Wasn't there a golem that developed from a sentient rock?
Kawaranu Mono <Kirameki no Gōremu>
https://www.novelupdates.com/series/kawaranu-mono-<kirameki-no-goremu>/
I feel like telling him that eggs aren't dangerous because he perceives them not to be dangerous, then proceeding to tell him that the egg you are holding could have been contaminated, if he didn't perceive it otherwise, then eating said egg, is a good way to end up poisoned. It certainly would have been possible for him, after being presented that information, to think, "So that egg could have been contaminated? What if it really is bad? What if she gets sick as a result of eating it?"
I'm with @Hollow7F though. His perception makes it easier for him to get the things he needs in good quality. He seems concerned about throwing off the balance of the world, but since he's not happy with them shipping some of the children off to that island, you would think that his goal is to throw off the balance of the world, to be able to change it. Start using that perception to your advantage. If you're worried about what happens if you are gone, use that perception to get useful items that will help them before that happens.