He was quickly rising through his church's ranks due to his kindness and diligence, but the older priests were jealous of him. Also the girl who accused was a pretty but manipulative church orphan angry that Sem refused to fall for her seductions. So she collaborated with the older priests to get revenge on Sem. She falsely accuses Sem of molesting her, while the other priests back her accusations. The rumour mill does the rest. The church higher up is certain of Sem's innocence, but considers the church's public reputation and inner stability more important than getting justice. So they decide to cut Sem off as a weak link. He's defrocked, jailed, and then kicked out of town. Sem later finds an inn run by a retired lady adventurer, one thing leads to another and he ends up sleeping with her. She suggests he try become an adventurer in the city, so he goes there. He does some herbalist/healing services in the city to support himself, but also gets into visiting the local brothels a lot.