I'm glad that MC's internal struggles and regret isn't completely glossed over and that it seems to be an integral part of his conflict moving forward.
I hope Hiyori reappears to him and is a regular force in the story. Like she tells him to forgive himself and the Oni and maybe eventually even lets or tells him pursue her in a relationship with her.
I do hope that we see them reunite in the afterlife when the series is over, though. As Oni don't seem able to die, so she'll be his partner in this world and Hiyori will be his partner in the next.