The summoning required something like 100 sacrifices in both worlds to open the gate to the other world.
This means that the 100 that were closest to him at the moment of his summoning were killed and sacrificed.
Also, in order to ensure the safe passage of his soul, 5-10 people of his familly were also sacrificed.
This means that, even if he found a way to return to Earth, he would have nothing to return to.