I bet it's his older brother. He got caught up in some bad shit and died but swore to keep little brother off the same path and so became his guardian spirit.
TFW Your standfreaky ghost stalker guardian spirit attracts more women than you.
Honestly I kind of want to see a manga with this concept explored. The glasses guy falls in love with the girl but she's in love with the spirit that follows him. Especially with @starburst98 backstory of the spirit being the guy's dead brother who wants to protect him. It would be ironic if the spirit trying to protect the guy would end up cucking him.