The spirits are being killed off unknowingly by the humans. When they turn into a black rock, they accumulate mana. But when the rock is shining and safe to touch, most humans don't realize it's a recovering spirit and turn into necklace, earrings, or some other form of jewelry, killing them.