I have read the manga quite a while ago, so I don't know who did it anymore, but in the 1962 movie adaptation and presumably in the original edo novel it was Ritsu's lover who was jealous of Shigeji because it was the lover who, at first, took care of the children and was gonna get to live with Ritsu before Shigeji came along and took all of them for himself. But the lover was cut from the manga adaptation anyway, so all I can say is "stay tuned" because I really don't remember myself.