With how clueless and innocently unassuming Papa Noh is, I assume his first wife was also probably a lady of wealth and/or status that he never realized liked him so much that she gave it up to be his wife (unless I missed/forgot a detail that said otherwise in the earlier story).