It only took him 33 chapters to meet her on her own terms instead of forcing her.
"It saddens me that the memories I cherish from the past don't mean the same to you"
Yeah because you weren't blackmailed to create those memories and exposed using the material of that blackmail.
It's so ridiculous that he can even think that. Shouldn't he be saying how he regrets his actions caused her to have a tough time? How can he look back happily on something that he knows was dark for her?
This means the writer does not intend to ever treat his blackmail and failure to return/protect/destroy the confession letter as blackmail and extreme carelessness towards something he knew gave her constant anxiety. At 15-17, I can forgive him somewhat but he's grown now.
He still doesn't really care about or want to understand her feelings.