Nope, he does not care.
He is perfectly capable of expressing himself, as shown with Bridey's younger siblings.
Try not to forget that dress the father gave her in the early chapters, it was clearly to mock her.
Bastard is probably only acting like this to try and make her a pawn easier since he...