What got me was when he touched her and she stiffened. It set off alarm bells in my head. He strikes me as an attentive person, and I bet he knows he's making her uncomfortable, and he just doesn't care.
Imagine if a friend of yours was telling you about a first date with a guy, and this was the date. Would you tell them to go on a second one? I sure wouldn't. Has he done anything overtly threatening? Not necessarily, but that depends on your dealbreakers. If he did something shady, would I be surprised? No, and that's a bad sign. But...I've heard of worse first dates. A friend of mine's date literally punched her in the face on their first date, and she went on a second one with him because he said it was an accident. So...everybody's different?
tldr; I agree, he's creepy. I'm still reading because I like the family and I want to see Mildred repair her relationship with her stepdaughter.