i don't know about the noble society in this setting but...
getting invited by the host himself and turning down an offer to dance from the party host is like humiliating the host imo
also it could affect their family face since the other guests are present..
though it's only a speculation lmao