I could've sworn the same thing, actually, but I just tested and yeah, it requires a page refresh. Could this be that 'Umbrella Effect' or whatever people always talk about?!😱😱😱?!
Edit: @Teasday is this intended behavior, and has it always been like this?