The brimhats ARE right, even if their methods are somewhat bad, you had an era of incredible world-changing magic and advancements, then comes 3 terrorists that decide for themselves that the world needs no freely available magic and wipe the world's memory, ¡hell they even say it was "a conspiracy"! now you have an institution made by those terrorists that defend their retrograde ideas and banned all the good magic: you have no more medical magic(so they condemned untold people to die of diseases that could be cured by magic alone plus all the other one that suffer with deformity and whatnot), no more world-changing advanced magic, everything that looks like it could make a truly good impact is "forbidden" and they enforce this unlawfully by strong-arming everyone into compliance.
That's pretty much the definition of an authoritarian regime.
the terrorists by themselves and their arbitrary rules erased untold centuries of magic knowledge and sent magitech back to the stone age and only allowed for trivialities, essentially turning magicians into parlor-trick clowns.
It's like having a super advanced civilization, then comes 3 clowns that decide that "advance bad hurr durrr we'll decide for humanity!" and single-handedly destroy all the advancement and send everything to the medieval age