They were there to hunt Kumoko because she's been terrorizing the other monsters. The scared monsters then ran away and converged near the entrances to the labyrinth, which became a big problem for the humans.
They were burning her nest because they want to capture her(or kill if capture is impossible), that's why there is a monster tamer included in the group.
All based on the WN(don't know about the LN though).
Edit: Seems like they used summoner here instead of monster tamer(which is more correct given his role in this party, to tame Kumoko).