Imo the Flayer is scarier cause Vecna is just a born psycho who loves torturing while the MF is a mysterious giant spider-shaped smoke monster/entity that tower over Hawkins.
My guy, if a dude without skin pulled up to my street, I would be somewhat anxious, but if a gigantic shadow spider that killed me in D&D once pulled up, I will be ded.