The industry has a term for broad, uncomplicated antagonists: “mustache-twirling.” Mustache-twirling comes up, for example, if the script isn’t working to provide the antagonist with humanity or depth, if their motivations are just to be eeeeeeevil. The more likely it seems an antagonist might go “NYAH-HA-HA! NOW I’VE GOT YOU!” the closer we are to having a mustache-twirling antagonist.
The best way to avoid mustache-twirling is to treat the antagonist with the same craft and attention with see with the protagonist. We should get a sense of personality, internal life, a background that has led the antagonist to these choices and actions. That is: a human being, a person. Even if the actions are bad, the antagonist has a thought process that justifies them.
There are a couple of reasons to avoid mustache-twirling. The first is creative; developing and grounding the antagonist helps to elevate the storytelling. The second is commercial; a well-developed antagonist has a better chance of luring name talent to the role.
Of course, there is a time and place for mustache-twirling. For example, in broader projects, like animated films intended for a very young audience, we can have one-dimensional antagonists; they’re just greedy, or mean, or hungry, etc.
Or certain genres/paradigms, like fantasy, favor mustache-twirling villains. For example Thulsa Doom in the original Conan the Barbarian serves an evil god and is driven by evil motivations. All we have to know is he’s evil, and it works.
In the first Star Wars: A New Hope, all we knew about Darth Vader was he was an evil cybernetic space-wizard who had fallen to the Dark Side, betrayed the Jedi, and “killed Luke’s father.” He might have had his reasons, but in that first movie his reasons were clear: Darth Vader is evil, nuff said.
Even in entertainment geared toward adults, we can find some mustache-twirling. Cunningham in Rob Roy is an unrepentant bastard, as is Longshanks in Braveheart, or Hans Gruber in Die Hard. They’re driven by motivations that are recognizably human (greed, power, etc.). Their single-minded, unalloyed focus on motivations we typically regard as moral failings might be seen as mustache-twirling only because they are clear and uncomplicated. If they are twirling a mustache, it’s to the delight of the audience, as we can’t wait for the protagonist to beat these guys.
The same could be said for Iago in Othello, or Hans Landa in Inglourious Basterds, though in their cases the characters’ cleverness and intelligence elevates and humanizes them. Either would laugh at someone who said, “NYAH-HA-HA!” Even a non-human antagonist like Pazuzu in The Exorcist (an actual demon) has personality, intelligence, and twisted motivation.
Overall point being: the more grounded the genre/paradigm, the more we want to develop and humanize our antagonist. Even in broader fare, we should look for ways to elevate the antagonist with intelligence and personality. Sometimes it is fun to twirl the ol’ mustache, but let’s do so only when it works for the story, and it’s a conscious choice.