Ambiguity in storytelling is rewarding but tricky. It’s rewarding because, when handled well, we can draw the audience into the narrative. If engaged, they wonder what’s going on and why. Like the characters, the audience actively thinks about the story, examining “clues” (literal or figurative) and making guesses.
Mystery as either a storytelling device or a genre is driven completely by ambiguity. Without ambiguity, there is no mystery; we know “whodunnit.”
However, ambiguity can also be tricky, because it involves some work to correctly pull off. I have read a lot of spec scripts that engage in only a single level of thinking, that being: (ambiguity/mystery) involves (confusion/the audience doesn’t know what’s going on), and thus (confusion) = (mystery).
But there is a big difference between an audience that thinks “I wonder what’s going on” versus an audience thinking, “I have no idea what’s going on.”
There are two ways to achieve the former: the engaged audience. The first is to show them that the ambiguous elements of the narrative are only “confusing” because the characters don’t yet have enough information. That is: even if we don’t know what something means in the moment, it does mean something. One good way to do this is to present a clue, then pay off that clue. Thus, the next time an ambiguous clue pops up, the audience has thus been taught that these choices are part of constructed narrative that has yet to be revealed, as opposed to a bunch of nonsense that’s dumped on the page just to cause empty-calories “confusion.”
The second way is to work on the characters in such a way that the audience cares about them. If we’re invested in the characters, we’re invested in their story. And if we’re invested in the characters’ story, and that story includes ambiguous elements, then we’re thus invested in figuring out those ambiguous elements.
For this reason, we can slip ambiguity into a story that’s otherwise fairly grounded… so long as we introduce it at the end, by which time we should be invested in the resolution of the characters’ story.
Here are two similar-yet-different examples. The first is TOTAL RECALL. At the end of this film, it’s left to the audience to decide if the protagonist has actually gone on this adventure, or if we have just been watching the mental vacation he purchased in the lab.
The other is THE FLORIDA PROJECT. This is a very grounded drama. But at the very end, the protagonist suddenly goes on a whimsical trip to Disney World that is wholly out of place with the rest of the film. We can guess she probably isn’t actually going on this trip, that she’s engaging in a bit of internal fantasy to escape the stress of her circumstance. But the movie doesn’t explain one way or the other. Thus, it's a tantalizing bit of ambiguity, sending the audience out of the theater with a question about a protagonist in whom they’re fully invested.