Attributed to the Russian author and playwright Anton Chekhov, from somewhere around the last decade or so of the ninetieth century, “Chekhov’s Gun” has been commonly translated as:
If in the first act you have hung a pistol on the wall, then in the following one it should be fired. Otherwise don’t put it there.
This is Chekhov reminding us that our readers, or the audiences of our play (or movie, or video game, or . . .) are making a mental inventory—whether or not they’re conscious of it—of everything we’ve told them, including that there’s a gun here, a sword there, a tuft of werewolf fur in the corner . . .
And any of those things that don’t pay off will feel like missing pieces.
In many cases your readers won’t even be specifically aware of what’s missing. They’ll have filed the gun (or whatever it is) away in their heads for later then might not ever be able to say, consciously, “Whatever happened to the gun? Why was that there? Did the author just forget about it? Did a scene get cut out? Did I miss something?”
And that last question is the worst. Now, there are times—many times, actually—that you want to leave your readers with questions: Is Deckard really a replicant? Was Dave Bowman time travelling or space travelling (or both) at the end of 2001: A Space Odyssey? But there’s as big a difference between establishing unnecessary details out of carelessness and intentionally leaving your readers to think for themselves and interpret the meaning of your story as there is between consciously making a stylistic decision to break a “rule” of grammar and not knowing the rule in the first place.
If anything in your story is there “just because” or “to provide color” I’d advise either cutting it, or looking at it as an opportunity to make some mundane detail into a pivotal, and surprising, plot point.
And as always, keep in mind that whatever you’re presenting as part of your characters’ world should have some emotional/psychological resonance. If the rifle hanging on the wall gives one character an uneasy feeling, that’s all the rifle needs to do. It tells us something about the character who’s uneasy and also tells us something about the owner of the rifle.
And not everyone who owns a rifle actually uses it to shoot someone! So no, the gun doesn’t literally have to “go off,” but it does need to advance your story in some way, otherwise it’s another example of why we never saw Captain Pickard go to the bathroom . . .