The movie opens with Agent K (Tommy Lee Jones) and his partner, D (Richard Hamilton) intercepting a truck containing illegal immigrants, and they allow everyone to cross into America except for an extraterrestrial, who is disguised as a human because he is an intergalactic fugitive. When a suspicious, United States Border Patrol agent, who saw K and D pose as fellow Border Patrol agents, sees the alien without its disguise, the alien attacks him, forcing K to destroy it and use the neuralyzer on the agent and his colleagues, claiming that the alien remains splattered all over them are from when the agent shot an underground gas main with his gun. D, feeling too old, asks K to use the neuralyzer on him so he can retire.
Meanwhile, James Darrell Edwards III (Will Smith) is a New York police officer pursuing a man on foot while his out-of-shape colleagues can't catch up. While chasing the incredibly fast and agile fugitive, he corners him and the man throws an odd looking gun away, which disintegrates, and escapes. When Edwards corners the man again at the top of the Guggenheim Museum, the man keeps saying "he's coming," and then tells him that the world's going to end, and then he sees the man blink with a set of secondary eyelids before jumping over the roof and killing himself.
Meanwhile, an alien craft crashes in front of a farmhouse, where the overbearing farmer and husband Edgar (Vincent D'Onofrio) and his wife reside. After hearing the crash outside his home, Edgar goes out with his shotgun, and when he arrives at the pit, a voice tells him to put his gun down, but he refuses, and a claw like hand grabs him and pulls him into the pit, and the mysterious alien kills him, rips off the skin, eats his body, and wears the skin as a disguise (and presumably even his eyeballs and mouth), and walks back in the house.
His depressed deadpan wife Beatrice asks him what happened, and instead, he strangely asks for sugar in water. She obeys, and when she notices that his skin is "hanging off his bones," and when he pulls his face back on his skull, Beatrice, to her horror, faints, and after doing so, the Bug goes back outside and rolls his ship out of the crater. Meanwhile, after investigating the crime scene that James had dealt with earlier, K arrives at the precinct station, (where no one believes Edwards's incredible story), questions Edwards about the incident, then takes him to a pawnshop run by a man named Jack Jeebs (Tony Shalhoub), an MIB informant alien who Edwards knew for dealing stolen goods, to identify the strange weapon the criminal drew on him.
While questioning Jeebs, K reveals Jeebs is an alien by blowing his head off with a weapon, which grows back immediately. After Edwards finds a weapon identical to the one he saw among Jeebs' hidden stash of alien technology, K realizes that the gun was meant to be used in an assassination, but Jeebs is unable to provide any further information. K uses the neuralyzer on Edwards, takes him to dinner, and leaves him an MIB business card. The next morning, an exterminator comes to kill a bug infestation in Edgar's barn where the bug confronts him in the barn, takes the exterminator's gas gun, sticking it in his mouth, where the poisonous gas kills him, and shoves his ship into the exterminator's truck, and take's the truck in its aid to hunt.
During that time, Edwards goes to the MIB's secret headquarters and competes with several others to qualify to join the agency. During one part of the test, a written exam, the candidates struggle to complete the test due to the oddly shaped chairs, until Edwards pulls a table over for him to use. At another part involving a shooting gallery, Edwards refuses to shoot several aliens because he thinks that they are doing normal everyday things, but shoots a little girl for carrying suspicious quantum physics textbooks, in the wrong place at the wrong time, especially for her age. After the tests, K takes him aside (while Zed uses the neuralyzer on the others for failing the tests) and offers him the position.
K explains that the catch of joining MIB means he will have to distance himself from any human contact to maintain secrecy, and that he will give him time to think it over. The next day, James accepts, and K shows him around, explaining everything about alien existence, and that the entire universe, and most of the aliens secretly residing on earth is being monitored, and J finally gets proof that he was right that his third grade teacher was from another planet. Meanwhile, Zed takes him aside, and has his identity erased, even burning off the fingerprints on his fingers, and can only eat, wear, and reside what M.I.B. allows him, and becomes Agent J.
Meanwhile, the bug tracks down what seems like an old jewelry shop owner, and the man goes to a restaurant to meet up with a friend, and the bug follows. The man is then revealed to be an alien in disguise, when they both speak in an alien language. They reveal to each other that a "bug" has landed on the planet, and that they must get them off of Earth, but, before they can even eat, the Bug finds them, and kills them, using his tail to stab them in the necks, grabs an object of what looks like would contain something that whatever the bug was looking for was in the object, and leaves the restaurant, knocking over tables as he leaves.
Meanwhile, at M.I.B, the two agents catch an alien named Reggie leaving past his limit, and K and J go to ask the alien about it. K asks Reggie why he's determined to leave, and Reggie seems disturbed and doesn't tell him much as to why he and his pregnant wife were leaving (while J was humorously trying to deliver Reggie's potential offspring.) Suspicious of why Reggie is so scared of something that he's leaving earth with a newborn, the two agents investigate the New York Tabloids, and notice an article that says "Alien Stole My Husband's Skin," and they track down the address to Edgar's farm, and question his shocked wife. After she tells the story, K neuralyzes Beatrice, and tells her that there was no UFO, and that "Swamp gas got trapped in a thermal pocket and refracted light from Venus.", and that her husband ran away.
J realizes that the neuralizers allow them to make up any story they want to cover things up and adds that Edgar left her for an old girlfriend, but then changes it up saying that she kicked him out, and suggests she do more shopping for herself. Meanwhile, when they scan the dirt in the crater that the ship crashed, and when the "green" light comes up, K, to his dismay, realizes that a bug has landed here, a member of a large, cockroach-like alien race, who have been at war with the Arqullians, a peaceful race of aliens. They decide to check the local morgues to watch for victims that the bug might have killed. They go to one where the body of the disguised aliens are taken and investigated by Dr. Laurel Weaver (Linda Fiorentino). During the investigation K and J arrive at the morgue posing as doctors. The face of the dead Arquillian "opens" while being examined by Dr Weaver and J revealing the actual Arquillian prince inside a robot body. Before dying the prince tells J that "to prevent war, the galaxy is on Orion's belt".
M.I.B. continue to research to understand what that meant, and they think J heard wrong, considering that the belt of the constellation Orion is just three stars, and that billions of stars make up a galaxy. Meanwhile, Zed notices that all the residing extraterrestrials are leaving the planet, due to the fear of the bug being around, and the two agents investigate the Arquillian's Jewelry store and find that the bug has already broken into it, though left everything behind, implying that he is looking for something. After J notices the bug passing by in front of the store, J attempts to shoot him, but misses, and the bug leaves. K angrily confronts him for discharging M.I.B's secret weapons in front of the public, and the two get in an argument about whether it's more important to keep themselves a secret or trying to proceed with the mission. When K calls Zed and tells him about the situation and to send M.I.B's Containment Crew to neuralyze the witnesses, Zed informs K that there's an Arquillian battle cruiser floating over earth, and they are holding M.I.B responsible for returning them the "galaxy" that the alien had brought up.
K decides to take J to the informant Frank the Pug (Tim Blaney), an alien disguised as a small lapdog, explains that the missing galaxy is a massive source of energy housed in a small jewel. After K concludes that if the Galaxy is on earth, then it can't be on Orion's belt. However, J figures out the galaxy is hanging on the collar of Rosenberg's cat Orion, who refuses to leave the prince's body at the morgue. Orion has been taken care of by Dr. Laurel Weaver. J arrives at the morgue just as the Bug grabs the galaxy from the cat's collar, and as the two agents arrive, the bug is holding Weaver hostage, and kidnaps her, and forces her to take her to New York State Pavilion at Flushing Meadows. Unable to track down the bug, the agents go back to headquarters, only to discover that the Arquillians have delivered an ultimatum to M.I.B. to secure the galaxy within an hour, or they will destroy Earth. However, J notices a portrait of the flying saucers disguised as attractions for the Pavilion that K had brought up earlier, when K was introducing him to their alien secrets earlier in the movie, and they realize where the bug is headed next.
The Bug arrives at the site of two disguised flying saucers, and decides to take Laurel with him, saying that he'll need a snack for the long trip. K and J are now on the way to Queens, New York, and they decide to take the tunnel, while K tells J to press "the little red button" which causes it to go into hyperdrive. K then puts on The Promised Land, and when J reminds K that Elvis is dead, K replies that Elvis went home, revealing his alien status. As the bug climbs up the tower to enter the saucer, Laurel tries to talk him out of it, and when she finally strikes him, the bug accidentally drops her in the trees, and decides not to waste any trouble trying to get her. He climbs into the saucer and flies off, but K and J arrive as he does so. K then pulls out space weapons to shoot the saucer down, causing it to crash through the Unisphere.
The bug comes out and angrily rants how it's over now, and that he's won, but the agents prepare to arrest him. J tells him to put his put his hands on his head. The bug sheds Edgar's skin, revealing it's true, cockroach-like form, and the agents prepare to shoot it, but the bug grabs the guns and swallows them, knocking them aside. K tells J to stop the Bug from getting into the other ship, and tells J that he's "gonna get his gun back." K then taunts the Bug to swallow him whole, and the bug does so. J does what he can to keep the bug from leaving, including throwing a rock at its head, waving branches on fire in front of him, and even tries tugging on its tail as it climbs up the tower, but the bug just keeps knocking J aside. J is knocked into a garbage can, and when he notices that it's full of cockroaches, he crushes them, taunting the bug in the process, infuriating him and managing to attract its attention. Just as the Bug prepares to eat J, he is blown open from the inside by K, who located his weapon and Orion's belt in the Bug's stomach.
As J and K sit on the ground covered in slime, K calls Zed, and tells him to tell the Arquillians that the bug is dead, and they now have the galaxy. However, the mortally wounded insect tries to attack again, but is finally blown to bits by Weaver using J's weapon. The three return to M.I.B. headquarters and K tells J that he has not been training him as a partner, but rather as a replacement, as he is tired of his life as an agent. J reluctantly uses the neuralyzer on K, using a story that he has been suffering in a coma for 35 years to allow him to return to his civilian life and the young woman he left behind.
A few days later, it is revealed that Weaver also joined M.I.B. and is now J's new partner, Agent L, and after reading some New York Tabloids, L tells J about an alien who wants to go to a sports game, and they drive off to deal with it. As they drive off in their car, the camera rapidly pulls back, showing that Earth and the Milky Way galaxy are also inside an alien marble being used in a marble game, revealing that, like the galaxy in a necklace, we are just a tiny part of something much greater.