In a Louisiana nursing home in 1999, Paul Edgecomb begins to cry while watching the 1935 film Top Hat. His elderly friend Elaine shows concern for him, and Paul tells her that the film reminded him of when he was a prison officer in charge of death row inmates at Cold Mountain Penitentiary during the summer of 1935. The scene shifts to 1935, where Paul works with fellow guards Brutus "Brutal" Howell, Harry Terwilliger, and Dean Stanton.
One day, John Coffey, a giant black man convicted of raping and killing two young white girls, arrives on death row. However, he is shy, soft-spoken, and emotional. John reveals extraordinary powers by healing Paul's urinary tract infection and resurrecting a mouse. Later, he heals the terminally ill wife of Warden Hal Moores. When John is asked to explain his power, he merely says that he "took it back."
Percy Wetmore, a sadist with a fierce temper, has recently begun working in the death row inmates block; his fellow guards dislike him, but cannot get rid of him because of his family connections to the governor. He demands to manage the execution of Eduard Delacroix, promising that afterward, he will transfer to an administrative post at a mental hospital. An agreement is made, but Percy then deliberately sabotages the execution: Instead of wetting the sponge used to conduct electricity and make executions quick and effective, he leaves it dry, causing the execution to malfunction dramatically.
Meanwhile, a violent, psychopathic prisoner named "Wild Bill" Wharton has arrived, to be executed for multiple murders committed during a robbery. At one point he seizes John's arm, and John psychically senses that Wharton is also responsible for the crime for which John was convicted and sentenced to death. John "takes back" the sickness in Hal's wife and regurgitates it into Percy, who then shoots Wharton to death and falls into a state of permanent catatonia. Percy is then admitted to Briar Ridge Mental Hospital as a patient rather than an administrator. In the wake of these events, Paul interrogates John, who says he "punished them bad men" and offers to show Paul what he saw. John takes Paul's hand and says he has to give Paul "a part of himself" in order for Paul to see what really happened to the girls.
Paul asks John what he should do, if he should open the door and let John walk away. John tells him that there is too much pain in the world, to which he is sensitive, and says he is "rightly tired of the pain" and is ready to rest. For his last request on the night before his execution, John watches the film Top Hat. When John is put in the electric chair, he asks Paul not to put the traditional black hood over his head because he is afraid of the dark. Paul agrees, shakes his hand, and John is executed.
As an elderly Paul finishes his story, he notes that he requested a transfer to a youth detention center, where he spent the remainder of his career. Elaine questions his statement that he had a fully grown son at the time, and Paul explains that he was 44 years old at the time of John's execution and that he is now 108. This is apparently a side effect of John giving a "part of himself" to Paul. Mr. Jingles, Del's mouse resurrected by John, is also still alive — but Paul believes his outliving all of his relatives and friends to be a punishment from God for having John executed, and wonders how long it will be before his own death