John Levy was born in Minneapolis. When he was a young boy, his family moved to Phoenix. His first exposure to poetry was in the sixth grade, when his older brother began playing recordings of Dylan Thomas reading his poetry. Later, Levy began to read e. e. cummings and at age 15, after finding a book of William Carlos Williams' poems, began writing poems.

Levy graduated from Oberlin College in 1974. He worked in a factory that summer and earned the money to fly to Kyoto, where he lived for a year and a half. For six months he worked as a waiter and dishwasher with the American poet Cid Corman in a coffee and ice cream shop that Corman had started with his wife Shizumi. He briefly returned to Arizona in early 1976, where he was a poet-in-residence at a private school for a month, having been awarded a grant by an arts commission. Levy then moved to Paris where he lived for just over a year, earning his living by babysitting a young Canadian boy and by working as a personal secretary for a retired diplomat.

Levy published his first collection of poetry, Suppose a Man , at the invitation of James L. Weil, publisher of The Elizabeth Press. Weil also published Levy's second collection, Among the Consonants (1980), and Weil became a generous and supportive friend until his death in 2006.

In 1980 Levy moved to Tucson, AZ, and continues to live there. After moving to Tucson, he worked as a carpenter with a high school friend who had started his own construction company. From 1983 to 1985, Levy moved to Meligalas, Greece where he taught English as a second language at private language schools in Kalamata. After returning to the United States, Levy took up the study of law in 1988 at the University of Arizona College of Law. After graduation, he clerked at the Court of Appeals (1991–1992), then undertook a solo practice for three years (doing both criminal and civil work). He then joined a small firm that specialised in plaintiff's securities fraud class action cases. In 1997 Levy joined the Pima County Public Defender's Office. He is now retired and continues to live in Tucson.