Cocker was born John Robert Cocker at 20 Tasker Road, Crookes, Sheffield, the youngest son of a civil servant. He left school early and became an apprentice gas fitter. In 1961, he started his musical career in the pubs of Sheffield.

Under the stage name Vance Arnold, Cocker began his career with Vance Arnold and the Avengers. In 1963 the band supported The Rolling Stones at Sheffield City Hall. In 1964 Cocker released his first single, a cover of The Beatles' "I'll Cry Instead" with a new band, Joe Cocker Big Blues.

This band ventured as far as France, where they played on American air bases. After a lull, Cocker teamed up with Chris Stainton, to form The Grease Band, in 1966. They were noticed by Denny Cordell, the producer of Procol Harum, The Moody Blues and Georgie Fame. Cordell set Cocker up with a residency at The Marquee Club in London.

After minor success in the U.S. with the single "Marjorine", he entered the big time with a groundbreaking rearrangement of "With a Little Help from My Friends," another Beatles cover, this time from the Sgt. Pepper's Lonely Hearts Club Band album, featuring lead guitar from Jimmy Page and backing vocals from Sue and Sunny. The eventual original touring line up of Cocker's Grease band featured Henry McCullough on lead guitar who would go on to briefly play with McCartney's Wings.

Cocker then had a second UK hit with the Leon Russell song, "Delta Lady". He had further success covering Beatles tunes in 1970 with his version of "She Came in Through the Bathroom Window" off their Abbey Road album. Though his British success proved difficult to sustain, he enjoyed several chart entries in the U.S. with "Cry Me a River" and "Feelin' Alright" by Dave Mason.

In 1970, his cover of the Box Tops' hit "The Letter", which appeared on the live album and film, Mad Dogs & Englishmen, became his first U.S. Top Ten hit. Cocker was backed in the film, soundtrack album and subsequent tour by a then unheard of band of more than 30 players (including three drummers, backing vocalist Rita Coolidge and pianist/bandleader Leon Russell).

Throughout 1969 he was featured on variety TV shows like The Ed Sullivan Show and "This Is Tom Jones." Onstage, he exhibited an idiosyncratic physical intensity, flailing his arms and playing air guitar, occasionally giving superfluous cues to his band.

In 1976, Cocker performed "Feelin' Alright" on Saturday Night Live. John Belushi joined him on stage doing his famous impersonation of Cocker's stage movements. In late 1975, he contributed vocals on a number of the tracks on Bo Diddley's The 20th Anniversary of Rock 'n' Roll all-star album.

In the beginning of the 1970s the "Sheffield Soul Shouter" dealt with drug abuse including alcohol. He managed to make a comeback in the 1980s and 1990s with several hits.

From 1988 to 1993, "With A Little Help From My Friends" was used as the theme music to ABC TV's The Wonder Years.

Cocker performed the opening set at Woodstock '94 as one of the few alumni who played at the original Woodstock Festival in 1969, and was very well received. He continues to tour sporadically, and currently lives on the Mad Dog Ranch in Crawford, Colorado, with his wife, Pam.

Cocker was awarded an OBE in the Queen's 2007 Birthday Honours list for services to music. To celebrate receiving his award in mid December 2007, Cocker played two concerts in London and in his home town of Sheffield.

In 2007, Cocker appeared playing minor characters in the film, Across the Universe, as the lead singer on another Beatles hit, "Come Together".