Harrison had a Billboard No.1 record in 1959 with the song "Kansas City". The song was written in 1951 and was one of the first credited collaborations by the team of Jerry Leiber and Mike Stoller. Harrison recorded "Kansas City" for Harlem music entrepreneur Bobby Robinson, which caused a furor with Herman Lubinsky and Savoy Records.

Harrison recorded for the Fire and Fury record labels, which were owned and operated by Bobby Robinson at his Harlem record shop, Bobby's Happy House of Hits on 125th Street, west of the Apollo Theatre.

Harrison's records are especially notable for the presence of the brilliant guitarist Wild Jimmy Spruill, whose solo on "Kansas City" is one of the most memorable in the history of rock and roll.

After this success, Harrison continued to perform and record but it would be another ten years before he recorded "Let's Work Together" that made it on the Billboard Hot 100 and was later a hit for Canned Heat.

This was a slightly modified re-cut of his 1962 single "Let's Stick Together". This, in turn, was later a big hit for Bryan Ferry in 1976. In 1970, he had some success with "My Heart Is Yours". He toured for many years with a band known as "Wilbert Harrison and The Roamers" as well as a solo act/one-man band.

Harrison died in 1994 in a Spencer, North Carolina nursing home at the age of 65. In 2001, his recording of "Kansas City" was given a Grammy Hall of Fame Award. His recording has also been named as one of The Rock and Roll Hall of Fame's 500 Songs that Shaped Rock and Roll.