Bassist Alan Longmuir, his younger brother Derek Longmuir, a drummer, along with school mate, lead singer Gordon "Nobby" Clark founded the group in Edinburgh, Scotland in 1967, as The Saxons. Shortly afterwards, seeking a less English-sounding moniker, they chose a new name allegedly by throwing a dart at a map of the United States.

The dart landed on the map in the state of Arkansas, but since "Arkansas Rollers" did not sound quite right, and might also lead to problems with pronunciation, they tried again and this time the dart landed near the community of Bay City, Michigan.

The Bay City Rollers were managed from early on by the imposing and controversial Tam Paton, himself a former big band leader. Paton was notorious for his rigid control over all aspects of the band's career, including the shuffling in and out of group members at a very high rate. Short term members from this period include David Paton (member 1969-1970) and Billy Lyall (member 1969-1971), who went on to be founding members of another successful Edinburgh band, Pilot.

The Bay City Rollers received their first break when prominent record executive Dick Leahy caught their act by chance in an Edinburgh club. After signing with Leahy's Bell Records, the band's first hit was "Keep on Dancing" (UK #9, 1971), a cover of a 1965 Gentrys hit, recorded at the suggestion of pop impresario and producer Jonathan King. (Singer Nobby Clark was backed on vocals on "Keep on Dancing" by King himself.)

Upon this release's success, they made guest appearances on BBC-TV's Top of the Pops. The group then won a Radio Luxembourg-sponsored song contest with the tune "Mañana", which was later popular in parts of Europe and in Israel.

Several non-charting singles were released over the following two years. This period did see the addition of long term members Eric Faulkner and Stuart "Woody" Wood.

In late 1973 they narrowly missed the UK chart with "Saturday Night", one of many singles written and produced for the band by the highly successful songwriting duo of Scotsman Bill Martin and Irishman Phil Coulter.

By the end of 1973, Nobby Clark had become disillusioned by the band's lack of success and decided to move on. He was replaced as lead singer by Leslie McKeown.

The five members at the very end of 1973 - the Longmuir brothers, Faulkner, Wood and McKeown - are generally referred to as the "classic line-up". In early 1974 McKeown hastily re-recorded lead vocals of the group's forthcoming single, "Remember (Sha La La La)", which became a sizable hit and a lead in to a series of UK chart hits.

Beginning with "Remember" (UK #6), the Rollers' popularity exploded, and they released a string of very successful hits on the British charts. Following in succession were "Shang-a-Lang" (UK #2), "Summerlove Sensation" (UK #3), and "All of Me Loves All of You" (UK #4).

By early 1975, they were one of the highest-selling acts in Britain. That year saw a successful UK tour (which prompted newspaper headlines about "Rollermania"), and a 20-week UK television series, Shang-a-Lang.

A cover of the Four Seasons' "Bye, Bye, Baby", stayed at #1 in the UK for six weeks in the spring of 1975, selling nearly a million copies to become the biggest seller of the year, and the subsequent single "Give a Little Love" topped the charts that summer, their second #1 hit. At the peak of their popularity in the UK, comparisons were even made to The Beatles.