Llewelyn was born in Newport, Monmouthshire, Wales, the son of Mia and Ivor Llewelyn, who was a coal mining engineer. He originally wanted to be a minister, but during his education at Radley public school, he worked as a stagehand in the school's productions and occasionally picked up small roles.

The outbreak of World War II in September 1939 halted his acting career; Llewelyn was commissioned as a second lieutenant in the British army, serving with the Royal Welch Fusiliers. In 1940, he was captured by the German army in France, and was held as a POW for five years.

Llewelyn was chosen for the role of Q because of his work with director Terence Young in the 1950 war film They Were Not Divided, in which he played a tank gunner. Beginning with From Russia with Love in 1963, Llewelyn appeared as Q, the quartermaster of the MI6 gadget lab (aka Q branch), in every Bond film until his death, except Live and Let Die (1973).

His last appearance as Q prior to his death was in The World Is Not Enough in 1999. During his briefing of 007 in the film, Q introduces John Cleese's character, R, as his heir presumptive, and the film alludes to Q's retirement, which Bond, after seeing R, expresses his hope that it won't be any time soon.

Q's response is to admonish Bond to "always have an escape plan", after which he lowers himself through the floor of his lab. Llewelyn had stated not long before his death that he had no plans to retire and that he would continue playing Q "as long as the producers want me and the Almighty doesn't."

In the 2002 film Die Another Day, John Cleese's character is the head of Q branch, having inherited the title of quartermaster from his predecessor. In all, Llewelyn appeared in 17 Bond films, more than any other actor, and worked with the first five James Bond actors.

He also portrayed Q in a 1967 EON-produced made-for-television special entitled "Welcome to Japan, Mr. Bond", which was included in the 2006 special edition DVD release of You Only Live Twice.