Hardy was born in Cheltenham, England, the son of Jocelyn (née Dugdale) and Henry Harrison Hardy. His father was headmaster of Cheltenham College. He was educated at Rugby School and Magdalen College, Oxford University where he gained a BA Honours degree in English.

Hardy began his career as a classical actor, appearing as Shakespeare's Henry V on stage and in television's An Age of Kings (1960), and subsequently played Coriolanus for the BBC.

It was while playing Henry V that Hardy developed an interest in medieval warfare, and he later wrote and presented an acclaimed television documentary on the subject of the Battle of Agincourt. He has also written two books on the subject of the Longbow, and The Great War Bow with Matthew Strickland. He was one of the experts consulted by the archaeologist responsible for raising the Mary Rose.

Over the years, Hardy has played a huge range of parts on television and film. His first continuing role in a TV series was as businessman Alec Stewart in the award-winning oil company drama The Troubleshooters for the BBC, which he played from 1966 to 1970.

In 1975, Hardy appeared in the episode "Such a Lovely Man" in the fifth season of the series Upstairs, Downstairs. But his most famous role was as Siegfried Farnon in the long-running All Creatures Great and Small (1978–1990), a stylish and much loved adaptation of James Herriot's novels. Some feel Hardy brought just the right touch of eccentricity and belligerence to the part of Siegfried.

He also made a memorable appearance in the 1987 ITV comedy series Hot Metal, in which he played a dual role, that of a newspaper proprietor, Twiggy Rathbone (who bore more than a passing resemblance to Rupert Murdoch) and his put-upon and harassed editor, Russell Spam.

In 1993 Hardy appeared in an episode of Inspector Morse, playing Andrew Baydon in "Twilight of the Gods". Hardy played the part of the successful businessman with a murky wartime past with a characteristic blend of the vulnerable and the bombast.

He has played former British Prime Minister Winston Churchill in several productions, most notably in Winston Churchill: The Wilderness Years (1981), for which he won a BAFTA award, but also in The Sittaford Mystery, Bomber Harris and War and Remembrance.

He also played Robert Dudley, Earl of Leicester in Elizabeth R, and Prince Albert in Edward the Seventh (known as Edward the King to the American audience). He took a leading role in the 1995 film version of Sense and Sensibility. His most recent roles were as Minister of Magic Cornelius Fudge in the Harry Potter movies, and as Lord Malan in His Master's Voice.

His voice performance as Robin Hood in Tale Spinners For Children Robin Hood, a LP from the 1960s, is considered one of the best Robin Hood renditions.

He was appointed a Commander of the Order of the British Empire (CBE) in the Queen's Birthday Honours of 1981.