An X Factor UK legend has passed away with the death of Simon Cowell’s bodyguard Tony Adkins, also known as Big Tony, who worked closely on the show for eight years with the famed judge.

X Factor viewers will remember Adkins being forced to remove contestants from the audition room, often when their performances didn’t go to plan on the hit reality show.

Adkins also played a giant in Harry Potter and the Deathly Hallows: Part Two in 2011.

Know the news with the 7NEWS app: Download today

Adkins’ family made the announcement on social media, saying he passed away on Easter Sunday during a rugby tour.

He was just 64, at the time of his death.

“It is with great sadness my family would like to announce that my Cousin Big Tony, Simon Cowell’s ex-bodyguard, passed away on Easter Sunday while on a rugby tour,” Tony’s family wrote.

“He was a loveable rogue and will be missed very much, RIP Big T.”

Hammersmith and Fulham RFC also shared a tribute to its website.

“It is with immense sadness that the club pays tribute to a true gent and great Hammer, Tony Adkins, who passed away whilst on tour with the Club in Poland on Easter Sunday,” the club wrote.

“Tony was at heart a touring Hammer, and he liked nothing more than taking the Hammers brand of Rugby beyond the boundaries of London.

“He represented the club all over Britain and Europe, his first tour being Brixham, before going on to earn touring badges in places as far afield as Barcelona, Prague and Amsterdam.”

Following his time on the show, Adkins spoke to The Mirror about Cowell.

“Simon was so laid-back and a very nice guy,” he said.

‘He was generous, too. Whenever the show finished — and at Christmas — you’d get an envelope as a thank you. The last one I got had £700 (A$1451) stuffed in it.

“He’s a very private person. He doesn’t give much away. You never see him drunk or out of control.”