Louis Jude Ferrigno Sr. was born in Brooklyn, New York, to Victoria and Matt Ferrigno, a police lieutenant. He is of Italian descent. Soon after he was born, Ferrigno says he believes he suffered a series of ear infections and lost 75 to 80% of his hearing, though his condition was not diagnosed until he was three years old. Hearing loss caused Ferrigno to be bullied by peers during his childhood: “They used to call me ‘deaf Louie, deaf mute’, because of my hearing and because of the way I sounded.”

Ferrigno started weight training at age 13, citing bodybuilder and Hercules star Steve Reeves as one of his role models. Because he could not afford to buy weights, he made his own using a broomstick and pails which he partially filled with cement. He was also a fan of the Hercules films that starred Reeves. Ferrigno attended St. Athanasius Grammar School and Brooklyn Technical High School, where he learned metal working.