Manfred Goldberg was born on April 21, 1930, in Kassel, Germany, to Rosa and Baruch Goldberg. He had a younger brother. Raised by Orthodox Jews, Manfred attended a Jewish primary school. His father was a textile merchant, and the family lived comfortably. However, after gentiles were restricted from doing business with Jews, Manfred's father's business fell apart.