Audiences Interact with Pinchas Gutter in Early New Dimensions in Testimony Pilot

Development took a major step forward this month for New Dimensions in Testimony, the three-dimensional, fully interactive display of Holocaust survivors created by USC’s Institute for Creative Technologies and Conscience Display. Audiences had the chance to interact with the pilot for the first time.

ICT and Conscience Display are working on the first pilot of New Dimensions in Testimony, in which Holocaust survivors will be digitized so they can interact with people in museums and classrooms around the world long afer they’re gone. Voice recognition software will allow viewers to ask them questions about their experiences during the Holocaust and their 3D, virtual double will answer in real time.

Holocaust survivor Pinchas Gutter is currently participating in the first pilot of NDT. During the first filming session, he answered hundreds of questions about his life while seated on ICT’s light stage, surrounded by 6,000 LED lights and 50 cameras. Gutter gave his testimony to USC Shoah Foundation’s Visual History Archive, and executive director Stephen Smith interviewed him for the first session.

Audiences at Los Angeles Museum of the Holocaust (LAMOTH) and USC were the first to test the interactive component of the pilot earlier this month, in order for the project staff to hear real questions audiences might ask of Gutter and begin to “train” the computer to recognize them and give the most appropriate answers.

At each of the test sessions at LAMOTH and USC, footage of Gutter’s filming session was displayed in front of the audience while project staff sat behind them, controlling the video. After watching a five-minute introduction in which Gutter briefly explained his experiences during the Holocaust, audience members could ask the virtual “Gutter” any questions they wanted. The staff would then search through the interview to find Gutter’s answer and play it back.

If a question was asked that Gutter hadn’t answered in his first interview session, the staff will make sure he answers it in the next one (currently scheduled for the end of August). For example, several people have asked Gutter when he immigrated to the United States – a question his interviewers hadn’t thought to ask him since he did not actually immigrate to the United States. But now that staff know it is a question audiences might ask, they will be sure Gutter has an answer for it.

At USC, a group of high school students participating in a neuroscience summer program attended one of the sessions. For over an hour, the students asked Gutter a wide range of questions, from “What happened the day you were liberated?” to “At any point did you experience a loss of faith or spirituality?” Gutter’s recorded answers were honest, descriptive and unedited, giving the session a conversational feel.

Though staff could not find answers to a few of the questions (“What did you think of the Nuremberg trials?”) they were able to find the answers to the majority of the questions very quickly – in anywhere from five to 30 seconds.

Staff were careful to note that the pilot is far from finished. It is lacking the voice-recognition software that will allow Gutter to engage in lifelike conversation without staff manually playing back answers. The video showed Gutter sitting in front of a green screen – when finished, he will be projected into the room as a 3D virtual display. Staff also noted that later, they will add a resting pose between each answer so Gutter appears to be listening and each answer blends seamlessly into the next.

But these missing components didn’t deter the students’ enthusiasm for the experience. At the end of the session, Conscience Display managing partner Heather Maio asked the students what they thought of the project.

“Sometimes I felt like it was an actual conversation,” one student said.

As the students got up to leave, staff played for them Gutter’s goodbye: “Thank you for listening. Thank you for coming,” he says, smiling.

Delighted, the students said goodbye to him, just as they would if Gutter had been physically there the whole time. “Thank you for having us,” one student added.

Smith and Gutter gave a presentation about NDT at March of the Living Canada’s conference "Holocaust Survivor Testimony in a Digital Age: Collection, Preservation, and Education" today in Toronto.