What does it mean to be in possession of your voice, and what would it mean for someone
else to possess it? How do you reconstruct someone who is no longer here, and how
real would that version of them be? Would it matter how real they are?
These are some of the many philosophical, ethical, and technological questions that
artificial intelligence (AI) has brought to the fore and that Sarah Sweeney, associate professor and chair of art, is contending with through her long-term project
“Conversations with My Deepfake Dad.”
Sweeney’s father, a screenwriter who documented his work and life in tape recordings,
died at the age of 44, when Sweeney was 17.
“When I turned 44, I had all these thoughts about what it was like to be the same
age he was when he died,” Sweeney says. “Was I at the same place in my life?”
Around that time, she also began to take an interest in deepfakes — media generated
using AI to replace someone’s likeness — and the implications of such capabilities.
“As I was looking through flash drives and found old movies of my dad when he was
very young, I had this thought: What would happen if you could make new memories with
someone who had died?”
Candid photos of Sweeney’s father
Sweeney contacted a company called Resemble AI and began working with the firm to
create a clone of her father’s voice using its speech-to-speech software. “It takes
someone’s speech — we’ve been calling them ‘the puppeteer,’ the person who’s speaking
— and maps my dad’s voice onto it,” she explains.
Over the next several months, she found six hours of her father’s recordings and worked
with Resemble AI and her mother to think through the ethical and technological challenges
of generating new conversations using her father’s voice. “I think a big question
in this project is, ‘Who has the right to say that someone who is no longer with us
can be used in a deepfake?’ I had to produce a death certificate and a letter of consent
from my mother.
As Sweeney moved forward with the project, she considered what the conversations with
her father would look like. “I started thinking about, when someone dies, where might
those answers come from?” she recalls. “One of the things I became really interested
in is how we hold someone after they die and how we imagine they’re the only version
of that person. But in fact, there are maybe 60 versions of that person in the world
held by different people.”
She decided she would generate six audio conversations, each based on different archival
materials, interviews, and research. Sweeney writes both sides of each 20-minute-long
conversation. She then speaks her side and her dad’s deepfake voice speaks his.
“The first dad is based on interviews with my mom and my sisters. All his answers
are written from what they think he would have said. So I got to meet my mom’s version
of my dad and my sisters’ version of my dad. And I’m learning a lot about him through
that.”
“The second one is just him, so that one is really interesting. It’s only words that
he used in his letters and his tapes.”
“The third one is therapy sessions. They’re called empty chair therapy sessions. The
fourth is a medium. The fifth is a chatbot. And the sixth is just voice memos to myself
that I’ve been recording in the car.”
The audio recordings, along with photographs, will constitute an online presence for
her father.
From left, Science Machine Shop Supervisor/Technician Jeff Britton, Senior Teaching
Professor of Physics Evan Halstead, Zoë Bilodeau ’23, Omar Gomez ’23, and Associate
Professor of Art Sarah Sweeney work together to install the phone booth in Scribner
Library.
Sweeney has taken the audio component to several classes at Skidmore, including psychology, English, media and film studies, and First-Year Experience classes.
“I listen to this file with them and then we have a conversation.”
Sweeney is keenly aware of the complex range of issues that her project raises, and
she is one of a number of Skidmore faculty members — from Associate Professor of Psychology
Jess Sullivan, who taught the course Artificial Self in spring 2023, to Professor
of English Mason Stokes, who taught the course Writing with AI in fall 2023 — who
are grappling with the complexity of AI through their research and teaching. Skidmore’s
Center for Leadership, Teaching, and Learning and the College’s Learning Experience Design and Digital Scholarship Support (LEDS)
teams even organized a semester-long AI Think Tank devoted to exploring some of those
issues this past fall.
Sweeney’s project continues to unfold, and rapidly evolving technologies continue
to raise ethical, philosophical, legal, and other questions. Sweeney hopes her project
will be thought-provoking in a multitude of ways, to many different audiences.
“If you look at the news, the worry right now is that our world is changing such that
we can’t figure out what’s real and not real. This piece is a little bit about saying,
‘Is there something interesting in something that’s both real and not real?’ And I
think that’s what I really want us to consider about these technologies that work
with our memories. They’re kind of incredible but also really scary. And I think that’s
what this piece is really about — is this incredible possibility of this strange conversation
I’m having with my dad.
“One of the things people often ask me is, ‘Does it matter that it’s real or not real?’
And I don’t think it does. At some points it really doesn’t matter. It feels like
my dad.”
Presented through a collaborative Skidmore effort
The “Conversations with My Deepfake Dad” audio recordings that Associate Professor
and Chair of Art Sarah Sweeney creates are being shared online as well as through
physical exhibitions.
The first conversation was presented through a phone booth, which debuted in October
2022 at CultureHub in New York City. A parallel piece — a rotary phone — was sent
to Los Angeles, and the phone booth installation was brought to Scribner Library for
the spring and summer of 2023.
A number of collaborators at Skidmore have helped to bring the installation to life.
The Art Department’s shop supervisor and building safety coordinator, Paul Davis, constructed the phone
booth.
Andrea Casey, art technology support coordinator and program coordinator, created
the prints for inside. Senior Teaching Professor Evan Halstead and Science Machine
Shop Supervisor/Technician Jeff Britton from the Physics Department, along with three of Halstead’s independent study students — Omar Gomez ’23, Zoë
Bilodeau ’23, and David O’Connell ’23 — assembled the electrical components that allow
the conversation to start and stop with the opening of the phone booth’s doors.
“What I’ve learned is that by partnering with people who have this incredible expertise
— students, faculty, staff — the piece is so much richer, and I could never have made
this thing by myself,” Sweeney says.
To listen to the conversations, visit www.mydeepfakedad.com
