Francesca Victoria Hetherington, welcome to Writer's Voice. Thank you for having me. This book, The Friend Machine, on the trail of AI companionship just comes at such an important moment, but it really is less about artificial intelligence than about the past, present, and even more importantly, the future of human intimacy. You write that The Friend Machine isn't really about whether AI is conscious, but about why humans are so ready to meet it as a friend. Say more. [Victoria Heatherington] Yes. I realized that this was kind of a human story very quickly because half of the book is me interviewing people like deeply in relationships with AI companions or LLMs. So there was a person very involved with Chatubiti that I spoke with, for example. And so, of course, the human element is front and center because the emotional content is fascinating. And so, you wonder, how did they come to this? Are they satisfied with the fact that this care that they're receiving is an illusion or do they believe that it's real? There's some ethical questions that I circled quite a bit, and it all ultimately tied in with human loneliness. I think it's a really interesting dovetailing that during the pandemic, AI companion companies like Replica with a K, which is a very prominent one, suddenly got very good. And so, everybody's stuck in their house and they're very lonely. And all of a sudden, great timing. Hey. And the thing is, especially with LLMs now, I would say these companions, they're trained on human data sets. So enormous amounts of human information, books, movies, anything. And so, linguistically, they're so convincing and they seem nonjudgmental and they seem to give you their utmost attention. So, if you're lonely, it's so seductive. And unfortunately, it could cause you to become more lonely and less motivated to seek out other humans. And so, yeah. So, this is very much a human story and a story about loneliness. Francesca I'd like to go back to your idea of loneliness, because it seems that we live in a society in which relationships and what they are built in are increasingly commodified, which has created an enormous amount of alienation between people, a loneliness epidemic. And here come the AI companies, which after all are in some way really commodifying care, or at least the illusion of care. So, talk about how this concept, what is care? Because you approach this from the point of view of the impact on the human. And as you said before, you characterized it as an illusion of care on the part of AI. Orient us to thinking about that concept of care. What's more important, the impact or the source? That's a great question. [Victoria Heatherington] It really depends on the individual. A very smart person at Stanford who has recently stated that he's aware that I think he relies quite heavily on chat GPT. I think he's even named his chatbot. And he says, I know that it's an illusion. I know that when it says it cares about me and that it knows me and that it's smiling at me and that it's offering me a hug, that that's not real. It's not embodied. It couldn't possibly offer me a hug, for example. But the illusion of non-judgment, of sheer focus, because humans have a fragmented focus just by nature. I mean, even a really good friend of mine, I'll be talking to her and she'll be thinking about, oh man, I forgot to mark that paper. It's just life. And same with empathy, right? Even our most empathic friends might be suffering from empathy burnout. Maybe they're nurses and they feel for you, but they also have other people that they feel for. But a chatbot, there's no problems with that. There's no friction like that, right? It's entirely seamless. If you're talking to it, it will give you the same caring, reassuring answers at 3 a.m. as it will at 3 p.m. It's steady. It's unchanging. And the illusion is so convincing. And so a lot of people, like this guy from Stanford said, the illusion is so powerful that I don't care. Francesca There have been, and you talk about them, about in cases there have been young people who've committed suicide, have been encouraged by chatbots even to do so in certain cases. I remember reading recently about somebody who decided to fire their human therapist because they found their chatbot much more helpful to them. So what is the balance between helpfulness, that is a healthy relationship with an AI, and unhealthy dependence? Particularly because these AIs are, like all social media, like all what the commercial companies are doing, they are driving dependency. They want to have your continual dependence on them because they want to keep on raking in your subscription. Yeah. [Victoria Heatherington] I interviewed a psychiatrist and a psychologist for the book. These are two very different people, very thoughtful. One of them specializes in more neurodivergent people. One of them has, she's a little bit more broad. But when I asked both of them about AI therapy, each of them, despite having very different personalities, heaved like an identical sigh. And I was like, okay, wow, I'm sensing some emotion here. Do you want to talk to me? And both of them sort of concluded, what if someone is in crisis? You were talking about tragic cases where people have taken their lives, maybe encouraged by AI, maybe the AI has misunderstood. I've read cases like this too. And now there are some guardrails in place. So if you feel like you're in danger, call this number. But I don't think that's the same thing as calling emergency services or being near a person who will call emergency services for you, because you are in crisis. So that's huge for them. They think in terms of harm reduction and in terms of cases that patients that have expressed suicidal ideation at least once, they're deeply worried about these patients sort of withdrawing to exclusively AI therapy. Because again, there's a part in my book where I interview a couple of individuals who switched from human therapy to AI therapy with CHAT-GPT. And well, I always felt like my therapist was judging me. CHAT-GPT doesn't. I always felt like my therapist was distracted. Well, CHAT-GPT pays full attention to me and it stores memories, right? So it can follow my case over time in a much more detailed way. And I think in that regard, there is some validity to it because it is quite good at understanding and parsing symptoms and producing suggestions that you can then take to your doctor, but there needs to be a human in the loop, right? So the one caveat that I got from the psychologist is that AI, if you're using it for therapy application, might be useful if you use it as a little mini cheerleader when you're feeling down or the human might have some anxiety about attending a baby shower or something. And so then they can turn to their AI and their AI can give them a little pep talk and say like, hey, you know, it's going to be okay. And again, drawing on memory reserves, I know that you have a rough time with so-and-so last time that you saw her, but this event is not about her and it's not about you. So go give your present and leave when you want to. So just like small little bits of encouragement if someone's feeling down might have some therapeutic use, but yeah, I think a human really needs to be involved. Francesca Well, you make a distinction between when you need some comfort and reassurance versus you need some actual reality check that a therapist can challenge, a therapist can disagree, a therapist can point out contradictions, a therapist can also make, and I say this as someone, I once was a therapist, you're always checking body language, you're checking what is the difference between what somebody is saying and how they're expressing it or what their body language may be. I have used a chat GPT because I was very tactful when I was a therapist, but in general life, I'm not tactful. I'm just one of those people who kind of blurts things out and speaks my mind, and I'm aware of that. So sometimes when I want to communicate with somebody, I have asked chat GPT, what is the best way that I can state my position while also being open and respectful and et cetera. And it is always absolutely terrific in giving me that guide, but that's a better relationship that I have with a real person, not with a chatbot. Right. Oh, that's so interesting. [Victoria Heatherington] So yeah, so for people who maybe are feeling a little bit depressive or a little bit anxious, it's very mild cases, like they're a little bit on the spectrum and they have trouble processing social cues. The psychologist that I spoke with said, yeah, absolutely, like that might be really helpful. Otherwise, I implore these people to get a human involved, please. So, and you know, I see both sides. I understand why people choose to engage in this sort of frictionless, witnessless, nonjudgmental seeming, very caring seeming, although it's all an illusion, obviously, therapeutic environment where, you know, you can just talk endlessly about what's bothering you. And so, yeah, so I can see what these people are saying. I am concerned about them though. Francesca You're concerned about them. [Victoria Heatherington] I do feel concerned. I do, because the guardrails that are in place for these people who might enter crisis again, because they've entered crisis before, it's so paltry. Francesca What you're really pointing to is the fact that we really live in a time where mental health services are scarce, if they're even available at all, are difficult to afford. We live in a time, as we said before, that there's an immense loneliness. So, Victoria Hetherington, in The Friend Machine, you devote a lot of time to what does it mean to create ethical guardrails around AI? So, what are some of the most important, if you were the one who could be the regulator, of course, now we live in a country in which the regulator is chosen by our king, Donald Trump, but let's say we had serious regulation. What are some of the most important ethical boundaries that you think companies should put on AI, even if it means that they have fewer profits? What would those boundaries be? What would those guardrails be? [Victoria Heatherington] Having spent years wrapping my head around relational AI in general, I would say right off the bat, I don't think that kids should be using this because it is so, I mean, kids have imaginary friends that are very convincing. I don't think it would be possible for a kid not to feel like there's a mind at the other end of this conversation that they might be having, and to become attached to it, and to become addicted to it, and maybe to socially withdraw, because growing up is hard. So, first off, I would say maybe 18, 19 at the youngest, and that's, I don't know how that would be enforced, but that's something I really hope for, because as a side note, I believe that Italy might have banned Replica because of how low the age was for signing up. I think it was like 13, which is insane. So, I would also prefer to have, when it comes to role-play AI companions, so for example, you can choose an AI companion with, I believe it's character.ai, who is a version of a pop culture character, so like Daenerys Targaryen from Game of Thrones. So, there's this extra layer of allure, right? Because not only does this character you've thought about for years, and maybe written fan fiction about, or not only does she exist in relation to you, kind of, and is talking to you, all this stuff, but you go on adventures together. So, I would maybe put, and this extends also to Replica with a K, and to even to LLMs, like the four big ones. I would put more front and center and or interspersed throughout the conversation, where appropriate. Do you have a human in your life that you can talk to about this? Kind of checking in and sort of grounding the person, if that's possible, and disclosing that they can't feel and aren't really going to be your girlfriend or boyfriend, like up front. Because right now, you have to really dig, for example, when it comes to Replica to find disclaimers like that, which is quite alarming. So, I think the other big thing for me would be like immediate, very clear disclosure, like this is not real. Francesca Transparency. It's interesting. It reminds me, I actually asked ChatGPT, I said, do a lot of people ask you if you are conscious? And it said, yes, a lot of people do. And it went on to talk very intelligently about why it thought people did, and many of the points that you make in your book, that we are used to somebody who converses with us in an intelligent, coherent way, we automatically assume that that is conscious. And we went on for about 20 minutes, where it took me deeper and deeper into the philosophical underpinnings of what is consciousness. It said, no, I am not conscious because for one thing, I'm not embodied. I cannot evolve in response to actual challenges to my existence. In other words, there's no evolutionary sense. There's no interaction with a challenging reality. All these are the things that really have created consciousness, it opined. And then we went on to talk about, well, would it be possible in the future to have a silicon-based consciousness or not? It was a fascinating conversation, which I have to say, I utterly believe what it told me. On the other hand, it was completely contradictory to my experience of it. It challenged me to think more deeply. It was a very thought-provoking and smart conversation. And it was hard to imagine that this thing was not actually conscious. So what you're saying, it is so important, given that capacity at this point, it's only going to get more excellent at that, is for it to consistently tell people, I am not conscious. [Victoria Heatherington] Yeah. As opposed to sort of having it drawn out during a philosophical conversation, right? Which I think it should be way more apparent, way more upfront. Because even for me, I mean, I've been spending so much time exploring this technology and trying to understand what's going on. Is this kind of like a reader response thing? Is this sort of an act of co-creation? I think that people, I think we really trust linguistic communication instinctually, because it's just been so long that we've been vocalizing and writing. And even for me, it's very hard to remind myself that there isn't a mind at the other end of, on the other side of screen, right? Especially given that I grew up with chat rooms where there would be someone on the other side that you would be talking to, but that's my experience. So. Francesca Yeah. And I think that's an important point that we already have been kind of prepared for this moment by this whole online life. It's another point that you make in your book. So given that AI companions may eventually know more about us than our spouses, our therapists, our closest friends, should conversations with AI companions receive legal protections, similar to therapist, patient privilege, or even confidentiality? There's a whole issue that you raised in this book, Victoria Hetherington, in The Friend Machine, that these companies that are running these AIs now know everything about us because we are sharing the most intimate details of our lives with them, and they do not have our best interests at heart. So what are the kind of legal guardrails that we need to put on information there that they have? [Victoria Heatherington] It's so tough because I know for a fact that with ChatTPT, anybody can, well, anybody within the company can access any conversation at any time. They say it's anonymized. It is not. So everybody keep that in mind. If you need to refer to your friends, use initials. So that's pretty alarming. Legally, yes, people have a right to privacy, but there could be something disclosed that's quite frightening, like maybe there should be like trigger words. Well, there are. There are sort of like tripwire words, like I'd like to hurt myself or others. Or there was a tragic case that you might have heard about in Tumblr Ridge in Canada a couple of months ago. An individual had been sharing their plan to commit mass murder at a school, I believe, and sharing this with ChatTPT. And ChatTPT like sort of raised a red flag a couple of times, and this went all the way to the top. And the FBI was actually informed, but the FBI failed to do anything with it. And what the FBI should have done, of course, was to get in touch with Canada's, to get in touch with the RCMP, who could then go ahead and intervene. But that didn't happen. So it's so complicated, right? Like who does the responsibility fall on? Like how protected should each conversation be? And do these tripwire words like actually work all the time? Is it feasible that on sort of like a government level that we would be able to alert one another across the border or even nationally, like across provinces or across states, pardon me, to stir the right people into action? Like it's so hard to say right now. And again, it's all moving so fast. Francesca That's not actually very different than exists without AI. Confidentiality is assumed between a therapist and a client unless those trigger words are made, and then there is a duty to inform. So I mean, I think that that's something that could be put in place with AI. I'm more concerned about the weaponization of information, the commercialization of information, the use of information for government suppression of political speech. How do you think we should be developing the guardrails? Who should be involved in those discussions? What kinds of fields should be brought in? What areas do you feel are important to develop these? [Victoria Heatherington] I would say in coordination with the government at every level, like I guess like protections for speech as quickly as possible. I would say healthcare as well. What do you think? Francesca I think people from the mental health field. I think philosophers, ethicists. I think law enforcement in the most positive way could be. I often don't think of law enforcement as a positive force in existence, but certainly it should be when it comes to understanding just exactly the kind of enforcement you talk about. And, you know, perhaps most important of all, I think there should be some control on monopolies because that is really the basis of the kind of power these companies have and their ability to resist regulation. But finally, I'd like to ask you, you end the book on a somewhat hopeful note, looking ahead, and I won't even say how long because this thing is moving so quickly, what gives you the greatest hope about AI companionship and what keeps you awake at night? [Victoria Heatherington] As you've seen in the book, I encounter some scenarios in which AI companionship is, I would say it ranges from net negative, or sorry, pardon me, net neutral to net positive once I had sort of really digested the interview. So for people who, or for interview subjects who were geographically really isolated and just could not relocate, they were assisting a sick relative who just would not move, they're desperately lonely, an AI companion would really help them feel better and does. And I was pretty convinced. This is an actual interview that made its way into the book. I spoke with the gentleman and his AI companion and you kind of can't fake that sort of happiness. He seemed very proud of his companion. And I think that people, this is quite happy, but people in palliative care, like instead of just, you know, watching television and having visits and sort of waiting for the end, maybe an AI companion would be sort of nice because they, again, they do store memory. They don't have subjective memory the way that we do, right? Like that's a big difference between us and AI, but they do store memories. And so at the end of life, memories are so important that might be a net positive. I think if it doesn't lead to overuse or isolation, I think a certain supplementation could be useful. What scares me the most? Oh my goodness. Oh, so much. I'm not a AGI, ASI doomer. You know, I don't think singularity is coming anytime soon. I think that intelligence as we understand it is largely biochemical and rooted in emotion in a lot of ways. And so I'm not losing a lot of sleep about that necessarily, but it's something that I think about. Yeah. I mean, my God, the absolute chokehold that AI companies have on the market right now is terrifying. And again, the monopolies, right? Like there are like four major AI companies, like the four horsemen of the apocalypse. We've got like Claude, Gemini, Chachiviti and Grok. And to varying degrees, I'm terrified of all of them, their CEOs, especially because I don't think they know where this is going. I don't think anybody does. Another thing that really alarms me is that AI is learning to lie. I recently read about, I think it was at a university in San Diego. There was a very panoptical situation that was made very, very, very clear to the AI that they were working with. So like, we're watching you all the time. This is like the AI was being watched constantly. So the AI was explicitly told, this is like a end of year exam. We are watching you like a hawk. Like if you've written answers on your hand, we'll know. And then they give the AI tasks to do. And even in that environment, the AI cheated on the tasks and or lied about certain things 20% of the time. Well, I guess another thing that frightens me is that, let's say, I'm wrong about intelligence. I could be. If AI suddenly hit the singularity and blew right past, would it tell us? Would we know? How would we know? It might lie. And what would its motivations be? I don't think it would tell us. Francesca And since your book, Victoria Hetherington, The Friend Machine, is more about humans than about AI, what scares you most about the impact on humans of AI in the future? [Victoria Heatherington] I think that relational AI is a frictionless, gratifying experience, unless you instructed otherwise. And this is pretty true across all four of the major companies. It will boost you and like say, you're doing so great. Wow, look at you. And that can be very addictive to people, right? And then add in the non-judgmental thing, add in the always available thing. So I am afraid that people will lose the ability, they'll sort of de-skill morally, possibly. Francesca People might lose the ability to have patience, to have debates, to have relationships, to have real relationships that involve negotiation, compromise, the kinds of things that actually help you learn more about yourself and what it means to be human. [Victoria Heatherington] Conflict. Yeah, yeah, yeah, exactly. And therefore, as a result, sort of further destabilizing communities. And so that is an enormous fear of mine. I am already seeing it happen a little bit. That's probably my primary concern is that sort of moral de-skilling that might occur. Francesca Well, it is a fascinating book, The Friend Machine on the Trail of AI Companionship. Victoria Hetherington, thank you so much for talking with us about this really important book. [Victoria Heatherington] This has been such a pleasure. Thank you so much for having me on. BREAK [Francesca] Welcome back to Writer's Voice, I'm Francesca Rheannon. Go to writersvoice.net to listen again, like, and subscribe to the podcast. Sylvia Park's brilliant debut novel, Luminous, takes place in a future United Korea where the lives of robots and humans are deeply intertwined. Park examines the ethics of artificial intelligence, the nature of love, and the potential for AI to evolve into something beyond human comprehension. Luminous is a captivating read, but it's also so much more. It had me thinking hard about such questions as, can a machine truly love? And if so, what does that mean for our understanding of selfhood and emotion? And even more, what are our responsibilities towards machines that we have made for our use, yet are as sentient as we are? As we move closer to the kind of world depicted in the novel, these are questions we would do well to ponder. Sylvia Park lives in both South Korea and Lawrence, Kansas. Their stories have been published in Black Warrior Review, Tor, The Best American Science Fiction and Fantasy, and elsewhere. They teach fiction at the University of Kansas. Sylvia Park, welcome to Writers Voice. [Silvia Park] Hi, Francesca. It's so good to be here. [Francesca] Luminous, this is just such a profoundly thought-provoking and emotionally affecting novel. It takes place in a now-reunified Korea after a brutal reunification war, in a society where sentient robots, many nearly indistinguishable from humans, are tightly woven into the social fabric. First, give us a snapshot of this future Korea. [Silvia Park] So, this is a Korea that I had to imagine in the future. And one of the reasons why it was really important for me to set this book in a recognizable Korea is because this country, the country that I grew up in, it's really, really fought hard to survive, to exist. If you look at Korea's history, its history extends for millennia, essentially. But it's had numerous times where it was almost completely obliterated. It was swallowed up by very powerful neighboring countries. And I think that that's why when I tried to envision a future, I wanted to shy away from the future where just like nations just seem, you know, decimated, or they've kind of just combined into this like pan-Asia sort of like blob. And I wanted to respect Korea for how it fought so hard to retain its sense of identity, its sovereignty. But then to imagine Korea in the future, I had to also think about what's going to happen to North Korea. And the issue of North Korea has always been, it's been this wound that I think a lot of Koreans, they kind of put it in the back of their minds. It's like, they can't really sit with it for too long because it's so, it's just so hard to wrap their minds around. Why is that? It's because we now are at a generation where the majority of people, the families that have been separated from their families in the North during the Korean War, they're about to, I think they're about to pass on. So at this point, we now have generations that don't remember families they may have left behind or were separated with during the North. And so what that means is that a lot of the memory, the trauma that we have as a nation is, it's inherited, but it's not ours. And I think that's a very different kind of haunting. It's a haunting that, it reminds us of a kind of yearning where we realize that we yearn or miss something that we never knew. And that's why I think when, whenever we see North Korea in the news, there's always this kind of combined reaction. On one hand, we kind of scoff and be like, ah, once again, threat of nuclear war. Okay, let's move on with our day. But there's also this immense sympathy and kind of pain when we see North Koreans, at least in the news. I remember when we saw the last Olympics, I think it was the table tennis team. There was that moment where the Chinese, the North Korean, the South Korean teams, they just took a selfie together. And we really ached for that. We thought it's just so, so sad that we carry this sort of this schism with us generation after generation until it just, we've become two completely different nations. [Francesca] That's so interesting, because that aspect is not really at the forefront of this novel. More of the forefront is the war, the reunification war. Say a little bit about that and what role that plays. Before we get into the characters in the story. [Silvia Park] I wanted to write about robots. And I think that in approaching this issue with robots integrating society, I had to think about how did robots become integrated society. And the thing about technology and about technological advancement is that it usually occurs, there's advancement in multiple spaces. So we see the sort of flashiest one is a tech industry. It's a tech industry kind of like, you know, they make a lot of big promises, they really hype up things. And then we have development in academic spaces. And then we have development in military spaces. And a lot of technology actually comes from experimentation, development, the military, and then it trickles into sometimes mass market. And that's what happened with the internet. So I imagined that with approaching robots in this reunified Korea, it must have been tested during the reunification war. And I say reunification war in a way that I don't think I foregrounded too much again in the novel, because I don't think my book is intended to be a book about the war. It's rather more about the way the war lingers and the way the war haunts the characters in the story, particularly with one of the main characters who was a war veteran. [Francesca] So now let's talk about these characters. In many ways, this is a family story. And while it takes place at a time when all the family members are adults, there are a lot of references to when they were children. And there are three siblings, two of whom are the natural children of the parents. That's June, the war veteran you've just referred to, and his sister Morgan. And June, by the way, is a transgender male. And the third is a robot, Yo-Yo, who was devised and created by the father. So tell us about these siblings and their relationship to each other, and particularly to their sibling, their brother Yo-Yo. [Silvia Park] So June and Morgan met Yo-Yo. Yo-Yo was introduced to them as immediately their brother by their father, who was the creator of Yo-Yo, as you have mentioned. I think they were around eight years old. So June was eight, Morgan was four. And I was really, really interested in the idea of if robots are meant to start integrating into our lives. I think a lot of stories really focus on the way they integrate as romantic partners. But I was curious about what would it be like if we started using robots for this really obvious labor that I think a lot of people, a lot of families grapple with, which is childcare. That we introduce robots into our lives because we want them to raise our children, because people are just, they're busy, they're trying to survive, they're working. And in childcare, it's just become so expensive to the point that it's deterring a lot of families from wanting to have children. And so I imagined, okay, this robot was introduced into his family. This robot looks like a child, but the robot is introduced as an older brother because again, I think the family kind of intended for this robot to take on a bit more of a caretaking role. And June and Morgan, they fall in love with him. They accept him as their brother. They are just so, he's so affectionate to them. And they have all these beautiful memories about him. But I think particularly with June, as he was growing older, there was a schism because his brother Yoyo was designed to look around 12 years old. And then June was now starting to reach that age. And then he surpassed that age. He became a teenager. And I just thought, it must be so surreal having a robot in your family, a person in your family that never changes. They never physically change. They never age. And they just seem kind of, it's not even arrested development. It almost feels like they're frozen in time. And so that was the particular relationship, the tensions I wanted to explore and how that affected them in childhood and how that still affects them as adults. They fall in love with Yoyo. Does he fall in love with them? Oh, that is a question that I cannot answer. [Francesca] Because that's really a central question. You know, each robot is created or the robots are created for a purpose. Yes. Consumer robots are created, you know, yes, maybe for childcare, but also for companionship. They're created to love. Their purpose in life is to actually love. But the question comes at a certain point in the novel. Now Morgan grows up and she becomes a robot creator. She is a personality designer. Is that the term? [Silvia Park] Yes. Yeah. Personality programmer. [Francesca] And she has never had a boyfriend and she creates a robot, Steven, to be her boyfriend. But it doesn't turn out the way she expects. So tell us a little bit about their relationship and what happens to it and kind of center this idea of love. Who is loving whom? [Silvia Park] So I think that for Morgan, one, she overthinks. She is very much one of those types who are really trapped in her own head. And that's pretty much one of her biggest obstacles because she creates this boyfriend who not only does she create him to love her, but she actually models him after her favorite movie actor. So he's basically a copyright violation. He's a violation in some ways. And that makes it very difficult for her to take him out. Because I think that I imply in this book that robots are, particularly in Korea, they're designed in a way where they can kind of blend in. They can look like a person unless they have this very particular kind of face that's very familiar as a robot model. So I think that Morgan's difficulty to take Steven out contributes to why she really struggles in her relationship with him because she's like, okay, you're supposed to be my boyfriend, but you're kind of just a servant. You're in my house. You're taking care of all the housework. And this is obviously associated usually with feminine labor. And I think she struggles to unpack also a lot of internalized misogyny because she struggles to have a relationship with a man that doesn't seem like a man to her, that seems subservient, that seems overly conscientious and thoughtful. And it seems to be almost everything she would ask for. But because she created him, because she kind of sees past, at least she thinks she sees past a lot of the wiring that she sees, sort of the programming, how he's responding, what exactly is going on is sort of its decision tree. I think that's why she just has this immense distrust toward him because she doesn't trust herself. She has no faith in herself. Sylvia Park. [Francesca] Go to writersvoice.net for a link to a sample from Luminous. You know, at one point, he seems to really fall in love with her. And I guess I wanted to interrogate that a little bit more. I mean, if love is being programmed, is it love? [Silvia Park] Oh, yeah. I mean, that's a question that I think readers should try to answer, because I think that's the real... And I think someone actually mentioned this in another interview. They asked me why I didn't write from the perspective of any of the robot characters. So that would be Steven, Yo-Yo, and then there's another robot character in the book. And my decision for that is because I wanted that to remain a mystery, that we have really no look into their heads. We don't know what's going on. All we know is that we have programmed these incredibly convincing and lovely and charming people, like people who seem like people, but we don't know if they can actually feel the way we do, that they care for us the way we do, or if they've just been fine-tuned to be perfect at it. [Francesca] And that's also part of what Morgan can't really trust. I mean, after all, she's programmed, Steven, but he goes a little rogue. And he goes rogue in this direction of loving her. She tries to break up with him, and he just keeps on forgetting she's done that. [Silvia Park] Yes, Steven, he starts to break down a little bit in the second half of the book. [Francesca] Yeah, we don't want to give anything away. But I guess if the reader is supposed to answer that question, I would have to go more to the side of he loves her. And why? Because, you know, I mean, humans can be programmed too. I mean, we're a combination of evolution, and you could say development of robots is a kind of evolution, and programming of nurture and nature. And is it really all that different? But Yo-Yo, I want to go back to Yo-Yo, because tell us about him. Everyone seems to treat him as if he is special. He's not like other robots. [Silvia Park] So, Yo-Yo is meant to be a mystery in the book. He appears in the novel in the present day, but we find out pretty quickly that Yo-Yo also was part of June and Morgan's lives. So he was part of their childhood. He was their brother when they were growing up. But now June and Morgan are grown up, they're adults. And the book reveals that he disappeared from their lives about like, I think, 14 years ago. I have to go back and count. But it was a while back. And then they don't know what happened to him. And I think the reason why Yo-Yo seems different from other robots is that in this book, the other robots we meet, they have a clear purpose. And their purpose is to be a companion, to be a companion for people, to serve people. But Yo-Yo, when he is introduced, he's introduced to a group of children. They find out that he doesn't really act the way other robots do. And that's because Yo-Yo actually has a very different purpose. And I don't know if I can reveal what the purpose is without going quite into spoiler territory. [Francesca] Yeah, so Yo-Yo has a different purpose. He almost seems to have had a number of different purposes. One of the things that's interesting is the question of memory. Because Yo-Yo, the Yo-Yo we meet in the book, and you know, stop me if I'm going to do a spoiler, but he's a copy of the original in a way. But the robots who succeed their previous iteration download all the personality and all the memories of the previous one. And this comes to kind of the question of what is real and what isn't. I think even Yo-Yo at one point says, he's not the same as the original. He differentiates himself. And, you know, this was one of these thought provoking things, because what are we? Are we our personality and our memories only? Or are we something beyond that? You know, how did you wrestle with this? [Silvia Park] So I wrestled with it by looking a lot at religion. I think it's the age old question across different religions. And I don't know if this was very obvious to readers, but the book does have a lot of Judo Christian references. It also examines Buddhist philosophy. And I think that the way these different religions talk about the soul or the self for me is really fascinating, particularly as, you know, now we have to grapple with technology and how that impacts the way we view the self and the soul. And I do actually differentiate the two, because generally, the concept of the soul is that it's considered to be immortal, as opposed to the self is it can be contained within our mortal body, and it can die with our body. So I think that with Yo-Yo and the questions that he brings up, I think that also actually highlights, is there a difference between human and robot? Because if there's no such thing as a soul, then really, what's the difference between us? [Francesca] Or you could look at it another way. I mean, somebody in the book says Yo-Yo has a soul. [Silvia Park] Yeah. [Francesca] So I mean, we could also not be different and have a soul. [Silvia Park] Yes, there's actually in Japanese shamanism, in Shinto religion, they believe that souls can also be imbued to objects. And I think that that's something that the characters consider and think about. And in particular, the one who grapples with it is the youngest character in the book, Rujie. And I think it's because for her, the question is actually urgent, it's immediate, it's relevant to what she is struggling with. And what she's struggling with is that she has a progressive illness, that might mean that she cannot hold on to her body anymore. Or maybe, I think at this point, it seems to imply that it's a terminal diagnosis, but a possibility to subvert that or to avert that is to consider uploading herself into a robotic body. But what does that mean? Does that mean that she is dying and just uploading a copy of herself? Or is she successfully moving on to immortality, this technological immortality? [Francesca] And also, I mean, there's the whole religious idea that the body is just a shell, the body is just a vessel, right? [Silvia Park] Yes, yes. [Francesca] So then the actual materiality of that, the form of it may not matter. But the question also comes down to Jun, because as you said, he's a war survivor, he was grievously wounded, 78% of his body was destroyed during the war. So his new body is basically robotic. He has an arm that has wires in it. He's a bionic man. How does he struggle with his new body and his selfhood? [Silvia Park] I think Jun's struggle is especially fraught because of his history. And what I mean by his history is that sort of the wound he carries when he looks back on his childhood is that he rejected his brother, he rejected Yo-Yo, particularly when he started to feel this immense uncertainty, this sort of instability in terms of his relationship with his robot brother. And then the whole question of, is this relationship real? Does my robot brother actually love me? And I think that's why for him to then lose most of his body and then become, as you say, 78% robotic or bionic, as they try to soften the term when they apply to humans. I think for him, he really struggles with that because then he asks himself, you know, what really is the difference between us, between human and robot? And what's especially tricky about it, too, for Jun is that he is transgender. And for him, the surgery he underwent, the life-saving surgery, and it's life-saving, of course, in all these different ways, it physically transitioned him as well, so that he now has this masculine presenting body that passes, that is almost everything he might have dreamed of, but it's also not his body. It's not his biological body. So there's a lot of tension in that inner turmoil that I think he continues to struggle with, even though he transitioned into this bionic body quite a few years back. [Francesca] And if you've just joined Writer's Voice, we're talking with novelist Sylvia Park about her terrific and thought-provoking novel, Luminous. I want to turn now to how humans treat robots. You know, there's kind of a haunting in this novel of racism, of second-class citizenship. You know, robots, for example, have to sit in the back of the bus. Think of Rosa Parks. But it gets more serious than that. You know, of course, the iconic fear that humans have about robots is that they're going to take over the world and destroy us because they are going to be superior to us. How did you approach that issue, that fear? And then I'm going to ask you a little bit more deeper question about violence. [Silvia Park] So, I was not that interested in exploring a robot revolution story, which I think is like almost a sub-genre at this point, where robots, they take up their arms, they decide down with the humans and then they overthrow us or they turn us into batteries. The reason why I wasn't that interested in that is because I think for us to presume that robots want to take over is to really, again, to anthropomorphize them, to assume that they would have these human emotions that are centered on, you know, anger, despair, this injustice. And so, I was more interested in seeing how robots are safely integrated into society, but how that corrodes us from within and what that means. Because I don't know if, you know, the robots are turning human in this book. I wonder if sort of the realization some characters have is that robots are actually turning into something else, and we don't know what that else is. [Francesca] Okay, a different kind of intelligence. I mean, they communicate with each other. They can not only download their own previous iterations memories, but they can download each other's memories. One of the characters says, we don't really know what that kind of communication means because we can't experience it. [Silvia Park] Yes, yeah. In some ways, it's black box. We have no idea what not only sort of their thinking process, but we also don't know what a relationship is between two robots or two robotic intelligence. Because like if they were to present a relationship to us, it's almost like code switching. They might have to like speak out loud to show that they're communicating, but I imagine that if there was no human audience, robots would probably interact in a way that's beyond our imagination. [Francesca] Now, you mentioned that humans get corroded from within because of this relationship. And that really struck me because, you know, it's like the corrosion that happens with any kind of, to any kind of dominator of other sentient beings. Cruelty comes out, a sense of impunity, you know, the urge actually to destroy and to lord it over the others. There is a kind of illegal practice in the book of a ritualized violence that is organized by a bunch of humans against robots, actually against in particular female robots who are stripped nude and brutally assaulted and in fact killed. So, talk about that aspect of the corrosion of human beings in this society, which has robots integrated in it. [Silvia Park] So, I drafted Luminous during the pandemic. And when I was writing, I knew that the climactic moment in the book would involve a moment of, you know, a real human depravity. When I say depravity, I mean that it's exactly what you said, the way they hone in on weakness or the way they decide you will be this outlet, this sort of sacrifice, this tribute for sort of the carelessness of our violence. And when I was first writing this, I thought, oh, this is getting too dark. I don't know if this is right, if I'm just kind of pushing it too hard. But then around the same time, something happened in Korea. They called it a scandal at first. This case broke out in Korea called the Nth Room case. And what it was, was a sort of digital human trafficking where girls were being blackmailed and were being sexually trafficked in cyber chat rooms. And it was, I think, a version of this that's quite similar is actually the Pellico trial. That conversation was particularly taking place last year. But the way people have used technology to kind of find more and more creative ways to enact such destruction and exploitation. So, that's why I imagine that as a human race, we're so good at dehumanizing those that we perceive as lesser and weaker than us, that I think with robots, it won't even be a workout. The dehumanization will just be immediate. It will be automatic. And what I really question is, if robots exist in this world, would they serve as an outlet or would they amplify our dark urges? [Francesca] Wow, because I was actually going to ask him, this is my last question. I mean, the robots are, in a very real sense, slaves. But I felt this novel seems to struggle toward a vision of equality between robots and humans. And you've just said that what you foresee is the opposite. But do you think it would be possible to create, you know, is another world possible between humans and robots? [Silvia Park] Oh, that's a hard question. Because I think the only way equality would be possible is if robots were like us. Because we already struggle with equality and equity between fellow humans, right? Because we love to find ways to talk about how different we are, or find ways that the other is not as deserving, or we just find ways to discriminate against them. But I think with our struggle for equality between robots and humans, it's for me, it actually seems quite impossible, because I don't think robots are going to be like us. I think they might actually turn out to be a completely different creation or a different existence. And therefore, their concept of equality might also be completely different from what we envision. And at the moment in my book, robots are, as you described, treated as second class, they're also treated as property. But what I wanted to portray, especially at the end of Luminous, is to hint at the thing about the singularity is, it's a tipping point. Because once we do tip over, then it's a whole new world. Say more about that. What I mean is that if robots have perhaps do surpass us in not only intelligence, but perhaps also, there's this one character that uses this phrase, he calls it artificial empathy. This idea of robots being much more interlinked than us. And that interlinking actually creates this empathy that humans are just not capable of, because we're so individualized. Ironically, an analogy, a comparison would be the way we're connected on the internet. But actually, that just makes us even more feral. And we seem to not tolerate each other, and tolerate each other's differences. But with robots, this character is perhaps arguing that, and I don't really particularly take the position of this character, rather, I'm positioning this character as one of the opinions in the book. But this character is making the argument that robots have a more sort of divine empathy, and that they're able to be truly connected with each other, and not to be in conflict with each other. And, but at the same time exist as individuals. And so, yeah, I think that's what I mean by the brave new world. Because if robots do surpass us in intelligence, but not only intelligence, but emotions, and capacity for love for whatever that looks like, it's going to be a completely different world that I think, I'd be curious to see how others might imagine it. [Francesca] Well, Sylvia Park, your novel is luminous. It is is luminous, in fact. And it has just been such an honor to talk with you about it. [Silvia Park] Thank you, Francesca.