Ep. 276: “Who’s the Child Passenger?”

“What the Hack?” is DeleteMe’s true cybercrime podcast hosted by Beau Friedlander.

Disclaimer: This transcript was auto-generated and may include minor errors.

Beau: You’ve probably seen this all over the place. There’s little interfaces. I don’t care what you’re using. There’s these little interfaces for AI. You wanna use AI with this? You want some AI with that? It’s like, “Do you want mayo with that?” Forget it. Meta’s doing it now. You may have seen it if you’re on Facebook or Instagram. There’s a little tiny question that pops up under a post, and it’ll be like, “Hey.” I mean, really? Get lost. I’m trying to do something here. They’re trying another way to get you to engage. Most of us scroll right past these things. For whatever reason, one day, our guest today, Kalie Robins, didn’t Is that good? Kalie Robins posted a video of herself singing in a car with her daughter. It’s the kind of thing parents post every day. Under it, the video, Facebook showed one of its suggested Meta AI questions.

Clip: Facebook put a Meta AI question underneath it that said, “Who’s the child passenger?”

Beau: Kalie thought Meta wanted her to tag her kid. She tapped it to say no. But it wasn’t that. It wasn’t even asking the questions. It was answering… it produced her daughters’ names, their birth details, photos, hobbies, favorite books, and beaches. Then Meta AI suggested another question: where does Kalie Robins live? Where does Kalie Robins live?

Clip: and then it says, “Pinpointing Kalie Robins’ location.”

Beau: If this were the only example, one bad prompt on one bad afternoon it would still be kind of horrible. But of course, it isn’t just that. Meta AI has been in the news a lot this year, and Meta has been in the news forever as the dark star of anti-privacy.

404 Podcast Clip: This is one of the wildest hacking stories I think has happened in quite a while. The headline is, “Hackers simply asked Meta AI to give them access to high-profile Instagram accounts. It worked.”

Beau: Don’t worry, Meta’s fixed it. And Meta always fixes it. Meta also says they fixed the whole question thing with Kalie’s daughter. No problem. Nothing to see here. Move along, Officer Barb Brady’s on the job. Meta keeps fixing things. Meta treats things like a bug, a weird prompt, a loophole. Even their founder said, “Let’s move fast and break things.” This is another example of moving fast and breaking things. Like don’t ask for permission, apologize. Apologize. What if it’s not a bug? What if this is just what happens when you have a company that’s whole reason for being is to know as much as possible about us. I’m Beau Friedlander, and this is “What the Hack?” The podcast that asks, in a world where your data is everywhere, how do you stay safe online?

Start

Beau: Kalie Robins, where are you in the world? Do you wanna say what city you’re in? That’s probably pretty easy to find out.

Kalie: There’s only a few hundred people, so we usually go by the town that’s closer that’s an actual town. So I would probably say Richfield.

Beau: Richfield. And not Richfield, Utah

Kalie: Yes. I’m actually from Scipio, and now we live in a smaller town.

[Ranh SFX / Pastoral Music, etc]

Beau: So I used to work on a cattle ranch in Antimony

Kalie: No, you didn’t

Beau: I did, yeah

Kalie: Why?

Beau: I had a cousin who became Mormon, did a mission, and met a family that had a big ranch down there. And, My parents sent me there in the summer to toughen up.

Kalie: That’ll do it.

Beau: I learned to eat over a campfire and even eat steak that was harvested on the way because a cow died, you know, when we were driving them up to pasture. So yeah.

Kalie: That’s real cowboy stuff

Beau: That’s real cowboy stuff, Kaely. And you’re from real cowboy land, though.

Kalie: I am, and my husband’s a real cowboy, so I’m very versed in that stuff.

[Beat]

Beau: You know, I made it clear to you that I know where you’re from, not in a creepy way, but in a meaningful way, which is this, that you’re pretty rural out there. I mean, how long would you figure it would take state police to get to you if you had actually had a problem?

Kalie: If it was, I mean, city police, they could be here pretty quick ’cause there’s not a lot going on. But if I needed state help, it would take about a half hour

Beau: Yeah. I mean, it is rural, and that, and it is concerning because you know, there’s a house to your right and a house to your left and folks don’t understand that. So

Kalie: That’s lost on a lot of people. I basically live in a field. Like, my address isn’t even populated on Google Maps. We’re nobody out here

Beau: Exactly. Well, and that can be a good thing too. You also live in an area of the world where, as I recall, there were even times when I was a teenager when somebody would hand me a revolver with a holster and say, “We’re going up on Griffin Top,” for example, “And there’s bears and there’s rattlesnakes, and you’re not going without a sidearm.”

Kalie: Yeah. Absolutely. We use them regularly around here, so we gotta be safe, but we do use them.

Story Start

Beau: Now, here’s another tool. It’s something that we all use to communicate with people online, and it is Facebook. It is all of the allied offerings of Meta. And I’m talking to you today because You posted a video that horrified me. And I wanna say it cracked me up ’cause it… you’re funny. If you don’t know, Kailee’s online personality, you’re really missing out. Kailee bills herself as a travel mom creator, but you’re not. That is… You’re much more than that. The handle on Instagram is KOnTheInside. So if you’re looking for some good parenting content, KOnTheInside is a way to go. Now, the video that I saw was, you had posted a video onto Facebook, and underneath it, Meta AI had popped up one of those questions it generates to get people to interact with its AI assistant and learn more about what they’re watching.

Clip: Yesterday, I posted a completely innocent video of me and my daughter singing in the car, and Facebook put a Meta AI question underneath it that said, “Who’s the child passenger?” First of all, why in the hell is that even a suggested question about a minor? But I clicked it, and this is where I started getting really uncomfortable.

[Beat/Music]

Kalie: I’m kind of new to all of this, and so I got a copyright violation notice. So I’m panicking. I’m like, “Oh my gosh, what did I do?” And I looked at the copyright violation, and underneath my video are these questions. And so I thought Meta was asking me to tag my child. Like, who’s the child? And I was gonna say, “Absolutely not. You don’t get to know that.” And when I clicked on it, it just gave everything you would want to know about my daughter, even my child that wasn’t in the video. It just kept offering more and more information. And honestly, I got, like, sick to my stomach, and I’m still, like, not okay about the whole thing. It was horrifying.

[give this some space]

Beau: So you posted the video and you got a message from Meta saying there’s a copyright problem with it?

Kalie: Yes.

Beau: What was the copyright problem?

Kalie: We recorded a video in the car and I had been playing music, and so it was saying I copied someone’s music. And so that’s, I was like, “Oh my gosh, do I need to delete the video? Do I need to change it? Why?” Like, I just didn’t understand you couldn’t use video from YouTube.

Beau: Okay, so hold on now. A quick word on the copyright notice. So you’ve got a song playing in the background of a video shot in your car. It doesn’t necessarily mean you did anything wrong, it just means the platform recognized the song and flagged it, and they are gonna take down your video or they’re gonna make you answer some questions. Do you remember what the copyrighted material was that was playing in the background of your video? Do you remember what the copyrighted material was that was playing in the background of your video?

Kalie: Yeah, it was just this DJ that me and my daughter listened to, and we were just jamming to his music.

Beau: How do we go from like, Okay, there’s a copyright violation. Cool. Totally understand. Hey, you know what? That DJ and his music deserves to get paid for every play, including on your social media maybe. I don’t know. Maybe. How do we go from there to who is the child in the video? ‘Cause that is the next question you were asked, correct?

Kalie: Ya. None that I can figure out. If it noticed that there was a child, that should have stopped right there.

Beau: Yeah. You would think so also because you know, there are laws against that.

Kalie: Exactly.

Beau: I mean, there’s COPPA, a federal law that puts limits on how online services collect personal information from Children,There are state laws that go even further. Lawmakers are still debating things like, Kids Online Safety Act, or KOSA, which is aimed at creating additional responsibilities for platforms when it comes to protecting minors. It did get through the Senate. It’s stalled right now. It’s not a surprise. And the question becomes, how do these protections even work when the technology itself is changing so quickly? Because this isn’t just about whether a stranger can identify a child in a photograph. It’s about a platform looking at a video of your family and using AI to generate questions about it, and then potentially revealing information you never intended to share, potentially. Doesn’t matter that AI is being pointed at this thing to generate questions. Means AI is pointed at this thing, your dossier, that index to the I that is you all the time.

Transition

Beau: My experience of the way that Meta interacts, it’s like today, I’ll give you an example. I posted a short video in 2015, not to date myself, of my children in ballet class, my children who are now 25 and 22. And context was they would never let me shoot a video of them doing ballet because it’s embarrassing, and, “Dad, knock it off.” So the caption was, “Creep shot from Dad.” And I filmed it when they didn’t know I was filming it. It was the cutest thing on earth. Meta today told me they were taking it down because I violated their terms because I was a creepy stalker. And I was like, “No, no, those are my kids, and you know those are my kids.” And this, that’s the sentence I want you to hear is, Meta, you know those are my children because I’m me, and you know who I am, and you know who they are. So, Kalie, when it asked who the child was, did it already know the answer?

Kalie: Yeah, I clicked on it and it knew everything.

Clip: It starts pulling completely separate information for each of my kids, their names, birth information, pictures and videos of them. It pulled a newborn picture, from my mom’s page from years ago, and then it pulled this. I deleted this picture years ago. I’m not going to pretend that I know how it works, but I know this picture has not been sitting on my page for years.

Beau: So daughter one Is immediately identified?

Kalie: Yes, by name. Her full name, her legal given name. Both of them are, even though daughter two was not in the video.

Beau: Okay. That’s very strange. And, were there photographs also to show that they understood who was who?

Kalie: Yes, it showed pictures and videos. That’s part of the reason I was so upset. A few of the videos that it did get from me, I did not name my kids. The information it gathered on their name, their birth dates came from like excited grandparents and aunties out of state. The information it gathered didn’t come from my page

Beau: It came from people they knew you were connected to.

Kalie: Yes.

Beau: So Kalie, I have a half-sister who is 17 years younger than me, and she has two children. And I remember when they were first born, she said, “Please do not post pictures of them online.” And I didn’t. And then one day I went online, and she had posted pictures of them online. And I was like, “Okay, I guess that’s over.” And they’re still minors. One is 10 and one is seven. And so they still don’t have any say over whether or not their images are online, but now Meta knows who they are. You didn’t have any control over your relatives, your excited aunties, posting stuff online with names, I’m guessing.

Kalie: Exactly. They were birth announcements.

Beau: It only takes a few seemingly harmless posts combined with information from relatives, connected accounts, potentially other data brokers for Meta to put together a surprisingly detailed picture of anyone, anyone. The problem is most people don’t really think about how this information is connected.

C Break

Beau: What’s the response to your video? Because, I mean, you got … I called you. Like, what else happened?

Kalie: A lot happened. I’ve spoken to legislators, lawmakers. I’m trying to get the ball moving on how to get, you know, more safety measures in place for kids. But when it comes to just general people online, I have some parents and grandparents who are very thankful for my video, but then I have so much hate and nasty messages and terrible emails that are frankly not just disgusting, but scary. Like, I’m genuinely concerned for my family’s safety.

Beau: Tell me about these messages. I don’t wanna know specifics, but why would someone be mad about what you posted?

Kalie: Basically that I was dumb and how could I not know that that was going to happen, and I mean, put a lot of blaming me for doing this to my kids. And as far as the nasty ones go, it’s just stuff that people want to do to my kids. I started deleting them, and every time I got a DM, I was just deleting. I don’t wanna see that. I don’t wanna look at that at all.

Clip: And yes, I already know what some of you are going to say. “Well, you posted your kids on the internet.” You’re right, I did. That’s why I’m making this video, because yesterday I did not understand posting separate, innocent pieces of our lives over years could mean an AI could sit there and assemble all those breadcrumbs into information about my minor children in seconds. I understand it now, and I’m not okay with it, so I’m done. I’m removing any identifiable pictures and videos of our kids. I’m going to ask friends and any family I haven’t spoken to to please save those precious memories and then remove my kids from your page. If I take everything off my page but you still have my kids on yours, it will go to your page. I’ve seen it. It already did it.

News Nation: People share photos and videos of kids on social media, but what happens when artificial intelligence starts connecting those dots between the posts?

YT Clip 1: This is exactly why you should stop posting your kids online.

Beau: 63,000 reshares and people have sent it to people they know 326,000 times. Okay, so we now know that this is a topic that people care about.

Clip: And then Facebook suggests that I ask Meta AI, “Where does Kalie Robins live?” Are you f- kidding me? So I clicked that, too. It starts digging through completely unrelated st from years of my life. Old posts connecting me to where I used to live, newer posts connecting me to where I live now, and then it says, “Pinpointing Kalie Robins’ location.” It pieced together my information to show people where I live. There was no location on that reel. I didn’t ask Facebook to build a profile of my family. I posted a video of me singing in the car with my kid. I’m not telling you what to do with your kids. Yesterday, I was you. I posted my kids. I’m telling you because if you post your children online, I think you deserve to know that this can happen.

[beat]

Beau: Why Meta put that question there is unclear. Suggested prompts may exist to get people using Meta AI. Could be. They may serve something else, though. Meta is famously super open about what they’re doing. Not. What we know is that a system decided a child in a video was worth asking about. Worth asking about. We have to assume there’s a dossier on that child already. Might have existed before they were born if the parents posted scans of those kids in utero. Meta has every post, every tag, every birth announcement tied to names and accounts. What Kalie saw goes beyond that. It shows how fast nowadays it all comes together. Call it an index, call it whatever you want. Dossier. It’s sitting there, organized, waiting. Waiting for someone to say, “Who is this? What is this? What can I sell them?” One question, and the AI assembles it into a profile of a child for whoever’s asking, and I’m sure they can probably infer what that child wants for Christmas.

Transition

Beau: So what do parents do now? What do you do? What do you do if you know someone who has children? Literally, what do you do if you’re a human being having to interface with this weird behemoth of a country, company, country of a behemoth?

Kalie: I don’t know the answers. I don’t know that once you delete something, it’s gone. I honestly don’t believe that once you delete something, it’s gone. But the advice I have been giving to parents is, you know, we can’t fix what we’ve done, but we can stop what we do moving forward. Don’t post your kids. If you do, which another thing, I am not shaming parents who do. I know a lot of people do it, you know, ’cause they’re excited about their kids or sometimes their family lifestyle is how they make money. There’s a lot of reasons people still post their kids, and my advice to them is make it so ridiculously hard to figure out who they are and where they can be found. Don’t post the back-to-school boards or the soccer game with the scoreboard in the background or team jerseys. Like, just be very aware that what you do post, all those little pieces can be compiled and just given.

Beau: No, can be, will be. And those little pieces are going to be pieced together, which is what’s so disingenuous. I’m gonna go back to my own story from 2015, which in my defense, we didn’t think it was gonna be quite this bad, 11 years ago. Meta knows darn well that those are my kids and that I did not do something untoward. And by the way, I think the post has 10 likes.

Kalie: not a big deal

Beau: it didn’t go viral, right? Now, if they can do that, we have to assume that the reason they’re pulling together these giant files is the reason you would guess first, right? It’s because they wanna sell that data to third parties so that you… So daughter one gets the present that she wants for Christmas or for her birthday, and daughter two gets the present she wants, and that they are both getting pummeled by other shopping opportunities that Meta can monetize. It’s really just that. But the downside of it is do you think about, do you worry at all about what happens if somebody isn’t a good person who works at Meta?

Kalie: A million percent. And it’s not even just someone who’s not good at Meta, it’s anybody that got to see that post and that question. I want to know what did they find out about my kids? And this is another point that I haven’t talked about much. I posted a video yesterday, day before yesterday. I was going back through the screen recording of what I was seeing in real time, and I was going through frame by frame and noticed five other kids that were not my own. I was given information on those kids. And so to me, when Meta says, “Kalie could only see that information. No one else could see it. It’s only the people who have access to it.” Well, I don’t know those five other kids. I’m not friends with them. I don’t follow them. So why was I able to see that?

Beau: When you went through the video again, you noticed that there were not just pictures of your children, but pictures of children you did not know.

Kalie: When I would click on a prompt, like one of them where I got two kids besides my own was, “What are child one’s hobbies?” And then I got a list and pictures of people with the same name, and it mixed up my daughter’s hobbies with their hobbies. So something as small as a hobby gave me two more kids

Beau: What other questions did it ask you? So it asked what are, what hobbies does child one enjoy, daughter one enjoy? What was another question they asked?

Kalie: So honestly, I just kept going through. So some of it was where do they like to camp? What’s their favorite beach? What’s their favorite book? Hobbies, what sports do they play? It just kept populating more and more answers or more and more questions.

Beau: Now, but was it answering them or it was just wanting you to answer the questions?

Kalie: it was giving me answers

Beau: And was it answering correctly?

Kalie: Yes. And this is why I started screen recording because it was giving correct answers, and then I started deleting posts. And when I was deleting posts, the answers got less and less accurate, but still accurate enough that it would be simple to know what you need to know about them.

Where are we now?

Beau: So where are we now? Meta took the question down. Come on. Nobody has said that there’s no more dossier. We know there’s a dossier. The answers are still there, organized in a certain way so that anyone who wants to know anything can find out what it is they wanna know pretty darn quick. Meta told “Futurism” that it missed the mark. I love that, Meta. You missed the mark. Oh, well, that’s what we are doing. It was much worse. And said the feature was fixed. Okay. Okay. Nothing to see here. Keep it moving. Hurry up. So according to “The Verge”, Meta said it would remove suggested prompts about personal topics. Great. I would like them to stop being so creepy in the first place. Anyway, Meta’s position is that Kalie was the only one who could see what she saw. Okay. She the only one who can see what she saw? I don’t buy that either. I bet you there’s people who can buy that information.

C Break 2

Beau: All right, so we’ve been working on this episode for my producer and editor, Andrew Steven, would like me to say a couple of weeks, but the fact is we haven’t really been working on it much for a couple weeks. It’s been sitting around waiting for us to figure out what to do with it, and one of the things that’s been sticking in my craw is what this means, right? ‘Cause you get these stories and you’re like, “Well, what does it mean? What should I do next?” And so Andrew had a very big reaction about what he wanted to do next, so I figured we’d just have him on. Andrew, what’s up?

Andrew: Hey. Yeah, I mean, honestly, this kind of feels like the final straw, and I’m 99% sure in the coming days I’m gonna be deleting my Instagram account.

Beau: What?

Andrew: yeah, it just sort of feels like… I don’t know. I mean, to be fair, I maybe posted five times in the last five years.

Beau: I was gonna say, I think I can count on– if I had lost all of my fingers and toes but I had one toe, I think I could still not be able to count any likes that I’ve ever gotten from you online.

Andrew: I just feel like social media has changed so much, especially the Meta-branded social media. Like, the way I see it is the promise of social media was a way to stay close to friends and family when you can’t be with them in real life. And maybe we knew in the back of our head we were giving away some sort of privacy or information, but it was just sort of like it was worth it. And now you hear more and more of these stories, let alone the actual products. Like When I do log on to my Instagram, you know, mostly what I see are posts from people or companies that I don’t follow, so I’m not even getting the connections with the people. And then you have stories like what happened with Kalie and her daughters and,

Andrew: I think so.

Beau: I wanna talk a little bit about the promise, because the promise of social media was always with one hand behind the back and fingers crossed

Andrew: Sure. But I think I was willing to trade X amount of my data and privacy to connect with friends and family who I can’t see on a regular basis.

Beau: You had a number in your head, so to speak, but with your data. And, how do you feel about your home address and phone number and previous addresses being available on people search sites? What do you think about that?

Andrew: I don’t like it, but the average person isn’t willing to jump through those hoops,

Beau: No, right. So but that’s an ante, that’s like a data ante that you’re willing to pay. You’re like, “It’s a couple pennies, whatever.” And Facebook, in the beginning, that’s kind of what it was. Is that so, that’s what you’re saying?

Andrew: Yeah. I mean, I think I knew I was the product. You know, if something’s free, you’re the product. But especially working on this show, seeing how big the data pool actually is, and now how easy it is to get ahold of.

Beau: Well, the way that I just recently saw a photograph of, somebody had transposed a photograph of the branches of a tree, and then the nerves and veins in a hand, and they were very similar. It’s quite beautiful. And I saw it on Instagram. I saw it on a meta platform. Now the thing is, like, in the beginning with these meta things, what I was providing them with was essentially a stick drawing of myself. Now they wanna know where every nerve and vein is in my body,

Andrew: and a 3D image scan

Beau: yeah. I think in the future, meta’s gonna know you have cancer before you do. And herein lies a problem because, do you really want a private company that is sort of Death Star shaped in its practice to be responsible for your life? I don’t

Andrew: And the thing too, it’s like I know myself, and I might even be able to justify if Meta were the only ones who knew that information about me. But the fact now that it’s leaking out, that it’s getting sent to Kalie, that could be any of us. The fact that, hackers just asked the Meta Help AI bot and reset over 20,000 passwords and took over accounts, there’s been personal data that’s been surfaced on the Discover feed, and it’s not just my data being owned by Facebook. It’s that Facebook is leaking this data to almost anyone who uses the platform.

Beau: Andrew, you sound like a person who’s been doing this show for five years. There’s no undoing that. You cannot go online comfortably with a head full of what the hack, and I’m proud of that.

TFS

Beau: Which brings us to our tinfoil swan, the paranoid takeaway you all need to keep safe on and offline. And here’s this week’s tinfoil swan couldn’t be simpler. If you don’t like what you’ve heard today Let this be the push that gets you to jump off the diving board into the sea or pool, if you don’t like sharks, where there’s no Meta anything. You just have to get off the platforms. It’s the only answer, and that’s WhatsApp, that’s Instagram, that’s Facebook, and that’s, you know… The introduction of AI in these systems has made it dangerous. How about that? I wanna put it as simply as possible. The introduction of AI has turned the stick figure of our data into this 3D model that Andrew was talking about, where every nerve, every vein, every cell is knowable, and they know where it is and what to sell it. And that is maybe a bridge too far for some of us. So if that’s what you heard and that’s what you’re thinking you should do, then don’t let me get in your way. How about that? Now, if you like what you heard or you think what you heard today is important, I want you to just share it with somebody. It’s super loving to send someone information about data privacy in a world where we increasingly don’t have much. So share this episode if you think this matters and you want someone, maybe an overuser of Meta like me, needs to hear it and might wanna get off the platform. Listen, it’s an audacious new world. I wouldn’t say it’s brave at all, but it certainly is mendacious and audacious and you need to be careful, and you need to be your own best friend when it comes to protecting yourself. All right. Stay safe. Have a good week, and we’ll see you soon.

Learn More:

• Read our recent article on Meta Muse and how it violates your privacy.
• Check out what ZDNet has to say about the worst AI agent for user privacy.
• Find out how to remove your personal information from the internet