Trying to Protect Children in Online Video Games Has No Easy Answers
Jenova Chen, known for games like Journey, now spends part of his time working with scared, anxious, and often overzealous parents.
More than ever, children are using video games as a social space. The era of the mall might be over, but hanging out on Roblox is alive and well, and the questions facing parents and game creators are equally thorny and without clear answers.
“What is the best way to actually make the parents feel safe, if your kid is chatting to some other kid online?” said Sky: Children of the Light designer Jenova Chen on a recent episode of on Crossplay’s podcast, Spawnpoint. “And when that person is anonymous, what is the best protection?”
You can listen to the entire podcast conversation here:
Game fans might better know Chen as the designer behind the beautiful and moving games like Journey and Flower. But for several years now, Chen’s focus has been on Sky, a game that adopts some of the themes that his studio, thatgamecompany, explored in Journey, and drops them into an ambitious multiplayer space. Sky is not about winning or losing, but about exploring with strangers and friends and connecting.
One of the game’s taglines is “a place where we shine brighter together.”
“I wanted to make a game that's four quadrant,” said Chen, referring to his hope to make a game that convinces everyone games are art. “This is before [my daughter was born]. I know that in order to really earn respect for the industry, you need to create an emotional experience that is accessible to everybody. That was always a challenge for me. But then once I did have [a] kid, seeing how she engaged with the game, it has definitely given me a lot more thought about ‘What is an ethical design?’”
Ethical design could mean, for example, how the game handles microtransactions. You can earn every cosmetic option in Sky by collecting its in-game currency, candles. You can also skip the line and buy them. They have no impact in-game beyond style.
(Though as Crossplay readers already know, even cosmetics can have consequences!)
“I'm a designer,” he said, “so I'm very sensitive about what they're trying to do to my kids, to myself. People [who] grew up in this society, they will get used to these. A lot of people just assume it's the norm. Pachinko is a norm in Japan. Anywhere you go, there's a gacha machine. So having a loot box seems totally normal. But in the Western world, it is not normal. But then, with so many games made by so many different countries, there's no clear regulations.”
But “ethical design” goes far beyond what you are—or aren’t—charging. Sky is a platform for social interactions, which means there’s a potential for harassment, bullying, and exploitation. Some of this you can anticipate, but even for a smart designer like Chen, much that you can’t, and it often falls into gray areas.
Players are anonymous in Sky and interactions are often fleeting, though you can become “friends.” Early in development, thatgamecompany had a Facebook group for players to discuss their time with the game and provide feedback. But some players started reporting harassment, and because it was a group built on Facebook, a platform that ties together important parts about a person’s identity—such as their name, their face, who they’re friends with—it accidentally became a harassment tool.
The studio moved from Facebook to Discord, the highly popular chat service, which brought its own set of problems.
“We have parents who tell us ‘Hey, my kid is talking to this person I think might be [an] adult and they engage in conversations that's not appropriate,’” he said, “And I would say, ‘Okay, this is pretty serious.’ We look into it. We don't see anything inappropriate in our game, because we have pretty heavy filters. They're just talking on Discord. How am I going to enforce what they say on Discord?”
One time, a parent called Chen upset their child had met in Sky was now following them to Minecraft and other games. The parent called this person a “pervert.”
“I looked into this person,” he said. “He's only been in this game for 40 minutes, [and] not much conversation happened. And I said, ‘Well, how can I trust that this person is indeed a pervert?’ In one of the cases we looked into, that person was a minor, but somehow he's pretending to be an adult. The parents were like, ‘Hey, you got to do something about this kid, the one who was talking to my daughter.’ We actually don't have any evidence to say this is indeed an adult.”
Chen said people who make such accusations are often wrong half the time.
Sky uses automated tools to detect bad behavior. It can detect grooming, where an adult establishes an emotional connection with a child with the goal of sexual abuse.
“I'm a parent, and my chief information officer is a parent, and both our daughters play Sky,” he said. “What is the best way to actually make the parents feel safe, if your kid is chatting to some other kid online? And when that person is anonymous, what is the best protection? Initially, we're using the AI approach, and then there's also the language problem. Because our game is running over 100 countries, there are languages that's currently not covered by chat filters.”
One of the hardest parts about being a parent is understanding your role in their life, and how it changes as they grow older. For so long, children are oblivious to the rules of the world, and your role is obvious. Don’t let them fall off things, don’t let them walk in the street without looking both ways. For a time, you are their whole world.
It’s lovely, but it does not last forever.
“What is the best way to actually make the parents feel safe, if your kid is chatting to some other kid online? And when that person is anonymous, what is the best protection?”
Soon, they begin to develop their own lives. You’re still part of it, of course, but you’re not all of it. They’re going to do things and feel things out of sight, and you have to hope that you’ve given them the tools to deal with it, or if it breaks bad, ask for help.
I do not want to be a helicopter parent. I do not want to snoop on my children. But it’s also my job to keep them safe, and it’s very understandable for parents to wonder if there are tools that might help them accomplish that job in the increasingly dominant digital realm. If the information is just sitting there, let me have it! I’ve profiled companies that will, in effect, provide parents ways of spying on their children.
If you were worried about bullying or grooming, why wouldn’t you be curious?
“I do have one question for you, parent to parent,” said Chen, towards the end of our chat. “Now, when the kid wants to play with me, I know what she's seeing, who she's meeting. There will be the teenager time where they don't want you to know. [Imagine] we provide you, the parent, a backend service where we will log the communication they’ve had with other online persons, which you don't know of, and you can browse through a chat log or activity log. Because these days, I even track how many minutes my kid is playing an app. I will set a limit of how long she can play. But if she's rebellious, if she's in that teenage time, do you want to have that control?”
It would be tempting, but absent legitimate danger, my impulse is to say no.
“My generation, I started playing games when I was older than my daughter,” he said. “If you see my generation as the generation that grew up with games, this is the first time that the second generation is growing up with a parent who grew up with games. You have whole new generational issues. How much should the parents’ generation have access to the children who're doing all kinds of things in the virtual world? I'm just a game developer.”
But he’s not just a game developer, and Chen knows that.
“We have seen all kinds of interesting extreme cases,” said Chen, before bringing up one such example. “You have one out of 100 million person who's really, really aggressive at suicide threat at other people. They're [a] minor, so you can't even take legal action against them. But they are threatening other folks in the community with self harm. It is pretty hard to navigate in a situation like that.”
I’d like to think—or hope—that wouldn’t happen to my child, but I can’t predict the future. Part of Sky’s future, however, might involve a “parent portal” with extra data.
Chen promised to invite me.
Have a story idea? Want to share a tip? Got a funny parenting story? Drop Patrick an email.
Also:
This approach of releasing a podcast, then releasing a written story related to that podcast, is my plan going forward. Please let me know if you have feedback!
I’d turned off Roblox chat on my daughter’s iPad, then goofed and somehow put it on again. For parents who let their kids on Roblox, how do you handle chat? (For the record, typing this out is what’s going to prompt me to turn it off again.)
Absent worry of legitimate danger, I am hoping to avoid invasive parenting. I do not want access to my child’s digital life, unless they want my help with it.
If Chen is so worried about children's wellbeing maybe they shouldn't go for that ridiculous "oh it's just cosmetics" excuse. If that were valid then brand-name clothing companies wouldn't exist. It's long-established that children get excluded and even outright bullied for daring to have parents that cannot afford that junk. Don't get me wrong, I understand and accept that running the servers costs a lot of money. So either charge a higher upfront free, or require a subscription. *DON'T* use shameless exploitation strategies like "oh it's just cosmetics" when everyone on the planet knows the social pressures and results of doing that.
Now, is that less exploitative and damaging than putting unregulated real-money gambling ("lootboxes") into games that are available to minors? Of course. But that just *cannot* be the standard we apply...