How 1.6M people watched a concert in Sky: Children of the Light | Jenova Chen interview

Earlier this year, Thatgamecompany announced it had more than 160 million people who downloaded Sky: Children of the Light, since the mobile game launched in 2019. And now the company has figured out a way to keep those players coming back through a virtual concert with the artist Aurora.

The title was an open world social indie adventure game, where players could have a cool experience in a beautiful world and share the emotional experience with friends. That enabled the company to raise $160 million in March 2022. During The Game Awards on December 8, the company showed off something new.

After more than a year of work, the company staged a virtual concert inside the game with the musical artist Aurora. She sang a series of beautiful songs and the game took the players on a kind of spiritual animated journey. More than 1.6 million people watched that concert, which is replaying every day for a month, said Jenova Chen, founder and CEO of Thatgamecompany, in an interview with GamesBeat.

Each instance of the concert has 4,000 players in it, all moving individually with the crowd. That’s far more people than online games can usually pack into a server space, and Chen said it took a lot of technical work to do. About eight engineers worked for less than a year to make it happen, mainly by springing from the tech created for an experience where they were able to put 7,000 fish in an aquarium in an earlier game.

Huge crowds of players poured into the game as it was announced at The Game Awards. And Chen said the virtual concert was a success. It also made him think about how to build multiplayer metaverse experiences in the future. The concert will continue to play in Sky until January 1.

Here’s an edited transcript of our interview.

GamesBeat: It looks like the concert was a great success for you.

Chen: We’ve worked on it for so long. Seeing the players say how much they loved it and how many different times they went to the same concert, it makes us feel like it was worth it. Seeing streamers play through the concert and seeing their faces was really the best reward for us, going into the holidays.

Our whole engineering team, they feel like they landed on the moon. Everyone was expecting something to go down. But we survived.

GamesBeat: What kind of numbers did you hit in terms of downloads Sky has had over the years?

Chen: Last time we announced it was about 160 million. To be honest, I haven’t checked recently. It’s probably gone up a bit. But I don’t have a new number at hand.

GamesBeat: How did the concert idea come about?

Chen: It was really after I saw the Fortnite concert. It was an impressive technological wonder, seeing that show. When they said they had 15 million people watching together, though, it didn’t really feel that way. In a game of Fortnite, you have maybe 40 people at most in the room. It didn’t really feel like a real concert. I’ve been to some real concerts, like seeing Taylor Swift at the Rose Bowl. It’s a different type of experience.

At thatgamecompany we always want to build positive emotional experiences between players. I felt like Fortnite didn’t really capture the connection between everyone going to the concert, these people who all love the same thing. There’s a sense that you’re part of something bigger. You’re not alone. I could imagine a better virtual concert, so then I thought, “Well, can I actually build that?” At the time it seemed like virtual concerts were the new thing for the future. At TGC we’ve always been trying to innovate, and sometimes subvert the emotional responses of games in certain genres. I wanted to build a concert where I could feel connected with tens of thousands of people.

The first thing I did was talk to our CTO. “Hey, can we actually do 20,000 or 30,000 people in a multiplayer co-op session?” Typically people would tell me that was impossible. But he actually spent some time thinking about it. We worked together to figure out that this was probably doable. Even though we only filled 4,000 people per stadium in this concert, we can pass about 40,000 people back and forth. It’s still doable. It’s kind of mind-boggling to put thousands of people in the same game room. But someone was crazy enough to ask, and someone was capable of making that happen. Now it’s really happening.

GamesBeat: I’d guess a lot of people have had the idea. I’m sure Epic had the idea. But nobody had the capability. That’s the surprising thing you were able to figure out here. You mentioned Improbable, but you have different technology here.

Chen: I think our advantage is that we’re vertically integrated. A lot of the time–if you were to say, “Make Fortnite with 4,000 people,” that’s just physically impossible. The level of data in that application, you just can’t do that. But with our company, we’re emotionally first. If we can put 4,000 people on the screen, we can compromise on the desired experience. We don’t need to show every single person, every single animation and every single bit of IK. We can sacrifice that. We don’t need to show every person as a full high-resolution, high-detail model. We can compromise on that.

GamesBeat: Do you have a technical explanation of how it worked?

Sam McGrath (CTO of Thatgamecompany): Up until the concert, the most players we were able to support in Sky was eight. So thinking about how we could increase that by almost three orders of magnitude was daunting, to say the least. The two main technical challenges were first, how to render that many players in the same space so that it could work across a wide range of hardware (Sky can be played on some pretty low-end devices, and we didn’t want to exclude them from the concert) and second, how to send all that information over the network efficiently so that it would work from anywhere in the world, even on a subpar internet connection.

One thing we really didn’t want to do was “cheat” by creating fake players in the Colosseum and other areas. While we did have to make some concessions, like using a simpler representation of the players than would be seen in a normal eight-player game, and creating a simpler emoji based interaction menu (vs our normal collection of hundreds of emotes), the other players you see in the audience during the concert are absolutely real! You can move and interact with the other 3,999 people in your server and enjoy an amazing shared experience.

In the end we were able to get this tech working in less than a year, with around eight engineers (including me) directly contributing to various aspects of it off and on, and our great backend team working to make sure our servers didn’t crash for what ended up being our largest ever event in Sky (over 1.6 million concurrent players). While there were about eight engineers responsible for implementing the concert tech, we also of course had many other talented artists, animators and designers who brought the whole thing to life! I’m super proud of our team for what we achieved from both a technical and artistic point of view, and the reactions players have had to the concert is incredibly heartwarming. I don’t think it’s hyperbole to say we achieved something that others could have easily written off as impossible. I’m really looking forward to pushing this system further next year; I feel we’re exploring a new frontier of game design with this type of massive shared experience, so it’s pretty exciting.

GamesBeat: You’re not showing faces on everyone.

Chen: Right. We just kept compromising on everything, but preserving the emotional goal, until we got to a point where we could abstract the remote players, the ones outside the observer, as these abstract characters. From that point on it becomes a problem we can solve. When you only need to track people by their position, their orientation, and their basic expressions, the data is contracted into something small enough that we could compress and send it over the network.

If we were to start with the scale of an actual game, something that might be 10 times that, that would be very difficult. It’s really the combination of, we have very clever technical mechanisms, and the design side is willing to work with the limitations of the technology. Then we get this magic happening on the screen. The magic is stretching technology to make something that you think is impossible possible. But it requires a lot of hacks and shortcuts.

GamesBeat: For the people who were experiencing the concerts, were they essentially able to identify themselves, where they were inside the concert and what their unique persona looks like compared to everyone else?

Chen: Yeah, let me–one thing we did is–I’m going to load up a video so you can see this. The color of every single cape–you can see that we’re passing high-definition data on this level here. There are eight players, and you can see every single person’s joints and animation. But the people behind us, you can only see the color of their cape and their height. We pass their height, the type of cape they wear, the color. But that’s all the data we’re passing.

GamesBeat: So you can interact with about eight people in your room?

Chen: But if you see here, if he walks somewhere else, there’s a button where you can transport to their room, join their room. You can see this person Jeremy here. He’s an abstract character, but because you’re his friend, you can tell that he’s your friend, even though he’s in the lower resolution. In order to see him in higher resolution, you can join his room. You can see that Jenova in the background is in the same room and you can see him with full detail. It’s two levels of detail, but you can still see someone’s name if they’re your friend. Technically, if you have 4,000 friends and they’re all sitting in the same stadium, you can see every single person’s name popping up.

GamesBeat: How is some of this different from what the Fortnite experience was like?

Chen: The Fortnite concert, each of the game rooms could only have up to 40 players watching the concert. The other 39 players would look like they do here. Just like the people sitting down in this row. But they wouldn’t have any of the audience in the background filling up the entire stadium. Also, in this case, each person in the audience here can control their own manta ray. You have about 3,900 manta rays being controlled by players in real time. Fortnite never had this level of co-op. They had up to 40. But you can see all these people here are making real choices.

The climax we have–here it is. Here are 3,000 people being drawn in and thrown out from a big explosion. All the players are flying out. This level of concurrence and realtime data, I don’t think any game has done that before. The biggest online battle in EVE Online, they were simulating about 6,000 spaceships shooting it out with each other, but their server was running so slow that they were basically updating one tick every five seconds. We have 10 updates per second with realtime data on 4,000 people, all moving and making decisions at the same time.

GamesBeat: How many sessions of 4,000 people did you have going at the same time?

Chen: The concurrence we had was 1.6 million. Divide that by 4,000, it’s 400 stadiums running at the same time. If we can increase the bandwidth of a stadium by 10 times, then we only need 40 of them. But that’s the kind of madness the engineering team was dealing with. It’s quite funny if you’re actually playing it. There’s a total sense of joy.

GamesBeat: You had that technology coming together. How long did it take to do that, and how long did it take to plan the content side at the same time?

Chen: It took us a very long time. We couldn’t have built this in just one quarter. We actually planned for about five quarters to do this. We had to make seasonal content at the same time we were building up the technology. This is very new. I don’t know if anyone will be developing their games like this. We have to keep the game live. But the first thing we did, I told the team, “I want this on the screen. I want 30,000 people with glow sticks.” First we worked on the data part. This was last year. We tried to merge every single game server that was running on the internet, regardless of which country they were in. We merged everybody’s traffic. You can see that even in the tutorial, there’s more than 1,000 people traveling. This data pipeline is a big investment in terms of technology. We allow realtime visualization of every single person in every single server.

With this data, the second thing we invested in was changing the rendering engine so we could render massive numbers of instanced individuals. In this case we made it underwater. We don’t have to render 7,000 fish, but we wanted to prepare for the concert, so we started testing by rendering this on a mobile phone, on Android, these 7,000 fish. Usually you’d do this on a console, but we wanted to see if we could do it on a phone. This is actually highly performant on a phone. That was a big push, to optimize the graphics rendering. This is also on the Switch.

From there we worked on having 5,000 players gathered in the home space. It’s very crowded. Then we started to work on the representation of the players. If another player gets really close to you, you don’t want to feel claustrophobic. We render the player into a tiny ember. Eventually you get to what you see in the game now.

Then we started to stream the data into the real game. This is our internal build, where we visualize the realtime data of every single person who’s playing on a server. We spent the next season working on the representation of this data. We started to do concepts and finding ways to render them in a way so that they look different, but they’re using the same instance. We designed the venue. Eventually we had this on the screen after about four seasons of work. That’s just to prepare the technology. For the creatives, we started to work with our singer, Aurora. She gave us a lot of input on making this AI-driven dialogue and managing the emotes coming from here. She also wrote the narrative behind each of the songs.

We didn’t want to make a concert that was just playing an artist’s songs. We wanted the songs to form a story, and we wanted the story to touch people, to make them cry in a good way. We started to work on a narrative with the artist. We designed the emotional arc in the same way we designed Journey. We designed the metaphor of the concert to be a metaphor about the tension between humanity and nature, the push and pull between nature and people.

When we worked with the artist, she would give us the story that she has in mind when she’s making the songs. We wanted to bring her soul onto the stage. A lot of the time I see these rather commercial virtual concerts, and I always feel like the character looks soulless. I don’t really feel like they’re on the stage. In order to make the artist feel like she’s present, I asked her to give us her idea of what the players would do in the stadium. I asked Aurora to give us the narrative that she imagined would happen alongside each of the song. It would have her DNA. Then I would weave these stories together into an emotional arc. We used to worship nature, and we’re the masters of nature now. That’s the narrative. Then we play it out through all of these audience interactions.

To me, this is an interactive concert. The audience shouldn’t just sit there and watch. They need to participate in the musical journey. The orange part here is the audience. They represent different aspects of nature throughout the concert. In the end, they become part of nature with Aurora, who represents a goddess of nature. It was a long process to creatively find where we are today. It’s not just a technical challenge. It’s a powerful piece. Emotionally, we were aiming for a high bar.

I can show you the climax here, where everybody dies and is brought back to life. They’re like butterflies. This is 3,000 butterflies, and the sun rises, which brings back Aurora, who represents Mother Nature. A lot of people were crying. You can see the emotes from the players. Then we have this giant version of Aurora. Instead of just jumping around, she reaches out and touches you and holds you, like your mother. A lot of the players, when they played this part of the concert, they were crying. It reminds them of their mothers, the love they have.

GamesBeat: It’s very emotionally constructed. You have real storytelling going on in the concert.

Chen: It’s definitely a therapeutic experience. It’s a concert with a lot of feeling. It wouldn’t be an overstatement–when we were trying to make this concert, we really aimed high. We wanted to see if we could push the envelope, to bring the interactive concert to the next level. Not just have a great show, but have the next level of interactivity between the people in the audience. The next level of emotions. We wanted to touch people and connect them together.

After the concert, we had a lot of people just lingering. They didn’t want to leave. This video is after the concert was over. People just hang around, because they’re so shocked by the experience. They want to prolong it. About two hours after the concert was over, the stadium still had about 800 people in it, trying to get to know each other. There’s a strong emotion, a shared memory among the people in the audience.

GamesBeat: How did you interact with Aurora? How did you find out that she was the artist you wanted to work with for this?

Chen: We want to make Sky a theme park, a brand that’s family-friendly. We didn’t want to necessarily pick a controversial artist. We wanted someone whose music is wholesome and positive. Aurora was the singer who provided the voice at the original climax of Sky. We had already gotten to know her back then. But she’d also become very popular on Instagram and Tiktok lately. She had a song that drew half a billion views, “Runaway.” She’s become much more popular in the past couple of years.

It made sense that we would work with her again, because all the songs she sings have very positive messages: protecting nature, fighting for human rights. She’s not the type of singer who just sings about sex or money. She sings about spirituality, about a deeper love. Her songs give me goosebumps when I hear her. There’s a connection to nature, is how most people describe her voice.

GamesBeat: Were you thinking that this might be close to a metaverse experience?

Chen: Oh, yeah, definitely. Before this, I wouldn’t say I believed in the metaverse, because there’s so much speculation. People say it’s just a fad, right? But after we experienced this concert–even myself, I’ve never played a game with 4,000 people at once. I feel like it’s more like a real society. I don’t know how to describe it. If you go to a concert, as an introverted person you feel exhausted afterward. It’s just too much going on, too many people. I feel the same way going to a real concert as I do attending this virtual concert. The emotion I felt is so real.

I can see, in the future, when games have the capacity to host more people–if we can do a concert like this, why don’t we do a convention? Something like CES could happen in a metaverse with tens of thousands of people. After this concert I’ve started to believe that’s totally possible, creating something similar to a real-world emotional experience. The other thing about this concert is, a lot of the people who came to see it had never played Sky. And they don’t need to. They just join the concert, walk around, and participate. There’s zero barrier to entry.

I think the metaverse has to be a zero-entry-barrier experience. It can’t require you to learn how to play a game – how to control a camera, how to jump, how to fly. That’s going to keep a lot of people from entering it. But with this concert, I was able to invite my four-year-old daughter and my parents. They could just drop in and all have a good time. That’s something new that I’ve never experienced, or even expected.

GamesBeat: It seems like a great idea to announce this at the Game Awards. You got in front of all those people, and then they all just joined in. It reminded me of the time Bethesda launched Fallout Shelter at E3. “It’s available now! You can go play it now.” They got something like 12 million people playing in a day.

Chen: We definitely saw a media spike from doing the show. At the same time, I’d say that the Game Awards crowd, the Game Awards players, they’re there for a very different type of game. The game before us and the game after us, they all had pretty strong gore on the screen. It makes me feel like–if we don’t fit in there, where do we belong? We’re a game, but we’re about art. We’re about emotion and wholesomeness. It doesn’t feel like there’s a venue for it, at least in the hardcore gaming media.

GamesBeat: Unless you were jumping off someone else’s concert, there aren’t that many places where you can find so many people that are interested in this kind of game, that are all gathering already for some other reason. But still, 1.6 million people is pretty good. You’re doing this over almost a whole month, right?

Chen: Yeah, I’d love for more people to try it out. The concert is 45 minutes, and it will be over after January 1. Personally, I feel sad that the concert is going to be over. It’s a year and a half of work, and it’ll be gone after the new year. If we can get more people to be aware of this experience, I hope they can give it a try. I’m hopeful for the possibilities of where games and the metaverse can go.

GamesBeat: Do you see an opportunity for this to help the game and help the company monetize? Or do you see this more as a technology demonstration?

Chen: We made the whole experience free. Technically this isn’t really a product, because we’re not selling it. What we want people to discover is that Sky as a game is a beautiful and wholesome community. If the concert can spur interest after they experience it – “Hey, what is this game?” – that’s really our focus, to reach out to a new audience that might have never known about our game. We just launched on the PlayStation, so we want more people to discover that this game exists.

GamesBeat: In that sense, then, if you had an evil monetization person, they’d call this user acquisition?

Chen: Sure. But it’s definitely not worth the money. If you think about it, working for a year and a half on this experience. It was a very expensive investment.

GamesBeat: What kind of opportunities do you see ahead? What do you think you can put on the road map now that you’ve done this successfully?

Chen: For me, I really haven’t experienced any game with more than 1,000 people in the same location. The feeling I had was so different that I’ve been inspired to push this envelope. Our hope is that in the next year, we’ll take this format and turn that into a genre. I don’t know what to call it right now, but hopefully we’ll give it a new name. It’s very different from an MMO game when it comes to interacting with people in a massive society.

I can only say–the metaverse is an interesting inspiration, because what do people do in real life when you have more than 10,000 people together? There are only a few things that people do together in such massive numbers. We’d like to use the technology we’ve built and keep exploring what other things we can do with tens of thousands of people together. Hopefully we’ll be on the frontier of pushing massive online games for a while.

Originally appeared on: TheSpuzz