Cevat Yerli created Room to blend video conferencing with 3D-animated scenes

Room is rolling out today as the latest collaboration tool combining video conferencing cameras with 3D-animated backgrounds. The social 3D communications engine was created by a team led by former Crytek game developer Cevat Yerli, who was behind the Far Cry, Crysis and Warface series of video games.

The rollout of Room (spelled ROOM by the company) is designed with the new reality of working in mind to make remote teamwork easier. The browser-based, one-click tool has been in open beta for creators, and a general rollout starting now.

Yerli is the latest game developer to try his hand at metaverse-like conferencing solutions. Last week, former VR game devs Alex Schwartz and Cy Wise of Absurd:joy launched their beta version of the collaboration tool Tangle. They were partly inspired by the success of game devs who created Slack and Discord. Both of those companies started out making games, but they both pivoted to make communications tools that have become extremely popular.

Room uses video capture from your webcam to take your live capture and place you in a 3D-animated scene. Your video is literally an overlay in a coffee shop or an auditorium or a theater. But what you’re seeing of the characters is live video.

The tool combines video capture from your webcam to take video of your face and upper body. Then it places you in a 3D-animated scene. It’s actually video of you, cut out and pasted into an animated background. The trick is making the blending appear to be seamless.

I asked for more explanation on how it works, and the company said, “Room has purposefully arranged the camera setting in a way that when sitting at the table and looking around, people are placed with a specific fob on the camera path (which Room calls ‘table view’) so that people naturally integrate with the 3D background and the 2D effect is not visible. You do not have to sit at the table to be captured. You can freely move around. However, Room can only show of people what their camera video is providing in real time. The team is working on solutions for people who are walking around to be shown integrated into the 3D environment as well. This will be part of a future release and is still in R&D stage.”

The opportunity is to produce something that is more joyful for employees to use in the metaverse age than the standard Zoom, Google Meet, Discord or Microsoft Teams. Meta is among the companies using solutions such as VR to take conferencing into the metaverse.

Yerli’s company isn’t pushing Room as a metaverse tool to be used to build virtual worlds. Rather, it is a communications tool that can support the metaverse strategies of other companies. It uses your camera to capture your reactions and converts them into expressions that your avatar can make. It does not require browser extensions.

Yerli will be in New York attending the Clinton Global Initiative and the Future Investment Initiative, where he’ll be debuting the product.

Luxembourg-based Room wants to make it easy to create lifelike or fantastical online meeting spaces using its proprietary social 3D communications engine, RealityOS. It replaces video camera calls with blended video and 3D animations that use lightweight data transfer so the communication happens at 60 frames per second, Yerli said in an interview with GamesBeat. That makes interaction more natural.

As tech giants grapple with arrangements to return to the office, retaining company culture is at an impasse. About 51% of senior leaders are worried that flexible work arrangements will make it difficult to maintain their organizations’ current culture, and the adoption of technology that supports and complements existing company cultures will be critical for future success, according to Mercer.

Research by INSEAD Knowledge found that a further 45% of workers said camaraderie and teamwork had declined since the start of the pandemic. Technology that bridges distance and creates a space for more uplifting and meaningful communication can result in better workdays and outcomes.

As part of the TMRW Foundation, founded by Yerli, Room uses real-time 3D, depth-of-field and real-time reflections that simulate a natural, first-person point of view where participants can freely move around the room and look at each other — just as they would in real life. The technology allows people to represent themselves realistically in 3D spaces and provides users with an experience that closely mirrors human nature, he said. Within the industry, Room features the highest level of social presence, Yerli said.

I tried out a demo of Room and it was pretty snappy. I didn’t see interaction delays like I normally do in video calls. When you look at the people in the room straight on, they look pretty realistic even though they are 3D animated. If you move to the side, you can see the characters are like cardboard cutouts. It would be more realistic if they were fully 3D animated, but this is how Room gets its speed.

Origins

Yerli started thinking about the tech after his twins were born in 2011. For this application, the technology that Crytek created for 3D-animated games would not work. Most of the work has been done in the last three years.

“It requires a completely different architecture and a completely different approach,” he said. “It was all about how I would see the world through the eyes of my children when they grew up. It was about what that next internet would look like. Other people reference the metaverse. For me, it was about information first, then people, social media, and life. I saw it as an internet of life.”

The idea is to share your life and the moments of life.

“The context defines what kind of transaction you do. If it’s a classroom, there is a learning transaction. If it’s shopping, there is a shopping transaction. For that smallest unit of life, contextual moments are shared. I was wondering how I could do this over the internet.”

In contrast to his career in games, Yerli is pushing a mass market technology with Room, not a high-end 3D graphics solution for a limited number of users.

“With Crytek, I was trying to push the envelope of graphics with Nvidia, AMD and Intel. Now we are pushing the boundaries of what you can do within one click away within a few seconds,” Yerli said. “We want you to enter a space and create lifelong experiences. My vision is to upgrade 2D to 3D first before going to VR.”

This approach also uses less processing power and so it’s better for the planet, said Stefanie Palomino, chief product officer of Room. Yerli said he thinks this lightweight approach will work better than selling a new generation of VR hardware to people. Yerli favors a 3D standard dubbed glTF right now.

Right now, about 16 people can participate in a Room meeting. The company is trying to push that upward over time.

“My whole life I have been pushing the high end. One thing I have learned is it is the masses that win the battle. We are always upholding that bar that it has to run on four billion devices,” Yerli said. “If I wanted to run on 20 million devices, then I would run on native apps and have complex avatars. That was my previous way of pushing tech for a limited audience. We want to make sure that we empower as many people as possible to connect from as many locations as possible.”

Key features

Room uses an immersive first-person perspective. It has real-time and AI-supported video presence in 3D, giving users physical depth and dimension. Participants are shown in the same shared space, giving them a realistic sense of presence and togetherness without the need for additional VR equipment.

It also features designer rooms that give shared context to meetings. Users have a wide range of Room options, including a hip New York coworking space, a cozy campfire in the woods, a Mediterranean beachfront, a Hollywood-style talk show set, an ethereal cloud room, and more. Each space has been with the help of interior decorators Claire Davies and Marcin Lubecki.

Room runs on RealityOS, a patent-protected social 3D communications platform. It is built with a new, browser-first 3D game engine core. The aim is to reduce video call fatigue and let users interact with objects and their peers in a natural and learned way.

It also has privacy protection, as users enter Room with the visual surroundings of their homes in the background automatically removed. All meetings are end-to-end encrypted. It is browser-ready and device-agnostic so users can access the platform by clicking a link within a web browser on any computer with a microphone and camera. No additional software downloads or VR headsets are required.

“We believe that digital spaces should be shaped by real life and with the presence of actual people, not anonymous avatars,” said Yerli. “Human nature is programmed to interact with others in a certain way. We close and deepen relationships face-to-face in shared spaces most of the time.”

He added, “When we meet online, we should do it in one room and with our actual selves. We know how to bring people together in digital spaces. We think that the concept of Room will profoundly change the way people get together in the next phase of the internet. Not as a replacement for real life — but as the second-best option. We call it the Internet of Life.”

Palomino said that creating the right culture and values remains important in the age of remote work.

“It is clear from the research that while remote working offers huge emotional and financial benefits to both organizations and individuals, without the right technology in place, there remains a significant risk to our culture of togetherness,” Palomino said. “Due to the recent and rapid virtualization of corporate interactions, we have pushed ourselves into these two-dimensional, flat spaces. Room has been designed to encourage people to interact in uplifting virtual 3D meetings, and ultimately support and enhance unique cultural norms.”

The Founders Plan features access to all rooms including the exclusive Founder Room, lifetime founder status and price, early access to upcoming features, unlimited meeting time and more.

I wrote one of the first stories ever on Yerli and his brothers as they launched Far Cry. Yerli served as CEO of Crytek from 1999 to 2018. While the company thrived on creating memorable games with high-end 3D graphics — Far Cry, Crysis, Warface, and more — it had trouble keeping up with the financial requirements and development demands of modern gaming.

He founded the TMRW Foundation in 2017 with the mission to create, acquire, and propel ideas, digital technology, and applications of the next iteration of the internet that have a positive impact on society and the way we define presence, interact as people, and generate profits. It has a portfolio of 3D simulations, VR, AR, and AI-powered products.

Palomino was the cofounder of Red Lab, a boutique consultancy started in 2015 to support clients in interactive, smart digital events and communications. The largely self-funded company started in 2017 and it has about 100 people. It is unrelated to Crytek.

The memory palace

Each Room is set up like a memory palace, a visualization technique that goes back to the ancient Romans and Greeks. With it, you create 3D spaces around the cool moments that happen in life. The 2020 video game Twin Mirror used this technique to solve mysteries.

Yerli believes the memory palace approach of Room is unique among the plentiful competition.

“The technology for avatars is pretty far away from being real, inclusive, accessible, and scalable,” he said. “So we came up with a technique based on AI inferencing. We can provide a presence or social presence quality that is unseen before.”

Yerli thinks this technique is more humanizing while enabling it to run in a browser that is only a click away.

“For me, the vision of the next internet is not so much about VR, it’s not so much about avatars, because often VR has a threshold, a technical barrier. And gamers don’t approve yet. The barrier is there, and the scale is not there. Our philosophy has been how can we build a foundation that can be operated by or clicked on and entered by four billion people. It has to be running on every browser, on any device. It has to be the most inclusive, the lowest lightweight engine.”

Accessibility doesn’t mean it has to sacrifice quality, as Room adds reflection shadows and AI inferencing to make it more realistic.

“What we’re trying to do is humanize technology,” Yerli said.

Originally appeared on: TheSpuzz