MeetKai rolls out creator tools for conversational AI and the metaverse

MeetKai has rolled out new creator tools as part of its goal to create a portfolio of metaverse and conversational AI technologies.

The Los Angeles-based company has more than 50 million global users thanks mainly to its conversational AI technology. Now MeetKai is heading off into the metaverse and showing off its tools at the CES 2023 tech trade show event in Las Vegas this week.

The company unveiled MeetKai Reality, MeetKai Metaverse Editor and MeetKai Cloud AI to help people create seamless metaverse experiences.

Rather than shoot for the extreme high-end of 3D graphics, MeetKai is targeting experiences that anyone can access via a web browser, without the need for expensive specialized hardware, CEO James Kaplan said in an interview with GamesBeat.

The company is headed by AI expert Kaplan and billionaire investor and semiconductor veteran Weili Dai, who is executive chairwoman.

Last year, MeetKai’s team went to New York to unveil the a metaverse demo, which you could access on your phone via a QR code on a billboard in Times Square. When you tapped on it with your smartphone camera app, the browser app generated an animated version of Times Square where you could roam about with your avatar. You could control where your avatar went on the phone.

Focusing on realism

Based on feedback from that demo, MeetKai started working on the tools it announced today, Dai said in an interview with GamesBeat.

“If you want to make the metaverse, it’s not like making a game,” said Dai. “If we try to build it all ourselves, I think we’ll fail just like Meta does. It’s a matter of building a ton of tools. It’s like you see a lot of high-quality websites. Some use a template and they have a high-quality website that works on mobile and the desktop at very low costs. It’s very efficient.”

One of the lessons was to focus on realism. As the buzz around the metaverse grew, Kaplan and his team disagreed with the movement away from the real world to the virtual world. He felt that blending the real world and the metaverse was the right move.

“We’ve been working on a lot since we did our Times Square trial run,” said Kaplan. “We learned a lot about what works, we learned a lot about what doesn’t work. And one of the things that we really took away from it was that interiors matter a lot. A lot of these metaverse experiences look blocky. So there’s a huge opportunity if you can make things that actually look good. And that’s what we’ve really set off to do.”

Creating custom projects

Kaplan said the team has made a lot of progress with visual fidelity for shadows and lighting. The physical-based rendering can capture details that look much better. That helped get the company a partnership with the Los Angeles Chargers last month.

“What we’re doing there is building out a Chargers metaverse experience around a store,” said Kaplan. “And then we’re potentially scanning in a number of key locations, like the locker room, the training facility, with the idea being they want to really provide a way for fans to interact with the brand. And doing it in a way that doesn’t look like a video game.”

The company has been doing a variety of custom projects like this for high-value brands.

“We do the work for them, basically going from the concept that they have in mind to something that looks good and have it work in the browser and be integrated,” said Kaplan. “This one is primarily being leveraged for activations we’ve been looking at doing for them. Companies love doing activations for festivals and special events. We have the capability to build these out extraordinarily rapidly.”

The company can build a store in a few days that looks identical to a store in real life, he said, and the imagery works in mobile, VR, desktops and more.

“The visual fidelity approaches much more what people would expect from a high-end production while still running in the browser without pixel streaming,” said Kaplan. “Most of these things that you see in the browser use pixel streaming, but it costs way too much money.”

MeetKai Reality

Besides the Chargers, the company has signed a deal with a large agricultural company where it is doing more custom work. At CES, MeetKai is doing demos where it can take a video of a room and convert it into a space where you can walk around in 3D within minutes. The quality of the rendering gets better with time, but you can publish something very quickly.

That tech is dubbed MeetKai Reality.

MeetKai Reality is a new software platform that supports users in digitizing physical spaces to create instant 3D replicas using nothing but a phone. The tech can be used to bring objects and spaces from the real world into the metaverse without the need for complicated scanning equipment or countless hours of custom 3-D modeling.

Kaplan believes that this could be useful across professions, and particularly for realtors, interior designers, architects, engineers and retailers. All users will need is a smartphone or any internet-connected device with a camera to record seconds of video and render their space.

Showing a demo of the Chargers space, Kaplan said, “This is entirely automated, like there’s no human that did anything here. It’s just an iPhone video that was taken. There’s no custom processing.”

Limitless potential

Sports companies have talked to MeetKai about enabling ecommerce stores because their physical stores can’t stock enough different types of products due to space limitations. Others want to scan the real world into a virtual model to train or plan. And sometimes training takes place in areas where a low-end smartphone is the most practical computing device.

“This concept of the browser is really about running this on any platform we want,” said Kaplan. “That’s why we did the Times Square test.”

And you shouldn’t have to build out a 3D space for hundreds of thousands of dollars just to get your stake in the metaverse. Kaplan said the company’s expertise in AI has paid off in terms of creating its tools for the metaverse. If you take a scan of a space, you need the AI to understand the details in that space.

Showing off a sneaker spinning in the air, he said, “We’re getting pretty good at dealing with shoes. There’s a lot of excitement around AI these days because of ChatGPT and Stable Diffusion. And we have a pretty interesting thing we can showcase too. There is more appetite in the consumer space for fun things you can create with AI.”

MeetKai Metaverse Editor

MeetKai Metaverse Editor will help usher in a new era of metaverse content creation. Unlike other virtual design platforms that struggle with object physics, clipping, or user-friendliness, Kaplan said MeetKai’s new software allows for easy virtual construction and space building without complicated tooling.

MeetKai’s platform also allows anyone to build their own metaverse content while working collaboratively in real time, making both personal and corporate metaverse projects more manageable.

On top of these advances, MeetKai continues to use its leading edge software to push the boundaries of conversational AI. The startup is developing its own large language model (LLM) to challenge current industry leaders and offer specialized enterprise use cases.

“I’m extremely proud of our team’s invention and innovation to be the first in the world to deploy such advanced tech with MeetKai Reality, MeetKai Cloud AI, and Meetkai Metaverse Editor,” said Dai. “This makes our reality-based vision for the metaverse attainable and practical from the start. As we announced early in 2022, we’re bringing the real world into the metaverse with super fast speed and effectiveness, for multiple industries such as real estate, retail, sports, architecture and entertainment.”

MeetKai Cloud AI

For now, MeetKai’s Cloud AI will add another level of immersion to virtual spaces by facilitating AI-human
interactions with the first cloud offering to enable domain-specific, knowledge-driven virtual humans.

Instead of just chitchat or fixed knowledge from the past, MeetKai Virtual Humans can leverage any form of multimedia to instantly interact with end users, at a fraction of the cost and 10 times the speed, using industry-first, real-time reasoning capabilities, the company said.

“As a company we have always believed that AI-first products drive differentiation; the metaverse is no different,” said Kaplan. “Our intent in building out our metaverse tools is to enable ourselves to build state of the art experiences in a fraction of the time with a fraction of the cost. We have a number of first-party experiences we are creating that weave together Reality, Editor, and Cloud AI to power unique user
functionality. An example of this is a virtual store that could feature an NPC capable of searching for and showing you scanned in objects.”

He said MeetKai’s metaverse tools can be used a la carte as well.

“We want to unlock the same cost and time savings for everyone else looking to build in the metaverse,” said Kaplan. “So if a brand already has a metaverse platform but wants to integrate MeetKai’s AI, they can do that. Likewise, if they have their own NPC tech, they could leverage MeetKai’s space building and rendering capabilities.”

A company that can scale?

The company has raised more than $20 million to date and has 50 employees. It might seem like a small effort, as there are more than 500 companies going after the metaverse, which McKinsey believes could be worth $5 trillion by 2030. But Dai can access a lot of resources.

She became wealthy as the cofounder of Marvell Technology Group, which became a major semiconductor company. She was named by Newsweek as one of the 150 Women Who Shake the World, but she also had some hard times. She and her husband Sehat Sutardja were fired from Marvell over an accounting scandal at Marvell 7 years ago. No outright fraud was uncovered, but regulators said there was constant sales pressure to hit targets.

Dai has since bounced back as a prolific investor. She believes her company will have the resources to scale on a global level, and the opportunity reminds her of what she and her husband did with Marvell.

“Now it’s just a matter of execution,” she said. “We always appeal to the masses using technology. It’s not only leading technology but very affordable. It can scale and go to the masses. With MeetKai, our strategy is no different than our semiconductor side of leading technology.”

Is the metaverse going to be real?

Dai has a lot of things in the works and she’s trying to stitch it all together.

As for the weak economy and the declining buzz around the metaverse, Kaplan said, “I think the problem last year was that crypto and metaverse got so intertwined. And then people saw the great train of crypto as a way to get around actually building things. Crypto is one of those weird things where it was possible to make money without actually building a product. It’s seems like a scam, right? It’s like a classic boiler room. But the metaverse has existed since Roblox and it has been around for a long time. And it’s only the crypto stuff that’s really in the downturn.”

He added, “The companies we’ve talked to have tremendous appetite for the metaverse as defined by 3D immersive spaces that are social, that people can do things together in, that have different brand activations. There’s less appetite for just selling NFTs and calling it the metaverse. And I think that’s actually very good for us.”

Will the metaverse take years to build out?

Kaplan looks at Minecraft, which has been radically successful among kids as they build it together and becomes a large world.

“To me, the problem of how we’re going to get to that metaverse is going to involve people building together, instead of one company just developing this gargantuan world with like everything in it,” said Kaplan. “MMOs have existed for a while. And there’s a reason why. To me, Minecraft has such a different appeal than World of Warcraft. Because in Minecraft you’re building this larger world.”

The key is just to make it easy to build things that are more realistic or have wider variety of artistry than something like Roblox, he said.

“Creators and builders can come to our platform, and we will have tools and all these people will be able to leverage this and create a world at same time with us,” said Dai. “And this is so affordable. That’s one of my passions, to enable everybody globally in parallel.”

Originally appeared on: TheSpuzz