Read
Announcer:
Today on Building the Open Metaverse.
Lincoln Wallen:
It puts a particular sort of perspective on when you say, well, what technologies don’t we have yet? You start to think a little bit about how a consumer expresses themselves. How do you get that? How do you get virtual production and the qualities of capture, whether facial capture or gesture capture into the metaverse so that even though you have thousands of people, when you turn to the person next to you, when you hear them coming within your sound bubble, so to speak, you can actually express yourself well. So I think those sorts of areas are going to become increasingly important for making these interactions meaningful.
Announcer:
Welcome to Building the Open Metaverse, where technology experts discuss how the community is building the open metaverse together, hosted by Patrick Cozzi from Cesium and Marc Petit from Epic Games.
Marc Petit:
So hello everybody. And welcome to our show. Building the Open Metaverse, the podcast where technologists share insight on how the community is building the metaverse together. Hello, my name is Marc Petit from Epic Games and my cohost is Patrick Cozzi from Cesium. Hello, Patrick. How are you today?
Patrick Cozzi:
Hi Marc. Hey everybody, doing great.
Marc Petit:
So we are kicking off our new season with a guest that has deep expertise in a number of topics. So expect us to cover quite a bit of ground for the next 45 minutes. It’s my extreme pleasure to welcome Lincoln Wallen, the CTO of Improbable to the show. Lincoln, welcome to the show.
Lincoln Wallen:
Thanks Marc. Great to speak to you. Thanks Patrick. Great to meet you.
Marc Petit:
So please, share with us your path with the metaverse. I mean, you had such a long and diverse path, so please take the time to run us through that.
Lincoln Wallen:
Yeah, so I, at least at the elevated age that I’ve reached today, I sort of break my career into two parts. So the hidden part, I guess, is I spent 20 years as an academic since starting off in the sort of physics and maths area, moved into AI and computing foundations and then spent 10 years as a professor at Oxford teaching mathematics and computing. But increasingly because of the context that I was in, working more and more with industry, more and more with sort of novel uses of computing technology and all sorts of places like control rooms, cockpits, dealing rooms and so on. So it very much at that sort of interaction between technology and people, what people are trying to do and how to even talk about that and the impact of technologies.
And that was really exciting and interesting because a lot of this is hard to study. But anyway, that’s what got me interested in the so-called real world. And so I stepped out of the academic world into a startup at the time called Math Engine that was aiming to build package libraries, to deliver of all things simulation and physics, which of course I knew a lot about computationally and from a physics point of view seemed easy when as an idea, incredibly hard. But that was a fun period building what was the early stages of the middleware market for video games, the use of middleware in web experiences. And of course the uptick of simulation in training and sort of government and agency type uses. So we address all those markets, which very interestingly is sort of a mirror of Improbable today hasn’t gone away. Simulation, digital twins, it’s still what the industry’s been aiming for, for 30, almost 40 years.
Long story, working with companies like Criterion saw the power of middleware, I think at its highlight Criterion software, which delivered the RenderWare suite of tools had about 30% of all consult titles built off of that graphics engine and sorted tools, which just illustrated how the industry could operate. And then of course, everything fell apart. Criterion was acquired by Electronic Arts. So I got to see the inside of a large scale publisher for real, and also the challenges of actually scaling content delivery, especially within the games industry, which is an unsolved problem. I would say interesting to talk about that as we talk about content delivery into the metaverse.
So I became the CTO for Electronic Arts Mobile, and really through the math engine days and up through the early days of EA Mobile. It was about how to disrupt the industry itself because we looked at Asia, we looked at Japan in particular from the late 90s, the mobile market was already a billion dollar market. Consumers had already moved to a more casual form of interaction, serviced free to play, micro transactions, all supported by platforms like IMO and so on. And getting that across to Europe or to the US where the mobile market was still in its infancy and consumer patents hadn’t really changed was actually quite difficult.
And I remember going to an EP of one of the major franchises. I won’t mention it now, it’s still around with a Nokia Series 40 phone with a little black and white game on it and saying, this is the future of gaming. He looked at it, it’s like the crocodile Dundee scene. It’s like, “That’s not a game. This is a game.” So the dissonance in terms of the commercial and consumer engagement versus the settled industry motion of now, tens of millions of dollars into console and PC product, we’re seeing that again with the metaverse I think. So, that’d be also something interesting to talk about.
Anyway, long story short, we built EA Mobile to about 200 million ARR in about four years, acquired a leading mobile publisher, and then tried to revolutionize EA from the inside. It didn’t work. Long story. I left at a change of leadership and joined Dreamworks Animation. So went in there as a CTO. And I would say that still represents the one and only media producing company that actually focused on the idea of the delivery of content at scale, because of the pressures of it being a public company, of monetization of movies being a settled market.
So if you’re going to scale as a public company has to, you have to scale through increased production. So it felt very much like a manufacturer and it approached the problem very much like a sort of lean manufacturing organization of teams, organization, and role of technology, common pipelines. And the outcome of that was substantial, but was somewhat limited. We were doing about what? a movie every year and a half, which was actually relatively quickly out of two studios. We then added a third on in Bangalore. So speed of light saying, well, can we run this sort of shared service, shared production service across the globe? And it led us directly to what has become cloud.
And so, I had the privilege to working with some fantastic teams, fantastic technologies, great partners like Intel and HP, and so on to actually work out what is production in the cloud? What does this actually mean? Data management and so on, just to give you a sense of the scale, half a billion digital files per movie, 12 movies simultaneously, you’ve got something larger than Salesforce, essentially as a sort of computing fabric, 10 of 1000s of cause that you need to activate continuously in order to manufacture the imagery and pipelines that need to be incredibly stable. We’ll come back to that when we talk about game development.
So one of the fantastic advantages, and this is I think one of the highlights of certainly my career and it taught me a lesson was that we asked the question, okay, we’re doing all this fantastic cloud computing and so on and so forth, but can we change the nature of content production? And what’s the thing that would make the biggest impact in terms of quality and cost?
And the answer was, well, if I can see what I’m creating in the same way that consumer does, then I can just change it and make it right. And you know as well as I, the sort of linear media production model its guess and check. You make some changes, you send it off to the farm, it comes back, you look at it, it’s wrong. So you refine it. And what the net result of saying, I want to see everything all the time in real time, we could do that because we were no longer limited to the compute resources in a single machine boundary. So as soon as you have that type of fabric, and you could say, no, my frame rate is fixed, scale my computing resources to make that workflow solid and stable, you could reorganize your entire production process around it.
And that’s the spirit of real-time. That’s why real-time is such magic because it feeds the creator, it feeds the efficiency of the process, and you cut out all the wastage. So we ended up putting four orders of magnitude into the throughput of the studio who went from that one movie every year and a half to three moves as a year with 1000 artists at the beginning and at the end, which is amazing productivity. But it taught me a lesson, which is when you have a problem, solve it directly, don’t compromise. And that was the lesson.
And then after Dreamworks, I try to do that same model to the consumer products industry, try to replace photography with digitally generated images like we do in the film industry. That’s still a work in progress. The industry’s still sort of on its way to adopting that.
But now we come to Improbable and Herman Narula sort of talked to me and said, look, we have this scale computing platform. We want to change the nature of the game experience. And I said, I’ve seen this before, been there in the games it’s wide and casual. You’re using scale compute to basically change the nature of the experience. I know what this feels like, and you want to sell the games industry. That’s pretty damn hard, but yeah, I’ll go back. So I joined him and it’s been a pretty interesting ride since. And we talk a little bit about some of the technologies later on, which are very relevant for the metaverse, but that’s me. So, I would describe myself as a sort of Silicon level, deep technologist, mathematician, but very, very interested and focused on the business applications of technology for creative consumer products, across the board.
Marc Petit:
I mean, you’re right to talk about the metaverse. I mean, you have all of that background about, science and mathematics and computer graphics and games. So it’s going to be interesting. So Patrick, kick it off.
Patrick Cozzi:
Sure. So Lincoln, let’s start off with Improbable and how you’re bringing scale to the metaverse, which is a very hot topic on the podcast. And tell us about what you’ve achieved so far and some of the tech underneath.
Lincoln Wallen:
Yeah. So the challenge that the founders of Improbable set themselves was they played Second Life. They were young guys interested in experiences and they wanted to create experiences with great meaning, sort of depth and lots of people in them. And they looked at the existing games and the only pattern in video games at the time that offered that potential was the MMO, the Massively Multiplayer Online model. And so they conceived the idea, well, what if we could make it easier for smaller companies is to basically build MMOs. There just aren’t enough of them around things that we want to play. So the idea of the company was born to take their distributed computing expertise from university and create a platform that would do two things, commoditize the delivery of MMOs, but also give those MMO scalable compute so that the richness and interactions and scale of those worlds could actually grow with no bound.
It’s a great aspiration. Many people have it, most games start out with it. And then the green light committees get them into the real world. So, that was the original pitch. And it took a long, many trials, different types of techniques, many iterations of technology. And essentially ultimately ended up with a product called SpatialOS, which was a whole in platform with some underlying networking technology. I joined the company around that time as it was just outgoing to market and sort of looked at the value proposition, if you like. And now you’ve got to look at the market and the customers and who you’re selling to and what they’re actually interested in buying. And what you discover is that, while this is a great idea and it was attractive for the mainstream market, they had different teams doing their own processes, building their own approaches to the problem.
And it’s incredibly difficult to have an all-in package, something we learned with RenderWare with the RenderWare Studio package, only the Japanese bought it, because they are very, very process centric, changing somebody’s processes is like pulling teeth. And so it was very, very difficult to get this platform adopted. So we started to pull the individual pieces apart, the networking, the operations, running the service, the online services and so on, so forth. And that was much, much more successful. So today we sort of built the service organization that really delivers code, technology, support to a wide range of game customers, many of the big publishers doing what they need. So basically helping them with their engine, helping them with optimization, helping them with online service development, we’re a service provider.
However, we didn’t lose sight of the fact that the underlying technology really did have some promise. And in the last sort of 18 months really cracked the problem of bringing thousands of players at AAA quality. So just standard Unreal gameplay type characters into the same space. So you think of it as the single shot, but now we’re talking 10,000 players at FBS quality. So when you compare that with what’s in the market right now, 150 is max or you go down to sort of sprites and very casual experiences. So when you’re talking about 10,000, UE four quality characters in the same space, and then you want them all to be different, which means your rendering pipeline has to differentiate everything you can see and then you want them to speak with each other, which means that the audio processing has to deal with that scale of information exchange.
When we break it down to sort of interactions, what we’ve essentially done is move the needle, almost six orders of magnitude beyond the current sort of game experience to actually produce environments in with you can have crowds, you can have stadium full of people. They can hear each other, they can interact with each other as if they were in a fully fledged game. Interestingly, when we go to the games industry and you say, okay, here it is. It’s like, well, what the hell am I supposed to do with that? And we didn’t know when we started to try all this stuff, we tried some gameplay before all massive battles, all that sort of stuff. But the reality is that only a few environments in the world really in those environments do consumers really know how to behave. They know how to behave at a performance. They know how to behave in a stadium. They actually know how to behave in a riot, but it’s not fun. But beyond that, there aren’t many environments in which large numbers of people come together and know intuitively how to behave, the military know how to do that.
And so we’re really into a new world design patterns where lots of people can get into a social space. And what we discovered was that while the technology delivers this, it’s actually people that deliver the experience. So we ended up having to put people into that experience to organize it, to communicate just as you do in the theme park, just as you do on a tour. And so that gives you a glimpse of what the metaverse is actually going to be like, it’s going to be people-centric, relationship centric, interaction centric. The rest is noise. So if you can show off yourself in a unique way, you can speak and interact in a natural way and you can do so with any number, any scale in a high fidelity real-time sense, now we’re talking. Now that’s what activates commerce. It activates self-identity. It activates creative endeavors.
I mean, I know you’ve had Rob Bredow as one of your previous speakers, but just think about virtual production, where you bring your actors, you bring your film crew, you just bring them, thousands of people into the same space, shoot the movie, you can now direct them by speaking to them, you don’t actually have to sort of have these stages of media production. Sports stadium, millions of fans experiencing a central performance or sport. It’s endless, the possibilities, but it comes down to the fact it’s people with people with real opportunities to interact. And that’s I think what we are facing now, which is fantastic.
Marc Petit:
So if I want to benefit from that scale and that, do I need to build the content differently? I mean, is this a constraint on the creative?
Lincoln Wallen:
So very little, there are some constraints because as I mentioned, rendering, you’ve got to actually fit within the sort of rendering budget. However, all of this is separate from the game engine itself. So we built this solution today into Unreal. So that if you’re an Unreal developer, you could just use the normal tools, blueprints, content development pipelines and so and so forth. And really, as I mentioned about the animation production problem in real time, real time animations. These are very complex characters, very rich scenes, usually offline render. But if you can scale compute, you just reduce the frame rate until you can actually author directly. In some respects, that’s what’s going on here. Only, not with the scale out on the compute side. It’s really about looking at that as an optimization problem, solving it directly. Looking at all the interactions, identifying the patterns and actually being able to solve for those patterns so that you can actually manage the frame rate. Effectively, you lock in the interaction rate and then everything else basically follows from that.
So the content creation is relatively normal and that can be integrated into any engine really, proprietary engines included. So that’s also exciting because as I know at my cost, changing people’s process is really hard. So if you can fit it into that process, you can win much more quickly.
Marc Petit:
So you mentioned that the games people were not, I mean, it’s a challenge to have so many people in a game that’s the way to put it. Yeah. So who’s going to be your adopter?
Lincoln Wallen:
Well, so not surprising where we went was live performances. So we talked with musicians for example, and you may have seen, go to our website, you’ll see the reference to the AleXa concert. So a K-pop artist getting to hang out with a couple thousand of her fans in a sort of fan party with her music and so on and so forth. That type of live event, that type of connection with a celebrity is a pattern that people understand. And in particular, within K-pop, there were some sort of gamey styles and interaction models that are real in real life ways of responding to the audience that we were able to reproduce within that virtual environment. So we reached for patterns in real life where lots of people come together and have fun and asked what are those patterns or how can those patterns be realized within the virtual so that people that aren’t co-located, that don’t have the friction, aren’t on the same continent can still participate with the creators they love and are following. So that’s one route, to look at sports, to look at entertainment.
The other route of course is design, it’s to work with people that are used to leading groups. So whether that’s studio back and forth, whether it’s comedians, whether it’s tour guides or security guards, it’s how do I show you something? How would I give you a tour of a building? So people are used to managing crowds in the real world. Some of these techniques become incredibly important when you’re dealing with thousands of people, but also you go to things like sports, sporting events where you can have form teams quickly, and you can collaborate on achieving something. Interestingly, these things have echoes of TV programs, where you have groups of people coming together and others cheering them on. So these are the types of experiences that come immediately to mind because we’ve seen them before. Of course, the creators of today and tomorrow are going to invent things we’ve never seen before, but that’s sort of where we’ve started.
Marc Petit:
Yeah. That’s fascinating, bringing that dilemma of scale and the social interaction, that’s going to be such a fundamental aspect of the metaverse.
Lincoln Wallen:
And I think it puts a particular sort of perspective on when you say, well, what technologies don’t we have yet, you start to think a little bit about how a consumer expresses themselves. How do you get that? How do you get virtual production and the qualities of capture whether facial capture or gesture capture into the metaverse so that even though you have thousands of people, when you turn to the person that next to you, and you hear them coming within your sound bubble, so to speak, you can actually express yourself well. So I think those sorts of areas are going to become increasingly important for making these interactions meaningful.
Marc Petit:
No, no, absolutely. That’s a very good point. So I wanted to switch gears a little bit. And you mentioned you were a CTO for mobile and online Electronic Arts during the rise of the mobile platform. And we feel we are living through the rise of the real-time 3D platform also known as the metaverse. So from your experience, is there one aspect of the mobile transition that you think is particularly relevant for the metaverse transition? Any learning that comes to your mind?
Lincoln Wallen:
Yeah, I would say, well, there are many, I perhaps pick out three quite substantial, first of all, the target audience. So mobile was a thing, not because people had big satellite phones in their BMWs or whatever, but because actually it became mass market. Everybody had a phone. And so you’re now dealing with content that needs to speak to everyone that casualisation, so mobile was one of the, it wasn’t the only driver cause the web also did this, but it was one of the primary drivers that moved gaming into the casual market. And also the control. This is a casual audience with a very, very limited control surface. So things have to be incredibly intuitive. You take much more control of the experience as a mobile developer than you would be used to with a controller or mouse and keyboard in the PC world or in the console world.
So you have to be willing to construct a really satisfying experience with very, very little input from the consumer. It’s much more produced. Those were the creative concerns. A second concern was the business model and the delivery model. There were standalone games which actually mirrored the business model of the games industry. Here’s a game, here’s a package. And it was the early form of digital distribution. But very quickly, you think these are networks. Why can I not continually update this game? And so it became one of the main drivers for sort of continuous delivery and games as a service, that by the way is a lesson that the Western industry at least is still learning. And it goes right to the heart of the development practice. How do you develop, because if you are continuously delivering content, you have to worry about things such as CICD. You have to worry about regression. All of these factors that are dealt with a big slash sledge channel in your typical sort of package delivery have to be dealt with in a much, much more agile fashion. And that really changes the development process.
So at Improbable, all our tooling, all our content delivery, and the way we support our customers as well is oriented towards zero regression, continuous integration, game is always live, game is always playable. And that’s a principle, not a consequence. Is something you have to start with, and then you can iterate and refine at a creative level with the consumer. So, that was another thing that came out of the mobile industry.
And the third thing I think is structural – the relationship between platforms, devices, software distribution, and infrastructure, and the need for all players to participate equitably in that organization. The mobile industry is highly optimized. There are handset manufacturers, there’s software providers, there’s network providers, operators have relationships with consumers and paying attention to the structure of how money’s made and fitting in and exploiting that, but doing so in a reasonable and value to he or she who provides value sort of way, really sort of makes a difference. And where you stray from that you create somehow these environments that are somewhat brittle, they can be quite frustrating. And I think, the metaverse is going to need to address the question of that layering, who gets paid for what, how to make things open, recognizing however that some people make big bets. When a platform owner spends billions of dollars to basically create a value proposition, they’re the ones creating a lot of value. They should get a return.
So platforms aren’t necessarily evil. It’s just a question of how they’re built. Are they built on open standards? Are they accessible to content providers? And are they sort of managing the value chain equitably? What cut are they taking? Is it commensurate with the investment that they’re making, and where that works, the platform succeeds, where it doesn’t. And I went through many, many platforms, things you will have forgotten now, smart TV platform, Sharp, Fujitsu, all of the Japanese electronics manufacturers went through iteration after iteration trying to launch game services, Engage, Gizmodo there were just many of them that crashed and burned in the mobile era. So it gave a really rapid laboratory in terms of how to set these ecosystems up and how not to set them up and when open and open source and open content equated to low quality and no consumer value and where open actually worked for the whole ecosystem. So it’s a nuanced conversation and demands sort of quite careful attendance as to what one means by open. But that’s another conversation.
Marc Petit:
What was interesting in the mobile era and particularly in Europe is the role of the telcos. They were both in the value chain, are we trading between hardware manufacturing, making sure handset could get in the hands of consumer to decent price, buying a lot of content as well. And so are they going to have play, what’s their role of potential in the metaverse? I mean, less the US, I mean, the telcos in the US play a less prominent role than a Vodafone or others in Europe, so what’s going to happen to them?
Lincoln Wallen:
Yeah. They used to play a massive role. I mean, it used to be, this is what kept Nokia out of the US market, is that Nokia wouldn’t succumb to this carrier centric device purchasing. Nokia believed at the time that the consumer should make the choice, which of course is what you broadly have today. So they weren’t wrong. It’s just that Apple beat them to it. And the rest is history. But I think the difference between the two markets as it emerged was that the US market was based on a set of individual operators, competing head to head across the entire kind. So AT&T, Verizon, T-Mobile being the three, but there were others, Sprint was in there and there were other players, but all of them took the entire country as their remit. Europe being of commensurate at size, but also national telecoms had a more competitive environment.
But nevertheless, the mobile industry saw the benefit to the consumer of organizing the underpinnings of that mobile network, especially the data exchange in a way that allowed consumers to connect through one network into another. And for them to share the value of the transmission of data across the two. And they did that through an independent group, the Mac group that basically organized that arbitrage between the networks. And that was a very, very effective way. And it led to, that’s why texting became a big thing in Europe, almost 10 years before it did in the US. In the US, if you remember, you used to have to pay to receive a text because it was actually sent from another network onto your carrier’s network. So, that carrier had no relationship with the sender. Therefore, they had to charge you for the transmission of that data.
So the consequences of how the industry is organized really can show through in terms of business preventing or value preventing structures. So I would say the Europe model, layers of technology, common interest in interoperability is obviously the way we would like the metaverse to go. I think, as creators, as consumers, and luckily the internet has intervened. It’s changed the telco model to more of an internet model. So I think we will go there, but it requires standardization, cooperation, interoperability, and a recognition that the consumer is the person we’re servicing and all the rest of the value chain needs to be thinking consumer-first. What’s valuable to the consumer, and if it’s valuable, then the value can be shared across those different layers in equitable ways.
Patrick Cozzi:
So Lincoln you’ve touched on open standards and interoperability a few times now, and I wanted to jump back. So at one point you were the CTO of Math Engine. That was one of the early middleware providers for physics, doing things like animation, rigging tools. So I was curious about your perspective for opens standards for the metaverse, especially around physics and animation?
Lincoln Wallen:
Yeah. Some things. So, yeah, I was part of a number of standards, open GIS, some of the data standards and OpenVDB at Dreamworks, for example, volumetric data, which has been very, very successful ODE, which was the open version of the Math Engine physics library that then I think became at least inspired physics, which became Nvidia’s solution. So I think open source, especially in this industry is a very, very important tool and standardization around those open solutions, as it’s proved extremely effective at allowing the industry to collectively move forward. Marc, Autodesk actually drove a lot of the ingestion of open source solutions or standard solutions that the entire animation industry, at least, and some of the games industry organized themselves around, helped develop integrated with the tool chains, with side effects and with Meyer and so on and moved everybody forward, sort of in lockstep. We compete on content, but the friction of content delivery and development is a cost. So we’re all motivated by reducing friction and costs.
It also helps in the movement of people through the industry, you get trained in things that then that skillset can go with you when you move to a different company. So lots of good benefits that come from that. Some areas like physics because of the dynamic character of it, are very, very difficult to standardize. You go back to, a programming language has a dynamic character. The only way of standardizing a programming language like Java is to actually standardize the execution model. And that’s sort of how OpenGL was standardized. So you have to come to a conclusion around the execution model for dynamics. The problem with that is that the way in which you solve differential equations, they break down into many, many different families and the solutions structure for each of those families is different.
So it’s very, very difficult to come up with a comprehensive engine that basically represents the dynamics layer and gives you sufficient variability in terms of the types of problems you can solve and standardization, solvers, you can have conjugate gradient solvers, and you can standardize a little bit around the structure of those. But in the end, there are faster techniques that you can use for cars and things of that kind without going into the solutions that really do articulated objects very well, like characters or hinge and jointed elements. So we tried quite hard. I was actually part of that effort for Java. But I’m not sure we understand enough. The mathematics isn’t there to unify this into a systematic standard. So what does that mean?
Well, when this happened to graphics and you had direct 3D out of RenderMorphics, I think you had RenderMorphics and RenderWare side by side as graphics libraries. RenderMorphics became direct 3D. So RenderWare became development tools, became an engine. You could say the same is true of Unreal. It used to be in the machines. And then as you standardize on processing or floating point processing or higher level or glass libraries and so on and so forth. So you end up with content development oriented packages and the two sit together quite nicely. So that to me is the model that for physics, it’s probably up in content development rather than down in device standards and network standards. But for each of these areas, you have to sort of do that type of analysis to work out. Does it even make sense?
Marc Petit:
So I remember, at Dreamworks, you guys had a huge investment in rigging tools. And so we have this fantastic format USD from Pixar that’s excellent at representing static assets, or is it the same problematic as for physics to extend into things like rigging or procedural items that are absolutely critical for concentration?
Lincoln Wallen:
I think this is the challenge. So there are two axes, one, the pipeline for content creation, especially in things like linear media, they’re really data formats that need to accommodate change and derivatives because the authoring operations. So you’ve got these operations of encapsulation, you say, okay, here’s a thing with these inputs and outputs. So now I can describe this entire family with this very compact description, but then you want an instance of that with some changes. And so you start getting these sort of overriding structures as a matter of the irritative iterative development of your desired outcome. Now that works in animation because the pipeline producing the images isn’t the output, you use the pipeline, it’s the image is the output. So almost as long as you can compile out that image, it doesn’t matter how well structured the intermediate stages are or what stage the pipeline’s reached. You can branch the pipeline, you can special case it and so on. You’ve even got compositing. So you can have a complete mess in terms of the image generation and still come out with the right set of images.
Dynamic real time software isn’t like that. Every frame is generated moment by moment, you have a settled fixed piece of code. So you have to refactor, regenerate the underlying code and dynamics so that for all the use cases you’re interested in, it can be generated by that single algorithm or that single procedure. That’s a much harder problem. It requires more and more refactoring of the code. And it’s one of the challenges of game development and why they can sometimes come to market in quite brittle ways. And it’s one of why continuous integration, continuous live, continuous test helps you pay the cost of that refactoring, where it’s cheapest, rather than accumulating it to a point where it’s just too expensive to actually pay that cost. And you ship something that’s very brittle. You can’t really change it. And that really fit the way the consumer engagement works today.
So the underlying nature of dynamic content, interactive content has some pretty profound implications on what we mean by standards. I’d almost say the development process itself, it is more important to standardize than the entire format for the media. It’s better to be able to produce it out of smaller standardized packages with a standard process than it is to try to standardize the media itself.
Patrick Cozzi:
Lincoln, are there any topics that we didn’t cover today that you wanted talk about?
Lincoln Wallen:
No, I think we’ve talked about standards. We talked about scale. I think I’d like to bring us back to the structure of the metaverse. I mean, at Improbable, we are trying to solve two things simultaneously. We’re trying to provide a fabric that is open to other content developers, other engines, to create content on at an operating model and process if you like that actually allows people to rapidly develop content all the way out to user generated content if needed, but put that in the hands of creators because we can’t invent everything. It’s really about ensuring developers and artists and creators can actually stimulate the consumer.
And then secondly, we’re really interested in servicing those companies, really helping them get their job done, whether that’s engineering, whether it’s production or otherwise, because I think the wider world, companies that are not in content creation are looking to use interactive media now in the way that they use 2D graphics, you could say that the website has brought 2D graphics to become a critical tool of commercial activity. If you don’t have a website, you don’t exist and what’s on your website, determines your brand and consumers engage with you through your website. That’s the web two era. And then the same is almost true today with a mobile app. If you’re not on mobile, you don’t have an app. You don’t engage consumers with that UX, you don’t exist. You’re not there. And the consumer will not recognize that you exist.
And the metaverse is saying, actually we’re moving. There’s another step. And now the media, it’s almost back to Flash. If you remember those days, interactive design, all these things are coming back where the interactions are much more intense. The experience is the core thing.
And that draws on the experience of media creators like in linear media, but also in the games industry in order to actually deliver and engage in consumer experience. So the relationship of the wider commercial world to the games industry is really about to change. Now, whether both sides are really ready for, that’s another matter. But the hunger to engage this way is there on the commercial side is the willingness to engage there on the game side, because now you are talking more of a service in the industry. More like what’s happened in the visual effects’ industry and linear media production. That’s my prediction for the next phase of the games industry it’s going to become even bigger is going to become more critical to everyday commerce for everyday companies. That’s what the metaverse means.
But if we lose sight of the fact that the reason people are there is because they can engage with each other. The opportunity from a social perspective, from an interactive perspective is the thing we need to keep our eyes on because if it’s just a commercial tool, the consumer won’t be there. It has to be a tool of self-expression, of identity, of being able to get to the people and the thing that consumers love and hang out with them, doing nothing if necessary, but doing things, ultimately experiencing things that they enjoy and sharing that enjoyment with other people. So I think we feel we’re at the really critical element, which is bringing people together and the rest of it will take care of itself. As long as people are together, they actually stimulate each other. That’s the lesson of the games industry, ripe large, it’s an enormously exciting time to stimulate both consumers and the wider economic world. Just think about the roles for people, the jobs and careers within the metaverse because it’s people-centric not technology-centric.
Marc Petit:
And absolutely. And now I understand better how Improbable can play a big role in putting a lot of people together so that scalability and solving, and then designing applications that do good work at that scale is going to be something fascinating to watch in the next few years. So last question. Is there any individual or organization that you’d like to give a shout out to today?
Lincoln Wallen:
I think the sort of two sides of the same coin, I do think the visual effects industry, the media industry I think has demonstrated a maturity and a sort of creativity that is a really rate pattern to emulate with one sort of downside, which is it getting squeezed to death by the owners of content, the margins being compressed over and over and over again. And I hope that this emergence of the metaverse, the recentering of the skills involved in both the interactive industry in the games industry and the visual effects industry really becomes more front and center of the entire economy and gets revalued. And you can start to see elements of that when make direct references, but you know what I’m talking about as to why that’s the case.
So really is a call out to those creative industries, the digital creative industries are really the heart of the metaverse and that’s where the consumer will be won and lost. And the skill sets in those industries are the 21st century skill sets. And so for me, it’s been very privileged and very exciting to be a technologist servicing that part of the economic world.
Marc Petit:
No, I wholeheartedly agree with you that the past 30 years of work from the visual effects and the game development industry are really enabling the metaverse today. And we’ve done it in a way, I say we collectively, as a committee, have done it in a way that is very open and collaborative, and you mentioned some of those efforts and hopefully, we get to carry that spirit into this brave new world of the metaverse. What we’ve done together, we keep on doing together and keeping open and accessible.
Lincoln Wallen:
I think that’s right. Together, let’s emulate that pattern as we go out into this more exciting and aggressive world.
Marc Petit:
It was a pleasure having you, because you’ve seen that all, you’ve been there all along for the past.
Lincoln Wallen:
No.
Marc Petit:
And yeah, no, look, it was fantastic. Thank you. I think your background is in academics shows you have a knack to explain things very, very clearly. So thank you so much. That was very instructive. Patrick, all good?
Patrick Cozzi:
Yeah. Fantastic. Thank you, Lincoln, for joining us.
Lincoln Wallen:
Well, thanks for inviting me really enjoyed it. And really, I’ve been listening to a number of your
podcasts. They’re really fantastic. What a resource. I mean, you’ve done something really quite great here. I don’t know why I haven’t come across it before, but I’m going to be getting all of our company to are actually listen to it because I think it’s really insightful.
Marc Petit:
Well, it’s easy when you happen to know the people who’ve made it happen for the past 30 years and you just put them on a soapbox and we get all the credit. So, no, I think it’s the reflection of the community and we want to make sure that SIGGRAPH plays a much bigger role moving forward. How just the recognition of the role that.
Lincoln Wallen:
Yeah. On that topic, Marc, I mean, in the math engine days, SIGGRAPH was the center of the games
industry, as well as the linear industry. In the intervening decade, the one in the middle, the two markets actually move quite substantially apart. This is an opportunity to bring them back together. I’m not sure in what state, either one is actually in its core business model, one is getting acquired and consolidated, the other one’s already been consolidated because it’s getting squeezed. So neither is in a fantastic place, even though in theory, there’s a hell of a lot of money, hell of a lot of production, hell of a lot of expansion going on. It doesn’t feel comfortable. So it would be great if SIGGRAPH could re-energize back to the late 90s.
Marc Petit:
And this podcast origin’s is at SIGGRAPH and we care passionately about this organization. And I think the metaverse is the opportunity. And the celebration of that 30 years of built up of technology that now enables the metaverse is, I think it can be very natural that SIGGRAPH was at the heart of that all along and has been one of the mitigators and the promoters of all of that. And all those collisions between games and movies and interactive content. And so I’m pretty hopeful.
Lincoln Wallen:
One of the things that I think this arc of technology illustrates is how the different tiers of technology, the device, the Silicon design, driver design, tools, process cloud. What we’re really saying now is that we’re looking across the entire infrastructure and computing fabric and asking what is the right shape to deliver the experience that we’re asking for. One of the insights that comes out of solving this large scale interaction problem is that the break down of those layers is not right. Which of course we can live with it because demonstrably, we can put ten thousand people in this experience, but if we wanted to put a million people in the same experience, whatever that would be, then these things have to change. And so it leads into, what is your silicon going to look like? What is your infrastructure actually going to look like? What are your network protocols going to look like? It’s a wholesale optimization of the computing infrastructure, which is why it’s right to call the metaverse the next generation of the internet.
It’s actually not just the marketing term. It will affect profoundly the architecture of everyone or every piece of that layer and the commercial relationships between the suppliers of those components, because they’ve been optimized for either voice or for packaged content delivery. And now they’re being optimized for large scale interactive activity, and that’s a different problem. And it’s a little bit like, the codec is different and therefore your hardware can be designed in a different way to accelerate it. And that’s very fascinating.
Marc Petit:
Absolutely. And we talked to Bill Vass about it, the VP of engineering at AWS. And we had a number of conversations alluding to the change of infrastructure and cascading of devices are going to have to do.
Lincoln Wallen:
Well, the end non-volatile memory, optical interconnect on the cloud side and so on and so forth. Edge and software to find networks and rooting through it. All of these things really that global optimal that if we can get the industry, including infrastructure providers to play ball like Nvidia did and Intel did with the games industry, HP did with the animation industry, then we can make fast changes and change the experience. So looking forward to sort of getting into that in the next couple of years.
Marc Petit:
Wonderful. We’re going to say goodbye for real now. Lincoln, thank you so much. It’s been fascinating. Patrick, thank you. We want to thank again, our audience, that’s going to hopefully, listen to us in season two, as we continue to talk about the open metaverse. So thank you everybody. Thank you again, Patrick.
