56: Unanimous AI with Louis Rosenberg, PhD

This episode has been automatically transcribed by AI, please excuse any typos or grammatical errors.

Chris: Louis Rosenberg is a pioneer in the fields of augmented reality, virtual reality and artificial intelligence. He developed the first functional augmented reality system for the air force research laboratory. In 1992, he then founded the early virtual reality company emerging Corp in 1993 and the early augmented reality company, Outland research in 2004. He’s currently CEO and chief scientist of unanimous. Accompany that amplifies group intelligence and shared environments. Rosenberg earned his PhD from Stanford university and has worked as a professor at Cal state university and has been awarded over 300 patents for AR VR and AI technologies. Louis, thanks for stopping by barcode.

Louis: Yeah, thanks for having me.

Chris: Absolutely, man. So first off, most listeners, including myself, would entertain. VR and AR technology as you would commonly see in the world today. Right? So you’re talking about your Oculus rift, your PlayStation, VR, and I’m interested to hear how you personally got into that field so early in your career, as I mentioned, it was, it was an early nineties, I believe, before it was really commercialized or, or went mainstream.

Louis: So. You know, the field of VR really started in research labs in the late eighties. And there was a very small amount of commercial activity in the late eighties.

Louis: I was a graduate student at Stanford. At that time I was exposed to the technology. I was interested in the technology and I decided, Hey, I’m going to pursue my, my doctoral work, my PhD work in, in this field. And I was fortunate enough to get. To get an opportunity to, to be a research associate at NASA in their VR lab.

Louis: Interesting. One of the few places that had VR really happening in the late eighties and early nineties. And so my, my earliest research in VR was on. Vision systems and looking at how you model interocular distance or the distance between your eyes in software to see if that, if you can optimize depth perception.

Louis: So, so these are the types of things we were looking at way back then was, you know, these very basic parameters. And so that was interesting research for me. But I actually. Found that experience of wearing VR headsets to be isolating and enclosing. And and I felt like, you know, this, this capability of, of creating immersive virtual worlds is amazing, but I, I just don’t like wearing the headsets.

Louis: And, and this was before the idea of augmented reality before that word even existed. But so I would describe to people I say, you know, what I really want to do is take this idea of VR and just splash it all over the real world. So it can be, you know, out there in the. And and so I pitched that concept to the U S air force and and they gave me a fellowship to go to Wright Patterson air force base in Dayton, Ohio, and build a system which was called the virtual fixtures platform.

Louis: And it was really the first functional interaction. Augmented reality system that allowed people to, to interact with the real world in the virtual world at the same time. And and that was also a great experience for me. And, and the point of that research was to show that we can enhance people’s performance, manual dexterity by putting virtual fixtures, virtual objects into the real world.

Louis: And so we showed that augmented reality could do this. But again, the, the, the thing that had the biggest impact on me, it was also surprising, which was it. Wasn’t the fact that this worked, it was what the, I did a lot of testing on human subjects and every single one of them would climb out of this system.

Louis: And they would just tell me how amazing the experience was and how excited they were and how this was going to change the world. And I believed it. And so in 1993, I founded immersion corporation, which is one of the early virtual reality. With the idea that, you know, in 10 years, this is going to be everywhere.

Louis: This, you know, virtual reality and augmented reality, what we call it now called the metaverse will be everywhere. And I, I guess I was off by 20 years cause it it’s taken really taking 30 years, not 10 for that to happen, but I think, I do think we’re now there, we’re now at the point where virtual reality and augmented reality now branded as the metaverse will will impact the lives of most people in the next five to 10.

Chris: Yeah, I think you’re right. I’m curious, what were some of the use cases back? As you’re trying to really prove this out as being a legit concept. Right. And, and how you envisioned it then, what were some of the use cases that really you know, was the, was the breakthrough for you?

Louis: Yeah, so when I was pitching the concept of evolvement reality to the air force I, you know, I gave them lots of different examples of what you could do with it, but the thing that got them, the most excited was applications and actually in medicine and the example I gave them was surgery. So imagine if imagine if your doctor performing surgery and and you had the ability to, to, to have virtual overlays that you could add to your world overlays that would provide you with information about the patient.Louis: Even potentially let you see into the patient, basically a project there. Imagery that you would take from cat scans, MRIs, and actually see, basically see through their skin, by Oak, by combining the real world in the virtual world. And, and then I propose this idea of virtual fixtures, which was to say, well, imagine if these things didn’t just look real, but they felt real that you could physically feel these things.

Louis: Then you could create a fixture that could actually guide your hand the way a ruler would guides a pencil. You could have a fixture that you could overlay on the patient and guide your hands. And make it, you know, make a perfectly precise incision or protect a scalpel from going too deep into the, into the body to protect vital organs.

Louis: And because these are virtual. They don’t have to be sterilized. They can just be, you know, in appear at the touch of a button, they could actually go penetrate below the surface of the skin. They can go through the body even, whereas real fixtures couldn’t. And so that was really the concept that got people excited and what we tested.

Louis: At Wright-Patterson air force base and to allow people to feel things, not just see and hear, but feel the human subjects that I had in these experiments, they wore this big upper body exoskeleton kind of like what you see in in aliens moving whether it’s exoskeleton that would track their hand motion and apply forces back so that if you, if your hand interacted with a virtual object, it stopped, you you’d feel it.

Louis: And And it worked. And, and so w what’s interesting is that, you know, that was the vision 30 years ago. This weekend, I was just at a medical conference in San Francisco where this technology has advanced so much over the last 30 years that there were research studies being shown that it was a group that a group, I think at Johns Hopkins that was showing how using augmented reality, they are performing surgeries right now.

Louis: To do. What’s called pedicle screw placement. These are screws that they put into people’s spine. And and, and, and when they’re wearing these augmented reality glasses, these, they get targets that appear on the patient, in the exact location where the screw has to go to help guide, guide them, performing these procedures.

Louis: And and this is, you know, cutting edge research. You know, my view is within five to 10 years, every surgeon is going to have the, the power of augmented reality when they’re performing procedures either to help give them navigational information while they’re performing the procedure, or to give them x-ray vision, to allow them to just see through the patient, by projecting, you know, 3d cat scan images right below the skin.

Louis: So you could actually see. So so medicine. I think is at the cutting edge of this right now, but those very same things could be used for every industry, you know, whatever your field is. I think augmented reality will give people superpowers. It will allow them to to visualize things and see things in ways that they, that they couldn’t you know, they, they currently just put on a flat screen.

Chris: I was going to ask you the question in terms of, you said five to 10 years, you actually see this becoming a reality. What does it take to get to that point? I understand it’s going to be more than a pitch. There’s going to be trials. Right. You know, I know with healthcare because I worked in security and healthcare in the past that everything with healthcare takes longer because there’s a patient on the table at the end of the day, we need to make sure that, you know, everyone’s safety is, is looked out for. So what, what are some of the steps that you need to do from taking. Conception to actually implementing it in the real world.

Louis: Yeah. So there’s, I mean, there’s research labs all over the world right now, exploring augmented reality for use in medicine for using all kinds of of procedures, including sophisticated surgeries.

Louis: They’re seeing good results. The technology is advancing really the, the, you know, for doing augmented reality, the hardest. Is to make sure that the virtual world and the real world are very well aligned, you know, physically lined spatially aligned. We call that spatial registration because, you know, it’s, if you want to be able to look into a patient’s body, that’s most powerful.

Louis: If, if what you see is exactly aligned with. The right place in the body. Right. And so they are, there are there technologies for doing that even in surgery with, you know, giving very, very accurate registration and it keeps getting better and better. It’s a little bit time consuming because right now, sometimes they, they do these, this registration with what they call markers.

Louis: And so these markers are. They could actually have to be surgically attached to the patient so that the system can register to these, you know, these reference points.

Chris: You can calibrate it to the actual patient, right.

Louis: Calibrate to the actual patient. And and that works and it works very, very accurately, but there’s new technologies coming out now, which I actually just use cameras and AI to to look at, you know, to look at the patient in real time.

Louis: And. Accurately calibrate the, you know, the 3d cat scan with the patient. And they’re not quite as accurate yet as when. When they use these physical markers, but I think within five years they will be. And when that happens now, the whole, the whole process becomes simple. It’s, you know, camera looks at the patient everything’s aligned and now you can have these really accurate augmented reality experiences.

Louis: So. The short answer to your question is, you know, there, there are a lot of procedures that don’t require that level of accuracy and registration, even, even just for planning a procedure. You know, if you can have a patient before you and look below the surface and see see what’s going on without having to look back and forth at a screen, which is, you know, the way they do it now, then.

Louis: It’s going to save you mental effort. You don’t have to do these mental transformations in your head of like, what am I seeing? Where is it in the body? And and even if it’s, you know, it’s not super precise, you know, down to the, you know, 10th of a millimeter. It’s still really valuable for the, for the doctor.

Louis: So, so there’s really no barriers right now for this impacting medicine. In terms of technical barriers, the barriers are in some sense, cultural barriers in terms of like it has to get accepted by the medical community. It has to get accepted by doctors. They have to feel comfortable wearing the headsets when they are asked to do these things in experimental trials.

Louis: The feedback is always really positive. They, they, you know, they might be skeptical at first, but then they realize, oh, this is easy and it’s intuitive and I don’t need to you to teach me anything. And in a lot of cases, I mean, that’s the thing about augmented reality is it puts the information where you want it to be.

Louis: And so it’s. It doesn’t require an explanation. It’s just, as soon as you try it, you realize like, oh, that’s the way it was supposed to happen. And, and so you get this very positive feedback. So my view is that once augmented reality is really introduced to the general public on a consumer scale, then those cultural issues for doctors and really every other profession will disappear and they’ll realize, oh, I want to use this in my professional life.

Louis: And, and And augmented reality glasses are really very close. I mean, all the big companies are working on consumer grade, augmented reality glasses, apple. Google, not Google glass. Well, Google glass was really what you would call smart glasses, where it was giving you your textual information. I mean, I credit Google for being ahead of time and that it was a failure of a product.

Louis: There’s a failure of a product, but you know what? They take risks. They were ahead of their time. You know, people called it augmented reality, but it was really just smart glasses. It wasn’t putting 3d information in your world in, in the spatially registered place. Set the tone. Then he did say it did set the tone.

Louis: It soured some people towards that. We’re now at a point where companies, including Google are developing real augmented reality glasses. So apple, Google, Microsoft, Sony, Samsung snap, all working on and Metta all working on augmented reality glasses. They’re a little harder than virtual reality glasses because they’re, they’re lighter weight, they’re smaller.

Louis: They have to look stylish, right? Because you’re going to wear them out in public. But there’s, you know, if you look at the industry, it’s very likely between 20 24, 20 25, maybe 20, 26. That’s the period. When these companies will launch augmented reality glasses. So if let’s say apple launches, augmented reality glasses in 2025.

Louis: I think that will change culture very, very quickly, because what that will mean is that when you’re walking down the street and you’re looking around, you will see content and you’ll see magical artistic content all around you instead of on your screen. And people will realize like, oh, that’s how computing should be.

Louis: And my view is that when, when apple does that, let’s say in 20, 25, Within five years, you know, 20, 30, maybe 20 30, 1, 20 32. I believe it will be very, very rapidly adopted because it, and you can think of it like the iPhone, like the iPhone was launched in 2007. Nobody knew they needed a smartphone. In 2007, everyone was, had flip phones, but it, they launched it.

Louis: It had value. It took off. And as soon as it started taking off, people started feeling like, well, if I don’t have a smartphone, I’m missing out. I’m missing out on capabilities. I’m missing out on content. I’m missing out on information. And so within five years, you pretty much had to buy a smart phone, even though it was 10 times more expensive.

Louis: Right? And so when, you know, when snap and apple and Metta and Microsoft launch augmented reality glasses, It’s going to be the same thing. If you don’t have them, you’re missing out on content. You’re missing out on information. You’re walking down the street and you’re not seeing the amazing things that other people are seeing.

Louis: And so I believe it will have rapid adoption to the point where augmented reality glasses. I think in the early 2030s will replace phones as the primary way we interact with content. And in 20, you know, in the early 2030s, we’ll look back at those movies from the 2000 and tens when people were walking down the.

Louis: Staring down at a phone, you know, people with their neck bent, staring down on a phone, what can I say? Like, that’s ridiculous. Like, why are they doing? Like the information could just be everywhere. And so I, I do believe it will happen. I believe it will happen quickly. The technology is almost there and again, these companies are working on it and it’s not, you know, small startups working on it.

Louis: The biggest companies in the world spending billions, because they know this is, this is the future of their industry, especially the future of mobile phones, all the, all the companies know that the future of their industry is augmented reality.

Louis: Yeah. That’s just it’s going to impact us all. And it’s going to be quick, like you said, and yeah.

Louis: Things that you look at your phone for now walking down the street, I mean restaurant review, right? Maybe there’s an overlay. When you look at a restaurant, how many stars he got on Yelp

Louis: or, and it will be immersive, meaning you you’ll be walking down the street and you will that restaurant could have virtual dishes, you know, on virtual tables that you walk past kind of like how a, a waiter or waitress might bring you the special.

Louis: You know, a virtual waiter might walk out the front door of the restaurant and show you the specialist as you walk past. And it will look real.

Chris: Hey, every, every table has a nice view of the skyline, although you’re really just looking at a brick wall, right.

Louis: It is absolutely possible. And then we’re getting a little bit into promotional content. I mean, there’s all these stores and restaurants will be filling our world with promotional content. You know, there’s there’s issues there that we could talk about that, the concern me, but they’ll also be artistic content and magical content.Louis: And you know, the creative things that developers can do with augmented reality throughout your life is, is amazing. It’s limitless, it’s limitless, and On the, on the flip side, the things that that advertisers can do in large corporations can do. And the companies that control the, the members can do are also limitless and and could become overwhelming in, in, in, you know, how much access they have to their consumers, especially if you’re wearing augmented reality glasses.

Louis: The way you use a phone today. I mean, people today, from the time they wake up to the time they go to sleep, they have their phone in their possession and that’s how, you know, augmented reality glasses would be. You would, you’d be wearing them all day. And and it will give third parties the ability to track what you’re doing throughout your day, and then hit you with promotional content throughout the day.

Louis: And so I’m a big proponent of Regulation of the metaverse to protect consumers so that we have the magical experiences, but we don’t have to worry about what the, the companies that control these technologies can do.

Chris: Yeah. That’s why I think the development of this technology is so critical in the way that it’s developed, the way the software’s developed the privacy implications that need to go into the, the thought process there, because I’m just curious to know.

Chris: In the metaverse or when things become more seamless to the user, how much easier those malicious acts who are privacy invasion situations may be, it may be easier for an attacker to take advantage of a user. You know, I can’t predict that, but I can see how that could be possible.

Louis: Yeah, I think it’s interesting to think of both the legitimate uses that are dangerous and the bad actor uses the, the the uses by an illegitimate party, but even just the legitimate uses. When you think of like, I like to compare, you know, the metaverse to social media, because people, you know, people are starting to realize over the last few years, that social media has, even though it had these utopian visions at the start, it has, there’s a lot of really negative, scary things about social media.

Louis: Because it gives the platforms, these very large amount of power, and then the platforms sell that power to third parties because that’s their business model. And so if you think of like, what’s the source of the danger of social media, their business model is really about tracking and targeting consumers.

Louis: And, and so the, you know, and they track. Pretty extensively, they track where you click and what you buy and who your friends are, and they use that to profile you over time. And then once they have these profiles, they can then sell access to. To third parties who target you with algorithmically generated content and or out algorithmically selected content.

Louis: And it’s newsfeeds it’s advertisements, it’s introductions to other friends. And this has created unexpected problems. It’s it’s amplifying people’s biases. It’s persuading people on behalf of sponsors. It’s perpetuating misinformation, disinformation. It has these damaging impacts. Okay. So now we can take that social media and think let’s think about the metaverse.

Louis: Well, in the metaverse they’re going to also track you and target you, but tracking you. It doesn’t just mean where you click tracking. You means where you go, what you do, what you look at, how long your gaze lingers on something. If you’re walking down the street and you slow down and you look in a store window, they will know that they’ll know how long your, you know, what are you slowing down to look at?

Louis: What are you, what are you walking past? They’ll be able to track your gate. Were you speeding up? Were you slowing down? There’ll be able to track your POS. And then from your posture, they can infer your, your emotion or your interest level. Very likely in the metaverse they’ll track, they’ll monitor your facial expressions, the monitor, your vocal inflections.

Louis: I believe even monitor your vital signs and PE when I tell people vital signs and go, oh, no one will agree to that. People already agree to that with smartwatches. People already agreed that with smartwatches. Those same technologies in the smartwatch has already there’s already companies working to build them into earbuds.

Louis: And so that ultimately your buds will, will monitor your blood pressure your heart rate, your respiration rate and, and okay. Like that’s interesting and valuable for medical applications, but. Well, lots of money in VC going into using all of these things, where you go, what you do, what you look at, what your, what your blood pressure is to advertise to you, to market, to you more efficiently.

Louis: So, so the first thing about the metaverse is from the especially in augmented reality, from the moment you wake up to the moment you go to sleep, they will know everything about you. And in some sense, they have to know that to be able to build the virtual world around. But we could regulate, they don’t have to store that information.

Louis: They don’t have to be able to to track that information and profile you over time. So they know exactly your behaviors. As you walk down every street and what stores you look at, and we don’t have to let them know that, but once they know that, then what are they going to do with that? Well, if they use the current business model of social media, they’ll use that to target you with promotional content, but targeting isn’t going to be.

Louis: Pop-up ads and, and, you know, videos going to your feed targeting is going to be immersive. And so there, so for example, they’ll use virtual product placements, meaning I walked down the street and I see, you know, a certain product. I see a certain car parked and I might think, oh, that’s. That’s a nice car.

Louis: I might not realize somebody placed that there specifically for me to see, or I’m walking down the street and I see people walking the other direction, holding shopping bags for a particular store. And I see it again and again, I think, wow, that that store is pretty popular where it must be around the corner.

Louis: And I might not realize like, no, I’m, I’m the only one seeing that that was put in that was injected into my world on behalf of a pink sponsor for me, or I see. And so this idea of virtual. Where they’re injecting into my world targeted experiences. That’s promotional con. Will happen and it will be just as real as everything else in my world.

Louis: And so I won’t be able to distinguish, did I just serendipitously come across these things or did somebody manipulate my world for me to, to have that experience? And that will happen? I believe it should be regulated. Meaning if somebody is putting virtual product placements into my world, It should be required, that it looks different than that, that I can tell the difference between seeing somebody walk down the street using a particular product and know that that’s just a real, another user in the world, or it was injected into the world on behalf of a sponsor.

Louis: I should, I should be required. They should require people to be transparent, to require sponsors, to be transparent so I can tell the difference. Now, virtual product placements will have. It won’t just be for, for products. It could be for political messaging. It could be. And I can see people walking down the street with a, you know, wearing a shirt and for a particular candidate, or I could see people protesting or, or, and so this, this idea that they can again, manipulate my world and I need to be able to know, did somebody pay to put that into my world or is that just a natural experience?

Louis: In my environment and then it will go even further from just virtual products to virtual. There’s lots of companies working on photo realistic virtual people that will fill the metaverse, whether it’s a fully virtual world or an augmented world. And and they will look real. In fact, it was just out, it was just a study that was released a couple months ago from Lancaster university and UC Berkeley, where they, they, they, they used the latest AI technology to generate artificial human face.

Louis: Computer-generated human faces. And then they did testing on hundreds of people to see. Can you tell the difference between a computer generated face completely. The people do not exist, but just completely jet computer generated and real photographs of people. And they found people cannot tell the difference that that state of the technology now is such, that people cannot tell the difference.

Louis: And that was, that was an interesting result. But the crazy result was they also asked people to judge the trustworthiness of these things. And they found that, that, that people judged the computer generated, faces more trustworthy than the real spaces. Yeah. Yes.

Chris: Yeah. It makes sense. I mean, there’s characteristics that make you want to believe somebody make them look more trustworthy.

Chris: So as they’re developing these algorithms…. what’s the one site thispersondoesnotexist.com. I don’t know if you’ve came across that one, but that’s a prime example right there. They all look trustworthy.

Louis: And so if you’re an advertiser, whether you’re a legitimate advertiser or pushing misinformation, disinformation, if people trust AI generated faces and AI generated people, virtual people, more than real people.

Louis: You’ll use virtual people and you don’t have to pay actors. You can use virtual people. And so the virtual people will happen. And so, and there’ll be two different types of uses of virtual people. I was what I would call passive and active. So passive might be, I might be in a virtual world, you know, fully virtual with avatars or augmented augmented world.

Louis: And Overhear a conversation between two people about some political issue. And I might think, oh, they’re just bills are just real people having a conversation. I might not realize that those are just, those were placed there for my behalf on behalf of a paying advertiser and taking that one step further.

Louis: We’re not that far away from these virtual avatars that. Real being being controlled by AI to engage you in PR promotional conversation. And so the ultimate, the ultimate form of advertising in the metaverse will be promotional conversation where AI driven agents that look real are engaging you in conversation to persuade you, to buy a product, to buy a service, to book, to buy into a political belief, to buy into a piece of disinformation.

Louis: They will engage you in these promotional conversations and they will be, if it’s not regulated, they will be armed with a full database of all your behaviors that have been tracked in the metaverse. There’ll be armed with tracking your facial expressions and vocal inflections in real time to track your emotions.

Louis: And the technology is very sophisticated already for real time emotion tracking. So imagine if you’re interacting with a, a virtual. Spokesperson that’s AI control that, that AI agent is tracking your emotions in real time, because it has, it’s even potentially looking at your blood pressure and heart rate while it’s talking to you.

Louis: And it’s adjusting its tactics to persuade you. That is where we’re headed, unless it’s regulated. People are very aware that AI can beat the world’s best chess player and beat the world’s best go player. How hard you think it would be for an AI like that, that looks human, that’s tracking your blood pressure and your vital signs to persuade you to buy a product you don’t need.

Louis: I think at easier than beating the world’s best chess player or to prove, or to persuade you to believe something that’s just nonsense or to persuade you to bleed something it’s not in your best interest. So this combination of. Of the metaverse and our artificial intelligence together makes a very potentially very scary world unless it’s regulated, unless the platforms are not allowed to use your vital signs to advertise to you are not allowed to do real time emotion profiling to advertise to you, to, to manipulate you.

Louis: You know, those, those types of regulations are I think, pretty reasonable. I don’t think people want their blood pressure to be. By advertisers. And now is the time to push for those regulations rather than, you know, end up like we did with social media where people waited 10 years too long to even think about regulating social media.

Louis: Cause we didn’t, we didn’t realize how damaging some of these practices would be with the metaphors. We have, you know, we have the benefit of social media as an example that, you know, giving the platform, the platform, sit in this position of power. Were there watching what people do and injecting information into their world and that power can distort your world.

Louis: And in the metaverse that power is much, much greater. I mean, what I like to tell people, like the whole point of virtual reality and augmented reality is to fool the senses. That’s what it’s for. It’s it’s trying to blur the boundaries between what’s real and what’s not right. That in the hands of a powerful corporation, you know, what could go wrong?

Chris: Yeah. Yeah. A hundred percent. Now, let me ask you this. On the flip side, you mentioned virtual avatars being controlled by it by AI. There’s also situations where you have virtual avatars being controlled by real people, for instance, in gaming. Right. And So Metta developed a game called horizon worlds.

Chris: And I don’t know if you remember this story, but back in December, a woman in the UK claimed that she was sexually harassed and raped in this metaverse gain. So when you think of regulations and governing, do you see that playing out as well? Where we’re going to see real life penalty? Enforced for virtual crimes committed.

Louis: Yeah. So I think there’s, I think there’s a lot of interesting issues there that are potentially scary in one, you know, the issue of harassment harassment happens online now. But it gets worse when it’s immersive. Whenever, you know, the more realistic things get the more that online harassment becomes like real in-person harassment and And we’re headed into a place where VR is virtual worlds will only be photorealistic.

Louis: And the potential harassment is almost as damaging as, as if it really happened.

Chris: And it’s not the same, although it could still inflict, you know, emotional trauma.

Louis: It’s not the same, but it will get closer and closer to the same every year. So, because we’re headed in a direction where it’s, you know, things become more and more real. And even to the point where, you know, physical touch will be in as part of the Metro versus well, it’s, that’s, that’s further out because the equipment is, is, is more expensive and there’s there’s other issues, but But the, the potential of what happens with, you know, avatars that are controlled by other people.

Louis: There’s this interesting. One of the big buzzwords in the w in the metaverse that a lot of companies are pushing is this idea of, you know, digital twins and, you know, digital twin would be, you know, oh, I can create a, I can create an avatar, looks exactly like me. And and so I can have a presence in the metaverse.

Louis: Photo realistic, but completely virtual. And that will happen. The thing I worry about is what I would call evil twins at where or an evil twin is. Okay. You have an avatar looks exactly like you. What if it’s not controlled by you? What if, what if it, I mean, that’s the ultimate form of identity theft, somebody hacks into engage control of your, your photorealistic avatar of you, or they just duplicate it.

Louis: And now there’s they’re using that to, to commit fraud and they’re not just emulating what you look like. They’re also using voice changing AI to, to emulate your voice. And so Deep fake in the metaverse. So it’s a real time deep, fake controlled by a third party. And so you could imagine that a coworker, somebody just pretends to be a coworker looks just like the coworker sounds just like the coworker and they’re using it as corporate, corporate espionage to get information out of you or and, and so this I, this, the potential of you know, of identity theft and phishing attacks and fraud.

Louis: In an immersive world where the technology is intended to fool the senses where there’s, you know, major corporations working on creating the technology so that they can make a photorealistic version of you even simulate your voice is dangerous. And and so it raises not just regulatory concerns, but security concerns, the same amount of effort that will go into regularly.

Louis: You know, corporations so that, that what’s considered legal is scaled back. We also then have to realize, well, there are parties who don’t care, what’s legal. And they will use all these techniques, whether it’s injecting, you know, fake things into your world. But that will include, you know, these evil twins that are, look just like somebody, you know, but aren’t controlled by them.

Louis: They’re controlled by somebody who is, who. Trying to manipulate you.

Chris: And as this technology develops, I’m sure that technical controls will develop as well to be able to hopefully detect these type of situations. But I also think, you know, awareness is super important too, for the public, the consumers that are going to be purchasing these devices, purchasing this software and just like not everybody’s aware of what a phishing attack is.

Chris: You know, it, it just becomes more important, I think, to stress the dangers of what happens, because again, it could be seamless at that point.

Louis: I mean that it ultimately will be seamless. I mean, that’s, you know, that’s what these companies are spending billions doing. And I mean the amount of money going into metaphors now.

Louis: It’s crazy. I mean, Metta spent over $10 billion last year, and that’s just the start to make these technologies seamless. And again, there’s amazing app. Like they’re like these companies will create amazing applications and magical experiences. And to me, the big thing is, okay, we, we know it has these, you know, people could potentially have great benefit let’s, you know, let’s just make sure that.

Louis: Or at the same time, putting just as much thought into, into restricting corporations from from exploiting consumers, with these, with these capabilities and also preventing bad actors from taking control of these capabilities because both of those things are just, you know, are going to be just as powerful as the, the positive applications.

Louis: Oh yeah, yeah, yeah.

Chris: We just can’t hinder the forward movement of what’s happening here, right? It’s just, we just need to run in parallel with it and make sure that, you know, the proper safeguards are in place because you don’t want to get to a point and then have to worry about that later because we see that far too many times.

Louis: Absolutely. And you know, and a lot of people get nervous when you talk about regulation. And I think in this case, There’s not really a need to regulate the, the content creators and the artists and the people who come up. It’s because it’s the biggest issue is to look at what are the parties that are putting on the position of extreme power.

Louis: Yes. Right. And you think about like, if you’re, if you’re a metaverse platform provider, which you really are, is you have, you, you’re basically tracking, you could be tracking a billion. Where they are, what they’re doing, what they’re looking at, what direction their gaze is looking either you’re tracking, you could mean you have this God’s eye view of a, of a whole world.

Louis: That’s a powerful position to be in. And you have the ability to just manipulate that world at will. You can have something appear, you know, right before them. And, and, and so I think educating the public that the power that the platforms will have. What makes social media look kind of quaint, right? Like, like social, like platform providers and social media have a lot of power, but, but they’re not, you know, basically looking down on a, on a whole world of people are knowing exactly what they’re doing.

Louis: And so, yeah, it’s, the power is extreme, which means, you know, thoughtful regulation makes sense. It’s, you know, the goal is not to suppress the industry. The goal is to, to allow these magical things to happen. But yeah. But allow consumers to feel, feel like they’re not being exploited by the companies that control the metamours and also are not a danger of fraudsters who are.

Louis: Basically popping into the metaverse and, you know, look like their good friend. They aren’t.

Chris: Yeah, exactly. So I would, I do want to shift over to a unanimous AI for a moment. And I’d like you to explain swarm technology to me. Is this a technology that you developed internally?

Louis: Yes. So so my current company is called unanimous AI and we’re an artificial intelligence company, but we, we take a really different approach to AI.

Louis: So most AI companies are. Work on, you know, processing huge amounts of data so that they can essentially replace people from certain tasks. Sometimes replacing people from repetitive tasks, sometimes replacing people from decision-making tasks. But there’s a big push in the AI world to say, oh, we can build AI that take people out of the loop unanimous AI, our coaches, different our approaches to say.

Louis: We think people are smart. We think people have amazing capabilities. Can we use AI to make people smarter, to make, to allow people to make better decisions? And, and especially we say, can we allow groups of people to make better decisions? And so we looked and like a lot of technologies, we look to nature for inspiration.

Louis: We say, well, you know, nature has spent millions and hundreds of millions of years. Involving different techniques for, for for different species. And there are some species that function in large groups, schools of fish, flocks of birds, swarms of bees. And and nature has developed something called swarm intelligence.

Louis: And so swarm intelligence is, is biologically. Process where a, a group, when they work together as a system, as a, as a tightly controlled system can become significantly smarter than the individuals who make up the system. And so think of a school of fish, a school of fish might have thousands of individuals.

Louis: Each of those individuals has a slightly different view of the world. There they have a slightly different history experiences, even slightly different personality. And. And they move together as a system. They’re actually detecting vibrations in the water between fish to make decisions. And so if you look at a school of fish, they move around through the ocean.

Louis: As a biologist will call them a superorganism. They, they, they, they move it’s almost like a single intelligence. And so a school there’s the thing about school of fish is there’s no leader. There’s no, there’s no one’s in charge. And yet the school of fish can navigate the whole ocean, navigate their lives, be a successful species.

Louis: That’s been around for hundreds of millions of years, much longer than you. And they’re smarter than the individuals, swarms of bees do the same thing. Swarms of bees make remarkable decisions in large groups that the individuals could never do. Flocks of birds do it. And so unanimous AI, we said, well, look, if flocks of birds and schools of fish and swarms of bees can get smarter by working together in systems as a swarm, why can’t humans do it?

Louis: And so we developed a technology called swarm AI that connects groups of people together over the. And allows them to make decisions together in groups. And it it’s very visual it in some, some people say it almost looks like a Weegee board. We have a group of people like, oh, moving together to make a decision.

Louis: Like when, imagine a review board that has, you know, a thousand people all moving the puck together. And, and so we develop that it uses AI to combine everybody’s insights. And we figured it works for nature should work for people. And when we started testing it, it works. It allows, it allows us to harness the intelligence of groups so that they can make significantly more accurate predictions and forecasts and estimations and significantly better decisions.

Louis: Basically, anytime you have a group of people who have some. Distribution of knowledge. If they come together as a swarm in our software is actually called swarm. They log in, they make a decision together. We can make them function as a, as a superintelligence

Chris: very interesting, and this could help minimize the threat of data poisoning or synthetic data injection on a mass scale because otherwise, your projected outcome or your trained algorithm is just completely altered.

Louis: You bring up a really interesting point, which is about what you really about AI in general. So most AI systems, I would say the vast majority do create intelligence by processing big data sets. So they can take a big dataset and they’re finding patterns in the data and they use those data, those patterns in the data to make for.

Louis: Okay. And it works, but it’s only as good as the data. So if you have bad data or corrupt corrupted data or old data, then it doesn’t work. Right. And so what we do at unanimous AI is that. We don’t process any big sets of data like that. We target what we would call the human database. Meaning if I pull, if I connect a hundred people, each one of those people has a brain filled with information and has a brain filled with knowledge and wisdom and insight and intuition.

Louis: The people are continuously updating the information. I mean, that’s what we are. We’re information collecting machines. And so like for example, every year we forecast the hospital. As a swarm, we do that. So we take on a lot of challenges from journalists because they’re always saying like, oh yeah, you can amplify intelligence.

Louis: Do this, predict the Oscars. And so we, we, it was actually, Newsweek was the first to ask us to do the Oscars that way we did it for them for a number of years. And as we pulled together 50 people, those 50 people are not experts. They’re just moving through. But they, but they live in the world. They they’ve seen they’ve seen some movies.

Louis: They’ve listened to the radio. They’ve watched TV, they’ve read something online. They have this distribution of information and knowledge about the movies and about the Oscars. We bring them together and we have them predict as a swarm. And then we compare it to professional forecasters, you know, movie critics and and every year we do this 50 regular movies.

Louis: And to predict the Oscars with like 90% accurate accuracy, 95% accuracy, maybe as low as 87% accuracy. But every year they outperform the New York times, the LA times variety magazine, vanity fair, which are, you know, professional experts and, and and the point is. People are smart. People have good information in their heads and we can connect them together and create the superintelligence and their information is up to date.

Louis: And you can compare that to traditional AI where you need this big database of historical data and that historical data is continuously getting old. And so the data you might have about last year’s Oscars, aren’t really going to help you predict this year’s Oscars cause it’s different movies, but we see the same thing even in the business world.

Louis: You know, we have a lot of customers who, who use our swarm technology for doing sales, forecasting, market forecasting message testing, but imagine Salesforce. If you’re trying to predict the sales of a product this holiday season, and you use data from last year’s holiday season, what you’re really doing is you’re predicting how well your product would have done last year.

Louis: But if you connect together groups of people who, who are aware of the today’s economic conditions and the sentiment of their world, and then we can make much more accurate. Product forecast or political forecast. We do the same thing with political forecasting. And again, people have vast amounts of knowledge and wisdom and intuition that’s up to date and they have a better sense of how to PR of how a political campaign will play out.

Louis: Then people using big databases of historical data because the data gets old. So.

Chris: I need to ask you about sports forecasting though, because the Kentucky Derby is tomorrow and at one point swarm technology predicted the superfecta of the, I believe it was a 2016 Kentucky Derby. And for those. Aren’t into horse racing.

Chris: That’s predicting the first four finishers, which happened to be at the time 540 to one odds. So if you don’t mind, could you talk to me about that a little bit and how that came to be?

Louis: Sure. So like I said, we, we take on challenges from journalists and I was a journalist from CBS interactive who came to us.

Louis: This was a few years back and said, you know, I seen you predict the Oscars. I see you predict the Grammys, predict the Kentucky. You said, okay, well, can do that. Is it well, no predict all four horses, first four horses in order, that was a challenge to you. That was a challenge. It was pretty the first four horses in order, we said, okay.

Louis: We, and we didn’t know anything about horse racing. You know, our technology has never seen horse racing or what our technology can do is take a group of people and connect them to. And make a superintelligence. And so we, we we found 20 horse racing enthusiasts just online that not experts, not professionals, just enthusiasts.

Louis: We had them come into our software called swarm and we said, okay, let’s predict let’s predict the Kentucky Derby it’s they predicted which H which words would come in first, second, third, fourth. And we gave the, we gave the predictions to the jury. We didn’t know she was going to do this, but she actually went to the Kentucky Derby and she placed a bet on the superfecta and she tweeted out her ticket which suddenly put a lot of pressure on us, like, okay.

Louis: And of course in tags, you know, and she told me she tagged you. So it was it was, you know, CVS interactive. So we got, you know, it got attention. And I placed a bet is where I placed a $20 bet. A lot of her readers placed bet. And the forces came in first, second, third, fourth. And, and so anyone who placed a $20 bet, like I did a one $11,000.

Louis: So I went $11,000 on that bet. One of her readers wrote into her and told her that they won $50,000 on that bet. And so it was it was an amazing example, but what was even more amazing about that is that we had all the data, we went back and we looked at okay, of these 20 people. Who came together in a swarm, you know, how many of them would have predicted the full horses right.

Louis: On their own, none of them. And we said, well, if they had just taken a vote, what would they, if they have just taken a boat, they would have gotten the winner. Right. But the other three wrong. Swarm is really very, it’s not a vote it’s, it’s a system. So it’s, so it’s a system, meaning, you know, people start pulling for different horses and, and it’s like a multi-directional tug of war.

Louis: Like, imagine it like what, it looks like a school of fish, making a decision they’re pulling and pushing on each other until they can find the one solution that they can best agree upon. And it’s it’s usually the best solution based on the information that everybody has. And the thing is like, when you do a one-off event like that which is, which is fun.

Louis: We do a lot of them like the Oscars. It’s hard to do to prove this statistically viable. And so we do, we, we we do rigorous studies with usually with universities as partners to do large amounts of data. And so, for example we do, we still do sports forecasting. We’re not forecasting the Kentucky Derby this year because we’re currently forecasting the, the entire NBA basketball season and, and major league baseball season.

Louis: So we’re predicting. You know, dozens of, of basketball and baseball games and also soccer games in the English premier league. And we track performance over time. And so the last season that just finished for us was we did the the NFL season. We did the, in every city, we predicted every single game of the NFL season against the spread, which in football is like the hardest sport really to gamble against, against the spread.

Louis: Cause there’s so much money in the, in the the, the handicappers are really, really good. And so if you’re a typical gambler, a betting on football against the spread, your performance is going to be 50%. You get half your bets, right. And then you’ll lose 3% every time. And you’ll you’ll over the course of season.

Louis: You’ll lose all your money. That’s what we’ll after will happen for most people. If you’re a professional gambler, if you can get 54% against the spread. 50 54% accuracy and your bets against the spread, then you’re making money and you’re professional gambler. If you’re like the top of the top, you might get to 55% accurate.

Louis: We use a group of 20 regular sports fans. We were 56% accurate against this. For the entire NFL season. And we see that for season after season. And the point is that people are smart, like regular people are smart sports fans. If, if we can connect them together and combine their insights and their knowledge and their wisdom, we can build a super intelligence that can predict sports that can predict financial markets.

Louis: We have we have customers who are hedge funds that that bring their analysts together and, and predict You know, predict particular particular equities. We even did a big research project a couple of years ago with Stanford medical school, where we had, where they wanted to see can they can, they amplify the intelligence of doctors.

Louis: And so they had groups of radiologists diagnose chest x-rays as a. And and it was small groups. It was just four or five doctors connected together. And they either diagnose the chest x-rays on their own diagnosed by taking a vote or diagnosed as a swarm. And when they work together as a swarm, they reduced their diagnostic area.

Louis: By 33%, 33% fewer errors when they work together as this one. And so so ultimately people are smart. When you connect groups of people together with AI, you can make them smarter and we’re not taking people out of the loop. We’re keeping, we’re keeping people into the process, which means we’re keeping their values and morals and sensibilities in the process.

Louis: We’re not replacing human decisions. With software, we’re amplifying human decisions with software.

Chris: Love it. So here at barcode, we have an AI robot named Booz bot and he takes inventory of all the drinks that our patrons order, and then calculates what someone will order at any given time.

Chris: Hypothetically, of course, Is this possible?

Louis: It is, Yes. And especially when we live in this augmented world where people are wearing AR AR glasses and, and you know, where they’ve been and what, what they’ve done and. An AI would be able to predict with high accuracy what drink you would order, and, and you might not even know what cues they’re picking up on.

Louis: It might, you know, it might, especially if it was really tracking all of your behaviors, it might be looking at, you know, the speed of your gate as you’re walking from the parking lot. Like when you’re, you know, when you walk in, you know, with a brisk gate that it knows you’re, you know, you’re in a certain mood and you’re going to order X.

Louis: And if you walk in sluggish and it knows you’re going to order Y and or it might be a combination of your gait and your facial expressions, it might. And so the point is that if a system can profile people by tracking where they are, what they doing, their gait, facial vital signs. It could accurately predict a lot about them and and people might appreciate you know a, a virtual bartender that’s, you know, that’s very insightful, but there’s just as many uses that people would not appreciate being able to be being predicted.

Chris: Absolutely. So you are in San Louis Obispo, California. That’s the the central coast region. So if I find myself out in that area where would you direct me to for good drink? Do you guys have any unique bars in that area or is that more wine country there?

Louis: It’s definitely wine country, but there’s also breweries.

Louis: But it’s definitely wine country. So, I mean, there are there are beautiful, really beautiful vineyards all around. And so I would I would, there’s an area called Edna valley that I would direct you towards with some really, really pretty vineyards. There’s also Avalon beach area has very pretty vineyards.

Louis: What most people will be looking for that’s unique in this area would be the, the the wineries and vineyards and it’s it’s wine country.

Chris: All right. So I just heard last call here. If you decided to open a cybersecurity or an AI themed bar what would the name be and what would your signature drink be called?

Louis: Great question. I’m not sure what the name of the bar would be, but I think a good name for a drink would be the evil twin. There you go on a theme of cybersecurity. I do think that that will become a, the future of identity theft.

Chris: It is going to be the future of identity theft. And I’m thinking that, you know, when you order that drink, you basically get two of the same.

Louis: Yeah, maybe that’s. There you go.

Chris: That’s awesome. Well, Louis has been a pleasure speaking with you before you go, could you let us know where we can all find you online?

Louis: Sure. So. Are the companies unanimous AI. And so if you want to find out about our sports forecasting or about how to amplify the intelligence of groups, just go to unanimous.ai.

Louis: You can find me there as well. If anyone wants to reach out to me, they could email me, Louis@unanimous.ai. And I’m also on LinkedIn at just Louis Rosenberg at LinkedIn. So you can find me there as well.

Chris: and everybody say hi, if they run into you in the metaverse at some point. And yeah, man, I’m betting on the Derby this year.

Chris: I’m betting on a horse called CyberKnife. I liked the name, but I’m also, I’m also betting on, on you and, you know, the future of unanimous AI really believe that you’re going to continue. Just trailblazing and innovating new ways to apply AI into the world. And you know, I commend your effort in securing us and doing it securely.

Chris: So again, appreciate your time. You take care.

Louis: Thanks.