Episode 168 - Urban Augmented Reality with Iracema Trevisan
Designer, artist and researcher Iracema Trevisan talks to Max about the future of Augmented Reality in cities.
About Iracema Trevisan
Iracema Trevisan is a creative director with a multidisciplinary work that includes Illustration, textile design, augmented reality, and motion graphics. She founded her studio in 2015 and has worked on a wide range of projects using Augmented Reality in fashion and marketing. Prior to that she was part of the music band CSS, with whom she did several tours around the world between 2006 and 2008. (hence the wikipedia page)
In 2019, as part of a masters program in Digital Humanities she wrote a comprehensive analysis of the use of AR in the urban space.
Twitter: @Iratrevisan
Website (click this it’s got really cool grpahics): irace.ma
Links
Article: The Possibility of AR on the Urban Space
Longer Paper: Augmented Cities by Iracema Trevisan
Headmap Manifesto (1999)
Marsbot for Airpods (2020)
Related Episodes
Episode 142 on Marsbot for Airpods
Episode 79 with Timoni West on Augmented and Mixed Reality
Episode 71 with Tassos Noulas on Urban Data Science
Transcript
Max Sklar: You're listening to The Local Maximum, Episode 168.
Time to expand your perspective. Welcome to The Local Maximum. Now here's your host, Max Sklar.
Max Sklar: Welcome everyone, you have reached another Local Maximum. I am really excited today to talk about augmented reality, something that I've worked on at work and something that I've thought about for a long time. This is the kind of the heads-up display, is the idea of North Star, wherever we go, we get annotation, maybe even get visuals and audio that are added to what we see around us. Whether it's in our cities, or on our roads, or even in our homes, we're still waiting for the augmented reality future. And I'm going to ask all of you in the audience at the end, what do you think about augmented reality technology? If AR becomes a reality, is it going to be some dystopian mess where no one is living in the real world, and our perceptions are tightly controlled by governments and advertisers? Or are you excited about this new medium for art, for design, for experiencing the world around you, whether it's knowing your history or finding places and products you want. Or knowing where your friends are and what people you have trusted have said about a place where maybe you're walking by for the first time, but where they have been there many times before?
I'll tell you right now, if AR becomes a reality, and there are many researchers working to make it happen, the fate of augmented reality, and humanity's experience with it, is deeply entwined with the fate of the internet itself right now, where we're having a bit of an identity crisis with big tech. So, folks, it's all on the line. To start out, I really just wanted to nerd out on this tech, and my good friend Tassos, who was on the show for urban data science. We talked about, that's his field, we talked about that in Episode 71 back in 2019, he recommended that I read this article by Iracema Trevisan, called “The Possibility of AI in Urban Space”. Turned out that Iracema researched both the history of the idea going back to a 1999 article. People have been talking about this for a long time. I mean, annotating the world around us that was in, what comes to mind, I was in like Robocop and Terminator in the 80s. Of course, it was only the robots that had their heads-up display. But people should have their heads up display, too. Actually I think Robocop was human, though, sorry. I don't remember that. That movie was not as good as Terminator. I don't know my 80s movies that well.
But anyway, Iracema also talked about, in addition to the history, she talked about the concept of the mirror world, where we coexist with digital objects. That was a little creepy, but also interesting to me, kind of like there are ghosts around of objects that people are putting in our environment that you don't know about, unless you wear a special kind of glasses, or you click around with your phone. So it's kind of like an alternate dimension. She also, maybe more, not importantly, or, what we're going to talk about. She also talked about what researchers and developers are planning for our future in AI. So Iracema also has a really interesting background. She was born in Brazil, and in the 2000s, in the O's, she was part of a fairly popular rock band, CSS, hence her Wikipedia page. Did some world tours, and now she works in design and illustration and graphics. Her research on augmented reality is part of a master's program, which I, she's either in or I think she just completed. So I'm like, “Hell yeah, I want to talk to her.” So she's living in Paris, we dealt with the time differences, as I have several times before, and talked to people in Europe. And we had a conversation about this, all the links will be in the show notes page at localmaxradio.com/168. I'm not going to waste any more time. Now, let's go to Paris and get started. Iracema Trevisan, welcome to The Local Maximum, you've reached The Local Maximum. Welcome to the show.
Iracema Trevisan: Thank you. Thank you for having me.
Max: So, very excited to have you on the show. I see you have a Wikipedia page. And—
Iracema: Oh, yeah.
Max: I don't have a Wikipedia page. So I was like, “Oh, this person’s serious.” So, tell me about your background. How did you get into augmented reality? And we're going to talk about a whole range of ideas and the mirror world in a second. But how did you get into all of this? It seems like you're into technology, design art, but just just give us a little background?
Iracema: Yeah, sure. So um, I usually, I say, I tell people that I'm trying to fit as many lives as I can inside one life. So I have kind of like a really eclectic background. But so the augmented reality I came to it about six, seven years ago, completely by chance. A friend of mine sent me an article about it. And I was working in fashion, I've been working in fashion for quite a long time as a designer, and then later as a lot in marketing communications. So doing videos, doing prints, doing graphic design, doing all those things. And I saw this article and I discovered augmented reality, and I just, I went crazy with it. It clicked something on me, to have this possibility to animate things in real life. And I think that what really draw me to augmented reality is kind of like the reality side of it as well. The fact that you're not isolating yourself inside a virtual world, but you're mixing the digital and the real world. So that's kind of like the thing that clicked on me. So I started doing projects for, still in my field. So most of my clients, they're in fashion and beauty and those things, so I started doing projects for them, like animating, packaging, and doing face filters for Instagram and stuff. And what happens is that—
Max: Wait. Is a face filter what I think it is? Is that like—
Iracema: Yeah, like Instagram stuff.
Max: Okay, okay.
Iracema: So.
Max: Yeah.
Iracema: And like one thing that came to my mind, really in the beginning was this possibility to put AR everywhere. So, one of the first things that I thought was like, “Oh, my God, that's so amazing. What if we had like running little monsters all around Paris? And like sculptures and colorful trees and all those things.” And I had this crazy idea, like to build all those things. And then I decided to go research what was possible, what was not. Because I'm not, I don't have a technical background. So I kind of decided to put my head down and just go research all those things. And that's where the article that you read came from, and that's kind of where most of my research on AR came from.
Max: Right. And your article was about urban AR, which I mean, I'm not sure what the distinction is between, like urban AR and non-urban AR, but I'm talking, I think I assume we're talking about places that are in usually public places in the middle of a city or populated area.
Iracema: Exactly, yeah.
Max: Sort of like the UX case that Foursquare has, or—
Iracema: Exactly, yeah. And I bumped into Foursquare a lot in my research. So the idea of this paper was to investigate the use of AR in public spaces, exactly, anything that's out in the street. And the reason that I got interested in it is that we see a lot of adoption on AR and things that are quite more controlled, like face filters or placing an Ikea furniture in your living room, and all those things.
Max: So it's like you already have your picture. And it's like you already kind of know what you want to do with it. There's no surprises there, really.
Iracema: Exactly, yeah. And I think that when you go out in the streets, you're in an environment that it's completely uncontrollable. Like you can have a car passing by, you can have a person bump into you, and all those things. So it's kind of like, I think it's the hardest way to do AR in a convincing, in a more practical way. So that's the reason that I got interested in that particular use.
Max: So one of the things you wrote is that a lot of the problems with implementing AR if we want to have an augmented reality future as we walk around our cities, you said a lot of the problems are not technical, but human. What do you mean?
Iracema: Yeah. What I mean by that is that I think that people underestimate how difficult it is for us to have an interface that it's not 2D, an interface that it's not like a screen of a phone or a screen of a computer. And so, that difficulty is come a lot from the way that when we interact with the computer, we’re doing it in a really precise way, with an input that it's really precise and binary in a way. Like you click or you don't click, or like you have an exact coordinates in a screen or something like this. And when we're interacting with something that it's in 3D, it's a lot more complicated than people think it is. And the reason that it's complicated, it's super human.
Max: It's a lot more space.
Iracema: Exactly. Yeah.
Max: But for one thing, I mean, just that z-coordinate on its own, and that's not the only problem, but I feel like just that z-coordinate on its own, it's tough enough.
Iracema: Yeah, of course. So one thing that I found, and it's funny because I cite an interview that you did with Timoni West. And she, I think she explains it really, really clearly. Because she's really deep in the research and the people that really have their hands on this problem, like how to solve our interactions in a 3D space. So I think, I don't want to butcher her explanations. I think that if people that are really interested in that I would totally tell them to go listen to her explaining the problem.
Max: Yeah. Well, that's that's Episode 79. So I'm going to link to it on the show notes.
Iracema: Exactly. Yeah.
Max: —this one. Yeah. So, yeah, so let's talk a little bit about, well, there's so many things we could talk about here. I want to talk about, there's a social aspect to it. Because I think a lot of the simple ones you're talking about, like going from Instagram filters and IKEA furniture to the real world. Those initial things don't have a social aspect to it. I mean, I guess, the Instagram, you’re trying to push it to a social network, but it's not like you can change your face. And then all of a sudden someone else comes in and messes it up. So you found this article, I guess let's start with the background. You found this article called the Headmap Manifesto that was written in 1999. Is that correct?
Iracema: Yeah, exactly. Russell, Ben Russell. Yeah, so the first thing that I did for this paper was to go a little bit back in the theory of computing science and futurologist, and people that try to see how this augmented city future would be. And his manifesto, it's super interesting. So it's called Headmap Manifesto. And he has this kind of schema drawing that it's quite close to the Google Glass, actually. And some people say that it was a little more inspired.
Max: Almost 14 years before Google Glass came out.
Iracema: Exactly. And what I found quite amazing is that it's quite poetic, actually. He speaks of this idea of software taking our world and software changing our world. So it's as if architecture was not only for architects anymore, but it was also for software designers. And he cites a lot of ideas or situations that we could see, for example, with Foursquare. Like the Foursquare withGoogle notation and all those things. And the fact that you could go to places and you could see traces that a friend of yours left there. He describes this scene of going to a restaurant and seeing a huge scone that's floating around. It was a friend of his that went there and had food poisoning or something and just left a clue for him. And I found it quite interesting. He talks about this in a poetic way that I find quite interesting.
Max: Yeah, yeah. So it's not just social interaction. Foursquare, for example, I could leave a tip or something like that. But it's also real-time interaction in the locations of your city. And then, now we're talking about integrating. So I guess Foursquare circa what 2009, 2011. You just had that, that location aware social aspect. Now, we've been talking for a long time now about augmented layers on top of that, and that has become kind of hard to do but everyone thinks it's coming. Let's start with audio augmented reality because that's something that I've worked on, which we kind of put out as Marsbot for AirPods recently, where you don't have to actually take out your phone and read something. You could actually hear what someone has to say as you walk by the restaurant. In that case, I guess you'd be walking by that place and your friends would say, “Hey, I got food poisoning here. You could.” So by the way, if you guys want to try Marsbot for AirPods, I highly recommend you try it out and tell me how it goes. But you tell me, where. I think a lot of people say, “Hey, visual augmented reality isn't ready yet.” So people are, we're starting with audio augmented reality. How do you look at it?
Iracema: So I think it's kind of the logical way to go. Because one of the things that I, one of the conclusions that I got into this paper is that visual augmented reality in an unpredictable place as it's in the street, it's still quite hard. So for example, I have the example of Google navigation that they developed their AR navigation. That's super interesting. And technologically, it's super impressive. But you always end up with a problem that you can't have a split attention when you're walking in the streets because you can always get hit by a car or something, just bump into a trashcan or something like this. So they had like some walk arounds this.
And then if you think, if you start from that principle, if you want to have something that's supposed to be used while you're walking down the street, it makes so much more sense to have something that it's audio only, that it’s not visual. Because for one thing, a lot of the audio inputs, they're already solved. We have vocal assistants for a long time now and they got better and better. So we got to a point that like we can talk to Alexa, we can talk to Google Home, we can talk like all those virtual assistants. So they're already trained, like the technology is already good. And the other thing, it's the attention thing because one of the problems with AR is the cognitive dissonance that you have. Like when you're looking at your phone, you're looking at a rendering of something that’s superposed to what you're seeing in front of you. So there's this, there are several problems.
Max: It's like maybe sensory overload, like you just have too much you're looking at too much at once.
Iracema: Yeah, you have that and you also have your vision that's kind of truncated by the screen of your phone, which is something quite curious. Like if you, for example, the Google Maps navigation, you have the tendency to see what your screen is looking. But if there's something that's coming away from the field of vision of your phone, you might not see it. So there's a lot of cognitive issues with the visual AR the way that it is today. I think that all of this can change the way, the day that we have head-mounted device
Max: Yeah. So—
Iracema: But that's kind of far-off. So I think that the audio thing, it's kind of like the lower hanging fruit right now. And I think that there's a lot of potential because audio can be super immersive, and audio is such an important aspect of taking people somewhere else and making the experience more immersive. So it should not be understated, I think.
Max: Right? There's a lot to do there. I kind of wish we can work on this more, at Foursquare for example. But we're all waiting for the head-mounted Google Glass-type solution. You said that's pretty far off. How far off do you think that is? Do you think it's something that we're gonna have trouble seeing in the next five years? Or do you think, I know that there are some hardware attempts coming. But I don't know, I've seen some demos. And then, you're like, “Okay, this company is coming out with something.” I think I saw a demo in 2019. And then it's like, “Well, they didn't really work out.” So it's just like, people keep trying.
Iracema: I don't know, I have no idea how much further we are, how much far we are from those becoming a consumer product. I think it's kind of hard to give a date, because there's a lot of people working on this. And there's even more people now that they were like, three years ago. It's almost like everyone is working on this right now. There was, this week, quite a few news on that, on the subject that was Niantic that kind of hinted at hardware and everything. But I think that before we do, before we have the glasses, I think we're gonna have a lot more geo-located AR. I think that's something that might come before the glasses come. And there's a lot of people working on 3D maps, augmented 3D maps of the city. So I think that that might, like a plot, like a platform that allows for adding content in the city in using those 3D maps. I think that's something that will come a lot earlier than the glasses itself.
Max: Is that what you mean when you talk about the mirror world? Or is that something else?
Iracema: Yeah, we wrote it one way to talk about it that people that use the word AR cloud, even metaverse, it's kind of like there's a lot of different denominations. But the basic principle of it is to build a 3D map that's annotated of the real world, and then to use that map to place contents and to place information on top of the real world. And that's something that every big company is working really hard on it, right now. It's kind of like using the real world as a next platform.
Max: What sort of information do you foresee? What kind of use cases do you think would be really cool with these 3D maps, or metaverse, or mirror world kind of sounds cooler, but—
Iracema: I know but I thought, I find that just the idea of it, I find quite poetic, actually. But, in some way, it's what Google did with its street view and with its maps.
Max: Yeah, like 3D maps on Google Earth, and we've had that for a long time. They're pretty cool.
Iracema: Yeah
Max: I thought that those were just gonna get, because 10 years ago, those were pretty awesome. And I thought that they were going to get more and more detailed with their 2D pictures. But it turns out, no, they actually had a lot of the world to cover on that. Just that was a lot to take care of. But yeah, what UX case do you foresee? Or do you have an interaction that you, like a dream interaction? For these sorts of things? Like what kind of information would you get that would be particularly where you'd be like, “Yeah, this is working.”
Iracema: I think that one thing that I'm really looking forward to see is that we're seeing more and more games becoming social platforms. And for the last year—
Max: Yeah, you mentioned Niantic, so yeah.
Iracema: Yeah, I think that they are in this intersection that's really interesting. But I think that we've seen in the last years, all those metaverses like Decentraland, and all those platforms where you can play and you can socialize and you kind of have the game and the social media into one place. And they all have this idea that you're in a parallel world. So you can buy lands, and you can go see people, and you can build things and you can share things. And I think that that’s something really interesting if we start doing that in the real world as well, using these platforms, using this structure that's being developed today. So I think that that's the UX case that I'm dreaming of.
Because one of the conclusions that I got in my paper, and I'm sure a lot of people will not agree with me, is that the UX cases that are really gonna work are the ones that are a lot more emotional than functional. So I see it working with entertainment, I see working with games, with social media a lot more than I see it working with navigation, for example. Because I think that there's a lot of the really practical functional things that we already solved. And they’re so much easier to look at a 2D map, something like Google Maps, like it already works. And it's this kind of UX case that we want it to be fast and we want it to be effective. And I think that AR, it's so much better if we want to make things, if we want to make people feel things. So I think it could work with, for example, an art exhibition or social, like meeting people or leaving messages to friends or leaving tags to friends or something like this. And games. So I think that it needs to engage the user emotionally to a point that, not just like being functional.
Max: It's interesting you mention that because the current projects we're working on now at Foursquare Labs is a location-based game where you can buy and sell venues. So hey, maybe I'll get you on the tester list if you’re interested.
Iracema: Yeah, I would love to. Yeah.
Max: Yeah. So maybe now that you mentioned it, maybe we should include something where you could improve those venues and build things on them. That sounds like a new feature. We're still uncertain as to what the overall goal is. It kind of seems like we're trying to fit. We need some game developers here because it all seems kind of pointless. It's like, “What do you want one out of it?” But that's our main thing to figure out. But—
Iracema: Yeah, the future of social media is games, I think. I really think that the next big social media is going to be a game platform.
Max: Okay, cool. Yeah, that's good to know, it's good to know then. It makes me feel better about working on one because we're trying to build the game as like non-game developers. It's like, “What are we doing here?” But I think we're discovering, we're learning a lot. So, okay. We're talking about having these 3D models of the world. Are these going to be continually updated? Are they going to be aware of like, who's there? And is there going to be a privacy concern? I would be concerned these days about privacy and human rights even.If all of a sudden in this cloud universe, we know where everyone is at all times in the real world. Or we might be kind of far from that. But I don't know, maybe not in London and a few other places.
Iracema: Yeah, I think that there's a whole set of issues regarding privacy that will come up. But I think I find it hard to talk about them right now because they're all hypotheses. But I think that we started seeing some glimpses of those issues, for example, Facebook. They're working a lot on AR right now. They have a huge amount of people working on that. And they're launching a smart glass. They said in two years or something like this with Luxottica, that glass company. And I think the first thing that you think is that those glasses, they have cameras. And the whole thing with those mappings is that it's a virtual circle of people using it filling up data that will be used to update the map. So the map is always, it's constantly updated.
Max: And they have facial recognition too.
Iracema: Exactly. So the whole issue with those glasses is that they have a camera on it. So they're filming everything the whole time. And we don't know how that's going to work with the privacy legislation. For example, here in Europe, we have really restrictive laws that you have to tell people when they're being filmed and all those things. How do you solve that when you have something in your heads that it's filming all the time. And by the way, that was the big issue with the Google Glass, there was, their biggest backlash was with that. So I'm not so sure how they're going to solve this. They speak about designing things that would give a clear sign, pin sign for the person that is in front of it that they're being recorded or something like this, but I don't know. I have no idea how they're going to solve this.
Max: Yeah. All right. So we're getting to the end here. But well, let's talk about desirability. I think that's related to what we're talking about. What do you think is the urban AR future that is most, I don't know, most desirable, most human friendly? And what kind of, I don't know, just tell me, your - and I know it never works out the way it. If it's like, “I want the world to be like this in five years. This is my dream for augmented reality.” It almost never seems to work out that way, but rather than trying to predict, what would be the best way that this technology ends up? Let's put it that way. I think that's a good—
Iracema: I think that if we're talking about metaverse and AR cloud, and all those things, I think that the best result we could have about it, it’s something that resembles the web. The way that the web started, the way that it was open, and the way that it allowed for a lot of people to create amazing products with it. Which means something that's open, and something that anyone can go and build on top of it. I think that's kind of like the most important.
Max: Right. Open and permissionless.
Iracema: Yeah, exactly. And I think that that's something that a lot of people are conscious about. And they're trying to, like there’s this open AR clouds that it's a consortium of different companies that do mapping and scanning into the technology for those users. And they’re pushing for something that is open, like the web is open. But then you have also all the other big companies like Google, Facebook, Apple that are going further and further ahead and in this space, so we are not sure if this platform is going to be open or not. So I think that that's kind of like the main issue that I see.
Max: Yeah, yeah, it's interesting how sometimes it works. Like sometimes they build walled gardens. And then sometimes in podcasting, for example, no one's able to build a walled garden monopoly. So you never know, could work out. Alright, Iracema, welcome, or thank you so much for coming on today. Do you have any last thoughts on this? And also, where can people find out more about you? I'll link to the article, of course, and I'll link everything on the show notes page. But where can people find you online?
Iracema: So my websites, it's irace.ma with a dot before the MA, ma, M-A. I-R-A-C-E dot M-A. And people can find me on Twitter, @iratrevisan. And, yeah, that's it.
Max: All right. Thank you so much for coming on the show.
Iracema: Thank you.
Max: Alright, once again, show notes pages are on localmaxradio.com\168. Let me know what you think about augmented reality on our Locals page, www.maximum.locals.com. Will it become a reality? Yes or no, that's the first question. And then if yes, are you looking forward to it, or are you terrified? It's either, well, or both. I was gonna say it's either one or the other could be both. So next week, I'm looking forward to having a conversation with you next week. I really want to have Aaron back on for the news update. There’s quite a few things we can talk about.
There was one headline I saw that was, was it EU or was it Britain? I think it was EU attempts to regulate Bayesian inference. And I was like, “Oh, my God, we got to talk about that.” And there's some other stuff in the news. So I'm hoping we can do that. And I also have a fairly exciting announcement to share about expanding The Local Maximum. It's not, maybe it won't be mind blowing, but it's pretty cool. So I'm looking forward to it.
I also want to point out I was saying, I don't have a Wikipedia page, but I do have an IMDB page that was the Internet Movie Database. I haven’t been in a movie, but it turns out that they have been cataloging some podcasts. So if getting an IMDB page is the only thing I get out of doing Local Maximum, I think it was worth it. Have a great week, everyone.
That's the show. To support The Local Maximum, sign up for exclusive content and our online community at www.maximum.locals.com. The Local Maximum is available wherever podcasts are found. If you want to keep up, remember to subscribe on your podcast app. Also, check out the website with show notes and additional materials at localmaxradio.com. If you want to contact me, the host, send an email to localmaxradio@gmail.com. Have a great week.
