Episode 142 - The Newest App for Augmented Reality from Foursquare, Marsbot for AirPods
Reality as we know it is shifting. Technology advancement has made it possible to experience reality more deeply than what our senses allow. One new app for augmented reality is Marsbot, an audio-based virtual assistant that gives context to your surroundings. With the release of apps like this, augmented reality will be more accessible from here on.
In today’s episode, Aaron interviews Max talk about the launch of Marsbot for AirPods. Maryam Aly, a senior software engineer at Foursquare, also joins the show to discuss the tech and specs of the app. Finally, they talk about recent developments in augmented reality and their implications on our ever-changing world.
Tune in to the episode to learn more about Marsbot and the future of augmented reality!
Here are three reasons why you should listen to the full episode:
Know the process of developing Marsbot for AirPods.
Discover the features, benefits, and potential of Marsbot.
Learn about Gowalla’s upcoming app for augmented reality.
Resources
Introducing Marsbot for AirPods on Foursquare blog
Download Marsbot for AirPods on the App Store.
Gowalla is being resurrected as an augmented reality social app on TechCrunch
Related Episodes
Episode 86 on audio augmented reality
Episode 66 with Dennis Crowley on the planned return of Marsbot
Episode Highlights
The Concept Behind Marsbot for AirPods
It was supposed to launch last March, but it got delayed.
The app augments your walks and travels through your environment.
Marsbot is an audio-based personal assistant that will give you more context as to where you are and what you're doing.
Marsbot is an app “you don’t have to use.” It means you don’t have to search and scroll through it endlessly to find something.
It works even on old earphones.
Features and Benefits
The user will hear facts or fun trivia as they pass by certain places.
Users can leave short five-second voice memos for other people to hear.
If you walk by someone with Marsbot audio, it will say their name.
You can also choose your walk-on music, which people can hear if they go near you.
It also works while you're in the car, but it tries to avoid interrupting phone calls and overriding podcasts or music.
The Journey of Marsbot
Foursquare Labs has been exploring the idea of an audio-based virtual assistant since 2016.
It took a long time to get it off the ground, and a summer intern named Matt Schwartz had been a big help.
They worked on the geofence concept and then tried to get it out to the public by March.
Coolest Experiences with Marsbot So Far
Leaving voice memos is even more exciting once you experience it firsthand.
You can have fun with the voice memos and leave witty quips instead.
Restaurants can play their jingles as people pass by.
On the Issue of Overcrowded Voice Memos
At the moment, everyone can hear everything.
Fortunately, the content isn’t enough to bombard users yet.
There will be moderation and limits in the future as the user base gets bigger.
Interesting Challenges and Opportunities
Storing audio files was something Foursquare hasn’t done before.
Audio processing may also be a future challenge to improve the app.
Marsbot can potentially be a content platform explored by artists and musicians.
At the moment, users upload standard audio files, but it may evolve, and behavior trends may change in the future.
Download Marsbot to discover this exciting virtual assistant!
Gowalla and Augmented Reality
Gowalla is working on an augmented reality, gamified social platform for 2021.
Nintendo’s Animal Crossing was an inspiration for the concept.
Max speculates that it may be something like Pokémon GO, where there’s more emphasis on visuals than audio.
Technology giants such as Facebook and Google have a lot of failed but exciting experiments.
5 Powerful Quotes from This Episode
“We're sort of over the idea that you have to open apps, and scroll, and search, and find things. That is just so tedious, and everybody does it, and there's too much of it. And a lot of those apps devolve into casinos, like, if they're monetized, they become empty calories.”
“The appeal of this being an app you don't have to use is that you're getting away from unlocking your phone, and notifications, and part of that dopamine loop. But I guess there's also a balance there that you're essentially creating audio notifications, and you want to find the sweet spot of still being useful and interesting but not bombarding the user.”
“If this does sound compelling to you, I encourage you to give it a try. Let me know what you think. We're not trying to take over your life here. We're trying to add to your life.”
“This is not some kind of Facebook-style honeypot to try to get all your information or something. It’s just really cool technology, and we're trying to play with what the future is, so that's gonna be fun.”
“Interestingly, we're not really seeing that on the tech side, that it's a lot of small companies being acquired and then maybe doesn't really pan out on the big picture. It's a different approach.”
Enjoy the Podcast?
Are you hungry to learn more about the future of augmented reality? Do you want to expand your perspective further? Subscribe to this podcast to learn more about A.I., technology, and society.
Leave us a review! If you loved this episode, we want to hear from you! Help us reach more audiences to bring them fresh perspectives on society and technology.
Do you want more people to check out Marsbot and other augmented reality apps? You can do it by simply sharing the takeaways you've learned from this episode on social media!
You can tune in to the show on Apple Podcasts, Soundcloud, and Stitcher. If you want to get in touch, visit the website, or find me on Twitter.
To expanding perspectives,
Max
Transcript
Max Sklar: You're listening to The Local Maximum Episode 142.
Time to expand your perspective. Welcome to The Local Maximum. Now here's your host, Max Sklar.
Max Sklar: Welcome, everyone. Welcome. You have reached another local maximum. Aaron, Thanks for joining me again today.
Aaron Bell: A pleasure as always.
Max: All right, we have—I've got to cover this today because there's some very exciting stuff happening at—for me at work at Foursquare today. Today, we are—last week, right after the last episode, actually, we launched a new product, a new app that I have been working on for the last year, plus, it's called Marsbot for AirPods.
Aaron: Long teased and finally delivered. So major congratulations due there.
Max: Yes. Yes. I mean, we were supposed to launch it on March 11th, I think, and that just didn't work out.
Aaron: I remember you mentioning you were looking forward to a trip to South by Southwest. And that all fell apart, along with everything else and everybody's life.
Max: Yes, yes. But we finally got to it because life has to move on.
Aaron: Absolutely. So this product, Marsbot for AirPods? What’s the elevator pitch? What is it? Why are people going to be interested?
Max: Okay. The elevator pitch is that it is a—it kind of augments your walks and travels through your environment. And it's a personal assistant that will give you more context as to where you are and what you're doing.
And so let me give you a little bit of background as to why we built this. There is a whole blog post on this at the Foursquare website, which I'll post on localmaxradio.com/142. But basically, at the lab, here at the lab, where we work, we were interested in a few concepts that are kind of themes into what Foursquare does. One is like apps that you don't have to use. We're sort of over the idea that you have to open apps and scroll and search and find things. That's just so tedious and everybody does it. And there's too much of it. And a lot of those apps devolve into kind of casinos, like if they're monetized, they become empty calories.
Aaron: I’m getting a little bit ahead of myself.
Max: Yes.
Aaron: But you guys had started doing some groundbreaking in that same area with what I guess is now called Marsbot classic.
Max: Yes. So that was the first Marsbot that came out of 2016. And we on the inside, we at Foursquare, have been calling that Marsbot classic, which was text message based so it texts you at the right time. This one, you don't even need to, is all audio based, so you don't even need to receive text. You don't even need to open your phone. We just noticed that people are walking around all day with their AirPods. And so we do want to build a virtual assistant. And basically, we want kind of an augmented reality in the world. But right now, augmented reality is audio only because the glasses aren't out yet. But we thought, “Hey, why wait for the glasses? Let's do something that's audio-based. We can get almost there. People already have their headphones in wherever they go.” And by the way, this works even if you don't use AirPods. It's more dramatic. We felt if we said it was for AirPods because that was like the new technology. But it works if you have like an old style headphones.
Aaron: Yes, I can speak to that because I've been in the beta for I can't even remember how long now and I do not own AirPods. I've been using my same Chinese knockoff brand bluetooth headphones for ages now. And no challenges there. It seems to interface with that flawlessly.
Max: Yes, yes. So let me tell you a little bit about some of the features. You're gonna hear stuff as you walk by certain places and maybe I will play some of the clips here. But basically, some of them are like text to speech. So sometimes you'll go by and it’ll say, “This place,” and they'll name the place and you'll see, you'll like be walking around, you'll look up and you'll see it. And it'll be, “This place is the number one ice cream shop in the neighborhood, or maybe in Gramercy,” or whatever. The battery parts...
Aaron: Do you use the Siri voice for that or there's a separate voice synthesizer?
Max: We use the Siri voice. Yes. We could use a different voice. We kind of adjusted the speed of the voice a little bit to make it sound more natural. But yes. And so sometimes it'll say, “X number of people have been here over the last two weeks,” or “Other Marsbot audio users have been here.” It might be a fun fact about the place. Like we import a lot of stuff from the History Channel. Yes, and sometimes….
Two really cool things about it. One thing is it could be a short voice memo that somebody else left and you yourself can leave a short voice memo for other people to hear. It's five seconds. But I have access to the back end where we can leave stuff that's longer than five seconds. So, we'll see what happens there. Or it could be a sound effect or something. Another thing is if you walk by another person who has Marsbot audio, it will say you're walking by that person and give you their name.
Aaron: That reminds me of—I feel like in the early days of Foursquare, probably before the Foursquare swarm split, there was some features in there. Maybe it was after the split. But it would basically tell you, of your friends, who's within a certain radius and try to encourage you to—well, there was initially that feature and then there was another feature that was like to try and set up plans as well with folks which seems to have some common DNA with this.
Max: Yes, so this might say if I walked by you on the street, but I don't notice, it might say, “Aaron is nearby,” or it might say “Aaron is inside the coffee shop,” if I walk by the coffee shop.
Aaron: Although in this case, it's not limited to like a friends list or anything that anyone who’s using the Marsbot for AirPods.
Max: Eventually we're going to have to limit it, but right now, because it's such a small beta—and but everybody's invited, but it's still pretty small, we're not going to see that much of it. So it'll be fine. But that I assume will change pretty quickly, especially if we get, we have hundreds of users. Let's say we get thousands, that might have changed pretty quickly.
Another cool thing that you could do is you could set your walk-on music. So this is kind of inspired by—I don't know if they do this, I don't know if you've been to a baseball game like in Yankee Stadium or something.
Aaron: Not a major league game in a long time. But we do have a family tradition for my father-in-law's birthday, going to a minor league baseball game in New Jersey.
Max: So I don't know if they do this in the minor leagues, but what they have is each player kind of chooses their walk-on music.
Aaron: Oh, yes, absolutely.
Max: And yes. So basically, whenever they're at bat, a certain clip of a certain song chosen by that player gets played. In Marsbot for AirPods, you can select your walk-on music, where if anytime I walk by you, I'll hear that song. So for example, my walk-on music is a clip from the introduction of the podcast, just be, “And now, here's your host, Max Sklar,” so that every time somebody walks by, they'll hear that.
Aaron: And how long are these welcome clips? Are those also the five-second snippets?
Max: Yes, but that's very controversial. I think we're going to increase that. People are not happy with their five seconds.
Aaron: I could see keeping the walk-on short because then you don't have to worry about copyright. If you snippet small enough, it's all fair use. But...
Max: Yes, mine's longer than five seconds, just because.
Aaron: You have superpowers.
Max: I can do whatever I want. Yes, perfect. Now, we've discussed this. We've been talking about this on the show for a long time. Just to link back, we talked about this on Episode 86, when I went to the Betaworks Hearing Voices conference because we were working on an audio application. And that was kind of the beginning of that. Actually, when I went there, we thought we were almost ready to launch but we didn't realize the whole people can actually leave clips thing, at that point. That was something we discovered later on.
And I also discovered this in Episode 66, when I was talking to—I was interviewing Dennis Crowley, who is running Foursquare Labs and who I work with every day on this. And he was—I was not working at Foursquare at the time, he was basically trying to recruit me right there in the interview and he was talking about, “Hey, we're going to restart Marsbot,” in Episode 66. So this has been a long time coming for long time listeners in The Local Maximum.
Aaron: Now, one of the things that I think I saw in—actually, I'm not sure if I saw this in some of the behind-the-scenes discussion or if this was announced n the official release. But for a while, a lot of these features were at least heavily focused on locations where you guys were doing development, so New York City, and San Francisco, and some other places. But it sounds like a lot of this capability was more or less switched on for all locations. So I'm looking forward to seeing what that means for me, being not just—not in New York, but I'm not even in the greater Boston area. I'm out in the boonies.
Max: Have you gotten anything yet?
Aaron: I haven't been out and about with my AirPods on or—excuse me—my bluetooth headset on recently, but I'm going up to the car dealership tomorrow. So maybe I'll click them on for that and see how it goes.
Max: So usually, so this is kind of an insider trick, if you look at Foursquare and you look at places that are rated like 9.0 or above, if you walk by those places, you'll probably get something. So that's kind of the trick.
Aaron: Well, the other thing that I did see mentioned in the release was that it will also work driving in the car. So I may intentionally test that capability out tomorrow.
Max: Yes, if you have it plugged in, it's not likely because the car can be going too fast. But if the car is—okay, it's not like you're going to be driving down the street, it's just gonna be going blah blah blah blah blah blah blah. But if you're kind of screwed at a stoplight, it might chime in.
Aaron: So I won't expect anything while I'm on the highway. But maybe, while I drive through downtown.
Max: Yes. I, personally, haven't gotten anything in the car. But other people have. Also, we did a lot of work to make sure that it doesn't interrupt your podcast, or it pauses the podcast. It will never interrupt your phone calls.
Aaron: Yes, I was going to ask about that. Does it do kind of a fade out or does it pause or?
Max: So if it's a phone call, you don't want your phone calls interrupted, so none of that.
Aaron: Right.
Max: If you are listening to a podcast, it will pause, safe thing, and then get back to the podcast. And music, I think it will lower the volume of the music. So we try to do it in a smart way.
Aaron: So it can tell the difference between music and a podcast in that context?
Max: Yes. It's not perfect. There are certain applications where it doesn't do exactly what you wanted to do. But let us know. Either you can let me know it localmaxradio@gmail.com. But better yet, you should probably—you could use the app to send feedback. So for something like that, I would do that.
Aaron: Well, and I guess part of it is that now that it's going into wider release, that it's a public open beta, is that the right term? Is it still technically a beta?
Max: I don't even know. We just do our thing, man.
Aaron: Let's not focus on the technicalities.
Max: Yes.
Aaron: Now that it is open to the public.
Max: Yes.
Aaron: The user generated content will hopefully, dramatically increase the number of possible pings you could be getting there. So it should only get more interesting as time goes on.
Max: Yes, well, the one milestone that I personally would like to see would be walking around in the city and getting pinged that somebody that I don't know is using it near me. That would be pretty exciting.
Aaron: So I guess part of what I was wondering about when we started talking about the pings and what it does for other audio is, part of the appeal of this being an app you don't have to use is that you're getting away from unlocking your phone and notifications and part of that dopamine loop, but I guess there's also a balance there that you're essentially creating audio notifications and you want to you want to find the sweet spot of still being useful and interesting, but not bombarding the user.
Max: Right. I mean, personally, I've been recently purging my phone of many, many apps because I realized a lot of them are just a waste of my time and detract from my life. So we'll see how that goes. And so I'm hoping that this will kind of add to the experience of walking around or visiting a city or visiting a small town like, in Connecticut, I've been adding some. So hopefully it does that and as more and more content comes on, hopefully, we can kind of filter it in a way where it does that very well. But that is the goal.
Aaron: Awesome. So we mentioned Marsbot classic a little bit earlier. Is that, in fact, still running strong in the background? I am a user but I can't tell, because like I hinted before, I haven't been getting out and about that much.
Max: Yes, so that's the text message based one. You can actually text it back and forth which is pretty cool. That we put out in 2016.
Aaron: I do remember trying to test the limits of the...
Max: Yes, it's not very, it's just...
Aaron: Maybe calling it an AI is too strong.
Max: Yes.
Aaron: The system behind it.
Max: It doesn't pass the Turing test but I did put some good easter eggs in it. Let's put it that way. So yes, that is still going strong. Still download that and use it. If you search Marsbot in your App Store if you have an iPhone, you'll get both. Both versions will come up. One is Marsbot for AirPods. One's Marsbot. They're very different. Marsbot is focused on when you stop at a place, not when you walk by. Marsbot for AirPods is focused on when you walk by, although there's a few things that could happen when you stop at a place too.
Aaron: And once you've installed it and set it up, you don't have to open it to—it'll just do its thing in the background whenever you have your AirPods hooked up or your earpods.
Max: Yes, AirPods. You were right the first time. What's the earpod again? I don't even know. It’s so confusing. Yes, exactly. You can go into the app and see your history, see what you've heard in the past, and use it to create clips of your own. But yes, you don't need it.
Aaron: Awesome.
Max: All right. So I had a discussion with Maryam, you remember Maryam Aly? She's been on the show a whole bunch of times. And so she is kind of an insider at Foursquare. She works there with me. She didn't work on Marsbot audio. So I had a discussion with her yesterday morning about Marsbot for AirPods. And we talked about some more technical stuff and some more Foursquare insider stuff. And I thought it was pretty interesting. Although you could tell in the interview that I was a little tired, but I think it's interesting. So why don't we pull that up, alright?
Aaron: Let's give it a listen.
Max: Maryam, you've reached another local maximum. Welcome to the show.
Maryam: Thanks, Max. It's really, really great to be back.
Max: Yes.
Maryam: It’s been a while.
Max: I'm back in New York and I don't think I even interviewed you about your exile from this. Is that what you call it or?
Maryam: Exactly.
Maryam: I always say that I was “trapped” in quotes. Because I probably could have gotten back. So what happened, I flew to London, in early March, thinking, “Okay, I'll be here for a week and a half.” But then, the virus happened. And that month and a half turned into five or a week and a half turned into five months.
Max: Yes, that's very—a lot of people had a week and a half, whether it's traveling or something else, out of the office become five months, but...
Maryam: Right, exactly.
Max: See this? There's a fourth new Foursquare hoodie. I don't know if you saw this. But did you get one?
Maryam: I didn't get the new hoodie. I haven't gotten any of the new swag except the new...
Max: You got to come to the office.
Maryam: That’s the thing.
Max: Yes, I think they were sending it out to everybody. And they're, “Oh, you guys are here. So just take it.” So, I don’t know. Alright, so let's talk, look, Marsbot for AirPods. There's a lot going on right now in the world. But this should be very exciting. I keep having to remind myself, this is a big deal that we got this app out. And so, let's talk about it. I wanted to give you a chance, maybe, to ask me some questions about maybe what this app is and what we were doing. And I know you've kind of been in and out talking to us as we've been working on it. So go ahead.
Maryam: Yes. I know that you all started this project a really long time ago, but I don't remember exactly when. Can you talk more about when it got started and the journey?
Max: Yes. So I mean, it started in, I mean, well—I look back at 2016, when we released what we now call Marsbot classic, which was basically…
Maryam: Right.
Max: And this whole idea that we've been—what's the word I want to use? For some reason, the word that pops into mind is pioneering. But I don't think that's the word. We've been exploring the idea of having this passive agent tell you what's going on as you walk around your town or around your city. Or it doesn't have to be a city, but we kind of think of it as a city.
Maryam: So it’s like friend in your pocket.
Max: Right. So Marsbot classic came. We did, we launched that in 2016. And that was text message based and it would text you as you stopped different places. It would tell you what to order, what to do next, all that stuff. And Dennis Crowley, who most of the listeners will know if you're longtime listeners, because he's been on the show, he's the founder of Foursquare and he kind of drives a lot of these labs initiatives. He was kind of interested at the time in can you make this an audio experience?
And so I really, I couldn't get the job done for two reasons. One was because I'm not a front engineer. So all I can do... And then secondly, we really didn't have a formal labs team where we were dedicated to it.
Maryam: Right.
Max: So, actually, when I was out of Foursquare for the first half of 2019, on that dedicated labs team, they kind of started working on this idea of, “Can we do like an audio first AirPods version?”
Maryam: And when you say they, who is they in this case?
Max: Oh, that would be Dennis and a couple people on Labs at the time. So we had an intern named Matt, but I wasn't there. So I can't tell you.
Maryam: That’s fair. I was there and I didn't have the visibility at the time.
Max: Yes. But it took a long time to get it off the ground. I mean, when I came in I had to kind of relearn Foursquare's tech stack and kind of figure out what was going on with Labs. We had trouble getting a front-end engineer, unfortunately.
Maryam: Before you rejoined, I guess before you left, before you'd rejoin, you had been working on something completely different.
Max: Yes.
Maryam: You had been working on Labs.
Max: Yes, attribution, right. Yes. So finally, we got, we kind of had some iPhone engineer and people asked, “Why isn't this available on Android?” Because we don't have an Android engineer. That was like, okay, so we got it. Okay, maybe I could have learned iOS. But that would have been a tall order. That would have pulled me all over the place.
Maryam: Right. It's not your forte and that's okay.
Max: Yes. So we finally got someone great, Anish, who I work with every day. And he joined, I think, in November, and yes we were working on... Well, basically, the technology we're working on over the summer of, not this year, but last year, with Matt—now I want to get his name because I feel really like I should have his last name. So let me…
Maryam: Get his name. He deserves credit because he worked on this. And especially as an intern, it's so impressive to have, come in, kind of help a project from the ground up, get started from the ground up.
Max: So as Matt Schwartz who was an intern over the summer, and we were working on was some very interesting stuff, because basically, if I want to send you a message just as you're walking by, I'm trying to think of an example like a Shake Shack, or McDonald's or, maybe I’ll think of a better example because, I want to think of it like a unique cool spot that you're walking by.
Maryam: Yes. Like, you're walking through a park and there's art that you can't see from where you're standing.
Max: Okay, yes. Right. So we want to hit you as you're walking by. And what we do is, we build a geofence—not build a geofence—but there's like an imaginary geofence around it that the technology can detect when you get into that range. And it's kind of far. You wouldn't want to ping someone just to stay into that range. But what we do is when they enter that range, then they get tracked more regularly, more often. And so then we know exactly when they're walking by that spot, to hit them with some audio.
And at the same time, we did all this calculation to figure out, okay, “What angle are they looking at?” and “Are they approaching or are they leaving?” “Are they making a U turn?” All that kind of matters. So that was in the summer. Then in the fall and winter of this year, we were kind of working hard to get it out by March with a niche, like actually building it into an app, building content, and all that. And then we were about to launch on March 11th and the world shut down.
Maryam: Right.
Max: Backup recently, and I was like, “We really should get this out because what are we going to do, wait ‘til 2022 or whatever?”
Maryam: Right. And honestly, now's a good time anyway, because people are walking around even if fewer things are open. People are still walking around. And the idea is, it's not just going to work in New York City. There's more content in New York City, but I can go and create content in Westport, Connecticut where I'm staying a lot or any small town you can go and create kind of content. But it's sort of—we're still on the fence of what content we want to create.
Some of the automated content, which is like text-to-speech will say, “You're passing whatever, which is the number two ice cream shop in East Village.” And that's kind of interesting because you kind of know the places you're walking by every day, you kind of know what's a landmark, what's the best of the best, or what has kind of a History Channel tip associated with it. So there's a little history in there.
But I feel like there's more interesting stuff that can be done with it. And so that's why we kind of allow people to allow their own voice to come in. And you actually hear from us and from users who have left kind of short voice messages at different places, which right now people are trying all sorts of different things, but I feel like no one's nailed it yet.
Maryam: So what's the coolest thing that Marsbot has whispered in your ear so far?
Max: Oh, that's a good question. I mean, the coolest moment I would say was the first time I left a voice because we were originally just going to do all speech-to-text, like it was all just a text. And at some point, we figured out, “Look, I could just play an MP3. So why don't I record my voice.” I’m a podcaster. I record my voice. I record my voice and leave an MP3 and I'm like, “Okay, that might be cool.” But it was like ten times, hundred times cooler than we thought it was when we actually used it.
I mean, it's exciting when, especially if it's the voice of someone you know comes on and tells you they've been there or something exciting about it. I don't know if I've gotten one that stands out as amazing yet. But I did some demos in Fort Greene Park in Brooklyn where I used to live and I was like, “Okay, I'm going to leave.” We're meeting there in the summer. I said, “Okay, I'm going to leave a message at every single venue there.”’ A lot of the venues were trees, and I would kind of, like, “There you go now, coming up to another tree.” I kind of make fun of the—there's a big drop with no fence to get to the playground. So I was like, I was making fun of that. It was just—I was just having little fun with it. So I kind of feel like more stuff can be done in that.
Maryam: Yes. So there's some users, if you go to Fort Greene, you're going to have a very delightful experience hearing Max talk about trees and make jokes about playgrounds.
Max: Yes, exactly. I reminded people it's illegal to go into playgrounds, and adult without a child, and all that stuff. So that's—I feel like, I kind of had a benefit, where I could leave clips of any length, but the user-generated stuff is only five seconds and people are already starting to complain like, “Five seconds? Are you kidding me?” And they're also recording on their phone, which sound quality, sometimes, is not so good.
Maryam: Right.
Max: We've got to work on that.
Maryam: Yes.
Max: I don't know what the answer is.
Maryam: I'm going to switch gears a bit because this kind of just popped into my head. So as you get more and more content, especially concentrated in specific areas, how do you limit what you say to the user so you're not bombarding them with a lot of information at once?
Max: Yes, that's a good question. And we're still figuring it out. Because right now, everyone can hear everything. We can remove stuff if we have to. I haven't seen anything that bad yet. I've seen muffled stuff. I've seen people say, “Uh, I think, uh,” and then they're five seconds is up. And for some reason they published it. But I haven't seen egregious that needs to be flagged yet.
But I think, right now, there is not a lot of content on there. So you're not going to be bombarded. And if we have some heavy moderation system, then you're almost going to hear nothing. So right now, it's everyone, everything. That's not going to scale if this thing becomes very popular.
Maryam: Right. So at this point, anyone who signs up can leave a voice note?
Max: As scary as that is, Maryam. Yes.
Maryam: Is that the term? What have you been calling them? Voice notes or?
Max: I don't even know if we have a term.
Maryam: Snippets. Whispers?
Max: I guess snippets. Yes, we haven't called them whispers. Audio messages? That sounds a little too technical. I don't know, we’re going to have to work on that.
Maryam: We’re going to come up with something. We could brainstorm some fun names later.
Max: Yes.
Maryam: This has been a very complex project as you've already alluded to. So I'm wondering what are some interesting things that you've learned along the way?
Max: Well, there were a lot of interesting challenges. First of all, not that I learned, but that was interesting new was kind of storing the audio files, because we at Foursquare have never stored audio files before.
Maryam: Right.
Max: And it turned out it wasn't that hard. But I learned a little bit about, okay, you kind of want to have, we kind of have it all in one directory and the file name, the entire file, all of the data in the file is hashed down to a single unique code. That’s what a hash function does. And then the filename is stored as that code. Also, it includes the user ID because if two people upload identical audio files, if one deletes it, you don't want them to delete the other one.
Maryam: Right.
Max: It’s, I don't know, pretty simple. It's pretty standard computer science stuff. But I thought that was somewhat interesting. And you're working with Amazon to store. It was somewhat interesting at working with Amazon, but you know Amazon services, to store it was somewhat interesting. And a lot of the—we're looking at audio processing. Do we want to use services that try to clear out background noise? Do we want to use services that try to do text-to-speech? We haven't done it yet. The economics look interesting because a lot of them are, “We charge x dollars per hour.” But with five second clips, the economics looks pretty good.
Maryam: Right.
Max: So yes. That's going to be interesting as well. Technically, I think the most interesting stuff was really in terms of the angle of approach and the whole dual system where we have the big geofence and then we track you until you hit the geofence, trying to get—we're not really perfect yet. But the idea is we want to be able to mark an “X” in the sidewalk. And then right as you walk on that “X,” it pings you with something.
And in some cases, we've gotten pretty close. In other cases, we don't know exactly where the venue is or the venue is marked as somewhere where it isn't or GPS kind of goes off. But I've had some pretty amazing experiences with it. It's just sometimes you get something that's across the street and things like that. Which is good, but, we want to be able to know, which is going to be tough.
Maryam: Right, yes. So it sounds like this. It's not—you're talking about this particular hard problem that you've already solved. And then you've kind of alluded to some things to that could be future hard problems like how to moderate, how to add a filter for quality, and filter out.
Max: Or more exact—I mean, we also have venue list so you could just put something on any along if you're walking along the trail and you find something interesting. Yes. Also, getting the audio to play at the right time and the right speed.
Maryam: Oh. The right speed. Wow.
Max: Getting the text-to-speech to work. Well, getting the text-to-speech and the audio files to work kind of seamlessly...
Maryam: Right.
Max: …was kind of a tough problem. I basically had to send the binary of the audio file down directly rather than have a URL and have—so the iPhone, we don't want the iPhone to download the message and then download the mp3 file separately. It's all in one shot is kind of an interesting way of doing it.
Maryam: I do see. So you were just talking about someone, kind of walking down a trail and hearing something.
Max: I'm excited about that. Because we've all been—all of us who live in the city have kind of wanted to get more out in nature recently.
Maryam: That's so true. It's really, so true. I think there's a potential for things that aren't venues to—I mean, which like you said, with the trails, but even in the city, things like street art, like getting more inside to particular street art. Like, “This artist is also featured at 45th and 7th if you want to see more of their work,” or something like that.
Max: Yes, exactly. I think somebody could also do something very interesting with sound effects. I'm not exactly sure. But there could be some artists or musicians that do something like that. I have the idea that this can be done but I don't really know what I would do in that area.
Maryam: Right.
Max: But it's kind of a platform that there's a lot that can be done with this, that can be explored. And so I want people to kind of think about this. If you have an idea, do it. And if you need more than five seconds, let me know and tell me what you're doing because I might be able to get you a little bit of a workaround there.
Maryam: You heard it here first.
Max: Yes.
Maryam: Max can get you the secret key.
Max: Yes, that's true.
Maryam: Unlimited power.
Max: Yes.
Maryam: I think it's really interesting because there are so many other short form creative things on the internet but none of them have been purely audio. You think about Tik Toks, right? Or vines? Really, really short videos that people have just taken to a whole another level. So much great art. I think you're right. There's so much potential here.
Max: Yes.
Maryam: More artists on the platform.
Max: Yes. Any medium that you have, there's always going to be people who take the medium like I take to podcasting. I tried blogging, I wasn't really up to it. And so, I feel like there's going to be some super users here who are going to be like, “Yeah, this is, I really want to create content on there.” There's always people. So I'm looking forward to finding.
Maryam: Well, if you had one request for listeners right now, what would it be?
Max: Go download Marsbot for AirPods if you have an iPhone and you can check it out. Basically, just go to your app store and look at Marsbot for AirPods. And yes, if you have any questions, email the show, localmaxradio@gmail.com. All right. I think this has been a good discussion. And I'm really excited to get this out. And Maryam, do you use it? You have an iPhone?
Maryam: I do have an iPhone and I do have it downloaded, although I don't have the App Store version downloaded yet so I need to do that.
Max: Okay. It's going to be similar. Have you heard anything interesting yet?
Maryam: Honestly, I haven't been out enough in New York. But before the office closure, I did get a nice message about Magnolia Bakery. Yes. I didn't know that there was one inside Penn Station, actually.
Max: Wow.
Maryam: Yes.
Max: It's pretty cool. All right. Excellent. Fantastic. All right, Maryam. Thanks for talking to me today and coming on the show.
Maryam: Thanks, Max. Thanks for having me
Max: All right, we're back. Pretty cool stuff. I think that does it for Marsbot for AirPods. If you are not convinced to download this right now or complain that you have Android and you can't download it, then I don't know what to tell you. But no, if this does sound compelling to you, I encourage you to give it a try and let me know what you think.
And look, we're not trying to take over your life here. We're trying to add to your life. So if it doesn't work out for you, it doesn't work out for you. But we're really trying to—this is not some kind of Facebook style honeypot to try to get all your information or something. It's just really cool technology. We're trying to play with what the future is. So that’s going to be fun.
Aaron: Before we jump on to the next thing, I have a question for you.
Max: Yes.
Aaron: So this, there may not be an answer to this since you're so deep inside of it, but have you heard something while using Marsbot for AirPods that surprised you?
Max: That surprised me?
Aaron: Or has it all been because you're involved on the back-end, nothing has caught you off guard?
Max: No, I pretty much, I wrote all of them. So no, not really. I mean, the last few days, I've been listening to all the clips that people have been adding. And I've just been listening to them over and over again. And I feel like people are going to change behaviors over time. Right now, they're kind of just doing the standard Foursquare stuff like, “Hey, this place has good burgers, check it out.” But I feel like people are going to shift over time and do different things with it.
Aaron: They maybe haven't really realized how open-ended the system is, at least for now.
Max: Yes, one—you could also do something that plays—you can't do something, but we can on the background. We can do something that plays every time you pass an instance of a certain chain and we have a little whistle that we do. It's a very short whistle that you almost don't notice every time you pass a McDonald's. And so it's interesting. We're just playing around with it to see what that feels like. And it's been kind of fun because I heard the whistle and it kind of makes me smile. I look up there's the McDonald's.
Aaron: I did see and I think it was in the article on the Foursquare website. But there was a link to Dennis's tweet, trying to get permission from McDonald's corporate to use their actual audio sting rather than one that is not a trademark.
Max: Yes. Well, we were practicing, rehearsing. We were testing with their whole da da da da dat da. And it was actually much better. But yes, we can't do that. The legal department was very upset and we had to change it.
Aaron: So this is not a sponsored thing that they've paid to have inserted into the…
Max: No. We can't even get them to let us use it for free. They're literally like, “You're not important for us to sign a document.” So, that's that. All right. So, interestingly enough, we launched this last week. It's gone very well, technically. I'm very happy with the launch. I'm happy with the users and the feedback we've been gotten even from the small number of users that we have.
Interestingly enough, on the same day that we launched Gowalla—which is an old company people might not remember—Gowalla announced that they are working on a location-based augmented reality game or app for 2021. And Gowalla is kind of one of Foursquare’s—I don't know—twin companies. It came about kind of at the time as Foursquare. It had some of the same features. They ended up getting bought by Facebook, which is, couldn't have been that much fun for the founders. Josh Williams is the founder of Gowalla. Now he's restarting it up. And he's actually friends with Dennis. So we're kind of seen as competitors but they certainly know each other.
Interestingly enough, they did not know—we did not know they were working on this, and probably vice versa. And we just happen to have the announcement at the same time, which is pretty cool, even though it's a long way…
Aaron: Goes to show that this kind of augmented reality is perhaps entering into an AR summer.
Max: Yes, an AR summer, that could be the name of a future episode if we talk about AR sometime in the next year, which we almost certainly will. So I don't really know what they're planning at Gowalla. But it says here, “Williams was cryptic when it came to details for what exactly the new augmented reality platform would look like. And when it launches, he did specify that it will feel more like a gamified social app than a social game. Though he'll also listed Nintendo's franchise, Animal Crossing as one of the platform's foundational inspirations.”
So I'm thinking something like Pokemon Go. I don't really know the difference between a social app and a social, a gamified social app and a social game. I'm not really sure what the—well, I guess a gamified app would be something more like Foursquare where you check in, you get utility, but you win apps and badges, right?
Aaron: I guess so. Yes, I'm…
Max: Pokemon Go would be…
Aaron: The terminology there is not crystallized enough for me to clearly separate those two.
Max: Yes.
Aaron: Although from the way they're talking about, it sounds like this would not be an audio first app in the way that Marsbot for AirPods is. Still in the AR space but much more likely to have a graphical aspect.
Max: You have to hold up your phone like Pokemon Go. But maybe they'll find that audio is way more interesting these days. So, we'll see.
Aaron: Unless they have the inside track and they know that Facebook is going to be coming out with the next generation of Google Glass, that they're trying to resurrect a technology in that space.
Max: Well, there's a lot of rumors about that, predating COVID. But yes.
Aaron: They've dumped so much into the Oculus brand, which hasn't really seemed to pan out in the widespread way that I think they were initially imagining it would.
Max: Yes, I mean, we did a whole episode on the Google Graveyard, stuff Google tried to think they were going to take over the world didn't. Soon, there's going to be a Facebook graveyard because they've got Libra. They've got Oculus. Oculus is actually, could do pretty well, just taking a lot longer. But we'll see.
Aaron: Yes, I know that there are other industries where you'll see there’s a particular company that they do a really good job taking innovative ideas and prototyping them, but they don't have the size and the power to bring it to the mass market. And so you will very typically see like 18 months to two and a half years after they come to market with a product, one of the big names in the business comes out with a much more polished version of that same product, but at mass market production levels.
And interestingly, we're not really seeing that on the tech side that it's, a lot of small companies being acquired, and then, maybe doesn't really pan out on the big picture. It's a different approach.
Max: Yes. Alright. Are you…
Aaron: Perhaps we're about to see a turning and how that all looks.
Max: Yes. Are you ready to go on to talk about next week's episode or we have anything else on this?
Aaron: Yes, I guess so if we have to. Okay. Next week.
Max: Well, okay. Look, next week's episode is coming out the night before the election. So the evening before the election. So we've got to do something election-related and then for those of you who roll your eyes, it'll be over finally. That's the last one.
Aaron: Maybe an election that does not age well. We shall see.
Max: Yes, surprisingly, our episodes have aged pretty well when I go back to them. But we're not Nostradamus over here. But well, even Nostradamus doesn’t—that's a whole other thing. But we haven't said that so incredibly bad. I'll step in it one day. Someday, I'll make a prediction that's so incredibly shortsighted that it'll look really bad like a week or two. But we're all about forecasting with accountability here. So we do a little bit better.
But next week, we're going to have some election related news. I want to give people a little bit perspective on this election, just so that you don't lose your minds and find some of the maybe lesser known stories that are going to be interesting to you, but maybe are not related to the doom scrolling that you're probably doing right now, regardless of what side that you're on. So I'm going to—that's going to be my goal. And we'll see if we meet it.
Aaron: Tangentially related to that, there is still time for folks to jump on to the 20/20 insight Forecasting Contest, which I don't know if we've mentioned that on the show before. But it's a joint effort between Metaculus and I think some other research organizations. So if you're listening to this before, I think it's the 29th of October. There are some election-related questions in there which is what brought my mind to it, but there's a chance to go in there. And if you can make predictions and submit your justifications for those predictions, you could gain some prize money.
So if you want to take your Bayesian skills and put them to the test, this is—I'm just starting to get into some of the questions there but particularly relevant for what we've got coming up next week would be... Let's see, for example, question one is, “Will either Joe Biden or Donald Trump can see the 2020 presidential elections by November 17?” And there's some more interesting stuff in there.
Max: Don’t worry about that. So many people, yeah... This could age very poorly, but I'm just saying.
Aaron: But particularly because, not only are they looking for folks to get the predictions right on these, they're looking for folks to show their work. So that's a little bit of a different methodology and I think some of the things that we've talked about on the show in the last 140+ episodes could serve you well there.
Max: Alright, great. So we'll follow up with that next week, everybody. Have a great week, everyone.
That's the show. Remember to check out the website at localmaxradio.com. If you want to contact me, the host or ask a question that I can answer on the show, send an email to localmaxradio@gmail.com. The show is available on iTunes, SoundCloud, Stitcher, and more. If you want to keep up, remember to subscribe to The Local maximum on one of these platforms and follow my Twitter account @MaxSklar. Have a great week.
