Episode 129 - What’s New in Language Translation Tech with João Graça
The world is connected now more than ever. We are free to access websites and places across the globe. In doing so, most of us have dealt with language translation tools to read online content or strike up conversations during our travels. While these tools have been helpful, you may have noticed they are still far from accurately translating complex human contexts, inflections, and sentiments.
In this episode, João Graça, CTO of multilingual translation platform Unbabel, talks about the past, present, and future of translation technology and the role of machine learning. He also discusses language translation tech’s applications and how they can still improve.
Tune in to find out the latest developments and challenges in automated translation.
About João Graça
João Graça is the co-founder and CTO of Unbabel, a multilingual translation platform working to tear down language barriers around the world. He has a doctorate degree in Natural Language Processing and Machine Learning and has authored several papers in machine learning. João also co-founded the Lisbon Machine Learning Summer School.
Check out his blog posts and LinkedIn to know more about his work.
Here are three reasons why you should listen to the full episode:
Discover the difference between natural language processing and natural language understanding.
What are the challenges faced by automated translation?
Learn why some languages are more difficult to translate than others.
Resources
Ronald L. Graham, Who Unlocked the Magic of Numbers, Dies at 84 on The New York Times
What is Graham's Number? (feat Ron Graham) by Numberphile
Translator Fails on YouTube
Google Translate Sings: "Africa" by Toto by Translator Fails
Related Episodes
Episode 23 on Natural Language Processing with Kris Conception
Episode 2 on Internationalization with Maryam Aly
Episode Highlights
What Is Automated Translation?
It is any system that can turn a given statement from one language to another.
The deep learning model Transformer is the best model there is so far.
Initial models in the 1940s and 1950s generated translations solely at a word level, with no consideration of context. It is taking decades of research to address this challenge.
Latest Breakthroughs
João doesn't think we've solved any problems we couldn't solve before.
Nonetheless, more useful applications were developed to address them better and at a higher performance level.
The Pain Points
Language is ambiguous. Some words have multiple meanings.
Also, the ordering of words in a sentence varies in every language.
The search base is also enormous because of the sheer number of words in a given language. Models must be constrained to search locally.
Even the best models today treat every sentence independently; in other words, they do not capture tone, context, and sentiment.
Low-resource languages also remain an unsolved problem. For example, if you want to translate from Portuguese to German, the model will most likely translate first from Portuguese to English and then from English to German.
Shortcomings of Machine Translation
Machine translation still produces a lot of critical errors.
It’s unfit for business communication, with some exceptions. For instance, you can use it in emails but not for your brand as a whole.
The new models are much more fluent. Their translations make sense; sometimes, however, they do not reflect what was said in the source.
Processing vs. Understanding
Natural language processing is running statistical algorithms over text, while natural language understanding is forming a semantic representation of the true meaning of the text.
Natural language understanding is the holy grail. However, in reality, it’s hard to make computationally efficient models for it.
Why Some Languages Are Harder to Translate than Others
The first difficulty lies in the word order. It’s fine if the two languages have the same order (e.g., English to Spanish). But it becomes more challenging when reordering is involved (e.g., English to Japanese).
The second difficulty lies in the inflection (e.g., gender, number) of a word.
Measuring the Accuracy of Translations
Automatic metrics score the quality of a translation. They compare the distance between the automated translation and the correct translation.
Multidimensional quality metric (MQM) is the industry-standard measurement, in which professional linguists look at the source of translation, mark all the errors, and group them into three categories: minor, major, or critical.
The score can range from 0 to 100; above 90 is considered professional quality. Having this measurement enables them to see improvement over time.
Applications in Society & Business
The technology is fit for simple conversations during traveling and first responders during catastrophes.
For João, the available technology is not yet entirely useful for the translation of websites or blogs.
João hopes future developments will allow a more inclusive world where we don’t stop and think about language issues.
Successful Techniques for Machine Learning
Deep learning is the universal hammer.
Unbabel mostly uses Python, except for machine translation.
For machine translation, they use a module on the C++ because it is more efficient for coding.
5 Powerful Quotes from This Episode
“I don’t think we’ve solved any problems that we couldn’t solve before. I think we’re solving them slightly better.”
“In dialogue, it can be useful because you can recover from the errors.”
“That (natural language understanding) is the holy grail since the ‘50s, specifically the triangle where you go from a source language to interlingua and then you generate.”
“For that kind of scenario, like traveling, easy conversations, the technology is fit for purpose. It might take you slightly a little bit longer, but it's definitely easier than not having it.”
“Think about it: you go to some websites, you go to some mobile app, and you're just talking on your language. You don't even stop to think that there's a language issue.”
Enjoy the Podcast?
Are you eager to learn more about automated translation? Do you want to expand your perspective further? Subscribe to this podcast to learn more about A.I., technology, and society.
Leave us a review! If you loved this episode, we want to hear from you! Help us reach more audiences to bring them fresh perspectives on society and technology.
Do you want more people to understand language translation tech? You can do it by simply sharing the takeaways you've learned from this episode on social media!
You can tune in to the show on Apple Podcasts, Soundcloud, and Stitcher. If you want to get in touch, visit the website, or find me on Twitter.
To expanding perspectives,
Max
Transcript
Max Sklar: You're listening to The Local Maximum Episode 129.
Time to expand your perspective. Welcome to the Local Maximum. Now here's your host, Max Sklar.
Max: You reached another Local Maximum. Welcome to the show. Last week, I played the first part of my summer news update with Aaron on what's happening in our cities. And boy am I going to be interested in that as I sit here with all my boxes and bubble tape all around me ready to move to Manhattan. Next week, we're gonna hear the rest of that conversation where we talk about the troubling political environment in science and engineering, and I'll tell you about it after today's interview.
Do you remember a time before you could translate sentences online? It's been a while. It's hard to remember that, but every time you got a document or phrase in another language, I guess if you're younger than, I don't know, if you're younger than 25, you might not remember this time. But there wasn't that much you could do about it, like, now we just take it for granted that you could just catch the meaning. And the tech has developed further ever since. But yeah, you got a document in another language, you're kind of stuck.
Not anymore. Today's topic is translation technology. I know we've talked about internationalization before, which is, you know, translating apps into other languages. But that's not necessarily automated. And we've talked about natural language processing before, but we haven't talked about translation yet. So that'll be fun.
In terms of emerging technology, this particular one is a case study on something that gets built up. Well, I don't know how to say it nicely. But translation technology gets built up excruciatingly slow over decades and decades. So, year to year, people are skeptical about whether we're making progress or not. But when you look back, it's really quite a lot, but the path from here to human-level understanding sometime in the future or real text to speech, like on Star Trek, that is a long path. So I want to talk about that today.
But before I get into that, I want to share a news item in my clippings—which are really just links that I sent myself—something that sometimes you find interesting stuff in the obituary section and this one is mathematician Ronald Graham, who died at 84. Now, probably not many of you have heard of Graham, but if you read the article, he's done math. He's had results in a lot of fields, but the one that captures the imagination—my imagination at least, probably others—is the number that he came up with. It's called Graham's number. And it's in the Guinness Book of World Records as the largest number, largest non-infinite number. Now obviously you can't have a largest non-infinite number, but there are roots because you could always say, “Hey, Graham's number one. Get me in the Guinness Book world records.” But, you know, you can't do it. So people come up with rules like, “No, it actually has to be a number that represents something meaningful.” So he came up with Graham's number, which does represent something meaningful. It's not usable, like an engineering sense in any way. I mean, the number is so huge, it almost gets meaningless at a certain point. When you talk about these huge numbers in terms of actually having something of that quantity, it just doesn't exist. It's more like that you're at that point, you're more talking about the complexity and describing the number, which, in this case, you actually have to sit down. And if I were to explain to you the number and have to sit down with someone and explain it to them for 10, 20, 30 minutes by a whiteboard or chalkboard in order to find the number, you can't just read the [laughs]. Even the number of digits is an astronomically high number.
So the problem that this number came up with, it was a graph theory problem. And it's essentially asking like, “What's the largest number for which this result must be true?” or “What's the number after which this result must be true?” The problem itself is actually kind of simple. You can learn it in a few minutes, you can learn it on Wikipedia, or you can link to the New York Times article that I found. So it's not totally inaccessible to an average person. It's more like, think of a graph, which is like a bunch of points connected together and coloring the edges in a certain way. But, yeah, when you're talking in terms of theoretical mathematics, and you're like, “When can this coloring be done in this way?” or “When can it be done this way?” and you want to know how big your graph has to be, some of these things come out to, it's not infinite. It's not small, but it's some ridiculous, ridiculous number. So really interesting. That's a really interesting thing. I would like to know, like, what the more practical applications of these numbers are, but certainly the problem itself is not totally impractical. But maybe we could talk about that another time.
All right. My next guest has many years of research and practical experience in the field of automated translation and is founder and CTO at Unbabel, a company with a multilingual translation platform working to tear down language barriers around the world. João Graça, you've reached The Local Maximum. Welcome to the show.
João Graça: Thank you.
Max: Alright, so your company is called Unbabel. You do automated translation. So first, I just want to know: what is the state of the art in automated translation these days? Maybe define automated translation for people who don't know what it is and what sort of problems are you working on?
João: Yeah, so basically automated translator or machine translation is basically any system that, given a sentence in one language, it can translate in other language. It's actually a very old problem. It started right after the Second World War where after [unintelligible] the German call, they decided that they could do the same for translation for language. So it's actually one of the first flaws of machine learning and natural language processing yet to be solved, and right now is pretty much everything in the field. The best model is a deep learning model called the Transformer. It's pretty much where we are.
Max: It seems like an automated translation system from the ‘40s and ‘50s couldn't be very good. So like, what were they trying to do?
João: No, it was not very good up to a point that there's a famous report that completely kills all the research in that area because the results were really bad. But it was very simple. So we just go based on the noisy-channel law. So the idea is like, every time I'm speaking something in French, I’m actually saying, or in English in this case, I'm actually saying something in Portuguese, and it's getting to, the noisy channel is like corrupting the signal, so they have to recover your original Portuguese and the model that existed was like, it would go word by word and say, “Okay, this word in English, what is the most likely translation in Portuguese?”
Max: Right.
João: Words in English and it was basically like word-level. And there were some constraints about what was the probability for word falling the other one. So those are like the first models, which are not that different from the ones that we have now, just better at representing words, in representing context.
Max: Right. So, it's kind of like you could look up each word today to a dictionary, like an English–French dictionary, and you can just write what the word meanings are side by side. So that's pretty easy to automate. But then when it comes to the meaning and the direction of the word, the problem is that like trying to find out the word and context, then it sounds like that's going to be much harder and that takes us into decades of research. So, what's the state of research now? There have been so many breakthroughs in machine learning or at least machine learning research kind of becoming mainstream over the last decade. So, what problems are we starting to solve now that we couldn't have solved maybe in 2015 or 2010?
João: Yeah, I mean, so I think that I'm not being cynical. I don't think we've solved any problems that we couldn't solve before. I think we're solving them slightly better. So, the performance level of most of the approaches in solving increased and then they start being, like, more useful applications for the problems. So, there is, like, a problem: “Okay, this was like a new form. It is like completely solved. You haven't seen it.”
Max: Right. So, what are the big pain points in machine translation that are tough to solve?
João: First of all, language is super ambiguous. So, if you think about a particular word like the word “bank” can mean very different things. Then when you translate to another language, that word ordering is different. So in Japanese, you have the verb but yet not in the beginning. So we not only have to translate, but I have to figure out how to reshuffle all of these things. And then language is, like, it's very ambiguous. So sometimes you have words that mean different things. Sometimes you have expressions that by themselves mean a different thing than the sum of the words. So that will still be charged. And then the search base is huge because of all the different words that you have on a given language. So you have to constrain your models to look very locally. So for instance, even now, the best models for machine translation, they treat every sentence independently. So it's like you put something on Google Translate, and the tone that you get for the sentence is falling formal will be different if you have a pronoun.
Max: Right. So it wouldn't know at all what the pronoun is talking about, and it wouldn't know what the conversation is, and sometimes the context of the conversation changes the meaning of it, even if it's not in the same sentence.
João: Completely. And then, I mean, it can’t capture, like, the sentiment of what you're talking about the domain. So there's a lot of limitations, and this is just talking for the main language. It’s where you have a lot of parallel data to the system. If you're not trying to train the system to Swahili, you don't have that. So low-resource language is pretty much an unsolved problem on natural language processing. Or if you want to do something like Portuguese to German, you're most likely going to the Portuguese to English, English to German just because of the availability there.
Max: So can you get compounding errors when you do that?
João: You get a little bit of the noisy channel, the broken phone deck.
Max: Right, right. So there are a couple things I want to ask about in terms of translation to Swahili. That's interesting. So you said there are some languages where we don't have that much data, but we certainly have some data. We have human translators in those languages. Do you think there's a way to build algorithms that could actually build decent translations? Even when there are less data? Is that something that could be researched? Or do you think it's more of now the only way out of this is to just gather more data?
João: No, I mean there's several answers to that question. So the first one is, yes, it's very little data. And I can tell a little bit about the project we did for crisis translation. You can do models. The project is deep learning models. They require a lot of them. You don't have to do feature engineering for the models. The reason you don't have to do it is because you have more data, and they can learn these features. So you really have lots of data. And what you do here is, you have research on doing these zero-shot models where basically, let's say it's when the models translate from one language to ten different languages. And so what you're trying to do is to compensate, it's “Okay, I’m going to learn to translate from English to Swahili and three other languages.”
Max: Right.
João: We try to learn some language and universals. So now I can do a better job in Swahili without it much there. And they don't work that well, but they weren't better than not having that at all. The other end is like if you really have a small amount of data, you’re basically better off potentially getting a statistical system, previous ones, because they'll require less than, but there's a very active area of research, like this is where, like, Google and Facebook are putting a lot of effort.
Max: Into statistical systems?
João: No, no, no, into companies like low-resource languages.
Max: Oh, low-resource languages, right. I mean, they're all statistical systems. So okay, what applications for automated translations would you say are, like, yes, we're dead on, we get these applications right now. And are there any application that people want to use this for and then they look into the state of the art and they're like, you know, that the way automated translation is now with in terms of its accuracy, it just doesn't work for this particular case. Are there any cases like that, like, where are we nailing it? And where is it? Like, you know, it's not good enough yet.
João: I mean, so it's mostly like machine translation right now. So, there's a lot of critical errors. So it's definitely not fit for purpose for, like, business communication, with some exceptions. So for instance, it's very useful for you to go, personally travel to a different country and just use a device and then you can, basically it's better than just talk. So, you know, in a dialogue, it can be useful because you can recover from the errors. So, you know, it’s something like if they don't understand, they say, “What?” And we say it’s a different thing, and you kind of build up on the conversation.
Max: Yeah, I've used that for traveling for many years now, almost like eight years probably. It's been good enough to, like, ask the taxi driver where to go and things like that.
João: But if you try to explain to the taxi drivers how you feel about life because your girlfriend broke up with you, you want to machine transition. It's like needed to that simple kind of scenario use case. And it’s important to understand that the digital space to do that, an email, which is something we will offer customer service, the quality is similar in most cases, so you can understand what the email says. But as a company, you don't want to expose your brand with that translation. So that's where human corrects the errors.
Now, this helped humans a lot. It's been improving a lot. There's also something very interesting which is, with this new model the translation become much more fluent. So they actually seem like produced by a professional writer. But sometimes they just generate gibberish. So they generate something which is not also the source. And then it becomes much harder to understand because if you're just reading, it kind of makes sense. It's just not what was said in the source. So that's an interesting challenge with these new models. But overall, we know emails, you can get along with. These personal travel, get a certification, you can get along. Emails are getting better, but there's a lot [unintelligible]. So if you think about you go to hotel, anyone to talk with the host in different languages, you can start using this kind of technology and don't have to speak English.
Max: Yeah. Okay, so one distinction that comes up is natural language processing versus natural language understanding. And, you know, from my point of view, I kind of see natural language processing is kind of just largely running statistical algorithms over text. Nothing wrong with that; I do it a lot. I've been very successful at it in the Foursquare text corpus to try to build ratings or sentiment and stuff like that, pull out key terms. But I see natural language understanding is where you actually build some representation of the true meaning of the text, some semantic representation. Do you think that natural language understanding is necessary for translation to be done right? In other words, do you think there has to be some kind of a translation from English to some representation of meaning and then some representation of meaning over to Portuguese or some other language?
João: I mean, so that is the Holy Grail since the ‘50s, specifically the triangle where you go from a source language to interlingua and then you generate. It will also make it much simpler because you cannot use monolingual data generate over language, the reality has never worked. So even the first layer of the going up the pyramid, which is you go from like, the word form to syntax, and you're trying to use syntax, we found a translation. It never worked. Neither translation, neither for speech recognition. And I don't know I think the question is, like, the formalisms are, like, human-made, which you don't see direct on the data. They're very hard to express and to come up with those models. So you know, the performance of natural language understanding, has handed the best, it’s not that neat. And the reason is because it's super hard to do those models, and make them computationally efficient. So right now, all the models that basically work, they just read the words. Now this is one way of seeing the world. I can give a completely different way of seeing the world, which is, you have this like very complex models, this type, the cascade of networks, and then you read an entire sentence and auto-generating is a vector. And then from this vector, you generate a target sentence. You can claim that this vector is actually the interlingua and it's getting really good results. Now, it's a little bit disappointing because people thought about interlingua, says something that was acceptable. Just look out, “Oh, it’s a vector. I found those numbers.”
Max: Right, so there's some internal representation in some of these models, but we can actually look under the hood and see what's going on.
João: Yeah, I mean, so there's like all this algebra of vectors where you say, if I have the vector for kink and I subtract the vector from man and I send the vector for woman, I get the vector, that is really closing space to me. But that's as much as you get.
Max: Yeah. So right. So that's like the Google Word2Vec system. Yeah. So, one question I wanted to ask and the one thing that we noticed when we were doing internationalization of the Foursquare app, and I talked about that with my coworker Maryam, who's sometimes a co-host on this program. On Episode 2, we talked about the difficulties of getting our text translated into a lot of different languages. These were human translations, but we realized that some languages were a lot harder than others and almost every language pose some unique problem when it came to translation. So, do you find that some language pairs are harder in terms of doing automated translation than others and in what way?
João: Well, so if you exclude the data problem, like, do you have enough data for the language barrier, I think there's two main things. One is like the word order. So machine translation primes when the order is the same, like English or Spanish. You know, you can go word by word and almost generate word by word. If you now go from English to Japanese, which has a lot of reordering, it becomes a challenge. And this has to do with local capability of the models. The other thing is like language has a lot of inflection. It's hard to generate, because I have to go from one English word and understand what's the gender, the number, and so it makes machine translation much harder, because of the choice and because the amount of data that you have to see all potential communities is much larger. So those two things are normally the main reasons for the difficulty of the machine translation system to perform.
Max: Cool. Is there some standard way to rank how accurate machine learning translations are? I was thinking, like, how difficult that would be to come up with a number: “My system performed with a 7.8, and this new one performed 8.3.” Are there any ways to rank the accuracy? And how do you guys think about that?
João: There's several ways. So there's an automatic metric, or there's actually a lot of automated metrics, that score the quality of a transaction, and the way they do it is you have a source text translated, and then you have a true translation, and you compare it to some sort of distance to grade it. And so this is how you train systems: you need a metric to train and this is a metric to use. And so there’s like a research track just on getting better metrics to correlate to human judgment, and there are numbers from zero to one. So you can say, “Well, my BLEU score is now 60%, and it used to be 52.” Now can you as a human being understand what that 60, 52 means? It's harder. And then there's another that uses like human judgments. So for example we have this, we use this method called MQM. That stands for multi-dimensional quality metric, which is an industry standard where basically you ask a professional linguist to look at the source text of the translation, and mark all the errors that are in the translation, and group them if they're minor, major, or critical. So critical means the meaning is different from source. Major means it's the same meaning but it's hard to understand. And minor, there is no linguistic detail.
Max: Yeah. So it sounds like they have qualitative judgments, and now you're aggregating them.
João: Yes. And you rate them for, like, critical, major, minor. And then to mark in typology say, this is grammar. This is syntax. This is domain. This is sentiment. It's a hard process, but at the end of the day, you get the value from 0 to 100. And normally what you say is like above 90 is considered professional quality. And obviously, this is very subjective because it depends if you're talking about a chat, an email, a marketing piece, but you have a number and you can see the improvement on machine translation over time, you can see the data that you have from humans on top of machine translation, so it’s a way to really well way to measure quality.
Max: So you said it's 0 to 100, and 90 is professional level. The professional human level is 90. So where have any machine translation systems gotten to 90, or where are they at now?
João: I mean, I can tell you what my personal experience.
Max: Okay.
João: On average, you don't get to 90. They’ll be like on 60, and this is for easy contents like emails. They'll be on 60-something. There are some examples where they get to 90, like text and Facebook status, like it just works very well for the engine. Some of them it gets really bad, but it's been improving a lot. It's complete to be around for different statistical systems, and now it is getting much better.
Max: Gotcha. Gotcha. So, I would have thought that with automated translation technology, I'd be having more non-English conversations these days. And, you know, sometimes maybe with customer service texts, you know, back and forth. I'm kind of translating with Google Translate. I was playing a game online the other day and someone was trolling me in German, and I had to translate what he was saying, and he was saying nasty things to me, but I beat them in the game, so it's okay. But what do you think are the main applications today because I feel like I personally wish I could use it more to have more international conversations, but it seems almost as difficult. Maybe it's not a translation problem. Maybe It's just an interface problem. I don't know.
João: So first thing, like, you're lucky that you also speak English. If you didn't speak English, if you basically only spoke, let’s say, Portuguese, then you will use translation technology much more, especially if you're traveling.
Max: Right.
João: Like for that kind of scenario, like traveling, easy conversations, the technology is fit for purpose. You know, it might take you slightly a little bit longer. But it's definitely easier than not having it. So on all scenarios like that, it is becoming more and more useful. And it is very useful for first responders. So if there's a catastrophe somewhere, and you have to go there in order to talk with the local people, normally the first responders do not come from the country where the people are, so there's a different language. The earthquake at Haiti was an amazing example where Microsoft was able to develop on the fly some engines that actually save lives. So I think there's a lot of like, scenarios it's always related to like traveling or security or something that you have to go to another country and interact with people. More on the form like, “Am I going to use machine translation to translate my website or my blog to 20 languages to get to the audience?” It's not there yet. I wouldn't do it.
Max: Right, right. So I'm working on some transcripts for this podcast. And I mean, anyone's obviously free to trade. Take the transcripts and put it through their favorite translation engine and see what comes out. But I don't know what will come out. So what do you think it looks like when this technology is deployed more broadly? What do you think the effect is on society and business? Let's say over the next 10, 20 years, how will my life be changed? Well, when assuming we're going to be traveling more, you know, hopefully soon, because right now traveling is a little bit tough. But what do you think this looks like in the future? Let's just start.
João: First thing, I think is like it's going to be a more inclusive world.
Max: Yeah.
João: People would be able to communicate more. The vision of the model will be like the translation layer. If you think about it, you go to some websites, you go to like some mobile app, and you're just talking on your language. You don't even stop to think that there's a language issue, the same way that when you want to go to Google, you don't think about the IP address because there's a layer that converts that. You just go on the site and everything is basically a new language.
Max: Yeah, so that'd be great.
João: It's one spirit. It's like it's seamless. At the same time, you walk with your mobile phone or actually with some device and you just talk and basically all this language disappears. And so learning the language becomes a hobby, not necessity.
Max: Right. Well, that would be great. There are some challenges though, like, when we were translating, and this is not really about language. But when we were translating the Foursquare apps, we realized that there were some inside jokes that only work for, let's say, a TV show that was in the United States or a TV show that was in English. And it's like, “Okay, we could translate that sentence word for word, but it really wouldn't make sense.”
João: No, but that is part, for me, of the translation problem is the cultural translation.
Max: Yeah.
João: Like you're translating like customer service for Japanese where culture is very different, like the domain is super important, super specific. And so, even the way that you write your original English has to be changed, so that it translates properly. So with the technology solved to pick up those details into a proper translation, so that was kind of like this fixed expression of jokes, it will not translate it. It won’t convey the same. So right now you can write a message that is very Americanized and you translate it, it comes out like in the other language, we can’t automate anymore because it didn't convey the sentiment. This is part of solving the problem.
Max: Yeah, I almost feel like there has to be, in some situations, you know, could translate automatically but in other situations, it has to go back to the author and say, “Hey, you know, there are a few ways I can convey this on the other side, and it has very different meanings or very different sentiments depending on how I convey it.” And in your home language, the richness of the language or translation you're translating in doesn't really match the one of your home languages. So you've got to make a decision here.
João: Yeah, definitely.
Max: I mean, the author is available, can make a decision on its own, but sometimes, like—
João: Remember, there was an example on my first natural language processing book about machine relation, which is like, if you talk about God, like “God is a shepherd,” it will probably translate it to a vector that will have sheep. They don't have the context. So all that cultural thing, but I really, from my point of view—
Max: Yeah, it could be “shepherd,” which by the way is somebody who herds sheep, but a sheep, it could explain the whole thing. Or it could come up with a new metaphor entirely.
João: But if you saw the policy we're talking about with natural language understanding, ideally, you'll be able to solve this thing because the system will bring you up to like that semantic meaning and now know how to generate all the appropriate culture. But we are very far from there.
Max: Yeah, I feel like that would almost have to be like a general AI problem. It would have to understand the world at a pretty high level.
João: Yeah. Supposedly general AI could solve language and vice versa.
Max: Vice versa, maybe two. Yeah. All right. So tell me a little bit about like, what's the sort of tech stack particularly in terms of machine learning you're using these days. What techniques are you finding the most success with and how are you using them?
João: Mostly, well, deep learning right now you'll just, kind of a universal hammer. There's some very interesting models based on these deep language models like Birch where you basically keep this large model and then just trend to the colored part to use them. We're using mostly Python, except for the machine translation. So for most of the areas we’re using Python and PyTorch and we contribute to that. And we build our models on top of PyTorch like quality estimation or MQ metrics. And then for machine translation, we’re actually using a module on the C++ because it was more efficient. On the coding [unintelligible], which is basically something important for obvious cost reasons. But that’s basically the stack, and we're now working a lot on automating all these processes. Basically, you know, press a button, you can train the model for all your customers. Every customer has their own model that keeps getting retrained every week. So all this automation in metrics and logging is something that we put a lot of effort on.
Max: Yeah, there's a lot of data engineering that has to go in the background for a lot of these systems.
João: Yeah.
Max: All right. Well, this is a fascinating discussion, João. Thanks. So do you have any last thoughts and where can people go to check out more about you and your company?
João: It is a very interesting conversation. Company, they can go to www.unbabel.com, LinkedIn or just send me an email at joao@unbabel.com.
Max: Okay, great. I'll link to it on the show notes page for this episode. João, thanks for coming on the program.
João: Thanks a lot.
Max: Alright, check out the show notes for this episode at localmaxradio.com/129. I'll link to João and I'll link to Unbabel but I also have to link to Translator Fails. I know we've done that before, but these songs are hilarious when she puts them through all the different languages and what's this one, Africa by Toto?
Okay, that's enough of that. You probably can't hear it. Get the real thing on YouTube. I'll link to that as well. Ah, gotta have something to do today. It's like 97 degrees out there. Next week on 130, Aaron and I are going to talk about an article in the Wall Street Journal called The Ideological Corruption of Science by physicist Lawrence Krauss and also we have some updates on the latest AI models, those GANs, those generated adversarial networks that are creating images of faces and people that look real but aren’t. We covered that before, but now you better believe it's already being used to fool people and trick the media into sharing doctored news stories. So, remember to subscribe so that you don't miss it. I am moving back to Manhattan this week. So, I'm happy that it's already recorded. I hope my week goes extra well, and yours, too. Have a great week everyone.
That's the show. Remember to check out the website at localmaxradio.com. If you want to contact me to host or ask a question that I can answer on the show, send an email to localmaxradio@gmail.com. The show is available on iTunes, SoundCloud, Stitcher and more. If you want to keep up, remember to subscribe to The Local Maximum on one of these platforms and to follow my Twitter account @maxsklar. Have a great week.
