After a little over a month of learning more languages to translate beyond Spanish, Google's
recently announced Neural Machine Translation system has used deep learning to
develop its own internal language. TechCrunch reports:
GNMT's creators were curious about something. If you teach the translation system to translate English to Korean and vice versa, and also English to Japanese and vice versa... could it translate Korean to Japanese, without resorting to English as a bridge between them? They made this helpful gif to illustrate the idea of what they call "zero-shot translation" (it's the orange one). As it turns out -- yes! It produces "reasonable" translations between two languages that it has not explicitly linked in any way. Remember, no English allowed. But this raised a second question. If the computer is able to make connections between concepts and words that have not been formally linked... does that mean that the computer has formed a concept of shared meaning for those words, meaning at a deeper level than simply that one word or phrase is the equivalent of another? In other words, has the computer developed its own internal language to represent the concepts it uses to translate between other languages? Based on how various sentences are related to one another in the memory space of the neural network, Google's language and AI boffins think that it has. The paper describing the researchers' work (primarily on efficient multi-language translation but touching on the mysterious interlingua) can be read at Arxiv.
Automation hits the white collar sector
By Anonymous Coward
•
2016-Nov-23 20:04
• Score: 4, Insightful
• Thread
As a translator, these last couple of years have been grim. For things like marketing efforts and full-length books, where a very polished translation is desired from the get-go, there's still work out there for human translators. However, the bread and butter of a lot of translators was things like multinationals' internal documentation, or catalogues that consist of lots of simple listings and not much actual prose, where polish and shine isn't as vital. Companies are increasingly running their material through Google Translate, and then hiring a native speaker of the target language to proofread and correct that clunky output a vastly lower price than human translation.
It has often been said here on Slashdot that the development of self-driving trucks will put 3 million people out of work in the US alone. But translation is a field where, very quietly, automation is hitting the white-collar sector hard.
Re:No, this seems wrong
By Rei
•
2016-Nov-23 20:46
• Score: 5, Informative
• Thread
But for example... if I train you that cat = gato in italian, and that cat = chat in french. And then ask you to spit out the french if give you the "gato" that's not exactly magic. It looks up 'gato' in italian and sees a reference to "chat" ...
the neural network is still basically encoding that chat (french) = cat (english); and cat (english) = gato (italian)
That would be nice if translating sentences was the same as looking up words in a dictionary. It's not. So pointing out that there are words that have correspondences is meaningless.
Languages have a fuzzy haze of concepts and ways to parse them. I could say "I feel sick" or "I am sick" in English and they're not the same, the latter expresses certainty. But in Icelandic you'd generally say "Ég er lasin(n)" or "Ég er veik(ur)" - aka, "I am sick" - for both of them. Not "I feel sick". You *can* say "I feel as if I'm sick", but that gives a sort of connotation as if you're doubting yourself, more than "I feel sick" does in English. The latter case is "Mér líður eins og ég sé veik(ur)", which is literally "Me (dative, not nominative) feels same and I would-be(pres.) sick (depends on gender)" There's an awful lot going on in there that a word-for-word translation just doesn't catch. Even if you catch phrases, like "eins og" -> "like" rather than literally "same and", you still don't have anything close to a one-to-one mapping.
And here we're talking two Germanic languages.
A neural net that can handle translations in a way where the results aren't terrible must have a concept of the fuzziness, the interplay of how different concepts are presented in different languages. And indeed, that's what the graphic that they show seems to suggest, where you have these branching clusters with varying pathways that dart between them for different languages. Perhaps calling that internal representation a "secret language" is a stretch, but it's most definitely nothing like having "English as a bridge language".
Re:No, this seems wrong
By Rei
•
2016-Nov-23 21:10
• Score: 5, Informative
• Thread
To follow up a bit further on that, there are some concepts that take whole sentences, paragraphs or more to describe. Back in the day I had a Japanese song, with English lyrics... except that one word in the middle remained untranslated ("Our satori are just floating in the core"). I asked a professor about what it means and it ended up as a whole lecture on Buddhist concepts and Japanese relations between the true self and the self that one presents to others in different contexts.
In Icelandic for me it often comes up in terms of geological terms. For example, someone will ask, "What does Reykjavík" mean, and I usually just give a quick "Smoking Cove" or "Smoking Bay" or something like that. But that's not really right, English doesn't really have a word that describes a "vík". A "vík" is where the coastline "víkur". To víkja is to give way, like if someone's tailgating you on the road and you pull off to the side to let them past. So where the coastline "víkur" - on a certain scale, at least - that's a "vík". It's often where a river empties out, but not all river mouths end in víkur, and not all víkur are river mouths, some are more like coves or small bays. But you wouldn't mistake a "vík" for a "fjörður" or anything like that. We divide "field" up into "akur", "tún", "völlur", maybe even more depending on the concept (melur maybe, if it's rocky? garður even in some contexts? Lots of possibilities). So, I mean, we can just pick a random word, but you'll lose context - and when you translate back you can come up with something that's just wrong.
Even the "smoking" part isn't quite right, as most people in English hear smoke and think of burning things, but "reykur" in Icelandic place names is often used to denote geothermal steam - even though it technically means smoke.
My favorite mismatched concept has to be the verb "nenna", generally used in the negative (e.g. "Ég nenni ekki!"). In the negative it's sort of like "can't be bothered to do X", "not in the mood to do X", "don't waaaanna do X", "it's not worth my time/effort to do X", or just plain "Meh". A lazy translation is often "can't be bothered", but it sounds weird as English speakers don't usually talk like that. I've noticed some people who learn Icelandic end up taking that verb back into English, or even noun-ifying it ("I don't have the nenn to do that right now...")
Paging Wittgenstein!
By brwski
•
2016-Nov-23 21:21
• Score: 3
• Thread
Paging Wittgenstein!
Not read, but ...
By tgv
•
2016-Nov-24 01:43
• Score: 3
• Thread
It's quite likely that there is a shared representation. That's what neural nets do: if you feed train them on similar input/output pairs, they will develop common activation patterns. They would do so regardless of the language, since they don't know which language is being presented.
Humans, OTOH, do know that they're being presented with a different language, and demonstrably do something called "code switching": a cognitive effort to use another language resource. Therefore, in the human brain, the shared connection is supposed to lie outside the language faculty (there are other reasons to assume it, too).