These days, it’s hard to avoid using a machine translation (MT) technology. The exponential growth in computing power and the ever-increasing demand for translation in today’s marketplace have led to their widespread adoption. And in the right hands, MT systems can be a handy resource. A rough translation of a large document delivered in seconds or minutes is more helpful than a good translation in three weeks, so they offer low-quality translations in those circumstances.
Despite the ease with which MT can be accessed, it is evident that the function and limitations of such systems are often misunderstood, and their capacity is widely overestimated. This essay aims to provide a high-level overview of MT systems and the optimal ways to employ them. Then, I’ll share data on current Internet-based MT usage to demonstrate the gap between the two and the continuing need to educate users on how to make the most of MT tools.
Mechanical Translators and Their Process
You might think that a computer translation program would combine the languages’ grammatical rules with a “dictionary” kept in memory to create a translation. This was, in fact, the case with some early systems. However, most state-of-the-art MT systems rely on a statistical method that is large “language-blind.” The algorithm is “trained” with a database of translated texts. A statistical model is produced by including factors like:
– “There is an X% chance that when the words (a, b, c) occur in succession in a sentence, the words (d, e, f) will occur in succession in the translation” (Note that the number of words in each pair need not be the same);
“if the word (a) ends in -X, then the word (b) has an X% chance of ending in -Y” is a statement describing the probabilities of endings in the target language.
After amassing an extensive database of such examples, the system can translate a sentence by evaluating potential translations constructed by stringing together words almost randomly (via some ‘naive selection’ process) and selecting the most probable one statistically.
Most people are taken aback when they hear this high-level explanation of MT, as they cannot fathom how a “linguistically blind” strategy could be successful. Even more unexpected is the fact that it often outperforms rule-based approaches. This is partly because the grammatical analysis is not without flaws (both automatic and human research have room for improvement) and, therefore, cannot be relied upon entirely. Because corpora of grammatically analyzed texts are small and scarce, training a system on “bare text” enables you to base a design on far more data than possible. There are trillions of pages of “bare text” available.
The downside of this method is that the accuracy of the translated text will rely heavily on the quality of the original data used to train the translator. Because sequences like “will be returned” are unlikely to have happened many times in the training corpus, the system will be hampered if you accidentally type it instead of “he will return” or “vous avez demander.” (or worse, may have occurred with a completely different meaning, as in they needed his will returned to the solicitor). Little grammatical context is available to the system, so it cannot determine that return is a form of return or that “the infinitive is likely after he will.”
It’s also possible to ask the system to translate a perfectly grammatical and everyday-use sentence containing less general characteristics in the training corpus. Technical or business papers, as well as transcripts of meetings of multilingual parliaments and conferences, are common training materials for MT systems. Because of this, MT systems have an inherent preference for more academic or specialized writing styles. Even if the training corpus includes common words and phrases, it may not account for everyday speech’s grammatical conventions (such as using t instead of usted in Spanish or the present tense instead of the future tense in different languages).
Real-World MT Systems
The public’s misunderstanding of the goals and constraints of computer translation systems has always been a significant concern for those working in the field. Somers (2003)[1] notes that the widespread use of MT on the internet and in chat rooms has “several side effects. […] There is certainly a need to educate the general public about the low quality of raw MT, and, importantly, why the quality is so low.” However, there is little evidence that users’ awareness of these issues has improved since 2003.
To demonstrate, I’ll use a small subset of the information accessible through the Espaol-Inglés website, where I provide a machine translation service from Spanish to English. The user input is taken, “cleaned up” (by fixing common orthographic errors and decoding “SMS-speak”), and then the service searches (a) a bank of examples from the site’s Spanish-English dictionary and (b) an MT engine for a translation. The MT engine currently used is Google Translate; however, a bespoke engine may be used shortly. We infer that most users are translating from their native language based on the results of an analysis of 549 Spanish-English queries submitted to the system by machines in Mexico[2].
To begin, why are individuals making use of MT? I made an educated “best guess” for each inquiry regarding the user’s motivation for requesting the translation. The intent is usually crystal clear, but there are some instances where it’s not. With that disclaimer in mind, I have concluded that the intended use is pretty clear-cut in about 88% of cases, and I have classified these uses as follows:
One-word or one-term lookups: 38%
Formal prose translation: 23%
Online conversation rate: 18%
Assignments: 9%
The fact that so many people use the translator to look up a particular word or term is surprising (and concerning). In reality, one-word searches made up 30% of all queries. This result is a little surprising because the same site also features a Spanish-English dictionary. It may indicate that website visitors conflate dictionaries and translators’ functions. The raw data doesn’t show it, but there were a few sequences of searches that looked like the user was trying to split up a sentence or phrase that would have been easier to translate if kept together.
For example, we see a request for cuarto para (“quarter to”) followed by request for a number, which may result from excessive vocabulary practice at school. There is a need to educate students and users about the distinction between the electronic dictionary and the machine translator[3]; precisely, that a dictionary will guide the user in choosing the appropriate translation given the context but only allows for single-word or single-phrase lookups, whereas a translator generally works best on whole sentences and, given a single word or term, will report the statistically most common translation.
My best guess is that around 75% of the time, the MT system is not being used for the official text translation or gisting it was “trained-for.” (and are entering an entire sentence, or at least partial sentence rather than an isolated noun phrase). Of course, without additional proof, we have no way of knowing if any of these translations were then meant for publication, which is not the point of the system.
In recent years, informal online chat translation has become almost as famous as formal text translation, even though MT systems are not usually trained for the latter context. Non-standard spelling, lack of punctuation, and colloquialisms not found in other written settings are all features prevalent in an online chat that present unique challenges for MT systems. A specialized system educated on a more appropriate (and perhaps custom-built) corpus would likely be necessary to translate chat sessions accurately.
It’s not shocking that some students use MT tools to complete their assignments. It’s worth noting if and how far this occurs. Fair use (learning from an activity) and attempts to “get the computer to do their homework” are both components of homework use. (with predictably dire results in some cases). Besides the explicit instructions for exercises, novice homework questions often feature sentences explaining overly broad concepts that would be glossed over in a standard text or discussion.
The frequency of errors in the original text likely to hinder the translation is a problem for users and designers of the system regardless of the application. In reality, more than 40% of queries had at least one such error; in some cases, there were multiples. The following were the most frequently made mistakes (questions containing only a single word or phrase were not included in the totals):
14% of all inquiries lack dialects.
Approximately one-fifth of the sentences had grammar errors.
Mistakes in spelling and grammar: 8%
Eight percent of sentences have grammatical errors.
Users appear to undervalue the importance of using standard orthography to give the best chance of a good translation, even though they were translating from their original language in most instances. Queries like “hoy es da de” are common because users don’t always realize that the translation of one word can rely on another or that it makes the translator’s job more difficult if grammatical constituents are incomplete. Because of the low probability of finding a sentence in the training corpus with, for example, a “dangling preposition” like this, such inquiries slow down the translation process.
There must be some… takeaways?
There is still a disconnect between MT systems’ capabilities and customer expectations. To me, it’s up to programmers and ends users/teachers to bridge this divide. Users should put more effort into making their source phrases “MT-friendly” and learning to evaluate the results of MT systems. These concerns should be integrated into language classes; proficiency in machine translation tools should be regarded as essential to language acquisition. I, as a developer, also need to consider how I can better meet the requirements of language learners with the resources I provide.
Notes
[1] Somers, “Machine Translation: the Latest Developments,” The Oxford Handbook of Computational Linguistics, Oxford University Press, 2003.
This peculiar count [2] results from the haphazard way in which inquiries satisfying the selection criteria were recorded during the allotted period. It should be mentioned that the method used to determine a computer’s location based on its IP address is not foolproof.
[3] If the user only inputs one word into the system, a message recommending that they consult the site’s dictionary appears below the translation.
A Spanish dictionary [http://www.espanol-ingles.com.mx/dictionary/spanish-english/], a section with Spanish phrases with audio recordings, grammar material, and online word games are just some tools available on the ESPANOL-INGLES website, which is also helpful for Spanish speakers learning English.
Read also: Studio Basics – Here’s What It Is Advisable To Record
