Solutions
Resources
Case Study
TAUS Estimate API as the Ultimate Risk Management Solution for a Global Technology Corporation
When MT becomes the standard way of communicating with customers, who will be responsible for correctness and quality?
This was the question that got the urgent attention of the localization team of a very large technology corporation. Tens of thousands of chat messages are machine-translated and exchanged between customers and agents every day. Who knows if the information is correct and complete, or even in the right language? Are we not insulting our customers, using abusive terms, and addressing them wrongly? How accurate are the translations?
To gain insights into these questions, the client integrated TAUS DeMT™ Estimate API into their content workflow as an MT risk management tool and a means for cost reduction.
Ready to get started?
The Client
A global technology corporation.
Fortune Global 100 company
Localization in 16 languages.
The Challenge
Human review at a global content scale is unthinkable, costly, and time-consuming.
The volume of the content - millions of words every month - and the speed of business - real-time - simply do not allow for that. Putting aside the cost of labor: reading, checking and correcting all these texts…. You can’t imagine.
“We have gone too far now on this path of automation, we’ve got to automate the quality review as well. There is no other way.” That was the conclusion of the director of the localization division.
But how are you going to do that? You basically need a machine checking another machine, a model that is capable of reviewing another model. Is that even possible?
The Solution
Automatic quality prediction
Based on examples of texts from the client, the TAUS NLP team generated a large dataset and customized a quality prediction model built on what’s known as sentence embeddings. The quality scores produced by the custom model were then compared with samples of human translations to determine the correlation between what the model thinks of the quality and what the human determined to be good quality.
The set goal was to reach an 85% accuracy rate, in other words for 85% of the machine-translated segments the quality prediction model agrees with the human judgment of quality.
TAUS Estimate API
The TAUS engineering team built the infrastructure and an API that allows the customer to plug the quality prediction model into their workflow assigning a score to every machine-translated segment in the communications stream between customers and agents, all within milliseconds.
“We keep analyzing everything that comes available on the global research scene to optimize our service. For every new customer, we assess the effectiveness of our models and undertake customization.”
Amir Kamran, Solution Architect and senior NLP engineer at TAUS
The ultimate risk management solution
And so, TAUS has solved a serious risk problem for one of the Fortune Global 100 companies that have already moved far ahead with MT as a global communications solution for customers and personnel.
Bad and unreliable translations are filtered out from the feeds fully automatically, Non-perfect or somewhat questionable translations are flagged with an alert to the user that this is an automatically generated translation.
The Estimate API solution also served the client as a big efficiency enhancement technology in post-editing workflows by filtering good from bad MT and only sending segments that require human review through the post-editing step. This saves a lot of time and cost.
The Result
The content workflow enhanced by the TAUS Estimate API is the way to get the best out of MT at scale and in real-time.
85% model accuracy rate
3 different scores: CUSTOM, TAUS QE, COMET
Simple and fast API integration
3x faster content rollout
Reducing post-editing costs
Minimizing risk of fatal or catastrophic errors
Discover more Case Studies
Enabling 15% Increase in Number of Perfect Translations for ING Hubs poland
ING Hubs Poland found out that training with TAUS datasets improves the number of perfect translations by 15% and with 95% precision.
.
Domain-Specific Training Data Generation for SYSTRAN
After the training with TAUS datasets in the pandemic domain, the SYSTRAN engines improved on average by 18% across all twelve language pairs compared to the baseline engines.
.
Customization of Amazon Active Custom Translate with TAUS Data
The customization of Amazon Translate with TAUS Data always improved the BLEU score measured on the test sets by more than 6 BLEU points on average and 2 BLEU points at a minimum.