Automatic quality prediction
Based on examples of texts from the client, the TAUS NLP team generated a large dataset and customized a quality prediction model built on what’s known as sentence embeddings. The quality scores produced by the custom model were then compared with samples of human translations to determine the correlation between what the model thinks of the quality and what the human determined to be good quality.
The set goal was to reach an 85% accuracy rate, in other words for 85% of the machine-translated segments the quality prediction model agrees with the human judgment of quality.
TAUS Estimate API
The TAUS engineering team built the infrastructure and an API that allows the customer to plug the quality prediction model into their workflow assigning a score to every machine-translated segment in the communications stream between customers and agents, all within milliseconds.
“We keep analyzing everything that comes available on the global research scene to optimize our service. For every new customer, we assess the effectiveness of our models and undertake customization.”
Amir Kamran, Solution Architect and senior NLP engineer at TAUS
The ultimate risk management solution
And so, TAUS has solved a serious risk problem for one of the Fortune Global 100 companies that have already moved far ahead with MT as a global communications solution for customers and personnel.
Bad and unreliable translations are filtered out from the feeds fully automatically, Non-perfect or somewhat questionable translations are flagged with an alert to the user that this is an automatically generated translation.
The Estimate API solution also served the client as a big efficiency enhancement technology in post-editing workflows by filtering good from bad MT and only sending segments that require human review through the post-editing step. This saves a lot of time and cost.