June 15, 2015, Amsterdam – TAUS and DFKI announce that they have completed the harmonization of their respective translation quality metrics, the Dynamic Quality Framework (DQF) error typology and the Multidimensional Quality Metrics (MQM). This harmonization effort is part of the European Union funded QT21 project. The new quality metric will be available to all stakeholders in the global translation industry looking for a standard way to categorize and measure translation quality.
The TAUS Dynamic Quality Framework (DQF) was developed over the last couple of years in consultation with its members. DQF includes various tools for the evaluation of translation quality, the error typology being one them. The Multidimensional Quality Metrics (MQM) is an error typology metric that was developed as part of the (EU-funded) QTLaunchPad project based on careful examination and extension of existing quality models. Despite the variety of approaches taken in industry and research, the two models turned out to be broadly similar, but they were also different in important ways due to their history. In a series of meetings the developers of MQM and DQF have agreed to make substantive changes to both frameworks to bring them into harmony. When the new versions of MQM and DQF are released this summer, users will no longer have to choose between the two because they will share the same underlying structure.
“The past few years have seen an explosion of interest in translation quality assessment methods and metrics,” says Attila Görög of TAUS. “Most companies use the LISA QA model or some interpretation of it or a variant of the SAE J2450 metric. Both of these models have not kept up with the times and lack the flexibility that is required in a world with much for diversified types of content. This newly harmonized metric offers translation professionals a standard and dynamic model that can be used in every context.”
DQF’s analytic method and the MQM hierarchy of translation quality issues have both been modified to share the same basic structure. DQF will use a subset of the full MQM hierarchy based on the experience of TAUS members, while MQM will continue to maintain a broader set of issue types designed to capture and describe the full range of quality assessment metrics currently in use. Users of the DQF analytic method will therefore be guaranteed to be compliant with MQM as well. Via MQM, DQF will gain an unambiguous mapping to the ITS 2.0 localization quality issue types that have been implemented in Ocelot and other ITS 2.0 compliant tools. This mapping will allow both MQM and DQF to interoperate with this W3C recommendation.
Arle Lommel (DFKI) states, “We are especially glad to have worked with TAUS on this harmonization effort because it reduces industry confusion about which framework to use and it simplifies the implementation process for everyone. Both MQM and DQF had to make significant changes, but the resulting shared framework is clearer and more useful for everyone. Moving forward we can expect to see more industry uptake of quality assessment best practices based on this shared resource.”
For the time being MQM and DQF will remain separate projects, with TAUS continuing to offer the DQF suite of online tools. Since this month TAUS also offers an API allowing translation technology and service providers to integrate DQF into their day-to-day work environment. The harmonized metric will be made available through a future release of the DQF API.
A document describing the harmonized version of DQF and MQM is now available online for comment at http://qt21.eu/mqm-definition. We are seeking public feedback on this draft through the end of June 2015.
DQF comprises of a set of tools for quality evaluation and productivity measurement, a content profiling wizard, and a knowledge base containing best practices and use cases. The framework has been developed in close cooperation with many of the TAUS member companies. In 2014 TAUS released the DQF tools running on the TAUS web site. The DQF API makes it possible for everyone to use DQF from within their own translation tool environment.
MQM is a framework for building task-specific translation metrics. With detailed descriptions of over 100 translation-related errors at various levels of detail, it allows users to create custom metrics that can be used for various assessment purposes. By providing a master vocabulary of error types, users can describe metrics in a fully transparent fashion. MQM has been implemented in a variety of commercial and open-source tools. For more information about MQM, please visit http://qt21.eu/mqm-definition.
TAUS is a resource center for the global language and translation industries. Our mission is to increase the size and significance of the translation industry to help the world communicate better.
We envision translation as a standard feature, a utility, similar to the internet, electricity and water. Translation available in all languages to all people in the world will push the evolution of human civilization to a much higher level of understanding, education and discovery.
We support buyers and providers of language services and technologies with a comprehensive suite of online services, software and knowledge that help them to grow and innovate their business. We extend the reach and growth of the translation industry through our vision of the Human Language Project and our execution with sharing translation memory data and quality evaluation metrics.
For more information about TAUS, please visit: https://www.taus.net
The German Research Center for Artificial Intelligence (DFKI)’s Language Technology lab conducts advanced research in language technology and provides novel computational techniques for processing text, speech and knowledge. We strive for a deeper understanding of human language and thought, studying the true needs of the end user and the demands of the market.
Key areas of activity include Text Analytics for Big Data, Machine Translation, and Human-Robot Interaction.
We address the global R&D community in language technology and neighboring technology areas as well as companies and other organizations planning to employ language technology applications. In addition to a wide variety of free service functions, the lab also offers highly specialized professional services to private customers.
For more information about DFKI’s language technology lab, please visit http://www.dfki.de/lt.
The project QT21 has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 645452.