dublin-skyline-banner

Hosted by Microsoft in Dublin on March 14-15, 2013

Overview

Solving an Industry Problem

Quality evaluation (QE) in the translation industry is problematic. It’s static. One translation quality fits all purposes and all content types. It’s subjective. It lacks industry-level agreed methods and metrics. The problem of QE is aggravated by the introduction of machine translation (MT) technology and the growing need to translate new content types.

Together with 16 of its enterprise members TAUS has developed a dynamic approach to QE. Variable factors like the type of content, turnaround time and intended audience determine the approach and method that is selected to evaluate the quality of translation.

An industry-shared platform for dynamic quality evaluation and MT productivity measurement has now been launched. This allows buyers and providers of translation to access common criteria for quality evaluation, establish practitioner best practices and benchmark performance in a neutral environment.

This Dynamic Quality Framework helps to bring credibility and transparency to translation quality evaluation, solving bottlenecks, saving time and costs and increasing efficiency and performance in the translation industry in general and particularly when deploying MT into production environments.

Objective of the Translation QE Summit

Buyers and providers of translation will review and agree the requirements, the use scenarios, the benchmark reporting and the implementation phases of the shared industry Dynamic Quality Framework (DQF).

Participants in the Summit are requested to familiarize themselves with the DQF knowledgebase and the DQF tools for translation quality evaluation and MT productivity measurement in preparation of the meeting.

Preparation

Preparation

The Translation QE Summit will start in the afternoon of March 14 with a Translation Buyer Roundtable. Representatives from software, hardware, automotive, government and other industry-sectors will meet to discuss the commonalities and differences in their needs for translation quality evaluation. The objective of this buyer-only meeting is to agree on a clear set of common requirements for transparent and dependable QE, which will lead to a more frictionless, faster and efficient translation process for all.

Translation vendors will join buyers for a full-day summit on March 15. Buyers will present their shared requirements. DQF use scenarios and approaches for quality evaluation will be presented by both buyers and vendors.

In break-out sessions participants will discuss and further refine the DQF in terms of requirements, metrics and benchmarking criteria.

The specific QE approaches that will be considered include adequacy/fluency, usability, error typology, readability, community-based evaluation, customer feedback, and productivity measurement. Conclusions of the break-out sessions will be presented to the plenary meeting.

Shared Research

In preparation of the Translation QE Summit participants will receive copies of TAUS research reports. Members can download these reports from the TAUS web site.

  • Dynamic Quality Framework Report. (November 2011). A thirty-two-page report documenting the rationale behind the DQF and the definition stage of the project with 16 enterprise members.
  • Advancing Best Practices in QE Report. (July 2012). A twenty-page report giving a comprehensive overview to the subject of MT quality evaluation, sharing of best practices, and documenting a number of brief use cases. The report explains how the TAUS DQF improves the state-of-the-art for machine translation evaluation in specific contexts.
  • Machine Translation Post-Editing Guidelines. Industry-shared criteria for “good enough” quality and “quality similar or equal to human translation”. Result of a dedicated workshop held in Portland (OR) in October 2009 and adopted as guidelines by the translation industry.
  • Common Criteria for Comparing Vendors’ Quality. Analysis and comparison of criteria applied to assessing quality of translation from different vendors. Report to be published in February 2013.

Non-members will receive an overview of the DQF knowledge base and shared tools for QE.

Program

Program

Thursday 14 March

Buyers Session

Buyers discuss and agree their common agenda for translation quality evaluation.

14:00 / Welcome and Program Objectives, Jaap van der Meer (TAUS)
14:10 / Introductions
14:40 / TAUS Dynamic Quality Framework, Rahzeb Choudhury (TAUS)

15:00 /REFRESHMENT BREAK

DQF Users Talk

Project participants highlight their QE requirements, while introducing their involvement and perspective on the DQF to other attendees

15:30 / Breen McInerney (Microsoft)
15:40 / Bettina Reichart (Oracle)
15:50 / Markus Foti (EC DGT)
16:00 / Morgan O'Brien (McAfee)
16:10 / Katrin Drescher (Symantec)
16:20 / Nancy Ferreira da Rocha (Spil Games)
16:30 / Sharon O'Brien (TAUS Labs)
16:40 / Lucia Specia (Sheffield University)

16:50 /  Plenary discussion of opportunities and barriers to adoption

17:45 /CLOSE

18:30 / Reception

Networking reception and dinner for all participants from days one and two. This will take place at the Dylan Hotel. Bus transfers are being organized for people staying at Bewley's Hotel or Buswells Hotel. Please email This email address is being protected from spambots. You need JavaScript enabled to view it.This email address is being protected from spambots. You need JavaScript enabled to view it.for questions about this.

Friday 15 March

Joint Session

Buyers and providers work together to agree definitions, establish best practices, and common metrics around translation quality.

Welcome and Overview

09:00 / Welcome and Program Objectives, Jaap van der Meer (TAUS)

09:15 / TAUS Dynamic Quality Framework, Rahzeb Choudhury (TAUS)

DQF Users Talk

Project participants highlight their QE requirements, while introducing their involvement and perspectives on the DQF to other attendees

09:45 / Breen McInerney (Microsoft)
09:53 / Katrin Drescher and Rafael Guzman (Symantec)
10:01 / Lucia Specia (Sheffield University)
10:09 / Karin Berghoefer (Appen Butler Hill)
10:17 / Bettina Reichart (Oracle)
10:25 / Christian Arno (Lingo24)

10:30 /REFRESHMENT BREAK

Breakout Discussions

11:00 /ROUND ONE
     1. Best Practices for Matching Translation Quality Requirements to Quality Evaluation Models
     (Chairs: Bettina Reichart, Oracle and Sharon O'Brien, TAUS Labs)
      Breakout aim: Review selection criteria in the DQF to establish best practices and create a TAUS Industry Guideline.
 
    2. Best Practices for Pricing MT Post- editing (Chairs: Katrin Drescher and Rafael Guzman, Symantec)
     Breakout aim: Recommendations on best practices for pricing MT post-editing leading to a TAUS Industry Guideline.

     3. Improving Automated Metrics (Chair: Lucia Specia, University of Sheffield)
     Breakout aim: Outline plan for industry supporting academia in improving automated metrics.

     4.  Best Practices for Adequacy, Fluency and/or Readability Evaluation (Chair: Karin Berghoefer, Appen Butler Hill)
     Breakout aim: Recommendations on best practices leading to a TAUS Industry Guideline.


12:00 /REPORTS BACK TO PLENARY
     Five-minute reports back by session leaders followed by short Q&A on any points for clarification.

12:30 /LUNCH

Introduction to Afternoon Program

14:00 / DQF TOOLS (Rahzeb Choudhury, TAUS)

DQF Users Talk

Project participants highlight their QE requirements, while introducing their involvement and perspectives on the DQF to other attendees.

14:05 / Morgan O'Brien (McAfee)
14:10 / Markus Foti (EC DGT)
 14:15 / Sharon O‘Brien (TAUS Labs)
14:25 / Ana Gueberof (Pactera)
 14:35 / Nancy Ferreira da Rocha (Spil Games)

Breakouts

14:40 /ROUND TWO
     1. Best Practices for Error Typology Quality Evaluation (Chair: Sharon O'Brien, TAUS Labs)
     Breakout aim: Recommendations on best practices leading to a TAUS Industry Guideline.

     2. Review and Improving the DQF Tools (Chair: Rahzeb Choudhury, TAUS)
     Breakout aim: List and prioritize ideas for improving the Dynamic Quality Framework.

     3. Best Practices Guidelines Quality and the Translator (Chair: Ana Gueberof, Pactera)
     Breakout aim: Recommendations on best practices leading to a TAUS Industry Guideline.

15:40 /REFRESHMENT BREAK

16:10 /REPORTS BACK TO PLENARY
     Five-minute reports back by session leaders followed by short Q&A on any points for clarification.

Closing Plenary

16:30 / A review of progress, priorities and next steps.

17:30 /ADJOURN

Participants

Participants

Who Should Attend

The Translation QE Summit is recommended to buyers of translation in different industries and government and non-government organizations (director level and vendor and quality management functions), and to small and large language service providers (CEO, director and quality management functions).

Participating Companies

ABBYY European Commission McAfee Spil Games
Amesto EMC Microsoft STP Nordic
Appen Butler Hill European Patent Office Moravia Symantec
Autodesk Google Oracle Tekom
Capita Intel Pactera Tilde
CLS Communication Jensen Localization Palex Languages & Software TransN
CNGL Jonckers Pangeanic University of Sheffield
Concorde LexWorks RR Donnelley WCS Group
Crestec Lingo24 Safaba Welocalize 
CrossLang Lionbridge Sajan Yamagata Europe 
Dell Logrus SAP  

Program Committee

The Program Committee of the Translation QE Summit consists of the members of the DQF enterprise work group: 

  • Wayne Bourland, Dell
  • Karen Combe, PTC
  • Fred Hollowood, Symantec
  • Breen McInerney, Microsoft
  • Christian Redmann, Siemens
  • Bettina Reichart, Oracle

Practical Information

Practical Information

The Translation QE Summit will be held at the Microsoft facilities in Dublin, Ireland.

Address: 
European Product Development Centre 2 (EPDC2)
South County Business Park, Leopardstown
Dublin 18
Click here to find directions to the Microsoft facilities.

A networking dinner will be offered for all participants on the evening of the 14th of March. Confirmation is required.

Hotel recommendations

Recommendable hotels in the area of the Microsoft facilities:

Bewley's Hotel Leopardstown
Address:
Central Park, Leopardstown, Dublin 18
T: +353 (0)1 293 5000 F: +353 (0)1 293 5099
Email: This email address is being protected from spambots. You need JavaScript enabled to view it.
Bewley’s Hotel Leopardstown is serviced by Aircoach, which runs from Dublin Airport every 20 minutes. Tickets can be purchased on the coach.
This email address is being protected from spambots. You need JavaScript enabled to view it.

Buswells Hotel
Address:
23-25 Molesworth Street, Dublin 2
T: 01 295 8900
Email: This email address is being protected from spambots. You need JavaScript enabled to view it.
Buswells Hotel is also serviced by Aircoach, which runs from Dublin Airport every 20 minutes. Tickets can be purchased on the coach.
This email address is being protected from spambots. You need JavaScript enabled to view it.

Recommendable hotels in Dublin city center:

The Westin Dublin
Address:
College Green, Westmoreland Street, Dublin 2
T: +353 (0)1 645 1000
Click here to find directions from various locations to the Westin Dublin.

The Westbury Hotel Dublin
Address:
Grafton Street, Dublin 2
T: +353 (0) 1 679 1122, Email: This email address is being protected from spambots. You need JavaScript enabled to view it.
Click here to find directions from various locations to the Westbury Hotel Dublin. 

The Shelbourne Dublin
Address:
27 St Stephen's Green · Dublin, 2
T: 353-1-6634500
Click here to find directions from various locations to the Shelbourne Dublin.