First Time Ever: Live Automatic Interpretation on the TAUS Stage
icons-action-calendar12 Feb 2018
5 minute read
For the first time ever, the live automatic interpretation powered by Microsoft will be on the TAUS Stage. Mark Seligman tells us all about that and what else is on his agenda.

The TAUS Asia Conference in Beijing on March 22-23 is shaping up as exciting. Here’s a sneak preview from my own vantage point.

presentation-translator-for-microsoft-PowerPoint.jpgFirst and most significant, the meeting will host a milestone in speech translation: for the first time anywhere, real-time automatic on-screen transcription and on-device multilingual text translation will be provided for all conference speakers. (Human interpreters for English and Mandarin will also be on the job.) Microsoft’s Presentation Translator, an add-on to Powerpoint, will power the experiment. The software represents a considerable development effort, supervised by translation chief Chris Wendt. Of particular interest for speech translation is the TrueText component, designed to handle the disfluencies of spontaneous speech – repetitions, hesitation syllables, stutters, etc. TrueText revises and cleans the preliminary speech recognition result for each utterance before displaying the regularized text on-screen as subtitles and then handing it off to machine translation. The translated text then streams to audience members on their own devices. The Presentation Translator also contains a built-in training tool, which can pre-scan the slides and speaker notes of a Powerpoint set to familiarize itself with the vocabulary. Speech recognition error rate reductions of as much as 30% are claimed. I'll be handling that training, along with the setup of wireless mics, etc., for the Beijing presentations.

As my second contribution, I'll be continuing my advocacy for TAUS collection of speech as well as text data. I'll reprise my San Jose talk on the topic, departing from the uncontroversial recognition that big data has been a decisive enabler in the separate development of machine translation and speech recognition. I'll go on to make the case that big (and good quality) data will be equally important in developing the combination of these technologies – speech translation – and that TAUS can play an important role in coordinating data collection and sharing. The hope is to foster a virtuous circle, in which speech translation data can improve speech translation, which can then produce more data. I'll also argue the importance of correction data – naturally enough, since my own company has concentrated on verification and correction of speech translation. I'll consider the economic impact of speech and speech translation data: Who will own it? How will it eventually affect employment? I'll touch on related ethical issues (privacy, security, prejudice), and on the use cases which might be lucrative or important enough to justify TAUS’s data collection efforts. Lastly, I'll consider the types of speech translation data that can be collected (monolingual vs. bilingual; spontaneous vs. scripted; etc.).

s2s.gifMy last contribution will address a topic especially dear to my heart: the role of semantics in machine translation, as it has been played in the past and as I think it may be played going forward. For this talk, the departure point will be the argument by John Searle and other influential theorists that machine translation and other natural language processing programs can never appreciate meaning in the deepest sense – in other words, that they can never really exhibit semantics at all. I'll concede that MT and many other NLP systems have made steady and impressive progress while use of explicit semantic processing has undergone a rise and fall; and I'll recognize that researchers haven’t yet agreed on the meaning of meaning. Still, I'll point to renewed interest in semantic representation and processing. And – my central point – I also foresee gradual adoption throughout natural language processing of semantic approaches grounded upon audio, visual, or other sensor-based input. These perceptually-grounded semantic approaches are distinct from most current perception-free methods.

From a philosophical viewpoint, I'll risk the suggestion that perceptually-grounded approaches to MT and other NLP can display intentionality, and consequently can provide the foundation for truly meaningful semantics. I think that perceptual grounding of this sort depends on the ability to learn and associate categories, and that this ability, in turn, is a necessary – though not sufficient – condition for higher cognitive processes. To lay the groundwork for these somewhat presumptuous claims, I'll be surveying the role of semantics in machine translation until now in terms of three paradigms: rule-based, statistical, and neural MT. Within the rule-based paradigm, we'll revisit direct, transfer-based, and interlingua-based variants.

Well, that ought to be enough for one conference!

To hear more from Mark Seligman and be a part of the live automatic interpretation experience while hearing what the Asian translation/localization market is up to, join us in Beijing at the TAUS Asia Conference! 


Dr. Mark Seligman is founder, President, and CEO of Spoken Translation, Inc. His early research concerned automatic generation of multi-paragraph discourses, inheritance-based grammars, and automatic grammar induction. During the 1980’s, he was the founding software trainer at IntelliCorp, Inc., a forefront developer of artificial intelligence programming tools. His research associations include ATR Institute International near Kyoto, where he studied numerous aspects of speech-to-speech translation; GETA (the Groupe d’Étude pour la Traduction Automatique) at the Université Joseph Fourier in Grenoble, France; and DFKI (Deutsches Forschungszentrum für Künstliche Intelligenz) in Saarbrücken, Germany. In the late 1990s’, he was Publications Manager at Inxight Software, Inc., commercializing linguistic and visualization programs developed at PARC. In 1997 and 1998, in cooperation with CompuServe, Inc., he organized the first speech translation system demonstrating broad coverage with acceptable quality. He established Spoken Translation, Inc. in 2002.

Related Articles
icons-action-calendar16 Aug 2022

It’s been almost three years since we’ve regularly come together all around the world to discuss the state of the industry. In our 15 years of events history at TAUS, we’ve discussed many topics. From localization workflows, pricing models, adopting quality evaluation metrics in our translation workflows, quality standards, and the many different translation technologies that help translators become more efficient, productive and creative. We brainstormed on solutions for the various challenges we came across, and in turn, TAUS started to create our own suite of products and services to address some of these challenges. The world has changed over the past years, and TAUS has changed as well. We’re so pleased to get together again and present the new TAUS, along with the exciting new subsector Language Data for AI, to you at our upcoming Massively Multilingual Conference & Expo in San Jose, CA on 11, 12 and 13 October 2022.

icons-action-calendar6 Apr 2020

Bit by bit, we are cracking the human language code. It’s happening fast and it marks a new beginning in both market expansion and the broader setting of human evolution. Language digitization in general is boosting economies, generating a spate of innovations, and opening up opportunities to push human civilization to higher levels of knowledge and mutual understanding.

icons-action-calendar1 May 2019

Japan in springtime is a truly magical place. Every year thousands of visitors come to Japan to celebrate the iconic sakura season, when the whole country lits up in the rosy glow of cherry blossoms. Strolling around the petal-covered streets of Tokyo, many of us don’t realize that sakura has a deep symbolic meaning in the Japanese culture. In Japan, the cherry blossom represents the fragility and the beauty of life. Carpe diem, seize the moment while it lasts.