Web Scraping for Parallel Corpora Creation

by Lisa Vasileva

01/10/2021

Web scraping is a common way to generate parallel data, making use of the immense source of multilingual data offered on the web. Here is how to do web scraping.

web-scraping-for-parallel-corpora-creation

Author

Lisa Vasileva

Lisa is a Data Curator at the NLP Team with TAUS. Using her background in linguistics and experience in the translation industry, she helps TAUS optimize the data offering and create new data solutions.

Data for AI

TAUS Data Sale to Boost Multilingual LLMs

by Anne-Maj van der Meer

11/03/2024

Purchase TAUS's exclusive data collection, featuring close to 7.4 billion words, covering 483 language pairs, now available at discounts exceeding 95% of the original value.

Data for AI

Transforming Translations: The Crucial Role of Language Data in the Age of Large Language Models and Generative AI

by Anne-Maj van der Meer

09/11/2023

Explore the crucial role of language data in training and fine-tuning LLMs and GenAI, ensuring high-quality, context-aware translations, fostering the symbiosis of human and machine in the localization sector.

Data for AI

Domain Adaptation: Types and Methods

by Anne-Maj van der Meer

19/12/2022

Domain adaptation approaches can be categorized into three categories according to the level of supervision used during the training process.