As AI is becoming more prominent in high-stakes industries like healthcare, education, construction, environment, autonomous machines, and law enforcement, we are finding an increased need to trust the decision-making process. These predictions often need to be extremely accurate, e.g. critical life or death situations in healthcare. Due to the critical and direct impact AI is having on our day-to-day lives, decision-makers need more insight and visibility into the mechanics of AI systems and the prediction process. Presently, often only technical experts such as data scientists or engineers understand the backend processes and algorithms being used, like the highly complex deep neural networks. The lack of interpretability has shown to be a means of disconnect between technical and non-technical practitioners. In an effort to make these AI systems more transparent, the field of Explainable AI (XAI) came into existence.
Explainable AI (XAI) is an emerging subset in AI that focuses on the readability of machine learning (ML) models. These tools help you understand and interpret your predictions, reducing the complexity and allowing for non-technically trained practitioners and stakeholders to be more aware of the modeling process. At its core, XAI aims to deconstruct black box decision-making processes in AI. XAI can answer questions like “Why was this prediction made?” or “How much confidence do I have in this prediction?” or “Why did this system fail?”
Natural Language Processing (NLP) is a subset of AI (artificial intelligence) and ML (machine learning) which aims to make sense of human language. NLP performs tasks such as topic classification, translation, sentiment analysis, and predictive spelling using text data. NLP has historically been based on interpretable models, referred to as white-box techniques. These techniques include easily interpretable models like decision trees, rule-based modeling, Markov models, logistic regression, and more. As of recent years, however, the level of interpretability has been reduced to black-box techniques, such as deep learning approaches and the use of language embedding features. With reduced interpretability comes reduced trust, especially with human-computer interactions (HCI) such as chatboxes, for example.
A group of researchers at IBM conducted a survey called A Survey of the State of Explainable AI for Natural Language Processing. As one of the few works on the intersection of NLP and XAI, the survey aims to provide an understanding of the current state of XAI and NLP, explain currently available techniques, and bring the research community’s attention to the presently existing gaps. The categories of explanations used consist of whether the explanation is for a single prediction (local) or the model’s prediction process as a whole (global). The main difference between these two categories is that the first explanation is outputted during the prediction process (self-explaining) whereas the second requires post-processing following the model’s prediction process (post-hoc). The authors further introduce additional explainability aspects, including techniques for serving the explanation and presentation type to the end-user.
The researchers in this study presented five major explainability techniques in NLP that characterize raw technical components to present a final explanation to the end-user. These are listed below:
XAI may be presented to users in different ways, depending on the model complexity and explainability technique used. The visualization ultimately used highly impacts the success of an XAI approach in NLP. Let’s look at the commonly used attention mechanism in NLP, which learns weights (importance scores) of a given set of features. Attention mechanisms are often visualized as raw scores or as a saliency heatmap. An example of a saliency heatmap visualization of an attention score can be seen in Figure 1.
Figure 1 - Weights assigned at each step of translation
Salience-based visualizations focus on making more important attributes or factors more visible to the end-user. Saliency is often used in XAI to depict the importance scores for different elements in an AI system. Examples of saliency-based visualizations include highlighting important words in a text and heatmaps.
Other visualization techniques of XAI for NLP include raw declarative representations and natural language explanations. Raw declarative representations assume that end-users are more advanced and can understand learned declarative representations, such as logic rules, trees, and programs. Natural language explanation is any human-comprehensible natural language, generated from sophisticated deep learning models. For example, these can be generated using a simple template-based approach or a more complex deep generative model. At its core, it turns rules and programs into human-readable language.
The survey presented displays the connection between XAI and NLP, specifically how XAI can be applied to an NLP-based system. The field of XAI is meant to add explainability as a much-desired feature to ML models, adding to the model’s overall prediction quality and interpretability. Explainability can be categorized into different sectors of the NLP model, as well as being depicted by different visualization techniques seen above. Because of the large-scale presence of NLP around us, including chat boxes, predictive typing, auto-correct, and machine translation, it is important for any end-user, especially in NLP-based organizations, to understand the behind-the-scenes grunt work of the model. XAI allows for the end-user to gain trust in the NLP application being used and therefore allowing for a positive feedback loop, to ultimately make the algorithm even better. As XAI is still a growing field, there is plenty of room for innovation on improving the explainability of NLP systems.
TAUS provides professional language data services to some of the world’s largest technology companies. Our data collection, annotation, processing and NLP capabilities and global data contributors at scale remain a source of competitive advantage for leaders in artificial intelligence (AI) and machine learning (ML) space.
Husna is a data scientist and has studied Mathematical Sciences at University of California, Santa Barbara. She also holds her master’s degree in Engineering, Data Science from University of California Riverside. She has experience in machine learning, data analytics, statistics, and big data. She enjoys technical writing when she is not working and is currently responsible for the data science-related content at TAUS.
In recent years, NMT systems are getting better and better, some even claiming human parity. If systems on-par with human translators could really be deployed, that would fulfill the “no-human in the loop” dream that the industry seems to indulge in more and more frequently.
The AI scene of the 2010s was shaped by breakthroughs in vision-enabled technologies, from advanced image searches to computer vision systems for medical image analysis or for detecting defective parts in manufacturing and assembly. The 2020s, however, are foreseen to be all about natural language technologies and language-based AI tasks. NLP, NLG, NLQ, NLU… The list of abbreviations starting with NL (Natural Language) seems to grow each day. Regardless of the technology domain, it’s observed that natural language technologies will be in a field-shaping position in a variety of areas from business intelligence and healthcare to fintech.