BERT for Named Entity Recognition (NER) on specialized corpora. An application on the biomedical field used in Meta, a research discovery tool

Transformer models have revolutionized the NLP field and are currently state-of-the art on a variety of tasks, such as named entity recognition, language inference or question answering. With new, more performant models being continuously developed (BERT, RoBERTa, AlBERT, ELECTRA, ERNIE, etc), these models are ubiquitous in virtually all domains that make use of natural language processing. So how can you apply these models on a specific task at hand, especially when the distribution of the data is different from the one the models have been trained on? In this talk, we will go over the process of using transformer models for the Named Entity Recognition (NER) task on specialized corpora. We will offer a specific example of building a BERT-based Named Entity Recognition model for mining for Experimental Methods and Datasets from full-text biomedical papers. The model has been deployed in a real-world production environment for users of Meta, a research discovery tool for biomedical researchers. We will discuss the entire ML pipeline, from building a training corpus, developing and evaluating the NER model, implementing it in a production environment and using the output on potential downstream applications. Even though we will focus on the biomedical field, the framework can be applied to any targeted domain.