{"id":2796,"date":"2023-06-13T01:08:23","date_gmt":"2023-06-13T00:08:23","guid":{"rendered":"https:\/\/archive.belbi.bg.ac.rs\/2023\/?post_type=abstract&#038;p=2796"},"modified":"2023-06-14T18:37:24","modified_gmt":"2023-06-14T17:37:24","slug":"zero-and-few-shot-machine-learning-for-named-entity-recognition-in-biomedical-texts","status":"publish","type":"abstract","link":"https:\/\/archive.belbi.bg.ac.rs\/2023\/abstract\/zero-and-few-shot-machine-learning-for-named-entity-recognition-in-biomedical-texts\/","title":{"rendered":"Zero- and Few-Shot Machine Learning for Named Entity Recognition in Biomedical Texts"},"content":{"rendered":"\n<p>Milo\u0161 Ko\u0161prdi\u0107<sup>1*<\/sup>, Nikola Prodanovi\u0107<sup>1<\/sup>, Adela Ljaji\u0107<sup>1<\/sup>, Bojana Ba\u0161aragin<sup>1<\/sup>, and Nikola Milo\u0161evi\u0107<sup>1,2<\/sup><\/p>\n\n\n\n<p class=\"affiliation-para\"><sup>1<\/sup>Institute for Artificial Intelligence Research and Development of Serbia, Fru\u0161kogorska 1, Novi Sad, Serbia<\/p>\n\n\n\n<p class=\"affiliation-para\"><sup>2<\/sup>Bayer A.G., Reaserch and Development, Mullerstrasse 173, Berlin, Germany<\/p>\n\n\n\n<p>milos.kosprdic [at] ivi.ac.rs<\/p>\n\n\n\n<p><strong>Abstract<\/strong><\/p>\n\n\n\n<p class=\"abstract-para\">Named entity recognition (NER) is an NLP that involves identifying and classifying named entities in text. Token classification is a crucial subtask of NER that assumes assigning labels to individual tokens within a text, indicating the named entity category to which they belong.  Fine-tuning large language models (LLMs) on labeled domain datasets has emerged as a powerful technique for improving NER performance. By training a pre-trained LLM such as BERT on domain-specific labeled data, the model learns to recognize named entities specific to that domain with high accuracy. This approach has been applied to a wide range of domains including biomedical and has demonstrated significant improvements in NER accuracy.  <\/p>\n\n\n\n<p class=\"abstract-para\">Still, data for fine-tuning pre-trained LLMs is large and labeling is a time-consuming and expensive process that requires expert domain knowledge.\u200b Also, domains with an open set of classes yield difficulties in traditional machine learning approaches since the number of classes to predict needs to be pre-defined.<\/p>\n\n\n\n<p class=\"abstract-para\">Our solution to the two mentioned problems is based on data transformation for factorizing the initial multiple classification problem into a binary one and applying cross-encoder-based BERT architecture for zero- and few-shot learning.<\/p>\n\n\n\n<p class=\"abstract-para\">To create our dataset, we transformed six widely used biomedical datasets that contain various biomedical entities such as genes, drugs, diseases, adverse events, chemicals, etc., into a uniform format. This transformation process enabled us to merge the datasets into a single cohesive dataset of 26 different named entity classes. <\/p>\n\n\n\n<p class=\"abstract-para\">We then fine-tuned two pre-trained language models: BioBERT and PubMedBERT for the NER task in zero- and few-shot settings. The results of the experiment for 9 classes in zero-shot mode are promising for semantically similar classes and improve significantly after providing only a few supporting examples for almost all classes. The best results were obtained using a fine-tuned PubMedBERT model, with average F1 scores of 35.44%, 50.10%, 69.94%, and 79.51% for zero-shot, one-shot, 10-shot, and 100-shot NER respectively.<\/p>\n\n\n\n<p class=\"abstract-para\"><strong>Keywords:<\/strong> zero-shot learning, machine learning, deep learning, natural language processing, biomedical named entity recognition<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Milo\u0161 Ko\u0161prdi\u0107<sup>1*<\/sup>, Nikola Prodanovi\u0107<sup>1<\/sup>, Adela Ljaji\u0107<sup>1<\/sup>, Bojana Ba\u0161aragin<sup>1<\/sup>, and Nikola Milo\u0161evi\u0107<sup>1,2<\/sup><\/p>\n<p class=\"affiliation-para\"><sup>1<\/sup>Institute for Artificial Intelligence Research and Development of Serbia, Fru\u0161kogorska 1, Novi Sad, Serbia<\/p>\n<p class=\"affiliation-para\"><sup>2<\/sup>Bayer A.G., Reaserch and Development, Mullerstrasse 173, Berlin, Germany<\/p>\n<p> <a class=\"continue-reading-link\" href=\"https:\/\/archive.belbi.bg.ac.rs\/2023\/abstract\/zero-and-few-shot-machine-learning-for-named-entity-recognition-in-biomedical-texts\/\"><span>Continue reading<\/span><i class=\"crycon-right-dir\"><\/i><\/a> <\/p>\n","protected":false},"author":162,"featured_media":0,"template":"","categories":[18],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v23.0 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Zero- and Few-Shot Machine Learning for Named Entity Recognition in Biomedical Texts - BelBi 2023<\/title>\n<meta name=\"robots\" content=\"noindex, follow\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Zero- and Few-Shot Machine Learning for Named Entity Recognition in Biomedical Texts - BelBi 2023\" \/>\n<meta property=\"og:description\" content=\"Milo\u0161 Ko\u0161prdi\u01071*, Nikola Prodanovi\u01071, Adela Ljaji\u01071, Bojana Ba\u0161aragin1, and Nikola Milo\u0161evi\u01071,21Institute for Artificial Intelligence Research and Development of Serbia, Fru\u0161kogorska 1, Novi Sad, Serbia2Bayer A.G., Reaserch and Development, Mullerstrasse 173, Berlin, Germany Continue reading\" \/>\n<meta property=\"og:url\" content=\"https:\/\/archive.belbi.bg.ac.rs\/2023\/abstract\/zero-and-few-shot-machine-learning-for-named-entity-recognition-in-biomedical-texts\/\" \/>\n<meta property=\"og:site_name\" content=\"BelBi 2023\" \/>\n<meta property=\"article:modified_time\" content=\"2023-06-14T17:37:24+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/archive.belbi.bg.ac.rs\/2023\/abstract\/zero-and-few-shot-machine-learning-for-named-entity-recognition-in-biomedical-texts\/\",\"url\":\"https:\/\/archive.belbi.bg.ac.rs\/2023\/abstract\/zero-and-few-shot-machine-learning-for-named-entity-recognition-in-biomedical-texts\/\",\"name\":\"Zero- and Few-Shot Machine Learning for Named Entity Recognition in Biomedical Texts - BelBi 2023\",\"isPartOf\":{\"@id\":\"https:\/\/archive.belbi.bg.ac.rs\/2023\/#website\"},\"datePublished\":\"2023-06-13T00:08:23+00:00\",\"dateModified\":\"2023-06-14T17:37:24+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/archive.belbi.bg.ac.rs\/2023\/abstract\/zero-and-few-shot-machine-learning-for-named-entity-recognition-in-biomedical-texts\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/archive.belbi.bg.ac.rs\/2023\/abstract\/zero-and-few-shot-machine-learning-for-named-entity-recognition-in-biomedical-texts\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/archive.belbi.bg.ac.rs\/2023\/abstract\/zero-and-few-shot-machine-learning-for-named-entity-recognition-in-biomedical-texts\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/archive.belbi.bg.ac.rs\/2023\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Zero- and Few-Shot Machine Learning for Named Entity Recognition in Biomedical Texts\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/archive.belbi.bg.ac.rs\/2023\/#website\",\"url\":\"https:\/\/archive.belbi.bg.ac.rs\/2023\/\",\"name\":\"BelBi 2023\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/archive.belbi.bg.ac.rs\/2023\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/archive.belbi.bg.ac.rs\/2023\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/archive.belbi.bg.ac.rs\/2023\/#organization\",\"name\":\"Belgrade Bioinformatics Conference\",\"url\":\"https:\/\/archive.belbi.bg.ac.rs\/2023\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/archive.belbi.bg.ac.rs\/2023\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/archive.belbi.bg.ac.rs\/2023\/wp-content\/uploads\/2023\/02\/145_97_171.png\",\"contentUrl\":\"https:\/\/archive.belbi.bg.ac.rs\/2023\/wp-content\/uploads\/2023\/02\/145_97_171.png\",\"width\":278,\"height\":500,\"caption\":\"Belgrade Bioinformatics Conference\"},\"image\":{\"@id\":\"https:\/\/archive.belbi.bg.ac.rs\/2023\/#\/schema\/logo\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Zero- and Few-Shot Machine Learning for Named Entity Recognition in Biomedical Texts - BelBi 2023","robots":{"index":"noindex","follow":"follow"},"og_locale":"en_US","og_type":"article","og_title":"Zero- and Few-Shot Machine Learning for Named Entity Recognition in Biomedical Texts - BelBi 2023","og_description":"Milo\u0161 Ko\u0161prdi\u01071*, Nikola Prodanovi\u01071, Adela Ljaji\u01071, Bojana Ba\u0161aragin1, and Nikola Milo\u0161evi\u01071,21Institute for Artificial Intelligence Research and Development of Serbia, Fru\u0161kogorska 1, Novi Sad, Serbia2Bayer A.G., Reaserch and Development, Mullerstrasse 173, Berlin, Germany Continue reading","og_url":"https:\/\/archive.belbi.bg.ac.rs\/2023\/abstract\/zero-and-few-shot-machine-learning-for-named-entity-recognition-in-biomedical-texts\/","og_site_name":"BelBi 2023","article_modified_time":"2023-06-14T17:37:24+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/archive.belbi.bg.ac.rs\/2023\/abstract\/zero-and-few-shot-machine-learning-for-named-entity-recognition-in-biomedical-texts\/","url":"https:\/\/archive.belbi.bg.ac.rs\/2023\/abstract\/zero-and-few-shot-machine-learning-for-named-entity-recognition-in-biomedical-texts\/","name":"Zero- and Few-Shot Machine Learning for Named Entity Recognition in Biomedical Texts - BelBi 2023","isPartOf":{"@id":"https:\/\/archive.belbi.bg.ac.rs\/2023\/#website"},"datePublished":"2023-06-13T00:08:23+00:00","dateModified":"2023-06-14T17:37:24+00:00","breadcrumb":{"@id":"https:\/\/archive.belbi.bg.ac.rs\/2023\/abstract\/zero-and-few-shot-machine-learning-for-named-entity-recognition-in-biomedical-texts\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/archive.belbi.bg.ac.rs\/2023\/abstract\/zero-and-few-shot-machine-learning-for-named-entity-recognition-in-biomedical-texts\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/archive.belbi.bg.ac.rs\/2023\/abstract\/zero-and-few-shot-machine-learning-for-named-entity-recognition-in-biomedical-texts\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/archive.belbi.bg.ac.rs\/2023\/"},{"@type":"ListItem","position":2,"name":"Zero- and Few-Shot Machine Learning for Named Entity Recognition in Biomedical Texts"}]},{"@type":"WebSite","@id":"https:\/\/archive.belbi.bg.ac.rs\/2023\/#website","url":"https:\/\/archive.belbi.bg.ac.rs\/2023\/","name":"BelBi 2023","description":"","publisher":{"@id":"https:\/\/archive.belbi.bg.ac.rs\/2023\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/archive.belbi.bg.ac.rs\/2023\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/archive.belbi.bg.ac.rs\/2023\/#organization","name":"Belgrade Bioinformatics Conference","url":"https:\/\/archive.belbi.bg.ac.rs\/2023\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/archive.belbi.bg.ac.rs\/2023\/#\/schema\/logo\/image\/","url":"https:\/\/archive.belbi.bg.ac.rs\/2023\/wp-content\/uploads\/2023\/02\/145_97_171.png","contentUrl":"https:\/\/archive.belbi.bg.ac.rs\/2023\/wp-content\/uploads\/2023\/02\/145_97_171.png","width":278,"height":500,"caption":"Belgrade Bioinformatics Conference"},"image":{"@id":"https:\/\/archive.belbi.bg.ac.rs\/2023\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/archive.belbi.bg.ac.rs\/2023\/wp-json\/wp\/v2\/abstract\/2796"}],"collection":[{"href":"https:\/\/archive.belbi.bg.ac.rs\/2023\/wp-json\/wp\/v2\/abstract"}],"about":[{"href":"https:\/\/archive.belbi.bg.ac.rs\/2023\/wp-json\/wp\/v2\/types\/abstract"}],"author":[{"embeddable":true,"href":"https:\/\/archive.belbi.bg.ac.rs\/2023\/wp-json\/wp\/v2\/users\/162"}],"wp:attachment":[{"href":"https:\/\/archive.belbi.bg.ac.rs\/2023\/wp-json\/wp\/v2\/media?parent=2796"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/archive.belbi.bg.ac.rs\/2023\/wp-json\/wp\/v2\/categories?post=2796"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/archive.belbi.bg.ac.rs\/2023\/wp-json\/wp\/v2\/tags?post=2796"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}