Named Entity Recognition (NER)

Overview : This study model is constructed as feed forward neural network on top of pretrained BERT model using Tensorflow/Keras framework. Model composed of tokenization layer, BERT layer, dropout layer and a dense layer with softmax activation layer. BERT model and tokenizer is downloaded from TF_Hub.

Data

CoNLL003 (English-version) CONLL Corpora (2003) which is not downloaded with the NLTK Data:

https://www.kaggle.com/datasets/alaakhaled/conll003-englishversion

File Description :

• data_loader.py : data preparation
• enc_2tags.pkl and tags_enc.pkl : pickle file of dictionaries for tags and encoded values
• helper_functions.py : small functions for encoders
• token_functions.py : token alignment for learning algorithm
• model.py : nueral network model
• train.py : training file for learning algorithm
• prediction.py : prediction file for deployment purpose
• bert_ner_weights.h5 : model weights for later usage

Usage : if necessary download repo and create an virtual env using following commands

download file

conda create --name exp_env
conda activate exp_env

find the folder directory in exp_env

pip install -r requirements.txt 

run train.py file
for deployment purpose prediction file created seperately as prediction.py

EVALUATION

Classification Report

F1 score: 0.917871611886156

precision    recall  f1-score   support

       B-LOC       0.69      0.72      0.71      2231
      B-MISC       0.70      0.62      0.66      1049
       B-ORG       0.78      0.64      0.70      1430
       B-PER       0.76      0.64      0.70      1973
       I-LOC       0.68      0.69      0.69       264
      I-MISC       0.74      0.41      0.52       367
       I-ORG       0.64      0.50      0.56       794
       I-PER       0.70      0.61      0.65      1358
           O       0.95      0.97      0.96     49527

    accuracy                           0.92     58993
   macro avg       0.74      0.64      0.68     58993
weighted avg       0.91      0.92      0.91     58993

Confusion Matrix For Evaluation Set

Example

test_sentence : ['CRICKET - LEICESTERSHIRE TAKE OVER AT TOP AFTER INNINGS VICTORY .']
test_tags : O O B-ORG O O O O O O O O
pred_tags : O O B-ORG O O O O O O O O