Introduction
The Babel NER (Named Entity Recognition) Project uses 18 named entity labels: CARDINAL, DATE, EVENT, FAC, GPE, LANGUAGE, LAW, LOC, MONEY, NORP, ORDINAL, ORG, PERCENT, PERSON, PRODUCT, QUANTITY, TIME, WORK_OF_ART, corresponding to the information about spaCy model available here: model card.
The model was prepared to make predictions on sentence-level data. That does not mean you should provide input that is segmented into sentences. Each line will be splitted into separate sentences automatically during evaluation, with unique identifiers derived from the text identifier. After evaluation, the label is shown in a separate column after each sentence, followed by it's corresponding value from the sentence in the next column, and so on. For each recognized label, it will insert two columns into the output. The NER Babel Machine currently supports English and Hungarian and uses a multilingual model for every other language.
You can upload your datasets here for automated NER coding. If you wish to submit multiple datasets one after another, please wait 5-10 minutes between each of your submissions. There is only one possibility for upload: non-coded datasets. The explanation of the form and the dataset requirement is available here.
The upload requires you to fill the following form on metadata regarding the dataset.
The non-coded datasets should contain an id and a text column. The column names must be in row 1. You are free to add supplementary variables to the dataset beyond the compulsory ones in the columns following them.
You are free to add supplementary variables to the dataset beyond the compulsory ones in the columns following them. Automatic processing requires to follow these rules.
After you upload your dataset and your file is successfully processed, you will receive the NER-coded dataset and a file (in CSV format) that includes the predictions by the spaCy model. If the files you would like to upload are larger than 1 GB, please reach out to us with the download link attached (such as Dropbox or Google Drive) using our contact form.
If you have any questions or feedback regarding the Babel Machine, please let us know using our contact form. Please remember that we can only get back to you on Hungarian business days.
Submit a dataset:
The research was supported by the Ministry of Innovation and Technology NRDI Office within the RRF-2.3.1-21-2022-00004 Artificial Intelligence National Laboratory project and received additional funding from the European Union's Horizon 2020 program under grant agreement no 101008468. We also thank the Babel Machine project and HUN-REN Cloud (Héder et al. 2022; https://science-cloud.hu) for their support. We used the machine learning service of the Slices RI infrastructure (https://www.slices-ri.eu/)
HOW TO CITE: If you use the Babel Machine for your work or research, please cite this paper:
Sebők, M., Máté, Á., Ring, O., Kovács, V., & Lehoczki, R. (2024). Leveraging Open Large Language Models for Multilingual Policy Topic Classification: The Babel Machine Approach. Social Science Computer Review, 0(0). https://doi.org/10.1177/08944393241259434