2024-03-28T09:06:16Z
https://kitami-it.repo.nii.ac.jp/oai
oai:kitami-it.repo.nii.ac.jp:00008937
2022-12-13T02:22:29Z
1:87
Improving Basic Natural Language Processing Tools for the Ainu Language
Nowakowski, Karol
Ptaszynski, Michal
Masui, Fumito
Momouchi, Yoshio
open access
Ainu language
endangered languages
normalization
word segmentation
part-of-speech tagging
Ainu is a critically endangered language spoken by the native inhabitants of northern Japan. This paper describes our research aimed at the development of technology for automatic processing of text in Ainu. In particular, we improved the existing tools for normalizing old transcriptions, word segmentation, and part-of-speech tagging. In the experiments we applied two Ainu language dictionaries from different domains (literary and colloquial) and created a new data set by combining them. The experiments revealed that expanding the lexicon had a positive impact on the overall performance of our tools, especially with test data unrelated to any of the training sets used.
MDPI
2019
eng
journal article
VoR
https://kitami-it.repo.nii.ac.jp/records/8937
https://doi.org/10.3390/info10110329
2078-2489
Information
10
11
329
https://kitami-it.repo.nii.ac.jp/record/8937/files/Information 2019, 10(11), 329.pdf
application/pdf
333.6 kB
2020-11-02