A small experiment using both Mecab and Tinysegmenter to create a tokenized list of Japanese sentences in JSON, taken from the Tatoeba corpus.
-
Updated
Mar 25, 2021 - Python
#
tinysegmenter
Here is 1 public repository matching this topic...
Add this topic to your repo
To associate your repository with the tinysegmenter topic, visit your repo's landing page and select "manage topics."