shamoji

gseby go-ego

Go efficient text segmentation; support english, chinese, japanese and other.

github.com/go-ego/gseGo 1.24+ 2 dependenciesv1.3.1updated 1d ago

gojiebaby yanyiwu

This is a Go implementation of jieba which a Chinese word splitting algorithm.

github.com/yanyiwu/gojiebaGo 1.17+ No dependencies v1.4.7updated 3mo ago

sentencesby neurosnap

Sentence tokenizer: converts text into a list of sentences.

github.com/neurosnap/sentencesGo 1.18+ No dependencies v1.1.2updated 29d ago

segmentby blevesearch

Go library for performing Unicode Text Segmentation as described in Unicode Standard Annex #29

github.com/blevesearch/segmentGo 1.18+ No dependencies updated 3mo ago

textcatby pebbe

Go package for n-gram based text categorization, with support for utf-8 and raw text.

github.com/pebbe/textcatGo 1.16+ 1 dependencyv2.3.0updated 1y ago

MMSEGOby awsong

This is a GO implementation of MMSEG which a Chinese word splitting algorithm.

updated 14y ago

stemmerby dchest

Stemmer packages for Go programming language. Includes English and German stemmers.

updated 9y ago

gotokenizerby xujiajun

A tokenizer based on the dictionary and Bigram language models for Golang. (Now only support chinese segmentation)

v1.1.0updated 7y ago