gse

gojiebaby yanyiwu

This is a Go implementation of jieba which a Chinese word splitting algorithm.

github.com/yanyiwu/gojiebaGo 1.17+ No dependencies v1.4.7updated 3mo ago

sentencesby neurosnap

Sentence tokenizer: converts text into a list of sentences.

github.com/neurosnap/sentencesGo 1.18+ No dependencies v1.1.2updated 29d ago

segmentby blevesearch

Go library for performing Unicode Text Segmentation as described in Unicode Standard Annex #29

github.com/blevesearch/segmentGo 1.18+ No dependencies updated 3mo ago

textcatby pebbe

Go package for n-gram based text categorization, with support for utf-8 and raw text.

github.com/pebbe/textcatGo 1.16+ 1 dependencyv2.3.0updated 1y ago

MMSEGOby awsong

This is a GO implementation of MMSEG which a Chinese word splitting algorithm.

updated 14y ago

stemmerby dchest

Stemmer packages for Go programming language. Includes English and German stemmers.

updated 9y ago

gotokenizerby xujiajun

A tokenizer based on the dictionary and Bigram language models for Golang. (Now only support chinese segmentation)

v1.1.0updated 7y ago

shamojiby osamingo

The shamoji is word filtering package written in Go.

github.com/osamingo/shamojiGo 1.24+ 5 dependenciesv1.3.0updated 1y ago