com.hankcs.hanlp.tokenizer

Class NotionalTokenizer

java.lang.Object

com.hankcs.hanlp.tokenizer.NotionalTokenizer

public class NotionalTokenizer
extends Object

实词分词器，自动移除停用词

Author:

hankcs

Field Summary

Fields

Modifier and Type	Field and Description
static Segment	SEGMENT 预置分词器

Constructor Summary

Constructors

Constructor and Description
NotionalTokenizer()

Method Summary

Modifier and Type	Method and Description
static List<List<Term>>	seg2sentence(String text) 切分为句子形式
static List<List<Term>>	seg2sentence(String text, boolean shortest) 分词断句 输出句子形式
static List<List<Term>>	seg2sentence(String text, Filter... filterArrayChain) 切分为句子形式
static List<Term>	segment(char[] text) 分词
static List<Term>	segment(String text)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SEGMENT

public static Segment SEGMENT

预置分词器

Constructor Detail

NotionalTokenizer

public NotionalTokenizer()

Method Detail

segment

public static List<Term> segment(String text)

segment

public static List<Term> segment(char[] text)

分词

Parameters:

text - 文本

Returns:

分词结果

seg2sentence

public static List<List<Term>> seg2sentence(String text)

切分为句子形式

Parameters:

text -

Returns:

seg2sentence

public static List<List<Term>> seg2sentence(String text,
                                            boolean shortest)

分词断句 输出句子形式

Parameters:

text - 待分词句子

shortest - 是否断句为最细的子句（将逗号也视作分隔符）

Returns:

句子列表，每个句子由一个单词列表组成

seg2sentence

public static List<List<Term>> seg2sentence(String text,
                                            Filter... filterArrayChain)

切分为句子形式

Parameters:

text -

filterArrayChain - 自定义过滤器链

Returns: