com.hankcs.hanlp.tokenizer

Class NLPTokenizer

java.lang.Object

com.hankcs.hanlp.tokenizer.NLPTokenizer

public class NLPTokenizer
extends Object

可供自然语言处理用的分词器，更重视准确率。

Author:

hankcs

Field Summary

Fields

Modifier and Type	Field and Description
static AbstractLexicalAnalyzer	ANALYZER 预置分词器

Constructor Summary

Constructors

Constructor and Description
NLPTokenizer()

Method Summary

Modifier and Type	Method and Description
static Sentence	analyze(String sentence) 词法分析
static List<List<Term>>	seg2sentence(String text) 切分为句子形式
static List<List<Term>>	seg2sentence(String text, boolean shortest) 分词断句 输出句子形式
static List<Term>	segment(char[] text) 分词
static List<Term>	segment(String text)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ANALYZER

public static AbstractLexicalAnalyzer ANALYZER

预置分词器

Constructor Detail

NLPTokenizer

public NLPTokenizer()

Method Detail

segment

public static List<Term> segment(String text)

segment

public static List<Term> segment(char[] text)

分词

Parameters:

text - 文本

Returns:

分词结果

seg2sentence

public static List<List<Term>> seg2sentence(String text)

切分为句子形式

Parameters:

text - 文本

Returns:

句子列表

analyze

public static Sentence analyze(String sentence)

词法分析

Parameters:

sentence -

Returns:

结构化句子

seg2sentence

public static List<List<Term>> seg2sentence(String text,
                                            boolean shortest)

分词断句 输出句子形式

Parameters:

text - 待分词句子

shortest - 是否断句为最细的子句（将逗号也视作分隔符）

Returns:

句子列表，每个句子由一个单词列表组成