com.hankcs.hanlp.mining.word

Class NewWordDiscover

java.lang.Object

com.hankcs.hanlp.mining.word.NewWordDiscover

public class NewWordDiscover
extends Object

新词发现工具
在实现上参考了：https://github.com/Moonshile/ChineseWordSegmentation

Author:

hankcs

Constructor Summary

Constructors

Constructor and Description
NewWordDiscover()
NewWordDiscover(int max_word_len, float min_freq, float min_entropy, float min_aggregation, boolean filter) 构造一个新词识别工具

Method Summary

Modifier and Type	Method and Description
List<WordInfo>	discover(BufferedReader reader, int size) 提取词语
List<WordInfo>	discover(String doc, int size) 提取词语

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

NewWordDiscover

public NewWordDiscover()

NewWordDiscover

public NewWordDiscover(int max_word_len,
                       float min_freq,
                       float min_entropy,
                       float min_aggregation,
                       boolean filter)

构造一个新词识别工具

Parameters:

max_word_len - 词语最长长度

min_freq - 词语最低频率

min_entropy - 词语最低熵

min_aggregation - 词语最低互信息

filter - 是否过滤掉HanLP中的词库中已存在的词语

Method Detail

discover

public List<WordInfo> discover(BufferedReader reader,
                               int size)
                        throws IOException

提取词语

Parameters:

reader - 大文本

size - 需要提取词语的数量

Returns:

一个词语列表

Throws:

IOException

discover

public List<WordInfo> discover(String doc,
                               int size)

提取词语

Parameters:

doc - 大文本

size - 需要提取词语的数量

Returns:

一个词语列表