com.hankcs.hanlp.mining.cluster

Class ClusterAnalyzer<K>

java.lang.Object

com.hankcs.hanlp.mining.cluster.ClusterAnalyzer<K>

Type Parameters:

K - 文档的id类型

public class ClusterAnalyzer<K>
extends Object

文本聚类

Author:

hankcs

Field Summary

Fields

Modifier and Type	Field and Description
protected HashMap<K,Document<K>>	documents_
protected Segment	segment
protected MutableDoubleArrayTrieInteger	vocabulary

Constructor Summary

Constructors

Constructor and Description
ClusterAnalyzer()

Method Summary

Modifier and Type	Method and Description
Document<K>	addDocument(K id, List<String> document) 添加文档
Document<K>	addDocument(K id, String document) 添加文档
static double	evaluate(String folderPath, String algorithm) 训练模型
protected int	id(String word)
List<Set<K>>	kmeans(int nclusters) k-means聚类
protected List<String>	preprocess(String document) 重载此方法实现自己的预处理逻辑（预处理、分词、去除停用词）
List<Set<K>>	repeatedBisection(double limit_eval) repeated bisection 聚类
List<Set<K>>	repeatedBisection(int nclusters) repeated bisection 聚类
List<Set<K>>	repeatedBisection(int nclusters, double limit_eval) repeated bisection 聚类
int	size() 已向聚类分析器添加的文档数量
protected SparseVector	toVector(List<String> wordList)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

documents_

protected HashMap<K,Document<K>> documents_

segment

protected Segment segment

vocabulary

protected MutableDoubleArrayTrieInteger vocabulary

Constructor Detail

ClusterAnalyzer

public ClusterAnalyzer()

Method Detail

id

protected int id(String word)

preprocess

protected List<String> preprocess(String document)

重载此方法实现自己的预处理逻辑（预处理、分词、去除停用词）

Parameters:

document - 文档

Returns:

单词列表

toVector

protected SparseVector toVector(List<String> wordList)

addDocument

public Document<K> addDocument(K id,
                               String document)

添加文档

Parameters:

id - 文档id

document - 文档内容

Returns:

文档对象

addDocument

public Document<K> addDocument(K id,
                               List<String> document)

添加文档

Parameters:

id - 文档id

document - 文档内容

Returns:

文档对象

kmeans

public List<Set<K>> kmeans(int nclusters)

k-means聚类

Parameters:

nclusters - 簇的数量

Returns:

指定数量的簇（Set）构成的集合

size

public int size()

已向聚类分析器添加的文档数量

Returns:

文档数量

repeatedBisection

public List<Set<K>> repeatedBisection(int nclusters)

repeated bisection 聚类

Parameters:

nclusters - 簇的数量

Returns:

指定数量的簇（Set）构成的集合

repeatedBisection

public List<Set<K>> repeatedBisection(double limit_eval)

repeated bisection 聚类

Parameters:

limit_eval - 准则函数增幅阈值

Returns:

指定数量的簇（Set）构成的集合

repeatedBisection

public List<Set<K>> repeatedBisection(int nclusters,
                                      double limit_eval)

repeated bisection 聚类

Parameters:

nclusters - 簇的数量

limit_eval - 准则函数增幅阈值

Returns:

指定数量的簇（Set）构成的集合

evaluate

public static double evaluate(String folderPath,
                              String algorithm)

训练模型

Parameters:

folderPath - 分类语料的根目录.目录必须满足如下结构:
根目录
├── 分类A
│ └── 1.txt
│ └── 2.txt
│ └── 3.txt
├── 分类B
│ └── 1.txt
│ └── ...
└── ...
文件不一定需要用数字命名,也不需要以txt作为后缀名,但一定需要是文本文件.

algorithm - kmeans 或 repeated bisection

Throws:

IOException - 任何可能的IO异常