org.apdplat.word.segmentation.impl

Class AbstractSegmentation

java.lang.Object

org.apdplat.word.segmentation.impl.AbstractSegmentation

All Implemented Interfaces:

DictionaryBasedSegmentation, Segmentation

Direct Known Subclasses:

BidirectionalMaximumMatching, BidirectionalMaximumMinimumMatching, BidirectionalMinimumMatching, FullSegmentation, MaximumMatching, MaxNgramScore, MinimalWordCount, MinimumMatching, ReverseMaximumMatching, ReverseMinimumMatching

public abstract class AbstractSegmentation
extends Object
implements DictionaryBasedSegmentation

基于词典的分词算法抽象类

Author:

杨尚川

Field Summary

Fields

Modifier and Type	Field and Description
protected org.slf4j.Logger	LOGGER

Constructor Summary

Constructors

Constructor and Description
AbstractSegmentation()

Method Summary

Modifier and Type	Method and Description
protected void	addWord(List<Word> result, String text, int start, int len) 将识别出的词放入队列
protected void	addWord(Stack<Word> result, String text, int start, int len) 将识别出的词入栈
Dictionary	getDictionary() 获取词典操作接口
int	getInterceptLength() 分词时截取的字符串的最大长度
protected Word	getWord(String text, int start, int len) 获取一个已经识别的词
boolean	isParallelSeg()
static void	main(String[] args)
Map<List<Word>,Float>	ngram(List<Word>... sentences) 利用ngram进行评分
boolean	ngramEnabled() 是否启用ngram
List<Word>	seg(String text) 默认分词算法实现： 1、把要分词的文本根据标点符号进行分割 2、对分割后的文本进行分词 3、组合分词结果
abstract List<Word>	segImpl(String text) 具体的分词实现，留待子类实现
void	setDictionary(Dictionary dictionary) 为基于词典的中文分词接口指定词典操作接口

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apdplat.word.segmentation.Segmentation

getSegmentationAlgorithm

Field Detail

LOGGER

protected final org.slf4j.Logger LOGGER

Constructor Detail

AbstractSegmentation

public AbstractSegmentation()

Method Detail

isParallelSeg

public boolean isParallelSeg()

setDictionary

public void setDictionary(Dictionary dictionary)

为基于词典的中文分词接口指定词典操作接口

Specified by:

setDictionary in interface DictionaryBasedSegmentation

Parameters:

dictionary - 词典操作接口

getDictionary

public Dictionary getDictionary()

获取词典操作接口

Specified by:

getDictionary in interface DictionaryBasedSegmentation

Returns:

词典操作接口

segImpl

public abstract List<Word> segImpl(String text)

具体的分词实现，留待子类实现

Parameters:

text - 文本

Returns:

分词结果

ngramEnabled

public boolean ngramEnabled()

是否启用ngram

Returns:

是或否

ngram

public Map<List<Word>,Float> ngram(List<Word>... sentences)

利用ngram进行评分

Parameters:

sentences - 多个分词结果

Returns:

评分后的结果

getInterceptLength

public int getInterceptLength()

分词时截取的字符串的最大长度

Returns:

seg

public List<Word> seg(String text)

默认分词算法实现： 1、把要分词的文本根据标点符号进行分割 2、对分割后的文本进行分词 3、组合分词结果

Specified by:

seg in interface Segmentation

Parameters:

text - 文本

Returns:

分词结果

addWord

protected void addWord(List<Word> result,
                       String text,
                       int start,
                       int len)

将识别出的词放入队列

Parameters:

result - 队列

text - 文本

start - 词开始索引

len - 词长度

addWord

protected void addWord(Stack<Word> result,
                       String text,
                       int start,
                       int len)

将识别出的词入栈

Parameters:

result - 栈

text - 文本

start - 词开始索引

len - 词长度

getWord

protected Word getWord(String text,
                       int start,
                       int len)

获取一个已经识别的词

Parameters:

text - 文本

start - 词开始索引

len - 词长度

Returns:

词或空

main

public static void main(String[] args)