Package org.apache.accumulo.tserver.compaction.strategies

Class BasicCompactionStrategy

java.lang.Object

org.apache.accumulo.tserver.compaction.CompactionStrategy

org.apache.accumulo.tserver.compaction.DefaultCompactionStrategy

org.apache.accumulo.tserver.compaction.strategies.BasicCompactionStrategy

public class BasicCompactionStrategy
extends DefaultCompactionStrategy

A compaction strategy that covers the following uses cases.

• Filtering out input files larger than a specified size. These are never considered for compaction.
• Compressing output files differently when the sum of the input files exceeds a specified size.

To filter out input files based on size set table.majc.compaction.strategy.opts.filter.size to the desired size.

To use a different compression for larger inputs set table.majc.compaction.strategy.opts.large.compress.threshold to bytes and table.majc.compaction.strategy.opts.large.compress.type to a compression type like gz or snappy. When setting one of these properties then the other must be set. When the total size of files being compacted is larger than the threshold then the specified compression type is used.

To use this strategy with Minor Compactions set table.file.compress.type=snappy and set a different compress type in table.majc.compaction.strategy.opts.large.compress.type for larger files.

The options that take sizes are in bytes and the suffixes K,M,and G can be used.

Field Summary

Fields

Modifier and Type	Field	Description
static String	LARGE_FILE_COMPRESSION_THRESHOLD	Threshold memory in bytes.
static String	LARGE_FILE_COMPRESSION_TYPE	Type of compression to use if large threshold is surpassed.
static String	SIZE_LIMIT_OPT

Constructor Summary

Constructors

Constructor	Description
BasicCompactionStrategy()

Method Summary

Modifier and Type	Method	Description
void	gatherInformation(MajorCompactionRequest request)	Called prior to obtaining the tablet lock, useful for examining metadata or indexes.
CompactionPlan	getCompactionPlan(MajorCompactionRequest request)	Get the plan for compacting a tablets files.
void	init(Map<String,String> options)	The settings for the compaction strategy pulled from zookeeper.
boolean	shouldCompact(MajorCompactionRequest request)	Determine if this tablet is eligible for a major compaction.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SIZE_LIMIT_OPT

public static final String SIZE_LIMIT_OPT

See Also:

Constant Field Values

LARGE_FILE_COMPRESSION_THRESHOLD

public static final String LARGE_FILE_COMPRESSION_THRESHOLD

Threshold memory in bytes. Files larger than this threshold will use table.majc.compaction.strategy.opts.file.large.compress.type for compression

See Also:

Constant Field Values

LARGE_FILE_COMPRESSION_TYPE

public static final String LARGE_FILE_COMPRESSION_TYPE

Type of compression to use if large threshold is surpassed. One of "gz","lzo","snappy", or "none"

See Also:

Constant Field Values

Constructor Detail

BasicCompactionStrategy

public BasicCompactionStrategy()

Method Detail

init

public void init(Map<String,String> options)

Description copied from class: CompactionStrategy

The settings for the compaction strategy pulled from zookeeper. The table.compacations.major.strategy.opts part of the setting will be removed.

Overrides:

init in class CompactionStrategy

shouldCompact

public boolean shouldCompact(MajorCompactionRequest request)

Description copied from class: CompactionStrategy

Determine if this tablet is eligible for a major compaction. It's ok if it later determines (through CompactionStrategy.gatherInformation(MajorCompactionRequest) and CompactionStrategy.getCompactionPlan(MajorCompactionRequest)) that it does not need to. Any state stored during shouldCompact will no longer exist when CompactionStrategy.gatherInformation(MajorCompactionRequest) and CompactionStrategy.getCompactionPlan(MajorCompactionRequest) are called.

Called while holding the tablet lock, so it should not be doing any blocking.

Since no blocking should be done in this method, then its unexpected that this method will throw IOException. However since its in the API, it can not be easily removed.

Overrides:

shouldCompact in class DefaultCompactionStrategy

gatherInformation

public void gatherInformation(MajorCompactionRequest request)
                       throws IOException

Description copied from class: CompactionStrategy

Called prior to obtaining the tablet lock, useful for examining metadata or indexes. State collected during this method will be available during the call the CompactionStrategy.getCompactionPlan(MajorCompactionRequest).

Overrides:

gatherInformation in class CompactionStrategy

Parameters:

request - basic details about the tablet

Throws:

IOException

getCompactionPlan

public CompactionPlan getCompactionPlan(MajorCompactionRequest request)

Description copied from class: CompactionStrategy

Get the plan for compacting a tablets files. Called while holding the tablet lock, so it should not be doing any blocking.

Since no blocking should be done in this method, then its unexpected that this method will throw IOException. However since its in the API, it can not be easily removed.

Overrides:

getCompactionPlan in class DefaultCompactionStrategy

Parameters:

request - basic details about the tablet

Returns:

the plan for a major compaction, or null to cancel the compaction.