org.apache.hadoop.hbase.wal

类 WALEdit

java.lang.Object

org.apache.hadoop.hbase.wal.WALEdit

所有已实现的接口:

HeapSize

@InterfaceAudience.LimitedPrivate(value={"Replication","Coprocesssor"})
public class WALEdit
extends Object
implements HeapSize

Used in HBase's transaction log (WAL) to represent a collection of edits (Cell/KeyValue objects) that came in as a single transaction. All the edits for a given transaction are written out as a single record, in PB format, followed (optionally) by Cells written via the WALCellEncoder.

This class is LimitedPrivate for CPs to read-only. The add(org.apache.hadoop.hbase.Cell, byte[]) methods are classified as private methods, not for use by CPs.

A particular WALEdit 'type' is the 'meta' type used to mark key operational events in the WAL such as compaction, flush, or region open. These meta types do not traverse hbase memstores. They are edits made by the hbase system rather than edit data submitted by clients. They only show in the WAL. These 'Meta' types have not been formally specified (or made into an explicit class type). They evolved organically. HBASE-8457 suggests codifying a WALEdit 'type' by adding a type field to WALEdit that gets serialized into the WAL. TODO. Would have to work on the consumption-side. Reading WALs on replay we seem to consume a Cell-at-a-time rather than by WALEdit. We are already in the below going out of our way to figure particular types -- e.g. if a compaction, replay, or close meta Marker -- during normal processing so would make sense to do this. Current system is an awkward marking of Cell columnfamily as METAFAMILY and then setting qualifier based off meta edit type. For replay-time where we read Cell-at-a-time, there are utility methods below for figuring meta type. See also createBulkLoadEvent(RegionInfo, WALProtos.BulkLoadDescriptor), etc., for where we create meta WALEdit instances.

WALEdit will accumulate a Set of all column family names referenced by the Cells add(Cell)'d. This is an optimization. Usually when loading a WALEdit, we have the column family name to-hand.. just shove it into the WALEdit if available. Doing this, we can save on a parse of each Cell to figure column family down the line when we go to add the WALEdit to the WAL file. See the hand-off in FSWALEntry Constructor.

另请参阅:

WALKey

字段概要

字段

限定符和类型	字段和说明
static byte[]	BULK_LOAD
static byte[]	COMPACTION 已过时。 Since 2.3.0. Make it protected, internal-use only. Use isCompactionMarker(Cell)
static byte[]	FLUSH 已过时。 Since 2.3.0. Make it protected, internal-use only.
static byte[]	METAFAMILY
static byte[]	METAROW 已过时。 Since 2.3.0. Not used.
static byte[]	REGION_EVENT 已过时。 Since 2.3.0. Remove. Not for external use. Not used.

构造器概要

构造器

构造器和说明
WALEdit()
WALEdit(boolean replay) 已过时。 since 2.0.1 and will be removed in 4.0.0. Use WALEdit(int, boolean) instead.
WALEdit(int cellCount) 已过时。 since 2.0.1 and will be removed in 4.0.0. Use WALEdit(int, boolean) instead.
WALEdit(int cellCount, boolean isReplay)

方法概要

限定符和类型	方法和说明
WALEdit	add(Cell cell)
WALEdit	add(Cell cell, byte[] family)
void	add(Map<byte[],List<Cell>> familyMap) Append the given map of family->edits to a WALEdit data structure.
static WALEdit	createBulkLoadEvent(RegionInfo hri, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.BulkLoadDescriptor bulkLoadDescriptor) Create a bulk loader WALEdit
static WALEdit	createCompaction(RegionInfo hri, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor c)
static WALEdit	createFlushWALEdit(RegionInfo hri, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor f)
static byte[]	createRegionEventDescriptorQualifier(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor.EventType t)
static WALEdit	createRegionEventWALEdit(byte[] rowForRegion, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor regionEventDesc)
static WALEdit	createRegionEventWALEdit(RegionInfo hri, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor regionEventDesc)
long	estimatedSerializedSizeOf()
static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.BulkLoadDescriptor	getBulkLoadDescriptor(Cell cell) Deserialized and returns a BulkLoadDescriptor from the passed in Cell
ArrayList<Cell>	getCells()
static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor	getCompaction(Cell kv) Deserialized and returns a CompactionDescriptor is the KeyValue contains one.
Set<byte[]>	getFamilies() For use by FSWALEntry ONLY.
static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor	getFlushDescriptor(Cell cell)
static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor	getRegionEventDescriptor(Cell cell)
static byte[]	getRowForRegion(RegionInfo hri)
long	heapSize()
static boolean	isCompactionMarker(Cell cell) Returns true if the given cell is a serialized WALProtos.CompactionDescriptor
boolean	isEmpty()
boolean	isMetaEdit()
static boolean	isMetaEditFamily(byte[] f) 已过时。 Since 2.3.0. Do not expose. Make protected.
static boolean	isMetaEditFamily(Cell cell) Replaying WALs can read Cell-at-a-time so need this method in those cases.
boolean	isRegionCloseMarker() Public so can be accessed from regionserver.wal package.
boolean	isReplay()
int	readFromCells(Codec.Decoder cellDecoder, int expectedCount) Reads WALEdit from cells.
void	setCells(ArrayList<Cell> cells) This is not thread safe.
int	size()
String	toString()

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

字段详细资料

METAFAMILY

public static final byte[] METAFAMILY

METAROW

@Deprecated
public static final byte[] METAROW

已过时。 Since 2.3.0. Not used.

COMPACTION

@Deprecated
public static final byte[] COMPACTION

已过时。 Since 2.3.0. Make it protected, internal-use only. Use isCompactionMarker(Cell)

FLUSH

@Deprecated
public static final byte[] FLUSH

已过时。 Since 2.3.0. Make it protected, internal-use only.

REGION_EVENT

@Deprecated
public static final byte[] REGION_EVENT

已过时。 Since 2.3.0. Remove. Not for external use. Not used.

BULK_LOAD

public static final byte[] BULK_LOAD

构造器详细资料

WALEdit

public WALEdit()

WALEdit

@Deprecated
public WALEdit(boolean replay)

已过时。 since 2.0.1 and will be removed in 4.0.0. Use WALEdit(int, boolean) instead.

另请参阅:

WALEdit(int, boolean), HBASE-20781

WALEdit

@Deprecated
public WALEdit(int cellCount)

已过时。 since 2.0.1 and will be removed in 4.0.0. Use WALEdit(int, boolean) instead.

另请参阅:

WALEdit(int, boolean), HBASE-20781

WALEdit

public WALEdit(int cellCount,
               boolean isReplay)

参数:

cellCount - Pass so can pre-size the WALEdit. Optimization.

方法详细资料

getFamilies

public Set<byte[]> getFamilies()

For use by FSWALEntry ONLY. An optimization.

返回:

All families in getCells(); may be null.

isMetaEditFamily

@Deprecated
public static boolean isMetaEditFamily(byte[] f)

已过时。 Since 2.3.0. Do not expose. Make protected.

返回:

True is f is METAFAMILY

isMetaEditFamily

public static boolean isMetaEditFamily(Cell cell)

Replaying WALs can read Cell-at-a-time so need this method in those cases.

isMetaEdit

public boolean isMetaEdit()

返回:

True if this is a meta edit; has one edit only and its columnfamily is METAFAMILY.

isReplay

public boolean isReplay()

返回:

True when current WALEdit is created by log replay. Replication skips WALEdits from replay.

add

@InterfaceAudience.Private
public WALEdit add(Cell cell,
                                              byte[] family)

add

@InterfaceAudience.Private
public WALEdit add(Cell cell)

isEmpty

public boolean isEmpty()

size

public int size()

getCells

public ArrayList<Cell> getCells()

setCells

@InterfaceAudience.Private
public void setCells(ArrayList<Cell> cells)

This is not thread safe. This will change the WALEdit and shouldn't be used unless you are sure that nothing else depends on the contents being immutable.

参数:

cells - the list of cells that this WALEdit now contains.

readFromCells

public int readFromCells(Codec.Decoder cellDecoder,
                         int expectedCount)
                  throws IOException

Reads WALEdit from cells.

参数:

cellDecoder - Cell decoder.

expectedCount - Expected cell count.

返回:

Number of KVs read.

抛出:

IOException

heapSize

public long heapSize()

指定者:

heapSize 在接口中 HeapSize

estimatedSerializedSizeOf

public long estimatedSerializedSizeOf()

toString

public String toString()

覆盖:

toString 在类中 Object

createFlushWALEdit

public static WALEdit createFlushWALEdit(RegionInfo hri,
                                         org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor f)

getFlushDescriptor

public static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor getFlushDescriptor(Cell cell)
                                                                                                      throws IOException

抛出:

IOException

createRegionEventWALEdit

public static WALEdit createRegionEventWALEdit(RegionInfo hri,
                                               org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor regionEventDesc)

返回:

A meta Marker WALEdit that has a single Cell whose value is the passed in regionEventDesc serialized and whose row is this region, columnfamily is METAFAMILY and qualifier is REGION_EVENT_PREFIX + WALProtos.RegionEventDescriptor.getEventType(); for example HBASE::REGION_EVENT::REGION_CLOSE.

createRegionEventWALEdit

public static WALEdit createRegionEventWALEdit(byte[] rowForRegion,
                                               org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor regionEventDesc)

createRegionEventDescriptorQualifier

public static byte[] createRegionEventDescriptorQualifier(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor.EventType t)

返回:

Cell qualifier for the passed in RegionEventDescriptor Type; e.g. we'll return something like a byte array with HBASE::REGION_EVENT::REGION_OPEN in it.

isRegionCloseMarker

public boolean isRegionCloseMarker()

Public so can be accessed from regionserver.wal package.

返回:

True if this is a Marker Edit and it is a RegionClose type.

getRegionEventDescriptor

public static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor getRegionEventDescriptor(Cell cell)
                                                                                                                  throws IOException

返回:

Returns a RegionEventDescriptor made by deserializing the content of the passed in cell, IFF the cell is a RegionEventDescriptor type WALEdit.

抛出:

IOException

createCompaction

public static WALEdit createCompaction(RegionInfo hri,
                                       org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor c)

返回:

A Marker WALEdit that has c serialized as its value

getRowForRegion

public static byte[] getRowForRegion(RegionInfo hri)

getCompaction

public static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor getCompaction(Cell kv)
                                                                                                      throws IOException

Deserialized and returns a CompactionDescriptor is the KeyValue contains one.

参数:

kv - the key value

返回:

deserialized CompactionDescriptor or null.

抛出:

IOException

isCompactionMarker

public static boolean isCompactionMarker(Cell cell)

Returns true if the given cell is a serialized WALProtos.CompactionDescriptor

另请参阅:

getCompaction(Cell)

createBulkLoadEvent

public static WALEdit createBulkLoadEvent(RegionInfo hri,
                                          org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.BulkLoadDescriptor bulkLoadDescriptor)

Create a bulk loader WALEdit

参数:

hri - The RegionInfo for the region in which we are bulk loading

bulkLoadDescriptor - The descriptor for the Bulk Loader

返回:

The WALEdit for the BulkLoad

getBulkLoadDescriptor

public static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.BulkLoadDescriptor getBulkLoadDescriptor(Cell cell)
                                                                                                            throws IOException

Deserialized and returns a BulkLoadDescriptor from the passed in Cell

参数:

cell - the key value

返回:

deserialized BulkLoadDescriptor or null.

抛出:

IOException

add

public void add(Map<byte[],List<Cell>> familyMap)

Append the given map of family->edits to a WALEdit data structure. This does not write to the WAL itself. Note that as an optimization, we will stamp the Set of column families into the WALEdit to save on our having to calculate column families subsequently down in the actual WAL writing.

参数:

familyMap - map of family->edits