org.apache.arrow.memory

Class BaseAllocator

java.lang.Object

org.apache.arrow.memory.BaseAllocator

All Implemented Interfaces:

AutoCloseable, BufferAllocator

Direct Known Subclasses:

RootAllocator

public abstract class BaseAllocator
extends Object
implements BufferAllocator

A base-class that implements all functionality of BufferAllocators.

The class is abstract to enforce usage of RootAllocator/ChildAllocator facades.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	BaseAllocator.Reservation Implementation of AllocationReservation that supports history tracking under DEBUG is true.
static class	BaseAllocator.Verbosity Enum for logging verbosity.

Field Summary

Fields

Modifier and Type	Field and Description
static boolean	DEBUG
static String	DEBUG_ALLOCATOR
static int	DEBUG_LOG_LENGTH
protected org.apache.arrow.memory.Accountant	parent The parent allocator.
protected long	reservation The amount of memory reserved for this allocator.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BaseAllocator(BaseAllocator parentAllocator, AllocationListener listener, String name, long initReservation, long maxAllocation, RoundingPolicy roundingPolicy) Initialize an allocator

Method Summary

Modifier and Type	Method and Description
void	assertOpen() Asserts (using java assertions) that the provided allocator is currently open.
ArrowBuf	buffer(int initialRequestSize) Allocate a new or reused buffer of the provided size.
ArrowBuf	buffer(int initialRequestSize, BufferManager manager) Allocate a new or reused buffer of the provided size.
void	close() Close this Accountant.
long	getAllocatedMemory() Return the current amount of allocated memory that this Accountant is managing accounting for.
ArrowByteBufAllocator	getAsByteBufAllocator() Returns the allocator this allocator falls back to when it needs more memory.
Collection<BufferAllocator>	getChildAllocators() Returns the set of child allocators.
ArrowBuf	getEmpty() Get a reference to the empty buffer associated with this allocator.
long	getHeadroom()
long	getInitReservation() Return the initial reservation.
long	getLimit() Return the current limit of this Accountant.
String	getName() Return the name of the accountant.
BufferAllocator	getParentAllocator() Returns the parent allocator.
long	getPeakMemoryAllocation() The peak memory allocated by this Accountant.
static StringBuilder	indent(StringBuilder sb, int indent) Specify an indentation amount when using a StringBuilder.
static boolean	isDebug()
boolean	isOverLimit()
BufferAllocator	newChildAllocator(String name, AllocationListener listener, long initReservation, long maxAllocation) Create a new child allocator.
BufferAllocator	newChildAllocator(String name, long initReservation, long maxAllocation) Create a new child allocator.
AllocationReservation	newReservation() Create an allocation reservation.
static int	nextPowerOfTwo(int val) Rounds up the provided value to the nearest power of two.
static long	nextPowerOfTwo(long val) Rounds up the provided value to the nearest power of two.
void	releaseBytes(long size)
void	setLimit(long newLimit) Set the maximum amount of memory that can be allocated in the this Accountant before failing an allocation.
String	toString()
String	toVerboseString() Provide a verbose string of the current allocator state.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.arrow.memory.BufferAllocator

getAllocatedMemory, getHeadroom, getInitReservation, getLimit, getPeakMemoryAllocation, isOverLimit, setLimit

Field Detail

DEBUG_ALLOCATOR

public static final String DEBUG_ALLOCATOR

See Also:

Constant Field Values

DEBUG_LOG_LENGTH

public static final int DEBUG_LOG_LENGTH

See Also:

Constant Field Values

DEBUG

public static final boolean DEBUG

parent

protected final org.apache.arrow.memory.Accountant parent

The parent allocator.

reservation

protected final long reservation

The amount of memory reserved for this allocator. Releases below this amount of memory will not be returned to the parent Accountant until this Accountant is closed.

Constructor Detail

BaseAllocator

protected BaseAllocator(BaseAllocator parentAllocator,
                        AllocationListener listener,
                        String name,
                        long initReservation,
                        long maxAllocation,
                        RoundingPolicy roundingPolicy)
                 throws OutOfMemoryException

Initialize an allocator

Parameters:

parentAllocator - parent allocator. null if defining a root allocator

listener - listener callback. Must be non-null -- use AllocationListener.NOOP if no listener desired

name - name of this allocator

initReservation - initial reservation. Cannot be modified after construction

maxAllocation - limit. Allocations past the limit fail. Can be modified after construction

Throws:

OutOfMemoryException

Method Detail

getParentAllocator

public BufferAllocator getParentAllocator()

Description copied from interface: BufferAllocator

Returns the parent allocator.

Specified by:

getParentAllocator in interface BufferAllocator

Returns:

parent allocator

getChildAllocators

public Collection<BufferAllocator> getChildAllocators()

Description copied from interface: BufferAllocator

Returns the set of child allocators.

Specified by:

getChildAllocators in interface BufferAllocator

Returns:

set of child allocators

nextPowerOfTwo

public static int nextPowerOfTwo(int val)

Rounds up the provided value to the nearest power of two.

Parameters:

val - An integer value.

Returns:

The closest power of two of that value.

nextPowerOfTwo

public static long nextPowerOfTwo(long val)

Rounds up the provided value to the nearest power of two.

Parameters:

val - A long value.

Returns:

The closest power of two of that value.

indent

public static StringBuilder indent(StringBuilder sb,
                                   int indent)

Specify an indentation amount when using a StringBuilder.

Parameters:

sb - StringBuilder to use

indent - Indentation amount

Returns:

the StringBuilder object with indentation applied

isDebug

public static boolean isDebug()

assertOpen

public void assertOpen()

Description copied from interface: BufferAllocator

Asserts (using java assertions) that the provided allocator is currently open. If assertions are disabled, this is a no-op.

Specified by:

assertOpen in interface BufferAllocator

getName

public String getName()

Return the name of the accountant.

Specified by:

getName in interface BufferAllocator

Returns:

name of accountant

getEmpty

public ArrowBuf getEmpty()

Description copied from interface: BufferAllocator

Get a reference to the empty buffer associated with this allocator. Empty buffers are special because we don't worry about them leaking or managing reference counts on them since they don't actually point to any memory.

Specified by:

getEmpty in interface BufferAllocator

Returns:

the empty buffer

buffer

public ArrowBuf buffer(int initialRequestSize)

Description copied from interface: BufferAllocator

Allocate a new or reused buffer of the provided size. Note that the buffer may technically be larger than the requested size for rounding purposes. However, the buffer's capacity will be set to the configured size.

Specified by:

buffer in interface BufferAllocator

Parameters:

initialRequestSize - The size in bytes.

Returns:

a new ArrowBuf, or null if the request can't be satisfied

buffer

public ArrowBuf buffer(int initialRequestSize,
                       BufferManager manager)

Description copied from interface: BufferAllocator

Allocate a new or reused buffer of the provided size. Note that the buffer may technically be larger than the requested size for rounding purposes. However, the buffer's capacity will be set to the configured size.

Specified by:

buffer in interface BufferAllocator

Parameters:

initialRequestSize - The size in bytes.

manager - A buffer manager to manage reallocation.

Returns:

a new ArrowBuf, or null if the request can't be satisfied

getAsByteBufAllocator

public ArrowByteBufAllocator getAsByteBufAllocator()

Description copied from interface: BufferAllocator

Returns the allocator this allocator falls back to when it needs more memory.

Specified by:

getAsByteBufAllocator in interface BufferAllocator

Returns:

the underlying allocator used by this allocator

newChildAllocator

public BufferAllocator newChildAllocator(String name,
                                         long initReservation,
                                         long maxAllocation)

Description copied from interface: BufferAllocator

Create a new child allocator.

Specified by:

newChildAllocator in interface BufferAllocator

Parameters:

name - the name of the allocator.

initReservation - the initial space reservation (obtained from this allocator)

maxAllocation - maximum amount of space the new allocator can allocate

Returns:

the new allocator, or null if it can't be created

newChildAllocator

public BufferAllocator newChildAllocator(String name,
                                         AllocationListener listener,
                                         long initReservation,
                                         long maxAllocation)

Description copied from interface: BufferAllocator

Create a new child allocator.

Specified by:

newChildAllocator in interface BufferAllocator

Parameters:

name - the name of the allocator.

listener - allocation listener for the newly created child

initReservation - the initial space reservation (obtained from this allocator)

maxAllocation - maximum amount of space the new allocator can allocate

Returns:

the new allocator, or null if it can't be created

newReservation

public AllocationReservation newReservation()

Description copied from interface: BufferAllocator

Create an allocation reservation. A reservation is a way of building up a request for a buffer whose size is not known in advance. See

Specified by:

newReservation in interface BufferAllocator

Returns:

the newly created reservation

See Also:

AllocationReservation

close

public void close()

Close this Accountant. This will release any reservation bytes back to a parent Accountant.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface BufferAllocator

toString

public String toString()

Overrides:

toString in class Object

toVerboseString

public String toVerboseString()

Provide a verbose string of the current allocator state. Includes the state of all child allocators, along with historical logs of each object and including stacktraces.

Specified by:

toVerboseString in interface BufferAllocator

Returns:

A Verbose string of current allocator state.

releaseBytes

public void releaseBytes(long size)

isOverLimit

public boolean isOverLimit()

getLimit

public long getLimit()

Return the current limit of this Accountant.

Returns:

Limit in bytes.

getInitReservation

public long getInitReservation()

Return the initial reservation.

Returns:

reservation in bytes.

setLimit

public void setLimit(long newLimit)

Set the maximum amount of memory that can be allocated in the this Accountant before failing an allocation.

Parameters:

newLimit - The limit in bytes.

getAllocatedMemory

public long getAllocatedMemory()

Return the current amount of allocated memory that this Accountant is managing accounting for. Note this does not include reservation memory that hasn't been allocated.

Returns:

Currently allocate memory in bytes.

getPeakMemoryAllocation

public long getPeakMemoryAllocation()

The peak memory allocated by this Accountant.

Returns:

The peak allocated memory in bytes.

getHeadroom

public long getHeadroom()