org.apache.beam.fn.harness.status

Class MemoryMonitor

java.lang.Object

org.apache.beam.fn.harness.status.MemoryMonitor

All Implemented Interfaces:

java.lang.Runnable

public class MemoryMonitor
extends java.lang.Object
implements java.lang.Runnable

A runnable which monitors a server for GC thrashing.

Note: Only one instance of this should be initialized per server and it should be done when the server starts running.

This runnable works as follows:

• It wakes up periodically and determines how much time was spend on garbage collection since the last time it woke up.
• If the time spent in garbage collection in the last period of time exceeds a certain threshold, that period is marked as "being in GC thrashing"
• It keeps track of the GC thrashing status of the last few periods.
• Every time the runnable's thread wakes up, it computes the ratio (# monitored periods in GC thrashing) / (# monitored periods).
• If this ratio exceeds a certain threshold, it is assumed that the server is in GC thrashing.
• It can also shutdown the current jvm runtime when a threshold of consecutive gc thrashing count is met. A heap dump is made before shutdown.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	MemoryMonitor.GCStatsProvider Abstract interface for providing GC stats (for testing).

Field Summary

Fields

Modifier and Type	Field and Description
static long	DEFAULT_SLEEP_TIME_MILLIS Amount of time (in ms) this thread must sleep between two consecutive iterations.

Method Summary

Modifier and Type	Method and Description
java.lang.String	describeMemory() Return a string describing the current memory state of the server.
java.io.File	dumpHeap() Dump the current heap profile to a file in the given directory and return its name.
static MemoryMonitor	fromOptions(org.apache.beam.sdk.options.PipelineOptions options)
boolean	isThrashing()
void	run() Runs this thread.
void	stop() Request the memory monitor stops.
@Nullable java.io.File	tryToDumpHeap() Dumps the heap to a file and return the name of the file, or null if the heap should not or could not be dumped.
void	waitForResources(java.lang.String context) Return only when the server is not in the GC thrashing state.
void	waitForThrashingState(boolean desiredThrashingState) For testing only: Wait for thrashing status to be updated to given value.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_SLEEP_TIME_MILLIS

public static final long DEFAULT_SLEEP_TIME_MILLIS

Amount of time (in ms) this thread must sleep between two consecutive iterations.

See Also:

Constant Field Values

Method Detail

fromOptions

public static MemoryMonitor fromOptions(org.apache.beam.sdk.options.PipelineOptions options)

waitForThrashingState

public void waitForThrashingState(boolean desiredThrashingState)

For testing only: Wait for thrashing status to be updated to given value.

stop

public void stop()

Request the memory monitor stops.

isThrashing

public boolean isThrashing()

tryToDumpHeap

public @Nullable java.io.File tryToDumpHeap()

Dumps the heap to a file and return the name of the file, or null if the heap should not or could not be dumped.

Returns:

The name of the file the heap was dumped to, otherwise null.

run

public void run()

Runs this thread.

Specified by:

run in interface java.lang.Runnable

waitForResources

public void waitForResources(java.lang.String context)

Return only when the server is not in the GC thrashing state.

dumpHeap

public java.io.File dumpHeap()
                      throws javax.management.MalformedObjectNameException,
                             javax.management.InstanceNotFoundException,
                             javax.management.ReflectionException,
                             javax.management.MBeanException,
                             java.io.IOException

Dump the current heap profile to a file in the given directory and return its name.

NOTE: We deliberately don't salt the heap dump filename so as to minimize disk impact of repeated dumps. These files can be of comparable size to the local disk.

Throws:

javax.management.MalformedObjectNameException

javax.management.InstanceNotFoundException

javax.management.ReflectionException

javax.management.MBeanException

java.io.IOException

describeMemory

public java.lang.String describeMemory()

Return a string describing the current memory state of the server.