org.deeplearning4j.ui.stats.impl

Class SbeStatsReport

java.lang.Object

org.deeplearning4j.ui.stats.impl.SbeStatsReport

All Implemented Interfaces:

Serializable, Persistable, StatsReport, AgronaPersistable

public class SbeStatsReport
extends Object
implements StatsReport, AgronaPersistable

An implementation of StatsReport using Simple Binary Encoding (SBE)

Author:

Alex Black

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SbeStatsReport()

Method Summary

Modifier and Type	Method and Description
void	decode(byte[] decode)
void	decode(ByteBuffer buffer)
void	decode(org.agrona.DirectBuffer buffer)
void	decode(InputStream inputStream)
byte[]	encode()
void	encode(ByteBuffer buffer)
void	encode(org.agrona.MutableDirectBuffer buffer)
void	encode(OutputStream outputStream)
int	encodingLengthBytes()
List<Serializable>	getDataSetMetaData() Get the DataSet metadata, if any (null otherwise).
String	getDataSetMetaDataClassName() Get the class
List<org.nd4j.linalg.primitives.Pair<String,int[]>>	getGarbageCollectionStats() Get the garbage collection stats: Pair contains GC name and the delta count/time values
Map<String,Histogram>	getHistograms(StatsType statsType) Get the histograms for all parameters, for a given StatsType (Parameters/Updates/Activations)
Map<String,Double>	getLearningRates() Get the learning rates by parameter
Map<String,Double>	getMean(StatsType statsType) Get the mean values for each parameter for the given StatsType (Parameters/Updates/Activations)
Map<String,Double>	getMeanMagnitudes(StatsType statsType) Get the mean magnitude values for each parameter for the given StatsType (Parameters/Updates/Activations)
String	getSessionID()
Map<String,Double>	getStdev(StatsType statsType) Get the standard deviation values for each parameter for the given StatsType (Parameters/Updates/Activations)
long	getTimeStamp()
String	getTypeID()
String	getWorkerID()
boolean	hasDataSetMetaData() Return whether any DataSet metadata is present or not
boolean	hasGarbageCollection() Return whether garbage collection information has been reported
boolean	hasHistograms(StatsType statsType) Return whether histograms have been reported, for the given stats type (Parameters, Updates, Activations)
boolean	hasLearningRates() Return whether the learning rates are present (have been reported)
boolean	hasMemoryUse() Return whether memory use has been reported
boolean	hasPerformance() Return whether performance stats (total time, total examples etc) have been reported
boolean	hasScore() Return whether the score is present (has been reported)
boolean	hasSummaryStats(StatsType statsType, SummaryType summaryType) Return whether the summary stats (mean, standard deviation, mean magnitudes) have been reported for the given stats type (Parameters, Updates, Activations)
void	reportDataSetMetaData(List<Serializable> dataSetMetaData, Class<?> metaDataClass) Report any metadata for the DataSet
void	reportDataSetMetaData(List<Serializable> dataSetMetaData, String metaDataClass) Report any metadata for the DataSet
void	reportGarbageCollection(String gcName, int deltaGCCount, int deltaGCTime) Report Garbage collection stats
void	reportHistograms(StatsType statsType, Map<String,Histogram> histogram) Report histograms for all parameters, for a given StatsType
void	reportIDs(String sessionID, String typeID, String workerID, long timeStamp)
void	reportIterationCount(int iterationCount) Report the current iteration number
void	reportLearningRates(Map<String,Double> learningRatesByParam) Report the learning rates by parameter
void	reportMean(StatsType statsType, Map<String,Double> mean) Report the mean values for each parameter, the given StatsType (Parameters/Updates/Activations)
void	reportMeanMagnitudes(StatsType statsType, Map<String,Double> meanMagnitudes) Report the mean magnitude values for each parameter for the given StatsType (Parameters/Updates/Activations)
void	reportMemoryUse(long jvmCurrentBytes, long jvmMaxBytes, long offHeapCurrentBytes, long offHeapMaxBytes, long[] deviceCurrentBytes, long[] deviceMaxBytes) Report the memory stats at this iteration
void	reportPerformance(long totalRuntimeMs, long totalExamples, long totalMinibatches, double examplesPerSecond, double minibatchesPerSecond) Report the performance stats (since the last report)
void	reportScore(double currentScore) Report model score at the current iteration
void	reportStatsCollectionDurationMS(int statsCollectionDurationMS) Report the number of milliseconds required to calculate all of the stats.
void	reportStdev(StatsType statsType, Map<String,Double> stdev) Report the standard deviation values for each parameter for the given StatsType (Parameters/Updates/Activations)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.deeplearning4j.ui.stats.api.StatsReport

getDeviceCurrentBytes, getDeviceMaxBytes, getExamplesPerSecond, getIterationCount, getJvmCurrentBytes, getJvmMaxBytes, getMinibatchesPerSecond, getOffHeapCurrentBytes, getOffHeapMaxBytes, getScore, getStatsCollectionDurationMs, getTotalExamples, getTotalMinibatches, getTotalRuntimeMs

Constructor Detail

SbeStatsReport

public SbeStatsReport()

Method Detail

reportIDs

public void reportIDs(String sessionID,
                      String typeID,
                      String workerID,
                      long timeStamp)

Specified by:

reportIDs in interface StatsReport

reportIterationCount

public void reportIterationCount(int iterationCount)

Description copied from interface: StatsReport

Report the current iteration number

Specified by:

reportIterationCount in interface StatsReport

reportStatsCollectionDurationMS

public void reportStatsCollectionDurationMS(int statsCollectionDurationMS)

Description copied from interface: StatsReport

Report the number of milliseconds required to calculate all of the stats. This is effectively the amount of listener overhead

Specified by:

reportStatsCollectionDurationMS in interface StatsReport

reportScore

public void reportScore(double currentScore)

Description copied from interface: StatsReport

Report model score at the current iteration

Specified by:

reportScore in interface StatsReport

reportLearningRates

public void reportLearningRates(Map<String,Double> learningRatesByParam)

Description copied from interface: StatsReport

Report the learning rates by parameter

Specified by:

reportLearningRates in interface StatsReport

getLearningRates

public Map<String,Double> getLearningRates()

Description copied from interface: StatsReport

Get the learning rates by parameter

Specified by:

getLearningRates in interface StatsReport

reportMemoryUse

public void reportMemoryUse(long jvmCurrentBytes,
                            long jvmMaxBytes,
                            long offHeapCurrentBytes,
                            long offHeapMaxBytes,
                            long[] deviceCurrentBytes,
                            long[] deviceMaxBytes)

Description copied from interface: StatsReport

Report the memory stats at this iteration

Specified by:

reportMemoryUse in interface StatsReport

Parameters:

jvmCurrentBytes - Current bytes used by the JVM

jvmMaxBytes - Max bytes usable by the JVM (heap)

offHeapCurrentBytes - Current off-heap bytes used

offHeapMaxBytes - Maximum off-heap bytes

deviceCurrentBytes - Current bytes used by each device (GPU, etc). May be null if no devices are present

deviceMaxBytes - Maximum bytes for each device (GPU, etc). May be null if no devices are present

reportPerformance

public void reportPerformance(long totalRuntimeMs,
                              long totalExamples,
                              long totalMinibatches,
                              double examplesPerSecond,
                              double minibatchesPerSecond)

Description copied from interface: StatsReport

Report the performance stats (since the last report)

Specified by:

reportPerformance in interface StatsReport

Parameters:

totalRuntimeMs - Overall runtime since initialization

totalExamples - Total examples processed since initialization

totalMinibatches - Total number of minibatches (iterations) since initialization

examplesPerSecond - Examples per second since last report

minibatchesPerSecond - Minibatches per second since last report

reportGarbageCollection

public void reportGarbageCollection(String gcName,
                                    int deltaGCCount,
                                    int deltaGCTime)

Description copied from interface: StatsReport

Report Garbage collection stats

Specified by:

reportGarbageCollection in interface StatsReport

Parameters:

gcName - Garbage collector name

deltaGCCount - Change in the total number of garbage collections, since last report

deltaGCTime - Change in the amount of time (milliseconds) for garbage collection, since last report

getGarbageCollectionStats

public List<org.nd4j.linalg.primitives.Pair<String,int[]>> getGarbageCollectionStats()

Description copied from interface: StatsReport

Get the garbage collection stats: Pair contains GC name and the delta count/time values

Specified by:

getGarbageCollectionStats in interface StatsReport

reportHistograms

public void reportHistograms(StatsType statsType,
                             Map<String,Histogram> histogram)

Description copied from interface: StatsReport

Report histograms for all parameters, for a given StatsType

Specified by:

reportHistograms in interface StatsReport

Parameters:

statsType - StatsType: Parameters, Updates, Activations

histogram - Histogram values for all parameters

getHistograms

public Map<String,Histogram> getHistograms(StatsType statsType)

Description copied from interface: StatsReport

Get the histograms for all parameters, for a given StatsType (Parameters/Updates/Activations)

Specified by:

getHistograms in interface StatsReport

Parameters:

statsType - Stats type (Params/updatse/activations) to get histograms for

Returns:

Histograms by parameter name, or null if not available

reportMean

public void reportMean(StatsType statsType,
                       Map<String,Double> mean)

Description copied from interface: StatsReport

Report the mean values for each parameter, the given StatsType (Parameters/Updates/Activations)

Specified by:

reportMean in interface StatsReport

Parameters:

statsType - Stats type to report

mean - Map of mean values, by parameter

getMean

public Map<String,Double> getMean(StatsType statsType)

Description copied from interface: StatsReport

Get the mean values for each parameter for the given StatsType (Parameters/Updates/Activations)

Specified by:

getMean in interface StatsReport

Parameters:

statsType - Stats type to get mean values for

Returns:

Map of mean values by parameter

reportStdev

public void reportStdev(StatsType statsType,
                        Map<String,Double> stdev)

Description copied from interface: StatsReport

Report the standard deviation values for each parameter for the given StatsType (Parameters/Updates/Activations)

Specified by:

reportStdev in interface StatsReport

Parameters:

statsType - Stats type to report std. dev values for

stdev - Map of std dev values by parameter

getStdev

public Map<String,Double> getStdev(StatsType statsType)

Description copied from interface: StatsReport

Get the standard deviation values for each parameter for the given StatsType (Parameters/Updates/Activations)

Specified by:

getStdev in interface StatsReport

Parameters:

statsType - Stats type to get std dev values for

Returns:

Map of stdev values by parameter

reportMeanMagnitudes

public void reportMeanMagnitudes(StatsType statsType,
                                 Map<String,Double> meanMagnitudes)

Description copied from interface: StatsReport

Report the mean magnitude values for each parameter for the given StatsType (Parameters/Updates/Activations)

Specified by:

reportMeanMagnitudes in interface StatsReport

Parameters:

statsType - Stats type to report mean magnitude values for

meanMagnitudes - Map of mean magnitude values by parameter

reportDataSetMetaData

public void reportDataSetMetaData(List<Serializable> dataSetMetaData,
                                  Class<?> metaDataClass)

Description copied from interface: StatsReport

Report any metadata for the DataSet

Specified by:

reportDataSetMetaData in interface StatsReport

Parameters:

dataSetMetaData - MetaData for the DataSet

metaDataClass - Class of the metadata. Can be later retieved using StatsReport.getDataSetMetaDataClassName()

reportDataSetMetaData

public void reportDataSetMetaData(List<Serializable> dataSetMetaData,
                                  String metaDataClass)

Description copied from interface: StatsReport

Report any metadata for the DataSet

Specified by:

reportDataSetMetaData in interface StatsReport

Parameters:

dataSetMetaData - MetaData for the DataSet

metaDataClass - Class of the metadata. Can be later retieved using StatsReport.getDataSetMetaDataClassName()

getMeanMagnitudes

public Map<String,Double> getMeanMagnitudes(StatsType statsType)

Description copied from interface: StatsReport

Get the mean magnitude values for each parameter for the given StatsType (Parameters/Updates/Activations)

Specified by:

getMeanMagnitudes in interface StatsReport

Parameters:

statsType - Stats type to get mean magnitude values for

Returns:

Map of mean magnitude values by parameter

getDataSetMetaData

public List<Serializable> getDataSetMetaData()

Description copied from interface: StatsReport

Get the DataSet metadata, if any (null otherwise). Note: due to serialization issues, this may in principle throw an unchecked exception related to class availability, serialization etc.

Specified by:

getDataSetMetaData in interface StatsReport

Returns:

List of DataSet metadata, if any.

getDataSetMetaDataClassName

public String getDataSetMetaDataClassName()

Description copied from interface: StatsReport

Get the class

Specified by:

getDataSetMetaDataClassName in interface StatsReport

Returns:

hasScore

public boolean hasScore()

Description copied from interface: StatsReport

Return whether the score is present (has been reported)

Specified by:

hasScore in interface StatsReport

hasLearningRates

public boolean hasLearningRates()

Description copied from interface: StatsReport

Return whether the learning rates are present (have been reported)

Specified by:

hasLearningRates in interface StatsReport

hasMemoryUse

public boolean hasMemoryUse()

Description copied from interface: StatsReport

Return whether memory use has been reported

Specified by:

hasMemoryUse in interface StatsReport

hasPerformance

public boolean hasPerformance()

Description copied from interface: StatsReport

Return whether performance stats (total time, total examples etc) have been reported

Specified by:

hasPerformance in interface StatsReport

hasGarbageCollection

public boolean hasGarbageCollection()

Description copied from interface: StatsReport

Return whether garbage collection information has been reported

Specified by:

hasGarbageCollection in interface StatsReport

hasHistograms

public boolean hasHistograms(StatsType statsType)

Description copied from interface: StatsReport

Return whether histograms have been reported, for the given stats type (Parameters, Updates, Activations)

Specified by:

hasHistograms in interface StatsReport

Parameters:

statsType - Stats type

hasSummaryStats

public boolean hasSummaryStats(StatsType statsType,
                               SummaryType summaryType)

Description copied from interface: StatsReport

Return whether the summary stats (mean, standard deviation, mean magnitudes) have been reported for the given stats type (Parameters, Updates, Activations)

Specified by:

hasSummaryStats in interface StatsReport

Parameters:

statsType - stats type (Parameters, Updates, Activations)

summaryType - Summary statistic type (mean, stdev, mean magnitude)

hasDataSetMetaData

public boolean hasDataSetMetaData()

Description copied from interface: StatsReport

Return whether any DataSet metadata is present or not

Specified by:

hasDataSetMetaData in interface StatsReport

Returns:

True if DataSet metadata is present

getSessionID

public String getSessionID()

Specified by:

getSessionID in interface Persistable

getTypeID

public String getTypeID()

Specified by:

getTypeID in interface Persistable

getWorkerID

public String getWorkerID()

Specified by:

getWorkerID in interface Persistable

getTimeStamp

public long getTimeStamp()

Specified by:

getTimeStamp in interface Persistable

encodingLengthBytes

public int encodingLengthBytes()

Specified by:

encodingLengthBytes in interface Persistable

encode

public byte[] encode()

Specified by:

encode in interface Persistable

encode

public void encode(ByteBuffer buffer)

Specified by:

encode in interface Persistable

encode

public void encode(org.agrona.MutableDirectBuffer buffer)

Specified by:

encode in interface AgronaPersistable

encode

public void encode(OutputStream outputStream)
            throws IOException

Specified by:

encode in interface Persistable

Throws:

IOException

decode

public void decode(byte[] decode)

Specified by:

decode in interface Persistable

decode

public void decode(ByteBuffer buffer)

Specified by:

decode in interface Persistable

decode

public void decode(org.agrona.DirectBuffer buffer)

Specified by:

decode in interface AgronaPersistable

decode

public void decode(InputStream inputStream)
            throws IOException

Specified by:

decode in interface Persistable

Throws:

IOException