ai.djl.basicdataset.utils

Class FixedBucketSampler

java.lang.Object

ai.djl.basicdataset.utils.FixedBucketSampler

All Implemented Interfaces:

ai.djl.training.dataset.Sampler

public class FixedBucketSampler
extends java.lang.Object
implements ai.djl.training.dataset.Sampler

FixedBucketSampler is a Sampler to be used with TextDataset, and PaddingStackBatchifier. It groups text data of same length, and samples them together so that the amount of padding required is minimised. It also makes sure that the sampling is random across epochs.

Nested Class Summary

Nested classes/interfaces inherited from interface ai.djl.training.dataset.Sampler

ai.djl.training.dataset.Sampler.SubSampler

Constructor Summary

Constructors

Constructor and Description
FixedBucketSampler(int batchSize) Constructs a new instance of FixedBucketSampler with the given number of buckets, and the given batch size.
FixedBucketSampler(int batchSize, int numBuckets) Constructs a new instance of FixedBucketSampler with the given number of buckets, and the given batch size.
FixedBucketSampler(int batchSize, int numBuckets, boolean dropLast, boolean shuffle) Constructs a new instance of FixedBucketSampler with the given number of buckets, and the given batch size.

Method Summary

Modifier and Type	Method and Description
int	getBatchSize()
java.util.Iterator<java.util.List<java.lang.Long>>	sample(ai.djl.training.dataset.RandomAccessDataset dataset)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FixedBucketSampler

public FixedBucketSampler(int batchSize,
                          int numBuckets,
                          boolean dropLast,
                          boolean shuffle)

Constructs a new instance of FixedBucketSampler with the given number of buckets, and the given batch size.

Parameters:

batchSize - the batch size

numBuckets - the number of buckets

dropLast - whether to drop the last incomplete batch

shuffle - whether to shuffle data randomyl while sampling

FixedBucketSampler

public FixedBucketSampler(int batchSize,
                          int numBuckets)

Constructs a new instance of FixedBucketSampler with the given number of buckets, and the given batch size.

Parameters:

batchSize - the batch size

numBuckets - the number of buckets

FixedBucketSampler

public FixedBucketSampler(int batchSize)

Constructs a new instance of FixedBucketSampler with the given number of buckets, and the given batch size.

Parameters:

batchSize - the batch size

Method Detail

sample

public java.util.Iterator<java.util.List<java.lang.Long>> sample(ai.djl.training.dataset.RandomAccessDataset dataset)

Specified by:

sample in interface ai.djl.training.dataset.Sampler

getBatchSize

public int getBatchSize()

Specified by:

getBatchSize in interface ai.djl.training.dataset.Sampler