com.linkedin.feathr.common.tensor.dense

Class DenseBytesTensor

java.lang.Object

com.linkedin.feathr.common.tensor.DenseTensor

com.linkedin.feathr.common.tensor.dense.ByteBufferDenseTensor

com.linkedin.feathr.common.tensor.dense.DenseBytesTensor

All Implemented Interfaces:

TensorData

public class DenseBytesTensor
extends ByteBufferDenseTensor

Dense tensor which can be converted from/to a TensorFlow tensor object. The underlying data is backed up by ByteBuffer, using TensorFlow-compatible format: https://github.com/tensorflow/tensorflow/blob/98c0deb828c2f98f0d6d77a12d32f3b33ed92887/tensorflow/c/c_api.h#L206.

All individual byte[] are added up in one ByteBuffer. ByteBuffer contains byte[] length (as a varint/ULEB128 https://en.wikipedia.org/wiki/LEB128#Unsigned_LEB128), followed by the contents of the byte[] at data[start_offset[i]]]

start_offset: array[uint64] data: byte[...] [offset A][offset B][offset C][length A][data A][length B][data B][length C][data C].

Field Summary

Fields inherited from class com.linkedin.feathr.common.tensor.dense.ByteBufferDenseTensor

_byteBuffer, _cardinality, _limit, _shape, _start, _types

Constructor Summary

Constructors

Constructor and Description
DenseBytesTensor(java.nio.ByteBuffer byteBuffer, long[] shape)
DenseBytesTensor(java.nio.ByteBuffer byteBuffer, long[] shape, Representable[] types, int cardinality)

Method Summary

Modifier and Type	Method and Description
java.util.List<?>	asList()
TensorIterator	iterator()

Methods inherited from class com.linkedin.feathr.common.tensor.dense.ByteBufferDenseTensor

cardinality, estimatedCardinality, getByteBuffer, getByteBuffer, getShape, getTypes

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.linkedin.feathr.common.tensor.TensorData

getArity, isEmpty

Constructor Detail

DenseBytesTensor

public DenseBytesTensor(java.nio.ByteBuffer byteBuffer,
                        long[] shape)

DenseBytesTensor

public DenseBytesTensor(java.nio.ByteBuffer byteBuffer,
                        long[] shape,
                        Representable[] types,
                        int cardinality)

Method Detail

iterator

public TensorIterator iterator()

Returns:

an iterator over entries in the mapping from dimensions to the value.

asList

public java.util.List<?> asList()

Specified by:

asList in class DenseTensor

Returns:

the list of values in their unraveled order. This in combination with the shape is enough to restore the entire tensor, as the dimensions can be calculated from the shape.