com.linkedin.feathr.common.tensor.dense

Class DenseStringTensor

java.lang.Object

com.linkedin.feathr.common.tensor.DenseTensor

com.linkedin.feathr.common.tensor.dense.ByteBufferDenseTensor

com.linkedin.feathr.common.tensor.dense.DenseStringTensor

All Implemented Interfaces:

TensorData

public class DenseStringTensor
extends ByteBufferDenseTensor

Dense tensor which can be converted from/to a TensorFlow tensor object. The underlying data is backed up by ByteBuffer, using TensorFlow-compatible format: https://github.com/tensorflow/tensorflow/blob/98c0deb828c2f98f0d6d77a12d32f3b33ed92887/tensorflow/c/c_api.h#L206. start_offset: array[uint64] data: byte[...] The string length (as a varint/ULEB128 https://en.wikipedia.org/wiki/LEB128#Unsigned_LEB128), followed by the contents of the string is encoded at data[start_offset[i]]], using UTF-8 encoding. In other words, [offset A][offset B][offset C][length A][data A][length B][data B][length C][data C].

Field Summary

Fields inherited from class com.linkedin.feathr.common.tensor.dense.ByteBufferDenseTensor

_byteBuffer, _cardinality, _limit, _shape, _start, _types

Constructor Summary

Constructors

Constructor and Description
DenseStringTensor(java.nio.ByteBuffer byteBuffer, long[] shape)
DenseStringTensor(java.nio.ByteBuffer byteBuffer, long[] shape, Representable[] types, int cardinality)

Method Summary

Modifier and Type	Method and Description
java.util.List<?>	asList()
TensorIterator	iterator()

Methods inherited from class com.linkedin.feathr.common.tensor.dense.ByteBufferDenseTensor

cardinality, estimatedCardinality, getByteBuffer, getByteBuffer, getShape, getTypes

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.linkedin.feathr.common.tensor.TensorData

getArity, isEmpty

Constructor Detail

DenseStringTensor

public DenseStringTensor(java.nio.ByteBuffer byteBuffer,
                         long[] shape)

DenseStringTensor

public DenseStringTensor(java.nio.ByteBuffer byteBuffer,
                         long[] shape,
                         Representable[] types,
                         int cardinality)

Method Detail

iterator

public TensorIterator iterator()

Returns:

an iterator over entries in the mapping from dimensions to the value.

asList

public java.util.List<?> asList()

Specified by:

asList in class DenseTensor

Returns:

the list of values in their unraveled order. This in combination with the shape is enough to restore the entire tensor, as the dimensions can be calculated from the shape.