org.apache.tez.runtime.library.cartesianproduct

Class CartesianProductConfig

java.lang.Object

org.apache.tez.runtime.library.cartesianproduct.CartesianProductConfig

@InterfaceStability.Evolving
public class CartesianProductConfig
extends Object

CartesianProductConfig is used to configure both CartesianProductVertexManager and CartesianProductEdgeManager. User need to specify the vertices and number of partitions of each vertices' output at least. In partitioned case, filter should be specified here also(via CartesianProductFilterDescriptor. User may also configure min/max fractions used in slow start.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	CartesianProductConfig(boolean isPartitioned, int[] numPartitions, String[] sources, CartesianProductFilterDescriptor filterDescriptor) create config for both cases, used by subclass
protected	CartesianProductConfig(int[] numPartitions, String[] sources, CartesianProductFilterDescriptor filterDescriptor) create config for partitioned case, with specified source vertices order
	CartesianProductConfig(List<String> sources) create config for fair cartesian product
	CartesianProductConfig(Map<String,Integer> vertexPartitionMap) create config for partitioned case without filter
	CartesianProductConfig(Map<String,Integer> vertexPartitionMap, CartesianProductFilterDescriptor filterDescriptor) create config for partitioned case with filter

Method Summary

Modifier and Type	Method and Description
protected void	checkNumPartitions()
protected static CartesianProductConfig	fromProto(CartesianProductUserPayload.CartesianProductConfigProto proto)
protected static CartesianProductConfig	fromUserPayload(org.apache.tez.dag.api.UserPayload payload)
CartesianProductFilterDescriptor	getFilterDescriptor()
boolean	getIsPartitioned()
List<Integer>	getNumPartitions()
List<String>	getSourceVertices()
protected CartesianProductUserPayload.CartesianProductConfigProto	toProto(org.apache.tez.dag.api.TezConfiguration conf)
org.apache.tez.dag.api.UserPayload	toUserPayload(org.apache.tez.dag.api.TezConfiguration conf)
protected static CartesianProductUserPayload.CartesianProductConfigProto	userPayloadToProto(org.apache.tez.dag.api.UserPayload payload)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CartesianProductConfig

public CartesianProductConfig(List<String> sources)

create config for fair cartesian product

Parameters:

sources - list of names of source vertices or vertex groups

CartesianProductConfig

public CartesianProductConfig(Map<String,Integer> vertexPartitionMap)

create config for partitioned case without filter

Parameters:

vertexPartitionMap - the map from vertex name to its number of partitions

CartesianProductConfig

public CartesianProductConfig(Map<String,Integer> vertexPartitionMap,
                              CartesianProductFilterDescriptor filterDescriptor)

create config for partitioned case with filter

Parameters:

vertexPartitionMap - the map from vertex name to its number of partitions

filterDescriptor -

CartesianProductConfig

protected CartesianProductConfig(int[] numPartitions,
                                 String[] sources,
                                 CartesianProductFilterDescriptor filterDescriptor)

create config for partitioned case, with specified source vertices order

Parameters:

numPartitions -

sources -

filterDescriptor -

CartesianProductConfig

protected CartesianProductConfig(boolean isPartitioned,
                                 int[] numPartitions,
                                 String[] sources,
                                 CartesianProductFilterDescriptor filterDescriptor)

create config for both cases, used by subclass

Method Detail

checkNumPartitions

protected void checkNumPartitions()

getSourceVertices

public List<String> getSourceVertices()

Returns:

the array of source vertices (or source vertex group) names

getNumPartitions

public List<Integer> getNumPartitions()

Returns:

the array of number of partitions, the order is same as result of getSourceVertices

getIsPartitioned

public boolean getIsPartitioned()

getFilterDescriptor

public CartesianProductFilterDescriptor getFilterDescriptor()

toUserPayload

public org.apache.tez.dag.api.UserPayload toUserPayload(org.apache.tez.dag.api.TezConfiguration conf)
                                                 throws IOException

Throws:

IOException

toProto

protected CartesianProductUserPayload.CartesianProductConfigProto toProto(org.apache.tez.dag.api.TezConfiguration conf)

userPayloadToProto

protected static CartesianProductUserPayload.CartesianProductConfigProto userPayloadToProto(org.apache.tez.dag.api.UserPayload payload)
                                                                                     throws com.google.protobuf.InvalidProtocolBufferException

Throws:

com.google.protobuf.InvalidProtocolBufferException

fromUserPayload

protected static CartesianProductConfig fromUserPayload(org.apache.tez.dag.api.UserPayload payload)
                                                 throws com.google.protobuf.InvalidProtocolBufferException

Throws:

com.google.protobuf.InvalidProtocolBufferException

fromProto

protected static CartesianProductConfig fromProto(CartesianProductUserPayload.CartesianProductConfigProto proto)