com.facebook.presto.hive

Class HiveMaterializedViewUtils

java.lang.Object

com.facebook.presto.hive.HiveMaterializedViewUtils

public class HiveMaterializedViewUtils
extends Object

Method Summary

Modifier and Type	Method and Description
static com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates	differenceDataPredicates(com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates baseTablePredicatesInfo, com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates viewPredicatesInfo, Map<String,String> viewToBaseTablePredicatesKeyMap)
static com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates	differenceDataPredicates(com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates baseTablePredicatesInfo, com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates viewPredicatesInfo, Map<String,String> viewToBaseTablePredicatesKeyMap, Map<String,String> viewToBaseTableIndirectMap) From given base table partitions, removes all partitions that are already used to compute related view partitions, only returning partitions that are not reflected in Materialized View.
static com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates	getEmptyMaterializedViewDataPredicates()
static com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates	getMaterializedDataPredicates(com.facebook.presto.hive.metastore.SemiTransactionalHiveMetastore metastore, com.facebook.presto.hive.metastore.MetastoreContext metastoreContext, com.facebook.presto.common.type.TypeManager typeManager, com.facebook.presto.hive.metastore.Table table, org.joda.time.DateTimeZone timeZone)
static Map<com.facebook.presto.spi.SchemaTableName,Map<String,String>>	getViewToBasePartitionMap(com.facebook.presto.hive.metastore.Table view, List<com.facebook.presto.hive.metastore.Table> baseTables, Map<String,Map<com.facebook.presto.spi.SchemaTableName,String>> viewToBaseColumnMap)
static void	validateMaterializedViewPartitionColumns(com.facebook.presto.hive.metastore.SemiTransactionalHiveMetastore metastore, com.facebook.presto.hive.metastore.MetastoreContext metastoreContext, com.facebook.presto.hive.metastore.Table viewTable, com.facebook.presto.spi.MaterializedViewDefinition viewDefinition) Validate the partition columns of a materialized view to ensure 1) a materialized view is partitioned; 2) it has at least one partition directly mapped to all base tables and 3) Outer join conditions have common partitions that are partitions in the view as well
static Optional<Map<String,String>>	viewToBaseTableOnOuterJoinSideIndirectMappedPartitions(com.facebook.presto.spi.MaterializedViewDefinition viewDefinition, com.facebook.presto.hive.metastore.Table baseTable)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

validateMaterializedViewPartitionColumns

public static void validateMaterializedViewPartitionColumns(com.facebook.presto.hive.metastore.SemiTransactionalHiveMetastore metastore,
                                                            com.facebook.presto.hive.metastore.MetastoreContext metastoreContext,
                                                            com.facebook.presto.hive.metastore.Table viewTable,
                                                            com.facebook.presto.spi.MaterializedViewDefinition viewDefinition)

Validate the partition columns of a materialized view to ensure 1) a materialized view is partitioned; 2) it has at least one partition directly mapped to all base tables and 3) Outer join conditions have common partitions that are partitions in the view as well

A column is directly mapped to a base table column if it is derived directly or transitively from the base table column, by only selecting a column or an aliased column without any function or operator applied. For example, with SELECT column_b AS column_a, column_a is directly mapped to column_b. With SELECT column_b + column_c AS column_a, column_a is not directly mapped to any column.

viewToBaseColumnMap only contains direct column mappings.

getMaterializedDataPredicates

public static com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates getMaterializedDataPredicates(com.facebook.presto.hive.metastore.SemiTransactionalHiveMetastore metastore,
                                                                                                                      com.facebook.presto.hive.metastore.MetastoreContext metastoreContext,
                                                                                                                      com.facebook.presto.common.type.TypeManager typeManager,
                                                                                                                      com.facebook.presto.hive.metastore.Table table,
                                                                                                                      org.joda.time.DateTimeZone timeZone)

viewToBaseTableOnOuterJoinSideIndirectMappedPartitions

public static Optional<Map<String,String>> viewToBaseTableOnOuterJoinSideIndirectMappedPartitions(com.facebook.presto.spi.MaterializedViewDefinition viewDefinition,
                                                                                                  com.facebook.presto.hive.metastore.Table baseTable)

differenceDataPredicates

public static com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates differenceDataPredicates(com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates baseTablePredicatesInfo,
                                                                                                                 com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates viewPredicatesInfo,
                                                                                                                 Map<String,String> viewToBaseTablePredicatesKeyMap)

differenceDataPredicates

public static com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates differenceDataPredicates(com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates baseTablePredicatesInfo,
                                                                                                                 com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates viewPredicatesInfo,
                                                                                                                 Map<String,String> viewToBaseTablePredicatesKeyMap,
                                                                                                                 Map<String,String> viewToBaseTableIndirectMap)

From given base table partitions, removes all partitions that are already used to compute related view partitions, only returning partitions that are not reflected in Materialized View. We assume that given materialized view partitions are still fresh, and in sync with base table partitions, i.e. related Materialized View partitions are already invalidated when new base table partitions land.

Parameters:

baseTablePredicatesInfo - Partitions info for base table

viewPredicatesInfo - Partitions info for view

viewToBaseTablePredicatesKeyMap - Partitions mapping from view to base table. Only includes direct mapping, i.e. excludes mapping from outer joins EQ clauses.

viewToBaseTableIndirectMap - Extra partitions mapping from view to base table, computed from viewToBaseTableOnOuterJoinSideIndirectMappedPartitions()

Returns:

Base Table partitions that have not been used to refresh view.

getEmptyMaterializedViewDataPredicates

public static com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates getEmptyMaterializedViewDataPredicates()

getViewToBasePartitionMap

public static Map<com.facebook.presto.spi.SchemaTableName,Map<String,String>> getViewToBasePartitionMap(com.facebook.presto.hive.metastore.Table view,
                                                                                                        List<com.facebook.presto.hive.metastore.Table> baseTables,
                                                                                                        Map<String,Map<com.facebook.presto.spi.SchemaTableName,String>> viewToBaseColumnMap)