org.apache.parquet.schema

Class Types

java.lang.Object

org.apache.parquet.schema.Types

public class Types
extends Object

This class provides fluent builders that produce Parquet schema Types.

The most basic use is to build primitive types:

   Types.required(INT64).named("id");
   Types.optional(INT32).named("number");
 

The #required(PrimitiveTypeName) factory method produces a primitive type builder, and the Types.Builder.named(String) builds the PrimitiveType. Between required and named, other builder methods can be used to add type annotations or other type metadata:

   Types.required(BINARY).as(UTF8).named("username");
   Types.optional(FIXED_LEN_BYTE_ARRAY).length(20).named("sha1");
 

Optional types are built using #optional(PrimitiveTypeName) to get the builder.

Groups are built similarly, using requiredGroup() (or the optional version) to return a group builder. Group builders provide required and optional to add primitive types, which return primitive builders like the versions above.

   // This produces:
   // required group User {
   //   required int64 id;
   //   optional binary email (UTF8);
   // }
   Types.requiredGroup()
            .required(INT64).named("id")
            .required(BINARY).as(UTF8).named("email")
        .named("User")
 

When required is called on a group builder, the builder it returns will add the type to the parent group when it is built and named will return its parent group builder (instead of the type) so more fields can be added.

Sub-groups can be created using requiredGroup() to get a group builder that will create the group type, add it to the parent builder, and return the parent builder for more fields.

   // required group User {
   //   required int64 id;
   //   optional binary email (UTF8);
   //   optional group address {
   //     required binary street (UTF8);
   //     required int32 zipcode;
   //   }
   // }
   Types.requiredGroup()
            .required(INT64).named("id")
            .required(BINARY).as(UTF8).named("email")
            .optionalGroup()
                .required(BINARY).as(UTF8).named("street")
                .required(INT32).named("zipcode")
            .named("address")
        .named("User")
 

Maps are built similarly, using requiredMap() (or the optionalMap() version) to return a map builder. Map builders provide key to add a primitive as key or a groupKey to add a group as key. key() returns a MapKey builder, which extends a primitive builder. On the other hand, groupKey() returns a MapGroupKey builder, which extends a group builder. A key in a map is always required.

Once a key is built, a primitive map value can be built using requiredValue() (or the optionalValue() version) that returns MapValue builder. A group map value can be built using requiredGroupValue() (or the optionalGroupValue() version) that returns MapGroupValue builder. // required group zipMap (MAP) { // repeated group map (MAP_KEY_VALUE) { // required float key // optional int32 value // } // } Types.requiredMap() .key(FLOAT) .optionalValue(INT32) .named("zipMap") // required group zipMap (MAP) { // repeated group map (MAP_KEY_VALUE) { // required group key { // optional int64 first; // required group second { // required float inner_id_1; // optional int32 inner_id_2; // } // } // optional group value { // optional group localGeoInfo { // required float inner_value_1; // optional int32 inner_value_2; // } // optional int32 zipcode; // } // } // } Types.requiredMap() .groupKey() .optional(INT64).named("id") .requiredGroup() .required(FLOAT).named("inner_id_1") .required(FLOAT).named("inner_id_2") .named("second") .optionalGroup() .optionalGroup() .required(FLOAT).named("inner_value_1") .optional(INT32).named("inner_value_2") .named("localGeoInfo") .optional(INT32).named("zipcode") .named("zipMap")

Message types are built using buildMessage() and function just like group builders.

   // message User {
   //   required int64 id;
   //   optional binary email (UTF8);
   //   optional group address {
   //     required binary street (UTF8);
   //     required int32 zipcode;
   //   }
   // }
   Types.buildMessage()
            .required(INT64).named("id")
            .required(BINARY).as(UTF8).named("email")
            .optionalGroup()
                .required(BINARY).as(UTF8).named("street")
                .required(INT32).named("zipcode")
            .named("address")
        .named("User")
 

These builders enforce consistency checks based on the specifications in the parquet-format documentation. For example, if DECIMAL is used to annotate a FIXED_LEN_BYTE_ARRAY that is not long enough for its maximum precision, these builders will throw an IllegalArgumentException:

   // throws IllegalArgumentException with message:
   // "FIXED(4) is not long enough to store 10 digits"
   Types.required(FIXED_LEN_BYTE_ARRAY).length(4)
        .as(DECIMAL).precision(10)
        .named("badDecimal");
 

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Types.BaseGroupBuilder<P,THIS extends Types.BaseGroupBuilder<P,THIS>>
static class	Types.BaseListBuilder<P,THIS extends Types.BaseListBuilder<P,THIS>>
static class	Types.BaseMapBuilder<P,THIS extends Types.BaseMapBuilder<P,THIS>>
static class	Types.BasePrimitiveBuilder<P,THIS extends Types.BasePrimitiveBuilder<P,THIS>>
static class	Types.Builder<THIS extends Types.Builder,P> A base builder for Type objects.
static class	Types.GroupBuilder<P> A builder for GroupType objects.
static class	Types.ListBuilder<P>
static class	Types.MapBuilder<P>
static class	Types.MessageTypeBuilder
static class	Types.PrimitiveBuilder<P> A builder for PrimitiveType objects.

Constructor Summary

Constructors

Constructor and Description
Types()

Method Summary

Methods

Modifier and Type	Method and Description
static Types.GroupBuilder<GroupType>	buildGroup(Type.Repetition repetition)
static Types.MessageTypeBuilder	buildMessage() Returns a builder to construct a MessageType.
static Types.ListBuilder<GroupType>	list(Type.Repetition repetition)
static Types.MapBuilder<GroupType>	map(Type.Repetition repetition)
static Types.PrimitiveBuilder<PrimitiveType>	optional(PrimitiveType.PrimitiveTypeName type) Returns a builder to construct an optional PrimitiveType.
static Types.GroupBuilder<GroupType>	optionalGroup() Returns a builder to construct an optional GroupType.
static Types.ListBuilder<GroupType>	optionalList()
static Types.MapBuilder<GroupType>	optionalMap()
static Types.PrimitiveBuilder<PrimitiveType>	primitive(PrimitiveType.PrimitiveTypeName type, Type.Repetition repetition)
static Types.PrimitiveBuilder<PrimitiveType>	repeated(PrimitiveType.PrimitiveTypeName type) Returns a builder to construct a repeated PrimitiveType.
static Types.GroupBuilder<GroupType>	repeatedGroup() Returns a builder to construct a repeated GroupType.
static Types.PrimitiveBuilder<PrimitiveType>	required(PrimitiveType.PrimitiveTypeName type) Returns a builder to construct a required PrimitiveType.
static Types.GroupBuilder<GroupType>	requiredGroup() Returns a builder to construct a required GroupType.
static Types.ListBuilder<GroupType>	requiredList()
static Types.MapBuilder<GroupType>	requiredMap()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Types

public Types()

Method Detail

buildMessage

public static Types.MessageTypeBuilder buildMessage()

Returns a builder to construct a MessageType.

Returns:

a Types.MessageTypeBuilder

primitive

public static Types.PrimitiveBuilder<PrimitiveType> primitive(PrimitiveType.PrimitiveTypeName type,
                                              Type.Repetition repetition)

required

public static Types.PrimitiveBuilder<PrimitiveType> required(PrimitiveType.PrimitiveTypeName type)

Returns a builder to construct a required PrimitiveType.

Parameters:

type - a PrimitiveType.PrimitiveTypeName for the constructed type

Returns:

a Types.PrimitiveBuilder

optional

public static Types.PrimitiveBuilder<PrimitiveType> optional(PrimitiveType.PrimitiveTypeName type)

Returns a builder to construct an optional PrimitiveType.

Parameters:

type - a PrimitiveType.PrimitiveTypeName for the constructed type

Returns:

a Types.PrimitiveBuilder

repeated

public static Types.PrimitiveBuilder<PrimitiveType> repeated(PrimitiveType.PrimitiveTypeName type)

Returns a builder to construct a repeated PrimitiveType.

Parameters:

type - a PrimitiveType.PrimitiveTypeName for the constructed type

Returns:

a Types.PrimitiveBuilder

buildGroup

public static Types.GroupBuilder<GroupType> buildGroup(Type.Repetition repetition)

requiredGroup

public static Types.GroupBuilder<GroupType> requiredGroup()

Returns a builder to construct a required GroupType.

Returns:

a Types.GroupBuilder

optionalGroup

public static Types.GroupBuilder<GroupType> optionalGroup()

Returns a builder to construct an optional GroupType.

Returns:

a Types.GroupBuilder

repeatedGroup

public static Types.GroupBuilder<GroupType> repeatedGroup()

Returns a builder to construct a repeated GroupType.

Returns:

a Types.GroupBuilder

map

public static Types.MapBuilder<GroupType> map(Type.Repetition repetition)

requiredMap

public static Types.MapBuilder<GroupType> requiredMap()

optionalMap

public static Types.MapBuilder<GroupType> optionalMap()

list

public static Types.ListBuilder<GroupType> list(Type.Repetition repetition)

requiredList

public static Types.ListBuilder<GroupType> requiredList()

optionalList

public static Types.ListBuilder<GroupType> optionalList()