6.Flume Channel Selectors

Flume Channel Selectors

If the type is not specified, then defaults to “replicating”.

Replicating Channel Selector (default)

Required properties are in bold.

Property Name Default Description selector.type replicating The component type name, needs to be replicating selector.optional – Set of channels to be marked as optional

Example for agent named a1 and it’s source called r1:

a1.sources = r1 a1.channels = c1 c2 c3 a1.sources.r1.selector.type = replicating a1.sources.r1.channels = c1 c2 c3 a1.sources.r1.selector.optional = c3

In the above configuration, c3 is an optional channel. Failure to write to c3 is simply ignored. Since c1 and c2 are not marked optional, failure to write to those channels will cause the transaction to fail.

Multiplexing Channel Selector

Required properties are in bold.

Property Name Default Description selector.type replicating The component type name, needs to be multiplexing selector.header flume.selector.header   selector.default –   selector.mapping.* –  

Example for agent named a1 and it’s source called r1:

a1.sources = r1 a1.channels = c1 c2 c3 c4 a1.sources.r1.selector.type = multiplexing a1.sources.r1.selector.header = state a1.sources.r1.selector.mapping.CZ = c1 a1.sources.r1.selector.mapping.US = c2 c3 a1.sources.r1.selector.default = c4

Custom Channel Selector

A custom channel selector is your own implementation of the ChannelSelector interface. A custom channel selector’s class and its dependencies must be included in the agent’s classpath when starting the Flume agent. The type of the custom channel selector is its FQCN.

Property Name Default Description selector.type – The component type name, needs to be your FQCN

Example for agent named a1 and its source called r1:

a1.sources = r1 a1.channels = c1 a1.sources.r1.selector.type = org.example.MyChannelSelector