Package org.voltdb.stream.api.pipeline

Interface VoltStreamSource<T>

Type Parameters:
T - type of data this source will produce
All Superinterfaces:
VoltLifecycle
All Known Implementing Classes:
GeneratorSource, IterableSource
public interface VoltStreamSource<T> extends VoltLifecycle
This class encapsulates any integration logic with an upstream system. The source is the first element of any pipeline. 

 Source's lifecycle is
 - configure
 - process*
 - commit*
 - destroy

  • Method Summary

    Modifier and Type
    Method
    Description
    default CommitResult
    commit(long batchId, ExecutionContext context)
    the callback is invoked when the source has to perform any logic assuring data has been successfully processed and upstream system can make progress.
    void
    process(long batchId, Consumer<T> consumer, ExecutionContext context)
    Whenever a scheduler has an idle worker it will bind a worker to the source allowing the source to starts processing data.

    Methods inherited from interface org.voltdb.stream.api.pipeline.VoltLifecycle

    configure, destroy

  • Method Details

    • process

      void process(long batchId, Consumer<T> consumer, ExecutionContext context)
      Whenever a scheduler has an idle worker it will bind a worker to the source allowing the source to starts processing data. This method can be called many times and a source implementation must decide to keep producing on cancelling the processing. See ExecutionContext.StreamExecutionContext.cancel() and ExecutionContext.StreamExecutionContext.isCanceled() to control the flow of the stream.

      Once the source has no more data to process the stream will be no longer scheduled and stream processing finishes.

      Parameters:
      batchId - of the current batch
      consumer - a downstream component this source emits data to
      context - runtime context

    • commit

      default CommitResult commit(long batchId, ExecutionContext context)
      the callback is invoked when the source has to perform any logic assuring data has been successfully processed and upstream system can make progress.
      Parameters:
      batchId - of the current batch
      context - runtime context
      Returns:
      result of the commit