Skip to main content

Common Patterns

Overview​

Connectors can be mostly grouped into one of three categories: Source, Aggregator, and Sink. The difference lies in their roles, and usually correspond to how they interact with the Starbloom message queue.

Source connector​

Source connectors are the simplest type of integration for Starbloom, allowing Starbloom to work with any type of third party data source. If you have data you'd like to index, this is where you start.

A Source connector extracts/ingests, cleans, and generalizes data from an external source, usually a blockchain, smart contract, or an API. It then passes on this data to the Starbloom message queue.

Examples:

  • Ingest blocks and transactions from a L1 or L2 blockchain
  • Ingest events from a smart contract, such as Uniswap, L2 bridge, or OpenSea
  • Ingest messages from a websocket API, such as Binance trade data, metrics data, or Telegram API
  • Ingest messages from polling a REST API, such as covid stats, Fear and Greed meter, or website rankings

Aggregator connector​

An Aggregator connector inputs data from the message queue, transforms/aggregates/augments it, then outputs data to the message queue.

Examples:

  • Generate statistics on a L1 or L2 blockchain's transactions
  • Create a price index across multiple markets

Sink connector​

note

This is an advanced topic and is not applicable for most integration use cases. If you are interested in building a sink connector, please reach out to our team.

A Sink connector inputs data from the message queue, and outputs it to storage or other output.

Examples:

  • SSE API
  • Websocket JSON API
  • Streaming API for other data formats
  • Write to database