Skip to content

Architectural Overview

This page is for informational purposes only

If you already understand what the main components of a Straming Integration Platform deployment are, you can skip this page.

architecture-diagram.png

This image broadly describes the environment that you will build today. Some of the pieces are already in place:

  • Application Server (represents our system or record)
  • Application Services (the Synergy DBL back end of our application)
  • Redpanda (Kafka) server
  • RustFS S3 storage server
  • SMTP4DEV development email server
  • Microsoft SQL Server
  • DbGate (browser-based database client)

But those are just the supporting products that every implementtion requires. During the workshop, you will install and configure all of the main Streaming Integration Platform components:

  • Configuration Server
  • Base Environment
  • Snapshot Agent
  • Message Broker
  • ISAM Agent
  • SQL Agent
  • JSON Agent

Here is a very quick overview of that each of those main components does.

Configuration Server

  • Presents a UI to allow admins to define and maintains configuration information for all components.
  • Acts as a certification authority for the SIP environment.
  • Provides TLS certificated and configuration information to known and trusted clients (other SIP components).

Snapshot Agent

  • On depand, creates snapshots of the system-of-record data.
  • Uploads those snapshots to S3 network storage.
  • Published information about available snapshots to a Kafka topic.

Message Broker

  • Receives events from the system-of-record as data changes occur.
  • Publishes CDC messages to Kafka topic for consumption by replication agents.

ISAM Agent

  • Initially hydrates a local replica SDMS data set.
  • Consumes CDC messages from the Kafka topic.
  • Mirrors the CDC events in the local data store to maintain a mirror of the system-of-record data.

SQL Agent

  • Initially hydrates a local replica SQL Server database data set.
  • Consumes CDC messages from the Kafka topic.
  • Mirrors the CDC events in the local data store to maintain a mirror of the system-of-record data.

JSON Agent

  • Consumes CDC messages from the Kafka topic.
  • Reformats the data of each message as JSON data.
  • Publishes the reformatted data to another Kafka topic.