CloudTDMS Architecture

CloudTDMS architecture consists of many segregated platforms aligned with the pricing model. In instance, we can differentiate 3 different type of platforms :

Starter platform : Shared platform for all free tier users.
Team platform : Shared platform for all team users (start-ups).
Business platform : Dedicated platform for each client.

Each CloudTDMS platform consists of several component:

Load balancer & reverse proxy.
Front-end.
Database.
Workflow engine and its workers.

The following image shows the high level architecture overview of CloudTDMS:

image info

The following table describes the architecture components:

Component	Description
Load balancer & Reverse proxy	Part of security components, installed after the network firewalls. It is providing high availability load balancer and reverse proxy for TCP and HTTP-based applications that spreads requests across multiple servers.
Front-end	A web-based client that you can use to perform all activities such as creating Applications, Streams and Attributes as well as making data discovery & profiling , and also generating data workflows responsible for creating your synthetic data.
Database	A relational database that stores the components that you define via CloudTDMS front-end server, such as Applications, Streams, Attributes and workflows. The database stores data providers and libraries as well as logs and notifications etc.
Workflow engine & workers	Workflow management platform for data engineering pipelines. CloudTDMS is using Apache Airflow. Airflow is written in Python, and workflows are created as Python scripts. Airflow uses directed acyclic graphs (DAGs) to manage workflow orchestration. Tasks and dependencies are defined in Python and then Airflow manages the scheduling and execution. DAGs can be run either on a defined schedule (e.g. hourly or daily) or based on external event triggers.