Data platforms
Context Engine sends plant data to your data platform and brings results back to the line. Each platform page shows every route between the two, what each side sets up, and where to start.
- Platforms
- Snowflake · Databricks · BigQuery · Microsoft Fabric · Amazon Redshift · Azure Synapse
- Runs on
- Context Engine, next to your machines
- Extra software
- None on the plant side
A data platform is where plant data meets the rest of the business: finance, supply chain, quality, models. Context Engine sits next to the machines, so every route starts or ends there. Some routes go straight to the platform; others pass through storage or a streaming service you already run. Coming back, a pipeline reads results on a schedule, or the platform calls Context Engine when something happens.
Every route uses connectors and nodes that are part of Context Engine. Nothing else is installed in the plant. What changes from route to route is how fresh the data is, what the platform charges for, and which connections your network has to allow.
Seven kinds of route
- Stream directly
- Context Engine writes rows straight into the platform's streaming ingest. Nothing in between.
- Write with SQL
- A SQL function inserts or merges rows through the platform's warehouse. Suits records that change.
- Land files
- Context Engine batches rows into Parquet files; the platform loads them. Suits volume and backfills.
- Through a streaming service
- Context Engine produces to Kafka, Event Hubs, Kinesis or Pub/Sub, and the platform reads from there.
- Start platform jobs
- A pipeline starts and follows a job inside the platform, for example after a batch or a shift.
- Read results back
- A scheduled pipeline reads results from the platform and hands them to the UNS or a machine.
- Platform calls MaestroHub
- The platform sends a webhook to Context Engine when something happens, and a pipeline acts on it.
All platforms at a glance
Select a platform to open its page, or a letter to go straight to that route.
| Platform | Stream directly | Write with SQL | Land files | Through a streaming service | Start platform jobs | Read results back | Platform calls MaestroHub |
|---|---|---|---|---|---|---|---|
| Snowflake | A | B | C | D | · | E | F |
| Databricks | · | B | AD | C | · | E | F |
| BigQuery | · | · | B | A | C | D | · |
| Microsoft Fabric | · | · | A | B | · | C | · |
| Amazon Redshift | · | A | B | C | · | D | · |
| Azure Synapse | · | A | B | C | D | E | · |
- AWhere to start on that platform
- BAlso available
- ·Not offered
How to choose
- Start with the highlighted route. It is the one most plants use for machine data on that platform.
- Add a route back when the platform produces something the line should use: a prediction, a quality limit, a production target.
- Let your network rules decide the rest. If the plant may reach only your own cloud storage or your streaming service, each page has a route for that.
Each platform page opens with a few questions that pick a route for you.