MaestroHub and Azure Synapse

Write rows into a dedicated SQL pool, bulk-load lake files with COPY INTO, stream through Event Hubs and Stream Analytics, start Synapse pipelines, and read results back to the line.

Connectors

Azure Synapse · Azure Blob · Event Hubs · Stream Analytics

Start with

B · Parquet files, then COPY INTO

Network

Outbound 1433 for SQL, HTTPS 443 for the rest

The Azure Synapse connector reaches a dedicated or serverless SQL pool and the workspace's pipelines. Most plants land Parquet files in Azure Blob storage and call Copy Into from the same pipeline.

Find your route

Answer a few questions; the drawing below follows your answer.

Where does the data go?

Into Azure SynapseBack to the plant

How fresh must it be?

SecondsMinutes are fine

What can the plant reach?

Synapse directlyOnly our storage accountOnly our Event Hubs

AWrite rows. Moderate volumes, or rows that change.

Architecture

Context Engine runs next to your machines, and every route starts or ends there. Select a route to trace it.

Into Azure Synapse

Back to the plant

• Set up in MaestroHub
• Your infrastructure
• Set up in Azure Synapse

BLand files, then COPY INTO. Minutes, as often as you load. The dedicated pool runs each load. What each side sets up

Routes

A

Into Azure Synapse

Write rows

For moderate rates and records

Freshness

Per call, seconds

Azure Synapse compute

The dedicated pool, billed while running

Delivery

Retried on failure; use MERGE so a retry cannot duplicate

In MaestroHub

1. Create an Azure Synapse connection to the dedicated pool, with a SQL login or Microsoft Entra ID.
2. Use Write, which creates or widens the table to fit.
3. Batch rows with a Buffer or Aggregator node so each call carries many rows.

In Azure Synapse

1. A database user for the identity, in db_datareader and db_datawriter.

Choose it when Moderate volumes, or rows that change.

Look elsewhere when High volume: use B.

Azure Synapse guide

B

Into Azure Synapse

Land files, then COPY INTO

Start here for machine data

Freshness

Minutes, as often as you load

Azure Synapse compute

The dedicated pool runs each load

Delivery

Load each file once; track loaded paths

In MaestroHub

1. Batch rows with a Buffer or Aggregator node so each call carries many rows.
2. Add Convert to File with format Parquet and a declared column schema.
3. Write with the Azure Blob node, Input Format base64.
4. Call the Synapse Copy Into function with the same path.

In your infrastructure

1. A storage account container for the landed files.

In Azure Synapse

1. Read access on the container for the pool, and the target table.

Choose it when Large volumes where minutes are fine.

Look elsewhere when You need seconds: use C.

Azure Synapse guideConvert to FileAzure Blob guide

C

Into Azure Synapse

Through Event Hubs

For second-level freshness

Freshness

Seconds

Azure Synapse compute

Stream Analytics streaming units, always on

Delivery

At least once

In MaestroHub

1. Create an Azure Event Hubs connection and use Send Batch per data set, with store-and-forward on.
2. Optionally start, stop and scale the job from a pipeline with the Azure Stream Analytics connector.

In your infrastructure

1. An event hub with a consumer group for the job.

In Azure Synapse

1. A Stream Analytics job that reads the hub and writes the dedicated pool.

Choose it when Freshness in seconds, often with windowed aggregates.

Look elsewhere when Minutes are fine: B costs less.

Azure Event Hubs guideStream Analytics guide

D

Into Azure Synapse

Run Synapse pipelines

Start loads on plant events

Freshness

When the plant says so

Azure Synapse compute

Whatever the pipeline runs

Delivery

Follow the run to its end; cancel on failure

In MaestroHub

1. Call Run Pipeline with parameters, follow it with Get Pipeline Run, and alert on failure or call Cancel Pipeline Run.

In Azure Synapse

1. A published Synapse pipeline. Give the identity the Synapse Artifact User and Synapse Credential User roles.

Choose it when A load or transform in Synapse should start on a plant event.

Look elsewhere when You only need to land rows: A or B is simpler.

Azure Synapse guide

E

Back to the plant

Read results back

Close the loop on a schedule

Freshness

Your schedule

Azure Synapse compute

Serverless bills data scanned; dedicated bills while running

Delivery

Reads current state; safe to repeat

In MaestroHub

1. Build a Query function. A result returns 1,000 rows by default, up to 100,000.
2. Pipeline: Schedule trigger, then the query, then publish to a UNS topic or write a setpoint through the machine's connector.

In Azure Synapse

1. A results view, or OPENROWSET over files in the lake, with SELECT granted.

Choose it when Analytics in Synapse produce values the plant should use.

Look elsewhere when Large exports: use Export To Storage and read the files.

Azure Synapse guideSchedule trigger

Compare the routes

Routes combine: most plants run one route in and one route back.

Route	Direction	Freshness	Azure Synapse compute	Delivery	Network	Best for
	Into Azure Synapse	Per call, seconds	The dedicated pool, billed while running	Retried on failure; use MERGE so a retry cannot duplicate	Outbound 1433 to Synapse	Records and moderate telemetry
	Into Azure Synapse	Minutes, as often as you load	The dedicated pool runs each load	Load each file once; track loaded paths	Outbound HTTPS 443 and 1433	High volume and backfills
	Into Azure Synapse	Seconds	Stream Analytics streaming units, always on	At least once	Outbound to Event Hubs	Live dashboards with windowed aggregates
	Into Azure Synapse	When the plant says so	Whatever the pipeline runs	Follow the run to its end; cancel on failure	Outbound HTTPS 443	Starting loads when a batch or a shift ends
	Back to the plant	Your schedule	Serverless bills data scanned; dedicated bills while running	Reads current state; safe to repeat	Outbound 1433 to Synapse	Targets, predictions, and lake files through OPENROWSET

Related

• All data platforms
• Azure Synapse
• Azure Blob
• Azure Stream Analytics