Skip to main content
Version: 3.0 (next)

MaestroHub and Microsoft Fabric

Land plant data in a Fabric lakehouse through OneLake, or stream it through Event Hubs into an Eventstream, and read lakehouse files back to the line.

Connectors
Microsoft Fabric (OneLake) · Azure Event Hubs
Start with
A · Parquet files in OneLake
Network
Outbound HTTPS 443 only

The Microsoft Fabric (OneLake) connector writes, reads and lists files in a lakehouse, signed in with Microsoft Entra ID. It writes files, not tables: a notebook, a pipeline or the Load Table API turns them into Delta tables. For freshness in seconds, Azure Event Hubs feeds an Eventstream.

Find your route​

Answer a few questions; the drawing below follows your answer.

Where does the data go?
How fresh must it be?
What can the plant reach?

BThrough Event Hubs. Send to the Eventstream's custom endpoint, which gives an Event Hubs connection string, or to your own event hub.

Architecture​

Context Engine runs next to your machines, and every route starts or ends there. Select a route to trace it.

Into Microsoft Fabric
Back to the plant
PLANTYOUR NETWORK OR CLOUD, OPTIONALMICROSOFT FABRICSourcesPLCs, OPC UAMQTT, MESHistoriansShop floorUNS, setpointsTeams, SlackContext Enginepipelines, UNSTablesand viewsAConvert to Parquetto OneLakeDFS endpoint over HTTPS 443Load to Tablesnotebook or Load Table APIBAzure Event HubsSendEvent hubor an Eventstream custom endpointEventstreamto an Eventhouse or lakehouseCOneLakeList, Read FileDFS endpoint over HTTPS 443Lakehouse filesresults a notebook writes
  • Set up in MaestroHub
  • Your infrastructure
  • Set up in Microsoft Fabric
ALand files in OneLake. Minutes, as often as you load. Fabric capacity while the load runs. What each side sets up

Routes​

A

Into Microsoft Fabric

Land files in OneLake​

Start here for machine data
Freshness
Minutes, as often as you load
Microsoft Fabric compute
Fabric capacity while the load runs
Delivery
One file per batch; a load reads only the files it names

In MaestroHub

  1. Batch rows with a Buffer or Aggregator node so each call carries many rows.
  2. Add Convert to File with format Parquet and a declared column schema.
  3. Write with OneLake Write File under Files/, Input Format base64.
  4. To turn the files into a table from the same pipeline, call the Fabric Load Table API from a REST connection.

In Microsoft Fabric

  1. A workspace and a lakehouse, with the service principal added to the workspace as Contributor.
  2. A notebook, a pipeline or Load to Tables that converts Files/ into a Delta table. A bare .parquet file under Tables/ is not a table.

Choose it when Plant data should land in the lakehouse and minutes are fine.

Look elsewhere when You need seconds: use B.

OneLake guideConvert to File

B

Into Microsoft Fabric

Through Event Hubs​

For second-level freshness
Freshness
Seconds
Microsoft Fabric compute
Fabric capacity runs the Eventstream
Delivery
At least once; remove duplicates on a message ID if needed

In MaestroHub

  1. Create an Azure Event Hubs connection, with a SAS connection string or Microsoft Entra ID.
  2. Use Send Batch per data set and keep store-and-forward on.

In your infrastructure

  1. An event hub with its own consumer group for Fabric, unless you send to the custom endpoint.

In Microsoft Fabric

  1. An Eventstream with the event hub, or a custom endpoint, as its source.
  2. A destination: an Eventhouse KQL table or a lakehouse table.

Choose it when Freshness in seconds matters.

Look elsewhere when Minutes are fine: A uses no capacity between loads.

Azure Event Hubs guide

C

Back to the plant

Read files back​

Close the loop on a schedule
Freshness
Your schedule
Microsoft Fabric compute
Only the notebook that writes the results
Delivery
Reads current files; safe to repeat

In MaestroHub

  1. Pipeline: Schedule trigger, then List Paths and Read File, then publish to a UNS topic.

In Microsoft Fabric

  1. A notebook that writes results as JSON or CSV under Files/results/.

Choose it when Fabric produces values the plant should use.

Look elsewhere when You want a SQL query against the warehouse: not available yet, see the note on this page.

OneLake guideSchedule trigger

No SQL read-back yet

The SQL Server connector signs in with a username and password, and Fabric's SQL analytics endpoint accepts only Microsoft Entra ID, so a pipeline cannot query a Fabric warehouse or lakehouse with SQL today. Read results back as files (C).

Compare the routes​

Routes combine: most plants run one route in and one route back.

RouteDirectionFreshnessMicrosoft Fabric computeDeliveryNetworkBest for
Into Microsoft FabricMinutes, as often as you loadFabric capacity while the load runsOne file per batch; a load reads only the files it namesOutbound HTTPS 443 to OneLakeTelemetry into lakehouse Delta tables and Direct Lake reports
Into Microsoft FabricSecondsFabric capacity runs the EventstreamAt least once; remove duplicates on a message ID if neededOutbound to Event HubsReal-time dashboards in Real-Time Intelligence
Back to the plantYour scheduleOnly the notebook that writes the resultsReads current files; safe to repeatOutbound HTTPS 443 to OneLakeModel outputs and targets written as files