AWS S3 Nodes

Fetch one object from, or write one object to, Amazon S3 or an S3-compatible store such as MinIO from your pipelines.

Needs

An S3 connection (bucket and prefix)

Operations

Fetch Object · Write Object

Defaults

Pattern match picks the most recently modified

Output

Fetch: base64 content plus object metadata

Amazon S3 and S3-compatible stores (MinIO, Ceph, Wasabi, …) hold the objects a plant produces and consumes: exports, reports, model artifacts, archives. MaestroHub provides two dedicated nodes for fetching an object into a pipeline and writing one out, against the bucket and prefix of the connection.

Configuration Quick Reference

Field	What you choose	Details
Parameters	Connection, Function, Function Parameters, Timeout Override	Select the connection profile, function, configure function parameters with expression support, and optionally override timeout.
Settings	Description, Timeout (seconds), Retry on Timeout, Retry on Fail, On Error	Node description, maximum execution time, retry behavior on timeout or failure, and error handling strategy. All execution settings default to pipeline-level values.

S3 Fetch Node

Read one object. objectKey may be an exact key, a wildcard or regex pattern, or a template; versionId reads a specific version of a versioned bucket. When a pattern matches several objects, the function's When Several Objects Match setting picks one: the most recently modified (the default) or the one whose key sorts last. If the listing under the pattern's prefix holds more objects than the discovery limit, the node fails rather than choosing among part of them.

Function Name	Purpose	Common Use Cases
Fetch Object	Read the object at objectKey (optionally versionId, maxFileSizeMB, discoveryLimit)	Daily report ingestion, picking up partner drops, hydrating a dataset

Output Structure

{
  "result": {
    "data": "aWQsbmFtZSx2YWx1ZQoxLGFscGhhLDEwMAo...",
    "metadata": {
      "objectKey": "reports/orders.csv",
      "fileName": "orders.csv",
      "fileType": "csv",
      "sizeBytes": 2048,
      "discoveryPattern": "^reports/.*\\.csv$",
      "selectedBy": "lastModified",
      "matchCount": 3,
      "contentType": "text/csv",
      "etag": "\"9b2cf535f27731c974343645a3985328\"",
      "lastModified": "2026-09-07T08:30:00Z",
      "storageClass": "STANDARD"
    }
  },
  "_metadata": { "success": true, "functionId": "<function-id>", "durationMs": 178, "timestamp": "2026-09-07T10:30:00Z" }
}

Field	Type	Description
result.data	string	The object content, always base64-encoded; decode it downstream, or feed it to a File Extractor with Input Field result.data
result.metadata.objectKey	string	The key read: the resolved one when objectKey was a pattern
result.metadata.fileName	string	Basename of the key
result.metadata.fileType	string	Extension without the leading dot
result.metadata.sizeBytes	number	Object size in bytes
result.metadata.discoveryPattern	string	The pattern that resolved the key (only when objectKey was a pattern)
result.metadata.selectedBy	string	The rule that picked the object among the matches: lastModified or name (only when objectKey was a pattern)
result.metadata.matchCount	number	How many objects the pattern matched (only when objectKey was a pattern)
result.metadata.contentType	string	MIME content type (when S3 reports it)
result.metadata.etag	string	Object ETag (when S3 reports it)
result.metadata.lastModified	string	Last modification timestamp, RFC 3339 UTC (when S3 reports it)
result.metadata.versionId	string	The version read (versioned buckets)
result.metadata.serverSideEncryption	string	Encryption at rest (AES256, aws:kms) when the object is encrypted
result.metadata.storageClass	string	Storage class (when S3 reports it)

---

S3 Write Node

Write one object from data to objectKey. Data is written as text, or decoded from base64 when the function's Input Format is base64 (output from a Data Serializer or an S3 Fetch needs this). Optional settings: contentType, storageClass, serverSideEncryption, cacheControl, metadata and tags. With Overwrite Existing off, an existing key is kept and a timestamped variant is written instead. Generate Presigned URL adds a time-limited download link to the result.

Function Name	Purpose	Common Use Cases
Write Object	Upload data to objectKey	Exporting pipeline results, publishing reports, archiving

Output Structure

{
  "result": {
    "objectKey": "exports/orders.csv",
    "bytesWritten": 4096,
    "contentType": "text/csv",
    "storageClass": "STANDARD",
    "etag": "\"9b2cf535f27731c974343645a3985328\"",
    "presignedUrl": "https://bucket.s3.amazonaws.com/exports/orders.csv?X-Amz-Signature=...",
    "presignedUrlExpiresAt": "2026-09-07T11:30:00Z",
    "presignedUrlExpiry": 3600
  },
  "_metadata": { "success": true, "functionId": "<function-id>", "durationMs": 245, "timestamp": "2026-09-07T10:30:00Z" }
}

Field	Type	Description
result.objectKey	string	The key written: a timestamped variant when the object existed and Overwrite Existing was off
result.bytesWritten	number	Bytes uploaded
result.contentType	string	The content type stored
result.storageClass	string	The storage class stored
result.etag	string	The object's ETag (when S3 reports it)
result.versionId	string	The version S3 assigned (versioned buckets)
result.serverSideEncryption	string	The encryption applied (when the object is encrypted at rest)
result.presignedUrl	string	A presigned GET URL, only when Generate Presigned URL is on and signing succeeded
result.presignedUrlExpiresAt	string	When the presigned URL stops working, ISO 8601 UTC, alongside result.presignedUrl
result.presignedUrlExpiry	number	Deprecated: the URL's lifetime in whole seconds. Read result.presignedUrlExpiresAt instead
result.presignedUrlError	string	Why the presigned URL could not be made, only when it was requested and signing failed; the write itself succeeded

---

Common Use Cases

• Pick up a partner drop: a Schedule trigger → S3 Fetch with objectKey inbound/orders-*.csv → File Extractor (Input Field result.data) → downstream.
• Publish a report: a SQL Query → Data Serializer (CSV) → S3 Write (Input Format base64, since the serializer emits base64 by default) with objectKey {{ 'reports/' + ($execution.startedAt | date: 'YYYY-MM-DD') + '/orders.csv' }} and Generate Presigned URL on → a Slack or Teams message carrying {{ $node["S3 Write"].result.presignedUrl }}.

The call's own facts ride along under _metadata next to the four every connected node carries: $node["S3 Fetch"]._metadata.bucket names the bucket the call ran against and $node["S3 Fetch"]._metadata.method the operation (s3.fetch.object or s3.write.object).