Elastic Agent — ECS ingest from MaestroHub

Poll /audit/export from Elastic Agent (Fleet-managed) using the HTTP-endpoint input. Records arrive as ECS-shaped nested JSON — Elastic Security's built-in detections index them without a custom mapping.

1. Create an OAuth2 client for the collector

Same as Splunk setup step 1 — grant type Client Credentials, scope audit_log:export.

2. Configure the HTTP-endpoint input

Elastic Agent integration policy → Add integration → Custom HTTPJSON Logs:

- type: httpjson
  data_stream:
    dataset: maestrohub.audit
    type: logs
  interval: 60s
  request.url: https://<your-maestro>/api/v1/audit/export
  request.method: GET
  request.transforms:
    - set:
        target: url.params.format
        value: ecs
    - set:
        target: url.params.from
        value: '[[.cursor.timestamp]]'
    - set:
        target: url.params.to
        value: '[[.now]]'
    - set:
        target: url.params.cursor
        value: '[[.cursor.opaque]]'
    - set:
        target: url.params.limit
        value: "1000"
  request.rate_limit.limit: 30
  request.rate_limit.remaining: 30
  auth.oauth2:
    client.id: ${MAESTRO_CLIENT_ID}
    client.secret: ${MAESTRO_CLIENT_SECRET}
    token_url: https://<your-maestro>/api/v1/oauth2/token
    scopes: ["audit_log:export"]
  cursor:
    timestamp:
      value: '[[.last_event.@timestamp]]'
    opaque:
      value: '[[.last_response.header.X-Audit-Next-Cursor]]'
  response.split:
    target: body
    type: array

Note the cursor block — Elastic Agent persists both timestamp (for from on next poll) and the opaque cursor (for cursor= on next poll). Both are advanced from the previous response.

3. Ingest pipeline

Elastic Security's Cloud Security detections key off ECS categories (iam, authentication, configuration). Our export ships those categories directly — no ingest-pipeline transform needed.

If you want to tag Maestro-specific fields for your dashboards, add an ingest pipeline in Kibana:

{
  "processors": [
    {
      "set": {
        "field": "event.provider",
        "value": "maestrohub"
      }
    }
  ]
}

4. Validate

Kibana Discover:

event.dataset : "maestrohub.audit"

Sample document:

{
  "@timestamp": "2026-07-21T14:33:12.000Z",
  "event": {
    "action": "role.assigned",
    "category": ["iam"],
    "outcome": "success",
    "severity": 5,
    "module": "authz",
    "kind": "event",
    "target": { "type": "role", "id": "Persona.DataEngineer" }
  },
  "user": {
    "id": "alice",
    "email": "alice@acme.com",
    "target": { "id": "bob" }
  },
  "source": { "ip": "10.4.2.9" },
  "user_agent": { "original": "curl/8.4.0" },
  "trace": { "id": "req_9f3a7c2b" },
  "organization": { "id": "acme" }
}

Elastic Security detection compatibility

Because we ship ECS-native categories, these built-in detections work out of the box:

• Identity and Access Audit rules (any change to role assignments, policy grants, or IAM state).
• Privilege Escalation via Role Assignment — triggers on event.action=role.assigned combined with event.severity>=7 (which our SIEM projection assigns to denied ops and impersonation starts).
• Impersonation events — Elastic reads user.effective.id which our ECS encoder maps to ImpersonatorID.

Field reference

See SIEM integration index — correlation and record semantics. Field paths mirror ECS 8.x.

Troubleshooting

Fleet shows the integration as "unhealthy" with a 401. Client credentials failed. Verify the client.id / client.secret in the Fleet policy match the OAuth2 client's secret in MaestroHub. Rotate the secret if in doubt.

Cursor block doesn't persist across restarts. Elastic Agent's cursor persistence is stored in its data dir. If you're running in an ephemeral container, mount /usr/share/elastic-agent/state to a persistent volume.

Documents are ingested but event.category is empty. The Maestro record was emitted before UC36 shipped (pre-existing rows carry empty categories). Query event.category : "iam" or event.action : * to see new records; older rows will backfill once they age out of retention.