Connector Config Conventions

Every field on a connector's connector.yaml and its Go Config struct lives at the boundary between operator input (UI form value), storage (JSONB / TEXT blob), and runtime execution (typed Go struct). Getting that boundary right without introducing silent unit bugs requires a small set of rules that this page spells out.

Rules 1-4 are enforced by TestConnectorConfigConsistency at apps/backend/modules/connectors/infrastructure/protocols/config_consistency_test.go. Rule 5 is enforced by TestNoHandRolledUnitConversion in config_duration_conversion_test.go alongside it. A CI failure on either means one of the rules below was broken.

New duration fields: type: duration

A new duration field is declared as a duration, with its unit in the value and its bounds beside it:

connectTimeout:
  type: duration
  default: "30s"
  minDuration: "1s"
  maxDuration: "5m"
  label: "Connection Timeout"

The Go field is a time.Duration, read through connector.PrepareConfig and connector.Unmarshal (rules 1 and 5). There is nothing to convert: "30s", "500ms" and "2h" all say their own unit, so rules 2-4 do not apply. ParseDefinition refuses a unit: on a type: duration field, and TestEveryUnitTaggedDurationDeclaresBounds requires the bounds. The form field is DurationField with durationStringSchema(...).

Rules 2-4 below govern the older shape, a type: integer field with a unit: tag. It remains only on fields 2.6 stored as bare numbers; do not use it for a new field.

The five rules

1. Duration fields are time.Duration in Go

Any config field that represents a duration MUST be typed as time.Duration in the Go Config struct. No int seconds, no int64 milliseconds, no float scalars.

Why: time.Duration carries the unit information in the type system. An int field labeled "timeout seconds" is a comment, not an enforcement — the next reader can misread it as ms and get a 1000× mistake. The runtime layer must trust the type.

// Correct
type Config struct {
    ConnectTimeout time.Duration `json:"connectTimeout"`
    FlushInterval  time.Duration `json:"flushInterval"`
}

// Wrong — recreates the ambiguity the type system exists to remove
type Config struct {
    ConnectTimeoutSeconds int `json:"connectTimeoutSeconds"`
    FlushIntervalMs       int `json:"flushIntervalMs"`
}

2. YAML duration fields carry a unit: tag

Any YAML field that a Go time.Duration reads MUST have unit: s or unit: ms set in connector.yaml. This is the wire-shape contract: operators enter a bare integer in the unit declared here, and connector.PrepareConfig converts it to nanoseconds before json.Unmarshal into the Go struct.

Why: without the tag, a wire value of 1000 reaches json.Unmarshal unchanged. Go's default time.Duration JSON decoder treats an integer as nanoseconds — so 1000 becomes 1 microsecond instead of 1 second (or 1 ms). The unit: tag is what tells PrepareConfig to multiply by time.Second or time.Millisecond.

# Correct
connectTimeout:
  type: integer
  unit: s
  default: 30
  label: "Connection Timeout (seconds)"

flushInterval:
  type: integer
  unit: ms
  default: 1000
  label: "Flush Interval (ms)"

# Wrong — a wire value of 30 will materialize as 30 nanoseconds
connectTimeout:
  type: integer
  default: 30
  label: "Connection Timeout (seconds)"

3. Field names carry NO unit suffix when a matching unit: tag exists

When a YAML field has a unit: tag, its name MUST NOT end in that unit's suffix.

• unit: s → field name must NOT end in Seconds
• unit: ms → field name must NOT end in Ms or Milliseconds

Why: the unit: tag IS the wire-shape source of truth. A connectTimeoutSeconds field with unit: s is either redundant (name matches tag) or misleading (name and tag drift). Either way, the suffix is a comment pretending to be a contract.

Fields without a unit: tag may legitimately carry a unit suffix — the name is doing the work the tag would. This is the pattern for domain-native integer fields (DefaultUpdateRateMs int on OPC-DA matches the underlying COM API's DWORD-in-ms parameter) or PLC register values where the "duration" is a scalar the hardware interprets.

Correct                        Wrong
────────────────────────────── ───────────────────────
connectTimeout + unit: s       connectTimeoutSeconds + unit: s
flushInterval  + unit: ms      flushIntervalMs        + unit: ms

connectTimeoutSeconds (no unit tag — name IS the unit indicator)
DefaultUpdateRateMs   (no unit tag — int type, COM API convention)

4. UI labels reflect the wire unit

The label: in YAML MUST make the wire unit visible to the operator. A unit: s field's label mentions seconds ("Seconds", "sec", " (s)"). A unit: ms field's label mentions ms ("ms", "milliseconds").

Why: the operator sees the label, not the tag. Without an explicit unit in the label they'll guess — and the wire is a bare integer, so guessing wrong ships silently.

# Correct
connectTimeout:
  unit: s
  label: "Connection Timeout (seconds)"

flushInterval:
  unit: ms
  label: "Flush Interval (ms)"

# Wrong — operator can't tell if 30 means seconds or ms
connectTimeout:
  unit: s
  label: "Connection Timeout"

5. Never hand-multiply a unit:-tagged field

Once connector.PrepareConfig has run, a unit:-tagged field holds nanoseconds. Reading it back out and multiplying by its unit again squares the unit. Take the prepared value as the time.Duration it already is:

// Correct — the whole struct, defaults and durations included.
prepared := connector.PrepareConfig(definition, decryptedConfig)
if err := connector.Unmarshal(prepared, &cfg); err != nil { ... }

// Also correct — reading one prepared field by hand.
cfg.ConnectTimeout = time.Duration(prepared["connectTimeout"].(int64))

// Wrong — squares the unit, and the type switch misses the int64
// that convertDurations actually emits, so the operator's value is
// dropped without a word.
if v, ok := prepared["connectTimeout"].(float64); ok {
    cfg.ConnectTimeout = time.Duration(v) * time.Second
}

Why: this is not hypothetical. MongoDB shipped the "wrong" form above and satisfied rules 1-4 completely — unit: s in YAML, a time.Duration in Go, no unit suffix, a "(seconds)" label. Because convertDurations emits an int64 and the switch tested only float64 and int, neither branch matched: every connection silently ran on the hardcoded 30s fallback no matter what the operator entered, so the Advanced tab's Connection Timeout was decoration for the life of the connector (#4827).

Multiplying is still correct for a field WITHOUT a unit: tag (mqtt's keepAlive is a plain int of seconds), and for code reading the RAW map before PrepareConfig. The test only fires inside packages that call PrepareConfig.

Defaults: the manifest and the form

default: in connector.yaml is consumed by PrepareConfig, by the generated docs, and by nothing at all in the frontend — every <X>ConnectorForm.tsx restates its defaults by hand in defaultValues and in the zod chain. That makes them two independent sources of truth for the same number, and they drift silently.

When they disagree, which side is wrong follows from how the backend sources its default — not from preference:

• The protocol builds its Go defaults FROM the manifest (configFromMap(map[string]any{}) = PrepareConfig + Unmarshal): the manifest is the backend's behaviour, so the form is the outlier.
• The protocol has an independent DefaultConfig(): that value breaks the tie, 2-of-3 wins. When Go and the form agree, the manifest is the outlier — this is why smtp's useTLS default was corrected to true rather than the form being weakened to false.

Run the audit after changing either side:

python3 scripts/audit_manifest_form_defaults.py

It reports parse coverage and its own blind spots, and carries an allowlist of deliberate divergences (a blank Host input, for one, so an operator cannot ship a connection silently pointed at localhost). It is an audit rather than a CI test for the same reason as its sibling audit_zod_manifest_requiredness.py: the extractor parses TypeScript text and could be blinded by a refactor, and a guard that goes blind silently is worse than a documented audit run.

The round-trip

The four rules together make one coherent flow:

1. UI    → operator enters `30` in a field labeled "(seconds)"
2. Wire  → frontend sends {"connectTimeout": 30}
3. DB    → stored as-is: {"connectTimeout": 30}
4. YAML  → connector.PrepareConfig reads `unit: s`,
           multiplies: config["connectTimeout"] = 30_000_000_000
5. Go    → json.Unmarshal into `ConnectTimeout time.Duration`
           = 30 * time.Second

Every layer's representation matches its natural scale — operator sees seconds, DB stores seconds, Go stores nanoseconds. No layer has to know the others' scales. The unit: tag is the sole conversion point.

What's NOT covered

• Engine (pipeline / node) configs. The engine has its own Duration wrapper at modules/engine/domain/node.go with different wire semantics — bare number defaults to milliseconds, not seconds. This is a known inconsistency; see internal-docs/architecture/timeout-hierarchy-design.md. A future unification will fold the engine into the same discipline.
• AWS SDK pass-through fields. SQS waitTimeSeconds and delaySeconds are AWS API-defined; we mirror the SDK naming so operator values pass through directly. Allowlisted in the test.
• External wire protocols. ODBC's Config is shaped by the .NET adapter's appsettings.json contract — we don't own the schema alone, so the field names track upstream. Fully carved out in the test.

When you're adding a new field

Quick checklist:

• Is it a duration? → time.Duration in Go, type: duration with minDuration/maxDuration in YAML (never a new unit:-tagged integer).
• Is the field name free of a unit suffix?
• Does the YAML label mention the unit?
• Does TestConnectorConfigConsistency still pass?
• Is the prepared value read as a Duration, never multiplied again?
• Does the form's default match the manifest's? (python3 scripts/audit_manifest_form_defaults.py)

If the answer to any of the first three is "no," the fourth will fail and tell you which rule broke.