Nuances: what to know before you rely on it

Every engine has behaviour that surprises people the first time. Ductus can't remove all of it, but it can tell you about it up front. These pages are the running list: what happens, why, and how to write your interfaces and set expectations with it in mind.

Each entry has a level (1 = everyone needs it on day one, 5 = expert). The levels match the planned course tracks, from first interface to expert.

For engine developers: whenever a change adds or reveals a nuance, add it to its section's page in the same commit, with a level, and link it from the guide page it affects (every_nuance_is_linked_from_a_guide fails otherwise).

Level	Who it's for
1 Foundations	Writing a first interface
2 Practitioner	Building and testing interfaces day to day
3 Professional	Databases, APIs, files, schedules, ordering, retries
4 Advanced	Isolation, performance, operations
5 Expert	Delivery guarantees and failure semantics under crashes and runaways

By subject

Messages and delivery (5)
The store (Postgres) (2)
Several engines on one database (4)
Retention (7)
Several destinations (7)
Order, retries and durability (7)
Resending and correcting messages (3)
Routing between interfaces (6)
Writing transforms (10)
Shared pool (3)
Runaway scripts (6)
Databases and APIs (5)
Impact and about: (8)
Checking connections (ductus doctor) (4)
Files, SFTP and schedules (4)
Operations: drain mode and the REST API (6)
The web UI (3)
Monitoring (10)
Alerts (10)
TLS on what the engine listens on (13)
Who may connect (allow:) (7)
TLS on what the engine connects to (5)
TLS to Postgres (6)
TLS to SQL Server (6)
SQL Server Windows logins (6)
Certificates (10)
People and access (19)
Local development (2)
Upgrades (15)
Windows (9)

Level 1: Foundations (24)

Retention is 1, 3 or 7 weeks, not any number of days (Retention)
Missing fields read as "" and never throw (Writing transforms)
return null filters; throwing quarantines (Writing transforms)
Fixtures can be read only inside a test (Writing transforms)
A quoted number in an environment file is text (Writing transforms)
A destination's owner is its interface's unless it says otherwise (Impact and about:)
Moving ownership in the web UI is a preview, not a change (Impact and about:)
An owner is a person when the text names exactly one account (Impact and about:)
Operators start on their own interfaces (Impact and about:)
Your own colors can be unreadable on purpose (The web UI)
There's no search by message content yet (Monitoring)
Silencing is not pausing, and acknowledging is not fixing (Alerts)
The API on the network needs tls (TLS on what the engine listens on)
Testing TLS without the real systems (TLS on what the engine listens on)
Cloud Postgres requires TLS (TLS to Postgres)
auth: windows: the engine's own account, or a named one (SQL Server Windows logins)
Theme and colors follow the account (People and access)
ductus dev is a development tool, not a way to run in production (Local development)
Your editor's help is the installed version's: run ductus types after an upgrade (Local development)
Run ductus upgrade --check with the new version before every upgrade (Upgrades)
Deploying project changes: systemctl reload (Upgrades)
ductus replay shows what a change would do to real traffic, before you make it (Upgrades)
The installer isn't signed yet: SmartScreen says "unknown publisher" (Windows)
Certificate files: write the drive (Windows)

Level 2: Practitioner (84)

Stopping the engine: what finishes and what waits (Messages and delivery)
A start waits 30 s for a database that refuses, then says why (The store (Postgres))
An interface runs on one engine at a time; a second one is refused (Several engines on one database)
Messages are kept at least their window, and gone within a period after (Retention)
Pending messages are never purged; past the window they're held (Retention)
Quarantined messages are kept until resolved, unless you say purge (Retention)
Shortening retention deletes messages, so it needs a confirmation (Retention)
Each destination runs its own function on the original message (Several destinations)
One destination down holds up only itself (Several destinations)
Pausing a destination: at once, no questions, and it holds (Several destinations)
Filtering and quarantine are per destination (Several destinations)
Resend sends a message again in place: first in its lane, not last (Resending and correcting messages)
The receiving interface's keys, dedupe and destinations apply (Routing between interfaces)
Routes are fixed in interface.yaml, and loops are refused (Routing between interfaces)
Imports used only for their side effects are removed (Writing transforms)
Copy what you change, not the whole thing through JSON (Writing transforms)
With split: records, input() gives a fixture's first record (Writing transforms)
Interfaces share a global scope (Shared pool)
Module-level caches live until the engine restarts (Shared pool)
timeout counts time without pausing, not total time (Runaway scripts)
A runaway message is quarantined; three in a row stop the interface (Runaway scripts)
api() headers: the API's, then yours (Databases and APIs)
Write resource names out: db("patient_db"), not db(name) (Databases and APIs)
ductus impact only knows what's named (Impact and about:)
Moving an owner moves their resources.yaml entries too (Impact and about:)
Your own documentation fields are checked by ductus validate, not by the engine (Impact and about:)
Doctor checks from where it runs (Checking connections (ductus doctor))
A pass means "can connect and log in", not "will accept your messages" (Checking connections (ductus doctor))
On Linux the service can write only /var/lib/ductus until you add a folder (Files, SFTP and schedules)
A start logs one line, not one per listener (Operations: drain mode and the REST API)
A console session acts as whoever signed in (The web UI)
In respond mode, calls run one after another unless you await them together (Monitoring)
Nothing is expected until you say so (Monitoring)
What each state means, and when it shows (Monitoring)
A receiver that's back can stay stalled for up to a minute (Monitoring)
Alerts tell you once, then remind; they don't repeat every check (Alerts)
A pause never alerts, and holds an alert open (Alerts)
An engine that is down sends nothing: use the heartbeat (Alerts)
Quiet hours hold warnings until morning; critical ones never wait (Alerts)
Who "owner" is (Alerts)
Webhook URLs are secrets (Alerts)
A certificate the engine can't use stops the start, with the reason (TLS on what the engine listens on)
Renewing: replace both files, no restart (TLS on what the engine listens on)
A certificate from the hospital's CA: request, then show (TLS on what the engine listens on)
HTTP interfaces on one port share its tls (TLS on what the engine listens on)
An allow list says who, which "one connection only" never did (Who may connect (allow:))
It's defence in depth, next to the firewall (Who may connect (allow:))
List 127.0.0.1 on the API if you use it from its own server (Who may connect (allow:))
HTTP interfaces on one port share its allow (Who may connect (allow:))
A private CA is trusted only where you name it (TLS on what the engine connects to)
A refused handshake is a temporary error: the message waits (TLS on what the engine connects to)
Our client certificate: tell the receiver what to expect (TLS on what the engine connects to)
An HTTP destination through an API uses the API's TLS (TLS on what the engine connects to)
A CA installed on this server counts (TLS to Postgres)
A refused Windows login: 18452 is Windows saying no, 18456 is SQL Server (SQL Server Windows logins)
The engine checks certificates by itself, and logs in to nothing (Certificates)
The earliest expiry in the chain counts, in whole days (Certificates)
A certificate alert is told at 30, 14 and 7 days, and when it expires (Certificates)
A check that can't connect keeps the last dates (Certificates)
Our certificates count down like the rest, with no check (Certificates)
A renewal that can't be used shows on its row, not as an alert (Certificates)
Passwords, lockout and what a disabled account loses (People and access)
The audit log records who did what, and what it leaves out (People and access)
Dismissing a quarantined message is not deleting it (People and access)
Signing in with the directory: local names first, groups decide the role (People and access)
Write several group DNs one per line, not [a, b] (People and access)
A person removed from the groups loses access within refresh (People and access)
Directory owners get their account when they're in a Ductus group (People and access)
A new directory account for the same person can't be linked to the old one (People and access)
A renamed directory person keeps one account; a reused name gets a new one (People and access)
When the directory is down (People and access)
A migration marked heavy holds up a table while it runs (Upgrades)
On Linux a restart refuses no connection: senders wait for it (Upgrades)
systemctl reload hands over: receiving never stops, sending pauses briefly (Upgrades)
A new version or project that can't start leaves the running one serving (Upgrades)
A replay compares outputs with the installed version from 0.6.6.4 on (Upgrades)
A replay's report holds patient data unless you hide it (Upgrades)
A stopped service, a failed start and a drain refuse connections, on purpose (Upgrades)
Windows and containers: a restart is a short gap (Upgrades)
The service's account: the default, a gMSA, or a named account (Windows)
An upgrade can't keep an account's password: the service waits for it (Windows)
An upgrade starts the service again, unless it was stopped on purpose (Windows)
What goes to the Event Log, and what only to the files (Windows)
The Windows service can only touch what you give it (Windows)

Level 3: Professional (67)

A message can be delivered twice (Messages and delivery)
Your transform can run more than once for the same message (Messages and delivery)
Only the original is stored; outputs are recomputed (Messages and delivery)
A dead engine's interfaces are free at once; a vanished host's within ~25 s (Several engines on one database)
Losing the database connection pauses sending, not receiving (Several engines on one database)
PgBouncer in transaction mode can't hold the engine's lock (Several engines on one database)
One retention change at a time per interface (Retention)
Adding, removing or renaming a destination (Several destinations)
Many destinations on one interface: when to split (Several destinations)
Order is per key, and one stuck message holds up its key (Order, retries and durability)
Without an ordering key, more workers don't help (Order, retries and durability)
Temporary failures retry forever by default (Order, retries and durability)
Database login failures make messages wait, not fail (Order, retries and durability)
Respond-mode HTTP requests aren't stored (Order, retries and durability)
A replay is the same message; an edited copy is a new one (Resending and correcting messages)
A message still pending can't be replaced yet (Resending and correcting messages)
A route is exactly once inside the engine, not beyond it (Routing between interfaces)
One query for many values, not one query per value (Writing transforms)
SQL Server: one result set, one batch per call (Databases and APIs)
fetch() and api() retry 5xx, 408 and 429 by default (Databases and APIs)
The documentation snapshot is as old as the last engine start (Impact and about:)
One working address is enough to pass, with a warning (Checking connections (ductus doctor))
A file is picked up one poll after it stops changing (Files, SFTP and schedules)
One file at a time, oldest first; a failure stops the rest (Files, SFTP and schedules)
Cron uses the server's local time; missed runs collapse to one (Files, SFTP and schedules)
Counts and timings start at zero when the engine starts (Monitoring)
Burst and slow receiver need a few weeks of history (Monitoring)
Finding the message or patient that holds a queue up (Monitoring)
Notifications are sent at least once, and a failing channel only shows in the UI (Alerts)
A restart doesn't alert on what it hasn't watched yet (Alerts)
Give the whole chain in cert_file (TLS on what the engine listens on)
Senders on TLS 1.0 or 1.1 can't connect (TLS on what the engine listens on)
A client certificate says which system sent, not who (TLS on what the engine listens on)
A refused handshake is logged once a minute per sender (TLS on what the engine listens on)
A CA's certificate as a server's (TLS on what the engine listens on)
Post-quantum key exchange: on by default, never in the way (TLS on what the engine listens on)
Host names are looked up at start and every 5 minutes (Who may connect (allow:))
Refusals are logged once a minute per listener and address (Who may connect (allow:))
Dead connections are noticed in about 30 seconds (Who may connect (allow:))
Doctor makes the engine's handshake (TLS on what the engine connects to)
Without sslmode, the database connection is TLS when offered, and nobody is checked (TLS to Postgres)
The database's certificate is watched when the engine checks it (TLS to Postgres)
A client certificate renews at the next connection (TLS to Postgres)
SQL Server connections are verified, and trust_server_certificate is for labs (TLS to SQL Server)
Connecting by a name the certificate doesn't list: name_in_certificate (TLS to SQL Server)
encrypt: strict is TDS 8: TLS before anything (TLS to SQL Server)
SQL Server 2022 on Linux doesn't accept encrypt: strict (TLS to SQL Server)
SQL Server's certificate is read through PRELOGIN, and watched when verified (TLS to SQL Server)
The service's own account is the computer on the network (SQL Server Windows logins)
Kerberos or NTLM: doctor and the start log say which (SQL Server Windows logins)
From Linux, a Windows login is always NTLM (SQL Server Windows logins)
Extended Protection set to Required refuses NTLM logins (SQL Server Windows logins)
What has a certificate the engine watches (Certificates)
Behind a load balancer, you see the balancer's certificate (Certificates)
Doctor reads the chain our listeners serve, but not the name senders use (Certificates)
Payload reads are logged before they are shown (People and access)
Copies of payloads and the audit log are kept for different times (People and access)
A crash shows in the audit log (People and access)
OpenLDAP: group membership follows a rename only with refint (People and access)
Wrong passwords count twice: here and in the directory (People and access)
The directory connection is TLS, verified like the rest (People and access)
A replay doesn't write: those messages can't be compared (Upgrades)
A difference from the record can be the data, not the code: drift (Upgrades)
In a replay the clock is the message's arrival, and Math.random repeats (Upgrades)
What a replay costs, and where (Upgrades)
"Log on as a service": the installer grants it, Group Policy can take it away (Windows)
A refused connection takes about a second on Windows, and localhost may be ::1 first (Windows)

Level 4: Advanced (29)

A query that holds the message tables for days can pause intake (The store (Postgres))
Shortening frees disk when partitions go, not at once (Retention)
Change ordering.partitions only with an empty queue (Order, retries and durability)
durability: relaxed can lose the last fraction of a second (Order, retries and durability)
Order holds across a route when the sender keeps it (Routing between interfaces)
A routed message waits up to 20 ms before the next interface sees it (Routing between interfaces)
Routes keep moving while draining (Routing between interfaces)
Short-lived objects are cheap; what you keep is what costs (Writing transforms)
Loops over thousands of items: give objects one shape (Writing transforms)
A shared library file is one copy per thread, not one per project (Shared pool)
Neighbours wait while runaway code runs (Runaway scripts)
The heap limit is shared by everything in an isolate (Runaway scripts)
Dedicated connection pools only for dedicated interfaces (Databases and APIs)
The commit time is the store's disk, and the ceiling per strict sender (Checking connections (ductus doctor))
Draining closes MLLP listeners; it doesn't answer AE (Operations: drain mode and the REST API)
Respond-mode HTTP keeps answering while draining (Operations: drain mode and the REST API)
A drain finishes only when every destination has caught up (Operations: drain mode and the REST API)
Drain mode isn't remembered across a restart (Operations: drain mode and the REST API)
The API listens on this machine only, until you open it (Operations: drain mode and the REST API)
The UI works over https, or at localhost on the engine's own server (The web UI)
Timings are estimates to a bucket, averages are exact (Monitoring)
Prometheus series grow with destinations (Monitoring)
Judging alerts costs little, and runs all the time (Alerts)
What TLS costs (TLS on what the engine listens on)
Opening a database connection over TLS costs ~60 ms, for now (TLS to Postgres)
Verification costs nothing measurable (TLS to SQL Server)
A refused certificate is said once, not logged as an error each time (Certificates)
Reading payloads is a permission, not part of a role (People and access)
Nested groups: Active Directory answers, other directories are walked (People and access)

Level 5: Expert (4)

A "done" marker can't prevent duplicates (Messages and delivery)
A runaway is blamed on the interface, not the destination (Several destinations)
A runaway after await interrupts every async transform on its thread (Runaway scripts)
A runaway in a shared library after await can't be blamed on anyone (Runaway scripts)