Nuances: people and access

What could surprise you here, why it is so, and what to do. Each entry has a level, 1 (day one) to 5 (expert): the index lists them all by level.

Reading payloads is a permission, not part of a role (level 4)

• What: GET /api/v1/messages/{id} (and the message page) returns the payload, and the saved output of each destination, only to admins and to users and tokens that were given the payloads permission. Without it the message page still shows its status, destinations and errors, and says the payload is hidden. Editing a message needs the permission too (correcting a message means reading it). Each read is in the audit log (next entries). Existing tokens were revoked when users came in (migration 0018); a new token has no permissions unless it is given them.
• What it doesn't hide: the ordering key (often an MRN), the control id, error texts (a quarantine's error can quote a field value) and sizes are still shown to anyone who can read the interface. The permission protects the message content, not every identifier the engine has seen. Write error messages in interface code the way you would write a log line that a support person may read.
• What to do: give payloads to the people who fix data problems and to nobody else; a monitoring token needs none. ductus user create ... --permission payloads, ductus token create ... --permission payloads.

Passwords, lockout and what a disabled account loses (level 2)

• What: people sign in with a username (not case sensitive) and a password of at least 12 characters (auth.password.min_length; never below 8) that isn't the username, one character repeated, or one of a short list of common ones. Passwords are stored as argon2id hashes. Five wrong passwords for one username within 15 minutes lock that account for 15 minutes (auth.lockout); the right password doesn't help meanwhile, and an admin can unlock at once. A wrong username and a wrong password give the same answer in about the same time.
• Why lock by username: per username, not per address, because sign-ins arrive through a proxy. The price is that someone who knows a username can keep that person locked out; an admin unlocks, and locking was the safer side of that choice for patient data.
• Disabling a user ends their sessions at once and stops every token of theirs (their scripts get 401). A password an admin chose (a new account, a reset) is temporary: the person's first sign-in can do nothing but change it. There is always at least one admin who can sign in: the last one can't be disabled or demoted.
• What to do: make accounts for people, tokens for scripts; let people make their own tokens under Account (they stop with their owner). Turn auth.token_sign_in off once everyone has an account, so a leaked token can't also open the UI. If you lock yourself out, ductus user on the server (it works on the database directly) makes or unlocks an admin.

The audit log records who did what, and what it leaves out (level 2)

• What: GET /api/v1/audit (the Audit page, ductus audit export) lists, newest first: sign-ins, failed sign-ins (and whether they locked the account), sign-outs, password changes and resets, user and token changes, payload reads, resends, replays, edits and dismissals of messages, pauses and resumes, drains, retention changes and confirmations, connection checks, exports of the log itself, and the engine's own starts, stops and stopped interfaces. Each row has who (a user, a token, system for the engine or someone at the server's command line, anonymous for a sign-in as a name nobody has), how (web UI, token, command line), the address, what it was done to, and a reason or details. Reading it needs the audit permission (admins have it); anyone signed in with a password sees their own activity under Account.
• What it leaves out: passwords, tokens and payloads are never in it. A failed sign-in for a name nobody has keeps no name: what people type in the username box is sometimes their password. Messages read through the database directly (psql) or by the engine's own code aren't API calls, so they aren't rows: the log covers the API, the web UI and the commands of ductus.
• The address is the connection's. Behind a TLS proxy (which the UI needs over a network) it's the proxy's address for everyone; the audit log can still say who, just not from where. A forwarded-for header is not trusted yet.
• What to do: give audit to the compliance or security person, not to operators. Ship the log to your SIEM with ductus audit export --since <day> (JSON lines, oldest first; the export is itself a row).

Payload reads are logged before they are shown (level 3)

• What: a read of a payload (GET /messages/{id} by someone with payloads) writes its audit row first. If the database can't take the row, the answer is 503 and no payload is returned: an unlogged read of patient data is worse than a refused one. The same message read again by the same person within audit.payload_read_window (10 minutes) is one row: a screen that reloads itself (the UI refetches when the window regains focus) would otherwise write a row every few seconds and bury the log. 0s logs every read. audit.payload_reads: false turns the rows off (the permission still applies).
• What to do: read "viewed message N between these times" as "at least once in a window", not as a count of looks. If you need every read, set the window to 0s and size the log for it (a row is about 300 bytes).

Dismissing a quarantined message is not deleting it (level 2)

• What: POST /messages/{id}/dismiss and /messages/dismiss (the Quarantine screen's "Dismiss selected") close a quarantined delivery nobody will fix or resend. The delivery becomes filtered with "dismissed by NAME: reason", leaves the quarantine, and nothing is sent; the message is settled again (sent, if another destination sent it). A reason is required. Only quarantined deliveries are touched: a pending one is being retried, a sent one isn't a problem. The message, its payload (while its retention lasts) and the dismissal stay visible on its page, and it can still be replayed later. In the held area a dismissal is how a held quarantine ends: the month's table drops once nothing in it is unfinished.
• What to do: write the reason for the next person ("lab retired the unit; these are the 37 results for it"), not for yourself. A dismissal is a decision about a patient's data not reaching a system; the audit log keeps who made it.

Copies of payloads and the audit log are kept for different times (level 3)

• What: every resend, edit and dismissal keeps what the delivery was before, including the output it had sent (message_actions.previous_output, for destinations with save_output: true). That is a copy of patient data, so it goes when the message's own retention ends: an hourly job sets it to NULL for actions older than the interface's retention (1, 3 or 7 weeks; an interface that is no longer configured gets 7); the row (who, when, what it was) stays. The rows themselves, and the audit log, are deleted after audit.keep: six years unless the project says otherwise (at least one year; ask your compliance office what your organization needs). At about a thousand rows a day that is some two million rows over six years.
• What to do: set audit.keep in ductus.yaml to your documented retention period, then leave it; shortening it deletes history at the next hourly run. The engine's log shows what the job cleared and deleted.

A crash shows in the audit log (level 3)

• What: each start writes engine.start (version, host, and how the previous run on this host ended); each clean stop writes engine.stop with the same instance. A start whose predecessor has no stop was killed, crashed or lost power; the new start's row says previous: unclean and the engine logs a warning. A stop that had to give up waiting (30 s) is written forced. With two engines on one database, each host is judged by its own rows.
• What to do: after an unclean start, nothing is lost (messages in hand are processed again, as the at-least-once rule says), but look at the receivers for duplicates around that time.

Signing in with the directory: local names first, groups decide the role (level 2)

• What: with auth.ldap in ductus.yaml (and its directory in resources.yaml), people sign in on the same page with their directory username and password. A name a local account holds is checked here and only here, never against the directory: a directory account can't take over the break-glass admin's name, and a directory person whose name a local account already has can't sign in until one of them is renamed. Any other name goes to the directory: the service account finds the person (user_filter, with what they typed escaped, so * or )( match nothing), the engine binds as them with their password, then reads their groups. The highest role group they are in wins; permissions groups only add to a role; in no role group, they are told they have no access (the password was right, so that tells nobody anything new). Their account is made at the first sign-in and shows as a directory account; role, permissions, name and email are the directory's at every sign-in, read-only here. An admin can still disable one here: then it can't sign in, whatever the directory says.
• What to do: make Ductus groups in the directory (admins, operators, viewers, and PHI readers for payloads), not one per person. Keep one local admin (ductus user create) for when the directory is down, with a long password kept where the on-call can get it. Don't map a role to a group everyone is in (Active Directory's primary group, Domain Users, isn't in member and wouldn't match anyway).

Write several group DNs one per line, not [a, b] (level 2)

• What: a group takes one DN or a list. In YAML's flow style (payloads: [CN=PHI,OU=Groups,DC=h, CN=Night,OU=Groups,DC=h]) every comma separates items, the commas inside each DN too, so the list becomes CN=PHI, OU=Groups, DC=h, ...: pieces that each look like a DN and match no group. The engine refuses to start (and ductus validate says) when a group "DN" has only one part, naming the piece.
• What to do: write lists in block style, one DN per line (- CN=PHI,OU=Groups,DC=h); a single DN can stay on the key's line.

A person removed from the groups loses access within refresh (level 2)

• What: right after the start (in the background) and then every auth.ldap.refresh (15 minutes by default, at least 1 minute) the engine reads again the groups and account of every directory person with a live session or token, and of every directory account an owner in its project names (since 0.6.7.3, whether or not they ever signed in: next entry). Removed from every role group, disabled in Active Directory (userAccountControl), or deleted: their sessions end, their tokens stop, the account shows "Refused by the directory", and the audit log has a user.directory_refused row by the engine. Changed groups change the role and permissions the same way (user.directory_change). Put back, they sign in again (and their tokens work again at the next refresh). Between two refreshes, a session keeps the role it had: a removal takes up to refresh.
• A sign-in the directory refuses (unknown name, disabled, no group) refuses the account with that name only when the entry it found is that account's person (or the account has no id yet): someone typing a renamed person's old name doesn't end their sessions; the refresh, by id, decides.
• What to do: for someone who must lose access now, disable the account on the Users page (at once), then remove them in the directory. A shorter refresh costs one search per person per group each time: 5 minutes is fine for a few hundred people.

Directory owners get their account when they're in a Ductus group (level 2)

• What: with auth.ldap, an owner written as an email address that reaches nobody (no account has it, now or before, and it isn't a team in owners:) is looked up in the directory by its email attribute (attributes.email) at each refresh round, the first right after the start. When exactly one entry has that email and it is in a Ductus role group (the same check as a sign-in), the engine makes their directory account then: made by the engine (audit user.create, "reason": "owner"), never signed in (the Users page says "Never signed in" and "made because they own something"). From then on it is an ordinary account: the Owners page shows the person, alerts mail them, Interfaces → Mine works at their first sign-in, and the round checks it like a signed-in one (a rename is seen within refresh; someone deleted or removed from the groups there is refused, so the owner shows as disabled, ductus owners --check fails on it, and no alert mail goes to their old address, even when the owner is written as that address). Anyone else stays an address (alerts go to the text itself): two entries with that email, or one in no Ductus group, make nothing, and the engine warns once per run ("in the directory but in no Ductus group: add them to one"). Owners written as a name or display name are never looked up: they must already be an account. A directory down, or slow, never delays a start: the round runs in the background, and tries again at the next refresh.
• Cost: one search per owner email that isn't an account yet, every round, plus the group reads for each one found; once an account, a check by id with its groups, like a signed-in person's. Measured against a directory on the same machine: 200 owner emails nobody there has, 75 ms a round; 200 owners' accounts, 0.38 s a round (nearly all of it the directory's searches: an account the directory says nothing new about writes nothing). Over a network, add about a millisecond per search.
• What to do: write owners as their email, and put the people who own interfaces in a Ductus group (a viewer group is enough to be an owner). For outside people (a vendor's support address), the address is fine: it is mailed as it is.

A new directory account for the same person can't be linked to the old one (level 2)

• What: a person deleted in the directory and made again (rather than renamed) is a new entry with a new id: Ductus can't tell it's them. Their old Ductus account is refused at the next round, and an owner that names it shows as disabled (no alert mail) until the new account exists. When the owner text is an email and the new entry has the same email and a Ductus group, the next round makes the new account, and it wins (at one level an enabled account beats disabled ones). Written another way, the owner stays disabled until it is reassigned.
• What to do: rename people in the directory rather than re-creating them. When one was re-created, check ductus owners --check, and ductus owners reassign --from <old text> --to <their email> where it still fails.

A renamed directory person keeps one account; a reused name gets a new one (level 2)

• What: a directory account is the directory's person, known by their directory id (auth.ldap.attributes.id: by default Active Directory's objectGUID, else entryUUID, which OpenLDAP, 389 Directory Server and FreeIPA give every entry), read at each sign-in and refresh and kept on the account. The refresh looks people up by that id, not by name. So when the directory changes someone's sign-in name (jsmith → jjones at a marriage), the next refresh or their sign-in with the new name renames their account: sessions, tokens, settings and history stay, and the old name is kept (audit user.directory_change with "username": [old, new]). When a departed person's sign-in name is given to someone new (a new entry, so a new id), the new person gets an account of their own; the earlier account is renamed <name>#<its id> out of the way and refused, so its tokens never work again, whoever holds them. Every change of a username, display name or email, by the directory or an admin, keeps the old value (for 0.6.7.2's owners). A person given a new directory account (deleted and made again, rather than renamed) is a new id too: Ductus can't link the two, so their old account is refused at the next refresh and they start with a new one.
• Before 0.6.7.1 no id was kept: an existing account takes its id at its first sign-in or refresh by name, so a name reused before that check can't be told apart (as before). With attributes.id: none, or a directory that gives no id, accounts are known by sign-in name only: ductus validate warns, and the engine logs it once.
• The sign-in name is read from the attribute user_filter matches {username} against (sAMAccountName by default); a filter that matches it another way needs attributes.username, or a refresh can't see the name change (validate warns).
• What to do: rename people in the directory rather than deleting and re-creating them, and revoke a person's tokens when they leave (the refusal stops them anyway; revoking makes it final). A rename onto a name a local account holds is refused and logged at each refresh: rename the local account.

OpenLDAP: group membership follows a rename only with refint (level 3)

• What: Active Directory's group member links follow a renamed or moved person by themselves. OpenLDAP's are plain DNs: without the refint overlay, a modrdn of a person leaves their old DN in every group, so after the rename they're in no Ductus group, and the refresh refuses them ("in no Ductus group") although Ductus found them by id. An owner who never signs in is refused the same way (it shows as disabled, and gets no alert mail).
• What to do: on OpenLDAP, turn on refint for member and uniqueMember (the tests' directory does: scripts/dev/slapd.sh), or update the groups by hand at each rename.

When the directory is down (level 2)

• What: a directory person's sign-in answers 503 "the directory can't be reached right now" (not "wrong password": they would retype it and lock themselves out). Local accounts keep working. Sessions already open keep working: a refresh that can't reach the directory changes nothing (an outage must not sign the whole hospital out), so an outage also delays a removal until the directory answers again. A start never waits for the directory (its refresh round runs in the background, measured: a start with the directory down took as long as one without auth.ldap). A wrong service account password (bind_password) looks the same as an outage: the engine logs it as an error, and ductus doctor (full) says which.
• What to do: watch the directory's certificate like the others (it is in the Connections table) and keep the break-glass admin. Point url at a name that fails over (a domain's DNS name that resolves to several controllers), not at one controller.

Wrong passwords count twice: here and in the directory (level 3)

• What: a wrong directory password counts toward Ductus's lockout for that username (5 in 15 minutes, auth.lockout) once the person has signed in at least once (before that there is no account here to count on), and the directory counts it toward its own lockout policy. Active Directory answers a disabled, locked or expired account like a wrong password; the person sees "wrong username or password" either way.
• What to do: set Ductus's lockout at or below the directory's, so Ductus refuses first and a typo storm doesn't lock the person out of everything else they use. A person locked here is unlocked on the Users page; locked in the directory, by the directory's admins.

Nested groups: Active Directory answers, other directories are walked (level 4)

• What: nested: true counts members of groups inside the configured groups. On Active Directory (seen in its root entry) one search per configured group uses its transitive-membership rule (member:1.2.840.113556.1.4.1941:=). Other directories (OpenLDAP) have no such rule: the engine walks up from the person's groups, at most 8 levels, searching group_base (default: the directory's naming context) for groups that contain each one. Without nested, only direct members count (member, or uniqueMember for groupOfUniqueNames).
• Not tested here: real Active Directory isn't available to the tests (OpenLDAP is); the transitive filter is unit-tested and is verified with the first pilot.
• What to do: prefer direct membership of the Ductus groups: it is one search per group and easy to audit. With deep or wide nesting on OpenLDAP, set group_base to the groups' OU so the walk searches less.

The directory connection is TLS, verified like the rest (level 3)

• What: url: ldaps://... (TLS from the start, port 636) or ldap://... with starttls: true. The certificate is checked against this server's trusted certificates, the same as HTTPS; a directory signed by the hospital's own CA that this server doesn't trust is refused unless ca: names that CA (a certificates: entry: hospital-ca: { ca: ${secret:HOSPITAL_CA} } or ca_file:), which is then trusted for this directory only. Plain ldap:// is refused unless insecure: true, which ductus validate and the engine warn about: people's passwords would cross the network readable.
• What to do: use ldaps:// or StartTLS everywhere but a test directory. On Windows servers joined to the domain the CA is usually installed already; on Linux servers it usually isn't (ca:).

Theme and colors follow the account (level 1)

• What: a signed-in user's theme and custom colors are saved on the server and applied in any browser they sign in to. A copy stays in the browser so the right theme shows before the page has loaded; signing out clears it, so the next person starts from the default. A session made with a token has no account: its choice stays in that browser.
• What to do: nothing; a person who had picked colors before accounts existed has them sent to their account the first time they sign in.