Skip to content

What rtb-config does not do

Every limit below is true of the current figment-backed crate. Some are deliberate design choices, some are consequences of the layering engine underneath, and a couple are sharp enough that you want to know about them before you build on top.

Can I use a TOML or JSON config file?

Not as an input, no. ConfigBuilder::user_file parses every file as YAML, whatever its extension. A .toml file registered as a user file fails with ConfigError::Parse complaining that a string was found where a map was expected.

JSON files happen to work, because YAML is a superset of JSON. That is a property of the parser, not a supported format.

This is asymmetric with writing. Under the mutable feature Config::write picks its format from the extension and will happily emit TOML, and that file cannot then be read back by this crate. If you write config out with the intention of reading it again, write YAML or JSON.

There is no way to register a file with an explicit format, and no other input format: no INI, no .env, no HCL.

Which YAML does it read?

YAML 1.2, parsed by serde-saphyr and held where it can be to the behaviour it had when figment parsed it with serde_yaml:

  • yes, no, on, off, y and n are strings. Only true and false are booleans.
  • << is an ordinary key. Merge keys are not expanded; anchors and aliases are.
  • When one map repeats a key, the last one wins, silently.
  • .inf, -.inf and .nan fail the parse, even for an f64 field. Under serde_yaml they reached an f64 field as infinity and NaN.
  • 1_000 is the number 1000. Under serde_yaml it was a string.
  • 0644 is the float 644.0, which fits neither a string nor an integer field. Under serde_yaml it was the string "0644". Quote a value with a leading zero that must stay a string, such as a file mode: mode: "0644".

Why does my underscored field ignore its environment variable?

Because the underscore in the variable name is read as a level of nesting, not as part of a key.

The environment layer is registered with underscore-splitting on, which is what makes MYTOOL_HTTP_PORT populate a nested http.port. The same rule applies to a flat field whose name contains an underscore:

struct MyConfig {
    max_retries: u32,   // cannot be set from the environment
}

MYTOOL_MAX_RETRIES=7 becomes the nested key max.retries, which your struct does not have. Unknown keys are ignored, so nothing fails; the value stays at whatever the file or the embedded default said, with no error and no warning.

There is no option to turn the splitting off. The workarounds are structural:

  • Use nested structs rather than underscored names. A http: HttpSection with a port field is reachable as MYTOOL_HTTP_PORT; a flat http_port is not.
  • Use single-word field names where a value has to be settable from the environment.
  • Set the value in a config file instead, where max_retries: 7 works normally.

The related trap is the prefix itself. env_prefixed("MYTOOL") without the trailing underscore leaves the separator on the front of every key and silently matches nothing. Register "MYTOOL_".

Which file changes does the watcher actually see?

All of the ordinary ones. The watcher follows the directory each user_file lives in, not the file itself, and reloads when an event in that directory names the file. So it sees:

  • In-place writes, the usual write(2) an application does.
  • Rename-replace saves, where a temporary file is written and renamed over the target. Editors that save atomically do this (vim's backupcopy setting controls the choice), and so does the temp-file-and-rename pattern Config::write tells you to implement yourself.
  • A file that does not exist yet. The directory has to exist when watch_files is called; the file can appear later.
  • A symlink swap. If the path is a symlink, the watcher also follows the directory of the file it points at, and re-resolves the link after every change in either directory. That is how a Kubernetes ConfigMap or Secret volume publishes an update: the kubelet writes a new directory and swaps a ..data symlink, so the file the path resolves to changes without the path itself being touched.

Up to v0.6.4 the watcher registered the file rather than its directory. A rename-replace save swapped the watched inode out from under it, and on Linux the first such save reloaded and nothing after it ever did, with no error and a live WatchHandle.

What has been checked, and where: every case above is a test in tests/watch.rs, run on Linux against the inotify backend. The ConfigMap case is tested against the same directory-and-symlink layout the kubelet builds, not against a real cluster. The macOS and Windows backends are not exercised by this crate's CI with the feature on, so treat them as unverified rather than known-good.

The watcher is still not a delivery guarantee. Events can be dropped when the OS queue overflows, and a reload that fails to parse is swallowed. A service that must not miss a change can still call Config::reload on a timer or a signal as well.

Can I find out which layer a value came from?

No. Once the layers are merged, a value is just a value. There is no provenance API, nothing that answers "did this port come from the file or from the environment?", and nothing that reports which keys an environment variable overrode.

This is the single biggest gap between rtb-config and the Go toolkit's config module, and it is the headline feature of the planned v2 store architecture. Today, if you need to show a user where a setting came from, you have to build the answer yourself by loading the layers separately.

Will writing config back preserve my file?

No. Config::write serialises the current in-memory value and replaces the target file wholesale. Three consequences:

  • Comments and key order are lost. A hand-written config file with explanatory comments comes back as machine-ordered YAML with none of them.
  • Every layer is flattened into the file. What gets written is the merged value, so embedded defaults become explicit keys, and any environment override active at that moment is baked in as if a human had typed it. A config set built naively on top of this turns a temporary MYTOOL_PORT=9999 into a permanent file entry.
  • The write is not atomic. A crash mid-write leaves a truncated file, and the next start fails to parse it.

Structure-preserving writes are the other half of the planned v2 work. Until then, a tool that edits a user's config file is better off editing that file directly than round-tripping it through Config::write.

Is there a CLI-flag layer?

No. The layering stops at environment variables. There is no provider for command-line arguments, and no merge entry point that would let you push a parsed clap struct in as the top layer. The source list is fixed at three kinds and is populated only through the builder.

Tools that want flags to win over the environment resolve that themselves after get(), or take the value from clap in preference to the config field at the point of use.

Are there remote or dynamic sources?

No. Files on the local filesystem, strings compiled into the binary, and the process environment. There is no support for Consul, etcd, S3, a secrets manager or an HTTP endpoint, and no plugin trait that would let you add one (Sources is a private struct with three fixed vectors).

A value fetched from a remote system has to be fetched by your code and supplied another way, for instance by setting an environment variable before build().

Does it validate anything beyond types?

Only what serde does. There is no range checking, no cross-field validation, no required-unless rules, and no schema enforcement on load. Config::schema generates a JSON Schema for export, but nothing in the crate validates against it.

Unknown keys are ignored by default, so a misspelled setting silently does nothing. #[serde(deny_unknown_fields)] on your struct turns that into an error, with the caveat noted in the API reference.

Does it do anything about secrets?

Almost nothing. Config fields are ordinary types, so a token in your config struct is a String in memory like any other, and it will appear in anything that serialises or debug-prints the struct, Config::write included.

The one protection is that Config's own Debug implementation never renders the stored value. It prints the file paths, environment prefixes and layer count instead. That stops tracing::debug!("{config:?}") leaking a secret, and nothing more.

Can I reload just one part of the configuration?

No. reload() re-reads every source and replaces the whole value. There is no per-key reload, no way to hold one section fixed, and no diff of what changed: a subscriber is woken and has to work out for itself whether the part it cares about moved.

Note also that a Config built with with_value or Default has no sources, so reloading one does not preserve the value it was given; it re-parses an empty source set. For a struct whose fields all default, that silently resets everything to Default.

What replaces these limits?

A v2 store architecture is approved for this repository. It adds value provenance and structure-preserving writes, which answer the two sections above on finding a value's layer and on preserving a file through Config::write. It is not implemented, no release carries it, and the figment-backed API documented here is the supported surface until it lands.

Next