Strict early feature-flagged config parsing

Just like user input of a program, application config are values often produced by a human being that too can make a program break.

I think there's an obvious value in strictly parsing configuration values, but it seems it's not so obvious how strict you need to define them and what behavior we're striving for in doing so.

In my work it's common to introduce new features toggled by feature flags that are disabled by default. This allows integrating a piece of code before it's finished, and it allows shipping disabled features to production environmnents while it's still being tested in a QA environment. That way a feature can be thouroughly tested without hindering a regular release cadence.

It's also common practice to use a blue-green deployment strategy. This is a strategy to detect bad releases and automatically roll back to a known good version based on metrics collected from a running application, the simplest form being wether or not it responds to a ping request.

An unfortunate effect of shipping disabled features is that it can encourage weakening the parsing of application config. A common scenario is that a feature introduces integration to a third-party service. At the time the feature is being developed, we typically only have access to authentication credentials for sandboxed development environments. And so to allow applications to keep starting up in production without these credentials, the definition of a configuration parameter might be changed from something like this:

SERVICE_URL = os.environ["SERVICE_URL"]

Into something like this, that wouldn't break when the value is missing:

SERVICE_URL = os.environ.get("SERVICE_URL")

The reason this is problematic is because it won't yield an error until a code path is visited that requires this configuration parameter, which can be long after the code is deployed and usually won't cause a blue-green deployment strategy to detect the faulty configuration.

And the problem I'm getting at is really that this in isolation looks completely fine. It's easy to reason along the lines that the value will just be configured later when the feature is enabled in production. This, in my experience, just doesn't hold true. I've witnessed multiple times how deployments like this breaks when being enabled in production months after the feature was first built, sometimes even deployed by a different developer than the original author.

And still, if the application would have just fallen over at parsing that configuration parameter, we wouldn't have had a problem, as our automated blue-green deployments would have immediately discarded the new version. This is the dilemma: we want to strictly parse our configuration parameter, but only if we know it will be used by an enabled feature.

Introducing feature-bound configuration

These conclusions lead me to build a new library that provides simple tooling to describe when parameters should be parsed.

This is an example of a tiny settings module that uses the new library (the API is a work-in-progress and I'd like to get rid of having the reference in the field metadata):

from dataclasses import dataclass, field
from typing import Final
from feature import Feature, load_spec

class RemoteAPIConf(Feature):
    BASE_URL: str
    TIMEOUT: int = 5

class OtherFeatureConf(Feature):
    NUMBER: int

@dataclass
class Settings:
    REMOTE_API: RemoteAPIConf = field(
        metadata={"reference": RemoteAPIConf}
    )
    OTHER_FEATURE: OtherFeatureConf = field(
        metadata={"reference": OtherFeatureConf}
    )

settings: Final = load_spec(Settings)

Calling load_spec() will load the env var ENABLED_FEATURES and from that decide which of the defined feature configurations it should load and parse.

The library will also have support for defining dependencies between features, and loading parameters from other sources than environment variables.

That's it for now, in conclusion: strictly parse your app config, it will make your sure you're being saved by blue-green deployments when bad configurations go out.