Environment variables: what they are and how to stop getting them wrong

Environment variables should be the easy part. You set a value, your app reads it, done. And yet I have lost entire afternoons to env var problems that turned out to be something embarrassingly simple. If you have ever stared at a "connection refused" error that made no sense, this one is for you.

The time I almost pushed secrets to GitHub

Everyone tells you to add .env to .gitignore right away. I knew that. What I did not know was that git add . does not care about your intentions. I had created the .env file before setting up .gitignore, and git was happily tracking it. I only caught it because I happened to run git status before pushing. The fix was git rm --cached .env, but the scare was real.

The worst version of this story is when you catch it after the push. Even if you delete the file in the next commit, the secret is still in your git history. Anyone who clones the repo can find it. At that point your only real option is to rotate the credential immediately. Tools like git-filter-repo can scrub the history, but you should assume the secret is compromised the moment it hits a remote.

Build-time vs runtime: the silent trap

This one bit me with a Next.js project deployed on Vercel. I had set API_URL in the Vercel dashboard, and my server-side code could read it just fine. But I was also referencing it in a helper function that ran during the build step. The build happened in a CI environment where the variable was not set yet, so the value got baked in as undefined. The app deployed, the pages rendered, and every API call went to undefined/api/users.

The lesson: variables that get inlined at build time are frozen at build time. If your framework does any static generation or bundling that evaluates process.env.SOMETHING, the value needs to exist when the build runs, not just when the server starts. This is different from a truly runtime variable that gets read on each request. Mixing up when a variable gets resolved is one of the most common env var bugs I see, and the error messages never point you in the right direction.

The Docker precedence nightmare

Docker has at least four places you can set environment variables, and they all have different priorities. I learned this the hard way when a containerized app kept connecting to the wrong database. I had updated the URL in my .env file, but the old value was still set in docker-compose.yml under the environment key. Compose was ignoring my .env change entirely because explicit values in the compose file take precedence.

Here is the rough precedence order, from highest to lowest:

1. Values passed on the command line (docker run -e VAR=value)
2. Values set in the environment section of docker-compose.yml
3. Values from env_file in docker-compose.yml
4. Values baked into the image via ENV in the Dockerfile

If you set the same variable in multiple places, the highest-priority one wins silently. No warning, no log message. You just get a value you did not expect. I now make it a habit to run docker compose config before deploying, which prints the fully resolved compose file with all variables substituted. It has saved me more than once.

When dev config leaks into production

This is the scariest one. I had a staging environment that pointed to a real (but non-production) database. At some point during a deploy, the staging env vars got copied to production because I was using a shared config template and forgot to override one variable. Production traffic hit the staging database for about twenty minutes before monitoring caught it.

The fix was simple in hindsight: validate environment variables at startup. Not just "does this variable exist" but "does this value make sense for this environment." A production app should probably not be connecting to a host called staging-db.internal. I started adding a small validation step that checks the current NODE_ENV against patterns in critical variables. It is not foolproof, but it catches the obvious mistakes.

What actually helped

After enough of these incidents, I settled on a few habits that reduced the pain:

• Check git status before every commit. Not just for env files, but it catches them too.
• Use docker compose config to verify what Docker actually sees after variable substitution.
• Validate at startup. A five-line check that throws on missing or suspicious variables is worth more than any amount of documentation.
• Keep one source of truth. If the value is in docker-compose.yml, do not also put it in an env_file. Pick one place per variable.
• Treat every leaked secret as compromised. Do not try to scrub history and hope nobody noticed. Rotate the credential.

None of this is complicated. The problem with environment variables is not that they are hard to understand. It is that they fail quietly, and the debugging feedback loop is terrible. You change a value, redeploy, and wait to see if the error goes away. The best defense is making mistakes loud and catching them before they reach production.

Sources

• Docker Compose environment variables precedence
• The Twelve-Factor App: Config
• git-filter-repo
• Next.js environment variables documentation