Managing Environment Variables Securely in Production

One of the first real challenges I faced as a DevOps engineer was figuring out how to properly manage environment variables in production. It sounds simple, but when you're dealing with sensitive data, multiple environments, and process managers like PM2, it gets complicated fast.

The Problem

We had a Node.js application running under PM2, and the team was storing sensitive configuration in various places:

• Some vars in .env files committed to git (bad)
• Others hardcoded in the application (worse)
• Database credentials passed as command line arguments (also bad)

This needed to be fixed, and it needed to be done right.

The Solution

Here's what I implemented:

1. Separate Configuration from Code

First rule: no secrets in the repository. Ever.

# ecosystem.config.js for PM2
module.exports = {
  apps: [{
    name: 'api',
    script: './dist/index.js',
    env_production: {
      NODE_ENV: 'production',
      // Reference to external config file
      CONFIG_PATH: '/opt/app/config/production.env'
    }
  }]
}

2. Secure File Permissions

Configuration files should only be readable by the application user:

# Set proper permissions
sudo chown app:app /opt/app/config/production.env
sudo chmod 600 /opt/app/config/production.env

3. Version Control for Config Structure

While we don't commit actual secrets, we do version control the structure:

# config/template.env - committed to git
DATABASE_URL=
API_KEY=
JWT_SECRET=
REDIS_URL=

# Actual values go in production.env - never committed

Key Lessons

1. Automate the boring stuff - Use scripts to validate config files exist and have required keys
2. Document everything - Future you will thank present you
3. Test your disaster recovery - What happens when the config file gets corrupted?

The implementation took a few iterations to get right, but now our environment variable management is secure, auditable, and maintainable.

Next up: I'll write about setting up proper monitoring for these configurations.