Weights & Biases (W&B) environment variables are the primary way you configure its behavior, especially within automated systems like CI/CD pipelines.

Here’s a W&B experiment running in a GitHub Actions CI/CD pipeline, logging metrics and artifacts to W&B:

name: Train and Log Model

on: [push]

jobs:
  train:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - name: Set up Python
      uses: actions/setup-python@v3
      with:
        python-version: '3.9'
    - name: Install dependencies
      run: pip install -r requirements.txt
    - name: Train model and log to W&B
      env:

        WANDB_API_KEY: ${{ secrets.WANDB_API_KEY }}

        WANDB_PROJECT: "my-ml-project"
        WANDB_RUN_GROUP: "ci-cd-runs"
      run: python train.py

The train.py script would contain something like:

import wandb
import os

# W&B will automatically pick up environment variables
# like WANDB_API_KEY, WANDB_PROJECT, WANDB_RUN_GROUP
wandb.init()

# Your training code here...
# For demonstration, we'll just log a dummy metric and save a dummy file
loss = 0.5
accuracy = 0.8
print(f"Training finished with loss: {loss}, accuracy: {accuracy}")

# Log metrics
wandb.log({"loss": loss, "accuracy": accuracy})

# Log an artifact (e.g., a trained model file)
with open("model.pth", "w") as f:
    f.write("dummy model weights")
wandb.log_artifact("model.pth", name="trained_model", type="model")

wandb.finish()

This setup allows you to train models, log results, and manage artifacts without manual intervention, ensuring reproducibility and traceability in your ML workflows.

The core problem W&B environment variables solve is decoupling configuration from code, especially crucial in dynamic CI/CD environments. Instead of hardcoding API keys, project names, or entity details into your training scripts, you inject them as environment variables. This makes your code more portable, secure (by not embedding secrets), and adaptable to different deployment stages (e.g., development vs. staging vs. production). W&B’s client libraries are designed to automatically detect and utilize these variables, simplifying integration.

Internally, when wandb.init() is called, the W&B Python library (or other language clients) scans the execution environment for specific environment variables. It prioritizes these variables over configuration files or arguments passed directly to wandb.init().

  • WANDB_API_KEY: This is the authentication token that links your local run to your W&B account. Without it, W&B cannot log data.
  • WANDB_PROJECT: Specifies the W&B project under which this run will be logged. If not set, W&B defaults to a project named "uncategorized".
  • WANDB_ENTITY: Your W&B username or team name. If you belong to multiple teams or have a specific entity you want to log to, this is essential. If not set, it defaults to the entity associated with the WANDB_API_KEY.
  • WANDB_RUN_GROUP: Allows you to group related runs together. This is incredibly useful for comparing experiments from a single CI/CD job or a specific hyperparameter sweep.
  • WANDB_NAME: The display name for the specific W&B run. By default, W&B generates a random name. Setting this allows for more descriptive run titles.
  • WANDB_MODE: Can be set to dryrun (no data is logged to W&B servers, useful for testing), offline (logs locally to a wandb/ directory, to be synced later), or online (the default, logs to W&B servers).
  • WANDB_NOTES: A longer description for the run.
  • WANDB_TAGS: A comma-separated list of tags to apply to the run.

Consider a scenario where your CI/CD pipeline needs to push models to a specific W&B project and entity, but also allow for local testing with a different project. You can set WANDB_PROJECT and WANDB_ENTITY in your CI/CD secrets, and then override them locally by setting WANDB_PROJECT=my-local-test-project in your terminal before running your script. The CI/CD environment will pick up the secrets, while your local environment will use the explicitly set local variable.

The WANDB_DISABLED environment variable is a powerful, often overlooked, way to completely disable W&B logging without altering your code. Setting WANDB_DISABLED=true will cause all W&B client calls (like wandb.init(), wandb.log(), wandb.finish()) to become no-ops. This is incredibly useful for debugging or when you want to run your training script in an environment where W&B logging is not desired or possible, without having to add conditional logic around every W&B call in your codebase. It acts as a global kill switch for W&B integration.

Once you have environment variables configured, the next logical step is to manage more complex configurations, such as setting up custom sweeps or integrating with distributed training frameworks that require specific W&B signaling.

Want structured learning?

Take the full Wandb course →