Workflow Environment Variables Guide

The GeniSpace Workflow Engine provides a powerful environment variable substitution feature that allows you to use the same configuration templates across different environments by dynamically replacing configuration values with environment variables. Environment variables enable you to reuse configurations across different workflow executions, avoiding the need to repeatedly set the same values in every node.

Overview

The environment variable substitution mechanism operates in two stages:

API-Side Substitution: During the task preparation stage, substitution is applied to operator URL configurations
Worker-Side Substitution: During the task execution stage, comprehensive substitution is applied to node inputs, configurations, and more

Environment variables can be centrally managed and used across multiple nodes in a workflow, supporting the use of the same configuration templates in different environments (development, testing, production).

Environment Variable Sources

Environment variables can be configured through the following three methods. The system automatically merges them according to priority:

1. Task-Level Environment Variables (Highest Priority)

Environment variables defined directly in the task configuration, applicable to all executions of a specific task. This method is suitable for task-specific configurations or temporary overrides.

{
  "envVars": [
    {
      "key": "DB_HOST",
      "value": "localhost",
      "isSecret": false,
      "description": "Database host address"
    },
    {
      "key": "DB_PASSWORD",
      "value": "secret123",
      "isSecret": true,
      "description": "Database password"
    }
  ]
}

2. ConfigMap Bindings (Medium Priority)

A task can bind to one or more ConfigMaps, and the variables within them are loaded as environment variables. This method is suitable for shared configurations and environment-specific configuration management.

{
  "executionConfig": {
    "configMaps": [
      {
        "id": "config-map-id",
        "name": "production-config"
      }
    ]
  }
}

tip

To learn how to create and manage ConfigMaps, refer to the ConfigMap Guide.

3. Default ConfigMap (Lowest Priority)

A team-level default configuration map that all tasks can access automatically. The system automatically loads ConfigMaps marked as default. This method is suitable for global common configurations.

tip

To learn how to set a default ConfigMap, refer to the ConfigMap Guide.

Priority Rules

Task-Level Environment Variables > ConfigMap Bindings > Default ConfigMap

When the same variable is defined in multiple sources, the higher-priority source overrides the lower-priority one. This allows you to flexibly manage configurations at different levels:

Global Configuration: Place in the default ConfigMap
Environment Configuration: Place in task-bound ConfigMaps
Task-Specific Configuration: Define directly in the task

📖 Learn More: For detailed information about configuration access rules, refer to ConfigMap Configuration Access Rules.

Two Stages of Environment Variable Substitution

Stage One: API-Side Substitution (Task Preparation Stage)

Before task execution, the API side performs environment variable substitution, supporting the use of environment variables in URL and headers configurations.

Substitution Scope

✅ Operator Node URL Configuration
- serverUrl and endpoint configuration (when building the full URL)
- Supports using environment variables at any position in the URL
✅ Operator Node Headers Configuration
- The value field of each header in the headers array
- Supports using environment variable templates in header values
❌ Not Yet Supported
- Other configuration items (such as timeout, retryPolicy, etc.)
- These configuration items need to use template syntax on the Worker side

Substitution Format

Use the {{variableName}} template syntax:

URL Configuration Example:

// In Operator configuration
{
  "serverUrl": "https://{{API_HOST}}",
  "endpoint": "/api/v1/{{RESOURCE}}"
}

// If environment variables are:
// API_HOST = "api.example.com"
// RESOURCE = "users"

// The final constructed URL is:
// "https://api.example.com/api/v1/users"

Headers Configuration Example:

// In Operator configuration
{
  "headers": [
    {
      "key": "Authorization",
      "value": "Bearer {{API_TOKEN}}"
    },
    {
      "key": "X-API-Key",
      "value": "{{API_KEY}}"
    },
    {
      "key": "Content-Type",
      "value": "application/json"
    }
  ]
}

// If environment variables are:
// API_TOKEN = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."
// API_KEY = "your-secret-api-key"

// The final substituted headers are:
// [
//   { "key": "Authorization", "value": "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..." },
//   { "key": "X-API-Key", "value": "your-secret-api-key" },
//   { "key": "Content-Type", "value": "application/json" }
// ]

Substitution Logic

The API side uses the buildUrl function to construct the full URL and substitutes environment variables during the process:

// Supported URL formats:
// 1. serverUrl + endpoint
"https://{{API_HOST}}" + "/api/{{VERSION}}/users"
// Result: "https://api.example.com/api/v1/users"

// 2. Full serverUrl (endpoint is empty)
"https://{{API_HOST}}/api/v1/users"
// Result: "https://api.example.com/api/v1/users"

// 3. Endpoint contains the full URL
"{{API_BASE_URL}}/users"
// Result: "https://api.example.com/users"

Handling Missing Variables

If an environment variable is not found, the original template format ({{variableName}}) is preserved, making it easier to debug and troubleshoot.

Stage Two: Worker-Side Substitution (Task Execution Stage)

When the Worker executes the workflow, it performs environment variable substitution on node inputs and configurations.

Substitution Scope

✅ Node Input Parameters (defaultInputs)
✅ Node Configuration Values
✅ Data Path References
✅ context.env Access in JavaScript Code

Substitution Format

The Worker side supports three ways to use environment variables:

1. Template Syntax: `{{variableName}}`

Use template syntax in node inputs and configurations:

{
  "defaultInputs": {
    "host": "{{DB_HOST}}",
    "port": "{{DB_PORT}}",
    "connectionString": "mysql://{{DB_USER}}:{{DB_PASSWORD}}@{{DB_HOST}}:{{DB_PORT}}/mydb"
  }
}

2. Data Path: `env.variableName`

Use data path references to access environment variables in edge configurations:

{
  "source": "nodeA:output",
  "target": "nodeB:host",
  "dataPath": "env.DB_HOST"
}

3. JavaScript: `context.env.variableName`

Access environment variables in transform nodes or other JavaScript-enabled nodes:

// In transform nodes or other JavaScript-enabled nodes
const apiUrl = context.env.API_BASE_URL;
const apiKey = context.env.API_KEY;

return {
  url: `${apiUrl}/endpoint`,
  headers: {
    'Authorization': `Bearer ${apiKey}`
  }
};

Recursive Processing of Environment Variable Substitution

Worker-side environment variable substitution supports recursive processing of complex data structures:

Strings: Substitute {{variableName}} templates within them
Arrays: Recursively process each element in the array
Objects: Recursively process all property values in the object

Example

{
  "config": {
    "api": {
      "baseUrl": "https://{{API_HOST}}",
      "endpoints": [
        "{{API_HOST}}/users",
        "{{API_HOST}}/products"
      ],
      "headers": {
        "Authorization": "Bearer {{API_TOKEN}}",
        "X-API-Key": "{{API_KEY}}"
      },
      "timeout": "{{REQUEST_TIMEOUT}}"
    }
  }
}

All {{variableName}} occurrences in nested string values are recursively substituted.

Handling Missing Environment Variables

API Side

If an environment variable is not found, the original template format ({{variableName}}) is preserved for debugging purposes. The system logs a warning but does not prevent task execution.

Worker Side

Template Syntax: Preserves the original template format ({{variableName}})
Data Path: Returns undefined
JavaScript: Returns undefined

It is recommended to check whether a variable exists before use:

// Safe access pattern
const apiUrl = context.env.API_BASE_URL || 'https://default-api.com';
const apiKey = context.env.API_KEY;
if (!apiKey) {
  throw new Error('API_KEY environment variable is not set');
}

Usage Scenario Examples

Scenario 1: Database Connection Configuration

{
  "envVars": [
    { "key": "DB_HOST", "value": "db.example.com" },
    { "key": "DB_PORT", "value": "5432" },
    { "key": "DB_NAME", "value": "mydb" },
    { "key": "DB_USER", "value": "admin" },
    { "key": "DB_PASSWORD", "value": "secret", "isSecret": true }
  ]
}

Used in a node:

{
  "defaultInputs": {
    "connectionString": "postgresql://{{DB_USER}}:{{DB_PASSWORD}}@{{DB_HOST}}:{{DB_PORT}}/{{DB_NAME}}"
  }
}

Scenario 2: API Service Configuration

Using in Operator Configuration (API-Side Substitution)

{
  "envVars": [
    { "key": "API_HOST", "value": "api.example.com" },
    { "key": "API_TOKEN", "value": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...", "isSecret": true },
    { "key": "API_KEY", "value": "your-api-key", "isSecret": true }
  ]
}

Used in Operator configuration:

{
  "configuration": {
    "values": {
      "serverUrl": "https://{{API_HOST}}",
      "headers": [
        {
          "key": "Authorization",
          "value": "Bearer {{API_TOKEN}}"
        },
        {
          "key": "X-API-Key",
          "value": "{{API_KEY}}"
        }
      ]
    }
  }
}

Using in JavaScript Code (Worker-Side Substitution)

{
  "envVars": [
    { "key": "API_BASE_URL", "value": "https://api.example.com" },
    { "key": "API_VERSION", "value": "v1" },
    { "key": "API_KEY", "value": "your-api-key", "isSecret": true }
  ]
}

async function callAPI(endpoint) {
  const baseUrl = context.env.API_BASE_URL;
  const version = context.env.API_VERSION;
  const apiKey = context.env.API_KEY;
  
  const url = `${baseUrl}/${version}/${endpoint}`;
  const response = await fetch(url, {
    headers: {
      'Authorization': `Bearer ${apiKey}`
    }
  });
  
  return response.json();
}

Scenario 3: Multi-Environment Configuration

Use ConfigMaps to manage configurations for different environments:

Development Environment ConfigMap:

name: dev-config
variables:
  API_HOST: "dev-api.example.com"
  DB_HOST: "dev-db.example.com"
  LOG_LEVEL: "debug"

Production Environment ConfigMap:

name: prod-config
variables:
  API_HOST: "api.example.com"
  DB_HOST: "prod-db.example.com"
  LOG_LEVEL: "info"

Bind the corresponding ConfigMap to a task to use the same configuration template across different environments.

Best Practices

1. Use ConfigMaps to Manage Environment Configurations

Common Configuration: Place in the default ConfigMap for use by all tasks
Environment-Specific Configuration: Place in task-bound ConfigMaps
Task-Specific Configuration: Define environment variables directly in the task

📖 Learn More: For information on creating, managing, and best practices for ConfigMaps, refer to the ConfigMap Guide.

2. Naming Conventions

Use uppercase letters and underscores: DB_HOST, API_KEY
Use meaningful names with prefixes:
- DB_ prefix: Database-related configurations
- API_ prefix: API service-related configurations
- REDIS_ prefix: Redis-related configurations
- LOG_ prefix: Logging-related configurations

3. Handling Sensitive Information

Mark sensitive information as isSecret: true
Sensitive information is automatically hidden in logs
Rotate sensitive credentials regularly
Consider using a dedicated secrets management service

4. Environment Variable Priority

Task-Level Environment Variables: Use for overriding in specific scenarios or temporary debugging
ConfigMaps: Use for managing shared configurations and environment-specific settings

5. Current Limitations and Notes

API-Side Support: Supports environment variable substitution in URLs and headers
Other Configuration Items: Items such as timeout, retryPolicy, etc. need to use template syntax on the Worker side
Missing Variables: It is recommended to check whether variables exist before use to avoid runtime errors
Type Conversion: Environment variable values are always strings; convert types as needed
Headers Format: Headers must be in array format, with each element containing key and value fields

6. Debugging Tips

Use meaningful variable names and descriptions
Test with non-sensitive values in the development environment
Check environment variable substitution records in the logs
Use default values to handle undefined variables

ConfigMap Guide - Learn how to create and manage ConfigMaps
Workflow Overview - Workflow engine overview
Workflow Builder - Workflow designer usage guide

Overview​

Environment Variable Sources​

1. Task-Level Environment Variables (Highest Priority)​

2. ConfigMap Bindings (Medium Priority)​

3. Default ConfigMap (Lowest Priority)​

Priority Rules​

Two Stages of Environment Variable Substitution​

Stage One: API-Side Substitution (Task Preparation Stage)​

Substitution Scope​

Substitution Format​

Substitution Logic​

Handling Missing Variables​

Stage Two: Worker-Side Substitution (Task Execution Stage)​

Substitution Scope​

Substitution Format​

1. Template Syntax: {{variableName}}​

2. Data Path: env.variableName​

3. JavaScript: context.env.variableName​

Recursive Processing of Environment Variable Substitution​

Example​

Handling Missing Environment Variables​

API Side​

Worker Side​

Usage Scenario Examples​

Scenario 1: Database Connection Configuration​

Scenario 2: API Service Configuration​

Using in Operator Configuration (API-Side Substitution)​

Using in JavaScript Code (Worker-Side Substitution)​

Scenario 3: Multi-Environment Configuration​

Best Practices​

1. Use ConfigMaps to Manage Environment Configurations​

2. Naming Conventions​

3. Handling Sensitive Information​

4. Environment Variable Priority​

5. Current Limitations and Notes​

6. Debugging Tips​

Related Documentation​

Overview

Environment Variable Sources

1. Task-Level Environment Variables (Highest Priority)

2. ConfigMap Bindings (Medium Priority)

3. Default ConfigMap (Lowest Priority)

Priority Rules

Two Stages of Environment Variable Substitution

Stage One: API-Side Substitution (Task Preparation Stage)

Substitution Scope

Substitution Format

Substitution Logic

Handling Missing Variables

Stage Two: Worker-Side Substitution (Task Execution Stage)

Substitution Scope

Substitution Format

1. Template Syntax: `{{variableName}}`

2. Data Path: `env.variableName`

3. JavaScript: `context.env.variableName`

Recursive Processing of Environment Variable Substitution

Example

Handling Missing Environment Variables

API Side

Worker Side

Usage Scenario Examples

Scenario 1: Database Connection Configuration

Scenario 2: API Service Configuration

Using in Operator Configuration (API-Side Substitution)

Using in JavaScript Code (Worker-Side Substitution)

Scenario 3: Multi-Environment Configuration

Best Practices

1. Use ConfigMaps to Manage Environment Configurations

2. Naming Conventions

3. Handling Sensitive Information

4. Environment Variable Priority

5. Current Limitations and Notes

6. Debugging Tips

Related Documentation