Understanding the Pipeline Flow

A pipeline is just a list of instructions that run automatically. Once you understand how it's structured — triggers, jobs, steps, artifacts — you can read any pipeline file in any CI tool.

The anatomy of a pipeline

A pipeline is a set of automated tasks triggered by an event — usually a code push. It's not magic; it's just a structured script running on a temporary server somewhere in the cloud.

Every pipeline has the same skeleton: something triggers it, jobs run (possibly in parallel), and each job contains sequential steps. Think of it like a recipe: the recipe is the workflow, a cooking stage is a job, and each individual instruction ("chop onions", "add salt") is a step.

Pipeline anatomy: Workflow → Jobs → Steps

Let's zoom in on the structure. A workflow is the entire pipeline definition — it's usually a YAML file. Inside a workflow, you have jobs. Each job runs on its own fresh server. Inside each job, you have steps, which are individual commands that run one after another.

Jobs vs steps — what's the difference?

Steps run sequentially inside a single job on a single server. If step 2 fails, step 3 never runs. They share the same filesystem — so a file written in step 2 is still there in step 3.

Jobs can run in parallel (or in a defined order using needs). Each job gets a brand-new, empty server. A file written in one job is not automatically available in the next job. That's exactly why artifacts exist.

The pipeline lifecycle

When you push code, your pipeline doesn't instantly start running. It moves through these states:

• Queued — the system received the trigger and is waiting for a free runner (the temporary server that actually runs your jobs).
• Running — a runner has been assigned and your jobs are executing.
• Passed (green) — all jobs completed successfully. Code is eligible for deployment.
• Failed (red) — at least one job or step returned an error. The pipeline stops and notifies you.
• Cancelled — someone manually cancelled it, or a newer push cancelled an older in-progress run.

Artifacts — passing files between jobs

Here's a real problem: you build your app in one job (producing a dist/ folder), then you want to deploy that exact built output in a second job. But jobs run on separate servers, so the dist/ folder doesn't exist on the deploy server.

The solution is artifacts. An artifact is a file or folder you explicitly save at the end of one job and download at the start of the next. Think of it like dropping a package in a shared locker — the build job puts the package in, and the deploy job picks it up.

Environment variables

An environment variable is a named value passed to your pipeline at runtime, outside of the code. Instead of writing const dbUrl = "postgres://prod-server/mydb" in your code (which would expose it in your git history), you store it as an environment variable and reference it like process.env.DATABASE_URL.

In a pipeline, environment variables come from two places:

• Workflow-level env — values you define directly in the YAML file, safe for non-sensitive configuration like NODE_ENV=production or a port number.
• Secrets — sensitive values (API keys, passwords, tokens) stored encrypted in your CI platform and injected at runtime. You'll learn how to set these up in Section 3.

.github/workflows/ci.yml (snippet)

env:
  NODE_ENV: production          # safe to put in the file
  PORT: 3000                    # safe to put in the file
  # DATABASE_URL comes from a secret — NEVER hardcode it here

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Run tests
        run: npm test
        env:
          DATABASE_URL: ${{ secrets.DATABASE_URL }}   # injected securely