What Is a DevOps Pipeline?

What is a DevOps pipeline: an automated sequence of stages that takes a code change from a developer’s commit to a running, monitored system without manual hand-offs. A typical pipeline has six stages, source, build, test, security scan, deploy and monitor, and each stage either passes the change to the next or stops it with a clear reason. The term overlaps with CI/CD pipeline; continuous integration covers the first four stages and continuous delivery or deployment covers the last two. This guide explains each stage, the tools used at each one, a starter setup for a small team, and the mistakes that make pipelines slow or distrusted.

The stages of a DevOps pipeline

1. Source

Everything starts in version control: application code, infrastructure definitions, pipeline definitions and configuration. A pull request is the trigger. Branch protection rules require a review and passing checks before anything merges, and short-lived branches keep merges small. The pipeline definition lives in the repository as YAML, so changes to the pipeline get the same review as changes to the code.

2. Build

The build stage compiles or bundles the code, restores dependencies and produces an artifact: a .NET publish folder, a container image, a zipped function package or a static site bundle. The artifact is versioned once and reused in every later stage, so what you tested is what you deploy. Dependency caching keeps this stage fast.

3. Test

Unit tests run on every pull request and must pass. Integration tests run against real dependencies in containers or an ephemeral environment. End-to-end and performance tests run on the main branch or nightly, because they are slower. A failing test blocks the merge; a flaky test gets fixed or deleted rather than retried until green.

4. Security scan

Four checks run in the pipeline: static analysis of the code (CodeQL, SonarQube), dependency scanning for known vulnerabilities (Dependabot, Snyk, GitHub Advanced Security), container image scanning (Trivy, Microsoft Defender for Containers) and infrastructure scanning (Checkov, tfsec). Secret detection runs on every commit. Each check has a severity threshold that blocks the merge, agreed with the team so the pipeline is strict without being ignored.

5. Deploy

Infrastructure changes apply first, through Terraform or Bicep with a plan reviewed in the pull request. The artifact then deploys to staging automatically, and to production after an approval gate or, for teams with the test coverage to support it, automatically. Deployment strategies such as slot swaps, blue-green and canary releases limit the blast radius, and rollback means redeploying the previous artifact, not fixing forward under pressure.

6. Monitor

Logs, metrics and traces flow from the running system into Application Insights, Azure Monitor, CloudWatch or Grafana, with alerts on error rate, latency and saturation. Deployment markers on the dashboards show which release changed what. What the monitoring finds goes back into the backlog, which is why the pipeline is drawn as a loop rather than a line.

Tools

Stage	Common tools	Notes
Source	GitHub, Azure Repos, GitLab	Branch protection, required reviews, CODEOWNERS
Build and orchestration	GitHub Actions, Azure Pipelines, GitLab CI, Jenkins	YAML in the repository; hosted or self-hosted runners
Test	xUnit, NUnit, Jest, Playwright, k6, Testcontainers	Unit on every pull request; slower suites on main or nightly
Security	CodeQL, SonarQube, Dependabot, Snyk, Trivy, Checkov	Severity thresholds that block; secret scanning on every commit
Artifacts	Azure Container Registry, Amazon ECR, GitHub Packages, Azure Artifacts	Immutable, versioned, scanned before push
Infrastructure	Terraform, Bicep, CloudFormation, Helm	Plan on pull request, apply on merge, remote state with locking
Deploy targets	App Service, Azure Container Apps, AKS, ECS, EKS, Lambda, Functions	Slots, blue-green or canary for low-risk rollouts
Monitor	Application Insights, Azure Monitor, CloudWatch, Prometheus, Grafana, OpenTelemetry	Alerts routed to people who can act; deployment markers

Tool choice matters less than the shape. A team on Azure DevOps and one on GitHub Actions end up with the same six stages; the YAML differs, the discipline does not.

Starter setup for a small team

A small team does not need Kubernetes, a service mesh or a platform team to get a working pipeline. The setup below covers a .NET or Node.js web application deploying to Azure App Service; the AWS equivalent (Elastic Beanstalk or ECS on Fargate) is the same shape.

1. Repository and branching. One repository with trunk-based development: short-lived feature branches, pull requests into main, branch protection requiring one review and the build check.
2. Build and test on every pull request. Restore, build, run unit tests, publish the artifact. Fail fast and keep it under ten minutes with dependency caching.
3. Scanning from day one. Dependabot for dependencies, CodeQL for static analysis, secret scanning on. Start with “high and critical block the merge” and tighten later.
4. Infrastructure in the same repository. A Bicep or Terraform folder defining the App Service plan, the web app, the database and Key Vault; plan on pull request, apply on merge.
5. Staging automatically, production with approval. Merges to main deploy to a staging slot; a named approver promotes to production with a slot swap, which also gives a one-click rollback.
6. No stored cloud keys. The pipeline authenticates to Azure or AWS with OpenID Connect federation, and application secrets come from Key Vault or Secrets Manager at deploy time.
7. Monitoring wired in. Application Insights or CloudWatch with alerts on error rate and latency, and a dashboard the team looks at after every deployment.

A GitHub Actions workflow for the first two steps looks like this; the deploy jobs add an environment with required reviewers and the login and deploy actions for your cloud.

name: build-and-test
on:
  pull_request:
  push:
    branches: [main]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-dotnet@v4
        with:
          dotnet-version: 8.0.x
      - run: dotnet restore
      - run: dotnet build --configuration Release --no-restore
      - run: dotnet test --configuration Release --no-build
      - run: dotnet publish src/Web --configuration Release --output publish
      - uses: actions/upload-artifact@v4
        with:
          name: web
          path: publish
  # deploy-staging: needs build, runs on main only, environment: staging
  # deploy-production: needs deploy-staging, environment: production (required reviewers)

Azure Pipelines expresses the same thing with stages, jobs and an environment that carries the approval. Either way the whole pipeline is a few dozen lines a new team member can read in an hour.

Common mistakes

• Building only on main. If the pull request does not run the build and tests, the review is guessing. Run them before merge, not after.
• Long-lived branches. A branch that lives for weeks produces a merge nobody understands. Small, frequent merges behind feature flags are safer than big ones.
• Hand-built environments. Staging configured by clicking in a portal drifts from production and cannot be rebuilt. Infrastructure as code is part of the pipeline, not a later project.
• Secrets in YAML or variables. Keys pasted into pipeline variables leak into logs and forks. Use federated identity for the pipeline and a vault for application secrets.
• Ignoring flaky tests. A pipeline that is re-run until it passes trains the team to ignore it. Quarantine the flaky test, fix it or delete it.
• Kubernetes too early. A cluster for three services adds upgrades, networking and on-call for no benefit a container service would not give. Our guide to monolith to microservices covers when the extra platform is justified.
• No rollback plan. If rolling back means a frantic hotfix, every deployment is a gamble. Keep artifacts, use slots or blue-green, and rehearse the rollback.
• Monitoring as an afterthought. A deployment that nobody watches is finished only in the pipeline’s opinion. Wire alerts and a dashboard into the first release.
• Nobody owns pipeline cost. Always-on self-hosted agents and ephemeral environments that are never deleted show up on the cloud bill; see our guide to cloud cost optimization.
• Automation without test design. A pipeline that runs a thin suite quickly proves little. Our guide on QA outsourcing and test automation covers building the suite the pipeline relies on.

Where PS3G fits

PS3G builds pipelines like the starter setup above and the larger versions that follow it. Our DevOps consulting covers the assessment of your current path to production, the pipeline and infrastructure-as-code setup, container platforms where they are justified, monitoring, and the security gates, on Azure DevOps, GitHub Actions or GitLab. For teams that do not want to staff a platform role, DevOps as a service means we run the pipelines, runners and platform under defined SLAs. The work is scoped for organizations of every size, from a founder with one application to a department that needs a platform built to its security standards.

Related reading

• Monolith to Microservices: When It’s Worth It and How to Do It
• Cloud Cost Optimization: 20 Ways to Cut Azure and AWS Bills
• QA Outsourcing and Test Automation: Get Value, Not Reports
• DevOps consulting and DevOps as a service from PS3G

Frequently asked questions

What is the difference between a DevOps pipeline and a CI/CD pipeline?

In everyday use they mean the same thing. Strictly, continuous integration covers source, build, test and scan, and continuous delivery or deployment covers the release to environments. A DevOps pipeline is the whole loop including monitoring and the feedback into the next change. The useful question is not the name but whether every stage runs automatically with a clear pass or fail.

What are the stages of a DevOps pipeline?

Source (version control and pull requests), build (compile and produce a versioned artifact), test (unit, integration, end-to-end), security scan (code, dependencies, containers, infrastructure, secrets), deploy (infrastructure first, then staging and production with gates) and monitor (logs, metrics, traces and alerts). Some teams split out an artifact stage and a release stage, but the shape is the same.

Which tool should a small team use, GitHub Actions or Azure DevOps?

Use the one your code already lives in. GitHub Actions is the natural choice for repositories on GitHub and has the tighter integration with Dependabot and code scanning. Azure DevOps suits teams that want boards, repositories, pipelines, artifact feeds and test plans in one product. Both support YAML pipelines, environments with approvals and federated identity to the cloud.

Do we need Kubernetes for a DevOps pipeline?

No. The pipeline is about automation and gates, not about the runtime. A web application on App Service, Container Apps, Elastic Beanstalk or ECS on Fargate gets the full pipeline with far less platform to operate. Kubernetes becomes worthwhile when you run many services with different scaling and isolation needs and have the people to run the cluster.

How long should a pipeline take?

Short enough that developers wait for it rather than context-switch: a pull-request build with unit tests in a few minutes, a deployment to staging within the same quarter hour. Dependency caching, parallel jobs and moving slow suites to the main branch or a nightly run keep it there. A pipeline that takes an hour gets bypassed, and bypassing it defeats the point.