Jenkins: A Declarative Pipeline

Jenkins is still the most common self-hosted automation server. Its strength is flexibility; its weakness is that a hand-clicked job configuration is unreviewable and unreproducible. Pipeline as code, a Jenkinsfile in the repository, fixes that: the build is versioned with the code and reviewed in pull requests.

Versions in use on this date. Jenkins 2.332 LTS (the 2.346 LTS line is due later this month) running on Java 11; Java 17 support is arriving in the next LTS. Always take plugin and core updates from the LTS stream and read the security advisories, because Jenkins and its plugins are a frequent target.

Architecture

flowchart TB
  DEV[Developer push] --> SCM[Git repository with Jenkinsfile]
  SCM -->|webhook| CTRL
  subgraph CTRL[Jenkins controller - schedules, no builds]
    Q[Build queue] --> PL[Pipeline interpreter]
  end
  PL -->|label docker| A1[Agent 1 ephemeral container or VM]
  PL -->|label docker| A2[Agent 2]
  A1 --> BUILD[Build and test in container]
  A2 --> SCAN[Scan and package]
  PL --> CRED[(Credentials store<br/>secrets masked in logs)]
  A1 --> REG[Registry or artifact store]
  PL --> NOTIFY[Notify Slack email]

Run builds on agents, never on the controller, set the controller’s executors to 0, and prefer ephemeral agents (Docker or Kubernetes) so every build starts clean.

A realistic Jenkinsfile

pipeline {
  agent none
  options {
    timestamps()
    timeout(time: 30, unit: 'MINUTES')
    buildDiscarder(logRotator(numToKeepStr: '20'))
    disableConcurrentBuilds()
  }
  environment {
    IMAGE = "registry.example.com/shop/api"
  }
  stages {
    stage('Test') {
      agent { docker { image 'node:16-alpine'; args '-u root:root' } }
      steps {
        sh 'npm ci'
        sh 'npm run lint'
        sh 'npm test -- --ci'
      }
      post { always { junit 'reports/junit.xml' } }
    }
    stage('Scan and build') {
      parallel {
        stage('Dependency audit') {
          agent { docker { image 'node:16-alpine' } }
          steps { sh 'npm audit --audit-level=high' }
        }
        stage('Image') {
          agent { label 'docker' }
          steps {
            sh 'docker build -t $IMAGE:$GIT_COMMIT .'
          }
        }
      }
    }
    stage('Push') {
      when { branch 'main' }
      agent { label 'docker' }
      steps {
        withCredentials([usernamePassword(credentialsId: 'registry-creds', usernameVariable: 'U', passwordVariable: 'P')]) {
          sh 'echo "$P" | docker login registry.example.com -u "$U" --password-stdin'
          sh 'docker push $IMAGE:$GIT_COMMIT'
        }
      }
    }
    stage('Deploy to production') {
      when { branch 'main' }
      steps {
        input message: 'Deploy to production?', ok: 'Deploy', submitter: 'release-managers'
        build job: 'deploy-prod', parameters: [string(name: 'TAG', value: env.GIT_COMMIT)]
      }
    }
  }
  post {
    failure { slackSend channel: '#ci', message: "FAILED ${env.JOB_NAME} #${env.BUILD_NUMBER} ${env.BUILD_URL}" }
    cleanup { cleanWs() }
  }
}

Reading it:

• agent none at the top, with an agent per stage, so waiting for approval does not hold an executor.
• options: timeouts avoid stuck builds; log rotation protects the controller’s disk; no concurrent builds of one branch.
• parallel runs independent checks together.
• when { branch 'main' } limits push and deploy to the main branch (in a Multibranch Pipeline).
• withCredentials injects secrets as masked variables only inside the block. Never echo them, and prefer --password-stdin.
• post actions handle notifications and workspace cleanup.
• input requires an authorised person to continue.

Use Replay to test Jenkinsfile edits quickly, and the Pipeline Syntax snippet generator for step options.

Make it maintainable

1. Multibranch Pipeline or the GitHub/GitLab organisation folder job: Jenkins discovers branches and pull requests automatically.
2. Shared libraries hold common steps (deployService(...)) so ten repositories do not copy-paste the same logic.
3. Configuration as code (JCasC plugin) describes Jenkins itself (security, credentials references, agents) in YAML in Git, so the controller can be rebuilt in minutes.
4. Back up JENKINS_HOME, and keep the plugin list pinned (a plugins.txt for the installer) and updated regularly.

Security essentials

• Authenticate with SSO or LDAP; use matrix or role-based authorisation; disable anonymous read.
• Keep the controller off the internet, or behind a reverse proxy with TLS.
• Use credentials scopes and folders to limit which jobs may use which secrets.
• Do not run untrusted pull-request code with access to secrets; build forks in isolated agents.
• Keep core, plugins and Java patched; review plugin advisories monthly.

Troubleshooting

• docker: not found inside a container stage: use an image that has the tool, or a labelled agent with Docker installed.
• Workspace permission errors with docker agents: user ID mismatch; set args '-u $(id -u):$(id -g)' or use root deliberately in throwaway containers.
• Pipeline stuck: executors busy, agent offline (Manage Jenkins, Nodes), or an input waiting.
• Slow builds: cache dependencies in a persistent volume, and run tests in parallel stages.