[FIXED] Jenkins Declarative: Kubernetes Plugin with multiple agents

Issue

I am trying to setup a Jenkins declarative pipeline to use two different agents during its execution. The agents are dynamically spawned by the Kubernetes plugin. For sake of argument and simplicity, let's assume I want to do this:

On Agent 1 (Cloud name: "ubuntu"):

• Run apt-get and some installs
• Run a shell script
• Additional steps

On Agent 2 (Cloud name: "fedora"):

• Run dnf and some installs
• Run a shell script
• Additional steps

The problem I have is that if if I use a global agent declaration:

pipeline {
  agent {
    kubernetes {
      cloud 'ubuntu'
      label "ubuntu-agent"
      containerTemplate {
        name 'support'
        image 'blenderfox/support'
        ttyEnabled true
        command 'cat'
      }
    }
  }
...
}

Then that is used across all the stages if I don't declare an agent on each of the stages.

If I use agent none:

pipeline {
  agent none
  ...
}

Then I have to declare an agent spec for each stage, for example:

    stage ("apt update") {
      agent {
        kubernetes {
          cloud 'ubuntu'
          label "ubuntu-agent"
          containerTemplate {
            name 'support'
            image 'blenderfox/support'
            ttyEnabled true
            command 'cat'
          }
        }
      }
      steps {
        sh """
        apt update
        """
      }
    }

While this would work for me in that I can declare per stage which agent I want, the problem this method causes, is that it spins up a new agent for each stage, meaning the state isn't carried between, for example, these two stages:

    stage ("apt-update") {
      agent {
        ....
      }
      steps {
        sh """
        apt update
        """
      }
    }
    stage ("apt-install") {
      agent {
        ....
      }
      steps {
        sh """
        apt install -y ....
        """
      }
    }

Can I reuse the same agent across stages? For example, something like this:

stage ("provision agent") {
  agent {
    ...
    label "ubuntu-agent"
    ...
  }
  steps {
    sh """
    echo "Provisioning agent"
    """
  }
}
stage ("apt-update") {
  agent {
    label "ubuntu-agent" //reuse agent from previous stage
  }
  steps {
    sh """
    apt update
    """
  }
}
stage ("apt-install") {
  agent {
    label "ubuntu-agent" //reuse agent from previous stage
  }
  steps {
    sh """
    apt install -y ....
    """
  }
}

Solution

Found a solution. Very hacky but it works:

pipeline {

  agent none

  stages {
    stage ("Provision dev agent") {
      agent {
        kubernetes {
          cloud 'dev-cloud'
          label "dev-agent-${env.BUILD_NUMBER}"
          slaveConnectTimeout 300
          idleMinutes 5
          yamlFile "jenkins-dev-agent.yaml"
        }
      }
      steps {
        sh """
        ## Do any agent init steps here
        """
      }

    }
    stage ("Do something on dev agent") {
      agent {
        kubernetes {
          label "dev-agent-${env.BUILD_NUMBER}"
        }
      }
      steps {
        sh """
        ## Do something here
        """
      }
    }


    stage ("Provision production agent") {
      agent {
        kubernetes {
          cloud 'prod-cloud'
          label "prod-agent-${env.BUILD_NUMBER}"
          slaveConnectTimeout 300
          idleMinutes 5
          yamlFile "jenkins-prod-agent.yaml"
        }
      }
      steps {
        sh """
        ## Do any agent init steps here
        """
      }
    }
    stage ("Do something on prod agent") {
      agent {
        kubernetes {
          label "prod-agent-${env.BUILD_NUMBER}"
        }
      }
      steps {
        sh """
        ## Do something here
        """
      }
    }
  }
}

The agent yamls vary, but you can do something like this:

spec:
   containers:
   - name: docker
     image: docker:18.06.1
     command: ["tail", "-f", "/dev/null"]
     imagePullPolicy: Always
     volumeMounts:
       - name: docker
         mountPath: /var/run/docker.sock
   volumes:
   - hostPath:
      path: "/var/run/docker.sock"
     name: "docker"

And then use the agent like so:

stage ("docker build") {
  agent {
    kubernetes {
      label "dev-agent-${env.BUILD_NUMBER}"
    }
  }
  steps {
    container('docker') {
      sh """
        ## docker build....
      """
    }
  }
}

Answered By - Blender Fox
Answer Checked By - Mary Flores (JavaFixing Volunteer)