[FIXED] How to run one job on controller node and another on agent node

Issue

I have 2 Jenkins jobs configured. One is parameterized web hook enabled job which has instructions to invoke another job which is multi-branch pipeline job. So when first job is executed, it will build the branch and checkout the sources to the workspace then started running multi-branch pipeline job.

This setup works well in case of controller node where docker Jenkins is running and having Jenkins home mounted to the container i.e. once 1st job completes its execution for building the branch then starts executing the 2nd job on the controller node itself.

But the requirement is to run the 1st job on controller node and once build is successful then it should start executing pipeline job on agent node.

I am not sure on how this can be achieved, because of following challenges:

1. how the already built branch details to be passed/copied to the agent node(s)?
2. how to instruct 1st job to run the 2nd job on the specified agent node? Here, important point is: pipeline job can be run on multiple agent nodes at the same time based on different parameters provided to run it on the specified node.

I have tried using Node and Parameter plugin to achieve this, so able to specify node when executing job ( 1st job ) remotely. But in this case if I specify node as a parameter then it tries to run both the jobs on the same node, which is not desired.

For example : I have 2 jobs named : First_job ( parameterized job ) and Second_job ( multibranch pipeline job ). On the First_job configuration page : I have 4 different parameters : 3 is of String type and 1 is node type. Then repository configuration to checkout the source code which will build the branch. And then on Execute Shell section , we have instruction to invoke a script which will build the branch for the 2nd job:

#!/bin/bash

bash bin/abc.sh \
"http://usr:pwd@localhost:8080/" \
"Second_job" \
"${param1}" \
"${param2}" \
"${param3}"

So, when executing it from remotely: curl --insecure -d 'param1=xxx&param2=xxx&param3=xxx&nodes=slave1' -X POST https://master_node_url/job/First_job/buildWithParameters?token=xxx

It should run First_job on controller node and Second_job should run on agent node and in order to enable it to running I may require to copy/share built branch details from controller to agent. But at present, First_job invokes Second_job on the same node.

As I am new to Jenkins so not sure how these can be achieved.

Solution

Install the Jenkins Parameterized Trigger plugin, and invoke that step. I'd also simply convert the first job to a pipeline and restrict to the "built-in" node (which is a bad practice btw. Set up an agent to run co-located to the controller using a different account if necessry), then restrict the second job to run on agent label "slave1' (also it's called agent now).

You second job should be restricted to an agent:

agent {
  label 'Agent'
}

While your first pipeline job is restricted to 'builtin' or 'master'(legacy), and your first job can simply call the second with parameters:

build job: 'folder/SecondJob', 
        parameters: [string(name: 'Param1', value: 'Value1'), string(name: 'Param2', value: 'Value2')]

You can also simply look at combining your two jobs into one, with a sequential build build step, followed by parallel stages or even get really fancy.

Answered By - Ian W
Answer Checked By - Candace Johnson (JavaFixing Volunteer)