[FIXED] Executing python script using Jenkins parameterized pipeline

Issue

I have set up a Jenkins parameterized job to execute python script using execute shell feature in Jenkin. The job has the following parameters:

user-name: string, order_area_name: comma-separated strings, country_name: string, country_code: string, and so on...

My use case is to split the order_area_name and execute the python script for every order_area_name sequentially. So, I wrote a script that looks something like this:

#!/bin/bash
export PYTHONHASHSEED=0
empty_string=""
parameters_list=""
IFS=","

#Checking every parameter if it is present or not
if [ "$user_name" != "$empty_string" ]
then
    parameters_list=$parameters_list" --user "$user_name
fi
if [ "$country_code" != "$empty_string" ]
then
    parameters_list=$parameters_list" --country_code "$country_code
fi
if [ "$country_category" != "$empty_string" ]
then
    parameters_list=$parameters_list" --country_category "$country_category
fi

parameters_list=$parameters_list" --aws_access_key_id "$aws_access_key_id
parameters_list=$parameters_list" --aws_secret_access_key "$aws_secret_access_key


##Checking if the parameter is present then splitting the string and storing it into array
##Then for each order_area_name executing the python script in sequential manner
if [ "$order_area_names" != "$empty_string" ]
then
    read -r -a order_area_name_array <<< "$order_area_names"
    for order_area in "${order_area_name_array[@]}";
    do      
        final_list=$parameters_list" --order_area_name "$order_area
        echo $final_list
        python3 ./main.py ${final_list}
    done
fi

exit

I am not able pass the final_list of values to the python script because of which the Jenkin job is failing. If I echo the final_list I see that the values are properly getting initialized:

--user [email protected] --mqs_level Q2 --num_parallel_pipelines 13 --sns_topic topicname --ramp_up_time 45 --max_duration_for_task 30 --batch_size 35 --lead_store_db_schema schema --airflow_k8s_web_server_pod_name airflow-web-xyz --aws_access_key_id 12345678 --aws_secret_access_key 12345678 --order_area_name London

The error looks something like this:

main.py: error: the following arguments are required: --user, --sns_topic, --aws_access_key_id, --aws_secret_access_key Build step 'Execute shell' marked build as failure Finished: FAILURE

I searched for this at a lot of places but didn't get any concrete answer for this. Could anyone please help me with this?

Solution

Instead of writing my command like:

python3 ./main.py ${final_list}

I used this and it worked very well:

echo $final_list | bash

"final_list" variable has the command which needs to be executed.

Answered By - Jay