Initialise Python with GoCD

Following the deployment of a python-aware agent, we can start creating a pipeline. The first stage would be to:

1. Clean up any previous data
2. Create a virtual environment for python
3. Install all dependencies from a requirements.txt file in our project

Note: we assume the global pipeline settings and the Materials are properly set up (e.g. out project will be checked out in ./project).

Since the above operations must be executed in order, I've set up a Setup stage with a job named virtualenv. The job has three tasks:

1. rm
2. virtualenv
3. pip install

rm

This task is a generic shell command rm. We delete the venv/ directory created by previous runs, so we'll use rm -rf venv

IMPORTANT: Make sure you don't have trailing spaces in the Args input widget for the Task! Otherwise, the various flags will be quoted (e.g. "-rf ") and the execution will fail.

virtualenv

This task will create a virtual environment named venv. We still use a shell command to run virtualenv through python like /usr/bin/python -m virtualenv venv

pip install

We need to execute pip install in the current virtual environment. Since GoCD command type doesn't execute multiple commands (which is arguably a feature), we need to run the command through python as well:

We're executing the command in our checked out project directory. Therefore, our virtual environment python is located at ../venv/bin/python and the command line would be:

../venv/bin/python -m pip install -r requirements.txt

Now, the value of all tools (python, c compiler, crypto libraries) we installed in our python-aware agent comes to shine :)

The XML

The XMl generated by the above steps is:

<job name="virtualenv">
  <tasks>
    <exec command="rm">
      <arg>-rf</arg>
      <arg>venv</arg>
      <runif status="passed" />
    </exec>
    <exec command="/usr/bin/python">
      <arg>-m</arg>
      <arg>virtualenv</arg>
      <arg>venv</arg>
      <runif status="passed" />
    </exec>
    <exec command="../venv/bin/python" workingdir="project">
      <arg>-m</arg>
      <arg>pip</arg>
      <arg>install</arg>
      <arg>-r</arg>
      <arg>requirements.txt</arg>
      <runif status="passed" />
    </exec>
  </tasks>
</job>

HTH,