Laur IVAN

Rundeck – Speeding Up Even More the Cleaning Process

As I was running the script, I’ve noticed it ran very slow. I have left it for a few days to run (700K executions), I’ve noticed the speed increased linearly over time. As Rundeck has both DB and files to store its data, it looked to me that it was the file system which was the slow part. Sure enough, when I tried to access the directory where the execution data was stored, it was slow – 1.4M files would take a while to get organised :) To short out…read more

Oct.09

Delete Executions in Rundeck – Wrapping it all Up

Posted by laur in IT, Software, Tutorial

The previous posts have built the components to allow us to get projects, jobs, executions and remove executions from a Rundeck instance via its API. Now, we can just wrap them up :) The list of packages we need is:

import time
import datetime
import sys
import logging

import time

import datetime

import sys

import logging

For convenience, I’ve declared a bunch of constants we can use throughout the project:

EXPIRE_DAYS = 30
SIZE=100
MAXCOUNT=1000
OFFSET=0
BATCH_SIZE=20
EXPIRE_MILISECONDS = EXPIRE_DAYS * 24 * 60 * 60 * 1000

log = logging.getLogger(__name__)

EXPIRE_DAYS = 30

SIZE=100

MAXCOUNT=1000

OFFSET=0

BATCH_SIZE=20

EXPIRE_MILISECONDS = EXPIRE_DAYS * 24 * 60 * 60 * 1000

log = logging.getLogger(__name__)

As we already have all the helper functions defined, we only need to wrap them in a convenient execution command:

def main():
    import argparse

    # The parser
    parser = argparse.ArgumentParser()
    parser.add_argument("host")
    parser.add_argument("port")
    parser.add_argument("api_key")

    args = parser.parse_args()

    projects = getProjectNames(listProjects(args.host, args.port, args.api_key))
    deleted = 0
    for project in projects:
        log.info('project:\t'+project)
        jobids = getJobIDs(listJobsForProject(args.host, args.port, args.api_key, project))
        for jobid in jobids:
            log.info('\tjobid:\t'+jobid)
            executions = getAllExecutionsForAJob(args.host, args.port, 
                args.api_key, jobid, MAXCOUNT)

            # Save the list of timestamps in a file
            with open(jobid + '.csv', 'wt') as f:
                for execution in executions:
                    f.write("%s,%s\n" % (execution,executions[execution]))
            deleted += checkDeletion(args.host, args.port, args.api_key, 
                executions)
        print('I have deleted :%d: jobs for %s' % (deleted,project))

def main():

import argparse

# The parser

parser = argparse.ArgumentParser()

parser.add_argument("host")

parser.add_argument("port")

parser.add_argument("api_key")

args = parser.parse_args()

projects = getProjectNames(listProjects(args.host, args.port, args.api_key))

deleted = 0

for project in projects:

log.info('project:\t'+project)

jobids = getJobIDs(listJobsForProject(args.host, args.port, args.api_key, project))

for jobid in jobids:

log.info('\tjobid:\t'+jobid)

executions = getAllExecutionsForAJob(args.host, args.port,

args.api_key, jobid, MAXCOUNT)

# Save the list of timestamps in a file

with open(jobid + '.csv', 'wt') as f:

for execution in executions:

f.write("%s,%s\n" % (execution,executions[execution]))

deleted += checkDeletion(args.host, args.port, args.api_key,

executions)

print('I have deleted :%d: jobs for %s' % (deleted,project))

The logis is fairly simple: Get the list of projects For each project,…read more

Oct.08

Remove Executions in Rundeck

Posted by laur in IT, Tutorial

TL;DR: You have to get the executions’ list and remove the appropriate ones using hte API. Once we’ve managed to get the list of job executions, we can use the API to remove the relevant executions. The command is rather simple:

def deleteExecutions(server, port, api_key, execution_ids):
    url =  server +':'+port+'/api/12/executions/delete?ids=' + 
        ','.join(execution_ids)
    headers = {'Content-Length': '0', 'X-RunDeck-Auth-Token': api_key }
    requests.post(url, headers=headers, verify=False)

def deleteExecutions(server, port, api_key, execution_ids):

url = server +':'+port+'/api/12/executions/delete?ids=' +

','.join(execution_ids)

headers = {'Content-Length': '0', 'X-RunDeck-Auth-Token': api_key }

requests.post(url, headers=headers, verify=False)

This method requires: server – the Rundeck’s server address port – the port on which we can connect to the Rundeck server api_key – the API key execution_ids – an array of valid execution IDs The function will try to remove all execution IDs in one go, since…read more

Oct.02

Get Job Executions in Rundeck

Posted by laur in IT, Tutorial

Following the previous articles, we are now in position to query job executions. The query is done via API as well. As a job can have an arbitrary number of executions, it’s advisable to use pagination. The code follows the same lines as the one in the previous entries:

# API call to get the list of the executions for a Job.
SIZE = 100
def getExecutionsForAJob(server, port, api_key, job_id, page=0, size=SIZE):
    url =  server +':'+port + \
        '/api/1/job/'+job_id + \
        '/executions?max='+('%d' % size) + \
        '&offset='+('%d' % (page * size))
    log.info(url)
    headers = {'Content-Type': 'application/json','X-RunDeck-Auth-Token': api_key }
    r = requests.get(url, headers=headers, verify=False)
    return r.text.encode('utf-8')

# API call to get the list of the executions for a Job.

SIZE = 100

def getExecutionsForAJob(server, port, api_key, job_id, page=0, size=SIZE):

url = server +':'+port + \

'/api/1/job/'+job_id + \

'/executions?max='+('%d' % size) + \

'&offset='+('%d' % (page * size))

log.info(url)

headers = {'Content-Type': 'application/json','X-RunDeck-Auth-Token': api_key }

r = requests.get(url, headers=headers, verify=False)

return r.text.encode('utf-8')

This function builds the URL for the call and returns an XML string. Its parameters are: server – the rundeck server port – the rundeck server’s port api_key – the api key for rundeck job_id – one of the job…read more