Run Celery 4.1.0 on Windows 10

TL;DR: Use set FORKED_BY_MULTIPROCESSING=1 on the worker script/shell When running Celery on Windows 10, I got the following error: ValueError: not enough values to unpack (expected 3, got 0) This is quite unfortunate as Windows is supported on “best effort”. A pull request has a change, but it’s not pretty to patch an installation. To […]

Convert Timestamp to DateTime for Pandas DataFrame

To convert a pandas data frame value from unix timestamp to python datetime you need to use:

where: timestamp is the column containing the timestamp value unit=’s’ defines the unit of the timestamp (seconds in this case) You can actually replace the column altogether:

Unfortunately, the conversion is not really aware of the […]

Get all Proposal Names from the Participant Portal

I’ve found out the participant portal for H2020 offers an API. This is quite neat as one can integrate with the data, e.g. by reading and listing the calls. I’ve written a proof of concept to list all calls from this and last years. First, we load the data in a JSON object:

Then, […]

Inject Proxy Settings in Selenium/Chrome

I have the following situation: I’m behind a corporate proxy and I need to run a test accessing an external website. To do this, I’ve picked up chrome/chromium as driver of choice. My current solution implies an authentication via plugin. The plugin The chrome plugin is essentially 2 files: a manifest file a JS containing […]

Strip HTML tags in python

Note: This is a post transferred from Laurii for historical and consolidation purposes. A common problem I have to deal with quite often is to remove all HTML tags from a document. While this is easy for XML (well formatted etc.) and you could do it by hand with a regexp, HTML has a bit […]

Cascade get value from dictionary in Python

Several times I’ve been faced with the problem: Get a value from a dictionary. If the key is not present, then get from another key. Normally, the implementation would go like this:

Or, even worse, with try-catch statements. That’s fine if the sequence is fixed, but what if you want to have some flexibility? […]

Change a django password manually

The other day I’ve been confronted with a strange situation: I forgot the admin password for a django application. I was playing with installing my own pypi repository, I’ve configured and deployed a test application and, after a couple of weeks, I forgot the password. I tried several times bu I knew that I used […]

Python and Outlook – An example

TL;DR: Install win32 extensions (e.g. using this to put it in your virtual env). Read some code below (there are folders with stuff, messages have CapitalizedPropertyNames). Profit. The other day I needed to perform some statistics on an Outlook mailbox. In particular, use the frequency per day to identify peaks of activity. I could get […]

Add XML to PostgreSQL from Python

One of the projects I worked on was to import a large number of XML files in a PostgreSQL database (as XML files). I chose python to do it. Here are the steps: Database The data source is the SCOPUS database which has a silly number of entries (approx 20,000,000), most containing 2 XML files […]

Rot Algorithm in Python

A while ago I wanted to send a bunch of emails to building companies in order to get some quotes and/or information. Unfortunately, the website in question (pagesdor.be) doesn’t provide emails in clear text to allow me copy/paste. Instead it uses a Rotating cypher algorithm (I guess it must be simple enough to be able […]

Scroll to top