Laur IVAN

Get all Proposal Names from the Participant Portal

Posted by in Software, Tutorial, Uncategorized

I’ve found out the participant portal for H2020 offers an API. This is quite neat as one can integrate with the data, e.g. by reading and listing the calls. I’ve written a proof of concept to list all calls from this and last years. First, we load the data in a JSON object:

Then, we extract the year:

Lastly, we iterate through all calls and extract the ones we need:

For the calls API, you can have access to the following information (fields): CallIdentifier FileName – e.g….read more

Inject Proxy Settings in Selenium/Chrome

Posted by in Software, Tutorial

I have the following situation: I’m behind a corporate proxy and I need to run a test accessing an external website. To do this, I’ve picked up chrome/chromium as driver of choice. My current solution implies an authentication via plugin. The plugin The chrome plugin is essentially 2 files: a manifest file a JS containing a listener Manifest The manifest file is:

The script Following script performs the proxy authentication:

,where the PROXY_* variables are the components of the proxy URL. Plugin The plugin is created by zipping…read more

0

Strip HTML tags in python

Posted by in Software, Tutorial

Note: This is a post transferred from Laurii for historical and consolidation purposes. A common problem I have to deal with quite often is to remove all HTML tags from a document. While this is easy for XML (well formatted etc.) and you could do it by hand with a regexp, HTML has a bit more problems. There are several solutions to get around this… Using a browser This is a bit overkill, but you could use a browser’s renderer to “display” the string content and then get it as…read more

Cascade get value from dictionary in Python

Posted by in Software, Tutorial

Several times I’ve been faced with the problem: Get a value from a dictionary. If the key is not present, then get from another key. Normally, the implementation would go like this:

Or, even worse, with try-catch statements. That’s fine if the sequence is fixed, but what if you want to have some flexibility? Well, my solution is like this:

Pretty, isn’t it? Note: This is a post transferred from Laurii for historical and consolidation purposes. A little experiment: If you find this post and ad below useful,…read more

Change a django password manually

Posted by in Software

The other day I’ve been confronted with a strange situation: I forgot the admin password for a django application. I was playing with installing my own pypi repository, I’ve configured and deployed a test application and, after a couple of weeks, I forgot the password. I tried several times bu I knew that I used something silly and… unfortunate. Even worse, the user name I thought I used was not the one I had set up for the test. Oops :) After looking online for options I’ve found the following:…read more

Python and Outlook – An example

Posted by in Software

TL;DR: Install win32 extensions (e.g. using this to put it in your virtual env). Read some code below (there are folders with stuff, messages have CapitalizedPropertyNames). Profit. The other day I needed to perform some statistics on an Outlook mailbox. In particular, use the frequency per day to identify peaks of activity. I could get an excel sheet going with VBA but it’s more fun in python. So, here it goes (mind you, it’s a bit long due to the included code). Preparation First, I’ve defined a class to help…read more

Add XML to PostgreSQL from Python

Posted by in Software

One of the projects I worked on was to import a large number of XML files in a PostgreSQL database (as XML files). I chose python to do it. Here are the steps: Database The data source is the SCOPUS database which has a silly number of entries (approx 20,000,000), most containing 2 XML files (article and citations). The publications with no citations don’t have the citations.xml file. Therefore, initial database has a single table named scpous_data with the following fields: id – string of 20 characters. This is the…read more

Rot Algorithm in Python

Posted by in Software

A while ago I wanted to send a bunch of emails to building companies in order to get some quotes and/or information. Unfortunately, the website in question (pagesdor.be) doesn’t provide emails in clear text to allow me copy/paste. Instead it uses a Rotating cypher algorithm (I guess it must be simple enough to be able to be decrypted by a browser). After a couple of emails, where the process of sending through the website costed me some time I figured that it would be easier to see and implement the…read more

Python classes to generate CSS

Posted by in Idea, Software

A while ago I got fed up with CSS and its cumbersomeness. As my poison of choice is python, I was thinking it would be quite useful to have a bunch of classes capable of generating CSS automatically in python. One could have a base class, a set of properties, a name-value dictionary and a way to nest them nicely so we could emulate LESS/SCSS. The idea is to use the power of python (or whatever language you’d want), with nesting etc. so we allow non-web designers mess around with…read more

D-BUS: DBUS and Mad scientists… – 2004 Style

Posted by in Software

TL;DR: This post is about an experience I had in 2004 and it’s part of my consolidating technical posts I wrote in time. It may or may not be relevant to today’s technologies. For the last while I was poking around with the glorious D-BUS… As it’s always the case with new cutting edge technology, it’s done by one or more very good, brilliant one might say (posh british accent here) people. I always have this image of “mad scientist” associated with that :). DBUS is great, is monumental, but…read more