Save Pandas DataFrame as Django Model

Posted by in Software, Tutorial

TL;DR: use bulk_upload coupled with a comprehension to speed up loading. Initially, I started to convert the data frame to a Model object row by row and save it. While easy, it’s also arguably the slowest method to load records, because the save() call performs a commit. Then I looked up how to save pandas data frames and I found out quite a few ways… Manual commit The first step would be to tweak the transaction to manually commit like described here:

SQL Other solution would be to use…read more

Load Templates with Pebble

Posted by in Software, Tutorial

Pebble is a templating engine. It looks particularly familiar to me because I have been using Django. You have access to blocks, include/import features, loops etc. This post is a simple exercise in loading some templates. Prerequisites I’ve been using Eclipse IDE, but you can use whatever you prefer. I’ve created a maven project and added pebble dependency:

The templates I’ve created 3 templates in the resources folder of the project: index.html


Note how the {{item}} parameter is taken from the loop in *index.html. inner.html

read more

Change a django password manually

Posted by in Software

The other day I’ve been confronted with a strange situation: I forgot the admin password for a django application. I was playing with installing my own pypi repository, I’ve configured and deployed a test application and, after a couple of weeks, I forgot the password. I tried several times bu I knew that I used something silly and… unfortunate. Even worse, the user name I thought I used was not the one I had set up for the test. Oops :) After looking online for options I’ve found the following:…read more

Python classes to generate CSS

Posted by in Idea, Software

A while ago I got fed up with CSS and its cumbersomeness. As my poison of choice is python, I was thinking it would be quite useful to have a bunch of classes capable of generating CSS automatically in python. One could have a base class, a set of properties, a name-value dictionary and a way to nest them nicely so we could emulate LESS/SCSS. The idea is to use the power of python (or whatever language you’d want), with nesting etc. so we allow non-web designers mess around with…read more

Formless Step in FormWizard

Posted by in Software

Sometimes you’d need to show a purely informational page as part of a FormWizard. As far as I could see, there are two solutions to the problem: If the info is something like a summary, you have it available in the “success” page (the one after the wizard is finished) Alternatively, you can make it a page in the wizard. The more challenging and versatile variant is the second one (versatile because it gives you the possibility to go back to a previous step to correct stuff). My Solution My…read more

Create Data Once in a FormWizard

Posted by in Software

At one of the sites I’m building I need to walk users through a registration process: Select a supported tool Generate an ID Create a configuration based on the previous two steps To do this, I chose to use a FormWizard with three steps, one for each of the above. The problem then becomes to generate the ID once per wizard iteration. Initial Approach(es) Various online solutions (and even in the FormWizard documentation) suggest using get_form_kwargs() or get_form_initial() to build step-relevant data coupled with the get_cleaned_data_from_step(). The get_form_initial() method is…read more

Easy django applications with cookiecutter

Posted by in Software

Now I’m trying to create a bunch of django applications and I’ve got tired of creating everything from scratch. So, here are the steps I’m using to easy create a new django app: Create a new virtual environment. This is vital. It allows you to set up your own versions, not pollute the default installation (removing packages is tricky IMO). If you don’t know what I’m talking about, spend a few minutes to find out. Install cookiecutter (pip install cookiecutter). This is according to some the best thing that happened…read more

Make pyflakes/pylint ignore unused imports

Posted by in Software

Sometimes, you need to add an import without explicitly using it, just to have stuff defined. In such cases, both pyflakes and pylint generate messages, which can be annoying. For example django-appconf has a class-based configuration like so:

If I’m using the file like this, then I’ll end up with both pylint and pyflakes complaining that settings is unused, although it actually is (when the file gets imported and parsed). You can silence pylint by placing a comment on the import line (or in a pylintrc file to have…read more

Decent image loader widget in django

Posted by in Software

I’m working on a website where I need to be able to load an image. The default widget is abysmal; it has a url to the current image, a checkbox and a file loader input element. Even styled, it looks pretty bad. So, I’ve taken upon myself to prettify it and get from this: to this: To accomplish the transformation, we need: Crispy forms – just because my app uses it and it gives nice bootstrap-based formatting Floppy forms – because it allow us to customise widgets markusslima’s bootstrap-filestyle –…read more

Speed-up easy-thumbnails on S3

Posted by in Software

I was developing a small app as a playground and confidence builder, choosing django, heroku and S3 as resources. One of the packages used is easy-thumbnails. Behaviour I chose to use the easy-thumbnails app with S3, a possibility granted by the storages framework. At a glance, everything works OK; thumbnails are created and rendered accordingly. However, on a thumbnail-heavy page, I (and my profiling app) noticed significant load times (3s for a page with 7 thumbnails). Drilling down, it became evident that something was checking the S3 bucket for each…read more