Django Tricks, Part 5 – Automatic App Settings

February 7, 2010 at 8:01 pm · Filed under technical

This post is the fifth in a series on Django tricks.

We started with automating Admin Model creation, and continued with automating initialization and installation, model mixins and forms.

This post describes how we automate incorporating new Django apps into our project.

Creating apps is a wonderful way to modularize work. Encapsulation, abstraction, re-use!

To reduce copy/paste/rewrite drudger we’ve automated our project’s settings.py file according to standard Django conventions.

APPS list

First, we need to list what apps are active in the project:

APPS = ('procrasdonate', 'crosstester', 'adwords', 'procrasdocoder')

Pathify utility

Let’s setup some convenience path functions.

path converts a path with unix path separator, /, into a path with the appropriate os-specific path separator.

pathify converts a list of path components into a string path with appropriate path separator.

Finally, PROJECT_PATH holds the projects root absolute path.

import os
def path(p):
    """
    @param p: path with unix path separator
    @return: string with os-specific path separator
    """
    return os.sep.join(p.split("/"))

import re
_space_replace = re.compile("([^\\\])( )")
def pathify(lst):
    """
    @param lst: list of path components
    @return: string with os-specific path separator between components 
    """
    # replaces spaces with raw spaces so that spaces in file names work
    repl = r"\g<1> "
    return os.sep.join([_space_replace.sub(repl, el) for el in lst])

PROJECT_PATH = os.path.dirname(os.path.realpath(__file__))

Middleware

Now we’re into the core of the work, which is to iterate through each app and append the appropriate path or module to the relevant tuple.

MIDDLEWARE_CLASSES = (
    'django.middleware.doc.XViewMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'ext.pagination.middleware.PaginationMiddleware',
    'django.contrib.csrf.middleware.CsrfMiddleware',
    'lib.ssl_middleware.SSLRedirect',
)
for app in APPS:
    if os.path.exists(pathify([PROJECT_PATH, app, 'middleware.py'], file_extension=True)):
        MIDDLEWARE_CLASSES += (
            '%s.middleware.%sMiddleware' % (app, app.capitalize()),
        )

Templates

Add template directories.

Note: Do you see how we also add the template directory of our Firefox extension? This is so that our server can reuse extension templates (OMGSQUEEL).

TEMPLATE_DIRS = (
    #'procrasdonate/ProcrasDonateFFExtn/content/templates',
    pathify([PROJECT_PATH, path('procrasdonate/ProcrasDonateFFExtn/content/templates')]),
)

for app in APPS:
    if os.path.exists(pathify([PROJECT_PATH, app, 'templates'])):
        TEMPLATE_DIRS += (
            pathify([PROJECT_PATH, path('%s/templates' % app)]),
        )

Installed Apps

INSTALLED_APPS = (
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.humanize',
    'ext.pagination',
    'django.contrib.auth',
    'django.contrib.admindocs',
    'django.contrib.admin',
)

for app in APPS:
    INSTALLED_APPS += (
        app,
    )

Context Processors

TEMPLATE_CONTEXT_PROCESSORS = (
    'lib.context.defaults',
    'django.core.context_processors.auth',
    'django.core.context_processors.debug',
    'django.core.context_processors.i18n',
    'django.core.context_processors.media',
    'django.core.context_processors.request',
)

for app in APPS:
    if os.path.exists(pathify([PROJECT_PATH, app, 'context.py'], file_extension=True)):
        TEMPLATE_CONTEXT_PROCESSORS += (
            '%s.context.defaults' % app,
        )

This only works if the app includes a “context.py” file with a function called “defaults” that returns the context dictionary:

context.py

def defaults(request):
    frame = {}
    frame.update(useful_settings())
    frame.update(header_data())
    ...
    return frame

def useful_settings():
    return { 'PDVERSION': settings.PDVERSION }

def header_data():
    return { ... }

Urls

The project’s base URL file is modified to automatically incorporate each app’s URLs.

For example, an app called “magic_cape” would be accessible from the URL “/magic_cape/”, assuming it contained a “urls.py” file inside a “views” directory.

To customize the urls for an app, it’s name is added to the CUSTOM_URLS_APPS tuple.

CUSTOM_URLS_APPS = ('procrasdonate',)

for app in settings.APPS:
    if app not in CUSTOM_URLS_APPS and os.path.exists(pathify([settings.PROJECT_PATH, app, 'views'])):
        urlpatterns += patterns('',
            (r'^%s/' % app, include('%s.views.urls' % app)),
        )

Proudly ProcrasDonating,

Lucy.

Comments (3)

Django Tricks, Part 4 – Forms

February 7, 2010 at 7:52 pm · Filed under technical

This post is the fourth in a series on Django tricks.

We started with automating Admin Model creation. This post describes how we automate the creation of form classes for models.

Automatic Model Form creation

The point of automatic forms is to reduce the effort that goes into writing views. Form classes for models should be created on the fly as we need them, with the same flexibility as if we had defined the form class directly.

Here’s an example that retrieves a form for the public fields of the Recipient model:

from lib.forms import get_form, EDIT_TYPE

@login_required
def edit_public_information(request):
    recipient = request.user.get_profile().recipient
    FormKlass = get_form(Recipient, EDIT_TYPE, includes=('name',
                                                         'category',
                                                         'mission',
                                                         'description',
                                                         'url'))

    if request.POST:
        form = FormKlass(request.POST, instance=recipient)
        if form.is_valid():
            form.save()

The get_form utility function takes the Model class as a required parameter. If that is the only parameter, it will return the default ModelForm for that model.

The includes and excludes fields normally specified in the ModelForm’s inner Meta class can be provided as parameters to get_form().

The forms module also specifies three ModelForm types, NEW_TYPE, EDIT_TYPE and ADMIN_TYPE. Originally, the includes and excludes were defined per type per model in the forms module. This is useful if there are multiple places in the web app where models are edited, and you don’t want to keep remembering which immutable or signal-computed fields to exclude.

Note: If we go back to using the NEW_, EDIT_ and ADMIN_TYPEs we would move their definitions to inside models themselves rather than editing a single dictionary in the forms module. Live, learn and move on.

The forms module is appended below.

Proudly ProcrasDonating,

Lucy.

from django.forms import ModelForm

"""
We want to create three kinds of forms:
1. Admin<Model>Form  admin forms.     these have all fields and are used for both creation and editing
2. New<Model>Form    user new forms.  these exclude hidden or contextually obvious fields (eg state, a review's node)
3. Edit<Model>Form   user edit forms. these exclude hidden fields
"""
# shows all fields
ADMIN_TYPE = 'admin'
# user facing form for creating new instance
NEW_TYPE = 'new'
# user facing form for editing existing instance
EDIT_TYPE = 'edit'

# users should not reference this cache directly.
# instead, use get_form
FORMS = { ADMIN_TYPE: {},
          NEW_TYPE: {},
          EDIT_TYPE: {},
          }

def get_form(model, type=ADMIN_TYPE, excludes=None, includes=None):
    """
    @param excludes: in addition to type-based excludes. for on-the-fly forms. 
    @param includes: for on-the-fly forms. 
    return form class for model and type
    """
    if model in FORMS[type] and not excludes and not includes:
        # don't cache on-the-fly forms
        form = FORMS[type][model]
    else:
        mname = model.__name__
        excludes = excludes or ()
        
        if type == ADMIN_TYPE:
            exec('Admin%sForm = model_form_class(model, excludes, includes)' % mname )
            form = locals()['%sForm' % mname]
                
        elif type == NEW_TYPE:
            if mname in new_forms_excludes:
                excludes += new_forms_excludes[mname]
            exec('New%sForm = model_form_class(model, excludes, includes)' % mname )
            form = locals()['New%sForm' % mname]

        elif type == EDIT_TYPE:
            if mname in edit_forms_excludes:
                excludes += edit_forms_excludes[mname]
            exec('Edit%sForm = model_form_class(model, excludes, includes)' % mname )
            form = locals()['Edit%sForm' % mname]
            
        else:
            raise "Type no good"
        
        if not excludes and not includes:
            # don't cache on-the-fly forms
            FORMS[type][model] = form
    
    return form


def model_form_class(_model, excludes=None, includes=None):
    class klass(ModelForm):
        class Meta:
            model = _model
            if excludes:
                exclude = excludes
            if includes:
                fields = includes
    return klass

new_forms_excludes = {#'OldModel':('url',),
                      #'OldModel2':('old_model','foo','bar'),
                      }

edit_forms_excludes = {
                       #'MyModel':('slug','immutable','calculated_from_post_save'),
                      }

Comments (1)

Django Tricks, Part 3 – Model mixins

February 7, 2010 at 6:52 pm · Filed under technical

This post is the third in a series on Django tricks.

In the last post we covered automating initialization. Continuing with initialization, this post will cover model mixins.

Model mixins mix convenient utilities into model classes.

Specified Mixins

Mixins is a general term for mixing attributes of an abstract class into a non-abstract subclass.

In this case, we want to mix convenience methods into Model Foo. The reason why these methods aren’t defined in Foo is because of clean, modular code. An example will help:

class Tagging(models.Model):
    tag = models.ForeignKey(Tag)
    post = models.ForeignKey(Post)

    @classmethod
    def Initialize(klass):
        model_utils.mixin(TaggingMixin, Post)

There are three models at play in the above example:

• Post – a blog post
• Tag – a tag
• Tagging – links a tag with a post

There is also a normal python class, TaggingMixin, that provides convenience methods for Posts:

class TaggingMixin(object):
    """ mixed into Post """
    
    @property
    def tags(self):
        return Tag.objects.filter(tagging_set__post=self)

In the Tagging model’s Initialization function, mixin mixes TaggingMixin into Post. This allows us to encapsulate all the Tag logic, including the ForeignKeys to Post, inside the Tagging models and the TagMixin class. The Post model remains clean, with only those functions that deal with its direct fields.

This separation becomes a necessity when Models are spread between different apps.

The mixin code is appended to the end of this post.

Standard Mixins

We automatically mixin in the following convenience functions to every Model:

• add
• find
• get
• get_or_none

get_or_none is a nice way to retrieve an object if it exists, such as:

lucy = User.get_or_none(name="Lucy")
if lucy:
    print lucy.favorite_number
else:
    print "Lucy is not in the database"

add is a convenient constructor for creating objects, though it requires adding a Make classmethod to every Model. The advantage is that we not only put constructor logic inside a constructor, but we also abstract that call into add…in case making things becomes more complicated later.

some view function

lucy = User.add("Lucy", 22)

models.py

class User(models.Model):
    name = models.CharField(max_length=222)
    favorite_number = models.IntegerField()

    @classmethod
    def make(klass, name, favnum):
        favnum = int(round(favnum))
        return User(name=name, favorite_number=favnum)

In order to mix these methods into every Model we resort to the __init__.py trick discussed previously:

from models import ALL_MODELS

from lib import model_utils
model_utils.mixin(model_utils.ModelMixin, ALL_MODELS)

Proudly ProcrasDonating,

Lucy

mixin definition

def mixin(mixin, klasses, last=0):
    if not isinstance(klasses, (list, tuple)):
        klasses = (klasses,)
    for klass in klasses:
        if mixin not in klass.__bases__:
            if last:
                klass.__bases__ = klass.__bases__+(mixin,)
            else:
                # sometimes this fails, but if change order seems to work
                try:
                    klass.__bases__ = (mixin,)+klass.__bases__
                except:
                    klass.__bases__ = klass.__bases__+(mixin,)

ModelMixin definition

class ModelMixin(object):
    """
    @summary:
    B{ModelMixin} is a mix-in used to provide common methods, attributes,
    and hooks across all models.
    
    At initialization, ModelMixin is mixed into all models listed in 
    """
    @classmethod
    def find(klass, ids=None, **kwargs):
        """
        @summary:
        Convenience method: B{find} simply passes arguments through to
        klass.objects.filter()
        """
        if isinstance(ids, (int, long)):
            return klass.objects.filter(id=ids, **kwargs)
        elif isinstance(ids, list):
            return klass.objects.filter(id__in=ids, **kwargs)
        else:
            return klass.objects.filter(**kwargs)

    @classmethod
    def get_or_none(klass, **kwargs):
        """
        @summary:
        Convenience method: B{get_or_none} simply passes arguments through to
        klass.objects.filter()
        """
        f = klass.objects.filter(**kwargs)
        if len(f) == 0:
            return None
        else:
            return f[0]
        
    @classmethod
    def get(klass, id=None, **kwargs):
        """
        Convenience method: B{get} simply passes arguments through to 
        klass.objects.get()
        
        @note: Raises exceptions anytime 'get' would.
        """
        if isinstance(id, (int, long)):
            #return klass.objects.get_object_or_404(id=id, **kwargs)
            return get_object_or_404(klass, id=id, **kwargs)
        elif len(kwargs) > 0:
            return klass.objects.get(**kwargs)
        else:
            raise RuntimeError("No id or conditions given to 'get'!")
    
    @classmethod
    def add(klass, *args, **kwargs):
        """
        @summary:
        This is a general method which simply calls 'make' with the same
        arguments and then saves the returned object.
        """
        o = klass.make(*args, **kwargs)
        o.save()
        #if publish:
        #    o.publish()
        return o

Comments (1)

Django Tricks, Part 2 – Initialize and Install

February 7, 2010 at 6:30 pm · Filed under technical

This post is the second in a series on Django tricks.

We started with automating Admin Model creation. Continuing with automation, this post will cover model initialization and installation.

Initialization is the stuff we would normally put in a constructor, such as register pre-save and post-save signals.

Models are meta-classes, so they don’t have normal constructors. Instead, we put all the initialization stuff in a classmethod called Initialize:

class MyModel(models.Model):
    @classmethod
    def Initialize(klass):
        models.signals.pre_save.connect(MyModel.sanitize_user_input, sender=MyModel)

In the app’s __init__.py file, we automatically look for Initialize methods to call.

__init__.py

from models import ALL_MODELS

for model in ALL_MODELS:
    if hasattr(model, 'Initialize'):
        model.Initialize()

We could put signal registration directly in the model’s file, but it’s nicer to encapsulate all this Initialization stuff in a function, and then automate that function’s execution.

For more neat initializations read part 3 of this series, Model mixins.

Installation is stuff we want to do once the model class is synched with the database, such as installing a fixture or initial data.

As with Initialization, we can optionally add a classmethod called install to a model. A post_syncdb function will look for install functions to call when iterating over models.

management.py

from django.dispatch import dispatcher
from django.db.models import signals
 
import models
 
"""
@summary: Install initial data when models are first created.
 
Automatically called by django.
Find out more be searching for "management.py" and "Extra special stuff" in
http://www.b-list.org/weblog/2006/sep/10/django-tips-laying-out-application/
document on wiki:
http://bilumi.org/trac/wiki/PostSyncdbInstall
"""
 
def post_syncdb(signal, sender, app, created_models, **kwargs):
    # only run when models we care about are first created
    if signal == signals.post_syncdb and app == models:
        for model in models.ALL_MODELS:
            if hasattr(model, 'install'):
                if kwargs['verbosity'] > 0:
                    print "Installing ", model
                model.install()
 
signals.post_syncdb.connect(post_syncdb)

Proudly ProcrasDonating,

Lucy.

Comments (3)