Jump to content

Check out our Community Blogs

Register and join over 40,000 other developers!

Recent Status Updates

View All Updates

- - - - -

Python on Google App Engine: Creating blog engine. Part 3

  • Please log in to reply
No replies to this topic

#1 Vladimir


    CC Resident

  • Just Joined
  • PipPipPipPip
  • 79 posts

Posted 05 December 2010 - 12:27 PM

In this part I am going to discuss some limitations I met at GAE. To be honest there are a lot of them, but here I listed only issues for which nice workaround was found.

1. pre- and post- hooks. Flask signals.
There is an excelent article by Nick Johnson: Pre- and post- put hooks for Datastore models - Nick's Blog . I fixed and enhanced code provided by Nick to support Flask signals and created BaseModel that extends db.Model:

from blinker import Namespace

model_signals = Namespace()
pre_put = model_signals.signal('pre-put')
post_put = model_signals.signal('post-put')

class BaseModel(db.Model):
    def pre_put(self):

    def post_put(self):

    def put(self, **kwargs):
        if self.pre_put() == False:
            return None # do not save model if pre_put() returned False
        for (receiver, ret) in signals.pre_put.send(self.__class__, model=self): # send signal to subscribers
            if ret == False:
                return None # do not save model if handler returned False

        key = super(BaseModel, self).put(**kwargs) # save model

        # here goes post hooks
        signals.post_put.send(self.__class__, model=self)

        return key

old_put = db.put

def put(models, **kwargs):
    # we can fallback to BaseModel.put() if user want to save only one model
    if not isinstance(models, (list, tuple)):
        return models.put()

    # call hooks and send signals to all models
    for model in models:
        if isinstance(model, BaseModel):
            if model.pre_put() == False:
            for (_, ret) in signals.pre_put.send(model.__class__, model):
                if ret == False:

    # check it there are models on the list
    if models:
        # batch save
        keys = old_put(models, **kwargs)
        keys = None

    # call hooks and send signals to all models
    for model in models:
        if isinstance(model, BaseModel):
            signals.post_put.send(model.__class__, model)

    return keys

# monkey path GAE ORM
db.put = put

This code is used in my app like this:

class Post(BaseModel):
    def pre_put(self):
        # autogenerate html from markdown markup
        self.content_html = markdown(self.content)

# here we listen for post_put signal on Post model to icrement post_count for post category
def increment_post_count_on_category(sender, model=None):
    category = model.category
    category.post_count += 1

2. Datastore transactions

Quote from docs:

All datastore operations in a transaction must operate on entities in the same entity group. This includes querying for entities by ancestor, retrieving entities by key, updating entities, and deleting entities. Notice that each root entity belongs to a separate entity group, so a single transaction cannot create or operate on more than one root entity.

In my case I want to check if category exists in database before saving it. To be able to do this I have to put all categories under the same root entity, but there is no suitable entity for this. The good news is that GAE does not really care if you provide real (saved) entity or just dummy key with random id. So I use dummy key as entity root to be able to use transaction:

Category(parent=db.Key.from_path(Category.kind(), sys.maxint))

Code that saves category looks like this:

        def create(cls, name):
            def txn():
                entity = Category.all().ancestor(our_dummy_key).filter('name =', name).get()
                if entity:
                    return (entity, False)

                entity = Category(name=name, parent=our_dummy_key)

                return (entity, True)

            return db.run_in_transaction(txn)

category, created = Category.create('name')

3. Denormalization and model instance method

GAE does not support joins. To be able to show user category name of the post I have to duplicate category name in the Post model:

class Post(BaseModel):
    # ...

    category_name = db.CategoryProperty(required=False)
    category = db.ReferenceProperty(Category, required=True)

Template looks like this:

<a href="{{ url_for('blog.category_view', slug=slugify(post.category_name)) }}">{{ post.category_name }}</a>

Not bad, but ususally above code looks like this:

<a href="{{ post.category.get_absolute_url() }}">{{ post.category }}</a>

To accomplish this I add such code to the post:

class Post(BaseModel):
    def category_c(self):
        """Creates and caches dummy category"""
        return Category(name=self.category_name)

The only change you need in template is replacing "category" with "category_c". Category must implement __unicode__() and get_absolute_url() methods:

class Category(BaseModel):
    # ...

    def get_absolute_url(self):
        return url_for('blog.category_view', slug=self.slug)

    def __unicode__(self):
        return unicode(self.name) # don't forget to force unicode conversion on property

4. key_id and key_name

It is not clear when I have to use key_id or key_name on the model and how to migrate from one to another. What do you think?

5.Model validation and required property

If True, the property cannot have a value of None. A model instance must initialize all required properties in its constructor so that the instance is not created with missing values. An attempt to create an instance without initializing a required property, or an attempt to assign None to a required property, raises a BadValueError.

I am not happy with this, because I definitely want most properties to present in datastore, but I also want to be able to autogenerate missing properties (and key_name) in pre_put hook. What do you think?
  • 0

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download