No replies to this topic

[Vladimir]

In this part I am going to discuss some limitations I met at GAE. To be honest there are a lot of them, but here I listed only issues for which nice workaround was found.

1. pre- and post- hooks. Flask signals.
There is an excelent article by Nick Johnson: Pre- and post- put hooks for Datastore models - Nick's Blog . I fixed and enhanced code provided by Nick to support Flask signals and created BaseModel that extends db.Model:

from blinker import Namespace



model_signals = Namespace()

pre_put = model_signals.signal('pre-put')

post_put = model_signals.signal('post-put')



class BaseModel(db.Model):

    def pre_put(self):

        pass


    def post_put(self):

        pass


    def put(self, **kwargs):

        if self.pre_put() == False:

            return None # do not save model if pre_put() returned False

        for (receiver, ret) in signals.pre_put.send(self.__class__, model=self): # send signal to subscribers

            if ret == False:

                return None # do not save model if handler returned False


        key = super(BaseModel, self).put(**kwargs) # save model


        # here goes post hooks

        self.post_put()

        signals.post_put.send(self.__class__, model=self)


        return key



old_put = db.put


def put(models, **kwargs):

    # we can fallback to BaseModel.put() if user want to save only one model

    if not isinstance(models, (list, tuple)):

        return models.put()


    # call hooks and send signals to all models

    for model in models:

        if isinstance(model, BaseModel):

            if model.pre_put() == False:

                models.remove(model)

            for (_, ret) in signals.pre_put.send(model.__class__, model):

                if ret == False:

                    models.remove(model)



    # check it there are models on the list

    if models:

        # batch save

        keys = old_put(models, **kwargs)

    else:

        keys = None


    # call hooks and send signals to all models

    for model in models:

        if isinstance(model, BaseModel):

            model.post_put()

            signals.post_put.send(model.__class__, model)


    return keys


# monkey path GAE ORM

db.put = put

This code is used in my app like this:

class Post(BaseModel):

    def pre_put(self):

        # autogenerate html from markdown markup

        self.content_html = markdown(self.content)



# here we listen for post_put signal on Post model to icrement post_count for post category

@signals.post_put.connect_via(Post)

def increment_post_count_on_category(sender, model=None):

    category = model.category

    category.post_count += 1

    category.put()

2. Datastore transactions

Quote from docs:

Quote

All datastore operations in a transaction must operate on entities in the same entity group. This includes querying for entities by ancestor, retrieving entities by key, updating entities, and deleting entities. Notice that each root entity belongs to a separate entity group, so a single transaction cannot create or operate on more than one root entity.

In my case I want to check if category exists in database before saving it. To be able to do this I have to put all categories under the same root entity, but there is no suitable entity for this. The good news is that GAE does not really care if you provide real (saved) entity or just dummy key with random id. So I use dummy key as entity root to be able to use transaction:

Category(parent=db.Key.from_path(Category.kind(), sys.maxint))

Code that saves category looks like this:

        @classmethod

        def create(cls, name):

            def txn():

                entity = Category.all().ancestor(our_dummy_key).filter('name =', name).get()

                if entity:

                    return (entity, False)


                entity = Category(name=name, parent=our_dummy_key)

                entity.put()


                return (entity, True)


            return db.run_in_transaction(txn)


category, created = Category.create('name')

3. Denormalization and model instance method

GAE does not support joins. To be able to show user category name of the post I have to duplicate category name in the Post model:

class Post(BaseModel):

    # ...


    category_name = db.CategoryProperty(required=False)

    category = db.ReferenceProperty(Category, required=True)

Template looks like this:

<a href="{{ url_for('blog.category_view', slug=slugify(post.category_name)) }}">{{ post.category_name }}</a>

Not bad, but ususally above code looks like this:

<a href="{{ post.category.get_absolute_url() }}">{{ post.category }}</a>

To accomplish this I add such code to the post:

class Post(BaseModel):

    @cached_property

    def category_c(self):

        """Creates and caches dummy category"""

        return Category(name=self.category_name)

The only change you need in template is replacing "category" with "category_c". Category must implement __unicode__() and get_absolute_url() methods:

class Category(BaseModel):

    # ...


    def get_absolute_url(self):

        return url_for('blog.category_view', slug=self.slug)


    def __unicode__(self):

        return unicode(self.name) # don't forget to force unicode conversion on property

4. key_id and key_name

It is not clear when I have to use key_id or key_name on the model and how to migrate from one to another. What do you think?

5.Model validation and required property

Quote

If True, the property cannot have a value of None. A model instance must initialize all required properties in its constructor so that the instance is not created with missing values. An attempt to create an instance without initializing a required property, or an attempt to assign None to a required property, raises a BadValueError.

I am not happy with this, because I definitely want most properties to present in datastore, but I also want to be able to autogenerate missing properties (and key_name) in pre_put hook. What do you think?