Bellicose Beliefs: Multiuser Chatroom with App Engine Channel API

The Part-1 of designing multiuser chat room with app engine deals with providing a basic information of how to create a simple chatroom/gameroom with App Engine Channel API. That article was more based on the tic-tac-toe example provided by the Google App Engine team.

In this post, I will talk about the ways in which this can be optimized using memcache.

But before that, let's look at the complications with the previous one

The Channel ID is a function of only userid. This means that one user can not login from multiple clients and can expect consistency. There will always be an inconsistency and improper outgoing messages on the channels.
Too much datastore operations. For every action, there is too much datastore operations going on. As the number of players keep on increasing, this problem becomes more intense. A developer reported that he had to wait several seconds for things to happen while doing it with ~30 players. This is an optimized version and will have much better serving time
Coding style. Well frankly, I was not satisfied with the coding style in prev version, so in the hope of creating better and beautiful code, I decided to rewrite it and I came up with this new file tournament.py, which is pretty neat.

First of all, in order to get an insight of effective memcaching in datastore entities, go through this post on Nick's blog. This has been completely used.

EfficientModel

In order to do good optimizations, I have a base class EfficientModel from which my datastore classes will be derived. Entities of kind EfficientModel are designed to be in memcache most of the time they are required. All the operations on these entities take place in memcache only. After the memcached entity has been updated "certain" number of times, the change is replicated in the datastore. The EfficientModel has an attribute mc_version, which stores the version number of this entity in memcache. The concept is that every entity has a revision number associated. Whenever there is any change in one of the attributes/property of the entity it's version number increases by one. mc_version stored the version number of entity in memcache. db_version stores the version number of entity in the datastore. The difference between memcache version and datastore version is called as FAULT NUMBER. When fault_number goes beyond a certain number known as fault_tolerance, then the memcache entity and the datastore entity are sync'd.

Properties, Classmethods and Functions

keyname, property. : Returns the key name of the entity. Equivalent to
```
.key().name()
```
memcache_key, property : The derived class is expected to define this property. This is supposed to return the key which will be used in memcache while storing and retrieving this entity. It is assumed at some places the memcache_key shall be same as keyname.
from_id(id), classmethod : This classmethod returns the entity based on the id passed. It first tries to fetch the entity from memcache. If not found, it attempts to fetch the entity from datastore. If that is not found, then it creates an entity with the key_name as the id passed and returns that.
_from_memcache, private function : Returns the memcached snapshot of the entity
_get, private function : Attempts to get self from memcache. If not found, goes through a get_or_insert call.
fetch_from_id(id), classmethod : The difference between fetch_from_id and from_is is that, fetch_from_id always creates an entity in the datastore if it does not already exists. While in case of from_id a dummy object (one which is not in the datastore, yet) is returned.
sync_from_db, function : Sync the entities from datastore to memcache. Essentially, the entity is copied to memcache from datastore and the versions are updated
sync_to_db, function : Sync the entities from memcache to data store. Essentially, the entity is copied from memcahe to datastore and the versions are updated
_store, private function : Stores the entity in memcache and updates the mc_version. If the fault becomes more than fault tolerance, it syncs the entity across datastore and memcache.
find_parent(id), classmethod : This is not implemented and the derived class is expected to work on this. This is supposed to return the key name of the parent entity of this entity.

Channels

The channel ids created are a function of userid as well as the time.time(). Whenever a channel id is created, it is stored as a property in the Player model as well as the Game model. Duplicacy of data, helps do better reads.

Player Class

The Player Class has more or less the same functions, but their writing style became little different due to change in coding style.

Game Class

The important parts of the class are shown here. For a more detailed version, have a look at the source code.

This is a better optimized version. I am still working on it on a bigger and wider scale and will keep updating about the scaling issues of this approach and how to fix them.