Performance should always be a consideration when developing any web application. Users expect a fast response and minimal delay when interacting with your application. While we are already utilising a highly scalable cloud setup, and a high performance datastore, we can further improve our performance by caching data. Reads and writes to datastore are higher in performance than most standard database systems, however reading and writing from datastore can be expensive in the context of performance. If we are able to remove some of the load on our datastore by serving data directly from our cache stored in memory, we stand to improve our application’s speed by serving directly from memory, and by freeing up our datastore for handling tasks that are datastore critical.

So what can we cache?

Any data that does not change frequently, or does not require complex queries to retrieve is ideal for caching. For example, you may have a datastore driven navigation system that stores details about pages and routes within your application. It is not likely that these will change often in your application, however the application will need the relevant data on nearly every page load. If we can cache this data, we can save ourselves a huge amount of datastore calls.

Caching Considerations

While caching is a great idea, it is important we handle our caching carefully. Google has many many warnings listed in the documentation stating that at any time the cache may be flushed (emptied), and all data stored in the cache may be lost. This is not ideal, as we may be relying on the data in order to successfully load a page. With this in mind, we need to be able to handle cache failures gracefully. When caching datastore information, we need to ensure that if the data we need is not found in the cache, we automatically look for the data in the datastore. This way we benefit from the speed and performance of the cache, but gain the reliability of datastore.

How do we cache our data?

Google App Engine uses Memcached to cache data. Memcached is a shared resource, which is available between all instances of our application. This means that if a user is served by instance A in one request, and instance B in the next request, cached data will be shared between instances. Memcached stored key, value pairs, essentially we can set a key, and set some data to be stored. You cannot query Memcached, you can only request information based on a known key. In order to achieve this, we will need to come up with a way to generate predictable keys, based on the data we are trying to store. For example, if we are going to store a User object, and the user’s datastore KeyID is 1234, we may set the datastore key as User_1234. In order to handle different object types and ID’s, we can create a caching class that can handle the key generation and storage/retrieval tasks for us.

Lets start by defining some unit tests for our key generation functions.

<?PHP // tests/AppCacheTest.php
	
class AppCacheTest extends BuziTest{
	public $Cache;
	
	public function setUp(){
		$this->Cache = Container::newAppCache();
	}
	
	public function testCanGenerateKeyFromEntityWithKeyId(){
		$User = new MockEntity();
		$User->setKeyID(md5(rand(1,1000)));
		
		$GenKey = $this->Cache->generateKeyFromEntity($User);
		$this->assertEquals('MockEntity_KeyId_'.$User->getKeyId(), $GenKey, 'The generated cache key does not match the expected cache key.');
	}
	
	public function testCanGenerateKeyFromEntityWithKeyName(){
		$User = new MockEntity();
		$User->setKeyName('mattpresland');
		
		$GenKey = $this->Cache->generateKeyFromEntity($User);
		$this->assertEquals('MockEntity_KeyName_'.$User->getKeyName(), $GenKey, 'The generated cache key does not match the expected cache key.');
	}
	
	public function testGenerateKeyFromEntityThrowsExceptionForNonEntity(){
		$User = 1;
		$this->setExpectedException('InvalidArgumentException', 'Invalid Argument');

		$GenKey = $this->Cache->generateKeyFromEntity($User);
		
				
		$User = 'Not an object';
		$this->setExpectedException('InvalidArgumentException','Invalid Argument');

		$GenKey = $this->Cache->generateKeyFromEntity($User);
		
				
		$User = false;
		$this->setExpectedException('InvalidArgumentException','Invalid Argument');
		$GenKey = $this->Cache->generateKeyFromEntity($User);
		
		
		
		$User = null;
		$this->setExpectedException('InvalidArgumentException','Invalid Argument');
		$GenKey = $this->Cache->generateKeyFromEntity($User);
	}
	
	public function testGenerateKeyFromEntityReturnsFalseWhenNoKeySet(){
		$User = new MockEntity();
		$User->Name = 'mattpresland';
		
		$this->assertFalse($this->Cache->generateKeyFromEntity($User), 'Generate key from entity does not return false if no key has been set');
	}

}

?>

We have set up 4 tests to ensure our cache keys are being generated correctly:

  • Can we generate a key from an entity with the KeyId set?
  • Can we generate a key from an entity with the KeyName set?
  • Does our function throw an exception if a non-entity is passed to the generate key function?
    • Integer
    • String
    • Boolean
    • Null
  • Does our function return false if no key has yet been set?

We can now begin writing our class and function to meet the requirements of our tests.

 

<?PHP // core/appcache.class.php
	
class AppCache{
	
	private $Cache;
	
	public function __construct($Cache){
		$this->setCache($Cache);
	}
	
	public function setCache($Cache){
		$this->Cache = $Cache;
	}
	
	public function generateKeyFromEntity($Object){
		if(is_a($Object, 'GDSEntity')){
			$ID = $this->getEntityKey($Object);
			$Type = get_class($Object);
			if(!$ID){
				return false;
			} else{
				$Key = $Type.'_'.$ID['Type'].'_'.$ID['Value'];
				return $Key;
			}
			
		} else{
			throw new InvalidArgumentException('Invalid Argument');
		}
		
	}
	
	public function getEntityKey($Entity){
		$KeyName = $Entity->getKeyName();
		$KeyID = $Entity->getKeyID();
		if(isset($KeyID)){
			$Key['Type'] = 'KeyId';
			$Key['Value'] = $KeyID;
			return $Key;
		} else{
			if(isset($KeyName)){
				$Key['Type'] = 'KeyName';
				$Key['Value'] = $KeyName;
				return $Key;
			} else{
				return false;
			}
		}
	}
}

	
?>

We have defined our AppCache class, and created a private $Cache variable to store our connection to Memcached. In our __construct() method we have simply stored the provided cache in our $Cache variable. At the end of the current file, we have created a helper method that returns an array with the key type and key value, from an entity we provide to the function. We used this exact method in our MockStore in the Testing Datastore article.

We have also defined a generateKeyFromEntity() method that will generate the Cache key from the datastore entity provided. The first check we do is to confirm if the $Object variable that has been passed in is an instance of the GDS\Entity class, or is an instance of a class that inherits/extends the GDS\Entity class. This will ensure we have a valid entity to perform our work on. If the $Object variable is not an Entity, we throw an exception, and this will satisfy our test for throwing an exception if a non-entity is provided.

Now that we know we are working with a valid entity, we can try and obtain the entity’s key type and value using our getEntityKey() method. If false is returned, it means that a key has not yet been set, so we cannot store this entity in our cache (we can, however this would not be an ideal solution as we want a re-producable key that can be generated with no knowledge of the contents of the entity), so we return false so our application can skip storing this entity in the cache. If we have a valid key returned to us, we can get the type of entity by requesting the class of the object. Next, we construct our key, which will be in the format EntityType_KeyType_KeyValue, and we can return our generated key value. If we save, and update our composer autoload using the composer dump command, we should be able to run our tests against our new generateKeyFromEntity() method.

Success! We can now rely on our generateKeyFromEntity() method to either provide a valid cache storage key, or return false indicating we do not need to store this entity at this point.

Next, we need to store our entity with our generated key. We will write a method that can handle saving the entity in our cache, but first, lets define our criteria/tests.

  • Can we store an entity in cache with valid key
  • Can we store multiple entities provided in an array
  • Does our method return false if the entity provided is not valid
  • Does our method return false if the entity provided cannot be stored in cache due to not having a key set.
<?PHP // tests/AppCacheTest.php
	
class AppCacheTest extends BuziTest{
	public $Cache;
	
	public function setUp(){
		$this->Cache = Container::newAppCache();
		$this->Cache->setCache(new MockCache());
	}
	
	public function testCanGenerateKeyFromEntityWithKeyId(){
		$User = new MockEntity();
		$User->setKeyID(md5(rand(1,1000)));
		
		$GenKey = $this->Cache->generateKeyFromEntity($User);
		$this->assertEquals('MockEntity_KeyId_'.$User->getKeyId(), $GenKey, 'The generated cache key does not match the expected cache key.');
	}
	
	public function testCanGenerateKeyFromEntityWithKeyName(){
		$User = new MockEntity();
		$User->setKeyName('mattpresland');
		
		$GenKey = $this->Cache->generateKeyFromEntity($User);
		$this->assertEquals('MockEntity_KeyName_'.$User->getKeyName(), $GenKey, 'The generated cache key does not match the expected cache key.');
	}
	
	public function testGenerateKeyFromEntityThrowsExceptionForNonEntity(){
		$User = 1;
		$this->setExpectedException('InvalidArgumentException', 'Invalid Argument');

		$GenKey = $this->Cache->generateKeyFromEntity($User);
		
				
		$User = 'Not an object';
		$this->setExpectedException('InvalidArgumentException','Invalid Argument');

		$GenKey = $this->Cache->generateKeyFromEntity($User);
		
				
		$User = false;
		$this->setExpectedException('InvalidArgumentException','Invalid Argument');
		$GenKey = $this->Cache->generateKeyFromEntity($User);
		
		
		
		$User = null;
		$this->setExpectedException('InvalidArgumentException','Invalid Argument');
		$GenKey = $this->Cache->generateKeyFromEntity($User);
	}
	
	public function testGenerateKeyFromEntityReturnsFalseWhenNoKeySet(){
		$User = new MockEntity();
		$User->Name = 'mattpresland';
		
		$this->assertFalse($this->Cache->generateKeyFromEntity($User), 'Generate key from entity does not return false if no key has been set');
	}
	
	public function testCanSaveEntityInCache(){
		$User = new MockEntity();
		$User->Name = 'Matt Presland';
		$User->setKeyID(md5(rand(1,1000)));
		
		$this->Cache->saveEntity($User);

		$MockCache = $this->Cache->getCache();
		var_dump($MockCache->Items);
		$this->assertEquals(1, sizeof($MockCache->Items), 'The cache does not contain the expected number of items');
	}
	
	public function testCanSaveArrayOfEntities(){
		$User = new MockEntity();
		$User->Name = 'Matt Presland';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Mickey Mouse';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Donald Duck';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		
		$this->Cache->saveEntities($Users);
		$MockCache = $this->Cache->getCache();
		$this->assertEquals(3, sizeof($MockCache->Items), 'The cache does not contain the expected number of items');
	}
	
	public function testCanSaveSingleEntityUsingSaveEntities(){
		$User = new MockEntity();
		$User->Name = 'Matt Presland';
		$User->setKeyID(md5(rand(1,1000)));
		
		$this->Cache->saveEntities($User);

		$MockCache = $this->Cache->getCache();
		var_dump($MockCache->Items);
		$this->assertEquals(1, sizeof($MockCache->Items), 'The cache does not contain the expected number of items');
	}
	
	public function testSaveEntityReturnsFalseIfNonEntityProvided(){
		$User = 1;
		$this->assertFalse($this->Cache->saveEntity($User), 'Expecting false to be returned with integer provided');
			
		$User = 'Not an object';
		$this->assertFalse($this->Cache->saveEntity($User), 'Expecting false to be returned with integer provided');
						
		$User = false;
		$this->assertFalse($this->Cache->saveEntity($User), 'Expecting false to be returned with integer provided');
		
		$User = null;
		$this->assertFalse($this->Cache->saveEntity($User), 'Expecting false to be returned with integer provided');
	}
	
	public function testSaveEntityReturnsFalseIfNoEntityKeySet(){
		$User = new MockEntity();
		$User->Name = 'Matt Presland';
		
		$this->assertFalse($this->Cache->saveEntity($User), 'Expecting false to be returned with integer provided');
	}

}

?>

We now have a set of tests to test 2 methods, saveEntity() and saveEntities(). We will use the saveEntity() method to store each individual entity, and the saveEntities() method to store an array of entities. You will also notice, we have a test defined to test if we can save a single entity using the saveEntities() method. This will allow us to use the saveEntities() method anywhere we need to save either an array or a single entity.

To satisfy our tests, our AppCache class should look like the one below.

<?PHP // core/appcache.class.php
	
class AppCache{
	
	private $Cache;
	
	public function __construct($Cache){
		$this->setCache($Cache);
	}
	
	public function setCache($Cache){
		$this->Cache = $Cache;
	}
	
	public function getCache(){
		return $this->Cache;
	}
	
	public function generateKeyFromEntity($Object){
		if(is_a($Object, 'GDSEntity')){
			$ID = $this->getEntityKey($Object);
			if(!$ID){
				return false;
			} else{
				$Type = get_class($Object);
				$Key = $Type.'_'.$ID['Type'].'_'.$ID['Value'];
				return $Key;
			}
			
		} else{
			throw new InvalidArgumentException('Invalid Argument');
		}
		
	}
	
	public function getEntityKey($Entity){
		$KeyName = $Entity->getKeyName();
		$KeyID = $Entity->getKeyID();
		if(isset($KeyID)){
			$Key['Type'] = 'KeyId';
			$Key['Value'] = $KeyID;
			return $Key;
		} else{
			if(isset($KeyName)){
				$Key['Type'] = 'KeyName';
				$Key['Value'] = $KeyName;
				return $Key;
			} else{
				return false;
			}
		}
	}
	
	public function saveEntity($Entity){
		try {
			$Key = $this->generateKeyFromEntity($Entity);
		} catch(Exception $e){
			$Key = false;
			//Log Non-entity provided to saveEntity()
		}
		if(!$Key){
			return false;
		} else{
			$this->Cache->set($Key, $Entity);
		}
	}
	
	public function saveEntities($Entities){
		if(!is_array($Entities)){
			$NewArray[] = $Entities;
			unset($Entities);
			$Entities = $NewArray;
		}
		foreach($Entities AS $Entity){
			$this->saveEntity($Entity);
		}
	}
}

	
?>

We have added our saveEntity() method, which accepts a single entity as input. The first task for our method is to try to get a key for the entity provided. If the data provided is not a valid entity, and exception will be thrown, and we will catch the exception, and set the key to false. You will notice I have left a comment here to log the exception. Our next article will discuss error logging, and we will complete our logging then. Once we have our key, we then check to make sure the key is valid, if the key is invalid, we return false to indicate we have not stored any data. If the key is valid, we use Memcached’s set($Key, $Data) method to store our entity.

The saveEntities() method accepts an array of entities. We first ensure we were given an array of data. If not, we create an array with the provided data as the first item in the array. We can then loop through the array using foreach, and use our saveEntity() method to save the entities. Our saveEntity method and getEntityKey method both check to ensure our data is a valid entity before storing the entity.

Our next task is to fetch data from our cache. When we looked at datastore, we had 4 methods that could fetch data based on either KeyId, or KeyName. To match our cache up with our datastore, we will create a set of matching cache methods. We will only need 2 methods, as we can specify KeyName or KeyId as one of the parameters, and use this to re-generate our cache Key. First, we need to define our tests to ensure our methods work as expected.

Our tests:

  • Can we generate a cache key from a request?
  • Can we fetch a single entity?
  • Can we fetch multiple entities?
  • Can we fetch a single entity using fetchEntities?
  • Does fetchEntity return false if no entity is found?
  • Does fetchEntities return false if no entities are found?
// tests/AppCacheTest.php -- additions only --

	public function testCanGenerateKeyFromFetchRequest(){
		$Key = $this->Cache->generateKeyFromFetchRequest('User', 'KeyName', 'mattpresland');
		$this->assertEquals('User_KeyName_mattpresland', $Key, 'The generated key does not match the expected key');
		
		$Key = $this->Cache->generateKeyFromFetchRequest('User', 'KeyId', '123456789');
		$this->assertEquals('User_KeyId_123456789', $Key, 'The generated key does not match the expected key');
	}
	
	public function testCanFetchSingleEntity(){
		$User = new MockEntity();
		$User->Name = 'Matt Presland';
		$User->setKeyName('mattpresland');
		$this->Cache->saveEntity($User);
		
		$Result = $this->Cache->fetchEntity('MockEntity', 'KeyName', 'mattpresland');
		$this->assertEquals($User, $Result, 'The Fetched entity does not match the expected entity');
	}
	
	public function testCanFetchEntities(){
		$User = new MockEntity();
		$User->Name = 'Matt Presland';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Mickey Mouse';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Donald Duck';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		
		$this->Cache->saveEntities($Users);
		
		$KeyValues[] = $Users[0]->getKeyId();
		$KeyValues[] = $Users[1]->getKeyId();
		$KeyValues[] = $Users[2]->getKeyId();
		
		$Result = $this->Cache->fetchEntities('MockEntity', 'KeyId', $KeyValues);
		
		$this->assertEquals($Users, $Result, 'The results do not match the expected results');
	}
	
	public function testCanFetchSingleEntityUsingFetchEntities(){
		$User = new MockEntity();
		$User->Name = 'Matt Presland';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Mickey Mouse';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Donald Duck';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		
		$this->Cache->saveEntities($Users);
		
		$Result = $this->Cache->fetchEntities('MockEntity','KeyId',$Users[1]->getKeyId());
		$this->assertEquals($Users[1], $Result[0], 'The fetched entity does not match the expected entity');

	}
	
	public function testFetchEntityReturnsFalseIfEntityNotFound(){
		$User = new MockEntity();
		$User->Name = 'Matt Presland';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Mickey Mouse';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Donald Duck';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		
		$this->Cache->saveEntities($Users);
		
		$this->assertFalse($this->Cache->fetchEntity('MockEntity', 'KeyId', '1234'), 'Fetch entity does not return false if entity not found');
	}
	
	public function testFetchEntitiesReturnsFalseIfEntitiesNotFound(){
		$User = new MockEntity();
		$User->Name = 'Matt Presland';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Mickey Mouse';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Donald Duck';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		
		$this->Cache->saveEntities($Users);
		$Keys[] = 1234;
		$Keys[] = 456789;
		
		$Result = $this->Cache->fetchEntities('MockEntity','KeyId',$Keys);
		var_dump($Result);
		
		$this->assertFalse($this->Cache->fetchEntities('MockEntity', 'KeyId', $Keys), 'Fetch entities does not return false if entity not found');
	}

We can now write the generateKeyFromFetchRequest(), fetchEntity() and fetchEntities() methods.

// core/appcache.class.php -- additions only --

	public function fetchEntity($EntityType, $KeyType, $KeyValue){
		$Key = $this->generateKeyFromFetchRequest($EntityType, $KeyType, $KeyValue);
		return $this->Cache->get($Key);
	}
	
	public function fetchEntities($EntityType, $KeyType, $KeyValues){
		if(!is_array($KeyValues)){
			$NewArray[] = $KeyValues;
			unset($KeyValues);
			$KeyValues = $NewArray;
		}
		foreach($KeyValues AS $KeyValue){
			$Entity = $this->fetchEntity($EntityType, $KeyType, $KeyValue);
			if($Entity != false){
				$Entities[] = $Entity;
			}
		}
		if(isset($Entities)){
			return $Entities;
		} else{
			return false;
		}
	}
	
	public function generateKeyFromFetchRequest($EntityType, $KeyType, $KeyValue){
		return $EntityType.'_'.$KeyType.'_'.$KeyValue;
	}

Our generateKeyFromFetchRequest() method simply takes the Entity type, the Key Type, and the Key value and combines them to re-create the cache Key that the entity would have been stored with. Our fetchEntity() method, calls on the generateKeyFromFetchRequest() method to obtain the cache key for the requested entity, and then uses Memcached’s get() method to retrieve the entity from our cache. We then return the fetched data. If no item was found in cache, the cache will return false, and our fetchEntity() method will also return false to indicate to our application that the item was not found.

Our fetchEntities() method checks to ensure that the array of KeyValues is an array, if not it wraps a single KeyValue in an array. We then loop through each KeyValue in the array to fetchEntity(), and if we receive a valid response, we add the entity to an array. If we have valid entities in our array, we return the array. If not, we return false to indicate that no entities were found.

We can now run our tests to ensure our methods work as expected.

Success! We now have a caching class that is able to store and retrieve entities. We may wish to expand this class later to handle data other than entities, however at this point this class will assist us with caching our datastore interactions. We can now use this class, in combination with our datastore Store class to automatically handle caching for our datastore entities.

We will run the following tests on our new class to ensure it functions as expected:

  • Can save single entity
  • Can save multiple entities
  • Can fetch by Id
  • Can fetch by Ids
  • Can fetch by name
  • Can fetch by names
  • Can fetch by Id with entity missing from cache
  • Can fetch by Ids with entities missing from cache
  • Can fetch by name with entity missing from cache
  • Can fetch by names with entities missing from cache
  • Can fetch by ids with some entities missing from cache
  • Can fetch by names with some entities missing from cache
  • Fetch by id returns false when no entity found
  • Fetch by ids returns empty array when no entity found
  • Fetch by name returns false when no entity found
  • Fetch by names returns empty array when no entity found

This is a long list of tests, however they are necessary to ensure our new class can handle situations where we are able to fetch from the cache, where we are not able to fetch from the cache, and where we can fetch part data from our cache, and the rest from our store. I won’t list the code for the tests just yet, we will have a look at the class we are working on first, and I will include the full tests at the end of the article.

We now need to write our class to pass all of those tests!

<?PHP // core/cachestore.class.php

class CacheStore {
	public $AppCache;
	public $Store;
	
	public function __construct($Store, $Cache){
		$this->AppCache = $Cache;
		$this->Store = $Store;
	}
	
	public function upsert($Entities){
		$this->AppCache->saveEntities($Entities);
		$this->Store->upsert($Entities);
	}
	
	public function fetchById($ID){
		$Result = $this->AppCache->fetchEntity($this->Store->obj_schema->getKind(), 'KeyId', $ID);
		if(!$Result){
			$Result = $this->Store->fetchById($ID);
		}
		return $Result;
	}
	
	public function fetchByIds($IDs){
		if(!is_array($IDs)){
			$NewArray[] = $IDs;
			unset($IDs);
			$IDs = $NewArray;
		}
		$Result = $this->AppCache->fetchEntities($this->Store->obj_schema->getKind(), 'KeyId', $IDs);
		if(!$Result){
			$Result = $this->Store->fetchByIds($IDs);
		} else{
			if(sizeof($Result) != sizeof($IDs)){
				foreach($Result AS $Item){
					$FoundIDs[] = $Item->getKeyId();
				}
				$UnfoundIDs = array_diff($IDs, $FoundIDs);
				$NewResults = $this->Store->fetchByIds($UnfoundIDs);
				$Result = array_merge($Result, $NewResults);
			}
		}
		return $Result;
	}
	
	public function fetchByName($Name){
		$Result = $this->AppCache->fetchEntity($this->Store->obj_schema->getKind(), 'KeyName', $Name);
		if(!$Result){
			$Result = $this->Store->fetchByName($Name);
		}
		return $Result;
	}
	
	public function fetchByNames($Names){
		if(!is_array($Names)){
			$NewArray[] = $Names;
			unset($Names);
			$Names = $NewArray;
		}
		$Result = $this->AppCache->fetchEntities($this->Store->obj_schema->getKind(), 'KeyName', $Names);
		if(!$Result){
			$Result = $this->Store->fetchByNames($Names);
		} else{
			if(sizeof($Result) != sizeof($Names)){
				foreach($Result AS $Item){
					$FoundNames[] = $Item->getKeyName();
				}
				$UnfoundNames = array_diff($Names, $FoundNames);
				$NewResults = $this->Store->fetchByNames($UnfoundNames);
				$Result = array_merge($Result, $NewResults);
			}
		}
		return $Result;
	}
	
}
	
?>

Our new CacheStore class accepts a Store and a Cache object when constructed. We simply set them to class variables, so we can access them from our class methods. Our first method, upsert() is quite simple, all we do is save our entities to both our AppCache, and to our Store. The entities should now be available both in the cache and datastore.

Next, we have written a fetchById() method, which first searches the cache for our entity, by passing in the Entity type, the KeyType and the KeyValue. Note here, we have auto-detected our EntityType using the Store’s obj_schema->getKind() method. When a store (in production) is instantiated, a schema is provided which gives us this information. We then test to see if we received a result, and if not, we then search our datastore for the entity. We then return whatever we find, if we have not found anything, false will be returned by the store.

Our fetchByIds() method checks to ensure it was given an array of Ids, otherwise it produces an array from the data is was provided. We then search our cache for the Ids. If we found no results, we then query our store for all the ids. If we received some results from the cache, but the number of results doesn’t match the number of Ids we searched, we make a list of the Id’s we did find, subtract them from the Ids provided, and search for the unfound Ids in our store. We then combine the results arrays and return the complete result set.

We can then write very similar methods for our fetchByName() and fetchByNames() methods, by simply changing the KeyId fields to KeyName and using the fetchByName() and fetchByNames() methods where appropriate.

That’s all there is to it! We first search our cache as we know the cache is much quicker, and if we don’t find what we are looking for, we search again in the datastore. This gives us the speed of cache, but the reliability of our datastore. We can now run our tests on this class, and ensure it all works as expected.

Success! We now have a way of automatically using both the cache and datastore, with no more effort required. Now any time we need a store with caching abilities, we can extend our new CacheStore class instead of the GDS\Store class.

In order to test the new class, I created a MockCacheStore class in the tests directory, which set an instance of our AppCache class, with a MockCache instance as its cache connection. I also set an instance of our MockStore class as the Store. Also in this mock class, I included helper functions to return the full cache contents, and the full datastore contents for testing, as well as a flushCache method to allow testing with an empty cache.

From there, We can run our tests on the MockCacheStore. I have included the code for both the MockCacheStore and CacheStoreTest files below.

<?PHP	// tests/mockcachestore.class.php

class MockCacheStore extends CacheStore{
	
	public $Cache;
	public $Gateway;
		
	public function buildSchema(){
		$Schema = new GDSSchema('MockEntity');
		$Schema->addString('Name', FALSE);
		return $Schema;
	}
	
	public function __construct(){
		$this->Cache = new AppCache(new MockCache());
		$this->Gateway = new MockGateway(null, null);
		$this->Store = new MockStore($this->Gateway);
		
		parent::__construct($this->buildSchema(), $this->Gateway, $this->Cache);
	}
	
	public function getCache(){
		return $this->AppCache->getCache();
	}
	
	public function getStore(){
		return $this->Gateway;
	}
	
	public function flushCache(){
		$this->AppCache->setCache(new MockCache());
	}
}
	
?>
	
<?PHP	// tests/CacheStoreTest.php

class CacheStoreTest extends BuziTest{
	
	public $CacheStore;
	public $StandardEntities;
	
	public $Gateway;
		
	public function setUp(){
		$this->Gateway = new MockGateway(null, null);
		$this->CacheStore = new MockCacheStore();
	}
		
	public function testCanSaveSingleEntity(){
		$User = new MockEntity();
		$User->Name = 'Matt Presland';
		$User->setKeyId('mattpresland');
		
		$this->CacheStore->upsert($User);
		
		$MockCache = $this->CacheStore->getCache();
		$MockStore = $this->CacheStore->getStore();
		
		$this->AssertEquals(1, sizeof($MockCache->Items), 'The size of the cache does not match the expected cache size');
		$this->AssertEquals(1, sizeof($MockStore->Entities), 'The size of the store does not match the expected store size');
				
		$this->AssertEquals($User, $MockCache->Items['MockEntity_KeyId_mattpresland'], 'The saved entity does not match the expected entity');
		$this->AssertEquals($User, $MockStore->Entities[0], 'The saved entity does not match the expected entity');
	}
	
	public function testCanSaveMultipleEntities(){
		$User = new MockEntity();
		$User->Name = 'Matt Presland';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Mickey Mouse';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Donald Duck';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		
		$this->CacheStore->upsert($Users);
		
		$MockCache = $this->CacheStore->getCache();
		$MockStore = $this->CacheStore->getStore();
		
		$this->AssertEquals(3, sizeof($MockCache->Items), 'The size of the cache does not match the expected cache size');
		$this->AssertEquals(3, sizeof($MockStore->Entities), 'The size of the store does not match the expected store size');
		
		$this->AssertEquals($Users[0], $MockCache->Items['MockEntity_KeyId_'.$Users[0]->getKeyId()], 'The saved entity does not match the expected entity');
		$this->AssertEquals($Users[1], $MockCache->Items['MockEntity_KeyId_'.$Users[1]->getKeyId()], 'The saved entity does not match the expected entity');
		$this->AssertEquals($Users[2], $MockCache->Items['MockEntity_KeyId_'.$Users[2]->getKeyId()], 'The saved entity does not match the expected entity');
		
		$this->AssertEquals($Users[0], $MockStore->Entities[0], 'The saved entity does not match the expected entity');
		$this->AssertEquals($Users[1], $MockStore->Entities[1], 'The saved entity does not match the expected entity');
		$this->AssertEquals($Users[2], $MockStore->Entities[2], 'The saved entity does not match the expected entity');
	}
	
	public function testCanFetchById(){
		$this->addStandardEntities();
		
		$Result = $this->CacheStore->fetchById($this->StandardEntities[1]->getKeyId());
		
		$this->assertEquals($this->StandardEntities[1], $Result, 'The fetched entity does not match the expected entity');
	}
	
	public function testCanFetchByIds(){
		$this->addStandardEntities();
		
		$IDs[] = $this->StandardEntities[1]->getKeyId();
		$IDs[] = $this->StandardEntities[2]->getKeyId();
		
		$Result = $this->CacheStore->fetchByIds($IDs);

		$this->assertEquals($this->StandardEntities[1], $Result[0], 'The fetched entity does not match the expected entity');
		$this->assertEquals($this->StandardEntities[2], $Result[1], 'The fetched entity does not match the expected entity');		
		
	}
	
	public function testCanFetchByName(){
		$this->addStandardEntities();
		
		$Result = $this->CacheStore->fetchByName('minniemouse');
		
		$this->assertEquals($this->StandardEntities[3], $Result, 'The fetched entity does not match the expected entity');
	}
	
	public function testCanFetchByNames(){
		$this->addStandardEntities();
	
		$Names[] = 'daffyduck';
		$Names[] = 'porkypig';
		
		$Result = $this->CacheStore->fetchByNames($Names);

		$this->assertEquals($this->StandardEntities[4], $Result[0], 'The fetched entity does not match the expected entity');
		$this->assertEquals($this->StandardEntities[5], $Result[1], 'The fetched entity does not match the expected entity');
	}
	
	public function addStandardEntities(){
		$User = new MockEntity();
		$User->Name = 'Matt Presland';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Mickey Mouse';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Donald Duck';
		$User->setKeyId(md5(rand(1,1000)));
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Minnie Mouse';
		$User->setKeyName('minniemouse');
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Daffy Duck';
		$User->setKeyName('daffyduck');
		$Users[] = $User;
		$User = new MockEntity();
		$User->Name = 'Porky Pig';
		$User->setKeyName('porkypig');
		$Users[] = $User;
		
		$this->StandardEntities = $Users;
		$this->CacheStore->upsert($Users);
	}
	
	public function testCanFetchByIdWithEntityMissingFromCache(){
		$this->addStandardEntities();
		
		$this->CacheStore->flushCache();
		
		$MockCache = $this->CacheStore->getCache();
		$MockStore = $this->CacheStore->getStore();
		
		//assert cache is empty
		$this->assertEquals(0, sizeof($MockCache->Items), 'The Cache is supposed to be empty');
		
		//try and fetch an entity
		$Result = $this->CacheStore->fetchById($this->StandardEntities[1]->getKeyId());
		
		$this->assertEquals($this->StandardEntities[1], $Result, 'The fetched entity does not match the expected entity');
	}
	
	public function testCanFetchByIdsWithEntitiesMissingFromCache(){
		$this->addStandardEntities();
		
		$IDs[] = $this->StandardEntities[1]->getKeyId();
		$IDs[] = $this->StandardEntities[2]->getKeyId();
		
		$this->CacheStore->flushCache();
		
		//assert cache is empty
		$MockCache = $this->CacheStore->getCache();
		$this->assertEquals(0, sizeof($MockCache->Items), 'The Cache is supposed to be empty');
		
		$Result = $this->CacheStore->fetchByIds($IDs);

		$this->assertEquals($this->StandardEntities[1], $Result[0], 'The fetched entity does not match the expected entity');
		$this->assertEquals($this->StandardEntities[2], $Result[1], 'The fetched entity does not match the expected entity');	
	}
	
	public function testCanFetchByNameWithEntityMissingFromCache(){
		$this->addStandardEntities();
		
		$this->CacheStore->flushCache();
		
		$MockCache = $this->CacheStore->getCache();
		//assert cache is empty
		$this->assertEquals(0, sizeof($MockCache->Items), 'The Cache is supposed to be empty');
		
		//try and fetch an entity
		$Result = $this->CacheStore->fetchByName('minniemouse');
		
		$this->assertEquals($this->StandardEntities[3], $Result, 'The fetched entity does not match the expected entity');	
	
	}
	
	public function testCanFetchByNamesWithEntitiesMissingFromCache(){
		$this->addStandardEntities();
		
		$Names[] = 'daffyduck';
		$Names[] = 'porkypig';
		
		$this->CacheStore->flushCache();
		
		//assert cache is empty
		$MockCache = $this->CacheStore->getCache();
		$this->assertEquals(0, sizeof($MockCache->Items), 'The Cache is supposed to be empty');
		
		$Result = $this->CacheStore->fetchByNames($Names);

		$this->assertEquals($this->StandardEntities[4], $Result[0], 'The fetched entity does not match the expected entity');
		$this->assertEquals($this->StandardEntities[5], $Result[1], 'The fetched entity does not match the expected entity');	
	}
	
	public function testCanFetchByIDsWithSomeEntitiesMissingFromCache(){
		$this->addStandardEntities();
		
		$this->CacheStore->flushCache();
		
		$MockCache = $this->CacheStore->getCache();
		//assert cache is empty
		$this->assertEquals(0, $MockCache->Items, 'The Cache is supposed to be empty');
		
		$User = new MockEntity();
		$User->Name = 'Felix the Cat';
		$User->setKeyId(md5(rand(1,1000)));
		
		$this->CacheStore->upsert($User);
		
		$MockCache = $this->CacheStore->getCache();
		//assert cache has one entity
		$this->assertEquals(1, sizeof($MockCache->Items), 'The Cache is supposed to have 1 item');
		
		$IDs[] = $User->getKeyId();
		$IDs[] = $this->StandardEntities[1]->getKeyId();
		
		//try and fetch an entity
		$Result = $this->CacheStore->fetchByIds($IDs);
		
		$this->assertEquals(2, sizeof($Result), 'number of fetched entities does not match');
		
		//the first result should be the one found in cache
		$this->assertEquals($User, $Result[0], 'The entity does not match the expected entity');
		//the second result should be the one found in the store
		$this->assertEquals($this->StandardEntities[1], $Result[1], 'The entity does not match the expected entity');	
	}
	
	public function testCanFetchByNamesWithSomeEntitiesMissingFromCache(){
		$this->addStandardEntities();
		
		$this->CacheStore->flushCache();
		
		$MockCache = $this->CacheStore->getCache();
		//assert cache is empty
		$this->assertEquals(0, $MockCache->Items, 'The Cache is supposed to be empty');
		
		$User = new MockEntity();
		$User->Name = 'Felix the Cat';
		$User->setKeyName('felixthecat');
		
		$this->CacheStore->upsert($User);
		
		$MockCache = $this->CacheStore->getCache();
		//assert cache has one entity
		$this->assertEquals(1, sizeof($MockCache->Items), 'The Cache is supposed to have 1 item');
		
		$Names[] = 'minniemouse';
		$Names[] = 'felixthecat';
		
		//try and fetch an entity
		$Result = $this->CacheStore->fetchByNames($Names);
		
		$this->assertEquals(2, sizeof($Result), 'number of fetched entities does not match');
		
		//the first result should be the one found in cache
		$this->assertEquals($User, $Result[0], 'The entity does not match the expected entity');
		//the second result should be the one found in the store
		$this->assertEquals($this->StandardEntities[3], $Result[1], 'The entity does not match the expected entity');	
	}
	
	public function testFetchByIdReturnsFalseWhenNoEntityFound(){
		$this->addStandardEntities();
		
		$Result = $this->CacheStore->fetchById('1234');
		$this->assertFalse($Result, 'Expecting fetch to return false when no entity is found');
	}
	
	public function testFetchByIdsReturnsEmptyArrayWhenNoEntityFound(){
		$this->addStandardEntities();
		
		$Result = $this->CacheStore->fetchByIds(['1234']);
		$this->assertEquals(0,sizeof($Result), 'Expecting fetch to return false when no entity is found');
	}
	
	public function testFetchByNameReturnsFalseWhenNoEntityFound(){
		$this->addStandardEntities();
		
		$Result = $this->CacheStore->fetchByName('donaldtrump');
		$this->assertFalse($Result, 'Expecting fetch to return false when no entity is found');
	}
	
	public function testFetchByNamesReturnsEmptyArrayWhenNoEntityFound(){
		$this->addStandardEntities();
		
		$Result = $this->CacheStore->fetchByNames(['donaldtrump']);
		$this->assertEquals(0,sizeof($Result), 'Expecting fetch to return false when no entity is found');
	}
	
		
}
	
?>

This was a long, in-depth article, it is worth the effort though, as we can now handle cache and datastore interactions seamlessly with minimal effort throughout the rest of our application. In the next article, we will look at how we can log exceptions and errors in our application for debugging.