In this series, we have seen datastore pop up in a few places, yet we have not conducted any unit testing on our datastore code. The reason for this, is that we cannot access datastore from the command line, as required by PHPUnit. Datastore is only available through the localhost port provided by the Google App Engine SDK. How do we test our datastore items?

The only way we are able to test our datastore services is to mock, or fake our connection to the datastore. Instead of using the PHP-GDS gateway class, we will need to make one that simulates some of the functionality of datastore. For the purposes of our testing, our mock datastore does not need to be complex. What we are mainly interested in, is whether our application sends data to the datastore correctly, and does it process data it retrieves from the datastore correctly. The PHP-GDS library we are using to connect to datastore has its own set of tests, which show it is able to handle the communication between our app and datastore.

To emulate datastore, what do we need to do? We need to be able to store entities, and retrieve entities. We can achieve a similar result with a PHP array, and write our own methods to emulate the gateway’s upsert, delete and fetch methods. With this in mind, we first need an array! Let’s start our MockGateway class now.

<?PHP // tests/mockgateway.class.php

class MockGateway{
	
	public $Entities;
	
	public function __construct($Dataset = null, $Namespace = null){
		$this->Entities = [];
	}
	
?>

Here we have created our class file, mockgateway.class.php and stored it in our tests/ directory. We only require this class for testing, so it makes sense to keep the file with our tests. Within the class we have defined a public property called Entities. This will allow us to manipulate the items in our mock store during our tests. We have also defined a __construct() method, which simply sets our $Entities property to an empty array. We now have an empty, mock datastore. Our next task is to accept entities and insert them, or add them to our store. If you look back to the last article, Google Datastore – A SaaS Experience, we learnt that the upsert() method was responsible for both inserting new entities, and updating existing entities. If you look at the definition of the GDS\Store class, the upsert() method calls putMulti($entities) on the gateway to store items. We can write our own putMulti() method to store our data.

<?PHP // tests/mockgateway.class.php

class MockGateway{
	
	public $Entities;
	
	public function __construct(){
		$this->Entities = [];
	}
	
	public function putMulti($Entities){
		
		if(!is_array($Entities)){
			$NewEntities[] = $Entities;
			unset($Entities);
			$Entities = $NewEntities;
		}
		
		foreach($Entities AS $Entity){
			unset($Key);
			$Key = $this->getEntityKey($Entity);
			if(!$Key){
				$Entity->setKeyId(md5(rand(1, 1000)));
				$this->Entities[] = $Entity;
			} else{
				$ItemKey = $this->searchObjectArray($this->Entities, $Key['Type'], $Key['Value']);
				
				if(!is_int($ItemKey)){
					$this->Entities[] = $Entity;
				} else{
					$this->Entities[$ItemKey] = $Entity;
				}
			}
		}
		return true;
	}
	
	public function searchObjectArray($Array, $Property, $Value){
		foreach($Array AS $Key => $Item){
			if($Property == 'KeyId'){
				$ArrayValue = $Item->getKeyId();
			}
			if($Property == 'KeyName'){
				$ArrayValue = $Item->getKeyName();
			}
			if($ArrayValue == $Value){
				return $Key;
			}
		}
		return false;
	}
	
	public function getEntityKey($Entity){
		$KeyName = $Entity->getKeyName();
		$KeyID = $Entity->getKeyID();
		if(isset($KeyID)){
			$Key['Type'] = 'KeyId';
			$Key['Value'] = $KeyID;
			return $Key;
		} else{
			if(isset($KeyName)){
				$Key['Type'] = 'KeyName';
				$Key['Value'] = $KeyName;
				return $Key;
			} else{
				return false;
			}
		}
	}
?>

There is a lot to digest here, but we will look at everything we have added individually. Firstly, you will see we have added 2 helper methods, that perform some logic on our entities, and on our object array. First, we will look at getEntityKey().

Datastore allows us to store 2 different types of keys, one being an automatically assigned ID KeyId, the other being a key we can set manually, KeyName. In order to work with our entities, we need to be able to detect which type of key is being used, and the value of the key. Our getEntityKey method accepts an entity object, and uses the entity class’s getKeyId(), and getKeyName() methods to find the values of the keys set. We then test to see if KeyID has been set, if so, we return a Key type of KeyId, along with the appropriate KeyId value.

If KeyName is set, we return a Key type of KeyName, with the appropriate value set. If neither KeyID or KeyName has been set, we return false, in order to allow our mock store class to assign a key.

The second helper method we added was searchObjectArray(). The purpose of this method is to search our object array, and return the Array Key, or the position the entity in the array. This will allow us to modify, or retrieve the correct entity in our mock store. To achieve this, we need an array to search, a key type to find, and a key value to find. We then loop through every item in our array, and extract the appropriate key type from each entity, and test it against the value we are searching for. If we find a match, we return the Array Key for that entity. If we reach the end of the array, and we have not found a match, we return false. Note that looping through large arrays to find a match is not very efficient. This class is simply for testing small interactions between our App and the gateway. This should never be done in production with large datasets.

With the helper methods set up, we can now look at our putMulti() method. The upsert() in the PHP-GDS library allows you to upsert an array of entities. In the function putMulti() declaration, we have required an argument $Entities, that will accept the array. The first section of our putMulti() method tests if the $Entities argument is an array, and if it is not an array, we create a new empty array, add the provided entity, and set the newly created array as our $Entities variable. We then loop through each of the entities that have been passed to the putMulti() method, we then use the unset($Key) command to reset any Key value that may remain from the previous loop. Next, we use our getEntityKey() helper method to determine if the current entity has a key set, if it does not, we can set a random key using md5(rand()), and add the entity to the end of our simulated datastore array.

If there is a key set, we search the simulated datastore array using our searchObjectArray() method, to see if it has previously been stored. If we do not get a valid array key back, we can simply add the entity to the store. If an array key is returned, we replace the existing data in the store array with the current entity. These actions handle adding new items, and also allow us to update existing items using the putMulti() method, which simulates the way the PHP-GDS Gateway class’s putMulti() method behaves.

Next, we will look at the gateway’s deleteMulti() method.

<?PHP // tests/mockgateway.class.php

class MockGateway{
	
	public $Entities;
	
	public function __construct(){
		$this->Entities = [];
	}
	
	public function putMulti($Entities){
		
		if(!is_array($Entities)){
			$NewEntities[] = $Entities;
			unset($Entities);
			$Entities = $NewEntities;
		}
		
		foreach($Entities AS $Entity){
			unset($Key);
			$Key = $this->getEntityKey($Entity);
			if(!$Key){
				$Entity->setKeyId(md5(rand(1, 1000)));
				$this->Entities[] = $Entity;
			} else{
				$ItemKey = $this->searchObjectArray($this->Entities, $Key['Type'], $Key['Value']);
				
				if(!is_int($ItemKey)){
					$this->Entities[] = $Entity;
				} else{
					$this->Entities[$ItemKey] = $Entity;
				}
			}
		}
		return true;
	}
	
	public function searchObjectArray($Array, $Property, $Value){
		foreach($Array AS $Key => $Item){
			if($Property == 'KeyId'){
				$ArrayValue = $Item->getKeyId();
			}
			if($Property == 'KeyName'){
				$ArrayValue = $Item->getKeyName();
			}
			if($ArrayValue == $Value){
				return $Key;
			}
		}
		return false;
	}
	
	public function getEntityKey($Entity){
		$KeyName = $Entity->getKeyName();
		$KeyID = $Entity->getKeyID();
		if(isset($KeyID)){
			$Key['Type'] = 'KeyId';
			$Key['Value'] = $KeyID;
			return $Key;
		} else{
			if(isset($KeyName)){
				$Key['Type'] = 'KeyName';
				$Key['Value'] = $KeyName;
				return $Key;
			} else{
				return false;
			}
		}
	}

	public function deleteMulti($Entities){
		if(!is_array($Entities)){
			$NewEntities[] = $Entities;
			unset($Entities);
			$Entities = $NewEntities;
		}
		
		foreach($Entities AS $Entity){
			$Key = $this->getEntityKey($Entity);
			if(!$Key){
				
			} else{
				$ItemKey = $this->searchObjectArray($this->Entities, $Key['Type'], $Key['Value']);
				if($ItemKey != false){
					unset($this->Entities[$ItemKey]);
				}
			}
		}
		return true;
	}
?>

Here, we accept an array of entities to delete, and as with our putMulti() method, we first need to ensure the argument provided is an array, if not we create one with the single entity that was supplied.

We then loop through the array of entities, and for each, we use our keyEntityKey to ensure a key has been set. If a key is not found, we do not need to take any action, as from our putMulti() method, we know that all entities stored in our simulated datastore have keys assigned to them. If a key is found, we then use our searchObjectArray to see if the entity exists in our store, if it does exist, we simply use the unset() command to remove the entity from the simulated store.

We can now add, update and delete our entities. We need to also be able to retrieve entities from the simulated store. The PHP-GDS provides methods to retrieve entities based on the KeyId, and KeyName. We will start with KeyID.

<?PHP // tests/mockgateway.class.php

class MockGateway{
	
	public $Entities;
	
	public function __construct(){
		$this->Entities = [];
	}
	
	public function putMulti($Entities){
		
		if(!is_array($Entities)){
			$NewEntities[] = $Entities;
			unset($Entities);
			$Entities = $NewEntities;
		}
		
		foreach($Entities AS $Entity){
			unset($Key);
			$Key = $this->getEntityKey($Entity);
			if(!$Key){
				$Entity->setKeyId(md5(rand(1, 1000)));
				$this->Entities[] = $Entity;
			} else{
				$ItemKey = $this->searchObjectArray($this->Entities, $Key['Type'], $Key['Value']);
				
				if(!is_int($ItemKey)){
					$this->Entities[] = $Entity;
				} else{
					$this->Entities[$ItemKey] = $Entity;
				}
			}
		}
		return true;
	}
	
	public function searchObjectArray($Array, $Property, $Value){
		foreach($Array AS $Key => $Item){
			if($Property == 'KeyId'){
				$ArrayValue = $Item->getKeyId();
			}
			if($Property == 'KeyName'){
				$ArrayValue = $Item->getKeyName();
			}
			if($ArrayValue == $Value){
				return $Key;
			}
		}
		return false;
	}
	
	public function getEntityKey($Entity){
		$KeyName = $Entity->getKeyName();
		$KeyID = $Entity->getKeyID();
		if(isset($KeyID)){
			$Key['Type'] = 'KeyId';
			$Key['Value'] = $KeyID;
			return $Key;
		} else{
			if(isset($KeyName)){
				$Key['Type'] = 'KeyName';
				$Key['Value'] = $KeyName;
				return $Key;
			} else{
				return false;
			}
		}
	}

	public function deleteMulti($Entities){
		if(!is_array($Entities)){
			$NewEntities[] = $Entities;
			unset($Entities);
			$Entities = $NewEntities;
		}
		
		foreach($Entities AS $Entity){
			$Key = $this->getEntityKey($Entity);
			if(!$Key){
				
			} else{
				$ItemKey = $this->searchObjectArray($this->Entities, $Key['Type'], $Key['Value']);
				if($ItemKey != false){
					unset($this->Entities[$ItemKey]);
				}
			}
		}
		return true;
	}

	public function fetchById($ID){
		$ItemKey = $this->searchObjectArray($this->Entities, 'KeyId', $ID);
		if(!$ItemKey){
			return false;
		}
		return $this->Entities[$ItemKey];
	}
	
	public function fetchByIds($IDs){
		$Items = [];
		foreach($IDs AS $ID){
			$Item = $this->fetchById($ID);
			if($Item != false){
				$Items[] = $Item;
			}
		}
		return $Items;
			
	}
	
?>

The fetchById() method is quite simple, it uses our searchObjectArray() method to see if we have an entity matching the KeyId provided. If we find an entity, we return it. If not, we simply return false.

The PHP-GDS library also provides a fetchByIds() method which allows an array of KeyIds to be provided. To implement this in our class, we can loop through each KeyId, and use our new fetchById() method to return the entity. If we receive false back from the fetchById() method, we ignore the ID, as the entity was not found. If we do find the entity, we add it to an $Items array and after looping through all the provided IDs, we can return the $Items array.

We do much the same for the KeyName fetch methods.

<?PHP // tests/mockgateway.class.php

class MockGateway{
	
	public $Entities;
	
	public function __construct(){
		$this->Entities = [];
	}
	
	public function putMulti($Entities){
		
		if(!is_array($Entities)){
			$NewEntities[] = $Entities;
			unset($Entities);
			$Entities = $NewEntities;
		}
		
		foreach($Entities AS $Entity){
			unset($Key);
			$Key = $this->getEntityKey($Entity);
			if(!$Key){
				$Entity->setKeyId(md5(rand(1, 1000)));
				$this->Entities[] = $Entity;
			} else{
				$ItemKey = $this->searchObjectArray($this->Entities, $Key['Type'], $Key['Value']);
				
				if(!is_int($ItemKey)){
					$this->Entities[] = $Entity;
				} else{
					$this->Entities[$ItemKey] = $Entity;
				}
			}
		}
		return true;
	}
	
	public function searchObjectArray($Array, $Property, $Value){
		foreach($Array AS $Key => $Item){
			if($Property == 'KeyId'){
				$ArrayValue = $Item->getKeyId();
			}
			if($Property == 'KeyName'){
				$ArrayValue = $Item->getKeyName();
			}
			if($ArrayValue == $Value){
				return $Key;
			}
		}
		return false;
	}
	
	public function getEntityKey($Entity){
		$KeyName = $Entity->getKeyName();
		$KeyID = $Entity->getKeyID();
		if(isset($KeyID)){
			$Key['Type'] = 'KeyId';
			$Key['Value'] = $KeyID;
			return $Key;
		} else{
			if(isset($KeyName)){
				$Key['Type'] = 'KeyName';
				$Key['Value'] = $KeyName;
				return $Key;
			} else{
				return false;
			}
		}
	}

	public function deleteMulti($Entities){
		if(!is_array($Entities)){
			$NewEntities[] = $Entities;
			unset($Entities);
			$Entities = $NewEntities;
		}
		
		foreach($Entities AS $Entity){
			$Key = $this->getEntityKey($Entity);
			if(!$Key){
				
			} else{
				$ItemKey = $this->searchObjectArray($this->Entities, $Key['Type'], $Key['Value']);
				if($ItemKey != false){
					unset($this->Entities[$ItemKey]);
				}
			}
		}
		return true;
	}

	public function fetchById($ID){
		$ItemKey = $this->searchObjectArray($this->Entities, 'KeyId', $ID);
		if(!$ItemKey){
			return false;
		}
		return $this->Entities[$ItemKey];
	}
	
	public function fetchByIds($IDs){
		$Items = [];
		foreach($IDs AS $ID){
			$Item = $this->fetchById($ID);
			if($Item != false){
				$Items[] = $Item;
			}
		}
		return $Items;
			
	}
	
	public function fetchByName($Name){
		$ItemKey = $this->searchObjectArray($this->Entities, 'KeyName', $Name);
		if(!$ItemKey){
			return false;
		}
		return $this->Entities[$ItemKey];
	}
	
	public function fetchByNames($Names){
		$Items = [];
		foreach($Names AS $Name){
			$Item = $this->fetchByName($Name);
			if($Item != false){
				$Items[] = $Item;
			}
		}
		return $Items;
	}
?>

We have again used our searchObjectArray() method to lookup entities, this time based on KeyName, not KeyId. The PHP-GDS library also allows queries to be run on the datastore to return matching entities. The queries that can be run are based on GQL, or Google Query Language, which is quite similar to MySQL queries. We cannot easily simulate running queries on our mock store, so instead, for each of the query methods from the PHP-GDS store class, we will simply return the entire $Entities array. We can manipulate the data stored in the mock store from our tests, to ensure we return data that will satisfy our tests. I have listed each of these query methods below, and have also included a number of mock methods to ensure our gateway covers all of the methods from the real gateway class.

Our final mockgateway.class.php file should look like the one below.

<?PHP	// tests/mockgateway.class.php

class MockGateway{
	
	public $Entities;
	
	public function __construct($Dataset = null, $Namespace = null){
		$this->Entities = [];
	}
	
	public function putMulti(array $Entities){
		if(!is_array($Entities)){
			$NewEntities[] = $Entities;
			unset($Entities);
			$Entities = $NewEntities;
		}
		
		foreach($Entities AS $Entity){
			unset($Key);
			$Key = $this->getEntityKey($Entity);
			if(!$Key){
				$Entity->setKeyId(md5(rand(1, 1000)));
				$this->Entities[] = $Entity;
			} else{
				$ItemKey = $this->searchObjectArray($this->Entities, $Key['Type'], $Key['Value']);
				
				if(!is_int($ItemKey)){
					$this->Entities[] = $Entity;
				} else{
					$this->Entities[$ItemKey] = $Entity;
				}
			}
		}
		return true;
	}
	
	public function deleteMulti(array $Entities){
		if(!is_array($Entities)){
			$NewEntities[] = $Entities;
			unset($Entities);
			$Entities = $NewEntities;
		}
		
		foreach($Entities AS $Entity){
			$Key = $this->getEntityKey($Entity);
			if(!$Key){
				
			} else{
				$ItemKey = $this->searchObjectArray($this->Entities, $Key['Type'], $Key['Value']);
				if($ItemKey != false){
					unset($this->Entities[$ItemKey]);
				}
			}
		}
		return true;
	}
	
	public function fetchByName($Name){
		$ItemKey = $this->searchObjectArray($this->Entities, 'KeyName', $Name);
		if($ItemKey === false){
			return false;
		}
		return $this->Entities[$ItemKey];
	}
	
	public function delete(GDSEntity $Entity){
		$Array[] = $Entity;
		$this->deleteMulti($Array);
	}
	
	public function fetchById($ID){
		$ItemKey = $this->searchObjectArray($this->Entities, 'KeyId', $ID);
		if($ItemKey === false){
			return false;
		}
		return $this->Entities[$ItemKey];
	}
	
	public function fetchByIds(array $IDs){
		$Items = [];
		foreach($IDs AS $ID){
			$Item = $this->fetchById($ID);
			if($Item != false){
				$Items[] = $Item;
			}
		}
		return $Items;
	}
	
	public function fetchByNames(array $Names){
		$Items = [];
		foreach($Names AS $Name){
			$Item = $this->fetchByName($Name);
			if($Item != false){
				$Items[] = $Item;
			}
		}
		return $Items;
	}
	
	public function gql($str_gql, $arr_params = NULL){
		return $this->Entities;
	}
	
	public function beginTransaction($bol_cross_group = false){
		//called by store, but not required in mock gateway
	}
	
	public function buildSchema(){
		$Schema = new GDSSchema('MockEntity');
		$Schema->addString('Username', true);
		$Schema->addString('Name', true);
		return $Schema;
	}
	
	public function withSchema(GDSSchema $Schema){
		//called by store, but not required in mock gateway
		$this->obj_schema = $Schema;
		return $this;
	}
	
	public function withTransaction($Transaction){
		//called by store, but not required in mock gateway
		return $this;
	}
	
	public function consumeTransaction(){
		return md5(rand(1,1000));
	}
		
	
	public function searchObjectArray($Array, $Property, $Value){
		foreach($Array AS $Key => $Item){
			if($Property == 'KeyId'){
				$ArrayValue = $Item->getKeyId();
			}
			if($Property == 'KeyName'){
				$ArrayValue = $Item->getKeyName();
			}
			if($ArrayValue == $Value){
				return $Key;
			}
		}
		return false;
	}
	
	public function getEntityKey($Entity){
		$KeyName = $Entity->getKeyName();
		$KeyID = $Entity->getKeyID();
		if(isset($KeyID)){
			$Key['Type'] = 'KeyId';
			$Key['Value'] = $KeyID;
			return $Key;
		} else{
			if(isset($KeyName)){
				$Key['Type'] = 'KeyName';
				$Key['Value'] = $KeyName;
				return $Key;
			} else{
				return false;
			}
		}
	}
	
	public function configureObjectValueParamForQuery($obj_val, $mix_value){
	
	}
	
	public function upsert(array $arr_entities){
		
	}
	
	public function extractAutoIDs(){
		
	}
	
	public function fetchByKeyPart(array $arr_key_parts, $str_setter){
		
	}
	
	public function getEndCursor(){
		
	}
	
	public function createMapper(){
		
	}
		
}
	
?>

We now have a mock gateway we can use to test our application against, in order to test the link between our app and the PHP-GDS store. We do need to also test this mock class, to ensure it will fulfil our needs.  I will not go through each test individually, instead give a list of each of the tests performed to show the level of coverage of testing on the class. This list is compiled using the phpunit –testdox feature we saw in the article Test Based Development – A SaaS Experience

  • [x] Can add single entity with no key set
  • [x] Can add single entity with key id set
  • [x] Can add single entity with key name set
  • [x] Can add multiple entities with mixed keys
  • [x] Can update single entity with key id
  • [x] Can update single entity with key name
  • [x] Can update multiple entities with mixed keys
  • [x] Can delete single item by key id
  • [x] Can delete single item by key name
  • [x] Can delete multiple items with mixed keys
  • [x] Can fetch by id
  • [x] Can fetch by ids
  • [x] Can fetch by name
  • [x] Can fetch by names
  • [x] Returns empty array when fetching by unknown id
  • [x] Returns empty array when fetching by unknown ids
  • [x] Returns empty array when fetching by unknown key name
  • [x] Returns empty array when fetching by unknown key names

We can now be confident that our mock gateway will perform well with all but the query methods. For query methods, we will need to manually manipulate the data in the store to ensure the desired entities are returned. When we looked at our custom session handler class in the article Session Management – A SaaS Experience I did not include any tests, as we had yet to come up with a way to test our datastore connection. As we have now found a solution, I have included a simple test that shows how we can test writing and reading sessions with our custom session handler.  You will also note, I have used a mock Cache as well, this is due to differences in the Google Memcached setup and PHP’s default Memcached setup. I have included the very simple source for this mock cache below as well.

<?PHP // tests/mockcache.class.php

class MockCache {
	public $Items;
	
	public function set($ID, $Data){
		$this->Items[$ID] = $Data;
		return true;
	}
	
	public function get($ID){
		if(isset($this->Items[$ID])){
			return $this->Items[$ID];
		} else{
			return false;
		}
	}
	
}
	
?>
<?PHP	// tests/SessionHandlerTest.php
ob_start();

class SessionHandlerTest extends PHPUnit_Framework_TestCase{
	
	public $Session;
	public $SessionHandler;
	public $Gateway;
		
	public function setUp(){
		$CustomSessionHandler = Container::newCustomSessionHandler();
		$this->Gateway = new MockGateway(null, null);
		$CustomSessionHandler->Store = new SessionStore($this->Gateway);
		$CustomSessionHandler->Cache = new MockCache();
		$this->SessionHandler = $CustomSessionHandler;
	}
		
	public function testCanWriteSession(){
		$ID = '1234';
		$Data = 'TestSession';
		
		$this->SessionHandler->write($ID, $Data);
		
		//Assert 1 Item in Store
		$this->AssertEquals(1, sizeof($this->Gateway->Entities), 'The size of the store does not match the number of items entered');
		$this->AssertEquals(1, sizeof($this->SessionHandler->Cache->Items), 'The size of the cache does not match the number of items entered');
	}
	
	public function testCanRetrieveSession(){
		$ID = '1234';
		$Data = 'TestSession';
		$this->SessionHandler->write($ID, $Data);
		$ID = '4567';
		$Data = 'Wrong Session';
		$this->SessionHandler->write($ID, $Data);
		
		$Session = $this->SessionHandler->read('1234');
		//Assert $Session contains the data 'TestSession'
		$this->assertEquals('TestSession', $Session, 'The returned session does not match the expected session');
	}
	
}	
	
?>

We can now test simple datastore interactions! This was quite a lengthy article! Database testing, and in particular Datastore testing is quite complex with PHPUnit, and currently mocking the store as we have done in this article is the only solution. I have been in contact with Tom Walder, the developer of the PHP-GDS library, and he is considering adding a more robust mock store for testing in the future. In the next article, we will explore how we can further use Memcached to improve performance in our application.