Retrieving Entities from MongoDB in Drupal 8

using external entities

Posted on: 2016-05-18 Last edited: 2016-05-18
Language:
  • English

Normally in Drupal, all the entities are stored in the same database, which is usually MySQL or other Relational Databases. What if we have data elsewhere, and want to access them in Drupal? I often use MongoDB, and I think it is a good idea if I could directly retrieve or edit from Drupal, without having to import data into Drupal.

Lets say in MongoDB I created a few items:

{ "_id" : ObjectId("573b436188bea58b559bb625"), "title" : "Entity One" }
{ "_id" : ObjectId("573b436c88bea58b559bb626"), "title" : "Another External Entity" }
{ "_id" : ObjectId("573b438988bea58b559bb627"), "title" : "MongoDB is cool" }

I want to load them in Drupal 8 as entities, and maybe edit them as well.

External Entities

What we could utilize is a module called External Entities. It does what its name says: allowing Drupal to treat external resources as a special type of entities. Similar modules include Remote Entity API (which doesn't have a Drupal 8 version yet).

Currently, External Entities comes with two build-in drivers: REST and Wiki. We will be creating a new driver for MongoDB. Today, we will just deal with retrieving data, not modifying.

Installing Dependencies


The first thing we need to install is the MongoDB driver for PHP.
On a Linux system, the installation is as simple as follows:

sudo pecl install mongodb

and adding the following line to your php.ini file:

extension=mongodb.so

We also gonna use MongoDB PHP library, which provides a high-level API for manipulating MongoDB collections. (Its API is also very similar to MongoDB drivers for other programming languages, such as pymongo in Python.)
To install it, all we need to do is use Composer. In your Drupal directory, execute:

composer require "mongodb/mongodb=^1.0.0"

And you are ready to go.

Writing a Plugin

Our MongoDB client for External Entities will be a plugin. But first let's create a module to hold this plugin.

create a external_entities_mongo.info.yml file.

name: External Entities MongoDB
type: module
description: MongoDB plugin for External Entities
core: 8.x
package: Custom
dependencies:
  - external_entities

Now create the plugin. You can find the source code of REST and Wiki clients and use them as a template.
src/Plugin/ExternalEntityStorageClient/MongoClient.php. Get the annotation correct.

<?php

/**
* @file
* Contains \Drupal\external_entities_mongo\Plugin\ExternalEntityStorageClient\MongoClient.
*/

namespace Drupal\external_entities_mongo\Plugin\ExternalEntityStorageClient;

use Drupal\external_entities\ExternalEntityStorageClientBase;

/**
* MongoDB implementation of an external entity storage client.
*
* @ExternalEntityStorageClient(
*   id = "Mongo_client",
*   name = "MongoDB"
* )
*/

Our Client will extend ExternalEntityStorageClientBase, which has four methods (corresponding to CRUD operations ): load, query, delete, and save. For the moment we will only implement load (for a single entity) and query (retrieving a list of entities).

class MongoClient extends ExternalEntityStorageClientBase {

private $client;

private $collection;

public function __construct(array $configuration, $plugin_id, $plugin_definition) {
  parent::__construct($configuration, $plugin_id, $plugin_definition);
  $this->client = new \MongoDB\Client($this->configuration['endpoint']);
  $this->collection = $this->client->demo->entities;
}

Here we hardcode the collection to use (which is Demo.entities). This is just for demonstration.

public function load($id) {
  $result = $this->collection->findOne(['_id'=>(new \MongoDB\BSON\ObjectId($id))]);
  return ((object) array(
          "_id" => (string)($result["_id"]),
          "title" => $result["title"]));
}


public function query(array $parameters) {
  $results = $this->collection->find();
  $r = [];
  foreach ($results as $result) {
    $r[] = ((object) array(
          "_id" => (string)($result["_id"]),
          "title" => $result["title"]));
  }
  return $r;
}

The return value of our method are generally a stdClass object (an array of stdClass objects). Note that the "_id" field of the result from MongoDB is a ObjectId object, which needs to be converted to a string.
External Entities needs you to ensure that each entities has a unique id, which could be either string or int.

public function save(\Drupal\external_entities\ExternalEntityInterface $entity) {
  //TODO
}
 
public function delete(\Drupal\external_entities\ExternalEntityInterface $entity) {
  //TODO
}
}

Now enable your module.

Then navigate to Structure -> External entity types and add a new entity type.

I will call it mongo. Remember check the "Read only" checkbox (because we haven't implement writing operations).

Now in the Field mappings section, enter "_id" for external entity ID, and "title" for Title.

You will notice that these are consistent with the keys in your return value.

In storage section, select "MongoDB" as Storage client, leave Format unchanged. For Endpoint fill in a MongoDB URL string such as mongodb://mongo:27017 (make sure MongoDB is running and your webserver has access to that URL.)

Leave other settings unchanged and save.

Now if you go to http://your.domain/external-entities/mongo, you should be able to see the three entities we created in MongoDB before.

Screenshot

Summary

That was just a simple experiment with External Entities. What I haven't shown you is the ability to create fields for your external entities and map them with whatever in your MongoDB.

I am very excited that we can take advantage of the flexibilities of NoSQL databases in Drupal. And with External Entities you can actually do more: for example, exposing employee data from your HR system as entities.

I will keep an eye on External Entities and in future posts we could polish this MongoDB client to make it suitable for more general use cases.