circle/doctrine-rest-driver

Use a REST API as if it was your local database

Motivation

What does a black sheep and a white sheep have in common? They produce wool.
What does a big bus and a small bus have in common? They drive people around.
And what does a SQL database and a REST API have in common? They store data.
, _(*1)

As blatantly obvious as this sounds the consequences are tremendous: With REST APIs being nothing more than data storage backends, we are able to reuse object relational mapping tools to access them., _(*2)

And because we have absolutely no idea how to write a programming language, we're tryin to do it like Rasmus and keep adding the next logical step on the way. So DoctrineRestDriver saw the light of day., _(*3)

Prerequisites

• You need composer to download the library

Installation

Add the driver to your project using composer:, _(*4)

composer require circle/doctrine-rest-driver

Change the following doctrine dbal configuration entries:, _(*5)

doctrine:
  dbal:
    driver_class: "Circle\\DoctrineRestDriver\\Driver"
    host:         "%default_api_url%"
    port:         "%default_api_port%"
    user:         "%default_api_username%"
    password:     "%default_api_password%"
    options:
      format:               "json" | "YourOwnNamespaceName" | if not specified json will be used
      authenticator_class:  "HttpAuthentication" | "YourOwnNamespaceName" | if not specified no authentication will be used

Additionally you can add CURL-specific options:, _(*6)

doctrine:
  dbal:
    driver_class: "Circle\\DoctrineRestDriver\\Driver"
    host:         "%default_api_url%"
    port:         "%default_api_port%"
    user:         "%default_api_username%"
    password:     "%default_api_password%"
    options:
      format:                         "json"
      authenticator_class:            "HttpAuthentication"
      CURLOPT_CURLOPT_FOLLOWLOCATION: true
      CURLOPT_HEADER:                 true

A full list of all possible options can be found here: http://php.net/manual/en/function.curl-setopt.php, _(*7)

By default, UPDATE queries are converted to PUT to work with the majority of APIs however, when persisting an updated entity, Doctrine will compare the edited entity to the original data and create a query that only contains the changed fields. In a REST API, this would be converted to a PATCH request as a PUT is meant to include the entire entity even if some properties have not changed., _(*8)

To use PATCH instead of PUT simply add a config value:, _(*9)

doctrine:
  dbal:
    options:
      use_patch: true

Usage

If your API routes follow these few conventions, using the driver is very easy:, _(*10)

• Each route must be structured the same: {apiHost}/{pathToApi}/{tableName}
• The PUT/PATCH, GET (single) and UPDATE routes need to contain an additional id: {apiHost}/{pathToApi}/{tableName}/{id}
• POST and GET (all) must follow the basic structure: {apiHost}/{pathToApi}/{tableName}

Don't worry, if this is not the case: Luckily, we provide a few annotations for you to configure your own routes., _(*11)

Responses

Your API is allowed to respond with a handful of different HTTP status codes to be deemed a successful response., _(*12)

Note that a 404 response is considered a "successful" response in some circumstances. This allows the driver to reflect a database being queried but returning no data due to an entity not being found, for example "/entity/1" will allow a 404 without causing an exception as it's perfectly acceptable to not find an entity from a database., _(*13)

The examples below show how to use the driver in a Symfony environment., _(*14)

If your API follows our conventions

First of all create your entities:, _(*15)

namespace CircleBundle\Entity;

use Doctrine\ORM\Mapping as ORM;

/**
 * This annotation marks the class as managed entity:
 *
 * @ORM\Entity
 * @ORM\Table("products")
 */
class Product {

    /**
     * @ORM\Column(type="integer")
     * @ORM\Id
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    private $id;

    /**
     * @ORM\Column(type="string", length=100)
     */
    private $name;

    public function getId() {
        return $this->id;
    }

    public function setName($name) {
        $this->name = $name;
        return $this;
    }

    public function getName() {
        return $this->name;
    }
}

Afterwards, you are able to use the created entity as if you were using a database., _(*16)

By using this setting, the driver is performing a lot of magic under the hood:, _(*17)

• It generally uses the request body to send data in JSON format
• It automatically maps the response into a valid entity if the status code matches the default expected status codes (200 for GET and PUT, 201 for POST 204 for DELETE)
• It saves the entity as managed doctrine entity
• It translates INSERT queries into POST requests to create new data
  • Urls have the following format: {apiHost}/{pathToApi}/{tableName}
• UPDATE queries will be turned into PUT requests:
  • Urls have the following format: {apiHost}/{pathToApi}/{tableName}/{id}
• The DELETE operation will remain:
  • Urls have the following format: {apiHost}/{pathToApi}/{tableName}/{id}
• SELECT queries become GET requests:
  • Urls have the following format: {apiHost}/{pathToApi}/{tableName}/{id} (if a single entity is requested) or {apiHost}/{pathToApi}/{tableName} (if all entities are requested)

Let's watch the driver in action by implementing some controller methods. In this example we assume that we have configured the host setting (chapter installation) with http://www.yourSite.com/api., _(*18)

<?php

namespace CircleBundle\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Symfony\HttpFoundation\Response;

class UserController extends Controller {

    /**
     * Sends the following request to the API:
     * POST http://www.yourSite.com/api/products HTTP/1.1
     * {"name": "Circle"}
     *
     * Let's assume the API responded with:
     * HTTP/1.1 201 Created
     * {"id": 1, "name": "Circle"}
     *
     * Response body is "1"
     */
    public function createAction() {
        $em     = $this->getDoctrine()->getManager();
        $entity = new CircleBundle\Entity\Product();
        $entity->setName('Circle');
        $em->persist($entity);
        $em->flush();

        return new Response($entity->getId());
    }

    /**
     * Sends the following request to the API by default:
     * GET http://www.yourSite.com/api/products/1 HTTP/1.1
     *
     * which might respond with:
     * HTTP/1.1 200 OK
     * {"id": 1, "name": "Circle"}
     *
     * Response body is "Circle"
     */
    public function readAction($id = 1) {
        $em     = $this->getDoctrine()->getManager();
        $entity = $em->find('CircleBundle\Entity\Product', $id);

        return new Response($entity->getName());
    }

    /**
     * Sends the following request to the API:
     * GET http://www.yourSite.com/api/products HTTP/1.1
     *
     * Example response:
     * HTTP/1.1 200 OK
     * [{"id": 1, "name": "Circle"}]
     *
     * Response body is "Circle"
     */
    public function readAllAction() {
        $em       = $this->getDoctrine()->getManager();
        $entities = $em->getRepository('CircleBundle\Entity\Product')->findAll();

        return new Response($entities->first()->getName());
    }

    /**
     * After sending a GET request (readAction) it sends the following 
     * request to the API by default:
     * PUT http://www.yourSite.com/api/products/1 HTTP/1.1
     * {"name": "myName"}
     *
     * Let's assume the API responded the GET request with:
     * HTTP/1.1 200 OK
     * {"id": 1, "name": "Circle"}
     *
     * and the PUT request with:
     * HTTP/1.1 200 OK
     * {"id": 1, "name": "myName"}
     *
     * Then the response body is "myName"
     */
    public function updateAction($id = 1) {
        $em     = $this->getDoctrine()->getManager();
        $entity = $em->find('CircleBundle\Entity\Product', $id);
        $entity->setName('myName');
        $em->flush();

        return new Response($entity->getName());
    }

    /**
     * After sending a GET request (readAction) it sends the following 
     * request to the API by default:
     * DELETE http://www.yourSite.com/api/products/1 HTTP/1.1
     *
     * If the response is:
     * HTTP/1.1 204 No Content
     *
     * the response body is ""
     */
    public function deleteAction($id = 1) {
        $em     = $this->getDoctrine()->getManager();
        $entity = $em->find('CircleBundle\Entity\Product', $id);
        $em->remove($entity);
        $em->flush();

        return new Response();
    }
}

If your API doesn't follow our conventions

Now it's time to introduce you to some annotations, which help you to configure your own routes. Be sure to use them only with Doctrine entities. All these annotations have the same structure so we will call them DataSource annotation:, _(*19)

@DataSource\SomeName("http://www.myRoute.com", method="POST", statusCode=200)

The ROUTE value is mandatory, method and statusCode are optional., _(*20)

To demonstrate their capabilities, let's customize some parts of the previous chapter with the DataSource annotation:, _(*21)

namespace CircleBundle\Entity;

use Doctrine\ORM\Mapping as ORM;
use Circle\DoctrineRestDriver\Annotations as DataSource;

/**
 * This annotation marks the class as managed entity:
 *
 * @ORM\Entity
 * @ORM\Table("products")
 * @DataSource\Select("http://www.yourSite.com/api/products/findOne/{id}")
 * @DataSource\Fetch("http://www.yourSite.com/api/products/findAll")
 * @DataSource\Insert("http://www.yourSite.com/api/products/insert", statusCodes={200})
 * @DataSource\Update("http://www.yourSite.com/api/products/update/{id}", method="POST")
 * @DataSource\Delete("http://www.yourSite.com/api/products/remove/{id}", method="POST", statusCodes={200})
 */
class Product {

    /**
     * @ORM\Column(type="integer")
     * @ORM\Id
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    private $id;

    /**
     * @ORM\Column(type="string", length=100)
     */
    private $name;

    public function getId() {
        return $this->id;
    }

    public function setName($name) {
        $this->name = $name;
        return $this;
    }

    public function getName() {
        return $this->name;
    }
}

If you have a list of acceptable status codes you can also pass an array to the statusCode option:, _(*22)

<?php
namespace CircleBundle\Entity;

use Doctrine\ORM\Mapping as ORM;
use Circle\DoctrineRestDriver\Annotations as DataSource;

/**
 * This annotation marks the class as managed entity:
 *
 * @ORM\Entity
 * @ORM\Table("products")
 * @DataSource\Select("http://www.yourSite.com/api/products/findOne/{id}", statusCodes={200, 203, 404})
 */
class Product {
   // ... //
}

The annotations tell the driver to send the requests to the configured URLs for each custom configuration. If you just want to define a specific route for one method, you don't need to use all annotations provided. The {id} act as a placeholder for the entity's identifier., _(*23)

<?php

namespace CircleBundle\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Symfony\HttpFoundation\Response;

class UserController extends Controller {

    /**
     * Sends the following request to the API:
     * POST http://www.yourSite.com/api/products/insert HTTP/1.1
     * {"name": "Circle"}
     *
     * Let's assume the API responded with:
     * HTTP/1.1 200 OK
     * {"id": 1, "name": "Circle"}
     *
     * Response body is "1"
     */
    public function createAction() {
        $em     = $this->getDoctrine()->getManager();
        $entity = new CircleBundle\Entity\Product();
        $entity->setName('Circle');
        $em->persist($entity);
        $em->flush();

        return new Response($entity->getId());
    }

    /**
     * Sends the following request to the API by default:
     * GET http://www.yourSite.com/api/products/findOne/1 HTTP/1.1
     *
     * which might respond with:
     * HTTP/1.1 200 OK
     * {"id": 1, "name": "Circle"}
     *
     * Response body is "Circle"
     */
    public function readAction($id = 1) {
        $em     = $this->getDoctrine()->getManager();
        $entity = $em->find('CircleBundle\Entity\Product', $id);

        return new Response($entity->getName());
    }

    /**
     * Sends the following request to the API:
     * GET http://www.yourSite.com/api/products/findAll HTTP/1.1
     *
     * Example response:
     * HTTP/1.1 200 OK
     * [{"id": 1, "name": "Circle"}]
     *
     * Response body is "Circle"
     */
    public function readAllAction() {
        $em       = $this->getDoctrine()->getManager();
        $entities = $em->getRepository('CircleBundle\Entity\Product')->findAll();

        return new Response($entities->first()->getName());
    }

    /**
     * After sending a GET request (readAction) it sends the following 
     * request to the API by default:
     * POST http://www.yourSite.com/api/products/update/1 HTTP/1.1
     * {"name": "myName"}
     *
     * Let's assume the API responded the GET request with:
     * HTTP/1.1 200 OK
     * {"id": 1, "name": "Circle"}
     *
     * and the POST request with:
     * HTTP/1.1 200 OK
     * {"id": 1, "name": "myName"}
     *
     * Then the response body is "myName"
     */
    public function updateAction($id = 1) {
        $em     = $this->getDoctrine()->getManager();
        $entity = $em->find('CircleBundle\Entity\Product', $id);
        $entity->setName('myName');
        $em->flush();

        return new Response($entity->getName());
    }

    /**
     * After sending a GET request (readAction) it sends the following 
     * request to the API by default:
     * POST http://www.yourSite.com/api/products/remove/1 HTTP/1.1
     *
     * If the response is:
     * HTTP/1.1 200 OK
     *
     * the response body is ""
     */
    public function deleteAction($id = 1) {
        $em     = $this->getDoctrine()->getManager();
        $entity = $em->find('CircleBundle\Entity\Product', $id);
        $em->remove($entity);
        $em->flush();

        return new Response();
    }
}

Pagination

Queries in Doctrine can be paginated using the OFFSET and LIMIT keywords. Pagination will be sent as request headers by default, but this can be configured to have pagination sent as query parameters:, _(*24)

doctrine:
  dbal:
    options:
      pagination_as_query: true

This will convert to the following:, _(*25)

SELECT name FROM users LIMIT 5 OFFSET 10

To:, _(*26)

https://api.example.com/users?per_page=5&page=3

The parameter keys used for per_page and page can also be set in the configuration file:, _(*27)

doctrine:
  dbal:
    options:
      per_page_param: count
      page_param: p

Examples

Need some more examples? Here they are:, _(*28)

Persisting entities

Imagine you have a REST API at http://www.your-url.com/api:, _(*29)

typedef UnregisteredAddress {
    street: String,
    city:   String
}

typedef RegisteredAddress {
    id:     Int,
    street: String,
    city:   String
}

Let's first configure Doctrine:, _(*30)

doctrine:
  dbal:
    driver_class: "Circle\\DoctrineRestDriver\\Driver"
    host:         "http://www.your-url.com/api"
    port:         "80"
    user:         ""
    password:     ""

Afterwards let's build the address entity:, _(*31)

<?php
namespace CircleBundle\Entity;

use Doctrine\ORM\Mapping as ORM;

/**
 * @ORM\Entity()
 * @ORM\Table("addresses")
 */
class Address {

    /**
     * @ORM\Column(type="integer")
     * @ORM\Id
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    private $id;

    /**
     * @ORM\Column(type="string")
     */
    private $street;

    /**
     * @ORM\Column(type="string")
     */
    private $city;

    public function setStreet($street) {
        $this->street = $street;
        return $this;
    }

    public function getStreet() {
        return $this->street;
    }

    public function setCity($city) {
        $this->city = $city;
        return $this;
    }

    public function getCity() {
        return $this->city;
    }

    public function getId() {
        return $this->id;
    }
}

And finally the controller and its action:, _(*32)

<?php

namespace CircleBundle\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Symfony\HttpFoundation\Response;

class AddressController extends Controller {

    public function createAction($street, $city) {
        $em      = $this->getDoctrine()->getManager();
        $address = new CircleBundle\Address();

        $address->setStreet($street)->setCity($city);
        $em->persist($address);

        try {
            $em->flush();
            return new Response('successfully registered');
        } catch(RequestFailedException) {
            return new Response('invalid address');
        }
    }
}

That's it. Each time the createAction is called it will send a POST request to the API., _(*33)

Associating entities

Let's extend the first example. Now we want to add a new entity type User which references the addresses defined in the previous example., _(*34)

The REST API offers the following additional routes:, _(*35)

typedef UnregisteredUser {
    name:     String,
    password: String,
    address:  Index
}

typedef RegisteredUser {
    id:       Int,
    name:     String,
    password: String,
    address:  Index
}

First, we need to build an additional entity "User":, _(*36)

namespace Circle\Entity;

use Doctrine\ORM\Mapping as ORM;

/**
 * @ORM\Entity()
 * @ORM\Table("users")
 */
class User {

    /**
     * @ORM\Column(type="integer")
     * @ORM\Id
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    private $id;

    /**
     * @ORM\Column(type="string")
     */
    private $name;

    /**
     * @ORM\OneToOne(targetEntity="CircleBundle\Address", cascade={"persist, remove"})
     */
    private $address;

    /**
     * @ORM\Column(type="string")
     */
    private $password;

    public function setName($name) {
        $this->name = $name;
        return $this;
    }

    public function getName() {
        return $this->name;
    }

    public function setPassword($password) {
        $this->password = $password;
        return $this;
    }

    public function getPassword() {
        return $this->password;
    }

    public function getId() {
        return $this->id;
    }

    public function setAddress(Address $address) {
        $this->address = $address;
        return $this;
    }

    public function getAddress() {
        return $this->address;
    }
}

The user has a relation to address. So let's have a look at what happens if we associate them:, _(*37)

<?php

namespace CircleBundle\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Symfony\HttpFoundation\Response;

class UserController extends Controller {

    public function createAction($name, $password, $addressId) {
        $em      = $this->getDoctrine()->getManager();
        $address = $em->find("CircleBundle\Entity\Address", $addressId);
        $user    = new User();

        $user->setName($name)
            ->setPassword($password)
            ->setAddress($address);

        $em->persist($user);
        $em->flush();

        return new Response('successfully registered');
    }
}

If we'd set name to username, password to secretPassword and addressId to 1 by triggering the createAction, the following requests would be sent by our driver:, _(*38)

GET  http://www.your-url.com/api/addresses/1 HTTP/1.1
POST http://www.your-url.com/api/users HTTP/1.1 {"name": "username", "password":"secretPassword", "address":1}

Because we have used the option cascade={"remove"} on the relation between users and addresses DELETE requests for addresses are automatically sent if the owning user is removed:, _(*39)

<?php

namespace CircleBundle\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Symfony\HttpFoundation\Response;

class UserController extends Controller {

    public function remove($id = 1) {
        $em = $this->getDoctrine()->getManager();
        $em->find('CircleBundle\Entity\User', $id);
        $em->remove($em);
        $em->flush();

        return new Response('successfully removed');
    }
}

For example, a DELETE request with the id 1 would trigger these requests:, _(*40)

DELETE http://www.your-url.com/api/addresses/1 HTTP/1.1
DELETE http://www.your-url.com/api/users/1 HTTP/1.1

Great, isn't it?, _(*41)

Using multiple Backends

In this last example we split the user and the address routes into two different REST APIs. This means we need multiple managers which is explained in the Doctrine documentation:, _(*42)

doctrine:
  dbal:
    default_connection: default
    connections:
      default:
        driver:   "pdo_mysql"
        host:     "localhost"
        port:     3306
        user:     "root"
        password: "root"
      user_api:
        driver_class: "Circle\\DoctrineRestDriver\\Driver"
        host:         "http://api.user.your-url.com"
        port:         80
        user:         "Circle"
        password:     "CircleUsers"
        options:
          authenticator_class:  "HttpAuthentication"
      address_api:
        driver_class: "Circle\\DoctrineRestDriver\\Driver"
        host:         "http://api.address.your-url.com"
        port:         80
        user:         "Circle"
        password:     "CircleAddresses"
        options:
          authenticator_class:  "HttpAuthentication"

Now it's getting crazy: We will try to read data from two different APIs and persist them into a MySQL database. Imagine the user API with the following route:, _(*43)

and the address API with this entry point:, _(*44)

We want to read a user from the user API and an address from the address API. After that we will associate and persist them in our MySQL database., _(*45)

<?php

namespace CircleBundle\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Symfony\HttpFoundation\Response;

class UserController extends Controller {

    public function createAction($userId, $addressId) {
        $emUsers       = $this->getDoctrine()->getManager('user_api');
        $emAddresses   = $this->getDoctrine()->getManager('address_api');
        $emPersistence = $this->getDoctrine()->getManager();

        $user    = $emUsers->find("CircleBundle\Entity\User", $userId);
        $address = $emAddresses->find("CircleBundle\Entity\Address", $addressId);
        $user->setAddress($address);

        $emPersistence->persist($address);
        $emPersistence->persist($user);
        $emPersistence->flush();

        return new Response('successfully persisted');
    }
}

As you can see in the request log both APIs are requested:, _(*46)

GET  http://api.users.your-url.com/users/1 HTTP/1.1
GET  http://api.addresses.your-url.com/addresses/1 HTTP/1.1

Afterwards both entities are persisted in the default MySQL database., _(*47)

Testing

To test the bundle just type:, _(*48)

make test

Contributing

If you want to contribute to this repository, please ensure ... - to follow the existing coding style. - to use the linting tools that are listed in the composer.json (which you get for free when using make). - to add and/or customize unit tests for any changed code. - to reference the corresponding issue in your pull request with a small description of your changes., _(*49)

All contributors are listed in the AUTHORS file, sorted by the time of their first contribution., _(*50)