DataLoaderPHP
DataLoaderPHP is a generic utility to be used as part of your application's data
fetching layer to provide a simplified and consistent API over various remote
data sources such as databases or web services via batching and caching., (*1)
[
][GA master]
, (*2)
Requirements
This library requires PHP >= 7.3 to work., (*3)
Getting Started
First, install DataLoaderPHP using composer., (*4)
composer require "overblog/dataloader-php"
To get started, create a DataLoader object., (*5)
Batching
Batching is not an advanced feature, it's DataLoader's primary feature.
Create loaders by providing a batch loading function., (*6)
use Overblog\DataLoader\DataLoader;
$myBatchGetUsers = function ($keys) { /* ... */ };
$promiseAdapter = new MyPromiseAdapter();
$userLoader = new DataLoader($myBatchGetUsers, $promiseAdapter);
A batch loading callable / callback accepts an Array of keys, and returns a Promise which
resolves to an Array of values., (*7)
Then load individual values from the loader. DataLoaderPHP will coalesce all
individual loads which occur within a single frame of execution (using await method)
and then call your batch function with all requested keys., (*8)
$userLoader->load(1)
->then(function ($user) use ($userLoader) { return $userLoader->load($user->invitedByID); })
->then(function ($invitedBy) { echo "User 1 was invited by $invitedBy"; });
// Elsewhere in your application
$userLoader->load(2)
->then(function ($user) use ($userLoader) { return $userLoader->load($user->invitedByID); })
->then(function ($invitedBy) { echo "User 2 was invited by $invitedBy"; });
// Synchronously waits on the promise to complete, if not using EventLoop.
$userLoader->await(); // or `DataLoader::await()`
A naive application may have issued four round-trips to a backend for the
required information, but with DataLoaderPHP this application will make at most
two., (*9)
DataLoaderPHP allows you to decouple unrelated parts of your application without
sacrificing the performance of batch data-loading. While the loader presents an
API that loads individual values, all concurrent requests will be coalesced and
presented to your batch loading function. This allows your application to safely
distribute data fetching requirements throughout your application and maintain
minimal outgoing data requests., (*10)
Batch Function
A batch loading function accepts an Array of keys, and returns a Promise which
resolves to an Array of values. There are a few constraints that must be upheld:, (*11)
- The Array of values must be the same length as the Array of keys.
- Each index in the Array of values must correspond to the same index in the Array of keys.
For example, if your batch function was provided the Array of keys: [ 2, 9, 6, 1 ],
and loading from a back-end service returned the values:, (*12)
[
['id' => 9, 'name' => 'Chicago'],
['id' => 1, 'name' => 'New York'],
['id' => 2, 'name' => 'San Francisco']
]
Our back-end service returned results in a different order than we requested, likely
because it was more efficient for it to do so. Also, it omitted a result for key 6,
which we can interpret as no value existing for that key., (*13)
To uphold the constraints of the batch function, it must return an Array of values
the same length as the Array of keys, and re-order them to ensure each index aligns
with the original keys [ 2, 9, 6, 1 ]:, (*14)
[
['id' => 2, 'name' => 'San Francisco'],
['id' => 9, 'name' => 'Chicago'],
null,
['id' => 1, 'name' => 'New York']
]
Caching (current PHP instance)
DataLoader provides a memoization cache for all loads which occur in a single
request to your application. After ->load() is called once with a given key,
the resulting value is cached to eliminate redundant loads., (*15)
In addition to relieving pressure on your data storage, caching results per-request
also creates fewer objects which may relieve memory pressure on your application:, (*16)
$userLoader = new DataLoader(...);
$promise1A = $userLoader->load(1);
$promise1B = $userLoader->load(1);
var_dump($promise1A === $promise1B); // bool(true)
Clearing Cache
In certain uncommon cases, clearing the request cache may be necessary., (*17)
The most common example when clearing the loader's cache is necessary is after
a mutation or update within the same request, when a cached value could be out of
date and future loads should not use any possibly cached value., (*18)
Here's a simple example using SQL UPDATE to illustrate., (*19)
use Overblog\DataLoader\DataLoader;
// Request begins...
$userLoader = new DataLoader(...);
// And a value happens to be loaded (and cached).
$userLoader->load(4)->then(...);
// A mutation occurs, invalidating what might be in cache.
$sql = 'UPDATE users WHERE id=4 SET username="zuck"';
if (true === $conn->query($sql)) {
$userLoader->clear(4);
}
// Later the value load is loaded again so the mutated data appears.
$userLoader->load(4)->then(...);
// Request completes.
Caching Errors
If a batch load fails (that is, a batch function throws or returns a rejected
Promise), then the requested values will not be cached. However if a batch
function returns an Error instance for an individual value, that Error will
be cached to avoid frequently loading the same Error., (*20)
In some circumstances you may wish to clear the cache for these individual Errors:, (*21)
$userLoader->load(1)->then(null, function ($exception) {
if (/* determine if error is transient */) {
$userLoader->clear(1);
}
throw $exception;
});
Disabling Cache
In certain uncommon cases, a DataLoader which does not cache may be desirable.
Calling new DataLoader(myBatchFn, new Option(['cache' => false ])) will ensure that every
call to ->load() will produce a new Promise, and requested keys will not be
saved in memory., (*22)
However, when the memoization cache is disabled, your batch function will
receive an array of keys which may contain duplicates! Each key will be
associated with each call to ->load(). Your batch loader should provide a value
for each instance of the requested key., (*23)
For example:, (*24)
$myLoader = new DataLoader(function ($keys) {
echo json_encode($keys);
return someBatchLoadFn($keys);
}, $promiseAdapter, new Option(['cache' => false ]));
$myLoader->load('A');
$myLoader->load('B');
$myLoader->load('A');
// [ 'A', 'B', 'A' ]
More complex cache behavior can be achieved by calling ->clear() or ->clearAll()
rather than disabling the cache completely. For example, this DataLoader will
provide unique keys to a batch function due to the memoization cache being
enabled, but will immediately clear its cache when the batch function is called
so later requests will load new values., (*25)
$myLoader = new DataLoader(function($keys) use ($identityLoader) {
$identityLoader->clearAll();
return someBatchLoadFn($keys);
}, $promiseAdapter);
API
class DataLoader
DataLoaderPHP creates a public API for loading data from a particular
data back-end with unique keys such as the id column of a SQL table or
document name in a MongoDB database, given a batch loading function., (*26)
Each DataLoaderPHP instance contains a unique memoized cache. Use caution when
used in long-lived applications or those which serve many users with different
access permissions and consider creating a new instance per web request., (*27)
new DataLoader(callable $batchLoadFn, PromiseAdapterInterface $promiseAdapter [, Option $options])
Create a new DataLoaderPHP given a batch loading instance and options., (*28)
load($key)
Loads a key, returning a Promise for the value represented by that key., (*35)
-
$key: An key value to load.
loadMany($keys)
Loads multiple keys, promising an array of values:, (*36)
list($a, $b) = DataLoader::await($myLoader->loadMany(['a', 'b']));
This is equivalent to the more verbose:, (*37)
list($a, $b) = DataLoader::await(\React\Promise\all([
$myLoader->load('a'),
$myLoader->load('b')
]));
-
$keys: An array of key values to load.
clear($key)
Clears the value at $key from the cache, if it exists. Returns itself for
method chaining., (*38)
-
$key: An key value to clear.
clearAll()
Clears the entire cache. To be used when some event results in unknown
invalidations across this particular DataLoaderPHP. Returns itself for
method chaining., (*39)
prime($key, $value)
Primes the cache with the provided key and value. If the key already exists, no
change is made. (To forcefully prime the cache, clear the key first with
$loader->clear($key)->prime($key, $value). Returns itself for method chaining., (*40)
static await([$promise][, $unwrap])
You can synchronously force promises to complete using DataLoaderPHP's await method.
When an await function is invoked it is expected to deliver a value to the promise or reject the promise.
Await method process all waiting promise in all dataLoaderPHP instances., (*41)
-
$promise: Optional promise to complete., (*42)
-
$unwrap: controls whether or not the value of the promise is returned for a fulfilled promise
or if an exception is thrown if the promise is rejected. Default true., (*43)
Using with Webonyx/GraphQL
DataLoader pairs nicely well with Webonyx/GraphQL. GraphQL fields are
designed to be stand-alone functions. Without a caching or batching mechanism,
it's easy for a naive GraphQL server to issue new database requests each time a
field is resolved., (*44)
Consider the following GraphQL request:, (*45)
{
me {
name
bestFriend {
name
}
friends(first: 5) {
name
bestFriend {
name
}
}
}
}
Naively, if me, bestFriend and friends each need to request the backend,
there could be at most 13 database requests!, (*46)
When using DataLoader, we could define the User type
at most 4 database requests,
and possibly fewer if there are cache hits., (*47)
<?php
use GraphQL\GraphQL;
use GraphQL\Type\Definition\ObjectType;
use GraphQL\Type\Definition\Type;
use Overblog\DataLoader\DataLoader;
use Overblog\DataLoader\Promise\Adapter\Webonyx\GraphQL\SyncPromiseAdapter;
use Overblog\PromiseAdapter\Adapter\WebonyxGraphQLSyncPromiseAdapter;
/**
* @var \PDO $dbh
*/
// ...
$graphQLPromiseAdapter = new SyncPromiseAdapter();
$dataLoaderPromiseAdapter = new WebonyxGraphQLSyncPromiseAdapter($graphQLPromiseAdapter);
$userLoader = new DataLoader(function ($keys) { /*...*/ }, $dataLoaderPromiseAdapter);
GraphQL::setPromiseAdapter($graphQLPromiseAdapter);
$userType = new ObjectType([
'name' => 'User',
'fields' => function () use (&$userType, $userLoader, $dbh) {
return [
'name' => ['type' => Type::string()],
'bestFriend' => [
'type' => $userType,
'resolve' => function ($user) use ($userLoader) {
$userLoader->load($user['bestFriendID']);
}
],
'friends' => [
'args' => [
'first' => ['type' => Type::int() ],
],
'type' => Type::listOf($userType),
'resolve' => function ($user, $args) use ($userLoader, $dbh) {
$sth = $dbh->prepare('SELECT toID FROM friends WHERE fromID=:userID LIMIT :first');
$sth->bindParam(':userID', $user['id'], PDO::PARAM_INT);
$sth->bindParam(':first', $args['first'], PDO::PARAM_INT);
$friendIDs = $sth->execute();
return $userLoader->loadMany($friendIDs);
}
]
];
}
]);
You can also see an example., (*48)
Using with Symfony
See the bundle., (*49)
Credits
Overblog/DataLoaderPHP is a port of dataLoader NodeJS version
by Facebook., (*50)
Also, large parts of the documentation have been ported from the dataLoader NodeJS version
Docs., (*51)
License
Overblog/DataLoaderPHP is released under the MIT license., (*52)
Wallogit.com
