Derafu Query
A query that travels in a URL
A data table, a report or an API client lets people filter your data, such as a grid made with DataTables or Tabulator. The easy ways to build it are to give them SQL, which is not safe, or to code every filter by hand, which does not end.
Derafu Query gives them a short, URL-safe expression instead. It travels in the URL, its values are always bound as parameters, and you decide which columns, paths and operators are allowed. For the queries that you write yourself, use your ORM or SQL: that is not what it is for.
How it works
From the user in the browser to your database, with you in the middle.
The user filters
total?>1000 and sends it in the URL.
You decide what is allowed
Derafu Query reads it
It joins your query
Where it plugs in
You keep the query builder that you already have.
What the user can say
Joins, operators, combinations and subqueries, in one string that fits in a URL.
// The filters of the user travel in the URL, for example from a data table.
// In API Platform, each parameter of the URL is a filter on one property.
// The value is joined to the property and read as an expression, so a value can add
// conditions on other paths with && or ||: declaring properties is not an allow-list.
#[ApiResource]
#[QueryParameter(key: 'price', property: 'price', filter: SmartFilter::class)]
#[QueryParameter(key: 'status', property: 'status', filter: SmartFilter::class)]
class Product
{
// ...
}
// GET /api/products?price=>1000&status=in:paid,issued
// price?>1000 and status?in:paid,issued
// Or one parameter that takes a whole expression. Encode it as one value,
// so that "&&" is not read as the separator of the URL parameters:
#[QueryParameter(key: 'filter', property: '__derafu_smart_filter', filter: SmartFilter::class)]
// GET /api/products?filter=status%3F%3Dactive%26%26price%3F%3E1000
# An expression is a path and a filter, separated by "?". It is plain text, so
# it fits in a URL (encode a whole expression as one value).
#
# customers __ invoices[on:id=customer_id] __ total ? >1000
# first table join and its condition column filter
#
# The path:
# segments are separated by "__": the first is the table, the middle ones are
# joined, the last one is the column.
# [on:id=customer_id] the join condition: "id" of the previous table is
# "customer_id" of this one
# [alias:c] an alias for the table
# [join:left] the type of join: inner (the default), left or right
#
# The filter: an operator and a value, such as ">1000", "in:a,b,c" or "is:null".
$qb->where('customers__invoices[on:id=customer_id]__total?>1000');
# The builder writes:
# INNER JOIN invoices ON customers.id = invoices.customer_id
# WHERE invoices.total > :param (1000 is a bound parameter)
price?=1000 price = :p
status?!=archived status != :p
total?>=500 total >= :p
name?^John name LIKE :p (:p = 'John%')
description?~~keyword description LIKE :p (:p = '%keyword%')
title?!~~*draft title NOT ILIKE :p (:p = '%draft%') PostgreSQL
status?in:paid,issued a list of values
created_at?date:20240101 a date
deleted_at?is:null no value at all
# ^ starts with, $ ends with, ~~ contains, and the ones that end with * ignore the case.
# The builder writes ILIKE, or the equivalent, for each database.
# The catalogue also has ranges, regular expressions and bitwise operators,
# and you can use your own catalogue.
# && is AND, || is OR, and parentheses group, as in any language.
status?=active&&total?>1000
status = :p1 AND total > :p2
category?=electronics||category?=hardware
category = :p1 OR category = :p2
status?=active&&(type?=person||tax_id?^78)
status = :p1 AND (type = :p2 OR tax_id LIKE :p3)
# && binds tighter than ||.
# Or pass a list: each element is added with AND. Handy when the filters are dynamic.
$qb->where(['status?=active', 'total?>1000']);
# A path that starts with "___" is a subquery instead of a join.
# It filters by the children of a row without repeating the row.
# Invoices that have a pending payment:
___payments[on:id=invoice_id]__status?=pending
EXISTS (SELECT 1 FROM payments
WHERE invoices.id = payments.invoice_id AND payments.status = :p)
# Invoices with no payments at all:
___payments[on:id=invoice_id]?is:empty
NOT EXISTS (SELECT 1 FROM payments WHERE ...)
# Aggregates: SUM, AVG, COUNT, MIN and MAX.
___payments[on:id=invoice_id]__SUM(amount)?>=1200
(SELECT SUM(p.amount) FROM payments p WHERE p.invoice_id = invoices.id) >= :p
// With your own builder: you keep it, and Derafu Query adds the conditions.
// The same two lines work with Doctrine DBAL, Doctrine ORM and Eloquent.
$condition = $parser->parse('status?=active&&total?>1000');
$applier = new IlluminateQueryBuilderConditionApplier();
$builder = Invoice::query();
$applier->apply($builder, $condition);
$invoices = $builder->get();
// With the builder of the package, from a configuration in an array, YAML or JSON:
$config = new QueryConfig([
'table' => 'products',
'select' => 'id, name, price',
'where' => 'category?=electronics',
'orderBy' => ['price' => 'DESC'],
'limit' => 10,
]);
$result = $config->applyTo($queryBuilder)->execute();
Frequently asked questions
What it does, and what it leaves to you.
join(), the ?E marker (the value is a column or an expression, not a bound parameter) and select() with sanitize: false. The joins that are written in the path with on: are parsed and sanitized.
InvalidArgumentException that you can log. The API Platform filter is the exception: it skips an invalid expression without a message, and the request is answered without that filter.
dev-main for now.
Where it fits
Other good tools exist. Choose by how you work.
| Approach | Examples | Choose it when | Derafu Query |
|---|---|---|---|
| The queries that you write | Your ORM, or SQL | Always, when you are the one who writes the query. | It is not for that. Use it when the query is written by a user. |
| Filters from an allow-list | spatie/laravel-query-builder (Laravel) | You want to declare which fields can be filtered, and nothing more, in Laravel. | A free expression with joins and subqueries that you restrict yourself. It is not tied to one framework. |
| Query languages that only parse | RQL parsers, search-syntax-parser | You need a syntax tree to translate yourself. | It also writes the SQL, with the joins, and applies it to your query builder. |
| Search systems | Rollerworks Search | You want a complete search system, with input formats and sets of fields. | A compact expression instead of sets of fields, with the joins in the path. |
| A standard for URL queries | OData | You want the standard and its clients. | A smaller syntax made for SQL, with the joins in the path. |
Install it
Start with the library, and add the bridge that you need.
# The Derafu packages are published as dev-main for now, so your composer.json needs:
# "minimum-stability": "dev", "prefer-stable": true
composer require derafu/query
# Optional, according to what you use:
# doctrine/dbal, doctrine/orm, illuminate/database, api-platform/core
# In a Symfony application:
composer require derafu/symfony-query-bundle
Next: the documentation has the whole list of operators, and the security guide explains what you must restrict.
Ready to write one expression?
Start with the syntax of an expression, then read the security guide before you open it to your users.