Derafu Query

Let your users write queries in the URL, without opening your SQL.

A query that travels in a URL

A data table, a report or an API client lets people filter your data, such as a grid made with DataTables or Tabulator. The easy ways to build it are to give them SQL, which is not safe, or to code every filter by hand, which does not end.

Derafu Query gives them a short, URL-safe expression instead. It travels in the URL, its values are always bound as parameters, and you decide which columns, paths and operators are allowed. For the queries that you write yourself, use your ORM or SQL: that is not what it is for.

How it works

From the user in the browser to your database, with you in the middle.

The user filters

A data table, a report or an API client builds an expression such as total?>1000 and sends it in the URL.

You decide what is allowed

Your application accepts only the columns, the paths and the operators that each endpoint exposes.

Derafu Query reads it

The value is checked against the pattern of its operator, the names are sanitized, and every value will be a bound parameter.

It joins your query

The condition, and the joins that its path needs, are added to your Doctrine, Eloquent or API Platform query, or to the builder of the package.

Where it plugs in

You keep the query builder that you already have.

What the user can say

Joins, operators, combinations and subqueries, in one string that fits in a URL.

// The filters of the user travel in the URL, for example from a data table.
// In API Platform, each parameter of the URL is a filter on one property.
// The value is joined to the property and read as an expression, so a value can add
// conditions on other paths with && or ||: declaring properties is not an allow-list.
#[ApiResource]
#[QueryParameter(key: 'price', property: 'price', filter: SmartFilter::class)]
#[QueryParameter(key: 'status', property: 'status', filter: SmartFilter::class)]
class Product
{
    // ...
}

// GET /api/products?price=>1000&status=in:paid,issued
//   price?>1000  and  status?in:paid,issued

// Or one parameter that takes a whole expression. Encode it as one value,
// so that "&&" is not read as the separator of the URL parameters:
#[QueryParameter(key: 'filter', property: '__derafu_smart_filter', filter: SmartFilter::class)]

// GET /api/products?filter=status%3F%3Dactive%26%26price%3F%3E1000
# An expression is a path and a filter, separated by "?". It is plain text, so
# it fits in a URL (encode a whole expression as one value).
#
#   customers __ invoices[on:id=customer_id] __ total ? >1000
#   first table   join and its condition       column   filter
#
# The path:
#   segments are separated by "__": the first is the table, the middle ones are
#   joined, the last one is the column.
#   [on:id=customer_id]  the join condition: "id" of the previous table is
#                        "customer_id" of this one
#   [alias:c]            an alias for the table
#   [join:left]          the type of join: inner (the default), left or right
#
# The filter: an operator and a value, such as ">1000", "in:a,b,c" or "is:null".

$qb->where('customers__invoices[on:id=customer_id]__total?>1000');

# The builder writes:
#   INNER JOIN invoices ON customers.id = invoices.customer_id
#   WHERE invoices.total > :param      (1000 is a bound parameter)
price?=1000                  price = :p
status?!=archived            status != :p
total?>=500                  total >= :p

name?^John                   name LIKE :p           (:p = 'John%')
description?~~keyword        description LIKE :p    (:p = '%keyword%')
title?!~~*draft              title NOT ILIKE :p     (:p = '%draft%')   PostgreSQL

status?in:paid,issued        a list of values
created_at?date:20240101     a date
deleted_at?is:null           no value at all

# ^ starts with, $ ends with, ~~ contains, and the ones that end with * ignore the case.
# The builder writes ILIKE, or the equivalent, for each database.
# The catalogue also has ranges, regular expressions and bitwise operators,
# and you can use your own catalogue.
# && is AND, || is OR, and parentheses group, as in any language.

status?=active&&total?>1000
    status = :p1 AND total > :p2

category?=electronics||category?=hardware
    category = :p1 OR category = :p2

status?=active&&(type?=person||tax_id?^78)
    status = :p1 AND (type = :p2 OR tax_id LIKE :p3)

# && binds tighter than ||.

# Or pass a list: each element is added with AND. Handy when the filters are dynamic.
$qb->where(['status?=active', 'total?>1000']);
# A path that starts with "___" is a subquery instead of a join.
# It filters by the children of a row without repeating the row.

# Invoices that have a pending payment:
___payments[on:id=invoice_id]__status?=pending
    EXISTS (SELECT 1 FROM payments
            WHERE invoices.id = payments.invoice_id AND payments.status = :p)

# Invoices with no payments at all:
___payments[on:id=invoice_id]?is:empty
    NOT EXISTS (SELECT 1 FROM payments WHERE ...)

# Aggregates: SUM, AVG, COUNT, MIN and MAX.
___payments[on:id=invoice_id]__SUM(amount)?>=1200
    (SELECT SUM(p.amount) FROM payments p WHERE p.invoice_id = invoices.id) >= :p
// With your own builder: you keep it, and Derafu Query adds the conditions.
// The same two lines work with Doctrine DBAL, Doctrine ORM and Eloquent.
$condition = $parser->parse('status?=active&&total?>1000');

$applier = new IlluminateQueryBuilderConditionApplier();
$builder = Invoice::query();
$applier->apply($builder, $condition);
$invoices = $builder->get();

// With the builder of the package, from a configuration in an array, YAML or JSON:
$config = new QueryConfig([
    'table'   => 'products',
    'select'  => 'id, name, price',
    'where'   => 'category?=electronics',
    'orderBy' => ['price' => 'DESC'],
    'limit'   => 10,
]);
$result = $config->applyTo($queryBuilder)->execute();

Frequently asked questions

What it does, and what it leaves to you.

It is safe against SQL injection: the values are bound as parameters, the names are sanitized and the value of each operator is validated. But safe is not the same as authorized. A valid expression can name any table or column that the database user can read. In API Platform, declaring the properties of a resource does not change that: the value of a parameter is joined to its property and read as an expression, so it can add conditions on other paths. You have to allow only the columns, the paths and the operators of each endpoint, keep the restrictions of tenant and owner in conditions that the server controls, and limit the length of the lists. The package does not do that for you: it is the job of the application.

No. For those, the ORM is the better choice, or SQL directly if you have no ORM or you do not like it. Derafu Query is for the queries that a user writes, in a URL.

Three things take raw SQL, so never give them input from a user: the condition of an explicit join(), the ?E marker (the value is a column or an expression, not a bound parameter) and select() with sanitize: false. The joins that are written in the path with on: are parsed and sanitized.

The parser checks it against the pattern of its operator before any SQL is made. A date that is not a date, an empty list or a decimal in a bitwise operator throws an InvalidArgumentException that you can log. The API Platform filter is the exception: it skips an invalid expression without a message, and the request is answered without that filter.

The SQL is written for PostgreSQL, MySQL and SQLite, and the DBAL bridge also detects SQL Server and Oracle. The builder of the package runs on PDO or Doctrine DBAL, and the bridges apply the expressions to Doctrine DBAL, Doctrine ORM, Eloquent and API Platform. DQL cannot write the date, bitwise, regular expression and ILIKE operators, so the ORM bridge throws an exception when it receives one. The API Platform filter skips it without a message, and the request is answered without that filter: keep that in mind if the filter is a restriction.

No. Doctrine, Illuminate and API Platform are optional: you install only the one that you use. The package itself needs PHP 8.5 or later, and the Derafu packages are published as dev-main for now.

Where it fits

Other good tools exist. Choose by how you work.

Approach Examples Choose it when Derafu Query
The queries that you write Your ORM, or SQL Always, when you are the one who writes the query. It is not for that. Use it when the query is written by a user.
Filters from an allow-list spatie/laravel-query-builder (Laravel) You want to declare which fields can be filtered, and nothing more, in Laravel. A free expression with joins and subqueries that you restrict yourself. It is not tied to one framework.
Query languages that only parse RQL parsers, search-syntax-parser You need a syntax tree to translate yourself. It also writes the SQL, with the joins, and applies it to your query builder.
Search systems Rollerworks Search You want a complete search system, with input formats and sets of fields. A compact expression instead of sets of fields, with the joins in the path.
A standard for URL queries OData You want the standard and its clients. A smaller syntax made for SQL, with the joins in the path.

Install it

Start with the library, and add the bridge that you need.

# The Derafu packages are published as dev-main for now, so your composer.json needs:
#   "minimum-stability": "dev", "prefer-stable": true
composer require derafu/query

# Optional, according to what you use:
#   doctrine/dbal, doctrine/orm, illuminate/database, api-platform/core

# In a Symfony application:
composer require derafu/symfony-query-bundle

Next: the documentation has the whole list of operators, and the security guide explains what you must restrict.

Ready to write one expression?

Start with the syntax of an expression, then read the security guide before you open it to your users.