--- title: "Data Handling Category" description: "Data Handling" type: "docs" category: "doc" tags: [] authors: [Anonymous] date: "2026-09-09" last_update: "2026-09-09" time_minutes: 1 draft: false unlisted: false url: "https://www.derafu.dev/docs/data" --- # Data Handling --- ## ORM Project Models and Entities # Models and Entities ![GitHub last commit](https://img.shields.io/github/last-commit/derafu/orm/main) ![CI Workflow](https://github.com/derafu/orm/actions/workflows/ci.yml/badge.svg?branch=main&event=push) ![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/derafu/orm) ![GitHub Issues](https://img.shields.io/github/issues-raw/derafu/orm) ![Total Downloads](https://poser.pugx.org/derafu/orm/downloads) ![Monthly Downloads](https://poser.pugx.org/derafu/orm/d/monthly) > Since 2006, [I participated in FLISoL](https://flisol.info/FLISOL2006/Chile/Valparaiso). In 2014, I presented [SowerPHP at UTEM (Santiago, Chile)](https://www.cnsl.cl/index.php/noticias/81-blog/isabel/1063-programa-flisol-santiago-2014). Years before, I presented MiPaGiNa at the University of Chile. On that occasion, someone in the audience said to me "Oh, you built an ORM," and I said, "What is that?" That day, during the presentation I was giving, I learned what an ORM was. This project compiles the ideas that have appeared in my mind over the years. Always with the goal of making everything "automagical." > > -- <cite>Esteban De La Fuente Rubio</cite> ## Ideas > [!NOTE] WIP > > Work In Progress. --- ## Query Project Derafu Query # Derafu Query --- ### Introduction Expressive Path-Based Query Builder for PHP # Expressive Path-Based Query Builder for PHP ![GitHub last commit](https://img.shields.io/github/last-commit/derafu/query/main) ![CI Workflow](https://github.com/derafu/query/actions/workflows/ci.yml/badge.svg?branch=main&event=push) ![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/derafu/query) ![GitHub Issues](https://img.shields.io/github/issues-raw/derafu/query) ![Total Downloads](https://poser.pugx.org/derafu/query/downloads) ![Monthly Downloads](https://poser.pugx.org/derafu/query/d/monthly) `derafu/query` is a PHP library for building and filtering SQL queries using a compact, URL-safe string syntax. A single expression like `customers[alias:c]__invoices[on:id=customer_id,alias:i]__total?>1000` carries the full join path, join conditions, and filter in one string — no separate join calls required. ## Why Derafu\Query? Traditional query builders require explicit join definitions, verbose relationship navigation, and different filter syntaxes across engines. `derafu/query` replaces all of that with: - A **single expression format** that encodes path, joins, and filter in one string. - **Automatic join generation** from path segments — no manual `join()` calls when using path syntax. - **50+ operators** covering comparisons, patterns, lists, ranges, dates, NULL, regex, bitwise, and subqueries. - **Database-specific SQL** generated from the same expression — PostgreSQL, MySQL, and SQLite without code changes. - **Framework bridges** for Doctrine DBAL, Doctrine ORM, Laravel's Illuminate query builder, and API Platform. | Feature | Derafu\Query | Traditional Query Builders | |------------------------------|:---:|:---:| | Path-Based Relationships | ✅ | ❌ | | Automatic Join Resolution | ✅ | ❌ | | Configurable Operators | ✅ | ❌ | | Framework Agnostic Core | ✅ | ⚠️ | | Unified Filter Syntax | ✅ | ⚠️ | | Multi-DB SQL Generation | ✅ | ⚠️ | --- ## Installation ```bash composer require derafu/query ``` --- ## Quick Start ### With SqlQueryBuilder (standalone) ```php use Derafu\Query\Builder\SqlQueryBuilder; use Derafu\Query\Engine\PdoEngine; use Derafu\Query\Filter\ExpressionParser; use Derafu\Query\Filter\PathParser; use Derafu\Query\Filter\FilterParser; use Derafu\Query\Filter\CompositeExpressionParser; use Derafu\Query\Operator\OperatorLoader; use Derafu\Query\Operator\OperatorManager; $pdo = new PDO('sqlite:/path/to/db.sqlite'); $engine = new PdoEngine($pdo); $loader = new OperatorLoader(); $manager = new OperatorManager($loader->loadFromFile('vendor/derafu/query/resources/operators.yaml')); $parser = new CompositeExpressionParser( new ExpressionParser(new PathParser(), new FilterParser($manager)) ); $qb = new SqlQueryBuilder($engine, $parser); // Simple filter. $rows = $qb->table('products')->where('price?>1000')->execute(); // Multi-table path (auto-generates the JOIN). $rows = $qb ->select('c.name AS customer_name, i.number, i.total') ->where('customers[alias:c]__invoices[on:id=customer_id,alias:i]__total?>1000') ->execute(); ``` ### With a Framework Bridge ```php // Doctrine DBAL. use Derafu\Query\Bridge\DoctrineDBALQueryBuilderConditionApplier; use Derafu\Query\Filter\CompositeExpressionParser; $applier = new DoctrineDBALQueryBuilderConditionApplier(); $condition = $parser->parse('status?=active&&total?>1000'); $applier->apply($dbalQueryBuilder, $condition); ``` --- ## The Expression Format Every filter is a string of the form: ``` path?filter ``` The `?` separates the **path** (which column or relationship to target) from the **filter** (which operator and value to apply). ``` price?>1000 ← column "price", operator ">", value "1000" status?in:paid,issued ← column "status", operator "in:", value "paid,issued" created_at?date:20240301 ← column "created_at", operator "date:", value "20240301" deleted_at?is:null ← column "deleted_at", operator "is:null", no value ``` Multi-segment paths navigate relationships and generate joins automatically: ``` customers[alias:c]__invoices[on:id=customer_id,alias:i]__total?>1000 ``` Subquery paths starting with `___` generate correlated EXISTS / aggregate subqueries: ``` ___payments[on:id=invoice_id]?is:empty ← NOT EXISTS (payments) ___payments[on:id=invoice_id]__status?=pending ← EXISTS with column filter ___payments[on:id=invoice_id]__SUM(amount)?>=500← aggregate scalar subquery ``` Multiple conditions can be combined with `&&` (AND), `||` (OR), and `()` (grouping): ``` status?=active&&total?>1000 category?=electronics||(category?=software&&price?<500) ``` --- ## Operator Overview `derafu/query` ships with over 50 operators across 10 types: | Type | Examples | SQL produced | |--------------|-------------------------------------------|-----------------------------------| | Standard | `=`, `!=`, `>`, `<`, `>=`, `<=` | `col = :p`, `col > :p` | | AutoLike | `^`, `$`, `~~`, `~~*`, `!~~` | `col LIKE :p` with auto `%` | | Like | `like:`, `ilike:`, `notlike:` | `col LIKE :p`, `col ILIKE :p` | | List | `in:`, `notin:` | `col IN (:p1, :p2, …)` | | Range | `between:`, `notbetween:` | `col BETWEEN :p1 AND :p2` | | Date | `date:`, `month:`, `year:`, `period:` | `DATE(col) = :p`, etc. | | NULL | `is:null`, `isnot:null`, `<=>` | `col IS NULL` | | Subquery | `is:empty`, `isnot:empty` | `NOT EXISTS (…)`, `EXISTS (…)` | | RegExp | `~`, `~*`, `!~`, `similarto:` | `col ~ :p`, `col SIMILAR TO :p` | | Binary | `b&`, `b\|`, `b^`, `b<<`, `b>>` | `col & :p`, etc. | --- ## What This Documentation Covers | Page | Content | |------|---------| | [Architecture](./architecture) | Layer structure, class responsibilities, data flow | | [Expression Syntax](./expression-syntax) | `path?filter` format, composite `&&`/`\|\|`/`()` | | [Path Syntax](./path-syntax) | Segments, options, join paths, subquery paths | | [Operators Reference](./operators) | All operators with SQL templates, validation, casting | | [Query Builder](./query-builder) | Fluent API and declarative `QueryConfig` | | [Framework Bridges](./bridges) | Doctrine DBAL/ORM, Illuminate, API Platform | | [Security Guide](./security-guide) | SQL sanitization and safe usage patterns | --- ### Architecture Architecture # Architecture `derafu/query` is organized into six cooperating layers. Understanding them helps you choose which classes to instantiate, which bridges to use, and where to add custom behavior. {.w-75 .mx-auto} ![Architecture diagram showing the six layers of derafu/query: Filter (parsing), Operator (management), Builder (SQL generation), Engine (execution), Bridge (framework integration), and Config (declarative queries). Arrows show the dependency flow from top to bottom.](https://www.derafu.dev/img/diagrams/content/docs/data/query/architecture-layers.svg) --- ## Layers at a Glance | Namespace | Responsibility | |-----------|---------------| | `Derafu\Query\Filter` | Parse string expressions into structured condition objects | | `Derafu\Query\Operator` | Load, validate, and manage operator definitions from YAML | | `Derafu\Query\Builder` | Build SQL strings and named-parameter arrays from conditions | | `Derafu\Query\Engine` | Execute SQL against a real database connection | | `Derafu\Query\Bridge` | Apply conditions to third-party query builders (Doctrine, Illuminate, etc.) | | `Derafu\Query\Config` | Load declarative query definitions from arrays, YAML, or JSON | --- ## Filter Layer The filter layer turns a raw string expression like `customers[alias:c]__invoices[on:id=customer_id,alias:i]__total?>1000` into an object tree. ### Key Classes | Class | Interface | Role | |-------|-----------|------| | `CompositeExpressionParser` | `CompositeExpressionParserInterface` | Entry point. Parses composite expressions with `&&`, `\|\|`, `()` into a tree of conditions. | | `ExpressionParser` | `ExpressionParserInterface` | Parses a single `path?filter` string into a `Condition`. | | `PathParser` | `PathParserInterface` | Splits the path part (`table__column`) into `Segment` objects. | | `FilterParser` | `FilterParserInterface` | Matches the filter part (`>1000`) against known operators. | | `Path` | `PathInterface` | Immutable value object holding an ordered list of `Segment` objects. | | `Segment` | `SegmentInterface` | Immutable value object for one path segment: name + options. | | `Filter` | `FilterInterface` | Immutable value object: operator + raw value string. | | `Condition` | `ConditionInterface` | Ties a `Path` to a `Filter`. Carries a `literal` flag. | | `CompositeCondition` | `CompositeConditionInterface` | AND or OR container of `Condition`/`CompositeCondition` objects. | ### Parsing Pipeline {.w-75 .mx-auto} ![Diagram showing the parsing pipeline: raw string → CompositeExpressionParser splits on &&/|| → ExpressionParser splits on ? → PathParser creates Path/Segments, FilterParser creates Filter → Condition object returned.](https://www.derafu.dev/img/diagrams/content/docs/data/query/parsing-pipeline.svg) ``` "status?=active&&total?>1000" │ ▼ CompositeExpressionParser CompositeCondition (AND) ├── Condition │ ├── Path [Segment("status")] │ └── Filter [Operator("="), value="active"] └── Condition ├── Path [Segment("total")] └── Filter [Operator(">"), value="1000"] ``` The `literal` flag on `Condition` distinguishes normal filter values from expression references (used with the `?E` marker for self-referencing conditions like `id?E!=other_alias.id`). --- ## Operator Layer Operators are defined in `resources/operators.yaml` and loaded at startup. Each operator carries: - A **symbol** (e.g. `>=`, `like:`, `b&`) - A **type** classifying its behavior - Engine-specific **SQL templates** with `{{column}}` / `{{value}}` / `{{values}}` / `{{value_1}}` / `{{value_2}}` placeholders - An optional **validation pattern** (regex) for the value - Optional **casting rules** that transform the value before binding - An optional **alias** pointing to another operator's SQL template ### Key Classes | Class | Interface | Role | |-------|-----------|------| | `Operator` | `OperatorInterface` | Immutable value object for one operator definition | | `OperatorLoader` | `OperatorLoaderInterface` | Reads and validates `operators.yaml` | | `OperatorManager` | `OperatorManagerInterface` | Registry; provides operators sorted by symbol length for greedy matching | | `OperatorManagerFactory` | `OperatorManagerFactoryInterface` | Convenience factory: loader + manager in one call | --- ## Builder Layer The builder layer converts condition objects into SQL strings with named parameters. ### Key Classes | Class | Interface | Role | |-------|-----------|------| | `SqlBuilderWhere` | `QueryBuilderWhereInterface` | Converts a single `Condition` or `CompositeCondition` into a WHERE fragment + parameters | | `SqlQueryBuilder` | `QueryBuilderInterface` | Full query builder: SELECT, FROM, JOIN, WHERE, GROUP BY, HAVING, ORDER BY, LIMIT, OFFSET | | `SqlQuery` | `QueryInterface` | Immutable value object holding SQL string and parameters. Implements `ArrayAccess` for `$q['sql']` and `$q['parameters']`. | | `SqlSanitizerTrait` | — | Sanitizes identifiers and expressions; supports a custom quoting callback | ### SqlBuilderWhere Internals `SqlBuilderWhere` is constructed with a database driver name (`pgsql`, `mysql`, `sqlite`). For each condition it: 1. Resolves the effective SQL template (operator's own `sql` key, or its base operator's). 2. Validates the raw value against the operator's pattern (if defined). 3. Normalizes the value: booleans → `0`/`1`, arrays → delimiter-joined string. 4. Applies casting rules (e.g. `like_start` appends `%`, `date` normalizes `YYYYMMDD` → `YYYY-MM-DD`). 5. Creates named parameters (`param_{column}_{uniqid}`). 6. Replaces `{{column}}`, `{{value}}`, `{{values}}`, `{{value_1}}`, `{{value_2}}` in the template. For composite conditions it recursively builds each child and joins them with `AND` / `OR` inside parentheses. For EXISTS paths (starting with `___`) it generates correlated `EXISTS (SELECT 1 FROM …)` or `NOT EXISTS (…)` subqueries. Aggregate column segments like `SUM(amount)` produce scalar subqueries: `(SELECT SUM(p.amount) FROM payments p WHERE …) >= :param`. ### Column Resolution from Paths When a path has more than one segment, `SqlBuilderWhere` qualifies the column with the previous segment's alias (or name): ``` authors__books__title → books.title authors[alias:a]__books[alias:b]__title → b.title ``` For `SqlQueryBuilder`, intermediate segments automatically become JOIN clauses (INNER by default; override with `join:left` option). --- ## Engine Layer The engine layer executes the SQL produced by the builder against a real connection. | Class | Interface | Role | |-------|-----------|------| | `PdoEngine` | `SqlEngineInterface` | Wraps a `PDO` instance; prepares, executes, and fetches rows | | `DoctrineEngine` | `SqlEngineInterface` | Wraps a Doctrine DBAL `Connection` | | `AbstractSqlEngine` | — | Shared `getDriver()` detection logic | --- ## Bridge Layer Bridges apply parsed conditions to existing third-party query builder instances, without requiring the full `SqlQueryBuilder`. | Class | Target QB | Notes | |-------|-----------|-------| | `DoctrineDBALQueryBuilderConditionApplier` | Doctrine DBAL `QueryBuilder` | Resolves driver via reflection; handles FROM inference and JOIN deduplication | | `DoctrineORMQueryBuilderConditionApplier` | Doctrine ORM `QueryBuilder` | Generates DQL; rewrites single-segment paths to qualify with root alias; throws `UnsupportedOperatorException` for DQL-incompatible operators | | `IlluminateQueryBuilderConditionApplier` | Illuminate `Query\Builder` / `Eloquent\Builder` | Accepts both; resolves Eloquent to base via `toBase()` | | `SmartFilter` (API Platform) | Doctrine ORM `QueryBuilder` | Wraps `DoctrineORMQueryBuilderConditionApplier` for use as an API Platform `#[QueryParameter]` filter | All bridge appliers share `ConditionApplierTrait`, which provides: - `buildConditionSql()` — compiles to SQL via `SqlBuilderWhere` - `extractPaths()` — collects all paths from a condition tree - `inferFromTable()` — detects the base table from multi-segment paths - `buildJoinSpecsFromPaths()` — builds JOIN specs for SQL-level bridges - `buildOrmJoinSpecsFromPaths()` — builds JOIN specs for Doctrine ORM DQL --- ## Config Layer The config layer provides a declarative alternative to the fluent builder API. | Class | Role | |-------|------| | `QueryConfig` | Wraps an array; calls builder methods when `applyTo($builder)` is invoked | | `YamlConfigLoader` | Loads YAML files or strings into arrays | | `JsonConfigLoader` | Loads JSON files or strings into arrays | `QueryConfig::fromFile()` auto-detects YAML vs JSON by file extension. --- ## Data Flow Summary {.w-75 .mx-auto} ![End-to-end data flow diagram: user string expression → CompositeExpressionParser → condition tree → SqlBuilderWhere (or bridge) → SQL + named parameters → PDO/Doctrine/Illuminate → result rows.](https://www.derafu.dev/img/diagrams/content/docs/data/query/data-flow.svg) ``` User provides string expression │ ▼ CompositeExpressionParser ├── ExpressionParser │ ├── PathParser → Path (Segments) │ └── FilterParser → Filter (Operator + value) └── returns Condition | CompositeCondition │ ▼ SqlQueryBuilder (or a Bridge applier) └── SqlBuilderWhere ├── Resolves SQL template (operator + engine) ├── Normalizes & casts value ├── Creates named parameters └── Returns SqlQuery { sql, parameters } │ ▼ Engine::execute(sql, parameters) │ ▼ array of result rows ``` --- ### Expression Syntax Expression Syntax # Expression Syntax An expression is the fundamental unit passed to `where()`, `andWhere()`, `orWhere()`, `having()`, and the bridge appliers. It encodes **what** to filter, **how** to filter it, and (optionally) how to combine multiple filters into one string. --- ## Single Expression: `path?filter` A single expression has the form: ``` path?filter ``` The `?` is the mandatory separator between the **path** (which column or relationship to target) and the **filter** (which operator and value to apply). ``` price?>1000 ^───── ^──── path filter ``` ### Path The path identifies a column. For a simple column in the driving table, it is just the column name: ``` price status created_at ``` For a column in a related table, segments are chained with double underscores `__`: ``` invoices__customers__name ``` For more details about the path syntax — including aliases, join conditions, and subquery paths — see [Path Syntax](./path-syntax). ### Filter The filter starts with an operator symbol immediately followed by the value (if any): ``` >1000 ← operator ">", value "1000" =active ← operator "=", value "active" in:a,b,c ← operator "in:", value "a,b,c" is:null ← operator "is:null", no value (the value part is empty) ^^start ← NOT valid — "^^" is not a registered operator ``` The `FilterParser` tries registered operators longest-first to avoid ambiguity between, for example, `!~*` and `!~`. For the complete list of operators with their symbols, value formats, and SQL output, see [Operators Reference](./operators). --- ## Composite Expressions Multiple conditions can be combined in a single string using: | Operator | Meaning | Precedence | |----------|---------|------------| | `&&` | AND | Higher | | `\|\|` | OR | Lower | | `(` `)` | Grouping | Overrides default precedence | The grammar follows standard boolean precedence: `&&` binds tighter than `||`. ``` A && B || C is parsed as (A && B) || C A || B && C is parsed as A || (B && C) (A || B) && C grouping overrides, result: (A || B) && C ``` ### Examples Simple AND: ``` status?=active&&total?>1000 ``` Produces: `status = :p1 AND total > :p2` Simple OR: ``` category?=electronics||category?=hardware ``` Produces: `category = :p1 OR category = :p2` AND with nested OR: ``` status?=active&&(type?=person||tax_id?^78) ``` Produces: `status = :p1 AND (type = :p2 OR tax_id LIKE :p3)` OR with nested AND: ``` category?=software||(category?=hardware&&price?>200) ``` Produces: `category = :p1 OR (category = :p2 AND price > :p3)` ### Parser Rules - `&&` and `||` are only split at **parenthesis depth 0**, so function calls like `SUM(amount)` inside paths are not split. - Whitespace around `&&` and `||` is trimmed. - A fully parenthesized expression like `(A&&B)` is unwrapped and parsed recursively. --- ## Multiple Expressions as an Array Instead of combining with `&&`/`||` in a string, you can pass an array of expressions to `where()` or `andWhere()`. Each element is ANDed together: ```php $qb->where(['status?=active', 'total?>1000']); // Equivalent to: status?=active&&total?>1000 ``` This is useful when expressions are generated dynamically: ```php $filters = []; if ($status) { $filters[] = 'status?=' . $status; } if ($minTotal) { $filters[] = 'total?>=' . $minTotal; } $qb->where($filters); ``` --- ## The `?E` Literal-Expression Marker By default the value part of a filter is treated as a **literal** — it becomes a bound parameter: ``` price?>1000 → price > :param with :param = '1000' ``` To reference another column or expression (not a literal value), replace `?` with `?E`: ``` id?E!=other_alias.id ``` This tells the parser that the value is an SQL identifier or expression, not a bound parameter. The value is sanitized as an identifier and inserted directly into the SQL — **no parameter binding**. Use this sparingly and only for trusted, controlled inputs, since the sanitizer strips characters but does not provide the same guarantees as prepared statement binding. Example from the test suite — simulating a self-join: ```php $qb->where([ 'products[alias:p1]__invoice_details[on:id=product_id,alias:id1]__invoice_id?isnot:null', 'products[alias:p1]__invoice_details[...alias:id2]__products[...alias:p2]__id?E!=p1.id', ]); ``` The second condition generates `p2.id != p1.id` (column reference, not a literal). --- ## How Expressions Are Parsed The entry point is `CompositeExpressionParser::parse(string $expression)`. 1. Trim whitespace. 2. Split on `||` at depth 0 → if more than one part, build an OR composite. 3. For each part, split on `&&` at depth 0 → if more than one part, build an AND composite. 4. For each atom, if wrapped in `(…)` unwrap and recurse; otherwise, call `ExpressionParser::parse()`. `ExpressionParser::parse()`: 1. Look for `?E` marker → set `literal = false`, replace `?E` with `?`. 2. Split on the first `?` → `pathExpression` and `filterExpression`. 3. Call `PathParser::parse(pathExpression)` → `Path`. 4. Call `FilterParser::parse(filterExpression)` → `Filter`. 5. Return `new Condition(path, filter, literal)`. `FilterParser::parse()`: 1. Retrieve operators sorted longest-first from `OperatorManager`. 2. Try each symbol as a prefix of `filterExpression`. 3. On match: extract value, create `Filter`, call `validate()`, return. 4. No match: throw `InvalidArgumentException`. --- ## Validation at Parse Time The `Filter::validate()` method checks the value against the operator's `pattern` field (if defined): - Operators whose symbol ends in `:` (e.g. `like:`, `in:`, `between:`) always require a non-empty value. - Specific patterns: dates must match `YYYYMMDD` or `YYMMDD`, bitwise values must be integers, list values must match the allowed character set. - NULL operators (`is:null`, `isnot:null`, `is:empty`, `isnot:empty`) require an **empty** value — the operator symbol itself is the full expression. Invalid expressions throw `InvalidArgumentException` immediately, before any SQL is generated. --- ### Path Syntax Path Syntax # Path Syntax The **path** is the left-hand side of a `path?filter` expression. It identifies which column to apply the filter to, and optionally encodes the JOIN chain required to reach that column. --- ## Simple Paths (Single Segment) A simple path is just a column name: ``` price status created_at deleted_at ``` When used with `SqlQueryBuilder.where()`, the column is unqualified — it refers to a column in the driving table set via `table()` or `from()`. When used with the Doctrine ORM bridge, single-segment paths are automatically qualified with the root entity alias (e.g. `status` → `c.status`). --- ## Multi-Segment Paths (Join Chains) Segments are separated by double underscores `__`. The first segment is the **base table**, intermediate segments are **join targets**, and the last segment is the **column**: ``` customers__name invoices__customers__name products__invoice_details__invoices__customers__type ``` ``` customers __ name ^─ table ^─ column invoices __ customers __ name ^─ table ^─ join target ^─ column ``` When `SqlQueryBuilder` processes a multi-segment path, it automatically generates INNER JOIN clauses for all intermediate segments that carry `on:` options. The base table is inferred from the first segment when no explicit `table()` call has been made. ### Column Qualification `SqlBuilderWhere` qualifies the column with the alias (or name) of the segment immediately before it: ``` invoices__customers__name → customers.name invoices[alias:i]__customers[alias:c]__name → c.name ``` --- ## Segment Options Each segment can carry metadata inside square brackets `[key:value,key2:value2]`: ``` customers[alias:c] invoices[on:id=customer_id,alias:i,join:left] ``` ### `alias:value` Sets an alias for the table in the generated SQL. The alias is used for column qualification, JOIN declarations, and correlation conditions. ``` customers[alias:c]__invoices[alias:i]__total → FROM customers AS c INNER JOIN invoices AS i … → i.total ``` ### `on:left_col=right_col` Defines the JOIN condition between the previous segment and this segment. `left_col` belongs to the previous table; `right_col` belongs to this table. ``` invoices__customers[on:customer_id=id]__name → … INNER JOIN customers ON invoices.customer_id = customers.id ``` Multiple `on:` pairs in the same brackets add multiple conditions (AND): ``` orders__items[on:order_id=id,on:branch_id=branch_id]__product → … INNER JOIN items ON orders.order_id = items.id AND orders.branch_id = items.branch_id ``` ### `join:type` Overrides the join type for this segment. Valid values: `inner` (default), `left`, `right`, `cross`. ``` customers__invoices[on:id=customer_id,join:left]__total → … LEFT JOIN invoices ON customers.id = invoices.customer_id ``` ### Combining Options All options can be combined in any order, separated by commas: ``` customers[alias:c]__invoices[on:id=customer_id,join:left,alias:i]__total ``` The segment above sets `alias=i`, `on.id=customer_id`, and `join=left` simultaneously. --- ## Exists/Subquery Paths (`___`) A path that starts with **triple underscore** `___` generates a correlated subquery instead of a JOIN. This is the mechanism for filtering by the existence or properties of child records. ### Basic Existence Check ``` ___payments[on:id=invoice_id] ``` No column segment → pure existence check. Combine with `is:empty` or `isnot:empty`: ``` ___payments[on:id=invoice_id]?is:empty → NOT EXISTS (SELECT 1 FROM payments WHERE …) ___payments[on:id=invoice_id]?isnot:empty → EXISTS (SELECT 1 FROM payments WHERE …) ``` ### Column Filter Inside the Subquery Adding a column segment after `__` filters on that column inside the `EXISTS`: ``` ___payments[on:id=invoice_id]__status?=pending → EXISTS (SELECT 1 FROM payments WHERE invoices.id = payments.invoice_id AND payments.status = :p) ``` Any operator can be used for the column filter inside the subquery: ``` ___payments[on:id=invoice_id]__amount?>500 ___payments[on:id=invoice_id]__method?in:card,transfer ___payments[on:id=invoice_id]__created_at?date:20240101 ``` ### Aggregate Scalar Subqueries When the column segment is an aggregate function call, a scalar subquery is generated: ``` ___payments[on:id=invoice_id]__SUM(amount)?>=1200 → (SELECT SUM(p.amount) FROM payments p WHERE p.invoice_id = invoices.id) >= :p ``` Supported aggregate functions: `SUM`, `AVG`, `COUNT`, `MIN`, `MAX`. ``` ___payments[on:id=invoice_id]__COUNT(*)?>1 ___invoices[on:id=customer_id]__AVG(total)?<1000 ___invoices[on:id=customer_id]__MIN(total)?>=100 ``` **`COUNT(*)` optimizations**: when the comparison is a pure existence check, it is automatically rewritten to `EXISTS` / `NOT EXISTS`: | Expression | Rewritten as | |------------|-------------| | `COUNT(*)?=0` | `NOT EXISTS (…)` | | `COUNT(*)?>0` | `EXISTS (…)` | | `COUNT(*)?!=0` | `EXISTS (…)` | | `COUNT(*)?>=1` | `EXISTS (…)` | | `COUNT(*)?<1` | `NOT EXISTS (…)` | | `COUNT(*)?<=0` | `NOT EXISTS (…)` | | `COUNT(*)?>1` | scalar subquery (kept as-is) | ### Aliases on Exists Segments You can add `alias:` to a subquery path segment: ``` ___payments[alias:p,on:id=invoice_id]__status?=pending → EXISTS (SELECT 1 FROM payments AS p WHERE invoices.id = p.invoice_id AND p.status = :p) ``` ### Nested Exists (Multi-Level) Chain multiple `___` separators to traverse deeper relationships: ``` ___payments[on:id=invoice_id]___items[on:id=payment_id]__amount?>100 ``` This generates an EXISTS subquery with an inner JOIN to `items`, filtering on `items.amount`. --- ## Function Notation in Paths The **last segment** may be an aggregate or SQL function call. This is used for HAVING conditions and similar: ``` AVG(price)?>500 ← in a having() call SUM(i.total)?>1000 ← qualified with a prior alias ``` When a parent segment exists, `SqlBuilderWhere` qualifies the function arguments automatically: ``` products[alias:p]__AVG(price)?>500 → AVG(p.price) > :p ``` --- ## Path Validation Rules `PathParser` enforces these rules: - A path cannot be empty. - Segment names cannot be empty — `author__` (trailing `__`) and `__books` (leading `__`) are invalid. - Segment names must start and end with `[a-zA-Z0-9_]` (closing `)` also allowed at the end, for function calls). - Option keys and values must both be non-empty. - An `on:` option value must contain `=` with non-empty parts on both sides. - Exists paths (`___`) must have a non-empty body after the triple underscore. --- ## Path Examples Reference | Path | Result | |------|--------| | `status` | `status` (no qualification) | | `invoices__status` | `invoices.status` | | `invoices[alias:i]__status` | `i.status` | | `customers[alias:c]__invoices[on:id=customer_id,alias:i]__total` | `i.total` with JOIN | | `products[alias:p]__invoice_details[on:id=product_id,alias:id]__invoices[on:invoice_id=id,alias:i]__customers[on:customer_id=id,alias:c]__name` | 3-level deep join | | `___payments[on:id=invoice_id]` | Subquery path, no column | | `___payments[on:id=invoice_id]__status` | Subquery path with column | | `___payments[on:id=invoice_id]__SUM(amount)` | Aggregate subquery | | `___payments[on:id=invoice_id]___items[on:id=payment_id]__amount` | Nested subquery | --- ## Paths in Doctrine ORM The Doctrine ORM bridge handles paths differently from SQL bridges: - **Single-segment paths** are automatically qualified with the root entity alias (e.g. `status` → `c.status`). - **Multi-segment paths** use the association name from Doctrine entity mappings, not SQL column names. The `on:` option is **ignored** — Doctrine resolves join conditions from the mapping. - **EXISTS paths** use DQL `SIZE(alias.assoc) = 0` for simple emptiness checks, and correlated `EXISTS(SELECT sub.id FROM EntityClass sub WHERE …)` for column filters. --- ### Operators Reference Operators Reference # Operators Reference Every operator has a **symbol** — the prefix that appears immediately after `?` in an expression. The filter parser matches operators longest-first, so `!~~*` is matched before `!~~` and `!~*` before `!~`. This page documents all built-in operators grouped by type. --- ## Value Format Conventions | Placeholder | Meaning | |-------------|---------| | `VALUE` | Any string value | | `PATTERN` | A LIKE pattern (may contain `%` and `_`) | | `DATE` | `YYYYMMDD` or `YYMMDD` | | `MONTH` | `MM` or `M` (1–12) | | `YEAR` | `YYYY` or `YY` | | `PERIOD` | `YYYYMM` or `YYMM` | | `INT` | Non-negative integer (no decimals) | | `V1,V2` | Two comma-separated values | | `V1,V2,…` | One or more comma-separated values | | _(empty)_ | No value; the symbol alone is the full filter | --- ## Standard Comparison Operators Map directly to SQL comparison operators. Accept any string or numeric value. | Symbol | Name | Value | SQL Template | |--------|------|-------|-------------| | `=` | Equals | `VALUE` | `{{column}} = {{value}}` | | `!=` | Not Equal | `VALUE` | `{{column}} != {{value}}` | | `!` | Not Equal (shorthand) | `VALUE` | alias of `!=` | | `<>` | Not Equal (SQL standard) | `VALUE` | alias of `!=` | | `>=` | Greater Than or Equal | `VALUE` | `{{column}} >= {{value}}` | | `<=` | Less Than or Equal | `VALUE` | `{{column}} <= {{value}}` | | `>` | Greater Than | `VALUE` | `{{column}} > {{value}}` | | `<` | Less Than | `VALUE` | `{{column}} < {{value}}` | ### Examples ``` price?=1000 → price = :p (:p = '1000') status?!=archived → status != :p (:p = 'archived') total?>=500 → total >= :p (:p = '500') created_at?<2025-01-01 → created_at < :p (:p = '2025-01-01') ``` **Tip**: Standard operators work with any data type — numbers, strings, ISO dates. The database engine applies its own type coercion. --- ## AutoLike Operators (Automatic Pattern Generation) These operators accept a plain text value and wrap it in `%` wildcards automatically before binding. They delegate to the `like:` / `ilike:` family for the actual SQL. ### Starts With | Symbol | Case | Value | Bound value | SQL | |--------|------|-------|-------------|-----| | `^` | Sensitive | `VALUE` | `VALUE%` | `col LIKE :p` (pgsql/sqlite), `col LIKE BINARY :p` (mysql) | | `^*` | Insensitive | `VALUE` | `VALUE%` | `col ILIKE :p` (pgsql), `col LIKE :p` (mysql/sqlite) | | `!^` | Sensitive | `VALUE` | `VALUE%` | `col NOT LIKE :p` | | `!^*` | Insensitive | `VALUE` | `VALUE%` | `col NOT ILIKE :p` / `col NOT LIKE :p` | ### Contains | Symbol | Case | Value | Bound value | SQL | |--------|------|-------|-------------|-----| | `~~` | Sensitive | `VALUE` | `%VALUE%` | `col LIKE :p` | | `~~*` | Insensitive | `VALUE` | `%VALUE%` | `col ILIKE :p` / `col LIKE :p` | | `!~~` | Sensitive | `VALUE` | `%VALUE%` | `col NOT LIKE :p` | | `!~~*` | Insensitive | `VALUE` | `%VALUE%` | `col NOT ILIKE :p` / `col NOT LIKE :p` | ### Ends With | Symbol | Case | Value | Bound value | SQL | |--------|------|-------|-------------|-----| | `$` | Sensitive | `VALUE` | `%VALUE` | `col LIKE :p` | | `$*` | Insensitive | `VALUE` | `%VALUE` | `col ILIKE :p` / `col LIKE :p` | | `!$` | Sensitive | `VALUE` | `%VALUE` | `col NOT LIKE :p` | | `!$*` | Insensitive | `VALUE` | `%VALUE` | `col NOT ILIKE :p` / `col NOT LIKE :p` | ### Examples ``` name?^John → name LIKE :p (:p = 'John%') name?$*LLC → name ILIKE :p (:p = '%LLC') (pgsql) description?~~keyword → description LIKE :p (:p = '%keyword%') title?!~~*draft → title NOT ILIKE :p (:p = '%draft%') (pgsql) ``` **Note on case sensitivity**: PostgreSQL natively supports `ILIKE`; MySQL uses `LIKE BINARY` for case-sensitive and plain `LIKE` for case-insensitive; SQLite uses plain `LIKE` for both (SQLite's `LIKE` is case-insensitive by default for ASCII). --- ## Pattern (LIKE) Operators Explicit LIKE operators where you supply the full pattern including `%` and `_` wildcards. | Symbol | Case | Value | SQL | |--------|------|-------|-----| | `like:` | Sensitive | `PATTERN` | pgsql: `col LIKE :p` · mysql: `col LIKE BINARY :p` · sqlite: `col LIKE :p` | | `notlike:` | Sensitive | `PATTERN` | pgsql: `col NOT LIKE :p` · mysql: `col NOT LIKE BINARY :p` | | `ilike:` | Insensitive | `PATTERN` | pgsql: `col ILIKE :p` · mysql: `col LIKE :p` · sqlite: `col LIKE :p` | | `notilike:` | Insensitive | `PATTERN` | pgsql: `col NOT ILIKE :p` · mysql/sqlite: `col NOT LIKE :p` | ### Examples ``` email?like:%.example.com → email LIKE :p (:p = '%.example.com') name?ilike:jo_n% → name ILIKE :p (:p = 'jo_n%') (pgsql) code?notlike:TEST_% → code NOT LIKE BINARY :p (mysql) ``` **Note**: `ilike:` and `notilike:` are **not supported** in Doctrine ORM DQL (only usable via SQL/DBAL/Illuminate bridges). --- ## List Operators Work with comma-separated lists of values. The list is split and each item becomes a separate named parameter. | Symbol | Value | SQL | |--------|-------|-----| | `in:` | `V1,V2,…` | `{{column}} IN (:p1, :p2, …)` | | `notin:` | `V1,V2,…` | `{{column}} NOT IN (:p1, :p2, …)` | ### Value Format Values are separated by commas. To include a literal comma in a value, escape it with `\,`: ``` status?in:active,pending,review → status IN (:p1, :p2, :p3) tags?in:a\,b,c\,d → tags IN (:p1, :p2) (:p1='a,b', :p2='c,d') ``` ### Validation The value must contain at least one item. Each item may contain word characters, dots, hyphens, and Unicode letters/marks. Whitespace inside values is not supported. ### Examples ``` status?in:paid,issued → status IN (:p1, :p2) category?notin:archived,draft → category NOT IN (:p1, :p2) id?in:1,2,3,4,5 → id IN (:p1, :p2, :p3, :p4, :p5) ``` --- ## Range Operators Filter values between two bounds (inclusive) or outside them. | Symbol | Value | SQL | |--------|-------|-----| | `between:` | `V1,V2` | `{{column}} BETWEEN {{value_1}} AND {{value_2}}` | | `notbetween:` | `V1,V2` | `{{column}} NOT BETWEEN {{value_1}} AND {{value_2}}` | ### Value Format Exactly two comma-separated values. Each value may contain word characters, dots, and hyphens (no spaces). ### Examples ``` price?between:100,500 → price BETWEEN :p1 AND :p2 price?notbetween:0,10 → price NOT BETWEEN :p1 AND :p2 created_at?between:2024-01-01,2024-12-31 → created_at BETWEEN :p1 AND :p2 ``` **Note**: `BETWEEN` is inclusive on both ends. For exclusive ranges, use `>` and `<` separately. --- ## Date Operators Specialized operators for date and time filtering. They apply SQL date functions and normalize the value. ### `date:` — Specific Date ``` date:YYYYMMDD or date:YYMMDD ``` Matches rows where the date portion of a datetime column equals a specific day. | Engine | SQL | |--------|-----| | All | `DATE({{column}}) = {{value}}` | **Value normalization**: - `YYYYMMDD` (8 digits) → `YYYY-MM-DD` - `YYMMDD` (6 digits) → `20YY-MM-DD` ``` created_at?date:20240823 → DATE(created_at) = :p (:p = '2024-08-23') created_at?date:240101 → DATE(created_at) = :p (:p = '2024-01-01') ``` ### `month:` — Month Number ``` month:MM or month:M ``` Matches rows in a specific calendar month, regardless of year. | Engine | SQL | |--------|-----| | All | `MONTH({{column}}) = {{value}}` | **Value normalization**: zero-padded to 2 digits (`8` → `08`). ``` created_at?month:08 → MONTH(created_at) = :p (:p = '08') created_at?month:3 → MONTH(created_at) = :p (:p = '03') ``` ### `year:` — Year ``` year:YYYY or year:YY ``` Matches rows in a specific year. | Engine | SQL | |--------|-----| | All | `YEAR({{column}}) = {{value}}` | **Value normalization**: 2-digit years are expanded to `20YY`. ``` created_at?year:2024 → YEAR(created_at) = :p (:p = '2024') created_at?year:24 → YEAR(created_at) = :p (:p = '2024') ``` ### `period:` — Year and Month ``` period:YYYYMM or period:YYMM ``` Matches rows in a specific year+month. | Engine | SQL | |--------|-----| | pgsql | `TO_CHAR({{column}}, "YYYYMM") = {{value}}` | | mysql | `DATE_FORMAT({{column}}, "%Y%m") = {{value}}` | | sqlite | `strftime("%Y%m", {{column}}) = {{value}}` | **Value normalization**: 4-digit `YYMM` is expanded to `20YYMM`. ``` created_at?period:202408 → strftime("%Y%m", created_at) = :p (:p = '202408') (sqlite) created_at?period:2408 → … (:p = '202408') ``` **Note**: Date operators use SQL functions (`DATE()`, `MONTH()`, etc.) and are **not compatible with Doctrine ORM DQL**. Use the Doctrine DBAL or Illuminate bridges for date filtering. --- ## NULL Operators | Symbol | Alias of | SQL | |--------|----------|-----| | `is:null` | — | `{{column}} IS NULL` | | `<=>` | `is:null` | `{{column}} IS NULL` | | `isnot:null` | — | `{{column}} IS NOT NULL` | ### Value Format The symbol is the complete filter — no value is expected after the symbol. `is:null` means the expression is literally `column?is:null`. ### Examples ``` deleted_at?is:null → deleted_at IS NULL deleted_at?<=> → deleted_at IS NULL (alternative syntax) email?isnot:null → email IS NOT NULL ``` **Note**: NULL operators produce no bound parameters. --- ## Subquery Operators Used exclusively with **exists paths** (`___`). They determine whether the correlated subquery checks for existence or absence. | Symbol | SQL | |--------|-----| | `is:empty` | `NOT EXISTS (SELECT 1 FROM … WHERE …)` | | `isnot:empty` | `EXISTS (SELECT 1 FROM … WHERE …)` | ``` ___payments[on:id=invoice_id]?is:empty → NOT EXISTS (SELECT 1 FROM payments WHERE invoices.id = payments.invoice_id) ___payments[on:id=invoice_id]?isnot:empty → EXISTS (SELECT 1 FROM payments WHERE invoices.id = payments.invoice_id) ``` In Doctrine ORM DQL, these generate `SIZE(alias.assoc) = 0` and `SIZE(alias.assoc) > 0` respectively. --- ## Regular Expression Operators These operators match POSIX extended regular expressions. Support varies by engine. ### Core Operators | Symbol | Case | SQL (pgsql) | SQL (mysql) | |--------|------|-------------|-------------| | `~` | Sensitive | `{{column}} ~ {{value}}` | `{{column}} REGEXP BINARY {{value}}` | | `~*` | Insensitive | `{{column}} ~* {{value}}` | `{{column}} REGEXP {{value}}` | | `!~` | Sensitive | `{{column}} !~ {{value}}` | `{{column}} NOT REGEXP BINARY {{value}}` | | `!~*` | Insensitive | `{{column}} !~* {{value}}` | `{{column}} NOT REGEXP {{value}}` | **SQLite** does not have built-in regex support — these operators have no SQLite template and will fail if used against a SQLite connection. ### MySQL Aliases | Symbol | Alias of | |--------|----------| | `regexp:` | `~` | | `notregexp:` | `!~` | | `rlike:` | `~` | | `notrlike:` | `!~` | These are provided for familiarity with MySQL's SQL syntax: ``` name?regexp:^John → name REGEXP BINARY :p (mysql) name?rlike:John.* → name REGEXP BINARY :p (mysql) ``` ### PostgreSQL SIMILAR TO | Symbol | SQL | |--------|-----| | `similarto:` | `{{column}} SIMILAR TO {{value}}` | | `notsimilarto:` | `{{column}} NOT SIMILAR TO {{value}}` | `SIMILAR TO` uses SQL-standard regex patterns (a hybrid of LIKE wildcards and regex). It is PostgreSQL-specific. Pattern rules: may contain `%`, `_`, `[a-z]`, `(a\|b)`, `?`, `*`, `+`. Unbalanced `(`, `)`, `[`, `]` are rejected at parse time. ``` name?similarto:John% → name SIMILAR TO :p code?similarto:[0-9]+ → code SIMILAR TO :p status?notsimilarto:(a|b)% → status NOT SIMILAR TO :p ``` ### Examples ``` email?~@example\.com → email ~ :p (pgsql) name?~*john → name ~* :p (pgsql) / name REGEXP :p (mysql) phone?!~^\+ → phone !~ :p (pgsql) ``` **Validation**: Regex patterns must contain at least one non-metacharacter character to prevent trivially-matching expressions. **Note**: Regex operators are **not compatible with Doctrine ORM DQL**. --- ## Bitwise Operators Operate on integer values at the bit level. All values must be non-negative integers (no decimals). | Symbol | Name | Value | SQL | |--------|------|-------|-----| | `b&` | Bitwise AND | `INT` | pgsql/sqlite: `col & :p` · mysql: `BIT_AND(col, :p)` | | `b\|` | Bitwise OR | `INT` | all: `col \| :p` | | `b^` | Bitwise XOR | `INT` | pgsql: `col # :p` · mysql: `BIT_XOR(col, :p)` · sqlite: `(col \| :p) & ~(col & :p)` | | `b<<` | Left Shift | `INT` | pgsql/sqlite: `col << :p` · mysql: `<< col, :p` | | `b>>` | Right Shift | `INT` | pgsql/sqlite: `col >> :p` · mysql: `>> col, :p` | | `b&~` | AND NOT | `INT` | all: `col & ~( :p )` | ### Common Use Case: Feature Flags Bitwise AND is commonly used to test whether a flag bit is set in a bitmask column: ``` flags?b&1 → flags & :p (:p = '1') — tests bit 0 (value 1) flags?b&2 → flags & :p (:p = '2') — tests bit 1 (value 2) flags?b&4 → flags & :p (:p = '4') — tests bit 2 (value 4) ``` ### Examples ``` permissions?b&4 → permissions & :p (:p = '4') (pgsql) permissions?b|3 → permissions | :p (:p = '3') version?b^255 → (version | :p) & ~(version & :p) (sqlite) offset?b<<2 → offset << :p (:p = '2') mask?b&~8 → mask & ~( :p ) (:p = '8') ``` **Validation**: Non-integer values (letters, decimals like `1.5`) are rejected at parse time. **Note**: Bitwise operators are **not compatible with Doctrine ORM DQL**. --- ## Operator Aliases Several operators are aliases that reuse another operator's SQL template: | Symbol | Alias of | Reason | |--------|----------|--------| | `!` | `!=` | Shorthand not-equal | | `<>` | `!=` | SQL-standard not-equal | | `<=>` | `is:null` | MySQL NULL-safe equality syntax | | `^` | `like:` | auto-casts with `like_start` | | `^*` | `ilike:` | auto-casts with `like_start` | | `!^` | `notlike:` | auto-casts with `like_start` | | `!^*` | `notilike:` | auto-casts with `like_start` | | `~~` | `like:` | auto-casts with `like` | | `~~*` | `ilike:` | auto-casts with `like` | | `!~~` | `notlike:` | auto-casts with `like` | | `!~~*` | `notilike:` | auto-casts with `like` | | `$` | `like:` | auto-casts with `like_end` | | `$*` | `ilike:` | auto-casts with `like_end` | | `!$` | `notlike:` | auto-casts with `like_end` | | `!$*` | `notilike:` | auto-casts with `like_end` | | `regexp:` | `~` | MySQL familiarity | | `notregexp:` | `!~` | MySQL familiarity | | `rlike:` | `~` | MySQL familiarity | | `notrlike:` | `!~` | MySQL familiarity | --- ## Casting Rules Casting rules transform the raw value string before it is bound as a parameter: | Rule | Input | Output | |------|-------|--------| | `like_start` | `John` | `John%` | | `like` | `John` | `%John%` | | `like_end` | `Smith` | `%Smith` | | `list` | `a,b,c` | `['a', 'b', 'c']` (split) | | `date` | `20240823` | `2024-08-23` | | `date` | `240823` | `2024-08-23` | | `month` | `8` | `08` | | `year` | `24` | `2024` | | `period` | `202408` | `202408` (unchanged) | | `period` | `2408` | `202408` | --- ## DQL Compatibility Matrix The Doctrine ORM bridge generates DQL, which does not support all SQL features: | Operator type | DQL compatible? | |---------------|:--------------:| | Standard (`=`, `!=`, `>`, etc.) | ✅ | | AutoLike (`^`, `~~`, `$`, etc.) | ✅ | | `like:`, `notlike:` | ✅ | | `ilike:`, `notilike:` | ❌ | | `in:`, `notin:` | ✅ | | `between:`, `notbetween:` | ✅ | | Date (`date:`, `month:`, `year:`, `period:`) | ❌ | | NULL (`is:null`, `isnot:null`) | ✅ | | Subquery (`is:empty`, `isnot:empty`) | ✅ (→ SIZE()) | | RegExp (`~`, `~*`, `similarto:`, etc.) | ❌ | | Binary (`b&`, `b\|`, etc.) | ❌ | For incompatible operators, use the Doctrine DBAL bridge or `IlluminateQueryBuilderConditionApplier` instead. In API Platform's `SmartFilter`, incompatible operators are silently skipped. --- ## Custom Operators Operators are defined in `resources/operators.yaml`. You can load a custom YAML file to add or override operators: ```yaml # my-operators.yaml version: '1.0.0' description: 'Custom operators' types: standard: name: 'Standard Operators' description: 'SQL comparison operators' operators: '=': type: standard name: Equals description: Exact match sql: '{{column}} = {{value}}' 'startswith:': type: like name: Starts With (verbose) description: Explicit starts-with LIKE operator sql: '{{column}} LIKE {{value}}' cast: ['like_start'] ``` ```php use Derafu\Query\Operator\OperatorLoader; use Derafu\Query\Operator\OperatorManagerFactory; $factory = new OperatorManagerFactory(new OperatorLoader()); $manager = $factory->create('/path/to/my-operators.yaml'); ``` Required fields per operator: `type`, `name`, `description`. The `type` key must reference an entry in the `types` section. --- ### Query Builder Query Builder # Query Builder `derafu/query` provides two complementary ways to build queries: 1. **Fluent API** (`SqlQueryBuilder`) — chainable method calls, useful in application code. 2. **Declarative Config** (`QueryConfig`) — array, YAML, or JSON definitions, useful for reusable templates and API-driven queries. Both produce the same output: a `SqlQuery` with an SQL string and named-parameter array. --- ## Setting Up SqlQueryBuilder `SqlQueryBuilder` requires an engine and an expression parser: ```php use Derafu\Query\Builder\SqlQueryBuilder; use Derafu\Query\Engine\PdoEngine; use Derafu\Query\Filter\CompositeExpressionParser; use Derafu\Query\Filter\ExpressionParser; use Derafu\Query\Filter\FilterParser; use Derafu\Query\Filter\PathParser; use Derafu\Query\Operator\OperatorLoader; use Derafu\Query\Operator\OperatorManager; // 1. Set up the operator registry. $loader = new OperatorLoader(); $manager = new OperatorManager( $loader->loadFromFile('vendor/derafu/query/resources/operators.yaml') ); // 2. Build the expression parser. $parser = new CompositeExpressionParser( new ExpressionParser(new PathParser(), new FilterParser($manager)) ); // 3. Create a database engine. $pdo = new PDO('pgsql:host=localhost;dbname=mydb', $user, $pass); $engine = new PdoEngine($pdo); // 4. Instantiate the builder. $qb = new SqlQueryBuilder($engine, $parser); ``` For Doctrine DBAL use `DoctrineEngine` instead of `PdoEngine`. For framework-integrated setups, consider the [bridges](./bridges) instead of `SqlQueryBuilder`. --- ## Fluent API Reference All mutating methods return `$this` (or a clone via `new()`) so they can be chained. ### `table(string $table, ?string $alias = null): self` Sets the `FROM` clause and resets the builder state via an internal `new()` clone. Call this first when starting a new query from a builder instance that may have prior state. ```php $qb->table('products')->where('price?>1000')->execute(); $qb->table('invoices', 'i')->select('i.id, i.total')->execute(); ``` ### `from(string $table, ?string $alias = null): self` Sets the `FROM` clause without resetting. Use `table()` for new queries; use `from()` internally or when you already have a fresh builder. ### `select(string|array $columns, bool $sanitize = true): self` Sets the `SELECT` columns. Accepts a comma-separated string or an array. ```php $qb->select('id, name, price'); $qb->select(['id', 'name', 'price']); $qb->select('c.name AS customer_name, i.total'); $qb->select('COUNT(*) AS total'); $qb->select('DISTINCT type'); // Use distinct() instead for the DISTINCT keyword ``` Pass `$sanitize = false` to skip identifier sanitization (use only for trusted, pre-validated expressions). ### `distinct(bool $distinct = true): self` Adds `DISTINCT` to the `SELECT` clause. ```php $qb->table('customers')->select('type')->distinct()->execute(); // → SELECT DISTINCT type FROM customers ``` ### `where(string|array|Condition|CompositeCondition $condition): self` Sets the `WHERE` clause. Resets any previous `WHERE` conditions. All elements are AND-combined. ```php $qb->where('status?=active'); $qb->where(['status?=active', 'total?>1000']); $qb->where('status?=active&&total?>1000'); // composite string ``` ### `andWhere(string|array|Condition|CompositeCondition $condition): self` Appends conditions to the existing `WHERE` with AND. If no `WHERE` exists yet, behaves like `where()`. ```php $qb->where('status?=active')->andWhere('total?>1000'); ``` ### `orWhere(string|array|Condition|CompositeCondition $condition): self` Wraps the existing `WHERE` and the new condition in an OR composite. ```php // WHERE (status = 'electronics' OR category = 'hardware') $qb->where('category?=electronics')->orWhere('category?=hardware'); // WHERE (category = 'software' OR (category = 'hardware' AND price > 200)) $qb->where('category?=software')->orWhere(['category?=hardware', 'price?>200']); // Multiple OR groups: // WHERE (status = 'cancelled' OR status = 'draft' OR (status = 'issued' AND total > 1000)) $qb->where('status?=cancelled') ->orWhere(['status?=draft', ['status?=issued', 'total?>1000']]); ``` ### `andWhereOr(array|Condition|CompositeCondition $conditions): self` Appends an OR composite to the existing AND WHERE. Each element of the array is one OR branch; arrays within the array become AND groups. ```php // WHERE status = 'active' AND (type = 'person' OR tax_id LIKE '78%') $qb->where('status?=active') ->andWhereOr(['type?=person', 'tax_id?^78']); // WHERE status = 'active' AND ((status = 'paid' AND total > 1000) OR (status = 'issued' AND date >= '2024-03-01')) $qb->where('customer_id?in:1,2') ->andWhereOr([ ['status?=paid', 'total?>1000'], ['status?=issued', 'date?>=2024-03-01'], ]); ``` ### `join(string $table, string $condition, string $type = 'INNER', ?string $alias = null): self` Adds an explicit JOIN clause. Valid types: `INNER`, `LEFT`, `RIGHT`, `CROSS`. Duplicates (same table+alias combination) are ignored. ```php $qb->join('customers', 'i.customer_id = c.id', 'INNER', 'c'); ``` Convenience wrappers: ```php $qb->innerJoin('customers', 'i.customer_id = c.id', 'c'); $qb->leftJoin('payments', 'i.id = p.invoice_id', 'p'); $qb->rightJoin('invoices', 'c.id = i.customer_id', 'i'); $qb->crossJoin('currencies'); ``` **Note**: When using path expressions in `where()`, joins are generated automatically from the path segments. Manual `join()` calls are only needed when you want explicit control, or when building the query without path expressions. ### `groupBy(string|array $columns): self` Adds `GROUP BY` columns. Multiple calls accumulate. ```php $qb->groupBy('category'); $qb->groupBy(['c.id', 'c.name']); ``` ### `having(string|array|Condition|CompositeCondition $condition): self` Sets the `HAVING` clause. Accepts the same expression formats as `where()`. ```php $qb->groupBy('category')->having('AVG(price)?>500'); $qb->groupBy('status')->having('COUNT(*)?>1'); ``` ### `orderBy(string|array $columns, string $direction = 'ASC'): self` Sets `ORDER BY`. Accepts a column string, or an associative array of `column => direction`. ```php $qb->orderBy('price', 'DESC'); $qb->orderBy(['category' => 'ASC', 'price' => 'DESC']); $qb->orderBy(['created_at', 'id']); // defaults to ASC ``` ### `limit(int $limit): self` / `offset(int $offset): self` Set pagination. `OFFSET` is only appended when both `LIMIT` and `OFFSET` are set. ```php $qb->limit(20)->offset(40); // → LIMIT 20 OFFSET 40 ``` ### `getQuery(): QueryInterface` Builds and returns the SQL without executing. Returns a `SqlQuery` that implements both `QueryInterface` and `ArrayAccess`: ```php $query = $qb->table('products')->where('price?>1000')->getQuery(); echo $query['sql']; // SELECT * FROM products WHERE (price > :param_price_...) echo $query['parameters']; // ['param_price_...' => '1000'] ``` ### `execute(): array` Builds and executes the query, returning rows as an array of associative arrays. ```php $rows = $qb->table('products')->where('price?>1000')->execute(); ``` --- ## Complete Query Examples ### Simple Filters ```php // Active customers. $qb->table('customers')->where('status?=active')->execute(); // Products in a price range. $qb->table('products')->where('price?between:100,1000')->execute(); // Soft-deleted records. $qb->table('users')->where('deleted_at?isnot:null')->execute(); // March 2024 invoices (SQLite). $qb->table('invoices')->where('date?period:202403')->execute(); ``` ### Composite Conditions ```php // AND: software products over $200. $qb->table('products') ->where(['category?=software', 'price?>200']) ->execute(); // OR: electronics or hardware. $qb->table('products') ->where('category?=electronics') ->orWhere('category?=hardware') ->execute(); // AND + OR: active customers who are persons OR whose tax_id starts with "78". $qb->table('customers') ->where('status?=active') ->andWhereOr(['type?=person', 'tax_id?^78']) ->execute(); ``` ### Path-Based Joins ```php // Auto-generated JOIN from path. $qb->select('c.name AS customer_name, i.number, i.total') ->where('customers[alias:c]__invoices[on:id=customer_id,alias:i]__total?>1000') ->execute(); // → SELECT … FROM customers AS c INNER JOIN invoices AS i ON c.id = i.customer_id WHERE i.total > :p // Multi-level join. $qb->select('p.name, i.number, c.name AS customer') ->where([ 'products[alias:p]__category?=electronics', 'products[alias:p]__invoice_details[on:id=product_id,alias:id]__invoices[on:invoice_id=id,alias:i]__status?=paid', 'products[alias:p]__invoice_details[on:id=product_id,alias:id]__invoices[on:invoice_id=id,alias:i]__customers[on:customer_id=id,alias:c]__type?=company', ]) ->execute(); ``` ### Subquery / EXISTS ```php // Invoices with no payments. $qb->table('invoices') ->where('___payments[on:id=invoice_id]?is:empty') ->execute(); // → SELECT * FROM invoices WHERE NOT EXISTS (SELECT 1 FROM payments WHERE invoices.id = payments.invoice_id) // Invoices with at least one pending payment. $qb->table('invoices') ->where('___payments[on:id=invoice_id]__status?=pending') ->execute(); // Invoices where total payments >= 1200. $qb->table('invoices') ->where('___payments[on:id=invoice_id]__SUM(amount)?>=1200') ->execute(); ``` ### Grouping, Having, Ordering, Pagination ```php // Category stats. $qb->table('products') ->select('category, AVG(price) AS avg_price') ->groupBy('category') ->having('AVG(price)?>500') ->orderBy('avg_price', 'DESC') ->limit(10) ->execute(); // Paginated customer list. $qb->table('customers') ->orderBy(['name' => 'ASC']) ->limit(25) ->offset(50) ->execute(); ``` ### Explicit Joins ```php // Manual JOIN for a complex ON clause. $qb->table('invoices', 'i') ->select('i.id, i.number, i.total, c.name AS customer_name') ->innerJoin('customers', 'i.customer_id = c.id', 'c') ->where('i.status?=paid') ->execute(); // LEFT JOIN with GROUP BY. $qb->table('customers', 'c') ->select('c.name, COUNT(i.id) AS invoice_count') ->leftJoin('invoices', 'c.id = i.customer_id', 'i') ->groupBy(['c.id', 'c.name']) ->execute(); ``` --- ## Declarative Query Configuration `QueryConfig` provides an array-based way to describe a query. It is particularly useful for storing query templates in files and for API-driven queries. ### Basic Usage ```php use Derafu\Query\Config\QueryConfig; $config = new QueryConfig([ 'table' => 'products', 'select' => 'id, name, price', 'where' => 'category?=electronics', 'orderBy' => ['price' => 'DESC'], 'limit' => 10, ]); $result = $config->applyTo($qb)->execute(); ``` `applyTo()` calls the corresponding builder methods in order and returns the modified builder, so you can chain additional calls after: ```php $builder = $config->applyTo($qb); if ($extraFilter) { $builder->andWhere('status?=active'); } $rows = $builder->execute(); ``` ### Loading from Files ```php // YAML file. $config = QueryConfig::fromYamlFile('/path/to/queries/product_report.yaml'); // JSON file. $config = QueryConfig::fromJsonFile('/path/to/queries/sales.json'); // Auto-detect by extension (.yml, .yaml, .json). $config = QueryConfig::fromFile('/path/to/query.yaml'); // From strings. $config = QueryConfig::fromYamlString($yamlString); $config = QueryConfig::fromJsonString($jsonString); ``` ### YAML Template Example ```yaml # recent_products.yaml table: products select: id, name, price, created_at where: deleted_at?is:null orderBy: created_at: DESC limit: 20 ``` ```php $config = QueryConfig::fromYamlFile('queries/recent_products.yaml'); $builder = $config->applyTo($qb); // Add extra filter at runtime. if ($category) { $builder->andWhere('category?=' . $category); } $rows = $builder->execute(); ``` ### Complete Configuration Reference All `QueryConfig` keys and their equivalent builder calls: | Key | Builder Method | Value Type | |-----|---------------|------------| | `table` | `table()` | string | | `alias` | `table($t, $alias)` | string | | `select` | `select()` | string or array | | `distinct` | `distinct(true)` | boolean | | `where` | `where()` | string, array, or nested | | `andWhere` | `andWhere()` | string or array | | `orWhere` | `orWhere()` | string or array | | `andWhereOr` | `andWhereOr()` | array of arrays | | `innerJoin` | `innerJoin()` | `{table, condition, alias?}` | | `leftJoin` | `leftJoin()` | `{table, condition, alias?}` | | `rightJoin` | `rightJoin()` | `{table, condition, alias?}` | | `crossJoin` | `crossJoin()` | `{table, alias?}` | | `groupBy` | `groupBy()` | string or array | | `having` | `having()` | string or array | | `orderBy` | `orderBy()` | `{column: direction}` | | `limit` | `limit()` | integer | | `offset` | `offset()` | integer | ```php $config = new QueryConfig([ 'table' => 'invoices', 'alias' => 'i', 'select' => 'i.id, i.number, c.name AS customer_name', 'distinct' => true, 'where' => 'i.status?=paid', 'andWhere' => 'i.total?>1000', 'orWhere' => 'i.date?period:202403', 'andWhereOr' => [ ['i.category?=service', 'i.total?>500'], ['i.category?=product', 'i.total?>1000'], ], 'innerJoin' => ['table' => 'customers', 'alias' => 'c', 'condition' => 'i.customer_id = c.id'], 'groupBy' => ['i.status'], 'having' => 'COUNT(*)?>1', 'orderBy' => ['i.created_at' => 'DESC'], 'limit' => 20, 'offset' => 40, ]); ``` ### API-Driven Queries `QueryConfig` is well-suited for accepting query parameters from an API: ```php $requestData = $request->getJsonBody(); $config = new QueryConfig([ 'table' => 'products', 'where' => $requestData['filters'] ?? [], 'orderBy' => $requestData['sort'] ?? ['id' => 'ASC'], 'limit' => min($requestData['limit'] ?? 20, 100), 'offset' => max($requestData['offset'] ?? 0, 0), ]); $rows = $config->applyTo($qb)->execute(); ``` Ensure the `where` filters are either pre-validated strings that only allow known column names, or use the full path syntax so that only correctly-structured expressions reach the parser. --- ### Framework Bridges Framework Bridges # Framework Bridges Bridges allow you to apply `derafu/query` expressions to existing third-party query builder instances — without replacing them. You keep your existing query builder and add Derafu filters on top. All bridges accept a parsed `ConditionInterface | CompositeConditionInterface` and apply it to the query builder. You parse expressions with `CompositeExpressionParser` and pass the result to the bridge. --- ## Shared Setup: The Expression Parser All bridges share the same expression parser setup: ```php use Derafu\Query\Filter\CompositeExpressionParser; use Derafu\Query\Filter\ExpressionParser; use Derafu\Query\Filter\FilterParser; use Derafu\Query\Filter\PathParser; use Derafu\Query\Operator\OperatorLoader; use Derafu\Query\Operator\OperatorManager; $loader = new OperatorLoader(); $manager = new OperatorManager( $loader->loadFromFile('vendor/derafu/query/resources/operators.yaml') ); $parser = new CompositeExpressionParser( new ExpressionParser(new PathParser(), new FilterParser($manager)) ); ``` --- ## Doctrine DBAL Bridge **Class**: `Derafu\Query\Bridge\DoctrineDBALQueryBuilderConditionApplier` Applies conditions to a Doctrine DBAL `QueryBuilder`. Supports all operators including date, regex, and bitwise operators. ### Methods ```php public function apply(object $queryBuilder, ConditionInterface|CompositeConditionInterface $condition): void public function applyHaving(object $queryBuilder, ConditionInterface|CompositeConditionInterface $condition): void ``` ### Usage ```php use Derafu\Query\Bridge\DoctrineDBALQueryBuilderConditionApplier; $applier = new DoctrineDBALQueryBuilderConditionApplier(); // Parse the expression. $condition = $parser->parse('status?=active&&total?>1000'); // Apply to an existing DBAL QueryBuilder. $applier->apply($dbalQb, $condition); // The QB now has: WHERE (status = :param_status_... AND total > :param_total_...) $rows = $dbalQb->executeQuery()->fetchAllAssociative(); ``` ### FROM and JOIN Inference When a condition uses multi-segment paths, the bridge: 1. **Infers `FROM`** from the first segment of the first multi-segment path, if no `FROM` has been set yet. 2. **Generates JOINs** from intermediate segments, with deduplication (same alias → skip). ```php // No from() call needed — it's inferred from the path. $condition = $parser->parse( 'customers[alias:c]__invoices[on:id=customer_id,alias:i]__total?>1000' ); $applier->apply($dbalQb, $condition); // Sets FROM customers AS c, adds INNER JOIN invoices AS i ON c.id = i.customer_id // WHERE i.total > :p ``` ### Driver Detection The bridge reads the database platform from the DBAL connection (via reflection on the internal `connection` property) and maps it to the driver name used by `SqlBuilderWhere`: | Platform | Driver | |----------|--------| | `PostgreSQLPlatform` | `pgsql` | | `MySQLPlatform` | `mysql` | | `SQLitePlatform` | `sqlite` | | `SQLServerPlatform` | `sqlsrv` | | `OraclePlatform` | `oci` | | Other | `pgsql` (fallback) | ### HAVING ```php $dbalQb->groupBy('status'); $condition = $parser->parse('COUNT(*)?>1'); $applier->applyHaving($dbalQb, $condition); ``` --- ## Doctrine ORM Bridge **Class**: `Derafu\Query\Bridge\DoctrineORMQueryBuilderConditionApplier` Applies conditions to a Doctrine ORM `QueryBuilder` as DQL. This bridge generates DQL-compatible fragments — it does not produce raw SQL. ### Methods ```php public function apply(object $queryBuilder, ConditionInterface|CompositeConditionInterface $condition): void public function applyHaving(object $queryBuilder, ConditionInterface|CompositeConditionInterface $condition): void ``` ### Usage ```php use Derafu\Query\Bridge\DoctrineORMQueryBuilderConditionApplier; $applier = new DoctrineORMQueryBuilderConditionApplier(); $condition = $parser->parse('status?=active&&total?>1000'); $ormQb = $em->createQueryBuilder() ->select('i') ->from(Invoice::class, 'i'); $applier->apply($ormQb, $condition); $invoices = $ormQb->getQuery()->getResult(); ``` ### Single-Segment Path Qualification Single-segment paths like `status` are automatically qualified with the root entity alias. The root alias is read from the first `FROM` clause: ``` status?=active → i.status = :param_status_... (when root alias is 'i') ``` Multi-segment paths are left as-is, allowing explicit alias control. ### JOIN Generation The ORM bridge adds JOINs to the QueryBuilder using Doctrine's association names — the `on:` option in path segments is **ignored**. Doctrine derives join conditions from entity mappings. ``` invoices[alias:i]__payments[alias:p]__status ``` If `Invoice` has a `payments` association, the bridge calls: ``` $ormQb->innerJoin('i.payments', 'p'); ``` ### EXISTS Paths in DQL Subquery paths (`___`) are translated to DQL: | Expression | DQL | |------------|-----| | `___payments?is:empty` | `SIZE(i.payments) = 0` | | `___payments?isnot:empty` | `SIZE(i.payments) > 0` | | `___payments__status?=pending` | `EXISTS(SELECT _payments0.id FROM App\Entity\Payment _payments0 WHERE _payments0.invoice = i AND _payments0.status = :p)` | | `___invoices__AVG(total)?<1000` | `(SELECT AVG(_i0.total) FROM App\Entity\Invoice _i0 WHERE _i0.customer = i) < :p` | | `___invoices__COUNT(*)?=0` | `SIZE(i.invoices) = 0` (optimized) | ### DQL-Incompatible Operators The following operator types throw `UnsupportedOperatorException` when used with this bridge: - **`date` type**: `date:`, `month:`, `year:`, `period:` — require SQL functions not available in DQL. - **`binary` type**: `b&`, `b|`, `b^`, etc. — bitwise SQL not supported in DQL. - **`regexp` type**: `~`, `~*`, `similarto:`, etc. — database-specific regex not in DQL. - **`ilike:` and `notilike:`** — ILIKE is not DQL. In the API Platform `SmartFilter`, these exceptions are caught and the filter is silently skipped. --- ## Illuminate (Laravel) Bridge **Class**: `Derafu\Query\Bridge\IlluminateQueryBuilderConditionApplier` Applies conditions to Illuminate's `Query\Builder` or Eloquent's `Builder`. Both are accepted — Eloquent builders are resolved to their underlying `Query\Builder` via `toBase()`. ### Methods ```php public function apply(object $queryBuilder, ConditionInterface|CompositeConditionInterface $condition): void public function applyHaving(object $queryBuilder, ConditionInterface|CompositeConditionInterface $condition): void ``` ### Usage ```php use Derafu\Query\Bridge\IlluminateQueryBuilderConditionApplier; $applier = new IlluminateQueryBuilderConditionApplier(); $condition = $parser->parse('status?=active&&total?>1000'); // With a plain Query\Builder. $qb = DB::table('invoices'); $applier->apply($qb, $condition); $rows = $qb->get(); // With an Eloquent Builder. $builder = Invoice::query(); $applier->apply($builder, $condition); $invoices = $builder->get(); ``` ### FROM and JOIN Inference Same behavior as the DBAL bridge: infers the `FROM` table from multi-segment paths and adds `join()`, `leftJoin()`, or `rightJoin()` calls as needed. Duplicate joins (same table reference) are skipped. ### Driver Detection The bridge reads the driver name directly from `Connection::getDriverName()`: | Driver name | SQL generated | |-------------|--------------| | `pgsql` | PostgreSQL SQL | | `mysql` | MySQL SQL | | `sqlite` | SQLite SQL | ### HAVING ```php $condition = $parser->parse('AVG(price)?>500'); $applier->applyHaving($qb, $condition); // Calls: $qb->havingRaw($sql, $params) ``` --- ## API Platform Bridge **Class**: `Derafu\Query\Bridge\ApiPlatform\SmartFilter` Integrates `derafu/query` as an API Platform filter via the `QueryParameter` attribute. When a request comes in, the filter builds a Derafu expression from the parameter's property name and value, parses it, and applies it to the ORM `QueryBuilder`. ### Registration ```php use ApiPlatform\Metadata\ApiResource; use ApiPlatform\Metadata\QueryParameter; use Derafu\Query\Bridge\ApiPlatform\SmartFilter; #[ApiResource] #[QueryParameter(key: 'price', property: 'price', filter: SmartFilter::class)] #[QueryParameter(key: 'status', property: 'status', filter: SmartFilter::class)] class Product { // … } ``` ### URL Filter Syntax The URL parameter value is the filter part (operator + value); the property name provides the path segment: ``` GET /api/products?price=>1000&status=in:paid,issued ``` The `SmartFilter` builds expressions internally: - `price` property + `>1000` value → expression `price?>1000` - `status` property + `in:paid,issued` value → expression `status?in:paid,issued` ### Generic SmartFilter Property For a single parameter that accepts a full composite expression, use the special `__derafu_smart_filter` property name: ```php #[QueryParameter(key: 'filter', property: '__derafu_smart_filter', filter: SmartFilter::class)] ``` ``` GET /api/products?filter=status?=active&&price?>1000 ``` ### Dependency Injection (Symfony) Register `CompositeExpressionParser` and `DoctrineORMQueryBuilderConditionApplier` as services and inject them into `SmartFilter`: ```yaml # services.yaml services: Derafu\Query\Bridge\ApiPlatform\SmartFilter: arguments: $compositeParser: '@Derafu\Query\Filter\Contract\CompositeExpressionParserInterface' $applier: '@Derafu\Query\Bridge\DoctrineORMQueryBuilderConditionApplier' ``` ### Error Handling `SmartFilter` silently catches two exceptions: - **`UnsupportedOperatorException`**: DQL-incompatible operators (`date:`, `b&`, `ilike:`, regex) — the filter is skipped. - **Any `Throwable`**: Malformed expressions — the filter is skipped. This prevents a bad filter parameter from crashing the entire collection endpoint. --- ## Bridge Comparison | Feature | Doctrine DBAL | Doctrine ORM | Illuminate | |---------|:---:|:---:|:---:| | SQL operators (all) | ✅ | ✅† | ✅ | | Date operators | ✅ | ❌ | ✅ | | Regex operators | ✅ | ❌ | ✅ | | Bitwise operators | ✅ | ❌ | ✅ | | `ilike:` / `notilike:` | ✅ | ❌ | ✅ | | FROM inference from paths | ✅ | ❌‡ | ✅ | | JOIN from path segments | ✅ | ✅ (ORM assoc) | ✅ | | EXISTS paths | ✅ | ✅ (SIZE/EXISTS DQL) | ✅ | | Aggregate subqueries | ✅ | ✅ | ✅ | † Compatible operators only — DQL-incompatible operators throw `UnsupportedOperatorException`. ‡ ORM bridge requires an explicit `FROM` / `from()` call before applying conditions. --- ### Security Guide Security Guide # Security Guide `derafu/query` uses prepared statements for all filter values. This page explains the security model, what is and is not protected, and how to use the library safely. --- ## How Values Are Protected All filter values — the part after the operator symbol in `column?operatorVALUE` — become **named parameters** in the generated SQL: ``` price?>1000 → price > :param_price_5f3a... with parameter binding: :param_price_5f3a... = '1000' ``` The SQL string itself never contains the user-supplied value. It goes through PDO/Doctrine/Illuminate's prepared statement mechanism, which prevents SQL injection in values. This holds for every operator type: standard comparisons, LIKE patterns, lists, ranges, dates, regex values, and bitwise operands are all bound as parameters. --- ## What SqlSanitizerTrait Protects `SqlSanitizerTrait` is used for **SQL identifiers** — column names, table names, aliases, and aggregate function names. It strips all characters except `[a-zA-Z0-9_]` from simple identifiers. Safe uses (identifiers derived from path segments or select expressions): ```php $qb->select('column_name'); // simple identifier $qb->select('table.column'); // qualified identifier $qb->select('column AS alias'); // with alias $qb->select('COUNT(*) AS total'); // aggregate function $qb->select('SUM(price) AS total'); // aggregate with column ``` Cases requiring attention: ```php // Complex expressions — arithmetic operators are detected and left as-is. $qb->select('price * quantity AS total'); // Multi-argument functions — outer function is sanitized; inner args are handled separately. $qb->select('COALESCE(column1, column2, 0) AS result'); ``` **The sanitizer is a defense-in-depth measure for identifier injection, not a substitute for input validation.** It strips unexpected characters but cannot reason about business logic (e.g. it does not know which columns a user is allowed to access). --- ## What Is NOT Automatically Protected ### Explicit JOIN Conditions The `join()` / `innerJoin()` / `leftJoin()` / `rightJoin()` methods accept a raw `$condition` string that is inserted verbatim into the SQL: ```php // This condition is NOT sanitized. $qb->innerJoin('customers', 'i.customer_id = c.id', 'c'); ``` **Never pass user-supplied input directly as a join condition.** Use path-based syntax in `where()` instead — those are parsed and sanitized: ```php // Safe: join condition comes from the path parser. $qb->where('invoices[alias:i]__customers[on:customer_id=id,alias:c]__name?isnot:null'); ``` ### The `?E` Expression Marker When a filter expression uses `?E` (instead of `?`), the value is treated as a column reference and sanitized as an identifier — **not** bound as a parameter: ``` id?E!=other_alias.id → id != other_alias.id (identifier, no parameter binding) ``` Only use `?E` for trusted, controlled values (e.g. comparing two known column aliases). Never pass user input as a `?E` value. ### `select()` with `$sanitize = false` ```php $qb->select($trustedExpression, sanitize: false); ``` When `$sanitize` is `false`, the expression is inserted without processing. Only use this for pre-validated, application-controlled strings. --- ## Parameters vs. Identifiers The key distinction in SQL security: | Type | Example | How it's handled | |------|---------|-----------------| | **Parameter** (value) | `price?>1000` — the `1000` | Bound via prepared statement | | **Identifier** (column/table) | Path segment names, `select()` columns | Sanitized by `SqlSanitizerTrait` | Never try to pass a value as an identifier or vice versa. --- ## Operator Value Validation Before SQL generation, `FilterParser` validates the raw value against each operator's `pattern` (if defined). Invalid values throw `InvalidArgumentException` immediately: ```php $parser->parse('price?date:not-a-date'); // throws: pattern mismatch $parser->parse('flags?b&1.5'); // throws: decimal not allowed for binary op $parser->parse('status?in:'); // throws: empty list ``` This prevents malformed inputs from reaching the SQL layer, even though the actual injection risk is eliminated by parameter binding. --- ## Input Validation Best Practices 1. **Whitelist columns**: If you accept filter column names from user input (e.g. in an API), validate them against a known-safe list before passing to the expression parser. 2. **Validate operator intent**: Consider whether a user should be allowed to use all operators. For example, regex operators can cause high-CPU queries on large tables; restrict them if necessary. 3. **Limit list sizes**: The `in:` and `notin:` operators accept arbitrarily long lists. Consider validating or capping list length before parsing. 4. **Principle of least privilege**: The database user used by your application should only have `SELECT` (and only `INSERT`/`UPDATE`/`DELETE` where needed). A read-only connection cannot be abused into DDL statements even if SQL injection were somehow possible. 5. **Audit HAVING and JOIN conditions**: `having()` accepts the same expression format as `where()` and is equally safe. Explicit `join()` conditions are not sanitized — see above. 6. **Log and monitor**: Log queries that raise `InvalidArgumentException` — they may indicate probing attempts. --- ## Composite Expression Safety Composite expressions (`&&`, `||`, `()`) are parsed structurally, not evaluated as SQL. The parser splits on `&&` and `||` at parenthesis depth 0 and recurses — it does not execute or interpolate anything. There is no risk of injection through the composite syntax itself. ``` A?=1&&B?=2 → two separate bound parameters (A?=1||B?=2) → same, with OR grouping ``` The value inside each leaf expression is always bound as a parameter (unless `?E` is used — see above). --- ## Summary | Area | Protection | |------|-----------| | Filter values | ✅ Prepared statement parameters | | Column/table identifiers | ✅ `SqlSanitizerTrait` stripping | | Operator validation patterns | ✅ Regex validation at parse time | | Composite expression parsing | ✅ Structural parsing, no SQL eval | | Explicit `join()` conditions | ⚠️ Not sanitized — keep application-controlled | | `?E` expression references | ⚠️ Sanitized as identifier — keep application-controlled | | `select($expr, sanitize: false)` | ⚠️ Raw insert — keep application-controlled | | Column name whitelisting | 🔲 Application responsibility | | Operator allowlisting | 🔲 Application responsibility | --- ### Query Config Declarative Query Configuration # Declarative Query Configuration The `QueryConfig` class provides a configuration-based alternative to the fluent builder API. It is covered in detail in the [Query Builder](./query-builder) page, under the **Declarative Query Configuration** section. ## Quick Reference ```php use Derafu\Query\Config\QueryConfig; // From an array. $config = new QueryConfig([ 'table' => 'products', 'select' => 'id, name, price', 'where' => 'category?=electronics', 'orderBy' => ['price' => 'DESC'], 'limit' => 10, ]); $result = $config->applyTo($queryBuilder)->execute(); // From a YAML file. $config = QueryConfig::fromYamlFile('queries/product_report.yaml'); // From a JSON file. $config = QueryConfig::fromJsonFile('queries/sales.json'); // Auto-detect format by extension. $config = QueryConfig::fromFile('queries/report.yaml'); ``` ### Supported Configuration Keys | Key | Description | |-----|-------------| | `table` | Table name (FROM clause) | | `alias` | Table alias | | `select` | Columns to select | | `distinct` | `true` to add DISTINCT | | `where` | WHERE condition(s) | | `andWhere` | Additional AND condition(s) | | `orWhere` | OR condition(s) | | `andWhereOr` | AND with nested OR groups | | `innerJoin` | `{table, condition, alias?}` | | `leftJoin` | `{table, condition, alias?}` | | `rightJoin` | `{table, condition, alias?}` | | `crossJoin` | `{table, alias?}` | | `groupBy` | GROUP BY column(s) | | `having` | HAVING condition(s) | | `orderBy` | `{column: direction}` pairs | | `limit` | Max rows to return | | `offset` | Rows to skip | For full documentation, examples, and API-driven query patterns, see [Query Builder → Declarative Query Configuration](./query-builder#declarative-query-configuration). --- ## ETL Project Derafu ETL # Derafu ETL --- ### Introduction From Spreadsheets to Databases Seamlessly # From Spreadsheets to Databases Seamlessly ![GitHub last commit](https://img.shields.io/github/last-commit/derafu/etl/main) ![CI Workflow](https://github.com/derafu/etl/actions/workflows/ci.yml/badge.svg?branch=main&event=push) ![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/derafu/etl) ![GitHub Issues](https://img.shields.io/github/issues-raw/derafu/etl) ![Total Downloads](https://poser.pugx.org/derafu/etl/downloads) ![Monthly Downloads](https://poser.pugx.org/derafu/etl/d/monthly) A PHP package that transforms spreadsheet data into database structures and content with minimal effort. ## Overview Derafu ETL provides a streamlined solution for converting data between spreadsheets and databases. With a clean, fluent API, it simplifies complex data integration tasks through a pipeline architecture. ```php $pipeline = new Pipeline(); $result = $pipeline ->extract('data.xlsx') // Extract data from a spreadsheet. ->transform($rules) // Apply transformations rules (optional). ->load('database.sqlite') // Load into a database. ->execute() ; ``` ## Key Features {.list-unstyled} - 📤 **Extract** data from various sources (XLSX, ODS, CSV, databases). - 🔄 **Transform** data with customizable rules. - 📥 **Load** data into different target systems. - 🔁 **Bidirectional** conversion between spreadsheets and databases. - 🏗️ **Schema management** with automatic table creation and structure updates. - 📊 **Data visualization** capabilities with schema export to Markdown, D2, and more. - 🧩 **Extensible** architecture for custom source and target systems. ## Installation Install via Composer: ```bash composer require derafu/etl ``` ## Quick Start ### Command Line The quickest way to use Derafu ETL is through the command line: ```bash php app/console.php derafu:etl data.xlsx database.sqlite ``` This extracts data from `data.xlsx` and loads it into a new SQLite database on `database.sqlite`. #### Example Run the example used in tests with: ```shell php app/console.php derafu:etl tests/fixtures/spreadsheet-data.xlsx ``` This will create a `spreadsheet-data.sqlite` in the current directory. ### PHP Code ```php use Derafu\ETL\Pipeline\Pipeline; $pipeline = new Pipeline(); $result = $pipeline ->extract('data.xlsx') // Load data from a XLSX. ->transform() // This will use default transformations. ->load([ // You can specify the configuration for Doctrine. 'doctrine' => [ 'driver' => 'pdo_sqlite', 'path' => 'database.sqlite', ] ]) ->execute(); // This is will run the process. echo "Rows loaded: " . $result->rowsLoaded(); ``` ## Understanding ETL Pipelines An ETL pipeline consists of three main steps: 1. **Extract**: Read data from a source (e.g., spreadsheet). 2. **Transform**: Apply rules and transformations to the data. 3. **Load**: Write the transformed data to a target (e.g., database). Derafu ETL provides a clean interface for each step while handling the complex details behind the scenes. ## More than just move data to a target ### Export Database Schema to Markdown ```php use Derafu\ETL\Database\DatabaseManager; use Derafu\ETL\Schema\Target\MarkdownSchemaTarget; $manager = new DatabaseManager(); $database = $manager->connect('database.sqlite'); $target = new MarkdownSchemaTarget(); $markdown = $target->applySchema($database->schema()); file_put_contents('schema.md', $markdown); ``` ### Generate Database Diagram ```php use Derafu\ETL\Database\DatabaseManager; use Derafu\ETL\Schema\Target\D2SchemaTarget; $manager = new DatabaseManager(); $database = $manager->connect('database.sqlite'); $target = new D2SchemaTarget(); $d2 = $target->applySchema($database->schema()); file_put_contents('schema.d2', $d2); ``` --- ### ETL Architecture Architecture # Architecture This document explains the architecture and design principles behind Derafu ETL. ## ETL Pattern The Extract-Transform-Load (ETL) pattern is a data integration process used to collect data from various sources, transform it to fit operational needs, and load it into a target database for analysis and storage. ### The Three Phases 1. **Extract**: Gathering data from source systems. 2. **Transform**: Converting the extracted data to satisfy operational requirements. 3. **Load**: Writing the transformed data to the target system. ## Derafu ETL Implementation Derafu ETL implements this pattern with a clean, object-oriented approach centered around the Pipeline concept. ### Core Components {.w-75 .mx-auto} ![ETL Pipeline](https://www.derafu.dev/img/diagrams/content/docs/data/etl/derafu-etl-core-components.svg) #### Extract Phase - `DataSource`: Encapsulates the data source (spreadsheet, database). - `DataExtractor`: Handles extraction logic. - `SchemaSource`: Extracts schema information from the source. #### Transform Phase - `DataRules`: Defines transformation rules. - `DataTransformer`: Applies transformations to extracted data. #### Load Phase - `DataTarget`: Represents the destination system. - `DataLoader`: Manages loading data into the target. - `SchemaTarget`: Applies schema to the target system. ### Pipeline Orchestration The `Pipeline` class orchestrates the entire ETL process, providing a fluent interface: ```php $pipeline ->extract($source) // Configure extraction. ->transform($rules) // Configure transformation. ->load($target) // Configure loading. ->execute() // Run the pipeline. ; ``` When `execute()` is called, the pipeline: 1. Validates the configuration. 2. Extracts data from the source. 3. Transforms the data according to rules. 4. Synchronizes the target schema with the source. 5. Loads the transformed data into the target. 6. Returns a result object with statistics. ## Key Abstractions ### Database The `Database` abstraction provides a unified interface for different database types: - `SpreadsheetDatabase`: Treats spreadsheets as databases using [Derafu Spreadsheet](https://www.derafu.dev/docs/data/spreadsheet). - `DoctrineDatabase`: Works with any database supported by Doctrine DBAL. ### Schema The `Schema` system represents database structure: - Tables, columns, indexes, foreign keys. - Import/export to various formats (Spreadsheet, Doctrine, Markdown, D2, etc.). ## Extension Points Derafu ETL is designed for extensibility: 1. **New Data Sources**: Implement `DataSourceInterface`. 2. **Custom Transformations**: Extend `DataRules`. 3. **New Data Targets**: Implement `DataTargetInterface`. 4. **Schema Visualization**: Implement `SchemaTargetInterface`. ## Design Principles 1. **Separation of Concerns**: Each component has a clear responsibility. 2. **Fluent Interface**: Expressive, chainable API. 3. **Flexibility**: Support for various formats and systems. 4. **Extensibility**: Easy to extend with custom components. --- ### Schema Visualization Schema Visualization # Schema Visualization Derafu ETL includes powerful tools for visualizing database schemas through various output formats. These visualizations are useful for documentation, planning, and understanding the structure of your data. ## Available Visualization Formats Derafu ETL supports multiple visualization formats through schema targets: - **Markdown**: Human-readable documentation. - **D2**: Interactive entity-relationship diagrams. - **Text**: Simple text representation. - **SQL**: Database creation scripts. - **Doctrine Schema**: Programmatic schema representation. ## Using Schema Targets Schema targets implement the `SchemaTargetInterface` and convert a schema to a specific format. ### Basic Usage All schema targets follow a similar pattern: ```php use Derafu\ETL\Database\DatabaseManager; use Derafu\ETL\Schema\Target\CustomSchemaTarget; // Just an example. // Connect to the database. $manager = new DatabaseManager(); $database = $manager->connect('database.sqlite'); // Get the schema. $schema = $database->schema(); // Create the target. $target = new CustomSchemaTarget(); // Apply the schema to the target. $output = $target->applySchema($schema); // Use the output. file_put_contents('output-file.custom', $output); ``` ## Markdown Schema The `MarkdownSchemaTarget` generates comprehensive Markdown documentation for your database schema. ```php use Derafu\ETL\Schema\Target\MarkdownSchemaTarget; $target = new MarkdownSchemaTarget(); $markdown = $target->applySchema($schema); file_put_contents('schema.md', $markdown); ``` The generated Markdown includes: - Table of contents. - Detailed table definitions. - Column information with types and constraints. - Primary keys, indexes, and foreign key relationships. Example output: ```markdown # Database Schema ## Table of Contents - [Table: users](#table-users) - [Table: posts](#table-posts) ## Table: users {#table-users} ### Columns | Column | Type | Attributes | Description | |------------|-------------|-------------------------|-------------| | id | integer | PRIMARY KEY / NOT NULL | | | username | string(100) | NOT NULL | | | email | string(255) | NOT NULL | | | created_at | datetime | NOT NULL | | ### Primary Key - Columns: `id` ### Indexes | Name | Columns | Type | Flags | |--------------|----------|--------|-------| | idx_username | username | INDEX | | | idx_email | email | UNIQUE | | ``` ## D2 Diagrams The `D2SchemaTarget` generates diagrams in [D2 format](https://d2lang.com/), a modern diagram scripting language. ```php use Derafu\ETL\Schema\Target\D2SchemaTarget; $target = new D2SchemaTarget( detailLevel: D2SchemaTarget::DETAIL_FULL, direction: D2SchemaTarget::DIRECTION_RIGHT, layout: D2SchemaTarget::LAYOUT_DEFAULT, includeIndexes: true ); $d2 = $target->applySchema($schema); file_put_contents('schema.d2', $d2); ``` Options include: - **Detail Level**: `DETAIL_FULL`, `DETAIL_KEYS_ONLY`, `DETAIL_MINIMAL`. - **Direction**: `DIRECTION_UP`, `DIRECTION_DOWN`, `DIRECTION_LEFT`, `DIRECTION_RIGHT`. - **Layout**: `LAYOUT_DEFAULT`, `LAYOUT_CLUSTERED`, `LAYOUT_HIERARCHICAL`. - **Include Indexes**: Whether to include indexes in the diagram. Example D2 output: ``` # Database Schema direction: right # Tables users: { shape: sql_table id: integer NOT NULL pk username: string(100) NOT NULL column email: string(255) NOT NULL column created_at: datetime NOT NULL column } posts: { shape: sql_table id: integer NOT NULL pk user_id: integer NOT NULL fk title: string(255) NOT NULL column content: text column created_at: datetime NOT NULL column } # Relationships posts -> users ``` ## Practical Applications ### Documentation Generate comprehensive documentation for your database: ```php use Derafu\ETL\Database\DatabaseManager; use Derafu\ETL\Schema\Target\MarkdownSchemaTarget; $manager = new DatabaseManager(); $database = $manager->connect('production.sqlite'); $target = new MarkdownSchemaTarget(); $docs = $target->applySchema($database->schema()); file_put_contents('database-schema.md', $docs); ``` ### Visual Modeling Create visual diagrams for presentations or analysis: ```php use Derafu\ETL\Schema\Target\D2SchemaTarget; // Filter to only show specific tables. $target = new D2SchemaTarget( tableFilter: ['users', 'posts', 'comments'] ); $diagram = $target->applySchema($schema); file_put_contents('entity-relationship.d2', $diagram); ``` ### Schema Migration Export a schema definition to create a new database: ```php use Derafu\ETL\Schema\Target\SqliteSchemaTarget; $target = new SqliteSchemaTarget(); $sql = $target->applySchema($schema); file_put_contents('schema.sql', $sql); ``` ## Integration with ETL Pipeline Schema visualization can be integrated with the ETL pipeline for comprehensive documentation: ```php use Derafu\ETL\Pipeline\Pipeline; use Derafu\ETL\Schema\Target\MarkdownSchemaTarget; // Run ETL pipeline. $pipeline = new Pipeline(); $result = $pipeline ->extract('data.xlsx') ->transform() ->load('database.sqlite') ->execute(); // Document the resulting database. $database = $result->target()->database(); $target = new MarkdownSchemaTarget(); $docs = $target->applySchema($database->schema()); file_put_contents('database-schema.md', $docs); ``` --- ## Spreadsheet Project Derafu Spreadsheet # Derafu Spreadsheet --- ### Introduction Unified Spreadsheet Processing for PHP # Unified Spreadsheet Processing for PHP ![GitHub last commit](https://img.shields.io/github/last-commit/derafu/spreadsheet/main) ![CI Workflow](https://github.com/derafu/spreadsheet/actions/workflows/ci.yml/badge.svg?branch=main&event=push) ![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/derafu/spreadsheet) ![GitHub Issues](https://img.shields.io/github/issues-raw/derafu/spreadsheet) ![Total Downloads](https://poser.pugx.org/derafu/spreadsheet/downloads) ![Monthly Downloads](https://poser.pugx.org/derafu/spreadsheet/d/monthly) Derafu Spreadsheet is a modern PHP library that provides a **single, consistent API** for working with spreadsheet files in multiple formats (XLSX, CSV, ODS, JSON, XML, YAML, and more). ## 🌟 Features - **Unified API** across all file formats. - **Multiple format support**: XLSX, XLS, CSV, ODS, JSON, XML, YAML, HTML, PDF. - **Smart type casting**: Automatically detects and converts data types (dates, numbers, booleans, JSON). - **Format-agnostic data manipulation**: Work with your data consistently regardless of source format. - **Minimal dependencies**: Use only what you need. - **PSR-7 compatible**: Generate HTTP responses with downloadable spreadsheets. - **Modern PHP**: Written for PHP 8 with strict typing. ## 🚀 Why Derafu Spreadsheet? While powerful libraries like PhpSpreadsheet exist, and this library use it under de hood, they often require different approaches for different formats and have complex APIs. Derafu Spreadsheet offers several key advantages: - **Simplified API**: Work with all spreadsheet formats through a consistent interface. - **Intelligent type handling**: Focus on your data, not type conversion. - **Format abstraction**: Write your code once, and it works with any format. - **Flexible format handlers**: Easily switch between formats or implement custom handlers. - **Minimal learning curve**: Clean, intuitive API with sensible defaults. ## 📦 Installation ```bash composer require derafu/spreadsheet ``` For specific format support, you may need additional dependencies: ```bash # For CSV support with League CSV (recommended). composer require league/csv # For XLSX, XLS, ODS, HTML support. composer require phpoffice/phpspreadsheet # For YAML support. composer require symfony/yaml # For HTTP response generation. composer require nyholm/psr7 ``` ## 📝 Basic Usage ```php <?php use Derafu\Spreadsheet\SpreadsheetLoader; use Derafu\Spreadsheet\SpreadsheetDumper; // Load a spreadsheet (format auto-detected from extension). // Under the hood it will create default Factory and Caster instances. If you // don't want that, inject your implementation. $loader = new Loader(); $spreadsheet = $loader->loadFromFile('data.xlsx'); // Access data from sheets. $sheet = $spreadsheet->getSheet('Sheet1'); $rows = $sheet->getRows(); // Modify data. $sheet->setCell(0, 0, 'Updated value'); $spreadsheet->createSheet('NewSheet', [['Header1', 'Header2'], [1, 2]]); // Save in different format. // Under the hood it will create default Factory and Caster instances. If you // don't want that, inject your implementation. $dumper = new Dumper(); $dumper->dumpToFile($spreadsheet, 'output.csv'); // Or convert to string $jsonString = $dumper->dumpToString($spreadsheet, 'json'); ``` ## 🔄 Working with Different Formats Derafu Spreadsheet handles format conversion automatically: ```php // Load Excel file. $spreadsheet = $loader->loadFromFile('data.xlsx'); // Save as CSV. $dumper->dumpToFile($spreadsheet, 'data.csv'); // Save as JSON. $dumper->dumpToFile($spreadsheet, 'data.json'); // Save as YAML. $dumper->dumpToFile($spreadsheet, 'data.yaml'); ``` ## 📋 Intelligent Type Casting One of Derafu Spreadsheet's key features is automatic type casting for both reading and writing: ```php $spreadsheet = $loader->loadFromFile('data.csv'); // String '123' is automatically cast to integer 123. // 'true' is cast to boolean true. // '2025-03-12' is cast to DateTimeImmutable object. // JSON strings are parsed to indexed arrays or associative arrays ("objects"). $cell = $sheet->getCell(0, 0); // Typed data, not just strings. // When writing, types are automatically converted to appropriate formats. $dumper->dumpToFile($spreadsheet, 'output.xlsx'); ``` --- ### Architecture Architecture # Architecture This document provides an overview of the architecture and design principles behind Derafu Spreadsheet. ## Core Components Derafu Spreadsheet is built around several key components that work together to provide a unified approach to spreadsheet processing: ![Architecture Diagram](https://www.derafu.dev/img/diagrams/content/docs/data/spreadsheet/derafu-spreadsheet-architecture-diagram.svg) ### Key Components 1. **Loader** - Handles loading spreadsheet data from files or strings. - Uses Factory to create appropriate Format Handlers. - Uses Caster to convert raw data to appropriate PHP types. 2. **Dumper** - Handles saving spreadsheet data to files or strings. - Uses Factory to create appropriate Format Handlers. - Uses Caster to convert PHP types to format-appropriate representations. 3. **Factory** - Creates and manages Format Handlers based on file extension or specified format. - Allows registering custom Format Handlers. - Provides format detection capabilities. 4. **Caster** - Converts raw data values to appropriate PHP types when reading. - Converts PHP types to appropriate string representations when writing. - Handles intelligent date, boolean, numeric, and JSON detection. 5. **Format Handlers** - Format-specific implementations that know how to read/write a particular format. - All implement a common FormatHandlerInterface. - Includes handlers for XLSX, XLS, CSV, ODS, JSON, XML, YAML, HTML, PDF. 6. **Data Model** - **Spreadsheet**: Top-level container for all data. - **Sheet**: Container for rows and cells with a name. - Both implement interfaces for consistent interaction. ## Design Principles Derafu Spreadsheet was designed with several key principles in mind: 1. **Unified API** - Consistent interface across all supported formats. - Same code works regardless of source or target format. 2. **Type Intelligence** - Automatic conversion between string values and appropriate PHP types. - No manual type casting required in application code. 3. **Separation of Concerns** - Format handling is separated from data model. - Type conversion is separated from data loading/saving. - Each component has a single, clear responsibility. 4. **Interface-Based Design** - All components define and implement interfaces. - Allows for custom implementations and extensions. 5. **Minimal Dependencies** - Core functionality has minimal dependencies. - Format-specific dependencies only required for formats you use. ## Data Flow When working with Derafu Spreadsheet, data flows through the components as follows: ### Loading Process: 1. **Loader** receives a file or string. 2. **Factory** creates appropriate Format Handler based on format. 3. **Format Handler** reads raw data into internal Spreadsheet structure. 4. **Caster** converts raw values to appropriate PHP types. 5. Typed **Spreadsheet** object is returned to application. ### Saving Process: 1. **Dumper** receives a Spreadsheet object and target format. 2. **Caster** converts PHP values to appropriate string representations. 3. **Factory** creates appropriate Format Handler for target format. 4. **Format Handler** writes structured data to file or string. 5. File path or string content is returned to application. ## Extensibility Derafu Spreadsheet is designed to be extensible: - **Custom Format Handlers**: Create handlers for proprietary or custom formats. - **Custom Casters**: Implement specialized type conversion logic. - **HTTP Integration**: Generate responses with downloadable spreadsheets. - **Framework Integration**: Easy to integrate with popular PHP frameworks. ## Directory Structure ``` src/ ├── Abstract/ │ └── AbstractPhpSpreadsheetFormatHandler.php ├── Contract/ │ ├── SpreadsheetCasterInterface.php │ ├── SpreadsheetDumperInterface.php │ ├── FactoryInterface.php │ ├── FormatHandlerInterface.php │ ├── SpreadsheetLoaderInterface.php │ ├── SheetInterface.php │ ├── SpreadsheetInterface.php │ └── Http/ │ └── SpreadsheetHttpResponseGeneratorInterface.php ├── Exception/ │ ├── SpreadsheetDumpException.php │ ├── SpreadsheetFileNotFoundException.php │ ├── SpreadsheetFormatNotSupportedException.php │ └── SpreadsheetLoadException.php ├── Format/ │ ├── CsvLeagueHandler.php │ ├── CsvPhpSpreadsheetHandler.php │ ├── HtmlHandler.php │ ├── JsonHandler.php │ ├── OdsHandler.php │ ├── PdfHandler.php │ ├── XlsHandler.php │ ├── XlsxHandler.php │ ├── XmlHandler.php │ └── YamlHandler.php ├── Http/ │ └── NyholmSpreadsheetHttpResponseGenerator.php ├── Caster.php ├── Dumper.php ├── Factory.php ├── Loader.php ├── Sheet.php └── Spreadsheet.php ``` --- ### Install Install the library # Install the library This guide will walk you through installing Derafu Spreadsheet and its dependencies. ## Basic Installation The simplest way to install Derafu Spreadsheet is through Composer: ```bash composer require derafu/spreadsheet ``` This installs the core package. Out of the box, only works with JSON and XML. Depending on which file formats you want to work with, you may need to install additional dependencies. ## Format-Specific Dependencies Derafu Spreadsheet supports multiple formats through different handlers. Each format may require additional dependencies: ### CSV Support For CSV support, you can choose between two handlers: ```bash # Recommended: League CSV (faster, more memory efficient). composer require league/csv # Alternative: PhpSpreadsheet CSV handling. composer require phpoffice/phpspreadsheet ``` ### Excel and OpenDocument Support For XLSX, XLS, and ODS support: ```bash composer require phpoffice/phpspreadsheet ``` ### YAML Support For YAML support: ```bash composer require symfony/yaml ``` ### PDF Support For PDF export: ```bash # Base requirement. composer require phpoffice/phpspreadsheet # Choose one PDF library: composer require mpdf/mpdf # For MpdfWriter (recommended). # OR composer require dompdf/dompdf # For DompdfWriter. # OR composer require tecnickcom/tcpdf # For TcpdfWriter. ``` ### HTTP Response Support For PSR-7 HTTP response integration: ```bash composer require nyholm/psr7 ``` If you don't want to use nyholm/psr7, you can use any PSR-7 compatible library implementing by your own `SpreadsheetHttpResponseGeneratorInterface`. ## Complete Installation (All Formats) If you want to support all formats, with the default handlers, you can install all dependencies at once: ```bash composer require derafu/spreadsheet league/csv phpoffice/phpspreadsheet \ symfony/yaml mpdf/mpdf nyholm/psr7 ``` ## Installation in Frameworks ### Symfony To use Derafu Spreadsheet in Symfony, install the package: ```bash composer require derafu/spreadsheet ``` Then define services in your `services.yaml`: ```yaml services: Derafu\Spreadsheet\Contract\SpreadsheetFactoryInterface: class: Derafu\Spreadsheet\Factory Derafu\Spreadsheet\Contract\SpreadsheetCasterInterface: class: Derafu\Spreadsheet\Caster Derafu\Spreadsheet\Contract\SpreadsheetLoaderInterface: class: Derafu\Spreadsheet\Loader arguments: - '@Derafu\Spreadsheet\Factory' - '@Derafu\Spreadsheet\Caster' Derafu\Spreadsheet\Contract\SpreadsheetDumperInterface: class: Derafu\Spreadsheet\Dumper arguments: - '@Derafu\Spreadsheet\Factory' - '@Derafu\Spreadsheet\Caster' ``` Check the constructors of the classes to see what arguments they expect. You can configure them as you want, for example change the delimiter for CSV. --- ### Basic Usage Basic Usage # Basic Usage This guide covers the fundamental operations with Derafu Spreadsheet: loading, manipulating, and saving spreadsheet data. ## Loading a Spreadsheet You can load a spreadsheet from a file or from a string: ```php <?php use Derafu\Spreadsheet\SpreadsheetLoader; // Create a loader. $loader = new Loader(); // From a file (format auto-detected from extension). $spreadsheet = $loader->loadFromFile('data.xlsx'); // From a string (must specify format). $csvString = "header1,header2\nvalue1,value2"; $spreadsheet = $loader->loadFromString($csvString, 'csv'); ``` ## Working with Sheets A spreadsheet contains one or more sheets, which you can access and manipulate: ```php // Get all sheet names. $sheetNames = $spreadsheet->getSheetNames(); // Get a specific sheet. $sheet = $spreadsheet->getSheet('Sheet1'); // Create a new sheet with data. $spreadsheet->createSheet('NewSheet', [ ['Header1', 'Header2', 'Header3'], ['Value1', 'Value2', 'Value3'], ['Value4', 'Value5', 'Value6'] ]); // Set active sheet. $spreadsheet->setActiveSheet('NewSheet'); // Get active sheet. $activeSheet = $spreadsheet->getActiveSheet(); // Check if a sheet exists. if ($spreadsheet->hasSheet('SomeSheet')) { // ... } // Remove a sheet. $spreadsheet->removeSheet('SomeSheet'); ``` ## Working with Rows and Cells Once you have a sheet, you can access and modify its data: ```php // Get all rows. $rows = $sheet->getRows(); // Get a specific row. $firstRow = $sheet->getRow(0); // Add a new row. $sheet->addRow(['New', 'Row', 'Data']); // Get a specific cell. $value = $sheet->getCell(0, 0); // Row 0, Column 0. // or with named columns (for associative sheets). $value = $sheet->getCell(0, 'column_name'); // Update a cell. $sheet->setCell(0, 0, 'New Value'); // Get header row. $headers = $sheet->getHeaderRow(); // Get data rows (excludes header for indexed sheets). $dataRows = $sheet->getDataRows(); ``` ## Data Types Thanks to the automatic type casting, you can work with native PHP types: ```php // Numbers are already cast to int/float. $number = $sheet->getCell(0, 0); // e.g., 123 (int). // Dates are cast to DateTimeImmutable. $date = $sheet->getCell(0, 1); // e.g., DateTimeImmutable object. echo $date->format('Y-m-d'); // "2025-03-12". // Booleans are properly typed. $bool = $sheet->getCell(0, 2); // e.g., true (bool). // You can set any type and it will be properly stored. $sheet->setCell(0, 3, new DateTimeImmutable()); $sheet->setCell(0, 4, ['array', 'values']); $sheet->setCell(0, 5, ['nested' => ['object' => 'structure']]); ``` ## Associative vs. Indexed Sheets Derafu Spreadsheet supports both associative (column names as keys) and indexed (numeric keys) sheets: ```php // Check if a sheet is associative. if ($sheet->isAssociative()) { // Work with associative data. $rowData = $sheet->getRow(0); echo $rowData['column_name']; } else { // Work with indexed data. $rowData = $sheet->getRow(0); echo $rowData[0]; // First column. } // Convert between formats. $associativeSheet = $sheet->toAssociative(); // First row becomes header. $indexedSheet = $sheet->toIndexed(); // Keys become first row. ``` ## Saving a Spreadsheet Once you're done modifying the data, you can save it to a file or get it as a string: ```php use Derafu\Spreadsheet\SpreadsheetDumper; // Create a dumper. $dumper = new Dumper(); // Save to a file (format detected from extension). $dumper->dumpToFile($spreadsheet, 'output.xlsx'); // Convert to a different format. $dumper->dumpToFile($spreadsheet, 'output.csv'); // Get as a string. $jsonString = $dumper->dumpToString($spreadsheet, 'json'); ``` ## Creating a Spreadsheet from Scratch You can also create a spreadsheet from scratch: ```php use Derafu\Spreadsheet\Spreadsheet; // Create an empty spreadsheet. $spreadsheet = new Spreadsheet(); // Create a sheet with data. $spreadsheet->createSheet('Sheet1', [ ['Name', 'Age', 'Email'], ['John Doe', 30, 'john@example.com'], ['Jane Smith', 25, 'jane@example.com'] ]); // Create from array. $data = [ 'Users' => [ ['Name', 'Age', 'Email'], ['John Doe', 30, 'john@example.com'], ['Jane Smith', 25, 'jane@example.com'] ], 'Products' => [ ['ID', 'Name', 'Price'], [1, 'Product A', 29.99], [2, 'Product B', 49.99] ] ]; $spreadsheet = Spreadsheet::fromArray($data); // Save it. $dumper = new Dumper(); $dumper->dumpToFile($spreadsheet, 'new_spreadsheet.xlsx'); ``` --- ### Format Handlers Format Handlers # Format Handlers Format handlers are a key component of Derafu Spreadsheet that enable the library to work with different file formats. Each format handler implements the `FormatHandlerInterface` and knows how to read and write a specific format. ## Supported Formats ![Format conversion diagram](https://www.derafu.dev/img/diagrams/content/docs/data/spreadsheet/derafu-spreadsheet-format-conversion-diagram.svg) Derafu Spreadsheet supports the following formats in the core package: | Format | Extension | Handler Class | Dependencies | |----------------------|-----------|----------------------------|-------------------------------| | Excel XLSX | .xlsx | `XlsxHandler` | PhpSpreadsheet | | Excel XLS | .xls | `XlsHandler` | PhpSpreadsheet | | OpenDocument | .ods | `OdsHandler` | PhpSpreadsheet | | CSV (League) | .csv | `CsvLeagueHandler` | League/CSV | | CSV (PhpSpreadsheet) | .csv | `CsvPhpSpreadsheetHandler` | PhpSpreadsheet | | JSON | .json | `JsonHandler` | PHP built-in | | XML | .xml | `XmlHandler` | PHP built-in | | YAML | .yaml | `YamlHandler` | Symfony/Yaml | | HTML | .html | `HtmlHandler` | PhpSpreadsheet | | PDF | .pdf | `PdfHandler` | PhpSpreadsheet + PDF renderer | ## How Format Handlers Work Each format handler is responsible for: 1. Loading data from a file into a `SpreadsheetInterface` object. 2. Dumping data from a `SpreadsheetInterface` object to a file or string (memory). 3. Providing metadata like the file extension and MIME type. The `Factory` class manages the format handlers and automatically selects the appropriate handler based on the file extension. ## Using Format Handlers Most of the time, you don't need to interact with format handlers directly. The `Loader` and `Dumper` classes handle this for you: ```php $loader = new Loader(); $spreadsheet = $loader->loadFromFile('data.xlsx'); $dumper = new Dumper(); $dumper->dumpToFile($spreadsheet, 'output.csv'); ``` However, if needed, you can access format handlers directly through the `Factory`: ```php $factory = new Factory(); $xlsxHandler = $factory->createFormatHandler('filepath.xlsx'); // or with explicit format $csvHandler = $factory->createFormatHandler('filepath.file', 'csv'); ``` ## Format-Specific Options Some format handlers support additional options in the constructor. ### CSV Options (CsvLeagueHandler) ```php $csvHandler = new CsvLeagueHandler( delimiter: ',', // Column delimiter. enclosure: '"', // Field enclosure character. escape: '\\', // Escape character. sheetName: 'Sheet' // Default sheet name when loading CSV. ); ``` ### OpenDocument/Excel Options (OdsHandler, XlsHandler, XlsxHandler) ```php $xlsxHandler = new XlsxHandler( readDataOnly: true // Read values only, ignore formatting. ); ``` ### PDF Options (PdfHandler) ```php $pdfHandler = new PdfHandler( writerType: 'Mpdf' // Can be 'Mpdf', 'Dompdf', or 'Tcpdf'. ); ``` ### JSON Options (JsonHandler) ```php $jsonHandler = new JsonHandler( encodeOptions: JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE, decodeOptions: JSON_OBJECT_AS_ARRAY, depth: 512, sheetName: 'Sheet' ); ``` ### YAML Options (YamlHandler) ```php use Symfony\Component\Yaml\Yaml; $yamlHandler = new YamlHandler( parseFlags: Yaml::PARSE_EXCEPTION_ON_INVALID_TYPE, dumpFlags: Yaml::DUMP_MULTI_LINE_LITERAL_BLOCK, sheetName: 'Sheet' ); ``` ## Custom Format Handlers You can create your own format handlers by implementing the `FormatHandlerInterface`: ```php use Derafu\Spreadsheet\Contract\SpreadsheetFormatHandlerInterface; class MyCustomHandler implements FormatHandlerInterface { // Implement required methods. } // Register with the factory. $factory = new Factory(); $factory->registerFormatHandler('custom', MyCustomHandler::class); // Now you can use it. $loader = new Loader($factory); $spreadsheet = $loader->loadFromFile('filepath.custom'); ``` ## Format Detection The `Factory` class detects the format based on the file extension. If the file doesn't have an extension or if you want to override it, you can explicitly specify the format: ```php $loader = new Loader(); $spreadsheet = $loader->loadFromFile('filepath', 'xlsx'); // Treat as XLSX. ``` You can also check which formats are supported: ```php $factory = new Factory(); $supportedFormats = $factory->getSupportedFormats(); // ['csv', 'xlsx', 'xls', 'ods', 'xml', 'json', 'yaml', 'html', 'pdf'] ``` ## Alternative Format Handlers For some formats like CSV, Derafu Spreadsheet provides multiple handlers. For example: - `CsvLeagueHandler` - Uses League/CSV (recommended for most cases). - `CsvPhpSpreadsheetHandler` - Uses PhpSpreadsheet. You can choose which handler to use by registering it for the format: ```php $factory = new Factory(); $factory->registerFormatHandler('csv', CsvPhpSpreadsheetHandler::class); $loader = new Loader($factory); // Now CSV files will use PhpSpreadsheet handler. ``` ## Format Handling and Type Casting When loading a file, the process works like this: 1. Format handler reads the raw data from the file. 2. Data is converted to a `SpreadsheetInterface` object. 3. `Caster` processes all values to convert them to the appropriate PHP types. When saving a file: 1. `Caster` converts PHP types to appropriate string representations. 2. Format handler writes the data to the file in the correct format. --- ### Type Casting Type Casting # Type Casting One of the most powerful features of Derafu Spreadsheet is its intelligent type casting system. This enables seamless conversion between spreadsheet file data and native PHP data types. ## The Type Casting Problem Spreadsheet files typically store everything as text, but your application needs properly typed data to work effectively. Other libraries often leave the type conversion to you, resulting in code like: ```php // Without automatic type casting. $value = $spreadsheet->getCell(0, 0); if (is_numeric($value)) { $value = (int)$value; } elseif ($value === 'true' || $value === 'false') { $value = $value === 'true'; } elseif (preg_match('/^\d{4}-\d{2}-\d{2}$/', $value)) { $value = new \DateTime($value); } // ... and so on for every cell. ``` ## Automatic Type Casting with Derafu Spreadsheet Under the hood, the `Caster` class automatically handles all of these conversions for you: ```php // With Derafu Spreadsheet - all values are properly typed. $spreadsheet = $loader->loadFromFile('data.csv'); $value = $spreadsheet->getSheet('Sheet1')->getCell(0, 0); // Already properly typed! ``` {.w-25 .mx-auto} ![Type casting flow diagram](https://www.derafu.dev/img/diagrams/content/docs/data/spreadsheet/derafu-spreadsheet-type-casting-flow-diagram.svg) ## Supported Type Conversions ### After doing a load (reading from file/data to PHP) | File Value | PHP Type | Example | |---------------|-----------------------|-----------------------------------------------| | Empty string | `null` | `""` → `null` | | Integer | `int` | `"123"` → `123` | | Decimal | `float` | `"123.45"` → `123.45` | | Boolean text | `bool` | `"true"` → `true` | | Date string | `DateTimeImmutable` | `"2025-03-12"` → `DateTimeImmutable` | | ISO 8601 date | `DateTimeImmutable` | `"2025-03-12T14:30:00"` → `DateTimeImmutable` | | JSON array | `array` | `"[1,2,3]"` → `[1, 2, 3]` | | JSON object | `array` (associative) | `"{"key":"value"}"` → `["key" => "value"]` | ### Before doing a dump (writing from PHP to file/data) | PHP Type | File Value | Example | |----------------------|-----------------------|-----------------------------------------------| | `null` | Empty string | `null` → `""` | | `int`/`float` | Preserved as is | `123.45` → `123.45` | | `bool` | String "true"/"false" | `true` → `"true"` | | `DateTimeInterface` | ISO 8601 / Date | `new DateTime()` → `"2025-03-12T14:30:00"` | | `array`/`object` | JSON string | `["a", "b"]` → `"["a","b"]"` | | `Stringable` objects | String representation | `$stringable` → `(string)$stringable` | ## Date Format Detection The library automatically detects various date formats: - `Y-m-d` (2025-03-12). - `d/m/Y` (12/03/2025). - `Y-m-d H:i:s` (2025-03-12 14:30:45). - `d/m/Y H:i:s` (12/03/2025 14:30:45). - `Y-m-d\TH:i:s` (2025-03-12T14:30:45). - `Y-m-d\TH:i:sP` (2025-03-12T14:30:45+00:00). All dates are automatically converted to UTC for consistency. ## JSON Detection and Parsing With strings that appear to be JSON (starting with `{` or `[`), the caster automatically will try to parse them into PHP arrays or objects: ```php // Cell contains: {"name":"John","age":30} $person = $sheet->getCell(0, 0); // Returns associative array: ["name" => "John", "age" => 30] ``` ## Custom Type Casting If you need different casting behavior, you can implement your own `SpreadsheetCasterInterface`: ```php use Derafu\Spreadsheet\Contract\SpreadsheetCasterInterface; class MyCaster implements SpreadsheetCasterInterface { // Your custom implementation. } // Then use it with your loader/dumper. $loader = new Loader(new Factory(), new MyCaster()); ``` ## How Type Casting Works Internally 1. When loading a file with `Loader`, after the raw data is read, `Caster::castAfterLoad()` is called. 2. When saving a file with `Dumper`, before writing data, `Caster::castBeforeDump()` is called. 3. The type casting is performed on every cell in every sheet, ensuring consistent typing. ## Performance Considerations Type casting is performed in-memory and typically adds minimal overhead. However, for extremely large spreadsheets with millions of cells, you might consider implementing a more selective casting strategy through a custom `SpreadsheetCasterInterface` implementation. If you don't want to use the Caster, you can use the Factory with the Format Handlers directly. If you bypass the Loader and Dumper, no type casting will be done. --- ### HTTP Responses Generating HTTP Responses # Generating HTTP Responses Derafu Spreadsheet makes it easy to generate downloadable spreadsheet files directly from your web application. This guide shows how to use the library to create HTTP responses for spreadsheet downloads. ## PSR-7 Response Integration Derafu Spreadsheet includes a PSR-7 compatible response generator for easy integration with frameworks that support PSR-7 standards. ### Requirements To use the HTTP response features, you need to install Nyholm's PSR-7 implementation: ```bash composer require nyholm/psr7 ``` If you don’t want to use nyholm/psr7, you can use any PSR-7 compatible library implementing by your own `SpreadsheetHttpResponseGeneratorInterface`. ### Basic Usage ```php <?php use Derafu\Spreadsheet\Http\NyholmSpreadsheetHttpResponseGenerator; use Derafu\Spreadsheet\Spreadsheet; // Create or load a spreadsheet. $spreadsheet = new Spreadsheet(); $spreadsheet->createSheet('Sheet1', [ ['Name', 'Email', 'Age'], ['John Doe', 'john@example.com', 30], ['Jane Smith', 'jane@example.com', 25] ]); // Create response generator. $responseGenerator = new NyholmSpreadsheetHttpResponseGenerator(); // Generate PSR-7 response with auto-detected format (xlsx). $response = $responseGenerator->createResponse( $spreadsheet, 'users-export.xlsx' ); // The response can now be sent by any PSR-7 compatible framework. ``` ### Specifying Format You can explicitly specify the format for the response: ```php $response = $responseGenerator->createResponse( $spreadsheet, 'users-export.csv', 'csv' // Explicitly specify format. ); ``` ### PSR-7 Compatible Frameworks (Slim, Mezzio, etc.) With PSR-7 compatible frameworks, you can directly use the `NyholmSpreadsheetHttpResponseGenerator`: ```php use Derafu\Spreadsheet\Http\NyholmSpreadsheetHttpResponseGenerator; use Derafu\Spreadsheet\SpreadsheetLoader; use Psr\Http\Message\ResponseInterface; use Psr\Http\Message\ServerRequestInterface; class ExportAction { public function __invoke(ServerRequestInterface $request): ResponseInterface { // Create or load your spreadsheet $loader = new Loader(); $spreadsheet = $loader->loadFromFile('data.xlsx'); // Modify it if needed // ... // Generate response $responseGenerator = new NyholmSpreadsheetHttpResponseGenerator(); return $responseGenerator->createResponse( $spreadsheet, 'exported-data.xlsx' ); } } ``` ## Custom Response Generation If you need to customize the response generation or use a different PSR-7 implementation, you can implement your own `SpreadsheetHttpResponseGeneratorInterface`: ```php <?php namespace App\Spreadsheet; use Derafu\Spreadsheet\Contract\Http\SpreadsheetHttpResponseGeneratorInterface; use Derafu\Spreadsheet\Contract\SpreadsheetInterface; use Psr\Http\Message\ResponseInterface; class CustomResponseGenerator implements SpreadsheetHttpResponseGeneratorInterface { public function createResponse( SpreadsheetInterface $spreadsheet, ?string $filename = null, ?string $format = null ): ResponseInterface { // Your custom implementation } } ``` ## MIME Types for Different Formats When generating HTTP responses, it's important to use the correct MIME type. Derafu Spreadsheet handles this for you, but here's a reference of the MIME types used for each format: | Format | MIME Type | |--------|-------------------------------------------------------------------| | XLSX | application/vnd.openxmlformats-officedocument.spreadsheetml.sheet | | XLS | application/vnd.ms-excel | | CSV | text/csv | | ODS | application/vnd.oasis.opendocument.spreadsheet | | JSON | application/json | | XML | application/xml | | YAML | application/yaml | | HTML | text/html | | PDF | application/pdf | ## Security Considerations When generating downloadable files from user data, be sure to: 1. **Sanitize data**: Ensure user-provided data doesn't contain malicious content. 2. **Validate filenames**: Clean and validate user-provided filenames. 3. **Set appropriate headers**: Ensure you're using the correct content type and disposition. 4. **Clean up temporary files**: Remove any temporary files after sending the response. ## Performance Tips For large spreadsheets, generating the response can be resource-intensive. Consider: 1. **Queuing exports**: For large exports, process them in a background job and notify the user when ready. 2. **Streaming responses**: Some frameworks support streaming responses to reduce memory usage. 3. **Pagination**: Consider exporting data in smaller batches if possible. --- ### Comparison Comparison with Alternative Libraries # Comparison with Alternative Libraries When choosing a spreadsheet library for PHP, it's important to understand the differences between available options. This guide compares Derafu Spreadsheet with some popular alternatives. ## Derafu Spreadsheet vs PhpSpreadsheet [PhpSpreadsheet](https://github.com/PHPOffice/PhpSpreadsheet) is the most widely used PHP spreadsheet library and is actually used internally by Derafu Spreadsheet for some format handlers. ### Advantages of Derafu Spreadsheet - **Simplified API**: The API is more streamlined and consistent across formats. - **Customizable type casting**: Has a custom and flexible type casting system. - **Format-agnostic code**: Write code once that works with any spreadsheet format. - **Easier data access**: Direct methods for working with rows and cells. - **Native handling of JSON/YAML**: Built-in support for these data exchange formats. - **JSON inside**: Allows you to easily store and retrieve JSON data inside spreadsheets. - **Modular approach**: Use only the format handlers you need. - **Less overhead**: Simpler in-memory structure for data representation. ### When to use PhpSpreadsheet instead - **Advanced Excel features**: If you need complex Excel features like conditional formatting, charts, etc. - **Cell styling**: If you need detailed control over cell appearance. - **Formula calculation**: If you need to evaluate Excel formulas. - **Mature ecosystem**: PhpSpreadsheet has been around longer and has more examples and community code. ## Derafu Spreadsheet vs league/csv [league/csv](https://csv.thephpleague.com/) is a focused library for working with CSV files. Internally, Derafu Spreadsheet uses it, as default, for CSV format handling. ### Advantages of Derafu Spreadsheet - **Multiple format support**: Work with many formats using the same code. - **Automatic type casting**: league/csv returns everything as strings. - **Higher-level abstractions**: Work with the concepts of sheets, rows, and cells. - **Easy format conversion**: Convert between CSV and other formats seamlessly. ### When to use league/csv instead - **CSV-only projects**: If you only need to work with CSV files. - **Memory efficiency for large files**: league/csv has specialized streaming capabilities. - **CSV-specific features**: If you need advanced CSV features like RFC compliance options. - **Simplicity**: If you want a more focused, single-purpose library. ## Code Comparison Examples ### Basic Reading Example **Derafu Spreadsheet**: ```php use Derafu\Spreadsheet\SpreadsheetLoader; $loader = new Loader(); $sheet = $loader->loadFromFile('data.xlsx')->getActiveSheet(); foreach ($sheet->getDataRows() as $row) { $id = $row[0]; // Already cast to proper type (int). $date = $row[1]; // Already a DateTimeImmutable. // Process... } ``` **PhpSpreadsheet**: ```php use DateTimeImmutable; use PhpOffice\PhpSpreadsheet\IOFactory; $spreadsheet = IOFactory::load('data.xlsx'); $sheet = $spreadsheet->getActiveSheet(); foreach ($sheet->getRowIterator(2) as $row) { $cellIterator = $row->getCellIterator(); $rowData = []; foreach ($cellIterator as $cell) { $rowData[] = $cell->getValue(); } $id = (int)$rowData[0]; // Depending on your app and code, manual casting needed. $date = new DateTimeImmutable($rowData[1]); // Manual conversion. // Process... } ``` **league/csv**: ```php use DateTimeImmutable; use League\Csv\Reader; $reader = Reader::createFromPath('data.csv'); $reader->setHeaderOffset(0); foreach ($reader->getRecords() as $record) { $id = (int)$record['id']; // Depending on your app and code, manual casting needed. $date = new DateTimeImmutable($record['date']); // Manual conversion. // Process... } ``` ### Format Conversion Example **Derafu Spreadsheet**: ```php use Derafu\Spreadsheet\SpreadsheetLoader; use Derafu\Spreadsheet\SpreadsheetDumper; $loader = new Loader(); $dumper = new Dumper(); // Load XLSX and save as CSV. $spreadsheet = $loader->loadFromFile('data.xlsx'); $dumper->dumpToFile($spreadsheet, 'data.csv'); // Load CSV and save as JSON. $spreadsheet = $loader->loadFromFile('data.csv'); $dumper->dumpToFile($spreadsheet, 'data.json'); ``` **With alternative libraries**: ```php use League\Csv\Reader as CsvReader; use PhpOffice\PhpSpreadsheet\IOFactory as PhpSpreadsheetIOFactory; use PhpOffice\PhpSpreadsheet\Writer\Csv as PhpSpreadsheetCsvWriter; // Load with PhpSpreadsheet, save as CSV. $spreadsheet = PhpSpreadsheetIOFactory::load('data.xlsx'); $writer = new PhpSpreadsheetCsvWriter($spreadsheet); $writer->save('data.csv'); // To convert CSV to JSON, would require: $csv = CsvReader::createFromPath('data.csv'); $csv->setHeaderOffset(0); $records = iterator_to_array($csv->getRecords()); file_put_contents('data.json', json_encode($records, JSON_PRETTY_PRINT)); ``` ## Feature Comparison Table | Feature | Derafu Spreadsheet | PhpSpreadsheet | league/csv | |----------------------------|-------------------------------------------------|--------------------------------|------------| | **Formats Supported** | XLSX, XLS, CSV, ODS, JSON, XML, YAML, HTML, PDF | XLSX, XLS, CSV, ODS, HTML, PDF | CSV only | | **API Consistency** | Excellent | Complex | Simple | | **Memory Efficiency** | Can be better with streaming support | Poor for large files | Excellent | | **Streaming Support** | Soon | Yes | Excellent | | **Customizable Casting** | ✅ | ❌ | ❌ | | **PSR-7 Integration** | ✅ | ❌ | ❌ | | **JSON/YAML Support** | ✅ | ❌ | ❌ | | **Formula Support** | ❌ | ✅ | ❌ | | **Cell Styling** | ❌ | ✅ | ❌ | ## Conclusion Derafu Spreadsheet is focused on simplicity and ease of use for handling the data inside spreadsheets, not the styling. It excels when you need: 1. **Unified handling** of multiple file formats. 2. **Clean, intuitive API** for spreadsheet operations. 3. **Automatic type handling** to reduce boilerplate code. 4. **Format conversion** capabilities. 5. **Modern PHP architecture** with a focus on developer experience. Other libraries may be better suited for specific use cases: - **PhpSpreadsheet**: For complex Excel features and formatting. - **league/csv**: For CSV-specific operations and streaming large files. Choose the tool that best matches your specific requirements and constraints. --- ## Data Processor Project Derafu Data Processor # Derafu Data Processor --- ### Introduction Four-Phase Data Processing Library # Four-Phase Data Processing Library ![GitHub last commit](https://img.shields.io/github/last-commit/derafu/data-processor/main) ![CI Workflow](https://github.com/derafu/data-processor/actions/workflows/ci.yml/badge.svg?branch=main&event=push) ![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/derafu/data-processor) ![GitHub Issues](https://img.shields.io/github/issues-raw/derafu/data-processor) ![Total Downloads](https://poser.pugx.org/derafu/data-processor/downloads) ![Monthly Downloads](https://poser.pugx.org/derafu/data-processor/d/monthly) A PHP library designed to process data through four distinct phases: casting, transformation, sanitization and validation. ## What Makes it Different? Unlike traditional validation libraries that focus solely on validation, Derafu Data Processor offers: - **Clear Phase Separation**: Each processing phase (cast → transform → sanitize → validate) has its own purpose and runs in a specific order. - **Independent Transformations**: A dedicated transformation layer for complex data conversions, separate from basic type casting. - **Type-Safe Operations**: Strong typing and clear error handling in each phase. - **Extensible Rule System**: Each type of rule (caster, transformer, sanitizer, validator) follows its own interface. - **Minimal Dependencies**: `intl` extension for specific format rules, `Carbon` for dates, and `derafu/translation` for translatable error messages (see [Translations](translations)). ## Features - 🎯 **Type-Safe Casting**: Convert between data types safely. - Basic types (string, int, float, bool). - Date and time handling. - 🔄 **Data Transformations**: Complex data conversions. - Base64 encoding/decoding. - JSON processing. - Slug generation. - Character transliteration. - 🧹 **Rich Sanitization Rules**: Clean and normalize input data. - String sanitization (trim, strip_tags, substring, etc.). - Regex-based cleaning (regex_remove, regex_keep). - HTML entity handling (htmlspecialchars, htmlentities, etc.). - Character set manipulation (remove_non_printable, remove_chars, etc.). - ✅ **Comprehensive Validation**: Various validation rules. - String validations (required, min_length, max_length, etc.). - Number validations (range, gte, lte, etc.). - Date validations (after, before, between, etc.). - Format validations (email, URL, UUID, etc.). - File validations (file, image, mime_types). - Array validations (min_items, max_items, not_empty, etc.). ## Installation ```bash composer require derafu/data-processor ``` ## Basic Usage ```php use Derafu\DataProcessor\ProcessorFactory; $processor = ProcessorFactory::create(); // Simple validation. $result = $processor->process('test@example.com', [ 'validate' => ['email'], ]); // Multi-phase processing. $result = $processor->process(' TEST@EXAMPLE.COM ', [ 'transform' => ['lowercase'], 'sanitize' => ['trim'], 'validate' => ['email', 'max_length:255'], ]); // Array validation. $result = $processor->process([1, 2, 2, 3], [ 'validate' => ['unique', 'max_items:5'], ]); // File validation. $result = $processor->process($_FILES['upload'], [ 'validate' => ['file:2M', 'mimetype:application/pdf,image/jpeg'] ]); ``` --- ### Rules Reference Rules Reference # Rules Reference ## Cast Rules | Rule | Description | Example | |------|-------------|---------| | `boolean`, `bool` | Convert value to boolean | `'cast' => 'boolean'` | | `date` | Convert value to Carbon date | `'cast' => 'date'` | | `datetime` | Convert value to Carbon datetime | `'cast' => 'datetime'` | | `float`, `decimal` | Convert value to float | `'cast' => 'float'` | | `integer`, `int` | Convert value to integer | `'cast' => 'integer'` | | `string`, `str` | Convert value to string | `'cast' => 'string'` | | `timestamp` | Convert value to Unix timestamp | `'cast' => 'timestamp'` | ## Transform Rules | Rule | Description | Example | |------|-------------|---------| | `base64_decode` | Decode base64 string | `'transform' => ['base64_decode']` | | `base64_encode` | Encode value to base64 | `'transform' => ['base64_encode']` | | `json_decode` | Decode JSON string | `'transform' => ['json_decode']`, `'transform' => ['json_decode:array']` | | `json_encode` | Encode value to JSON | `'transform' => ['json_encode']`, `'transform' => ['json_encode:pretty']` | | `json_to_array` | Convert JSON string to array | `'transform' => ['json_to_array']` | | `json_to_object` | Convert JSON string to object | `'transform' => ['json_to_object']` | | `round` | Round numeric value | `'transform' => ['round:2']` | | `lowercase`, `lower` | Convert string to lowercase | `'transform' => ['lowercase']` | | `slug` | Convert string to URL friendly format | `'transform' => ['slug']` | | `transliterate` | Convert characters to ASCII | `'transform' => ['transliterate']` | | `uppercase`, `upper` | Convert string to uppercase | `'transform' => ['uppercase']` | ## Sanitize Rules | Rule | Description | Example | |------|-------------|---------| | `addslashes` | Add slashes before quotes | `'sanitize' => ['addslashes']` | | `htmlentities` | Convert characters to HTML entities | `'sanitize' => ['htmlentities']` | | `htmlspecialchars` | Convert special characters to HTML entities | `'sanitize' => ['htmlspecialchars']` | | `regex_keep` | Keep only characters matching pattern | `'sanitize' => ['regex_keep:/[0-9]/']` | | `regex_remove` | Remove characters matching pattern | `'sanitize' => ['regex_remove:/[^a-z]/']` | | `remove_chars` | Remove specific characters | `'sanitize' => ['remove_chars:@#$']` | | `remove_non_printable` | Remove non-printable characters | `'sanitize' => ['remove_non_printable']` | | `remove_prefix` | Remove string prefix | `'sanitize' => ['remove_prefix:http']` | | `remove_suffix` | Remove string suffix | `'sanitize' => ['remove_suffix:.txt']` | | `spaces` | Normalize multiple spaces to single | `'sanitize' => ['spaces']` | | `strip_tags` | Remove HTML and PHP tags | `'sanitize' => ['strip_tags']`, `'sanitize' => ['strip_tags:<p><br>']` | | `substring` | Extract part of string | `'sanitize' => ['substring:5']`, `'sanitize' => ['substring:0,10']` | | `trim` | Remove whitespace from ends | `'sanitize' => ['trim']` | ## Validate Rules ### Array Validation | Rule | Description | Example | |------|-------------|---------| | `in`, `choices` | Check if value is in list | `'validate' => ['in:a,b,c']` | | `max_items` | Check array maximum length | `'validate' => ['max_items:5']` | | `min_items` | Check array minimum length | `'validate' => ['min_items:1']` | | `notempty` | Check if array is not empty | `'validate' => ['notempty']` | | `unique` | Check if array values are unique | `'validate' => ['unique']` | ### Date Validation | Rule | Description | Example | |------|-------------|---------| | `after` | Check if date is after another | `'validate' => ['after:2024-01-01']` | | `after_or_equal` | Check if date is after or equal | `'validate' => ['after_or_equal:2024-01-01']` | | `before` | Check if date is before another | `'validate' => ['before:2024-12-31']` | | `before_or_equal` | Check if date is before or equal | `'validate' => ['before_or_equal:2024-12-31']` | | `between` | Check if date is between two dates | `'validate' => ['between:2024-01-01,2024-12-31']` | | `date_equals` | Check if date equals another | `'validate' => ['date_equals:2024-01-01']` | | `date_format` | Check if date matches format | `'validate' => ['date_format:Y-m-d']` | | `weekday` | Check if date is weekday | `'validate' => ['weekday']` | | `weekend` | Check if date is weekend | `'validate' => ['weekend']` | ### File Validation | Rule | Description | Example | |------|-------------|---------| | `file` | Validate file with size limit | `'validate' => ['file:2M']` | | `image` | Validate image file with size limit | `'validate' => ['image:2M']` | | `mimetype` | Validate file mime type | `'validate' => ['mimetype:application/pdf,image/jpeg']` | ### Financial Validation | Rule | Description | Example | |------|-------------|---------| | `bic` | Validate BIC/SWIFT code | `'validate' => ['bic']` | | `card_number` | Validate credit card number | `'validate' => ['card_number']` | | `iban` | Validate IBAN | `'validate' => ['iban']` | ### I18n Validation | Rule | Description | Example | |------|-------------|---------| | `country` | Validate country code | `'validate' => ['country']` | | `currency` | Validate currency code | `'validate' => ['currency']` | | `language` | Validate language code | `'validate' => ['language']` | | `locale` | Validate locale code | `'validate' => ['locale']` | | `timezone` | Validate timezone identifier | `'validate' => ['timezone']` | ### ID Validation | Rule | Description | Example | |------|-------------|---------| | `uuid` | Validate UUID string | `'validate' => ['uuid']` | ### Internet Validation | Rule | Description | Example | |------|-------------|---------| | `email` | Validate email address | `'validate' => ['email']` | | `hostname` | Validate hostname | `'validate' => ['hostname']` | | `ip` | Validate IP address | `'validate' => ['ip']`, `'validate' => ['ip:v4']`, `'validate' => ['ip:v6']` | | `url` | Validate URL | `'validate' => ['url']` | ### Numeric Validation | Rule | Description | Example | |------|-------------|---------| | `decimal`, `float`, `real` | Validate decimal number | `'validate' => ['decimal:2']` | | `digits_range` | Validate number of digits range | `'validate' => ['digits_range:3,5']` | | `digits` | Validate exact number of digits | `'validate' => ['digits:4']` | | `gt` | Greater than | `'validate' => ['gt:10']` | | `gte`, `min` | Greater than or equal | `'validate' => ['gte:10']` | | `integer`, `int` | Validate integer | `'validate' => ['integer']` | | `lt` | Less than | `'validate' => ['lt:10']` | | `lte`, `max` | Less than or equal | `'validate' => ['lte:10']` | | `multiple_of` | Check if number is multiple | `'validate' => ['multiple_of:5']` | | `numeric` | Validate numeric value | `'validate' => ['numeric']` | | `range` | Validate number in range | `'validate' => ['range:1,100']` | ### String Validation | Rule | Description | Example | |------|-------------|---------| | `alpha` | Only alphabetic characters | `'validate' => ['alpha']` | | `alpha_dash` | Alphanumeric with dashes/underscores | `'validate' => ['alpha_dash']` | | `alpha_num` | Only alphanumeric characters | `'validate' => ['alpha_num']` | | `base64` | Validate base64 string | `'validate' => ['base64']` | | `ends_with` | String ends with value | `'validate' => ['ends_with:Test']` | | `json` | Validate JSON string | `'validate' => ['json']` | | `length` | Exact string length | `'validate' => ['length:10']` | | `max_length` | Maximum string length | `'validate' => ['max_length:255']` | | `min_length` | Minimum string length | `'validate' => ['min_length:3']` | | `not_regex` | Not match regular expression | `'validate' => ['not_regex:/[0-9]/']` | | `no_whitespace` | No whitespace in string | `'validate' => ['no_whitespace']` | | `regex` | Match regular expression | `'validate' => ['regex:/^[A-Z]+$/']` | | `required` | Value is required | `'validate' => ['required']` | | `slug` | Validate URL friendly string | `'validate' => ['slug']` | | `starts_with` | String starts with value | `'validate' => ['starts_with:Test']` | --- ### Custom Registries Creating Custom Registries # Creating Custom Registries You can create your own rule registry to customize which rules are available: ```php use Derafu\DataProcessor\RuleRegistry; use Derafu\DataProcessor\Processor; use Derafu\DataProcessor\RuleResolver; // Create a custom registry. $registry = new RuleRegistry(); // Register only the rules you need. $registry ->addTransformerRule('base64_encode', Base64EncodeRule::class) ->addSanitizerRule('trim', TrimRule::class) ->addCasterRule('integer', IntegerRule::class) ->addValidatorRule('email', EmailRule::class); // Create resolver, parser and processor with your registry. $resolver = new RuleResolver($registry); $parser = new RuleParser(); $processor = new Processor($resolver, $parser); ``` --- ### Custom Rules Creating Custom Rules # Creating Custom Rules ## Custom Cast Rule ```php use Derafu\DataProcessor\Contract\CasterRuleInterface; final class CustomCastRule implements CasterRuleInterface { public function cast(mixed $value, array $parameters = []): mixed { // Your casting logic here. } } // Register the rule. $registry->addCasterRule('custom_cast', CustomCastRule::class); ``` ## Custom Transform Rule ```php use Derafu\DataProcessor\Contract\TransformerRuleInterface; final class CustomTransformRule implements TransformerRuleInterface { public function transform(mixed $value, array $parameters = []): mixed { // Your transformation logic here. } } // Register the rule. $registry->addTransformerRule('custom_transform', CustomTransformRule::class); ``` ## Custom Sanitizer Rule ```php use Derafu\DataProcessor\Contract\SanitizerRuleInterface; final class CustomSanitizerRule implements SanitizerRuleInterface { public function sanitize(mixed $value, array $parameters = []): mixed { // Your sanitization logic here. } } // Register the rule. $registry->addSanitizerRule('custom_sanitize', CustomSanitizerRule::class); ``` ## Custom Validator Rule ```php use Derafu\DataProcessor\Contract\ValidatorRuleInterface; final class CustomValidatorRule implements ValidatorRuleInterface { public function validate(mixed $value, array $parameters = []): void { // Your validation logic here. } } // Register the rule. $registry->addValidatorRule('custom_validate', CustomValidatorRule::class); ``` ## Using Custom Rules Once registered, you can use your custom rules just like built-in ones: ```php $processor->process('field', $value, [ 'cast' => 'custom_cast', 'transform' => ['custom_transform'], 'sanitize' => ['custom_sanitize'], 'validate' => ['custom_validate'], ]); ``` --- ### Custom Registrar Creating Custom Rule Registrar # Creating Custom Rule Registrar For better organization, you can create a custom rule registrar: ```php use Derafu\DataProcessor\Contract\RuleRegistrarInterface; use Derafu\DataProcessor\Contract\RuleRegistryInterface; final class CustomRuleRegistrar implements RuleRegistrarInterface { public function register(RuleRegistryInterface $registry): void { // Register caster rules. $registry ->addCasterRule('custom_type_1', CustomType1Rule::class) ->addCasterRule('custom_type_2', CustomType2Rule::class); // Register transform rules. $registry ->addTransformerRule('custom_transform_1', CustomTransform1Rule::class) ->addTransformerRule('custom_transform_2', CustomTransform2Rule::class); // Register sanitizer rules. $registry ->addSanitizerRule('custom_sanitize_1', CustomSanitize1Rule::class) ->addSanitizerRule('custom_sanitize_2', CustomSanitize2Rule::class); // Register validator rules. $registry ->addValidatorRule('custom_validate_1', CustomValidate1Rule::class) ->addValidatorRule('custom_validate_2', CustomValidate2Rule::class); } } // Usage. $processor = ProcessorFactory::create( new CustomRuleRegistrar(), withDefaultRules: false // Create without default rules using the factory. ); ``` This allows you to: - Create domain-specific rule sets. - Override default rules with custom implementations. - Group related rules together. - Control which rules are available in your application. --- ### Custom Parser Creating Custom Rule Parser # Creating Custom Rule Parser You can create your own rule parser to customize how rules are parsed from strings or arrays: ```php use Derafu\DataProcessor\Contract\RuleParserInterface; final class CustomRuleParser implements RuleParserInterface { public function parse(string|array $rules): array { if (is_array($rules)) { return $this->parseArray($rules); } return $this->parseString($rules); } private function parseArray(array $rules): array { // Your array parsing logic here. $parsed = []; foreach ($rules as $type => $typeRules) { // Handle different formats for rules. $parsed[$type] = $this->parseTypeRules($typeRules); } return $parsed; } private function parseString(string $rules): array { // Your string parsing logic here. // Example: parse rules like "t(rule1|rule2) s(rule3) required|email" return [ 'transform' => ['rule1', 'rule2'], 'sanitize' => ['rule3'], 'validate' => ['required', 'email'], ]; } private function parseTypeRules(string|array $rules): array|string { // Handle parsing of individual rule types. if (is_string($rules)) { return explode('|', $rules); } return $rules; } } // Usage with processor factory. $processor = ProcessorFactory::create( parser: new CustomRuleParser() ); // Or manual instantiation. $registry = new RuleRegistry(); // Then register the rules you need. $resolver = new RuleResolver($registry); $parser = new CustomRuleParser(); $processor = new Processor($resolver, $parser); // Using your custom parser. $result = $processor->process('test@example.com', 't(lowercase) s(trim) required|email'); // Or. $result = $processor->process('test@example.com', [ 'transform' => 'lowercase', 'sanitize' => 'trim', 'validate' => 'required|email', ]); ``` This allows you to: - Create custom rule parsing formats. - Support different string formats for rules. - Add custom shorthand notations. - Implement domain-specific rule syntax. - Handle complex rule configurations. --- ### Translations Translations # Translations `derafu/data-processor`'s exceptions can be translated. This page explains what's translatable, what ships out of the box, and how to activate it in your application. ## What's Translatable Four exception classes implement `Derafu\Translation\Contract\TranslatableInterface` (via `TranslatableExceptionTrait`, from [`derafu/translation`](https://www.derafu.dev/docs/core/translation)): - `Derafu\DataProcessor\Exception\ValidationException` - `Derafu\DataProcessor\Exception\SanitizationException` - `Derafu\DataProcessor\Exception\TransformationException` - `Derafu\DataProcessor\Exception\CastingException` `Derafu\DataProcessor\Exception\RuleNotFoundException` is **not** translatable on purpose — it signals a misconfigured rule name (a programming/configuration error), not something an end user should ever see. All four use the `errors` domain (`TranslatableExceptionTrait`'s default). Every message thrown by a built-in rule is written in ICU MessageFormat syntax and uses the literal English text as its translation id — for example, `'Value must be greater than {value}.'` — following the same convention documented in [derafu/translation's Exceptions guide](https://www.derafu.dev/docs/core/translation). Without any translator configured, `getMessage()` already returns the fully formatted English text — this library works exactly as before, whether or not you set up translation at all: ```php use Derafu\DataProcessor\Exception\ValidationException; $e = new ValidationException(['Array must not contain more than {max} items.', 'max' => 5]); echo $e->getMessage(); // "Array must not contain more than 5 items." ``` ## What Ships `derafu/data-processor` includes: - `resources/translations/errors+intl-icu.es.php` — a Spanish translation for every distinct message used by the four exception classes above (the `+intl-icu` domain suffix is required for `derafu/translation`'s ICU placeholders like `{max}` to be substituted — see [ICU Formatting](https://www.derafu.dev/docs/core/translation)). - `Derafu\DataProcessor\Translation\DataProcessorTranslationResourceProvider` — a `TranslationResourceProviderInterface` implementation pointing at that directory, ready to hand to a `TranslationResourceRegistrar` or a dependency-injection container. Neither of these does anything by itself — `derafu/data-processor` is a library, it doesn't build or own a `Translator`. Activating translation is entirely up to the application that uses it. ## Activating It (Plain PHP) ```php use Derafu\DataProcessor\Exception\ValidationException; use Derafu\DataProcessor\Translation\DataProcessorTranslationResourceProvider; use Derafu\Translation\TranslatorFactory; $translator = TranslatorFactory::create( defaultLocale: 'es', fallbackLocales: ['es', 'en'], resourceProviders: [new DataProcessorTranslationResourceProvider()], ); $e = new ValidationException(['Array must not contain more than {max} items.', 'max' => 5]); echo $e->trans($translator, 'es'); // "El arreglo no debe contener más de 5 elementos." ``` If you'd rather point at the directory directly (for example, to combine it with your own translation files) without going through the provider class, use `TranslationResourceRegistrar` instead: ```php use Derafu\Translation\TranslationResourceRegistrar; $registrar = new TranslationResourceRegistrar($translator); $registrar->registerDirectory(__DIR__ . '/vendor/derafu/data-processor/resources/translations'); $registrar->registerDirectory(__DIR__ . '/translations'); // Your own, registered last: it can override. ``` ## Activating It (Dependency Injection) If your application uses `symfony/dependency-injection`, import both recipe files and feed the tagged providers into your own `Translator` registration: ```yaml imports: - { resource: '../vendor/derafu/translation/resources/config/translation-services.yaml' } - { resource: '../vendor/derafu/data-processor/resources/config/data-processor-services.yaml' } services: Symfony\Component\Translation\Translator: factory: ['Derafu\Translation\TranslatorFactory', 'create'] arguments: $defaultLocale: 'es' $fallbackLocales: ['es', 'en'] $resourceProviders: !tagged_iterator derafu_translation.resource_provider ``` `data-processor-services.yaml` already tags `DataProcessorTranslationResourceProvider` with `derafu_translation.resource_provider`, so it's picked up automatically — nothing else to configure. Anything in your own application that type-hints `Symfony\Contracts\Translation\TranslatorInterface` (as `derafu/translation`'s own exceptions do) resolves to this `Translator` via the alias `translation-services.yaml` provides. ## A Note on Rule Names vs. Messages Rule names (`'email'`, `'max_length'`, the strings you pass to `process()`) are never translated — they're an internal API, not user-facing text. Only the **exception messages** thrown when a rule fails go through translation. --- ## Repository Project Lightweight Data Source Management for PHP # Lightweight Data Source Management for PHP ![GitHub last commit](https://img.shields.io/github/last-commit/derafu/repository/main) ![CI Workflow](https://github.com/derafu/repository/actions/workflows/ci.yml/badge.svg?branch=main&event=push) ![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/derafu/repository) ![GitHub Issues](https://img.shields.io/github/issues-raw/derafu/repository) ![Total Downloads](https://poser.pugx.org/derafu/repository/downloads) ![Monthly Downloads](https://poser.pugx.org/derafu/repository/d/monthly) A lightweight, flexible PHP library for managing data repositories with multiple data sources and seamless integration with PHP frameworks. ## Why Derafu\Repository? ### 🚀 **Simplified Data Management** Traditional PHP data repositories often: - Are tightly coupled to specific storage mechanisms. - Require complex configuration. - Lack flexibility across different data sources. ### 🔥 **What Makes Derafu\Repository Unique?** | Feature | Derafu\Repository | Traditional Repositories | |-----------------------------|-------------------|--------------------------| | **File-Based Data Sources** | ✅ Yes | ❌ No | | **Lightweight Design** | ✅ Yes | ❌ No | | **Simple Configuration** | ✅ Yes | ❌ No | | **Framework Agnostic** | ✅ Yes | ⚠️ Varies | --- ## Features - ✅ **Multiple Data Source Support** – Load data from PHP arrays, JSON, YAML files. - ✅ **Generic Entity Handling** – Work with any type of data structure. - ✅ **Flexible Querying** – Find, filter, and order data with ease. - ✅ **Doctrine Collections Compatible** – Use Doctrine Criteria for advanced filtering. - ✅ **Zero Heavy Dependencies** – Lightweight and performance-focused. - ✅ **PHP 8+ Ready** – Leverages modern PHP features. --- ## Installation Install via Composer: ```bash composer require derafu/repository ``` ## Basic Usage ```php use Derafu\Repository\Repository; // Load data from a PHP file or array. $data = [ 'products' => [ 'prod-001' => [ 'name' => 'Laptop XPS', 'category' => 'computers', 'price' => 1299.99, ], // More products... ], ]; // Create a repository. $repository = new Repository($data, idAttribute: 'id'); // Find all products. $allProducts = $repository->findAll(); // Find products by criteria. $computerProducts = $repository->findBy([ 'category' => 'computers' ]); // Find a single product. $laptop = $repository->findOneBy([ 'name' => 'Laptop XPS', ]); ``` ## Advanced Usage with Doctrine Criteria ```php use Doctrine\Common\Collections\Criteria; use Doctrine\Common\Collections\Order; // Advanced filtering with Doctrine Criteria. $expensiveProducts = $repository->findByCriteria( Criteria::create() ->where(Criteria::expr()->gt('price', 1000)) ->orderBy(['price' => Order::Descending]) ); ``` ## Supported Data Sources - PHP Arrays. - PHP Files returning arrays. - JSON Files. - YAML Files. ## Performance Considerations - Optimized for small to medium-sized datasets. - In-memory data management. - Recommended for configuration, lookup tables, and static data. --- ## Config Project Yet Another Config Lib # Yet Another Config Lib ![GitHub last commit](https://img.shields.io/github/last-commit/derafu/config/main) ![CI Workflow](https://github.com/derafu/config/actions/workflows/ci.yml/badge.svg?branch=main&event=push) ![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/derafu/config) ![GitHub Issues](https://img.shields.io/github/issues-raw/derafu/config) ![Total Downloads](https://poser.pugx.org/derafu/config/downloads) ![Monthly Downloads](https://poser.pugx.org/derafu/config/d/monthly) ## Installation Install via Composer: ```bash composer require derafu/config ``` ## Basic Usage ```php use Derafu\Config\Configuration; // Or `Options`. $configuration = new Configuration([ 'app' => [ 'name' => 'Derafu Config', ], ]); echo $configuration['app']['name']; // "Derafu Config" echo $configuration['app.name']; // "Derafu Config" echo $configuration->get('app.name'); // "Derafu Config" echo $configuration->get('app')->get('name'); // "Derafu Config" ``` --- ## Container Project Flexible Data Containers for PHP # Flexible Data Containers for PHP ![GitHub last commit](https://img.shields.io/github/last-commit/derafu/container/main) ![CI Workflow](https://github.com/derafu/container/actions/workflows/ci.yml/badge.svg?branch=main&event=push) ![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/derafu/container) ![GitHub Issues](https://img.shields.io/github/issues-raw/derafu/container) ![Total Downloads](https://poser.pugx.org/derafu/container/downloads) ![Monthly Downloads](https://poser.pugx.org/derafu/container/d/monthly) A collection of specialized PHP data containers that provide different levels of structure and validation, from simple bags to schema-validated stores. ## Features - 🎯 Purpose-built containers for different needs. - 🔍 JSON Schema validation support. - 📦 Simple to complex data structures. - 🔄 Sequential data handling. - 🛡️ Type-safe operations. - 🧩 Common interface across containers. - 🪶 Minimal dependencies. - 🧪 Comprehensive test coverage. ## Why Derafu\Container? Unlike generic data structures, Derafu\Container provides specialized containers each designed for specific use cases: - **Bag**: Simple, flexible data storage without restrictions. - **Vault**: Basic structured data with validation. - **Store**: Advanced data validation using JSON Schema. - **Journal**: Sequential data storage with LIFO access. ## Installation Install via Composer: ```bash composer require derafu/container ``` ## Basic Usage ### Bag - Flexible Container ```php use Derafu\Container\Bag; $bag = new Bag(); $bag->set('user.name', 'John'); $bag->set('user.email', 'john@example.com'); echo $bag->get('user.name'); // "John" ``` ### Vault - Structured Container ```php use Derafu\Container\Vault; $vault = new Vault([], [ 'user' => [ 'type' => 'array', 'required' => true, 'schema' => [ 'name' => ['type' => 'string'], 'age' => ['type' => 'integer'], ], ], ]); $vault->set('user', [ 'name' => 'John', 'age' => 30, ]); ``` ### Store - JSON Schema Container ```php use Derafu\Container\Store; $schema = [ 'type' => 'object', 'properties' => [ 'user' => [ 'type' => 'object', 'required' => ['name', 'email'], 'properties' => [ 'name' => ['type' => 'string'], 'email' => ['type' => 'string', 'format' => 'email'], ], ], ], ]; $store = new Store([], $schema); $store->set('user', [ 'name' => 'John', 'email' => 'john@example.com', ]); ``` ### Journal - Sequential Container ```php use Derafu\Container\Journal; $journal = new Journal(); $journal->add('First Entry'); $journal->add('Second Entry'); print_r($journal->reverse()); // Shows entries newest first. ``` ## Container Comparison | Feature | Bag | Vault | Store | Journal | |------------------|-----|--------|--------|----------| | Schema | No | Simple | JSON | No | | Validation | No | Basic | Full | No | | Nested Data | Yes | Yes | Yes | No | | Sequential | No | No | No | Yes | | Dot Notation | Yes | Yes | Yes | No | ## Common Interface All containers implement `ContainerInterface`: ```php interface ContainerInterface extends ArrayAccess { public function set(string $key, mixed $value): static; public function get(string $key, mixed $default = null): mixed; public function has(string $key): bool; public function clear(?string $key = null): void; } ``` ## Advanced Usage ### Custom Validation in Vault ```php use Derafu\Container\Vault; $vault = new Vault([], [ 'age' => [ 'type' => 'integer', 'validator' => fn ($value) => $value >= 18 ] ]); ``` ### Complex JSON Schema in Store ```php use Derafu\Container\Store; $store = new Store([], [ 'type' => 'object', 'properties' => [ 'users' => [ 'type' => 'array', 'items' => [ 'type' => 'object', 'required' => ['id', 'name'], 'properties' => [ 'id' => ['type' => 'integer'], 'name' => ['type' => 'string'], 'email' => ['type' => 'string', 'format' => 'email'], ], ], ], ], ]); ``` ### Journal with Filtering ```php use Derafu\Container\Journal; $journal = new Journal(); $journal->add(['level' => 'info', 'message' => 'System started']); $journal->add(['level' => 'error', 'message' => 'Connection failed']); $errors = array_filter($journal->reverse(), fn($entry) => $entry['level'] === 'error' ); ``` --- ## Selector Project Derafu Selector # Derafu Selector --- ### Introduction Elegant Data Structure Navigation for PHP # Elegant Data Structure Navigation for PHP ![GitHub last commit](https://img.shields.io/github/last-commit/derafu/selector/main) ![CI Workflow](https://github.com/derafu/selector/actions/workflows/ci.yml/badge.svg?branch=main&event=push) ![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/derafu/selector) ![GitHub Issues](https://img.shields.io/github/issues-raw/derafu/selector) ![Total Downloads](https://poser.pugx.org/derafu/selector/downloads) ![Monthly Downloads](https://poser.pugx.org/derafu/selector/d/monthly) A powerful, flexible, and efficient PHP library for querying, extracting, and modifying complex data structures using a **declarative selector syntax**. ## Why Derafu\Selector? ### 🚀 **A Smarter Way to Work with Data Structures** Traditional PHP methods for accessing and modifying nested data structures (`$array['key']['subkey']`) are: - **Error-prone**: Risk of `Undefined index` or `Trying to access array offset on null` errors. - **Verbose & Hard to Read**: Requires multiple `isset()` checks and deep nesting. - **Limited**: No built-in support for **filters, conditional selectors, or transformations**. ### 🔥 **What Makes Derafu\Selector Unique?** | Feature | Derafu\Selector | PHP Native (`isset()` & loops) | Other Libraries | |--------------------------------------|-----------------|--------------------------------|-----------------| | **Dot-Notation Access** | ✅ Yes | ❌ No | ⚠️ Limited | | **Nested Selection with Conditions** | ✅ Yes | ❌ No | ⚠️ Limited | | **Dynamic Path Resolution** | ✅ Yes | ❌ No | ⚠️ Varies | | **Auto-Handles Undefined Keys** | ✅ Yes | ❌ No | ⚠️ Partial | | **Filters & Expressions** | ✅ Yes | ❌ No | ⚠️ Partial | --- ## 🔍 Overview Derafu Selector gives you a clean, robust way to work with nested arrays and objects in PHP. It eliminates the need for repetitive null checks, nested loops, and error-prone array access, replacing them with a powerful, declarative syntax. ```php // Instead of this: $userName = isset($data['user']) && isset($data['user']['profile']) && isset($data['user']['profile']['name']) ? $data['user']['profile']['name'] : null; // Simply write this: $userName = Selector::get($data, 'user.profile.name'); // Or even filter arrays with conditions: $adminEmail = Selector::get($data, 'users[role=admin:email]'); ``` ## ✨ Features {.list-unstyled} - ✅ **Powerful Dot Notation** - Access deeply nested data with simple paths (`user.profile.name`). - ✅ **Array Access** - Use numeric indices to access array elements (`items[0].name`). - ✅ **Filtering** - Find array elements by key/value conditions (`users[id=123:email]`). - ✅ **Conditional Logic** - Use ternary-like expressions (`((type) = "admin" ? (admin_role) : (user_role))`). - ✅ **Multiple Query Languages** - Support for native syntax, JSONPath, and JMESPath. - ✅ **Error Handling** - No more "undefined index" or "trying to access array offset on null" errors. - ✅ **Write Support** - Modify data structures with the same powerful syntax. - ✅ **Zero Dependencies** - Lightweight core with optional support for JSONPath and JMESPath. --- ## 📦 Installation ```bash composer require derafu/selector ``` ## 🚀 Quick Start ### Reading Data ```php use Derafu\Selector\Selector; $data = [ 'user' => [ 'name' => 'John Doe', 'email' => 'john@example.com', 'profile' => [ 'avatar' => 'john.jpg', 'settings' => ['theme' => 'dark', 'notifications' => true], ], 'roles' => ['editor', 'contributor'], ], 'products' => [ ['id' => 101, 'name' => 'Laptop', 'price' => 999], ['id' => 102, 'name' => 'Phone', 'price' => 699], ['id' => 103, 'name' => 'Headphones', 'price' => 149], ], ]; // Simple property access. $name = Selector::get($data, 'user.name'); // "John Doe" // Nested properties. $theme = Selector::get($data, 'user.profile.settings.theme'); // "dark" // Array elements by index. $firstRole = Selector::get($data, 'user.roles[0]'); // "editor" // Array filtering. $laptop = Selector::get($data, 'products[id=101:name]'); // "Laptop" // With default values. $address = Selector::get($data, 'user.address', 'Not specified'); // "Not specified" // Check if a path exists. $hasAvatar = Selector::has($data, 'user.profile.avatar'); // true ``` ### Writing Data ```php // Set a simple value. Selector::set($data, 'user.address', '123 Main St'); // Create nested structures automatically. Selector::set($data, 'user.profile.settings.language', 'en'); // Update array elements. Selector::set($data, 'products[id=101:price]', 899); // Remove a value. Selector::clear($data, 'user.profile.settings.notifications'); ``` ### Advanced Selector Syntax ```php // Conditional selectors. $role = Selector::get($data, '((user.type) = "admin" ? (admin_role) : (user_role))'); // String concatenation. $greeting = Selector::get($data, '"Hello, "(user.name)"!"'); // "Hello, John Doe!" // OR operator for fallbacks. $contactInfo = Selector::get($data, 'user.phone||user.email'); // Uses email if phone is null // JSONPath integration. $expensiveProducts = Selector::get($data, '$.products[?(@.price > 500)].name'); // ["Laptop", "Phone"] // JMESPath integration. $productNames = Selector::get($data, 'jmespath:products[*].name'); // ["Laptop", "Phone", "Headphones"] ``` --- ### Basic Usage Basic Usage # Basic Usage This guide covers the fundamentals of working with Derafu Selector for navigating and manipulating data structures. ## Reading Data The most common operation with Selector is to read values from complex data structures. The `get()` method provides a clean, safe way to extract values without worry about undefined indices or null values. ### Basic Dot Notation ```php use Derafu\Selector\Selector; $data = [ 'user' => [ 'name' => 'John Doe', 'email' => 'john@example.com', 'profile' => [ 'avatar' => 'john.jpg', ], ], ]; // Simple property access. $name = Selector::get($data, 'user.name'); // "John Doe" // Nested properties. $avatar = Selector::get($data, 'user.profile.avatar'); // "john.jpg" // Non-existent path returns null. $age = Selector::get($data, 'user.age'); // null ``` ### Default Values You can provide a default value that will be returned if the selector path doesn't exist: ```php // With default value $address = Selector::get($data, 'user.address', 'No address provided'); // "No address provided" ``` ### Array Access Access array elements by index: ```php $data = [ 'items' => ['apple', 'banana', 'cherry'], ]; $firstItem = Selector::get($data, 'items[0]'); // "apple" $lastItem = Selector::get($data, 'items[2]'); // "cherry" ``` ### Combining Dot Notation and Array Access ```php $data = [ 'categories' => [ 'fruits' => ['apple', 'banana', 'cherry'], 'vegetables' => ['carrot', 'broccoli', 'spinach'], ], ]; $firstFruit = Selector::get($data, 'categories.fruits[0]'); // "apple" $secondVegetable = Selector::get($data, 'categories.vegetables[1]'); // "broccoli" ``` ## Writing Data Writing data is just as easy as reading it. The `set()` method handles creating intermediate structures as needed. ### Setting Simple Values ```php $data = ['user' => ['name' => 'John']]; // Update an existing value. Selector::set($data, 'user.name', 'Jane'); // $data is now ['user' => ['name' => 'Jane']] // Create a new property. Selector::set($data, 'user.email', 'jane@example.com'); // $data is now ['user' => ['name' => 'Jane', 'email' => 'jane@example.com']] ``` ### Creating Nested Structures Selector will automatically create intermediate arrays: ```php $data = []; // Create a deep structure. Selector::set($data, 'settings.theme.colors.primary', '#3366FF'); // $data is now ['settings' => ['theme' => ['colors' => ['primary' => '#3366FF']]]] ``` ### Working with Arrays ```php $data = ['items' => ['apple', 'banana']]; // Add or update an array element. Selector::set($data, 'items[2]', 'cherry'); // $data is now ['items' => ['apple', 'banana', 'cherry']] // Update a specific element. Selector::set($data, 'items[0]', 'red apple'); // $data is now ['items' => ['red apple', 'banana', 'cherry']] ``` ### Merging Arrays When setting an array value to an existing array path, the arrays are merged: ```php $data = [ 'settings' => [ 'display' => ['theme' => 'light'], ], ]; Selector::set($data, 'settings.display', ['fontSize' => 'large']); // $data is now ['settings' => ['display' => ['theme' => 'light', 'fontSize' => 'large']]] ``` ## Checking if Paths Exist The `has()` method checks if a value exists at the specified path: ```php $data = [ 'user' => [ 'name' => 'John', 'email' => null, ], ]; Selector::has($data, 'user.name'); // true Selector::has($data, 'user.email'); // false (because the value is null) Selector::has($data, 'user.address'); // false (path doesn't exist) ``` ## Clearing Values Remove values from the data structure with the `clear()` method: ```php $data = [ 'user' => [ 'name' => 'John', 'email' => 'john@example.com', 'settings' => ['theme' => 'dark'], ], ]; // Remove a simple property. Selector::clear($data, 'user.email'); // $data is now ['user' => ['name' => 'John', 'settings' => ['theme' => 'dark']]] // Remove a nested property. Selector::clear($data, 'user.settings.theme'); // $data is now ['user' => ['name' => 'John', 'settings' => []]] // Parent arrays are cleaned up if they become empty. Selector::clear($data, 'user.name'); // $data is now ['user' => ['settings' => []]] ``` ## Working with Different Data Types Selector handles various data types properly: ```php $data = [ 'values' => [ 'string' => 'text', 'number' => 42, 'boolean' => true, 'null_value' => null, 'array' => [1, 2, 3], 'object' => ['key' => 'value'], ], ]; Selector::get($data, 'values.string'); // "text" Selector::get($data, 'values.number'); // 42 Selector::get($data, 'values.boolean'); // true Selector::get($data, 'values.null_value'); // null Selector::get($data, 'values.array'); // [1, 2, 3] Selector::get($data, 'values.object'); // ['key' => 'value'] ``` ## Error Handling Selector is designed to be forgiving and avoid common PHP errors: ```php // No "undefined index" errors. Selector::get($data, 'non.existent.path'); // returns null // No "trying to access array offset on null" errors. $data = ['user' => null]; Selector::get($data, 'user.name'); // returns null // However, if you have syntax errors in your selector, you'll get a SelectorException. try { Selector::get($data, 'user..name'); // Invalid selector with double dot } catch (\Derafu\Selector\Exception\SelectorException $e) { echo $e->getMessage(); // "Failed to read using selector 'user..name'..." } ``` --- For more advanced usage, check out the other guides in the documentation. --- ### Working with Arrays and Filters Working with Arrays and Filters # Working with Arrays and Filters This guide focuses on techniques for working with arrays and using filters in Derafu Selector. ## Basic Array Access Arrays can be accessed using both dot notation and square bracket syntax: ```php use Derafu\Selector\Selector; $data = [ 'items' => ['apple', 'banana', 'cherry'], 'counts' => [5, 10, 15], ]; // Access entire array. $allItems = Selector::get($data, 'items'); // ['apple', 'banana', 'cherry'] // First element. $firstItem = Selector::get($data, 'items[0]'); // 'apple' // Last element. $lastItem = Selector::get($data, 'items[2]'); // 'cherry' // Non-existent index. $nonExistent = Selector::get($data, 'items[5]'); // null ``` ## Numeric Indexing ```php $data = [ 'matrix' => [ [1, 2, 3], [4, 5, 6], [7, 8, 9], ], ]; // Accessing multi-dimensional arrays. $center = Selector::get($data, 'matrix[1][1]'); // 5 // Alternative dot notation. $center = Selector::get($data, 'matrix.1.1'); // 5 // Mixed notation. $firstRow = Selector::get($data, 'matrix[0]'); // [1, 2, 3] $firstRowThirdElement = Selector::get($data, 'matrix[0][2]'); // 3 ``` ## Filtering Arrays by Condition The most powerful feature for array handling is filtering by key/value conditions: ```php $data = [ 'users' => [ ['id' => 1, 'name' => 'John', 'role' => 'admin'], ['id' => 2, 'name' => 'Jane', 'role' => 'user'], ['id' => 3, 'name' => 'Bob', 'role' => 'user'], ] ]; // Find user by ID. $userName = Selector::get($data, 'users[id=2:name]'); // 'Jane' // Find user by role. $adminName = Selector::get($data, 'users[role=admin:name]'); // 'John' // Non-matching filter. $nonExistent = Selector::get($data, 'users[id=99:name]'); // null ``` ### Syntax for Dependent Selectors The syntax for array filtering with dependent selectors is: ``` arrayKey[requiredKey=requiredValue:dependentKey] ``` Where: - `arrayKey` is the key containing the array to search in. - `requiredKey` is the key to match in each array element. - `requiredValue` is the value that `requiredKey` should equal. - `dependentKey` is the key to extract from the matching element. ## Complex Filtering You can combine filters with further navigation: ```php $data = [ 'orders' => [ [ 'id' => 1001, 'customer' => 'John', 'items' => [ ['product' => 'Laptop', 'price' => 999], ['product' => 'Mouse', 'price' => 25], ], ], [ 'id' => 1002, 'customer' => 'Jane', 'items' => [ ['product' => 'Phone', 'price' => 699], ['product' => 'Headphones', 'price' => 149], ], ], ], ]; // Get the first item from order 1002. $product = Selector::get($data, 'orders[id=1002:items][0].product'); // 'Phone' // Get the price of the second item from order 1001. $price = Selector::get($data, 'orders[id=1001:items][1].price'); // 25 // Find an item by product name within an order. $data = [ 'orders' => [ [ 'id' => 1001, 'items' => [ ['product' => 'Laptop', 'price' => 999], ['product' => 'Mouse', 'price' => 25], ], ], ], ]; $mousePrice = Selector::get($data, 'orders[id=1001:items][product=Mouse:price]'); // 25 ``` ## Nested Arrays For deeply nested structures: ```php $data = [ 'departments' => [ [ 'name' => 'Engineering', 'teams' => [ [ 'name' => 'Frontend', 'members' => [ ['id' => 101, 'name' => 'Alice'], ['id' => 102, 'name' => 'Bob'], ] ], [ 'name' => 'Backend', 'members' => [ ['id' => 201, 'name' => 'Charlie'], ['id' => 202, 'name' => 'Diana'], ], ], ], ], ], ]; // Find Charlie's ID through the department and team. $charlieId = Selector::get($data, 'departments[name=Engineering:teams][name=Backend:members][name=Charlie:id]' ); // 201 ``` ## Array Operations ### Writing to Arrays ```php $data = [ 'users' => [ ['id' => 1, 'name' => 'John'], ['id' => 2, 'name' => 'Jane'], ], ]; // Update a value. Selector::set($data, 'users[id=1:name]', 'Johnny'); // $data['users'][0]['name'] is now 'Johnny' // Add a new property. Selector::set($data, 'users[id=2:email]', 'jane@example.com'); // $data['users'][1]['email'] is now 'jane@example.com' // Add a new element. Selector::set($data, 'users[3]', ['id' => 3, 'name' => 'Bob']); // $data['users'][3] is now ['id' => 3, 'name' => 'Bob'] // If no matching element exists, a new one is created. Selector::set($data, 'users[id=4:name]', 'Alice'); // Adds a new element to $data['users'] with id=4 and name='Alice' ``` ### Clearing Array Elements ```php $data = [ 'users' => [ ['id' => 1, 'name' => 'John', 'email' => 'john@example.com'], ['id' => 2, 'name' => 'Jane', 'email' => 'jane@example.com'], ], ]; // Remove a property. Selector::clear($data, 'users[id=1:email]'); // John's email is now removed. // Remove an entire array element. Selector::clear($data, 'users[id=2]'); // Jane's record is now completely removed. // Clean up empty array elements. $data = [ 'teams' => [ ['id' => 1, 'members' => ['Alice', 'Bob']], ['id' => 2, 'members' => ['Charlie']], ], ]; // Remove the last member from team 2. Selector::clear($data, 'teams[id=2:members][0]'); // Now teams[id=2:members] is an empty array. // Derafu Selector automatically cleans up empty arrays: // $data['teams'][1]['members'] is now an empty array. // If we clear team 1's members too, it would clean up further. Selector::clear($data, 'teams[id=1:members]'); // Now both teams have empty members arrays. ``` ### Working with Mixed Array Types Selector handles both associative and indexed arrays seamlessly: ```php $data = [ 'products' => [ 'electronics' => [ ['id' => 'e1', 'name' => 'Laptop', 'price' => 999], ['id' => 'e2', 'name' => 'Phone', 'price' => 699], ], 'books' => [ ['id' => 'b1', 'name' => 'Novel', 'price' => 15], ['id' => 'b2', 'name' => 'Textbook', 'price' => 50], ], ], ]; // Access by category and index. $firstElectronic = Selector::get($data, 'products.electronics[0].name'); // 'Laptop' // Access by category and ID. $textbookPrice = Selector::get($data, 'products.books[id=b2:price]'); // 50 // Numeric indices still work with filtering. $secondBookName = Selector::get($data, 'products.books[1].name'); // 'Textbook' ``` ### Handling Empty or Non-existent Arrays Selector provides safe handling for empty or non-existent arrays: ```php $data = [ 'categories' => [ 'active' => [ ['id' => 1, 'name' => 'Electronics'], ['id' => 2, 'name' => 'Books'], ], 'inactive' => [], ], ]; // Access on empty array. $inactiveCategory = Selector::get($data, 'categories.inactive[0]'); // null // Filter on empty array. $result = Selector::get($data, 'categories.inactive[id=1:name]'); // null // Non-existent array path. $result = Selector::get($data, 'categories.archived[0].name'); // null // Providing defaults. $result = Selector::get($data, 'categories.archived[0].name', 'Not found'); // 'Not found' ``` ### Modifying Array Elements in Place You can modify array elements directly: ```php $data = [ 'cart' => [ ['id' => 'p1', 'name' => 'Laptop', 'quantity' => 1], ['id' => 'p2', 'name' => 'Mouse', 'quantity' => 2], ], ]; // Increase quantity. $currentQuantity = Selector::get($data, 'cart[id=p1:quantity]'); Selector::set($data, 'cart[id=p1:quantity]', $currentQuantity + 1); // Now p1's quantity is 2 // Add a new property to an array element. Selector::set($data, 'cart[id=p2:price]', 25); // Mouse now has a price property // Modify a nested property. $data = [ 'orders' => [ [ 'id' => 1, 'items' => [ ['product' => 'Laptop', 'price' => 999], ], ], ], ]; // Apply discount. Selector::set($data, 'orders[id=1:items][product=Laptop:price]', 899); // Price is now 899 ``` --- By leveraging these array handling capabilities, you can work with complex nested data structures in a clean, readable way without worrying about undefined indices or complex nesting logic. --- ### Advanced Syntax Advanced Selector Syntax # Advanced Selector Syntax This guide covers the advanced selector syntax features that make Derafu Selector powerful for complex data manipulation. ## String Literals and Concatenation You can include literal strings in your selectors and concatenate them with values from your data: ```php $data = ['user' => ['name' => 'John']]; // Simple string concatenation. $greeting = Selector::get($data, '"Hello, "(user.name)"!"'); // "Hello, John!" // Multiple concatenations. $message = Selector::get($data, '"User "(user.name)" logged in at "(timestamp)'); // "User John logged in at 2023-04-15 10:30:45" ``` ## OR Operator for Fallbacks The `||` operator provides fallback values when a selector path doesn't exist or returns null: ```php $data = [ 'user' => [ 'name' => 'John', 'email' => 'john@example.com', // phone is not set ], ]; // Use email if phone doesn't exist. $contact = Selector::get($data, 'user.phone||user.email'); // "john@example.com" // Chains of fallbacks. $identifier = Selector::get($data, 'user.id||user.username||user.email'); // "john@example.com" // Fallback to a literal string. $phone = Selector::get($data, 'user.phone||"Not provided"'); // "Not provided" // Combine with string concatenation. $phoneDisplay = Selector::get($data, '"Phone: "(user.phone||"N/A")'); // "Phone: N/A" ``` ## Conditional (Ternary) Selectors Conditionals let you choose between different selector paths based on conditions: ```php $data = [ 'user' => [ 'type' => 'admin', 'admin_role' => 'super_admin', 'user_role' => 'regular', ], ]; // Simple condition using equality. $role = Selector::get($data, '((user.type) = "admin" ? (user.admin_role) : (user.user_role))' ); // "super_admin" // Using not equal. $accessLevel = Selector::get($data, '((user.type) != "guest" ? ("authenticated") : ("anonymous"))' ); // "authenticated" // Numeric comparisons. $data = ['score' => 85]; $grade = Selector::get($data, '((score) >= "90" ? ("A") : ((score) >= "80" ? ("B") : ("C")))' ); // "B" ``` ## Dependent Selectors Find and access specific elements in arrays based on key/value matching: ```php $data = [ 'users' => [ ['id' => 101, 'name' => 'John', 'email' => 'john@example.com'], ['id' => 102, 'name' => 'Jane', 'email' => 'jane@example.com'], ['id' => 103, 'name' => 'Bob', 'email' => 'bob@example.com'], ], ]; // Get the name of user with id 102. $name = Selector::get($data, 'users[id=102:name]'); // "Jane" // Get the email of user with id 101. $email = Selector::get($data, 'users[id=101:email]'); // "john@example.com" // Nested properties. $data = [ 'orders' => [ [ 'id' => 1001, 'customer' => ['id' => 101, 'name' => 'John'], 'items' => [['product' => 'Laptop', 'price' => 999]], ], [ 'id' => 1002, 'customer' => ['id' => 102, 'name' => 'Jane'], 'items' => [['product' => 'Phone', 'price' => 699]], ], ], ]; // Get Jane's first order item. $janeProduct = Selector::get($data, 'orders[id=1002:items][0].product'); // "Phone" // Access by string values with special characters. $data = [ 'categories' => [ ['id' => 'cat-1', 'name' => 'Electronics'], ['id' => 'cat-2', 'name' => 'Books'] ] ]; $catName = Selector::get($data, 'categories[id=cat-1:name]'); // "Electronics" ``` ## Comparison Operators Conditional selectors support several comparison operators: ```php $data = [ 'product' => [ 'price' => 50, 'stock' => 10, 'name' => 'Gadget', 'tags' => ['electronics', 'new'], ], ]; // Equality. $isGadget = Selector::get($data, '((product.name) = "Gadget" ? ("Yes") : ("No"))' ); // "Yes" // Not equal. $isExpensive = Selector::get($data, '((product.price) != "100" ? ("Affordable") : ("Expensive"))' ); // "Affordable" // Greater than. $priceTier = Selector::get($data, '((product.price) > "75" ? ("Premium") : ("Standard"))' ); // "Standard" // Less than or equal. $stockStatus = Selector::get($data, '((product.stock) <= "5" ? ("Low Stock") : ("In Stock"))' ); // "In Stock" // Contains (for arrays and strings). $isNew = Selector::get($data, '((product.tags) contains "new" ? ("New Arrival") : ("Regular Item"))' ); // "New Arrival" // Length check. $nameLength = Selector::get($data, '((product.name) length "6" ? ("Six Letters") : ("Other Length"))' ); // "Six Letters" // Null check. $data['product']['description'] = null; $hasDescription = Selector::get($data, '((product.description) is "null" ? ("No Description") : ("Has Description"))' ); // "No Description" ``` ## Special Functions The selector syntax includes several special functions: ```php $data = [ 'product' => [ 'name' => 'Gadget', 'price' => 49.99, 'tags' => ['electronics', 'gadget', 'new'], ], ]; // contains - check if an array or string contains a value. $hasTag = Selector::get($data, '((product.tags) contains "gadget" ? ("Tagged as gadget") : ("Not tagged"))' ); // "Tagged as gadget" // length - check the length of a string or array. $tagCount = Selector::get($data, '((product.tags) length "3" ? ("Has 3 tags") : ("Has another tag count"))' ); // "Has 3 tags" $nameLength = Selector::get($data, '((product.name) length "6" ? ("Name has 6 chars") : ("Name has different length"))' ); // "Name has 6 chars" // is - check the type of a value (currently supports null checks). $hasDescription = Selector::get($data, '((product.description) is "null" ? ("No description") : ("Has description"))' ); // "No description" ``` ## Parentheses and Escaping The selector syntax uses parentheses and quotes extensively: ```php // (selector) extracts a value from the data // "string" is a literal string $data = ['message' => 'Hello World']; // Simple extraction. $msg = Selector::get($data, '(message)'); // "Hello World" // Literal string. $literal = Selector::get($data, '"Static text"'); // "Static text" // Concatenating extractions and literals. $full = Selector::get($data, '(message)" - welcome!"'); // "Hello World - welcome!" // Escaping quotes and parentheses. $escaped = Selector::get($data, '"This is a \\"quoted\\" string"'); // 'This is a "quoted" string' $parentheses = Selector::get($data, '"Formula: (x + y)"'); // "Formula: (x + y)" // Nested parentheses for complex expressions. $nested = Selector::get($data, '((message) = "Hello World" ? ("Greeting") : ("Other message"))'); // "Greeting" ``` --- For even more advanced capabilities, explore the integration with JSONPath and JMESPath in their respective guides. --- ### JSONPath Integration JSONPath Integration # JSONPath Integration This guide explains how to use JSONPath with Derafu Selector for powerful data structure querying. ## Introduction to JSONPath JSONPath is a query language for JSON, similar to XPath for XML. It provides a powerful way to extract data from complex JSON structures. Derafu Selector integrates JSONPath to give you even more flexibility in navigating your data. All JSONPath expressions in Derafu Selector start with `$.` to distinguish them from regular selectors. ## Basic JSONPath Queries Here are some basic JSONPath examples: ```php use Derafu\Selector\Selector; $data = [ 'store' => [ 'books' => [ ['title' => 'Book 1', 'price' => 10], ['title' => 'Book 2', 'price' => 20], ['title' => 'Book 3', 'price' => 30], ], 'bicycles' => [ ['model' => 'Model A', 'price' => 100], ['model' => 'Model B', 'price' => 200], ], ], 'user' => [ 'name' => 'John', 'email' => 'john@example.com', ], ]; // Access a simple property. $userName = Selector::get($data, '$.user.name'); // "John" // Access an array element by index. $firstBook = Selector::get($data, '$.store.books[0].title'); // "Book 1" // Get all book titles. $allTitles = Selector::get($data, '$.store.books[*].title'); // ["Book 1", "Book 2", "Book 3"] // Get the entire books array. $allBooks = Selector::get($data, '$.store.books'); // The complete books array ``` ## Advanced JSONPath Features JSONPath offers powerful filtering capabilities: ```php // Filter books by price (more than 15). $expensiveBooks = Selector::get($data, '$.store.books[?(@.price > 15)].title'); // ["Book 2", "Book 3"] // Filter books by title (exact match). $specificBook = Selector::get($data, '$.store.books[?(@.title == "Book 2")]'); // Returns the complete book object with title "Book 2" // Multiple conditions with AND. $filtered = Selector::get($data, '$.store.books[?(@.price > 15 && @.price < 25)].title' ); // ["Book 2"] // Regular expression matching (if supported by the JSONPath implementation). $booksWithPattern = Selector::get($data, '$.store.books[?(@.title =~ /Book [1-2]/)].title'); // ["Book 1", "Book 2"] // Array slicing. $firstTwoBooks = Selector::get($data, '$.store.books[0:2].title'); // ["Book 1", "Book 2"] ``` ### Common JSONPath Syntax Elements | Symbol | Description | |-------------------|----------------------------------------------------------| | `$` | The root object/element | | `@` | The current object/element | | `.` | Child operator | | `..` | Recursive descent (find all matches at any level) | | `*` | Wildcard (all objects/elements) | | `[n]` | Array index (0-based) | | `[n:m]` | Array slice from n to m | | `[?()]` | Filter expression | | `==` `!=` | Equality operators | | `>` `<` `>=` `<=` | Comparison operators | | `&&` `\|\|` | Logical AND and OR | ## Combining JSONPath with Derafu Selectors You can mix JSONPath with regular Derafu selectors for maximum flexibility: ```php // Use JSONPath to get a value and then concatenate with text. $message = Selector::get($data, '"User: "($.user.name)'); // "User: John" // Use JSONPath as part of an OR selector. $contact = Selector::get($data, '$.user.phone||$.user.email'); // Falls back to "john@example.com" if phone doesn't exist // Use JSONPath result in a conditional expression. $userType = Selector::get($data, '(($.user.email) contains "example.com" ? ("Standard") : ("Premium"))' ); // "Standard" // Format array results. $bookList = Selector::get($data, '"Available books: "($.store.books[*].title)' ); // "Available books: [\"Book 1\", \"Book 2\", \"Book 3\"]" // Mix JSONPath with regular selectors. $data = [ 'config' => ['theme' => 'dark'], 'preferences' => [ 'admin' => ['theme' => 'light'], 'user' => ['theme' => 'system'] ] ]; $theme = Selector::get($data, '(($.user.role) = "admin" ? (preferences.admin.theme) : (config.theme))' ); // Falls back to "dark" if user.role doesn't exist ``` ## Performance Considerations While JSONPath provides powerful querying capabilities, it may have different performance characteristics compared to native Derafu selectors: 1. **Initialization Cost**: JSONPath queries require initializing the JSONPath processor, which adds some overhead. 2. **Complex Queries**: For very complex queries with multiple conditions, JSONPath may be more efficient than chaining multiple native selectors. 3. **Large Data Sets**: For very large data structures, JSONPath's optimized filtering can be more efficient than iterating through the data manually. 4. **Writing Operations**: Note that JSONPath in Derafu Selector supports read operations only. For writing operations, you need to use native selectors. **Best Practices**: - Use native Derafu selectors for simple path access (`user.profile.name`). - Use JSONPath for complex filtering and array operations. - Consider caching results of expensive JSONPath queries if they're used repeatedly. - Avoid using `..` (recursive descent) on very large data structures if performance is critical. ```php // Less efficient for simple access. $name = Selector::get($data, '$.user.name'); // More efficient native syntax for simple access. $name = Selector::get($data, 'user.name'); // JSONPath is more efficient for complex filtering. $expensiveItems = Selector::get($data, '$.items[?(@.price > 100 && @.category == "electronics")].name' ); ``` --- By understanding when to use JSONPath versus native selectors, you can optimize your code for both readability and performance. --- ### JMESPath Integration JMESPath Integration # JMESPath Integration This guide explains how to use JMESPath with Derafu Selector for powerful data querying. ## Introduction to JMESPath JMESPath is a query language for JSON that allows you to extract and transform elements from complex data structures. It provides a more structured and feature-rich alternative to JSONPath with a clear specification. All JMESPath expressions in Derafu Selector start with `jmespath:` to distinguish them from regular selectors. ## Basic JMESPath Queries Here are some basic JMESPath examples: ```php use Derafu\Selector\Selector; $data = [ 'people' => [ [ 'name' => 'John', 'age' => 30, 'phones' => ['home' => '555-1234', 'mobile' => '555-5678'], ], [ 'name' => 'Jane', 'age' => 25, 'phones' => ['home' => '555-4321', 'mobile' => '555-8765'], ], ], 'locations' => [ 'Seattle' => ['state' => 'WA', 'population' => 724305], 'New York' => ['state' => 'NY', 'population' => 8804190], ], ]; // Access a simple path. $firstPerson = Selector::get($data, 'jmespath:people[0]'); // Complete first person object. // Access a specific property. $firstName = Selector::get($data, 'jmespath:people[0].name'); // "John" // Access all names. $allNames = Selector::get($data, 'jmespath:people[*].name'); // ["John", "Jane"] // Access a nested property. $firstMobile = Selector::get($data, 'jmespath:people[0].phones.mobile'); // "555-5678" // Use bracket notation for keys with special characters. $nyState = Selector::get($data, 'jmespath:locations."New York".state'); // "NY" ``` ## Advanced JMESPath Features JMESPath offers powerful filtering, projection, and transformation capabilities: ### Filtering ```php // Filter people over age 25. $over25 = Selector::get($data, 'jmespath:people[?age > `25`]'); // Returns the complete person object for John. // Filter with multiple conditions. $filtered = Selector::get($data, 'jmespath:people[?age > `25` && contains(name, `Jo`)]' ); // Returns John's complete object. // Get just the names of filtered results. $names = Selector::get($data, 'jmespath:people[?age > `25`].name'); // ["John"] // Filter on nested properties. $withMobile = Selector::get($data, 'jmespath:people[?phones.mobile != null].name' ); // ["John", "Jane"] ``` ### Multi-select and Projections ```php // Select specific fields (projection). $simplified = Selector::get($data, 'jmespath:people[*].{n: name, a: age}'); // [{"n":"John","a":30}, {"n":"Jane","a":25}] // Multi-select on a single object. $details = Selector::get($data, 'jmespath:people[0].{name: name, mobile: phones.mobile}'); // {"name":"John","mobile":"555-5678"} // Flatten nested structures. $phones = Selector::get($data, 'jmespath:people[*].phones.*'); // ["555-1234", "555-5678", "555-4321", "555-8765"] ``` ### Functions JMESPath supports a variety of built-in functions: ```php // String functions. $upperNames = Selector::get($data, 'jmespath:people[*].name | [*].to_upper(@)'); // ["JOHN", "JANE"] // Length/count. $peopleCount = Selector::get($data, 'jmespath:length(people)'); // 2 // Min/max of values. $data = ['values' => [5, 3, 8, 1, 7]]; $maxValue = Selector::get($data, 'jmespath:max(values)'); // 8 // Sort function. $sortedNames = Selector::get($data, 'jmespath:sort(people[*].name)'); // ["Jane", "John"] // Map function for transformations. $ages = Selector::get($data, 'jmespath:people[*].age | map(&to_string(@), @)'); // ["30", "25"] ``` ### Slicing and Indexing ```php // Array slicing. $firstPerson = Selector::get($data, 'jmespath:people[:1]'); // [Complete first person object] // Negative indices. $lastPerson = Selector::get($data, 'jmespath:people[-1]'); // Complete last person object (Jane). // Step values. $everyOtherPerson = Selector::get($data, 'jmespath:people[::2]'); // Every other person in the array. ``` ## Combining JMESPath with Derafu Selectors You can mix JMESPath with regular Derafu selectors: ```php // Use JMESPath to get a value and then concatenate with text. $greeting = Selector::get($data, '"Hello, "(jmespath:people[0].name)"!"'); // "Hello, John!" // Use JMESPath as part of an OR selector. $location = Selector::get($data, 'jmespath:current_location||jmespath:locations."Seattle".state' ); // Falls back to "WA" if current_location doesn't exist. // Use JMESPath within a conditional. $message = Selector::get($data, '((jmespath:people | length(@)) > "1" ? ("Multiple people") : ("One person"))' ); // "Multiple people" // Format array results. $nameList = Selector::get($data, '"People: "(jmespath:people[*].name)' ); // "People: [\"John\", \"Jane\"]" ``` ## Choosing Between JMESPath and JSONPath Derafu Selector supports both JMESPath and JSONPath. Here are some considerations for choosing between them: ### JMESPath Advantages - **Formal Specification**: JMESPath has a formal, well-defined specification. - **Functions**: Built-in functions for transformation and manipulation. - **Multi-select Projection**: Create new structures with the data you extract. - **Expressions**: More powerful expression language. - **Pipe Operator**: Chain operations together. ### JSONPath Advantages - **Simplicity**: More straightforward syntax for basic queries. - **Familiarity**: If you're already using JSONPath elsewhere. - **Recursive Descent**: The `..` operator for finding elements at any level. ### Examples of the Same Query in Both ```php $data = [ 'products' => [ ['name' => 'Laptop', 'price' => 999, 'category' => 'electronics'], ['name' => 'Phone', 'price' => 699, 'category' => 'electronics'], ['name' => 'Book', 'price' => 15, 'category' => 'media'], ], ]; // Get electronics products with JSONPath. $electronics = Selector::get($data, '$.products[?(@.category == "electronics")].name' ); // ["Laptop", "Phone"] // Same query with JMESPath. $electronics = Selector::get($data, 'jmespath:products[?category == `electronics`].name' ); // ["Laptop", "Phone"] // Filter by price and sort with JSONPath (sorting may not be available in all implementations). // May require multiple selector calls. // With JMESPath, this is built-in. $sortedProducts = Selector::get($data, 'jmespath:products[?price > `500`].name | sort(@)' ); // ["Laptop", "Phone"] (sorted) ``` --- By understanding the strengths of each query language, you can choose the right tool for your specific data navigation needs. --- ## XML Project Derafu XML # Derafu XML --- ### Introduction Library for XML manipulation # Library for XML manipulation ![GitHub last commit](https://img.shields.io/github/last-commit/derafu/xml/main) ![CI Workflow](https://github.com/derafu/xml/actions/workflows/ci.yml/badge.svg?branch=main&event=push) ![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/derafu/xml) ![GitHub Issues](https://img.shields.io/github/issues-raw/derafu/xml) ![Total Downloads](https://poser.pugx.org/derafu/xml/downloads) ![Monthly Downloads](https://poser.pugx.org/derafu/xml/d/monthly) A comprehensive PHP library for XML manipulation, providing robust tools for encoding, decoding, validating, and querying XML documents. ## Features - **Conversion**: Transform between XML and PHP arrays in both directions. - **Validation**: Validate XML documents against XSD schemas. - **Querying**: Powerful XPath querying with parameter support. - **Canonicalization**: Support for C14N and ISO-8859-1 encoding. - **Encoding**: Proper handling of character encoding between UTF-8 and ISO-8859-1. - **Special Characters**: Automatic handling of XML special characters and entities. ## Installation ```bash composer require derafu/xml ``` ## Basic Usage ### Creating XML from an Array ```php use Derafu\Xml\Service\XmlEncoder; $encoder = new XmlEncoder(); // Create an array to convert to XML. $data = [ 'root' => [ 'element1' => 'value1', 'element2' => 'value2', 'element3' => [ '@attributes' => [ 'attr1' => 'attrValue', ], '@value' => 'value3', ], 'repeatedElement' => ['value4', 'value5', 'value6'], ], ]; // Convert the array to XML. $xmlDocument = $encoder->encode($data); // Save as XML string. $xmlString = $xmlDocument->saveXml(); // Get the XML without the XML declaration. $xmlContent = $xmlDocument->getXml(); // Get a canonicalized version of the XML. $c14nXml = $xmlDocument->C14N(); // Get a canonicalized version with ISO-8859-1 encoding. $isoXml = $xmlDocument->C14NWithIso88591Encoding(); ``` ### Converting XML to an Array ```php use Derafu\Xml\Service\XmlDecoder; use Derafu\Xml\XmlDocument; $decoder = new XmlDecoder(); // Load an existing XML string. $xmlContent = '<?xml version="1.0" encoding="UTF-8"?><root><element>value</element></root>'; $document = new XmlDocument(); $document->loadXml($xmlContent); // Convert to an array. $array = $decoder->decode($document); // Now $array contains ['root' => ['element' => 'value']] ``` ### Working with XPath ```php use Derafu\Xml\XPathQuery; $xmlContent = '<?xml version="1.0"?><root><item id="1">First</item><item id="2">Second</item></root>'; // Create XPath query instance. $query = new XPathQuery($xmlContent); // Get a specific value. $value = $query->getValue('/root/item[@id="2"]'); // "Second" // Get multiple values. $values = $query->getValues('/root/item'); // ["First", "Second"] // Get structured array. $array = $query->get('/root'); // Result: ['item' => ['First', 'Second']] ``` ### XML Validation ```php use Derafu\Xml\Exception\XmlException; use Derafu\Xml\Service\XmlValidator; $validator = new XmlValidator(); // Validate an XML document against a schema. try { $validator->validate($xmlDocument, '/path/to/schema.xsd'); echo "XML is valid!"; } catch (XmlException $e) { echo "Validation failed: " . $e->getMessage(); $errors = $e->getErrors(); // Get detailed error information. } ``` ## Advanced Usage ### Working with Namespaces ```php // Create XPath query with namespace support. $namespaces = ['ns' => 'http://example.com/namespace']; $query = new XPathQuery($xmlContent, $namespaces); // Query with namespace. $result = $query->get('//ns:Element'); ``` ### Array Structure for XML Creation When creating XML from arrays, the structure follows these conventions: - Simple key-value pairs become element nodes. - Use `@attributes` for node attributes. - Use `@value` for node text content when attributes are present. - Arrays of values create repeated nodes with the same name. Example: ```php $data = [ 'root' => [ 'element' => [ '@attributes' => ['id' => '123'], '@value' => 'content', ], 'items' => [ 'item' => ['value1', 'value2', 'value3'], ], ], ]; ``` Produces: ```xml <root> <element id="123">content</element> <items> <item>value1</item> <item>value2</item> <item>value3</item> </items> </root> ``` --- ### Array Structure XML-Array Conversion Structure # XML-Array Conversion Structure The Derafu XML library uses a specific convention for representing the structure of XML documents as PHP arrays and vice versa. Understanding this structure is essential for effectively using the encoding and decoding features. ## Basic Structure At its most basic level, an XML element with a value is represented as a key-value pair in the array: XML: ```xml <element>value</element> ``` Array: ```php ['element' => 'value'] ``` ## Nested Elements Nested elements are represented as nested arrays: XML: ```xml <parent> <child>value</child> </parent> ``` Array: ```php [ 'parent' => [ 'child' => 'value' ] ] ``` ## Attributes XML attributes are handled using the special `@attributes` key: XML: ```xml <element id="123" type="example">value</element> ``` Array: ```php [ 'element' => [ '@attributes' => [ 'id' => '123', 'type' => 'example' ], '@value' => 'value' ] ] ``` Note the use of `@value` to hold the element's text content when attributes are present. ## Repeated Elements Elements with the same name are collected into arrays: XML: ```xml <parent> <child>value1</child> <child>value2</child> <child>value3</child> </parent> ``` Array: ```php [ 'parent' => [ 'child' => [ 'value1', 'value2', 'value3' ] ] ] ``` ## Complex Structures The conventions can be combined to represent complex XML structures: XML: ```xml <root> <items> <item id="1"> <name>Item 1</name> <price currency="USD">19.99</price> </item> <item id="2"> <name>Item 2</name> <price currency="EUR">29.99</price> </item> </items> <summary total="2">Multiple items</summary> </root> ``` Array: ```php [ 'root' => [ 'items' => [ 'item' => [ [ '@attributes' => ['id' => '1'], 'name' => 'Item 1', 'price' => [ '@attributes' => ['currency' => 'USD'], '@value' => '19.99' ] ], [ '@attributes' => ['id' => '2'], 'name' => 'Item 2', 'price' => [ '@attributes' => ['currency' => 'EUR'], '@value' => '29.99' ] ] ] ], 'summary' => [ '@attributes' => ['total' => '2'], '@value' => 'Multiple items' ] ] ] ``` ## Special Cases ### Empty Elements Empty elements are represented as `null` or empty strings: XML: ```xml <element></element> ``` Array: ```php ['element' => null] ``` or when creating XML: ```php ['element' => ''] ``` ### Skipping Elements When creating XML, certain values cause an element to be skipped: ```php [ 'included' => 'This will be in the XML', 'excluded' => null, // This will be skipped. 'alsoExcluded' => false, // This will be skipped. 'emptyArraySkipped' => [] // This will be skipped. ] ``` ## Array to XML Encoding Rules When using `XmlEncoder` to create XML from arrays, these rules apply: 1. Simple key-value pairs become elements with text content. 2. Nested arrays become nested elements. 3. The special key `@attributes` defines attributes for the parent element. 4. The special key `@value` defines the text content when attributes are present. 5. Arrays of values create multiple elements with the same name. 6. Null, false, and empty arrays are skipped (not included in the XML). 7. Empty strings ('') create empty elements. ## XML to Array Decoding Rules When using `XmlDecoder` to convert XML to arrays, these rules apply: 1. Elements become keys in the array. 2. Text content becomes the value of the key. 3. Attributes are collected under the `@attributes` key. 4. When attributes are present, text content is stored under the `@value` key. 5. Multiple elements with the same name are collected into arrays. 6. Empty elements become null values. 7. Complex nested structures are preserved in the array hierarchy. ## Usage Examples ### Creating Complex XML ```php use Derafu\Xml\Service\XmlEncoder; $data = [ 'invoice' => [ '@attributes' => [ 'id' => 'INV-2025-001', 'date' => '2025-03-05' ], 'customer' => [ 'name' => 'Acme Inc.', 'address' => [ 'street' => '123 Main St', 'city' => 'Anytown', 'zipcode' => '12345' ] ], 'items' => [ 'item' => [ [ '@attributes' => ['sku' => 'PROD1'], 'description' => 'Product One', 'quantity' => '2', 'price' => '10.99' ], [ '@attributes' => ['sku' => 'PROD2'], 'description' => 'Product Two', 'quantity' => '1', 'price' => '24.99' ] ] ], 'total' => '46.97' ] ]; $encoder = new XmlEncoder(); $xmlDocument = $encoder->encode($data); echo $xmlDocument->saveXml(); ``` ### Processing XML into Arrays ```php use Derafu\Xml\Service\XmlDecoder; $xmlString = ' <catalog> <book id="bk101"> <author>Gambardella, Matthew</author> <title>XML Developer\'s Guide</title> <genre>Computer</genre> <price>44.95</price> <publish_date>2025-01-15</publish_date> </book> <book id="bk102"> <author>Ralls, Kim</author> <title>Midnight Rain</title> <genre>Fantasy</genre> <price>5.95</price> <publish_date>2025-02-20</publish_date> </book> </catalog>'; $document = new \Derafu\Xml\XmlDocument(); $document->loadXml($xmlString); $decoder = new XmlDecoder(); $array = $decoder->decode($document); // Access data directly. $firstBookTitle = $array['catalog']['book'][0]['title']; // "XML Developer's Guide" $secondBookPrice = $array['catalog']['book'][1]['price']; // "5.95" ``` ## Tips for Working with the Array Structure 1. **Plan Your Structure**: When creating XML, plan your array structure carefully to match the desired XML output. 2. **Handle Repeated Elements**: Always use numeric arrays for repeated elements with the same name. 3. **Use `@attributes` and `@value`**: Remember to use these special keys when working with elements that have both attributes and text content. 4. **Validate Your Output**: Always validate the generated XML against your schema if one exists. 5. **Check for Empty Values**: When processing arrays from XML, check for null values to handle empty elements properly. --- ### XML Document XmlDocument Class # XmlDocument Class The `XmlDocument` class extends PHP's native `DOMDocument` with additional functionality for XML manipulation, transformation, and querying. This class is the core component for working with XML documents in the Derafu XML library. ## Key Features - Extended XML loading with character encoding handling. - Simplified access to XML metadata (namespace, schema). - Canonicalization support with encoding options. - XPath query integration. - Array conversion capabilities. - Signature node handling for XML digital signatures. ## Basic Usage ### Creating a New Document ```php use Derafu\Xml\XmlDocument; // Create with default version (1.0) and encoding (ISO-8859-1). $document = new XmlDocument(); // Or specify version and encoding. $document = new XmlDocument('1.0', 'UTF-8'); ``` ### Loading XML Content ```php // Load from a string. $xmlString = '<root><element>value</element></root>'; $document->loadXml($xmlString); // The method handles encoding conversion automatically. $utf8Xml = '<?xml version="1.0" encoding="UTF-8"?><root><element>Árbol</element></root>'; $document->loadXml($utf8Xml); // Will convert to ISO-8859-1 if needed. ``` ### Accessing Document Information ```php // Get the root element name. $rootName = $document->getName(); // e.g., "root" // Get the XML namespace (if any). $namespace = $document->getNamespace(); // e.g., "http://example.com" // Get the schema location (if any). $schema = $document->getSchema(); // e.g., "schema.xsd" ``` ### Saving and Serializing ```php // Get the complete XML document with declaration. $xmlString = $document->saveXml(); // Get only the XML content without the declaration. $xmlContent = $document->getXml(); // Get the canonicalized (C14N) version. $canonXml = $document->C14N(); // Get canonicalized version with ISO-8859-1 encoding. $isoCanonXml = $document->C14NWithIso88591Encoding(); // Get flattened canonicalized version (whitespace removed between tags). $flatXml = $document->C14NWithIso88591EncodingFlattened(); ``` ## XPath Querying The XmlDocument class has integrated XPath querying capabilities: ```php // Execute an XPath query and get result as string or array. $result = $document->query('/root/element'); // Get DOMNodeList from an XPath query. $nodes = $document->getNodes('/root/element'); // Use parameters in XPath queries. $params = ['id' => '123']; $result = $document->query('/root/element[@id=:id]', $params); ``` ## Array Conversion ```php // Convert the entire document to an array. $array = $document->toArray(); // Access a specific part using dot notation. $value = $document->get('root.element'); // With a default value if not found. $value = $document->get('root.missing', 'default value'); ``` ## Working with XML Digital Signatures If the document contains an XML digital signature, you can extract it: ```php // Get the signature node as XML. $signatureXml = $document->getSignatureNodeXml(); // Returns null if no signature is present. if ($signatureXml !== null) { // Process signature... } ``` ## Handling Special Characters and Entities The `XmlDocument` class works with the `XmlHelper` to properly handle special characters and entities: ```php // When saving XML, entities are handled correctly. $document->loadXml('<root><element>Text with & < > " \'</element></root>'); $xml = $document->saveXml(); // Produces: <root><element>Text with &amp; &lt; &gt; &quot; &apos;</element></root> ``` ## Advanced Canonicalization ### Working with Specific Nodes You can apply canonicalization to specific parts of the document: ```php // Canonicalize only a portion of the document. $canonXml = $document->C14NWithIso88591Encoding('/root/section'); ``` ### Differences Between Canonicalization Methods The class offers several canonicalization methods: 1. **C14N()**: Standard canonicalization, always outputs UTF-8. 2. **C14NWithIso88591Encoding()**: Canonicalization with conversion to ISO-8859-1. 3. **C14NWithIso88591EncodingFlattened()**: ISO-8859-1 canonicalization with whitespace removal. These methods are particularly useful when working with digital signatures or when XML needs to be processed by systems with specific encoding requirements. ## Error Handling The `loadXml()` method throws descriptive exceptions when it encounters errors: ```php use Derafu\Xml\Exception\XmlException; try { $document->loadXml($potentiallyInvalidXml); } catch (XmlException $e) { echo "Error loading XML: " . $e->getMessage(); $errors = $e->getErrors(); // Get detailed error information if available. } ``` ## Performance Considerations - When working with large XML documents, prefer using XPath queries to extract specific sections rather than converting the entire document to an array. - The canonicalization methods perform more processing, so use them only when needed. - The `getXml()` method is more efficient than `saveXml()` when you only need the XML content without the declaration. --- ### XPath Query XPathQuery Class # XPathQuery Class The `XPathQuery` class is a powerful tool for extracting data from XML documents using XPath expressions. It provides a convenient interface to query XML with support for namespaces, parameterized queries, and complex data structures. ## Basic Usage ```php use Derafu\Xml\XPathQuery; // Initialize with XML string or DOMDocument. $query = new XPathQuery($xmlString); // Get a single value. $value = $query->getValue('/root/element'); // Get multiple values. $values = $query->getValues('/root/items/item'); // Get DOM nodes. $nodes = $query->getNodes('/root/items/item'); // Get structured data. $data = $query->get('/root'); ``` ## Working with Namespaces ```php // Initialize with namespace support. $namespaces = [ 'ns' => 'http://example.com/ns', 'xs' => 'http://www.w3.org/2001/XMLSchema' ]; $query = new XPathQuery($xmlString, $namespaces); // Use namespaces in queries. $result = $query->get('//ns:Element/xs:Type'); ``` ## Parameterized Queries The `XPathQuery` class supports parameterized queries, making it safer and easier to build dynamic XPath expressions: ```php // Query with parameters. $params = [ 'id' => '123', 'type' => 'product' ]; $result = $query->get('//item[@id=:id and @type=:type]', $params); ``` This automatically handles escaping of values, including proper handling of values containing quotes. ## Context Nodes You can limit the scope of your query by providing a context node: ```php // Get a context node. $contextNode = $query->getNodes('//section')->item(0); // Query within that context. $result = $query->get('./item', [], $contextNode); ``` ## Return Value Handling The `get()` method is intelligent about what it returns: 1. `null` if no matching nodes are found. 2. A string if a single node with no children is found. 3. An array if multiple nodes are found. 4. A structured array if nodes with child elements are found. ### Example of Structured Results For XML like: ```xml <root> <person> <name>John</name> <age>30</age> </person> <person> <name>Jane</name> <age>25</age> </person> </root> ``` The query `$query->get('/root/person')` would return: ```php [ [ 'name' => 'John', 'age' => '30' ], [ 'name' => 'Jane', 'age' => '25' ] ] ``` ## Special Features ### Using Without Namespaces If you need to query XML with namespaces but don't want to specify them all, you can use the class without registering namespaces: ```php $query = new XPathQuery($xmlString); // No namespaces registered. // This will match any element named 'Element' regardless of its namespace. $result = $query->get('//Element'); ``` ### Value Quoting The class handles the quoting of values in XPath expressions automatically, even when values contain both single and double quotes: ```php $params = ['complex' => 'Value with "double" and \'single\' quotes']; $result = $query->get('//element[text()=:complex]', $params); ``` ### Error Handling The class provides clear error messages when there are issues with the XML document or XPath expressions: ```php try { $result = $query->get('//invalid[xpath'); } catch (InvalidArgumentException $e) { // Handle the error. } ``` --- ### XML Service XmlService Class # XmlService Class The `XmlService` class is the central component of the Derafu XML library, providing a unified interface for encoding, decoding, and validating XML documents. It follows a service-oriented architecture by delegating the actual work to specialized components. ## Service Structure The `XmlService` relies on three key components: 1. **XmlEncoder**: Converts PHP arrays to XML documents. 2. **XmlDecoder**: Converts XML documents to PHP arrays. 3. **XmlValidator**: Validates XML documents against XSD schemas. ## Basic Usage ```php use Derafu\Xml\Service\XmlEncoder; use Derafu\Xml\Service\XmlDecoder; use Derafu\Xml\Service\XmlValidator; use Derafu\Xml\Service\XmlService; // Initialize the components. $encoder = new XmlEncoder(); $decoder = new XmlDecoder(); $validator = new XmlValidator(); // Create the service. $xmlService = new XmlService($encoder, $decoder, $validator); ``` ## Encoding (Array to XML) The `encode()` method converts a PHP array to an XML document: ```php $data = [ 'invoice' => [ '@attributes' => [ 'id' => '12345', 'date' => '2025-03-05' ], 'client' => [ 'name' => 'Acme Corporation', 'tax_id' => '123456789' ], 'items' => [ 'item' => [ [ '@attributes' => ['id' => '1'], 'description' => 'Product A', 'quantity' => '2', 'price' => '19.99' ], [ '@attributes' => ['id' => '2'], 'description' => 'Product B', 'quantity' => '1', 'price' => '29.99' ] ] ], 'total' => '69.97' ] ]; $xmlDocument = $xmlService->encode($data); ``` This produces: ```xml <?xml version="1.0" encoding="ISO-8859-1"?> <invoice id="12345" date="2025-03-05"> <client> <name>Acme Corporation</name> <tax_id>123456789</tax_id> </client> <items> <item id="1"> <description>Product A</description> <quantity>2</quantity> <price>19.99</price> </item> <item id="2"> <description>Product B</description> <quantity>1</quantity> <price>29.99</price> </item> </items> <total>69.97</total> </invoice> ``` ### Using Namespaces You can specify XML namespaces during encoding: ```php $namespace = ['http://example.com/invoice', 'inv']; $xmlDocument = $xmlService->encode($data, $namespace); ``` This generates: ```xml <?xml version="1.0" encoding="ISO-8859-1"?> <inv:invoice id="12345" date="2025-03-05" xmlns:inv="http://example.com/invoice"> <!-- Content with namespace prefixes --> </inv:invoice> ``` ## Decoding (XML to Array) The `decode()` method converts an XML document to a PHP array: ```php // Assuming $xmlDocument is a Derafu\Xml\XmlDocument instance. $array = $xmlService->decode($xmlDocument); // Or starting from a DOMElement. $element = $xmlDocument->getDocumentElement(); $array = $xmlService->decode($element); ``` ### Handling Repeated Elements By default, elements with the same name are collected into arrays: ```xml <root> <item>Value 1</item> <item>Value 2</item> <item>Value 3</item> </root> ``` Becomes: ```php [ 'root' => [ 'item' => [ 'Value 1', 'Value 2', 'Value 3' ] ] ] ``` You can control this behavior with the `$twinsAsArray` parameter: ```php $array = $xmlService->decode($xmlDocument, null, twinsAsArray: true); ``` ## Validation The `validate()` method checks an XML document against an XSD schema: ```php use Derafu\Xml\Exception\XmlException; try { $xmlService->validate($xmlDocument, '/path/to/schema.xsd'); echo "XML is valid!"; } catch (XmlException $e) { echo "Validation failed: " . $e->getMessage(); // Get detailed error information. $errors = $e->getErrors(); foreach ($errors as $error) { echo "- $error\n"; } } ``` ### Schema Auto-Detection If the XML document includes a `schemaLocation` attribute, you can omit the schema path: ```php // XML with xsi:schemaLocation="http://example.com/ns schema.xsd" try { $xmlService->validate($xmlDocument); // Will use schema.xsd } catch (XmlException $e) { // Handle validation error. } ``` ### Error Translations The validator can translate technical libxml error messages into more user-friendly messages: ```php $translations = [ 'element1' => 'Customer Info', 'element2' => 'Product Details' ]; try { $xmlService->validate($xmlDocument, '/path/to/schema.xsd', $translations); } catch (XmlException $e) { // Error messages will use the translated element names. } ``` ## Integration Example This complete example shows how to use the XML service for a typical workflow: ```php // Initialize the service. $encoder = new XmlEncoder(); $decoder = new XmlDecoder(); $validator = new XmlValidator(); $xmlService = new XmlService($encoder, $decoder, $validator); // Create XML from array. $data = ['root' => ['element' => 'value']]; $xmlDocument = $xmlService->encode($data); // Save to a file. file_put_contents('document.xml', $xmlDocument->saveXml()); // Load from a file. $loadedXml = new \Derafu\Xml\XmlDocument(); $loadedXml->loadXml(file_get_contents('document.xml')); // Validate. try { $xmlService->validate($loadedXml, 'schema.xsd'); // Convert back to array. $array = $xmlService->decode($loadedXml); // Process the data. // ... } catch (\Derafu\Xml\Exception\XmlException $e) { // Handle validation errors. } ``` --- ## Certificate Project Derafu Certificate # Derafu Certificate --- ### Introduction Library for digital certificates # Library for digital certificates ![GitHub last commit](https://img.shields.io/github/last-commit/derafu/certificate/main) ![CI Workflow](https://github.com/derafu/certificate/actions/workflows/ci.yml/badge.svg?branch=main&event=push) ![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/derafu/certificate) ![GitHub Issues](https://img.shields.io/github/issues-raw/derafu/certificate) ![Total Downloads](https://poser.pugx.org/derafu/certificate/downloads) ![Monthly Downloads](https://poser.pugx.org/derafu/certificate/d/monthly) A comprehensive PHP library for working with digital certificates, providing tools for loading, validating and generating certificates. ## Features - **Certificate Loading**: Load certificates from files, data, arrays or keys. - **Certificate Validation**: Validate certificates against specific requirements. - **Certificate Generation**: Create self-signed certificates for testing. - **Certificate Information**: Extract key information from certificates (ID, name, email, etc.). - **Key Management**: Work with public and private keys, modulus, and exponent. ## Installation ```bash composer require derafu/certificate ``` ## Basic Usage ### Loading a Certificate ```php use Derafu\Certificate\Service\CertificateLoader; // Create a loader. $loader = new CertificateLoader(); // Load from a file. $certificate = $loader->loadFromFile('/path/to/certificate.p12', 'password'); // Load from data. $certificate = $loader->loadFromData($certificateData, 'password'); // Load from array. $certificate = $loader->loadFromArray([ 'cert' => $publicKey, 'pkey' => $privateKey ]); // Load from keys. $certificate = $loader->loadFromKeys($publicKey, $privateKey); ``` ### Accessing Certificate Information ```php // Get basic certificate information. $id = $certificate->getId(); // e.g., "12345678-9" $name = $certificate->getName(); // e.g., "John Doe" $email = $certificate->getEmail(); // e.g., "john.doe@example.com" // Check certificate validity. $isActive = $certificate->isActive(); // true if certificate is valid. $expirationDays = $certificate->getExpirationDays(); // days until expiration. // Get validity dates. $validFrom = $certificate->getFrom(); // e.g., "2025-01-01T00:00:00" $validTo = $certificate->getTo(); // e.g., "2026-01-01T00:00:00" // Get certificate issuer. $issuer = $certificate->getIssuer(); // e.g., "Example CA" // Get key components. $modulus = $certificate->getModulus(); $exponent = $certificate->getExponent(); // Get raw keys. $publicKey = $certificate->getPublicKey(); // with headers. $privateKey = $certificate->getPrivateKey(); // with headers. $cleanPublicKey = $certificate->getPublicKey(true); // without headers. $cleanPrivateKey = $certificate->getPrivateKey(true); // without headers. ``` ### Validating a Certificate ```php use Derafu\Certificate\Exception\CertificateException; use Derafu\Certificate\Service\CertificateValidator; $validator = new CertificateValidator(); try { $validator->validate($certificate); echo "Certificate is valid"; } catch (CertificateException $e) { echo "Certificate validation failed: " . $e->getMessage(); } ``` ### Creating a Fake Certificate for Testing ```php use Derafu\Certificate\Service\CertificateLoader; use Derafu\Certificate\Service\CertificateFaker; $loader = new CertificateLoader(); $faker = new CertificateFaker($loader); // Create a fake certificate with default values. $certificate = $faker->createFake(); // Create a fake certificate with custom values. $certificate = $faker->createFake( id: '12345678-9', name: 'John Doe', email: 'john.doe@example.com', password: 'secure_password' ); // Export to PKCS#12 format. $pkcs12Data = $certificate->getPkcs12('password'); file_put_contents('certificate.p12', $pkcs12Data); ``` ### Using the Service The `CertificateService` provides a unified interface to all library functionality: ```php use Derafu\Certificate\Service\CertificateLoader; use Derafu\Certificate\Service\CertificateFaker; use Derafu\Certificate\Service\CertificateValidator; use Derafu\Certificate\Service\CertificateService; // Create the service with its dependencies. $loader = new CertificateLoader(); $faker = new CertificateFaker($loader); $validator = new CertificateValidator(); $service = new CertificateService($faker, $loader, $validator); // Use the service for certificate operations. $certificate = $service->loadFromFile('/path/to/certificate.p12', 'password'); $service->validate($certificate); // Create a fake certificate for testing. $fakeCertificate = $service->createFake( '12345678-9', 'John Doe', 'john.doe@example.com' ); ``` ## Advanced Usage ### Creating a Self-Signed Certificate For more control over certificate generation: ```php use Derafu\Certificate\SelfSignedCertificate; use Derafu\Certificate\Service\CertificateLoader; // Create a self-signed certificate with custom values. $selfSigned = new SelfSignedCertificate(); $selfSigned->setSubject( C: 'US', ST: 'California', L: 'San Francisco', O: 'Example Organization', OU: 'IT Department', CN: 'John Doe', emailAddress: 'john.doe@example.com', serialNumber: '12345678-9' ); $selfSigned->setIssuer( CN: 'Example CA' ); $selfSigned->setValidity(365); // Valid for 1 year. $selfSigned->setPassword('secure_password'); // Get the certificate array. $certArray = $selfSigned->toArray(); // Load as a Certificate object. $loader = new CertificateLoader(); $certificate = $loader->loadFromArray($certArray); ``` ### Working with Asymmetric Keys ```php use Derafu\Certificate\AsymmetricKeyHelper; // Normalize a public key (add headers if missing). $normalizedPublicKey = AsymmetricKeyHelper::normalizePublicKey($rawPublicKey); // Normalize a private key (add headers if missing). $normalizedPrivateKey = AsymmetricKeyHelper::normalizePrivateKey($rawPrivateKey); // Generate a public key from modulus and exponent. // Requirements: composer require phpseclib/phpseclib $publicKey = AsymmetricKeyHelper::generatePublicKeyFromModulusExponent( $modulus, $exponent ); ``` --- ### Certificate Class Certificate Class # Certificate Class The `Certificate` class is the core component of the Derafu Certificate library, representing a digital certificate with its public and private keys and providing methods to access certificate details and properties. ## Overview The `Certificate` class implements the `CertificateInterface` and provides functionality to: - Access public and private keys. - Extract certificate metadata (ID, name, email, issuer, etc.). - Check certificate validity. - Get certificate validity dates. - Retrieve cryptographic components (modulus, exponent). - Generate PKCS#12 data. ## Usage ### Creating a Certificate To create a `Certificate` instance, you need a public key (certificate) and a private key: ```php use Derafu\Certificate\Certificate; $certificate = new Certificate($publicKey, $privateKey); ``` The constructor automatically normalizes the keys using `AsymmetricKeyHelper`. ### Getting Keys ```php // Get both keys as an array. $keys = $certificate->getKeys(); // Result: ['cert' => '...public key...', 'pkey' => '...private key...'] // Get both keys without headers/footers. $cleanKeys = $certificate->getKeys(clean: true); // Get just the public key. $publicKey = $certificate->getPublicKey(); // Alternatively. $publicKey = $certificate->getCertificate(); // Get just the private key. $privateKey = $certificate->getPrivateKey(); // Get clean keys (without headers/footers). $cleanPublicKey = $certificate->getPublicKey(clean: true); $cleanPrivateKey = $certificate->getPrivateKey(clean: true); ``` ### Certificate Metadata ```php // Get certificate ID (usually a document/tax number). $id = $certificate->getId(); // e.g., "12345678-9" $id = $certificate->getId(forceUpper:false); // with original case, e.g., "12345678-k" // Get certificate owner's name. $name = $certificate->getName(); // Get certificate owner's email. $email = $certificate->getEmail(); // Get certificate issuer. $issuer = $certificate->getIssuer(); ``` ### Certificate Validity ```php // Check if certificate is valid (not expired). $isActive = $certificate->isActive(); // Check if certificate is valid at a specific date. $isActiveOn = $certificate->isActive('2025-06-01'); // Get certificate validity period. $validFrom = $certificate->getFrom(); // e.g., "2025-01-01T00:00:00" $validTo = $certificate->getTo(); // e.g., "2026-01-01T00:00:00" // Get total validity period in days. $totalDays = $certificate->getTotalDays(); // Get days remaining until expiration. $daysRemaining = $certificate->getExpirationDays(); // Get days remaining from a specific date. $daysRemainingFrom = $certificate->getExpirationDays('2025-06-01'); ``` ### Cryptographic Components ```php // Get certificate raw data. $data = $certificate->getData(); // Get private key details. $privateKeyDetails = $certificate->getPrivateKeyDetails(); // Get modulus and exponent. $modulus = $certificate->getModulus(); $exponent = $certificate->getExponent(); // With custom word wrapping. $modulus = $certificate->getModulus(wordwrap: 75); $exponent = $certificate->getExponent(wordwrap: 75); ``` ### Exporting a Certificate ```php // Export certificate as PKCS#12 data. $pkcs12 = $certificate->getPkcs12('password'); // Save to a file. file_put_contents('certificate.p12', $pkcs12); ``` ## Error Handling The `Certificate` class throws `CertificateException` when it encounters problems such as: - Unable to extract the certificate ID. - Unable to find the certificate name. - Unable to find the certificate email. - Unable to access the modulus or exponent. Example: ```php use Derafu\Certificate\Exception\CertificateException; try { $id = $certificate->getId(); $name = $certificate->getName(); $email = $certificate->getEmail(); } catch (CertificateException $e) { echo "Certificate error: " . $e->getMessage(); } ``` ## Implementation Details ### ID Extraction The `getId()` method attempts to extract the certificate ID through multiple methods: 1. First, it checks the `serialNumber` field in the subject. 2. If not found, it looks for the ID in the Subject Alternative Name (SAN) extension. 3. If still not found, it throws a `CertificateException`. ### Certificate Validation The `isActive()` method checks if the certificate is valid at the current date (or a specified date) by comparing it against the certificate's validity period. ### Cryptographic Components The `getModulus()` and `getExponent()` methods extract these values from the private key details and return them as base64-encoded strings, which can be used for various cryptographic operations. --- ### Self-Signed Certificate SelfSignedCertificate Class # SelfSignedCertificate Class The `SelfSignedCertificate` class provides functionality to generate self-signed certificates for testing and development purposes. It allows customization of certificate properties such as subject, issuer, validity period, and password protection. ## Overview This class creates a digital certificate that includes: 1. A Certificate Authority (CA) certificate. 2. A user certificate signed by the CA. 3. Both packaged together in PKCS#12 format. The generated certificate is suitable for testing electronic signature functionality without requiring a certificate from a commercial Certificate Authority. ## Basic Usage ```php use Derafu\Certificate\SelfSignedCertificate; use Derafu\Certificate\Service\CertificateLoader; // Create with default settings. $selfSigned = new SelfSignedCertificate(); $certificateArray = $selfSigned->toArray(); // Load as a Certificate object. $loader = new CertificateLoader(); $certificate = $loader->loadFromArray($certificateArray); ``` ## Configuration Methods ### Setting the Subject The subject represents the owner of the certificate: ```php $selfSigned->setSubject( C: 'US', // Country code (2 letters). ST: 'California', // State/Province. L: 'San Francisco', // Locality/City. O: 'Example Corporation', // Organization. OU: 'IT Department', // Organizational Unit. CN: 'John Doe', // Common Name (full name). emailAddress: 'john.doe@example.com', // Email address. serialNumber: '12345678-9', // ID/Serial number (tax ID, etc.). title: 'System Administrator' // Title. ); ``` Only `CN`, `emailAddress`, and `serialNumber` are strictly required. The method will throw a `CertificateException` if any of these are empty. ### Setting the Issuer The issuer represents the Certificate Authority that issues the certificate: ```php $selfSigned->setIssuer( C: 'US', // Country code ST: 'Washington', // State/Province L: 'Seattle', // Locality/City O: 'Example CA', // Organization OU: 'Certificate Authority', // Organizational Unit CN: 'Example Root CA', // CA Name emailAddress: 'ca@example.com', // CA Email serialNumber: '87654321-0' // CA ID ); ``` ### Setting the Validity Period ```php // Set validity to 730 days (2 years). $selfSigned->setValidity(730); ``` The default validity period is 365 days (1 year). ### Setting the Password ```php // Set password for the private key. $selfSigned->setPassword('secure_password'); ``` The default password is `'i_love_derafu'`. ## Generating the Certificate ### Getting the Certificate Array ```php // Get the certificate as an array with 'cert' and 'pkey' elements. $certificateArray = $selfSigned->toArray(); // The array contains: // [ // 'cert' => '...public key certificate...', // 'pkey' => '...private key...' // ] ``` ### Getting the Raw PKCS#12 Data ```php // Get the toPkcs12() method result (PKCS#12 binary data). $pkcs12Data = $selfSigned->toPkcs12(); // Save to a file. file_put_contents('certificate.p12', $pkcs12Data); ``` ## Certificate Structure The generated certificate consists of: 1. An issuer (CA) certificate that is self-signed. 2. A subject certificate that is signed by the issuer. 3. Both certificates' private keys. The PKCS#12 container includes both certificates and is password-protected using the specified password. ## Default Values If you create a `SelfSignedCertificate` without any customization, it will use the following default values: ### Default Subject - Country: `CL` (Chile). - State: `Colchagua`. - Locality: `Santa Cruz`. - Organization: `Intergalactic Robots Organization`. - Organizational Unit: `Technology`. - Common Name: `Daniel Bot`. - Email: `daniel.bot@example.com`. - Serial Number: `11222333-9`. - Title: `Bot`. ### Default Issuer - Country: `CL` (Chile). - State: `Colchagua`. - Locality: `Santa Cruz`. - Organization: `Derafu`. - Organizational Unit: `Technology`. - Common Name: `Derafu Test Certificate Authority`. - Email: `fakes-certificates@derafu.org`. - Serial Number: `76192083-9`. ### Other Defaults - Validity: 365 days. - Password: `i_love_derafu`. ## Implementation Details The `SelfSignedCertificate` class uses the PHP OpenSSL extension to: 1. Generate a key pair for the issuer. 2. Create a Certificate Signing Request (CSR) for the issuer. 3. Self-sign the issuer's CSR to create the issuer certificate. 4. Generate a key pair for the subject. 5. Create a CSR for the subject. 6. Sign the subject's CSR with the issuer's certificate. 7. Package everything into a PKCS#12 container. The process mimics a real CA-issued certificate but is entirely self-contained and suitable for testing purposes. --- ### Asymmetric Key Helper AsymmetricKeyHelper Class # AsymmetricKeyHelper Class The `AsymmetricKeyHelper` class provides utilities for working with RSA certificates and keys. It offers methods to normalize public and private keys and to generate public keys from modulus and exponent values. ## Overview When working with digital certificates and electronic signatures, you often need to handle RSA key components in different formats. The `AsymmetricKeyHelper` class solves common problems related to key formatting and generation: 1. Adding standard PEM headers and footers to raw key data. 2. Converting between key components (modulus, exponent) and full key format. ## Key Normalization ### Normalizing a Public Key ```php use Derafu\Certificate\AsymmetricKeyHelper; // Raw public key without headers. $rawPublicKey = "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvTLIKu... (more base64 data)"; // Normalize the public key (add headers and footers). $normalizedPublicKey = AsymmetricKeyHelper::normalizePublicKey($rawPublicKey); // Result: // -----BEGIN CERTIFICATE----- // MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvTLIKu... // (more base64 data, wrapped, by default, at 64 characters per line) // -----END CERTIFICATE----- ``` ### Normalizing a Private Key ```php // Raw private key without headers. $rawPrivateKey = "MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkA... (more base64 data)"; // Normalize the private key (add headers and footers). $normalizedPrivateKey = AsymmetricKeyHelper::normalizePrivateKey($rawPrivateKey); // Result: // -----BEGIN PRIVATE KEY----- // MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkA... // (more base64 data, wrapped, by default, at 64 characters per line) // -----END PRIVATE KEY----- ``` ### Custom Line Length Both normalization methods support a custom line length for wrapping the key data: ```php // Wrap lines at 75 characters instead of the default 64. $normalizedPublicKey = AsymmetricKeyHelper::normalizePublicKey($rawPublicKey, 75); $normalizedPrivateKey = AsymmetricKeyHelper::normalizePrivateKey($rawPrivateKey, 75); ``` ## Key Generation ### Generating a Public Key from Modulus and Exponent You can generate a complete public key if you have the modulus and exponent components: ```php // Base64-encoded modulus and exponent. $modulus = "wKhpaf5AYomI+0/tLxJtvjHVCveRYYZ9j0yDlL..."; $exponent = "AQAB"; // Generate public key. $publicKey = AsymmetricKeyHelper::generatePublicKeyFromModulusExponent( $modulus, $exponent ); // Result is a complete public key in PKCS1 format: // -----BEGIN RSA PUBLIC KEY----- // MIIBCgKCAQEAwKhpaf5AYomI+0/tLxJtvjHVCveRYYZ9j0yDlL... // -----END RSA PUBLIC KEY----- ``` This method requires the `phpseclib3/phpseclib` library. If the library is not installed, it will throw a `LogicException` with instructions to install it. ## Common Use Cases ### Preparing Keys for Use with OpenSSL Functions Many OpenSSL functions require properly formatted keys with headers and footers: ```php // Normalize keys before using with OpenSSL. $normalizedPublicKey = AsymmetricKeyHelper::normalizePublicKey($rawPublicKey); $normalizedPrivateKey = AsymmetricKeyHelper::normalizePrivateKey($rawPrivateKey); // Now use with OpenSSL functions. $signature = openssl_sign($data, $signature, $normalizedPrivateKey, OPENSSL_ALGO_SHA256); $result = openssl_verify($data, $signature, $normalizedPublicKey, OPENSSL_ALGO_SHA256); ``` ### Reconstructing Public Keys In some systems, especially with hardware security modules (HSMs) or smart cards, you might only have access to the modulus and exponent components: ```php // Reconstruct the public key from components. $publicKey = AsymmetricKeyHelper::generatePublicKeyFromModulusExponent( $modulus, $exponent ); // Use for verification. $result = openssl_verify($data, $signature, $publicKey, OPENSSL_ALGO_SHA256); ``` ## Implementation Details ### Public Key Normalization The `normalizePublicKey()` method: 1. Checks if the key already contains the "BEGIN CERTIFICATE" header. 2. If not, adds the appropriate BEGIN/END headers. 3. Wraps the content to the specified line length. 4. Returns the normalized key. ### Private Key Normalization The `normalizePrivateKey()` method: 1. Checks if the key already contains the "BEGIN PRIVATE KEY" header. 2. If not, adds the appropriate BEGIN/END headers. 3. Wraps the content to the specified line length. 4. Returns the normalized key. ### Public Key Generation The `generatePublicKeyFromModulusExponent()` method: 1. Decodes the base64-encoded modulus and exponent. 2. Creates BigInteger instances from the binary data. 3. Loads these values into a RSA key object using phpseclib. 4. Exports the key in PKCS1 format. 5. Returns the complete public key string. --- ### Certificate Service CertificateService Class # CertificateService Class The `CertificateService` is the main entry point for working with digital certificates in the Derafu Certificate library. It provides a unified interface to all the key functionality like loading, validating, and creating certificates. ## Overview The `CertificateService` follows the service pattern, acting as a facade for: - `CertificateFaker`: For creating test certificates. - `CertificateLoader`: For loading certificates from various sources. - `CertificateValidator`: For validating certificates. By using the service, you can access all the library's functionality through a single interface. ## Basic Usage ### Setting Up the Service ```php use Derafu\Certificate\Service\CertificateLoader; use Derafu\Certificate\Service\CertificateFaker; use Derafu\Certificate\Service\CertificateValidator; use Derafu\Certificate\Service\CertificateService; // Create dependencies. $loader = new CertificateLoader(); $faker = new CertificateFaker($loader); $validator = new CertificateValidator(); // Create the service. $service = new CertificateService($faker, $loader, $validator); ``` ### Loading Certificates The service provides methods to load certificates from various sources: ```php // Load from a PKCS#12 file. $certificate = $service->loadFromFile('/path/to/certificate.p12', 'password'); // Load from PKCS#12 data. $certificate = $service->loadFromData($pkcs12Data, 'password'); // Load from a certificate array. $certificate = $service->loadFromArray([ 'cert' => $publicKey, 'pkey' => $privateKey ]); // Load directly from key strings. $certificate = $service->loadFromKeys($publicKey, $privateKey); ``` ### Validating Certificates ```php use Derafu\Certificate\Exception\CertificateException; try { $service->validate($certificate); echo "Certificate is valid!"; } catch (CertificateException $e) { echo "Validation failed: " . $e->getMessage(); } ``` ### Creating Test Certificates ```php // Create with default values. $certificate = $service->createFake(); // Create with custom values. $certificate = $service->createFake( id: '12345678-9', name: 'John Doe', email: 'john.doe@example.com' ); // Use the certificate. $id = $certificate->getId(); $name = $certificate->getName(); $isValid = $certificate->isActive(); ``` ## Complete Example ```php // Initialize the service. $loader = new CertificateLoader(); $faker = new CertificateFaker($loader); $validator = new CertificateValidator(); $service = new CertificateService($faker, $loader, $validator); // Create a test certificate. $certificate = $service->createFake( id: '12345678-9', name: 'John Doe', email: 'john.doe@example.com' ); // Extract certificate data. echo "Certificate ID: " . $certificate->getId() . "\n"; echo "Certificate Name: " . $certificate->getName() . "\n"; echo "Valid until: " . $certificate->getTo() . "\n"; echo "Days remaining: " . $certificate->getExpirationDays() . "\n"; // Validate the certificate. try { $service->validate($certificate); echo "Certificate is valid\n"; } catch (CertificateException $e) { echo "Invalid certificate: " . $e->getMessage() . "\n"; } // Export the certificate. $pkcs12Data = $certificate->getPkcs12('password'); file_put_contents('certificate.p12', $pkcs12Data); // Later, load the certificate back. $loadedCertificate = $service->loadFromFile('certificate.p12', 'password'); ``` ## Customizing the Service ### Custom Validator You can create a custom validator by implementing the `CertificateValidatorInterface`: ```php use Derafu\Certificate\Contract\CertificateInterface; use Derafu\Certificate\Contract\CertificateValidatorInterface; use Derafu\Certificate\Exception\CertificateException; class MyCustomValidator implements CertificateValidatorInterface { public function validate(CertificateInterface $certificate): void { // Implement your validation logic. if (!$certificate->isActive()) { throw new CertificateException( 'Certificate is expired.' ); } // Add specific validation for your use case. if ($certificate->getExpirationDays() < 30) { throw new CertificateException( 'Certificate will expire in less than 30 days.' ); } } } // Use your custom validator. $customValidator = new MyCustomValidator(); $service = new CertificateService($faker, $loader, $customValidator); ``` ## Service Components ### CertificateLoader The `CertificateLoader` component handles loading certificates from various sources: - PKCS#12 files. - PKCS#12 binary data. - Array of keys. - Individual public and private keys. ### CertificateFaker The `CertificateFaker` component provides a way to create self-signed certificates for testing purposes: - Create certificates with custom subject information. - Create certificates with custom issuer information. - Create certificates with custom validity periods. Under the hood, it uses the `SelfSignedCertificate` class. ### CertificateValidator The `CertificateValidator` component validates certificates according to specific requirements: - Checks if the certificate ID (RUN) is properly formatted. - Ensures that "K" in the ID is uppercase. - Verifies that the certificate is not expired. The default validator is designed for Chilean certificates used with the SII (Servicio de Impuestos Internos), but you can implement your own validator for different requirements. --- ## Signature Project Derafu Signature # Derafu Signature --- ### Introduction Library for digital signatures # Library for digital signatures ![GitHub last commit](https://img.shields.io/github/last-commit/derafu/signature/main) ![CI Workflow](https://github.com/derafu/signature/actions/workflows/ci.yml/badge.svg?branch=main&event=push) ![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/derafu/signature) ![GitHub Issues](https://img.shields.io/github/issues-raw/derafu/signature) ![Total Downloads](https://poser.pugx.org/derafu/signature/downloads) ![Monthly Downloads](https://poser.pugx.org/derafu/signature/d/monthly) A comprehensive PHP library for creating and validating digital signatures, with special focus on XML digital signatures (XML-DSIG). ## Features - **Digital Signatures**: Sign and validate any data with RSA key pairs. - **XML Signatures**: Full support for XML Digital Signatures (XML-DSIG). - **Signature Verification**: Validate signatures against public keys. - **Reference Support**: Sign specific sections of XML documents using ID references. - **Integration**: Works seamlessly with Derafu Certificate and Derafu XML libraries. ## Installation ```bash composer require derafu/signature ``` ## Basic Usage ### Signing Data ```php use Derafu\Signature\Service\SignatureGenerator; use Derafu\Signature\Service\SignatureService; use Derafu\Signature\Service\SignatureValidator; use Derafu\Xml\Service\XmlDecoder; use Derafu\Xml\Service\XmlEncoder; use Derafu\Xml\Service\XmlService; use Derafu\Xml\Service\XmlValidator; // Set up the signature service. $xmlEncoder = new XmlEncoder(); $xmlDecoder = new XmlDecoder(); $xmlValidator = new XmlValidator(); $xmlService = new XmlService($xmlEncoder, $xmlDecoder, $xmlValidator); $signatureGenerator = new SignatureGenerator($xmlService); $signatureValidator = new SignatureValidator($signatureGenerator, $xmlService); $signatureService = new SignatureService($signatureGenerator, $signatureValidator); // Sign simple data. $privateKey = '...'; $data = 'Hello, world!'; $signature = $signatureService->sign($data, $privateKey); // Validate the signature. $isValid = $signatureService->validate($data, $signature, $publicKey); ``` ### Signing XML Documents ```php use Derafu\Certificate\Service\CertificateLoader; // Load a certificate. $certificateLoader = new CertificateLoader(); $certificate = $certificateLoader->loadFromFile( '/path/to/certificate.p12', 'password' ); // Load XML to sign. $xml = file_get_contents('document.xml'); // Sign the entire XML document. $signedXml = $signatureService->signXml($xml, $certificate); // Sign a specific element in the XML document (identified by ID). $signedXml = $signatureService->signXml($xml, $certificate, 'elementId'); // Save the signed XML. file_put_contents('signed_document.xml', $signedXml); ``` ### Validating XML Signatures ```php use Derafu\Signature\Exception\SignatureException; // Load signed XML. $signedXml = file_get_contents('signed_document.xml'); try { // Validate the XML signature. $signatureService->validateXml($signedXml); echo "Signature is valid!"; } catch (SignatureException $e) { echo "Signature validation failed: " . $e->getMessage(); } ``` ## Advanced Usage ### Detailed XML Signature Validation For more detailed control over the validation process: ```php // Create a signature node from the signed XML. $signatureNode = $signatureService->createSignatureNode($signatureXml); // Validate the digest value (integrity of the signed content). $signatureService->validateXmlDigestValue($xmlDocument, $signatureNode); // Validate the signature value (authenticity of the signer). $signatureService->validateXmlSignatureValue($signatureNode); ``` ### Calculating Digest Values ```php use Derafu\Xml\XmlDocument; // Load XML document. $xmlDoc = new XmlDocument(); $xmlDoc->loadXml($xml); // Calculate digest value for the entire document. $digestValue = $signatureService->generateXmlDigestValue($xmlDoc); // Calculate digest value for a specific element. $digestValue = $signatureService->generateXmlDigestValue($xmlDoc, 'elementId'); ``` ## XML-DSIG Implementation Details The library implements XML Digital Signatures according to the [W3C XML Signature Syntax and Processing](https://www.w3.org/TR/xmldsig-core/) specification: 1. The `Signature` element is created with the following components: - `SignedInfo`: Contains information about what was signed. - `SignatureValue`: Contains the actual signature value. - `KeyInfo`: Contains information about the key used to validate the signature. 2. Canonicalization is performed using the C14N algorithm (http://www.w3.org/TR/2001/REC-xml-c14n-20010315). 3. Signatures are created using RSA-SHA1 (http://www.w3.org/2000/09/xmldsig#rsa-sha1). 4. Digests are created using SHA1 (http://www.w3.org/2000/09/xmldsig#sha1). ### XML-DSIG Structure When signing an XML document, the resulting signature will have the following structure: ```xml <Signature xmlns="http://www.w3.org/2000/09/xmldsig#"> <SignedInfo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"/> <SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/> <Reference URI="#elementId"> <Transforms> <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/> </Transforms> <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/> <DigestValue>...</DigestValue> </Reference> </SignedInfo> <SignatureValue>...</SignatureValue> <KeyInfo> <KeyValue> <RSAKeyValue> <Modulus>...</Modulus> <Exponent>...</Exponent> </RSAKeyValue> </KeyValue> <X509Data> <X509Certificate>...</X509Certificate> </X509Data> </KeyInfo> </Signature> ``` ## Integration with Other Derafu Libraries This library is designed to work seamlessly with other Derafu libraries: - **Derafu Certificate**: For handling digital certificates and key pairs. - **Derafu XML**: For handling XML documents and operations. --- ### Signature Class Signature Class # Signature Class The `Signature` class is the core component of the Derafu Signature library, representing an XML digital signature node according to the XML-DSIG standard. ## Overview The `Signature` class implements the `SignatureInterface` and represents the XML `<Signature>` element that contains all the information related to a digital signature in an XML document: - The digest value of the signed content. - The signature value. - The public key information for signature verification. This class is used both when creating new signatures and when validating existing ones. ## Usage ### Creating a New Signature Node ```php use Derafu\Signature\Signature; use Derafu\Certificate\Service\CertificateLoader; // Load a certificate. $certificateLoader = new CertificateLoader(); $certificate = $certificateLoader->loadFromFile( '/path/to/certificate.p12', 'password' ); // Create a signature node. $signatureNode = new Signature(); // Configure with digest value, certificate, and optional reference. $signatureNode->configureSignatureData( digestValue: 'bWFpbkRpZ2VzdFZhbHVlQmFzZTY0', certificate: $certificate, reference: 'documentId' // Optional, specify to sign a specific element. ); ``` ### Working with an Existing Signature Node ```php // Typically you'd use the SignatureValidator to create a signature node from XML. $signatureNode = $signatureService->createSignatureNode($signatureXml); // Access signature properties. $reference = $signatureNode->getReference(); $digestValue = $signatureNode->getDigestValue(); $signatureValue = $signatureNode->getSignatureValue(); $x509Certificate = $signatureNode->getX509Certificate(); ``` ## API Reference ### Setting and Getting Data ```php // Set raw data structure. $signatureNode->setData($dataArray); // Get the current data structure. $dataArray = $signatureNode->getData(); ``` ### Configuring the Signature ```php // Configure all signature components at once. $signatureNode->configureSignatureData( digestValue: 'base64DigestValue', certificate: $certificate, reference: 'elementId' ); ``` ### Working with XML ```php // Set the XML representation of the signature. $signatureNode->setXml($xmlDocument); // Get the XML representation of the signature. $xmlDocument = $signatureNode->getXml(); ``` ### Setting Signature Value ```php // Set the calculated signature value (after signing the SignedInfo element). $signatureNode->setSignatureValue('base64SignatureValue'); ``` ### Getting Signature Components ```php // Get the reference URI (without # prefix). $reference = $signatureNode->getReference(); // Get the digest value. $digestValue = $signatureNode->getDigestValue(); // Get the X.509 certificate (without headers/footers). $certificate = $signatureNode->getX509Certificate(); // Get the signature value. $signatureValue = $signatureNode->getSignatureValue(); ``` ## Data Structure The `Signature` class maintains an internal data array that represents the XML structure of the signature. This structure follows the XML-DSIG standard: ```php [ 'Signature' => [ '@attributes' => [ 'xmlns' => 'http://www.w3.org/2000/09/xmldsig#', ], 'SignedInfo' => [ '@attributes' => [ 'xmlns:xsi' => 'http://www.w3.org/2001/XMLSchema-instance', ], 'CanonicalizationMethod' => [ '@attributes' => [ 'Algorithm' => 'http://www.w3.org/TR/2001/REC-xml-c14n-20010315', ], ], 'SignatureMethod' => [ '@attributes' => [ 'Algorithm' => 'http://www.w3.org/2000/09/xmldsig#rsa-sha1', ], ], 'Reference' => [ '@attributes' => [ 'URI' => '', // Reference URI, empty for entire document. ], 'Transforms' => [ 'Transform' => [ '@attributes' => [ 'Algorithm' => 'http://www.w3.org/2000/09/xmldsig#enveloped-signature', ], ], ], 'DigestMethod' => [ '@attributes' => [ 'Algorithm' => 'http://www.w3.org/2000/09/xmldsig#sha1', ], ], 'DigestValue' => '', // Will contain the digest value. ], ], 'SignatureValue' => '', // Will contain the signature value. 'KeyInfo' => [ 'KeyValue' => [ 'RSAKeyValue' => [ 'Modulus' => '', // Will contain the certificate modulus. 'Exponent' => '', // Will contain the certificate exponent. ], ], 'X509Data' => [ 'X509Certificate' => '', // Will contain the certificate. ], ], ], ] ``` ## Implementation Details ### XML Invalidation The `Signature` class automatically invalidates the XML representation when data is modified: ```php // This will cause the internal XML to be invalidated. $signatureNode->setData($newData); // This will also invalidate the XML. $signatureNode->setSignatureValue($newSignatureValue); ``` After invalidation, the XML must be regenerated (typically by the `SignatureGenerator` class) before `getXml()` can be called again. ### Reference URIs Reference URIs are handled according to the XML-DSIG standard: - An empty URI (`""`) means the entire document is signed. - A URI starting with `#` refers to an element with the specified ID. - The `getReference()` method returns the reference without the `#` prefix. - The `configureSignatureData()` method automatically adds the `#` prefix if not present. ### Transformation Algorithm The transformation algorithm changes based on whether a reference is provided: - With a reference: `http://www.w3.org/TR/2001/REC-xml-c14n-20010315` (standard C14N). - Without a reference: `http://www.w3.org/2000/09/xmldsig#enveloped-signature` (enveloped signature transformation). This ensures that the signature is correctly calculated for both whole-document signatures and element signatures. --- ### Signature Generator SignatureGenerator Class # SignatureGenerator Class The `SignatureGenerator` class is responsible for creating digital signatures, both for general data and specifically for XML documents according to the XML-DSIG standard. ## Overview The `SignatureGenerator` implements the `SignatureGeneratorInterface` and provides mechanisms to: 1. Sign any data using a private key. 2. Sign XML documents using a certificate. 3. Calculate digest values for XML documents or specific elements within them. This class is a core component of the Derafu Signature library and is typically used through the `SignatureService` facade. ## Basic Usage ### Initialization ```php use Derafu\Signature\Service\SignatureGenerator; use Derafu\Xml\Service\XmlDecoder; use Derafu\Xml\Service\XmlEncoder; use Derafu\Xml\Service\XmlService; use Derafu\Xml\Service\XmlValidator; // Initialize required XML services. $xmlEncoder = new XmlEncoder(); $xmlDecoder = new XmlDecoder(); $xmlValidator = new XmlValidator(); $xmlService = new XmlService($xmlEncoder, $xmlDecoder, $xmlValidator); // Create the signature generator. $generator = new SignatureGenerator($xmlService); ``` ### Signing Data ```php // Sign data with a private key. $data = 'Data to be signed'; $signature = $generator->sign($data, $privateKey); // Optional: Specify a different signature algorithm. $signature = $generator->sign($data, $privateKey, OPENSSL_ALGO_SHA256); ``` ### Signing XML Documents ```php use Derafu\Certificate\Service\CertificateLoader; // Load a certificate. $loader = new CertificateLoader(); $certificate = $loader->loadFromFile('/path/to/certificate.p12', 'password'); // Sign an XML string. $xmlString = '<root><element>data</element></root>'; $signedXml = $generator->signXml($xmlString, $certificate); // Sign a specific element in the XML (identified by ID). $xmlWithIds = '<root><element ID="myElement">data</element></root>'; $signedXml = $generator->signXml($xmlWithIds, $certificate, 'myElement'); ``` ### Calculating Digest Values ```php // Create an XML document. $xmlDoc = new \Derafu\Xml\XmlDocument(); $xmlDoc->loadXml('<root><element ID="myElement">data</element></root>'); // Calculate digest value for the entire document. $digestValue = $generator->generateXmlDigestValue($xmlDoc); // Calculate digest value for a specific element. $digestValue = $generator->generateXmlDigestValue($xmlDoc, 'myElement'); ``` ## API Reference ### Data Signing ```php /** * Sign the provided data using a private key. * * @param string $data Data to be signed. * @param string $privateKey Private key to be used for signing. * @param string|int $signatureAlgorithm Algorithm to be used for signing (default SHA1). * @return string Digital signature in base64. * @throws SignatureException If the signing operation fails. */ public function sign( string $data, string $privateKey, string|int $signatureAlgorithm = OPENSSL_ALGO_SHA1 ): string; ``` ### XML Signing ```php /** * Sign an XML document using RSA and SHA1. * * @param XmlDocumentInterface|string $xml XML document to be signed. * @param CertificateInterface $certificate Digital certificate to be used for signing. * @param ?string $reference Reference to which the signature is made. If not * specified, the digest of the entire XML document will be signed. * @return string String XML with the generated signature included in the * "Signature" tag at the end of the XML (last element within the root node). * @throws SignatureException If any problem occurs while signing. */ public function signXml( XmlDocumentInterface|string $xml, CertificateInterface $certificate, ?string $reference = null ): string; ``` ### Digest Value Generation ```php /** * Generate the SHA1 ("DigestValue") of a node of the XML with a certain * reference. This can be used later to generate the XML signature. * * If no reference is specified, the "DigestValue" will be calculated over the * entire XML (root node). * * @param XmlDocumentInterface $doc XML document to be signed. * @param ?string $reference Reference to which the signature is made. * @return string Data of the XML that must be digested. * @throws XmlException If the reference is not found in the XML. */ public function generateXmlDigestValue( XmlDocumentInterface $doc, ?string $reference = null ): string; ``` ## Implementation Details ### Digital Signature Process For general data, the signing process is straightforward: 1. The data is signed using the private key and the specified algorithm. 2. The resulting binary signature is base64-encoded. 3. The base64-encoded signature is returned. ### XML Signature Process For XML documents, the signing process follows the XML-DSIG standard: 1. The XML document is loaded and parsed. 2. If a reference is provided, the referenced element is located. 3. The digest value (SHA1 hash) of the canonicalized (C14N) content is calculated. 4. A `Signature` node is created with the digest value and certificate information. 5. The `SignedInfo` element of the signature is canonicalized. 6. The canonicalized `SignedInfo` is signed using the certificate's private key. 7. The signature value is added to the `SignatureValue` element. 8. The complete signature node is added to the XML document. 9. The signed XML document is returned. ### Canonicalization The class uses the `C14NWithIso88591Encoding` method for canonicalization, which: 1. Applies XML canonicalization (C14N) according to the W3C standard. 2. Converts the result to ISO-8859-1 encoding. 3. Ensures consistent representation across different systems. This process is crucial for ensuring that the same signature is generated regardless of the XML document's formatting or encoding. ### Reference Handling When a reference is provided: 1. The reference must be an ID attribute value in the XML document. 2. The digest is calculated only for the referenced element. 3. The signature's `Reference` element includes a URI attribute (`#elementId`). 4. The transform algorithm is set to standard C14N. When no reference is provided: 1. The digest is calculated for the entire document (excluding any existing signature). 2. The signature's `Reference` element has an empty URI attribute. 3. The transform algorithm is set to "enveloped signature transformation". ### Signature Node Creation The class creates a `Signature` node with the following components: 1. `SignedInfo`: Contains information about what data was signed. - `CanonicalizationMethod`: Specifies the C14N algorithm. - `SignatureMethod`: Specifies RSA-SHA1. - `Reference`: Points to the signed content and includes the digest value. 2. `SignatureValue`: Contains the actual signature of the `SignedInfo` element. 3. `KeyInfo`: Contains information about the certificate used for signing. - `KeyValue/RSAKeyValue`: Contains the modulus and exponent from the certificate. - `X509Data/X509Certificate`: Contains the certificate itself. --- ### Signature Validator SignatureValidator Class # SignatureValidator Class The `SignatureValidator` class is responsible for validating digital signatures, both for general data and specifically for XML documents that follow the XML-DSIG standard. ## Overview The `SignatureValidator` implements the `SignatureValidatorInterface` and provides mechanisms to: 1. Validate signatures for any data against a public key. 2. Validate signatures in XML documents. 3. Extract and parse signature nodes from XML documents. 4. Validate specific aspects of XML signatures (digest values and signature values). This class is a core component of the Derafu Signature library and is typically used through the `SignatureService` facade. ## Basic Usage ### Initialization ```php use Derafu\Signature\Service\SignatureGenerator; use Derafu\Signature\Service\SignatureValidator; use Derafu\Xml\Service\XmlDecoder; use Derafu\Xml\Service\XmlEncoder; use Derafu\Xml\Service\XmlService; use Derafu\Xml\Service\XmlValidator; // Initialize required XML services. $xmlEncoder = new XmlEncoder(); $xmlDecoder = new XmlDecoder(); $xmlValidator = new XmlValidator(); $xmlService = new XmlService($xmlEncoder, $xmlDecoder, $xmlValidator); // Create the signature generator (needed by validator). $generator = new SignatureGenerator($xmlService); // Create the signature validator. $validator = new SignatureValidator($generator, $xmlService); ``` ### Validating Data Signatures ```php // Validate a signature for some data. $data = 'Data that was signed'; $signature = 'base64EncodedSignature'; $publicKey = 'publicKeyPEM'; $isValid = $validator->validate($data, $signature, $publicKey); // Optional: Specify the signature algorithm. $isValid = $validator->validate($data, $signature, $publicKey, OPENSSL_ALGO_SHA256); if ($isValid) { echo "Signature is valid!"; } else { echo "Signature is invalid!"; } ``` ### Validating XML Signatures ```php use Derafu\Signature\Exception\SignatureException; // Validate an XML document with a signature. $signedXml = file_get_contents('signed_document.xml'); try { $validator->validateXml($signedXml); echo "XML signature is valid!"; } catch (SignatureException $e) { echo "XML signature validation failed: " . $e->getMessage(); } ``` ### Working with Signature Nodes ```php // Extract and parse a signature node from XML. $signatureXml = '<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">...</Signature>'; $signatureNode = $validator->createSignatureNode($signatureXml); // Access signature components. $reference = $signatureNode->getReference(); $digestValue = $signatureNode->getDigestValue(); $signatureValue = $signatureNode->getSignatureValue(); $x509Certificate = $signatureNode->getX509Certificate(); ``` ### Detailed XML Signature Validation ```php // Create a signature node from a signed XML document. $signatureNode = $validator->createSignatureNode($signatureXml); // Validate just the digest value (content integrity). try { $validator->validateXmlDigestValue($xmlDoc, $signatureNode); echo "Content integrity verified!"; } catch (SignatureException $e) { echo "Content may have been tampered with: " . $e->getMessage(); } // Validate just the signature value (signer authenticity). try { $validator->validateXmlSignatureValue($signatureNode); echo "Signature authenticity verified!"; } catch (SignatureException $e) { echo "Signature validation failed: " . $e->getMessage(); } ``` ## API Reference ### Data Signature Validation ```php /** * Validate the digital signature of data. * * @param string $data Data to be verified. * @param string $signature Digital signature of the data in base64. * @param string $publicKey Public key of the signature of the data. * @param string|int $signatureAlgorithm Algorithm used to sign (default SHA1). * @return bool `true` if the signature is valid, `false` if it is invalid. * @throws SignatureException If there was an error while validating. */ public function validate( string $data, string $signature, string $publicKey, string|int $signatureAlgorithm = OPENSSL_ALGO_SHA1 ): bool; ``` ### XML Signature Validation ```php /** * Validate the validity of an XML signature using RSA and SHA1. * * @param XmlDocumentInterface|string $xml XML string to be validated. * @return void * @throws SignatureException If there was an error while validating. */ public function validateXml(XmlDocumentInterface|string $xml): void; ``` ### Signature Node Handling ```php /** * Creates the `Signature` instance from a string XML with the signature node. * * @param string $xml String with the XML of the `Signature` node. * @return SignatureInterface */ public function createSignatureNode(string $xml): SignatureInterface; ``` ### Detailed Validation Methods ```php /** * Validate the DigestValue of the signed data. * * @param XmlDocumentInterface|string $xml Document to be validated. * @param SignatureInterface $signatureNode Signature node to be validated. * @return void * @throws SignatureException If the DigestValue is invalid. */ public function validateXmlDigestValue( XmlDocumentInterface|string $xml, SignatureInterface $signatureNode ): void; /** * Validate the signature of the `SignedInfo` node of the XML using the X509 * certificate. * * @param SignatureInterface $signatureNode Signature node to be validated. * @throws SignatureException If the XML signature is invalid. */ public function validateXmlSignatureValue( SignatureInterface $signatureNode ): void; ``` ## Implementation Details ### Signature Validation Process For general data signatures, the validation process is as follows: 1. The public key is normalized (headers and footers are added if missing). 2. The base64-encoded signature is decoded to binary. 3. The `openssl_verify` function is called with the data, decoded signature, and public key. 4. If the result is 1, the signature is valid; if 0, it's invalid; if -1, an error occurred. ### XML Signature Validation Process For XML signatures, the validation process follows the XML-DSIG standard: 1. The XML document is loaded and parsed. 2. All `Signature` elements in the document are located. 3. For each signature element: - A `Signature` node object is created from the element's XML. - The digest value is validated to ensure content integrity. - The signature value is validated to ensure signer authenticity. ### Digest Value Validation The digest value validation ensures that the content being signed hasn't been modified: 1. The XML document is loaded. 2. The reference from the signature node is extracted (if any). 3. The digest value is calculated for the referenced content or the entire document. 4. The calculated digest value is compared to the one in the signature. 5. If they don't match, a `SignatureException` is thrown. ### Signature Value Validation The signature value validation ensures that the signature was created with the corresponding private key: 1. The `SignedInfo` element is extracted and canonicalized. 2. The base64-encoded signature value is obtained from the `SignatureValue` element. 3. The public key is extracted from the `X509Certificate` element. 4. The signature is validated using the canonicalized `SignedInfo`, the signature value, and the public key. 5. If the validation fails, a `SignatureException` is thrown. ### Error Handling The `SignatureValidator` provides detailed error messages when validation fails: - For data signatures, it indicates when an error occurred during validation. - For XML signatures, it indicates whether the problem is with the digest value (content integrity) or the signature value (signer authenticity). - For missing signatures or malformed XML, it provides clear error messages. These detailed error messages help diagnose the exact cause of validation failures. ### Dependency on SignatureGenerator The `SignatureValidator` requires an instance of `SignatureGeneratorInterface` to calculate digest values for XML documents. This dependency ensures that the same digest calculation algorithm is used for both signing and validating. ### XML Handling The class uses the Derafu XML library for XML operations: - Loading and parsing XML documents. - Extracting signature elements. - Canonicalizing XML for digest and signature validation. - Converting between XML and PHP arrays. This integration ensures consistent handling of XML across the library. --- ### Signature Service SignatureService Class # SignatureService Class The `SignatureService` class is the main entry point for working with digital signatures in the Derafu Signature library. It provides a unified interface for generating and validating signatures for both general data and XML documents. ## Overview The `SignatureService` implements the `SignatureServiceInterface`, which combines the functionality of: - `SignatureGeneratorInterface`: For creating digital signatures. - `SignatureValidatorInterface`: For validating digital signatures. This service acts as a facade over the signature generation and validation components, making it easy to use the library's full functionality through a single interface. ## Basic Usage ### Setting Up the Service ```php use Derafu\Signature\Service\SignatureGenerator; use Derafu\Signature\Service\SignatureService; use Derafu\Signature\Service\SignatureValidator; use Derafu\Xml\Service\XmlDecoder; use Derafu\Xml\Service\XmlEncoder; use Derafu\Xml\Service\XmlService; use Derafu\Xml\Service\XmlValidator; // Initialize required XML services. $xmlEncoder = new XmlEncoder(); $xmlDecoder = new XmlDecoder(); $xmlValidator = new XmlValidator(); $xmlService = new XmlService($xmlEncoder, $xmlDecoder, $xmlValidator); // Create generator and validator. $generator = new SignatureGenerator($xmlService); $validator = new SignatureValidator($generator, $xmlService); // Create the signature service. $signatureService = new SignatureService($generator, $validator); ``` ### Signing Data ```php // Sign simple data with a private key. $data = 'Data to be signed'; $signature = $signatureService->sign($data, $privateKey); // Optional: Specify a different signature algorithm. $signature = $signatureService->sign($data, $privateKey, OPENSSL_ALGO_SHA256); ``` ### Validating a Signature ```php // Validate a signature using a public key. $isValid = $signatureService->validate($data, $signature, $publicKey); // Optional: Specify the same signature algorithm used for signing. $isValid = $signatureService->validate($data, $signature, $publicKey, OPENSSL_ALGO_SHA256); if ($isValid) { echo "Signature is valid!"; } else { echo "Signature is invalid!"; } ``` ### Signing XML ```php use Derafu\Certificate\Service\CertificateLoader; use Derafu\Xml\XmlDocument; // Load a certificate. $loader = new CertificateLoader(); $certificate = $loader->loadFromFile('/path/to/certificate.p12', 'password'); // Sign an XML string. $xmlString = '<root><element>data</element></root>'; $signedXml = $signatureService->signXml($xmlString, $certificate); // Sign an XmlDocument object. $xmlDoc = new XmlDocument(); $xmlDoc->loadXml($xmlString); $signedXml = $signatureService->signXml($xmlDoc, $certificate); // Sign a specific element in the XML (identified by ID). $xmlWithIds = '<root><element ID="myElement">data</element></root>'; $signedXml = $signatureService->signXml($xmlWithIds, $certificate, 'myElement'); ``` ### Validating XML Signatures ```php use Derafu\Signature\Exception\SignatureException; try { // Validate a signed XML document. $signatureService->validateXml($signedXml); echo "XML signature is valid!"; } catch (SignatureException $e) { echo "XML signature validation failed: " . $e->getMessage(); } ``` ## Advanced Usage ### Calculating Digest Values ```php // Calculate the digest value for an XML document. $digestValue = $signatureService->generateXmlDigestValue($xmlDoc); // Calculate the digest value for a specific element. $digestValue = $signatureService->generateXmlDigestValue($xmlDoc, 'elementId'); ``` ### Working with Signature Nodes ```php // Extract signature node from a signed XML. $signatureXml = $signedXml; // The signature node XML. $signatureNode = $signatureService->createSignatureNode($signatureXml); // Validate just the digest value (content integrity). try { $signatureService->validateXmlDigestValue($xmlDoc, $signatureNode); echo "XML content integrity verified!"; } catch (SignatureException $e) { echo "Digest validation failed: " . $e->getMessage(); } // Validate just the signature value (signer authenticity). try { $signatureService->validateXmlSignatureValue($signatureNode); echo "Signature authenticity verified!"; } catch (SignatureException $e) { echo "Signature validation failed: " . $e->getMessage(); } ``` ## API Reference ### Data Signing and Validation ```php // Generate a digital signature for data. public function sign( string $data, string $privateKey, string|int $signatureAlgorithm = OPENSSL_ALGO_SHA1 ): string; // Validate a digital signature for data. public function validate( string $data, string $signature, string $publicKey, string|int $signatureAlgorithm = OPENSSL_ALGO_SHA1 ): bool; ``` ### XML Signing and Validation ```php // Sign an XML document. public function signXml( XmlDocumentInterface|string $xml, CertificateInterface $certificate, ?string $reference = null ): string; // Validate an XML signature. public function validateXml(XmlDocumentInterface|string $xml): void; // Calculate the digest value for an XML document or element. public function generateXmlDigestValue( XmlDocumentInterface $doc, ?string $reference = null ): string; ``` ### Signature Node Operations ```php // Create a signature node from XML. public function createSignatureNode(string $xml): SignatureInterface; // Validate the digest value of a signature. public function validateXmlDigestValue( XmlDocumentInterface|string $xml, SignatureInterface $signatureNode ): void; // Validate the signature value of a signature. public function validateXmlSignatureValue( SignatureInterface $signatureNode ): void; ``` ## Implementation Details ### Dependency Injection The `SignatureService` uses dependency injection to receive its required components: ```php public function __construct( private readonly SignatureGeneratorInterface $generator, private readonly SignatureValidatorInterface $validator ) ``` This design allows for flexibility and testability, as the generator and validator components can be replaced with custom implementations if needed. ### Delegation Pattern The service uses the delegation pattern to forward method calls to the appropriate components: - Signing methods are delegated to the `SignatureGeneratorInterface` implementation. - Validation methods are delegated to the `SignatureValidatorInterface` implementation. This separation of concerns keeps the codebase clean and maintainable. ### Error Handling Validation methods throw `SignatureException` when validation fails, providing detailed error messages about what went wrong: - Invalid digest values (content has been modified). - Invalid signature values (signature was not created with the expected private key). - Missing signature nodes. - Malformed XML. These exceptions should be caught and handled appropriately in your application code. --- ### XML-DSIG Standard XML Digital Signature (XML-DSIG) # XML Digital Signature (XML-DSIG) This document provides an in-depth explanation of how the Derafu Signature library implements the XML Digital Signature (XML-DSIG) standard, including the structure of signature elements, the signing process, and the validation process. ## XML-DSIG Overview XML Digital Signatures provide integrity, message authentication, and signer authentication for XML data. The XML-DSIG standard is defined by the W3C in the [XML Signature Syntax and Processing](https://www.w3.org/TR/xmldsig-core/) specification. ## Signature Structure The Derafu Signature library creates XML signatures with the following structure: ```xml <Signature xmlns="http://www.w3.org/2000/09/xmldsig#"> <SignedInfo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"/> <SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/> <Reference URI="#elementId"> <Transforms> <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/> </Transforms> <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/> <DigestValue>base64EncodedDigestValue</DigestValue> </Reference> </SignedInfo> <SignatureValue>base64EncodedSignatureValue</SignatureValue> <KeyInfo> <KeyValue> <RSAKeyValue> <Modulus>base64EncodedModulus</Modulus> <Exponent>base64EncodedExponent</Exponent> </RSAKeyValue> </KeyValue> <X509Data> <X509Certificate>base64EncodedCertificate</X509Certificate> </X509Data> </KeyInfo> </Signature> ``` ### Key Components 1. **SignedInfo**: Contains information about what data is being signed. - **CanonicalizationMethod**: Specifies how the XML is normalized before signing. - **SignatureMethod**: Specifies the algorithm used for signing (RSA-SHA1). - **Reference**: Points to the data being signed. - **Transforms**: Describes transformations applied to the data before digesting. - **DigestMethod**: Specifies the algorithm used for the digest (SHA1). - **DigestValue**: Contains the base64-encoded digest of the data. 2. **SignatureValue**: Contains the base64-encoded signature of the canonicalized SignedInfo element. 3. **KeyInfo**: Contains information about the key used to validate the signature. - **KeyValue/RSAKeyValue**: Contains the RSA key parameters (modulus and exponent). - **X509Data/X509Certificate**: Contains the X.509 certificate used for signing. ## Signing Process The Derafu Signature library implements XML signing following these steps: ### 1. Preparing the Data - If a reference ID is specified, the referenced element is located in the XML document. - Otherwise, the entire XML document is used (excluding any existing Signature elements). ### 2. Calculating the Digest Value - The data is canonicalized using the C14N algorithm. - The canonicalized data is converted to ISO-8859-1 encoding. - The SHA1 digest of the data is calculated and base64-encoded. ### 3. Creating the SignedInfo Element - The SignedInfo element is created with the appropriate CanonicalizationMethod, SignatureMethod, and Reference elements. - The DigestValue is included in the Reference element. ### 4. Calculating the Signature Value - The SignedInfo element is canonicalized and converted to ISO-8859-1 encoding. - The canonicalized SignedInfo is signed using the private key from the certificate. - The resulting signature is base64-encoded and included in the SignatureValue element. ### 5. Including Key Information - The public key components (modulus and exponent) are extracted from the certificate. - The certificate itself is included in the X509Certificate element. ### 6. Adding the Signature to the Document - The complete Signature element is added to the XML document, typically as the last child of the root element. ## Validation Process The Derafu Signature library validates XML signatures following these steps: ### 1. Locating Signature Elements - All Signature elements in the XML document are located. - Each signature is validated independently. ### 2. Validating the Digest Value - The reference in the signature is extracted. - The referenced data (or the entire document) is canonicalized and converted to ISO-8859-1 encoding. - The SHA1 digest of the data is calculated and base64-encoded. - The calculated digest is compared to the DigestValue in the signature. - If they don't match, the content integrity check fails. ### 3. Validating the Signature Value - The SignedInfo element is canonicalized and converted to ISO-8859-1 encoding. - The X.509 certificate is extracted from the X509Certificate element. - The signature value is validated using the canonicalized SignedInfo, the SignatureValue, and the public key from the certificate. - If the validation fails, the signer authenticity check fails. ## Reference Types The Derafu Signature library supports two types of references: ### 1. Element References - Specified by a URI attribute with a value starting with `#` (e.g., `URI="#elementId"`). - Only the referenced element is signed. - The transform algorithm is set to standard C14N. ### 2. Whole Document References - Specified by an empty URI attribute (`URI=""`). - The entire document is signed, excluding any Signature elements. - The transform algorithm is set to "enveloped signature transformation". ## Canonicalization The Derafu Signature library uses the following canonicalization algorithms: ### 1. XML Canonicalization (C14N) - Algorithm: `http://www.w3.org/TR/2001/REC-xml-c14n-20010315` - Ensures consistent XML representation regardless of formatting differences. - Applied to data before calculating digests and to SignedInfo before calculating signatures. ### 2. Enveloped Signature Transform - Algorithm: `http://www.w3.org/2000/09/xmldsig#enveloped-signature` - Excludes the signature itself when signing the entire document. - Prevents circular references in the signing process. ## Working with References and IDs When using element references, the referenced element must have an `ID` attribute: ```xml <root> <element ID="myElement">data</element> </root> ``` The reference in the signature would be: ```xml <Reference URI="#myElement"> <!-- ... --> </Reference> ``` The `SignatureGenerator.signXml()` method accepts the ID as a parameter: ```php $signedXml = $signatureGenerator->signXml($xml, $certificate, 'myElement'); ``` ## Security Considerations ### Digest Algorithm The library uses SHA1 for digest calculation. While SHA1 is considered cryptographically weak for certain applications, it remains the standard algorithm specified in the XML-DSIG specification. ### Signature Algorithm The library uses RSA-SHA1 for signature calculation, which is the standard algorithm specified in the XML-DSIG specification. ### Certificate Handling The library includes the entire X.509 certificate in the signature, which allows for complete validation including certificate chain verification (although this is not currently implemented in the library). ## Compatibility The XML signatures generated by the Derafu Signature library follow the W3C XML-DSIG standard and should be compatible with other XML-DSIG implementations. However, different implementations may have subtle differences in canonicalization or other aspects of the signing process. ## Common Issues ### Invalid References If a reference ID is specified but doesn't exist in the XML document, the signing process will fail. ### Malformed XML If the XML document is not well-formed, both signing and validation will fail. ### Character Encoding The library uses ISO-8859-1 encoding for canonicalized XML to ensure consistent digest and signature calculation across different systems. ## Example Usage ### Signing with a Reference ```php $xml = '<root><element ID="myElement">data</element></root>'; $signedXml = $signatureService->signXml($xml, $certificate, 'myElement'); ``` ### Signing the Entire Document ```php $xml = '<root><element>data</element></root>'; $signedXml = $signatureService->signXml($xml, $certificate); ``` ### Validating a Signature ```php try { $signatureService->validateXml($signedXml); echo "Signature is valid!"; } catch (SignatureException $e) { echo "Signature validation failed: " . $e->getMessage(); } ``` --- Last updated on 09/09/2026