2017 © Pedro Peláez
 

library compiler

The Hoa\Compiler library.

image

hoa/compiler

The Hoa\Compiler library.

  • Tuesday, March 6, 2018
  • by Hoa
  • Repository
  • 17 Watchers
  • 160 Stars
  • 874,155 Installations
  • PHP
  • 15 Dependents
  • 0 Suggesters
  • 30 Forks
  • 11 Open issues
  • 18 Versions
  • 6 % Grown

The README.md

Hoa , (*1)

Build status Code coverage Packagist License , (*2)

Hoa is a modular, extensible and structured set of PHP libraries.
Moreover, Hoa aims at being a bridge between industrial and research worlds. , (*3)

Hoa\Compiler

Help on IRC Help on Gitter Documentation Board, (*4)

This library allows to manipulate LL(1) and LL(k) compiler compilers. A dedicated grammar description language is provided for the last one: the PP language., (*5)

Learn more., (*6)

Installation

With Composer, to include this library into your dependencies, you need to require hoa/compiler:, (*7)

$ composer require hoa/compiler '~3.0'

For more installation procedures, please read the Source page., (*8)

Testing

Before running the test suites, the development dependencies must be installed:, (*9)

$ composer install

Then, to run all the test suites:, (*10)

$ vendor/bin/hoa test:run

For more information, please read the contributor guide., (*11)

Quick usage

As a quick overview, we will look at the PP language and the LL(k) compiler compiler., (*12)

The PP language

A grammar is constituted by tokens (the units of a word) and rules (please, see the documentation for an introduction to the language theory). The PP language declares tokens with the following construction:, (*13)

%token [source_namespace:]name value [-> destination_namespace]

The default namespace is default. The value of a token is represented by a PCRE. We can skip tokens with the %skip construction., (*14)

As an example, we will take the simplified grammar of the JSON language. The complete grammar is in the hoa://Library/Json/Grammar.pp file. Thus:, (*15)

%skip   space          \s
// Scalars.
%token  true           true
%token  false          false
%token  null           null
// Strings.
%token  quote_         "        -> string
%token  string:string  [^"]+
%token  string:_quote  "        -> default
// Objects.
%token  brace_         {
%token _brace          }
// Arrays.
%token  bracket_       \[
%token _bracket        \]
// Rest.
%token  colon          :
%token  comma          ,
%token  number         \d+

value:
    <true> | <false> | <null> | string() | object() | array() | number()

string:
    ::quote_:: <string> ::_quote::

number:
    <number>

#object:
    ::brace_:: pair() ( ::comma:: pair() )* ::_brace::

#pair:
    string() ::colon:: value()

#array:
    ::bracket_:: value() ( ::comma:: value() )* ::_bracket::

We can see the PP constructions:, (*16)

  • rule() to call a rule;
  • <token> and ::token:: to declare a token;
  • | for a disjunction;
  • (…) to group multiple declarations;
  • e? to say that e is optional;
  • e+ to say that e can appear at least 1 time;
  • e* to say that e can appear 0 or many times;
  • e{x,y} to say that e can appear between x and y times;
  • #node to create a node the AST (resulting tree);
  • token[i] to unify tokens value between them.

Unification is very useful. For example, if we have a token that expresses a quote (simple or double), we could have:, (*17)

%token  quote   "|'
%token  handle  \w+

string:
    ::quote:: <handle> ::quote::

So, the data "foo" and 'foo' will be valid, but also "foo' and 'foo"! To avoid this, we can add a new constraint on token value by unifying them, thus:, (*18)

string:
    ::quote[0]:: <handle> ::quote[0]::

All quote[0] for the rule instance must have the same value. Another example is the unification of XML tags name., (*19)

LL(k) compiler compiler

The Hoa\Compiler\Llk\Llk class provide helpers to manipulate (load or save) a compiler. The following code will use the previous grammar to create a compiler, and we will parse a JSON string. If the parsing succeed, it will produce an AST (stands for Abstract Syntax Tree) we can visit, for example to dump the AST:, (*20)

// 1. Load grammar.
$compiler = Hoa\Compiler\Llk\Llk::load(new Hoa\File\Read('Json.pp'));

// 2. Parse a data.
$ast = $compiler->parse('{"foo": true, "bar": [null, 42]}');

// 3. Dump the AST.
$dump = new Hoa\Compiler\Visitor\Dump();
echo $dump->visit($ast);

/**
 * Will output:
 *     >  #object
 *     >  >  #pair
 *     >  >  >  token(string, foo)
 *     >  >  >  token(true, true)
 *     >  >  #pair
 *     >  >  >  token(string, bar)
 *     >  >  >  #array
 *     >  >  >  >  token(null, null)
 *     >  >  >  >  token(number, 42)
 */

Pretty simple., (*21)

Compiler in CLI

This library proposes a script to parse and apply a visitor on a data with a specific grammar. Very useful. Moreover, we can use pipe (because Hoa\File\Read —please, see the Hoa\File library— supports 0 as stdin), thus:, (*22)

$ echo '[1, [1, [2, 3], 5], 8]' | hoa compiler:pp Json.pp 0 --visitor dump
>  #array
>  >  token(number, 1)
>  >  #array
>  >  >  token(number, 1)
>  >  >  #array
>  >  >  >  token(number, 2)
>  >  >  >  token(number, 3)
>  >  >  token(number, 5)
>  >  token(number, 8)

You can apply any visitor classes., (*23)

Errors

Errors are well-presented:, (*24)

$ echo '{"foo" true}' | hoa compiler:pp Json.pp 0 --visitor dump
Uncaught exception (Hoa\Compiler\Exception\UnexpectedToken):
Hoa\Compiler\Llk\Parser::parse(): (0) Unexpected token "true" (true) at line 1
and column 8:
{"foo" true}
       ↑
in hoa://Library/Compiler/Llk/Parser.php at line 1

Samplers

Some algorithms are available to generate data based on a grammar. We will give only one example with the coverage-based generation algorithm that will activate all branches and tokens in the grammar:, (*25)

$sampler = new Hoa\Compiler\Llk\Sampler\Coverage(
    // Grammar.
    Hoa\Compiler\Llk\Llk::load(new Hoa\File\Read('Json.pp')),
    // Token sampler.
    new Hoa\Regex\Visitor\Isotropic(new Hoa\Math\Sampler\Random())
);

foreach ($sampler as $i => $data) {
    echo $i, ' => ', $data, "\n";
}

/**
 * Will output:
 *     0 => true
 *     1 => {" )o?bz " : null , " %3W) " : [false, 130    , " 6"   ]  }
 *     2 => [{" ny  " : true } ]
 *     3 => {" Ne;[3 " :[ true , true ] , " th: " : true," C[8} " :   true }
 */

Research papers

Documentation

The hack book of Hoa\Compiler contains detailed information about how to use this library and how it works., (*26)

To generate the documentation locally, execute the following commands:, (*27)

$ composer require --dev hoa/devtools
$ vendor/bin/hoa devtools:documentation --open

More documentation can be found on the project's website: hoa-project.net., (*28)

Getting help

There are mainly two ways to get help:, (*29)

Contribution

Do you want to contribute? Thanks! A detailed contributor guide explains everything you need to know., (*30)

License

Hoa is under the New BSD License (BSD-3-Clause). Please, see LICENSE for details., (*31)

The Versions

06/03 2018

dev-master

9999999-dev https://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

The Development Requires

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

08/08 2017

3.17.08.08

3.17.08.08 https://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

The Development Requires

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

10/01 2017

3.17.01.10

3.17.01.10 https://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

The Development Requires

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

24/10 2016

3.16.10.24

3.16.10.24 https://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

The Development Requires

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

15/08 2016

3.16.08.15

3.16.08.15 http://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

The Development Requires

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

14/01 2016

3.16.01.14

3.16.01.14 http://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

The Development Requires

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

11/01 2016

3.16.01.11

3.16.01.11 http://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

The Development Requires

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

29/10 2015

2.15.10.29

2.15.10.29 http://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

The Development Requires

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

21/10 2015

2.15.10.21

2.15.10.21 http://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

The Development Requires

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

25/08 2015

2.15.08.25

2.15.08.25 http://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

The Development Requires

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

29/05 2015

2.15.05.29

2.15.05.29 http://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

The Development Requires

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

17/02 2015

2.15.02.17

2.15.02.17 http://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

The Development Requires

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

09/12 2014

2.14.12.10

2.14.12.10 http://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

The Development Requires

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

25/11 2014

2.14.11.26

2.14.11.26 http://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

The Development Requires

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

09/11 2014

2.14.11.09

2.14.11.09 http://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

The Development Requires

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

23/09 2014

2.14.09.23

2.14.09.23 http://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

17/09 2014

2.14.09.17

2.14.09.17 http://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk

16/09 2014

1.14.09.16

1.14.09.16 http://hoa-project.net/

The Hoa\Compiler library.

  Sources   Download

BSD-3-Clause

The Requires

 

by Ivan Enderlin
by Hoa community

language parser library token lexer syntax random trace compiler coverage rule ast grammar context-free regular sampler algebraic pp isotropic uniform exhaustive ll1 llk