Parser Grammar Engine


The Parser Grammar Engine is a compiler and runtime for Raku rules for the Parrot virtual machine. PGE uses these rules to convert a parsing expression grammar into Parrot bytecode. It is therefore compiling rules into a program, unlike most virtual machines and runtimes, which store regular expressions in a secondary internal format that is then interpreted at runtime by a regular expression engine. The rules format used by PGE can express any regular expression and most formal grammars, and as such it forms the first link in the compiler chain for all of Parrot's front-end languages.
When executed, the bytecode generated by PGE will parse text as described in the input rules, generating a parse tree. The parse tree can be manipulated directly, or fed into the next stage of the Parrot compiler toolchain in order to generate an AST from which code generation can occur.

History

Originally named P6GE and written in C, PGE was translated to native Parrot and renamed not long after its initial release in November 2004. Its author is Patrick R. Michaud. PGE was written in order to reduce the amount of work required to implement a compiler on top of Parrot. It was also written to allow Perl 6 to easily self-host, though current Pugs development no longer uses PGE as its primary rules back-end in favor of a native engine called PCR.

Internals

PGE combines three styles of parsing:
The primary form is Raku rules, so a PGE rule might look like this for an addition-only grammar:

rule term
rule number
rule expr

The operator precedence parser allows an operator table to be built and used directly in a Perl 6 rule style parser like so:

rule expr is optable
rule term
rule number
proto term: is precedence
is parsed
proto infix:+ is looser

This accomplishes the same goal of defining a simple, addition-only grammar, but does so using a combination of a Raku style regex/rules for term and number and a shift-reduce optable for everything else.

Code generation

Though PGE outputs code which will parse the grammar described by a rule, and can be used at run time to handle simple grammars and regular expressions found in code, its primary purpose is for the parsing of high level languages.
The Parrot compiler toolchain is broken into several parts, of which PGE is the first. PGE converts source code to parse trees. The Tree Grammar Engine then converts these into Parrot Abstract Syntax Trees. A second TGE pass then converts a PAST into Parrot Opcode Syntax Trees which can be directly transformed into executable bytecode.