diff options
| author | Jinkun Jang <jinkun.jang@samsung.com> | 2013-03-13 01:44:50 +0900 |
|---|---|---|
| committer | Jinkun Jang <jinkun.jang@samsung.com> | 2013-03-13 01:44:50 +0900 |
| commit | 7e13e5fc62b81ffc5728006ddb560478ac3f1576 (patch) | |
| tree | 813d127306e4518e2ab20cf03f7dea33d14a4210 | |
| parent | 12056158053532946b53b6249cb0e6cfd4580051 (diff) | |
| download | ragel-7e13e5fc62b81ffc5728006ddb560478ac3f1576.tar.gz ragel-7e13e5fc62b81ffc5728006ddb560478ac3f1576.tar.bz2 ragel-7e13e5fc62b81ffc5728006ddb560478ac3f1576.zip | |
Tizen 2.1 base
306 files changed, 56369 insertions, 3790 deletions
@@ -0,0 +1 @@ +See CREDITS. @@ -2,24 +2,40 @@ Ragel State Machine Compiler -- CREDITS ======================================= -* Written by Adrian Thurston <thurston@cs.queensu.ca>. -* Objective-C output contributed by Eric Ocean. +* Ragel was designed and written by Adrian Thurston <thurston@complang.org>. + +Many others have helped out along the way. My apolgies to anyone who has been +missed. + +* Many thanks to Arbor Networks for supporting development of Ragel. + +* Objective-C output and valuable feedback contributed by Erich Ocean. * D output and many great ideas contributed by Alan West. * Conditionals inspired by David Helder. * Java code generation contributions, bug reports, fixes, test cases - and suggestions from Colin Fleming + and suggestions from Colin Fleming. + +* Useful discussions and bug reports due to Carlos Antunes. + +* Ruby code generation contributed by Victor Hugo Borja. -* Useful discussion and bug from Carlos Antunes. +* C# code generation contributed by Daniel Tang. * Feedback, Packaging, and Fixes provided by: - Bob Tennent, Robert Lemmen, Tobias Jahn, Cris Bailiff, Buddy Betts, Scott - Dixon, Steven Handerson, Michael Somos, Bob Paddock, Istvan Buki, David - Drai, Matthias Rahlf, Zinx Verituse, Markus W. Weissmann, Marc Liyanage, - Eric Ocean, Alan West, Steven Kibbler, Laurent Boulard, Jon Oberheide, - David Helder, Lexington Luthor, Jason Jobe, Colin Fleming, Carlos Antunes, - Steve Horne + Bob Tennent, Robert Lemmen, Tobias Jahn, Cris Bailiff, Buddy Betts, + Scott Dixon, Steven Handerson, Michael Somos, Bob Paddock, Istvan Buki, + David Drai, Matthias Rahlf, Zinx Verituse, Markus W. Weissmann, + Marc Liyanage, Erich Ocean, Alan West, Steven Kibbler, Laurent Boulard, + Jon Oberheide, David Helder, Lexington Luthor, Jason Jobe, Colin Fleming, + Carlos Antunes, Steve Horne, Matt Mower, Josef Goettgens, Zed Shaw, + Marcus Rueckert, Jeremy Hinegardner, Aaron Campbell, Josh Purinton, + Judson Lester, Barry Arthur, Tim Potter, Ryan Phelps, David Waite, + Kenny MacDermid, MenTaLguY, Manoj Rajagopalan, Tim Chklovski, + Mikkel Fahnøe Jørgensen, Andrei Polushin, Evan Phoenix, David Balmain, + Ross Thomas, Mitchell Foral, John D. Mitchell, Diego 'Flameeyes' Pettenò, + Jose Quinteiro, William Morgan, _why, Iñaki Baz Castillo @@ -1,14 +1,427 @@ -For Next Release -================ +Ragel 6.6 - Dec 2, 2009 +======================= + -Applied a number of patches from Diego Elio 'Flameeyes' Pettenò. Should not + be modifying the program's arguments. Problem raised by const correctness in + gcc 4.4. Other const-correctness and include fixes provided. + -Fixed improper allocation of checks in makeIncludePathChecks. + -Fixed segfault when there are no machine instantiations. + -Fixed wrong line directives. Line directives need to use the fileName stored + in the InputLoc stuctures from the parse trees, not the root source file, + otherwise actions in included files will have the wrong source file names + associated with the text. + -Made a number of build system improvements. We locate the DIST file using + $srcdir and source it. It contains settings for build_parsers and + build_manual. This allows the user of a dist to enable only one. + -Added missing files to doc/Makefile.am and examples/Makefile.am. + -Added checks for pdflatex and fig2dev is build_manual is on. + -Use automake --foreign so we don't need to have INSTALL and NEWS present. + -Ragel VIM syntax files should be specialized by host language. Updated the + VIM syntax files. + -Just discovered that $srcdir is available in the configure script for + checking for the existence of a file created by dist-hook. This lets us write + a test that knows the difference between a distribution tarball and something + from the repos. The building of the parsers and the manual can now be + automatically turned off in a make dist tarball. + -Added examples to the dist. Added unicode2ragel.rb to EXTRA_DIST in contrib. + -Moved unicode2ragel.rb to the contrib directory. + +Ragel 6.5 - May 18, 2009 +======================== + -Fixed a bug in graphviz generation. Ragel crashed when using -V and -M and + the specified machine referenced another machine that wasn't included in the + build. + -The name "CS" is in use on OpenSolaris, changed to vCS to ease compiling + Ragel there. + -Converted to automake. + -REALLY fixed a bug that was intended to be fixed in 6.4: + Fixed a problem reading hex numbers that have the high bit set when the + alphabet is signed and we are on 64 bit. This was reported by _why. The + fix was provided by Wialliam Morgan. The literal 0xffffffff was used for + a fully set long when -1L should be used instead. + A null patch (whitespace changes) must have gotten checked after I was + testing with and without the critical one-line patch and I forgot to enable + make sure it was enabled in the final checkin version. + +Ragel 6.4 - Mar 22, 2009 +======================== + -Moved back to a single executable. The old intermediate format can still be + generated using the -x option. Ragel was split into frontend and backend + programs in version 5.0. This was done to encourage interoperability with + other tools. Since then, ragel has been made to work with qfsm, with ragel + producing the intermediate format and qfsm consuming it. However, there has + been no use of Ragel as a consumer of state machine data, with Ragel used as + a code generator for DFAs. This is not surprising given that much of the + complexity of Ragel is in the frontend, where the regular language to DFA + compilation happens. Since the full benefits of the split have not + materialized, and the split increases the complexity for users, Ragel has + been made once again into a single executable. + -Applied a fix to the documentation Makefile from John D. Mitchell. + -Use CXXFLAGS instead of CFLAGS for C++ compiling. Patch from Diego + 'Flameeyes' Pettenò. + -Added support for DESTDIR variable. Patch from Diego 'Flameeyes' Pettenò. + -Added a script called unicode2ragel.rb for generating unicode machines to + the examples directory. From Rakan El-Khalil. + -Fixed a copy-paste error in the documentation that was reported by Jose + Quinteiro. + -Added three new write commands: + write start; + write first_final; + write error; + These generate a reference to the start, first final and error state. When + there are many different machine specifications in one file it is easy to + get the prefix for these wrong (especially when you do a lot of copy-pasting + of boilerplate). The problem can be avoided by using write commands. + -Fixed a problem reading hex numbers that have the high bit set when the + alphabet is signed and we are on 64 bit. This was reported by _why. The fix + was provided by Wialliam Morgan. The literal 0xffffffff was used for a fully + set long when -1L should be used instead. + +Ragel 6.3 - Aug 29, 2008 +======================== + -Fixed an assertion that is too strong. In the condition code we need to copy + transitions that have non-empty lmActionTable arrays so we don't assert + emptiness in the constructor. Lift out the assertion and copy the array in + the constructor. + -Fixed and improved multiple include prevention. We now track the entire + include history of a parser state to prevent duplicates. + -Fixed crash on failed lookup of goto/call/etc target. + +Ragel 6.2 - May 9, 2008 +======================= + -Bug fix: The lm_switch actions needs to set p from tokend when there is no + user action. + -Bug fix: when not using indicies we can't use a transitions's id to identify + the eof transition to take. Instead add the transition to the end of the + transition list and store its position in a new var called pos. The pos var + is then used as the index. + -Bug fix: an fnext followed by an fbreak in -G2 was not working. The fbreak + was not aware that the fnext causes the cs variable to be forced active. In + this case fbreak does not need to save cs because it is already current. + -Bug fix: need to compute the low and high character-space keys from the + condition-trans overlap when computing an expansion. Can't use the range + supplied from the condition overlap since they may not match. An incorrect + machine that accepted 1(!cond1, !cond2) was generated for the following + grammar. This bug was reported by Tim Chklovski. + c = 2 @matched_c; + sc1 = 1..2 when cond1; + sc2 = 1..2 when cond2; + main := sc1 | c | sc2; + -Bug fix: error messages in start label analysis of join operations were + causing assertion failures because location info was not set. Fixed by + adding locations. + -Include and import file searching now searches for the file name given based + on the location of the current file, not ragel's current path. + Additional search locations can be given using the -I option. + -Rubinius code generation was updated to the latest Rubinius. Patch from Evan + Phoenix. + -Switched from strcasecmp to strcmp for testing long arguments. + -Applied a patch from Andrei Polushin for setting the error message format. + --error-format=gnu (default) + --error-fromat=msvc + -Now using the _WIN32 define instead of _WIN32. Other MSVC compilation + improvments from Andrei Polushin. + -Added the hyperref package to the manual. + +Ragel 6.1 - Mar 26, 2008 +======================== + -Scanners now ensure that any leaving actions at the end of a pattern are + executed. They are always executed before the pattern action. + -Added an option -d for turning off the removal of duplicate actions from + actions lists. + -Need to unset the final state status of the start state in kleene star if it + is set. It is possible to crash ragel when the warning is ignored. + -In the dot file generation we need to print any actions that are in + State::eofTrans. These come from scanners only. + -Use @docdir@ for the docdir Makefile variable. + -Check for ar and ranlib in the configure script. + +Ragel 6.0 - Jan 12, 2008 +======================== + -Removed the 'noend' write option from examples/atoi.rl. This example is + referenced a lot as a first example and as such it shouldn't contain a + special purpose write option like 'noend'. + -Introcuded the "eof" variable for indicating the end of file. The p variable + is checked against eof when the processing loop reaches the end of a block. + If p == eof at this time then the EOF actions are executed. The variable is + required only when EOF actions have been emebedded. + -The "write eof" command is no longer needed and was removed. + -Scanners now use EOF actions to generate tokens. This eliminates the need to + flush the last token. + -Restructured the Java driver; a switch statement with fallthrough cases are + now used to emulate gotos. + -Ruby code generation was also restructured. Gotos are elmulated using a + series of if tests. + -Went back to 3.X semantics for >, % and error actions. The > operator also + embeds a leaving action/priority into the start state if it is final. If EOF + happens in a state with a leaving operator then the leaving action is + executed. If EOF happens in a non-final state that has an error action, the + error action is executed. + -The "ragel" program now executes frontend and backend processes separately, + connecting them with a temporary file in the current directory. Without the + -x option the "ragel" program marshals arguments and calls the frontend and + backend. With the -x option the "ragel" program acts as the frontend only. + -Added name finding for executables. If any forward slash is found in argv0 + then it is assumed that the path is explicit and the path to the backend + executable should be derived from that. Whe check that location and also go + up one then inside a directory of the same name in case we are executing + from the source tree. If no forward slash is found it is assumed the file is + being run from the installed location. The PREFIX supplied during + configuration is used. + -On windows GetModuleFileNameEx is used to find out where the the current + process's binary is. That location is searched first. If that fails then we + go up one directory and look for the executable inside a directory of the + same name in case we are executing from the source tree. + -Changed the -l option in rlgen-cd to -L because it is covered in the + frontend. Added a passthrough in the frontend for the backend options. + -Dot file generation can now be invoked using the -V option to ragel. We + now require an input file. If standard in is used then we don't have a file + name on which to base the output. + -Able to build native windows executables using Cygwin+MinGW. + -Patch from David Waite: Large arrays are now created by copying in the data + from smaller arrays using System.arraycopy(). This eliminates the debug data + associated with explicit initialization statements. It is also much easier + on the java compiler which can run out of memory compiling very large + machines. The downside is that it takes slightly longer to initialize static + data at run time. + -The fbreak statement now advances p. + -In the :> :>> and <: operators it was possible for the priority assignment + to be bypassed via the zero length string. In :> this was fixed + automatically with the semantics change to the entering priority operator. + If the start state is final it now embeds a leaving action into it, + preventing persistance through the zero length string. In :>> and <: this + was fixed explicitly. With <: the entering priority operator was used and + with :> a special exception was added. Since it uses the finishing + transition operator it also adds a leaving priority to the start state if it + is final. + -Ranlib is now run on the archives. Patch from Kenny MacDermid. + -The case statement syntax in ruby code generation used a form depreciated in + Ruby 1.9. Updated it. + -Made a number of fixes that eliminate warnings in GCC 4.3. Mostly concern + the now depreciate automatic conversion of string contsants to "char*" type. + Other fixes include adding parenthesis around && within ||. + -The "tokstart" and "tokend" variables were changed to "ts" and "te". + +Ragel 5.25 - Dec 24, 2007 +========================= + -Fixed segfault reported by Ryan Phelps. Affected Java and Ruby code + generation. The dataExpr variable was not initialized. + -Fixed incorrect case label in test/runtests. Caused Objective-C tests to be + ignored. + -Added missing include to common.cpp. + +Ragel 5.24 - Sep 16, 2007 +========================= + -Applied patch from Victor Hugo Borja <vic@rubyforge.org>. This patch + implements -T1 -F0 -F1 and -G0 in the ruby code generator. Goto-driven code + generation is experimental and requires rubinius asm directives (specify + with --rbx option). These code generators pass all the ruby tests. + -If the condition embedding code runs out of available characters in the + keyspace an error message is emitted. + -The first example that appeared in the manual used the special-purpose + 'noend' write option. This caused confusion. Now a basic example appears + first. + -Added two new statements: prepush and postpop. These are code blocks that + are written out during call and return statements. The prepush code is + written immediately before pushing the current state to the state stack + during a call. The postpop code is written immediately after popping the + current state during return. These can be used to implement a dynamically + resizable stack. + +Ragel 5.23 - Jul 24, 2007 +========================= + -Eliminated the use of callcc as an alternative to goto. Instead, the named + breaks implementation used in the Java code generator is imitated using + control flow variables. + -Improved the error message given when there is a write statement but no + machine instantiations and hence no state machine. + -Documentation improvements: updates to "Machine Instantiation", "Write Init" + and "Write Exports" sectons. Added the "Variables Used by Ragel" section. + -Renamed "Entering Actions" to "Starting Actions." + -Other documentation updates. + +Ragel 5.22 - June 14, 2007 +========================== + -Bug fix: need to isolate the start state of a scanner before setting the + to-state and from-state actions which clear and set tokstart. This affected + very simple scanners only. Most scanners have an isolated start state due to + the pattern structure. + -Bug fix: when -S or -M was given the ragel version number was not emitted, + causing the backend to reject the intermediate format. From Tim Potter. + -The p varialbe is now set up at the beginning of a scanner action, rather + than at the end. This leaves scanner actions free to manipulate p and + removes the need for the special holdTE and execTE (TE for tokend) versions + of hold and exec. It also removes the need to set p = tokend-1 immediately + before any control flow. We loose the ability to determine where in the + input stream a scanner action is executed, however this information is of + little use because it is primarily an artifact of the scanner implementation + (sometimes the last char, other times later on). The gains of this change + are consistency and simplicity. + -The "data" variable (used in Java and Ruby code generation only) can now be + overridden using the variable statement. + +Ragel 5.21 - May 9, 2007 +======================== + -Fixed an inconsistency in the value of p following an error. In the C + directly executable code (rlgen-cd -G2) p is left at the character where + the error occurred, which is correct. In all other code generators it was + left at the following character. This was fixed. Now in all code generators + p is left at the character where the error occurred. + -Bug fix: when fhold was used in scanner pattern actions which get executed + on the last character of the pattern (pattern matches which do not require + any lookahead), fhold was modifying p instead of tokend. This was fixed and + the patact.rl test was modified to cover the case. + -Fixed typos in the guide, improved the state action embedding operator + section and added subsections on the variable, import, and export + statements. + -Implemented a better solution than the pri hack for resolving the '-' + ambiguity: force a shortest match of term. + -Fixed bugs in the binary searching for condition keys in both the Ruby and + Java code generation. + -Can now embed the negative sense of a condition. Added a language- + independent test case for this feature and the necessary transformation + support. + -Added new condition embedding syntax: + expr inwhen cond - The transitions into the machine (starting transitions). + expr outwhen cond - The pending transitions out of the machine. + -The argument to the variable statement which affects the name of the current + state variable was changed from "curstate" to "cs" (the default name used + for the current state) + -Implemented the other variables names in the variable statement. Now all + variables (p, pe, cs, top, stack, act, tokstart, tokend) can be renamed. + -Parse errors in the intermediate XML file now cause the backend to exit + immediately rather then forge on. The recovery infrastructure isn't there + and segfaults are likely. + -When no input is given to the backend program, it should not print an error + message, it should just return a non-zero exit status. The assumption is + that the frontend printed an error. + -The version number is now included in the intermediate file. An error is + emitted if there is a mismatch. + -The alphabet type is now communicated from the frontend to the backend using + a one-word internal name instead of an array offset. + -The Ruby host language types had been just copied from Java. Reduced them to + two basic types: char and int, both signed with the usual C sizes. + +Ragel 5.20 - Apr 7, 2007 +======================== + -The cs variable is now always initialized, unless the "nocs" option is given + to the write init command. If there is no main machine, cs is initialized to + the entry point defined by the last machine instantiation. + -A number of fixes were made to the Ruby code generator. + -The frontend now scans ruby comments and regular expressions. + -A transformation for Ruby was added to the language-independent test suite. + The Ruby code generator passes on all the language-independent tests. + -A new Ruby test and two language-independent tests were added. + -Some portability fixes were made (Patches from Josef Goettgens and Aaron + Campbell). + -Fixed a make dependency bug which caused a problem for parallel building + (Patch from Jeremy Hinegardner). + +Ragel 5.19 - Mar 14, 2007 +========================= + -Added an import statement to ragel. This statement takes a literal string as + an argument, interprets it as a file name, then scrapes the file for + sequences of tokens that match the following forms. Tokens inside ragel + sections are ignored. An example is in test/import1.rl + name = number + name = lit_string + "define" name number + "define" name lit_string + -Added an export mechanism which writes defines for single character machines + that have been tagged with the export keyword in their definition. Defines + are used for C, ints for D, Java and Ruby. Examples of the export feature + are in test/export*.rl. + -All machine instantiations are now always generated, even if they are not + referenced. In the backend, entry points for all instantiations are written + out alongside start, error and first final states. + -If the main machine is not present then do not emit an error. Generate the + machine without a start state and do not initialize cs in the write init + code. + -Added an option -l to rlgen-cd which inhibits the writing of #line + directives. + -Added a new syntax for verbose embeddings. This adds parentheses: + $from(action_name); + Verbose embeddings without parentheses can make code difficult to read + because they force a space in the middle of an action embedding. There is a + tendency to associtate spaces with concatenation. Without syntax + highlighting to make it clear that the embedding type is a keyword, the + problem is especially bad. The danger is that a verbose embedding could be + read as an embedding of the keyword representing the empbedding type. With + parentheses, verbose embeddings read much more clearly. + -Conditions now have a forced order when more than one is executed on a + single character. Previously ordering relied on pointers, which caused + results to vary by compiler. Ordering is now done using conditon action + declaration order. This fixes the failure of cond4.rl which occured with + g++ 4.1 and other compiler versions. + -In the port from flex to ragel, the name separator :: in Ragel code was + lost. Added it back. + -In the examples directory switched from rlcodegen to rlgen-cd. Silenced a + warning in statechart.rl. + -In the root makefile the distclean target was fixed. It was calling clean in + the subdirs. In docs, the clean target was not deleting the new manpages for + the rlgen-* programs. Fixed. + -Portability and other fixes from Josef Goettgens were applied. + -The @datadir@ and @mandir@ variables are made use of in doc/Makefile.in for + specifying where documentation should be installed. Patch from Marcus + Rueckert. + +Ragel 5.18 - Feb 13, 2007 +========================= + -There is now a 100% correspondence between state id numbers in the + intermediate XML file, Graphviz dot files and generated code. This was + achieved by moving code which determines if the error state is necessary + into the frontend, and then assigning state numbers before writing out the + intermediate file. + -Backened class structure was reorganized to make it easier to add new code + generators without having to also modify the existing code generators. + -The C and D code generation executable was changed to rlgen-cd. + -The Java code generation was split out into it's own exectuable (rlgen-java) + to allow it to freely diverge from the C/D-based code generation. + -The graphviz dot file generation was also split out to it's own executable + (rlgen-dot). + -The Ruby code generation patch from Victor Hugo Borja was added. This is + highly experimental code and is not yet completely functional. It is in the + executable rlgen-ruby. + -The problem with large state machine machines in Java was fixed. This + problem was discovered by Colin Fleming, who also contributed a patch. + Rather than specify arrays as comma-separated lists of literals, array + initialization is now done in a static function. This is the approach used + by the Java compiler. Unlike the compiler Ragel is careful split large + initilization functions. + -The manual was expanded and reorganized somewhat. + -Eliminated per-example directories in examples/. + -Made some fixes to the pullscan.rl example. + -In the frontend CR characters are now treated as whitespace. + -Updated to the latest aapl. This completely eliminates the shallowCopy + function. With that, a definitive memory leak is fixed. + -Control codes with escape sequences are now printable characters (-p + option). Also, the space character is now printed as SP. + -Fixed the null dereference and consequential segfault which occurred when + trying to create empty machines with [] and // and /a[]b/. + -Fixed the segfault which occured when a machine reference failed. + -Discontinuing ragel.spec. It is more appropriate for this to be written by + package maintenance developers. + +Ragel 5.17 - Jan 28, 2007 +========================= + -The scanners and parsers in both the frontend and backend programs were + completely rewritten using Ragel and Kelbt. -The '%when condition' syntax was functioning like '$when condition'. This was fixed. - + -In the Vim syntax file fixes to the matching of embedding operators were + made. Also, improvements to the sync patterns were made. + -Added pullscan.rl to the examples directory. It is an example of doing + pull-based scanning. Also, xmlscan.rl in rlcodegen is a pull scanner. + -The introduction chapter of the manual was improved. The manually-drawn + figures for the examples were replaced with graphviz-drawn figures. + Ragel 5.16 - Nov 20, 2006 ========================= - -Bug fix: the fhold and fexec directives did not function correctly in + -Policy change: the fhold and fexec directives did not function correctly in scanner pattern actions. In this context manipulations of p may be lost or - made invalid. To fix this, fexec and fhold now manipulate tokend, which is - now always used to update p when the action terminates. + made invalid. In the previous version of Ragel they were banned because of + this. Instead of banning these directives they have been fixed. The fexec + and fhold directives now manipulate tokend, which is now always used to + update p when the action terminates. Ragel 5.15 - Oct 31, 2006 ========================= @@ -552,7 +965,7 @@ Ragel 3.7 - Oct 31, 2004 -Changed examples/cppscan to use fexec and thereby go much faster. -Implemented flex and re2c versions of examples/cppscan. Ragel version goes faster than flex version but not as fast as re2c version. - -Merged in Objective-C patch from Eric Ocean. + -Merged in Objective-C patch from Erich Ocean. -Turned off syncing with stdio in C++ tests to make them go faster. -Renamed C++ code generaion classes with the Cpp Prefix instead of CC to make them easier to read. @@ -708,7 +1121,7 @@ Ragel 3.4 - May 8, 2004 running of tests. Ragel 3.3 - Mar 7, 2004 -========================= +======================= -Portability bug fixes were made. Minimum and maximum integer values are now taken from the system. An alignment problem on 64bit systems was fixed. @@ -1056,8 +1469,8 @@ Ragel 1.3.1 - Oct 2, 2002 -A proper user guide was started. Chapter 1: Specifying Ragel Programs was written. It even has some diagrams :) -Ragel 1.3.0 - Sept 4, 2002 -========================== +Ragel 1.3.0 - Sep 4, 2002 +========================= -NULL keyword no longer used in table output. -Though not yet in use, underlying graph structure changed to support range transitions. As a result, most of the code that walks transition lists is now @@ -1117,8 +1530,8 @@ Ragel 1.2.0 - May 3, 2002 -If -e is given, but minimization is not turned on, print a warning. -Makefiles use automatic dependencies. -Ragel 1.1.0 - April 15, 2002 -============================ +Ragel 1.1.0 - Apr 15, 2002 +========================== -Added goto fsm: much faster than any other fsm style. -Default operator (if two machines are side by side with no operator between them) is concatenation. First showed up in 1.0.4. @@ -1147,8 +1560,8 @@ Ragel 1.1.0 - April 15, 2002 -Code generation section in docs. -Uses the latests aapl. -Ragel 1.0.5 - March 3, 2002 -=========================== +Ragel 1.0.5 - Mar 3, 2002 +========================= -Bugfix in SetErrorState that caused an assertion failure when compiling simple machines that did not have full transition tables (and thus did not show up on any example machines). Assertion failure did not occur @@ -1164,8 +1577,8 @@ Ragel 1.0.5 - March 3, 2002 -Started a README with compilation notes. Started an AUTHORS file. -Started the user documentation. Describes basic machines and operators. -Ragel 1.0.4 - March 1, 2002 -=========================== +Ragel 1.0.4 - Mar 1, 2002 +========================= -Ported to the version of Aapl just after 2.2.0 release. See http://www.ragel.ca/aapl/ for details on aapl. -Fixed a bug in the clang example: the newline machine was not stared. @@ -0,0 +1,6 @@ +#!/bin/sh + +# Change to yes to enable building of parsers or manual. Reconfigure +# afterwards. +build_parsers=no; +build_manual=no; diff --git a/Makefile.am b/Makefile.am new file mode 100644 index 0000000..bbed893 --- /dev/null +++ b/Makefile.am @@ -0,0 +1,19 @@ + +SUBDIRS = ragel doc +DIST_SUBDIRS = $(SUBDIRS) aapl contrib examples test + +dist_doc_DATA = CREDITS ChangeLog +EXTRA_DIST = ragel.vim + +# This file is checked for by the configure script and its existence causes the +# parsers and the manual to not be built when the distribution is built. +dist-hook: + ( \ + echo "#!/bin/sh"; \ + echo ""; \ + echo "# Change to yes to enable building of parsers or manual. Reconfigure"; \ + echo "# afterwards."; \ + echo "build_parsers=no;"; \ + echo "build_manual=no;"; \ + ) > $(distdir)/DIST + @@ -5,50 +5,28 @@ 1. Build Requirements --------------------- - * GNU Make + * Make * g++ If you would like to modify Ragel and need to build Ragel's scanners and -parsers from the specifications then set BUILD_PARSERS=true in the configure -script and then run it. To build the parsers you will need the following -programs: +parsers from the specifications then set "build_parsers=yes" the DIST file and +reconfigure. This variable is normally set to "no" in the distribution tarballs +and "yes" in version control. You will need the following programs: - * flex - * bison (recent version and not bison++, see below) - * gperf + * ragel (the most recent version) + * kelbt (the most recent version) -To build the user guide the following extra programs are needed: +To build the user guide set "build_manual=yes" in the DIST file and +reconfigure. You will need the following extra programs: * fig2dev * pdflatex +2. Compilation and Installation +------------------------------- -2. Compilation --------------- +Ragel uses autoconf and automake. -To configure type './configure'. The makefiles honour the --prefix option to -specify where the program is to be installed to. - -To build the ragel program type 'make'. - -To build all the documentation cd to 'doc' and type 'make'. If you don't have -all of the programs to build the user guide and just want the man page use -'make ragel.1 rlcodegen.1'. - - -3. Installing -------------- - -The command 'make install' will build the programs and install them to $PREFIX/bin/. -A 'make install' in the doc directory will make and install all the -documentation. The man pages install to $PREFIX/man/man1/ and the user guide -and ChangeLog install to $PREFIX/share/doc/ragel/. To install just the man page -use 'make man-install'. - - -4. Why Ragel cannot be built with Bison++ ------------------------------------------ -Ragel is written in C++ using a C-style parser. Bison++ sees that we are using -C++ and generates classes, which breaks the build. As of last investigation, -this can't be stopped. Bison++ is therefore only compatible with Bison if you -are implementing a C-style parser in C. +$ ./configure --prefix=PREFIX +$ make +$ make install @@ -1,9 +1,59 @@ +Guard against including a ragel file more than once (newsgroup). + +Line numbers in included files refer to the master file. This needs to be +fixed -- from Manoj. + +Remove old action embedding and condition setting syntax. + fbreak should advance the current char. Depreciate fbreak and add fctl_break; fctl_return <expr>; fctl_goto <label>; +This is needed for better support of pull scanners. + +Eliminate tokend, replace it with the marker variable and add ftokend/ftoklen. + +Add the accept action embedding operator: like eof but only for states that end +up final. Add the combined pending/accept action. This becomes a good idea when +eof action execution is moved into the main loop. + +Add a prefix operator which sets every state final. + +Minimization should remove a condition when the character allows both +the positive and negative sense of the condition. This happens in: +test_every_10_chars = ( ( c when test_len ) c{0,9} )**; +In this example there is non-determinsm that is killed by the priorities, but +since conditions are expanded before priorities are tested, many transitions +end up with test_len || !test_len. + +Should be possible to include scanner definitions in another scanner. + +Need an "entry name;" feature, causing name to get written out with the other +entry points in the data. + +Possibly bring back the old semantics of > in a new operator, or allow it to be +defined somehow. + +When priorities are embedded without a name, the name of the current machine is +used. Perhaps a unique name for each instance of the current machine should be +used instead. This idea could work well if applied to user-defined embeddings. + +User defined embeddings <-name(a1,a2,...). +User defined operators expr1 <name> expr2. + +Doesn't make make sense for [] to be the lambda (zero-length) machine. This +should be the empty set (the empty machine). But then would it be invalid in a +regular expression? + +The |> guarded operator and the <| guarded operator need to be added. + +An option to turn off the removal of duplicate actions might be useful for +analyzing unintentional nondeterminism. + +Might be a good idea to add in some protection against using up all of a +system's memory. This protection could then be removed by people when someone +is sure they want to use a lot of memory. -It should be possible to import/export definitions. If a scanner can be optimized into a pure state machine, maybe permit it to be referenced as a machine definition. Alternately: inline scanners with an @@ -32,6 +82,10 @@ A debugger would be nice. Ragel could emit a special debug version that prompted for debug commands that allowed the user to step through the machine and get details about where they are in their RL. +A quick and easy alternative would be a trace code generation option. This +would open a trace file and list all the active machines at each step of the +input. + Frontend should allow the redefinition of fsm section delimiters. Do more to obscure ragel's private variables. Just a leading underscore is not diff --git a/aapl/Makefile.am b/aapl/Makefile.am new file mode 100644 index 0000000..4a7f30f --- /dev/null +++ b/aapl/Makefile.am @@ -0,0 +1,10 @@ +noinst_HEADERS = \ + avlbasic.h avlimel.h avlmap.h bstcommon.h compare.h insertsort.h \ + sbstset.h avlcommon.h avlimelkey.h avlmel.h bstmap.h dlcommon.h \ + mergesort.h sbsttable.h avlibasic.h avliset.h avlmelkey.h bstset.h \ + dlist.h quicksort.h svector.h avlikeyless.h avlitree.h avlset.h \ + bsttable.h dlistmel.h resize.h table.h avlimap.h avlkeyless.h avltree.h \ + bubblesort.h dlistval.h sbstmap.h vector.h + +EXTRA_DIST = README COPYING + diff --git a/aapl/avlbasic.h b/aapl/avlbasic.h index 780ef07..ed826f3 100644 --- a/aapl/avlbasic.h +++ b/aapl/avlbasic.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/avlcommon.h b/aapl/avlcommon.h index 1984531..06983bc 100644 --- a/aapl/avlcommon.h +++ b/aapl/avlcommon.h @@ -1,5 +1,5 @@ /* - * Copyright 2001 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. @@ -598,7 +598,9 @@ template <AVLMEL_TEMPDEF> void AvlTree<AVLMEL_TEMPUSE>:: root = other.root; #ifdef WALKABLE - BASELIST::shallowCopy( other ); + BASELIST::head = other.BASELIST::head; + BASELIST::tail = other.BASELIST::tail; + BASELIST::listLen = other.BASELIST::listLen; #else head = other.head; tail = other.tail; @@ -636,7 +638,9 @@ template <AVLMEL_TEMPDEF> void AvlTree<AVLMEL_TEMPUSE>:: root = other.root; #ifdef WALKABLE - BASELIST::shallowCopy( other ); + BASELIST::head = other.BASELIST::head; + BASELIST::tail = other.BASELIST::tail; + BASELIST::listLen = other.BASELIST::listLen; #else head = other.head; tail = other.tail; diff --git a/aapl/avlibasic.h b/aapl/avlibasic.h index a48faaa..b916f74 100644 --- a/aapl/avlibasic.h +++ b/aapl/avlibasic.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/avlikeyless.h b/aapl/avlikeyless.h index 559b75a..0c60608 100644 --- a/aapl/avlikeyless.h +++ b/aapl/avlikeyless.h @@ -1,5 +1,5 @@ /* - * Copyright 2002, 2003 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002, 2003 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/avlimap.h b/aapl/avlimap.h index 38bfff7..c207dc5 100644 --- a/aapl/avlimap.h +++ b/aapl/avlimap.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/avlimel.h b/aapl/avlimel.h index 9442a99..bceddcd 100644 --- a/aapl/avlimel.h +++ b/aapl/avlimel.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/avlimelkey.h b/aapl/avlimelkey.h index faa56e8..52d3a40 100644 --- a/aapl/avlimelkey.h +++ b/aapl/avlimelkey.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/avliset.h b/aapl/avliset.h index cf5be36..9594e7d 100644 --- a/aapl/avliset.h +++ b/aapl/avliset.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/avlitree.h b/aapl/avlitree.h index b053c96..28394bb 100644 --- a/aapl/avlitree.h +++ b/aapl/avlitree.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/avlkeyless.h b/aapl/avlkeyless.h index 3080513..fecf8bd 100644 --- a/aapl/avlkeyless.h +++ b/aapl/avlkeyless.h @@ -1,5 +1,5 @@ /* - * Copyright 2002, 2003 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002, 2003 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/avlmap.h b/aapl/avlmap.h index e4e1566..378613c 100644 --- a/aapl/avlmap.h +++ b/aapl/avlmap.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/avlmel.h b/aapl/avlmel.h index 7bfad3b..6d0deb7 100644 --- a/aapl/avlmel.h +++ b/aapl/avlmel.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/avlmelkey.h b/aapl/avlmelkey.h index 9261cc8..5a66c9c 100644 --- a/aapl/avlmelkey.h +++ b/aapl/avlmelkey.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/avlset.h b/aapl/avlset.h index 224ee59..579378a 100644 --- a/aapl/avlset.h +++ b/aapl/avlset.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/avltree.h b/aapl/avltree.h index cf15359..2aa8e15 100644 --- a/aapl/avltree.h +++ b/aapl/avltree.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/bstcommon.h b/aapl/bstcommon.h index bd390cd..888717f 100644 --- a/aapl/bstcommon.h +++ b/aapl/bstcommon.h @@ -1,5 +1,5 @@ /* - * Copyright 2001 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/bstmap.h b/aapl/bstmap.h index 5154b86..30c8e3c 100644 --- a/aapl/bstmap.h +++ b/aapl/bstmap.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/bstset.h b/aapl/bstset.h index ce710ee..4a0f88e 100644 --- a/aapl/bstset.h +++ b/aapl/bstset.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/bsttable.h b/aapl/bsttable.h index 9898ebf..4e4babc 100644 --- a/aapl/bsttable.h +++ b/aapl/bsttable.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/bubblesort.h b/aapl/bubblesort.h index 20e0f6f..bcc2fb6 100644 --- a/aapl/bubblesort.h +++ b/aapl/bubblesort.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/compare.h b/aapl/compare.h index e537736..3106df9 100644 --- a/aapl/compare.h +++ b/aapl/compare.h @@ -1,5 +1,5 @@ /* - * Copyright 2001 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/dlcommon.h b/aapl/dlcommon.h index 5ce9bd3..91dff25 100644 --- a/aapl/dlcommon.h +++ b/aapl/dlcommon.h @@ -1,5 +1,5 @@ /* - * Copyright 2001 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/dlist.h b/aapl/dlist.h index eaf3e5d..e34c8da 100644 --- a/aapl/dlist.h +++ b/aapl/dlist.h @@ -1,5 +1,5 @@ /* - * Copyright 2001 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/dlistmel.h b/aapl/dlistmel.h index 3433139..17de543 100644 --- a/aapl/dlistmel.h +++ b/aapl/dlistmel.h @@ -1,5 +1,5 @@ /* - * Copyright 2001 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/dlistval.h b/aapl/dlistval.h index 6f24999..4fcf33f 100644 --- a/aapl/dlistval.h +++ b/aapl/dlistval.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/insertsort.h b/aapl/insertsort.h index eb3e264..94aef7b 100644 --- a/aapl/insertsort.h +++ b/aapl/insertsort.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/mergesort.h b/aapl/mergesort.h index d017511..68b8426 100644 --- a/aapl/mergesort.h +++ b/aapl/mergesort.h @@ -1,5 +1,5 @@ /* - * Copyright 2001, 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001, 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/quicksort.h b/aapl/quicksort.h index 9bb96ef..bb6941e 100644 --- a/aapl/quicksort.h +++ b/aapl/quicksort.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/resize.h b/aapl/resize.h index 24edc16..9e8491a 100644 --- a/aapl/resize.h +++ b/aapl/resize.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/sbstmap.h b/aapl/sbstmap.h index 9436a47..e3975a1 100644 --- a/aapl/sbstmap.h +++ b/aapl/sbstmap.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/sbstset.h b/aapl/sbstset.h index fe8ddf6..3487ee7 100644 --- a/aapl/sbstset.h +++ b/aapl/sbstset.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/sbsttable.h b/aapl/sbsttable.h index 100b87e..348f1fd 100644 --- a/aapl/sbsttable.h +++ b/aapl/sbsttable.h @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/svector.h b/aapl/svector.h index ff9e97c..7dcae62 100644 --- a/aapl/svector.h +++ b/aapl/svector.h @@ -1,5 +1,5 @@ /* - * Copyright 2002, 2006 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002, 2006 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. @@ -103,9 +103,6 @@ public: /* Shallow copy. */ SVector( const SVector &v ); - /* Shallow copy. */ - SVector(STabHead *head); - /** * \brief Free all memory used by the vector. * @@ -128,9 +125,6 @@ public: /* Perform a shallow copy of another vector. */ SVector &operator=( const SVector &v ); - /* Perform a shallow copy of another vector by the header. */ - SVector &operator=( STabHead *head ); - /*@{*/ /** @@ -490,32 +484,6 @@ protected: void downResizeDup(long len); }; -#if 0 -/* Create a vector with an intial number of elements and size. */ -template <class T, class Resize> SVector<T, Resize>:: - SVector( long size, long allocLen ) -{ - /* Allocate the space if we are given a positive allocLen. */ - if ( allocLen > 0 ) { - /* Allocate the data needed. */ - STabHead *head = (STabHead*) malloc( sizeof(STabHead) + - sizeof(T) * allocLen ); - if ( head == 0 ) - throw std::bad_alloc(); - - /* Set up the header and save the data pointer. */ - head->refCount = 1; - head->allocLen = allocLen; - head->tabLen = 0; - BaseTable::data = (T*) (head + 1); - } - - /* Grow to the size specified. If we did not have enough space - * allocated that is ok. Table will be grown to the right size. */ - setAsNew( size ); -} -#endif - /** * \brief Perform a shallow copy of the vector. * @@ -535,26 +503,6 @@ template <class T, class Resize> SVector<T, Resize>:: } } -#if 0 -/** - * \brief Perform a shallow copy of the vector from only the header. - * - * Takes a reference to the contents specified by the header. - */ -template <class T, class Resize> SVector<T, Resize>:: - SVector(STabHead *head) -{ - /* Take a reference to other, if the header is no-null. */ - if ( head == 0 ) - BaseTable::data = 0; - else { - head->refCount += 1; - BaseTable::data = (T*) (head + 1); - } -} -#endif - - /** * \brief Shallow copy another vector into this vector. * @@ -581,30 +529,6 @@ template <class T, class Resize> SVector<T, Resize> & return *this; } -/** - * \brief Shallow copy another vector into this vector from only the header. - * - * Takes a reference to the other header vector. The contents of this vector - * are first emptied. - * - * \returns A reference to this. - */ -template <class T, class Resize> SVector<T, Resize> & - SVector<T, Resize>::operator=( STabHead *head ) -{ - /* First clean out the current contents. */ - empty(); - - /* Take a reference to other, if the header is no-null. */ - if ( head == 0 ) - BaseTable::data = 0; - else { - head->refCount += 1; - BaseTable::data = (T*) (head + 1); - } - return *this; -} - /* Init a vector iterator with just a vector. */ template <class T, class Resize> SVector<T, Resize>:: Iter::Iter( const SVector &v ) diff --git a/aapl/table.h b/aapl/table.h index c1f2b7b..c218281 100644 --- a/aapl/table.h +++ b/aapl/table.h @@ -1,5 +1,5 @@ /* - * Copyright 2001, 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001, 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. diff --git a/aapl/vector.h b/aapl/vector.h index c33e35b..b29c749 100644 --- a/aapl/vector.h +++ b/aapl/vector.h @@ -1,5 +1,5 @@ /* - * Copyright 2002, 2006 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002, 2006 Adrian Thurston <thurston@complang.org> */ /* This file is part of Aapl. @@ -106,9 +106,9 @@ public: /* Abandon the contents of the vector without deleteing. */ void abandon(); - /* Performs a shallow copy of another vector into this vector. If this - * vector is non-empty then its contents are lost (not freed). */ - void shallowCopy( const Vector &v ); + /* Transfers the elements of another vector into this vector. First emptys + * the current vector. */ + void transfer( Vector &v ); /* Perform a deep copy of another vector into this vector. */ Vector &operator=( const Vector &v ); @@ -470,25 +470,6 @@ protected: void downResize(long len); }; -#if 0 -/* Create a vector with an intial number of elements and size. */ -template<class T, class Resize> Vector<T, Resize>:: - Vector( long size, long allocLen ) -{ - /* Allocate the space if we are given a positive allocLen. */ - BaseTable::allocLen = allocLen; - if ( allocLen > 0 ) { - BaseTable::data = (T*) malloc(sizeof(T) * BaseTable::allocLen); - if ( BaseTable::data == 0 ) - throw std::bad_alloc(); - } - - /* Grow to the size specified. If we did not have enough space - * allocated that is ok. Table will be grown to right size. */ - setAsNew( size ); -} -#endif - /* Init a vector iterator with just a vector. */ template <class T, class Resize> Vector<T, Resize>::Iter::Iter( const Vector &v ) { @@ -624,20 +605,22 @@ template<class T, class Resize> void Vector<T, Resize>:: } /** - * \brief Shallow copy another vector into this vector. + * \brief Transfer the contents of another vector into this vector. * - * The dynamic array of the other vector is copied into this vector by - * reference. If this vector is non-empty then its contents are lost. This - * routine must be used with care. After a shallow copy one vector should - * abandon its contents to prevent both destructors from attempting to free - * the common array. + * The dynamic array of the other vector is moved into this vector by + * reference. If this vector is non-empty then its contents are first deleted. + * Afterward the other vector will be empty. */ template<class T, class Resize> void Vector<T, Resize>:: - shallowCopy( const Vector &v ) + transfer( Vector &v ) { + empty(); + BaseTable::data = v.data; BaseTable::tabLen = v.tabLen; BaseTable::allocLen = v.allocLen; + + v.abandon(); } /** diff --git a/autogen.sh b/autogen.sh new file mode 100755 index 0000000..25fb200 --- /dev/null +++ b/autogen.sh @@ -0,0 +1,7 @@ +#!/bin/bash +# + +aclocal +autoheader +automake --foreign --add-missing +autoconf diff --git a/configure.in b/configure.in index a8e55e6..e00e89c 100644 --- a/configure.in +++ b/configure.in @@ -1,5 +1,5 @@ dnl -dnl Copyright 2001-2006 Adrian Thurston <thurston@cs.queensu.ca> +dnl Copyright 2001-2009 Adrian Thurston <thurston@complang.org> dnl dnl This file is part of Ragel. @@ -18,21 +18,35 @@ dnl You should have received a copy of the GNU General Public License dnl along with Ragel; if not, write to the Free Software dnl Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA -AC_INIT(ragel/main.cpp) -AC_CONFIG_HEADER(common/config.h) +AC_INIT(ragel, 6.6) +PUBDATE="Dec 2009" -dnl Set true if build system should generate parsers from flex, bison, and -dnl gperf sources. Set false if generated files are included and not to be -dnl built (production). -AC_SUBST(BUILD_PARSERS,true) +AM_INIT_AUTOMAKE([foreign]) +AC_SUBST(PUBDATE) +AC_CONFIG_HEADER(ragel/config.h) -dnl Checks for programs. -AC_PROG_CC -AC_DEFINE_UNQUOTED(CC,$CC) +dnl Choose defaults for the build_parsers and build_manual vars. If the dist +dnl file is present in the root then default to no, otherwise go for it. +AC_CHECK_FILES( $srcdir/DIST, + [ . $srcdir/DIST; ], + [ build_parsers=yes; build_manual=yes; ] ) + +dnl Set to true if the build system should generate parsers from ragel and kelbt +dnl sources. Set to false if generated files are included and not to be built +dnl (production). +AM_CONDITIONAL(BUILD_PARSERS, [test "x$build_parsers" = "xyes"]) + +dnl Set to true if the manual should be built. +AM_CONDITIONAL(BUILD_MANUAL, [test "x$build_manual" = "xyes"]) dnl Checks for programs. +AC_PROG_CC AC_PROG_CXX -AC_DEFINE_UNQUOTED(CXX,$CXX) +AC_CHECK_TOOL(AR, ar) +AC_PROG_RANLIB + +# Default flags. +CXXFLAGS="-g" dnl Set test on c++ compiler. AC_LANG_CPLUSPLUS @@ -40,50 +54,50 @@ AC_LANG_CPLUSPLUS dnl Check for definition of MAKE. AC_PROG_MAKE_SET -if test $BUILD_PARSERS = true; then +# Checks to carry out if we are building parsers. +if test "x$build_parsers" = "xyes"; then -dnl Check for flex -AC_CHECK_PROG(FLEX, flex, flex) -if test -z "$FLEX"; then +AC_CHECK_PROG(RAGEL, ragel, ragel) +if test -z "$RAGEL"; then echo - echo "error: flex is required to compile ragel" + echo "error: ragel is required to build the parsers" echo exit 1 fi -dnl Check for gperf -AC_CHECK_PROG(GPERF, gperf, gperf) -if test -z "$GPERF"; then +AC_CHECK_PROG(KELBT, kelbt, kelbt) +if test -z "$KELBT"; then echo - echo "error: gperf is required to compile ragel" + echo "error: kelbt is required to build the parsers" echo exit 1 fi -dnl Check for bison -AC_CHECK_PROG(BISON, bison, bison) -if test -z "$BISON"; then +fi + +# Checks to carry out if we are building the manual. +if test "x$build_manual" = "xyes"; then + +AC_CHECK_PROG(FIG2DEV, fig2dev, fig2dev) +if test -z "$FIG2DEV"; then echo - echo "error: bison is required to compile ragel" + echo "error: fig2dev is required to build the manual" echo exit 1 fi -dnl Sorry, Ragel will not compile with bison++. -if "$BISON" --version | grep 'bison++'; then +AC_CHECK_PROG(PDFLATEX, pdflatex, pdflatex) +if test -z "$PDFLATEX"; then echo - echo "error: sorry, ragel cannot be compiled with bison++" + echo "error: pdflatex is required to build the manual" echo exit 1 fi -fi # BUILD_PARSERS +fi dnl Check for the D compiler AC_CHECK_PROG(GDC, gdc, gdc) -if test -n "$GDC"; then - AC_DEFINE_UNQUOTED(GDC,$GDC) -fi dnl Check for the Objective-C compiler AC_MSG_CHECKING([for the Objective-C compiler]) @@ -94,7 +108,6 @@ GOBJC="" if gcc -x objective-c conftest.m -o conftest.bin 2>/dev/null; then GOBJC="gcc -x objective-c" AC_MSG_RESULT([yes]) - AC_DEFINE_UNQUOTED(GOBJC,$GOBJC) else AC_MSG_RESULT([no]) fi @@ -102,17 +115,26 @@ AC_SUBST(GOBJC) dnl Check for the Java compiler. AC_CHECK_PROG(JAVAC, javac, javac) -if test -n "$JAVAC"; then - AC_DEFINE_UNQUOTED(JAVAC,$JAVAC) -fi dnl Check for TXL. AC_CHECK_PROG(TXL, txl, txl) -if test -n "$TXL"; then - AC_DEFINE_UNQUOTED(TXL,$TXL) -fi + +dnl Check for Ruby. +AC_CHECK_PROG(RUBY, ruby, ruby) + +dnl Check for the C# compiler. +AC_CHECK_PROG(GMCS, gmcs, gmcs) dnl write output files -AC_OUTPUT(Makefile common/Makefile ragel/Makefile rlcodegen/Makefile doc/Makefile test/Makefile) +AC_OUTPUT( + [ + Makefile ragel/Makefile aapl/Makefile + doc/Makefile doc/ragel.1 + contrib/Makefile + test/Makefile test/runtests + examples/Makefile + ], + [chmod +x test/runtests] +) echo "configuration of ragel complete" diff --git a/contrib/Makefile.am b/contrib/Makefile.am new file mode 100644 index 0000000..7ef7e8d --- /dev/null +++ b/contrib/Makefile.am @@ -0,0 +1,2 @@ + +EXTRA_DIST = ragel.make ragel.m4 unicode2ragel.rb diff --git a/contrib/ragel.m4 b/contrib/ragel.m4 new file mode 100644 index 0000000..c5cb9e9 --- /dev/null +++ b/contrib/ragel.m4 @@ -0,0 +1,43 @@ +dnl Check for presence of the Ragel State Machine generator. +dnl +dnl This macro checks for the presence of the ragel tool in the system, +dnl and whether the ragel tool is absolutely needed for a complete +dnl build. +dnl +dnl To check for the need for Ragel, you have to provide the relative +dnl path of a source file generated through Ragel: if the file is +dnl present in the source tree, a missing ragel command will not cause +dnl the configure to abort. + +AC_DEFUN([_RAGEL_VARS], [ + AC_ARG_VAR([RAGEL], [Ragel generator command]) + AC_ARG_VAR([RAGELFLAGS], [Ragel generator flags]) +]) + +AC_DEFUN([CHECK_RAGEL], [ + AC_REQUIRE([_RAGEL_VARS]) + AC_CHECK_PROG([RAGEL], [ragel], [ragel], [no]) + + dnl We set RAGEL to false so that it would execute the "false" + dnl command if needed. + AS_IF([test x"$RAGEL" = x"no"], [RAGEL=false]) + + dnl Only test the need if not found + AS_IF([test x"$RAGEL" = x"false"], [ + AC_MSG_CHECKING([whether we need ragel to regenerate sources]) + AS_IF([test -a ${srcdir}/$1], [ragel_needed=no], [ragel_needed=yes]) + AC_MSG_RESULT([$ragel_needed]) + + AS_IF([test x"$ragel_needed" = x"yes"], + [AC_MSG_ERROR([dnl +You need Ragel to build from development sources. +You can find Ragel at http://www.complang.org/ragel/dnl + ])]) + ]) +]) + +AC_DEFUN([CHECK_RAGEL_AM], [ + CHECK_RAGEL([$1]) + + AM_CONDITIONAL([HAVE_RAGEL], [test x"$RAGEL" != x"false"]) +]) diff --git a/contrib/ragel.make b/contrib/ragel.make new file mode 100644 index 0000000..f7a71b5 --- /dev/null +++ b/contrib/ragel.make @@ -0,0 +1,6 @@ +# -*- Makefile -*-
+
+SUFFIXES = .rl
+
+.rl.c:
+ $(RAGEL) $(RAGELFLAGS) -C $< -o $@
diff --git a/contrib/unicode2ragel.rb b/contrib/unicode2ragel.rb new file mode 100644 index 0000000..d64e601 --- /dev/null +++ b/contrib/unicode2ragel.rb @@ -0,0 +1,305 @@ +#!/usr/bin/env ruby +# +# This script uses the unicode spec to generate a Ragel state machine +# that recognizes unicode alphanumeric characters. It generates 5 +# character classes: uupper, ulower, ualpha, udigit, and ualnum. +# Currently supported encodings are UTF-8 [default] and UCS-4. +# +# Usage: unicode2ragel.rb [options] +# -e, --encoding [ucs4 | utf8] Data encoding +# -h, --help Show this message +# +# This script was originally written as part of the Ferret search +# engine library. +# +# Author: Rakan El-Khalil <rakan@well.com> + +require 'optparse' +require 'open-uri' + +ENCODINGS = [ :utf8, :ucs4 ] +ALPHTYPES = { :utf8 => "unsigned char", :ucs4 => "unsigned int" } +CHART_URL = "http://www.unicode.org/Public/5.1.0/ucd/DerivedCoreProperties.txt" + +### +# Display vars & default option + +TOTAL_WIDTH = 80 +RANGE_WIDTH = 23 +@encoding = :utf8 + +### +# Option parsing + +cli_opts = OptionParser.new do |opts| + opts.on("-e", "--encoding [ucs4 | utf8]", "Data encoding") do |o| + @encoding = o.downcase.to_sym + end + opts.on("-h", "--help", "Show this message") do + puts opts + exit + end +end + +cli_opts.parse(ARGV) +unless ENCODINGS.member? @encoding + puts "Invalid encoding: #{@encoding}" + puts cli_opts + exit +end + +## +# Downloads the document at url and yields every alpha line's hex +# range and description. + +def each_alpha( url, property ) + open( url ) do |file| + file.each_line do |line| + next if line =~ /^#/; + next if line !~ /; #{property} #/; + + range, description = line.split(/;/) + range.strip! + description.gsub!(/.*#/, '').strip! + + if range =~ /\.\./ + start, stop = range.split '..' + else start = stop = range + end + + yield start.hex .. stop.hex, description + end + end +end + +### +# Formats to hex at minimum width + +def to_hex( n ) + r = "%0X" % n + r = "0#{r}" unless (r.length % 2).zero? + r +end + +### +# UCS4 is just a straight hex conversion of the unicode codepoint. + +def to_ucs4( range ) + rangestr = "0x" + to_hex(range.begin) + rangestr << "..0x" + to_hex(range.end) if range.begin != range.end + [ rangestr ] +end + +## +# 0x00 - 0x7f -> 0zzzzzzz[7] +# 0x80 - 0x7ff -> 110yyyyy[5] 10zzzzzz[6] +# 0x800 - 0xffff -> 1110xxxx[4] 10yyyyyy[6] 10zzzzzz[6] +# 0x010000 - 0x10ffff -> 11110www[3] 10xxxxxx[6] 10yyyyyy[6] 10zzzzzz[6] + +UTF8_BOUNDARIES = [0x7f, 0x7ff, 0xffff, 0x10ffff] + +def to_utf8_enc( n ) + r = 0 + if n <= 0x7f + r = n + elsif n <= 0x7ff + y = 0xc0 | (n >> 6) + z = 0x80 | (n & 0x3f) + r = y << 8 | z + elsif n <= 0xffff + x = 0xe0 | (n >> 12) + y = 0x80 | (n >> 6) & 0x3f + z = 0x80 | n & 0x3f + r = x << 16 | y << 8 | z + elsif n <= 0x10ffff + w = 0xf0 | (n >> 18) + x = 0x80 | (n >> 12) & 0x3f + y = 0x80 | (n >> 6) & 0x3f + z = 0x80 | n & 0x3f + r = w << 24 | x << 16 | y << 8 | z + end + + to_hex(r) +end + +def from_utf8_enc( n ) + n = n.hex + r = 0 + if n <= 0x7f + r = n + elsif n <= 0xdfff + y = (n >> 8) & 0x1f + z = n & 0x3f + r = y << 6 | z + elsif n <= 0xefffff + x = (n >> 16) & 0x0f + y = (n >> 8) & 0x3f + z = n & 0x3f + r = x << 10 | y << 6 | z + elsif n <= 0xf7ffffff + w = (n >> 24) & 0x07 + x = (n >> 16) & 0x3f + y = (n >> 8) & 0x3f + z = n & 0x3f + r = w << 18 | x << 12 | y << 6 | z + end + r +end + +### +# Given a range, splits it up into ranges that can be continuously +# encoded into utf8. Eg: 0x00 .. 0xff => [0x00..0x7f, 0x80..0xff] +# This is not strictly needed since the current [5.1] unicode standard +# doesn't have ranges that straddle utf8 boundaries. This is included +# for completeness as there is no telling if that will ever change. + +def utf8_ranges( range ) + ranges = [] + UTF8_BOUNDARIES.each do |max| + if range.begin <= max + return ranges << range if range.end <= max + + ranges << range.begin .. max + range = (max + 1) .. range.end + end + end + ranges +end + +def build_range( start, stop ) + size = start.size/2 + left = size - 1 + return [""] if size < 1 + + a = start[0..1] + b = stop[0..1] + + ### + # Shared prefix + + if a == b + return build_range(start[2..-1], stop[2..-1]).map do |elt| + "0x#{a} " + elt + end + end + + ### + # Unshared prefix, end of run + + return ["0x#{a}..0x#{b} "] if left.zero? + + ### + # Unshared prefix, not end of run + # Range can be 0x123456..0x56789A + # Which is equivalent to: + # 0x123456 .. 0x12FFFF + # 0x130000 .. 0x55FFFF + # 0x560000 .. 0x56789A + + ret = [] + ret << build_range(start, a + "FF" * left) + + ### + # Only generate middle range if need be. + + if a.hex+1 != b.hex + max = to_hex(b.hex - 1) + max = "FF" if b == "FF" + ret << "0x#{to_hex(a.hex+1)}..0x#{max} " + "0x00..0xFF " * left + end + + ### + # Don't generate last range if it is covered by first range + + ret << build_range(b + "00" * left, stop) unless b == "FF" + ret.flatten! +end + +def to_utf8( range ) + utf8_ranges( range ).map do |r| + build_range to_utf8_enc(r.begin), to_utf8_enc(r.end) + end.flatten! +end + +## +# Perform a 3-way comparison of the number of codepoints advertised by +# the unicode spec for the given range, the originally parsed range, +# and the resulting utf8 encoded range. + +def count_codepoints( code ) + code.split(' ').inject(1) do |acc, elt| + if elt =~ /0x(.+)\.\.0x(.+)/ + if @encoding == :utf8 + acc * (from_utf8_enc($2) - from_utf8_enc($1) + 1) + else + acc * ($2.hex - $1.hex + 1) + end + else + acc + end + end +end + +def is_valid?( range, desc, codes ) + spec_count = 1 + spec_count = $1.to_i if desc =~ /\[(\d+)\]/ + range_count = range.end - range.begin + 1 + + sum = codes.inject(0) { |acc, elt| acc + count_codepoints(elt) } + sum == spec_count and sum == range_count +end + +## +# Generate the state maching to stdout + +def generate_machine( name, property ) + pipe = " " + puts " #{name} = " + each_alpha( CHART_URL, property ) do |range, desc| + + codes = (@encoding == :ucs4) ? to_ucs4(range) : to_utf8(range) + + raise "Invalid encoding of range #{range}: #{codes.inspect}" unless + is_valid? range, desc, codes + + range_width = codes.map { |a| a.size }.max + range_width = RANGE_WIDTH if range_width < RANGE_WIDTH + + desc_width = TOTAL_WIDTH - RANGE_WIDTH - 11 + desc_width -= (range_width - RANGE_WIDTH) if range_width > RANGE_WIDTH + + if desc.size > desc_width + desc = desc[0..desc_width - 4] + "..." + end + + codes.each_with_index do |r, idx| + desc = "" unless idx.zero? + code = "%-#{range_width}s" % r + puts " #{pipe} #{code} ##{desc}" + pipe = "|" + end + end + puts " ;" + puts "" +end + +puts <<EOF +# The following Ragel file was autogenerated with #{$0} +# from: #{CHART_URL} +# +# It defines ualpha, udigit, ualnum. +# +# To use this, make sure that your alphtype is set to #{ALPHTYPES[@encoding]}, +# and that your input is in #{@encoding}. + +%%{ + machine WChar; +EOF +generate_machine( :ualpha, "Alphabetic" ) +generate_machine( :ulower, "Lowercase" ) +generate_machine( :uupper, "Uppercase" ) +puts <<EOF + udigit = '0'..'9'; + ualnum = ualpha | udigit; +}%% +EOF diff --git a/debian/changelog b/debian/changelog new file mode 100644 index 0000000..e1cc30f --- /dev/null +++ b/debian/changelog @@ -0,0 +1,347 @@ +ragel (6.6-1slp2) unstable; urgency=low + + * repackage for SLP + + -- Mike McCormack <mj.mccormack@samsung.com> Wed, 03 Aug 2011 17:17:01 +0900 + +ragel (6.6-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Tue, 05 Jan 2010 15:16:17 +0000 + +ragel (6.5-1) unstable; urgency=low + + * New upstream release + * Bumped standards version + + -- Robert Lemmen <robertle@semistable.com> Thu, 17 Sep 2009 19:59:27 +0100 + +ragel (6.3-2) unstable; urgency=low + + * Removed pre-built pdfs + * Upped debhelper version + + -- Robert Lemmen <robertle@semistable.com> Mon, 29 Jun 2009 16:45:45 +0100 + +ragel (6.3-1) unstable; urgency=low + + * New upstream release (closes: #502931) + * updated description (closes: #494147) + + -- Robert Lemmen <robertle@semistable.com> Wed, 22 Oct 2008 20:01:12 +0100 + +ragel (6.2-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Wed, 18 Jun 2008 19:05:03 +0100 + +ragel (6.1-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Thu, 8 May 2008 22:55:27 +0100 + +ragel (6.0-3) unstable; urgency=low + + * Added workaround for tetex transition + + -- Robert Lemmen <robertle@semistable.com> Wed, 5 Mar 2008 11:38:54 +0000 + +ragel (6.0-2) unstable; urgency=low + + * Chnages the gs-gpl dependency to ghostscript (ghostscript transisiton) + * Hacked my way around #457568 + + -- Robert Lemmen <robertle@semistable.com> Tue, 4 Mar 2008 13:58:16 +0000 + +ragel (6.0-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Fri, 29 Feb 2008 10:02:02 +0000 + +ragel (5.23-3) unstable; urgency=low + + * Dependency changed from gs to gs-gpl + + -- Robert Lemmen <robertle@semistable.com> Wed, 17 Oct 2007 12:45:26 +0200 + +ragel (5.23-2) unstable; urgency=low + + * Depend on gs to make ragel build on all architectures (closes: #443294) + + -- Robert Lemmen <robertle@semistable.com> Fri, 21 Sep 2007 14:54:37 +0200 + +ragel (5.23-1) unstable; urgency=low + + * New upstream release + * Is now handling the nostrip build option correctly (closes: #437904) + + -- Robert Lemmen <robertle@semistable.com> Fri, 14 Sep 2007 13:31:55 +0200 + +ragel (5.16-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Mon, 4 Dec 2006 20:50:15 +0100 + +ragel (5.14-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Thu, 5 Oct 2006 21:53:18 +0200 + +ragel (5.10-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Tue, 8 Aug 2006 14:56:03 +0200 + +ragel (5.9-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Thu, 20 Jul 2006 16:39:43 +0200 + +ragel (5.7-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Tue, 16 May 2006 00:07:18 +0200 + +ragel (5.6-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Sun, 16 Apr 2006 14:09:28 +0200 + +ragel (5.5-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Wed, 29 Mar 2006 15:30:28 +0200 + +ragel (5.4-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Thu, 23 Mar 2006 11:03:46 +0100 + +ragel (5.3-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Tue, 31 Jan 2006 09:44:16 +0100 + +ragel (5.1-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Thu, 22 Dec 2005 11:14:51 +0100 + +ragel (4.2-2) unstable; urgency=low + + * Dropped dependency on dvips (closes: #337451) + + -- Robert Lemmen <robertle@semistable.com> Wed, 9 Nov 2005 10:11:01 +0100 + +ragel (4.2-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Wed, 28 Sep 2005 11:25:12 +0200 + +ragel (4.1-4) unstable; urgency=low + + * Make the build system work around bug #324458 + + -- Robert Lemmen <robertle@semistable.com> Mon, 29 Aug 2005 11:13:33 +0200 + +ragel (4.1-3) unstable; urgency=low + + * Use -O2 instead of -O3 for the tests on m68k to work around compiler + bugs (closes: #317475) + * Further cleanups + * Respects DEB_BUILD_OPTIONS now + + -- Robert Lemmen <robertle@semistable.com> Thu, 4 Aug 2005 15:27:36 +0200 + +ragel (4.1-2) unstable; urgency=low + + * Rebuild for the g++ transition + * Smallish cleanups in the makefiles + * Updated to new standards version + + -- Robert Lemmen <robertle@semistable.com> Tue, 19 Jul 2005 10:29:16 +0200 + +ragel (4.1-1) unstable; urgency=low + + * New upstream release + * This version does the manual build again, for a smaller diff! + + -- Robert Lemmen <robertle@semistable.com> Tue, 28 Jun 2005 14:03:00 +0200 + +ragel (4.0-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Wed, 8 Jun 2005 14:15:13 +0200 + +ragel (3.7-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Wed, 3 Nov 2004 20:37:23 +0100 + +ragel (3.6-2) unstable; urgency=low + + * Patched to make buildable again on some archs + + -- Robert Lemmen <robertle@semistable.com> Sun, 26 Sep 2004 20:15:02 +0200 + +ragel (3.6-1) unstable; urgency=low + + * New upstream release, see upstream ChangeLog for details + + -- Robert Lemmen <robertle@semistable.com> Mon, 12 Jul 2004 23:31:51 +0200 + +ragel (3.5-1) unstable; urgency=low + + * New upstream release (closes: #254836) + + -- Robert Lemmen <robertle@semistable.com> Fri, 18 Jun 2004 00:33:16 +0200 + +ragel (3.4-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Thu, 13 May 2004 13:01:51 +0200 + +ragel (3.3-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Mon, 8 Mar 2004 16:05:12 +0100 + +ragel (3.2-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Mon, 1 Mar 2004 17:34:09 +0100 + +ragel (3.1-1) unstable; urgency=low + + * New upstream release + * fixed copyright + + -- Robert Lemmen <robertle@semistable.com> Tue, 24 Feb 2004 11:51:07 +0100 + +ragel (3.0-1) unstable; urgency=low + + * New upstream release with many improvements + + -- Robert Lemmen <robertle@semistable.com> Fri, 23 Jan 2004 00:11:48 +0100 + +ragel (2.2-1) unstable; urgency=low + + * New upstream release + * does self-test during build now + + -- Robert Lemmen <robertle@semistable.com> Mon, 13 Oct 2003 13:09:58 +0200 + +ragel (2.1-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Tue, 30 Sep 2003 12:53:11 +0200 + +ragel (2.0-1) unstable; urgency=low + + * New upstream release (includes postfix repetition operators, named + priorities and a c/c++ interface to indicate whena machine can no longer + accept. + * 2.0 is not always comaptible with earlier versions, see PORTING_TO_V2 + * now supports DEB_BUILD_OPTIONS + + -- Robert Lemmen <robertle@semistable.com> Mon, 8 Sep 2003 13:29:38 +0200 + +ragel (1.5.4-2) unstable; urgency=low + + * reverted the change from 1.5.4-1, somehow the sgml build doesn't seem to + work on all computers.. + + -- Robert Lemmen <robertle@semistable.com> Wed, 23 Jul 2003 13:56:12 +0200 + +ragel (1.5.4-1) unstable; urgency=low + + * New upstream release + * rebuild the documentation during make instead of shipping it in the diff. + This makes the diff much smaller, but increases the build-dependencies. + + -- Robert Lemmen <robertle@semistable.com> Mon, 14 Jul 2003 20:18:00 +0200 + +ragel (1.5.1-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Sat, 21 Jun 2003 19:38:33 +0200 + +ragel (1.5.0-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Wed, 18 Jun 2003 17:19:35 +0200 + +ragel (1.4.1-1) unstable; urgency=low + + * New upstream release (Closes: #175637) + (see /usr/share/doc/ragel/changelog.gz) + + -- Robert Lemmen <robertle@semistable.com> Tue, 7 Jan 2003 18:40:16 +0700 + +ragel (1.3.3-1) unstable; urgency=low + + * New upstream release (see /usr/share/doc/ragel/changelog.gz) + + -- Robert Lemmen <robertle@semistable.com> Fri, 24 Oct 2002 19:57:39 +0700 + +ragel (1.3.1-1) unstable; urgency=low + + * New upstream release (see /usr/share/doc/ragel/changelog.gz) + + -- Robert Lemmen <robertle@semistable.com> Fri, 4 Oct 2002 20:51:09 +0700 + +ragel (1.3.0-1) unstable; urgency=low + + * New upstream release + * build-dependency on awk is not necessary any more + * fhs transition issues resolved + * man page is not generated any more, but taken from source tree. + + -- Robert Lemmen <robertle@semistable.com> Sun, 22 Sep 2002 23:56:24 +0700 + +ragel (1.2.2-1) unstable; urgency=low + + * New upstream release + + -- Robert Lemmen <robertle@semistable.com> Sun, 26 May 2002 16:33:50 +0200 + +ragel (1.2.1-1) unstable; urgency=low + + * New upstream release + * works with g++ 3.0 now + (Closes: Bug#146536) + + -- Robert Lemmen <robertle@semistable.com> Mon, 13 May 2002 18:59:27 +0200 + +ragel (1.1.0-1) unstable; urgency=low + + * Initial Release. + * fixed directories in makefile to comply with the fhs + * added manual page (from homepage) + * added examples from homepage + + -- Robert Lemmen <robertle@semistable.com> Mon, 15 Apr 2002 20:31:21 +0200 + diff --git a/debian/compat b/debian/compat new file mode 100644 index 0000000..7f8f011 --- /dev/null +++ b/debian/compat @@ -0,0 +1 @@ +7 diff --git a/debian/control b/debian/control new file mode 100644 index 0000000..abb8ed6 --- /dev/null +++ b/debian/control @@ -0,0 +1,21 @@ +Source: ragel +Section: devel +Priority: optional +Maintainer: Mike McCormack <mj.mccormack@samsung.com> +## Maintainer: Robert Lemmen <robertle@semistable.com> +Build-Depends: debhelper (>> 7.0.0), autotools-dev, flex, bison +Standards-Version: 3.8.3 + +Package: ragel +Architecture: any +Depends: ${shlibs:Depends}, ${misc:Depends} +Description: compiles finite state machines into code in various languages + Ragel compiles finite state machines from regular languages into C, C++, + Objective-C, D, Ruby or Java code. Ragel allows the programmer to embed + actions at any point in a regular language. Non-determinism can be + controlled through the use of embedded priorities and guarded regular + language operators. Ragel also supports the construction of scanners and + the building of state machines using state-charts. Ragel can be used to + create robust recognizers and parsers which run very fast. It can work + with integer-sized alphabets and can compile large state machines. + The generated code has no dependencies. diff --git a/debian/copyright b/debian/copyright new file mode 100644 index 0000000..206bef3 --- /dev/null +++ b/debian/copyright @@ -0,0 +1,27 @@ +This package was debianized by Robert Lemmen <robertle@semistable.com> on +Mon, 15 Apr 2002 20:31:21 +0200. + +It was downloaded from http://www.ragel.ca/ragel/ + +Upstream Author: Adrian Thurston <thurston@cs.queensu.ca> + +Copyright: 2002-2006 Adrian Thurston + +License: + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License with + the Debian GNU/Linux distribution in file /usr/share/common-licenses/GPL; + if not, write to the Free Software Foundation, Inc., 51 Franklin St, + Fifth Floor, Boston, MA 02110-1301, USA. + + On Debian systems, the complete text of the GNU General Public + License, version 2, can be found in /usr/share/common-licenses/GPL-2. diff --git a/debian/docs b/debian/docs new file mode 100644 index 0000000..cc5135d --- /dev/null +++ b/debian/docs @@ -0,0 +1,2 @@ +TODO +ragel.vim diff --git a/debian/examples b/debian/examples new file mode 100644 index 0000000..0f0096f --- /dev/null +++ b/debian/examples @@ -0,0 +1 @@ +examples/** diff --git a/debian/manpages b/debian/manpages new file mode 100644 index 0000000..7bc45ca --- /dev/null +++ b/debian/manpages @@ -0,0 +1,2 @@ +doc/ragel.1 +doc/rlgen-*.1 diff --git a/debian/rules b/debian/rules new file mode 100755 index 0000000..2f340ba --- /dev/null +++ b/debian/rules @@ -0,0 +1,59 @@ +#!/usr/bin/make -f + +CFLAGS = -g -Wall +ifneq (,$(findstring noopt,$(DEB_BUILD_OPTIONS))) + CFLAGS += -O0 +else + CFLAGS += -O2 +endif +export CFLAGS + +# for the m68k hack +DEB_BUILD_ARCH?=$(shell dpkg-architecture -qDEB_BUILD_ARCH) +export DEB_BUILD_ARCH + +configure: + ./autogen.sh + +config.status: configure + dh_testdir + ./configure --prefix=/usr + +build: build-stamp + +build-stamp: config.status + dh_testdir + $(MAKE) SUBDIRS=ragel + touch build-stamp + +clean: + dh_testdir + dh_testroot + rm -f build-stamp + [ ! -f Makefile ] || $(MAKE) distclean + [ ! -f test/Makefile ] || $(MAKE) -C test distclean + dh_clean + +install: build + dh_testdir + dh_testroot + dh_prep + dh_installdirs + $(MAKE) install SUBDIRS=ragel DESTDIR=$(CURDIR)/debian/ragel + +binary-indep: build install + +binary-arch: build install + dh_testdir + dh_testroot + dh_link + dh_strip + dh_fixperms + dh_installdeb + dh_shlibdeps + dh_gencontrol + dh_md5sums + dh_builddeb + +binary: binary-indep binary-arch +.PHONY: build clean binary-indep binary-arch binary install diff --git a/debian/watch b/debian/watch new file mode 100644 index 0000000..a512598 --- /dev/null +++ b/debian/watch @@ -0,0 +1,7 @@ +# Example watch control file for uscan +# Rename this file to "watch" and then you can run the "uscan" command +# to check for upstream updates and more. +version=3 +# Site Directory Pattern Version Script +#http://www.ragel.ca /ragel ragel-(.*)\.tar\.gz debian uupdate +http://www.cs.queensu.ca/~thurston/ragel/ ragel-(.*)\.tar\.gz diff --git a/doc/Makefile.am b/doc/Makefile.am new file mode 100644 index 0000000..1cbc742 --- /dev/null +++ b/doc/Makefile.am @@ -0,0 +1,60 @@ +# +# Copyright 2001-2007 Adrian Thurston <thurston@complang.org> +# + +# This file is part of Ragel. +# +# Ragel is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# Ragel is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with Ragel; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + +man_MANS = ragel.1 + +dist_doc_DATA = ragel-guide.pdf + +EXTRA_DIST = ragel-guide.tex \ + bmconcat.fig bmregex.fig dropdown.fig exdoneact.fig \ + exoutact1.fig exstrongsubtr.fig lines2.fig smallscanner.fig bmnull.fig \ + comments1.fig entryguard.fig exinter.fig exoutact2.fig exsubtr.fig \ + lmkleene.fig stembed.fig bmnum.fig comments2.fig exaction.fig \ + exnegate.fig explus.fig finguard.fig opconcat.fig bmor.fig conds1.fig \ + exallact.fig exoption.fig exstact.fig leftguard.fig opor.fig \ + bmrange.fig conds2.fig exconcat.fig exor.fig exstar.fig lines1.fig \ + opstar.fig + +if BUILD_MANUAL + +.fig.pdf: + fig2dev -L pdf $< $@ + +.tex.pdf: + pdflatex -interaction=nonstopmode $< >/dev/null + pdflatex -interaction=nonstopmode $< >/dev/null + pdflatex -interaction=nonstopmode $< >/dev/null + +version.tex: Makefile + echo '|def|version{$(PACKAGE_VERSION)}' | tr '|' '\\' > version.tex + echo '|def|pubdate{$(PUBDATE)}' | tr '|' '\\' >> version.tex + +ragel-guide.pdf: version.tex + +ragel-guide.pdf: bmconcat.pdf bmregex.pdf dropdown.pdf exdoneact.pdf \ + exoutact1.pdf exstrongsubtr.pdf lines2.pdf smallscanner.pdf bmnull.pdf \ + comments1.pdf entryguard.pdf exinter.pdf exoutact2.pdf exsubtr.pdf \ + lmkleene.pdf stembed.pdf bmnum.pdf comments2.pdf exaction.pdf \ + exnegate.pdf explus.pdf finguard.pdf opconcat.pdf bmor.pdf conds1.pdf \ + exallact.pdf exoption.pdf exstact.pdf leftguard.pdf opor.pdf \ + bmrange.pdf conds2.pdf exconcat.pdf exor.pdf exstar.pdf lines1.pdf \ + opstar.pdf + +endif diff --git a/doc/RELEASE_NOTES_V6 b/doc/RELEASE_NOTES_V6 new file mode 100644 index 0000000..b08b8a3 --- /dev/null +++ b/doc/RELEASE_NOTES_V6 @@ -0,0 +1,95 @@ + + RELEASE NOTES Ragel 6.X + +This file describes the changes in Ragel version 6.X that are not backwards +compatible. For a list of all the changes see the ChangeLog file. + +Leaving Actions in Scanners (new in 6.1) +======================================== + +Scanners now ensure that any leaving actions at the end of a pattern are +executed. They are always executed before the pattern action. + +The EOF Event +============= + +There is a new execution variable called "eof". This should be set to pe on the +execution of the last buffer block. When p == eof the state machine's EOF +actions are executed. The variable is required only when EOF actions have been +embedded. + +The advantage of this over "write eof" is that EOF actions are now executed in +the same context as regular actions. They are free to manipulate p, and jump to +a new portion of the machine to reprocess input. This was not possible with +"write eof". + +The "write eof" directive was consequently removed. + +Scanners now use EOF actions to to flush out the last token, if needed. This +eliminates the need to manually flush the last token. + +Semantics of > % and Error Actions +================================== + +Ragel has gone back to the 3.X semantics for >, % and error actions. + +Those that have been using Ragel since the 3.X days will remember that the +entering operator > embedded a leaving action/priority into the start state +when it was final. The leaving operator % would embed EOF actions when the +final states stayed final all the way to the end of compilation. Also, error +actions would embed EOF actions when at the end of compilation the states the +error actions were embedded into were not final. + +The problem before was that EOF actions and regular actions were executed in +different contexts ("write exec" and "write eof"), and a single action block +could easily end up in two different functions. This could lead to compile +errors and other subtle errors. Now that all actions are executed in the same +context ("write exec") these problems go away. The original semantics has been +restored. + +Backend Automatically Executed +============================== + +The "ragel" program now automatically executes the appropriate backend. If you +need the intermediate XML format you can use the -x option. + +The fbreak Statement +==================== + +The fbreak statement now advances p. It is now possible to break out of the +machine and restart it without having to fix p first. Originally, fbreak did +not advance p because it was intended to be used to terminate processing. +Advancing p was more work than necessary in that case. But fbreak turns out to +be useful for stopping to return a token as well. In this case the failure to +advance p is an inconvenience. + +Guarded Concatenation Operators are Stronger +============================================ + +The :> :>> and <: guarded concatenation operators have been strengthened. In +the previous version of Ragel is was possible for the priority assignments to +be bypassed via the the zero length string. Running the following examples +through 5.25 you will see that the a and b actions are executed on a single +transition, showing the guard fails. This happens because the operators did not +consider that the middle machine might have a start state that is final. In 6.0 +these cases have been fixed. + + (' '@a)* <: 'x'* . ' '@b; + (' '@a)* :> 'x'? . ' '@b; + (' '@a)* :>> 'xyz'? . ' '@b; + +The tokstart and tokend Variables Renamed +========================================= + +The "tokstart" and "tokend" variables were changed to "ts" and "te". These +variables get referenced a lot in scanner actions. They should be shorter. + +To update your code simply search and replace: + tokstart => ts + tokend => te + +Options +======= + +The -l option in rlgen-cd was changed to -L because -l is used in the frontend, +which now must pass options through. diff --git a/doc/bmconcat.fig b/doc/bmconcat.fig index a47f13b..921f131 100644 --- a/doc/bmconcat.fig +++ b/doc/bmconcat.fig @@ -1,40 +1,78 @@ #FIG 3.2 -Landscape +Portrait Center Metric -A4 +A4 100.00 Single -2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: bmconcat +# Pages: 1 1200 2 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1440 450 135 135 1440 450 1575 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2115 450 135 135 2115 450 2250 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 765 450 135 135 765 450 900 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2790 450 135 135 2790 450 2925 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 3465 450 135 135 3465 450 3600 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 4140 450 135 135 4140 450 4275 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 4140 450 90 90 4140 450 4230 450 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 900 450 1305 450 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1575 450 1980 450 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 2250 450 2655 450 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 2925 450 3330 450 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 3600 450 4005 450 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 225 495 315 360 405 630 495 450 540 450 630 450 - 0.000 1.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 0 10 0.0000 4 105 75 1035 405 h\001 -4 0 0 50 0 0 10 0.0000 4 75 60 1710 405 e\001 -4 0 0 50 0 0 10 0.0000 4 105 60 2385 405 l\001 -4 0 0 50 0 0 10 0.0000 4 105 60 3060 405 l\001 -4 0 0 50 0 0 10 0.0000 4 75 75 3735 405 o\001 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 2550 33 33 33 2550 66 2583 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 2550 383 383 1333 2550 1716 2933 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 1333 2633 0\001 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 66 2550 139 2550 237 2550 354 2550 485 2550 624 2550 766 2550 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 766 2483 933 2550 766 2600 766 2483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 500 2500 IN\001 +# 5 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 9400 2550 383 383 9400 2550 9783 2933 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 9400 2550 450 450 9400 2550 9850 3000 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 9400 2633 5\001 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 2966 2550 383 383 2966 2550 3349 2933 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 2966 2633 1\001 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 1733 2550 1837 2550 1946 2550 2060 2550 2175 2550 2289 2550 2400 2550 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 2400 2483 2566 2550 2400 2600 2400 2483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 2150 2500 'h'\001 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4566 2550 383 383 4566 2550 4949 2933 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 4566 2633 2\001 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 3366 2550 3467 2550 3571 2550 3677 2550 3783 2550 3891 2550 4000 2550 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 4000 2483 4166 2550 4000 2600 4000 2483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 3766 2500 'e'\001 +# 3 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 6133 2550 383 383 6133 2550 6516 2933 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 6133 2633 3\001 +# 2 -> 3 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 4966 2550 5060 2550 5159 2550 5260 2550 5362 2550 5465 2550 5566 2550 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 5566 2483 5733 2550 5566 2600 5566 2483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 5350 2500 'l'\001 +# 4 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 7700 2550 383 383 7700 2550 8083 2933 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 7700 2633 4\001 +# 3 -> 4 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 6533 2550 6627 2550 6725 2550 6827 2550 6929 2550 7032 2550 7133 2550 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 7133 2483 7300 2550 7133 2600 7133 2483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 6916 2500 'l'\001 +# 4 -> 5 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 8100 2550 8202 2550 8309 2550 8420 2550 8534 2550 8650 2550 8766 2550 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 8766 2483 8933 2550 8766 2600 8766 2483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 8516 2500 'o'\001 +# end of FIG file diff --git a/doc/bmnull.fig b/doc/bmnull.fig index 1b85885..a56e2ad 100644 --- a/doc/bmnull.fig +++ b/doc/bmnull.fig @@ -1,15 +1,28 @@ #FIG 3.2 -Landscape +Portrait Center Metric -A4 +A4 100.00 Single -2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: bmnull +# Pages: 1 1200 2 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 765 450 135 135 765 450 900 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 765 450 90 90 765 450 855 450 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 225 495 315 360 405 630 495 450 540 450 630 450 - 0.000 1.000 1.000 1.000 1.000 0.000 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 2550 33 33 33 2550 66 2583 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 2550 383 383 1400 2550 1783 2933 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 2550 450 450 1400 2550 1850 3000 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 1400 2633 0\001 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 66 2550 132 2550 225 2550 341 2550 474 2550 617 2550 766 2550 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 766 2483 933 2550 766 2600 766 2483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 500 2500 IN\001 +# end of FIG file diff --git a/doc/bmnum.fig b/doc/bmnum.fig index 5160114..f00eec4 100644 --- a/doc/bmnum.fig +++ b/doc/bmnum.fig @@ -1,20 +1,38 @@ #FIG 3.2 -Landscape +Portrait Center Metric -A4 +A4 100.00 Single -2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: bmnum +# Pages: 1 1200 2 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 765 450 135 135 765 450 900 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1665 450 135 135 1665 450 1800 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1665 450 90 90 1665 450 1755 450 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 900 450 1530 450 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 225 495 315 360 405 630 495 450 540 450 630 450 - 0.000 1.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 0 10 0.0000 4 75 270 1035 405 num\001 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 2550 33 33 33 2550 66 2583 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 2550 383 383 1333 2550 1716 2933 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 1333 2633 0\001 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 66 2550 139 2550 237 2550 354 2550 485 2550 624 2550 766 2550 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 766 2483 933 2550 766 2600 766 2483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 500 2500 IN\001 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3066 2550 383 383 3066 2550 3449 2933 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3066 2550 450 450 3066 2550 3516 3000 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 3066 2633 1\001 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 1733 2550 1838 2550 1951 2550 2070 2550 2192 2550 2314 2550 2433 2550 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 2433 2483 2600 2550 2433 2600 2433 2483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 2166 2500 42\001 +# end of FIG file diff --git a/doc/bmor.fig b/doc/bmor.fig index 69c6da0..093c4f2 100644 --- a/doc/bmor.fig +++ b/doc/bmor.fig @@ -1,28 +1,38 @@ #FIG 3.2 -Landscape +Portrait Center Metric -A4 +A4 100.00 Single -2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: bmor +# Pages: 1 1200 2 -5 1 0 2 0 7 50 0 -1 0.000 0 1 1 0 1327.500 103.500 810 585 1305 810 1845 585 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 1 1 0 1327.500 -472.500 900 495 1305 585 1755 495 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 1327.500 796.500 810 315 1305 90 1845 315 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 1327.500 1372.500 900 405 1305 315 1755 405 - 1 1 2.00 60.00 60.00 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 765 450 135 135 765 450 900 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1890 450 90 90 1890 450 1980 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1890 450 135 135 1890 450 2025 450 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 225 495 315 360 405 630 495 450 540 450 630 450 - 0.000 1.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 0 10 0.0000 4 105 75 1305 45 h\001 -4 0 0 50 0 0 10 0.0000 4 75 60 1305 270 e\001 -4 0 0 50 0 0 10 0.0000 4 105 60 1305 540 l\001 -4 0 0 50 0 0 10 0.0000 4 75 75 1305 765 o\001 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 2550 33 33 33 2550 66 2583 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 2550 383 383 1333 2550 1716 2933 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 1333 2633 0\001 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 66 2550 139 2550 237 2550 354 2550 485 2550 624 2550 766 2550 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 766 2483 933 2550 766 2600 766 2483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 500 2500 IN\001 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3933 2550 383 383 3933 2550 4316 2933 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3933 2550 450 450 3933 2550 4383 3000 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 3933 2633 1\001 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 1733 2550 1960 2550 2217 2550 2491 2550 2771 2550 3045 2550 3300 2550 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 3300 2483 3466 2550 3300 2600 3300 2483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 2600 2500 'e', 'h', 'l', 'o'\001 +# end of FIG file diff --git a/doc/bmrange.fig b/doc/bmrange.fig index 7ad3693..7c85d8e 100644 --- a/doc/bmrange.fig +++ b/doc/bmrange.fig @@ -1,20 +1,38 @@ #FIG 3.2 -Landscape +Portrait Center Metric -A4 +A4 100.00 Single -2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: bmrange +# Pages: 1 1200 2 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 765 450 135 135 765 450 900 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1710 450 135 135 1710 450 1845 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1710 450 90 90 1710 450 1800 450 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 900 450 1575 450 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 225 495 315 360 405 630 495 450 540 450 630 450 - 0.000 1.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 0 10 0.0000 4 105 285 1080 405 l .. u\001 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 2550 33 33 33 2550 66 2583 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 2550 383 383 1333 2550 1716 2933 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 1333 2633 0\001 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 66 2550 139 2550 237 2550 354 2550 485 2550 624 2550 766 2550 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 766 2483 933 2550 766 2600 766 2483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 500 2500 IN\001 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3333 2550 383 383 3333 2550 3716 2933 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3333 2550 450 450 3333 2550 3783 3000 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 3333 2633 1\001 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 1733 2550 1881 2550 2039 2550 2204 2550 2371 2550 2538 2550 2700 2550 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 2700 2483 2866 2550 2700 2600 2700 2483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 2300 2500 'a'..'z'\001 +# end of FIG file diff --git a/doc/bmregex.fig b/doc/bmregex.fig index 5823524..881f534 100644 --- a/doc/bmregex.fig +++ b/doc/bmregex.fig @@ -1,42 +1,109 @@ -#FIG 3.2 -Landscape +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait Center Metric A4 100.00 Single -2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: bmregex +# Pages: 1 1200 2 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 3420.000 240.000 3330 360 3420 90 3510 360 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 1440.000 240.000 1350 360 1440 90 1530 360 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 2340.000 240.000 2250 360 2340 90 2430 360 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 2880.000 266.250 3375 585 2880 855 2385 585 - 1 1 2.00 60.00 60.00 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 765 450 135 135 765 450 900 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1440 450 135 135 1440 450 1575 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2340 450 135 135 2340 450 2475 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 3420 450 135 135 3420 450 3555 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 3420 450 90 90 3420 450 3510 450 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 900 450 1305 450 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1575 450 2205 450 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 2475 450 3285 450 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 225 495 315 360 405 630 495 450 540 450 630 450 - 0.000 1.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 0 10 0.0000 4 75 60 1035 405 a\001 -4 0 0 50 0 0 10 0.0000 4 105 75 1395 45 b\001 -4 0 0 50 0 12 10 0.0000 4 105 180 2250 45 df\001 -4 0 0 50 0 0 10 0.0000 4 135 315 2700 405 1,2,3\001 -4 0 0 50 0 0 10 0.0000 4 75 195 1800 405 c-z\001 -4 0 0 50 0 0 10 0.0000 4 135 315 3285 45 1,2,3\001 -4 0 0 50 0 12 10 0.0000 4 105 180 2790 810 df\001 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 3850 33 33 33 3850 66 3883 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 3850 383 383 1333 3850 1716 4233 +# 3 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 6866 3850 383 383 6866 3850 7249 4233 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 6866 3850 450 450 6866 3850 7316 4300 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4866 3850 383 383 4866 3850 5249 4233 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 2933 3850 383 383 2933 3850 3316 4233 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 3783 933 3850 766 3900 766 3783 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 6583 3350 6550 3516 6483 3350 6583 3350 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5333 4166 5200 4050 5383 4066 5333 4166 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2366 3783 2533 3850 2366 3900 2366 3783 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2700 3400 2666 3566 2600 3400 2700 3400 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4300 3783 4466 3850 4300 3900 4300 3783 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 6233 3783 6400 3850 6233 3883 6233 3783 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4633 3400 4600 3566 4533 3400 4633 3400 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + -90 2835 7515 2835 7515 4365 -90 4365 -90 2835 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 3850 139 3850 237 3850 354 3850 485 3850 624 3850 + 766 3850 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 3 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 7183 3516 7208 3410 7201 3312 7162 3227 7093 3159 6994 3115 + 6866 3100 6772 3108 6691 3131 6625 3168 6575 3218 6544 3279 + 6533 3350 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 3 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 6466 4066 6407 4091 6345 4115 6283 4137 6220 4156 6159 4172 + 6100 4183 6001 4198 5917 4209 5839 4214 5760 4212 5672 4203 + 5566 4183 5533 4174 5499 4166 5464 4156 5428 4145 5390 4132 + 5350 4116 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 1733 3850 1834 3850 1938 3850 2043 3850 2150 3850 2258 3850 + 2366 3850 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 3200 3566 3220 3467 3216 3374 3185 3291 3128 3225 3044 3182 + 2933 3166 2849 3174 2782 3197 2729 3233 2690 3280 2664 3336 + 2650 3400 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 3333 3850 3481 3850 3639 3850 3804 3850 3971 3850 4138 3850 + 4300 3850 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 2 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 5266 3850 5316 3848 5366 3845 5416 3841 5466 3837 5516 3834 + 5566 3833 5676 3833 5764 3833 5839 3833 5913 3833 5996 3833 + 6100 3833 6123 3833 6145 3833 6166 3833 6187 3833 6209 3833 + 6233 3833 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 2 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 5133 3566 5154 3467 5149 3374 5118 3291 5061 3225 4977 3182 + 4866 3166 4783 3174 4715 3197 4662 3233 4623 3280 4597 3336 + 4583 3400 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1333 3933 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 3800 IN\001 +4 1 0 0 0 0 14 0.0000 2 150 120 6866 3933 3\001 +4 1 0 0 0 0 14 0.0000 2 165 540 6866 3050 '1'..'3'\001 +4 1 0 0 0 0 14 0.0000 2 150 120 4866 3933 2\001 +4 1 0 0 0 0 14 0.0000 2 150 465 5833 4116 DEF\001 +4 1 0 0 0 0 14 0.0000 2 165 120 2933 3933 1\001 +4 1 0 0 0 0 14 0.0000 2 150 195 2133 3800 'a'\001 +4 1 0 0 0 0 14 0.0000 2 150 210 2933 3116 'b'\001 +4 1 0 0 0 0 14 0.0000 2 150 510 3900 3800 'c'..'z'\001 +4 1 0 0 0 0 14 0.0000 2 165 540 5833 3783 '1'..'3'\001 +4 1 0 0 0 0 14 0.0000 2 150 465 4866 3116 DEF\001 diff --git a/doc/comments1.fig b/doc/comments1.fig new file mode 100644 index 0000000..066de16 --- /dev/null +++ b/doc/comments1.fig @@ -0,0 +1,167 @@ +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait +Center +Metric +A4 +100.00 +Single +-2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: comments1 +# Pages: 1 +1200 2 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 5833 33 33 33 5833 66 5866 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 5833 383 383 1333 5833 1716 6216 +# 5 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 12833 5216 383 383 12833 5216 13216 5599 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 12833 5216 450 450 12833 5216 13283 5666 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4533 5833 383 383 4533 5833 4916 6216 +# 3 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 7383 5900 383 383 7383 5900 7766 6283 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 2900 5833 383 383 2900 5833 3283 6216 +# 4 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 10333 6300 383 383 10333 6300 10716 6683 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 5766 933 5833 766 5883 766 5766 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5033 5583 4866 5616 4983 5483 5033 5583 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 7933 5766 7750 5766 7900 5666 7933 5766 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2333 5766 2500 5833 2333 5883 2333 5766 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3966 5766 4133 5833 3966 5883 3966 5766 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4300 5383 4266 5550 4200 5383 4300 5383 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 6816 5800 6983 5866 6816 5900 6816 5800 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5000 6150 4866 6033 5050 6050 5000 6150 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 7150 5450 7116 5616 7050 5450 7150 5450 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 9816 6066 9966 6166 9783 6166 9816 6066 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 12300 5516 12466 5483 12350 5616 12300 5516 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4800 6316 4683 6183 4850 6216 4800 6316 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 7833 6250 7716 6116 7883 6150 7833 6250 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + 13410 4365 -45 4365 -45 6840 13410 6840 13410 4365 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 5833 139 5833 237 5833 354 5833 485 5833 624 5833 + 766 5833 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 5 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 12383 5100 11763 4966 10941 4822 9975 4700 8919 4633 7830 4655 + 6766 4800 6447 4883 6127 4993 5816 5122 5522 5261 5252 5401 + 5016 5533 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 5 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 12366 5216 11883 5230 11275 5256 10577 5302 9824 5370 9053 5468 + 8300 5600 8233 5616 8167 5633 8102 5652 8038 5671 7976 5692 + 7916 5716 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 1733 5833 1827 5833 1925 5833 2027 5833 2129 5833 2232 5833 + 2333 5833 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 3300 5833 3403 5833 3513 5833 3627 5833 3741 5833 3855 5833 + 3966 5833 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 2 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 4800 5550 4820 5450 4816 5357 4785 5275 4728 5209 4644 5165 + 4533 5150 4449 5158 4382 5180 4329 5216 4290 5263 4264 5319 + 4250 5383 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 2 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 4933 5833 4983 5832 5033 5829 5083 5825 5133 5820 5183 5817 + 5233 5816 5478 5815 5679 5812 5856 5810 6032 5809 6228 5810 + 6466 5816 6525 5823 6583 5829 6641 5833 6700 5837 6758 5842 + 6816 5850 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 3 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 7016 6033 6930 6065 6840 6093 6747 6118 6653 6139 6559 6155 + 6466 6166 6228 6204 6032 6233 5856 6247 5679 6244 5478 6218 + 5233 6166 5199 6158 5166 6149 5131 6139 5095 6128 5057 6115 + 5016 6100 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 3 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 7650 5616 7670 5517 7666 5424 7635 5341 7578 5275 7494 5232 + 7383 5216 7299 5224 7232 5247 7179 5283 7140 5330 7114 5386 + 7100 5450 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 3 -> 4 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 7783 5900 8021 5909 8296 5924 8595 5943 8909 5970 9225 6005 + 9533 6050 9576 6059 9620 6070 9666 6083 9712 6095 9757 6107 + 9800 6116 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 4 -> 5 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 10716 6200 10908 6150 11125 6085 11360 6008 11601 5919 11840 5821 + 12066 5716 12108 5691 12149 5666 12191 5641 12233 5616 12274 5591 + 12316 5566 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 4 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 10000 6516 9933 6529 9864 6537 9791 6541 9713 6546 9628 6553 + 9533 6566 8997 6655 8556 6719 8162 6758 7765 6769 7316 6750 + 6766 6700 6483 6658 6150 6602 5793 6533 5438 6453 5109 6363 + 4833 6266 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 4 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 9950 6383 9722 6416 9462 6446 9181 6464 8887 6464 8590 6440 + 8300 6383 8225 6363 8150 6339 8077 6310 8004 6277 7934 6240 + 7866 6200 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1333 5916 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 5783 IN\001 +4 1 0 0 0 0 14 0.0000 2 150 120 12833 5300 5\001 +4 1 0 0 0 0 14 0.0000 2 150 120 4533 5916 2\001 +4 1 0 0 0 0 14 0.0000 2 150 1230 8916 4600 DEF / comm\001 +4 1 0 0 0 0 14 0.0000 2 150 120 7383 5983 3\001 +4 1 0 0 0 0 14 0.0000 2 150 975 10333 5266 '*' / comm\001 +4 1 0 0 0 0 14 0.0000 2 165 120 2900 5916 1\001 +4 1 0 0 0 0 14 0.0000 2 150 150 2116 5783 '/'\001 +4 1 0 0 0 0 14 0.0000 2 90 210 3716 5783 '*'\001 +4 1 0 0 0 0 14 0.0000 2 150 1230 4533 5100 DEF / comm\001 +4 1 0 0 0 0 14 0.0000 2 150 975 5850 5766 '*' / comm\001 +4 1 0 0 0 0 14 0.0000 2 150 1230 5850 6116 DEF / comm\001 +4 1 0 0 0 0 14 0.0000 2 150 975 7383 5166 '*' / comm\001 +4 1 0 0 0 0 14 0.0000 2 150 120 10333 6383 4\001 +4 1 0 0 0 0 14 0.0000 2 150 915 8916 5883 '/' / comm\001 +4 1 0 0 0 0 14 0.0000 2 150 915 11600 5666 ' ' / comm\001 +4 1 0 0 0 0 14 0.0000 2 150 1230 7383 6650 DEF / comm\001 +4 1 0 0 0 0 14 0.0000 2 150 975 8916 6333 '*' / comm\001 diff --git a/doc/comments2.fig b/doc/comments2.fig new file mode 100644 index 0000000..3417edc --- /dev/null +++ b/doc/comments2.fig @@ -0,0 +1,110 @@ +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait +Center +Metric +A4 +100.00 +Single +-2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: comments2 +# Pages: 1 +1200 2 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 3716 33 33 33 3716 66 3749 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 3716 383 383 1333 3716 1716 4099 +# 4 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 8800 3716 383 383 8800 3716 9183 4099 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 8800 3716 450 450 8800 3716 9250 4166 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 2900 3716 383 383 2900 3716 3283 4099 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4533 3716 383 383 4533 3716 4916 4099 +# 3 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 7166 3716 383 383 7166 3716 7549 4099 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 3650 933 3716 766 3766 766 3650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2333 3650 2500 3716 2333 3766 2333 3650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3966 3650 4133 3716 3966 3766 3966 3650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4300 3266 4266 3433 4200 3266 4300 3266 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 6600 3650 6766 3716 6600 3750 6600 3650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 8166 3650 8333 3716 8166 3766 8166 3650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5000 4033 4866 3916 5050 3933 5000 4033 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 6933 3266 6900 3433 6833 3266 6933 3266 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + 9315 2745 -45 2745 -45 4185 9315 4185 9315 2745 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 3716 139 3716 237 3716 354 3716 485 3716 624 3716 + 766 3716 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 1733 3716 1827 3716 1925 3716 2027 3716 2129 3716 2232 3716 + 2333 3716 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 3300 3716 3403 3716 3513 3716 3627 3716 3741 3716 3855 3716 + 3966 3716 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 2 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 4800 3433 4820 3334 4816 3240 4785 3158 4728 3092 4644 3049 + 4533 3033 4449 3041 4382 3064 4329 3100 4290 3146 4264 3203 + 4250 3266 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 2 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 4933 3716 4983 3715 5033 3712 5083 3708 5133 3704 5183 3701 + 5233 3700 5478 3700 5679 3700 5856 3700 6032 3700 6228 3700 + 6466 3700 6490 3700 6512 3700 6533 3700 6554 3700 6576 3700 + 6600 3700 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 3 -> 4 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 7566 3716 7660 3716 7759 3716 7860 3716 7962 3716 8065 3716 + 8166 3716 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 3 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 6833 3916 6774 3948 6716 3977 6656 4002 6595 4022 6532 4038 + 6466 4050 6228 4098 6032 4127 5856 4137 5679 4127 5478 4098 + 5233 4050 5199 4041 5166 4032 5131 4022 5095 4011 5057 3998 + 5016 3983 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 3 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 7433 3433 7454 3334 7449 3240 7418 3158 7361 3092 7277 3049 + 7166 3033 7083 3041 7015 3064 6962 3100 6923 3146 6897 3203 + 6883 3266 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1333 3800 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 3666 IN\001 +4 1 0 0 0 0 14 0.0000 2 150 120 8800 3800 4\001 +4 1 0 0 0 0 14 0.0000 2 165 120 2900 3800 1\001 +4 1 0 0 0 0 14 0.0000 2 150 150 2116 3666 '/'\001 +4 1 0 0 0 0 14 0.0000 2 150 120 4533 3800 2\001 +4 1 0 0 0 0 14 0.0000 2 90 210 3716 3666 '*'\001 +4 1 0 0 0 0 14 0.0000 2 150 1230 4533 2983 DEF / comm\001 +4 1 0 0 0 0 14 0.0000 2 150 120 7166 3800 3\001 +4 1 0 0 0 0 14 0.0000 2 150 975 5850 3650 '*' / comm\001 +4 1 0 0 0 0 14 0.0000 2 150 150 7950 3666 '/'\001 +4 1 0 0 0 0 14 0.0000 2 150 1230 5850 3983 DEF / comm\001 +4 1 0 0 0 0 14 0.0000 2 150 975 7166 2983 '*' / comm\001 diff --git a/doc/conds1.fig b/doc/conds1.fig new file mode 100644 index 0000000..3436e89 --- /dev/null +++ b/doc/conds1.fig @@ -0,0 +1,96 @@ +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait +Center +Metric +A4 +100.00 +Single +-2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: conds1 +# Pages: 1 +1200 2 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 4650 33 33 33 4650 66 4683 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 4650 383 383 1400 4650 1783 5033 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 4650 450 450 1400 4650 1850 5100 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3233 4650 383 383 3233 4650 3616 5033 +# 3 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 8466 4650 383 383 8466 4650 8849 5033 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 8466 4650 450 450 8466 4650 8916 5100 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5500 5683 383 383 5500 5683 5883 6066 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 4583 933 4650 766 4700 766 4583 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2666 4583 2833 4650 2666 4700 2666 4583 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 8183 4150 8150 4316 8083 4150 8183 4150 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3783 4683 3616 4633 3783 4566 3783 4683 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5016 5400 5150 5516 4966 5500 5016 5400 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 7850 4800 8033 4800 7883 4900 7850 4800 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5266 5233 5233 5400 5166 5233 5266 5233 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + 9360 3645 -90 3645 -90 6165 9360 6165 9360 3645 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 4650 132 4650 225 4650 341 4650 474 4650 617 4650 + 766 4650 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 1866 4650 1995 4650 2129 4650 2266 4650 2403 4650 2537 4650 + 2666 4650 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 3 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 8783 4316 8808 4210 8801 4112 8762 4027 8693 3959 8594 3915 + 8466 3900 8372 3908 8291 3931 8225 3968 8175 4018 8144 4079 + 8133 4150 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 3 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 8016 4633 7959 4632 7904 4629 7852 4625 7800 4620 7750 4617 + 7700 4616 7136 4601 6679 4589 6272 4581 5865 4577 5403 4577 + 4833 4583 4661 4591 4483 4600 4302 4608 4122 4616 3947 4625 + 3783 4633 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 3600 4816 3800 4908 4029 5014 4275 5127 4525 5241 4771 5351 + 5000 5450 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 2 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 5866 5550 6147 5455 6477 5342 6835 5218 7200 5090 7550 4965 + 7866 4850 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 2 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 5766 5400 5787 5300 5782 5207 5752 5125 5695 5059 5611 5015 + 5500 5000 5416 5008 5348 5030 5295 5066 5256 5113 5230 5169 + 5216 5233 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1400 4733 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 4600 IN\001 +4 1 0 0 0 0 14 0.0000 2 165 120 3233 4733 1\001 +4 1 0 0 0 0 14 0.0000 2 165 360 2350 4600 100\001 +4 1 0 0 0 0 14 0.0000 2 150 120 8466 4733 3\001 +4 1 0 0 0 0 14 0.0000 2 210 1590 8466 3850 97..122(test_len)\001 +4 1 0 0 0 0 14 0.0000 2 210 1305 5500 4533 100(!test_len)\001 +4 1 0 0 0 0 14 0.0000 2 150 120 5500 5766 2\001 +4 1 0 0 0 0 14 0.0000 2 150 600 4233 4966 48..57\001 +4 1 0 0 0 0 14 0.0000 2 180 1245 7083 4900 58 / rec_num\001 +4 1 0 0 0 0 14 0.0000 2 150 600 5500 4950 48..57\001 diff --git a/doc/conds2.fig b/doc/conds2.fig new file mode 100644 index 0000000..d5a5557 --- /dev/null +++ b/doc/conds2.fig @@ -0,0 +1,107 @@ +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait +Center +Metric +A4 +100.00 +Single +-2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: conds2 +# Pages: 1 +1200 2 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 4350 33 33 33 4350 66 4383 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 4350 383 383 1333 4350 1716 4733 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 11633 5016 383 383 11633 5016 12016 5399 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 11633 5016 450 450 11633 5016 12083 5466 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 9183 4433 383 383 9183 4433 9566 4816 +# 3 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4766 5383 383 383 4766 5383 5149 5766 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 4283 933 4350 766 4400 766 4283 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 8616 4316 8783 4383 8616 4416 8616 4316 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4233 5166 4383 5266 4200 5266 4233 5166 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 11033 4816 11183 4916 11000 4916 11033 4816 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 8950 3983 8916 4150 8850 3983 8950 3983 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 11016 5050 11183 5083 11016 5150 11016 5050 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 8633 4500 8800 4516 8650 4600 8633 4500 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4533 4933 4500 5100 4433 4933 4533 4933 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + 12285 3465 -135 3465 -135 5895 12285 5895 12285 3465 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 4350 139 4350 237 4350 354 4350 485 4350 624 4350 + 766 4350 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 25 + 1716 4283 1773 4282 1828 4278 1881 4272 1932 4266 1983 4258 + 2033 4250 2352 4232 2611 4227 2839 4231 3066 4238 3321 4246 + 3633 4250 4551 4259 5297 4260 5956 4260 6613 4267 7355 4288 + 8266 4333 8325 4340 8383 4345 8441 4350 8499 4354 8558 4359 + 8616 4366 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 0.000 +# 0 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 1716 4466 2062 4570 2479 4694 2935 4829 3398 4966 3835 5098 + 4216 5216 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 9566 4516 9774 4564 10009 4622 10260 4685 10518 4750 10773 4811 + 11016 4866 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 9450 4150 9470 4050 9466 3957 9435 3875 9378 3809 9294 3765 + 9183 3750 9099 3758 9032 3780 8979 3816 8940 3863 8914 3919 + 8900 3983 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 3 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 5166 5383 5685 5380 6364 5370 7158 5352 8024 5324 8919 5284 + 9800 5233 10010 5215 10222 5195 10433 5172 10638 5149 10833 5124 + 11016 5100 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 3 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 5150 5300 5623 5200 6208 5075 6854 4937 7508 4796 8118 4663 + 8633 4550 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 3 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 5033 5100 5054 5000 5049 4907 5018 4825 4961 4759 4877 4715 + 4766 4700 4683 4708 4615 4730 4562 4766 4523 4813 4497 4869 + 4483 4933 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1333 4433 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 4300 IN\001 +4 1 0 0 0 0 14 0.0000 2 150 120 11633 5100 2\001 +4 1 0 0 0 0 14 0.0000 2 165 120 9183 4516 1\001 +4 1 0 0 0 0 14 0.0000 2 210 1665 4766 4216 97..122(!test_len)\001 +4 1 0 0 0 0 14 0.0000 2 150 120 4766 5466 3\001 +4 1 0 0 0 0 14 0.0000 2 210 1590 2833 4533 97..122(test_len)\001 +4 1 0 0 0 0 14 0.0000 2 165 780 10483 4633 10 / two\001 +4 1 0 0 0 0 14 0.0000 2 195 1440 9183 3700 48..57, 97..122\001 +4 1 0 0 0 0 14 0.0000 2 195 1245 9183 5183 10 / one, two\001 +4 1 0 0 0 0 14 0.0000 2 210 2385 7083 4616 48..57, 97..122(!test_len)\001 +4 1 0 0 0 0 14 0.0000 2 210 1590 4766 4650 97..122(test_len)\001 diff --git a/doc/dropdown.fig b/doc/dropdown.fig new file mode 100644 index 0000000..29fefec --- /dev/null +++ b/doc/dropdown.fig @@ -0,0 +1,107 @@ +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait +Center +Metric +A4 +100.00 +Single +-2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: dropdown +# Pages: 1 +1200 2 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 3950 33 33 33 3950 66 3983 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 3950 383 383 1333 3950 1716 4333 +# 3 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 9266 4133 383 383 9266 4133 9649 4516 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 9266 4133 450 450 9266 4133 9716 4583 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5383 3550 383 383 5383 3550 5766 3933 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 7566 4133 383 383 7566 4133 7949 4516 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 3883 933 3950 766 4000 766 3883 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1100 3500 1066 3666 1000 3500 1100 3500 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4816 3433 4983 3500 4816 3533 4816 3433 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1883 4066 1716 4000 1883 3966 1883 4066 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 7033 3933 7183 4033 7000 4033 7033 3933 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 8633 4066 8800 4133 8633 4183 8633 4066 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1833 4216 1666 4150 1833 4116 1833 4216 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 7333 3683 7300 3850 7233 3683 7333 3683 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + -90 2970 9765 2970 9765 4680 -90 4680 -90 2970 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 3950 139 3950 237 3950 354 3950 485 3950 624 3950 + 766 3950 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 1600 3666 1620 3567 1616 3474 1585 3391 1528 3325 1444 3282 + 1333 3266 1249 3274 1182 3297 1129 3333 1090 3380 1064 3436 + 1050 3500 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 1666 3750 1726 3725 1787 3700 1850 3677 1912 3654 1973 3634 + 2033 3616 2528 3528 3036 3474 3537 3450 4013 3447 4446 3460 + 4816 3483 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 5033 3716 4892 3768 4738 3823 4575 3877 4406 3926 4235 3968 + 4066 4000 3689 4046 3300 4070 2912 4077 2538 4067 2190 4046 + 1883 4016 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 5766 3650 5954 3695 6161 3747 6379 3804 6600 3863 6815 3923 + 7016 3983 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 2 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 7966 4133 8069 4133 8176 4133 8287 4133 8401 4133 8516 4133 + 8633 4133 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 2 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 7183 4233 7062 4258 6935 4282 6804 4304 6669 4323 6534 4339 + 6400 4350 5539 4415 4838 4443 4216 4433 3595 4384 2893 4295 + 2033 4166 1999 4166 1966 4166 1933 4166 1899 4166 1866 4166 + 1833 4166 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 2 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 7833 3850 7854 3750 7849 3657 7818 3575 7761 3509 7677 3465 + 7566 3450 7483 3458 7415 3480 7362 3516 7323 3563 7297 3619 + 7283 3683 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1333 4033 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 3900 IN\001 +4 1 0 0 0 0 14 0.0000 2 150 120 9266 4216 3\001 +4 1 0 0 0 0 14 0.0000 2 150 1170 1333 3216 DEF / bchar\001 +4 1 0 0 0 0 14 0.0000 2 165 120 5383 3633 1\001 +4 1 0 0 0 0 14 0.0000 2 195 165 3050 3416 ']'\001 +4 1 0 0 0 0 14 0.0000 2 195 2055 3050 3950 DEF / bbrack1, bchar\001 +4 1 0 0 0 0 14 0.0000 2 150 120 7566 4216 2\001 +4 1 0 0 0 0 14 0.0000 2 195 165 6783 3883 ']'\001 +4 1 0 0 0 0 14 0.0000 2 150 225 8383 4083 '>'\001 +4 1 0 0 0 0 14 0.0000 2 195 2055 5383 4300 DEF / bbrack2, bchar\001 +4 1 0 0 0 0 14 0.0000 2 210 1110 7566 3400 ']' / bbrack1\001 diff --git a/doc/entryguard.fig b/doc/entryguard.fig new file mode 100644 index 0000000..c848716 --- /dev/null +++ b/doc/entryguard.fig @@ -0,0 +1,73 @@ +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait +Center +Metric +A4 +100.00 +Single +-2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: exstpri +# Pages: 1 +1200 2 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 3550 33 33 33 3550 66 3583 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 3550 383 383 1333 3550 1716 3933 +# 3 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 6266 3550 383 383 6266 3550 6649 3933 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 6266 3550 450 450 6266 3550 6716 4000 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 2966 3550 383 383 2966 3550 3349 3933 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4533 3550 383 383 4533 3550 4916 3933 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 3483 933 3550 766 3600 766 3483 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1100 3100 1066 3266 1000 3100 1100 3100 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2400 3483 2566 3550 2400 3600 2400 3483 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3966 3483 4133 3550 3966 3600 3966 3483 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5633 3483 5800 3550 5633 3600 5633 3483 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + -45 2565 6795 2565 6795 4185 -45 4185 -45 2565 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 3550 139 3550 237 3550 354 3550 485 3550 624 3550 + 766 3550 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 1600 3266 1620 3167 1616 3074 1585 2991 1528 2925 1444 2882 + 1333 2866 1249 2874 1182 2897 1129 2933 1090 2980 1064 3036 + 1050 3100 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 1733 3550 1837 3550 1946 3550 2060 3550 2175 3550 2289 3550 + 2400 3550 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 3366 3550 3460 3550 3559 3550 3660 3550 3762 3550 3865 3550 + 3966 3550 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 2 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 4933 3550 5038 3550 5151 3550 5270 3550 5392 3550 5514 3550 + 5633 3550 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1333 3633 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 3500 IN\001 +4 1 0 0 0 0 14 0.0000 2 150 120 6266 3633 3\001 +4 1 0 0 0 0 14 0.0000 2 150 465 1333 2816 DEF\001 +4 1 0 0 0 0 14 0.0000 2 165 120 2966 3633 1\001 +4 1 0 0 0 0 14 0.0000 2 150 225 2150 3500 'F'\001 +4 1 0 0 0 0 14 0.0000 2 150 120 4533 3633 2\001 +4 1 0 0 0 0 14 0.0000 2 150 165 3750 3500 'I'\001 +4 1 0 0 0 0 14 0.0000 2 150 270 5366 3500 'N'\001 diff --git a/doc/exaction.fig b/doc/exaction.fig index e41ef2e..9829f06 100644 --- a/doc/exaction.fig +++ b/doc/exaction.fig @@ -1,37 +1,64 @@ -#FIG 3.2 Produced by xfig version 3.2.5-alpha5 -Landscape +#FIG 3.2 +# Generated by Graphviz version 2.12 (Tue Sep 4 16:56:48 UTC 2007) +# For: (thurston) Adrian Thurston,,, +# Title: exaction +# Pages: 1 +Portrait Center -Metric -A4 +Inches +Letter 100.00 Single -2 1200 2 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 1620.000 400.500 1530 495 1620 270 1710 495 - 1 1 2.00 60.00 60.00 -6 1377 810 1872 990 -4 0 0 50 0 0 10 0.0000 4 120 315 1557 945 /C,N\001 -4 0 0 50 0 12 10 0.0000 4 105 180 1377 945 nl\001 --6 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 585 585 135 135 585 585 720 585 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1620 585 135 135 1620 585 1755 585 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2655 585 135 135 2655 585 2790 585 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2655 585 90 90 2655 585 2745 585 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 720 585 1485 585 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1755 585 2520 585 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 45 630 135 495 225 765 315 585 360 585 450 585 - 0.000 1.000 1.000 1.000 1.000 0.000 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 630 720 765 900 1305 1035 1935 1035 2475 900 2610 720 - 0.000 1.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 0 10 0.0000 4 120 495 855 540 a-z/A,B\001 -4 0 0 50 0 0 10 0.0000 4 105 330 1485 225 a-z/B\001 -4 0 0 50 0 12 10 0.0000 4 105 180 1935 540 nl\001 -4 0 0 50 0 0 10 0.0000 4 120 315 2115 540 /C,N\001 +0 32 #d3d3d3 +2 3 0 1 7 7 2 0 20 0.0 0 0 0 0 0 5 + 0 2460 0 0 8680 0 8680 2460 0 2460 +# ENTRY +1 1 0 1 0 0 1 0 20 0.000 0 0.0000 120 720 40 -40 120 720 160 680 +# 1 +1 1 0 1 0 7 1 0 -1 0.000 0 0.0000 1640 720 420 -430 1640 720 2060 290 +4 1 0 1 0 0 14.0 0.0000 4 0.0 0.0 1640 800 1\001 +# ENTRY->1 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 160 720 247 720 364 720 505 720 662 720 829 720 1000 720 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 1000 650 1200 720 1000 790 1000 650 +4 1 0 0 0 0 14.0 0.0000 4 0.0 0.0 680 600 IN\001 +# 3 +1 1 0 1 0 7 1 0 -1 0.000 0 0.0000 8080 720 422 -430 8080 720 8502 290 +1 1 0 1 0 7 1 0 -1 0.000 0 0.0000 8080 720 500 -510 8080 720 8580 210 +4 1 0 1 0 0 14.0 0.0000 4 0.0 0.0 8080 800 3\001 +# 1->3 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 19 + 2060 620 2121 610 2185 600 2250 590 2315 580 2379 570 2440 560 3383 488 4150 465 4828 480 5504 521 6265 579 7200 640 7229 641 7255 645 7280 650 7305 655 7331 659 7360 660 + 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 7368 590 7560 680 7354 730 7368 590 +4 1 0 0 0 0 14.0 0.0000 4 0.0 0.0 4980 360 '\\n' / A, C, N\001 +# 2 +1 1 0 1 0 7 1 0 -1 0.000 0 0.0000 4980 1960 420 -430 4980 1960 5400 1530 +4 1 0 1 0 0 14.0 0.0000 4 0.0 0.0 4980 2040 2\001 +# 1->2 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 2040 860 2376 985 2767 1133 3188 1293 3613 1454 4019 1607 4380 1740 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 4409 1675 4580 1800 4368 1810 4409 1675 +4 1 0 0 0 0 14.0 0.0000 4 0.0 0.0 3160 920 'a'..'z' / A, B\001 +# 2->3 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 5380 1800 5669 1681 6002 1547 6363 1405 6731 1259 7090 1116 7420 980 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 7389 917 7600 900 7446 1045 7389 917 +4 1 0 0 0 0 14.0 0.0000 4 0.0 0.0 6640 1000 '\\n' / C, N\001 +# 2->2 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 13 + 4580 1780 4432 1645 4368 1509 4390 1383 4499 1278 4695 1206 4980 1180 5220 1199 5401 1250 5523 1328 5586 1423 5592 1530 5540 1640 + 0 1 1 1 1 1 1 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 5577 1701 5380 1780 5484 1596 5577 1701 +4 1 0 0 0 0 14.0 0.0000 4 0.0 0.0 4980 1060 'a'..'z' / B\001 +# end of FIG file diff --git a/doc/exallact.fig b/doc/exallact.fig index 40f4fcb..593ffd3 100644 --- a/doc/exallact.fig +++ b/doc/exallact.fig @@ -1,25 +1,48 @@ #FIG 3.2 -Landscape +Portrait Center Metric -A4 +A4 100.00 Single -2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: exallact +# Pages: 1 1200 2 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 630 495 135 135 630 495 765 495 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1530 495 135 135 1530 495 1665 495 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2430 495 135 135 2430 495 2565 495 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2430 495 90 90 2430 495 2520 495 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 765 495 1395 495 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1665 495 2295 495 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 90 540 180 405 270 675 360 495 405 495 495 495 - 0.000 1.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 0 10 0.0000 4 105 285 945 450 m/A\001 -4 0 0 50 0 0 10 0.0000 4 135 360 1800 450 1,2/A\001 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 2550 33 33 33 2550 66 2583 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 2550 383 383 1333 2550 1716 2933 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 1333 2633 0\001 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 66 2550 139 2550 237 2550 354 2550 485 2550 624 2550 766 2550 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 766 2483 933 2550 766 2600 766 2483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 500 2500 IN\001 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5733 2550 383 383 5733 2550 6116 2933 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5733 2550 450 450 5733 2550 6183 3000 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 5733 2633 2\001 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3366 2550 383 383 3366 2550 3749 2933 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 3366 2633 1\001 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 1733 2550 1893 2550 2069 2550 2254 2550 2441 2550 2626 2550 2800 2550 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 2800 2483 2966 2550 2800 2600 2800 2483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 2350 2500 'm' / A\001 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 3766 2550 3962 2550 4179 2550 4408 2550 4643 2550 4876 2550 5100 2550 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 5100 2483 5266 2550 5100 2600 5100 2483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 4516 2500 '1'..'2' / A\001 +# end of FIG file diff --git a/doc/exconcat.fig b/doc/exconcat.fig index 21bf76f..e634139 100644 --- a/doc/exconcat.fig +++ b/doc/exconcat.fig @@ -1,93 +1,217 @@ -#FIG 3.2 -Landscape +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait Center Metric A4 100.00 Single -2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: exconcat +# Pages: 1 1200 2 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1845 1080 135 135 1845 1080 1980 1080 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 3105 1080 135 135 3105 1080 3240 1080 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 3105 1080 90 90 3105 1080 3195 1080 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1845 1845 135 135 1845 1845 1980 1845 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1125 1575 135 135 1125 1575 1260 1575 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 585 1080 135 135 585 1080 720 1080 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2565 1575 135 135 2565 1575 2700 1575 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 720 1080 1710 1080 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1215 1485 1755 1170 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 675 1170 1035 1485 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1260 1620 1710 1800 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1845 1710 1845 1215 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 2655 1485 3015 1170 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 2970 1080 1980 1080 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1980 1800 2430 1620 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 2475 1485 1935 1170 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 5 - 1 1 2.00 60.00 60.00 - 1755 1935 1485 2115 900 2070 405 1530 495 1170 - 0.000 1.000 1.000 1.000 0.000 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 5 - 1 1 2.00 60.00 60.00 - 1035 1665 945 1755 765 1755 540 1530 585 1215 - 0.000 1.000 1.000 1.000 0.000 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 3 - 1 1 2.00 60.00 60.00 - 1755 990 1215 675 675 990 - 0.000 1.000 0.000 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 450 1035 225 810 180 675 225 630 315 675 540 945 - 0.000 1.000 1.000 1.000 1.000 0.000 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 1800 945 1800 765 1800 675 1890 675 1890 810 1890 945 - 0.000 1.000 1.000 1.000 1.000 0.000 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 5 - 1 1 2.00 60.00 60.00 - 3105 945 3105 405 900 405 675 765 630 945 - 0.000 1.000 1.000 1.000 0.000 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 45 1125 135 990 225 1260 315 1080 360 1080 450 1080 - 0.000 1.000 1.000 1.000 1.000 0.000 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 9 - 1 1 2.00 60.00 60.00 - 3105 1215 3105 1350 3060 1620 2880 1845 2565 2070 2115 2160 - 1710 2115 1350 1980 1170 1710 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 10216 33 33 33 10216 66 10249 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 10216 383 383 1333 10216 1716 10599 +# 5 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 8466 9950 383 383 8466 9950 8849 10333 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 8466 9950 450 450 8466 9950 8916 10400 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3000 9283 383 383 3000 9283 3383 9666 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 10400 9283 383 383 10400 9283 10783 9666 +# 3 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4666 8466 383 383 4666 8466 5049 8849 +# 4 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 6533 9000 383 383 6533 9000 6916 9383 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 10150 933 10216 766 10266 766 10150 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1866 10400 1716 10316 1883 10300 1866 10400 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3550 9400 3383 9333 3550 9300 3550 9400 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 9850 9416 10033 9416 9883 9516 9850 9416 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1100 9766 1066 9933 1000 9766 1100 9766 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2566 9650 2733 9583 2650 9733 2566 9650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 10116 9750 10250 9633 10216 9800 10116 9750 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1750 9833 1583 9916 1666 9750 1750 9833 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 10250 8750 10283 8916 10150 8800 10250 8750 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4150 8666 4316 8633 4200 8766 4150 8666 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1400 10783 1383 10600 1500 10733 1400 10783 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 10166 8833 10133 9000 10066 8833 10166 8833 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1500 9716 1366 9833 1400 9666 1500 9716 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 9916 8983 10066 9083 9883 9083 9916 8983 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 6000 8800 6150 8900 5966 8900 6000 8800 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 7916 9650 8050 9766 7866 9750 7916 9650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 9850 9183 10016 9250 9850 9283 9850 9183 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + 11160 7065 -45 7065 -45 12015 11160 12015 11160 7065 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 10216 139 10216 237 10216 354 10216 485 10216 624 10216 + 766 10216 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 5 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 8050 10133 7841 10211 7604 10290 7347 10364 7078 10425 6804 10467 + 6533 10483 5044 10483 4046 10483 3441 10483 3130 10483 3016 10483 + 3000 10483 2806 10478 2610 10463 2416 10441 2228 10414 2049 10383 + 1883 10350 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 5 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 8000 9900 7398 9822 6642 9727 5806 9625 4962 9522 4186 9427 + 3550 9350 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 5 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 8900 9800 9049 9747 9209 9691 9377 9633 9545 9575 9710 9519 + 9866 9466 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 1600 9933 1620 9834 1616 9740 1585 9658 1528 9592 1444 9549 + 1333 9533 1249 9541 1182 9564 1129 9600 1090 9646 1064 9703 + 1050 9766 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 0 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 1716 10150 1810 10122 1908 10089 2008 10052 2108 10010 2205 9965 + 2300 9916 2356 9889 2411 9856 2462 9820 2511 9782 2556 9741 + 2600 9700 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 1533 10550 1735 10781 1935 10961 2147 11093 2386 11183 2665 11234 + 3000 11250 3025 11250 3202 11250 3683 11250 4619 11250 6163 11250 + 8466 11250 8921 11199 9270 11054 9541 10829 9762 10533 9962 10181 + 10166 9783 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 2 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 2616 9366 2522 9393 2424 9422 2325 9456 2225 9493 2127 9536 + 2033 9583 1963 9623 1902 9661 1847 9697 1797 9732 1748 9766 + 1700 9800 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 2 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 25 + 3250 8983 3275 8967 3300 8953 3325 8937 3350 8919 3375 8896 + 3400 8866 3659 8554 3824 8222 3952 7906 4097 7638 4317 7452 + 4666 7383 4684 7383 4807 7383 5141 7383 5792 7383 6865 7383 + 8466 7383 8911 7429 9260 7560 9539 7768 9772 8045 9984 8380 + 10200 8766 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 12 10 0.0000 4 105 180 675 1575 nl\001 -4 0 0 50 0 0 10 0.0000 4 105 90 855 1260 E\001 -4 0 0 50 0 12 10 0.0000 4 105 180 1125 720 nl\001 -4 0 0 50 0 12 10 0.0000 4 105 180 1125 1035 df\001 -4 0 0 50 0 12 10 0.0000 4 105 180 180 585 nl\001 -4 0 0 50 0 12 10 0.0000 4 105 180 1755 630 df\001 -4 0 0 50 0 12 10 0.0000 4 105 180 2475 1035 df\001 -4 0 0 50 0 12 10 0.0000 4 105 180 990 1980 nl\001 -4 0 0 50 0 12 10 0.0000 4 105 180 1755 360 nl\001 -4 0 0 50 0 12 10 0.0000 4 105 180 2205 1305 df\001 -4 0 0 50 0 12 10 0.0000 4 105 180 2655 1305 nl\001 -4 0 0 50 0 12 10 0.0000 4 105 180 1305 1305 df\001 -4 0 0 50 0 0 10 0.0000 4 105 105 1485 1665 O\001 -4 0 0 50 0 0 10 0.0000 4 105 90 2115 1665 F\001 -4 0 0 50 0 12 10 0.0000 4 105 180 1620 1485 df\001 -4 0 0 50 0 0 10 0.0000 4 105 90 2295 2025 E\001 +# 2 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 3350 9116 3472 9055 3606 8990 3745 8922 3888 8853 4029 8784 + 4166 8716 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 31 + 10366 9666 10365 9691 10362 9715 10358 9737 10354 9756 10351 9772 + 10350 9783 10276 10083 10211 10331 10137 10545 10038 10746 9898 10952 + 9700 11183 9509 11404 9339 11583 9170 11722 8982 11821 8754 11880 + 8466 11900 6163 11900 4619 11900 3683 11900 3202 11900 3025 11900 + 3000 11900 2609 11865 2294 11764 2037 11600 1822 11374 1631 11089 + 1450 10750 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 10666 9000 10687 8900 10682 8807 10652 8725 10595 8659 10511 8615 + 10400 8600 10316 8608 10248 8630 10195 8666 10156 8713 10130 8769 + 10116 8833 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 3 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 4266 8466 4030 8471 3764 8488 3477 8520 3180 8572 2884 8647 + 2600 8750 2326 8880 2098 8999 1906 9122 1740 9267 1591 9448 + 1450 9683 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 3 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 31 + 5050 8400 5107 8391 5161 8383 5214 8375 5266 8366 5316 8358 + 5366 8350 5462 8344 5539 8342 5606 8343 5671 8346 5744 8348 + 5833 8350 6455 8390 6959 8432 7402 8483 7840 8551 8332 8643 + 8933 8766 9088 8804 9213 8837 9322 8868 9430 8901 9551 8938 + 9700 8983 9740 8991 9777 9000 9812 9008 9844 9016 9873 9025 + 9900 9033 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 3 -> 4 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 5050 8566 5190 8609 5343 8654 5504 8702 5667 8750 5828 8800 + 5983 8850 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 4 -> 5 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 6866 9200 6926 9233 6987 9266 7050 9297 7112 9328 7173 9357 + 7233 9383 7342 9434 7454 9487 7566 9541 7679 9595 7790 9648 + 7900 9700 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 4 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 6933 9016 7047 9024 7172 9032 7304 9039 7438 9045 7571 9048 + 7700 9050 8099 9080 8422 9108 8706 9133 8988 9158 9307 9185 + 9700 9216 9725 9217 9750 9220 9775 9225 9800 9229 9825 9232 + 9850 9233 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1333 10300 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 10166 IN\001 +4 1 0 0 0 0 14 0.0000 2 150 120 8466 10033 5\001 +4 1 0 0 0 0 14 0.0000 2 165 240 4666 10433 10\001 +4 1 0 0 0 0 14 0.0000 2 150 120 3000 9366 2\001 +4 1 0 0 0 0 14 0.0000 2 150 240 5600 9550 'E'\001 +4 1 0 0 0 0 14 0.0000 2 165 120 10400 9366 1\001 +4 1 0 0 0 0 14 0.0000 2 150 465 9466 9500 DEF\001 +4 1 0 0 0 0 14 0.0000 2 165 240 1333 9483 10\001 +4 1 0 0 0 0 14 0.0000 2 150 240 2166 9866 'E'\001 +4 1 0 0 0 0 14 0.0000 2 150 465 5600 11200 DEF\001 +4 1 0 0 0 0 14 0.0000 2 165 240 2166 9450 10\001 +4 1 0 0 0 0 14 0.0000 2 150 465 6533 7333 DEF\001 +4 1 0 0 0 0 14 0.0000 2 150 120 4666 8550 3\001 +4 1 0 0 0 0 14 0.0000 2 150 270 3833 8800 'O'\001 +4 1 0 0 0 0 14 0.0000 2 165 240 5600 11850 10\001 +4 1 0 0 0 0 14 0.0000 2 150 465 10400 8550 DEF\001 +4 1 0 0 0 0 14 0.0000 2 165 240 3000 8516 10\001 +4 1 0 0 0 0 14 0.0000 2 150 465 7466 8433 DEF\001 +4 1 0 0 0 0 14 0.0000 2 150 120 6533 9083 4\001 +4 1 0 0 0 0 14 0.0000 2 150 225 5600 8633 'F'\001 +4 1 0 0 0 0 14 0.0000 2 165 240 7466 9333 10\001 +4 1 0 0 0 0 14 0.0000 2 150 465 8466 9050 DEF\001 diff --git a/doc/exdoneact.fig b/doc/exdoneact.fig index a9904af..3930e5a 100644 --- a/doc/exdoneact.fig +++ b/doc/exdoneact.fig @@ -1,24 +1,51 @@ -#FIG 3.2 -Landscape +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait Center Metric A4 100.00 Single -2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: exdoneact +# Pages: 1 1200 2 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 630.000 310.500 540 405 630 180 720 405 - 1 1 2.00 60.00 60.00 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 630 495 135 135 630 495 765 495 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1530 495 90 90 1530 495 1620 495 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1530 495 135 135 1530 495 1665 495 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 765 495 1395 495 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 90 540 180 405 270 675 360 495 405 495 495 495 - 0.000 1.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 0 10 0.0000 4 75 195 540 135 a-z\001 -4 0 0 50 0 12 10 0.0000 4 105 180 900 450 sp\001 -4 0 0 50 0 0 10 0.0000 4 105 165 1080 450 /A\001 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 3550 33 33 33 3550 66 3583 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 3550 383 383 1333 3550 1716 3933 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3333 3550 383 383 3333 3550 3716 3933 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3333 3550 450 450 3333 3550 3783 4000 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 3483 933 3550 766 3600 766 3483 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2700 3483 2866 3550 2700 3600 2700 3483 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1100 3100 1066 3266 1000 3100 1100 3100 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + -270 2610 3915 2610 3915 4140 -270 4140 -270 2610 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 3550 139 3550 237 3550 354 3550 485 3550 624 3550 + 766 3550 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 1733 3550 1881 3550 2039 3550 2204 3550 2371 3550 2538 3550 + 2700 3550 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 1600 3266 1620 3167 1616 3074 1585 2991 1528 2925 1444 2882 + 1333 2866 1249 2874 1182 2897 1129 2933 1090 2980 1064 3036 + 1050 3100 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1333 3633 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 3500 IN\001 +4 1 0 0 0 0 14 0.0000 2 165 120 3333 3633 1\001 +4 1 0 0 0 0 14 0.0000 2 150 510 2300 3500 ' ' / A\001 +4 1 0 0 0 0 14 0.0000 2 150 510 1333 2816 'a'..'z'\001 diff --git a/doc/exinter.fig b/doc/exinter.fig index 51bc5df..cc802f6 100644 --- a/doc/exinter.fig +++ b/doc/exinter.fig @@ -1,48 +1,75 @@ #FIG 3.2 -Landscape +Portrait Center Metric -A4 +A4 100.00 Single -2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: exinter +# Pages: 1 1200 2 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 1125.000 1777.500 765 360 1125 315 1485 360 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 1 1 0 1125.000 -877.500 765 540 1125 585 1485 540 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 1 1 0 2025.000 -877.500 1665 540 2025 585 2385 540 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 1 1 0 2925.000 -877.500 2565 540 2925 585 3285 540 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 1 1 0 3825.000 -877.500 3465 540 3825 585 4185 540 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 2025.000 1777.500 1665 360 2025 315 2385 360 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 2925.000 1777.500 2565 360 2925 315 3285 360 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 3825.000 1777.500 3465 360 3825 315 4185 360 - 1 1 2.00 60.00 60.00 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 675 450 135 135 675 450 810 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1575 450 135 135 1575 450 1710 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2475 450 135 135 2475 450 2610 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 3375 450 135 135 3375 450 3510 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 4275 450 135 135 4275 450 4410 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 4275 450 90 90 4275 450 4365 450 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 135 495 225 360 315 630 405 450 450 450 540 450 - 0.000 1.000 1.000 1.000 1.000 0.000 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 5 - 1 1 2.00 60.00 60.00 - 4275 585 4320 990 2475 1215 630 990 675 585 - 0.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 12 10 0.0000 4 105 180 2385 1080 nl\001 -4 0 0 50 0 12 10 0.0000 4 105 180 1035 540 sp\001 -4 0 0 50 0 12 10 0.0000 4 105 180 1935 540 sp\001 -4 0 0 50 0 12 10 0.0000 4 105 180 2835 540 sp\001 -4 0 0 50 0 12 10 0.0000 4 105 180 3735 540 sp\001 -4 0 0 50 0 0 10 0.0000 4 75 195 3735 270 a-z\001 -4 0 0 50 0 0 10 0.0000 4 75 195 2835 270 a-z\001 -4 0 0 50 0 0 10 0.0000 4 75 195 1935 270 a-z\001 -4 0 0 50 0 0 10 0.0000 4 75 195 1035 270 a-z\001 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 3816 33 33 33 3816 66 3849 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 3816 383 383 1400 3816 1783 4199 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 3816 450 450 1400 3816 1850 4266 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 1400 3900 0\001 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 66 3816 132 3816 225 3816 341 3816 474 3816 617 3816 766 3816 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 766 3750 933 3816 766 3866 766 3750 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 500 3766 IN\001 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3666 3266 383 383 3666 3266 4049 3649 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 3666 3350 1\001 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 1850 3700 2044 3654 2256 3603 2477 3550 2699 3496 2915 3445 3116 3400 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 3116 3350 3283 3366 3133 3450 3116 3350 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 2566 3416 ' ', 'a'..'z'\001 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5866 3216 383 383 5866 3216 6249 3599 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 5866 3300 2\001 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 4066 3250 4252 3248 4456 3245 4670 3241 4887 3237 5099 3234 5300 3233 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 5300 3166 5466 3233 5300 3283 5300 3166 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 4766 3200 ' ', 'a'..'z'\001 +# 3 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 8066 3300 383 383 8066 3300 8449 3683 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 8066 3383 3\001 +# 2 -> 3 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 6266 3233 6452 3241 6656 3249 6870 3258 7087 3266 7299 3274 7500 3283 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 7500 3216 7666 3283 7500 3333 7500 3216 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 6966 3216 ' ', 'a'..'z'\001 +# 4 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 10266 3816 383 383 10266 3816 10649 4199 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 10266 3900 4\001 +# 3 -> 4 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 8450 3383 8637 3428 8845 3479 9064 3533 9288 3587 9508 3637 9716 3683 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 9733 3633 9883 3716 9716 3733 9733 3633 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 9166 3433 ' ', 'a'..'z'\001 +# 4 -> 0 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 19 + 9883 3916 9643 3969 9364 4025 9056 4079 8730 4124 8396 4155 8066 4166 6212 4166 4970 4166 4216 4166 3829 4166 3687 4166 3666 4166 3378 4158 3087 4137 2797 4104 2518 4062 2255 4016 2016 3966 + 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 2000 4016 1850 3933 2016 3916 2000 4016 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 5866 4116 10\001 +# end of FIG file diff --git a/doc/exnegate.fig b/doc/exnegate.fig index ceb4a90..487fda5 100644 --- a/doc/exnegate.fig +++ b/doc/exnegate.fig @@ -1,31 +1,52 @@ -#FIG 3.2 -Landscape +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait Center Metric A4 100.00 Single -2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: exnegate +# Pages: 1 1200 2 -6 1350 180 1710 765 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 1530.000 375.000 1440 495 1530 225 1620 495 - 1 1 2.00 60.00 60.00 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1530 585 135 135 1530 585 1665 585 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1530 585 90 90 1530 585 1620 585 --6 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 585 585 135 135 585 585 720 585 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 585 585 90 90 585 585 675 585 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 720 585 1395 585 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 585 450 900 135 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 45 630 135 495 225 765 315 585 360 585 450 585 - 0.000 1.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 12 10 0.0000 4 105 180 1440 180 df\001 -4 0 0 50 0 12 10 0.0000 4 105 180 900 540 df\001 -4 0 0 50 0 0 10 0.7854 4 105 210 585 360 0-9\001 -4 0 0 50 0 22 10 0.0000 4 105 165 945 135 Err\001 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 3683 33 33 33 3683 66 3716 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 3683 383 383 1400 3683 1783 4066 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 3683 450 450 1400 3683 1850 4133 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4400 3683 383 383 4400 3683 4783 4066 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4400 3683 450 450 4400 3683 4850 4133 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 3616 933 3683 766 3733 766 3616 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3766 3616 3933 3683 3766 3733 3766 3616 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4116 3183 4083 3350 4016 3183 4116 3183 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + 4950 2655 -90 2655 -90 4230 4950 4230 4950 2655 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 3683 132 3683 225 3683 341 3683 474 3683 617 3683 + 766 3683 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 1866 3683 2147 3683 2462 3683 2797 3683 3137 3683 3465 3683 + 3766 3683 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 4716 3350 4741 3243 4734 3145 4695 3060 4626 2993 4527 2949 + 4400 2933 4305 2941 4224 2964 4158 3002 4108 3051 4077 3112 + 4066 3183 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1400 3766 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 3633 IN\001 +4 1 0 0 0 0 14 0.0000 2 165 120 4400 3766 1\001 +4 1 0 0 0 0 14 0.0000 2 195 1455 2900 3633 -128..'/', ':'..127\001 +4 1 0 0 0 0 14 0.0000 2 150 465 4400 2883 DEF\001 diff --git a/doc/exoption.fig b/doc/exoption.fig index b59f46e..fafc107 100644 --- a/doc/exoption.fig +++ b/doc/exoption.fig @@ -1,37 +1,84 @@ -#FIG 3.2 -Landscape +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait Center Metric A4 100.00 Single -2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: exoption +# Pages: 1 1200 2 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 1395.000 330.000 1305 450 1395 180 1485 450 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 3015.000 330.000 2925 450 3015 180 3105 450 - 1 1 2.00 60.00 60.00 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 585 540 135 135 585 540 720 540 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1395 540 90 90 1395 540 1485 540 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1395 540 135 135 1395 540 1530 540 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2205 540 135 135 2205 540 2340 540 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 3015 540 135 135 3015 540 3150 540 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 3015 540 90 90 3015 540 3105 540 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 720 540 1260 540 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1530 540 2070 540 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 2340 540 2880 540 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 45 585 135 450 225 720 315 540 360 540 450 540 - 0.000 1.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 0 10 0.0000 4 105 210 900 495 0-9\001 -4 0 0 50 0 0 10 0.0000 4 105 210 1305 135 0-9\001 -4 0 0 50 0 0 10 0.0000 4 15 45 1755 495 .\001 -4 0 0 50 0 0 10 0.0000 4 105 210 2520 495 0-9\001 -4 0 0 50 0 0 10 0.0000 4 105 210 2925 135 0-9\001 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 3683 33 33 33 3683 66 3716 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 3683 383 383 1333 3683 1716 4066 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3333 3683 383 383 3333 3683 3716 4066 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3333 3683 450 450 3333 3683 3783 4133 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4966 3683 383 383 4966 3683 5349 4066 +# 3 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 6966 3683 383 383 6966 3683 7349 4066 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 6966 3683 450 450 6966 3683 7416 4133 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 3616 933 3683 766 3733 766 3616 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3050 3183 3016 3350 2950 3183 3050 3183 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4400 3616 4566 3683 4400 3733 4400 3616 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 6683 3183 6650 3350 6583 3183 6683 3183 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2700 3616 2866 3683 2700 3733 2700 3616 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 6333 3616 6500 3683 6333 3733 6333 3616 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + 7515 2655 -45 2655 -45 4230 7515 4230 7515 2655 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 3683 139 3683 237 3683 354 3683 485 3683 624 3683 + 766 3683 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 3650 3350 3675 3243 3667 3145 3629 3060 3559 2993 3460 2949 + 3333 2933 3239 2941 3158 2964 3091 3002 3041 3051 3010 3112 + 3000 3183 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 3800 3683 3894 3683 3992 3683 4093 3683 4196 3683 4298 3683 + 4400 3683 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 3 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 7283 3350 7308 3243 7301 3145 7262 3060 7193 2993 7094 2949 + 6966 2933 6872 2941 6791 2964 6725 3002 6675 3051 6644 3112 + 6633 3183 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 1733 3683 1881 3683 2039 3683 2204 3683 2371 3683 2538 3683 + 2700 3683 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 2 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 5366 3683 5514 3683 5672 3683 5837 3683 6004 3683 6171 3683 + 6333 3683 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1333 3766 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 3633 IN\001 +4 1 0 0 0 0 14 0.0000 2 165 120 3333 3766 1\001 +4 1 0 0 0 0 14 0.0000 2 150 540 3333 2883 '0'..'9'\001 +4 1 0 0 0 0 14 0.0000 2 150 120 4966 3766 2\001 +4 1 0 0 0 0 14 0.0000 2 150 150 4183 3633 '.'\001 +4 1 0 0 0 0 14 0.0000 2 150 120 6966 3766 3\001 +4 1 0 0 0 0 14 0.0000 2 150 540 6966 2883 '0'..'9'\001 +4 1 0 0 0 0 14 0.0000 2 150 540 2300 3633 '0'..'9'\001 +4 1 0 0 0 0 14 0.0000 2 150 540 5933 3633 '0'..'9'\001 diff --git a/doc/exor.fig b/doc/exor.fig index 5d30b16..62e0184 100644 --- a/doc/exor.fig +++ b/doc/exor.fig @@ -1,65 +1,128 @@ -#FIG 3.2 -Landscape +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait Center Metric A4 100.00 Single -2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: exor +# Pages: 1 1200 2 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 720 990 135 135 720 990 855 990 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1800 990 135 135 1800 990 1935 990 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1800 990 90 90 1800 990 1890 990 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1800 360 90 90 1800 360 1890 360 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1800 1620 90 90 1800 1620 1890 1620 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1800 360 135 135 1800 360 1935 360 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1800 1620 135 135 1800 1620 1935 1620 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2700 540 135 135 2700 540 2835 540 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 3825 900 135 135 3825 900 3960 900 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 3825 900 90 90 3825 900 3915 900 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 765 855 1665 360 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 855 990 1665 990 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 765 1125 1665 1620 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1935 360 2565 495 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 2835 585 3690 855 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1800 495 1800 855 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 180 1035 270 900 360 1170 450 990 495 990 585 990 - 0.000 1.000 1.000 1.000 1.000 0.000 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 1935 1665 2745 1665 2880 1665 2880 1575 2745 1575 1935 1575 - 0.000 1.000 1.000 1.000 1.000 0.000 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 1935 1035 2250 1035 2385 1035 2385 945 2250 945 1935 945 - 0.000 1.000 1.000 1.000 1.000 0.000 -3 2 0 1 7 7 50 -1 -1 0.000 0 0 0 2 - 4455 540 4455 1035 - 0.000 0.000 -3 2 0 2 0 7 50 -1 -1 0.000 0 1 0 4 - 1 1 2.00 60.00 60.00 - 3690 945 3555 1305 4095 1305 3960 945 - 0.000 -1.000 -1.000 0.000 -4 0 0 50 0 0 10 0.0000 4 105 195 1530 675 0-9\001 -4 0 0 50 0 0 10 0.0000 4 105 195 1125 945 1-9\001 -4 0 0 50 0 0 10 5.7770 4 120 435 1035 1215 a-z,A-Z\001 -4 0 0 50 0 0 10 0.5061 4 105 75 1080 630 0\001 -4 0 0 50 0 0 10 0.0000 4 105 195 2070 900 0-9\001 -4 0 0 50 0 0 10 0.0000 4 120 660 2070 1530 0-9,a-z,A-Z\001 -4 0 0 50 0 0 12 6.0214 4 75 90 2160 360 x\001 -4 0 0 50 0 0 10 5.9865 4 120 645 2925 540 0-9,a-f,A-F\001 -4 0 0 50 0 0 10 0.0000 4 120 645 3510 1575 0-9,a-f,A-F\001 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 8700 33 33 33 8700 66 8733 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 8700 383 383 1333 8700 1716 9083 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4066 7683 383 383 4066 7683 4449 8066 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4066 7683 450 450 4066 7683 4516 8133 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 6133 8700 383 383 6133 8700 6516 9083 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 6133 8700 450 450 6133 8700 6583 9150 +# 3 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 6133 7000 383 383 6133 7000 6516 7383 +# 4 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 9533 7000 383 383 9533 7000 9916 7383 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 9533 7000 450 450 9533 7000 9983 7450 +# 5 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4066 10316 383 383 4066 10316 4449 10699 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4066 10316 450 450 4066 10316 4516 10766 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 8633 933 8700 766 8750 766 8633 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5600 8366 5716 8500 5550 8466 5600 8366 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5583 7116 5766 7116 5616 7216 5583 7116 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5850 8200 5816 8366 5750 8200 5850 8200 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 9250 6500 9216 6666 9150 6500 9250 6500 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3783 9816 3750 9983 3683 9816 3783 9816 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3450 7850 3633 7850 3483 7950 3450 7850 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5500 8633 5666 8700 5500 8750 5500 8633 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3533 9966 3650 10100 3483 10066 3533 9966 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 8900 6933 9066 7000 8900 7050 8900 6933 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + 10530 5985 -90 5985 -90 10845 10530 10845 10530 5985 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 8700 139 8700 237 8700 354 8700 485 8700 624 8700 + 766 8700 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 4483 7883 4650 7963 4830 8051 5018 8143 5208 8237 5392 8329 + 5566 8416 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 4500 7533 4672 7479 4855 7419 5043 7356 5233 7291 5420 7227 + 5600 7166 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 2 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 6450 8366 6475 8260 6467 8162 6429 8077 6359 8009 6260 7965 + 6133 7950 6039 7958 5958 7981 5891 8018 5841 8068 5810 8129 + 5800 8200 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 4 -> 4 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 9850 6666 9875 6560 9867 6462 9829 6377 9759 6309 9660 6265 + 9533 6250 9439 6258 9358 6281 9291 6318 9241 6368 9210 6429 + 9200 6500 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 5 -> 5 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 4383 9983 4408 9877 4401 9779 4362 9693 4293 9626 4194 9582 + 4066 9566 3972 9574 3891 9598 3825 9635 3775 9685 3744 9746 + 3733 9816 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 1700 8566 1950 8473 2239 8364 2552 8245 2871 8124 3181 8007 + 3466 7900 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 1733 8700 2226 8700 2850 8700 3547 8700 4260 8700 4930 8700 + 5500 8700 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 5 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 1650 8933 1709 8976 1771 9020 1835 9066 1900 9112 1966 9157 + 2033 9200 2280 9349 2535 9496 2791 9639 3041 9775 3280 9901 + 3500 10016 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 3 -> 4 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 6533 7000 6862 7000 7250 7000 7672 7000 8104 7000 8522 7000 + 8900 7000 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1333 8783 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 8650 IN\001 +4 1 0 0 0 0 14 0.0000 2 165 120 4066 7766 1\001 +4 1 0 0 0 0 14 0.0000 2 150 120 6133 8783 2\001 +4 1 0 0 0 0 14 0.0000 2 150 540 5100 7966 '0'..'9'\001 +4 1 0 0 0 0 14 0.0000 2 150 120 6133 7083 3\001 +4 1 0 0 0 0 14 0.0000 2 150 210 5100 7200 'x'\001 +4 1 0 0 0 0 14 0.0000 2 150 540 6133 7900 '0'..'9'\001 +4 1 0 0 0 0 14 0.0000 2 150 120 9533 7083 4\001 +4 1 0 0 0 0 14 0.0000 2 180 1875 9533 6200 '0'..'9', 'A'..'F', 'a'..'f'\001 +4 1 0 0 0 0 14 0.0000 2 150 120 4066 10400 5\001 +4 1 0 0 0 0 14 0.0000 2 180 1920 4066 9516 '0'..'9', 'A'..'Z', 'a'..'z'\001 +4 1 0 0 0 0 14 0.0000 2 150 210 2666 7916 '0'\001 +4 1 0 0 0 0 14 0.0000 2 165 540 4066 8650 '1'..'9'\001 +4 1 0 0 0 0 14 0.0000 2 180 1260 2666 9100 'A'..'Z', 'a'..'z'\001 +4 1 0 0 0 0 14 0.0000 2 180 1875 7833 6950 '0'..'9', 'A'..'F', 'a'..'f'\001 diff --git a/doc/exoutact1.fig b/doc/exoutact1.fig new file mode 100644 index 0000000..5ebc332 --- /dev/null +++ b/doc/exoutact1.fig @@ -0,0 +1,62 @@ +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait +Center +Metric +A4 +100.00 +Single +-2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: exoutact1 +# Pages: 1 +1200 2 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 3550 33 33 33 3550 66 3583 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 3550 383 383 1333 3550 1716 3933 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5333 3550 383 383 5333 3550 5716 3933 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5333 3550 450 450 5333 3550 5783 4000 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3266 3550 383 383 3266 3550 3649 3933 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 3483 933 3550 766 3600 766 3483 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2700 3483 2866 3550 2700 3600 2700 3483 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4700 3483 4866 3550 4700 3600 4700 3483 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3033 3100 3000 3266 2933 3100 3033 3100 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + -135 2610 5895 2610 5895 4140 -135 4140 -135 2610 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 3550 139 3550 237 3550 354 3550 485 3550 624 3550 + 766 3550 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 1733 3550 1881 3550 2039 3550 2204 3550 2371 3550 2538 3550 + 2700 3550 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 3666 3550 3824 3550 3993 3550 4170 3550 4350 3550 4528 3550 + 4700 3550 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 3533 3266 3554 3167 3549 3074 3518 2991 3461 2925 3377 2882 + 3266 2866 3183 2874 3115 2897 3062 2933 3023 2980 2997 3036 + 2983 3100 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1333 3633 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 3500 IN\001 +4 1 0 0 0 0 14 0.0000 2 150 120 5333 3633 2\001 +4 1 0 0 0 0 14 0.0000 2 165 120 3266 3633 1\001 +4 1 0 0 0 0 14 0.0000 2 150 510 2300 3500 'a'..'z'\001 +4 1 0 0 0 0 14 0.0000 2 165 600 4266 3500 10 / A\001 +4 1 0 0 0 0 14 0.0000 2 150 510 3266 2816 'a'..'z'\001 diff --git a/doc/exoutact2.fig b/doc/exoutact2.fig new file mode 100644 index 0000000..7e01a18 --- /dev/null +++ b/doc/exoutact2.fig @@ -0,0 +1,78 @@ +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait +Center +Metric +A4 +100.00 +Single +-2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: exoutact2 +# Pages: 1 +1200 2 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 3716 33 33 33 3716 66 3749 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 3716 383 383 1333 3716 1716 4099 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 8500 3716 383 383 8500 3716 8883 4099 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 8500 3716 450 450 8500 3716 8950 4166 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4733 3716 383 383 4733 3716 5116 4099 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 3650 933 3716 766 3766 766 3650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4166 3650 4333 3716 4166 3750 4166 3650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 7866 3650 8033 3716 7866 3766 7866 3650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1800 4033 1666 3916 1850 3933 1800 4033 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4500 3266 4466 3433 4400 3266 4500 3266 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + -135 2745 9090 2745 9090 4320 -135 4320 -135 2745 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 3716 139 3716 237 3716 354 3716 485 3716 624 3716 + 766 3716 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 1733 3716 1783 3715 1833 3712 1883 3708 1933 3704 1983 3701 + 2033 3700 2432 3700 2755 3700 3039 3700 3322 3700 3640 3700 + 4033 3700 4057 3700 4079 3700 4100 3700 4120 3700 4142 3700 + 4166 3700 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 5133 3716 5508 3716 5960 3716 6456 3716 6961 3716 7443 3716 + 7866 3716 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 4400 3916 4341 3948 4282 3977 4222 4002 4161 4022 4098 4038 + 4033 4050 3645 4126 3325 4172 3039 4187 2751 4172 2428 4126 + 2033 4050 1999 4041 1966 4032 1931 4022 1895 4011 1857 3998 + 1816 3983 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 5000 3433 5020 3334 5016 3240 4985 3158 4928 3092 4844 3049 + 4733 3033 4649 3041 4582 3064 4529 3100 4490 3146 4464 3203 + 4450 3266 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1333 3800 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 3666 IN\001 +4 1 0 0 0 0 14 0.0000 2 150 120 8500 3800 2\001 +4 1 0 0 0 0 14 0.0000 2 165 120 4733 3800 1\001 +4 1 0 0 0 0 14 0.0000 2 165 1230 3033 3650 'a'..'z' / lower\001 +4 1 0 0 0 0 14 0.0000 2 195 2325 6583 3666 10 / term_word, newline\001 +4 1 0 0 0 0 14 0.0000 2 210 2010 3033 3983 ' ' / term_word, space\001 +4 1 0 0 0 0 14 0.0000 2 165 1230 4733 2983 'a'..'z' / lower\001 diff --git a/doc/explus.fig b/doc/explus.fig index cb42300..81636f0 100644 --- a/doc/explus.fig +++ b/doc/explus.fig @@ -1,23 +1,51 @@ -#FIG 3.2 -Landscape +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait Center Metric A4 100.00 Single -2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: explus +# Pages: 1 1200 2 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 1845.000 375.000 1755 495 1845 225 1935 495 - 1 1 2.00 60.00 60.00 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 585 585 135 135 585 585 720 585 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1845 585 135 135 1845 585 1980 585 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1845 585 90 90 1845 585 1935 585 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 720 585 1710 585 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 45 630 135 495 225 765 315 585 360 585 450 585 - 0.000 1.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 0 10 0.0000 4 135 765 810 540 0-9,a-z,A-Z\001 -4 0 0 50 0 0 10 0.0000 4 135 765 1485 180 0-9,a-z,A-Z\001 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 3683 33 33 33 3683 66 3716 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 3683 383 383 1333 3683 1716 4066 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4700 3683 383 383 4700 3683 5083 4066 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4700 3683 450 450 4700 3683 5150 4133 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 3616 933 3683 766 3733 766 3616 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4416 3183 4383 3350 4316 3183 4416 3183 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4066 3616 4233 3683 4066 3733 4066 3616 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + 5850 2655 -90 2655 -90 4230 5850 4230 5850 2655 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 3683 139 3683 237 3683 354 3683 485 3683 624 3683 + 766 3683 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 5016 3350 5041 3243 5034 3145 4995 3060 4926 2993 4827 2949 + 4700 2933 4605 2941 4524 2964 4458 3002 4408 3051 4377 3112 + 4366 3183 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 1733 3683 2055 3683 2438 3683 2856 3683 3283 3683 3695 3683 + 4066 3683 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1333 3766 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 3633 IN\001 +4 1 0 0 0 0 14 0.0000 2 165 120 4700 3766 1\001 +4 1 0 0 0 0 14 0.0000 2 180 1920 4700 2883 '0'..'9', 'A'..'Z', 'a'..'z'\001 +4 1 0 0 0 0 14 0.0000 2 180 1920 2983 3633 '0'..'9', 'A'..'Z', 'a'..'z'\001 diff --git a/doc/exstact.fig b/doc/exstact.fig index 699324e..849f8ed 100644 --- a/doc/exstact.fig +++ b/doc/exstact.fig @@ -1,33 +1,64 @@ -#FIG 3.2 Produced by xfig version 3.2.5-alpha5 -Landscape +#FIG 3.2 +# Generated by Graphviz version 2.12 (Tue Sep 4 16:56:48 UTC 2007) +# For: (thurston) Adrian Thurston,,, +# Title: exstact +# Pages: 1 +Portrait Center -Metric -A4 +Inches +Letter 100.00 Single -2 1200 2 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 1530.000 310.500 1440 405 1530 180 1620 405 - 1 1 2.00 60.00 60.00 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 630 495 135 135 630 495 765 495 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1530 495 135 135 1530 495 1665 495 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2430 495 90 90 2430 495 2520 495 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2430 495 135 135 2430 495 2565 495 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 765 495 1395 495 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1665 495 2295 495 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 90 540 180 405 270 675 360 495 405 495 495 495 - 0.000 1.000 1.000 1.000 1.000 0.000 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 720 585 855 765 1215 900 1845 900 2205 765 2340 585 - 0.000 1.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 0 10 0.0000 4 105 360 900 450 a-z/A\001 -4 0 0 50 0 12 10 0.0000 4 105 180 1890 450 sp\001 -4 0 0 50 0 12 10 0.0000 4 105 180 1440 810 sp\001 -4 0 0 50 0 0 10 0.0000 4 75 195 1427 127 a-z\001 +0 32 #d3d3d3 +2 3 0 1 7 7 2 0 20 0.0 0 0 0 0 0 5 + 0 2340 0 0 6920 0 6920 2340 0 2340 +# ENTRY +1 1 0 1 0 0 1 0 20 0.000 0 0.0000 120 600 40 -40 120 600 160 560 +# 1 +1 1 0 1 0 7 1 0 -1 0.000 0 0.0000 1640 600 420 -430 1640 600 2060 170 +4 1 0 1 0 0 14.0 0.0000 4 0.0 0.0 1640 680 1\001 +# ENTRY->1 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 160 600 247 600 364 600 505 600 662 600 829 600 1000 600 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 1000 530 1200 600 1000 670 1000 530 +4 1 0 0 0 0 14.0 0.0000 4 0.0 0.0 680 480 IN\001 +# 3 +1 1 0 1 0 7 1 0 -1 0.000 0 0.0000 6320 600 422 -430 6320 600 6742 170 +1 1 0 1 0 7 1 0 -1 0.000 0 0.0000 6320 600 500 -510 6320 600 6820 90 +4 1 0 1 0 0 14.0 0.0000 4 0.0 0.0 6320 680 3\001 +# 1->3 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 19 + 2080 560 2140 551 2200 545 2260 540 2320 535 2380 529 2440 520 2902 495 3277 479 3608 470 3936 468 4306 472 4760 480 4901 490 5044 500 5188 510 5329 520 5467 530 5600 540 + 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 5608 470 5800 560 5594 610 5608 470 +4 1 0 0 0 0 14.0 0.0000 4 0.0 0.0 4320 360 SP / A\001 +# 2 +1 1 0 1 0 7 1 0 -1 0.000 0 0.0000 4320 1840 420 -430 4320 1840 4740 1410 +4 1 0 1 0 0 14.0 0.0000 4 0.0 0.0 4320 1920 2\001 +# 1->2 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 2020 780 2273 899 2560 1032 2868 1173 3180 1315 3483 1453 3760 1580 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 3786 1515 3940 1660 3729 1643 3786 1515 +4 1 0 0 0 0 14.0 0.0000 4 0.0 0.0 2980 880 'a'..'z' / A\001 +# 2->3 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 4680 1620 4829 1525 4993 1423 5168 1315 5347 1204 5526 1091 5700 980 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 5671 916 5880 880 5739 1038 5671 916 +4 1 0 0 0 0 14.0 0.0000 4 0.0 0.0 5280 1080 SP\001 +# 2->2 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 13 + 4020 1540 3970 1421 3960 1309 3990 1210 4060 1131 4170 1079 4320 1060 4430 1070 4520 1097 4590 1140 4640 1196 4670 1264 4680 1340 + 0 1 1 1 1 1 1 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 4745 1369 4620 1540 4610 1328 4745 1369 +4 1 0 0 0 0 14.0 0.0000 4 0.0 0.0 4320 940 'a'..'z'\001 +# end of FIG file diff --git a/doc/exstar.fig b/doc/exstar.fig index cca7963..3d82215 100644 --- a/doc/exstar.fig +++ b/doc/exstar.fig @@ -1,32 +1,77 @@ -#FIG 3.2 -Landscape +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait Center Metric A4 100.00 Single -2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: exstar +# Pages: 1 1200 2 -5 1 0 2 0 7 50 0 -1 0.000 0 1 0 1 1035.000 -742.500 675 675 1035 720 1395 675 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 1035.000 1912.500 675 495 1035 450 1395 495 - 1 1 2.00 60.00 60.00 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 585 585 90 90 585 585 675 585 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 585 585 135 135 585 585 720 585 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1485 585 135 135 1485 585 1620 585 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 540 450 540 315 540 180 630 180 630 315 630 450 - 0.000 1.000 1.000 1.000 1.000 0.000 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 45 630 135 495 225 765 315 585 360 585 450 585 - 0.000 1.000 1.000 1.000 1.000 0.000 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 1440 450 1440 315 1440 180 1530 180 1530 315 1530 450 - 0.000 1.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 0 10 0.0000 4 75 195 945 405 a-z\001 -4 0 0 50 0 12 10 0.0000 4 105 180 945 675 nl\001 -4 0 0 50 0 12 10 0.0000 4 105 180 495 135 nl\001 -4 0 0 50 0 0 10 0.0000 4 75 195 1395 135 a-z\001 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 3850 33 33 33 3850 66 3883 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 3850 383 383 1400 3850 1783 4233 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 3850 450 450 1400 3850 1850 4300 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3400 3850 383 383 3400 3850 3783 4233 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 3783 933 3850 766 3900 766 3783 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1116 3350 1083 3516 1016 3350 1116 3350 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2833 3783 3000 3850 2833 3883 2833 3783 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1950 4183 1816 4066 2000 4083 1950 4183 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3166 3400 3133 3566 3066 3400 3166 3400 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + 3915 2835 -90 2835 -90 4410 3915 4410 3915 2835 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 3850 132 3850 225 3850 341 3850 474 3850 617 3850 + 766 3850 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 1716 3516 1741 3410 1734 3312 1695 3227 1626 3159 1527 3115 + 1400 3100 1305 3108 1224 3131 1158 3168 1108 3218 1077 3279 + 1066 3350 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 1866 3850 1916 3842 1966 3838 2016 3835 2066 3833 2116 3833 + 2166 3833 2276 3833 2364 3833 2439 3833 2513 3833 2596 3833 + 2700 3833 2723 3833 2745 3833 2766 3833 2787 3833 2809 3833 + 2833 3833 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 1 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 3066 4050 3008 4081 2949 4110 2889 4135 2828 4156 2765 4172 + 2700 4183 2601 4203 2517 4212 2439 4214 2360 4209 2272 4198 + 2166 4183 2133 4175 2100 4166 2066 4158 2033 4150 2000 4141 + 1966 4133 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 3666 3566 3687 3467 3682 3374 3652 3291 3595 3225 3511 3182 + 3400 3166 3316 3174 3248 3197 3195 3233 3156 3280 3130 3336 + 3116 3400 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1400 3933 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 3800 IN\001 +4 1 0 0 0 0 14 0.0000 2 165 240 1400 3050 10\001 +4 1 0 0 0 0 14 0.0000 2 165 120 3400 3933 1\001 +4 1 0 0 0 0 14 0.0000 2 150 510 2433 3783 'a'..'z'\001 +4 1 0 0 0 0 14 0.0000 2 165 240 2433 4116 10\001 +4 1 0 0 0 0 14 0.0000 2 150 510 3400 3116 'a'..'z'\001 diff --git a/doc/exstrongsubtr.fig b/doc/exstrongsubtr.fig index 1aca526..2f25699 100644 --- a/doc/exstrongsubtr.fig +++ b/doc/exstrongsubtr.fig @@ -7,59 +7,114 @@ A4 Single -2 # Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) -# For: (age) Adrian Thurston,,, -# Title: foo +# For: (age) Adrian Thurston +# Title: exstrongsubtr # Pages: 1 1200 2 0 32 #d2d2d2 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 1470.000 376.000 1380 496 1470 226 1560 496 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 2306.000 376.000 2216 496 2306 226 2396 496 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 3130.000 364.000 3040 484 3130 214 3220 484 - 1 1 2.00 60.00 60.00 -5 1 0 2 0 7 50 0 -1 0.000 0 0 1 0 2721.519 538.911 3088 714 2714 945 2356 716 - 1 1 2.00 60.00 60.00 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 3716 33 33 33 3716 66 3749 # 0 -1 1 0 2 0 7 50 0 -1 0.000 0 0.0000 603 591 135 135 603 591 738 591 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 3716 383 383 1333 3716 1716 4099 +# 4 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 8433 3716 383 383 8433 3716 8816 4099 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 8433 3716 450 450 8433 3716 8883 4166 # 1 -1 1 0 2 0 7 50 0 -1 0.000 0 0.0000 1474 596 135 135 1474 596 1609 596 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3266 3716 383 383 3266 3716 3649 4099 # 2 -1 1 0 2 0 7 50 0 -1 0.000 0 0.0000 2311 590 135 135 2311 590 2446 590 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4833 3716 383 383 4833 3716 5216 4099 # 3 -1 1 0 2 0 7 50 0 -1 0.000 0 0.0000 3135 591 135 135 3135 591 3270 591 -1 1 0 2 0 7 50 0 -1 0.000 0 0.0000 3938 589 135 135 3938 589 4073 589 -# 4 -1 1 0 2 0 7 50 0 -1 0.000 0 0.0000 3938 589 90 90 3938 589 4028 589 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 67 640 157 505 247 775 337 595 382 595 472 595 - 0.000 1.000 1.000 1.000 1.000 0.000 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 6700 3716 383 383 6700 3716 7083 4099 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 3650 933 3716 766 3766 766 3650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2700 3650 2866 3716 2700 3766 2700 3650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3033 3266 3000 3433 2933 3266 3033 3266 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4266 3650 4433 3716 4266 3766 4266 3650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4600 3266 4566 3433 4500 3266 4600 3266 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 6133 3650 6300 3716 6133 3750 6133 3650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 7800 3650 7966 3716 7800 3766 7800 3650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5300 4033 5166 3916 5350 3933 5300 4033 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 6466 3266 6433 3433 6366 3266 6466 3266 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + 9135 2745 -180 2745 -180 4365 9135 4365 9135 2745 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 3716 139 3716 237 3716 354 3716 485 3716 624 3716 + 766 3716 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 # 0 -> 1 -3 4 0 2 0 7 50 0 -1 0.000 0 1 0 2 - 1 1 2.00 60.00 60.00 - 747 589 1341 592 - 0.000 0.000 -# 1 -> 2 -3 4 0 2 0 7 50 0 -1 0.000 0 1 0 2 - 1 1 2.00 60.00 60.00 - 1619 597 2179 594 - 0.000 0.000 -# 1 -> 2 -3 4 0 2 0 7 50 0 -1 0.000 0 1 0 2 - 1 1 2.00 60.00 60.00 - 2457 590 3002 590 - 0.000 0.000 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 1733 3716 1881 3716 2039 3716 2204 3716 2371 3716 2538 3716 + 2700 3716 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 3533 3433 3554 3334 3549 3240 3518 3158 3461 3092 3377 3049 + 3266 3033 3183 3041 3115 3064 3062 3100 3023 3146 2997 3203 + 2983 3266 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 # 1 -> 2 -3 4 0 2 0 7 50 0 -1 0.000 0 1 0 2 - 1 1 2.00 60.00 60.00 - 3284 589 3810 588 - 0.000 0.000 -4 0 0 50 0 0 10 0.0000 4 75 240 885 536 a..z\001 -4 0 0 50 0 12 10 0.0000 4 105 210 3451 538 nl\001 -4 0 0 50 0 12 10 0.0000 4 105 210 2209 190 df\001 -4 0 0 50 0 0 10 0.0000 4 75 45 1832 542 :\001 -4 0 0 50 0 12 10 0.0000 4 105 210 2624 893 df\001 -4 0 0 50 0 0 10 0.0000 4 75 240 1348 184 a..z\001 -4 0 0 50 0 12 10 0.0000 4 75 210 2610 540 cr\001 -4 0 0 50 0 12 10 0.0000 4 75 210 3015 180 cr\001 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 3666 3716 3760 3716 3859 3716 3960 3716 4062 3716 4165 3716 + 4266 3716 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 2 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 5100 3433 5120 3334 5116 3240 5085 3158 5028 3092 4944 3049 + 4833 3033 4749 3041 4682 3064 4629 3100 4590 3146 4564 3203 + 4550 3266 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 2 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 5233 3716 5283 3715 5333 3712 5383 3708 5433 3704 5483 3701 + 5533 3700 5629 3700 5706 3700 5772 3700 5838 3700 5911 3700 + 6000 3700 6023 3700 6045 3700 6066 3700 6087 3700 6109 3700 + 6133 3700 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 3 -> 4 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 7100 3716 7205 3716 7318 3716 7437 3716 7559 3716 7681 3716 + 7800 3716 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 3 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 6366 3916 6308 3948 6249 3977 6189 4002 6128 4022 6065 4038 + 6000 4050 5911 4063 5838 4072 5772 4075 5706 4072 5629 4063 + 5533 4050 5499 4041 5466 4032 5431 4022 5395 4011 5357 3998 + 5316 3983 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 3 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 6966 3433 6987 3334 6982 3240 6952 3158 6895 3092 6811 3049 + 6700 3033 6616 3041 6548 3064 6495 3100 6456 3146 6430 3203 + 6416 3266 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1333 3800 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 3666 IN\001 +4 1 0 0 0 0 14 0.0000 2 150 120 8433 3800 4\001 +4 1 0 0 0 0 14 0.0000 2 165 120 3266 3800 1\001 +4 1 0 0 0 0 14 0.0000 2 150 510 2300 3666 'a'..'z'\001 +4 1 0 0 0 0 14 0.0000 2 150 510 3266 2983 'a'..'z'\001 +4 1 0 0 0 0 14 0.0000 2 150 120 4833 3800 2\001 +4 1 0 0 0 0 14 0.0000 2 150 150 4050 3666 ':'\001 +4 1 0 0 0 0 14 0.0000 2 150 465 4833 2983 DEF\001 +4 1 0 0 0 0 14 0.0000 2 150 120 6700 3800 3\001 +4 1 0 0 0 0 14 0.0000 2 165 240 5766 3650 13\001 +4 1 0 0 0 0 14 0.0000 2 165 240 7533 3666 10\001 +4 1 0 0 0 0 14 0.0000 2 150 465 5766 3983 DEF\001 +4 1 0 0 0 0 14 0.0000 2 165 240 6700 2983 13\001 diff --git a/doc/exsubtr.fig b/doc/exsubtr.fig index 0e35990..4a2ead2 100644 --- a/doc/exsubtr.fig +++ b/doc/exsubtr.fig @@ -1,87 +1,141 @@ #FIG 3.2 -Landscape +Portrait Center Metric -A4 +A4 100.00 Single -2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: exsubtr +# Pages: 1 1200 2 -6 1395 270 3555 630 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1575 450 135 135 1575 450 1710 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2475 450 135 135 2475 450 2610 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 3375 450 135 135 3375 450 3510 450 --6 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 675 1215 135 135 675 1215 810 1215 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2475 1215 90 90 2475 1215 2565 1215 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2475 1215 135 135 2475 1215 2610 1215 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 3375 1980 135 135 3375 1980 3510 1980 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2475 1980 135 135 2475 1980 2610 1980 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1575 1980 135 135 1575 1980 1710 1980 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2475 450 90 90 2475 450 2565 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1575 450 90 90 1575 450 1665 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1575 1980 90 90 1575 1980 1665 1980 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2475 1980 90 90 2475 1980 2565 1980 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 720 1080 1440 450 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 720 1350 1440 1980 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 810 1215 2340 1215 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1665 540 2385 1125 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1665 1890 2385 1305 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 2475 1845 2475 1350 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1710 1980 2340 1980 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 2610 1980 3240 1980 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1710 450 2340 450 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 2610 450 3240 450 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 2475 585 2475 1080 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 3285 540 2565 1125 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 3285 1890 2565 1305 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 135 1260 225 1125 315 1395 405 1215 450 1215 540 1215 - 0.000 1.000 1.000 1.000 1.000 0.000 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 2610 1260 3015 1260 3150 1260 3150 1170 3015 1170 2610 1170 - 0.000 1.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 0 10 0.0000 4 105 45 990 720 i\001 -4 0 0 50 0 0 10 0.0000 4 105 60 1125 1620 f\001 -4 0 0 50 0 0 10 0.0000 4 135 660 1215 1170 a-e,g-h,j-z\001 -4 0 0 50 0 0 10 0.6807 4 75 195 2880 810 a-z\001 -4 0 0 50 0 0 10 5.6025 4 75 195 2925 1530 a-z\001 -4 0 0 50 0 0 10 0.0000 4 75 210 2520 720 u-z\001 -4 0 0 50 0 0 10 0.0000 4 105 195 2205 1755 a-q\001 -4 0 0 50 0 0 10 0.0000 4 75 195 2520 1755 s-z\001 -4 0 0 50 0 0 10 0.0000 4 75 180 2205 720 a-s\001 -4 0 0 50 0 0 10 0.0000 4 75 75 1980 1935 o\001 -4 0 0 50 0 0 10 0.0000 4 75 60 2835 1935 r\001 -4 0 0 50 0 0 10 0.0000 4 90 60 2835 405 t\001 -4 0 0 50 0 0 10 0.0000 4 75 75 1935 405 n\001 -4 0 0 50 0 0 10 5.6025 4 105 495 1845 630 a-m,o-z\001 -4 0 0 50 0 0 10 0.6807 4 105 450 1800 1710 a-n,p-z\001 -4 0 0 50 0 0 10 0.0000 4 75 195 2835 1125 a-z\001 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 9100 33 33 33 9100 66 9133 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 9100 383 383 1333 9100 1716 9483 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 1333 9183 0\001 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 66 9100 139 9100 237 9100 354 9100 485 9100 624 9100 766 9100 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 766 9033 933 9100 766 9150 766 9033 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 500 9050 IN\001 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 10600 8400 383 383 10600 8400 10983 8783 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 10600 8400 450 450 10600 8400 11050 8850 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 10600 8483 1\001 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 13 + 10916 8066 10941 7960 10934 7862 10895 7777 10826 7709 10727 7665 10600 7650 10505 7658 10424 7681 10358 7718 10308 7768 10277 7829 10266 7900 + 0 1 1 1 1 1 1 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 10316 7900 10283 8066 10216 7900 10316 7900 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 10600 7600 'a'..'z'\001 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 2966 7633 383 383 2966 7633 3349 8016 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 2966 7633 450 450 2966 7633 3416 8083 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 2966 7716 2\001 +# 2 -> 1 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 13 + 3433 7650 3917 7670 4530 7701 5237 7741 6002 7793 6790 7856 7566 7933 7997 7981 8433 8038 8862 8102 9271 8166 9649 8228 9983 8283 + 0 1 1 1 1 1 1 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 10000 8233 10150 8316 9983 8333 10000 8233 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 6950 7783 'a'..'n', 'p'..'z'\001 +# 3 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5150 6716 383 383 5150 6716 5533 7099 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5150 6716 450 450 5150 6716 5600 7166 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 5150 6800 3\001 +# 2 -> 3 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 3400 7450 3576 7372 3769 7291 3972 7208 4180 7125 4385 7044 4583 6966 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 4550 6916 4733 6900 4600 7016 4550 6916 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 3850 7200 'o'\001 +# 3 -> 1 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 13 + 5600 6616 6025 6539 6548 6468 7137 6422 7762 6420 8393 6478 9000 6616 9362 6750 9657 6893 9900 7058 10103 7256 10282 7499 10450 7800 + 0 1 1 1 1 1 1 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 10500 7766 10516 7950 10400 7816 10500 7766 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 8433 6383 'a'..'q', 's'..'z'\001 +# 4 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 8433 7150 383 383 8433 7150 8816 7533 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 8433 7233 4\001 +# 3 -> 4 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 5600 6783 5938 6827 6321 6875 6729 6927 7139 6979 7531 7032 7883 7083 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 7883 7033 8050 7100 7883 7133 7883 7033 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 6950 6850 'r'\001 +# 5 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 2966 9100 383 383 2966 9100 3349 9483 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 2966 9100 450 450 2966 9100 3416 9550 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 2966 9183 5\001 +# 5 -> 1 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 19 + 3416 9166 3546 9177 3685 9191 3829 9208 3975 9224 4122 9239 4266 9250 5378 9281 6280 9248 7075 9158 7864 9017 8749 8834 9833 8616 9866 8615 9899 8612 9931 8608 9961 8604 9990 8601 10016 8600 + 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 10016 8550 10183 8583 10016 8650 10016 8550 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 6950 9033 'a'..'m', 'o'..'z'\001 +# 6 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5150 8650 383 383 5150 8650 5533 9033 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5150 8650 450 450 5150 8650 5600 9100 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 5150 8733 6\001 +# 5 -> 6 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 3416 9000 3590 8965 3776 8928 3968 8889 4162 8849 4352 8808 4533 8766 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 4533 8716 4700 8733 4550 8816 4533 8716 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 3850 8883 'n'\001 +# 6 -> 1 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 13 + 5616 8650 5881 8656 6185 8660 6516 8660 6864 8656 7218 8647 7566 8633 7992 8612 8425 8586 8854 8556 9262 8524 9638 8494 9966 8466 + 0 1 1 1 1 1 1 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 9966 8416 10133 8450 9966 8516 9966 8416 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 8433 8516 'a'..'s', 'u'..'z'\001 +# 6 -> 4 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 13 + 5616 8616 5947 8587 6318 8548 6697 8497 7053 8435 7353 8358 7566 8266 7694 8183 7811 8087 7918 7981 8016 7867 8104 7750 8183 7633 + 0 1 1 1 1 1 1 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 8133 7616 8266 7500 8233 7666 8133 7616 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 6950 8216 't'\001 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 19 + 1566 9416 1727 9603 1922 9801 2147 9989 2400 10148 2674 10258 2966 10300 2991 10300 3169 10300 3650 10300 4586 10300 6130 10300 8433 10300 8906 10255 9276 10127 9572 9925 9823 9655 10056 9327 10300 8950 + 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 10266 8900 10400 8800 10350 8966 10266 8900 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 5150 10250 'a'..'e', 'g'..'h', 'j'..'z'\001 +# 0 -> 2 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 1633 8833 1765 8717 1909 8589 2060 8454 2212 8316 2360 8179 2500 8050 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 2466 8000 2633 7933 2550 8083 2466 8000 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 2116 8316 'f'\001 +# 0 -> 5 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 7 + 1733 9100 1827 9100 1925 9100 2027 9100 2129 9100 2232 9100 2333 9100 + 0 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 2333 9033 2500 9100 2333 9150 2333 9033 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 2116 9050 'i'\001 +# 4 -> 1 +3 4 0 1 0 0 0 0 -1 0.0 0 0 0 13 + 8800 7283 8957 7345 9127 7416 9304 7493 9483 7578 9661 7669 9833 7766 9883 7801 9933 7837 9983 7875 10033 7912 10083 7948 10133 7983 + 0 1 1 1 1 1 1 1 1 1 1 1 0 +2 3 0 1 0 0 0 0 20 0.0 0 0 0 0 0 4 + 10166 7933 10266 8083 10100 8033 10166 7933 +4 1 0 0 0 0 14.0 0.0000 2 0.0 0.0 9566 7450 'a'..'z'\001 +# end of FIG file diff --git a/doc/extract.awk b/doc/extract.awk new file mode 100644 index 0000000..2874456 --- /dev/null +++ b/doc/extract.awk @@ -0,0 +1,41 @@ +#!/usr/bin/awk +# + +BEGIN { + in_generate = 0; + in_verbatim = 0; + return_val = 1; +} + +/^% GENERATE: *[a-z0-9A-Z_\.\-]+ *$/ && $3 == exname { + in_generate = 1; + return_val = 0; + next; +} + +/^% END GENERATE$/ { + in_generate = 0; + next; +} + +in_generate && /\\begin\{verbatim\}/ { + in_generate = 0; + in_verbatim = 1; + next; +} + +in_verbatim && /\\end\{verbatim\}/ { + in_generate = 1; + in_verbatim = 0; + next; +} + +in_generate && /^%/ { + print substr( $0, 2 ); +} + +in_verbatim { + print $0; +} + +END { exit return_val; } diff --git a/doc/finguard.fig b/doc/finguard.fig new file mode 100644 index 0000000..325eeb7 --- /dev/null +++ b/doc/finguard.fig @@ -0,0 +1,119 @@ +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait +Center +Metric +A4 +100.00 +Single +-2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: exdonepri +# Pages: 1 +1200 2 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 4983 33 33 33 4983 66 5016 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1333 4983 383 383 1333 4983 1716 5366 +# 3 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 6566 4983 383 383 6566 4983 6949 5366 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 6566 4983 450 450 6566 4983 7016 5433 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3200 4466 383 383 3200 4466 3583 4849 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4833 4983 383 383 4833 4983 5216 5366 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 4916 933 4983 766 5033 766 4916 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1100 4533 1066 4700 1000 4533 1100 4533 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2633 4483 2800 4500 2650 4583 2633 4483 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1866 5150 1716 5066 1883 5050 1866 5150 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2966 4016 2933 4183 2866 4016 2966 4016 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4350 4683 4483 4800 4300 4783 4350 4683 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5933 4916 6100 4983 5933 5033 5933 4916 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1816 5250 1650 5216 1816 5150 1816 5250 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3550 4900 3450 4766 3616 4816 3550 4900 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + -90 3510 7155 3510 7155 5535 -90 5535 -90 3510 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 4983 139 4983 237 4983 354 4983 485 4983 624 4983 + 766 4983 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 1600 4700 1620 4600 1616 4507 1585 4425 1528 4359 1444 4315 + 1333 4300 1249 4308 1182 4330 1129 4366 1090 4413 1064 4469 + 1050 4533 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 1683 4800 1741 4775 1799 4750 1858 4725 1916 4700 1974 4675 + 2033 4650 2127 4626 2225 4604 2327 4585 2429 4567 2532 4550 + 2633 4533 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 2950 4766 2892 4821 2833 4869 2768 4912 2694 4953 2606 4992 + 2500 5033 2398 5068 2295 5091 2191 5104 2087 5108 1984 5106 + 1883 5100 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 3466 4183 3487 4084 3482 3990 3452 3908 3395 3842 3311 3799 + 3200 3783 3116 3791 3048 3814 2995 3850 2956 3896 2930 3953 + 2916 4016 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 3583 4533 3669 4544 3759 4559 3852 4579 3946 4601 4040 4625 + 4133 4650 4166 4665 4200 4679 4233 4691 4266 4704 4300 4717 + 4333 4733 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 2 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 5233 4983 5338 4983 5451 4983 5570 4983 5692 4983 5814 4983 + 5933 4983 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 2 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 4483 5166 4354 5206 4228 5229 4097 5241 3954 5248 3791 5254 + 3600 5266 3295 5296 3045 5304 2822 5295 2598 5272 2344 5239 + 2033 5200 1992 5194 1954 5192 1918 5193 1883 5196 1850 5198 + 1816 5200 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 2 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 4450 5033 4362 5038 4270 5038 4175 5033 4079 5022 3987 5005 + 3900 4983 3836 4965 3777 4945 3725 4925 3677 4904 3636 4884 + 3600 4866 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1333 5066 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 4933 IN\001 +4 1 0 0 0 0 14 0.0000 2 150 120 6566 5066 3\001 +4 1 0 0 0 0 14 0.0000 2 150 465 1333 4250 DEF\001 +4 1 0 0 0 0 14 0.0000 2 165 120 3200 4550 1\001 +4 1 0 0 0 0 14 0.0000 2 150 225 2266 4533 'F'\001 +4 1 0 0 0 0 14 0.0000 2 150 465 2266 4983 DEF\001 +4 1 0 0 0 0 14 0.0000 2 150 225 3200 3733 'F'\001 +4 1 0 0 0 0 14 0.0000 2 150 120 4833 5066 2\001 +4 1 0 0 0 0 14 0.0000 2 150 165 4016 4550 'I'\001 +4 1 0 0 0 0 14 0.0000 2 150 270 5666 4933 'N'\001 +4 1 0 0 0 0 14 0.0000 2 150 465 3200 5216 DEF\001 +4 1 0 0 0 0 14 0.0000 2 150 225 4016 4933 'F'\001 diff --git a/doc/fixbackbox.awk b/doc/fixbackbox.awk new file mode 100644 index 0000000..434fd20 --- /dev/null +++ b/doc/fixbackbox.awk @@ -0,0 +1,10 @@ +#!/usr/bin/awk +# + +NF == 16 && $16 == 5 { + $7 = 1 + print $0 + next; +} + +{ print $0; } diff --git a/doc/genfigs.sh b/doc/genfigs.sh new file mode 100755 index 0000000..8d52107 --- /dev/null +++ b/doc/genfigs.sh @@ -0,0 +1,18 @@ +#!/bin/bash +# + +input=ragel-guide.tex + +for fig; do + if awk -f extract.awk -vexname=$fig $input > /dev/null; then + echo generating ${fig}.dot + opt=`awk -f extract.awk -vexname=$fig $input | + sed '/^ *OPT:/s/^.*: *//p;d'` + awk -f extract.awk -vexname=$fig $input > ${fig}.rl + ../ragel/ragel -V -p ${fig}.rl > ${fig}.dot + else + echo "$0: internal error: figure $fig not found in $input" >&2 + exit 1 + fi +done + diff --git a/doc/leftguard.fig b/doc/leftguard.fig new file mode 100644 index 0000000..0fdb7f7 --- /dev/null +++ b/doc/leftguard.fig @@ -0,0 +1,94 @@ +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait +Center +Metric +A4 +100.00 +Single +-2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: leftguard +# Pages: 1 +1200 2 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 5183 33 33 33 5183 66 5216 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 5183 383 383 1400 5183 1783 5566 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 5183 450 450 1400 5183 1850 5633 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4033 4550 383 383 4033 4550 4416 4933 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 4033 4550 450 450 4033 4550 4483 5000 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 7500 5183 383 383 7500 5183 7883 5566 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 7500 5183 450 450 7500 5183 7950 5633 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 5116 933 5183 766 5233 766 5116 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3416 4650 3583 4666 3433 4750 3416 4650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 6883 5216 7050 5250 6883 5316 6883 5216 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3750 4050 3716 4216 3650 4050 3750 4050 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 6900 5016 7050 5100 6883 5116 6900 5016 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 7400 4600 7333 4766 7300 4600 7400 4600 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 7033 4766 7116 4933 6950 4850 7033 4766 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + 8190 3510 -45 3510 -45 5715 8190 5715 8190 3510 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 5183 132 5183 225 5183 341 5183 474 5183 617 5183 + 766 5183 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 1850 5083 2082 5028 2341 4965 2614 4897 2891 4829 3162 4761 + 3416 4700 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 1866 5266 2062 5292 2280 5320 2512 5347 2753 5374 2995 5397 + 3233 5416 3891 5447 4564 5444 5227 5416 5851 5372 6412 5319 + 6883 5266 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 4350 4216 4375 4110 4367 4012 4329 3927 4259 3859 4160 3815 + 4033 3800 3939 3808 3858 3831 3791 3868 3741 3918 3710 3979 + 3700 4050 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 4483 4633 4827 4692 5227 4764 5658 4843 6094 4924 6511 5001 + 6883 5066 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 2 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 7666 4766 7662 4678 7649 4598 7627 4531 7595 4479 7552 4445 + 7500 4433 7462 4438 7431 4454 7406 4479 7385 4512 7366 4552 + 7350 4600 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 2 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 7883 4933 8038 4752 8106 4570 8085 4402 7977 4262 7781 4168 + 7500 4133 7243 4160 7055 4236 6937 4350 6888 4491 6909 4650 + 7000 4816 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1400 5266 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 5133 IN\001 +4 1 0 0 0 0 14 0.0000 2 165 120 4033 4633 1\001 +4 1 0 0 0 0 14 0.0000 2 150 720 2550 4800 ' ' / start\001 +4 1 0 0 0 0 14 0.0000 2 150 120 7500 5266 2\001 +4 1 0 0 0 0 14 0.0000 2 225 1575 4033 5366 'a'..'z' / fin, alpha\001 +4 1 0 0 0 0 14 0.0000 2 150 150 4033 3750 ' '\001 +4 1 0 0 0 0 14 0.0000 2 225 1575 5933 4716 'a'..'z' / fin, alpha\001 +4 1 0 0 0 0 14 0.0000 2 150 600 7500 4383 ' ' / ws\001 +4 1 0 0 0 0 14 0.0000 2 225 1200 7500 4083 'a'..'z' / alpha\001 diff --git a/doc/lines1.fig b/doc/lines1.fig new file mode 100644 index 0000000..dfaf219 --- /dev/null +++ b/doc/lines1.fig @@ -0,0 +1,164 @@ +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait +Center +Metric +A4 +100.00 +Single +-2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: lines1 +# Pages: 1 +1200 2 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 6150 33 33 33 6150 66 6183 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 6150 383 383 1400 6150 1783 6533 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 6150 450 450 1400 6150 1850 6600 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3933 6150 383 383 3933 6150 4316 6533 +# 4 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5666 5416 383 383 5666 5416 6049 5799 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5666 5416 450 450 5666 5416 6116 5866 +# 5 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 8633 5716 383 383 8633 5716 9016 6099 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 10433 5716 383 383 10433 5716 10816 6099 +# 3 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 12833 5066 383 383 12833 5066 13216 5449 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 6083 933 6150 766 6200 766 6083 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3366 6083 3533 6150 3366 6200 3366 6083 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 8116 5466 8266 5566 8083 5566 8116 5466 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5050 5616 5233 5600 5100 5716 5050 5616 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3700 5700 3666 5866 3600 5700 3700 5700 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 9900 5916 10083 5900 9950 6016 9900 5916 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 12283 5083 12450 5116 12283 5183 12283 5083 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 6266 5216 6083 5216 6216 5100 6266 5216 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 10966 5900 10816 5816 10983 5800 10966 5900 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 12600 4616 12566 4783 12500 4616 12600 4616 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 6166 5800 6050 5666 6216 5700 6166 5800 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 9866 5650 10033 5716 9866 5766 9866 5650 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 8400 5266 8366 5433 8300 5266 8400 5266 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + 13455 4095 -45 4095 -45 6660 13455 6660 13455 4095 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 6150 132 6150 225 6150 341 6150 474 6150 617 6150 + 766 6150 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 1866 6150 2091 6150 2340 6150 2604 6150 2870 6150 3128 6150 + 3366 6150 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 4 -> 5 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 6116 5400 6370 5393 6658 5391 6968 5397 7291 5413 7617 5441 + 7933 5483 7959 5484 7987 5488 8016 5493 8045 5500 8073 5508 + 8100 5516 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 4 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 4300 6000 4414 5947 4540 5891 4672 5833 4809 5775 4947 5719 + 5083 5666 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 4200 5866 4220 5767 4216 5674 4185 5591 4128 5525 4044 5482 + 3933 5466 3849 5474 3782 5497 3729 5533 3690 5580 3664 5636 + 3650 5700 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 4316 6216 4847 6281 5543 6347 6356 6395 7240 6408 8148 6365 + 9033 6250 9191 6218 9348 6176 9502 6127 9651 6073 9796 6018 + 9933 5966 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 2 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 10783 5533 10841 5508 10900 5483 10958 5458 11016 5433 11075 5408 + 11133 5383 11322 5327 11520 5277 11720 5233 11918 5194 12107 5161 + 12283 5133 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 3 -> 4 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 25 + 12450 4933 12399 4916 12349 4900 12297 4885 12245 4871 12190 4859 + 12133 4850 11611 4766 10966 4689 10252 4627 9522 4588 8831 4581 + 8233 4616 7870 4671 7577 4725 7320 4785 7066 4857 6781 4948 + 6433 5066 6400 5082 6367 5095 6335 5108 6304 5120 6276 5134 + 6250 5150 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 0.000 +# 3 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 12616 5383 12554 5459 12483 5534 12406 5606 12321 5670 12230 5725 + 12133 5766 11943 5828 11746 5866 11545 5883 11348 5883 11158 5871 + 10983 5850 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 3 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 13100 4783 13120 4684 13116 4590 13085 4508 13028 4442 12944 4399 + 12833 4383 12749 4391 12682 4414 12629 4450 12590 4496 12564 4553 + 12550 4616 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 5 -> 4 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 8250 5800 8003 5840 7719 5874 7410 5895 7085 5897 6756 5873 + 6433 5816 6392 5807 6354 5795 6316 5783 6279 5770 6240 5759 + 6200 5750 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 5 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 9033 5716 9163 5716 9301 5716 9443 5716 9587 5716 9729 5716 + 9866 5716 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 5 -> 5 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 8900 5433 8920 5334 8916 5240 8885 5158 8828 5092 8744 5049 + 8633 5033 8549 5041 8482 5064 8429 5100 8390 5146 8364 5203 + 8350 5266 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1400 6233 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 6100 IN\001 +4 1 0 0 0 0 14 0.0000 2 165 120 3933 6233 1\001 +4 1 0 0 0 0 14 0.0000 2 150 1050 2700 6100 'a'..'z' / first\001 +4 1 0 0 0 0 14 0.0000 2 150 120 5666 5500 4\001 +4 1 0 0 0 0 14 0.0000 2 150 120 8633 5800 5\001 +4 1 0 0 0 0 14 0.0000 2 195 1455 7183 5350 'a'..'z' / tail, first\001 +4 1 0 0 0 0 14 0.0000 2 165 240 4766 5733 10\001 +4 1 0 0 0 0 14 0.0000 2 150 1050 3933 5416 'a'..'z' / first\001 +4 1 0 0 0 0 14 0.0000 2 150 120 10433 5800 2\001 +4 1 0 0 0 0 14 0.0000 2 180 390 7183 6350 9, ' '\001 +4 1 0 0 0 0 14 0.0000 2 150 120 12833 5150 3\001 +4 1 0 0 0 0 14 0.0000 2 165 975 11633 5133 'a'..'z' / tail\001 +4 1 0 0 0 0 14 0.0000 2 165 240 9533 4550 10\001 +4 1 0 0 0 0 14 0.0000 2 180 390 11633 5716 9, ' '\001 +4 1 0 0 0 0 14 0.0000 2 165 975 12833 4333 'a'..'z' / tail\001 +4 1 0 0 0 0 14 0.0000 2 165 240 7183 5766 10\001 +4 1 0 0 0 0 14 0.0000 2 180 390 9533 5666 9, ' '\001 +4 1 0 0 0 0 14 0.0000 2 195 1455 8633 4983 'a'..'z' / first, tail\001 diff --git a/doc/lines2.fig b/doc/lines2.fig new file mode 100644 index 0000000..9d9d713 --- /dev/null +++ b/doc/lines2.fig @@ -0,0 +1,121 @@ +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait +Center +Metric +A4 +100.00 +Single +-2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: lines2 +# Pages: 1 +1200 2 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 4883 33 33 33 4883 66 4916 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 4883 383 383 1400 4883 1783 5266 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 4883 450 450 1400 4883 1850 5333 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3933 4416 383 383 3933 4416 4316 4799 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5733 4416 383 383 5733 4416 6116 4799 +# 3 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 8133 4883 383 383 8133 4883 8516 5266 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 4816 933 4883 766 4933 766 4816 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3366 4350 3533 4416 3366 4466 3366 4350 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2000 5066 1850 4983 2016 4966 2000 5066 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3700 3966 3666 4133 3600 3966 3700 3966 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5166 4350 5333 4416 5166 4466 5166 4350 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 7666 4566 7783 4700 7616 4666 7666 4566 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1933 5183 1766 5150 1933 5083 1933 5183 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 6133 4816 6033 4666 6200 4716 6133 4816 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 7900 4433 7866 4600 7800 4433 7900 4433 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + 8775 3465 -45 3465 -45 5400 8775 5400 8775 3465 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 4883 132 4883 225 4883 341 4883 474 4883 617 4883 + 766 4883 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 1816 4683 1875 4658 1933 4633 1991 4610 2050 4588 2108 4567 + 2166 4550 2370 4505 2577 4470 2785 4445 2988 4429 3184 4419 + 3366 4416 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 3650 4683 3590 4733 3527 4781 3460 4827 3389 4868 3313 4904 + 3233 4933 3035 4988 2829 5021 2618 5037 2409 5039 2207 5031 + 2016 5016 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 4200 4133 4220 4034 4216 3940 4185 3858 4128 3792 4044 3749 + 3933 3733 3849 3741 3782 3764 3729 3800 3690 3846 3664 3903 + 3650 3966 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 4333 4416 4463 4416 4601 4416 4743 4416 4887 4416 5029 4416 + 5166 4416 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 2 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 6133 4416 6321 4418 6529 4425 6752 4439 6981 4463 7210 4499 + 7433 4550 7466 4558 7499 4567 7533 4577 7566 4588 7599 4601 + 7633 4616 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 3 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 25 + 7800 5066 7748 5072 7693 5074 7635 5075 7572 5075 7505 5077 + 7433 5083 6610 5156 5940 5185 5345 5183 4748 5164 4069 5143 + 3233 5133 3025 5142 2856 5151 2706 5158 2554 5159 2381 5151 + 2166 5133 2125 5127 2083 5125 2043 5127 2004 5129 1967 5132 + 1933 5133 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 0.000 +# 3 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 7750 4950 7561 4971 7349 4984 7122 4985 6889 4970 6657 4938 + 6433 4883 6384 4872 6338 4856 6293 4837 6250 4815 6208 4791 + 6166 4766 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 3 -> 3 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 8400 4600 8420 4500 8416 4407 8385 4325 8328 4259 8244 4215 + 8133 4200 8049 4208 7982 4230 7929 4266 7890 4313 7864 4369 + 7850 4433 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1400 4966 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 4833 IN\001 +4 1 0 0 0 0 14 0.0000 2 165 120 3933 4500 1\001 +4 1 0 0 0 0 14 0.0000 2 150 1050 2700 4400 'a'..'z' / first\001 +4 1 0 0 0 0 14 0.0000 2 165 240 2700 4883 10\001 +4 1 0 0 0 0 14 0.0000 2 150 1050 3933 3683 'a'..'z' / first\001 +4 1 0 0 0 0 14 0.0000 2 150 120 5733 4500 2\001 +4 1 0 0 0 0 14 0.0000 2 180 390 4833 4366 9, ' '\001 +4 1 0 0 0 0 14 0.0000 2 150 120 8133 4966 3\001 +4 1 0 0 0 0 14 0.0000 2 165 975 6933 4400 'a'..'z' / tail\001 +4 1 0 0 0 0 14 0.0000 2 165 240 4833 5116 10\001 +4 1 0 0 0 0 14 0.0000 2 180 390 6933 4833 9, ' '\001 +4 1 0 0 0 0 14 0.0000 2 165 975 8133 4150 'a'..'z' / tail\001 diff --git a/doc/lmkleene.fig b/doc/lmkleene.fig new file mode 100644 index 0000000..b0207af --- /dev/null +++ b/doc/lmkleene.fig @@ -0,0 +1,116 @@ +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait +Center +Metric +A4 +100.00 +Single +-2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: exfinpri +# Pages: 1 +1200 2 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 6166 33 33 33 6166 66 6199 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 6166 383 383 1400 6166 1783 6549 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 6166 450 450 1400 6166 1850 6616 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3466 5416 383 383 3466 5416 3849 5799 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 3466 5416 450 450 3466 5416 3916 5866 +# 2 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5866 6433 383 383 5866 6433 6249 6816 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5866 6433 450 450 5866 6433 6316 6883 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 6100 933 6166 766 6216 766 6100 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1116 5666 1083 5833 1016 5666 1116 5666 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2833 5366 3000 5400 2833 5466 2833 5366 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5233 6350 5400 6416 5233 6450 5233 6350 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 2033 6166 1866 6133 2033 6066 2033 6166 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 3183 4916 3150 5083 3083 4916 3183 4916 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5300 6133 5433 6250 5250 6233 5300 6133 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1950 6483 1816 6366 2000 6383 1950 6483 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 5583 5933 5550 6100 5483 5933 5583 5933 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + -45 4365 6615 4365 6615 7020 -45 7020 -45 4365 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 6166 132 6166 225 6166 341 6166 474 6166 617 6166 + 766 6166 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 1716 5833 1741 5727 1734 5629 1695 5543 1626 5476 1527 5432 + 1400 5416 1305 5424 1224 5448 1158 5485 1108 5535 1077 5596 + 1066 5666 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 1733 5833 1794 5782 1860 5730 1931 5679 2006 5630 2084 5587 + 2166 5550 2270 5512 2380 5482 2493 5458 2608 5440 2722 5426 + 2833 5416 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 0 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 1866 6200 2033 6216 2216 6232 2408 6247 2606 6261 2804 6273 + 3000 6283 3391 6307 3790 6328 4185 6347 4565 6366 4918 6383 + 5233 6400 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 3116 5700 3049 5742 2982 5786 2914 5829 2845 5869 2773 5904 + 2700 5933 2590 5979 2479 6017 2366 6050 2254 6076 2142 6098 + 2033 6116 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 3783 5083 3808 4977 3801 4879 3762 4793 3693 4726 3594 4682 + 3466 4666 3372 4674 3291 4698 3225 4735 3175 4785 3144 4846 + 3133 4916 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 1 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 3883 5600 4095 5688 4327 5784 4570 5885 4816 5987 5056 6088 + 5283 6183 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 2 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 5416 6533 5099 6593 4727 6650 4314 6695 3878 6722 3434 6720 + 3000 6683 2783 6650 2595 6618 2427 6583 2271 6542 2120 6493 + 1966 6433 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 2 -> 2 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 6183 6100 6208 5993 6201 5895 6162 5810 6093 5743 5994 5699 + 5866 5683 5772 5691 5691 5714 5625 5752 5575 5801 5544 5862 + 5533 5933 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1400 6250 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 6116 IN\001 +4 1 0 0 0 0 14 0.0000 2 150 150 1400 5366 ' '\001 +4 1 0 0 0 0 14 0.0000 2 165 120 3466 5500 1\001 +4 1 0 0 0 0 14 0.0000 2 150 540 2433 5400 '0'..'9'\001 +4 1 0 0 0 0 14 0.0000 2 150 120 5866 6516 2\001 +4 1 0 0 0 0 14 0.0000 2 150 510 3466 6233 'a'..'z'\001 +4 1 0 0 0 0 14 0.0000 2 150 495 2433 5883 ' ' / B\001 +4 1 0 0 0 0 14 0.0000 2 150 540 3466 4616 '0'..'9'\001 +4 1 0 0 0 0 14 0.0000 2 150 855 4666 5733 'a'..'z' / B\001 +4 1 0 0 0 0 14 0.0000 2 150 510 3466 6633 ' ' / A\001 +4 1 0 0 0 0 14 0.0000 2 180 1170 5866 5633 '0'..'9', 'a'..'z'\001 diff --git a/doc/opconcat.fig b/doc/opconcat.fig index 312e301..d46084e 100644 --- a/doc/opconcat.fig +++ b/doc/opconcat.fig @@ -1,4 +1,4 @@ -#FIG 3.2 +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 Landscape Center Metric @@ -7,37 +7,33 @@ A4 Single -2 1200 2 -6 225 180 1530 1080 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 585 630 135 135 585 630 720 630 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1215 450 90 90 1215 450 1305 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1215 450 135 135 1215 450 1350 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1215 810 90 90 1215 810 1305 810 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1215 810 135 135 1215 810 1350 810 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 585 630 135 135 585 630 720 630 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1215 810 90 90 1215 810 1305 810 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1215 810 135 135 1215 810 1350 810 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1215 450 135 135 1215 450 1350 450 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1215 450 90 90 1215 450 1305 450 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 2340 630 135 135 2340 630 2475 630 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 2970 450 90 90 2970 450 3060 450 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 2970 450 135 135 2970 450 3105 450 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 2970 810 135 135 2970 810 3105 810 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 2970 810 90 90 2970 810 3060 810 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 45.00 60.00 + 1350 450 2205 585 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 45.00 60.00 + 1350 810 2205 675 3 1 0 1 0 7 50 0 -1 0.000 0 0 0 8 225 630 495 270 1125 180 1485 270 1530 630 1485 990 1125 1080 495 990 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 --6 -6 1980 180 3285 1080 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2970 450 90 90 2970 450 3060 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2970 450 135 135 2970 450 3105 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2340 630 135 135 2340 630 2475 630 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2970 810 135 135 2970 810 3105 810 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2970 810 90 90 2970 810 3060 810 3 1 0 1 0 7 50 0 -1 0.000 0 0 0 8 1980 630 2250 270 2880 180 3240 270 3285 630 3240 990 2880 1080 2250 990 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 --6 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1350 810 2205 675 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 1350 450 2205 585 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 +3 0 0 1 0 7 50 0 -1 0.000 0 1 0 6 + 1 1 1.00 45.00 60.00 45 675 135 540 225 810 315 630 360 630 450 630 0.000 1.000 1.000 1.000 1.000 0.000 -4 0 0 50 0 32 10 0.0000 4 75 75 1710 450 e\001 -4 0 0 50 0 32 10 0.0000 4 75 75 1710 900 e\001 +4 0 0 50 0 32 10 0.0000 4 90 75 1710 450 e\001 +4 0 0 50 0 32 10 0.0000 4 90 75 1710 900 e\001 diff --git a/doc/opor.fig b/doc/opor.fig index 7dbb8ca..edeefb4 100644 --- a/doc/opor.fig +++ b/doc/opor.fig @@ -1,4 +1,4 @@ -#FIG 3.2 +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 Landscape Center Metric @@ -7,29 +7,23 @@ A4 Single -2 1200 2 -6 0 765 765 1170 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 585 945 135 135 585 945 720 945 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 45 990 135 855 225 1125 315 945 360 945 450 945 - 0.000 1.000 1.000 1.000 1.000 0.000 --6 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1215 1440 135 135 1215 1440 1350 1440 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1845 1260 90 90 1845 1260 1935 1260 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1845 1260 135 135 1845 1260 1980 1260 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1845 270 90 90 1845 270 1935 270 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1845 270 135 135 1845 270 1980 270 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1215 450 135 135 1215 450 1350 450 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1845 630 135 135 1845 630 1980 630 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1845 630 90 90 1845 630 1935 630 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1845 1620 90 90 1845 1620 1935 1620 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1845 1620 135 135 1845 1620 1980 1620 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 - 675 855 1125 540 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 585 945 135 135 585 945 720 945 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1215 450 135 135 1215 450 1350 450 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1845 630 135 135 1845 630 1980 630 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1845 630 90 90 1845 630 1935 630 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1845 270 135 135 1845 270 1980 270 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1845 270 90 90 1845 270 1935 270 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1845 1260 90 90 1845 1260 1935 1260 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1845 1260 135 135 1845 1260 1980 1260 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1215 1440 135 135 1215 1440 1350 1440 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1845 1620 90 90 1845 1620 1935 1620 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1845 1620 135 135 1845 1620 1980 1620 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 45.00 60.00 675 1035 1125 1350 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 45.00 60.00 + 675 855 1125 540 3 1 0 1 0 7 50 0 -1 0.000 0 0 0 8 855 1440 1125 1080 1755 990 2115 1080 2160 1440 2115 1800 1755 1890 1125 1800 @@ -38,5 +32,9 @@ Single 855 450 1125 90 1755 0 2115 90 2160 450 2115 810 1755 900 1125 810 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 -4 0 0 50 0 32 10 0.0000 4 75 75 720 1260 e\001 -4 0 0 50 0 32 10 0.0000 4 75 75 720 720 e\001 +3 0 0 1 0 7 50 0 -1 0.000 0 1 0 6 + 1 1 1.00 45.00 60.00 + 45 990 135 855 225 1125 315 945 360 945 450 945 + 0.000 1.000 1.000 1.000 1.000 0.000 +4 0 0 50 0 32 10 0.0000 4 90 75 720 1260 e\001 +4 0 0 50 0 32 10 0.0000 4 90 75 720 720 e\001 diff --git a/doc/opstar.fig b/doc/opstar.fig index 5bac654..cb2cac6 100644 --- a/doc/opstar.fig +++ b/doc/opstar.fig @@ -7,43 +7,37 @@ A4 Single -2 1200 2 -6 360 495 1125 900 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 945 675 135 135 945 675 1080 675 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 405 720 495 585 585 855 675 675 720 675 810 675 - 0.000 1.000 1.000 1.000 1.000 0.000 --6 -6 2070 135 2430 495 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2250 315 90 90 2250 315 2340 315 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2250 315 135 135 2250 315 2385 315 --6 -6 969 -122 1329 238 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1149 58 90 90 1149 58 1239 58 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1149 58 135 135 1149 58 1284 58 --6 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 1620 495 135 135 1620 495 1755 495 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2250 675 135 135 2250 675 2385 675 -1 3 0 2 0 7 50 0 -1 0.000 1 0.0000 2250 675 90 90 2250 675 2340 675 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 945 675 135 135 945 675 1080 675 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1149 58 90 90 1149 58 1239 58 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1149 58 135 135 1149 58 1284 58 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 1620 495 135 135 1620 495 1755 495 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 2250 315 90 90 2250 315 2340 315 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 2250 315 135 135 2250 315 2385 315 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 2250 675 135 135 2250 675 2385 675 +1 3 0 1 0 7 50 0 -1 0.000 1 0.0000 2250 675 90 90 2250 675 2340 675 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 45.00 60.00 1080 630 1485 540 -2 1 0 2 0 7 50 0 -1 0.000 0 0 -1 1 0 2 - 1 1 2.00 60.00 60.00 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 45.00 60.00 973 543 1103 203 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 2385 360 2700 630 2700 1215 1980 1395 1260 1125 978 801 - 0.000 1.000 1.000 1.000 1.000 0.000 -3 0 0 2 0 7 50 0 -1 0.000 0 1 0 6 - 1 1 2.00 60.00 60.00 - 2385 720 2520 855 2475 1125 1935 1215 1395 1035 1067 730 - 0.000 1.000 1.000 1.000 1.000 0.000 3 1 0 1 0 7 50 0 -1 0.000 0 0 0 8 1260 495 1530 135 2160 45 2520 135 2565 495 2520 855 2160 945 1530 855 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 -4 0 0 50 0 32 10 0.0000 4 75 75 1845 1125 e\001 -4 0 0 50 0 32 10 0.0000 4 75 75 1845 1440 e\001 -4 0 0 50 0 32 10 0.0000 4 75 75 1156 549 e\001 -4 0 0 50 0 32 10 0.0000 4 75 75 896 442 e\001 +3 0 0 1 0 7 50 0 -1 0.000 0 1 0 6 + 1 1 1.00 45.00 60.00 + 2385 360 2700 630 2700 1215 1980 1395 1260 1125 978 801 + 0.000 1.000 1.000 1.000 1.000 0.000 +3 0 0 1 0 7 50 0 -1 0.000 0 1 0 6 + 1 1 1.00 45.00 60.00 + 405 720 495 585 585 855 675 675 720 675 810 675 + 0.000 1.000 1.000 1.000 1.000 0.000 +3 0 0 1 0 7 50 0 -1 0.000 0 1 0 6 + 1 1 1.00 45.00 60.00 + 2385 720 2520 855 2475 1125 1935 1215 1395 1035 1067 730 + 0.000 1.000 1.000 1.000 1.000 0.000 +4 0 0 50 0 32 10 0.0000 4 90 75 1845 1125 e\001 +4 0 0 50 0 32 10 0.0000 4 90 75 1845 1440 e\001 +4 0 0 50 0 32 10 0.0000 4 90 75 1156 549 e\001 +4 0 0 50 0 32 10 0.0000 4 90 75 896 442 e\001 diff --git a/doc/ragel-guide.tex b/doc/ragel-guide.tex index db5f88f..4cca738 100644 --- a/doc/ragel-guide.tex +++ b/doc/ragel-guide.tex @@ -1,5 +1,5 @@ % -% Copyright 2001-2006 Adrian Thurston <thurston@cs.queensu.ca> +% Copyright 2001-2009 Adrian Thurston <thurston@complang.org> % % This file is part of Ragel. @@ -18,17 +18,23 @@ % along with Ragel; if not, write to the Free Software % Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA -\documentclass[letterpaper,12pt,oneside]{book} -\usepackage{pslatex} -\usepackage{graphics} +% TODO: Need a section on the different strategies for handline recursion. + +\documentclass[letterpaper,11pt,oneside]{book} +\usepackage{graphicx} \usepackage{comment} \usepackage{multicol} -\usepackage[medium]{titlesec} - -\topmargin 0in +\usepackage[ + colorlinks=true, + linkcolor=black, + citecolor=green, + filecolor=black, + urlcolor=black]{hyperref} + +\topmargin -0.20in \oddsidemargin 0in \textwidth 6.5in -\textheight 8.5in +\textheight 9in \setlength{\parskip}{0pt} \setlength{\topsep}{0pt} @@ -48,6 +54,8 @@ \setcounter{topnumber}{50} \setcounter{bottomnumber}{50} +\newenvironment{inline_code}{\def\baselinestretch{1}\vspace{12pt}\small}{} + \begin{document} % @@ -73,7 +81,7 @@ by\\ % \chapter*{License} Ragel version \version, \pubdate\\ -Copyright \copyright\ 2003, 2004, 2005, 2006 Adrian Thurston +Copyright \copyright\ 2003-2007 Adrian Thurston \vspace{6mm} {\bf\it\noindent This document is part of Ragel, and as such, this document is @@ -112,28 +120,29 @@ License along with Ragel; if not, write to the Free Software Foundation, Inc., \section{Abstract} Regular expressions are used heavily in practice for the purpose of specifying -parsers. However, they are normally used as black boxes linked together with -program logic. User actions are associated with entire expressions and matched -text is extracted from input. With these facilities it is not possible to -specify an entire parser with a single regular expression because practical -parsing tasks invariably involve the execution of arbitrary user code -throughout the course of parsing. - -Ragel is a software development tool which allows the user to embed actions into -regular expressions without disrupting the regular expression syntax. -Consequently, one can specify an entire parser using a single regular -experssion. The single-expression model affords concise -and elegant descriptions of languages and the generation of very simple, -fast and robust code. Ragel compiles finite state machines from a high level -regular language notation to executable C, C++, Objective-C or D. +parsers. They are normally used as black boxes linked together with program +logic. User actions are executed in between invocations of the regular +expression engine. Adding actions before a pattern terminates requires patterns +to be broken and pasted back together with program logic. The more user actions +are needed, the less the advantages of regular expressions are seen. + +Ragel is a software development tool that allows user actions to be +embedded into the transitions of a regular expression's corresponding state +machine, eliminating the need to switch from the regular expression engine and +user code execution environment and back again. As a result, expressions can be +maximally continuous. One is free to specify an entire parser using a single +regular expression. The single-expression model affords concise and elegant +descriptions of languages and the generation of very simple, fast and robust +code. Ragel compiles executable finite state machines from a high level regular language +notation. Ragel targets C, C++, Objective-C, D, Java and Ruby. In addition to building state machines from regular expressions, Ragel allows the programmer to directly specify state machines with state charts. These two -notations may also be freely combined. There are facilities for controlling -nondeterminism in the resulting machines and building scanners using the -longest-match paradigm. Ragel can produce code that runs as fast as manually -constructed machines. Ragel can handle integer-sized alphabets and can compile -very large state machines. +notations may be freely combined. There are also facilities for controlling +nondeterminism in the resulting machines and building scanners using patterns +that themselves have embedded actions. Ragel can produce code that is small and +runs very fast. Ragel can handle integer-sized alphabets and can compile very +large state machines. \section{Motivation} @@ -142,47 +151,44 @@ context-free language there are many tools to choose from. It is quite common to generate useful and efficient parsers for programming languages from a formal grammar. It is also quite common for programmers to avoid such tools when making parsers for simple computer languages, such as file formats and -communication protocols. Such languages often meet the criteria for the -regular languages. Tools for processing the context-free languages are simply -too heavyweight for the purpose of parsing regular languages because the extra -run-time effort required for supporting the recursive nature of context-free -languages is wasted. - -Regular expressions are more appropriate than context-free grammars for a large -number of parsing probelems. Parsers based on them have many advantages over -hand written parsers. Regular expression syntax is convenient, -concise and easy to maintain. Existing -parsing tools based on regular expressions, such as Lex, Re2C, Sed, Awk and -Perl, are normally split into two levels: a regular expression matching engine -and some kind of program logic for linking patterns together and executing user -code. - -As an example, Lex requires the user to consider a language as a sequence -of independent patterns. -Unfortunately, there are many computer languages that are considered regular, -which do not fit this model. This model also places restrictions on when action -code may be executed. Since action code can only be associated with complete -patterns, if action code must be executed before an entire pattern is matched -then the pattern must be broken into smaller units. Instead of being forced to -disrupt the regular expression syntax, it is desirable to retain a single -expression and embed code for performing actions directly into the transitions -which move over the characters. After all we know the transitions are there. - -Perl allows one to link patterns together using arbitrary program code. This -is very flexible and powerful, however we can be more concise, clear and robust -if we avoid gluing together regular expressions with if statements and while -loops, and instead only compose parsers with regular expression operators. To -achieve this we require an action execution model for associating code with the -sub-expressions of a regular expression in a way that does not disrupt its -syntax. - -The primary goal of Ragel is therefore to provide developers with an ability to embed -actions into the transitions and states of a regular expression in support the -definition of entire parsers or large sections of parsers using a single -regular expression that is compiled to a simple state machine. From the -regular expression we gain a clear and concise statement of our language. From -the state machine we obtain a very fast and robust executable that lends itself -to many kinds of analysis and visualization. +communication protocols. Such languages are often regular and tools for +processing the context-free languages are viewed as too heavyweight for the +purpose of parsing regular languages. The extra run-time effort required for +supporting the recursive nature of context-free languages is wasted. + +When we turn to the regular expression-based parsing tools, such as Lex, Re2C, +and scripting languages such as Sed, Awk and Perl we find that they are split +into two levels: a regular expression matching engine and some kind of program +logic for linking patterns together. For example, a Lex program is composed of +sets of regular expressions. The implied program logic repeatedly attempts to +match a pattern in the current set. When a match is found the associated user +code executed. It requires the user to consider a language as a sequence of +independent tokens. Scripting languages and regular expression libraries allow +one to link patterns together using arbitrary program code. This is very +flexible and powerful, however we can be more concise and clear if we avoid +gluing together regular expressions with if statements and while loops. + +This model of execution, where the runtime alternates between regular +expression matching and user code exectution places restrictions on when +action code may be executed. Since action code can only be associated with +complete patterns, any action code that must be executed before an entire +pattern is matched requires that the pattern be broken into smaller units. +Instead of being forced to disrupt the regular expression syntax and write +smaller expressions, it is desirable to retain a single expression and embed +code for performing actions directly into the transitions that move over the +characters. After all, capable programmers are astutely aware of the machinery +underlying their programs, so why not provide them with access to that +machinery? To achieve this we require an action execution model for associating +code with the sub-expressions of a regular expression in a way that does not +disrupt its syntax. + +The primary goal of Ragel is to provide developers with an ability to embed +actions into the transitions and states of a regular expression's state machine +in support of the definition of entire parsers or large sections of parsers +using a single regular expression. From the regular expression we gain a clear +and concise statement of our language. From the state machine we obtain a very +fast and robust executable that lends itself to many kinds of analysis and +visualization. \section{Overview} @@ -193,7 +199,7 @@ deterministic finite state automaton. Since every regular language has a state machine representation and vice versa, the terms regular language and state machine (or just machine) will be used interchangeably in this document. -Ragel outputs machines to C, C++, Objective-C, or D code. The output is +Ragel outputs machines to C, C++, Objective-C, D, Java or Ruby code. The output is designed to be generic and is not bound to any particular input or processing method. A Ragel machine expects to have data passed to it in buffer blocks. When there is no more input, the machine can be queried for acceptance. In @@ -201,96 +207,100 @@ this way, a Ragel machine can be used to simply recognize a regular language like a regular expression library. By embedding code into the regular language, a Ragel machine can also be used to parse input. -The Ragel input language has many operators for constructing and manipulating +The Ragel language has many operators for constructing and manipulating machines. Machines are built up from smaller machines, to bigger ones, to the final machine representing the language that needs to be recognized or parsed. -The core state machine construction operators are those found in most ``Theory -of Computation'' textbooks. They date back to the 1950s and are widely studied. +The core state machine construction operators are those found in most theory +of computation textbooks. They date back to the 1950s and are widely studied. They are based on set operations and permit one to think of languages as a set -of strings. They are Union, Intersection, Subtraction, Concatenation and Kleene +of strings. They are Union, Intersection, Difference, Concatenation and Kleene Star. Put together, these operators make up what most people know as regular -expressions. Ragel also provides a longest-match construction for easily -building scanners and provides operators for explicitly constructing machines -using a state chart method. In the state chart method one joins machines +expressions. Ragel also provides a scanner construction operator +and provides operators for explicitly constructing machines +using a state chart method. In the state chart method, one joins machines together without any implied transitions and then explicitly specifies where epsilon transitions should be drawn. The state machine manipulation operators are specific to Ragel. They allow the -programmer to access the states and transitions of regular languages. There are -two uses of the manipulation operators. The first and primary use is to embed -code into transitions and states, allowing the programmer to specify the -actions of the state machine. - -Following a number of action embeddings, a single transition can have a number -of actions embedded in it. When making a nondeterministic specification into a -DFA using machines that have embedded actions, new transitions are often made -that have the combined actions of several source transitions. Ragel ensures -that multiple actions associated with a single transition are ordered -consistently with respect to the order of reference and the natural ordering -implied by the construction operators. - -The second use of the manipulation operators is to assign priorities in +programmer to access the states and transitions of regular language's +corresponding machine. There are two uses of the manipulation operators. The +first and primary use is to embed code into transitions and states, allowing +the programmer to specify the actions of the state machine. + +Ragel attempts to make the action embedding facility as intuitive as possible. +To do so, a number of issues need to be addressed. For example, when making a +nondeterministic specification into a DFA using machines that have embedded +actions, new transitions are often made that have the combined actions of +several source transitions. Ragel ensures that multiple actions associated with +a single transition are ordered consistently with respect to the order of +reference and the natural ordering implied by the construction operators. + +The second use of the manipulation operators is to assign priorities to transitions. Priorities provide a convenient way of controlling any nondeterminism introduced by the construction operators. Suppose two transitions leave from the same state and go to distinct target states on the same character. If these transitions are assigned conflicting priorities, then during the determinization process the transition with the higher priority will take precedence over the transition with the lower priority. The lower priority -transition gets abandoned. The transitions would otherwise be combined to a new -transition that goes to a new state which is a combination of the original +transition gets abandoned. The transitions would otherwise be combined into a new +transition that goes to a new state that is a combination of the original target states. Priorities are often required for segmenting machines. The most common uses of priorities have been encoded into a set of simple operators -which should be used instead of priority embeddings whenever possible. +that should be used instead of priority embeddings whenever possible. -There are four operators for embedding actions and priorities into the -transitions of a state machine, these correspond to the different -classes of transitions in a machine. It is possible to embed into start -transitions, finishing transitions, all transitions or pending out -transitions. The embedding of pending out transitions is a special case. -These transition embeddings gets stored in the final states of a machine. They -are transferred to any transitions that may be made going out of the machine by -a concatenation or kleene star operator. +For the purposes of embedding, Ragel divides transitions and states into +different classes. There are four operators for embedding actions and +priorities into the transitions of a state machine. It is possible to embed +into entering transitions, finishing transitions, all transitions and leaving +transitions. The embedding into leaving transitions is a special case. +These transition embeddings get stored in the final states of a machine. They +are transferred to any transitions that are made going out of the machine by +future concatenation or kleene star operations. There are several more operators for embedding actions into states. Like the transition embeddings, there are various different classes of states that the embedding operators access. For example, one can access start states, final -states or all states, among others. Unlike the transition -embeddings, there -are several different types of state action embeddings. These are executed at various -different times during the processing of input. It is possible to embed -actions which are exectued on all transitions into a state, all transitions out of a state, -transitions taken on the error event or on the EOF event. +states or all states, among others. Unlike the transition embeddings, there are +several different types of state action embeddings. These are executed at +various different times during the processing of input. It is possible to embed +actions that are exectued on transitions into a state, on transitions out of a +state, on transitions taken on the error event, or on transitions taken on the +EOF event. Within actions, it is possible to influence the behaviour of the state machine. The user can write action code that jumps or calls to another portion of the machine, changes the current character being processed, or breaks out of the processing loop. With the state machine calling feature Ragel can be used to -parse languages which are not regular. For example, one can parse balanced -parentheses by calling into a parser when an open bracket character is seen and -returning to the state on the top of the stack when the corresponding closing -bracket character is seen. More complicated context-free languages such as -expressions in C, are out of the scope of Ragel. - -Ragel provides a longest-match construction operator which eases the task of -building scanners. This construction behaves much like the primary processing -model of Lex. The generated code, which relies on user-defined variables for -backtracking, repeatedly tries to match patterns to the input, favouring longer -patterns over shorter ones and patterns that appear ahead of others when the -lengths of the possible matches are identical. When a pattern is matched the -associated action is executed. Longest-match machines take Ragel out of the -domain of pure state machines and require the user to maintain the backtracking -related variables. However, longest-match machines integrate well with regular -state machine instantiations. They can be called to or jumped to only when -needed, or they can be called out of or jumped out of when a simpler, pure -state machine model is needed. +parse languages that are not regular. For example, one can parse balanced +parentheses by calling into a parser when an open parenthesis character is seen +and returning to the state on the top of the stack when the corresponding +closing parenthesis character is seen. More complicated context-free languages +such as expressions in C are out of the scope of Ragel. + +Ragel also provides a scanner construction operator that can be used to build +scanners much the same way that Lex is used. The Ragel generated code, which +relies on user-defined variables for backtracking, repeatedly tries to match +patterns to the input, favouring longer patterns over shorter ones and patterns +that appear ahead of others when the lengths of the possible matches are +identical. When a pattern is matched the associated action is executed. + +The key distinguishing feature between scanners in Ragel and scanners in Lex is +that Ragel patterns may be arbitrary Ragel expressions and can therefore +contain embedded code. With a Ragel-based scanner the user need not wait until +the end of a pattern before user code can be executed. + +Scanners do take Ragel out of the domain of pure state machines and require the +user to maintain the backtracking related variables. However, scanners +integrate well with regular state machine instantiations. They can be called to +or jumped to only when needed, or they can be called out of or jumped out of +when a simpler, pure state machine model is appropriate. Two types of output code style are available. Ragel can produce a table-driven -machine or a directly executable machine. The directly executable machine is much -faster than the table-driven. On the other hand, the table-driven machine is -more compact and less demanding on the host language compiler. It is better -suited to compiling large state machines and in the future will be used for -coverage statistics gathering and debugging. +machine or a directly executable machine. The directly executable machine is +much faster than the table-driven. On the other hand, the table-driven machine +is more compact and less demanding on the host language compiler. It is better +suited to compiling large state machines. \section{Related Work} @@ -322,8 +332,8 @@ help, but employing a heavyweight processing loop that imposes a stream example of this kind of scenario is the conversion of floating point numbers contained in a string to their corresponding numerical values. -Another drawback is that -Lex patterns are black boxes. It is not possbile to execute a user action while +Another drawback is the very issue that Ragel attempts to solve. +It is not possible to execute a user action while matching a character contained inside a pattern. For example, if scanning a programming language and string literals can contain newlines which must be counted, a Lex user must break up a string literal pattern so as to associate @@ -382,7 +392,7 @@ How ragel is different from Lex. \end{comment} The Re2C program defines an input processing model similar to that of Lex. -Unlike Lex, Re2C focuses on making generated state machines run very fast and +Re2C focuses on making generated state machines run very fast and integrate easily into any program, free of dependencies. Re2C generates directly executable code and is able to claim that generated parsers run nearly as fast as their hand-coded equivalents. This is very important for user @@ -407,16 +417,22 @@ They are instead interpreted and involve backtracking. This is shown by the following Perl program. When it is fed the input \verb|abcd| the interpretor attempts to match the first alternative, printing \verb|a1 b1|. When this possibility fails it backtracks and tries the second possibility, printing -\verb|a2 b2|, at which point it succeeds. A similar parser expressed in Ragel -will attempt both of the alternatives concurrently, printing -\verb|a1 a2 b1 b2|. +\verb|a2 b2|, at which point it succeeds. -\verbspace +\begin{inline_code} \begin{verbatim} print "YES\n" if ( <STDIN> =~ /( a (?{ print "a1 "; }) b (?{ print "b1 "; }) cX ) | ( a (?{ print "a2 "; }) b (?{ print "b2 "; }) cd )/x ) \end{verbatim} +\end{inline_code} +\verbspace + +In Ragel there is no regular expression interpretor. Aside from the scanner +operator, all Ragel expressions are made into deterministic machines and the +run time simply moves from state to state as it consumes input. An equivalent +parser expressed in Ragel would attempt both of the alternatives concurrently, +printing \verb|a1 a2 b1 b2|. \section{Development Status} @@ -428,16 +444,13 @@ compatibility. Though in the past this has not always held true: changes that break code have crept into minor version number changes. Typically, the documentation lags behind the development in the interest of documenting only the lasting features. The latest changes are always documented in the ChangeLog -file. As Ragel stabilizes, which is expected in the 5.x line, the version -numbering rules will become more strict and the documentation will become more -plentiful. - +file. \chapter{Constructing State Machines} \section{Ragel State Machine Specifications} -A Ragel input file consists of a host language code file with embedded machine +A Ragel input file consists of a program in the host language that contains embedded machine specifications. Ragel normally passes input straight to output. When it sees a machine specification it stops to read the Ragel statements and possibly generate code in place of the specification. @@ -454,10 +467,10 @@ interpret preprocessor directives itself so includes, defines and ifdef logic cannot be used to alter the parse of a Ragel input file. It is therefore not possible to use an \verb|#if 0| directive to comment out a machine as is commonly done in C code. As an alternative, a machine can be prevented from -causing any generated output by commenting out the write statements. +causing any generated output by commenting out write statements. -In Figure \ref{cmd-line-parsing}, a multi-line machine is used to define the -machine and single line machines are used to trigger the writing of the machine +In Figure \ref{cmd-line-parsing}, a multi-line specification is used to define the +machine and single line specifications are used to trigger the writing of the machine data and execution code. \begin{figure} @@ -474,7 +487,7 @@ data and execution code. 0 @{ res = 1; }; }%% -%% write data noerror nofinal; +%% write data; \end{verbatim} \columnbreak \begin{verbatim} @@ -496,7 +509,6 @@ int main( int argc, char **argv ) \label{cmd-line-parsing} \end{figure} - \subsection{Naming Ragel Blocks} \begin{verbatim} @@ -511,21 +523,7 @@ previous specification name exists then this is an error. Because FSM specifications persist in memory, a machine's statements can be spread across multiple machine specifications. This allows one to break up a machine across several files or draw in statements that are common to multiple machines using -the include statement. - -\subsection{Including Ragel Code} - -\begin{verbatim} -include FsmName "inputfile.rl"; -\end{verbatim} -\verbspace - -The \verb|include| statement can be used to draw in the statements of another FSM -specification. Both the name and input file are optional, however at least one -must be given. Without an FSM name, the given input file is searched for an FSM -of the same name as the current specification. Without an input file the -current file is searched for a machine of the given name. If both are present, -the given input file is searched for a machine of the given name. +the \verb|include| statement. \subsection{Machine Definition} \label{definition} @@ -535,8 +533,8 @@ the given input file is searched for a machine of the given name. \end{verbatim} \verbspace -The machine definition statement associates an FSM expression with a name. Machine -expressions assigned to names can later be referenced by other expressions. A +The machine definition statement associates an FSM expression with a name. Machine +expressions assigned to names can later be referenced in other expressions. A definition statement on its own does not cause any states to be generated. It is simply a description of a machine to be used later. States are generated only when a definition is instantiated, which happens when a definition is referenced in an instantiated @@ -550,38 +548,78 @@ expression. \end{verbatim} \verbspace -The machine instantiation statement generates a set of states representing an expression and -associates a name with the entry point. Each instantiation generates a distinct -set of states. At a very minimum the \verb|main| machine must be instantiated. -Other machines may be instantiated and control passed to them by use of -\verb|fcall|, \verb|fgoto| or \verb|fnext| statements. +The machine instantiation statement generates a set of states representing an +expression. Each instantiation generates a distinct set of states. The starting +state of the instantiation is written in the data section of the generated code +using the instantiation name. If a machine named +\verb|main| is instantiated, its start state is used as the +specification's start state and is assigned to the \verb|cs| variable by the +\verb|write init| command. If no \verb|main| machine is given, the start state +of the last machine instantiation to appear is used as the specification's +start state. -\begin{comment} -\subsection{Write Statement} +From outside the execution loop, control may be passed to any machine by +assigning the entry point to the \verb|cs| variable. From inside the execution +loop, control may be passed to any machine instantiation using \verb|fcall|, +\verb|fgoto| or \verb|fnext| statements. + +\subsection{Including Ragel Code} \begin{verbatim} -write <component> [options]; +include FsmName "inputfile.rl"; \end{verbatim} \verbspace -The write statement is used to generate parts of the machine. There are four -components that can be generated: the state machine's static data, the -initialization code, the execution code and the EOF action execution code. The -write statement is described in detail in Section \ref{write-statement}. -\end{comment} +The \verb|include| statement can be used to draw in the statements of another FSM +specification. Both the name and input file are optional, however at least one +must be given. Without an FSM name, the given input file is searched for an FSM +of the same name as the current specification. Without an input file the +current file is searched for a machine of the given name. If both are present, +the given input file is searched for a machine of the given name. + +Ragel searches for included files from the location of the current file. +Additional directories can be added to the search path using the \verb|-I| +option. + +\subsection{Importing Definitions} +\label{import} -\section{Lexical Analysis of an FSM Specification} +\begin{verbatim} +import "inputfile.h"; +\end{verbatim} +\verbspace + +The \verb|import| statement scrapes a file for sequences of tokens that match +the following forms. Ragel treats these forms as state machine definitions. + +\begin{itemize} + \setlength{\itemsep}{-2mm} + \item \verb|name '=' number| + \item \verb|name '=' lit_string| + \item \verb|'define' name number| + \item \verb|'define' name lit_string| +\end{itemize} + +If the input file is a Ragel program then tokens inside any Ragel +specifications are ignored. See Section \ref{export} for a description of +exporting machine definitions. + +Ragel searches for imported files from the location of the current file. +Additional directories can be added to the search path using the \verb|-I| +option. + +\section{Lexical Analysis of a Ragel Block} \label{lexing} -Within a machine specification the following lexical rules apply to the parse -of the input. +Within a machine specification the following lexical rules apply to the input. \begin{itemize} \item The \verb|#| symbol begins a comment that terminates at the next newline. \item The symbols \verb|""|, \verb|''|, \verb|//|, \verb|[]| behave as the -delimiters of literal strings. With them, the following escape sequences are interpreted: +delimiters of literal strings. Within them, the following escape sequences +are interpreted: \verb| \0 \a \b \t \n \v \f \r| @@ -594,7 +632,7 @@ expressions in Section \ref{basic}. \item The symbols \verb|{}| delimit a block of host language code that will be embedded into the machine as an action. Within the block of host language -code, basic lexical analysis of C/C++ comments and strings is done in order to +code, basic lexical analysis of comments and strings is done in order to correctly find the closing brace of the block. With the exception of FSM commands embedded in code blocks, the entire block is preserved as is for identical reproduction in the output code. @@ -603,7 +641,7 @@ identical reproduction in the output code. Integers used for specifying machines may be negative only if the alphabet type is signed. Integers used for specifying priorities may be positive or negative. -\item The pattern \verb|0x[0-9a-fA-f]+| denotes an integer in hexadecimal +\item The pattern \verb|0x[0-9A-Fa-f]+| denotes an integer in hexadecimal format. \item The keywords are \verb|access|, \verb|action|, \verb|alphtype|, @@ -638,11 +676,6 @@ The basic machines are the base operands of regular language expressions. They are the smallest unit to which machine construction and manipulation operators can be applied. -In the diagrams that follow the symbol \verb|df| represents -the default transition, which is taken if no other transition can be taken. The -symbol \verb|cr| represents the carriage return character, \verb|nl| represents the newline character (aka line feed) and the symbol -\verb|sp| represents the space character. - \begin{itemize} \item \verb|'hello'| -- Concatenation Literal. Produces a machine that matches @@ -650,8 +683,20 @@ the sequence of characters in the quoted string. If there are 5 characters there will be 6 states chained together with the characters in the string. See Section \ref{lexing} for information on valid escape sequences. -\begin{center} -\includegraphics{bmconcat} +% GENERATE: bmconcat +% OPT: -p +% %%{ +% machine bmconcat; +\begin{comment} +\begin{verbatim} +main := 'hello'; +\end{verbatim} +\end{comment} +% }%% +% END GENERATE + +\begin{center} +\includegraphics[scale=0.55]{bmconcat} \end{center} It is possible @@ -663,39 +708,75 @@ the string, for example \verb|'cmd'i|. \item \verb|[hello]| -- Or Expression. Produces a union of characters. There will be two states with a transition for each unique character between the two states. The \verb|[]| delimiters behave like the quotes of a literal string. For example, -\verb|[ \t]| means tab or space. The or expression supports character ranges +\verb|[ \t]| means tab or space. The \verb|or| expression supports character ranges with the \verb|-| symbol as a separator. The meaning of the union can be negated using an initial \verb|^| character as in standard regular expressions. See Section \ref{lexing} for information on valid escape sequences -in or expressions. +in \verb|or| expressions. + +% GENERATE: bmor +% OPT: -p +% %%{ +% machine bmor; +\begin{comment} +\begin{verbatim} +main := [hello]; +\end{verbatim} +\end{comment} +% }%% +% END GENERATE \begin{center} -\includegraphics{bmor} +\includegraphics[scale=0.55]{bmor} \end{center} \item \verb|''|, \verb|""|, and \verb|[]| -- Zero Length Machine. Produces a machine that matches the zero length string. Zero length machines have one state that is both a start state and a final state. +% GENERATE: bmnull +% OPT: -p +% %%{ +% machine bmnull; +\begin{comment} +\begin{verbatim} +main := ''; +\end{verbatim} +\end{comment} +% }%% +% END GENERATE + \begin{center} -\includegraphics{bmnull} +\includegraphics[scale=0.55]{bmnull} \end{center} -\item \verb|number| -- Simple Machine. Produces a two state machine with one +% FIXME: More on the range of values here. +\item \verb|42| -- Numerical Literal. Produces a two state machine with one transition on the given number. The number may be in decimal or hexadecimal format and should be in the range allowed by the alphabet type. The minimum and maximum values permitted are defined by the host machine that Ragel is compiled on. For example, numbers in a \verb|short| alphabet on an i386 machine should be in the range \verb|-32768| to \verb|32767|. +% GENERATE: bmnum +% %%{ +% machine bmnum; +\begin{comment} +\begin{verbatim} +main := 42; +\end{verbatim} +\end{comment} +% }%% +% END GENERATE + \begin{center} -\includegraphics{bmnum} +\includegraphics[scale=0.55]{bmnum} \end{center} \item \verb|/simple_regex/| -- Regular Expression. Regular expressions are -parsed as a series of expressions that will be concatenated together. Each +parsed as a series of expressions that are concatenated together. Each concatenated expression -may be a literal character, the any character specified by the \verb|.| +may be a literal character, the ``any'' character specified by the \verb|.| symbol, or a union of characters specified by the \verb|[]| delimiters. If the first character of a union is \verb|^| then it matches any character not in the list. Within a union, a range of characters can be given by separating the first @@ -709,22 +790,49 @@ trailing option. Use it to produce case-insensitive machines, as in \verb|/GET/i Ragel does not support very complex regular expressions because the desired results can always be achieved using the more general machine construction operators listed in Section \ref{machconst}. The following diagram shows the -result of compiling \verb|/ab*[c-z].*[123]/|. +result of compiling \verb|/ab*[c-z].*[123]/|. \verb|DEF| represents the default +transition, which is taken if no other transition can be taken. + + +% GENERATE: bmregex +% OPT: -p +% %%{ +% machine bmregex; +\begin{comment} +\begin{verbatim} +main := /ab*[c-z].*[123]/; +\end{verbatim} +\end{comment} +% }%% +% END GENERATE \begin{center} -\includegraphics{bmregex} +\includegraphics[scale=0.55]{bmregex} \end{center} -\item \verb|lit .. lit| -- Range. Produces a machine that matches any +\item \verb|'a' .. 'z'| -- Range. Produces a machine that matches any characters in the specified range. Allowable upper and lower bounds of the -range are concatenation literals of length one and number literals. For +range are concatenation literals of length one and numerical literals. For example, \verb|0x10..0x20|, \verb|0..63|, and \verb|'a'..'z'| are valid ranges. The bounds should be in the range allowed by the alphabet type. +% GENERATE: bmrange +% OPT: -p +% %%{ +% machine bmrange; +\begin{comment} +\begin{verbatim} +main := 'a' .. 'z'; +\end{verbatim} +\end{comment} +% }%% +% END GENERATE + \begin{center} -\includegraphics{bmrange} +\includegraphics[scale=0.55]{bmrange} \end{center} + \item \verb|variable_name| -- Lookup the machine definition assigned to the variable name given and use an instance of it. See Section \ref{definition} for an important note on what it means to reference a variable name. @@ -814,28 +922,31 @@ in the same precedence group are evaluated from left to right. \label{machconst} When using Ragel it is helpful to have a sense of how it constructs machines. -Sometimes this the determinization process can cause results that appear unusual to someone -unfamiliar with it. Ragel does not make use of any nondeterministic -intermediate state machines. All operators accept and return deterministic -machines. However, to ease the discussion, the operations are defined in terms -epsilon transitions. - -To draw an epsilon transition between two states \verb|x| and \verb|y|, is to +The determinization process can produce results that seem unusual to someone +not familiar with the NFA to DFA conversion algorithm. In this section we +describe Ragel's state machine operators. Though the operators are defined +using epsilon transitions, it should be noted that this is for discussion only. +The epsilon transitions described in this section do not persist, but are +immediately removed by the determinization process which is executed at every +operation. Ragel does not make use of any nondeterministic intermediate state +machines. + +To create an epsilon transition between two states \verb|x| and \verb|y| is to copy all of the properties of \verb|y| into \verb|x|. This involves drawing in -all of \verb|y|'s to-state actions, EOF actions, etc., as well as its +all of \verb|y|'s to-state actions, EOF actions, etc., in addition to its transitions. If \verb|x| and \verb|y| both have a transition out on the same character, then the transitions must be combined. During transition -combination a new transition is made which goes to a new state that is the +combination a new transition is made that goes to a new state that is the combination of both target states. The new combination state is created using the same epsilon transition method. The new state has an epsilon transition -drawn to all the states that compose it. Since every time an epsilon transition -is drawn the creation of new epsilon transitions may be triggered, the process -of drawing epsilon transitions is repeated until there are no more epsilon -transitions to be made. +drawn to all the states that compose it. Since the creation of new epsilon +transitions may be triggered every time an epsilon transition is drawn, the +process of drawing epsilon transitions is repeated until there are no more +epsilon transitions to be made. A very common error that is made when using Ragel is to make machines that do -too much at once. That is, to create machines that have unintentional -nondeterminism. This usually results from being unaware of the common strings +too much. That is, to create machines that have unintentional +nondetermistic properties. This usually results from being unaware of the common strings between machines that are combined together using the regular language operators. This can involve never leaving a machine, causing its actions to be propagated through all the following states. Or it can involve an alternation @@ -848,15 +959,10 @@ one portion of the machine to the next. See Chapter \ref{controlling-nondeterminism} for more on this problem and how to solve it. The Graphviz tool is an immense help when debugging improperly compiled -machines or otherwise learning how to use Ragel. In many cases, practical -parsing programs will be too large to completely visualize with Graphviz. The -proper approach is to reduce the language to the smallest subset possible that -still exhibits the characteristics that one wishes to learn about or to fix. -This can be done without modifying the source code using the \verb|-M| and -\verb|-S| options at the frontend. If a machine cannot be easily reduced, -embeddings of unique actions can be very useful for tracing a -particular component of a larger machine specification, since action names are -written out on transition labels. +machines or otherwise learning how to use Ragel. Graphviz Dot files can be +generated from Ragel programs using the \verb|-V| option. See Section +\ref{visualization} for more information. + \subsection{Union} @@ -882,16 +988,22 @@ shown below. The following example demonstrates the union of three machines representing common tokens. -\verbspace +% GENERATE: exor +% OPT: -p +% %%{ +% machine exor; +\begin{inline_code} \begin{verbatim} # Hex digits, decimal digits, or identifiers main := '0x' xdigit+ | digit+ | alpha alnum*; \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE \graphspace \begin{center} -\includegraphics{exor} +\includegraphics[scale=0.55]{exor} \end{center} \subsection{Intersection} @@ -900,7 +1012,7 @@ main := '0x' xdigit+ | digit+ | alpha alnum*; \verbspace Intersection produces a machine that matches any -string which is in both machine one and machine two. To achieve intersection, a +string that is in both machine one and machine two. To achieve intersection, a union is performed on the two machines. After the result has been made deterministic, any final state that is not a combination of final states from both machines has its final state status revoked. To complete the operation, @@ -909,7 +1021,11 @@ if there are any such paths in either of the expressions they will be removed by the intersection operator. Intersection can be used to require that two independent patterns be simultaneously satisfied as in the following example. -\verbspace +% GENERATE: exinter +% OPT: -p +% %%{ +% machine exinter; +\begin{inline_code} \begin{verbatim} # Match lines four characters wide that contain # words separated by whitespace. @@ -917,11 +1033,13 @@ main := /[^\n][^\n][^\n][^\n]\n/* & (/[a-z][a-z]*/ | [ \n])**; \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE \graphspace \begin{center} -\includegraphics{exinter} +\includegraphics[scale=0.55]{exinter} \end{center} \subsection{Difference} @@ -930,7 +1048,7 @@ main := \verbspace The difference operation produces a machine that matches -strings which are in machine one but which are not in machine two. To achieve subtraction, +strings that are in machine one but are not in machine two. To achieve subtraction, a union is performed on the two machines. After the result has been made deterministic, any final state that came from machine two or is a combination of states involving a final state from machine two has its final state status @@ -939,18 +1057,27 @@ that does not lead to a final state. The following example demonstrates the use of subtraction to exclude specific cases from a set. \verbspace + +% GENERATE: exsubtr +% OPT: -p +% %%{ +% machine exsubtr; +\begin{inline_code} \begin{verbatim} # Subtract keywords from identifiers. main := /[a-z][a-z]*/ - ( 'for' | 'int' ); \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE \graphspace \begin{center} -\includegraphics{exsubtr} +\includegraphics[scale=0.55]{exsubtr} \end{center} \graphspace + \subsection{Strong Difference} \label{strong_difference} @@ -958,19 +1085,28 @@ main := /[a-z][a-z]*/ - ( 'for' | 'int' ); \verbspace Strong difference produces a machine that matches any string of the first -machine which does not have any string of the second machine as a substring. In +machine that does not have any string of the second machine as a substring. In the following example, strong subtraction is used to excluded \verb|CRLF| from -a sequence. +a sequence. In the corresponding visualization, the label \verb|DEF| is short +for default. The default transition is taken if no other transition can be +taken. -\verbspace +% GENERATE: exstrongsubtr +% OPT: -p +% %%{ +% machine exstrongsubtr; +\begin{inline_code} \begin{verbatim} crlf = '\r\n'; main := [a-z]+ ':' ( any* -- crlf ) crlf; \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE +\graphspace \begin{center} -\includegraphics{exstrongsubtr} +\includegraphics[scale=0.55]{exstrongsubtr} \end{center} \graphspace @@ -989,10 +1125,10 @@ expr - ( any* expr any* ) Concatenation produces a machine that matches all the strings in machine one followed by all the strings in machine two. Concatenation draws epsilon transitions from the final states of the first machine to the start state of the second machine. The -final states of the first machine loose their final state status, unless the +final states of the first machine lose their final state status, unless the start state of the second machine is final as well. Concatenation is the default operator. Two machines next to each other with no -operator between them results in the machines being concatenated together. +operator between them results in concatenation. \graphspace \begin{center} @@ -1001,30 +1137,37 @@ operator between them results in the machines being concatenated together. \graphspace The opportunity for nondeterministic behaviour results from the possibility of -the final states of the first machine accepting a string which is also accepted +the final states of the first machine accepting a string that is also accepted by the start state of the second machine. -The most common scenario that this happens in is the +The most common scenario in which this happens is the concatenation of a machine that repeats some pattern with a machine that gives -a termination string, but the repetition machine does not exclude the -termination string. The example in Section \ref{strong_difference} +a terminating string, but the repetition machine does not exclude the +terminating string. The example in Section \ref{strong_difference} guards against this. Another example is the expression \verb|("'" any* "'")|. -When exectued the thread of control will +When executed the thread of control will never leave the \verb|any*| machine. This is a problem especially if actions -are embedded to processes the characters of the \verb|any*| component. +are embedded to process the characters of the \verb|any*| component. In the following example, the first machine is always active due to the nondeterministic nature of concatenation. This particular nondeterminism is intended however because we wish to permit EOF strings before the end of the input. -\verbspace +% GENERATE: exconcat +% OPT: -p +% %%{ +% machine exconcat; +\begin{inline_code} \begin{verbatim} # Require an eof marker on the last line. main := /[^\n]*\n/* . 'EOF\n'; \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE +\graphspace \begin{center} -\includegraphics{exconcat} +\includegraphics[scale=0.55]{exconcat} \end{center} \graphspace @@ -1035,13 +1178,12 @@ adjacent machines there is an ambiguity between subtraction of a positive numerical literal and concatenation of a negative numerical literal. For example, \verb|(x-7)| could be interpreted as \verb|(x . -7)| or \verb|(x - 7)|. In the Ragel language, the subtraction operator always takes precedence -over concatenation of a negative literal. Precedence was given to the -subtraction-based interpretation so as to adhere to the rule that the default +over concatenation of a negative literal. We adhere to the rule that the default concatenation operator takes effect only when there are no other operators between two machines. Beware of writing machines such as \verb|(any -1)| when what is -desired is a concatenation of \verb|any| and -1. Instead write -\verb|(any . -1)| or \verb|(any (-1))|. If in doubt of the meaning of your program do not -rely on the default concatenation operator, always use the \verb|.| symbol. +desired is a concatenation of \verb|any| and \verb|-1|. Instead write +\verb|(any . -1)| or \verb|(any (-1))|. If in doubt of the meaning of your program do not +rely on the default concatenation operator; always use the \verb|.| symbol. \subsection{Kleene Star} @@ -1072,8 +1214,8 @@ arising from nondeterministic behavior, this is discussed in more detail in Chap by using the longest-match construction discussed in Section \ref{generating-scanners} on scanners. -In this simple -example, there is no nondeterminism introduced by the exterior kleene star due +In this +example, there is no nondeterminism introduced by the exterior kleene star due to the newline at the end of the regular expression. Without the newline the exterior kleene star would be redundant and there would be ambiguity between repeating the inner range of the regular expression and the entire regular @@ -1081,16 +1223,24 @@ expression. Though it would not cause a problem in this case, unnecessary nondeterminism in the kleene star operator often causes undesired results for new Ragel users and must be guarded against. -\verbspace +% GENERATE: exstar +% OPT: -p +% %%{ +% machine exstar; +\begin{inline_code} \begin{verbatim} # Match any number of lines with only lowercase letters. main := /[a-z]*\n/*; \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE +\graphspace \begin{center} -\includegraphics{exstar} +\includegraphics[scale=0.55]{exstar} \end{center} +\graphspace \subsection{One Or More Repetition} @@ -1099,19 +1249,26 @@ main := /[a-z]*\n/*; This operator produces the concatenation of the machine with the kleene star of itself. The result will match one or more repetitions of the machine. The plus -operator is equivalent to \verb|(expr . expr*)|. The plus operator makes -repetitions that cannot be zero length. +operator is equivalent to \verb|(expr . expr*)|. -\verbspace +% GENERATE: explus +% OPT: -p +% %%{ +% machine explus; +\begin{inline_code} \begin{verbatim} # Match alpha-numeric words. main := alnum+; \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE +\graphspace \begin{center} -\includegraphics{explus} +\includegraphics[scale=0.55]{explus} \end{center} +\graphspace \subsection{Optional} @@ -1121,18 +1278,27 @@ main := alnum+; The {\em optional} operator produces a machine that accepts the machine given or the zero length string. The optional operator is equivalent to \verb/(expr | '' )/. In the following example the optional operator is used to -extend a token. +possibly extend a token. -\verbspace +% GENERATE: exoption +% OPT: -p +% %%{ +% machine exoption; +\begin{inline_code} \begin{verbatim} # Match integers or floats. main := digit+ ('.' digit+)?; \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE +\graphspace \begin{center} -\includegraphics{exoption} +\includegraphics[scale=0.55]{exoption} \end{center} +\graphspace + \subsection{Repetition} @@ -1154,16 +1320,25 @@ main := digit+ ('.' digit+)?; Negation produces a machine that matches any string not matched by the given machine. Negation is equivalent to \verb|(any* - expr)|. -\verbspace +% GENERATE: exnegate +% OPT: -p +% %%{ +% machine exnegate; +\begin{inline_code} \begin{verbatim} # Accept anything but a string beginning with a digit. main := ! ( digit any* ); \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE +\graphspace \begin{center} -\includegraphics{exnegate} +\includegraphics[scale=0.55]{exnegate} \end{center} +\graphspace + \subsection{Character-Level Negation} @@ -1172,432 +1347,8 @@ main := ! ( digit any* ); Character-level negation produces a machine that matches any single character not matched by the given machine. Character-Level Negation is equivalent to -\verb|(any - expr)|. - -\section{State Charts} - -It is not uncommon for programmers to implement -parsers as manually-coded state machines, either using a switch statement or a -state map compiler which takes a list of states, transitions and actions, and -generates code. - -This method can be a very effective programming technique for producing robust -code. The key disadvantage becomes clear when one attempts to comprehend such a -parser. Machines coded in this way usually require many lines, causing logic to -be spread out over large distances in the source file. Remembering the function -of a large number of states can be difficult and organizing the parser in a -sensible way requires discipline because branches and repetition present many -file layout options. This kind of programming takes a specification with -inherent structure such as looping, alternation and concatenation and expresses -it in a flat form. - -If we could take an isolated component of a manually programmed state chart, -that is, a subset of states that has only one entry point, and implement it -using regular language operators then we could eliminate all the explicit -naming of the states contained in it. By eliminating explicitly named states -and replacing them with higher-level specifications we simplify a parser -specification. - -For example, sometimes chains of states are needed, with only a small number of -possible characters appearing along the chain. These can easily be replaced -with a concatenation of characters. Sometimes a group of common states -implement a loop back to another single portion of the machine. Rather than -manually duplicate all the transitions that loop back, we may be able to -express the loop using a kleene star operator. - -Ragel allows one to take this state map simplification approach. We can build -state machines using a state map model and implement portions of the state map -using regular languages. In place of any transition in the state machine, -entire sub-state machines can be given. These can encapsulate functionality -defined elsewhere. An important aspect of the Ragel approach is that when we -wrap up a collection of states using a regular expression we do not loose -access to the states and transitions. We can still execute code on the -transitions that we have encapsulated. - -\subsection{Join} - -\verb|expr , expr , ...| -\verbspace - -Join a list of machines together without -drawing any transitions, without setting up a start state, and without -designating any final states. Transitions between the machines may be specified -using labels and epsilon transitions. The start state must be explicity -specified with the ``start'' label. Final states may be specified with the an -epsilon transition to the implicitly created ``final'' state. The join -operation allows one to build machines using a state chart model. - -\subsection{Label} - -\verb|label: expr| -\verbspace - -Attaches a label to an expression. Labels can be -used as the target of epsilon transitions and explicit control transfer -statements such \verb|fgoto| and \verb|fnext| in action -code. - -\subsection{Epsilon} - -\verb|expr -> label| -\verbspace - -Draws an epsilon transition to the state defined -by \verb|label|. Epsilon transitions are made deterministic when join -operators are evaluated. Epsilon transitions that are not in a join operation -are made deterministic when the machine definition that contains the epsilon is -complete. See Section \ref{labels} for information on referencing labels. - - -\section{Scanners} -\label{generating-scanners} - -The longest-match operator can be used to construct scanners. The generated -machine repeatedly attempts to match one of the given patterns, first favouring -longer pattern matches over shorter ones. If there is a choice between equal -length matches, the match of the pattern which appears first is chosen. - -\verbspace -\begin{verbatim} -<machine_name> := |* - pattern1 => action1; - pattern2 => action2; - ... - *|; -\end{verbatim} -\verbspace - -The longest-match construction operator is not a pure state machine operator. -It relies on the \verb|tokstart|, \verb|tokend| and \verb|act| variables to be -present so that it can backtrack and make pointers to the matched text -available to the user. If input is processed using multiple calls to the -execute code then the user must ensure that when a token is only partially -matched that the prefix is preserved on the subsequent invocation of the -execute code. - -The \verb|tokstart| variable must be defined as a pointer to the input data. -It is used for recording where the current token match begins. This variable -may be used in action code for retrieving the text of the current match. Ragel -ensures that in between tokens and outside of the longest-match machines that -this pointer is set to null. In between calls to the execute code the user must -check if \verb|tokstart| is set and if so, ensure that the data it points to is -preserved ahead of the next buffer block. This is described in more detail -below. - -The \verb|tokend| variable must also be defined as a pointer to the input data. -It is used for recording where a match ends and where scanning of the next -token should begin. This can also be used in action code for retrieving the -text of the current match. - -The \verb|act| variable must be defined as an integer type. It is used for -recording the identity of the last pattern matched when the scanner must go -past a matched pattern in an attempt to make a longer match. If the longer -match fails it may need to consult the act variable. In some cases use of the act -variable can be avoided because the value of the current state is enough -information to determine which token to accept, however in other cases this is -not enough and so the \verb|act| variable is used. - -When the longest-match operator is in use, the user's driver code must take on -some buffer management functions. The following algorithm gives an overview of -the steps that should be taken to properly use the longest-match operator. - -\begin{itemize} -\setlength{\parskip}{0pt} -\item Read a block of input data. -\item Run the execute code. -\item If \verb|tokstart| is set, the execute code will expect the incomplete -token to be preserved ahead of the buffer on the next invocation of the execute -code. -\begin{itemize} -\item Shift the data beginning at \verb|tokstart| and ending at \verb|pe| to the -beginning of the input buffer. -\item Reset \verb|tokstart| to the beginning of the buffer. -\item Shift \verb|tokend| by the distance from the old value of \verb|tokstart| -to the new value. The \verb|tokend| variable may or may not be valid. There is -no way to know if it holds a meaningful value because it is not kept at null -when it is not in use. It can be shifted regardless. -\end{itemize} -\item Read another block of data into the buffer, immediately following any -preserved data. -\item Run the scanner on the new data. -\end{itemize} - -Figure \ref{preserve_example} shows the required handling of an input stream in -which a token is broken by the input block boundaries. After processing up to -and including the ``t'' of ``characters'', the prefix of the string token must be -retained and processing should resume at the ``e'' on the next iteration of -the execute code. - -If one uses a large input buffer for collecting input then the number of times -the shifting must be done will be small. Furthermore, if one takes care not to -define tokens that are allowed to be very long and instead processes these -items using pure state machines or sub-scanners, then only a small amount of -data will ever need to be shifted. - -\begin{figure} -\begin{verbatim} - a) A stream "of characters" to be scanned. - | | | - p tokstart pe - - b) "of characters" to be scanned. - | | | - tokstart p pe -\end{verbatim} -\caption{Following an invocation of the execute code there may be a partially -matched token (a). The data of the partially matched token -must be preserved ahead of the new data on the next invocation (b).} -\label{preserve_example} -\end{figure} - -Since scanners attempt to make the longest possible match of input, in some -cases they are not able to identify a token upon parsing its final character, -they must wait for a lookahead character. For example if trying to match words, -the token match must be triggered on following whitespace in case more -characters of the word have yet to come. The user must therefore arrange for an -EOF character to be sent to the scanner to flush out any token that has not yet -been matched. The user can exclude a single character from the entire scanner -and use this character as the EOF character, possibly specifying an EOF action. -For most scanners, zero is a suitable choice for the EOF character. - -Alternatively, if whitespace is not significant and ignored by the scanner, the -final real token can be flushed out by simply sending an additional whitespace -character on the end of the stream. If the real stream ends with whitespace -then it will simply be extended and ignored. If it does not, then the last real token is -guaranteed to be flushed and the dummy EOF whitespace ignored. -An example scanner processing loop is given in Figure \ref{scanner-loop}. - -\begin{figure} -\small -\begin{verbatim} - int have = 0; - bool done = false; - while ( !done ) { - /* How much space is in the buffer? */ - int space = BUFSIZE - have; - if ( space == 0 ) { - /* Buffer is full. */ - cerr << "TOKEN TOO BIG" << endl; - exit(1); - } - - /* Read in a block after any data we already have. */ - char *p = inbuf + have; - cin.read( p, space ); - int len = cin.gcount(); - - /* If no data was read, send the EOF character. - if ( len == 0 ) { - p[0] = 0, len++; - done = true; - } - - char *pe = p + len; - %% write exec; - - if ( cs == RagelScan_error ) { - /* Machine failed before finding a token. */ - cerr << "PARSE ERROR" << endl; - exit(1); - } - - if ( tokstart == 0 ) - have = 0; - else { - /* There is a prefix to preserve, shift it over. */ - have = pe - tokstart; - memmove( inbuf, tokstart, have ); - tokend = inbuf + (tokend-tokstart); - tokstart = inbuf; - } - } -\end{verbatim} -\caption{A processing loop for a scanner.} -\label{scanner-loop} -\end{figure} - - -\section{Write Statement} -\label{write-statement} - -\begin{verbatim} -write <component> [options]; -\end{verbatim} -\verbspace - - -The write statement is used to generate parts of the machine. -There are four -components that can be generated by a write statement. These components are the -state machine's data, initialization code, execution code and EOF action -execution code. A write statement may appear before a machine is fully defined. -This allows one to write out the data first then later define the machine where -it is used. An example of this is show in Figure \ref{fbreak-example}. - -\subsection{Write Data} -\begin{verbatim} -write data [options]; -\end{verbatim} -\verbspace - -The write data statement causes Ragel to emit the constant static data needed -by the machine. In table-driven output styles (see Section \ref{genout}) this -is a collection of arrays that represent the states and transitions of the -machine. In goto-driven machines much less data is emitted. At the very -minimum a start state \verb|name_start| is generated. All variables written -out in machine data have both the \verb|static| and \verb|const| properties and -are prefixed with the name of the machine and an -underscore. The data can be placed inside a class, inside a function, or it can -be defined as global data. - -Two variables are written that may be used to test the state of the machine -after a buffer block has been processed. The \verb|name_error| variable gives -the id of the state that the machine moves into when it cannot find a valid -transition to take. The machine immediately breaks out of the processing loop when -it finds itself in the error state. The error variable can be compared to the -current state to determine if the machine has failed to parse the input. If the -machine is complete, that is from every state there is a transition to a proper -state on every possible character of the alphabet, then no error state is required -and this variable will be set to -1. - -The \verb|name_first_final| variable stores the id of the first final state. All of the -machine's states are sorted by their final state status before having their ids -assigned. Checking if the machine has accepted its input can then be done by -checking if the current state is greater-than or equal to the first final -state. - -Data generation has several options: - -\begin{itemize} -\item \verb|noerror| - Do not generate the integer variable that gives the -id of the error state. -\item \verb|nofinal| - Do not generate the integer variable that gives the -id of the first final state. -\item \verb|noprefix| - Do not prefix the variable names with the name of the -machine. -\end{itemize} - -\subsection{Write Init} -\begin{verbatim} -write init; -\end{verbatim} -\verbspace - -The write init statement causes Ragel to emit initialization code. This should -be executed once before the machine is started. At a very minimum this sets the -current state to the start state. If other variables are needed by the -generated code, such as call -stack variables or longest-match management variables, they are also -initialized here. - -\subsection{Write Exec} -\begin{verbatim} -write exec [options]; -\end{verbatim} -\verbspace - -The write exec statement causes Ragel to emit the state machine's execution code. -Ragel expects several variables to be available to this code. At a very minimum, the -generated code needs access to the current character position \verb|p|, the ending -position \verb|pe| and the current state \verb|cs|, though \verb|pe| -can be excluded by specifying the \verb|noend| write option. -The \verb|p| variable is the cursor that the execute code will -used to traverse the input. The \verb|pe| variable should be set up to point to one -position past the last valid character in the buffer. - -Other variables are needed when certain features are used. For example using -the \verb|fcall| or \verb|fret| statements requires \verb|stack| and -\verb|top| variables to be defined. If a longest-match construction is used, -variables for managing backtracking are required. - -The write exec statement has one option. The \verb|noend| option tells Ragel -to generate code that ignores the end position \verb|pe|. In this -case the user must explicitly break out of the processing loop using -\verb|fbreak|, otherwise the machine will continue to process characters until -it moves into the error state. This option is useful if one wishes to process a -null terminated string. Rather than traverse the string to discover then length -before processing the input, the user can break out when the null character is -seen. The example in Figure \ref{fbreak-example} shows the use of the -\verb|noend| write option and the \verb|fbreak| statement for processing a string. - -\begin{figure} -\small -\begin{verbatim} -#include <stdio.h> -%% machine foo; -int main( int argc, char **argv ) -{ - %% write data noerror nofinal; - int cs, res = 0; - if ( argc > 1 ) { - char *p = argv[1]; - %%{ - main := - [a-z]+ - 0 @{ res = 1; fbreak; }; - write init; - write exec noend; - }%% - } - printf("execute = %i\n", res ); - return 0; -} -\end{verbatim} -\caption{Use of {\tt noend} write option and the {\tt fbreak} statement for -processing a string.} -\label{fbreak-example} -\end{figure} - - -\subsection{Write EOF Actions} -\begin{verbatim} -write eof; -\end{verbatim} -\verbspace - -The write EOF statement causes Ragel to emit code that executes EOF actions. -This write statement is only relevant if EOF actions have been embedded, -otherwise it does not generate anything. The EOF action code requires access to -the current state. - -\section{Referencing Names} -\label{labels} - -This section describes how to reference names in epsilon transitions and -action-based control-flow statements such as \verb|fgoto|. There is a hierarchy -of names implied in a Ragel specification. At the top level are the machine -instantiations. Beneath the instantiations are labels and references to machine -definitions. Beneath those are more labels and references to definitions, and -so on. - -Any name reference may contain multiple components separated with the \verb|::| -compound symbol. The search for the first component of a name reference is -rooted at the join expression that the epsilon transition or action embedding -is contained in. If the name reference is not not contained in a join, -the search is rooted at the machine definition that that the epsilon transition or -action embedding is contained in. Each component after the first is searched -for beginning at the location in the name tree that the previous reference -component refers to. - -In the case of action-based references, if the action is embedded more than -once, the local search is performed for each embedding and the result is the -union of all the searches. If no result is found for action-based references then -the search is repeated at the root of the name tree. Any action-based name -search may be forced into a strictly global search by prefixing the name -reference with \verb|::|. - -The final component of the name reference must resolve to a unique entry point. -If a name is unique in the entire name tree it can be referenced as is. If it -is not unique it can be specified by qualifying it with names above it in the -name tree. However, it can always be renamed. - -% FIXME: Should fit this in somewhere. -% Some kinds of name references are illegal. Cannot call into longest-match -% machine, can only call its start state. Cannot make a call to anywhere from -% any part of a longest-match machine except a rule's action. This would result -% in an eventual return to some point inside a longest-match other than the -% start state. This is banned for the same reason a call into the LM machine is -% banned. +\verb|(any - expr)|. It must be applied only to machines that match strings of +length one. \section{State Machine Minimization} @@ -1615,8 +1366,56 @@ listed in memory. Though exact analysis is very difficult, Ragel minimization runs close to $O(n \times log(n))$ and requires $O(n)$ temporary storage where $n$ is the number of states. +\section{Visualization} +\label{visualization} + +%In many cases, practical +%parsing programs will be too large to completely visualize with Graphviz. The +%proper approach is to reduce the language to the smallest subset possible that +%still exhibits the characteristics that one wishes to learn about or to fix. +%This can be done without modifying the source code using the \verb|-M| and +%\verb|-S| options. If a machine cannot be easily reduced, +%embeddings of unique actions can be very useful for tracing a +%particular component of a larger machine specification, since action names are +%written out on transition labels. + +Ragel is able to emit compiled state machines in Graphviz's Dot file format. +This is done using the \verb|-V| option. +Graphviz support allows users to perform +incremental visualization of their parsers. User actions are displayed on +transition labels of the graph. + +If the final graph is too large to be +meaningful, or even drawn, the user is able to inspect portions of the parser +by naming particular regular expression definitions with the \verb|-S| and +\verb|-M| options to the \verb|ragel| program. Use of Graphviz greatly +improves the Ragel programming experience. It allows users to learn Ragel by +experimentation and also to track down bugs caused by unintended +nondeterminism. + +Ragel has another option to help debugging. The \verb|-x| option causes Ragel +to emit the compiled machine in an XML format. + \chapter{User Actions} +Ragel permits the user to embed actions into the transitions of a regular +expression's corresponding state machine. These actions are executed when the +generated code moves over a transition. Like the regular expression operators, +the action embedding operators are fully compositional. They take a state +machine and an action as input, embed the action and yield a new state machine +that can be used in the construction of other machines. Due to the +compositional nature of embeddings, the user has complete freedom in the +placement of actions. + +A machine's transitions are categorized into four classes. The action embedding +operators access the transitions defined by these classes. The {\em entering +transition} operator \verb|>| isolates the start state, then embeds an action +into all transitions leaving it. The {\em finishing transition} operator +\verb|@| embeds an action into all transitions going into a final state. The +{\em all transition} operator \verb|$| embeds an action into all transitions of +an expression. The {\em leaving transition} operator \verb|%| provides access +to the yet-unmade transitions moving out of the machine via the final states. + \section{Embedding Actions} \begin{verbatim} @@ -1632,54 +1431,83 @@ Action names can be referenced by the action embedding operators in expressions. Though actions need not be named in this way (literal blocks of code can be embedded directly when building machines), defining reusable blocks of code whenever possible is good practice because it potentially increases the -degree to which the machine can be minimized. Within an action some Ragel expressions -and statements are parsed and translated. These allow the user to interact with the machine -from action code. See Section \ref{vals} for a complete list of statements and -values available in code blocks. +degree to which the machine can be minimized. + +Within an action some Ragel expressions and statements are parsed and +translated. These allow the user to interact with the machine from action code. +See Section \ref{vals} for a complete list of statements and values available +in code blocks. \subsection{Entering Action} \verb|expr > action| \verbspace -The entering operator embeds an action into the starting transitions. The -action is executed on all transitions that enter into the machine from the -start state. If the start state is a final state then it is possible for the -machine to never be entered and the starting transitions bypassed. In the -following example, the action is executed on the first transition of the -machine. If the repetition machine is bypassed the action is not executed. +The entering action operator embeds an action into all transitions +that enter into the machine from the start state. If the start state is final, +then the action is also embedded into the start state as a leaving action. This +means that if a machine accepts the zero-length string and control passes +through the start state then the entering action is executed. Note +that this can happen on both a following character and on the EOF event. + +In some machines the start state has transtions coming in from within the +machine. In these cases the start state is first isolated from the rest of the +machine ensuring that the entering actions are exected once only. \verbspace + +% GENERATE: exstact +% OPT: -p +% %%{ +% machine exstact; +\begin{inline_code} \begin{verbatim} # Execute A at the beginning of a string of alpha. +action A {} main := ( lower* >A ) . ' '; \end{verbatim} +\end{inline_code} +% }%% +% END GENERATE +\graphspace \begin{center} -\includegraphics{exstact} +\includegraphics[scale=0.55]{exstact} \end{center} +\graphspace \subsection{Finishing Action} \verb|expr @ action| \verbspace -The finishing action operator embeds an action into any transitions that go into a -final state. Whether or not the machine accepts is not determined at the point -the action is executed. Further input may move the machine out of the accepting -state, but keep it in the machine. As in the following example, the -into-final-state operator is most often used when no lookahead is necessary. +The finishing action operator embeds an action into any transitions that move +the machine into a final state. Further input may move the machine out of the +final state, but keep it in the machine. Therefore finishing actions may be +executed more than once if a machine has any internal transitions out of a +final state. In the following example the final state has no transitions out +and the finishing action is executed only once. -\verbspace +% GENERATE: exdoneact +% OPT: -p +% %%{ +% machine exdoneact; +% action A {} +\begin{inline_code} \begin{verbatim} # Execute A when the trailing space is seen. main := ( lower* ' ' ) @A; \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE +\graphspace \begin{center} -\includegraphics{exdoneact} +\includegraphics[scale=0.55]{exdoneact} \end{center} +\graphspace + \subsection{All Transition Action} @@ -1690,94 +1518,145 @@ The all transition operator embeds an action into all transitions of a machine. The action is executed whenever a transition of the machine is taken. In the following example, A is executed on every character matched. -\verbspace +% GENERATE: exallact +% OPT: -p +% %%{ +% machine exallact; +% action A {} +\begin{inline_code} \begin{verbatim} -# Execute A on any characters of machine one or two. +# Execute A on any characters of the machine. main := ( 'm1' | 'm2' ) $A; \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE +\graphspace \begin{center} -\includegraphics{exallact} +\includegraphics[scale=0.55]{exallact} \end{center} +\graphspace + -\subsection{Pending Out Actions} +\subsection{Leaving Actions} \label{out-actions} \verb|expr % action| \verbspace -The pending out action operator embeds an action into the pending out -transitions of a machine. The action is first embedded into the final states of -the machine and later transferred to any transitions made going out of the -machine. The transfer can be caused either by a concatenation or kleene star -operation. This mechanism allows one to associate an action with the -termination of a sequence, without being concerned about what particular -character terminates the sequence. In the following example, A is executed -when leaving the alpha machine by the newline character. +The leaving action operator queues an action for embedding into the transitions +that go out of a machine via a final state. The action is first stored in +the machine's final states and is later transferred to any transitions that are +made going out of the machine by a kleene star or concatenation operation. -\verbspace +If a final state of the machine is still final when compilation is complete +then the leaving action is also embedded as an EOF action. Therefore, leaving +the machine is defined as either leaving on a character or as state machine +acceptance. + +This operator allows one to associate an action with the termination of a +sequence, without being concerned about what particular character terminates +the sequence. In the following example, A is executed when leaving the alpha +machine on the newline character. + +% GENERATE: exoutact1 +% OPT: -p +% %%{ +% machine exoutact1; +% action A {} +\begin{inline_code} \begin{verbatim} -# Match a word followed by an newline. Execute A when +# Match a word followed by a newline. Execute A when # finishing the word. main := ( lower+ %A ) . '\n'; \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE +\graphspace \begin{center} -\includegraphics{exfinact} +\includegraphics[scale=0.55]{exoutact1} \end{center} \graphspace - In the following example, the \verb|term_word| action could be used to register the appearance of a word and to clear the buffer that the \verb|lower| action used to store the text of it. -\verbspace +% GENERATE: exoutact2 +% OPT: -p +% %%{ +% machine exoutact2; +% action lower {} +% action space {} +% action term_word {} +% action newline {} +\begin{inline_code} \begin{verbatim} word = ( [a-z] @lower )+ %term_word; main := word ( ' ' @space word )* '\n' @newline; \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE -% FIXME: add -%\begin{center} -%\includegraphics[scale=0.4]{outact.ps} -%\end{center} +\graphspace +\begin{center} +\includegraphics[scale=0.55]{exoutact2} +\end{center} +\graphspace -In this final example of the action embedding operators, A is executed upon -entering the alpha machine, B is executed on all transitions of the alpha -machine, C is executed when the alpha machine accepts by moving into the +In this final example of the action embedding operators, A is executed upon entering +the alpha machine, B is executed on all transitions of the +alpha machine, C is executed when the alpha machine is exited by moving into the newline machine and N is executed when the newline machine moves into a final state. -\verbspace +% GENERATE: exaction +% OPT: -p +% %%{ +% machine exaction; +% action A {} +% action B {} +% action C {} +% action N {} +\begin{inline_code} \begin{verbatim} # Execute A on starting the alpha machine, B on every transition # moving through it and C upon finishing. Execute N on the newline. main := ( lower* >A $B %C ) . '\n' @N; \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE +\graphspace \begin{center} -\includegraphics{exaction} +\includegraphics[scale=0.55]{exaction} \end{center} +\graphspace + \section{State Action Embedding Operators} The state embedding operators allow one to embed actions into states. Like the transition embedding operators, there are several different classes of states -that the operators access. The meanings of the symbols are partially related to -the meanings of the symbols used by the transition embedding operators. +that the operators access. The meanings of the symbols are similar to the +meanings of the symbols used for the transition embedding operators. The design +of the state selections was driven by a need to cover the states of an +expression with exactly one error action. -The state embedding operators are different from the transition embedding -operators in that there are various kinds of events that embedded actions can -be associated with, requiring them to be distinguished by these different types -of events. The state embedding operators have two components. The first, which -is the first one or two characters, specifies the class of states that the -action will be embedded into. The second component specifies the type of event -the action will be executed on. +Unlike the transition embedding operators, the state embedding operators are +also distinguished by the different kinds of events that embedded actions can +be associated with. Therefore the state embedding operators have two +components. The first, which is the first one or two characters, specifies the +class of states that the action will be embedded into. The second component +specifies the type of event the action will be executed on. The symbols of the +second component also have equivalent kewords. + +\vspace{10pt} \def\fakeitem{\hspace*{12pt}$\bullet$\hspace*{10pt}} @@ -1785,44 +1664,38 @@ the action will be executed on. \begin{multicols}{2} \raggedcolumns \noindent The different classes of states are:\\ -\fakeitem \verb|> | -- the start state \\ +\fakeitem \verb|> | -- the start state\\ +\fakeitem \verb|< | -- any state except the start state\\ \fakeitem \verb|$ | -- all states\\ \fakeitem \verb|% | -- final states\\ -\fakeitem \verb|< | -- any state except the start state\\ \fakeitem \verb|@ | -- any state except final states\\ \fakeitem \verb|<>| -- any except start and final (middle) \columnbreak \noindent The different kinds of embeddings are:\\ -\fakeitem \verb|~| -- to-state actions\\ -\fakeitem \verb|*| -- from-state actions\\ -\fakeitem \verb|/| -- EOF actions\\ -\fakeitem \verb|!| -- error actions\\ -\fakeitem \verb|^| -- local error actions\\ +\fakeitem \verb|~| -- to-state actions (\verb|to|)\\ +\fakeitem \verb|*| -- from-state actions (\verb|from|)\\ +\fakeitem \verb|/| -- EOF actions (\verb|eof|)\\ +\fakeitem \verb|!| -- error actions (\verb|err|)\\ +\fakeitem \verb|^| -- local error actions (\verb|lerr|)\\ \end{multicols} \end{minipage} -%\label{state-act-embed} -%\caption{The two components of state embedding operators. The class of states -%to select comes first, followed by the type of embedding.} -% -%\begin{figure}[t] -%\centering -%\includegraphics{stembed} -%\caption{Summary of state manipulation operators} -%\label{state-act-embed-chart} -%\end{figure} - -%\noindent Putting these two components together we get a matrix of state -%embedding operators. The entire set is given in Figure \ref{state-act-embed-chart}. - \subsection{To-State and From-State Actions} \subsubsection{To-State Actions} -\verb| >~ $~ %~ <~ @~ <>~ | -\verbspace +\def\sasp{\hspace*{40pt}} + +\sasp\verb|>~action >to(name) >to{...} | -- the start state\\ +\sasp\verb|<~action <to(name) <to{...} | -- any state except the start state\\ +\sasp\verb|$~action $to(name) $to{...} | -- all states\\ +\sasp\verb|%~action %to(name) %to{...} | -- final states\\ +\sasp\verb|@~action @to(name) @to{...} | -- any state except final states\\ +\sasp\verb|<>~action <>to(name) <>to{...}| -- any except start and final (middle) +\vspace{12pt} + To-state actions are executed whenever the state machine moves into the specified state, either by a natural movement over a transition or by an @@ -1841,8 +1714,13 @@ of to-state actions. \subsubsection{From-State Actions} -\verb| >* $* %* <* @* <>* | -\verbspace +\sasp\verb|>*action >from(name) >from{...} | -- the start state\\ +\sasp\verb|<*action <from(name) <from{...} | -- any state except the start state\\ +\sasp\verb|$*action $from(name) $from{...} | -- all states\\ +\sasp\verb|%*action %from(name) %from{...} | -- final states\\ +\sasp\verb|@*action @from(name) @from{...} | -- any state except final states\\ +\sasp\verb|<>*action <>from(name) <>from{...}| -- any except start and final (middle) +\vspace{12pt} From-state actions are executed whenever the state machine takes a transition from a state, either to itself or to some other state. These actions are executed @@ -1854,101 +1732,175 @@ embeddings, from-state embeddings stay with the state. \subsection{EOF Actions} -\verb| >/ $/ %/ </ @/ <>/ | -\verbspace +\sasp\verb|>/action >eof(name) >eof{...} | -- the start state\\ +\sasp\verb|</action <eof(name) <eof{...} | -- any state except the start state\\ +\sasp\verb|$/action $eof(name) $eof{...} | -- all states\\ +\sasp\verb|%/action %eof(name) %eof{...} | -- final states\\ +\sasp\verb|@/action @eof(name) @eof{...} | -- any state except final states\\ +\sasp\verb|<>/action <>eof(name) <>eof{...}| -- any except start and final (middle) +\vspace{12pt} -The EOF action embedding operators enable the user to embed EOF actions into -different classes of -states. EOF actions are stored in states and generated with the \verb|write eof| -statement. The generated EOF code switches on the current state and executes the EOF -actions associated with it. +The EOF action embedding operators enable the user to embed actions that are +executed at the end of the input stream. EOF actions are stored in states and +generated in the \verb|write exec| block. They are run when \verb|p == pe == eof| +as the execute block is finishing. EOF actions are free to adjust \verb|p| and +jump to another part of the machine to restart execution. \subsection{Handling Errors} +In many applications it is useful to be able to react to parsing errors. The +user may wish to print an error message that depends on the context. It +may also be desirable to consume input in an attempt to return the input stream +to some known state and resume parsing. To support error handling and recovery, +Ragel provides error action embedding operators. There are two kinds of error +actions: global error actions and local error actions. +Error actions can be used to simply report errors, or by jumping to a machine +instantiation that consumes input, can attempt to recover from errors. + \subsubsection{Global Error Actions} -\verb| >! $! %! <! @! <>! | -\verbspace +\sasp\verb|>!action >err(name) >err{...} | -- the start state\\ +\sasp\verb|<!action <err(name) <err{...} | -- any state except the start state\\ +\sasp\verb|$!action $err(name) $err{...} | -- all states\\ +\sasp\verb|%!action %err(name) %err{...} | -- final states\\ +\sasp\verb|@!action @err(name) @err{...} | -- any state except final states\\ +\sasp\verb|<>!action <>err(name) <>err{...}| -- any except start and final (middle) +\vspace{12pt} -Error actions are stored in states until the final state machine has been fully -constructed. They are then transferred to the transitions that move into the -error state. This transfer entails the creation of a transition from the state -to the error state that is taken on all input characters which are not already -covered by the state's transitions. In other words it provides a default -action. Error actions can induce a recovery by altering \verb|p| and then jumping back -into the machine with \verb|fgoto|. +Global error actions are stored in the states they are embedded into until +compilation is complete. They are then transferred to the transitions that move +into the error state. These transitions are taken on all input characters that +are not already covered by the state's transitions. If a state with an error +action is not final when compilation is complete, then the action is also +embedded as an EOF action. + +Error actions can be used to recover from errors by jumping back into the +machine with \verb|fgoto| and optionally altering \verb|p|. \subsubsection{Local Error Actions} -\verb| >^ $^ %^ <^ @^ <>^ | -\verbspace +\sasp\verb|>^action >lerr(name) >lerr{...} | -- the start state\\ +\sasp\verb|<^action <lerr(name) <lerr{...} | -- any state except the start state\\ +\sasp\verb|$^action $lerr(name) $lerr{...} | -- all states\\ +\sasp\verb|%^action %lerr(name) %lerr{...} | -- final states\\ +\sasp\verb|@^action @lerr(name) @lerr{...} | -- any state except final states\\ +\sasp\verb|<>^action <>lerr(name) <>lerr{...}| -- any except start and final (middle) +\vspace{12pt} + +Like global error actions, local error actions are also stored in the states +they are embedded into until a transfer point. The transfer point is different +however. Each local error action embedding is associated with a name. When a +machine definition has been fully constructed, all local error action +embeddings associated with the same name as the machine definition are +transferred to the error transitions. At this time they are also embedded as +EOF actions in the case of non-final states. + +Local error actions can be used to specify an action to take when a particular +section of a larger state machine fails to match. A particular machine +definition's ``thread'' may die and the local error actions executed, however +the machine as a whole may continue to match input. + +There are two forms of local error action embeddings. In the first form the +name defaults to the current machine. In the second form the machine name can +be specified. This is useful when it is more convenient to specify the local +error action in a sub-definition that is used to construct the machine +definition that the local error action is associated with. To embed local +error actions and +explicitly state the machine definition on which the transfer is to happen use +\verb|(name, action)| as the action. + +\subsubsection{Example} + +The following example uses error actions to report an error and jump to a +machine that consumes the remainder of the line when parsing fails. After +consuming the line, the error recovery machine returns to the main loop. + +% GENERATE: erract +% %%{ +% machine erract; +% ws = ' '; +% address = 'foo@bar.com'; +% date = 'Monday May 12'; +\begin{inline_code} +\begin{verbatim} +action cmd_err { + printf( "command error\n" ); + fhold; fgoto line; +} +action from_err { + printf( "from error\n" ); + fhold; fgoto line; +} +action to_err { + printf( "to error\n" ); + fhold; fgoto line; +} + +line := [^\n]* '\n' @{ fgoto main; }; + +main := ( + ( + 'from' @err(cmd_err) + ( ws+ address ws+ date '\n' ) $err(from_err) | + 'to' @err(cmd_err) + ( ws+ address '\n' ) $err(to_err) + ) +)*; +\end{verbatim} +\end{inline_code} +% }%% +% %% write data; +% void f() +% { +% %% write init; +% %% write exec; +% } +% END GENERATE -Like global error actions, local error actions are also stored in states until -a transfer point. The transfer point is different however. Each local error action -embedding is associated with a name. When a machine definition has been fully -constructed, all local error actions embeddings associated the same name as the -machine are transferred to error transitions. Local error actions can be used -to specify an action to take when a particular section of a larger state -machine fails to make a match. A particular machine definition's ``thread'' may -die and the local error actions executed, however the machine as a whole may -continue to match input. - -There are two forms of local error action embeddings. In the first form the name defaults -to the current machine. In the second form the machine name can be specified. This -is useful when it is more convenient to specify the local error action in a -sub-definition that is used to construct the machine definition where the -transfer should happen. To embed local error actions and explicitly state the -machine on which the transfer is to happen use \verb|(name, action)| as the -action. -\begin{comment} -\begin{itemize} -\setlength{\parskip}{0in} -\item \verb|expr >^ (name, action) | -- Start state. -\item \verb|expr $^ (name, action) | -- All states. -\item \verb|expr %^ (name, action) | -- Final states. -\item \verb|expr <^ (name, action) | -- Not start state. -\item \verb|expr <>^ (name, action)| -- Not start and not final states. -\end{itemize} -\end{comment} \section{Action Ordering and Duplicates} -When building a parser by combining smaller expressions which themselves have -embedded actions, it is often the case that transitions are made which need to -execute a number of actions on one input character. For example when we leave -an expression, we may execute the expression's pending out action and the -subsequent expression's starting action on the same input character. We must -therefore devise a method for ordering actions that is both intuitive and -predictable for the user and repeatable by the state machine compiler. The -determinization processes cannot simply order actions by the time at which they -are introduced into a transition -- otherwise the programmer will be at the -mercy of luck. - -We associate with the embedding of each action a distinct timestamp which is +When combining expressions that have embedded actions it is often the case that +a number of actions must be executed on a single input character. For example, +following a concatenation the leaving action of the left expression and the +entering action of the right expression will be embedded into one transition. +This requires a method of ordering actions that is intuitive and +predictable for the user, and repeatable for the compiler. + +We associate with the embedding of each action a unique timestamp that is used to order actions that appear together on a single transition in the final -compiled state machine. To accomplish this we traverse the parse tree of -regular expressions and assign timestamps to action embeddings. This algorithm -is recursive in nature and quite simple. When it visits a parse tree node it -assigns timestamps to all {\em starting} action embeddings, recurses on the -parse tree, then assigns timestamps to the remaining {\em all}, {\em -finishing}, and {\em leaving} embeddings in the order in which they appear. - -Ragel does not permit actions (defined or unnamed) to appear multiple times in -an action list. When the final machine has been created, actions which appear -more than once in single transition or EOF action list have their duplicates -removed. The first appearance of the action is preserved. This is useful in a -number of scenarios. First, it allows us to union machines with common -prefixes without worrying about the action embeddings in the prefix being -duplicated. Second, it prevents pending out actions from being transferred multiple times -when a concatenation follows a kleene star and the two machines begin with a common -character. +state machine. To accomplish this we recursively traverse the parse tree of +regular expressions and assign timestamps to action embeddings. References to +machine definitions are followed in the traversal. When we visit a +parse tree node we assign timestamps to all {\em entering} action embeddings, +recurse on the parse tree, then assign timestamps to the remaining {\em all}, +{\em finishing}, and {\em leaving} embeddings in the order in which they +appear. + +By default Ragel does not permit a single action to appear multiple times in an action +list. When the final machine has been created, actions that appear more than +once in a single transition, to-state, from-state or EOF action list have their +duplicates removed. +The first appearance of the action is preserved. This is useful in a number of +scenarios. First, it allows us to union machines with common prefixes without +worrying about the action embeddings in the prefix being duplicated. Second, it +prevents leaving actions from being transferred multiple times. This can +happen when a machine is repeated, then followed with another machine that +begins with a common character. For example: \verbspace \begin{verbatim} word = [a-z]+ %act; main := word ( '\n' word )* '\n\n'; \end{verbatim} +\verbspace + +Note that Ragel does not compare action bodies to determine if they have +identical program text. It simply checks for duplicates using each action +block's unique location in the program. + +The removal of duplicates can be turned off using the \verb|-d| option. \section{Values and Statements Available in Code Blocks} \label{vals} @@ -1975,8 +1927,8 @@ entry point \verb|label|. The integer value returned will be a compile time constant. This number is suitable for later use in control flow transfer statements that take an expression. This value should not be compared against the current state because any given label can have multiple states representing -it. The value returned by \verb|fentry| will be one of the possibly multiple states the -label represents. +it. The value returned by \verb|fentry| can be any one of the multiple states that +it represents. \end{itemize} \noindent The following statements are available in code blocks: @@ -1987,24 +1939,19 @@ label represents. data in multiple buffer blocks, the \verb|fhold| statement should only be used once in the set of actions executed on a character. Multiple calls may result in backing up over the beginning of the buffer block. The \verb|fhold| -statement does not imply any transfer of control. In actions embedded into -transitions, it is equivalent to the \verb|p--;| statement. In scanner pattern -actions any changes made to \verb|p| are lost. In this context, \verb|fhold| is -equivalent to \verb|tokend--;|. +statement does not imply any transfer of control. It is equivalent to the +\verb|p--;| statement. \item \verb|fexec <expr>;| -- Set the next character to process. This can be used to backtrack to previous input or advance ahead. Unlike \verb|fhold|, which can be used anywhere, \verb|fexec| requires the user to ensure that the target of the backtrack is in the current buffer block or is known to be somewhere ahead of -it. The machine will continue iterating forward until \verb|pe| is arrived, +it. The machine will continue iterating forward until \verb|pe| is arrived at, \verb|fbreak| is called or the machine moves into the error state. In actions embedded into transitions, the \verb|fexec| statement is equivalent to setting \verb|p| to one position ahead of the next character to process. If the user also modifies \verb|pe|, it is possible to change the buffer block entirely. -In scanner pattern actions any changes made to \verb|p| are lost. In this -context, \verb|fexec| is equivalent to setting \verb|tokend| to the next -character to process. \item \verb|fgoto <label>;| -- Jump to an entry point defined by \verb|<label>|. The \verb|fgoto| statement immediately transfers control to @@ -2027,6 +1974,7 @@ of the transition on which the call was made. Use of \verb|fcall| requires the declaration of a call stack. An array of integers named \verb|stack| and a single integer named \verb|top| must be declared. With the \verb|fcall| construct, control is immediately transferred to the destination state. +See section \ref{modularization} for more information. \item \verb|fcall *<expr>;| -- Push the current state and jump to the entry point given by \verb|<expr>|. The expression must evaluate to an integer value @@ -2034,18 +1982,16 @@ representing a state. \item \verb|fret;| -- Return to the target state of the transition on which the last \verb|fcall| was made. Use of \verb|fret| requires the declaration of a -call stack with \verb|fstack| in the struct block. Control is immediately -transferred to the destination state. - -\item \verb|fbreak;| -- Save the current state and immediately break out of the -execute loop. This statement is useful in conjunction with the \verb|noend| -write option. Rather than process input until the end marker of the input -buffer is arrived at, the fbreak statement can be used to stop processing input -upon seeing some end-of-string marker. It can also be used for handling -exceptional circumstances. The fbreak statement does not change the pointer to -the current character. After an \verb|fbreak| call the \verb|p| variable will point to -the character that was being traversed over when the action was -executed. The current state will be the target of the current transition. +call stack. Control is immediately transferred to the destination state. + +\item \verb|fbreak;| -- Advance \verb|p|, save the target state to \verb|cs| +and immediately break out of the execute loop. This statement is useful +in conjunction with the \verb|noend| write option. Rather than process input +until \verb|pe| is arrived at, the fbreak statement +can be used to stop processing from an action. After an \verb|fbreak| +statement the \verb|p| variable will point to the next character in the input. The +current state will be the target of the current transition. Note that \verb|fbreak| +causes the target state's to-state actions to be skipped. \end{itemize} @@ -2055,7 +2001,7 @@ to other machine construction operators. If an action jumps to another state then unioning any transition that executes that action with another transition that follows some other path will cause that other path to be lost. Using commands that manually jump around a machine takes us out of the domain of -regular languages because transitions that may be conditional and that the +regular languages because transitions that the machine construction operators are not aware of are introduced. These commands should therefore be used with caution. @@ -2065,22 +2011,19 @@ commands should therefore be used with caution. Along with the flexibility of arbitrary action embeddings comes a need to control nondeterminism in regular expressions. If a regular expression is -ambiguous, then sup-components of a parser other than the intended parts may be -active at any given time. This means that actions which are irrelevant to the +ambiguous, then sub-components of a parser other than the intended parts may become +active. This means that actions that are irrelevant to the current subset of the parser may be executed, causing problems for the programmer. -Tools which are based on regular expression engines and which are used for +Tools that are based on regular expression engines and that are used for recognition tasks will usually function as intended regardless of the presence of ambiguities. It is quite common for users of scripting languages to write regular expressions that are heavily ambiguous and it generally does not matter. As long as one of the potential matches is recognized, there can be any -number of other matches present. - -In some systems, matched text is attributed to a portion of a regular -expression. Armed with the knowledge that the regular expression engine always -pursues the longest match or the shortest match, the user is able to compose -their patterns accordingly. +number of other matches present. In some parsing systems the run-time engine +can employ a strategy for resolving ambiguities, for example always pursuing +the longest possible match and discarding others. In Ragel, there is no regular expression run-time engine, just a simple state machine execution model. When we begin to embed actions and face the @@ -2088,67 +2031,170 @@ possibility of spurious action execution, it becomes clear that controlling nondeterminism at the machine construction level is very important. Consider the following example. -\verbspace +% GENERATE: lines1 +% OPT: -p +% %%{ +% machine lines1; +% action first {} +% action tail {} +% word = [a-z]+; +\begin{inline_code} \begin{verbatim} -lines = ( word ( space word )* '\n' )*; +ws = [\n\t ]; +line = word $first ( ws word $tail )* '\n'; +lines = line*; \end{verbatim} -\verbspace +\end{inline_code} +% main := lines; +% }%% +% END GENERATE -Since the \verb|space| built-in expression includes the newline character, we will -not leave the line expression when a newline character is seen. We will +\begin{center} +\includegraphics[scale=0.53]{lines1} +\end{center} +\graphspace + +Since the \verb|ws| expression includes the newline character, we will +not finish the \verb|line| expression when a newline character is seen. We will simultaneously pursue the possibility of matching further words on the same -line and the possibility of matching a second line. The solution here is easy: -simply exclude the newline character from the \verb|space| expression. Solving -this kind of problem is straightforward because the string that terminates the -sequence is a single character long. When it is multiple characters long we -have a more difficult problem, as shown by the following example. +line and the possibility of matching a second line. Evidence of this fact is +in the state tables. On several transitions both the \verb|first| and +\verb|tail| actions are executed. The solution here is simple: exclude +the newline character from the \verb|ws| expression. -\verbspace +% GENERATE: lines2 +% OPT: -p +% %%{ +% machine lines2; +% action first {} +% action tail {} +% word = [a-z]+; +\begin{inline_code} \begin{verbatim} -comment = '/*' any* '*/'; +ws = [\t ]; +line = word $first ( ws word $tail )* '\n'; +lines = line*; \end{verbatim} -\verbspace +\end{inline_code} +% main := lines; +% }%% +% END GENERATE + +\begin{center} +\includegraphics[scale=0.55]{lines2} +\end{center} +\graphspace + +Solving this kind of problem is straightforward when the ambiguity is created +by strings that are a single character long. When the ambiguity is created by +strings that are multiple characters long we have a more difficult problem. +The following example is an incorrect attempt at a regular expression for C +language comments. + +% GENERATE: comments1 +% OPT: -p +% %%{ +% machine comments1; +% action comm {} +\begin{inline_code} +\begin{verbatim} +comment = '/*' ( any @comm )* '*/'; +main := comment ' '; +\end{verbatim} +\end{inline_code} +% }%% +% END GENERATE + +\begin{center} +\includegraphics[scale=0.55]{comments1} +\end{center} +\graphspace Using standard concatenation, we will never leave the \verb|any*| expression. We will forever entertain the possibility that a \verb|'*/'| string that we see is contained in a longer comment and that, simultaneously, the comment has -ended. One way to approach the problem is to exclude the terminating string +ended. The concatenation of the \verb|comment| machine with \verb|SP| is done +to show this. When we match space, we are also still matching the comment body. + +One way to approach the problem is to exclude the terminating string from the \verb|any*| expression using set difference. We must be careful to exclude not just the terminating string, but any string that contains it as a substring. A verbose, but proper specification of a C comment parser is given -by the following regular expression. Note that this operation is the basis of the -strong subtraction operator. +by the following regular expression. -\verbspace +% GENERATE: comments2 +% OPT: -p +% %%{ +% machine comments2; +% action comm {} +\begin{inline_code} \begin{verbatim} -comment = '/*' ( any* - ( any* '*/' any* ) ) '*/'; +comment = '/*' ( ( any @comm )* - ( any* '*/' any* ) ) '*/'; \end{verbatim} -\verbspace +\end{inline_code} +% main := comment; +% }%% +% END GENERATE + +\graphspace +\begin{center} +\includegraphics[scale=0.55]{comments2} +\end{center} +\graphspace +Note that Ragel's strong subtraction operator \verb|--| can also be used here. +In doing this subtraction we have phrased the problem of controlling non-determinism in +terms of excluding strings common to two expressions that interact when +combined. We can also phrase the problem in terms of the transitions of the state machines that implement these expressions. During the concatenation of \verb|any*| and \verb|'*/'| we will be making transitions that are composed of -both the loop of the first expression and the characters of the second. +both the loop of the first expression and the final character of the second. At this time we want the transition on the \verb|'/'| character to take precedence over and disallow the transition that originated in the \verb|any*| loop. -In another scenario, we wish to implement a lightweight tokenizer that we can +In another parsing problem, we wish to implement a lightweight tokenizer that we can utilize in the composition of a larger machine. For example, some HTTP headers -have a token stream as a sub-language. +have a token stream as a sub-language. The following example is an attempt +at a regular expression-based tokenizer that does not function correctly due to +unintended nondeterminism. -\verbspace +\newpage + +% GENERATE: smallscanner +% OPT: -p +% %%{ +% machine smallscanner; +% action start_str {} +% action on_char {} +% action finish_str {} +\begin{inline_code} \begin{verbatim} -header_contents = ( lower+ | digit+ | ' ' )*; +header_contents = ( + lower+ >start_str $on_char %finish_str | + ' ' +)*; \end{verbatim} -\verbspace +\end{inline_code} +% main := header_contents; +% }%% +% END GENERATE + +\begin{center} +\includegraphics[scale=0.55]{smallscanner} +\end{center} +\graphspace In this case, the problem with using a standard kleene star operation is that -there is an ambiguity between extending a token and wrapping around the -machine to begin a new token. Using the standard operator, we get -an undesirable nondeterministic behaviour. What is required is for the +there is an ambiguity between extending a token and wrapping around the machine +to begin a new token. Using the standard operator, we get an undesirable +nondeterministic behaviour. Evidence of this can be seen on the transition out +of state one to itself. The transition extends the string, and simultaneously, +finishes the string only to immediately begin a new one. What is required is +for the transitions that represent an extension of a token to take precedence over the -transitions that represent the beginning of a new token. For this problem, -there is no simple solution that uses standard regular expressions. +transitions that represent the beginning of a new token. For this problem +there is no simple solution that uses standard regular expression operators. \section{Priorities} @@ -2171,14 +2217,6 @@ eliminates the need to ever delete them. Such a scheme allows the user to choose a unique name, embed two different priority values using that name and be confident that the priority embedding will be free of any side effects. -\section{Priority Assignment} - -Priorities are integer values assigned to names within transitions. -Only priorities with the same name are allowed to interact. When the machine -construction process is combining transitions that have different priorities -assiged to the same name, the transition with the higher priority is preserved -and the lower priority is dropped. - In the first form of priority embedding the name defaults to the name of the machine definition that the priority is assigned in. In this sense priorities are by default local to the current machine definition or instantiation. Beware of @@ -2192,10 +2230,10 @@ boundaries. \item \verb|expr > int| -- Sets starting transitions to have priority int. \item \verb|expr @ int| -- Sets transitions that go into a final state to have priority int. \item \verb|expr $ int| -- Sets all transitions to have priority int. -\item \verb|expr % int| -- Sets pending out transitions from final states to -have priority int.\\ When a transition is made going out of the machine (either -by concatenation or kleene star) its priority is immediately set to the pending -out priority. +\item \verb|expr % int| -- Sets leaving transitions to +have priority int. When a transition is made going out of the machine (either +by concatenation or kleene star) its priority is immediately set to the +leaving priority. \end{itemize} The second form of priority assignment allows the programmer to specify the name @@ -2203,19 +2241,21 @@ to which the priority is assigned. \begin{itemize} \setlength{\parskip}{0in} -\item \verb|expr > (name, int)| -- Entering transitions. -\item \verb|expr @ (name, int)| -- Transitions into final state. +\item \verb|expr > (name, int)| -- Starting transitions. +\item \verb|expr @ (name, int)| -- Finishing transitions (into a final state). \item \verb|expr $ (name, int)| -- All transitions. -\item \verb|expr % (name, int)| -- Pending out transitions. +\item \verb|expr % (name, int)| -- Leaving transitions. \end{itemize} \section{Guarded Operators that Encapsulate Priorities} -Priorities can be very confusing for the user. They force the user to imagine -the transitions inside machines and work out the precise effects of regular -expression operations. When we consider that this problem is worsened by the +Priority embeddings are a very expressive mechanism. At the same time they +can be very confusing for the user. They force the user to imagine +the transitions inside two interacting expressions and work out the precise +effects of the operations between them. When we consider +that this problem is worsened by the potential for side effects caused by unintended priority name collisions, we -see that exposing the user to priorities is rather undesirable. +see that exposing the user to priorities is undesirable. Fortunately, in practice the use of priorities has been necessary only in a small number of scenarios. This allows us to encapsulate their functionality @@ -2223,56 +2263,82 @@ into a small set of operators and fully hide them from the user. This is advantageous from a language design point of view because it greatly simplifies the design. -\begin{comment} -Example from 2 page poster paper. -% GENERATE: lmkleene -% %%{ -% machine lmkleene; -% action id {} -% action number {} -% action ws {} -% action mark {} +Going back to the C comment example, we can now properly specify +it using a guarded concatenation operator which we call {\em finish-guarded +concatenation}. From the user's point of view, this operator terminates the +first machine when the second machine moves into a final state. It chooses a +unique name and uses it to embed a low priority into all +transitions of the first machine. A higher priority is then embedded into the +transitions of the second machine that enter into a final state. The following +example yields a machine identical to the example in Section +\ref{controlling-nondeterminism}. + +\begin{inline_code} \begin{verbatim} -main := ( lower+ ':' ' '* <: ( - ( lower ( lower | digit )* ) >mark %id | - digit+ >mark %number | - ' '+ >mark %ws -)** '\n' )*; +comment = '/*' ( any @comm )* :>> '*/'; \end{verbatim} -% }%% -% END GENERATE +\end{inline_code} -% FIXME: Add -%\begin{center} -%\includegraphics[scale=0.4]{lmkleene.ps} -%\end{center} -\end{comment} +\graphspace +\begin{center} +\includegraphics[scale=0.55]{comments2} +\end{center} +\graphspace -\subsection{Entry-Guarded Contatenation} +Another guarded operator is {\em left-guarded concatenation}, given by the +\verb|<:| compound symbol. This operator places a higher priority on all +transitions of the first machine. This is useful if one must forcibly separate +two lists that contain common elements. For example, one may need to tokenize a +stream, but first consume leading whitespace. + +Ragel also includes a {\em longest-match kleene star} operator, given by the +\verb|**| compound symbol. This +guarded operator embeds a high +priority into all transitions of the machine. +A lower priority is then embedded into the leaving transitions. When the +kleene star operator makes the epsilon transitions from +the final states into the new start state, the lower priority will be transferred +to the epsilon transitions. In cases where following an epsilon transition +out of a final state conflicts with an existing transition out of a final +state, the epsilon transition will be dropped. + +Other guarded operators are conceivable, such as guards on union that cause one +alternative to take precedence over another. These may be implemented when it +is clear they constitute a frequently used operation. +In the next section we discuss the explicit specification of state machines +using state charts. + +\subsection{Entry-Guarded Concatenation} \verb|expr :> expr| \verbspace This operator concatenates two machines, but first assigns a low priority to all transitions -of the first machine and a high priority to the entering transitions of the +of the first machine and a high priority to the starting transitions of the second machine. This operator is useful if from the final states of the first -machine, it is possible to accept the characters in the start transitions of +machine it is possible to accept the characters in the entering transitions of the second machine. This operator effectively terminates the first machine -immediately upon entering the second machine, where otherwise they would be +immediately upon starting the second machine, where otherwise they would be pursued concurrently. In the following example, entry-guarded concatenation is used to move out of a machine that matches everything at the first sign of an end-of-input marker. -\verbspace +% GENERATE: entryguard +% OPT: -p +% %%{ +% machine entryguard; +\begin{inline_code} \begin{verbatim} # Leave the catch-all machine on the first character of FIN. main := any* :> 'FIN'; \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE \begin{center} -\includegraphics{exstpri} +\includegraphics[scale=0.55]{entryguard} \end{center} \graphspace @@ -2283,7 +2349,7 @@ Entry-guarded concatenation is equivalent to the following: expr $(unique_name,0) . expr >(unique_name,1) \end{verbatim} -\subsection{Finish-Guarded Contatenation} +\subsection{Finish-Guarded Concatenation} \verb|expr :>> expr| \verbspace @@ -2297,19 +2363,28 @@ machine only when the second accepts. In the following example, finish-guarded concatenation causes the move out of the machine that matches everything to be delayed until the full end-of-input marker has been matched. -\verbspace +% GENERATE: finguard +% OPT: -p +% %%{ +% machine finguard; +\begin{inline_code} \begin{verbatim} # Leave the catch-all machine on the last character of FIN. main := any* :>> 'FIN'; \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE \begin{center} -\includegraphics{exdonepri} +\includegraphics[scale=0.55]{finguard} \end{center} \graphspace -Finish-guarded concatenation is equivalent to the following: +Finish-guarded concatenation is equivalent to the following, with one +exception. If the right machine's start state is final, the higher priority is +also embedded into it as a leaving priority. This prevents the left machine +from persisting via the zero-length string. \verbspace \begin{verbatim} @@ -2327,11 +2402,27 @@ sequence with another sequence composed of some of the same characters. For example, one can consume leading whitespace before tokenizing a sequence of whitespace-separated words as in: -\verbspace +% GENERATE: leftguard +% OPT: -p +% %%{ +% machine leftguard; +% action alpha {} +% action ws {} +% action start {} +% action fin {} +\begin{inline_code} \begin{verbatim} -( ' '* <: ( ' '+ | [a-z]+ )** ) +main := ( ' '* >start %fin ) <: ( ' ' $ws | [a-z] $alpha )*; \end{verbatim} -\verbspace +\end{inline_code} +% }%% +% END GENERATE + +\graphspace +\begin{center} +\includegraphics[scale=0.55]{leftguard} +\end{center} +\graphspace Left-guarded concatenation is equivalent to the following: @@ -2354,6 +2445,14 @@ longest-match kleene star to an alternation of token patterns, as in the following. \verbspace + +% GENERATE: lmkleene +% OPT: -p +% %%{ +% machine exfinpri; +% action A {} +% action B {} +\begin{inline_code} \begin{verbatim} # Repeat tokens, but make sure to get the longest match. main := ( @@ -2362,11 +2461,12 @@ main := ( ' ' )**; \end{verbatim} - -\verbspace +\end{inline_code} +% }%% +% END GENERATE \begin{center} -\includegraphics{exfinpri} +\includegraphics[scale=0.55]{lmkleene} \end{center} \graphspace @@ -2380,20 +2480,139 @@ equivalent to: \end{verbatim} \verbspace -When the kleene star is applied, transitions are made out of the machine which -go back into it. These are assigned a priority of zero by the pending out -transition mechanism. This is less than the priority of the transitions out of -the final states that do not leave the machine. When two transitions clash on -the same character, the differing priorities causes the transition which -stays in the machine to take precedence. The transition that wraps around is -dropped. +When the kleene star is applied, transitions that go out of the machine and +back into it are made. These are assigned a priority of zero by the leaving +transition mechanism. This is less than the priority of one assigned to the +transitions leaving the final states but not leaving the machine. When +these transitions clash on the same character, the +transition that stays in the machine takes precedence. The transition +that wraps around is dropped. -Note that this operator does not build a scanner in the traditional sense because -there is never any backtracking. To build a scanner in the traditional sense -use the Longest-Match machine construction described Section \ref{generating-scanners}. +Note that this operator does not build a scanner in the traditional sense +because there is never any backtracking. To build a scanner with backtracking +use the Longest-Match machine construction described in Section +\ref{generating-scanners}. \chapter{Interface to Host Program} +The Ragel code generator is very flexible. The generated code has no +dependencies and can be inserted in any function, perhaps inside a loop if +desired. The user is responsible for declaring and initializing a number of +required variables, including the current state and the pointer to the input +stream. These can live in any scope. Control of the input processing loop is +also possible: the user may break out of the processing loop and return to it +at any time. + +In the case of C and D host languages, Ragel is able to generate very +fast-running code that implements state machines as directly executable code. +Since very large files strain the host language compiler, table-based code +generation is also supported. In the future we hope to provide a partitioned, +directly executable format that is able to reduce the burden on the host +compiler by splitting large machines across multiple functions. + +In the case of Java and Ruby, table-based code generation is the only code +style supported. In the future this may be expanded to include other code +styles. + +Ragel can be used to parse input in one block, or it can be used to parse input +in a sequence of blocks as it arrives from a file or socket. Parsing the input +in a sequence of blocks brings with it a few responsibilities. If the parser +utilizes a scanner, care must be taken to not break the input stream anywhere +but token boundaries. If pointers to the input stream are taken during +parsing, care must be taken to not use a pointer that has been invalidated by +movement to a subsequent block. If the current input data pointer is moved +backwards it must not be moved past the beginning of the current block. + +Figure \ref{basic-example} shows a simple Ragel program that does not have any +actions. The example tests the first argument of the program against a number +pattern and then prints the machine's acceptance status. + +\begin{figure} +\small +\begin{verbatim} +#include <stdio.h> +#include <string.h> +%%{ + machine foo; + write data; +}%% +int main( int argc, char **argv ) +{ + int cs; + if ( argc > 1 ) { + char *p = argv[1]; + char *pe = p + strlen( p ); + %%{ + main := [0-9]+ ( '.' [0-9]+ )?; + + write init; + write exec; + }%% + } + printf("result = %i\n", cs >= foo_first_final ); + return 0; +} +\end{verbatim} +\caption{A basic Ragel example without any actions.} +\label{basic-example} +\end{figure} + +\section{Variables Used by Ragel} + +There are a number of variables that Ragel expects the user to declare. At a +very minimum the \verb|cs|, \verb|p| and \verb|pe| variables must be declared. +In Java and Ruby code the \verb|data| variable must also be declared. If +EOF actions are used then the \verb|eof| variable is required. If +stack-based state machine control flow statements are used then the +\verb|stack| and \verb|top| variables are required. If a scanner is declared +then the \verb|act|, \verb|ts| and \verb|te| variables must be +declared. + +\begin{itemize} + +\item \verb|cs| - Current state. This must be an integer and it should persist +across invocations of the machine when the data is broken into blocks that are +processed independently. This variable may be modified from outside the +execution loop, but not from within. + +\item \verb|p| - Data pointer. In C/D code this variable is expected to be a +pointer to the character data to process. It should be initialized to the +beginning of the data block on every run of the machine. In Java and Ruby it is +used as an offset to \verb|data| and must be an integer. In this case it should +be initialized to zero on every run of the machine. + +\item \verb|pe| - Data end pointer. This should be initialized to \verb|p| plus +the data length on every run of the machine. In Java and Ruby code this should +be initialized to the data length. + +\item \verb|eof| - End of file pointer. This should be set to \verb|pe| when +the buffer block being processed is the last one, otherwise it should be set to +null. In Java and Ruby code \verb|-1| must be used instead of null. If the EOF +event can be known only after the final buffer block has been processed, then +it is possible to set \verb|p = pe = eof| and run the execute block. + +\item \verb|data| - This variable is only required in Java and Ruby code. It +must be an array containting the data to process. + +\item \verb|stack| - This must be an array of integers. It is used to store +integer values representing states. If the stack must resize dynamically the +Pre-push and Post-Pop statements can be used to do this (Sections +\ref{prepush} and \ref{postpop}). + +\item \verb|top| - This must be an integer value and will be used as an offset +to \verb|stack|, giving the next available spot on the top of the stack. + +\item \verb|act| - This must be an integer value. It is a variable sometimes +used by scanner code to keep track of the most recent successful pattern match. + +\item \verb|ts| - This must be a pointer to character data. In Java and +Ruby code this must be an integer. See Section \ref{generating-scanners} for +more information. + +\item \verb|te| - Also a pointer to character data. + +\end{itemize} + \section{Alphtype Statement} \begin{verbatim} @@ -2402,11 +2621,47 @@ alphtype unsigned int; \verbspace The alphtype statement specifies the alphabet data type that the machine -operates on. During the compilation of the machine, integer literals are expected to -be in the range of possible values of the alphtype. Supported alphabet types -are \verb|char|, \verb|unsigned char|, \verb|short|, \verb|unsigned short|, -\verb|int|, \verb|unsigned int|, \verb|long|, and \verb|unsigned long|. -The default is \verb|char|. +operates on. During the compilation of the machine, integer literals are +expected to be in the range of possible values of the alphtype. The default +is always \verb|char|. + +\begin{multicols}{2} +\setlength{\columnseprule}{1pt} +C/C++/Objective-C: +\begin{verbatim} + char unsigned char + short unsigned short + int unsigned int + long unsigned long +\end{verbatim} + +Java: +\begin{verbatim} + char + byte + short + int +\end{verbatim} + + +\columnbreak + +D: +\begin{verbatim} + char + byte ubyte + short ushort + wchar + int uint + dchar +\end{verbatim} + +Ruby: +\begin{verbatim} + char + int +\end{verbatim} +\end{multicols} \section{Getkey Statement} @@ -2415,14 +2670,14 @@ getkey fpc->id; \end{verbatim} \verbspace -Specify to Ragel how to retrieve the character that the machine operates on +This statement specifies to Ragel how to retrieve the current character from from the pointer to the current element (\verb|p|). Any expression that returns a value of the alphabet type may be used. The getkey statement may be used for looking into element structures or for translating the character to process. The getkey expression defaults to \verb|(*p)|. In goto-driven machines the getkey expression may be evaluated more than once per element processed, therefore it should not incur a -large cost and preclude optimization. +large cost nor preclude optimization. \section{Access Statement} @@ -2431,31 +2686,261 @@ access fsm->; \end{verbatim} \verbspace -The access statement allows one to tell Ragel how the generated code should +The access statement specifies how the generated code should access the machine data that is persistent across processing buffer blocks. -This includes all variables except \verb|p| and \verb|pe|. This includes -\verb|cs|, \verb|top|, \verb|stack|, \verb|tokstart|, \verb|tokend| and \verb|act|. -This is useful if a machine is to be encapsulated inside a -structure in C code. The access statement can be used to give the name of +This applies to all variables except \verb|p|, \verb|pe| and \verb|eof|. This includes +\verb|cs|, \verb|top|, \verb|stack|, \verb|ts|, \verb|te| and \verb|act|. +The access statement is useful if a machine is to be encapsulated inside a +structure in C code. It can be used to give the name of a pointer to the structure. +\section{Variable Statement} + +\begin{verbatim} +variable p fsm->p; +\end{verbatim} +\verbspace + +The variable statement specifies how to access a specific +variable. All of the variables that are declared by the user and +used by Ragel can be changed. This includes \verb|p|, \verb|pe|, \verb|eof|, \verb|cs|, +\verb|top|, \verb|stack|, \verb|ts|, \verb|te| and \verb|act|. +In Ruby and Java code generation the \verb|data| variable can also be changed. + +\section{Pre-Push Statement} +\label{prepush} + +\begin{verbatim} +prepush { + /* stack growing code */ +} +\end{verbatim} +\verbspace + +The prepush statement allows the user to supply stack management code that is +written out during the generation of fcall, immediately before the current +state is pushed to the stack. This statement can be used to test the number of +available spaces and dynamically grow the stack if necessary. + +\section{Post-Pop Statement} +\label{postpop} + +\begin{verbatim} +postpop { + /* stack shrinking code */ +} +\end{verbatim} +\verbspace + +The postpop statement allows the user to supply stack management code that is +written out during the generation of fret, immediately after the next state is +popped from the stack. This statement can be used to dynamically shrink the +stack. + +\section{Write Statement} +\label{write-statement} + +\begin{verbatim} +write <component> [options]; +\end{verbatim} +\verbspace + +The write statement is used to generate parts of the machine. +There are seven +components that can be generated by a write statement. These components make up the +state machine's data, initialization code, execution code, and export definitions. +A write statement may appear before a machine is fully defined. +This allows one to write out the data first then later define the machine where +it is used. An example of this is shown in Figure \ref{fbreak-example}. + +\subsection{Write Data} +\begin{verbatim} +write data [options]; +\end{verbatim} +\verbspace + +The write data statement causes Ragel to emit the constant static data needed +by the machine. In table-driven output styles (see Section \ref{genout}) this +is a collection of arrays that represent the states and transitions of the +machine. In goto-driven machines much less data is emitted. At the very +minimum a start state \verb|name_start| is generated. All variables written +out in machine data have both the \verb|static| and \verb|const| properties and +are prefixed with the name of the machine and an +underscore. The data can be placed inside a class, inside a function, or it can +be defined as global data. + +Two variables are written that may be used to test the state of the machine +after a buffer block has been processed. The \verb|name_error| variable gives +the id of the state that the machine moves into when it cannot find a valid +transition to take. The machine immediately breaks out of the processing loop when +it finds itself in the error state. The error variable can be compared to the +current state to determine if the machine has failed to parse the input. If the +machine is complete, that is from every state there is a transition to a proper +state on every possible character of the alphabet, then no error state is required +and this variable will be set to -1. + +The \verb|name_first_final| variable stores the id of the first final state. All of the +machine's states are sorted by their final state status before having their ids +assigned. Checking if the machine has accepted its input can then be done by +checking if the current state is greater-than or equal to the first final +state. + +Data generation has several options: + +\begin{itemize} +\setlength{\itemsep}{-2mm} +\item \verb|noerror | - Do not generate the integer variable that gives the +id of the error state. +\item \verb|nofinal | - Do not generate the integer variable that gives the +id of the first final state. +\item \verb|noprefix | - Do not prefix the variable names with the name of the +machine. +\end{itemize} + +\begin{figure} +\small +\begin{verbatim} +#include <stdio.h> +%% machine foo; +%% write data; +int main( int argc, char **argv ) +{ + int cs, res = 0; + if ( argc > 1 ) { + char *p = argv[1]; + %%{ + main := + [a-z]+ + 0 @{ res = 1; fbreak; }; + write init; + write exec noend; + }%% + } + printf("execute = %i\n", res ); + return 0; +} +\end{verbatim} +\caption{Use of {\tt noend} write option and the {\tt fbreak} statement for +processing a string.} +\label{fbreak-example} +\end{figure} + +\subsection{Write Start, First Final and Error} + +\begin{verbatim} +write start; +write first_final; +write error; +\end{verbatim} +\verbspace + +These three write statements provide an alternative means of accessing the +\verb|start|, \verb|first_final| and \verb|error| states. If there are many +different machine specifications in one file it is easy to get the prefix for +these wrong. This is especially true if the state machine boilerplate is +frequently made by a copy-paste-edit process. These write statements allow the +problem to be avoided. They can be used as follows: + +\verbspace + +{ +\small +\begin{verbatim} +/* Did parsing succeed? */ +if ( cs < %%{ write first_final; }%% ) { + result = ERR_PARSE_ERROR; + goto fail; +} +\end{verbatim} +} + + +\subsection{Write Init} +\begin{verbatim} +write init [options]; +\end{verbatim} +\verbspace + +The write init statement causes Ragel to emit initialization code. This should +be executed once before the machine is started. At a very minimum this sets the +current state to the start state. If other variables are needed by the +generated code, such as call stack variables or scanner management +variables, they are also initialized here. + +The \verb|nocs| option to the write init statement will cause ragel to skip +intialization of the cs variable. This is useful if the user wishes to use +custom logic to decide which state the specification should start in. + +\subsection{Write Exec} +\begin{verbatim} +write exec [options]; +\end{verbatim} +\verbspace + +The write exec statement causes Ragel to emit the state machine's execution code. +Ragel expects several variables to be available to this code. At a very minimum, the +generated code needs access to the current character position \verb|p|, the ending +position \verb|pe| and the current state \verb|cs| (though \verb|pe| +can be omitted using the \verb|noend| write option). +The \verb|p| variable is the cursor that the execute code will +used to traverse the input. The \verb|pe| variable should be set up to point to one +position past the last valid character in the buffer. + +Other variables are needed when certain features are used. For example using +the \verb|fcall| or \verb|fret| statements requires \verb|stack| and +\verb|top| variables to be defined. If a longest-match construction is used, +variables for managing backtracking are required. + +The write exec statement has one option. The \verb|noend| option tells Ragel +to generate code that ignores the end position \verb|pe|. In this +case the user must explicitly break out of the processing loop using +\verb|fbreak|, otherwise the machine will continue to process characters until +it moves into the error state. This option is useful if one wishes to process a +null terminated string. Rather than traverse the string to discover then length +before processing the input, the user can break out when the null character is +seen. The example in Figure \ref{fbreak-example} shows the use of the +\verb|noend| write option and the \verb|fbreak| statement for processing a string. + +\subsection{Write Exports} +\label{export} + +\begin{verbatim} +write exports; +\end{verbatim} +\verbspace + +The export feature can be used to export simple machine definitions. Machine definitions +are marked for export using the \verb|export| keyword. + +\verbspace +\begin{verbatim} +export machine_to_export = 0x44; +\end{verbatim} +\verbspace + +When the write exports statement is used these machines are +written out in the generated code. Defines are used for C and constant integers +are used for D, Java and Ruby. See Section \ref{import} for a description of the +import statement. + \section{Maintaining Pointers to Input Data} In the creation of any parser it is not uncommon to require the collection of the data being parsed. It is always possible to collect data into a growable buffer as the machine moves over it, however the copying of data is a somewhat -wasteful use of processor cycles. The most efficient way to collect data -from the parser is to set pointers into the input. This poses a problem for -uses of Ragel where the input data arrives in blocks, such as over a socket or -from a file. The program will error if a pointer is set in one buffer block but -must be used while parsing a following buffer block. - -The longest-match constructions exhibit this problem, requiring the maintenance +wasteful use of processor cycles. The most efficient way to collect data from +the parser is to set pointers into the input then later reference them. This +poses a problem for uses of Ragel where the input data arrives in blocks, such +as over a socket or from a file. If a pointer is set in one buffer block but +must be used while parsing a following buffer block, some extra consideration +to correctness must be made. + +The scanner constructions exhibit this problem, requiring the maintenance code described in Section \ref{generating-scanners}. If a longest-match construction has been used somewhere in the machine then it is possible to take advantage of the required prefix maintenance code in the driver program to ensure pointers to the input are always valid. If laying down a pointer one can -set \verb|tokstart| at the same spot or ahead of it. When data is shifted in +set \verb|ts| at the same spot or ahead of it. When data is shifted in between loops the user must also shift the pointer. In this way it is possible to maintain pointers to the input that will always be consistent. @@ -2515,24 +3000,18 @@ first found newline. On the next input read, the new data is placed after the partially read line and processing continues from the beginning of the line. An example of line-oriented processing is given in Figure \ref{line-oriented}. +\section{Specifying the Host Language} -\section{Running the Executables} - -Ragel is broken down into two executables: a frontend which compiles machines -and emits them in an XML format, and a backend which generates code or a -Graphviz Dot file from the XML data. The purpose of the XML-based intermediate -format is to allow users to inspect their compiled state machines and to -interface Ragel to other tools such as custom visualizers, code generators or -analysis tools. The intermediate format will provide a better platform for -extending Ragel to support new host languages. The split also serves to reduce -complexity of the Ragel program by strictly separating the data structures and -algorithms that are used to compile machines from those that are used to -generate code. +The \verb|ragel| program has a number of options for specifying the host +language. The host-language options are: -\verbspace -\begin{verbatim} -[user@host] myproj: ragel file.rl | rlcodegen -G2 -o file.c -\end{verbatim} +\begin{itemize} +\item \verb|-C | for C/C++/Objective-C code (default) +\item \verb|-D | for D code. +\item \verb|-J | for Java code. +\item \verb|-R | for Ruby code. +\item \verb|-A | for C\# code. +\end{itemize} \section{Choosing a Generated Code Style} \label{genout} @@ -2591,38 +3070,628 @@ preferred output format. \verbspace \begin{center} -\begin{tabular}{|c|c|} +\begin{tabular}{|c|c|c|} \hline -\multicolumn{2}{|c|}{\bf Code Output Style Options} \\ +\multicolumn{3}{|c|}{\bf Code Output Style Options} \\ \hline -\verb|-T0|&binary search table-driven\\ +\verb|-T0|&binary search table-driven&C/D/Java/Ruby/C\#\\ \hline -\verb|-T1|&binary search, expanded actions\\ +\verb|-T1|&binary search, expanded actions&C/D/Ruby/C\#\\ \hline -\verb|-F0|&flat table-driven\\ +\verb|-F0|&flat table-driven&C/D/Ruby/C\#\\ \hline -\verb|-F1|&flat table, expanded actions\\ +\verb|-F1|&flat table, expanded actions&C/D/Ruby/C\#\\ \hline -\verb|-G0|&goto-driven\\ +\verb|-G0|&goto-driven&C/D/C\#\\ \hline -\verb|-G1|&goto, expanded actions\\ +\verb|-G1|&goto, expanded actions&C/D/C\#\\ \hline -\verb|-G2|&goto, in-place actions\\ +\verb|-G2|&goto, in-place actions&C/D\\ \hline \end{tabular} \end{center} -\section{Graphviz} +\chapter{Beyond the Basic Model} + +\section{Parser Modularization} +\label{modularization} + +It is possible to use Ragel's machine construction and action embedding +operators to specify an entire parser using a single regular expression. In +many cases this is the desired way to specify a parser in Ragel. However, in +some scenarios the language to parse may be so large that it is difficult to +think about it as a single regular expression. It may also shift between distinct +parsing strategies, in which case modularization into several coherent blocks +of the language may be appropriate. + +It may also be the case that patterns that compile to a large number of states +must be used in a number of different contexts and referencing them in each +context results in a very large state machine. In this case, an ability to reuse +parsers would reduce code size. + +To address this, distinct regular expressions may be instantiated and linked +together by means of a jumping and calling mechanism. This mechanism is +analogous to the jumping to and calling of processor instructions. A jump +command, given in action code, causes control to be immediately passed to +another portion of the machine by way of setting the current state variable. A +call command causes the target state of the current transition to be pushed to +a state stack before control is transferred. Later on, the original location +may be returned to with a return statement. In the following example, distinct +state machines are used to handle the parsing of two types of headers. + +% GENERATE: call +% %%{ +% machine call; +\begin{inline_code} +\begin{verbatim} +action return { fret; } +action call_date { fcall date; } +action call_name { fcall name; } -Ragel is able to emit compiled state machines in Graphviz's Dot file format. -Graphviz support allows users to perform -incremental visualization of their parsers. User actions are displayed on -transition labels of the graph. If the final graph is too large to be -meaningful, or even drawn, the user is able to inspect portions of the parser -by naming particular regular expression definitions with the \verb|-S| and -\verb|-M| options to the \verb|ragel| program. Use of Graphviz greatly -improves the Ragel programming experience. It allows users to learn Ragel by -experimentation and also to track down bugs caused by unintended -nondeterminism. +# A parser for date strings. +date := [0-9][0-9] '/' + [0-9][0-9] '/' + [0-9][0-9][0-9][0-9] '\n' @return; + +# A parser for name strings. +name := ( [a-zA-Z]+ | ' ' )** '\n' @return; + +# The main parser. +headers = + ( 'from' | 'to' ) ':' @call_name | + ( 'departed' | 'arrived' ) ':' @call_date; + +main := headers*; +\end{verbatim} +\end{inline_code} +% }%% +% %% write data; +% void f() +% { +% %% write init; +% %% write exec; +% } +% END GENERATE + +Calling and jumping should be used carefully as they are operations that take +one out of the domain of regular languages. A machine that contains a call or +jump statement in one of its actions should be used as an argument to a machine +construction operator only with considerable care. Since DFA transitions may +actually represent several NFA transitions, a call or jump embedded in one +machine can inadvertently terminate another machine that it shares prefixes +with. Despite this danger, theses statements have proven useful for tying +together sub-parsers of a language into a parser for the full language, +especially for the purpose of modularizing code and reducing the number of +states when the machine contains frequently recurring patterns. + +Section \ref{vals} describes the jump and call statements that are used to +transfer control. These statements make use of two variables that must be +declared by the user, \verb|stack| and \verb|top|. The \verb|stack| variable +must be an array of integers and \verb|top| must be a single integer, which +will point to the next available space in \verb|stack|. Sections \ref{prepush} +and \ref{postpop} describe the Pre-Push and Post-Pop statements which can be +used to implement a dynamically resizable array. + +\section{Referencing Names} +\label{labels} + +This section describes how to reference names in epsilon transitions (Section +\ref{state-charts}) and +action-based control-flow statements such as \verb|fgoto|. There is a hierarchy +of names implied in a Ragel specification. At the top level are the machine +instantiations. Beneath the instantiations are labels and references to machine +definitions. Beneath those are more labels and references to definitions, and +so on. + +Any name reference may contain multiple components separated with the \verb|::| +compound symbol. The search for the first component of a name reference is +rooted at the join expression that the epsilon transition or action embedding +is contained in. If the name reference is not contained in a join, +the search is rooted at the machine definition that the epsilon transition or +action embedding is contained in. Each component after the first is searched +for beginning at the location in the name tree that the previous reference +component refers to. + +In the case of action-based references, if the action is embedded more than +once, the local search is performed for each embedding and the result is the +union of all the searches. If no result is found for action-based references then +the search is repeated at the root of the name tree. Any action-based name +search may be forced into a strictly global search by prefixing the name +reference with \verb|::|. + +The final component of the name reference must resolve to a unique entry point. +If a name is unique in the entire name tree it can be referenced as is. If it +is not unique it can be specified by qualifying it with names above it in the +name tree. However, it can always be renamed. + +% FIXME: Should fit this in somewhere. +% Some kinds of name references are illegal. Cannot call into longest-match +% machine, can only call its start state. Cannot make a call to anywhere from +% any part of a longest-match machine except a rule's action. This would result +% in an eventual return to some point inside a longest-match other than the +% start state. This is banned for the same reason a call into the LM machine is +% banned. + + +\section{Scanners} +\label{generating-scanners} + +Scanners are very much intertwined with regular-languages and their +corresponding processors. For this reason Ragel supports the definition of +scanners. The generated code will repeatedly attempt to match patterns from a +list, favouring longer patterns over shorter patterns. In the case of +equal-length matches, the generated code will favour patterns that appear ahead +of others. When a scanner makes a match it executes the user code associated +with the match, consumes the input then resumes scanning. + +\verbspace +\begin{verbatim} +<machine_name> := |* + pattern1 => action1; + pattern2 => action2; + ... + *|; +\end{verbatim} +\verbspace + +On the surface, Ragel scanners are similar to those defined by Lex. Though +there is a key distinguishing feature: patterns may be arbitrary Ragel +expressions and can therefore contain embedded code. With a Ragel-based scanner +the user need not wait until the end of a pattern before user code can be +executed. + +Scanners can be used to process sub-languages, as well as for tokenizing +programming languages. In the following example a scanner is used to tokenize +the contents of a header field. + +\begin{inline_code} +\begin{verbatim} +word = [a-z]+; +head_name = 'Header'; + +header := |* + word; + ' '; + '\n' => { fret; }; +*|; + +main := ( head_name ':' @{ fcall header; } )*; +\end{verbatim} +\end{inline_code} +\verbspace + +The scanner construction has a purpose similar to the longest-match kleene star +operator \verb|**|. The key +difference is that a scanner is able to backtrack to match a previously matched +shorter string when the pursuit of a longer string fails. For this reason the +scanner construction operator is not a pure state machine construction +operator. It relies on several variables that enable it to backtrack and make +pointers to the matched input text available to the user. For this reason +scanners must be immediately instantiated. They cannot be defined inline or +referenced by another expression. Scanners must be jumped to or called. + +Scanners rely on the \verb|ts|, \verb|te| and \verb|act| +variables to be present so that they can backtrack and make pointers to the +matched text available to the user. If input is processed using multiple calls +to the execute code then the user must ensure that when a token is only +partially matched that the prefix is preserved on the subsequent invocation of +the execute code. + +The \verb|ts| variable must be defined as a pointer to the input data. +It is used for recording where the current token match begins. This variable +may be used in action code for retrieving the text of the current match. Ragel +ensures that in between tokens and outside of the longest-match machines that +this pointer is set to null. In between calls to the execute code the user must +check if \verb|ts| is set and if so, ensure that the data it points to is +preserved ahead of the next buffer block. This is described in more detail +below. + +The \verb|te| variable must also be defined as a pointer to the input data. +It is used for recording where a match ends and where scanning of the next +token should begin. This can also be used in action code for retrieving the +text of the current match. + +The \verb|act| variable must be defined as an integer type. It is used for +recording the identity of the last pattern matched when the scanner must go +past a matched pattern in an attempt to make a longer match. If the longer +match fails it may need to consult the \verb|act| variable. In some cases, use +of the \verb|act| +variable can be avoided because the value of the current state is enough +information to determine which token to accept, however in other cases this is +not enough and so the \verb|act| variable is used. + +When the longest-match operator is in use, the user's driver code must take on +some buffer management functions. The following algorithm gives an overview of +the steps that should be taken to properly use the longest-match operator. + +\begin{itemize} +\setlength{\parskip}{0pt} +\item Read a block of input data. +\item Run the execute code. +\item If \verb|ts| is set, the execute code will expect the incomplete +token to be preserved ahead of the buffer on the next invocation of the execute +code. +\begin{itemize} +\item Shift the data beginning at \verb|ts| and ending at \verb|pe| to the +beginning of the input buffer. +\item Reset \verb|ts| to the beginning of the buffer. +\item Shift \verb|te| by the distance from the old value of \verb|ts| +to the new value. The \verb|te| variable may or may not be valid. There is +no way to know if it holds a meaningful value because it is not kept at null +when it is not in use. It can be shifted regardless. +\end{itemize} +\item Read another block of data into the buffer, immediately following any +preserved data. +\item Run the scanner on the new data. +\end{itemize} + +Figure \ref{preserve_example} shows the required handling of an input stream in +which a token is broken by the input block boundaries. After processing up to +and including the ``t'' of ``characters'', the prefix of the string token must be +retained and processing should resume at the ``e'' on the next iteration of +the execute code. + +If one uses a large input buffer for collecting input then the number of times +the shifting must be done will be small. Furthermore, if one takes care not to +define tokens that are allowed to be very long and instead processes these +items using pure state machines or sub-scanners, then only a small amount of +data will ever need to be shifted. + +\begin{figure} +\begin{verbatim} + a) A stream "of characters" to be scanned. + | | | + p ts pe + + b) "of characters" to be scanned. + | | | + ts p pe +\end{verbatim} +\caption{Following an invocation of the execute code there may be a partially +matched token (a). The data of the partially matched token +must be preserved ahead of the new data on the next invocation (b).} +\label{preserve_example} +\end{figure} + +Since scanners attempt to make the longest possible match of input, patterns +such as identifiers require one character of lookahead in order to trigger a +match. In the case of the last token in the input stream the user must ensure +that the \verb|eof| variable is set so that the final token is flushed out. + +An example scanner processing loop is given in Figure \ref{scanner-loop}. + +\begin{figure} +\small +\begin{verbatim} + int have = 0; + bool done = false; + while ( !done ) { + /* How much space is in the buffer? */ + int space = BUFSIZE - have; + if ( space == 0 ) { + /* Buffer is full. */ + cerr << "TOKEN TOO BIG" << endl; + exit(1); + } + + /* Read in a block after any data we already have. */ + char *p = inbuf + have; + cin.read( p, space ); + int len = cin.gcount(); + + char *pe = p + len; + char *eof = 0; + + /* If no data was read indicate EOF. */ + if ( len == 0 ) { + eof = pe; + done = true; + } + + %% write exec; + + if ( cs == Scanner_error ) { + /* Machine failed before finding a token. */ + cerr << "PARSE ERROR" << endl; + exit(1); + } + + if ( ts == 0 ) + have = 0; + else { + /* There is a prefix to preserve, shift it over. */ + have = pe - ts; + memmove( inbuf, ts, have ); + te = inbuf + (te-ts); + ts = inbuf; + } + } +\end{verbatim} +\caption{A processing loop for a scanner.} +\label{scanner-loop} +\end{figure} + +\section{State Charts} +\label{state-charts} + +In addition to supporting the construction of state machines using regular +languages, Ragel provides a way to manually specify state machines using +state charts. The comma operator combines machines together without any +implied transitions. The user can then manually link machines by specifying +epsilon transitions with the \verb|->| operator. Epsilon transitions are drawn +between the final states of a machine and entry points defined by labels. This +makes it possible to build machines using the explicit state-chart method while +making minimal changes to the Ragel language. + +An interesting feature of Ragel's state chart construction method is that it +can be mixed freely with regular expression constructions. A state chart may be +referenced from within a regular expression, or a regular expression may be +used in the definition of a state chart transition. + +\subsection{Join} + +\verb|expr , expr , ...| +\verbspace + +Join a list of machines together without +drawing any transitions, without setting up a start state, and without +designating any final states. Transitions between the machines may be specified +using labels and epsilon transitions. The start state must be explicity +specified with the ``start'' label. Final states may be specified with an +epsilon transition to the implicitly created ``final'' state. The join +operation allows one to build machines using a state chart model. + +\subsection{Label} + +\verb|label: expr| +\verbspace + +Attaches a label to an expression. Labels can be +used as the target of epsilon transitions and explicit control transfer +statements such as \verb|fgoto| and \verb|fnext| in action +code. + +\subsection{Epsilon} + +\verb|expr -> label| +\verbspace + +Draws an epsilon transition to the state defined +by \verb|label|. Epsilon transitions are made deterministic when join +operators are evaluated. Epsilon transitions that are not in a join operation +are made deterministic when the machine definition that contains the epsilon is +complete. See Section \ref{labels} for information on referencing labels. + +\subsection{Simplifying State Charts} + +There are two benefits to providing state charts in Ragel. The first is that it +allows us to take a state chart with a full listing of states and transitions +and simplify it in selective places using regular expressions. + +The state chart method of specifying parsers is very common. It is an +effective programming technique for producing robust code. The key disadvantage +becomes clear when one attempts to comprehend a large parser specified in this +way. These programs usually require many lines, causing logic to be spread out +over large distances in the source file. Remembering the function of a large +number of states can be difficult and organizing the parser in a sensible way +requires discipline because branches and repetition present many file layout +options. This kind of programming takes a specification with inherent +structure such as looping, alternation and concatenation and expresses it in a +flat form. + +If we could take an isolated component of a manually programmed state chart, +that is, a subset of states that has only one entry point, and implement it +using regular language operators then we could eliminate all the explicit +naming of the states contained in it. By eliminating explicitly named states +and replacing them with higher-level specifications we simplify a state machine +specification. + +For example, sometimes chains of states are needed, with only a small number of +possible characters appearing along the chain. These can easily be replaced +with a concatenation of characters. Sometimes a group of common states +implement a loop back to another single portion of the machine. Rather than +manually duplicate all the transitions that loop back, we may be able to +express the loop using a kleene star operator. + +Ragel allows one to take this state map simplification approach. We can build +state machines using a state map model and implement portions of the state map +using regular languages. In place of any transition in the state machine, +entire sub-machines can be given. These can encapsulate functionality +defined elsewhere. An important aspect of the Ragel approach is that when we +wrap up a collection of states using a regular expression we do not lose +access to the states and transitions. We can still execute code on the +transitions that we have encapsulated. + +\subsection{Dropping Down One Level of Abstraction} +\label{down} + +The second benefit of incorporating state charts into Ragel is that it permits +us to bypass the regular language abstraction if we need to. Ragel's action +embedding operators are sometimes insufficient for expressing certain parsing +tasks. In the same way that is useful for C language programmers to drop down +to assembly language programming using embedded assembler, it is sometimes +useful for the Ragel programmer to drop down to programming with state charts. + +In the following example, we wish to buffer the characters of an XML CDATA +sequence. The sequence is terminated by the string \verb|]]>|. The challenge +in our application is that we do not wish the terminating characters to be +buffered. An expression of the form \verb|any* @buffer :>> ']]>'| will not work +because the buffer will always contain the characters \verb|]]| on the end. +Instead, what we need is to delay the buffering of \hspace{0.25mm} \verb|]| +characters until a time when we +abandon the terminating sequence and go back into the main loop. There is no +easy way to express this using Ragel's regular expression and action embedding +operators, and so an ability to drop down to the state chart method is useful. + +% GENERATE: dropdown +% OPT: -p +% %%{ +% machine dropdown; +\begin{inline_code} +\begin{verbatim} +action bchar { buff( fpc ); } # Buffer the current character. +action bbrack1 { buff( "]" ); } +action bbrack2 { buff( "]]" ); } + +CDATA_body = +start: ( + ']' -> one | + (any-']') @bchar ->start +), +one: ( + ']' -> two | + [^\]] @bbrack1 @bchar ->start +), +two: ( + '>' -> final | + ']' @bbrack1 -> two | + [^>\]] @bbrack2 @bchar ->start +); +\end{verbatim} +\end{inline_code} +% main := CDATA_body; +% }%% +% END GENERATE + +\graphspace +\begin{center} +\includegraphics[scale=0.55]{dropdown} +\end{center} + + +\section{Semantic Conditions} +\label{semantic} + +Many communication protocols contain variable-length fields, where the length +of the field is given ahead of the field as a value. This +problem cannot be expressed using regular languages because of its +context-dependent nature. The prevalence of variable-length fields in +communication protocols motivated us to introduce semantic conditions into +the Ragel language. + +A semantic condition is a block of user code that is executed immediately +before a transition is taken. If the code returns a value of true, the +transition may be taken. We can now embed code that extracts the length of a +field, then proceed to match $n$ data values. + +% GENERATE: conds1 +% OPT: -p +% %%{ +% machine conds1; +% number = digit+; +\begin{inline_code} +\begin{verbatim} +action rec_num { i = 0; n = getnumber(); } +action test_len { i++ < n } +data_fields = ( + 'd' + [0-9]+ %rec_num + ':' + ( [a-z] when test_len )* +)**; +\end{verbatim} +\end{inline_code} +% main := data_fields; +% }%% +% END GENERATE + +\begin{center} +\includegraphics[scale=0.55]{conds1} +\end{center} +\graphspace + +The Ragel implementation of semantic conditions does not force us to give up the +compositional property of Ragel definitions. For example, a machine that tests +the length of a field using conditions can be unioned with another machine +that accepts some of the same strings, without the two machines interfering with +one another. The user need not be concerned about whether or not the result of the +semantic condition will affect the matching of the second machine. + +To see this, first consider that when a user associates a condition with an +existing transition, the transition's label is translated from the base character +to its corresponding value in the space that represents ``condition $c$ true''. Should +the determinization process combine a state that has a conditional transition +with another state that has a transition on the same input character but +without a condition, then the condition-less transition first has its label +translated into two values, one to its corresponding value in the space that +represents ``condition $c$ true'' and another to its corresponding value in the +space that represents ``condition $c$ false''. It +is then safe to combine the two transitions. This is shown in the following +example. Two intersecting patterns are unioned, one with a condition and one +without. The condition embedded in the first pattern does not affect the second +pattern. + +% GENERATE: conds2 +% OPT: -p +% %%{ +% machine conds2; +% number = digit+; +\begin{inline_code} +\begin{verbatim} +action test_len { i++ < n } +action one { /* accept pattern one */ } +action two { /* accept pattern two */ } +patterns = + ( [a-z] when test_len )+ %one | + [a-z][a-z0-9]* %two; +main := patterns '\n'; +\end{verbatim} +\end{inline_code} +% }%% +% END GENERATE + +\begin{center} +\includegraphics[scale=0.55]{conds2} +\end{center} +\graphspace + +There are many more potential uses for semantic conditions. The user is free to +use arbitrary code and may therefore perform actions such as looking up names +in dictionaries, validating input using external parsing mechanisms or +performing checks on the semantic structure of input seen so far. In the +next section we describe how Ragel accommodates several common parser +engineering problems. + +\vspace{10pt} + +\noindent {\large\bf Note:} The semantic condition feature works only with +alphabet types that are smaller in width than the \verb|long| type. To +implement semantic conditions Ragel needs to be able to allocate characters +from the alphabet space. Ragel uses these allocated characters to express +"character C with condition P true" or "C with P false." Since internally Ragel +uses longs to store characters there is no room left in the alphabet space +unless an alphabet type smaller than long is used. + +\section{Implementing Lookahead} + +There are a few strategies for implementing lookahead in Ragel programs. +Leaving actions, which are described in Section \ref{out-actions}, can be +used as a form of lookahead. Ragel also provides the \verb|fhold| directive +which can be used in actions to prevent the machine from advancing over the +current character. It is also possible to manually adjust the current character +position by shifting it backwards using \verb|fexec|, however when this is +done, care must be taken not to overstep the beginning of the current buffer +block. In both the use of \verb|fhold| and \verb|fexec| the user must be +cautious of combining the resulting machine with another in such a way that the +transition on which the current position is adjusted is not combined with a +transition from the other machine. + +\section{Parsing Recursive Language Structures} + +In general Ragel cannot handle recursive structures because the grammar is +interpreted as a regular language. However, depending on what needs to be +parsed it is sometimes practical to implement the recursive parts using manual +coding techniques. This often works in cases where the recursive structures are +simple and easy to recognize, such as in the balancing of parentheses + +One approach to parsing recursive structures is to use actions that increment +and decrement counters or otherwise recognize the entry to and exit from +recursive structures and then jump to the appropriate machine defnition using +\verb|fcall| and \verb|fret|. Alternatively, semantic conditions can be used to +test counter variables. + +A more traditional approach is to call a separate parsing function (expressed +in the host language) when a recursive structure is entered, then later return +when the end is recognized. \end{document} diff --git a/doc/ragel.1.in b/doc/ragel.1.in index cdae3e9..cfcc0f5 100644 --- a/doc/ragel.1.in +++ b/doc/ragel.1.in @@ -1,5 +1,5 @@ .\" -.\" Copyright 2001-2005 Adrian Thurston <thurston@cs.queensu.ca> +.\" Copyright 2001-2007 Adrian Thurston <thurston@complang.org> .\" .\" This file is part of Ragel. @@ -29,12 +29,8 @@ ragel \- compile regular languages into executable state machines .RI [ options ] .I file .SH DESCRIPTION -.B Note: -this is the frontend component of Ragel, which generates an intermediate -file format that must be processed by rlcodegen(1). - -Ragel compiles finite state machines from regular languages into executable -code. Ragel can generate C, C++, Objective-C, D, or Java code. Ragel state +Ragel compiles executable finite state machines from regular languages. +Ragel can generate C, C++, Objective-C, D, or Java code. Ragel state machines can not only recognize byte sequences as regular expression machines do, but can also execute code at arbitrary points in the recognition of a regular language. User code is @@ -59,11 +55,30 @@ into the application. The generated code has no dependencies. .BR \-h ", " \-H ", " \-? ", " \-\-help Display help and exit. .TP +.B \-v +Print version information and exit. +.TP .B \-o " file" Write output to file. If -o is not given, a default file name is chosen by -replacing the suffix of the input. For source files ending in .rh the suffix .h -is used. For all other source files a suffix based on the output language -is used (.c, .cpp, .m, .dot) +replacing the file extenstion of the input file. For source files ending in .rh +the suffix .h is used. For all other source files a suffix based on the output +language is used (.c, .cpp, .m, etc.). If -o is not given for Graphviz output +the generated dot file is written to standard output. +.TP +.B \-s +Print some statistics on standard error. +.TP +.B \--error-format=gnu +Print error messages using the format "file:line:column:" (default) +.TP +.B \--error-format=msvc +Print error messages using the format "file(line,column):" +.TP +.B \-d +Do not remove duplicate actions from action lists. +.TP +.B \-I " dir" +Add dir to the list of directories to search for included and imported files .TP .B \-n Do not perform state minimization. @@ -78,11 +93,21 @@ are minimized once at the end. This is the default minimization option. .B \-e Minimize after every operation. .TP +.B \-x +Compile the state machines and emit an XML representation of the host data and +the machines. +.TP +.B \-V +Generate a dot file for Graphviz. +.TP +.B \-p +Display printable characters on labels. +.TP .B \-S <spec> -FSM specification to output for -V +FSM specification to output. .TP .B \-M <machine> -Machine definition/instantiation to output for -V +Machine definition/instantiation to output. .TP .B \-C The host language is C, C++, Obj-C or Obj-C++. This is the default host language option. @@ -92,6 +117,58 @@ The host language is D. .TP .B \-J The host language is Java. +.TP +.B \-R +The host language is Ruby. +.TP +.B \-L +Inhibit writing of #line directives. +.TP +.B \-T0 +(C/D/Java/Ruby/C#) Generate a table driven FSM. This is the default code style. +The table driven +FSM represents the state machine as static data. There are tables of states, +transitions, indicies and actions. The current state is stored in a variable. +The execution is a loop that looks that given the current state and current +character to process looks up the transition to take using a binary search, +executes any actions and moves to the target state. In general, the table +driven FSM produces a smaller binary and requires a less expensive host language +compile but results in slower running code. The table driven FSM is suitable +for any FSM. +.TP +.B \-T1 +(C/D/Ruby/C#) Generate a faster table driven FSM by expanding action lists in the action +execute code. +.TP +.B \-F0 +(C/D/Ruby/C#) Generate a flat table driven FSM. Transitions are represented as an array +indexed by the current alphabet character. This eliminates the need for a +binary search to locate transitions and produces faster code, however it is +only suitable for small alphabets. +.TP +.B \-F1 +(C/D/Ruby/C#) Generate a faster flat table driven FSM by expanding action lists in the action +execute code. +.TP +.B \-G0 +(C/D/C#) Generate a goto driven FSM. The goto driven FSM represents the state machine +as a series of goto statements. While in the machine, the current state is +stored by the processor's instruction pointer. The execution is a flat function +where control is passed from state to state using gotos. In general, the goto +FSM produces faster code but results in a larger binary and a more expensive +host language compile. +.TP +.B \-G1 +(C/D/C#) Generate a faster goto driven FSM by expanding action lists in the action +execute code. +.TP +.B \-G2 +(C/D) Generate a really fast goto driven FSM by embedding action lists in the state +machine control code. +.TP +.B \-P<N> +(C/D) N-Way Split really fast goto-driven FSM. + .SH RAGEL INPUT NOTE: This is a very brief description of Ragel input. Ragel is described in more detail in the user guide available from the homepage (see below). @@ -128,6 +205,9 @@ Specify how to access the persistent state machine variables. .TP .I Write: Write some component of the machine. +.TP +.I Variable: +Override the default variable names (p, pe, cs, act, etc). .SH BASIC MACHINES The basic machines are the base operands of the regular language expressions. .TP @@ -248,14 +328,27 @@ Produces a machine that matches any string that is in both machine one and machine two. .TP .I expr - expr -Produces a machine that matches string that is in machine one but not in +Produces a machine that matches any string that is in machine one but not in machine two. +.TP +.I expr -- expr +Strong Subtraction. Matches any string in machine one that does not have any string +in machine two as a substring. .LP .B GROUP 3: .TP .I expr . expr Produces a machine that matches all the strings in machine one followed by all the strings in machine two. +.TP +.I expr :> expr +Entry-Guarded Concatenation: terminates machine one upon entry to machine two. +.TP +.I expr :>> expr +Finish-Guarded Concatenation: terminates machine one when machine two finishes. +.TP +.I expr <: expr +Left-Guarded Concatenation: gives a higher priority to machine one. .LP NOTE: Concatenation is the default operator. Two machines next to each other with no operator between them results in the concatenation operation. @@ -426,7 +519,7 @@ defaults to the name of the machine definition the priority is assigned in. Transitions do not have default priorities. .TP .I expr > int -Assigns the priority int in all transitions entering into the machine. +Assigns the priority int in all transitions leaving the start state. .TP .I expr @ int Assigns the priority int in all transitions that go into a final state. @@ -442,7 +535,7 @@ to which the priority is assigned, allowing interactions to cross machine definition boundaries. .TP .I expr > (name,int) -Assigns the priority int to name in all transitions entering into the machine. +Assigns the priority int to name in all transitions leaving the start state. .TP .I expr @ (name, int) Assigns the priority int to name in all transitions that go into a final state. @@ -460,7 +553,7 @@ Produces the kleene star of a machine. Matches zero or more repetitions of the machine. .TP .I expr ** -Longest Matching Kleene Star. This version of kleene star puts a higher +Longest-Match Kleene Star. This version of kleene star puts a higher priority on staying in the machine over wrapping around and starting over. This operator is equivalent to ( ( expr ) $0 %1 )*. .TP @@ -489,6 +582,10 @@ Produces a machine that matches n to m repetitions of expr. .I ! expr Produces a machine that matches any string not matched by the given machine. This operator is equivalent to ( *extend - expr ). +.TP +.I ^ expr +Character-Level Negation. Matches any single character not matched by the +single character machine expr. .LP .B GROUP 9: .TP @@ -547,15 +644,13 @@ Return to the target state of the transition on which the last fcall was made. .TP .I fbreak; Save the current state and immediately break out of the machine. -.SH BUGS -Ragel is still under development and has not yet matured. There are probably -many bugs. .SH CREDITS -Ragel was written by Adrian Thurston <thurston@cs.queensu.ca>. Objective-C -output contributed by Eric Ocean. D output contributed by Alan West. +Ragel was written by Adrian Thurston <thurston@complang.org>. Objective-C +output contributed by Erich Ocean. D output contributed by Alan West. Ruby +output contributed by Victor Hugo Borja. C Sharp code generation contributed by +Daniel Tang. Contributions to Java code generation by Colin Fleming. .SH "SEE ALSO" -.BR rlcodegen (1), .BR re2c (1), .BR flex (1) -Homepage: http://www.cs.queensu.ca/home/thurston/ragel/ +Homepage: http://www.complang.org/ragel/ diff --git a/doc/smallscanner.fig b/doc/smallscanner.fig new file mode 100644 index 0000000..c86750e --- /dev/null +++ b/doc/smallscanner.fig @@ -0,0 +1,78 @@ +#FIG 3.2 Produced by xfig version 3.2.5-alpha5 +Portrait +Center +Metric +A4 +100.00 +Single +-2 +# Generated by dot version 2.2.1 (Fri Sep 30 13:22:44 UTC 2005) +# For: (age) Adrian Thurston +# Title: smallscanner +# Pages: 1 +1200 2 +0 32 #d2d2d2 +# ENTRY +1 1 0 1 0 0 0 0 20 0.000 0 0.0000 33 3850 33 33 33 3850 66 3883 +# 0 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 3850 383 383 1400 3850 1783 4233 +1 1 0 1 0 32 0 0 -1 0.000 0 0.0000 1400 3850 450 450 1400 3850 1850 4300 +# 1 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5266 3850 383 383 5266 3850 5649 4233 +1 1 0 1 0 0 0 0 -1 0.000 0 0.0000 5266 3850 450 450 5266 3850 5716 4300 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 766 3783 933 3850 766 3900 766 3783 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1116 3350 1083 3516 1016 3350 1116 3350 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4633 3783 4800 3850 4633 3883 4633 3783 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 1950 4183 1816 4066 2000 4083 1950 4183 +2 3 0 1 0 0 0 0 20 0.000 0 0 0 0 0 4 + 4983 3350 4950 3516 4883 3350 4983 3350 +2 2 0 0 0 7 50 -1 -1 0.000 0 0 -1 0 0 5 + 7110 2835 -90 2835 -90 4455 7110 4455 7110 2835 +# ENTRY -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 7 + 66 3850 132 3850 225 3850 341 3850 474 3850 617 3850 + 766 3850 + 0.000 1.000 1.000 1.000 1.000 1.000 0.000 +# 0 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 1716 3516 1741 3410 1734 3312 1695 3227 1626 3159 1527 3115 + 1400 3100 1305 3108 1224 3131 1158 3168 1108 3218 1077 3279 + 1066 3350 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +# 0 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 1866 3850 1916 3842 1966 3838 2016 3835 2066 3833 2116 3833 + 2166 3833 2632 3833 3008 3833 3339 3833 3669 3833 4041 3833 + 4500 3833 4523 3833 4545 3833 4566 3833 4587 3833 4609 3833 + 4633 3833 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 1 -> 0 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 19 + 4866 4066 4807 4091 4745 4115 4683 4137 4620 4156 4559 4172 + 4500 4183 4050 4266 3676 4316 3339 4333 3001 4316 2623 4266 + 2166 4183 2133 4175 2100 4166 2066 4158 2033 4150 2000 4141 + 1966 4133 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 0.000 +# 1 -> 1 +3 4 0 1 0 0 0 0 -1 0.000 0 0 0 13 + 5583 3516 5608 3410 5601 3312 5562 3227 5493 3159 5394 3115 + 5266 3100 5172 3108 5091 3131 5025 3168 4975 3218 4944 3279 + 4933 3350 + 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 + 1.000 1.000 1.000 1.000 0.000 +4 1 0 0 0 0 14 0.0000 2 150 120 1400 3933 0\001 +4 1 0 0 0 0 14 0.0000 2 150 255 500 3800 IN\001 +4 1 0 0 0 0 14 0.0000 2 150 150 1400 3050 ' '\001 +4 1 0 0 0 0 14 0.0000 2 165 120 5266 3933 1\001 +4 1 0 0 0 0 14 0.0000 2 180 2310 3333 3783 'a'..'z' / start_str, on_char\001 +4 1 0 0 0 0 14 0.0000 2 180 1200 3333 4116 ' ' / finish_str\001 +4 1 0 0 0 0 14 0.0000 2 180 3300 5266 3050 'a'..'z' / on_char, finish_str, start_str\001 diff --git a/examples/Makefile.am b/examples/Makefile.am new file mode 100644 index 0000000..b4c6a94 --- /dev/null +++ b/examples/Makefile.am @@ -0,0 +1,91 @@ +# +# Copyright 2002-2009 Adrian Thurston <thurston@complang.org> +# + +# This file is part of Ragel. +# +# Ragel is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# Ragel is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with Ragel; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + +RAGEL = ../ragel/ragel +FLEX = flex +RE2C = re2c + +noinst_PROGRAMS = \ + atoi awkemu clang concurrent cppscan format gotocallret mailbox params \ + pullscan rlscan statechart + +EXTRA_DIST = \ + gotocallret.rl pullscan.rl concurrent.rl rlscan.rl statechart.rl \ + params.rl clang.rl cppscan.rl format.rl awkemu.rl mailbox.rl atoi.rl + +gotocallret_SOURCES = gotocallret.cpp +pullscan_SOURCES = pullscan.c +concurrent_SOURCES = concurrent.cpp +rlscan_SOURCES = rlscan.cpp +statechart_SOURCES = statechart.cpp +params_SOURCES = params.c +clang_SOURCES = clang.c +cppscan_SOURCES = cppscan.cpp +format_SOURCES = format.c +awkemu_SOURCES = awkemu.c +mailbox_SOURCES = mailbox.cpp +atoi_SOURCES = atoi.cpp + +gotocallret.cpp: gotocallret.rl + $(RAGEL) -G2 -o gotocallret.cpp gotocallret.rl + +pullscan.c: pullscan.rl $(RAGEL) + $(RAGEL) -G2 -o $@ pullscan.rl + +concurrent.cpp: concurrent.rl $(RAGEL) + $(RAGEL) -G2 -o concurrent.cpp concurrent.rl + +rlscan.cpp: rlscan.rl + $(RAGEL) -G2 -o rlscan.cpp rlscan.rl + +statechart.cpp: statechart.rl + $(RAGEL) -G2 -o statechart.cpp statechart.rl + +params.c: params.rl + $(RAGEL) -G2 -o params.c params.rl + +clang.c: clang.rl + $(RAGEL) -G2 -o clang.c clang.rl + +cppscan.cpp: cppscan.rl + $(RAGEL) -G2 -o $@ cppscan.rl + +format.c: format.rl + $(RAGEL) -G2 -o format.c format.rl + +awkemu.c: awkemu.rl + $(RAGEL) -G2 -o awkemu.c awkemu.rl + +mailbox.cpp: mailbox.rl + $(RAGEL) -G2 -o mailbox.cpp mailbox.rl + +atoi.cpp: atoi.rl + $(RAGEL) -G2 -o atoi.cpp atoi.rl + +### + +lex-cppscan.cpp: cppscan.lex + $(FLEX) -f -o $@ $< + +re2c-cppscan.cpp: cppscan.rec + $(RE2C) -s $< > $@ + +example.cpp: example.rec + $(RE2C) -s $< > $@ diff --git a/examples/atoi.rl b/examples/atoi.rl new file mode 100644 index 0000000..7164b68 --- /dev/null +++ b/examples/atoi.rl @@ -0,0 +1,59 @@ +/* + * Convert a string to an integer. + */ + +#include <stdlib.h> +#include <string.h> +#include <stdio.h> + +%%{ + machine atoi; + write data; +}%% + +long long atoi( char *str ) +{ + char *p = str, *pe = str + strlen( str ); + int cs; + long long val = 0; + bool neg = false; + + %%{ + action see_neg { + neg = true; + } + + action add_digit { + val = val * 10 + (fc - '0'); + } + + main := + ( '-'@see_neg | '+' )? ( digit @add_digit )+ + '\n'; + + # Initialize and execute. + write init; + write exec; + }%% + + if ( neg ) + val = -1 * val; + + if ( cs < atoi_first_final ) + fprintf( stderr, "atoi: there was an error\n" ); + + return val; +}; + + +#define BUFSIZE 1024 + +int main() +{ + char buf[BUFSIZE]; + while ( fgets( buf, sizeof(buf), stdin ) != 0 ) { + long long value = atoi( buf ); + printf( "%lld\n", value ); + } + return 0; +} diff --git a/examples/awkemu.rl b/examples/awkemu.rl new file mode 100644 index 0000000..6615943 --- /dev/null +++ b/examples/awkemu.rl @@ -0,0 +1,116 @@ +/* + * Perform the basic line parsing of input performed by awk. + */ + +#include <stdio.h> +#include <string.h> +#include <stdlib.h> +#include <unistd.h> + +%%{ + machine awkemu; + + action start_word { + ws[nwords] = fpc; + } + + action end_word { + we[nwords++] = fpc; + } + + action start_line { + nwords = 0; + ls = fpc; + } + + action end_line { + printf("endline(%i): ", nwords ); + fwrite( ls, 1, p - ls, stdout ); + printf("\n"); + + for ( i = 0; i < nwords; i++ ) { + printf(" word: "); + fwrite( ws[i], 1, we[i] - ws[i], stdout ); + printf("\n"); + } + } + + # Words in a line. + word = ^[ \t\n]+; + + # The whitespace separating words in a line. + whitespace = [ \t]; + + # The components in a line to break up. Either a word or a single char of + # whitespace. On the word capture characters. + blineElements = word >start_word %end_word | whitespace; + + # Star the break line elements. Just be careful to decrement the leaving + # priority as we don't want multiple character identifiers to be treated as + # multiple single char identifiers. + line = ( blineElements** '\n' ) >start_line @end_line; + + # Any number of lines. + main := line*; +}%% + +%% write data noerror nofinal; + +#define MAXWORDS 256 +#define BUFSIZE 4096 +char buf[BUFSIZE]; + +int main() +{ + int i, nwords = 0; + char *ls = 0; + char *ws[MAXWORDS]; + char *we[MAXWORDS]; + + int cs; + int have = 0; + + %% write init; + + while ( 1 ) { + char *p, *pe, *data = buf + have; + int len, space = BUFSIZE - have; + /* fprintf( stderr, "space: %i\n", space ); */ + + if ( space == 0 ) { + fprintf(stderr, "buffer out of space\n"); + exit(1); + } + + len = fread( data, 1, space, stdin ); + /* fprintf( stderr, "len: %i\n", len ); */ + if ( len == 0 ) + break; + + /* Find the last newline by searching backwards. This is where + * we will stop processing on this iteration. */ + p = buf; + pe = buf + have + len - 1; + while ( *pe != '\n' && pe >= buf ) + pe--; + pe += 1; + + /* fprintf( stderr, "running on: %i\n", pe - p ); */ + + %% write exec; + + /* How much is still in the buffer. */ + have = data + len - pe; + if ( have > 0 ) + memmove( buf, pe, have ); + + /* fprintf(stderr, "have: %i\n", have ); */ + + if ( len < space ) + break; + } + + if ( have > 0 ) + fprintf(stderr, "input not newline terminated\n"); + return 0; +} diff --git a/examples/awkequiv.awk b/examples/awkequiv.awk new file mode 100755 index 0000000..9877dd3 --- /dev/null +++ b/examples/awkequiv.awk @@ -0,0 +1,10 @@ +#!/usr/bin/awk -f +# + + +{ + print "endline(" NF "): " $0 + for ( i = 1; i <= NF; i++ ) { + print " word: " $i + } +} diff --git a/examples/clang.rl b/examples/clang.rl new file mode 100644 index 0000000..60491e5 --- /dev/null +++ b/examples/clang.rl @@ -0,0 +1,150 @@ +/* + * A mini C-like language scanner. + */ + +#include <stdio.h> +#include <string.h> +#include <stdlib.h> + +%%{ + machine clang; + + newline = '\n' @{curline += 1;}; + any_count_line = any | newline; + + # Consume a C comment. + c_comment := any_count_line* :>> '*/' @{fgoto main;}; + + main := |* + + # Alpha numberic characters or underscore. + alnum_u = alnum | '_'; + + # Alpha charactres or underscore. + alpha_u = alpha | '_'; + + # Symbols. Upon entering clear the buffer. On all transitions + # buffer a character. Upon leaving dump the symbol. + ( punct - [_'"] ) { + printf( "symbol(%i): %c\n", curline, ts[0] ); + }; + + # Identifier. Upon entering clear the buffer. On all transitions + # buffer a character. Upon leaving, dump the identifier. + alpha_u alnum_u* { + printf( "ident(%i): ", curline ); + fwrite( ts, 1, te-ts, stdout ); + printf("\n"); + }; + + # Single Quote. + sliteralChar = [^'\\] | newline | ( '\\' . any_count_line ); + '\'' . sliteralChar* . '\'' { + printf( "single_lit(%i): ", curline ); + fwrite( ts, 1, te-ts, stdout ); + printf("\n"); + }; + + # Double Quote. + dliteralChar = [^"\\] | newline | ( '\\' any_count_line ); + '"' . dliteralChar* . '"' { + printf( "double_lit(%i): ", curline ); + fwrite( ts, 1, te-ts, stdout ); + printf("\n"); + }; + + # Whitespace is standard ws, newlines and control codes. + any_count_line - 0x21..0x7e; + + # Describe both c style comments and c++ style comments. The + # priority bump on tne terminator of the comments brings us + # out of the extend* which matches everything. + '//' [^\n]* newline; + + '/*' { fgoto c_comment; }; + + # Match an integer. We don't bother clearing the buf or filling it. + # The float machine overlaps with int and it will do it. + digit+ { + printf( "int(%i): ", curline ); + fwrite( ts, 1, te-ts, stdout ); + printf("\n"); + }; + + # Match a float. Upon entering the machine clear the buf, buffer + # characters on every trans and dump the float upon leaving. + digit+ '.' digit+ { + printf( "float(%i): ", curline ); + fwrite( ts, 1, te-ts, stdout ); + printf("\n"); + }; + + # Match a hex. Upon entering the hex part, clear the buf, buffer characters + # on every trans and dump the hex on leaving transitions. + '0x' xdigit+ { + printf( "hex(%i): ", curline ); + fwrite( ts, 1, te-ts, stdout ); + printf("\n"); + }; + + *|; +}%% + +%% write data nofinal; + +#define BUFSIZE 128 + +void scanner() +{ + static char buf[BUFSIZE]; + int cs, act, have = 0, curline = 1; + char *ts, *te = 0; + int done = 0; + + %% write init; + + while ( !done ) { + char *p = buf + have, *pe, *eof = 0; + int len, space = BUFSIZE - have; + + if ( space == 0 ) { + /* We've used up the entire buffer storing an already-parsed token + * prefix that must be preserved. */ + fprintf(stderr, "OUT OF BUFFER SPACE\n" ); + exit(1); + } + + len = fread( p, 1, space, stdin ); + pe = p + len; + + /* Check if this is the end of file. */ + if ( len < space ) { + eof = pe; + done = 1; + } + + %% write exec; + + if ( cs == clang_error ) { + fprintf(stderr, "PARSE ERROR\n" ); + break; + } + + if ( ts == 0 ) + have = 0; + else { + /* There is a prefix to preserve, shift it over. */ + have = pe - ts; + memmove( buf, ts, have ); + te = buf + (te-ts); + ts = buf; + } + } +} + +int main() +{ + scanner(); + return 0; +} + diff --git a/examples/concurrent.rl b/examples/concurrent.rl new file mode 100644 index 0000000..224f960 --- /dev/null +++ b/examples/concurrent.rl @@ -0,0 +1,126 @@ +/* + * Show off concurrent abilities. + */ + +#include <iostream> +#include <stdlib.h> +#include <stdio.h> + +using namespace std; + +#define BUFSIZE 2048 + +struct Concurrent +{ + int cur_char; + int start_word; + int start_comment; + int start_literal; + + int cs; + + int init( ); + int execute( const char *data, int len, bool isEof ); + int finish( ); +}; + +%%{ + machine Concurrent; + + action next_char { + cur_char += 1; + } + + action start_word { + start_word = cur_char; + } + action end_word { + cout << "word: " << start_word << + " " << cur_char-1 << endl; + } + + action start_comment { + start_comment = cur_char; + } + action end_comment { + cout << "comment: " << start_comment << + " " << cur_char-1 << endl; + } + + action start_literal { + start_literal = cur_char; + } + action end_literal { + cout << "literal: " << start_literal << + " " << cur_char-1 << endl; + } + + # Count characters. + chars = ( any @next_char )*; + + # Words are non-whitespace. + word = ( any-space )+ >start_word %end_word; + words = ( ( word | space ) $1 %0 )*; + + # Finds C style comments. + comment = ( '/*' any* :>> '*/' ) >start_comment %end_comment; + comments = ( comment | any )**; + + # Finds single quoted strings. + literalChar = ( any - ['\\] ) | ( '\\' . any ); + literal = ('\'' literalChar* '\'' ) >start_literal %end_literal; + literals = ( ( literal | (any-'\'') ) $1 %0 )*; + + main := chars | words | comments | literals; +}%% + +%% write data; + +int Concurrent::init( ) +{ + %% write init; + cur_char = 0; + return 1; +} + +int Concurrent::execute( const char *data, int len, bool isEof ) +{ + const char *p = data; + const char *pe = data + len; + const char *eof = isEof ? pe : 0; + + %% write exec; + + if ( cs == Concurrent_error ) + return -1; + if ( cs >= Concurrent_first_final ) + return 1; + return 0; +} + +int Concurrent::finish( ) +{ + if ( cs == Concurrent_error ) + return -1; + if ( cs >= Concurrent_first_final ) + return 1; + return 0; +} + +Concurrent concurrent; +char buf[BUFSIZE]; + +int main() +{ + concurrent.init(); + while ( 1 ) { + int len = fread( buf, 1, BUFSIZE, stdin ); + concurrent.execute( buf, len, len != BUFSIZE ); + if ( len != BUFSIZE ) + break; + } + + if ( concurrent.finish() <= 0 ) + cerr << "concurrent: error parsing input" << endl; + return 0; +} diff --git a/examples/cppscan.lex b/examples/cppscan.lex new file mode 100644 index 0000000..fb66253 --- /dev/null +++ b/examples/cppscan.lex @@ -0,0 +1,143 @@ +/* + * flex equivalent to cppscan.rl + */ + +%{ + +#include <stdio.h> + +#define TK_Dlit 256 +#define TK_Slit 257 +#define TK_Float 258 +#define TK_Id 259 +#define TK_NameSep 260 +#define TK_Arrow 261 +#define TK_PlusPlus 262 +#define TK_MinusMinus 263 +#define TK_ArrowStar 264 +#define TK_DotStar 265 +#define TK_ShiftLeft 266 +#define TK_ShiftRight 267 +#define TK_IntegerDecimal 268 +#define TK_IntegerOctal 269 +#define TK_IntegerHex 270 +#define TK_EqualsEquals 271 +#define TK_NotEquals 272 +#define TK_AndAnd 273 +#define TK_OrOr 274 +#define TK_MultAssign 275 +#define TK_DivAssign 276 +#define TK_PercentAssign 277 +#define TK_PlusAssign 278 +#define TK_MinusAssign 279 +#define TK_AmpAssign 280 +#define TK_CaretAssign 281 +#define TK_BarAssign 282 +#define TK_DotDotDot 283 +#define TK_Whitespace 284 +#define TK_Comment 285 + +int line = 1, col = 1; + +void token( int tok, char *data, int len ) +{ + printf( "<%i> ", tok ); + for ( int i = 0; i < len; i++ ) + fputc( data[i], stdout ); + fputc( '\n', stdout ); + + /* Count newlines and columns. This code is here mainly for having some + * code in the token routine when commenting out the above output during + * performance testing. */ + for ( int i = 0; i < len; i ++ ) { + if ( data[i] == '\n' ) { + line += 1; + col = 1; + } + else { + col += 1; + } + } +} + + +%} + +%x COMMENT + +FRACT_CONST [0-9]*\.[0-9]+|[0-9]+\. +EXPONENT [eE][+\-]?[0-9]+ +FLOAT_SUFFIX [flFL] + +%% + + /* Single and double literals. */ +L?\'([^\'\\\n]|\\.)*\' { + token( TK_Slit, yytext, yyleng ); +} + +L?\"([^\"\\\n]|\\.)*\" { + token( TK_Dlit, yytext, yyleng ); +} + +[a-zA-Z_][a-zA-Z0-9_]* { + token( TK_Id, yytext, yyleng ); +} + +{FRACT_CONST}{EXPONENT}?{FLOAT_SUFFIX}?|[0-9]+{EXPONENT}{FLOAT_SUFFIX}? { + token( TK_Float, yytext, yyleng ); +} + +(0|[1-9][0-9]*)[ulUL]{0,3} { + token( TK_IntegerDecimal, yytext, yyleng ); +} + +0[0-9]+[ulUL]{0,2} { + token( TK_IntegerOctal, yytext, yyleng ); +} + +0x[0-9a-fA-F]+[ulUL]{0,2} { + token( TK_IntegerHex, yytext, yyleng ); +} + +:: token( TK_NameSep, yytext, yyleng ); +== token( TK_EqualsEquals, yytext, yyleng ); +!= token( TK_NotEquals, yytext, yyleng ); +&& token( TK_AndAnd, yytext, yyleng ); +\|\| token( TK_OrOr, yytext, yyleng ); +\*= token( TK_MultAssign, yytext, yyleng ); +\/= token( TK_DivAssign, yytext, yyleng ); +%= token( TK_PercentAssign, yytext, yyleng ); +\+= token( TK_PlusAssign, yytext, yyleng ); +-= token( TK_MinusAssign, yytext, yyleng ); +&= token( TK_AmpAssign, yytext, yyleng ); +^= token( TK_CaretAssign, yytext, yyleng ); +\|= token( TK_BarAssign, yytext, yyleng ); +\+\+ token( TK_PlusPlus, yytext, yyleng ); +-- token( TK_MinusMinus, yytext, yyleng ); +-> token( TK_Arrow, yytext, yyleng ); +->\* token( TK_ArrowStar, yytext, yyleng ); +\.\* token( TK_DotStar, yytext, yyleng ); +\.\.\. token( TK_DotDotDot, yytext, yyleng ); + +\/\* BEGIN(COMMENT); +<COMMENT>\*\/ BEGIN(INITIAL); +<COMMENT>(.|\n) { } + +\/\/.*\n {} +[^!-~]+ {} + +[!-/:-@\[-`{-~] token( yytext[0], yytext, yyleng ); + +%% + +int yywrap() +{ + /* Once the input is done, no more. */ + return 1; +} + +int main() +{ + yylex(); +} diff --git a/examples/cppscan.rec b/examples/cppscan.rec new file mode 100644 index 0000000..43f297d --- /dev/null +++ b/examples/cppscan.rec @@ -0,0 +1,183 @@ +#include <stdio.h> +#include <string.h> +#include <stdlib.h> + +#define TK_Dlit 256 +#define TK_Slit 257 +#define TK_Float 258 +#define TK_Id 259 +#define TK_NameSep 260 +#define TK_Arrow 261 +#define TK_PlusPlus 262 +#define TK_MinusMinus 263 +#define TK_ArrowStar 264 +#define TK_DotStar 265 +#define TK_ShiftLeft 266 +#define TK_ShiftRight 267 +#define TK_IntegerDecimal 268 +#define TK_IntegerOctal 269 +#define TK_IntegerHex 270 +#define TK_EqualsEquals 271 +#define TK_NotEquals 272 +#define TK_AndAnd 273 +#define TK_OrOr 274 +#define TK_MultAssign 275 +#define TK_DivAssign 276 +#define TK_PercentAssign 277 +#define TK_PlusAssign 278 +#define TK_MinusAssign 279 +#define TK_AmpAssign 280 +#define TK_CaretAssign 281 +#define TK_BarAssign 282 +#define TK_DotDotDot 283 +#define TK_Whitespace 284 +#define TK_Comment 285 + +int line = 1, col = 1; + +void token( int tok, char *data, int len ) +{ + printf( "<%i> ", tok ); + for ( int i = 0; i < len; i++ ) + fputc( data[i], stdout ); + fputc( '\n', stdout ); + + /* Count newlines and columns. This code is here mainly for having some + * code in the token routine when commenting out the above output during + * performance testing. */ + for ( int i = 0; i < len; i ++ ) { + if ( data[i] == '\n' ) { + line += 1; + col = 1; + } + else { + col += 1; + } + } +} + +#define BUFSIZE 8192 +char buf[BUFSIZE]; + +void fill( int n ) +{ + printf("fill(%i)\n", n); + exit(1); +} + +int main() +{ + char *start, *p = buf, *lim = buf, *marker; + int len, have, want, shift; + int done = 0; + +#define YYCTYPE char + +#define YYCURSOR p +#define YYLIMIT lim +#define YYMARKER marker + +#define YYFILL(n) { \ + if ( ! done ) { \ + have = lim-start; \ + if ( start > buf ) { \ + shift = start-buf; \ + memmove( buf, start, have ); \ + start -= shift; \ + p -= shift; \ + lim -= shift; \ + marker -= shift; \ + } \ + want = BUFSIZE - have - 1; \ + len = fread( lim, 1, want, stdin ); \ + lim += len; \ + if ( len < want ) { \ + *lim++ = 0; \ + done = 1; \ + } \ + } \ + } + +again: + start = p; + +/*!re2c + +ANY = [\000-\377]; +FRACTCONST = ( [0-9]* "." [0-9]+ ) | [0-9]+ "."; +EXPONENT = [eE] [+\-]? [0-9]+; +FLOATSUFFIX = [flFL]; + + "L"? "\'" ( ANY \ [\'\\\n] | "\\" ANY )* "\'" { + token( TK_Slit, start, p-start ); + goto again; + } + + "L"? "\"" ( ANY \ [\"\\\n] | "\\" ANY )* "\"" { + token( TK_Dlit, start, p-start ); + goto again; + } + + [a-zA-Z_][a-zA-Z0-9_]* { + token( TK_Id, start, p-start ); + goto again; + } + + ( FRACTCONST EXPONENT? FLOATSUFFIX? ) | ( [0-9]+ EXPONENT FLOATSUFFIX? ) { + token( TK_Float, start, p-start ); + goto again; + } + + + ( "0" | [1-9][0-9]* ) [ulUL]* { + token( TK_IntegerDecimal, start, p-start ); + goto again; + } + + "0" [0-9]+ [ulUL]* { + token( TK_IntegerOctal, start, p-start ); + goto again; + } + + "0x" [0-9a-fA-F]+[ulUL]* { + token( TK_IntegerHex, start, p-start ); + goto again; + } + + "::" { token( TK_NameSep, start, p-start ); goto again; } + "==" { token( TK_EqualsEquals, start, p-start ); goto again; } + "!=" { token( TK_NotEquals, start, p-start ); goto again; } + "&&" { token( TK_AndAnd, start, p-start ); goto again; } + "||" { token( TK_OrOr, start, p-start ); goto again; } + "*=" { token( TK_MultAssign, start, p-start ); goto again; } + "/=" { token( TK_DivAssign, start, p-start ); goto again; } + "%=" { token( TK_PercentAssign, start, p-start ); goto again; } + "+=" { token( TK_PlusAssign, start, p-start ); goto again; } + "-=" { token( TK_MinusAssign, start, p-start ); goto again; } + "&=" { token( TK_AmpAssign, start, p-start ); goto again; } + "^=" { token( TK_CaretAssign, start, p-start ); goto again; } + "|=" { token( TK_BarAssign, start, p-start ); goto again; } + "++" { token( TK_PlusPlus, start, p-start ); goto again; } + "--" { token( TK_MinusMinus, start, p-start ); goto again; } + "->" { token( TK_Arrow, start, p-start ); goto again; } + "->*" { token( TK_ArrowStar, start, p-start ); goto again; } + ".*" { token( TK_DotStar, start, p-start ); goto again; } + "..." { token( TK_DotDotDot, start, p-start ); goto again; } + + "/*" { goto comment; } + "//" (ANY\"\n")* "\n" { goto again; } + [\001-\040\177]+ { goto again; } + + [\041-\057\072-\100\133-\140\173-\176] { + token( *start, start, p-start ); + goto again; + } + "\000" { return 0; } +*/ + +comment: +/*!re2c + "*/" { goto again; } + ANY { goto comment; } +*/ +} diff --git a/examples/cppscan.rl b/examples/cppscan.rl new file mode 100644 index 0000000..1ead5aa --- /dev/null +++ b/examples/cppscan.rl @@ -0,0 +1,208 @@ +/* + * A C++ scanner. Uses the longest match construction. + * << <= <<= >> >= >>= are left out since angle brackets are used in templates. + */ + +#include <string.h> +#include <stdlib.h> +#include <iostream> + +#define TK_Dlit 256 +#define TK_Slit 257 +#define TK_Float 258 +#define TK_Id 259 +#define TK_NameSep 260 +#define TK_Arrow 261 +#define TK_PlusPlus 262 +#define TK_MinusMinus 263 +#define TK_ArrowStar 264 +#define TK_DotStar 265 +#define TK_ShiftLeft 266 +#define TK_ShiftRight 267 +#define TK_IntegerDecimal 268 +#define TK_IntegerOctal 269 +#define TK_IntegerHex 270 +#define TK_EqualsEquals 271 +#define TK_NotEquals 272 +#define TK_AndAnd 273 +#define TK_OrOr 274 +#define TK_MultAssign 275 +#define TK_DivAssign 276 +#define TK_PercentAssign 277 +#define TK_PlusAssign 278 +#define TK_MinusAssign 279 +#define TK_AmpAssign 280 +#define TK_CaretAssign 281 +#define TK_BarAssign 282 +#define TK_DotDotDot 283 +#define TK_Whitespace 284 +#define TK_Comment 285 + +#define BUFSIZE 16384 + +/* EOF char used to flush out that last token. This should be a whitespace + * token. */ + +#define LAST_CHAR 0 + +using std::cerr; +using std::cout; +using std::cin; +using std::endl; + +static char buf[BUFSIZE]; +static int line = 1, col = 1; +static char *ts, *te; +static int act, have = 0; +static int cs; + +%%{ + machine Scanner; + write data nofinal; + + # Floating literals. + fract_const = digit* '.' digit+ | digit+ '.'; + exponent = [eE] [+\-]? digit+; + float_suffix = [flFL]; + + c_comment := + any* :>> '*/' + @{ fgoto main; }; + + main := |* + + # Single and double literals. + ( 'L'? "'" ( [^'\\\n] | /\\./ )* "'" ) + {token( TK_Slit );}; + ( 'L'? '"' ( [^"\\\n] | /\\./ )* '"' ) + {token( TK_Dlit );}; + + # Identifiers + ( [a-zA-Z_] [a-zA-Z0-9_]* ) + {token( TK_Id );}; + + # Floating literals. + ( fract_const exponent? float_suffix? | digit+ exponent float_suffix? ) + {token( TK_Float );}; + + # Integer decimal. Leading part buffered by float. + ( ( '0' | [1-9] [0-9]* ) [ulUL]{0,3} ) + {token( TK_IntegerDecimal );}; + + # Integer octal. Leading part buffered by float. + ( '0' [0-9]+ [ulUL]{0,2} ) + {token( TK_IntegerOctal );}; + + # Integer hex. Leading 0 buffered by float. + ( '0' ( 'x' [0-9a-fA-F]+ [ulUL]{0,2} ) ) + {token( TK_IntegerHex );}; + + # Only buffer the second item, first buffered by symbol. */ + '::' {token( TK_NameSep );}; + '==' {token( TK_EqualsEquals );}; + '!=' {token( TK_NotEquals );}; + '&&' {token( TK_AndAnd );}; + '||' {token( TK_OrOr );}; + '*=' {token( TK_MultAssign );}; + '/=' {token( TK_DivAssign );}; + '%=' {token( TK_PercentAssign );}; + '+=' {token( TK_PlusAssign );}; + '-=' {token( TK_MinusAssign );}; + '&=' {token( TK_AmpAssign );}; + '^=' {token( TK_CaretAssign );}; + '|=' {token( TK_BarAssign );}; + '++' {token( TK_PlusPlus );}; + '--' {token( TK_MinusMinus );}; + '->' {token( TK_Arrow );}; + '->*' {token( TK_ArrowStar );}; + '.*' {token( TK_DotStar );}; + + # Three char compounds, first item already buffered. */ + '...' {token( TK_DotDotDot );}; + + # Single char symbols. + ( punct - [_"'] ) {token( ts[0] );}; + + # Comments and whitespace. + '/*' { fgoto c_comment; }; + '//' [^\n]* '\n'; + ( any - 33..126 )+; + + *|; +}%% + +void token( int tok ) +{ + char *data = ts; + int len = te - ts; + + cout << '<' << tok << "> "; + cout.write( data, len ); + cout << '\n'; + + /* Count newlines and columns. This code is here mainly for having some + * code in the token routine when commenting out the above output during + * performance testing. */ + for ( int i = 0; i < len; i ++ ) { + if ( data[i] == '\n' ) { + line += 1; + col = 1; + } + else { + col += 1; + } + } +} + +int main() +{ + std::ios::sync_with_stdio(false); + + %% write init; + + /* Do the first read. */ + bool done = false; + while ( !done ) { + char *p = buf + have; + int space = BUFSIZE - have; + + if ( space == 0 ) { + /* We filled up the buffer trying to scan a token. */ + cerr << "OUT OF BUFFER SPACE" << endl; + exit(1); + } + + cin.read( p, space ); + int len = cin.gcount(); + char *pe = p + len; + char *eof = 0; + + /* If we see eof then append the EOF char. */ + if ( cin.eof() ) { + eof = pe; + done = true; + } + + %% write exec; + + /* Check if we failed. */ + if ( cs == Scanner_error ) { + /* Machine failed before finding a token. */ + cerr << "PARSE ERROR" << endl; + exit(1); + } + + /* Now set up the prefix. */ + if ( ts == 0 ) + have = 0; + else { + /* There is data that needs to be shifted over. */ + have = pe - ts; + memmove( buf, ts, have ); + te -= (ts-buf); + ts = buf; + } + } + + return 0; +} diff --git a/examples/format.rl b/examples/format.rl new file mode 100644 index 0000000..f8a37be --- /dev/null +++ b/examples/format.rl @@ -0,0 +1,191 @@ +/* + * Partial printf implementation. + */ + +#define BUFLEN 1024 +#include <stdio.h> + +typedef void (*WriteFunc)( char *data, int len ); + +struct format +{ + char buf[BUFLEN+1]; + int buflen; + WriteFunc write; + + int flags; + int width; + int prec; + int cs; +}; + +void do_conv( struct format *fsm, char c ) +{ + printf( "flags: %x\n", fsm->flags ); + printf( "width: %i\n", fsm->width ); + printf( "prec: %i\n", fsm->prec ); + printf( "conv: %c\n", c ); + printf( "\n" ); +} + +#define FL_HASH 0x01 +#define FL_ZERO 0x02 +#define FL_DASH 0x04 +#define FL_SPACE 0x08 +#define FL_PLUS 0x10 + +#define FL_HAS_WIDTH 0x0100 +#define FL_WIDTH_ARG 0x0200 +#define FL_HAS_PREC 0x0400 +#define FL_PREC_ARG 0x0800 + +#define FL_LEN_H 0x010000 +#define FL_LEN_HH 0x020000 +#define FL_LEN_L 0x040000 +#define FL_LEN_LL 0x080000 + +%%{ + machine format; + access fsm->; + + action clear { + fsm->flags = 0; + fsm->width = 0; + fsm->prec = 0; + } + + # A non-zero number. + nznum = [1-9] [0-9]*; + + # Width + action width_num { fsm->width = 10 * fsm->width + (fc-'0'); } + action width_arg { fsm->flags |= FL_WIDTH_ARG; } + action width { fsm->flags |= FL_HAS_WIDTH; } + width = ( ( nznum $width_num | '*' @width_arg ) %width )?; + + # Precision + action prec_num { fsm->prec = 10 * fsm->prec + (fc-'0'); } + action prec_arg { fsm->flags |= FL_PREC_ARG; } + action prec { fsm->flags |= FL_HAS_PREC; } + precision = ( '.' ( digit* $prec_num %prec | '*' @prec_arg ) )?; + + # Flags + action flags_hash { fsm->flags |= FL_HASH; } + action flags_zero { fsm->flags |= FL_ZERO; } + action flags_dash { fsm->flags |= FL_DASH; } + action flags_space { fsm->flags |= FL_SPACE; } + action flags_plus { fsm->flags |= FL_PLUS; } + + flags = ( + '#' @flags_hash | + '0' @flags_zero | + '-' @flags_dash | + ' ' @flags_space | + '+' @flags_plus )*; + + action length_h { fsm->flags |= FL_LEN_H; } + action length_l { fsm->flags |= FL_LEN_L; } + action length_hh { fsm->flags |= FL_LEN_HH; } + action length_ll { fsm->flags |= FL_LEN_LL; } + + # Must use leaving transitions on 'h' and 'l' because they are + # prefixes for 'hh' and 'll'. + length = ( + 'h' %length_h | + 'l' %length_l | + 'hh' @length_hh | + 'll' @length_ll )?; + + action conversion { + do_conv( fsm, fc ); + } + + conversion = [diouxXcsp] @conversion; + + fmt_spec = + '%' @clear + flags + width + precision + length + conversion; + + action emit { + if ( fsm->buflen == BUFLEN ) { + fsm->write( fsm->buf, fsm->buflen ); + fsm->buflen = 0; + } + fsm->buf[fsm->buflen++] = fc; + } + + action finish_ok { + if ( fsm->buflen > 0 ) + fsm->write( fsm->buf, fsm->buflen ); + } + action finish_err { + printf("EOF IN FORMAT\n"); + } + action err_char { + printf("ERROR ON CHAR: 0x%x\n", fc ); + } + + main := ( + [^%] @emit | + '%%' @emit | + fmt_spec + )* @/finish_err %/finish_ok $!err_char; +}%% + +%% write data; + +void format_init( struct format *fsm ) +{ + fsm->buflen = 0; + %% write init; +} + +void format_execute( struct format *fsm, const char *data, int len, int isEof ) +{ + const char *p = data; + const char *pe = data + len; + const char *eof = isEof ? pe : 0; + + %% write exec; +} + +int format_finish( struct format *fsm ) +{ + if ( fsm->cs == format_error ) + return -1; + if ( fsm->cs >= format_first_final ) + return 1; + return 0; +} + + +#define INPUT_BUFSIZE 2048 + +struct format fsm; +char buf[INPUT_BUFSIZE]; + +void write(char *data, int len ) +{ + fwrite( data, 1, len, stdout ); +} + +int main() +{ + fsm.write = write; + format_init( &fsm ); + while ( 1 ) { + int len = fread( buf, 1, INPUT_BUFSIZE, stdin ); + int eof = len != INPUT_BUFSIZE; + format_execute( &fsm, buf, len, eof ); + if ( eof ) + break; + } + if ( format_finish( &fsm ) <= 0 ) + printf("FAIL\n"); + return 0; +} + diff --git a/examples/gotocallret.rl b/examples/gotocallret.rl new file mode 100644 index 0000000..32c01a2 --- /dev/null +++ b/examples/gotocallret.rl @@ -0,0 +1,96 @@ +/* + * Demonstrate the use of goto, call and return. This machine expects either a + * lower case char or a digit as a command then a space followed by the command + * arg. If the command is a char, then the arg must be an a string of chars. + * If the command is a digit, then the arg must be a string of digits. This + * choice is determined by action code, rather than though transition + * desitinations. + */ + +#include <iostream> +#include <stdlib.h> +#include <stdio.h> +#include <string.h> + +using namespace std; + +struct GotoCallRet +{ + char comm; + int cs, top, stack[32]; + + int init( ); + int execute( const char *data, int len, bool isEof ); + int finish( ); +}; + +%%{ + machine GotoCallRet; + + # Error machine, consumes to end of + # line, then starts the main line over. + garble_line := ( + (any-'\n')*'\n' + ) >{cout << "error: garbling line" << endl;} @{fgoto main;}; + + # Look for a string of alphas or of digits, + # on anything else, hold the character and return. + alp_comm := alpha+ $!{fhold;fret;}; + dig_comm := digit+ $!{fhold;fret;}; + + # Choose which to machine to call into based on the command. + action comm_arg { + if ( comm >= 'a' ) + fcall alp_comm; + else + fcall dig_comm; + } + + # Specifies command string. Note that the arg is left out. + command = ( + [a-z0-9] @{comm = fc;} ' ' @comm_arg '\n' + ) @{cout << "correct command" << endl;}; + + # Any number of commands. If there is an + # error anywhere, garble the line. + main := command* $!{fhold;fgoto garble_line;}; +}%% + +%% write data; + +int GotoCallRet::init( ) +{ + %% write init; + return 1; +} + +int GotoCallRet::execute( const char *data, int len, bool isEof ) +{ + const char *p = data; + const char *pe = data + len; + const char *eof = isEof ? pe : 0; + + %% write exec; + if ( cs == GotoCallRet_error ) + return -1; + if ( cs >= GotoCallRet_first_final ) + return 1; + return 0; +} + +#define BUFSIZE 1024 + +int main() +{ + char buf[BUFSIZE]; + + GotoCallRet gcr; + gcr.init(); + while ( fgets( buf, sizeof(buf), stdin ) != 0 ) + gcr.execute( buf, strlen(buf), false ); + + gcr.execute( 0, 0, true ); + if ( gcr.cs < GotoCallRet_first_final ) + cerr << "gotocallret: error: parsing input" << endl; + return 0; +} diff --git a/examples/mailbox.rl b/examples/mailbox.rl new file mode 100644 index 0000000..94590fd --- /dev/null +++ b/examples/mailbox.rl @@ -0,0 +1,207 @@ +/* + * Parses unix mail boxes into headers and bodies. + */ + +#include <iostream> +#include <stdlib.h> +#include <stdio.h> +#include <string.h> + +using namespace std; + +#define BUFSIZE 2048 + +/* A growable buffer for collecting headers. */ +struct Buffer +{ + Buffer() : data(0), allocated(0), length(0) { } + ~Buffer() { empty(); } + + void append( char p ) { + if ( ++length > allocated ) + upAllocate( length*2 ); + data[length-1] = p; + } + + void clear() { length = 0; } + void upAllocate( int len ); + void empty(); + + char *data; + int allocated; + int length; +}; + + +struct MailboxScanner +{ + Buffer headName; + Buffer headContent; + + int cs, top, stack[1]; + + int init( ); + int execute( const char *data, int len, bool isEof ); + int finish( ); +}; + +%%{ + machine MailboxScanner; + + # Buffer the header names. + action bufHeadName { headName.append(fc); } + + # Prints a blank line after the end of the headers of each message. + action blankLine { cout << endl; } + + # Helpers we will use in matching the date section of the from line. + day = /[A-Z][a-z][a-z]/; + month = /[A-Z][a-z][a-z]/; + year = /[0-9][0-9][0-9][0-9]/; + time = /[0-9][0-9]:[0-9][0-9]/ . ( /:[0-9][0-9]/ | '' ); + letterZone = /[A-Z][A-Z][A-Z]/; + numZone = /[+\-][0-9][0-9][0-9][0-9]/; + zone = letterZone | numZone; + dayNum = /[0-9 ][0-9]/; + + # These are the different formats of the date minus an obscure + # type that has a funny string 'remote from xxx' on the end. Taken + # from c-client in the imap-2000 distribution. + date = day . ' ' . month . ' ' . dayNum . ' ' . time . ' ' . + ( year | year . ' ' . zone | zone . ' ' . year ); + + # From lines separate messages. We will exclude fromLine from a message + # body line. This will cause us to stay in message line up until an + # entirely correct from line is matched. + fromLine = 'From ' . (any-'\n')* . ' ' . date . '\n'; + + # The types of characters that can be used as a header name. + hchar = print - [ :]; + + # Simply eat up an uninteresting header. Return at the first non-ws + # character following a newline. + consumeHeader := ( + [^\n] | + '\n' [ \t] | + '\n' [^ \t] @{fhold; fret;} + )*; + + action hchar {headContent.append(fc);} + action hspace {headContent.append(' ');} + + action hfinish { + headContent.append(0); + cout << headContent.data << endl; + headContent.clear(); + fhold; + fret; + } + + # Display the contents of a header as it is consumed. Collapses line + # continuations to a single space. + printHeader := ( + [^\n] @hchar | + ( '\n' ( [ \t]+ '\n' )* [ \t]+ ) %hspace + )** $!hfinish; + + action onHeader + { + headName.append(0); + if ( strcmp( headName.data, "From" ) == 0 || + strcmp( headName.data, "To" ) == 0 || + strcmp( headName.data, "Subject" ) == 0 ) + { + /* Print the header name, then jump to a machine the will display + * the contents. */ + cout << headName.data << ":"; + headName.clear(); + fcall printHeader; + } + + headName.clear(); + fcall consumeHeader; + } + + header = hchar+ $bufHeadName ':' @onHeader; + + # Exclude fromLine from a messageLine, otherwise when encountering a + # fromLine we will be simultaneously matching the old message and a new + # message. + messageLine = ( [^\n]* '\n' - fromLine ); + + # An entire message. + message = ( fromLine . header* . '\n' @blankLine . messageLine* ); + + # File is a series of messages. + main := message*; +}%% + +%% write data; + +int MailboxScanner::init( ) +{ + %% write init; + return 1; +} + +int MailboxScanner::execute( const char *data, int len, bool isEof ) +{ + const char *p = data; + const char *pe = data + len; + const char *eof = isEof ? pe : 0; + + %% write exec; + + if ( cs == MailboxScanner_error ) + return -1; + if ( cs >= MailboxScanner_first_final ) + return 1; + return 0; +} + +int MailboxScanner::finish( ) +{ + if ( cs == MailboxScanner_error ) + return -1; + if ( cs >= MailboxScanner_first_final ) + return 1; + return 0; +} + + +void Buffer::empty() +{ + if ( data != 0 ) { + free( data ); + + data = 0; + length = 0; + allocated = 0; + } +} + +void Buffer::upAllocate( int len ) +{ + if ( data == 0 ) + data = (char*) malloc( len ); + else + data = (char*) realloc( data, len ); + allocated = len; +} + +MailboxScanner mailbox; +char buf[BUFSIZE]; + +int main() +{ + mailbox.init(); + while ( 1 ) { + int len = fread( buf, 1, BUFSIZE, stdin ); + mailbox.execute( buf, len, len != BUFSIZE ); + if ( len != BUFSIZE ) + break; + } + if ( mailbox.finish() <= 0 ) + cerr << "mailbox: error parsing input" << endl; + return 0; +} diff --git a/examples/params.rl b/examples/params.rl new file mode 100644 index 0000000..a8ffeae --- /dev/null +++ b/examples/params.rl @@ -0,0 +1,102 @@ +/* + * Parse command line arguments. + */ + +#include <stdio.h> +#include <string.h> + +#define BUFLEN 1024 + +struct params +{ + char buffer[BUFLEN+1]; + int buflen; + int cs; +}; + +%%{ + machine params; + access fsm->; + + # A buffer to collect argurments + + # Append to the buffer. + action append { + if ( fsm->buflen < BUFLEN ) + fsm->buffer[fsm->buflen++] = fc; + } + + # Terminate a buffer. + action term { + if ( fsm->buflen < BUFLEN ) + fsm->buffer[fsm->buflen++] = 0; + } + + # Clear out the buffer + action clear { fsm->buflen = 0; } + + action help { printf("help\n"); } + action version { printf("version\n"); } + action output { printf("output: \"%s\"\n", fsm->buffer); } + action spec { printf("spec: \"%s\"\n", fsm->buffer); } + action mach { printf("machine: \"%s\"\n", fsm->buffer); } + + # Helpers that collect strings + string = [^\0]+ >clear $append %term; + + # Different arguments. + help = ( '-h' | '-H' | '-?' | '--help' ) 0 @help; + version = ( '-v' | '--version' ) 0 @version; + output = '-o' 0? string 0 @output; + spec = '-S' 0? string 0 @spec; + mach = '-M' 0? string 0 @mach; + + main := ( + help | + version | + output | + spec | + mach + )*; +}%% + +%% write data; + +void params_init( struct params *fsm ) +{ + fsm->buflen = 0; + %% write init; +} + +void params_execute( struct params *fsm, const char *data, int len ) +{ + const char *p = data; + const char *pe = data + len; + + %% write exec; +} + +int params_finish( struct params *fsm ) +{ + if ( fsm->cs == params_error ) + return -1; + if ( fsm->cs >= params_first_final ) + return 1; + return 0; +} + +#define BUFSIZE 2048 + +int main( int argc, char **argv ) +{ + int a; + struct params params; + + params_init( ¶ms ); + for ( a = 1; a < argc; a++ ) + params_execute( ¶ms, argv[a], strlen(argv[a])+1 ); + if ( params_finish( ¶ms ) != 1 ) + fprintf( stderr, "params: error processing arguments\n" ); + + return 0; +} diff --git a/examples/pullscan.rl b/examples/pullscan.rl new file mode 100644 index 0000000..d9e8a57 --- /dev/null +++ b/examples/pullscan.rl @@ -0,0 +1,170 @@ +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +#define BUFSIZE 4096 + +typedef struct _Scanner { + /* Scanner state. */ + int cs; + int act; + int have; + int curline; + char *ts; + char *te; + char *p; + char *pe; + char *eof; + FILE *file; + int done; + + /* Token data */ + char *data; + int len; + int value; + + char buf[BUFSIZE]; +} Scanner; + + +%%{ + machine Scanner; + write data; +}%% + +void scan_init( Scanner *s, FILE *file ) +{ + memset (s, '\0', sizeof(Scanner)); + s->curline = 1; + s->file = file; + s->eof = 0; + %% write init; +} + +#define TK_NO_TOKEN (-1) +#define TK_ERR 128 +#define TK_EOF 129 +#define TK_Identifier 130 +#define TK_Number 131 +#define TK_String 132 + +#define ret_tok( _tok ) token = _tok; s->data = s->ts + +int scan( Scanner *s ) +{ + int token = TK_NO_TOKEN; + int space, readlen; + + while ( 1 ) { + if ( s->p == s->pe ) { + printf("scanner: need more data\n"); + + if ( s->ts == 0 ) + s->have = 0; + else { + /* There is data that needs to be shifted over. */ + printf("scanner: buffer broken mid token\n"); + s->have = s->pe - s->ts; + memmove( s->buf, s->ts, s->have ); + s->te -= (s->ts-s->buf); + s->ts = s->buf; + } + + s->p = s->buf + s->have; + space = BUFSIZE - s->have; + + if ( space == 0 ) { + /* We filled up the buffer trying to scan a token. */ + printf("scanner: out of buffer space\n"); + return TK_ERR; + } + + if ( s->done ) { + printf("scanner: end of file\n"); + s->p[0] = 0; + readlen = 1; + } + else { + readlen = fread( s->p, 1, space, s->file ); + if ( readlen < space ) + s->done = 1; + } + + s->pe = s->p + readlen; + } + + %%{ + machine Scanner; + access s->; + variable p s->p; + variable pe s->pe; + variable eof s->eof; + + main := |* + + # Identifiers + ( [a-zA-Z_] [a-zA-Z0-9_]* ) => + { ret_tok( TK_Identifier ); fbreak; }; + + # Whitespace + [ \t\n]; + + '"' ( [^\\"] | '\\' any ) * '"' => + { ret_tok( TK_String ); fbreak; }; + + # Number + digit+ => + { ret_tok( TK_Number ); fbreak; }; + + # EOF + 0 => + { ret_tok( TK_EOF ); fbreak; }; + + # Anything else + any => + { ret_tok( *s->p ); fbreak; }; + + *|; + + write exec; + }%% + + if ( s->cs == Scanner_error ) + return TK_ERR; + + if ( token != TK_NO_TOKEN ) { + s->len = s->p - s->data; + return token; + } + } +} + + +int main (int argc, char** argv) +{ + Scanner ss; + int tok; + + scan_init(&ss, stdin); + + while ( 1 ) { + tok = scan (&ss); + if ( tok == TK_EOF ) { + printf ("parser: EOF\n"); + break; + } + else if ( tok == TK_ERR ) { + printf ("parser: ERR\n"); + break; + } + else { + printf ("parser: %d \"", tok); + fwrite ( ss.data, 1, ss.len, stdout ); + printf ("\"\n" ); + } + } + + return 0; +} + + diff --git a/examples/rlscan.rl b/examples/rlscan.rl new file mode 100644 index 0000000..d4d4bf9 --- /dev/null +++ b/examples/rlscan.rl @@ -0,0 +1,300 @@ +/* + * Lexes Ragel input files. + */ + +#include <iostream> +#include <stdlib.h> +#include <stdio.h> +#include <string.h> + +using namespace std; + +void escapeXML( char *data ) +{ + while ( *data != 0 ) { + switch ( *data ) { + case '<': cout << "<"; break; + case '>': cout << ">"; break; + case '&': cout << "&"; break; + default: cout << *data; break; + } + data += 1; + } +} + +void escapeXML( char c ) +{ + switch ( c ) { + case '<': cout << "<"; break; + case '>': cout << ">"; break; + case '&': cout << "&"; break; + default: cout << c; break; + } +} + +void escapeXML( char *data, int len ) +{ + for ( char *end = data + len; data != end; data++ ) { + switch ( *data ) { + case '<': cout << "<"; break; + case '>': cout << ">"; break; + case '&': cout << "&"; break; + default: cout << *data; break; + } + } +} + +inline void write( const char *data ) +{ + cout << data; +} + +inline void write( char c ) +{ + cout << c; +} + +inline void write( char *data, int len ) +{ + cout.write( data, len ); +} + + +%%{ + machine RagelScan; + + word = [a-zA-Z_][a-zA-Z_0-9]*; + integer = [0-9]+; + hex = '0x' [0-9a-fA-F] [0-9a-fA-F]*; + + default = ^0; + EOF = 0; + + # Handles comments in outside code and inline blocks. + c_comment := + ( default* :>> '*/' ) + ${ escapeXML( fc ); } + @{ fret; }; + + action emit { + escapeXML( ts, te-ts ); + } + + # + # Inline action code + # + + ilscan := |* + + "'" ( [^'\\] | /\\./ )* "'" => emit; + '"' ( [^"\\] | /\\./ )* '"' => emit; + '/*' { + write( "/*" ); + fcall c_comment; + }; + '//' [^\n]* '\n' => emit; + + '{' { + write( '{' ); + inline_depth += 1; + }; + + '}' { + write( '}' ); + /* If dropping down to the last } then return + * to ragel code. */ + if ( --inline_depth == 0 ) { + write( "</inline>\n" ); + fgoto rlscan; + } + }; + + default => { escapeXML( *ts ); }; + *|; + + # + # Ragel Tokens + # + + rlscan := |* + '}%%' { + if ( !single_line ) { + write( "</section>\n" ); + fgoto main; + } + }; + + '\n' { + if ( single_line ) { + write( "</section>\n" ); + fgoto main; + } + }; + + # Word + word { + write( "<word>" ); + write( ts, te-ts ); + write( "</word>\n" ); + }; + + # Decimal integer. + integer { + write( "<int>" ); + write( ts, te-ts ); + write( "</int>\n" ); + }; + + # Hexidecimal integer. + hex { + write( "<hex>" ); + write( ts, te-ts ); + write( "</hex>\n" ); + }; + + # Consume comments. + '#' [^\n]* '\n'; + + # Single literal string. + "'" ( [^'\\] | /\\./ )* "'" { + write( "<single_lit>" ); + escapeXML( ts, te-ts ); + write( "</single_lit>\n" ); + }; + + # Double literal string. + '"' ( [^"\\] | /\\./ )* '"' { + write( "<double_lit>" ); + escapeXML( ts, te-ts ); + write( "</double_lit>\n" ); + }; + + # Or literal. + '[' ( [^\]\\] | /\\./ )* ']' { + write( "<or_lit>" ); + escapeXML( ts, te-ts ); + write( "</or_lit>\n" ); + }; + + # Regex Literal. + '/' ( [^/\\] | /\\./ ) * '/' { + write( "<re_lit>" ); + escapeXML( ts, te-ts ); + write( "</re_lit>\n" ); + }; + + # Open an inline block + '{' { + inline_depth = 1; + write( "<inline>{" ); + fgoto ilscan; + }; + + punct { + write( "<symbol>" ); + escapeXML( fc ); + write( "</symbol>\n" ); + }; + + default; + *|; + + # + # Outside code. + # + + main := |* + + "'" ( [^'\\] | /\\./ )* "'" => emit; + '"' ( [^"\\] | /\\./ )* '"' => emit; + + '/*' { + escapeXML( ts, te-ts ); + fcall c_comment; + }; + + '//' [^\n]* '\n' => emit; + + '%%{' { + write( "<section>\n" ); + single_line = false; + fgoto rlscan; + }; + + '%%' { + write( "<section>\n" ); + single_line = true; + fgoto rlscan; + }; + + default { + escapeXML( *ts ); + }; + + # EOF. + EOF; + *|; +}%% + +%% write data nofinal; + +#define BUFSIZE 2048 + +int main() +{ + std::ios::sync_with_stdio(false); + + int cs, act; + char *ts, *te; + int stack[1], top; + + static char inbuf[BUFSIZE]; + bool single_line = false; + int inline_depth = 0; + + %% write init; + + bool done = false; + int have = 0; + while ( !done ) { + /* How much space is in the buffer? */ + int space = BUFSIZE - have; + if ( space == 0 ) { + /* Buffer is full. */ + cerr << "TOKEN TOO BIG" << endl; + exit(1); + } + + /* Read in a block. */ + char *p = inbuf + have; + cin.read( p, space ); + int len = cin.gcount(); + char *pe = p + len; + char *eof = 0; + + /* Check for EOF. */ + if ( len == 0 ) { + eof = pe; + done = true; + } + + %% write exec; + + if ( cs == RagelScan_error ) { + /* Machine failed before finding a token. */ + cerr << "PARSE ERROR" << endl; + exit(1); + } + + if ( ts == 0 ) + have = 0; + else { + /* There is a prefix to preserve, shift it over. */ + have = pe - ts; + memmove( inbuf, ts, have ); + te = inbuf + (te-ts); + ts = inbuf; + } + } + return 0; +} diff --git a/examples/statechart.rl b/examples/statechart.rl new file mode 100644 index 0000000..a04471b --- /dev/null +++ b/examples/statechart.rl @@ -0,0 +1,116 @@ +/* + * Demonstrate the use of labels, the epsilon operator, and the join operator + * for creating machines using the named state and transition list paradigm. + * This implementes the same machine as the atoi example. + */ + +#include <iostream> +#include <stdlib.h> +#include <stdio.h> +#include <string.h> + +using namespace std; + +struct StateChart +{ + bool neg; + int val; + int cs; + + int init( ); + int execute( const char *data, int len ); + int finish( ); +}; + +%%{ + machine StateChart; + + action begin { + neg = false; + val = 0; + } + + action see_neg { + neg = true; + } + + action add_digit { + val = val * 10 + (fc - '0'); + } + + action finish { + if ( neg ) + val = -1 * val; + } + + atoi = ( + start: ( + '-' @see_neg ->om_num | + '+' ->om_num | + [0-9] @add_digit ->more_nums + ), + + # One or more nums. + om_num: ( + [0-9] @add_digit ->more_nums + ), + + # Zero ore more nums. + more_nums: ( + [0-9] @add_digit ->more_nums | + '' -> final + ) + ) >begin %finish; + + main := ( atoi '\n' @{ cout << val << endl; } )*; +}%% + +%% write data; + +int StateChart::init( ) +{ + neg = false; + val = false; + %% write init; + return 1; +} + +int StateChart::execute( const char *data, int len ) +{ + const char *p = data; + const char *pe = data + len; + + %% write exec; + + if ( cs == StateChart_error ) + return -1; + if ( cs >= StateChart_first_final ) + return 1; + return 0; +} + +int StateChart::finish( ) +{ + if ( cs == StateChart_error ) + return -1; + if ( cs >= StateChart_first_final ) + return 1; + return 0; +} + + +#define BUFSIZE 1024 + +int main() +{ + char buf[BUFSIZE]; + + StateChart atoi; + atoi.init(); + while ( fgets( buf, sizeof(buf), stdin ) != 0 ) { + atoi.execute( buf, strlen(buf) ); + } + if ( atoi.finish() <= 0 ) + cerr << "statechart: error: parsing input" << endl; + return 0; +} diff --git a/examples/uri.rl b/examples/uri.rl new file mode 100644 index 0000000..185a76c --- /dev/null +++ b/examples/uri.rl @@ -0,0 +1,31 @@ +%%{ + machine uri; + + action scheme {} + action loc {} + action item {} + action query {} + action last {} + action nothing {} + + main := + # Scheme machine. This is ambiguous with the item machine. We commit + # to the scheme machine on colon. + ( [^:/?#]+ ':' @(colon,1) @scheme )? + + # Location machine. This is ambiguous with the item machine. We remain + # ambiguous until a second slash, at that point and all points after + # we place a higher priority on staying in the location machine over + # moving into the item machine. + ( ( '/' ( '/' [^/?#]* ) $(loc,1) ) %loc %/loc )? + + # Item machine. Ambiguous with both scheme and location, which both + # get a higher priority on the characters causing ambiguity. + ( ( [^?#]+ ) $(loc,0) $(colon,0) %item %/item )? + + # Last two components, the characters that initiate these machines are + # not supported in any previous components, therefore there are no + # ambiguities introduced by these parts. + ( '?' [^#]* %query %/query)? + ( '#' any* %/last )?; +}%% diff --git a/packaging/no-doc.patch b/packaging/no-doc.patch new file mode 100644 index 0000000..6227025 --- /dev/null +++ b/packaging/no-doc.patch @@ -0,0 +1,11 @@ +Index: ragel-6.6/Makefile.am +=================================================================== +--- ragel-6.6.orig/Makefile.am ++++ ragel-6.6/Makefile.am +@@ -1,5 +1,5 @@ + +-SUBDIRS = ragel doc ++SUBDIRS = ragel + DIST_SUBDIRS = $(SUBDIRS) aapl contrib examples test + + dist_doc_DATA = CREDITS ChangeLog diff --git a/packaging/ragel.spec b/packaging/ragel.spec new file mode 100644 index 0000000..502b503 --- /dev/null +++ b/packaging/ragel.spec @@ -0,0 +1,46 @@ +Name: ragel +Summary: Ragel +Version: 6.6 +Release: 1 +Group: TO_BE/FILLED_IN +License: TO BE FILLED IN +Source0: %{name}-%{version}.tar.gz +Patch0: no-doc.patch + + +%description +compiles finite state machines into code in various languages +Ragel compiles finite state machines from regular languages into C, C++, +Objective-C, D, Ruby or Java code. Ragel allows the programmer to embed +actions at any point in a regular language. Non-determinism can be +controlled through the use of embedded priorities and guarded regular +language operators. Ragel also supports the construction of scanners and +the building of state machines using state-charts. Ragel can be used to +create robust recognizers and parsers which run very fast. It can work +with integer-sized alphabets and can compile large state machines. +The generated code has no dependencies. + + +%prep +%setup -q +%patch0 -p1 + + +%build +./autogen.sh +./configure --prefix=%{_prefix} + +make %{?jobs:-j%jobs} + + +%install +rm -rf %{buildroot} +%make_install +mkdir -p %{buildroot}/usr/share/license +cp %{_builddir}/%{buildsubdir}/COPYING %{buildroot}/usr/share/license/%{name} + + +%files +%{_bindir}/ragel +%{_defaultdocdir}/* +/usr/share/license/%{name} diff --git a/ragel-d.vim b/ragel-d.vim new file mode 100644 index 0000000..ac7c51d --- /dev/null +++ b/ragel-d.vim @@ -0,0 +1,156 @@ +" Vim syntax file +" +" Language: Ragel +" Author: Adrian Thurston + +syntax clear + +" +" Outside code +" + +" Comments +syntax region ocComment start="\/\*" end="\*\/" +syntax match ocComment "\/\/.*$" + +" Anything preprocessor +syntax match ocPreproc "#.*$" + +" Strings +syntax match ocLiteral "'\(\\.\|[^'\\]\)*'" +syntax match ocLiteral "\"\(\\.\|[^\"\\]\)*\"" + +" C/C++ Keywords +syntax keyword ocType unsigned signed void char short int long float double bool +syntax keyword ocType inline static extern register const volatile auto +syntax keyword ocType union enum struct class typedef +syntax keyword ocType namespace template typename mutable +syntax keyword ocKeyword break continue default do else for +syntax keyword ocKeyword goto if return switch while +syntax keyword ocKeyword new delete this using friend public private protected sizeof +syntax keyword ocKeyword throw try catch operator typeid +syntax keyword ocKeyword and bitor xor compl bitand and_eq or_eq xor_eq not not_eq +syntax keyword ocKeyword static_cast dynamic_cast + +" D Keywords +syntax keyword ocType wchar dchar bit byte ubyte ushort uint ulong cent ucent +syntax keyword ocType cfloat ifloat cdouble idouble real creal ireal +syntax keyword ocKeyword abstract alias align asm assert body cast debug delegate +syntax keyword ocKeyword deprecated export final finally foreach function import in inout +syntax keyword ocKeyword interface invariant is mixin module out override package pragma +syntax keyword ocKeyword super synchronized typeof unittest version with + +" Numbers +syntax match ocNumber "[0-9][0-9]*" +syntax match ocNumber "0x[0-9a-fA-F][0-9a-fA-F]*" + +" Booleans +syntax keyword ocBoolean true false + +" Identifiers +syntax match anyId "[a-zA-Z_][a-zA-Z_0-9]*" + +" Inline code only +syntax keyword fsmType fpc fc fcurs fbuf fblen ftargs fstack +syntax keyword fsmKeyword fhold fgoto fcall fret fentry fnext fexec fbreak + +syntax cluster rlItems contains=rlComment,rlLiteral,rlAugmentOps,rlOtherOps,rlKeywords,rlWrite,rlCodeCurly,rlCodeSemi,rlNumber,anyId,rlLabelColon,rlExprKeywords + +syntax region machineSpec1 matchgroup=beginRL start="%%{" end="}%%" contains=@rlItems +syntax region machineSpec2 matchgroup=beginRL start="%%[^{]"rs=e-1 end="$" keepend contains=@rlItems +syntax region machineSpec2 matchgroup=beginRL start="%%$" end="$" keepend contains=@rlItems + +" Comments +syntax match rlComment "#.*$" contained + +" Literals +syntax match rlLiteral "'\(\\.\|[^'\\]\)*'[i]*" contained +syntax match rlLiteral "\"\(\\.\|[^\"\\]\)*\"[i]*" contained +syntax match rlLiteral /\/\(\\.\|[^\/\\]\)*\/[i]*/ contained +syntax match rlLiteral "\[\(\\.\|[^\]\\]\)*\]" contained + +" Numbers +syntax match rlNumber "[0-9][0-9]*" contained +syntax match rlNumber "0x[0-9a-fA-F][0-9a-fA-F]*" contained + +" Operators +syntax match rlAugmentOps "[>$%@]" contained +syntax match rlAugmentOps "<>\|<" contained +syntax match rlAugmentOps "[>\<$%@][!\^/*~]" contained +syntax match rlAugmentOps "[>$%]?" contained +syntax match rlAugmentOps "<>[!\^/*~]" contained +syntax match rlAugmentOps "=>" contained +syntax match rlOtherOps "->" contained + +syntax match rlOtherOps ":>" contained +syntax match rlOtherOps ":>>" contained +syntax match rlOtherOps "<:" contained + +" Keywords +" FIXME: Enable the range keyword post 5.17. +" syntax keyword rlKeywords machine action context include range contained +syntax keyword rlKeywords machine action context include import export prepush postpop contained +syntax keyword rlExprKeywords when inwhen outwhen err lerr eof from to contained + +" Case Labels +syntax keyword caseLabelKeyword case contained +syntax cluster caseLabelItems contains=ocComment,ocPreproc,ocLiteral,ocType,ocKeyword,caseLabelKeyword,ocNumber,ocBoolean,anyId,fsmType,fsmKeyword +syntax match caseLabelColon "case" contains=@caseLabelItems +syntax match caseLabelColon "case[\t ]\+.*:$" contains=@caseLabelItems +syntax match caseLabelColon "case[\t ]\+.*:[^=:]"me=e-1 contains=@caseLabelItems + +" Labels +syntax match ocLabelColon "^[\t ]*[a-zA-Z_][a-zA-Z_0-9]*[ \t]*:$" contains=anyLabel +syntax match ocLabelColon "^[\t ]*[a-zA-Z_][a-zA-Z_0-9]*[ \t]*:[^=:]"me=e-1 contains=anyLabel + +syntax match rlLabelColon "[a-zA-Z_][a-zA-Z_0-9]*[ \t]*:$" contained contains=anyLabel +syntax match rlLabelColon "[a-zA-Z_][a-zA-Z_0-9]*[ \t]*:[^=:>]"me=e-1 contained contains=anyLabel +syntax match anyLabel "[a-zA-Z_][a-zA-Z_0-9]*" contained + +" All items that can go in a code block. + +syntax cluster inlineItems contains=rlCodeCurly,ocComment,ocPreproc,ocLiteral,ocType,ocKeyword,ocNumber,ocBoolean,ocLabelColon,anyId,fsmType,fsmKeyword,caseLabelColon + +" Blocks of code. rlCodeCurly is recursive. +syntax region rlCodeCurly matchgroup=NONE start="{" end="}" contained contains=@inlineItems +syntax region rlCodeSemi matchgroup=Type start="\<alphtype\>" start="\<getkey\>" start="\<access\>" start="\<variable\>" matchgroup=NONE end=";" contained contains=@inlineItems + +syntax region rlWrite matchgroup=Type start="\<write\>" matchgroup=NONE end="[;)]" contained contains=rlWriteKeywords,rlWriteOptions + +syntax keyword rlWriteKeywords init data exec exports start error first_final contained +syntax keyword rlWriteOptions noerror nofinal noprefix noend nocs contained + +" +" Sync at the start of machine specs. +" +" Match The ragel delimiters only if there quotes no ahead on the same line. +" On the open marker, use & to consume the leader. +syntax sync match ragelSyncPat grouphere NONE "^[^\'\"%]*%%{&^[^\'\"%]*" +syntax sync match ragelSyncPat grouphere NONE "^[^\'\"%]*%%[^{]&^[^\'\"%]*" +syntax sync match ragelSyncPat grouphere NONE "^[^\'\"]*}%%" + +" +" Specifying Groups +" +hi link ocComment Comment +hi link ocPreproc Macro +hi link ocLiteral String +hi link ocType Type +hi link ocKeyword Keyword +hi link ocNumber Number +hi link ocBoolean Boolean +hi link rlComment Comment +hi link rlNumber Number +hi link rlLiteral String +hi link rlAugmentOps Keyword +hi link rlExprKeywords Keyword +hi link rlWriteKeywords Keyword +hi link rlWriteOptions Keyword +hi link rlKeywords Type +hi link fsmType Type +hi link fsmKeyword Keyword +hi link anyLabel Label +hi link caseLabelKeyword Keyword +hi link beginRL Type + +let b:current_syntax = "ragel" diff --git a/ragel-java.vim b/ragel-java.vim new file mode 100644 index 0000000..227481e --- /dev/null +++ b/ragel-java.vim @@ -0,0 +1,151 @@ +" Vim syntax file +" +" Language: Ragel +" Author: Adrian Thurston + +syntax clear + +" +" Outside code +" + +" Comments +syntax region ocComment start="\/\*" end="\*\/" +syntax match ocComment "\/\/.*$" + +" Anything preprocessor +syntax match ocPreproc "#.*$" + +" Strings +syntax match ocLiteral "'\(\\.\|[^'\\]\)*'" +syntax match ocLiteral "\"\(\\.\|[^\"\\]\)*\"" + +" C/C++ Keywords +syntax keyword ocType unsigned signed void char short int long float double bool +syntax keyword ocType inline static extern register const volatile auto +syntax keyword ocType union enum struct class typedef +syntax keyword ocType namespace template typename mutable +syntax keyword ocKeyword break continue default do else for +syntax keyword ocKeyword goto if return switch while +syntax keyword ocKeyword new delete this using friend public private protected sizeof +syntax keyword ocKeyword throw try catch operator typeid +syntax keyword ocKeyword and bitor xor compl bitand and_eq or_eq xor_eq not not_eq +syntax keyword ocKeyword static_cast dynamic_cast + +" Java Keywords +syntax keyword ocType byte short char int + +" Numbers +syntax match ocNumber "[0-9][0-9]*" +syntax match ocNumber "0x[0-9a-fA-F][0-9a-fA-F]*" + +" Booleans +syntax keyword ocBoolean true false + +" Identifiers +syntax match anyId "[a-zA-Z_][a-zA-Z_0-9]*" + +" Inline code only +syntax keyword fsmType fpc fc fcurs fbuf fblen ftargs fstack +syntax keyword fsmKeyword fhold fgoto fcall fret fentry fnext fexec fbreak + +syntax cluster rlItems contains=rlComment,rlLiteral,rlAugmentOps,rlOtherOps,rlKeywords,rlWrite,rlCodeCurly,rlCodeSemi,rlNumber,anyId,rlLabelColon,rlExprKeywords + +syntax region machineSpec1 matchgroup=beginRL start="%%{" end="}%%" contains=@rlItems +syntax region machineSpec2 matchgroup=beginRL start="%%[^{]"rs=e-1 end="$" keepend contains=@rlItems +syntax region machineSpec2 matchgroup=beginRL start="%%$" end="$" keepend contains=@rlItems + +" Comments +syntax match rlComment "#.*$" contained + +" Literals +syntax match rlLiteral "'\(\\.\|[^'\\]\)*'[i]*" contained +syntax match rlLiteral "\"\(\\.\|[^\"\\]\)*\"[i]*" contained +syntax match rlLiteral /\/\(\\.\|[^\/\\]\)*\/[i]*/ contained +syntax match rlLiteral "\[\(\\.\|[^\]\\]\)*\]" contained + +" Numbers +syntax match rlNumber "[0-9][0-9]*" contained +syntax match rlNumber "0x[0-9a-fA-F][0-9a-fA-F]*" contained + +" Operators +syntax match rlAugmentOps "[>$%@]" contained +syntax match rlAugmentOps "<>\|<" contained +syntax match rlAugmentOps "[>\<$%@][!\^/*~]" contained +syntax match rlAugmentOps "[>$%]?" contained +syntax match rlAugmentOps "<>[!\^/*~]" contained +syntax match rlAugmentOps "=>" contained +syntax match rlOtherOps "->" contained + +syntax match rlOtherOps ":>" contained +syntax match rlOtherOps ":>>" contained +syntax match rlOtherOps "<:" contained + +" Keywords +" FIXME: Enable the range keyword post 5.17. +" syntax keyword rlKeywords machine action context include range contained +syntax keyword rlKeywords machine action context include import export prepush postpop contained +syntax keyword rlExprKeywords when inwhen outwhen err lerr eof from to contained + +" Case Labels +syntax keyword caseLabelKeyword case contained +syntax cluster caseLabelItems contains=ocComment,ocPreproc,ocLiteral,ocType,ocKeyword,caseLabelKeyword,ocNumber,ocBoolean,anyId,fsmType,fsmKeyword +syntax match caseLabelColon "case" contains=@caseLabelItems +syntax match caseLabelColon "case[\t ]\+.*:$" contains=@caseLabelItems +syntax match caseLabelColon "case[\t ]\+.*:[^=:]"me=e-1 contains=@caseLabelItems + +" Labels +syntax match ocLabelColon "^[\t ]*[a-zA-Z_][a-zA-Z_0-9]*[ \t]*:$" contains=anyLabel +syntax match ocLabelColon "^[\t ]*[a-zA-Z_][a-zA-Z_0-9]*[ \t]*:[^=:]"me=e-1 contains=anyLabel + +syntax match rlLabelColon "[a-zA-Z_][a-zA-Z_0-9]*[ \t]*:$" contained contains=anyLabel +syntax match rlLabelColon "[a-zA-Z_][a-zA-Z_0-9]*[ \t]*:[^=:>]"me=e-1 contained contains=anyLabel +syntax match anyLabel "[a-zA-Z_][a-zA-Z_0-9]*" contained + +" All items that can go in a code block. + +syntax cluster inlineItems contains=rlCodeCurly,ocComment,ocPreproc,ocLiteral,ocType,ocKeyword,ocNumber,ocBoolean,ocLabelColon,anyId,fsmType,fsmKeyword,caseLabelColon + +" Blocks of code. rlCodeCurly is recursive. +syntax region rlCodeCurly matchgroup=NONE start="{" end="}" contained contains=@inlineItems +syntax region rlCodeSemi matchgroup=Type start="\<alphtype\>" start="\<getkey\>" start="\<access\>" start="\<variable\>" matchgroup=NONE end=";" contained contains=@inlineItems + +syntax region rlWrite matchgroup=Type start="\<write\>" matchgroup=NONE end="[;)]" contained contains=rlWriteKeywords,rlWriteOptions + +syntax keyword rlWriteKeywords init data exec exports start error first_final contained +syntax keyword rlWriteOptions noerror nofinal noprefix noend nocs contained + +" +" Sync at the start of machine specs. +" +" Match The ragel delimiters only if there quotes no ahead on the same line. +" On the open marker, use & to consume the leader. +syntax sync match ragelSyncPat grouphere NONE "^[^\'\"%]*%%{&^[^\'\"%]*" +syntax sync match ragelSyncPat grouphere NONE "^[^\'\"%]*%%[^{]&^[^\'\"%]*" +syntax sync match ragelSyncPat grouphere NONE "^[^\'\"]*}%%" + +" +" Specifying Groups +" +hi link ocComment Comment +hi link ocPreproc Macro +hi link ocLiteral String +hi link ocType Type +hi link ocKeyword Keyword +hi link ocNumber Number +hi link ocBoolean Boolean +hi link rlComment Comment +hi link rlNumber Number +hi link rlLiteral String +hi link rlAugmentOps Keyword +hi link rlExprKeywords Keyword +hi link rlWriteKeywords Keyword +hi link rlWriteOptions Keyword +hi link rlKeywords Type +hi link fsmType Type +hi link fsmKeyword Keyword +hi link anyLabel Label +hi link caseLabelKeyword Keyword +hi link beginRL Type + +let b:current_syntax = "ragel" diff --git a/ragel-m.vim b/ragel-m.vim new file mode 100644 index 0000000..c8d9904 --- /dev/null +++ b/ragel-m.vim @@ -0,0 +1,154 @@ +" Vim syntax file +" +" Language: Ragel +" Author: Adrian Thurston + +syntax clear + +" +" Outside code +" + +" Comments +syntax region ocComment start="\/\*" end="\*\/" +syntax match ocComment "\/\/.*$" + +" Anything preprocessor +syntax match ocPreproc "#.*$" + +" Strings +syntax match ocLiteral "'\(\\.\|[^'\\]\)*'" +syntax match ocLiteral "\"\(\\.\|[^\"\\]\)*\"" + +" C/C++ Keywords +syntax keyword ocType unsigned signed void char short int long float double bool +syntax keyword ocType inline static extern register const volatile auto +syntax keyword ocType union enum struct class typedef +syntax keyword ocType namespace template typename mutable +syntax keyword ocKeyword break continue default do else for +syntax keyword ocKeyword goto if return switch while +syntax keyword ocKeyword new delete this using friend public private protected sizeof +syntax keyword ocKeyword throw try catch operator typeid +syntax keyword ocKeyword and bitor xor compl bitand and_eq or_eq xor_eq not not_eq +syntax keyword ocKeyword static_cast dynamic_cast + +" Objective-C Directives +syntax match ocKeyword "@public\|@private\|@protected" +syntax match ocKeyword "@interface\|@implementation" +syntax match ocKeyword "@class\|@end\|@defs" +syntax match ocKeyword "@encode\|@protocol\|@selector" + +" Numbers +syntax match ocNumber "[0-9][0-9]*" +syntax match ocNumber "0x[0-9a-fA-F][0-9a-fA-F]*" + +" Booleans +syntax keyword ocBoolean true false + +" Identifiers +syntax match anyId "[a-zA-Z_][a-zA-Z_0-9]*" + +" Inline code only +syntax keyword fsmType fpc fc fcurs fbuf fblen ftargs fstack +syntax keyword fsmKeyword fhold fgoto fcall fret fentry fnext fexec fbreak + +syntax cluster rlItems contains=rlComment,rlLiteral,rlAugmentOps,rlOtherOps,rlKeywords,rlWrite,rlCodeCurly,rlCodeSemi,rlNumber,anyId,rlLabelColon,rlExprKeywords + +syntax region machineSpec1 matchgroup=beginRL start="%%{" end="}%%" contains=@rlItems +syntax region machineSpec2 matchgroup=beginRL start="%%[^{]"rs=e-1 end="$" keepend contains=@rlItems +syntax region machineSpec2 matchgroup=beginRL start="%%$" end="$" keepend contains=@rlItems + +" Comments +syntax match rlComment "#.*$" contained + +" Literals +syntax match rlLiteral "'\(\\.\|[^'\\]\)*'[i]*" contained +syntax match rlLiteral "\"\(\\.\|[^\"\\]\)*\"[i]*" contained +syntax match rlLiteral /\/\(\\.\|[^\/\\]\)*\/[i]*/ contained +syntax match rlLiteral "\[\(\\.\|[^\]\\]\)*\]" contained + +" Numbers +syntax match rlNumber "[0-9][0-9]*" contained +syntax match rlNumber "0x[0-9a-fA-F][0-9a-fA-F]*" contained + +" Operators +syntax match rlAugmentOps "[>$%@]" contained +syntax match rlAugmentOps "<>\|<" contained +syntax match rlAugmentOps "[>\<$%@][!\^/*~]" contained +syntax match rlAugmentOps "[>$%]?" contained +syntax match rlAugmentOps "<>[!\^/*~]" contained +syntax match rlAugmentOps "=>" contained +syntax match rlOtherOps "->" contained + +syntax match rlOtherOps ":>" contained +syntax match rlOtherOps ":>>" contained +syntax match rlOtherOps "<:" contained + +" Keywords +" FIXME: Enable the range keyword post 5.17. +" syntax keyword rlKeywords machine action context include range contained +syntax keyword rlKeywords machine action context include import export prepush postpop contained +syntax keyword rlExprKeywords when inwhen outwhen err lerr eof from to contained + +" Case Labels +syntax keyword caseLabelKeyword case contained +syntax cluster caseLabelItems contains=ocComment,ocPreproc,ocLiteral,ocType,ocKeyword,caseLabelKeyword,ocNumber,ocBoolean,anyId,fsmType,fsmKeyword +syntax match caseLabelColon "case" contains=@caseLabelItems +syntax match caseLabelColon "case[\t ]\+.*:$" contains=@caseLabelItems +syntax match caseLabelColon "case[\t ]\+.*:[^=:]"me=e-1 contains=@caseLabelItems + +" Labels +syntax match ocLabelColon "^[\t ]*[a-zA-Z_][a-zA-Z_0-9]*[ \t]*:$" contains=anyLabel +syntax match ocLabelColon "^[\t ]*[a-zA-Z_][a-zA-Z_0-9]*[ \t]*:[^=:]"me=e-1 contains=anyLabel + +syntax match rlLabelColon "[a-zA-Z_][a-zA-Z_0-9]*[ \t]*:$" contained contains=anyLabel +syntax match rlLabelColon "[a-zA-Z_][a-zA-Z_0-9]*[ \t]*:[^=:>]"me=e-1 contained contains=anyLabel +syntax match anyLabel "[a-zA-Z_][a-zA-Z_0-9]*" contained + +" All items that can go in a code block. + +syntax cluster inlineItems contains=rlCodeCurly,ocComment,ocPreproc,ocLiteral,ocType,ocKeyword,ocNumber,ocBoolean,ocLabelColon,anyId,fsmType,fsmKeyword,caseLabelColon + +" Blocks of code. rlCodeCurly is recursive. +syntax region rlCodeCurly matchgroup=NONE start="{" end="}" contained contains=@inlineItems +syntax region rlCodeSemi matchgroup=Type start="\<alphtype\>" start="\<getkey\>" start="\<access\>" start="\<variable\>" matchgroup=NONE end=";" contained contains=@inlineItems + +syntax region rlWrite matchgroup=Type start="\<write\>" matchgroup=NONE end="[;)]" contained contains=rlWriteKeywords,rlWriteOptions + +syntax keyword rlWriteKeywords init data exec exports start error first_final contained +syntax keyword rlWriteOptions noerror nofinal noprefix noend nocs contained + +" +" Sync at the start of machine specs. +" +" Match The ragel delimiters only if there quotes no ahead on the same line. +" On the open marker, use & to consume the leader. +syntax sync match ragelSyncPat grouphere NONE "^[^\'\"%]*%%{&^[^\'\"%]*" +syntax sync match ragelSyncPat grouphere NONE "^[^\'\"%]*%%[^{]&^[^\'\"%]*" +syntax sync match ragelSyncPat grouphere NONE "^[^\'\"]*}%%" + +" +" Specifying Groups +" +hi link ocComment Comment +hi link ocPreproc Macro +hi link ocLiteral String +hi link ocType Type +hi link ocKeyword Keyword +hi link ocNumber Number +hi link ocBoolean Boolean +hi link rlComment Comment +hi link rlNumber Number +hi link rlLiteral String +hi link rlAugmentOps Keyword +hi link rlExprKeywords Keyword +hi link rlWriteKeywords Keyword +hi link rlWriteOptions Keyword +hi link rlKeywords Type +hi link fsmType Type +hi link fsmKeyword Keyword +hi link anyLabel Label +hi link caseLabelKeyword Keyword +hi link beginRL Type + +let b:current_syntax = "ragel" @@ -14,7 +14,8 @@ syntax region ocComment start="\/\*" end="\*\/" syntax match ocComment "\/\/.*$" " Anything preprocessor -syntax match ocPreproc "#.*$" +syntax match ocPreproc "#\(.\|\\\n\)*$" +syntax region ocPreproc start="#" end="[^\\]$" " Strings syntax match ocLiteral "'\(\\.\|[^'\\]\)*'" @@ -32,23 +33,6 @@ syntax keyword ocKeyword throw try catch operator typeid syntax keyword ocKeyword and bitor xor compl bitand and_eq or_eq xor_eq not not_eq syntax keyword ocKeyword static_cast dynamic_cast -" D Keywords -syntax keyword ocType wchar dchar bit byte ubyte ushort uint ulong cent ucent -syntax keyword ocType cfloat ifloat cdouble idouble real creal ireal -syntax keyword ocKeyword abstract alias align asm assert body cast debug delegate -syntax keyword ocKeyword deprecated export final finally foreach function import in inout -syntax keyword ocKeyword interface invariant is mixin module out override package pragma -syntax keyword ocKeyword super synchronized typeof unittest version with - -" Java Keywords -syntax keyword ocType byte short char int - -" Objective-C Directives -syntax match ocKeyword "@public\|@private\|@protected" -syntax match ocKeyword "@interface\|@implementation" -syntax match ocKeyword "@class\|@end\|@defs" -syntax match ocKeyword "@encode\|@protocol\|@selector" - " Numbers syntax match ocNumber "[0-9][0-9]*" syntax match ocNumber "0x[0-9a-fA-F][0-9a-fA-F]*" @@ -96,8 +80,10 @@ syntax match rlOtherOps ":>>" contained syntax match rlOtherOps "<:" contained " Keywords -syntax keyword rlKeywords machine action context include range contained -syntax keyword rlExprKeywords when err lerr eof from to contained +" FIXME: Enable the range keyword post 5.17. +" syntax keyword rlKeywords machine action context include range contained +syntax keyword rlKeywords machine action context include import export prepush postpop contained +syntax keyword rlExprKeywords when inwhen outwhen err lerr eof from to contained " Case Labels syntax keyword caseLabelKeyword case contained @@ -122,17 +108,19 @@ syntax cluster inlineItems contains=rlCodeCurly,ocComment,ocPreproc,ocLiteral,oc syntax region rlCodeCurly matchgroup=NONE start="{" end="}" contained contains=@inlineItems syntax region rlCodeSemi matchgroup=Type start="\<alphtype\>" start="\<getkey\>" start="\<access\>" start="\<variable\>" matchgroup=NONE end=";" contained contains=@inlineItems -syntax region rlWrite matchgroup=Type start="\<write\>" matchgroup=NONE end=";" contained contains=rlWriteKeywords,rlWriteOptions +syntax region rlWrite matchgroup=Type start="\<write\>" matchgroup=NONE end="[;)]" contained contains=rlWriteKeywords,rlWriteOptions -syntax keyword rlWriteKeywords init data exec eof contained -syntax keyword rlWriteOptions noerror nofinal noprefix noend contained +syntax keyword rlWriteKeywords init data exec exports start error first_final contained +syntax keyword rlWriteOptions noerror nofinal noprefix noend nocs contained " " Sync at the start of machine specs. " -syntax sync match ragelSyncPat grouphere NONE "%%{&" -syntax sync match ragelSyncPat grouphere NONE "%%[^{]&" -syntax sync match ragelSyncPat grouphere NONE "}%%" +" Match The ragel delimiters only if there quotes no ahead on the same line. +" On the open marker, use & to consume the leader. +syntax sync match ragelSyncPat grouphere NONE "^[^\'\"%]*%%{&^[^\'\"%]*" +syntax sync match ragelSyncPat grouphere NONE "^[^\'\"%]*%%[^{]&^[^\'\"%]*" +syntax sync match ragelSyncPat grouphere NONE "^[^\'\"]*}%%" " " Specifying Groups diff --git a/ragel/Makefile.am b/ragel/Makefile.am new file mode 100644 index 0000000..db3610c --- /dev/null +++ b/ragel/Makefile.am @@ -0,0 +1,52 @@ + +INCLUDES = -I$(top_srcdir)/aapl + +bin_PROGRAMS = ragel + +ragel_CXXFLAGS = -Wall + +ragel_SOURCES = \ + buffer.h cdgoto.h cscodegen.h csipgoto.h inputdata.h rbxgoto.h \ + rubyflat.h cdcodegen.h cdipgoto.h csfflat.h cssplit.h javacodegen.h \ + redfsm.h rubyftable.h cdfflat.h cdsplit.h csfgoto.h cstable.h \ + parsedata.h rlparse.h rubytable.h cdfgoto.h cdtable.h csflat.h \ + dotcodegen.h parsetree.h rlscan.h version.h cdflat.h common.h \ + csftable.h fsmgraph.h pcheck.h rubycodegen.h xmlcodegen.h cdftable.h \ + csgoto.h gendata.h ragel.h rubyfflat.h \ + main.cpp parsetree.cpp parsedata.cpp fsmstate.cpp fsmbase.cpp \ + fsmattach.cpp fsmmin.cpp fsmgraph.cpp fsmap.cpp rlscan.cpp rlparse.cpp \ + inputdata.cpp common.cpp redfsm.cpp gendata.cpp cdcodegen.cpp \ + cdtable.cpp cdftable.cpp cdflat.cpp cdfflat.cpp cdgoto.cpp cdfgoto.cpp \ + cdipgoto.cpp cdsplit.cpp javacodegen.cpp rubycodegen.cpp rubytable.cpp \ + rubyftable.cpp rubyflat.cpp rubyfflat.cpp rbxgoto.cpp cscodegen.cpp \ + cstable.cpp csftable.cpp csflat.cpp csfflat.cpp csgoto.cpp csfgoto.cpp \ + csipgoto.cpp cssplit.cpp dotcodegen.cpp xmlcodegen.cpp + +BUILT_SOURCES = \ + rlscan.cpp rlparse.h rlparse.cpp version.h + +version.h: Makefile + echo '#define VERSION "$(PACKAGE_VERSION)"' > version.h + echo '#define PUBDATE "$(PUBDATE)"' >> version.h + +EXTRA_DIST = rlscan.rl rlparse.kh rlparse.kl + +if BUILD_PARSERS + +CLEANFILES = \ + rlscan.cpp rlparse.h rlparse.cpp + +rlparse.h: rlparse.kh + kelbt -o $@ $< + +rlparse.cpp: rlparse.kl rlparse.kh + kelbt -o $@ $< + +# This dependency comes from the import of the parser defines +# into the scanner. +rlscan.cpp: rlparse.h + +rlscan.cpp: rlscan.rl + ragel -G2 -I$(builddir) -o $@ $< + +endif diff --git a/ragel/buffer.h b/ragel/buffer.h new file mode 100644 index 0000000..9c49c1f --- /dev/null +++ b/ragel/buffer.h @@ -0,0 +1,55 @@ +/* + * Copyright 2003 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _BUFFER_H +#define _BUFFER_H + +#define BUFFER_INITIAL_SIZE 4096 + +/* An automatically grown buffer for collecting tokens. Always reuses space; + * never down resizes. */ +struct Buffer +{ + Buffer() + { + data = (char*) malloc( BUFFER_INITIAL_SIZE ); + allocated = BUFFER_INITIAL_SIZE; + length = 0; + } + ~Buffer() { free(data); } + + void append( char p ) + { + if ( length == allocated ) { + allocated *= 2; + data = (char*) realloc( data, allocated ); + } + data[length++] = p; + } + + void clear() { length = 0; } + + char *data; + int allocated; + int length; +}; + +#endif diff --git a/ragel/cdcodegen.cpp b/ragel/cdcodegen.cpp new file mode 100644 index 0000000..a0a882c --- /dev/null +++ b/ragel/cdcodegen.cpp @@ -0,0 +1,856 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "cdcodegen.h" +#include "ragel.h" +#include "redfsm.h" +#include "gendata.h" +#include <sstream> +#include <string> +#include <assert.h> + + +using std::ostream; +using std::ostringstream; +using std::string; +using std::cerr; +using std::endl; +using std::istream; +using std::ifstream; +using std::ostream; +using std::ios; +using std::cin; +using std::cout; +using std::cerr; +using std::endl; + + +extern int numSplitPartitions; +extern bool noLineDirectives; + +void cdLineDirective( ostream &out, const char *fileName, int line ) +{ + if ( noLineDirectives ) + out << "/* "; + + /* Write the preprocessor line info for to the input file. */ + out << "#line " << line << " \""; + for ( const char *pc = fileName; *pc != 0; pc++ ) { + if ( *pc == '\\' ) + out << "\\\\"; + else + out << *pc; + } + out << '"'; + + if ( noLineDirectives ) + out << " */"; + + out << '\n'; +} + +void FsmCodeGen::genLineDirective( ostream &out ) +{ + std::streambuf *sbuf = out.rdbuf(); + output_filter *filter = static_cast<output_filter*>(sbuf); + cdLineDirective( out, filter->fileName, filter->line + 1 ); +} + + +/* Init code gen with in parameters. */ +FsmCodeGen::FsmCodeGen( ostream &out ) +: + CodeGenData(out) +{ +} + +unsigned int FsmCodeGen::arrayTypeSize( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + return arrayType->size; +} + +string FsmCodeGen::ARRAY_TYPE( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + + string ret = arrayType->data1; + if ( arrayType->data2 != 0 ) { + ret += " "; + ret += arrayType->data2; + } + return ret; +} + + +/* Write out the fsm name. */ +string FsmCodeGen::FSM_NAME() +{ + return fsmName; +} + +/* Emit the offset of the start state as a decimal integer. */ +string FsmCodeGen::START_STATE_ID() +{ + ostringstream ret; + ret << redFsm->startState->id; + return ret.str(); +}; + +/* Write out the array of actions. */ +std::ostream &FsmCodeGen::ACTIONS_ARRAY() +{ + out << "\t0, "; + int totalActions = 1; + for ( GenActionTableMap::Iter act = redFsm->actionMap; act.lte(); act++ ) { + /* Write out the length, which will never be the last character. */ + out << act->key.length() << ", "; + /* Put in a line break every 8 */ + if ( totalActions++ % 8 == 7 ) + out << "\n\t"; + + for ( GenActionTable::Iter item = act->key; item.lte(); item++ ) { + out << item->value->actionId; + if ( ! (act.last() && item.last()) ) + out << ", "; + + /* Put in a line break every 8 */ + if ( totalActions++ % 8 == 7 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +string FsmCodeGen::ACCESS() +{ + ostringstream ret; + if ( accessExpr != 0 ) + INLINE_LIST( ret, accessExpr, 0, false, false ); + return ret.str(); +} + + +string FsmCodeGen::P() +{ + ostringstream ret; + if ( pExpr == 0 ) + ret << "p"; + else { + ret << "("; + INLINE_LIST( ret, pExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::PE() +{ + ostringstream ret; + if ( peExpr == 0 ) + ret << "pe"; + else { + ret << "("; + INLINE_LIST( ret, peExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::vEOF() +{ + ostringstream ret; + if ( eofExpr == 0 ) + ret << "eof"; + else { + ret << "("; + INLINE_LIST( ret, eofExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::vCS() +{ + ostringstream ret; + if ( csExpr == 0 ) + ret << ACCESS() << "cs"; + else { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, csExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::TOP() +{ + ostringstream ret; + if ( topExpr == 0 ) + ret << ACCESS() + "top"; + else { + ret << "("; + INLINE_LIST( ret, topExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::STACK() +{ + ostringstream ret; + if ( stackExpr == 0 ) + ret << ACCESS() + "stack"; + else { + ret << "("; + INLINE_LIST( ret, stackExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::ACT() +{ + ostringstream ret; + if ( actExpr == 0 ) + ret << ACCESS() + "act"; + else { + ret << "("; + INLINE_LIST( ret, actExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::TOKSTART() +{ + ostringstream ret; + if ( tokstartExpr == 0 ) + ret << ACCESS() + "ts"; + else { + ret << "("; + INLINE_LIST( ret, tokstartExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::TOKEND() +{ + ostringstream ret; + if ( tokendExpr == 0 ) + ret << ACCESS() + "te"; + else { + ret << "("; + INLINE_LIST( ret, tokendExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::GET_WIDE_KEY() +{ + if ( redFsm->anyConditions() ) + return "_widec"; + else + return GET_KEY(); +} + +string FsmCodeGen::GET_WIDE_KEY( RedStateAp *state ) +{ + if ( state->stateCondList.length() > 0 ) + return "_widec"; + else + return GET_KEY(); +} + +string FsmCodeGen::GET_KEY() +{ + ostringstream ret; + if ( getKeyExpr != 0 ) { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, getKeyExpr, 0, false, false ); + ret << ")"; + } + else { + /* Expression for retrieving the key, use simple dereference. */ + ret << "(*" << P() << ")"; + } + return ret.str(); +} + +/* Write out level number of tabs. Makes the nested binary search nice + * looking. */ +string FsmCodeGen::TABS( int level ) +{ + string result; + while ( level-- > 0 ) + result += "\t"; + return result; +} + +/* Write out a key from the fsm code gen. Depends on wether or not the key is + * signed. */ +string FsmCodeGen::KEY( Key key ) +{ + ostringstream ret; + if ( keyOps->isSigned || !hostLang->explicitUnsigned ) + ret << key.getVal(); + else + ret << (unsigned long) key.getVal() << 'u'; + return ret.str(); +} + +void FsmCodeGen::EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ) +{ + /* The parser gives fexec two children. The double brackets are for D + * code. If the inline list is a single word it will get interpreted as a + * C-style cast by the D compiler. */ + ret << "{" << P() << " = (("; + INLINE_LIST( ret, item->children, targState, inFinish, false ); + ret << "))-1;}"; +} + +void FsmCodeGen::LM_SWITCH( ostream &ret, GenInlineItem *item, + int targState, int inFinish, bool csForced ) +{ + ret << + " switch( " << ACT() << " ) {\n"; + + bool haveDefault = false; + for ( GenInlineList::Iter lma = *item->children; lma.lte(); lma++ ) { + /* Write the case label, the action and the case break. */ + if ( lma->lmId < 0 ) { + ret << " default:\n"; + haveDefault = true; + } + else + ret << " case " << lma->lmId << ":\n"; + + /* Write the block and close it off. */ + ret << " {"; + INLINE_LIST( ret, lma->children, targState, inFinish, csForced ); + ret << "}\n"; + + ret << " break;\n"; + } + + if ( hostLang->lang == HostLang::D && !haveDefault ) + ret << " default: break;"; + + ret << + " }\n" + "\t"; +} + +void FsmCodeGen::SET_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = " << item->lmId << ";"; +} + +void FsmCodeGen::SET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + /* The tokend action sets tokend. */ + ret << TOKEND() << " = " << P(); + if ( item->offset != 0 ) + out << "+" << item->offset; + out << ";"; +} + +void FsmCodeGen::GET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + ret << TOKEND(); +} + +void FsmCodeGen::INIT_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << NULL_ITEM() << ";"; +} + +void FsmCodeGen::INIT_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = 0;"; +} + +void FsmCodeGen::SET_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << P() << ";"; +} + +void FsmCodeGen::SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish, bool csForced ) +{ + if ( item->children->length() > 0 ) { + /* Write the block and close it off. */ + ret << "{"; + INLINE_LIST( ret, item->children, targState, inFinish, csForced ); + ret << "}"; + } +} + + +/* Write out an inline tree structure. Walks the list and possibly calls out + * to virtual functions than handle language specific items in the tree. */ +void FsmCodeGen::INLINE_LIST( ostream &ret, GenInlineList *inlineList, + int targState, bool inFinish, bool csForced ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Text: + ret << item->data; + break; + case GenInlineItem::Goto: + GOTO( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Call: + CALL( ret, item->targState->id, targState, inFinish ); + break; + case GenInlineItem::Next: + NEXT( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Ret: + RET( ret, inFinish ); + break; + case GenInlineItem::PChar: + ret << P(); + break; + case GenInlineItem::Char: + ret << GET_KEY(); + break; + case GenInlineItem::Hold: + ret << P() << "--;"; + break; + case GenInlineItem::Exec: + EXEC( ret, item, targState, inFinish ); + break; + case GenInlineItem::Curs: + CURS( ret, inFinish ); + break; + case GenInlineItem::Targs: + TARGS( ret, inFinish, targState ); + break; + case GenInlineItem::Entry: + ret << item->targState->id; + break; + case GenInlineItem::GotoExpr: + GOTO_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::CallExpr: + CALL_EXPR( ret, item, targState, inFinish ); + break; + case GenInlineItem::NextExpr: + NEXT_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::LmSwitch: + LM_SWITCH( ret, item, targState, inFinish, csForced ); + break; + case GenInlineItem::LmSetActId: + SET_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokEnd: + SET_TOKEND( ret, item ); + break; + case GenInlineItem::LmGetTokEnd: + GET_TOKEND( ret, item ); + break; + case GenInlineItem::LmInitTokStart: + INIT_TOKSTART( ret, item ); + break; + case GenInlineItem::LmInitAct: + INIT_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokStart: + SET_TOKSTART( ret, item ); + break; + case GenInlineItem::SubAction: + SUB_ACTION( ret, item, targState, inFinish, csForced ); + break; + case GenInlineItem::Break: + BREAK( ret, targState, csForced ); + break; + } + } +} +/* Write out paths in line directives. Escapes any special characters. */ +string FsmCodeGen::LDIR_PATH( char *path ) +{ + ostringstream ret; + for ( char *pc = path; *pc != 0; pc++ ) { + if ( *pc == '\\' ) + ret << "\\\\"; + else + ret << *pc; + } + return ret.str(); +} + +void FsmCodeGen::ACTION( ostream &ret, GenAction *action, int targState, + bool inFinish, bool csForced ) +{ + /* Write the preprocessor line info for going into the source file. */ + cdLineDirective( ret, action->loc.fileName, action->loc.line ); + + /* Write the block and close it off. */ + ret << "\t{"; + INLINE_LIST( ret, action->inlineList, targState, inFinish, csForced ); + ret << "}\n"; +} + +void FsmCodeGen::CONDITION( ostream &ret, GenAction *condition ) +{ + ret << "\n"; + cdLineDirective( ret, condition->loc.fileName, condition->loc.line ); + INLINE_LIST( ret, condition->inlineList, 0, false, false ); +} + +string FsmCodeGen::ERROR_STATE() +{ + ostringstream ret; + if ( redFsm->errState != 0 ) + ret << redFsm->errState->id; + else + ret << "-1"; + return ret.str(); +} + +string FsmCodeGen::FIRST_FINAL_STATE() +{ + ostringstream ret; + if ( redFsm->firstFinState != 0 ) + ret << redFsm->firstFinState->id; + else + ret << redFsm->nextStateId; + return ret.str(); +} + +void FsmCodeGen::writeInit() +{ + out << " {\n"; + + if ( !noCS ) + out << "\t" << vCS() << " = " << START() << ";\n"; + + /* If there are any calls, then the stack top needs initialization. */ + if ( redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "\t" << TOP() << " = 0;\n"; + + if ( hasLongestMatch ) { + out << + " " << TOKSTART() << " = " << NULL_ITEM() << ";\n" + " " << TOKEND() << " = " << NULL_ITEM() << ";\n" + " " << ACT() << " = 0;\n"; + } + out << " }\n"; +} + +string FsmCodeGen::DATA_PREFIX() +{ + if ( !noPrefix ) + return FSM_NAME() + "_"; + return ""; +} + +/* Emit the alphabet data type. */ +string FsmCodeGen::ALPH_TYPE() +{ + string ret = keyOps->alphType->data1; + if ( keyOps->alphType->data2 != 0 ) { + ret += " "; + ret += + keyOps->alphType->data2; + } + return ret; +} + +/* Emit the alphabet data type. */ +string FsmCodeGen::WIDE_ALPH_TYPE() +{ + string ret; + if ( redFsm->maxKey <= keyOps->maxKey ) + ret = ALPH_TYPE(); + else { + long long maxKeyVal = redFsm->maxKey.getLongLong(); + HostType *wideType = keyOps->typeSubsumes( keyOps->isSigned, maxKeyVal ); + assert( wideType != 0 ); + + ret = wideType->data1; + if ( wideType->data2 != 0 ) { + ret += " "; + ret += wideType->data2; + } + } + return ret; +} + +void FsmCodeGen::STATE_IDS() +{ + if ( redFsm->startState != 0 ) + STATIC_VAR( "int", START() ) << " = " << START_STATE_ID() << ";\n"; + + if ( !noFinal ) + STATIC_VAR( "int" , FIRST_FINAL() ) << " = " << FIRST_FINAL_STATE() << ";\n"; + + if ( !noError ) + STATIC_VAR( "int", ERROR() ) << " = " << ERROR_STATE() << ";\n"; + + out << "\n"; + + if ( entryPointNames.length() > 0 ) { + for ( EntryNameVect::Iter en = entryPointNames; en.lte(); en++ ) { + STATIC_VAR( "int", DATA_PREFIX() + "en_" + *en ) << + " = " << entryPointIds[en.pos()] << ";\n"; + } + out << "\n"; + } +} + +void FsmCodeGen::writeStart() +{ + out << START_STATE_ID(); +} + +void FsmCodeGen::writeFirstFinal() +{ + out << FIRST_FINAL_STATE(); +} + +void FsmCodeGen::writeError() +{ + out << ERROR_STATE(); +} + +/* + * Language specific, but style independent code generators functions. + */ + +string CCodeGen::PTR_CONST() +{ + return "const "; +} + +std::ostream &CCodeGen::OPEN_ARRAY( string type, string name ) +{ + out << "static const " << type << " " << name << "[] = {\n"; + return out; +} + +std::ostream &CCodeGen::CLOSE_ARRAY() +{ + return out << "};\n"; +} + +std::ostream &CCodeGen::STATIC_VAR( string type, string name ) +{ + out << "static const " << type << " " << name; + return out; +} + +string CCodeGen::UINT( ) +{ + return "unsigned int"; +} + +string CCodeGen::ARR_OFF( string ptr, string offset ) +{ + return ptr + " + " + offset; +} + +string CCodeGen::CAST( string type ) +{ + return "(" + type + ")"; +} + +string CCodeGen::NULL_ITEM() +{ + return "0"; +} + +string CCodeGen::POINTER() +{ + return " *"; +} + +std::ostream &CCodeGen::SWITCH_DEFAULT() +{ + return out; +} + +string CCodeGen::CTRL_FLOW() +{ + return ""; +} + +void CCodeGen::writeExports() +{ + if ( exportList.length() > 0 ) { + for ( ExportList::Iter ex = exportList; ex.lte(); ex++ ) { + out << "#define " << DATA_PREFIX() << "ex_" << ex->name << " " << + KEY(ex->key) << "\n"; + } + out << "\n"; + } +} + +/* + * D Specific + */ + +string DCodeGen::NULL_ITEM() +{ + return "null"; +} + +string DCodeGen::POINTER() +{ + // multiple items seperated by commas can also be pointer types. + return "* "; +} + +string DCodeGen::PTR_CONST() +{ + return ""; +} + +std::ostream &DCodeGen::OPEN_ARRAY( string type, string name ) +{ + out << "static const " << type << "[] " << name << " = [\n"; + return out; +} + +std::ostream &DCodeGen::CLOSE_ARRAY() +{ + return out << "];\n"; +} + +std::ostream &DCodeGen::STATIC_VAR( string type, string name ) +{ + out << "static const " << type << " " << name; + return out; +} + +string DCodeGen::ARR_OFF( string ptr, string offset ) +{ + return "&" + ptr + "[" + offset + "]"; +} + +string DCodeGen::CAST( string type ) +{ + return "cast(" + type + ")"; +} + +string DCodeGen::UINT( ) +{ + return "uint"; +} + +std::ostream &DCodeGen::SWITCH_DEFAULT() +{ + out << " default: break;\n"; + return out; +} + +string DCodeGen::CTRL_FLOW() +{ + return "if (true) "; +} + +void DCodeGen::writeExports() +{ + if ( exportList.length() > 0 ) { + for ( ExportList::Iter ex = exportList; ex.lte(); ex++ ) { + out << "static const " << ALPH_TYPE() << " " << DATA_PREFIX() << + "ex_" << ex->name << " = " << KEY(ex->key) << ";\n"; + } + out << "\n"; + } +} + +/* + * End D-specific code. + */ + +void FsmCodeGen::finishRagelDef() +{ + if ( codeStyle == GenGoto || codeStyle == GenFGoto || + codeStyle == GenIpGoto || codeStyle == GenSplit ) + { + /* For directly executable machines there is no required state + * ordering. Choose a depth-first ordering to increase the + * potential for fall-throughs. */ + redFsm->depthFirstOrdering(); + } + else { + /* The frontend will do this for us, but it may be a good idea to + * force it if the intermediate file is edited. */ + redFsm->sortByStateId(); + } + + /* Choose default transitions and the single transition. */ + redFsm->chooseDefaultSpan(); + + /* Maybe do flat expand, otherwise choose single. */ + if ( codeStyle == GenFlat || codeStyle == GenFFlat ) + redFsm->makeFlat(); + else + redFsm->chooseSingle(); + + /* If any errors have occured in the input file then don't write anything. */ + if ( gblErrorCount > 0 ) + return; + + if ( codeStyle == GenSplit ) + redFsm->partitionFsm( numSplitPartitions ); + + if ( codeStyle == GenIpGoto || codeStyle == GenSplit ) + redFsm->setInTrans(); + + /* Anlayze Machine will find the final action reference counts, among + * other things. We will use these in reporting the usage + * of fsm directives in action code. */ + analyzeMachine(); + + /* Determine if we should use indicies. */ + calcIndexSize(); +} + +ostream &FsmCodeGen::source_warning( const InputLoc &loc ) +{ + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": warning: "; + return cerr; +} + +ostream &FsmCodeGen::source_error( const InputLoc &loc ) +{ + gblErrorCount += 1; + assert( sourceFileName != 0 ); + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": "; + return cerr; +} + diff --git a/ragel/cdcodegen.h b/ragel/cdcodegen.h new file mode 100644 index 0000000..23b2228 --- /dev/null +++ b/ragel/cdcodegen.h @@ -0,0 +1,223 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDCODEGEN_H +#define _CDCODEGEN_H + +#include <iostream> +#include <string> +#include <stdio.h> +#include "common.h" +#include "gendata.h" + +using std::string; +using std::ostream; + +/* Integer array line length. */ +#define IALL 8 + +/* Forwards. */ +struct RedFsmAp; +struct RedStateAp; +struct CodeGenData; +struct GenAction; +struct NameInst; +struct GenInlineItem; +struct GenInlineList; +struct RedAction; +struct LongestMatch; +struct LongestMatchPart; + +string itoa( int i ); + +/* + * class FsmCodeGen + */ +class FsmCodeGen : public CodeGenData +{ +public: + FsmCodeGen( ostream &out ); + virtual ~FsmCodeGen() {} + + virtual void finishRagelDef(); + virtual void writeInit(); + virtual void writeStart(); + virtual void writeFirstFinal(); + virtual void writeError(); + +protected: + string FSM_NAME(); + string START_STATE_ID(); + ostream &ACTIONS_ARRAY(); + string GET_WIDE_KEY(); + string GET_WIDE_KEY( RedStateAp *state ); + string TABS( int level ); + string KEY( Key key ); + string LDIR_PATH( char *path ); + void ACTION( ostream &ret, GenAction *action, int targState, + bool inFinish, bool csForced ); + void CONDITION( ostream &ret, GenAction *condition ); + string ALPH_TYPE(); + string WIDE_ALPH_TYPE(); + string ARRAY_TYPE( unsigned long maxVal ); + + virtual string ARR_OFF( string ptr, string offset ) = 0; + virtual string CAST( string type ) = 0; + virtual string UINT() = 0; + virtual string NULL_ITEM() = 0; + virtual string POINTER() = 0; + virtual string GET_KEY(); + virtual ostream &SWITCH_DEFAULT() = 0; + + string P(); + string PE(); + string vEOF(); + + string ACCESS(); + string vCS(); + string STACK(); + string TOP(); + string TOKSTART(); + string TOKEND(); + string ACT(); + + string DATA_PREFIX(); + string PM() { return "_" + DATA_PREFIX() + "partition_map"; } + string C() { return "_" + DATA_PREFIX() + "cond_spaces"; } + string CK() { return "_" + DATA_PREFIX() + "cond_keys"; } + string K() { return "_" + DATA_PREFIX() + "trans_keys"; } + string I() { return "_" + DATA_PREFIX() + "indicies"; } + string CO() { return "_" + DATA_PREFIX() + "cond_offsets"; } + string KO() { return "_" + DATA_PREFIX() + "key_offsets"; } + string IO() { return "_" + DATA_PREFIX() + "index_offsets"; } + string CL() { return "_" + DATA_PREFIX() + "cond_lengths"; } + string SL() { return "_" + DATA_PREFIX() + "single_lengths"; } + string RL() { return "_" + DATA_PREFIX() + "range_lengths"; } + string A() { return "_" + DATA_PREFIX() + "actions"; } + string TA() { return "_" + DATA_PREFIX() + "trans_actions"; } + string TT() { return "_" + DATA_PREFIX() + "trans_targs"; } + string TSA() { return "_" + DATA_PREFIX() + "to_state_actions"; } + string FSA() { return "_" + DATA_PREFIX() + "from_state_actions"; } + string EA() { return "_" + DATA_PREFIX() + "eof_actions"; } + string ET() { return "_" + DATA_PREFIX() + "eof_trans"; } + string SP() { return "_" + DATA_PREFIX() + "key_spans"; } + string CSP() { return "_" + DATA_PREFIX() + "cond_key_spans"; } + string START() { return DATA_PREFIX() + "start"; } + string ERROR() { return DATA_PREFIX() + "error"; } + string FIRST_FINAL() { return DATA_PREFIX() + "first_final"; } + string CTXDATA() { return DATA_PREFIX() + "ctxdata"; } + + void INLINE_LIST( ostream &ret, GenInlineList *inlineList, + int targState, bool inFinish, bool csForced ); + virtual void GOTO( ostream &ret, int gotoDest, bool inFinish ) = 0; + virtual void CALL( ostream &ret, int callDest, int targState, bool inFinish ) = 0; + virtual void NEXT( ostream &ret, int nextDest, bool inFinish ) = 0; + virtual void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) = 0; + virtual void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) = 0; + virtual void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, + int targState, bool inFinish ) = 0; + virtual void RET( ostream &ret, bool inFinish ) = 0; + virtual void BREAK( ostream &ret, int targState, bool csForced ) = 0; + virtual void CURS( ostream &ret, bool inFinish ) = 0; + virtual void TARGS( ostream &ret, bool inFinish, int targState ) = 0; + void EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void LM_SWITCH( ostream &ret, GenInlineItem *item, int targState, + int inFinish, bool csForced ); + void SET_ACT( ostream &ret, GenInlineItem *item ); + void INIT_TOKSTART( ostream &ret, GenInlineItem *item ); + void INIT_ACT( ostream &ret, GenInlineItem *item ); + void SET_TOKSTART( ostream &ret, GenInlineItem *item ); + void SET_TOKEND( ostream &ret, GenInlineItem *item ); + void GET_TOKEND( ostream &ret, GenInlineItem *item ); + void SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish, bool csForced ); + void STATE_IDS(); + + string ERROR_STATE(); + string FIRST_FINAL_STATE(); + + virtual string PTR_CONST() = 0; + virtual ostream &OPEN_ARRAY( string type, string name ) = 0; + virtual ostream &CLOSE_ARRAY() = 0; + virtual ostream &STATIC_VAR( string type, string name ) = 0; + + virtual string CTRL_FLOW() = 0; + + ostream &source_warning(const InputLoc &loc); + ostream &source_error(const InputLoc &loc); + + unsigned int arrayTypeSize( unsigned long maxVal ); + + bool outLabelUsed; + bool testEofUsed; + bool againLabelUsed; + bool useIndicies; + + void genLineDirective( ostream &out ); + +public: + /* Determine if we should use indicies. */ + virtual void calcIndexSize() {} +}; + +class CCodeGen : virtual public FsmCodeGen +{ +public: + CCodeGen( ostream &out ) : FsmCodeGen(out) {} + + virtual string NULL_ITEM(); + virtual string POINTER(); + virtual ostream &SWITCH_DEFAULT(); + virtual ostream &OPEN_ARRAY( string type, string name ); + virtual ostream &CLOSE_ARRAY(); + virtual ostream &STATIC_VAR( string type, string name ); + virtual string ARR_OFF( string ptr, string offset ); + virtual string CAST( string type ); + virtual string UINT(); + virtual string PTR_CONST(); + virtual string CTRL_FLOW(); + + virtual void writeExports(); +}; + +class DCodeGen : virtual public FsmCodeGen +{ +public: + DCodeGen( ostream &out ) : FsmCodeGen(out) {} + + virtual string NULL_ITEM(); + virtual string POINTER(); + virtual ostream &SWITCH_DEFAULT(); + virtual ostream &OPEN_ARRAY( string type, string name ); + virtual ostream &CLOSE_ARRAY(); + virtual ostream &STATIC_VAR( string type, string name ); + virtual string ARR_OFF( string ptr, string offset ); + virtual string CAST( string type ); + virtual string UINT(); + virtual string PTR_CONST(); + virtual string CTRL_FLOW(); + + virtual void writeExports(); +}; + +#endif diff --git a/ragel/cdfflat.cpp b/ragel/cdfflat.cpp new file mode 100644 index 0000000..d2c1c78 --- /dev/null +++ b/ragel/cdfflat.cpp @@ -0,0 +1,384 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cdfflat.h" +#include "redfsm.h" +#include "gendata.h" + +std::ostream &FFlatCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &FFlatCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &FFlatCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + out << act; + return out; +} + +/* Write out the function for a transition. */ +std::ostream &FFlatCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + int action = 0; + if ( trans->action != 0 ) + action = trans->action->actListId+1; + out << action; + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &FFlatCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &FFlatCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &FFlatCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &FFlatCodeGen::ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +void FFlatCodeGen::writeData() +{ + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpan), CSP() ); + COND_KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCond), C() ); + CONDS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondIndexOffset), CO() ); + COND_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSpan), SP() ); + KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxFlatIndexOffset), IO() ); + FLAT_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void FFlatCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << + " {\n" + " int _slen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << ";\n"; + out << " int _trans"; + + if ( redFsm->anyConditions() ) + out << ", _cond"; + + out << ";\n"; + + out << + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_keys;\n" + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxIndex) << POINTER() << "_inds;\n"; + + if ( redFsm->anyConditions() ) { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxCond) << POINTER() << "_conds;\n" + " " << WIDE_ALPH_TYPE() << " _widec;\n"; + } + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " switch ( " << FSA() << "[" << vCS() << "] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] == 0 )\n" + " goto _again;\n" + "\n" + " switch ( " << TA() << "[_trans] ) {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " switch ( " << TSA() << "[" << vCS() << "] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " goto _eof_trans;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " switch ( " << EA() << "[" << vCS() << "] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/ragel/cdfflat.h b/ragel/cdfflat.h new file mode 100644 index 0000000..406bca9 --- /dev/null +++ b/ragel/cdfflat.h @@ -0,0 +1,75 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDFFLAT_H +#define _CDFFLAT_H + +#include <iostream> +#include "cdflat.h" + +/* Forwards. */ +struct CodeGenData; + +/* + * FFlatCodeGen + */ +class FFlatCodeGen : public FlatCodeGen +{ +protected: + FFlatCodeGen( ostream &out ) : FsmCodeGen(out), FlatCodeGen(out) {} + + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); +}; + +/* + * CFFlatCodeGen + */ +struct CFFlatCodeGen + : public FFlatCodeGen, public CCodeGen +{ + CFFlatCodeGen( ostream &out ) : + FsmCodeGen(out), FFlatCodeGen(out), CCodeGen(out) {} +}; + +/* + * DFFlatCodeGen + */ +struct DFFlatCodeGen + : public FFlatCodeGen, public DCodeGen +{ + DFFlatCodeGen( ostream &out ) : + FsmCodeGen(out), FFlatCodeGen(out), DCodeGen(out) {} +}; + +#endif diff --git a/ragel/cdfgoto.cpp b/ragel/cdfgoto.cpp new file mode 100644 index 0000000..3b229de --- /dev/null +++ b/ragel/cdfgoto.cpp @@ -0,0 +1,296 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cdfgoto.h" +#include "redfsm.h" +#include "gendata.h" +#include "bstmap.h" + +std::ostream &FGotoCodeGen::EXEC_ACTIONS() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* We are at the start of a glob, write the case. */ + out << "f" << redAct->actListId << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tgoto _again;\n"; + } + } + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &FGotoCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &FGotoCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &FGotoCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &FGotoCodeGen::FINISH_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* States that are final and have an out action need a case. */ + if ( st->eofAction != 0 ) { + /* Write the case label. */ + out << "\t\tcase " << st->id << ": "; + + /* Jump to the func. */ + out << "goto f" << st->eofAction->actListId << ";\n"; + } + } + + return out; +} + +unsigned int FGotoCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + return act; +} + +unsigned int FGotoCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + return act; +} + +unsigned int FGotoCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + return act; +} + +void FGotoCodeGen::writeData() +{ + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void FGotoCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << " {\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " int _ps = 0;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " switch ( " << FSA() << "[" << vCS() << "] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + out << + " switch ( " << vCS() << " ) {\n"; + STATE_GOTOS(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + TRANSITIONS() << + "\n"; + + if ( redFsm->anyRegActions() ) + EXEC_ACTIONS() << "\n"; + + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " switch ( " << TSA() << "[" << vCS() << "] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " switch ( " << vCS() << " ) {\n"; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << " case " << st->id << ": goto tr" << st->eofTrans->id << ";\n"; + } + + SWITCH_DEFAULT() << + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " switch ( " << EA() << "[" << vCS() << "] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/ragel/cdfgoto.h b/ragel/cdfgoto.h new file mode 100644 index 0000000..802522d --- /dev/null +++ b/ragel/cdfgoto.h @@ -0,0 +1,75 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDFGOTO_H +#define _CDFGOTO_H + +#include <iostream> +#include "cdgoto.h" + +/* Forwards. */ +struct CodeGenData; + + +/* + * class FGotoCodeGen + */ +class FGotoCodeGen : public GotoCodeGen +{ +public: + FGotoCodeGen( ostream &out ) : FsmCodeGen(out), GotoCodeGen(out) {} + + std::ostream &EXEC_ACTIONS(); + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &FINISH_CASES(); + std::ostream &EOF_ACTION_SWITCH(); + unsigned int TO_STATE_ACTION( RedStateAp *state ); + unsigned int FROM_STATE_ACTION( RedStateAp *state ); + unsigned int EOF_ACTION( RedStateAp *state ); + + virtual void writeData(); + virtual void writeExec(); +}; + +/* + * class CFGotoCodeGen + */ +struct CFGotoCodeGen + : public FGotoCodeGen, public CCodeGen +{ + CFGotoCodeGen( ostream &out ) : + FsmCodeGen(out), FGotoCodeGen(out), CCodeGen(out) {} +}; + +/* + * class DFGotoCodeGen + */ +struct DFGotoCodeGen + : public FGotoCodeGen, public DCodeGen +{ + DFGotoCodeGen( ostream &out ) : + FsmCodeGen(out), FGotoCodeGen(out), DCodeGen(out) {} +}; + +#endif diff --git a/ragel/cdflat.cpp b/ragel/cdflat.cpp new file mode 100644 index 0000000..7fa2a9d --- /dev/null +++ b/ragel/cdflat.cpp @@ -0,0 +1,851 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cdflat.h" +#include "redfsm.h" +#include "gendata.h" + +std::ostream &FlatCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + out << act; + return out; +} + +std::ostream &FlatCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + out << act; + return out; +} + +std::ostream &FlatCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + out << act; + return out; +} + +std::ostream &FlatCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + out << act; + return out; +} + +std::ostream &FlatCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &FlatCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &FlatCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, true, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &FlatCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &FlatCodeGen::FLAT_INDEX_OFFSET() +{ + out << "\t"; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the index offset ahead. */ + if ( st->transList != 0 ) + curIndOffset += keyOps->span( st->lowKey, st->highKey ); + + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + out << "\n"; + return out; +} + +std::ostream &FlatCodeGen::KEY_SPANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + unsigned long long span = 0; + if ( st->transList != 0 ) + span = keyOps->span( st->lowKey, st->highKey ); + out << span; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &FlatCodeGen::TO_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + TO_STATE_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &FlatCodeGen::FROM_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + FROM_STATE_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &FlatCodeGen::EOF_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + EOF_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &FlatCodeGen::EOF_TRANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + out << trans; + + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +std::ostream &FlatCodeGen::COND_KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit just cond low key and cond high key. */ + out << KEY( st->condLowKey ) << ", "; + out << KEY( st->condHighKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &FlatCodeGen::COND_KEY_SPANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + unsigned long long span = 0; + if ( st->condList != 0 ) + span = keyOps->span( st->condLowKey, st->condHighKey ); + out << span; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &FlatCodeGen::CONDS() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->condList != 0 ) { + /* Walk the singles. */ + unsigned long long span = keyOps->span( st->condLowKey, st->condHighKey ); + for ( unsigned long long pos = 0; pos < span; pos++ ) { + if ( st->condList[pos] != 0 ) + out << st->condList[pos]->condSpaceId + 1 << ", "; + else + out << "0, "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &FlatCodeGen::COND_INDEX_OFFSET() +{ + out << "\t"; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the index offset ahead. */ + if ( st->condList != 0 ) + curIndOffset += keyOps->span( st->condLowKey, st->condHighKey ); + } + out << "\n"; + return out; +} + + +std::ostream &FlatCodeGen::KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit just low key and high key. */ + out << KEY( st->lowKey ) << ", "; + out << KEY( st->highKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &FlatCodeGen::INDICIES() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->transList != 0 ) { + /* Walk the singles. */ + unsigned long long span = keyOps->span( st->lowKey, st->highKey ); + for ( unsigned long long pos = 0; pos < span; pos++ ) { + out << st->transList[pos]->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) + out << st->defTrans->id << ", "; + + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &FlatCodeGen::TRANS_TARGS() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalStates = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Save the position. Needed for eofTargs. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + out << trans->targ->id; + if ( t < redFsm->transSet.length()-1 ) { + out << ", "; + if ( ++totalStates % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + + +std::ostream &FlatCodeGen::TRANS_ACTIONS() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalAct = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + TRANS_ACTION( trans ); + if ( t < redFsm->transSet.length()-1 ) { + out << ", "; + if ( ++totalAct % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + +void FlatCodeGen::LOCATE_TRANS() +{ + out << + " _keys = " << ARR_OFF( K(), "(" + vCS() + "<<1)" ) << ";\n" + " _inds = " << ARR_OFF( I(), IO() + "[" + vCS() + "]" ) << ";\n" + "\n" + " _slen = " << SP() << "[" << vCS() << "];\n" + " _trans = _inds[ _slen > 0 && _keys[0] <=" << GET_WIDE_KEY() << " &&\n" + " " << GET_WIDE_KEY() << " <= _keys[1] ?\n" + " " << GET_WIDE_KEY() << " - _keys[0] : _slen ];\n" + "\n"; +} + +void FlatCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{" << vCS() << " = " << gotoDest << "; " << + CTRL_FLOW() << "goto _again;}"; +} + +void FlatCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; +} + +void FlatCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void FlatCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void FlatCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void FlatCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << ");"; +} + +void FlatCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = " << + callDest << "; " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + + +void FlatCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish, false ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + + +void FlatCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << TOP() << "];"; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false, false ); + ret << "}"; + } + + ret << CTRL_FLOW() << "goto _again;}"; +} + +void FlatCodeGen::BREAK( ostream &ret, int targState, bool csForced ) +{ + outLabelUsed = true; + ret << "{" << P() << "++; " << CTRL_FLOW() << "goto _out; }"; +} + +void FlatCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpan), CSP() ); + COND_KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCond), C() ); + CONDS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondIndexOffset), CO() ); + COND_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSpan), SP() ); + KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxFlatIndexOffset), IO() ); + FLAT_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void FlatCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << GET_KEY() << ";\n"; + + out << + " _keys = " << ARR_OFF( CK(), "(" + vCS() + "<<1)" ) << ";\n" + " _conds = " << ARR_OFF( C(), CO() + "[" + vCS() + "]" ) << ";\n" + "\n" + " _slen = " << CSP() << "[" << vCS() << "];\n" + " _cond = _slen > 0 && _keys[0] <=" << GET_WIDE_KEY() << " &&\n" + " " << GET_WIDE_KEY() << " <= _keys[1] ?\n" + " _conds[" << GET_WIDE_KEY() << " - _keys[0]] : 0;\n" + "\n"; + + out << + " switch ( _cond ) {\n"; + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " case " << condSpace->condSpaceId + 1 << ": {\n"; + out << TABS(2) << "_widec = " << CAST(WIDE_ALPH_TYPE()) << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "));\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(2) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } + + out << " }\n"; + out << " break;\n"; + } + + SWITCH_DEFAULT(); + + out << + " }\n"; +} + +void FlatCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << + " {\n" + " int _slen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << + ";\n" + " int _trans"; + + if ( redFsm->anyConditions() ) + out << ", _cond"; + out << ";\n"; + + if ( redFsm->anyToStateActions() || + redFsm->anyRegActions() || redFsm->anyFromStateActions() ) + { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxActArrItem) << POINTER() << "_acts;\n" + " " << UINT() << " _nacts;\n"; + } + + out << + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_keys;\n" + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxIndex) << POINTER() << "_inds;\n"; + + if ( redFsm->anyConditions() ) { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxCond) << POINTER() << "_conds;\n" + " " << WIDE_ALPH_TYPE() << " _widec;\n"; + } + + out << "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << ARR_OFF( A(), FSA() + "[" + vCS() + "]" ) << ";\n" + " _nacts = " << CAST(UINT()) << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] == 0 )\n" + " goto _again;\n" + "\n" + " _acts = " << ARR_OFF( A(), TA() + "[_trans]" ) << ";\n" + " _nacts = " << CAST(UINT()) << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *(_acts++) )\n {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << ARR_OFF( A(), TSA() + "[" + vCS() + "]" ) << ";\n" + " _nacts = " << CAST(UINT()) << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " goto _eof_trans;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxActArrItem) << + POINTER() << "__acts = " << + ARR_OFF( A(), EA() + "[" + vCS() + "]" ) << ";\n" + " " << UINT() << " __nacts = " << CAST(UINT()) << " *__acts++;\n" + " while ( __nacts-- > 0 ) {\n" + " switch ( *__acts++ ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/ragel/cdflat.h b/ragel/cdflat.h new file mode 100644 index 0000000..1a43e3e --- /dev/null +++ b/ragel/cdflat.h @@ -0,0 +1,108 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDFLAT_H +#define _CDFLAT_H + +#include <iostream> +#include "cdcodegen.h" + +/* Forwards. */ +struct CodeGenData; +struct NameInst; +struct RedTransAp; +struct RedStateAp; + +/* + * FlatCodeGen + */ +class FlatCodeGen : virtual public FsmCodeGen +{ +public: + FlatCodeGen( ostream &out ) : FsmCodeGen(out) {} + virtual ~FlatCodeGen() { } + +protected: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &FLAT_INDEX_OFFSET(); + std::ostream &KEY_SPANS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + void LOCATE_TRANS(); + + std::ostream &COND_INDEX_OFFSET(); + void COND_TRANSLATE(); + std::ostream &CONDS(); + std::ostream &COND_KEYS(); + std::ostream &COND_KEY_SPANS(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState, bool csForced ); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); +}; + +/* + * CFlatCodeGen + */ +struct CFlatCodeGen + : public FlatCodeGen, public CCodeGen +{ + CFlatCodeGen( ostream &out ) : + FsmCodeGen(out), FlatCodeGen(out), CCodeGen(out) {} +}; + +/* + * DFlatCodeGen + */ +struct DFlatCodeGen + : public FlatCodeGen, public DCodeGen +{ + DFlatCodeGen( ostream &out ) : + FsmCodeGen(out), FlatCodeGen(out), DCodeGen(out) {} +}; + +#endif diff --git a/ragel/cdftable.cpp b/ragel/cdftable.cpp new file mode 100644 index 0000000..cb5a4ff --- /dev/null +++ b/ragel/cdftable.cpp @@ -0,0 +1,436 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cdftable.h" +#include "redfsm.h" +#include "gendata.h" + +/* Determine if we should use indicies or not. */ +void FTabCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActListId) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActListId) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +std::ostream &FTabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &FTabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &FTabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + out << act; + return out; +} + + +/* Write out the function for a transition. */ +std::ostream &FTabCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + int action = 0; + if ( trans->action != 0 ) + action = trans->action->actListId+1; + out << action; + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &FTabCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &FTabCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &FTabCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &FTabCodeGen::ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +void FTabCodeGen::writeData() +{ + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + "\n"; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void FTabCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << + " {\n" + " int _klen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << + ";\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_keys;\n" + " int _trans;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + out << "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " switch ( " << FSA() << "[" << vCS() << "] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + out << "_match:\n"; + + if ( useIndicies ) + out << " _trans = " << I() << "[_trans];\n"; + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] == 0 )\n" + " goto _again;\n" + "\n" + " switch ( " << TA() << "[_trans] ) {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " switch ( " << TSA() << "[" << vCS() << "] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " goto _eof_trans;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " switch ( " << EA() << "[" << vCS() << "] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/ragel/cdftable.h b/ragel/cdftable.h new file mode 100644 index 0000000..d075723 --- /dev/null +++ b/ragel/cdftable.h @@ -0,0 +1,77 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDFTABLE_H +#define _CDFTABLE_H + +#include <iostream> +#include "cdtable.h" + +/* Forwards. */ +struct CodeGenData; + + +/* + * FTabCodeG\verb|e + */ +class FTabCodeGen : public TabCodeGen +{ +protected: + FTabCodeGen( ostream &out ) : FsmCodeGen(out), TabCodeGen(out) {} + + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + virtual void writeData(); + virtual void writeExec(); + virtual void calcIndexSize(); +}; + + +/* + * CFTabCodeGen + */ +struct CFTabCodeGen + : public FTabCodeGen, public CCodeGen +{ + CFTabCodeGen( ostream &out ) : + FsmCodeGen(out), FTabCodeGen(out), CCodeGen(out) {} +}; + +/* + * class DFTabCodeGen + */ +struct DFTabCodeGen + : public FTabCodeGen, public DCodeGen +{ + DFTabCodeGen( ostream &out ) : + FsmCodeGen(out), FTabCodeGen(out), DCodeGen(out) {} +}; + +#endif diff --git a/ragel/cdgoto.cpp b/ragel/cdgoto.cpp new file mode 100644 index 0000000..d6f6bdf --- /dev/null +++ b/ragel/cdgoto.cpp @@ -0,0 +1,801 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cdgoto.h" +#include "redfsm.h" +#include "bstmap.h" +#include "gendata.h" + +/* Emit the goto to take for a given transition. */ +std::ostream &GotoCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + out << TABS(level) << "goto tr" << trans->id << ";"; + return out; +} + +std::ostream &GotoCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &GotoCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &GotoCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, true, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &GotoCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +void GotoCodeGen::GOTO_HEADER( RedStateAp *state ) +{ + /* Label the state. */ + out << "case " << state->id << ":\n"; +} + + +void GotoCodeGen::emitSingleSwitch( RedStateAp *state ) +{ + /* Load up the singles. */ + int numSingles = state->outSingle.length(); + RedTransEl *data = state->outSingle.data; + + if ( numSingles == 1 ) { + /* If there is a single single key then write it out as an if. */ + out << "\tif ( " << GET_WIDE_KEY(state) << " == " << + KEY(data[0].lowKey) << " )\n\t\t"; + + /* Virtual function for writing the target of the transition. */ + TRANS_GOTO(data[0].value, 0) << "\n"; + } + else if ( numSingles > 1 ) { + /* Write out single keys in a switch if there is more than one. */ + out << "\tswitch( " << GET_WIDE_KEY(state) << " ) {\n"; + + /* Write out the single indicies. */ + for ( int j = 0; j < numSingles; j++ ) { + out << "\t\tcase " << KEY(data[j].lowKey) << ": "; + TRANS_GOTO(data[j].value, 0) << "\n"; + } + + /* Emits a default case for D code. */ + SWITCH_DEFAULT(); + + /* Close off the transition switch. */ + out << "\t}\n"; + } +} + +void GotoCodeGen::emitRangeBSearch( RedStateAp *state, int level, int low, int high ) +{ + /* Get the mid position, staying on the lower end of the range. */ + int mid = (low + high) >> 1; + RedTransEl *data = state->outRange.data; + + /* Determine if we need to look higher or lower. */ + bool anyLower = mid > low; + bool anyHigher = mid < high; + + /* Determine if the keys at mid are the limits of the alphabet. */ + bool limitLow = data[mid].lowKey == keyOps->minKey; + bool limitHigh = data[mid].highKey == keyOps->maxKey; + + if ( anyLower && anyHigher ) { + /* Can go lower and higher than mid. */ + out << TABS(level) << "if ( " << GET_WIDE_KEY(state) << " < " << + KEY(data[mid].lowKey) << " ) {\n"; + emitRangeBSearch( state, level+1, low, mid-1 ); + out << TABS(level) << "} else if ( " << GET_WIDE_KEY(state) << " > " << + KEY(data[mid].highKey) << " ) {\n"; + emitRangeBSearch( state, level+1, mid+1, high ); + out << TABS(level) << "} else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else if ( anyLower && !anyHigher ) { + /* Can go lower than mid but not higher. */ + out << TABS(level) << "if ( " << GET_WIDE_KEY(state) << " < " << + KEY(data[mid].lowKey) << " ) {\n"; + emitRangeBSearch( state, level+1, low, mid-1 ); + + /* if the higher is the highest in the alphabet then there is no + * sense testing it. */ + if ( limitHigh ) { + out << TABS(level) << "} else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + } + else if ( !anyLower && anyHigher ) { + /* Can go higher than mid but not lower. */ + out << TABS(level) << "if ( " << GET_WIDE_KEY(state) << " > " << + KEY(data[mid].highKey) << " ) {\n"; + emitRangeBSearch( state, level+1, mid+1, high ); + + /* If the lower end is the lowest in the alphabet then there is no + * sense testing it. */ + if ( limitLow ) { + out << TABS(level) << "} else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_WIDE_KEY(state) << " >= " << + KEY(data[mid].lowKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + } + else { + /* Cannot go higher or lower than mid. It's mid or bust. What + * tests to do depends on limits of alphabet. */ + if ( !limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << KEY(data[mid].lowKey) << " <= " << + GET_WIDE_KEY(state) << " && " << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else if ( limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else if ( !limitLow && limitHigh ) { + out << TABS(level) << "if ( " << KEY(data[mid].lowKey) << " <= " << + GET_WIDE_KEY(state) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + /* Both high and low are at the limit. No tests to do. */ + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + } +} + +void GotoCodeGen::STATE_GOTO_ERROR() +{ + /* Label the state and bail immediately. */ + outLabelUsed = true; + RedStateAp *state = redFsm->errState; + out << "case " << state->id << ":\n"; + out << " goto _out;\n"; +} + +void GotoCodeGen::COND_TRANSLATE( GenStateCond *stateCond, int level ) +{ + GenCondSpace *condSpace = stateCond->condSpace; + out << TABS(level) << "_widec = " << CAST(WIDE_ALPH_TYPE()) << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "));\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(level) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } +} + +void GotoCodeGen::emitCondBSearch( RedStateAp *state, int level, int low, int high ) +{ + /* Get the mid position, staying on the lower end of the range. */ + int mid = (low + high) >> 1; + GenStateCond **data = state->stateCondVect.data; + + /* Determine if we need to look higher or lower. */ + bool anyLower = mid > low; + bool anyHigher = mid < high; + + /* Determine if the keys at mid are the limits of the alphabet. */ + bool limitLow = data[mid]->lowKey == keyOps->minKey; + bool limitHigh = data[mid]->highKey == keyOps->maxKey; + + if ( anyLower && anyHigher ) { + /* Can go lower and higher than mid. */ + out << TABS(level) << "if ( " << GET_KEY() << " < " << + KEY(data[mid]->lowKey) << " ) {\n"; + emitCondBSearch( state, level+1, low, mid-1 ); + out << TABS(level) << "} else if ( " << GET_KEY() << " > " << + KEY(data[mid]->highKey) << " ) {\n"; + emitCondBSearch( state, level+1, mid+1, high ); + out << TABS(level) << "} else {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else if ( anyLower && !anyHigher ) { + /* Can go lower than mid but not higher. */ + out << TABS(level) << "if ( " << GET_KEY() << " < " << + KEY(data[mid]->lowKey) << " ) {\n"; + emitCondBSearch( state, level+1, low, mid-1 ); + + /* if the higher is the highest in the alphabet then there is no + * sense testing it. */ + if ( limitHigh ) { + out << TABS(level) << "} else {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + } + else if ( !anyLower && anyHigher ) { + /* Can go higher than mid but not lower. */ + out << TABS(level) << "if ( " << GET_KEY() << " > " << + KEY(data[mid]->highKey) << " ) {\n"; + emitCondBSearch( state, level+1, mid+1, high ); + + /* If the lower end is the lowest in the alphabet then there is no + * sense testing it. */ + if ( limitLow ) { + out << TABS(level) << "} else {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_KEY() << " >= " << + KEY(data[mid]->lowKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + } + else { + /* Cannot go higher or lower than mid. It's mid or bust. What + * tests to do depends on limits of alphabet. */ + if ( !limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << KEY(data[mid]->lowKey) << " <= " << + GET_KEY() << " && " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else if ( limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else if ( !limitLow && limitHigh ) { + out << TABS(level) << "if ( " << KEY(data[mid]->lowKey) << " <= " << + GET_KEY() << " )\n {"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else { + /* Both high and low are at the limit. No tests to do. */ + COND_TRANSLATE(data[mid], level); + } + } +} + +std::ostream &GotoCodeGen::STATE_GOTOS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st == redFsm->errState ) + STATE_GOTO_ERROR(); + else { + /* Writing code above state gotos. */ + GOTO_HEADER( st ); + + if ( st->stateCondVect.length() > 0 ) { + out << " _widec = " << GET_KEY() << ";\n"; + emitCondBSearch( st, 1, 0, st->stateCondVect.length() - 1 ); + } + + /* Try singles. */ + if ( st->outSingle.length() > 0 ) + emitSingleSwitch( st ); + + /* Default case is to binary search for the ranges, if that fails then */ + if ( st->outRange.length() > 0 ) + emitRangeBSearch( st, 1, 0, st->outRange.length() - 1 ); + + /* Write the default transition. */ + TRANS_GOTO( st->defTrans, 1 ) << "\n"; + } + } + return out; +} + +std::ostream &GotoCodeGen::TRANSITIONS() +{ + /* Emit any transitions that have functions and that go to + * this state. */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* Write the label for the transition so it can be jumped to. */ + out << " tr" << trans->id << ": "; + + /* Destination state. */ + if ( trans->action != 0 && trans->action->anyCurStateRef() ) + out << "_ps = " << vCS() << ";"; + out << vCS() << " = " << trans->targ->id << "; "; + + if ( trans->action != 0 ) { + /* Write out the transition func. */ + out << "goto f" << trans->action->actListId << ";\n"; + } + else { + /* No code to execute, just loop around. */ + out << "goto _again;\n"; + } + } + return out; +} + +std::ostream &GotoCodeGen::EXEC_FUNCS() +{ + /* Make labels that set acts and jump to execFuncs. Loop func indicies. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + out << " f" << redAct->actListId << ": " << + "_acts = " << ARR_OFF(A(), itoa( redAct->location+1 ) ) << ";" + " goto execFuncs;\n"; + } + } + + out << + "\n" + "execFuncs:\n" + " _nacts = *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + " goto _again;\n"; + return out; +} + +unsigned int GotoCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + return act; +} + +unsigned int GotoCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + return act; +} + +unsigned int GotoCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + return act; +} + +std::ostream &GotoCodeGen::TO_STATE_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = TO_STATE_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &GotoCodeGen::FROM_STATE_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = FROM_STATE_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &GotoCodeGen::EOF_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = EOF_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &GotoCodeGen::FINISH_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* States that are final and have an out action need a case. */ + if ( st->eofAction != 0 ) { + /* Write the case label. */ + out << "\t\tcase " << st->id << ": "; + + /* Write the goto func. */ + out << "goto f" << st->eofAction->actListId << ";\n"; + } + } + + return out; +} + +void GotoCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{" << vCS() << " = " << gotoDest << "; " << + CTRL_FLOW() << "goto _again;}"; +} + +void GotoCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; +} + +void GotoCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void GotoCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void GotoCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void GotoCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << ");"; +} + +void GotoCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = " << + callDest << "; " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void GotoCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish, false ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void GotoCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << TOP() << "];"; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false, false ); + ret << "}"; + } + + ret << CTRL_FLOW() << "goto _again;}"; +} + +void GotoCodeGen::BREAK( ostream &ret, int targState, bool csForced ) +{ + outLabelUsed = true; + ret << "{" << P() << "++; " << CTRL_FLOW() << "goto _out; }"; +} + +void GotoCodeGen::writeData() +{ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void GotoCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << " {\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " int _ps = 0;\n"; + + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxActArrItem) << POINTER() << "_acts;\n" + " " << UINT() << " _nacts;\n"; + } + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + out << "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << ARR_OFF( A(), FSA() + "[" + vCS() + "]" ) << ";\n" + " _nacts = " << CAST(UINT()) << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + out << + " switch ( " << vCS() << " ) {\n"; + STATE_GOTOS(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + TRANSITIONS() << + "\n"; + + if ( redFsm->anyRegActions() ) + EXEC_FUNCS() << "\n"; + + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << ARR_OFF( A(), TSA() + "[" + vCS() + "]" ) << ";\n" + " _nacts = " << CAST(UINT()) << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " switch ( " << vCS() << " ) {\n"; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << " case " << st->id << ": goto tr" << st->eofTrans->id << ";\n"; + } + + SWITCH_DEFAULT() << + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxActArrItem) << + POINTER() << "__acts = " << + ARR_OFF( A(), EA() + "[" + vCS() + "]" ) << ";\n" + " " << UINT() << " __nacts = " << CAST(UINT()) << " *__acts++;\n" + " while ( __nacts-- > 0 ) {\n" + " switch ( *__acts++ ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/ragel/cdgoto.h b/ragel/cdgoto.h new file mode 100644 index 0000000..7de8e9b --- /dev/null +++ b/ragel/cdgoto.h @@ -0,0 +1,109 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDGOTO_H +#define _CDGOTO_H + +#include <iostream> +#include "cdcodegen.h" + +/* Forwards. */ +struct CodeGenData; +struct NameInst; +struct RedTransAp; +struct RedStateAp; +struct GenStateCond; + +/* + * Goto driven fsm. + */ +class GotoCodeGen : virtual public FsmCodeGen +{ +public: + GotoCodeGen( ostream &out ) : FsmCodeGen(out) {} + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + std::ostream &STATE_GOTOS(); + std::ostream &TRANSITIONS(); + std::ostream &EXEC_FUNCS(); + std::ostream &FINISH_CASES(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState, bool csForced ); + + virtual unsigned int TO_STATE_ACTION( RedStateAp *state ); + virtual unsigned int FROM_STATE_ACTION( RedStateAp *state ); + virtual unsigned int EOF_ACTION( RedStateAp *state ); + + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + + void COND_TRANSLATE( GenStateCond *stateCond, int level ); + void emitCondBSearch( RedStateAp *state, int level, int low, int high ); + void STATE_CONDS( RedStateAp *state, bool genDefault ); + + virtual std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + + void emitSingleSwitch( RedStateAp *state ); + void emitRangeBSearch( RedStateAp *state, int level, int low, int high ); + + /* Called from STATE_GOTOS just before writing the gotos */ + virtual void GOTO_HEADER( RedStateAp *state ); + virtual void STATE_GOTO_ERROR(); + + virtual void writeData(); + virtual void writeExec(); +}; + +/* + * class CGotoCodeGen + */ +struct CGotoCodeGen + : public GotoCodeGen, public CCodeGen +{ + CGotoCodeGen( ostream &out ) : + FsmCodeGen(out), GotoCodeGen(out), CCodeGen(out) {} +}; + +/* + * class DGotoCodeGen + */ +struct DGotoCodeGen + : public GotoCodeGen, public DCodeGen +{ + DGotoCodeGen( ostream &out ) : + FsmCodeGen(out), GotoCodeGen(out), DCodeGen(out) {} +}; + +#endif diff --git a/ragel/cdipgoto.cpp b/ragel/cdipgoto.cpp new file mode 100644 index 0000000..298bf36 --- /dev/null +++ b/ragel/cdipgoto.cpp @@ -0,0 +1,444 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cdipgoto.h" +#include "redfsm.h" +#include "gendata.h" +#include "bstmap.h" + +bool IpGotoCodeGen::useAgainLabel() +{ + return redFsm->anyRegActionRets() || + redFsm->anyRegActionByValControl() || + redFsm->anyRegNextStmt(); +} + +void IpGotoCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{" << CTRL_FLOW() << "goto st" << gotoDest << ";}"; +} + +void IpGotoCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << targState << + "; " << CTRL_FLOW() << "goto st" << callDest << ";}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void IpGotoCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << targState << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void IpGotoCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << TOP() << "];"; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false, false ); + ret << "}"; + } + + ret << CTRL_FLOW() << "goto _again;}"; +} + +void IpGotoCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; +} + +void IpGotoCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void IpGotoCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << ");"; +} + +void IpGotoCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void IpGotoCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << targState; +} + +void IpGotoCodeGen::BREAK( ostream &ret, int targState, bool csForced ) +{ + outLabelUsed = true; + ret << "{" << P() << "++; "; + if ( !csForced ) + ret << vCS() << " = " << targState << "; "; + ret << CTRL_FLOW() << "goto _out;}"; +} + +bool IpGotoCodeGen::IN_TRANS_ACTIONS( RedStateAp *state ) +{ + bool anyWritten = false; + + /* Emit any transitions that have actions and that go to this state. */ + for ( int it = 0; it < state->numInTrans; it++ ) { + RedTransAp *trans = state->inTrans[it]; + if ( trans->action != 0 && trans->labelNeeded ) { + /* Remember that we wrote an action so we know to write the + * line directive for going back to the output. */ + anyWritten = true; + + /* Write the label for the transition so it can be jumped to. */ + out << "tr" << trans->id << ":\n"; + + /* If the action contains a next, then we must preload the current + * state since the action may or may not set it. */ + if ( trans->action->anyNextStmt() ) + out << " " << vCS() << " = " << trans->targ->id << ";\n"; + + /* Write each action in the list. */ + for ( GenActionTable::Iter item = trans->action->key; item.lte(); item++ ) { + ACTION( out, item->value, trans->targ->id, false, + trans->action->anyNextStmt() ); + } + + /* If the action contains a next then we need to reload, otherwise + * jump directly to the target state. */ + if ( trans->action->anyNextStmt() ) + out << "\tgoto _again;\n"; + else + out << "\tgoto st" << trans->targ->id << ";\n"; + } + } + + return anyWritten; +} + +/* Called from GotoCodeGen::STATE_GOTOS just before writing the gotos for each + * state. */ +void IpGotoCodeGen::GOTO_HEADER( RedStateAp *state ) +{ + bool anyWritten = IN_TRANS_ACTIONS( state ); + + if ( state->labelNeeded ) + out << "st" << state->id << ":\n"; + + if ( state->toStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->toStateAction->key; item.lte(); item++ ) { + ACTION( out, item->value, state->id, false, + state->toStateAction->anyNextStmt() ); + } + } + + /* Advance and test buffer pos. */ + if ( state->labelNeeded ) { + if ( !noEnd ) { + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _test_eof" << state->id << ";\n"; + } + else { + out << + " " << P() << " += 1;\n"; + } + } + + /* Give the state a switch case. */ + out << "case " << state->id << ":\n"; + + if ( state->fromStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->fromStateAction->key; item.lte(); item++ ) { + ACTION( out, item->value, state->id, false, + state->fromStateAction->anyNextStmt() ); + } + } + + if ( anyWritten ) + genLineDirective( out ); + + /* Record the prev state if necessary. */ + if ( state->anyRegCurStateRef() ) + out << " _ps = " << state->id << ";\n"; +} + +void IpGotoCodeGen::STATE_GOTO_ERROR() +{ + /* In the error state we need to emit some stuff that usually goes into + * the header. */ + RedStateAp *state = redFsm->errState; + bool anyWritten = IN_TRANS_ACTIONS( state ); + + /* No case label needed since we don't switch on the error state. */ + if ( anyWritten ) + genLineDirective( out ); + + if ( state->labelNeeded ) + out << "st" << state->id << ":\n"; + + /* Break out here. */ + outLabelUsed = true; + out << vCS() << " = " << state->id << ";\n"; + out << " goto _out;\n"; +} + + +/* Emit the goto to take for a given transition. */ +std::ostream &IpGotoCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + if ( trans->action != 0 ) { + /* Go to the transition which will go to the state. */ + out << TABS(level) << "goto tr" << trans->id << ";"; + } + else { + /* Go directly to the target state. */ + out << TABS(level) << "goto st" << trans->targ->id << ";"; + } + return out; +} + +std::ostream &IpGotoCodeGen::EXIT_STATES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->outNeeded ) { + testEofUsed = true; + out << " _test_eof" << st->id << ": " << vCS() << " = " << + st->id << "; goto _test_eof; \n"; + } + } + return out; +} + +std::ostream &IpGotoCodeGen::AGAIN_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + out << + " case " << st->id << ": goto st" << st->id << ";\n"; + } + return out; +} + +std::ostream &IpGotoCodeGen::FINISH_CASES() +{ + bool anyWritten = false; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofAction != 0 ) { + if ( st->eofAction->eofRefs == 0 ) + st->eofAction->eofRefs = new IntSet; + st->eofAction->eofRefs->insert( st->id ); + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << " case " << st->id << ": goto tr" << st->eofTrans->id << ";\n"; + } + + for ( GenActionTableMap::Iter act = redFsm->actionMap; act.lte(); act++ ) { + if ( act->eofRefs != 0 ) { + for ( IntSet::Iter pst = *act->eofRefs; pst.lte(); pst++ ) + out << " case " << *pst << ": \n"; + + /* Remember that we wrote a trans so we know to write the + * line directive for going back to the output. */ + anyWritten = true; + + /* Write each action in the eof action list. */ + for ( GenActionTable::Iter item = act->key; item.lte(); item++ ) + ACTION( out, item->value, STATE_ERR_STATE, true, false ); + out << "\tbreak;\n"; + } + } + + if ( anyWritten ) + genLineDirective( out ); + return out; +} + +void IpGotoCodeGen::setLabelsNeeded( GenInlineList *inlineList ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Goto: case GenInlineItem::Call: { + /* Mark the target as needing a label. */ + item->targState->labelNeeded = true; + break; + } + default: break; + } + + if ( item->children != 0 ) + setLabelsNeeded( item->children ); + } +} + +/* Set up labelNeeded flag for each state. */ +void IpGotoCodeGen::setLabelsNeeded() +{ + /* If we use the _again label, then we the _again switch, which uses all + * labels. */ + if ( useAgainLabel() ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = true; + } + else { + /* Do not use all labels by default, init all labelNeeded vars to false. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = false; + + /* Walk all transitions and set only those that have targs. */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* If there is no action with a next statement, then the label will be + * needed. */ + if ( trans->action == 0 || !trans->action->anyNextStmt() ) + trans->targ->labelNeeded = true; + + /* Need labels for states that have goto or calls in action code + * invoked on characters (ie, not from out action code). */ + if ( trans->action != 0 ) { + /* Loop the actions. */ + for ( GenActionTable::Iter act = trans->action->key; act.lte(); act++ ) { + /* Get the action and walk it's tree. */ + setLabelsNeeded( act->value->inlineList ); + } + } + } + } + + if ( !noEnd ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st != redFsm->errState ) + st->outNeeded = st->labelNeeded; + } + } +} + +void IpGotoCodeGen::writeData() +{ + STATE_IDS(); +} + +void IpGotoCodeGen::writeExec() +{ + /* Must set labels immediately before writing because we may depend on the + * noend write option. */ + setLabelsNeeded(); + testEofUsed = false; + outLabelUsed = false; + + out << " {\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " int _ps = 0;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( useAgainLabel() ) { + out << + " goto _resume;\n" + "\n" + "_again:\n" + " switch ( " << vCS() << " ) {\n"; + AGAIN_CASES() << + " default: break;\n" + " }\n" + "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + else { + out << + " " << P() << " += 1;\n"; + } + + out << "_resume:\n"; + } + + out << + " switch ( " << vCS() << " )\n {\n"; + STATE_GOTOS(); + SWITCH_DEFAULT() << + " }\n"; + EXIT_STATES() << + "\n"; + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n" + " switch ( " << vCS() << " ) {\n"; + FINISH_CASES(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << + " }\n"; +} diff --git a/ragel/cdipgoto.h b/ragel/cdipgoto.h new file mode 100644 index 0000000..cd5107e --- /dev/null +++ b/ragel/cdipgoto.h @@ -0,0 +1,95 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDIPGOTO_H +#define _CDIPGOTO_H + +#include <iostream> +#include "cdgoto.h" + +/* Forwards. */ +struct CodeGenData; + +/* + * class FGotoCodeGen + */ +class IpGotoCodeGen : public GotoCodeGen +{ +public: + IpGotoCodeGen( ostream &out ) : FsmCodeGen(out), GotoCodeGen(out) {} + + std::ostream &EXIT_STATES(); + std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + std::ostream &FINISH_CASES(); + std::ostream &AGAIN_CASES(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void RET( ostream &ret, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void BREAK( ostream &ret, int targState, bool csForced ); + + virtual void writeData(); + virtual void writeExec(); + +protected: + bool useAgainLabel(); + + /* Called from GotoCodeGen::STATE_GOTOS just before writing the gotos for + * each state. */ + bool IN_TRANS_ACTIONS( RedStateAp *state ); + void GOTO_HEADER( RedStateAp *state ); + void STATE_GOTO_ERROR(); + + /* Set up labelNeeded flag for each state. */ + void setLabelsNeeded( GenInlineList *inlineList ); + void setLabelsNeeded(); +}; + + +/* + * class CIpGotoCodeGen + */ +struct CIpGotoCodeGen + : public IpGotoCodeGen, public CCodeGen +{ + CIpGotoCodeGen( ostream &out ) : + FsmCodeGen(out), IpGotoCodeGen(out), CCodeGen(out) {} +}; + +/* + * class DIpGotoCodeGen + */ +struct DIpGotoCodeGen + : public IpGotoCodeGen, public DCodeGen +{ + DIpGotoCodeGen( ostream &out ) : + FsmCodeGen(out), IpGotoCodeGen(out), DCodeGen(out) {} +}; + +#endif diff --git a/ragel/cdsplit.cpp b/ragel/cdsplit.cpp new file mode 100644 index 0000000..f494b73 --- /dev/null +++ b/ragel/cdsplit.cpp @@ -0,0 +1,526 @@ +/* + * Copyright 2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + + +#include "ragel.h" +#include "cdsplit.h" +#include "gendata.h" +#include <assert.h> + +using std::ostream; +using std::ios; +using std::endl; + +/* Emit the goto to take for a given transition. */ +std::ostream &SplitCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + if ( trans->targ->partition == currentPartition ) { + if ( trans->action != 0 ) { + /* Go to the transition which will go to the state. */ + out << TABS(level) << "goto tr" << trans->id << ";"; + } + else { + /* Go directly to the target state. */ + out << TABS(level) << "goto st" << trans->targ->id << ";"; + } + } + else { + if ( trans->action != 0 ) { + /* Go to the transition which will go to the state. */ + out << TABS(level) << "goto ptr" << trans->id << ";"; + trans->partitionBoundary = true; + } + else { + /* Go directly to the target state. */ + out << TABS(level) << "goto pst" << trans->targ->id << ";"; + trans->targ->partitionBoundary = true; + } + } + return out; +} + +/* Called from before writing the gotos for each state. */ +void SplitCodeGen::GOTO_HEADER( RedStateAp *state, bool stateInPartition ) +{ + bool anyWritten = IN_TRANS_ACTIONS( state ); + + if ( state->labelNeeded ) + out << "st" << state->id << ":\n"; + + if ( state->toStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->toStateAction->key; item.lte(); item++ ) { + ACTION( out, item->value, state->id, false, + state->toStateAction->anyNextStmt() ); + } + } + + /* Advance and test buffer pos. */ + if ( state->labelNeeded ) { + if ( !noEnd ) { + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _out" << state->id << ";\n"; + } + else { + out << + " " << P() << " += 1;\n"; + } + } + + /* Give the state a switch case. */ + out << "case " << state->id << ":\n"; + + if ( state->fromStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->fromStateAction->key; item.lte(); item++ ) { + ACTION( out, item->value, state->id, false, + state->fromStateAction->anyNextStmt() ); + } + } + + if ( anyWritten ) + genLineDirective( out ); + + /* Record the prev state if necessary. */ + if ( state->anyRegCurStateRef() ) + out << " _ps = " << state->id << ";\n"; +} + +std::ostream &SplitCodeGen::STATE_GOTOS( int partition ) +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->partition == partition ) { + if ( st == redFsm->errState ) + STATE_GOTO_ERROR(); + else { + /* We call into the base of the goto which calls back into us + * using virtual functions. Set the current partition rather + * than coding parameter passing throughout. */ + currentPartition = partition; + + /* Writing code above state gotos. */ + GOTO_HEADER( st, st->partition == partition ); + + if ( st->stateCondVect.length() > 0 ) { + out << " _widec = " << GET_KEY() << ";\n"; + emitCondBSearch( st, 1, 0, st->stateCondVect.length() - 1 ); + } + + /* Try singles. */ + if ( st->outSingle.length() > 0 ) + emitSingleSwitch( st ); + + /* Default case is to binary search for the ranges, if that fails then */ + if ( st->outRange.length() > 0 ) + emitRangeBSearch( st, 1, 0, st->outRange.length() - 1 ); + + /* Write the default transition. */ + TRANS_GOTO( st->defTrans, 1 ) << "\n"; + } + } + } + return out; +} + + +std::ostream &SplitCodeGen::PART_TRANS( int partition ) +{ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + if ( trans->partitionBoundary ) { + out << + "ptr" << trans->id << ":\n"; + + if ( trans->action != 0 ) { + /* If the action contains a next, then we must preload the current + * state since the action may or may not set it. */ + if ( trans->action->anyNextStmt() ) + out << " " << vCS() << " = " << trans->targ->id << ";\n"; + + /* Write each action in the list. */ + for ( GenActionTable::Iter item = trans->action->key; item.lte(); item++ ) { + ACTION( out, item->value, trans->targ->id, false, + trans->action->anyNextStmt() ); + } + } + + out << + " goto pst" << trans->targ->id << ";\n"; + trans->targ->partitionBoundary = true; + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->partitionBoundary ) { + out << + " pst" << st->id << ":\n" + " " << vCS() << " = " << st->id << ";\n"; + + if ( st->toStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + for ( GenActionTable::Iter item = st->toStateAction->key; item.lte(); item++ ) { + ACTION( out, item->value, st->id, false, + st->toStateAction->anyNextStmt() ); + } + genLineDirective( out ); + } + + ptOutLabelUsed = true; + out << " goto _pt_out; \n"; + } + } + return out; +} + +std::ostream &SplitCodeGen::EXIT_STATES( int partition ) +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->partition == partition && st->outNeeded ) { + outLabelUsed = true; + out << " _out" << st->id << ": " << vCS() << " = " << + st->id << "; goto _out; \n"; + } + } + return out; +} + + +std::ostream &SplitCodeGen::PARTITION( int partition ) +{ + outLabelUsed = false; + ptOutLabelUsed = false; + + /* Initialize the partition boundaries, which get set during the writing + * of states. After the state writing we will */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + trans->partitionBoundary = false; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->partitionBoundary = false; + + out << " " << ALPH_TYPE() << " *p = *_pp, *pe = *_ppe;\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " int _ps = 0;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( useAgainLabel() ) { + out << + " goto _resume;\n" + "\n" + "_again:\n" + " switch ( " << vCS() << " ) {\n"; + AGAIN_CASES() << + " default: break;\n" + " }\n" + "\n"; + + + if ( !noEnd ) { + outLabelUsed = true; + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _out;\n"; + + } + else { + out << + " " << P() << " += 1;\n"; + } + + out << + "_resume:\n"; + } + + out << + " switch ( " << vCS() << " )\n {\n"; + STATE_GOTOS( partition ); + SWITCH_DEFAULT() << + " }\n"; + PART_TRANS( partition ); + EXIT_STATES( partition ); + + if ( outLabelUsed ) { + out << + "\n" + " _out:\n" + " *_pp = p;\n" + " *_ppe = pe;\n" + " return 0;\n"; + } + + if ( ptOutLabelUsed ) { + out << + "\n" + " _pt_out:\n" + " *_pp = p;\n" + " *_ppe = pe;\n" + " return 1;\n"; + } + + return out; +} + +std::ostream &SplitCodeGen::PART_MAP() +{ + int *partMap = new int[redFsm->stateList.length()]; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + partMap[st->id] = st->partition; + + out << "\t"; + int totalItem = 0; + for ( int i = 0; i < redFsm->stateList.length(); i++ ) { + out << partMap[i]; + if ( i != redFsm->stateList.length() - 1 ) { + out << ", "; + if ( ++totalItem % IALL == 0 ) + out << "\n\t"; + } + } + + delete[] partMap; + return out; +} + +void SplitCodeGen::writeData() +{ + out << + "static const int " << START() << " = " << START_STATE_ID() << ";\n" + "\n"; + + if ( !noFinal ) { + out << + "static const int " << FIRST_FINAL() << " = " << FIRST_FINAL_STATE() << ";\n" + "\n"; + } + + if ( !noError ) { + out << + "static const int " << ERROR() << " = " << ERROR_STATE() << ";\n" + "\n"; + } + + + OPEN_ARRAY( ARRAY_TYPE(numSplitPartitions), PM() ); + PART_MAP(); + CLOSE_ARRAY() << + "\n"; + + for ( int p = 0; p < redFsm->nParts; p++ ) { + out << "int partition" << p << "( " << ALPH_TYPE() << " **_pp, " << ALPH_TYPE() << + " **_ppe, struct " << FSM_NAME() << " *fsm );\n"; + } + out << "\n"; +} + +std::ostream &SplitCodeGen::ALL_PARTITIONS() +{ + /* compute the format string. */ + int width = 0, high = redFsm->nParts - 1; + while ( high > 0 ) { + width++; + high /= 10; + } + assert( width <= 8 ); + char suffFormat[] = "_%6.6d.c"; + suffFormat[2] = suffFormat[4] = ( '0' + width ); + + for ( int p = 0; p < redFsm->nParts; p++ ) { + char suffix[10]; + sprintf( suffix, suffFormat, p ); + const char *fn = fileNameFromStem( sourceFileName, suffix ); + const char *include = fileNameFromStem( sourceFileName, ".h" ); + + /* Create the filter on the output and open it. */ + output_filter *partFilter = new output_filter( fn ); + partFilter->open( fn, ios::out|ios::trunc ); + if ( !partFilter->is_open() ) { + error() << "error opening " << fn << " for writing" << endl; + exit(1); + } + + /* Attach the new file to the output stream. */ + std::streambuf *prev_rdbuf = out.rdbuf( partFilter ); + + out << + "#include \"" << include << "\"\n" + "int partition" << p << "( " << ALPH_TYPE() << " **_pp, " << ALPH_TYPE() << + " **_ppe, struct " << FSM_NAME() << " *fsm )\n" + "{\n"; + PARTITION( p ) << + "}\n\n"; + out.flush(); + + /* Fix the output stream. */ + out.rdbuf( prev_rdbuf ); + } + return out; +} + + +void SplitCodeGen::writeExec() +{ + /* Must set labels immediately before writing because we may depend on the + * noend write option. */ + setLabelsNeeded(); + out << + " {\n" + " int _stat = 0;\n"; + + if ( !noEnd ) { + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _out;\n"; + } + + out << " goto _resume;\n"; + + /* In this reentry, to-state actions have already been executed on the + * partition-switch exit from the last partition. */ + out << "_reenter:\n"; + + if ( !noEnd ) { + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _out;\n"; + } + else { + out << + " " << P() << " += 1;\n"; + } + + out << "_resume:\n"; + + out << + " switch ( " << PM() << "[" << vCS() << "] ) {\n"; + for ( int p = 0; p < redFsm->nParts; p++ ) { + out << + " case " << p << ":\n" + " _stat = partition" << p << "( &p, &pe, fsm );\n" + " break;\n"; + } + out << + " }\n" + " if ( _stat )\n" + " goto _reenter;\n"; + + if ( !noEnd ) + out << " _out: {}\n"; + + out << + " }\n"; + + ALL_PARTITIONS(); +} + +void SplitCodeGen::setLabelsNeeded( RedStateAp *fromState, GenInlineList *inlineList ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Goto: case GenInlineItem::Call: { + /* In split code gen we only need labels for transitions across + * partitions. */ + if ( fromState->partition == item->targState->partition ){ + /* Mark the target as needing a label. */ + item->targState->labelNeeded = true; + } + break; + } + default: break; + } + + if ( item->children != 0 ) + setLabelsNeeded( fromState, item->children ); + } +} + +void SplitCodeGen::setLabelsNeeded( RedStateAp *fromState, RedTransAp *trans ) +{ + /* In the split code gen we don't need labels for transitions across + * partitions. */ + if ( fromState->partition == trans->targ->partition ) { + /* If there is no action with a next statement, then the label will be + * needed. */ + trans->labelNeeded = true; + if ( trans->action == 0 || !trans->action->anyNextStmt() ) + trans->targ->labelNeeded = true; + } + + /* Need labels for states that have goto or calls in action code + * invoked on characters (ie, not from out action code). */ + if ( trans->action != 0 ) { + /* Loop the actions. */ + for ( GenActionTable::Iter act = trans->action->key; act.lte(); act++ ) { + /* Get the action and walk it's tree. */ + setLabelsNeeded( fromState, act->value->inlineList ); + } + } +} + +/* Set up labelNeeded flag for each state. */ +void SplitCodeGen::setLabelsNeeded() +{ + /* If we use the _again label, then we the _again switch, which uses all + * labels. */ + if ( useAgainLabel() ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = true; + } + else { + /* Do not use all labels by default, init all labelNeeded vars to false. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = false; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + trans->labelNeeded = false; + + /* Walk all transitions and set only those that have targs. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + for ( RedTransList::Iter tel = st->outRange; tel.lte(); tel++ ) + setLabelsNeeded( st, tel->value ); + + for ( RedTransList::Iter tel = st->outSingle; tel.lte(); tel++ ) + setLabelsNeeded( st, tel->value ); + + if ( st->defTrans != 0 ) + setLabelsNeeded( st, st->defTrans ); + } + } + + if ( !noEnd ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->outNeeded = st->labelNeeded; + } + else { + if ( redFsm->errState != 0 ) + redFsm->errState->outNeeded = true; + + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* Any state with a transition in that has a break will need an + * out label. */ + if ( trans->action != 0 && trans->action->anyBreakStmt() ) + trans->targ->outNeeded = true; + } + } +} + diff --git a/ragel/cdsplit.h b/ragel/cdsplit.h new file mode 100644 index 0000000..c00fc90 --- /dev/null +++ b/ragel/cdsplit.h @@ -0,0 +1,70 @@ +/* + * Copyright 2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDSPLIT_H +#define _CDSPLIT_H + +#include "cdipgoto.h" + +class SplitCodeGen : public IpGotoCodeGen +{ +public: + SplitCodeGen( ostream &out ) : FsmCodeGen(out), IpGotoCodeGen(out) {} + + bool ptOutLabelUsed; + + std::ostream &PART_MAP(); + std::ostream &EXIT_STATES( int partition ); + std::ostream &PART_TRANS( int partition ); + std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + void GOTO_HEADER( RedStateAp *state, bool stateInPartition ); + std::ostream &STATE_GOTOS( int partition ); + std::ostream &PARTITION( int partition ); + std::ostream &ALL_PARTITIONS(); + void writeData(); + void writeExec(); + void writeParts(); + + void setLabelsNeeded( RedStateAp *fromState, GenInlineList *inlineList ); + void setLabelsNeeded( RedStateAp *fromState, RedTransAp *trans ); + void setLabelsNeeded(); + + int currentPartition; +}; + +struct CSplitCodeGen + : public SplitCodeGen, public CCodeGen +{ + CSplitCodeGen( ostream &out ) : + FsmCodeGen(out), SplitCodeGen(out), CCodeGen(out) {} +}; + +/* + * class DIpGotoCodeGen + */ +struct DSplitCodeGen + : public SplitCodeGen, public DCodeGen +{ + DSplitCodeGen( ostream &out ) : + FsmCodeGen(out), SplitCodeGen(out), DCodeGen(out) {} +}; + +#endif diff --git a/ragel/cdtable.cpp b/ragel/cdtable.cpp new file mode 100644 index 0000000..827d3ad --- /dev/null +++ b/ragel/cdtable.cpp @@ -0,0 +1,1096 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cdtable.h" +#include "redfsm.h" +#include "gendata.h" + +/* Determine if we should use indicies or not. */ +void TabCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActionLoc) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActionLoc) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +std::ostream &TabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + out << act; + return out; +} + +std::ostream &TabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + out << act; + return out; +} + +std::ostream &TabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + out << act; + return out; +} + + +std::ostream &TabCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + out << act; + return out; +} + +std::ostream &TabCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &TabCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &TabCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, true, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &TabCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &TabCodeGen::COND_OFFSETS() +{ + out << "\t"; + int totalStateNum = 0, curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + out << curKeyOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the key offset ahead. */ + curKeyOffset += st->stateCondList.length(); + } + out << "\n"; + return out; +} + +std::ostream &TabCodeGen::KEY_OFFSETS() +{ + out << "\t"; + int totalStateNum = 0, curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + out << curKeyOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the key offset ahead. */ + curKeyOffset += st->outSingle.length() + st->outRange.length()*2; + } + out << "\n"; + return out; +} + + +std::ostream &TabCodeGen::INDEX_OFFSETS() +{ + out << "\t"; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the index offset ahead. */ + curIndOffset += st->outSingle.length() + st->outRange.length(); + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + out << "\n"; + return out; +} + +std::ostream &TabCodeGen::COND_LENS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + out << st->stateCondList.length(); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +std::ostream &TabCodeGen::SINGLE_LENS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + out << st->outSingle.length(); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &TabCodeGen::RANGE_LENS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit length of range index. */ + out << st->outRange.length(); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &TabCodeGen::TO_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + TO_STATE_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &TabCodeGen::FROM_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + FROM_STATE_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &TabCodeGen::EOF_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + EOF_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &TabCodeGen::EOF_TRANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + out << trans; + + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +std::ostream &TabCodeGen::COND_KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Lower key. */ + out << KEY( sc->lowKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + + /* Upper key. */ + out << KEY( sc->highKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &TabCodeGen::COND_SPACES() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Cond Space id. */ + out << sc->condSpace->condSpaceId << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &TabCodeGen::KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + out << KEY( stel->lowKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Loop the state's transitions. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + /* Lower key. */ + out << KEY( rtel->lowKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + + /* Upper key. */ + out << KEY( rtel->highKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &TabCodeGen::INDICIES() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + out << stel->value->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + out << rtel->value->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + out << st->defTrans->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &TabCodeGen::TRANS_TARGS() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* The state's default target state. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Add any eof transitions that have not yet been written out above. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + trans->pos = totalTrans; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + + +std::ostream &TabCodeGen::TRANS_ACTIONS() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Add any eof transitions that have not yet been written out above. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &TabCodeGen::TRANS_TARGS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalStates = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Record the position, need this for eofTrans. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + out << trans->targ->id; + if ( t < redFsm->transSet.length()-1 ) { + out << ", "; + if ( ++totalStates % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + + +std::ostream &TabCodeGen::TRANS_ACTIONS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalAct = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + TRANS_ACTION( trans ); + if ( t < redFsm->transSet.length()-1 ) { + out << ", "; + if ( ++totalAct % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + +void TabCodeGen::LOCATE_TRANS() +{ + out << + " _keys = " << ARR_OFF( K(), KO() + "[" + vCS() + "]" ) << ";\n" + " _trans = " << IO() << "[" << vCS() << "];\n" + "\n" + " _klen = " << SL() << "[" << vCS() << "];\n" + " if ( _klen > 0 ) {\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_lower = _keys;\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_mid;\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_upper = _keys + _klen - 1;\n" + " while (1) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = _lower + ((_upper-_lower) >> 1);\n" + " if ( " << GET_WIDE_KEY() << " < *_mid )\n" + " _upper = _mid - 1;\n" + " else if ( " << GET_WIDE_KEY() << " > *_mid )\n" + " _lower = _mid + 1;\n" + " else {\n" + " _trans += (_mid - _keys);\n" + " goto _match;\n" + " }\n" + " }\n" + " _keys += _klen;\n" + " _trans += _klen;\n" + " }\n" + "\n" + " _klen = " << RL() << "[" << vCS() << "];\n" + " if ( _klen > 0 ) {\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_lower = _keys;\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_mid;\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_upper = _keys + (_klen<<1) - 2;\n" + " while (1) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = _lower + (((_upper-_lower) >> 1) & ~1);\n" + " if ( " << GET_WIDE_KEY() << " < _mid[0] )\n" + " _upper = _mid - 2;\n" + " else if ( " << GET_WIDE_KEY() << " > _mid[1] )\n" + " _lower = _mid + 2;\n" + " else {\n" + " _trans += ((_mid - _keys)>>1);\n" + " goto _match;\n" + " }\n" + " }\n" + " _trans += _klen;\n" + " }\n" + "\n"; +} + +void TabCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{" << vCS() << " = " << gotoDest << "; " << + CTRL_FLOW() << "goto _again;}"; +} + +void TabCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; +} + +void TabCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void TabCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void TabCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void TabCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << ");"; +} + +void TabCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = " << + callDest << "; " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void TabCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish, false ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void TabCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << + TOP() << "]; "; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false, false ); + ret << "}"; + } + + ret << CTRL_FLOW() << "goto _again;}"; +} + +void TabCodeGen::BREAK( ostream &ret, int targState, bool csForced ) +{ + outLabelUsed = true; + ret << "{" << P() << "++; " << CTRL_FLOW() << "goto _out; }"; +} + +void TabCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + "\n"; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void TabCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << GET_KEY() << ";\n" + " _klen = " << CL() << "[" << vCS() << "];\n" + " _keys = " << ARR_OFF( CK(), "(" + CO() + "[" + vCS() + "]*2)" ) << ";\n" + " if ( _klen > 0 ) {\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_lower = _keys;\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_mid;\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_upper = _keys + (_klen<<1) - 2;\n" + " while (1) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = _lower + (((_upper-_lower) >> 1) & ~1);\n" + " if ( " << GET_WIDE_KEY() << " < _mid[0] )\n" + " _upper = _mid - 2;\n" + " else if ( " << GET_WIDE_KEY() << " > _mid[1] )\n" + " _lower = _mid + 2;\n" + " else {\n" + " switch ( " << C() << "[" << CO() << "[" << vCS() << "]" + " + ((_mid - _keys)>>1)] ) {\n"; + + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " case " << condSpace->condSpaceId << ": {\n"; + out << TABS(2) << "_widec = " << CAST(WIDE_ALPH_TYPE()) << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "));\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(2) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } + + out << + " break;\n" + " }\n"; + } + + SWITCH_DEFAULT(); + + out << + " }\n" + " break;\n" + " }\n" + " }\n" + " }\n" + "\n"; +} + +void TabCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << + " {\n" + " int _klen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << + ";\n" + " " << UINT() << " _trans;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxActArrItem) << + POINTER() << "_acts;\n" + " " << UINT() << " _nacts;\n"; + } + + out << + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_keys;\n" + "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << ARR_OFF( A(), FSA() + "[" + vCS() + "]" ) << ";\n" + " _nacts = " << CAST(UINT()) << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + out << "_match:\n"; + + if ( useIndicies ) + out << " _trans = " << I() << "[_trans];\n"; + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] == 0 )\n" + " goto _again;\n" + "\n" + " _acts = " << ARR_OFF( A(), TA() + "[_trans]" ) << ";\n" + " _nacts = " << CAST(UINT()) << " *_acts++;\n" + " while ( _nacts-- > 0 )\n {\n" + " switch ( *_acts++ )\n {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << ARR_OFF( A(), TSA() + "[" + vCS() + "]" ) << ";\n" + " _nacts = " << CAST(UINT()) << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " goto _eof_trans;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxActArrItem) << + POINTER() << "__acts = " << + ARR_OFF( A(), EA() + "[" + vCS() + "]" ) << ";\n" + " " << UINT() << " __nacts = " << CAST(UINT()) << " *__acts++;\n" + " while ( __nacts-- > 0 ) {\n" + " switch ( *__acts++ ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/ragel/cdtable.h b/ragel/cdtable.h new file mode 100644 index 0000000..b038283 --- /dev/null +++ b/ragel/cdtable.h @@ -0,0 +1,114 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDTABLE_H +#define _CDTABLE_H + +#include <iostream> +#include "cdcodegen.h" + +/* Forwards. */ +struct CodeGenData; +struct NameInst; +struct RedTransAp; +struct RedStateAp; + +/* + * TabCodeGen + */ +class TabCodeGen : virtual public FsmCodeGen +{ +public: + TabCodeGen( ostream &out ) : FsmCodeGen(out) {} + virtual ~TabCodeGen() { } + virtual void writeData(); + virtual void writeExec(); + +protected: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + std::ostream &COND_KEYS(); + std::ostream &COND_SPACES(); + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &COND_OFFSETS(); + std::ostream &KEY_OFFSETS(); + std::ostream &INDEX_OFFSETS(); + std::ostream &COND_LENS(); + std::ostream &SINGLE_LENS(); + std::ostream &RANGE_LENS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + std::ostream &TRANS_TARGS_WI(); + std::ostream &TRANS_ACTIONS_WI(); + void LOCATE_TRANS(); + + void COND_TRANSLATE(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState, bool csForced ); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + virtual void calcIndexSize(); +}; + + +/* + * CTabCodeGen + */ +struct CTabCodeGen + : public TabCodeGen, public CCodeGen +{ + CTabCodeGen( ostream &out ) : + FsmCodeGen(out), TabCodeGen(out), CCodeGen(out) {} +}; + +/* + * DTabCodeGen + */ +struct DTabCodeGen + : public TabCodeGen, public DCodeGen +{ + DTabCodeGen( ostream &out ) : + FsmCodeGen(out), TabCodeGen(out), DCodeGen(out) {} +}; + +#endif diff --git a/ragel/common.cpp b/ragel/common.cpp new file mode 100644 index 0000000..d5e945d --- /dev/null +++ b/ragel/common.cpp @@ -0,0 +1,355 @@ +/* + * Copyright 2006-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "pcheck.h" +#include "common.h" +#include "stdlib.h" +#include <string.h> +#include <assert.h> + +HostType hostTypesC[] = +{ + { "char", 0, "char", true, CHAR_MIN, CHAR_MAX, sizeof(char) }, + { "unsigned", "char", "uchar", false, 0, UCHAR_MAX, sizeof(unsigned char) }, + { "short", 0, "short", true, SHRT_MIN, SHRT_MAX, sizeof(short) }, + { "unsigned", "short", "ushort", false, 0, USHRT_MAX, sizeof(unsigned short) }, + { "int", 0, "int", true, INT_MIN, INT_MAX, sizeof(int) }, + { "unsigned", "int", "uint", false, 0, UINT_MAX, sizeof(unsigned int) }, + { "long", 0, "long", true, LONG_MIN, LONG_MAX, sizeof(long) }, + { "unsigned", "long", "ulong", false, 0, ULONG_MAX, sizeof(unsigned long) } +}; + +#define S8BIT_MIN -128 +#define S8BIT_MAX 127 + +#define U8BIT_MIN 0 +#define U8BIT_MAX 255 + +#define S16BIT_MIN -32768 +#define S16BIT_MAX 32767 + +#define U16BIT_MIN 0 +#define U16BIT_MAX 65535 + +#define S32BIT_MIN –2147483648l +#define S32BIT_MAX 2147483647l + +#define U32BIT_MIN 0 +#define U32BIT_MAX 4294967295l + +#define S64BIT_MIN –9223372036854775808ll +#define S64BIT_MAX 9223372036854775807ll + +#define U64BIT_MIN 0 +#define U64BIT_MAX 18446744073709551615ll + +HostType hostTypesD[] = +{ + { "byte", 0, "byte", true, CHAR_MIN, CHAR_MAX, 1 }, + { "ubyte", 0, "ubyte", false, 0, UCHAR_MAX, 1 }, + { "char", 0, "char", false, 0, UCHAR_MAX, 1 }, + { "short", 0, "short", true, SHRT_MIN, SHRT_MAX, 2 }, + { "ushort", 0, "ushort", false, 0, USHRT_MAX, 2 }, + { "wchar", 0, "wchar", false, 0, USHRT_MAX, 2 }, + { "int", 0, "int", true, INT_MIN, INT_MAX, 4 }, + { "uint", 0, "uint", false, 0, UINT_MAX, 4 }, + { "dchar", 0, "dchar", false, 0, UINT_MAX, 4 } +}; + +HostType hostTypesJava[] = +{ + { "byte", 0, "byte", true, CHAR_MIN, CHAR_MAX, 1 }, + { "short", 0, "short", true, SHRT_MIN, SHRT_MAX, 2 }, + { "char", 0, "char", false, 0, USHRT_MAX, 2 }, + { "int", 0, "int", true, INT_MIN, INT_MAX, 4 }, +}; + +/* What are the appropriate types for ruby? */ +HostType hostTypesRuby[] = +{ + { "char", 0, "char", true, CHAR_MIN, CHAR_MAX, 1 }, + { "int", 0, "int", true, INT_MIN, INT_MAX, 4 }, +}; + +HostType hostTypesCSharp[] = +{ + { "sbyte", 0, "sbyte", true, CHAR_MIN, CHAR_MAX, 1 }, + { "byte", 0, "byte", false, 0, UCHAR_MAX, 1 }, + { "short", 0, "short", true, SHRT_MIN, SHRT_MAX, 2 }, + { "ushort", 0, "ushort", false, 0, USHRT_MAX, 2 }, + { "char", 0, "char", false, 0, USHRT_MAX, 2 }, + { "int", 0, "int", true, INT_MIN, INT_MAX, 4 }, + { "uint", 0, "uint", false, 0, UINT_MAX, 4 }, + { "long", 0, "long", true, LONG_MIN, LONG_MAX, 8 }, + { "ulong", 0, "ulong", false, 0, ULONG_MAX, 8 } +}; + +HostLang hostLangC = { HostLang::C, hostTypesC, 8, hostTypesC+0, true }; +HostLang hostLangD = { HostLang::D, hostTypesD, 9, hostTypesD+2, true }; +HostLang hostLangJava = { HostLang::Java, hostTypesJava, 4, hostTypesJava+2, false }; +HostLang hostLangRuby = { HostLang::Ruby, hostTypesRuby, 2, hostTypesRuby+0, false }; +HostLang hostLangCSharp = { HostLang::CSharp, hostTypesCSharp, 9, hostTypesCSharp+4, true }; + +HostLang *hostLang = &hostLangC; + +HostType *findAlphType( const char *s1 ) +{ + for ( int i = 0; i < hostLang->numHostTypes; i++ ) { + if ( strcmp( s1, hostLang->hostTypes[i].data1 ) == 0 && + hostLang->hostTypes[i].data2 == 0 ) + { + return hostLang->hostTypes + i; + } + } + + return 0; +} + +HostType *findAlphType( const char *s1, const char *s2 ) +{ + for ( int i = 0; i < hostLang->numHostTypes; i++ ) { + if ( strcmp( s1, hostLang->hostTypes[i].data1 ) == 0 && + hostLang->hostTypes[i].data2 != 0 && + strcmp( s2, hostLang->hostTypes[i].data2 ) == 0 ) + { + return hostLang->hostTypes + i; + } + } + + return 0; +} + +HostType *findAlphTypeInternal( const char *s1 ) +{ + for ( int i = 0; i < hostLang->numHostTypes; i++ ) { + if ( strcmp( s1, hostLang->hostTypes[i].internalName ) == 0 ) + return hostLang->hostTypes + i; + } + + return 0; +} + +/* Construct a new parameter checker with for paramSpec. */ +ParamCheck::ParamCheck( const char *paramSpec, int argc, const char **argv ) +: + state(noparam), + argOffset(0), + curArg(0), + iCurArg(1), + paramSpec(paramSpec), + argc(argc), + argv(argv) +{ +} + +/* Check a single option. Returns the index of the next parameter. Sets p to + * the arg character if valid, 0 otherwise. Sets parg to the parameter arg if + * there is one, NULL otherwise. */ +bool ParamCheck::check() +{ + bool requiresParam; + + if ( iCurArg >= argc ) { /* Off the end of the arg list. */ + state = noparam; + return false; + } + + if ( argOffset != 0 && *argOffset == 0 ) { + /* We are at the end of an arg string. */ + iCurArg += 1; + if ( iCurArg >= argc ) { + state = noparam; + return false; + } + argOffset = 0; + } + + if ( argOffset == 0 ) { + /* Set the current arg. */ + curArg = argv[iCurArg]; + + /* We are at the beginning of an arg string. */ + if ( argv[iCurArg] == 0 || /* Argv[iCurArg] is null. */ + argv[iCurArg][0] != '-' || /* Not a param. */ + argv[iCurArg][1] == 0 ) { /* Only a dash. */ + parameter = 0; + paramArg = 0; + + iCurArg += 1; + state = noparam; + return true; + } + argOffset = argv[iCurArg] + 1; + } + + /* Get the arg char. */ + char argChar = *argOffset; + + /* Loop over all the parms and look for a match. */ + const char *pSpec = paramSpec; + while ( *pSpec != 0 ) { + char pSpecChar = *pSpec; + + /* If there is a ':' following the char then + * it requires a parm. If a parm is required + * then move ahead two in the parmspec. Otherwise + * move ahead one in the parm spec. */ + if ( pSpec[1] == ':' ) { + requiresParam = true; + pSpec += 2; + } + else { + requiresParam = false; + pSpec += 1; + } + + /* Do we have a match. */ + if ( argChar == pSpecChar ) { + if ( requiresParam ) { + if ( argOffset[1] == 0 ) { + /* The param must follow. */ + if ( iCurArg + 1 == argc ) { + /* We are the last arg so there + * cannot be a parameter to it. */ + parameter = argChar; + paramArg = 0; + iCurArg += 1; + argOffset = 0; + state = invalid; + return true; + } + else { + /* the parameter to the arg is the next arg. */ + parameter = pSpecChar; + paramArg = argv[iCurArg + 1]; + iCurArg += 2; + argOffset = 0; + state = match; + return true; + } + } + else { + /* The param for the arg is built in. */ + parameter = pSpecChar; + paramArg = argOffset + 1; + iCurArg += 1; + argOffset = 0; + state = match; + return true; + } + } + else { + /* Good, we matched the parm and no + * arg is required. */ + parameter = pSpecChar; + paramArg = 0; + argOffset += 1; + state = match; + return true; + } + } + } + + /* We did not find a match. Bad Argument. */ + parameter = argChar; + paramArg = 0; + argOffset += 1; + state = invalid; + return true; +} + +/* Counts newlines before sending sync. */ +int output_filter::sync( ) +{ + line += 1; + return std::filebuf::sync(); +} + +/* Counts newlines before sending data out to file. */ +std::streamsize output_filter::xsputn( const char *s, std::streamsize n ) +{ + for ( int i = 0; i < n; i++ ) { + if ( s[i] == '\n' ) + line += 1; + } + return std::filebuf::xsputn( s, n ); +} + +/* Scans a string looking for the file extension. If there is a file + * extension then pointer returned points to inside the string + * passed in. Otherwise returns null. */ +const char *findFileExtension( const char *stemFile ) +{ + const char *ppos = stemFile + strlen(stemFile) - 1; + + /* Scan backwards from the end looking for the first dot. + * If we encounter a '/' before the first dot, then stop the scan. */ + while ( 1 ) { + /* If we found a dot or got to the beginning of the string then + * we are done. */ + if ( ppos == stemFile || *ppos == '.' ) + break; + + /* If we hit a / then there is no extension. Done. */ + if ( *ppos == '/' ) { + ppos = stemFile; + break; + } + ppos--; + } + + /* If we got to the front of the string then bail we + * did not find an extension */ + if ( ppos == stemFile ) + ppos = 0; + + return ppos; +} + +/* Make a file name from a stem. Removes the old filename suffix and + * replaces it with a new one. Returns a newed up string. */ +const char *fileNameFromStem( const char *stemFile, const char *suffix ) +{ + long len = strlen( stemFile ); + assert( len > 0 ); + + /* Get the extension. */ + const char *ppos = findFileExtension( stemFile ); + + /* If an extension was found, then shorten what we think the len is. */ + if ( ppos != 0 ) + len = ppos - stemFile; + + /* Make the return string from the stem and the suffix. */ + char *retVal = new char[ len + strlen( suffix ) + 1 ]; + strncpy( retVal, stemFile, len ); + strcpy( retVal + len, suffix ); + + return retVal; +} + +exit_object endp; + +void operator<<( std::ostream &out, exit_object & ) +{ + out << std::endl; + exit(1); +} diff --git a/ragel/common.h b/ragel/common.h new file mode 100644 index 0000000..a9ec27a --- /dev/null +++ b/ragel/common.h @@ -0,0 +1,378 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _COMMON_H +#define _COMMON_H + +#include <fstream> +#include <climits> +#include "dlist.h" + +/* Location in an input file. */ +struct InputLoc +{ + const char *fileName; + long line; + long col; +}; + + +typedef unsigned long long Size; + +struct Key +{ +private: + long key; + +public: + friend inline Key operator+(const Key key1, const Key key2); + friend inline Key operator-(const Key key1, const Key key2); + friend inline Key operator/(const Key key1, const Key key2); + friend inline long operator&(const Key key1, const Key key2); + + friend inline bool operator<( const Key key1, const Key key2 ); + friend inline bool operator<=( const Key key1, const Key key2 ); + friend inline bool operator>( const Key key1, const Key key2 ); + friend inline bool operator>=( const Key key1, const Key key2 ); + friend inline bool operator==( const Key key1, const Key key2 ); + friend inline bool operator!=( const Key key1, const Key key2 ); + + friend struct KeyOps; + + Key( ) {} + Key( const Key &key ) : key(key.key) {} + Key( long key ) : key(key) {} + + /* Returns the value used to represent the key. This value must be + * interpreted based on signedness. */ + long getVal() const { return key; }; + + /* Returns the key casted to a long long. This form of the key does not + * require any signedness interpretation. */ + long long getLongLong() const; + + /* Returns the distance from the key value to the maximum value that the + * key implementation can hold. */ + Size availableSpace() const; + + bool isUpper() const { return ( 'A' <= key && key <= 'Z' ); } + bool isLower() const { return ( 'a' <= key && key <= 'z' ); } + bool isPrintable() const + { + return ( 7 <= key && key <= 13 ) || ( 32 <= key && key < 127 ); + } + + Key toUpper() const + { return Key( 'A' + ( key - 'a' ) ); } + Key toLower() const + { return Key( 'a' + ( key - 'A' ) ); } + + void operator+=( const Key other ) + { + /* FIXME: must be made aware of isSigned. */ + key += other.key; + } + + void operator-=( const Key other ) + { + /* FIXME: must be made aware of isSigned. */ + key -= other.key; + } + + void operator|=( const Key other ) + { + /* FIXME: must be made aware of isSigned. */ + key |= other.key; + } + + /* Decrement. Needed only for ranges. */ + inline void decrement(); + inline void increment(); +}; + +struct HostType +{ + const char *data1; + const char *data2; + const char *internalName; + bool isSigned; + long long minVal; + long long maxVal; + unsigned int size; +}; + +struct HostLang +{ + /* Target language. */ + enum Lang + { + C, D, Java, Ruby, CSharp + }; + + Lang lang; + HostType *hostTypes; + int numHostTypes; + HostType *defaultAlphType; + bool explicitUnsigned; +}; + +extern HostLang *hostLang; + +extern HostLang hostLangC; +extern HostLang hostLangD; +extern HostLang hostLangJava; +extern HostLang hostLangRuby; +extern HostLang hostLangCSharp; + +HostType *findAlphType( const char *s1 ); +HostType *findAlphType( const char *s1, const char *s2 ); +HostType *findAlphTypeInternal( const char *s1 ); + +/* An abstraction of the key operators that manages key operations such as + * comparison and increment according the signedness of the key. */ +struct KeyOps +{ + /* Default to signed alphabet. */ + KeyOps() : + isSigned(true), + alphType(0) + {} + + /* Default to signed alphabet. */ + KeyOps( bool isSigned ) + :isSigned(isSigned) {} + + bool isSigned; + Key minKey, maxKey; + HostType *alphType; + + void setAlphType( HostType *alphType ) + { + this->alphType = alphType; + isSigned = alphType->isSigned; + if ( isSigned ) { + minKey = (long) alphType->minVal; + maxKey = (long) alphType->maxVal; + } + else { + minKey = (long) (unsigned long) alphType->minVal; + maxKey = (long) (unsigned long) alphType->maxVal; + } + } + + /* Compute the distance between two keys. */ + Size span( Key key1, Key key2 ) + { + return isSigned ? + (unsigned long long)( + (long long)key2.key - + (long long)key1.key + 1) : + (unsigned long long)( + (unsigned long)key2.key) - + (unsigned long long)((unsigned long)key1.key) + 1; + } + + Size alphSize() + { return span( minKey, maxKey ); } + + HostType *typeSubsumes( long long maxVal ) + { + for ( int i = 0; i < hostLang->numHostTypes; i++ ) { + if ( maxVal <= hostLang->hostTypes[i].maxVal ) + return hostLang->hostTypes + i; + } + return 0; + } + + HostType *typeSubsumes( bool isSigned, long long maxVal ) + { + for ( int i = 0; i < hostLang->numHostTypes; i++ ) { + if ( ( ( isSigned && hostLang->hostTypes[i].isSigned ) || !isSigned ) && + maxVal <= hostLang->hostTypes[i].maxVal ) + return hostLang->hostTypes + i; + } + return 0; + } +}; + +extern KeyOps *keyOps; + +inline bool operator<( const Key key1, const Key key2 ) +{ + return keyOps->isSigned ? key1.key < key2.key : + (unsigned long)key1.key < (unsigned long)key2.key; +} + +inline bool operator<=( const Key key1, const Key key2 ) +{ + return keyOps->isSigned ? key1.key <= key2.key : + (unsigned long)key1.key <= (unsigned long)key2.key; +} + +inline bool operator>( const Key key1, const Key key2 ) +{ + return keyOps->isSigned ? key1.key > key2.key : + (unsigned long)key1.key > (unsigned long)key2.key; +} + +inline bool operator>=( const Key key1, const Key key2 ) +{ + return keyOps->isSigned ? key1.key >= key2.key : + (unsigned long)key1.key >= (unsigned long)key2.key; +} + +inline bool operator==( const Key key1, const Key key2 ) +{ + return key1.key == key2.key; +} + +inline bool operator!=( const Key key1, const Key key2 ) +{ + return key1.key != key2.key; +} + +/* Decrement. Needed only for ranges. */ +inline void Key::decrement() +{ + key = keyOps->isSigned ? key - 1 : ((unsigned long)key)-1; +} + +/* Increment. Needed only for ranges. */ +inline void Key::increment() +{ + key = keyOps->isSigned ? key+1 : ((unsigned long)key)+1; +} + +inline long long Key::getLongLong() const +{ + return keyOps->isSigned ? (long long)key : (long long)(unsigned long)key; +} + +inline Size Key::availableSpace() const +{ + if ( keyOps->isSigned ) + return (long long)LONG_MAX - (long long)key; + else + return (unsigned long long)ULONG_MAX - (unsigned long long)(unsigned long)key; +} + +inline Key operator+(const Key key1, const Key key2) +{ + /* FIXME: must be made aware of isSigned. */ + return Key( key1.key + key2.key ); +} + +inline Key operator-(const Key key1, const Key key2) +{ + /* FIXME: must be made aware of isSigned. */ + return Key( key1.key - key2.key ); +} + +inline long operator&(const Key key1, const Key key2) +{ + /* FIXME: must be made aware of isSigned. */ + return key1.key & key2.key; +} + +inline Key operator/(const Key key1, const Key key2) +{ + /* FIXME: must be made aware of isSigned. */ + return key1.key / key2.key; +} + +/* Filter on the output stream that keeps track of the number of lines + * output. */ +class output_filter : public std::filebuf +{ +public: + output_filter( const char *fileName ) : fileName(fileName), line(1) { } + + virtual int sync(); + virtual std::streamsize xsputn(const char* s, std::streamsize n); + + const char *fileName; + int line; +}; + +class cfilebuf : public std::streambuf +{ +public: + cfilebuf( char *fileName, FILE* file ) : fileName(fileName), file(file) { } + char *fileName; + FILE *file; + + int sync() + { + fflush( file ); + return 0; + } + + int overflow( int c ) + { + if ( c != EOF ) + fputc( c, file ); + return 0; + } + + std::streamsize xsputn( const char* s, std::streamsize n ) + { + std::streamsize written = fwrite( s, 1, n, file ); + return written; + } +}; + +class costream : public std::ostream +{ +public: + costream( cfilebuf *b ) : + std::ostream(b), b(b) {} + + ~costream() + { delete b; } + + void fclose() + { ::fclose( b->file ); } + + cfilebuf *b; +}; + + +const char *findFileExtension( const char *stemFile ); +const char *fileNameFromStem( const char *stemFile, const char *suffix ); + +struct Export +{ + Export( const char *name, Key key ) + : name(name), key(key) {} + + const char *name; + Key key; + + Export *prev, *next; +}; + +typedef DList<Export> ExportList; + +struct exit_object { }; +extern exit_object endp; +void operator<<( std::ostream &out, exit_object & ); + +#endif diff --git a/ragel/cscodegen.cpp b/ragel/cscodegen.cpp new file mode 100644 index 0000000..c04511a --- /dev/null +++ b/ragel/cscodegen.cpp @@ -0,0 +1,817 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cscodegen.h" +#include "redfsm.h" +#include "gendata.h" +#include <sstream> +#include <iomanip> +#include <string> +#include <assert.h> + +using std::ostream; +using std::ostringstream; +using std::string; +using std::cerr; +using std::endl; + +using std::istream; +using std::ifstream; +using std::ostream; +using std::ios; +using std::cin; +using std::cout; +using std::cerr; +using std::endl; + +void csharpLineDirective( ostream &out, const char *fileName, int line ) +{ + if ( noLineDirectives ) + out << "/* "; + + /* Write the preprocessor line info for to the input file. */ + out << "#line " << line << " \""; + for ( const char *pc = fileName; *pc != 0; pc++ ) { + if ( *pc == '\\' ) + out << "\\\\"; + else + out << *pc; + } + out << '"'; + + if ( noLineDirectives ) + out << " */"; + + out << '\n'; +} + +void CSharpFsmCodeGen::genLineDirective( ostream &out ) +{ + std::streambuf *sbuf = out.rdbuf(); + output_filter *filter = static_cast<output_filter*>(sbuf); + csharpLineDirective( out, filter->fileName, filter->line + 1 ); +} + + +/* Init code gen with in parameters. */ +CSharpFsmCodeGen::CSharpFsmCodeGen( ostream &out ) +: + CodeGenData(out) +{ +} + +unsigned int CSharpFsmCodeGen::arrayTypeSize( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + return arrayType->size; +} + +string CSharpFsmCodeGen::ARRAY_TYPE( unsigned long maxVal ) +{ + return ARRAY_TYPE( maxVal, false ); +} + +string CSharpFsmCodeGen::ARRAY_TYPE( unsigned long maxVal, bool forceSigned ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType; + if (forceSigned) + arrayType = keyOps->typeSubsumes(true, maxValLL); + else + arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + + string ret = arrayType->data1; + if ( arrayType->data2 != 0 ) { + ret += " "; + ret += arrayType->data2; + } + return ret; +} + +/* Write out the fsm name. */ +string CSharpFsmCodeGen::FSM_NAME() +{ + return fsmName; +} + +/* Emit the offset of the start state as a decimal integer. */ +string CSharpFsmCodeGen::START_STATE_ID() +{ + ostringstream ret; + ret << redFsm->startState->id; + return ret.str(); +}; + +/* Write out the array of actions. */ +std::ostream &CSharpFsmCodeGen::ACTIONS_ARRAY() +{ + out << "\t0, "; + int totalActions = 1; + for ( GenActionTableMap::Iter act = redFsm->actionMap; act.lte(); act++ ) { + /* Write out the length, which will never be the last character. */ + out << act->key.length() << ", "; + /* Put in a line break every 8 */ + if ( totalActions++ % 8 == 7 ) + out << "\n\t"; + + for ( GenActionTable::Iter item = act->key; item.lte(); item++ ) { + out << item->value->actionId; + if ( ! (act.last() && item.last()) ) + out << ", "; + + /* Put in a line break every 8 */ + if ( totalActions++ % 8 == 7 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +string CSharpFsmCodeGen::ACCESS() +{ + ostringstream ret; + if ( accessExpr != 0 ) + INLINE_LIST( ret, accessExpr, 0, false ); + return ret.str(); +} + + +string CSharpFsmCodeGen::P() +{ + ostringstream ret; + if ( pExpr == 0 ) + ret << "p"; + else { + ret << "("; + INLINE_LIST( ret, pExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::PE() +{ + ostringstream ret; + if ( peExpr == 0 ) + ret << "pe"; + else { + ret << "("; + INLINE_LIST( ret, peExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::vEOF() +{ + ostringstream ret; + if ( eofExpr == 0 ) + ret << "eof"; + else { + ret << "("; + INLINE_LIST( ret, eofExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::vCS() +{ + ostringstream ret; + if ( csExpr == 0 ) + ret << ACCESS() << "cs"; + else { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, csExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::TOP() +{ + ostringstream ret; + if ( topExpr == 0 ) + ret << ACCESS() + "top"; + else { + ret << "("; + INLINE_LIST( ret, topExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::STACK() +{ + ostringstream ret; + if ( stackExpr == 0 ) + ret << ACCESS() + "stack"; + else { + ret << "("; + INLINE_LIST( ret, stackExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::ACT() +{ + ostringstream ret; + if ( actExpr == 0 ) + ret << ACCESS() + "act"; + else { + ret << "("; + INLINE_LIST( ret, actExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::TOKSTART() +{ + ostringstream ret; + if ( tokstartExpr == 0 ) + ret << ACCESS() + "ts"; + else { + ret << "("; + INLINE_LIST( ret, tokstartExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::TOKEND() +{ + ostringstream ret; + if ( tokendExpr == 0 ) + ret << ACCESS() + "te"; + else { + ret << "("; + INLINE_LIST( ret, tokendExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::GET_WIDE_KEY() +{ + if ( redFsm->anyConditions() ) + return "_widec"; + else + return GET_KEY(); +} + +string CSharpFsmCodeGen::GET_WIDE_KEY( RedStateAp *state ) +{ + if ( state->stateCondList.length() > 0 ) + return "_widec"; + else + return GET_KEY(); +} + +string CSharpFsmCodeGen::GET_KEY() +{ + ostringstream ret; + if ( getKeyExpr != 0 ) { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, getKeyExpr, 0, false ); + ret << ")"; + } + else { + /* Expression for retrieving the key, use simple dereference. */ + ret << "(*" << P() << ")"; + } + return ret.str(); +} + +/* Write out level number of tabs. Makes the nested binary search nice + * looking. */ +string CSharpFsmCodeGen::TABS( int level ) +{ + string result; + while ( level-- > 0 ) + result += "\t"; + return result; +} + +/* Write out a key from the fsm code gen. Depends on wether or not the key is + * signed. */ +string CSharpFsmCodeGen::KEY( Key key ) +{ + ostringstream ret; + if ( keyOps->isSigned || !hostLang->explicitUnsigned ) + ret << key.getVal(); + else + ret << (unsigned long) key.getVal() << 'u'; + return ret.str(); +} + +string CSharpFsmCodeGen::ALPHA_KEY( Key key ) +{ + ostringstream ret; + if (key.getVal() > 0xFFFF) { + ret << key.getVal(); + } else { + ret << "'\\u" << std::hex << std::setw(4) << std::setfill('0') << + key.getVal() << "'"; + } + //ret << "(char) " << key.getVal(); + return ret.str(); +} + +void CSharpFsmCodeGen::EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ) +{ + /* The parser gives fexec two children. The double brackets are for D + * code. If the inline list is a single word it will get interpreted as a + * C-style cast by the D compiler. */ + ret << "{" << P() << " = (("; + INLINE_LIST( ret, item->children, targState, inFinish ); + ret << "))-1;}"; +} + +void CSharpFsmCodeGen::LM_SWITCH( ostream &ret, GenInlineItem *item, + int targState, int inFinish ) +{ + ret << + " switch( " << ACT() << " ) {\n"; + + for ( GenInlineList::Iter lma = *item->children; lma.lte(); lma++ ) { + /* Write the case label, the action and the case break. */ + if ( lma->lmId < 0 ) + ret << " default:\n"; + else + ret << " case " << lma->lmId << ":\n"; + + /* Write the block and close it off. */ + ret << " {"; + INLINE_LIST( ret, lma->children, targState, inFinish ); + ret << "}\n"; + + ret << " break;\n"; + } + + ret << + " }\n" + "\t"; +} + +void CSharpFsmCodeGen::SET_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = " << item->lmId << ";"; +} + +void CSharpFsmCodeGen::SET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + /* The tokend action sets tokend. */ + ret << TOKEND() << " = " << P(); + if ( item->offset != 0 ) + out << "+" << item->offset; + out << ";"; +} + +void CSharpFsmCodeGen::GET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + ret << TOKEND(); +} + +void CSharpFsmCodeGen::INIT_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << NULL_ITEM() << ";"; +} + +void CSharpFsmCodeGen::INIT_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = 0;"; +} + +void CSharpFsmCodeGen::SET_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << P() << ";"; +} + +void CSharpFsmCodeGen::SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish ) +{ + if ( item->children->length() > 0 ) { + /* Write the block and close it off. */ + ret << "{"; + INLINE_LIST( ret, item->children, targState, inFinish ); + ret << "}"; + } +} + + +/* Write out an inline tree structure. Walks the list and possibly calls out + * to virtual functions than handle language specific items in the tree. */ +void CSharpFsmCodeGen::INLINE_LIST( ostream &ret, GenInlineList *inlineList, + int targState, bool inFinish ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Text: + ret << item->data; + break; + case GenInlineItem::Goto: + GOTO( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Call: + CALL( ret, item->targState->id, targState, inFinish ); + break; + case GenInlineItem::Next: + NEXT( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Ret: + RET( ret, inFinish ); + break; + case GenInlineItem::PChar: + ret << P(); + break; + case GenInlineItem::Char: + ret << GET_KEY(); + break; + case GenInlineItem::Hold: + ret << P() << "--;"; + break; + case GenInlineItem::Exec: + EXEC( ret, item, targState, inFinish ); + break; + case GenInlineItem::Curs: + CURS( ret, inFinish ); + break; + case GenInlineItem::Targs: + TARGS( ret, inFinish, targState ); + break; + case GenInlineItem::Entry: + ret << item->targState->id; + break; + case GenInlineItem::GotoExpr: + GOTO_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::CallExpr: + CALL_EXPR( ret, item, targState, inFinish ); + break; + case GenInlineItem::NextExpr: + NEXT_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::LmSwitch: + LM_SWITCH( ret, item, targState, inFinish ); + break; + case GenInlineItem::LmSetActId: + SET_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokEnd: + SET_TOKEND( ret, item ); + break; + case GenInlineItem::LmGetTokEnd: + GET_TOKEND( ret, item ); + break; + case GenInlineItem::LmInitTokStart: + INIT_TOKSTART( ret, item ); + break; + case GenInlineItem::LmInitAct: + INIT_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokStart: + SET_TOKSTART( ret, item ); + break; + case GenInlineItem::SubAction: + SUB_ACTION( ret, item, targState, inFinish ); + break; + case GenInlineItem::Break: + BREAK( ret, targState ); + break; + } + } +} +/* Write out paths in line directives. Escapes any special characters. */ +string CSharpFsmCodeGen::LDIR_PATH( char *path ) +{ + ostringstream ret; + for ( char *pc = path; *pc != 0; pc++ ) { + if ( *pc == '\\' ) + ret << "\\\\"; + else + ret << *pc; + } + return ret.str(); +} + +void CSharpFsmCodeGen::ACTION( ostream &ret, GenAction *action, int targState, bool inFinish ) +{ + /* Write the preprocessor line info for going into the source file. */ + csharpLineDirective( ret, action->loc.fileName, action->loc.line ); + + /* Write the block and close it off. */ + ret << "\t{"; + INLINE_LIST( ret, action->inlineList, targState, inFinish ); + ret << "}\n"; +} + +void CSharpFsmCodeGen::CONDITION( ostream &ret, GenAction *condition ) +{ + ret << "\n"; + csharpLineDirective( ret, condition->loc.fileName, condition->loc.line ); + INLINE_LIST( ret, condition->inlineList, 0, false ); +} + +string CSharpFsmCodeGen::ERROR_STATE() +{ + ostringstream ret; + if ( redFsm->errState != 0 ) + ret << redFsm->errState->id; + else + ret << "-1"; + return ret.str(); +} + +string CSharpFsmCodeGen::FIRST_FINAL_STATE() +{ + ostringstream ret; + if ( redFsm->firstFinState != 0 ) + ret << redFsm->firstFinState->id; + else + ret << redFsm->nextStateId; + return ret.str(); +} + +void CSharpFsmCodeGen::writeInit() +{ + out << " {\n"; + + if ( !noCS ) + out << "\t" << vCS() << " = " << START() << ";\n"; + + /* If there are any calls, then the stack top needs initialization. */ + if ( redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "\t" << TOP() << " = 0;\n"; + + if ( hasLongestMatch ) { + out << + " " << TOKSTART() << " = " << NULL_ITEM() << ";\n" + " " << TOKEND() << " = " << NULL_ITEM() << ";\n" + " " << ACT() << " = 0;\n"; + } + out << " }\n"; +} + +string CSharpFsmCodeGen::DATA_PREFIX() +{ + if ( !noPrefix ) + return FSM_NAME() + "_"; + return ""; +} + +/* Emit the alphabet data type. */ +string CSharpFsmCodeGen::ALPH_TYPE() +{ + string ret = keyOps->alphType->data1; + if ( keyOps->alphType->data2 != 0 ) { + ret += " "; + ret += + keyOps->alphType->data2; + } + return ret; +} + +/* Emit the alphabet data type. */ +string CSharpFsmCodeGen::WIDE_ALPH_TYPE() +{ + string ret; + if ( redFsm->maxKey <= keyOps->maxKey ) + ret = ALPH_TYPE(); + else { + long long maxKeyVal = redFsm->maxKey.getLongLong(); + HostType *wideType = keyOps->typeSubsumes( keyOps->isSigned, maxKeyVal ); + assert( wideType != 0 ); + + ret = wideType->data1; + if ( wideType->data2 != 0 ) { + ret += " "; + ret += wideType->data2; + } + } + return ret; +} + +void CSharpFsmCodeGen::STATE_IDS() +{ + if ( redFsm->startState != 0 ) + STATIC_VAR( "int", START() ) << " = " << START_STATE_ID() << ";\n"; + + if ( !noFinal ) + STATIC_VAR( "int" , FIRST_FINAL() ) << " = " << FIRST_FINAL_STATE() << ";\n"; + + if ( !noError ) + STATIC_VAR( "int", ERROR() ) << " = " << ERROR_STATE() << ";\n"; + + out << "\n"; + + if ( entryPointNames.length() > 0 ) { + for ( EntryNameVect::Iter en = entryPointNames; en.lte(); en++ ) { + STATIC_VAR( "int", DATA_PREFIX() + "en_" + *en ) << + " = " << entryPointIds[en.pos()] << ";\n"; + } + out << "\n"; + } +} + + +void CSharpFsmCodeGen::writeStart() +{ + out << START_STATE_ID(); +} + +void CSharpFsmCodeGen::writeFirstFinal() +{ + out << FIRST_FINAL_STATE(); +} + +void CSharpFsmCodeGen::writeError() +{ + out << ERROR_STATE(); +} + +/* + * C# Specific + */ +string CSharpCodeGen::GET_KEY() +{ + ostringstream ret; + if ( getKeyExpr != 0 ) { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, getKeyExpr, 0, false ); + ret << ")"; + } + else { + /* Expression for retrieving the key, use simple dereference. */ + ret << "data[" << P() << "]"; + } + return ret.str(); +} +string CSharpCodeGen::NULL_ITEM() +{ + return "-1"; +} + +string CSharpCodeGen::POINTER() +{ + // XXX C# has no pointers + // multiple items seperated by commas can also be pointer types. + return " "; +} + +string CSharpCodeGen::PTR_CONST() +{ + return ""; +} + +std::ostream &CSharpCodeGen::OPEN_ARRAY( string type, string name ) +{ + out << "static readonly " << type << "[] " << name << " = "; + /* + if (type == "char") + out << "Encoding.ASCII.Get"; + else */ + out << "new " << type << " [] {\n"; + return out; +} + +std::ostream &CSharpCodeGen::CLOSE_ARRAY() +{ + return out << "};\n"; +} + +std::ostream &CSharpCodeGen::STATIC_VAR( string type, string name ) +{ + out << "const " << type << " " << name; + return out; +} + +string CSharpCodeGen::ARR_OFF( string ptr, string offset ) +{ + // XXX C# can't do pointer arithmetic + return "&" + ptr + "[" + offset + "]"; +} + +string CSharpCodeGen::CAST( string type ) +{ + return "(" + type + ")"; +} + +string CSharpCodeGen::UINT( ) +{ + return "uint"; +} + +std::ostream &CSharpCodeGen::SWITCH_DEFAULT() +{ + out << " default: break;\n"; + return out; +} + +string CSharpCodeGen::CTRL_FLOW() +{ + return "if (true) "; +} + +void CSharpCodeGen::writeExports() +{ + if ( exportList.length() > 0 ) { + for ( ExportList::Iter ex = exportList; ex.lte(); ex++ ) { + out << "const " << ALPH_TYPE() << " " << DATA_PREFIX() << + "ex_" << ex->name << " = " << KEY(ex->key) << ";\n"; + } + out << "\n"; + } +} + +/* + * End C#-specific code. + */ + +void CSharpFsmCodeGen::finishRagelDef() +{ + if ( codeStyle == GenGoto || codeStyle == GenFGoto || + codeStyle == GenIpGoto || codeStyle == GenSplit ) + { + /* For directly executable machines there is no required state + * ordering. Choose a depth-first ordering to increase the + * potential for fall-throughs. */ + redFsm->depthFirstOrdering(); + } + else { + /* The frontend will do this for us, but it may be a good idea to + * force it if the intermediate file is edited. */ + redFsm->sortByStateId(); + } + + /* Choose default transitions and the single transition. */ + redFsm->chooseDefaultSpan(); + + /* Maybe do flat expand, otherwise choose single. */ + if ( codeStyle == GenFlat || codeStyle == GenFFlat ) + redFsm->makeFlat(); + else + redFsm->chooseSingle(); + + /* If any errors have occured in the input file then don't write anything. */ + if ( gblErrorCount > 0 ) + return; + + if ( codeStyle == GenSplit ) + redFsm->partitionFsm( numSplitPartitions ); + + if ( codeStyle == GenIpGoto || codeStyle == GenSplit ) + redFsm->setInTrans(); + + /* Anlayze Machine will find the final action reference counts, among + * other things. We will use these in reporting the usage + * of fsm directives in action code. */ + analyzeMachine(); + + /* Determine if we should use indicies. */ + calcIndexSize(); +} + +ostream &CSharpFsmCodeGen::source_warning( const InputLoc &loc ) +{ + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": warning: "; + return cerr; +} + +ostream &CSharpFsmCodeGen::source_error( const InputLoc &loc ) +{ + gblErrorCount += 1; + assert( sourceFileName != 0 ); + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": "; + return cerr; +} + diff --git a/ragel/cscodegen.h b/ragel/cscodegen.h new file mode 100644 index 0000000..0471ea9 --- /dev/null +++ b/ragel/cscodegen.h @@ -0,0 +1,205 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CSCODEGEN_H +#define _CSCODEGEN_H + +#include <iostream> +#include <string> +#include <stdio.h> +#include "common.h" +#include "gendata.h" + +using std::string; +using std::ostream; + +/* Integer array line length. */ +#define IALL 8 + +/* Forwards. */ +struct RedFsmAp; +struct RedStateAp; +struct CodeGenData; +struct GenAction; +struct NameInst; +struct GenInlineItem; +struct GenInlineList; +struct RedAction; +struct LongestMatch; +struct LongestMatchPart; + +string itoa( int i ); + +/* + * class CSharpFsmCodeGen + */ +class CSharpFsmCodeGen : public CodeGenData +{ +public: + CSharpFsmCodeGen( ostream &out ); + virtual ~CSharpFsmCodeGen() {} + + virtual void finishRagelDef(); + virtual void writeInit(); + virtual void writeStart(); + virtual void writeFirstFinal(); + virtual void writeError(); + +protected: + string FSM_NAME(); + string START_STATE_ID(); + ostream &ACTIONS_ARRAY(); + string GET_WIDE_KEY(); + string GET_WIDE_KEY( RedStateAp *state ); + string TABS( int level ); + string KEY( Key key ); + string ALPHA_KEY( Key key ); + string LDIR_PATH( char *path ); + void ACTION( ostream &ret, GenAction *action, int targState, bool inFinish ); + void CONDITION( ostream &ret, GenAction *condition ); + string ALPH_TYPE(); + string WIDE_ALPH_TYPE(); + string ARRAY_TYPE( unsigned long maxVal ); + string ARRAY_TYPE( unsigned long maxVal, bool forceSigned ); + + virtual string ARR_OFF( string ptr, string offset ) = 0; + virtual string CAST( string type ) = 0; + virtual string UINT() = 0; + virtual string NULL_ITEM() = 0; + virtual string POINTER() = 0; + virtual string GET_KEY(); + virtual ostream &SWITCH_DEFAULT() = 0; + + string P(); + string PE(); + string vEOF(); + + string ACCESS(); + string vCS(); + string STACK(); + string TOP(); + string TOKSTART(); + string TOKEND(); + string ACT(); + + string DATA_PREFIX(); + string PM() { return "_" + DATA_PREFIX() + "partition_map"; } + string C() { return "_" + DATA_PREFIX() + "cond_spaces"; } + string CK() { return "_" + DATA_PREFIX() + "cond_keys"; } + string K() { return "_" + DATA_PREFIX() + "trans_keys"; } + string I() { return "_" + DATA_PREFIX() + "indicies"; } + string CO() { return "_" + DATA_PREFIX() + "cond_offsets"; } + string KO() { return "_" + DATA_PREFIX() + "key_offsets"; } + string IO() { return "_" + DATA_PREFIX() + "index_offsets"; } + string CL() { return "_" + DATA_PREFIX() + "cond_lengths"; } + string SL() { return "_" + DATA_PREFIX() + "single_lengths"; } + string RL() { return "_" + DATA_PREFIX() + "range_lengths"; } + string A() { return "_" + DATA_PREFIX() + "actions"; } + string TA() { return "_" + DATA_PREFIX() + "trans_actions"; } + string TT() { return "_" + DATA_PREFIX() + "trans_targs"; } + string TSA() { return "_" + DATA_PREFIX() + "to_state_actions"; } + string FSA() { return "_" + DATA_PREFIX() + "from_state_actions"; } + string EA() { return "_" + DATA_PREFIX() + "eof_actions"; } + string ET() { return "_" + DATA_PREFIX() + "eof_trans"; } + string SP() { return "_" + DATA_PREFIX() + "key_spans"; } + string CSP() { return "_" + DATA_PREFIX() + "cond_key_spans"; } + string START() { return DATA_PREFIX() + "start"; } + string ERROR() { return DATA_PREFIX() + "error"; } + string FIRST_FINAL() { return DATA_PREFIX() + "first_final"; } + string CTXDATA() { return DATA_PREFIX() + "ctxdata"; } + + void INLINE_LIST( ostream &ret, GenInlineList *inlineList, int targState, bool inFinish ); + virtual void GOTO( ostream &ret, int gotoDest, bool inFinish ) = 0; + virtual void CALL( ostream &ret, int callDest, int targState, bool inFinish ) = 0; + virtual void NEXT( ostream &ret, int nextDest, bool inFinish ) = 0; + virtual void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) = 0; + virtual void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) = 0; + virtual void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, + int targState, bool inFinish ) = 0; + virtual void RET( ostream &ret, bool inFinish ) = 0; + virtual void BREAK( ostream &ret, int targState ) = 0; + virtual void CURS( ostream &ret, bool inFinish ) = 0; + virtual void TARGS( ostream &ret, bool inFinish, int targState ) = 0; + void EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void LM_SWITCH( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void SET_ACT( ostream &ret, GenInlineItem *item ); + void INIT_TOKSTART( ostream &ret, GenInlineItem *item ); + void INIT_ACT( ostream &ret, GenInlineItem *item ); + void SET_TOKSTART( ostream &ret, GenInlineItem *item ); + void SET_TOKEND( ostream &ret, GenInlineItem *item ); + void GET_TOKEND( ostream &ret, GenInlineItem *item ); + void SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish ); + void STATE_IDS(); + + string ERROR_STATE(); + string FIRST_FINAL_STATE(); + + virtual string PTR_CONST() = 0; + virtual ostream &OPEN_ARRAY( string type, string name ) = 0; + virtual ostream &CLOSE_ARRAY() = 0; + virtual ostream &STATIC_VAR( string type, string name ) = 0; + + virtual string CTRL_FLOW() = 0; + + ostream &source_warning(const InputLoc &loc); + ostream &source_error(const InputLoc &loc); + + unsigned int arrayTypeSize( unsigned long maxVal ); + + bool outLabelUsed; + bool testEofUsed; + bool againLabelUsed; + bool useIndicies; + +public: + /* Determine if we should use indicies. */ + virtual void calcIndexSize() {} + + void genLineDirective( ostream &out ); +}; + +class CSharpCodeGen : virtual public CSharpFsmCodeGen +{ +public: + CSharpCodeGen( ostream &out ) : CSharpFsmCodeGen(out) {} + + virtual string GET_KEY(); + virtual string NULL_ITEM(); + virtual string POINTER(); + virtual ostream &SWITCH_DEFAULT(); + virtual ostream &OPEN_ARRAY( string type, string name ); + virtual ostream &CLOSE_ARRAY(); + virtual ostream &STATIC_VAR( string type, string name ); + virtual string ARR_OFF( string ptr, string offset ); + virtual string CAST( string type ); + virtual string UINT(); + virtual string PTR_CONST(); + virtual string CTRL_FLOW(); + + virtual void writeExports(); +}; + +#define MAX(a, b) (a > b ? a : b) + +#endif diff --git a/ragel/csfflat.cpp b/ragel/csfflat.cpp new file mode 100644 index 0000000..f504eb2 --- /dev/null +++ b/ragel/csfflat.cpp @@ -0,0 +1,389 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "csfflat.h" +#include "redfsm.h" +#include "gendata.h" + +std::ostream &CSharpFFlatCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &CSharpFFlatCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &CSharpFFlatCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + out << act; + return out; +} + +/* Write out the function for a transition. */ +std::ostream &CSharpFFlatCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + int action = 0; + if ( trans->action != 0 ) + action = trans->action->actListId+1; + out << action; + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &CSharpFFlatCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &CSharpFFlatCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpFFlatCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &CSharpFFlatCodeGen::ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +void CSharpFFlatCodeGen::writeData() +{ + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpan), CSP() ); + COND_KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCond), C() ); + CONDS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondIndexOffset), CO() ); + COND_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSpan), SP() ); + KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxFlatIndexOffset), IO() ); + FLAT_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void CSharpFFlatCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + initVarTypes(); + + out << + " {\n" + " " << slenType << " _slen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << ";\n"; + out << " " << transType << " _trans"; + + if ( redFsm->anyConditions() ) + out << ", _cond"; + + out << ";\n"; + + out << + " " << "int _keys;\n" + " " << indsType << " _inds;\n"; + /* + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_keys;\n" + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxIndex) << POINTER() << "_inds;\n";*/ + + if ( redFsm->anyConditions() ) { + out << + " " << condsType << " _conds;\n" + " " << WIDE_ALPH_TYPE() << " _widec;\n"; + } + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " switch ( " << FSA() << "[" << vCS() << "] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] == 0 )\n" + " goto _again;\n" + "\n" + " switch ( " << TA() << "[_trans] ) {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " switch ( " << TSA() << "[" << vCS() << "] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << CAST(transType) << " (" << ET() << + "[" << vCS() << "] - 1);\n" + " goto _eof_trans;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " switch ( " << EA() << "[" << vCS() << "] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/ragel/csfflat.h b/ragel/csfflat.h new file mode 100644 index 0000000..b102fe5 --- /dev/null +++ b/ragel/csfflat.h @@ -0,0 +1,55 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CSFFLAT_H +#define _CSFFLAT_H + +#include <iostream> +#include "csflat.h" + +/* Forwards. */ +struct CodeGenData; + +/* + * CSharpFFlatCodeGen + */ +class CSharpFFlatCodeGen : public CSharpFlatCodeGen +{ +public: + CSharpFFlatCodeGen( ostream &out ) : CSharpFsmCodeGen(out), CSharpFlatCodeGen(out) {} +private: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); +}; + +#endif diff --git a/ragel/csfgoto.cpp b/ragel/csfgoto.cpp new file mode 100644 index 0000000..5a26424 --- /dev/null +++ b/ragel/csfgoto.cpp @@ -0,0 +1,296 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "csfgoto.h" +#include "redfsm.h" +#include "gendata.h" +#include "bstmap.h" + +std::ostream &CSharpFGotoCodeGen::EXEC_ACTIONS() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* We are at the start of a glob, write the case. */ + out << "f" << redAct->actListId << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tgoto _again;\n"; + } + } + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &CSharpFGotoCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &CSharpFGotoCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpFGotoCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &CSharpFGotoCodeGen::FINISH_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* States that are final and have an out action need a case. */ + if ( st->eofAction != 0 ) { + /* Write the case label. */ + out << "\t\tcase " << st->id << ": "; + + /* Jump to the func. */ + out << "goto f" << st->eofAction->actListId << ";\n"; + } + } + + return out; +} + +unsigned int CSharpFGotoCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + return act; +} + +unsigned int CSharpFGotoCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + return act; +} + +unsigned int CSharpFGotoCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + return act; +} + +void CSharpFGotoCodeGen::writeData() +{ + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void CSharpFGotoCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << " {\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " int _ps = 0;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " switch ( " << FSA() << "[" << vCS() << "] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + out << + " switch ( " << vCS() << " ) {\n"; + STATE_GOTOS(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + TRANSITIONS() << + "\n"; + + if ( redFsm->anyRegActions() ) + EXEC_ACTIONS() << "\n"; + + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " switch ( " << TSA() << "[" << vCS() << "] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " switch ( " << vCS() << " ) {\n"; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << " case " << st->id << ": goto tr" << st->eofTrans->id << ";\n"; + } + + SWITCH_DEFAULT() << + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " switch ( " << EA() << "[" << vCS() << "] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/ragel/csfgoto.h b/ragel/csfgoto.h new file mode 100644 index 0000000..fa9447b --- /dev/null +++ b/ragel/csfgoto.h @@ -0,0 +1,55 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CSFGOTO_H +#define _CSFGOTO_H + +#include <iostream> +#include "csgoto.h" + +/* Forwards. */ +struct CodeGenData; + + +/* + * class CSharpFGotoCodeGen + */ +class CSharpFGotoCodeGen : public CSharpGotoCodeGen +{ +public: + CSharpFGotoCodeGen( ostream &out ) : CSharpFsmCodeGen(out), CSharpGotoCodeGen(out) {} + + std::ostream &EXEC_ACTIONS(); + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &FINISH_CASES(); + std::ostream &EOF_ACTION_SWITCH(); + unsigned int TO_STATE_ACTION( RedStateAp *state ); + unsigned int FROM_STATE_ACTION( RedStateAp *state ); + unsigned int EOF_ACTION( RedStateAp *state ); + + virtual void writeData(); + virtual void writeExec(); +}; + +#endif diff --git a/ragel/csflat.cpp b/ragel/csflat.cpp new file mode 100644 index 0000000..3762839 --- /dev/null +++ b/ragel/csflat.cpp @@ -0,0 +1,884 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "csflat.h" +#include "redfsm.h" +#include "gendata.h" + +std::ostream &CSharpFlatCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + out << act; + return out; +} + +std::ostream &CSharpFlatCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + out << act; + return out; +} + +std::ostream &CSharpFlatCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + out << act; + return out; +} + +std::ostream &CSharpFlatCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + out << act; + return out; +} + +std::ostream &CSharpFlatCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpFlatCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpFlatCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, true ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &CSharpFlatCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &CSharpFlatCodeGen::FLAT_INDEX_OFFSET() +{ + out << "\t"; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the index offset ahead. */ + if ( st->transList != 0 ) + curIndOffset += keyOps->span( st->lowKey, st->highKey ); + + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + out << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::KEY_SPANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + unsigned long long span = 0; + if ( st->transList != 0 ) + span = keyOps->span( st->lowKey, st->highKey ); + out << span; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::TO_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + TO_STATE_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::FROM_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + FROM_STATE_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::EOF_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + EOF_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::EOF_TRANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + out << trans; + + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +std::ostream &CSharpFlatCodeGen::COND_KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit just cond low key and cond high key. */ + out << ALPHA_KEY( st->condLowKey ) << ", "; + out << ALPHA_KEY( st->condHighKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << "(char) " << 0 << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::COND_KEY_SPANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + unsigned long long span = 0; + if ( st->condList != 0 ) + span = keyOps->span( st->condLowKey, st->condHighKey ); + out << span; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::CONDS() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->condList != 0 ) { + /* Walk the singles. */ + unsigned long long span = keyOps->span( st->condLowKey, st->condHighKey ); + for ( unsigned long long pos = 0; pos < span; pos++ ) { + if ( st->condList[pos] != 0 ) + out << st->condList[pos]->condSpaceId + 1 << ", "; + else + out << "0, "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::COND_INDEX_OFFSET() +{ + out << "\t"; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the index offset ahead. */ + if ( st->condList != 0 ) + curIndOffset += keyOps->span( st->condLowKey, st->condHighKey ); + } + out << "\n"; + return out; +} + + +std::ostream &CSharpFlatCodeGen::KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit just low key and high key. */ + out << ALPHA_KEY( st->lowKey ) << ", "; + out << ALPHA_KEY( st->highKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << "(char) " << 0 << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::INDICIES() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->transList != 0 ) { + /* Walk the singles. */ + unsigned long long span = keyOps->span( st->lowKey, st->highKey ); + for ( unsigned long long pos = 0; pos < span; pos++ ) { + out << st->transList[pos]->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) + out << st->defTrans->id << ", "; + + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::TRANS_TARGS() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalStates = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Record the position, need this for eofTrans. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + out << trans->targ->id; + if ( t < redFsm->transSet.length()-1 ) { + out << ", "; + if ( ++totalStates % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + + +std::ostream &CSharpFlatCodeGen::TRANS_ACTIONS() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalAct = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + TRANS_ACTION( trans ); + if ( t < redFsm->transSet.length()-1 ) { + out << ", "; + if ( ++totalAct % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + +void CSharpFlatCodeGen::LOCATE_TRANS() +{ + out << + " _keys = " << vCS() << "<<1;\n" + " _inds = " << IO() << "[" << vCS() << "];\n" + "\n" + " _slen = " << SP() << "[" << vCS() << "];\n" + " _trans = " << I() << "[_inds + (\n" + " _slen > 0 && " << K() << "[_keys] <=" << GET_WIDE_KEY() << " &&\n" + " " << GET_WIDE_KEY() << " <= " << K() <<"[_keys+1] ?\n" + " " << GET_WIDE_KEY() << " - " << K() << "[_keys] : _slen ) ];\n" + "\n"; +} + +void CSharpFlatCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{" << vCS() << " = " << gotoDest << "; " << + CTRL_FLOW() << "goto _again;}"; +} + +void CSharpFlatCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; +} + +void CSharpFlatCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void CSharpFlatCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void CSharpFlatCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void CSharpFlatCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + +void CSharpFlatCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = " << + callDest << "; " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + + +void CSharpFlatCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + + +void CSharpFlatCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << TOP() << "];"; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false ); + ret << "}"; + } + + ret << CTRL_FLOW() << "goto _again;}"; +} + +void CSharpFlatCodeGen::BREAK( ostream &ret, int targState ) +{ + outLabelUsed = true; + ret << "{" << P() << "++; " << CTRL_FLOW() << "goto _out; }"; +} + +void CSharpFlatCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpan), CSP() ); + COND_KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCond), C() ); + CONDS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondIndexOffset), CO() ); + COND_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSpan), SP() ); + KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxFlatIndexOffset), IO() ); + FLAT_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void CSharpFlatCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << GET_KEY() << ";\n"; + + out << + " _keys = " << vCS() << "<<1;\n" + " _conds = " << CO() << "[" << vCS() << "];\n" +// " _keys = " << ARR_OFF( CK(), "(" + vCS() + "<<1)" ) << ";\n" +// " _conds = " << ARR_OFF( C(), CO() + "[" + vCS() + "]" ) << ";\n" + "\n" + " _slen = " << CSP() << "[" << vCS() << "];\n" + " if (_slen > 0 && " << CK() << "[_keys] <=" + << GET_WIDE_KEY() << " &&\n" + " " << GET_WIDE_KEY() << " <= " << CK() << "[_keys+1])\n" + " _cond = " << C() << "[_conds+" << GET_WIDE_KEY() << " - " << + CK() << "[_keys]];\n" + " else\n" + " _cond = 0;" + "\n"; + /* XXX This version of the code doesn't work because Mono is weird. Works + * fine in Microsoft's csc, even though the bug report filed claimed it + * didn't. + " _slen = " << CSP() << "[" << vCS() << "];\n" + " _cond = _slen > 0 && " << CK() << "[_keys] <=" + << GET_WIDE_KEY() << " &&\n" + " " << GET_WIDE_KEY() << " <= " << CK() << "[_keys+1] ?\n" + " " << C() << "[_conds+" << GET_WIDE_KEY() << " - " << CK() + << "[_keys]] : 0;\n" + "\n"; + */ + out << + " switch ( _cond ) {\n"; + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " case " << condSpace->condSpaceId + 1 << ": {\n"; + out << TABS(2) << "_widec = " << CAST(WIDE_ALPH_TYPE()) << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "));\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(2) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } + + out << " }\n"; + out << " break;\n"; + } + + SWITCH_DEFAULT(); + + out << + " }\n"; +} + +void CSharpFlatCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + initVarTypes(); + + out << + " {\n" + " " << slenType << " _slen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << + ";\n" + " " << transType << " _trans"; + + if ( redFsm->anyConditions() ) + out << ", _cond"; + out << ";\n"; + + if ( redFsm->anyToStateActions() || + redFsm->anyRegActions() || redFsm->anyFromStateActions() ) + { + out << + " " << actsType << " _acts;\n" + " " << nactsType << " _nacts;\n"; + } + + out << + " " << "int _keys;\n" + " " << indsType << " _inds;\n"; + /* + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_keys;\n" + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxIndex) << POINTER() << "_inds;\n";*/ + + if ( redFsm->anyConditions() ) { + out << + " " << condsType << " _conds;\n" + " " << WIDE_ALPH_TYPE() << " _widec;\n"; + } + + out << "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << FSA() << "[" << vCS() << "];\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] == 0 )\n" + " goto _again;\n" + "\n" + " _acts = " << TA() << "[_trans];\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] )\n {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << TSA() << "[" << vCS() << "];\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << CAST(transType) << " (" << ET() << + "[" << vCS() << "] - 1);\n" + " goto _eof_trans;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxActArrItem) << + POINTER() << "__acts = " << + EA() << "[" << vCS() << "];\n" + " " << UINT() << " __nacts = " << CAST(UINT()) << " " << + A() << "[__acts++];\n" + " while ( __nacts-- > 0 ) {\n" + " switch ( " << A() << "[__acts++] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} + +void CSharpFlatCodeGen::initVarTypes() +{ + slenType = ARRAY_TYPE(MAX(redFsm->maxSpan, redFsm->maxCondSpan)); + transType = ARRAY_TYPE(redFsm->maxIndex+1); + actsType = ARRAY_TYPE(redFsm->maxActionLoc); + nactsType = ARRAY_TYPE(redFsm->maxActArrItem); + indsType = ARRAY_TYPE(redFsm->maxFlatIndexOffset); + condsType = ARRAY_TYPE(redFsm->maxCondIndexOffset); +} diff --git a/ragel/csflat.h b/ragel/csflat.h new file mode 100644 index 0000000..9f55f61 --- /dev/null +++ b/ragel/csflat.h @@ -0,0 +1,91 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CSFLAT_H +#define _CSFLAT_H + +#include <iostream> +#include "cscodegen.h" + +/* Forwards. */ +struct CodeGenData; +struct NameInst; +struct RedTransAp; +struct RedStateAp; + +/* + * CSharpFlatCodeGen + */ +class CSharpFlatCodeGen : virtual public CSharpFsmCodeGen, public CSharpCodeGen +{ +public: + CSharpFlatCodeGen( ostream &out ) : CSharpFsmCodeGen(out), CSharpCodeGen(out) {} + virtual ~CSharpFlatCodeGen() { } + +protected: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &FLAT_INDEX_OFFSET(); + std::ostream &KEY_SPANS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + void LOCATE_TRANS(); + + std::ostream &COND_INDEX_OFFSET(); + void COND_TRANSLATE(); + std::ostream &CONDS(); + std::ostream &COND_KEYS(); + std::ostream &COND_KEY_SPANS(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState ); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); + + void initVarTypes(); + string slenType, transType, actsType, nactsType, indsType, condsType; +}; + +#endif diff --git a/ragel/csftable.cpp b/ragel/csftable.cpp new file mode 100644 index 0000000..44378e8 --- /dev/null +++ b/ragel/csftable.cpp @@ -0,0 +1,438 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "csftable.h" +#include "redfsm.h" +#include "gendata.h" + +/* Determine if we should use indicies or not. */ +void CSharpFTabCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActListId) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActListId) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +std::ostream &CSharpFTabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &CSharpFTabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &CSharpFTabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + out << act; + return out; +} + + +/* Write out the function for a transition. */ +std::ostream &CSharpFTabCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + int action = 0; + if ( trans->action != 0 ) + action = trans->action->actListId+1; + out << action; + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &CSharpFTabCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &CSharpFTabCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpFTabCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &CSharpFTabCodeGen::ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +void CSharpFTabCodeGen::writeData() +{ + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + "\n"; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void CSharpFTabCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + initVarTypes(); + + out << + " {\n" + " " << klenType << " _klen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << + ";\n" + " " << keysType << " _keys;\n" + " " << transType << " _trans;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + out << "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " switch ( " << FSA() << "[" << vCS() << "] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + out << "_match:\n"; + + if ( useIndicies ) + out << " _trans = " << CAST(transType) << I() << "[_trans];\n"; + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] == 0 )\n" + " goto _again;\n" + "\n" + " switch ( " << TA() << "[_trans] ) {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " switch ( " << TSA() << "[" << vCS() << "] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << CAST(transType) << " (" << ET() << + "[" << vCS() << "] - 1);\n" + " goto _eof_trans;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " switch ( " << EA() << "[" << vCS() << "] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/ragel/csftable.h b/ragel/csftable.h new file mode 100644 index 0000000..612ec32 --- /dev/null +++ b/ragel/csftable.h @@ -0,0 +1,56 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CSFTABLE_H +#define _CSFTABLE_H + +#include <iostream> +#include "cstable.h" + +/* Forwards. */ +struct CodeGenData; + + +/* + * CSharpFTabCodeGen + */ +class CSharpFTabCodeGen : public CSharpTabCodeGen +{ +public: + CSharpFTabCodeGen( ostream &out ) : CSharpFsmCodeGen(out), CSharpTabCodeGen(out) {} +private: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + virtual void writeData(); + virtual void writeExec(); + virtual void calcIndexSize(); +}; + +#endif diff --git a/ragel/csgoto.cpp b/ragel/csgoto.cpp new file mode 100644 index 0000000..4bb2ec1 --- /dev/null +++ b/ragel/csgoto.cpp @@ -0,0 +1,801 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "csgoto.h" +#include "redfsm.h" +#include "bstmap.h" +#include "gendata.h" + +/* Emit the goto to take for a given transition. */ +std::ostream &CSharpGotoCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + out << TABS(level) << "goto tr" << trans->id << ";"; + return out; +} + +std::ostream &CSharpGotoCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpGotoCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpGotoCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, true ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpGotoCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +void CSharpGotoCodeGen::GOTO_HEADER( RedStateAp *state ) +{ + /* Label the state. */ + out << "case " << state->id << ":\n"; +} + + +void CSharpGotoCodeGen::emitSingleSwitch( RedStateAp *state ) +{ + /* Load up the singles. */ + int numSingles = state->outSingle.length(); + RedTransEl *data = state->outSingle.data; + + if ( numSingles == 1 ) { + /* If there is a single single key then write it out as an if. */ + out << "\tif ( " << GET_WIDE_KEY(state) << " == " << + KEY(data[0].lowKey) << " )\n\t\t"; + + /* Virtual function for writing the target of the transition. */ + TRANS_GOTO(data[0].value, 0) << "\n"; + } + else if ( numSingles > 1 ) { + /* Write out single keys in a switch if there is more than one. */ + out << "\tswitch( " << GET_WIDE_KEY(state) << " ) {\n"; + + /* Write out the single indicies. */ + for ( int j = 0; j < numSingles; j++ ) { + out << "\t\tcase " << ALPHA_KEY(data[j].lowKey) << ": "; + TRANS_GOTO(data[j].value, 0) << "\n"; + } + + /* Emits a default case for D code. */ + SWITCH_DEFAULT(); + + /* Close off the transition switch. */ + out << "\t}\n"; + } +} + +void CSharpGotoCodeGen::emitRangeBSearch( RedStateAp *state, int level, int low, int high ) +{ + /* Get the mid position, staying on the lower end of the range. */ + int mid = (low + high) >> 1; + RedTransEl *data = state->outRange.data; + + /* Determine if we need to look higher or lower. */ + bool anyLower = mid > low; + bool anyHigher = mid < high; + + /* Determine if the keys at mid are the limits of the alphabet. */ + bool limitLow = data[mid].lowKey == keyOps->minKey; + bool limitHigh = data[mid].highKey == keyOps->maxKey; + + if ( anyLower && anyHigher ) { + /* Can go lower and higher than mid. */ + out << TABS(level) << "if ( " << GET_WIDE_KEY(state) << " < " << + KEY(data[mid].lowKey) << " ) {\n"; + emitRangeBSearch( state, level+1, low, mid-1 ); + out << TABS(level) << "} else if ( " << GET_WIDE_KEY(state) << " > " << + KEY(data[mid].highKey) << " ) {\n"; + emitRangeBSearch( state, level+1, mid+1, high ); + out << TABS(level) << "} else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else if ( anyLower && !anyHigher ) { + /* Can go lower than mid but not higher. */ + out << TABS(level) << "if ( " << GET_WIDE_KEY(state) << " < " << + KEY(data[mid].lowKey) << " ) {\n"; + emitRangeBSearch( state, level+1, low, mid-1 ); + + /* if the higher is the highest in the alphabet then there is no + * sense testing it. */ + if ( limitHigh ) { + out << TABS(level) << "} else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + } + else if ( !anyLower && anyHigher ) { + /* Can go higher than mid but not lower. */ + out << TABS(level) << "if ( " << GET_WIDE_KEY(state) << " > " << + KEY(data[mid].highKey) << " ) {\n"; + emitRangeBSearch( state, level+1, mid+1, high ); + + /* If the lower end is the lowest in the alphabet then there is no + * sense testing it. */ + if ( limitLow ) { + out << TABS(level) << "} else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_WIDE_KEY(state) << " >= " << + KEY(data[mid].lowKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + } + else { + /* Cannot go higher or lower than mid. It's mid or bust. What + * tests to do depends on limits of alphabet. */ + if ( !limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << KEY(data[mid].lowKey) << " <= " << + GET_WIDE_KEY(state) << " && " << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else if ( limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else if ( !limitLow && limitHigh ) { + out << TABS(level) << "if ( " << KEY(data[mid].lowKey) << " <= " << + GET_WIDE_KEY(state) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + /* Both high and low are at the limit. No tests to do. */ + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + } +} + +void CSharpGotoCodeGen::STATE_GOTO_ERROR() +{ + /* Label the state and bail immediately. */ + outLabelUsed = true; + RedStateAp *state = redFsm->errState; + out << "case " << state->id << ":\n"; + out << " goto _out;\n"; +} + +void CSharpGotoCodeGen::COND_TRANSLATE( GenStateCond *stateCond, int level ) +{ + GenCondSpace *condSpace = stateCond->condSpace; + out << TABS(level) << "_widec = " << CAST(WIDE_ALPH_TYPE()) << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "));\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(level) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } +} + +void CSharpGotoCodeGen::emitCondBSearch( RedStateAp *state, int level, int low, int high ) +{ + /* Get the mid position, staying on the lower end of the range. */ + int mid = (low + high) >> 1; + GenStateCond **data = state->stateCondVect.data; + + /* Determine if we need to look higher or lower. */ + bool anyLower = mid > low; + bool anyHigher = mid < high; + + /* Determine if the keys at mid are the limits of the alphabet. */ + bool limitLow = data[mid]->lowKey == keyOps->minKey; + bool limitHigh = data[mid]->highKey == keyOps->maxKey; + + if ( anyLower && anyHigher ) { + /* Can go lower and higher than mid. */ + out << TABS(level) << "if ( " << GET_KEY() << " < " << + KEY(data[mid]->lowKey) << " ) {\n"; + emitCondBSearch( state, level+1, low, mid-1 ); + out << TABS(level) << "} else if ( " << GET_KEY() << " > " << + KEY(data[mid]->highKey) << " ) {\n"; + emitCondBSearch( state, level+1, mid+1, high ); + out << TABS(level) << "} else {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else if ( anyLower && !anyHigher ) { + /* Can go lower than mid but not higher. */ + out << TABS(level) << "if ( " << GET_KEY() << " < " << + KEY(data[mid]->lowKey) << " ) {\n"; + emitCondBSearch( state, level+1, low, mid-1 ); + + /* if the higher is the highest in the alphabet then there is no + * sense testing it. */ + if ( limitHigh ) { + out << TABS(level) << "} else {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + } + else if ( !anyLower && anyHigher ) { + /* Can go higher than mid but not lower. */ + out << TABS(level) << "if ( " << GET_KEY() << " > " << + KEY(data[mid]->highKey) << " ) {\n"; + emitCondBSearch( state, level+1, mid+1, high ); + + /* If the lower end is the lowest in the alphabet then there is no + * sense testing it. */ + if ( limitLow ) { + out << TABS(level) << "} else {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_KEY() << " >= " << + KEY(data[mid]->lowKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + } + else { + /* Cannot go higher or lower than mid. It's mid or bust. What + * tests to do depends on limits of alphabet. */ + if ( !limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << KEY(data[mid]->lowKey) << " <= " << + GET_KEY() << " && " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else if ( limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else if ( !limitLow && limitHigh ) { + out << TABS(level) << "if ( " << KEY(data[mid]->lowKey) << " <= " << + GET_KEY() << " )\n {"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else { + /* Both high and low are at the limit. No tests to do. */ + COND_TRANSLATE(data[mid], level); + } + } +} + +std::ostream &CSharpGotoCodeGen::STATE_GOTOS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st == redFsm->errState ) + STATE_GOTO_ERROR(); + else { + /* Writing code above state gotos. */ + GOTO_HEADER( st ); + + if ( st->stateCondVect.length() > 0 ) { + out << " _widec = " << GET_KEY() << ";\n"; + emitCondBSearch( st, 1, 0, st->stateCondVect.length() - 1 ); + } + + /* Try singles. */ + if ( st->outSingle.length() > 0 ) + emitSingleSwitch( st ); + + /* Default case is to binary search for the ranges, if that fails then */ + if ( st->outRange.length() > 0 ) + emitRangeBSearch( st, 1, 0, st->outRange.length() - 1 ); + + /* Write the default transition. */ + TRANS_GOTO( st->defTrans, 1 ) << "\n"; + } + } + return out; +} + +std::ostream &CSharpGotoCodeGen::TRANSITIONS() +{ + /* Emit any transitions that have functions and that go to + * this state. */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* Write the label for the transition so it can be jumped to. */ + out << " tr" << trans->id << ": "; + + /* Destination state. */ + if ( trans->action != 0 && trans->action->anyCurStateRef() ) + out << "_ps = " << vCS() << ";"; + out << vCS() << " = " << trans->targ->id << "; "; + + if ( trans->action != 0 ) { + /* Write out the transition func. */ + out << "goto f" << trans->action->actListId << ";\n"; + } + else { + /* No code to execute, just loop around. */ + out << "goto _again;\n"; + } + } + return out; +} + +std::ostream &CSharpGotoCodeGen::EXEC_FUNCS() +{ + /* Make labels that set acts and jump to execFuncs. Loop func indicies. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + out << " f" << redAct->actListId << ": " << + "_acts = " << itoa( redAct->location+1 ) << ";" + " goto execFuncs;\n"; + } + } + + out << + "\n" + "execFuncs:\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + " goto _again;\n"; + return out; +} + +unsigned int CSharpGotoCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + return act; +} + +unsigned int CSharpGotoCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + return act; +} + +unsigned int CSharpGotoCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + return act; +} + +std::ostream &CSharpGotoCodeGen::TO_STATE_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = TO_STATE_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &CSharpGotoCodeGen::FROM_STATE_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = FROM_STATE_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &CSharpGotoCodeGen::EOF_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = EOF_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &CSharpGotoCodeGen::FINISH_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* States that are final and have an out action need a case. */ + if ( st->eofAction != 0 ) { + /* Write the case label. */ + out << "\t\tcase " << st->id << ": "; + + /* Write the goto func. */ + out << "goto f" << st->eofAction->actListId << ";\n"; + } + } + + return out; +} + +void CSharpGotoCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{" << vCS() << " = " << gotoDest << "; " << + CTRL_FLOW() << "goto _again;}"; +} + +void CSharpGotoCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; +} + +void CSharpGotoCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void CSharpGotoCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void CSharpGotoCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void CSharpGotoCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + +void CSharpGotoCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = " << + callDest << "; " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void CSharpGotoCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void CSharpGotoCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << TOP() << "];"; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false ); + ret << "}"; + } + + ret << CTRL_FLOW() << "goto _again;}"; +} + +void CSharpGotoCodeGen::BREAK( ostream &ret, int targState ) +{ + outLabelUsed = true; + ret << "{" << P() << "++; " << CTRL_FLOW() << "goto _out; }"; +} + +void CSharpGotoCodeGen::writeData() +{ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void CSharpGotoCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << " {\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " int _ps = 0;\n"; + + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + { + out << + " " << ARRAY_TYPE(redFsm->maxActionLoc) << " _acts;\n" + " " << ARRAY_TYPE(redFsm->maxActArrItem) << " _nacts;\n"; + } + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + out << "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << FSA() << "[" << vCS() << "];\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + out << + " switch ( " << vCS() << " ) {\n"; + STATE_GOTOS(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + TRANSITIONS() << + "\n"; + + if ( redFsm->anyRegActions() ) + EXEC_FUNCS() << "\n"; + + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << TSA() << "[" << vCS() << "];\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " switch ( " << vCS() << " ) {\n"; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << " case " << st->id << ": goto tr" << st->eofTrans->id << ";\n"; + } + + SWITCH_DEFAULT() << + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " " << ARRAY_TYPE(redFsm->maxActionLoc) << " __acts = " << + EA() << "[" << vCS() << "];\n" + " " << ARRAY_TYPE(redFsm->maxActArrItem) << " __nacts = " << + A() << "[__acts++];\n" + " while ( __nacts-- > 0 ) {\n" + " switch ( " << A() << "[__acts++] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/ragel/csgoto.h b/ragel/csgoto.h new file mode 100644 index 0000000..7491884 --- /dev/null +++ b/ragel/csgoto.h @@ -0,0 +1,89 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _GOTOCODEGEN_H +#define _GOTOCODEGEN_H + +#include <iostream> +#include "cscodegen.h" + +/* Forwards. */ +struct CodeGenData; +struct NameInst; +struct RedTransAp; +struct RedStateAp; +struct GenStateCond; + +/* + * Goto driven fsm. + */ +class CSharpGotoCodeGen : virtual public CSharpFsmCodeGen, public CSharpCodeGen +{ +public: + CSharpGotoCodeGen( ostream &out ) : CSharpFsmCodeGen(out), CSharpCodeGen(out) {} + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + std::ostream &STATE_GOTOS(); + std::ostream &TRANSITIONS(); + std::ostream &EXEC_FUNCS(); + std::ostream &FINISH_CASES(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState ); + + virtual unsigned int TO_STATE_ACTION( RedStateAp *state ); + virtual unsigned int FROM_STATE_ACTION( RedStateAp *state ); + virtual unsigned int EOF_ACTION( RedStateAp *state ); + + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + + void COND_TRANSLATE( GenStateCond *stateCond, int level ); + void emitCondBSearch( RedStateAp *state, int level, int low, int high ); + void STATE_CONDS( RedStateAp *state, bool genDefault ); + + virtual std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + + void emitSingleSwitch( RedStateAp *state ); + void emitRangeBSearch( RedStateAp *state, int level, int low, int high ); + + /* Called from STATE_GOTOS just before writing the gotos */ + virtual void GOTO_HEADER( RedStateAp *state ); + virtual void STATE_GOTO_ERROR(); + + virtual void writeData(); + virtual void writeExec(); +}; + +#endif diff --git a/ragel/csipgoto.cpp b/ragel/csipgoto.cpp new file mode 100644 index 0000000..0e3168a --- /dev/null +++ b/ragel/csipgoto.cpp @@ -0,0 +1,438 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "csipgoto.h" +#include "redfsm.h" +#include "gendata.h" +#include "bstmap.h" + +bool CSharpIpGotoCodeGen::useAgainLabel() +{ + return redFsm->anyRegActionRets() || + redFsm->anyRegActionByValControl() || + redFsm->anyRegNextStmt(); +} + +void CSharpIpGotoCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{" << CTRL_FLOW() << "goto st" << gotoDest << ";}"; +} + +void CSharpIpGotoCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << targState << + "; " << CTRL_FLOW() << "goto st" << callDest << ";}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void CSharpIpGotoCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << targState << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void CSharpIpGotoCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << TOP() << "];"; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false ); + ret << "}"; + } + + ret << CTRL_FLOW() << "goto _again;}"; +} + +void CSharpIpGotoCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; +} + +void CSharpIpGotoCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void CSharpIpGotoCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + +void CSharpIpGotoCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void CSharpIpGotoCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << targState; +} + +void CSharpIpGotoCodeGen::BREAK( ostream &ret, int targState ) +{ + /* FIXME: If this code generator is made active then BREAK generation + * needs to check csForced. */ + outLabelUsed = true; + ret << "{" << P() << "++; " << vCS() << " = " << targState << + "; " << CTRL_FLOW() << "goto _out;}"; +} + +bool CSharpIpGotoCodeGen::IN_TRANS_ACTIONS( RedStateAp *state ) +{ + bool anyWritten = false; + + /* Emit any transitions that have actions and that go to this state. */ + for ( int it = 0; it < state->numInTrans; it++ ) { + RedTransAp *trans = state->inTrans[it]; + if ( trans->action != 0 && trans->labelNeeded ) { + /* Remember that we wrote an action so we know to write the + * line directive for going back to the output. */ + anyWritten = true; + + /* Write the label for the transition so it can be jumped to. */ + out << "tr" << trans->id << ":\n"; + + /* If the action contains a next, then we must preload the current + * state since the action may or may not set it. */ + if ( trans->action->anyNextStmt() ) + out << " " << vCS() << " = " << trans->targ->id << ";\n"; + + /* Write each action in the list. */ + for ( GenActionTable::Iter item = trans->action->key; item.lte(); item++ ) + ACTION( out, item->value, trans->targ->id, false ); + + /* If the action contains a next then we need to reload, otherwise + * jump directly to the target state. */ + if ( trans->action->anyNextStmt() ) + out << "\tgoto _again;\n"; + else + out << "\tgoto st" << trans->targ->id << ";\n"; + } + } + + return anyWritten; +} + +/* Called from GotoCodeGen::STATE_GOTOS just before writing the gotos for each + * state. */ +void CSharpIpGotoCodeGen::GOTO_HEADER( RedStateAp *state ) +{ + bool anyWritten = IN_TRANS_ACTIONS( state ); + + if ( state->labelNeeded ) + out << "st" << state->id << ":\n"; + + if ( state->toStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->toStateAction->key; item.lte(); item++ ) + ACTION( out, item->value, state->id, false ); + } + + /* Advance and test buffer pos. */ + if ( state->labelNeeded ) { + if ( !noEnd ) { + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _test_eof" << state->id << ";\n"; + } + else { + out << + " " << P() << " += 1;\n"; + } + } + + /* Give the state a switch case. */ + out << "case " << state->id << ":\n"; + + if ( state->fromStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->fromStateAction->key; item.lte(); item++ ) + ACTION( out, item->value, state->id, false ); + } + + if ( anyWritten ) + genLineDirective( out ); + + /* Record the prev state if necessary. */ + if ( state->anyRegCurStateRef() ) + out << " _ps = " << state->id << ";\n"; +} + +void CSharpIpGotoCodeGen::STATE_GOTO_ERROR() +{ + /* In the error state we need to emit some stuff that usually goes into + * the header. */ + RedStateAp *state = redFsm->errState; + bool anyWritten = IN_TRANS_ACTIONS( state ); + + /* No case label needed since we don't switch on the error state. */ + if ( anyWritten ) + genLineDirective( out ); + + if ( state->labelNeeded ) + out << "st" << state->id << ":\n"; + + /* Break out here. */ + outLabelUsed = true; + out << vCS() << " = " << state->id << ";\n"; + out << " goto _out;\n"; +} + + +/* Emit the goto to take for a given transition. */ +std::ostream &CSharpIpGotoCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + if ( trans->action != 0 ) { + /* Go to the transition which will go to the state. */ + out << TABS(level) << "goto tr" << trans->id << ";"; + } + else { + /* Go directly to the target state. */ + out << TABS(level) << "goto st" << trans->targ->id << ";"; + } + return out; +} + +std::ostream &CSharpIpGotoCodeGen::EXIT_STATES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->outNeeded ) { + testEofUsed = true; + out << " _test_eof" << st->id << ": " << vCS() << " = " << + st->id << "; goto _test_eof; \n"; + } + } + return out; +} + +std::ostream &CSharpIpGotoCodeGen::AGAIN_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + out << + " case " << st->id << ": goto st" << st->id << ";\n"; + } + return out; +} + +std::ostream &CSharpIpGotoCodeGen::FINISH_CASES() +{ + bool anyWritten = false; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofAction != 0 ) { + if ( st->eofAction->eofRefs == 0 ) + st->eofAction->eofRefs = new IntSet; + st->eofAction->eofRefs->insert( st->id ); + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << " case " << st->id << ": goto tr" << st->eofTrans->id << ";\n"; + } + + for ( GenActionTableMap::Iter act = redFsm->actionMap; act.lte(); act++ ) { + if ( act->eofRefs != 0 ) { + for ( IntSet::Iter pst = *act->eofRefs; pst.lte(); pst++ ) + out << " case " << *pst << ": \n"; + + /* Remember that we wrote a trans so we know to write the + * line directive for going back to the output. */ + anyWritten = true; + + /* Write each action in the eof action list. */ + for ( GenActionTable::Iter item = act->key; item.lte(); item++ ) + ACTION( out, item->value, STATE_ERR_STATE, true ); + out << "\tbreak;\n"; + } + } + + if ( anyWritten ) + genLineDirective( out ); + return out; +} + +void CSharpIpGotoCodeGen::setLabelsNeeded( GenInlineList *inlineList ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Goto: case GenInlineItem::Call: { + /* Mark the target as needing a label. */ + item->targState->labelNeeded = true; + break; + } + default: break; + } + + if ( item->children != 0 ) + setLabelsNeeded( item->children ); + } +} + +/* Set up labelNeeded flag for each state. */ +void CSharpIpGotoCodeGen::setLabelsNeeded() +{ + /* If we use the _again label, then we the _again switch, which uses all + * labels. */ + if ( useAgainLabel() ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = true; + } + else { + /* Do not use all labels by default, init all labelNeeded vars to false. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = false; + + /* Walk all transitions and set only those that have targs. */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* If there is no action with a next statement, then the label will be + * needed. */ + if ( trans->action == 0 || !trans->action->anyNextStmt() ) + trans->targ->labelNeeded = true; + + /* Need labels for states that have goto or calls in action code + * invoked on characters (ie, not from out action code). */ + if ( trans->action != 0 ) { + /* Loop the actions. */ + for ( GenActionTable::Iter act = trans->action->key; act.lte(); act++ ) { + /* Get the action and walk it's tree. */ + setLabelsNeeded( act->value->inlineList ); + } + } + } + } + + if ( !noEnd ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st != redFsm->errState ) + st->outNeeded = st->labelNeeded; + } + } +} + +void CSharpIpGotoCodeGen::writeData() +{ + STATE_IDS(); +} + +void CSharpIpGotoCodeGen::writeExec() +{ + /* Must set labels immediately before writing because we may depend on the + * noend write option. */ + setLabelsNeeded(); + testEofUsed = false; + outLabelUsed = false; + + out << " {\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " int _ps = 0;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( useAgainLabel() ) { + out << + " goto _resume;\n" + "\n" + "_again:\n" + " switch ( " << vCS() << " ) {\n"; + AGAIN_CASES() << + " default: break;\n" + " }\n" + "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + else { + out << + " " << P() << " += 1;\n"; + } + + out << "_resume:\n"; + } + + out << + " switch ( " << vCS() << " )\n {\n"; + STATE_GOTOS(); + SWITCH_DEFAULT() << + " }\n"; + EXIT_STATES() << + "\n"; + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n" + " switch ( " << vCS() << " ) {\n"; + FINISH_CASES(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << + " }\n"; +} diff --git a/ragel/csipgoto.h b/ragel/csipgoto.h new file mode 100644 index 0000000..4aeb47d --- /dev/null +++ b/ragel/csipgoto.h @@ -0,0 +1,75 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _IPGCODEGEN_H +#define _IPGCODEGEN_H + +#include <iostream> +#include "csgoto.h" + +/* Forwards. */ +struct CodeGenData; + +/* + * class CSharpIpGotoCodeGen + */ +class CSharpIpGotoCodeGen : public CSharpGotoCodeGen +{ +public: + CSharpIpGotoCodeGen( ostream &out ) : CSharpFsmCodeGen(out), + CSharpGotoCodeGen(out) {} + + std::ostream &EXIT_STATES(); + std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + std::ostream &FINISH_CASES(); + std::ostream &AGAIN_CASES(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void RET( ostream &ret, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void BREAK( ostream &ret, int targState ); + + virtual void writeData(); + virtual void writeExec(); + +protected: + bool useAgainLabel(); + + /* Called from GotoCodeGen::STATE_GOTOS just before writing the gotos for + * each state. */ + bool IN_TRANS_ACTIONS( RedStateAp *state ); + void GOTO_HEADER( RedStateAp *state ); + void STATE_GOTO_ERROR(); + + /* Set up labelNeeded flag for each state. */ + void setLabelsNeeded( GenInlineList *inlineList ); + void setLabelsNeeded(); +}; + +#endif diff --git a/ragel/cssplit.cpp b/ragel/cssplit.cpp new file mode 100644 index 0000000..8c7464f --- /dev/null +++ b/ragel/cssplit.cpp @@ -0,0 +1,518 @@ +/* + * Copyright 2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + + +#include "ragel.h" +#include "cssplit.h" +#include "gendata.h" +#include <assert.h> + +using std::ostream; +using std::ios; +using std::endl; + +/* Emit the goto to take for a given transition. */ +std::ostream &CSharpSplitCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + if ( trans->targ->partition == currentPartition ) { + if ( trans->action != 0 ) { + /* Go to the transition which will go to the state. */ + out << TABS(level) << "goto tr" << trans->id << ";"; + } + else { + /* Go directly to the target state. */ + out << TABS(level) << "goto st" << trans->targ->id << ";"; + } + } + else { + if ( trans->action != 0 ) { + /* Go to the transition which will go to the state. */ + out << TABS(level) << "goto ptr" << trans->id << ";"; + trans->partitionBoundary = true; + } + else { + /* Go directly to the target state. */ + out << TABS(level) << "goto pst" << trans->targ->id << ";"; + trans->targ->partitionBoundary = true; + } + } + return out; +} + +/* Called from before writing the gotos for each state. */ +void CSharpSplitCodeGen::GOTO_HEADER( RedStateAp *state, bool stateInPartition ) +{ + bool anyWritten = IN_TRANS_ACTIONS( state ); + + if ( state->labelNeeded ) + out << "st" << state->id << ":\n"; + + if ( state->toStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->toStateAction->key; item.lte(); item++ ) + ACTION( out, item->value, state->id, false ); + } + + /* Advance and test buffer pos. */ + if ( state->labelNeeded ) { + if ( !noEnd ) { + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _out" << state->id << ";\n"; + } + else { + out << + " " << P() << " += 1;\n"; + } + } + + /* Give the state a switch case. */ + out << "case " << state->id << ":\n"; + + if ( state->fromStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->fromStateAction->key; item.lte(); item++ ) + ACTION( out, item->value, state->id, false ); + } + + if ( anyWritten ) + genLineDirective( out ); + + /* Record the prev state if necessary. */ + if ( state->anyRegCurStateRef() ) + out << " _ps = " << state->id << ";\n"; +} + +std::ostream &CSharpSplitCodeGen::STATE_GOTOS( int partition ) +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->partition == partition ) { + if ( st == redFsm->errState ) + STATE_GOTO_ERROR(); + else { + /* We call into the base of the goto which calls back into us + * using virtual functions. Set the current partition rather + * than coding parameter passing throughout. */ + currentPartition = partition; + + /* Writing code above state gotos. */ + GOTO_HEADER( st, st->partition == partition ); + + if ( st->stateCondVect.length() > 0 ) { + out << " _widec = " << GET_KEY() << ";\n"; + emitCondBSearch( st, 1, 0, st->stateCondVect.length() - 1 ); + } + + /* Try singles. */ + if ( st->outSingle.length() > 0 ) + emitSingleSwitch( st ); + + /* Default case is to binary search for the ranges, if that fails then */ + if ( st->outRange.length() > 0 ) + emitRangeBSearch( st, 1, 0, st->outRange.length() - 1 ); + + /* Write the default transition. */ + TRANS_GOTO( st->defTrans, 1 ) << "\n"; + } + } + } + return out; +} + + +std::ostream &CSharpSplitCodeGen::PART_TRANS( int partition ) +{ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + if ( trans->partitionBoundary ) { + out << + "ptr" << trans->id << ":\n"; + + if ( trans->action != 0 ) { + /* If the action contains a next, then we must preload the current + * state since the action may or may not set it. */ + if ( trans->action->anyNextStmt() ) + out << " " << vCS() << " = " << trans->targ->id << ";\n"; + + /* Write each action in the list. */ + for ( GenActionTable::Iter item = trans->action->key; item.lte(); item++ ) + ACTION( out, item->value, trans->targ->id, false ); + } + + out << + " goto pst" << trans->targ->id << ";\n"; + trans->targ->partitionBoundary = true; + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->partitionBoundary ) { + out << + " pst" << st->id << ":\n" + " " << vCS() << " = " << st->id << ";\n"; + + if ( st->toStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + for ( GenActionTable::Iter item = st->toStateAction->key; item.lte(); item++ ) + ACTION( out, item->value, st->id, false ); + genLineDirective( out ); + } + + ptOutLabelUsed = true; + out << " goto _pt_out; \n"; + } + } + return out; +} + +std::ostream &CSharpSplitCodeGen::EXIT_STATES( int partition ) +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->partition == partition && st->outNeeded ) { + outLabelUsed = true; + out << " _out" << st->id << ": " << vCS() << " = " << + st->id << "; goto _out; \n"; + } + } + return out; +} + + +std::ostream &CSharpSplitCodeGen::PARTITION( int partition ) +{ + outLabelUsed = false; + ptOutLabelUsed = false; + + /* Initialize the partition boundaries, which get set during the writing + * of states. After the state writing we will */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + trans->partitionBoundary = false; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->partitionBoundary = false; + + out << " " << ALPH_TYPE() << " *p = *_pp, *pe = *_ppe;\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " int _ps = 0;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( useAgainLabel() ) { + out << + " goto _resume;\n" + "\n" + "_again:\n" + " switch ( " << vCS() << " ) {\n"; + AGAIN_CASES() << + " default: break;\n" + " }\n" + "\n"; + + + if ( !noEnd ) { + outLabelUsed = true; + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _out;\n"; + + } + else { + out << + " " << P() << " += 1;\n"; + } + + out << + "_resume:\n"; + } + + out << + " switch ( " << vCS() << " )\n {\n"; + STATE_GOTOS( partition ); + SWITCH_DEFAULT() << + " }\n"; + PART_TRANS( partition ); + EXIT_STATES( partition ); + + if ( outLabelUsed ) { + out << + "\n" + " _out:\n" + " *_pp = p;\n" + " *_ppe = pe;\n" + " return 0;\n"; + } + + if ( ptOutLabelUsed ) { + out << + "\n" + " _pt_out:\n" + " *_pp = p;\n" + " *_ppe = pe;\n" + " return 1;\n"; + } + + return out; +} + +std::ostream &CSharpSplitCodeGen::PART_MAP() +{ + int *partMap = new int[redFsm->stateList.length()]; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + partMap[st->id] = st->partition; + + out << "\t"; + int totalItem = 0; + for ( int i = 0; i < redFsm->stateList.length(); i++ ) { + out << partMap[i]; + if ( i != redFsm->stateList.length() - 1 ) { + out << ", "; + if ( ++totalItem % IALL == 0 ) + out << "\n\t"; + } + } + + delete[] partMap; + return out; +} + +void CSharpSplitCodeGen::writeData() +{ + out << + "const int " << START() << " = " << START_STATE_ID() << ";\n" + "\n"; + + if ( !noFinal ) { + out << + "const int " << FIRST_FINAL() << " = " << FIRST_FINAL_STATE() << ";\n" + "\n"; + } + + if ( !noError ) { + out << + "const int " << ERROR() << " = " << ERROR_STATE() << ";\n" + "\n"; + } + + + OPEN_ARRAY( ARRAY_TYPE(numSplitPartitions), PM() ); + PART_MAP(); + CLOSE_ARRAY() << + "\n"; + + for ( int p = 0; p < redFsm->nParts; p++ ) { + out << "int partition" << p << "( " << ALPH_TYPE() << " **_pp, " << ALPH_TYPE() << + " **_ppe, struct " << FSM_NAME() << " *fsm );\n"; + } + out << "\n"; +} + +std::ostream &CSharpSplitCodeGen::ALL_PARTITIONS() +{ + /* compute the format string. */ + int width = 0, high = redFsm->nParts - 1; + while ( high > 0 ) { + width++; + high /= 10; + } + assert( width <= 8 ); + char suffFormat[] = "_%6.6d.c"; + suffFormat[2] = suffFormat[4] = ( '0' + width ); + + for ( int p = 0; p < redFsm->nParts; p++ ) { + char suffix[10]; + sprintf( suffix, suffFormat, p ); + const char *fn = fileNameFromStem( sourceFileName, suffix ); + const char *include = fileNameFromStem( sourceFileName, ".h" ); + + /* Create the filter on the output and open it. */ + output_filter *partFilter = new output_filter( fn ); + partFilter->open( fn, ios::out|ios::trunc ); + if ( !partFilter->is_open() ) { + error() << "error opening " << fn << " for writing" << endl; + exit(1); + } + + /* Attach the new file to the output stream. */ + std::streambuf *prev_rdbuf = out.rdbuf( partFilter ); + + out << + "#include \"" << include << "\"\n" + "int partition" << p << "( " << ALPH_TYPE() << " **_pp, " << ALPH_TYPE() << + " **_ppe, struct " << FSM_NAME() << " *fsm )\n" + "{\n"; + PARTITION( p ) << + "}\n\n"; + out.flush(); + + /* Fix the output stream. */ + out.rdbuf( prev_rdbuf ); + } + return out; +} + + +void CSharpSplitCodeGen::writeExec() +{ + /* Must set labels immediately before writing because we may depend on the + * noend write option. */ + setLabelsNeeded(); + out << + " {\n" + " int _stat = 0;\n"; + + if ( !noEnd ) { + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _out;\n"; + } + + out << " goto _resume;\n"; + + /* In this reentry, to-state actions have already been executed on the + * partition-switch exit from the last partition. */ + out << "_reenter:\n"; + + if ( !noEnd ) { + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _out;\n"; + } + else { + out << + " " << P() << " += 1;\n"; + } + + out << "_resume:\n"; + + out << + " switch ( " << PM() << "[" << vCS() << "] ) {\n"; + for ( int p = 0; p < redFsm->nParts; p++ ) { + out << + " case " << p << ":\n" + " _stat = partition" << p << "( &p, &pe, fsm );\n" + " break;\n"; + } + out << + " }\n" + " if ( _stat )\n" + " goto _reenter;\n"; + + if ( !noEnd ) + out << " _out: {}\n"; + + out << + " }\n"; + + ALL_PARTITIONS(); +} + +void CSharpSplitCodeGen::setLabelsNeeded( RedStateAp *fromState, GenInlineList *inlineList ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Goto: case GenInlineItem::Call: { + /* In split code gen we only need labels for transitions across + * partitions. */ + if ( fromState->partition == item->targState->partition ){ + /* Mark the target as needing a label. */ + item->targState->labelNeeded = true; + } + break; + } + default: break; + } + + if ( item->children != 0 ) + setLabelsNeeded( fromState, item->children ); + } +} + +void CSharpSplitCodeGen::setLabelsNeeded( RedStateAp *fromState, RedTransAp *trans ) +{ + /* In the split code gen we don't need labels for transitions across + * partitions. */ + if ( fromState->partition == trans->targ->partition ) { + /* If there is no action with a next statement, then the label will be + * needed. */ + trans->labelNeeded = true; + if ( trans->action == 0 || !trans->action->anyNextStmt() ) + trans->targ->labelNeeded = true; + } + + /* Need labels for states that have goto or calls in action code + * invoked on characters (ie, not from out action code). */ + if ( trans->action != 0 ) { + /* Loop the actions. */ + for ( GenActionTable::Iter act = trans->action->key; act.lte(); act++ ) { + /* Get the action and walk it's tree. */ + setLabelsNeeded( fromState, act->value->inlineList ); + } + } +} + +/* Set up labelNeeded flag for each state. */ +void CSharpSplitCodeGen::setLabelsNeeded() +{ + /* If we use the _again label, then we the _again switch, which uses all + * labels. */ + if ( useAgainLabel() ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = true; + } + else { + /* Do not use all labels by default, init all labelNeeded vars to false. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = false; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + trans->labelNeeded = false; + + /* Walk all transitions and set only those that have targs. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + for ( RedTransList::Iter tel = st->outRange; tel.lte(); tel++ ) + setLabelsNeeded( st, tel->value ); + + for ( RedTransList::Iter tel = st->outSingle; tel.lte(); tel++ ) + setLabelsNeeded( st, tel->value ); + + if ( st->defTrans != 0 ) + setLabelsNeeded( st, st->defTrans ); + } + } + + if ( !noEnd ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->outNeeded = st->labelNeeded; + } + else { + if ( redFsm->errState != 0 ) + redFsm->errState->outNeeded = true; + + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* Any state with a transition in that has a break will need an + * out label. */ + if ( trans->action != 0 && trans->action->anyBreakStmt() ) + trans->targ->outNeeded = true; + } + } +} + diff --git a/ragel/cssplit.h b/ragel/cssplit.h new file mode 100644 index 0000000..9ff2d8f --- /dev/null +++ b/ragel/cssplit.h @@ -0,0 +1,53 @@ +/* + * Copyright 2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _SPLITCODEGEN_H +#define _SPLITCODEGEN_H + +#include "csipgoto.h" + +class CSharpSplitCodeGen : public CSharpIpGotoCodeGen +{ +public: + CSharpSplitCodeGen( ostream &out ) : CSharpFsmCodeGen(out), CSharpIpGotoCodeGen(out) {} + + bool ptOutLabelUsed; + + std::ostream &PART_MAP(); + std::ostream &EXIT_STATES( int partition ); + std::ostream &PART_TRANS( int partition ); + std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + void GOTO_HEADER( RedStateAp *state, bool stateInPartition ); + std::ostream &STATE_GOTOS( int partition ); + std::ostream &PARTITION( int partition ); + std::ostream &ALL_PARTITIONS(); + void writeData(); + void writeExec(); + void writeParts(); + + void setLabelsNeeded( RedStateAp *fromState, GenInlineList *inlineList ); + void setLabelsNeeded( RedStateAp *fromState, RedTransAp *trans ); + void setLabelsNeeded(); + + int currentPartition; +}; + +#endif diff --git a/ragel/cstable.cpp b/ragel/cstable.cpp new file mode 100644 index 0000000..cde4ba9 --- /dev/null +++ b/ragel/cstable.cpp @@ -0,0 +1,1117 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cstable.h" +#include "redfsm.h" +#include "gendata.h" + +/* Determine if we should use indicies or not. */ +void CSharpTabCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActionLoc) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActionLoc) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +std::ostream &CSharpTabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + out << act; + return out; +} + +std::ostream &CSharpTabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + out << act; + return out; +} + +std::ostream &CSharpTabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + out << act; + return out; +} + + +std::ostream &CSharpTabCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + out << act; + return out; +} + +std::ostream &CSharpTabCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpTabCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpTabCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, true ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &CSharpTabCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpTabCodeGen::COND_OFFSETS() +{ + out << "\t"; + int totalStateNum = 0, curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + out << curKeyOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the key offset ahead. */ + curKeyOffset += st->stateCondList.length(); + } + out << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::KEY_OFFSETS() +{ + out << "\t"; + int totalStateNum = 0, curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + out << curKeyOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the key offset ahead. */ + curKeyOffset += st->outSingle.length() + st->outRange.length()*2; + } + out << "\n"; + return out; +} + + +std::ostream &CSharpTabCodeGen::INDEX_OFFSETS() +{ + out << "\t"; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the index offset ahead. */ + curIndOffset += st->outSingle.length() + st->outRange.length(); + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + out << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::COND_LENS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + out << st->stateCondList.length(); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +std::ostream &CSharpTabCodeGen::SINGLE_LENS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + out << st->outSingle.length(); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::RANGE_LENS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit length of range index. */ + out << st->outRange.length(); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::TO_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + TO_STATE_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::FROM_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + FROM_STATE_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::EOF_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + EOF_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::EOF_TRANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + out << trans; + + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +std::ostream &CSharpTabCodeGen::COND_KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Lower key. */ + out << ALPHA_KEY( sc->lowKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + + /* Upper key. */ + out << ALPHA_KEY( sc->highKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::COND_SPACES() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Cond Space id. */ + out << sc->condSpace->condSpaceId << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + out << ALPHA_KEY( stel->lowKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Loop the state's transitions. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + /* Lower key. */ + out << ALPHA_KEY( rtel->lowKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + + /* Upper key. */ + out << ALPHA_KEY( rtel->highKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << "(char) " << 0 << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::INDICIES() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + out << stel->value->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + out << rtel->value->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + out << st->defTrans->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::TRANS_TARGS() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* The state's default target state. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + trans->pos = totalTrans; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + + +std::ostream &CSharpTabCodeGen::TRANS_ACTIONS() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::TRANS_TARGS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalStates = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Record the position, need this for eofTrans. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + out << trans->targ->id; + if ( t < redFsm->transSet.length()-1 ) { + out << ", "; + if ( ++totalStates % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + + +std::ostream &CSharpTabCodeGen::TRANS_ACTIONS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalAct = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + TRANS_ACTION( trans ); + if ( t < redFsm->transSet.length()-1 ) { + out << ", "; + if ( ++totalAct % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + +void CSharpTabCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{" << vCS() << " = " << gotoDest << "; " << + CTRL_FLOW() << "goto _again;}"; +} + +void CSharpTabCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; +} + +void CSharpTabCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void CSharpTabCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void CSharpTabCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void CSharpTabCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + +void CSharpTabCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = " << + callDest << "; " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void CSharpTabCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void CSharpTabCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << + TOP() << "]; "; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false ); + ret << "}"; + } + + ret << CTRL_FLOW() << "goto _again;}"; +} + +void CSharpTabCodeGen::BREAK( ostream &ret, int targState ) +{ + outLabelUsed = true; + ret << "{" << P() << "++; " << CTRL_FLOW() << "goto _out; }"; +} + +void CSharpTabCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + "\n"; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void CSharpTabCodeGen::LOCATE_TRANS() +{ + out << + " _keys = " << KO() + "[" + vCS() + "]" << ";\n" + " _trans = " << CAST(transType) << IO() << "[" << vCS() << "];\n" + "\n" + " _klen = " << SL() << "[" << vCS() << "];\n" + " if ( _klen > 0 ) {\n" + " " << signedKeysType << " _lower = _keys;\n" + " " << signedKeysType << " _mid;\n" + " " << signedKeysType << " _upper = " << CAST(signedKeysType) << + " (_keys + _klen - 1);\n" + " while (true) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = " << CAST(signedKeysType) << + " (_lower + ((_upper-_lower) >> 1));\n" + " if ( " << GET_WIDE_KEY() << " < " << K() << "[_mid] )\n" + " _upper = " << CAST(signedKeysType) << " (_mid - 1);\n" + " else if ( " << GET_WIDE_KEY() << " > " << K() << "[_mid] )\n" + " _lower = " << CAST(signedKeysType) << " (_mid + 1);\n" + " else {\n" + " _trans += " << CAST(transType) << " (_mid - _keys);\n" + " goto _match;\n" + " }\n" + " }\n" + " _keys += " << CAST(keysType) << " _klen;\n" + " _trans += " << CAST(transType) << " _klen;\n" + " }\n" + "\n" + " _klen = " << RL() << "[" << vCS() << "];\n" + " if ( _klen > 0 ) {\n" + " " << signedKeysType << " _lower = _keys;\n" + " " << signedKeysType << " _mid;\n" + " " << signedKeysType << " _upper = " << CAST(signedKeysType) << + " (_keys + (_klen<<1) - 2);\n" + " while (true) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = " << CAST(signedKeysType) << + " (_lower + (((_upper-_lower) >> 1) & ~1));\n" + " if ( " << GET_WIDE_KEY() << " < " << K() << "[_mid] )\n" + " _upper = " << CAST(signedKeysType) << " (_mid - 2);\n" + " else if ( " << GET_WIDE_KEY() << " > " << K() << "[_mid+1] )\n" + " _lower = " << CAST(signedKeysType) << " (_mid + 2);\n" + " else {\n" + " _trans += " << CAST(transType) << "((_mid - _keys)>>1);\n" + " goto _match;\n" + " }\n" + " }\n" + " _trans += " << CAST(transType) << " _klen;\n" + " }\n" + "\n"; +} + +void CSharpTabCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << GET_KEY() << ";\n" + " _klen = " << CL() << "[" << vCS() << "];\n" + " _keys = " << CAST(keysType) << " ("<< CO() << "[" << vCS() << "]*2);\n" + " if ( _klen > 0 ) {\n" + " " << signedKeysType << " _lower = _keys;\n" + " " << signedKeysType << " _mid;\n" + " " << signedKeysType << " _upper = " << CAST(signedKeysType) << + " (_keys + (_klen<<1) - 2);\n" + " while (true) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = " << CAST(signedKeysType) << + " (_lower + (((_upper-_lower) >> 1) & ~1));\n" + " if ( " << GET_WIDE_KEY() << " < " << CK() << "[_mid] )\n" + " _upper = " << CAST(signedKeysType) << " (_mid - 2);\n" + " else if ( " << GET_WIDE_KEY() << " > " << CK() << "[_mid+1] )\n" + " _lower = " << CAST(signedKeysType) << " (_mid + 2);\n" + " else {\n" + " switch ( " << C() << "[" << CO() << "[" << vCS() << "]" + " + ((_mid - _keys)>>1)] ) {\n"; + + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " case " << condSpace->condSpaceId << ": {\n"; + out << TABS(2) << "_widec = " << CAST(WIDE_ALPH_TYPE()) << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "));\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(2) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } + + out << + " break;\n" + " }\n"; + } + + SWITCH_DEFAULT(); + + out << + " }\n" + " break;\n" + " }\n" + " }\n" + " }\n" + "\n"; +} + +void CSharpTabCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + initVarTypes(); + + out << + " {\n" + " " << klenType << " _klen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << + ";\n" + " " << transType << " _trans;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + { + out << + " " << actsType << " _acts;\n" + " " << nactsType << " _nacts;\n"; + } + + out << + " " << keysType << " _keys;\n" + "\n"; +// " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_keys;\n" + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << FSA() << "[" + vCS() + "]" << ";\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + out << "_match:\n"; + + if ( useIndicies ) + out << " _trans = " << CAST(transType) << I() << "[_trans];\n"; + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] == 0 )\n" + " goto _again;\n" + "\n" + " _acts = " << TA() << "[_trans]" << ";\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 )\n {\n" + " switch ( " << A() << "[_acts++] )\n {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << TSA() << "[" << vCS() << "]" << ";\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << CAST(transType) << " (" << ET() << + "[" << vCS() << "] - 1);\n" + " goto _eof_trans;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " " << actsType << " __acts = " << + EA() << "[" << vCS() << "]" << ";\n" + " " << nactsType << " __nacts = " << + A() << "[__acts++];\n" + " while ( __nacts-- > 0 ) {\n" + " switch ( " << A() << "[__acts++] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} + +void CSharpTabCodeGen::initVarTypes() +{ + int klenMax = MAX(MAX(redFsm->maxCondLen, redFsm->maxRangeLen), + redFsm->maxSingleLen); + int keysMax = MAX(MAX(redFsm->maxKeyOffset, klenMax), + redFsm->maxCondOffset); + int transMax = MAX(MAX(redFsm->maxIndex+1, redFsm->maxIndexOffset), keysMax); + transMax = MAX(transMax, klenMax); + transType = ARRAY_TYPE(transMax); + klenType = ARRAY_TYPE(klenMax); + keysType = ARRAY_TYPE(keysMax); + signedKeysType = ARRAY_TYPE(keysMax, true); + actsType = ARRAY_TYPE(redFsm->maxActionLoc); + nactsType = ARRAY_TYPE(redFsm->maxActArrItem); +} diff --git a/ragel/cstable.h b/ragel/cstable.h new file mode 100644 index 0000000..623f670 --- /dev/null +++ b/ragel/cstable.h @@ -0,0 +1,102 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _TABCODEGEN_H +#define _TABCODEGEN_H + +#include <iostream> +#include "cscodegen.h" + +/* Forwards. */ +struct CodeGenData; +struct NameInst; +struct RedTransAp; +struct RedStateAp; + +/* + * TabCodeGen + */ +class CSharpTabCodeGen : virtual public CSharpFsmCodeGen, public CSharpCodeGen +{ +public: + CSharpTabCodeGen( ostream &out ) : CSharpFsmCodeGen(out), CSharpCodeGen(out) {} + virtual ~CSharpTabCodeGen() { } + virtual void writeData(); + virtual void writeExec(); + +protected: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + std::ostream &COND_KEYS(); + std::ostream &COND_SPACES(); + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &COND_OFFSETS(); + std::ostream &KEY_OFFSETS(); + std::ostream &INDEX_OFFSETS(); + std::ostream &COND_LENS(); + std::ostream &SINGLE_LENS(); + std::ostream &RANGE_LENS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + std::ostream &TRANS_TARGS_WI(); + std::ostream &TRANS_ACTIONS_WI(); + + void LOCATE_TRANS(); + + void COND_TRANSLATE(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState ); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + virtual void calcIndexSize(); + + void initVarTypes(); + string klenType; + string keysType; + string signedKeysType; + string transType; + string actsType; + string nactsType; +}; + +#endif diff --git a/ragel/dotcodegen.cpp b/ragel/dotcodegen.cpp new file mode 100644 index 0000000..a61bdb7 --- /dev/null +++ b/ragel/dotcodegen.cpp @@ -0,0 +1,319 @@ +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "dotcodegen.h" +#include "gendata.h" + +using std::istream; +using std::ifstream; +using std::ostream; +using std::ios; +using std::cin; +using std::cout; +using std::cerr; +using std::endl; + +/* Override this so that write statement processing is ignored */ +void GraphvizDotGen::writeStatement( InputLoc &, int, char ** ) +{ +} + +std::ostream &GraphvizDotGen::KEY( Key key ) +{ + if ( displayPrintables && key.isPrintable() ) { + // Output values as characters, ensuring we escape the quote (") character + char cVal = (char) key.getVal(); + switch ( cVal ) { + case '"': case '\\': + out << "'\\" << cVal << "'"; + break; + case '\a': + out << "'\\\\a'"; + break; + case '\b': + out << "'\\\\b'"; + break; + case '\t': + out << "'\\\\t'"; + break; + case '\n': + out << "'\\\\n'"; + break; + case '\v': + out << "'\\\\v'"; + break; + case '\f': + out << "'\\\\f'"; + break; + case '\r': + out << "'\\\\r'"; + break; + case ' ': + out << "SP"; + break; + default: + out << "'" << cVal << "'"; + break; + } + } + else { + if ( keyOps->isSigned ) + out << key.getVal(); + else + out << (unsigned long) key.getVal(); + } + + return out; +} + +std::ostream &GraphvizDotGen::TRANS_ACTION( RedStateAp *fromState, RedTransAp *trans ) +{ + int n = 0; + RedAction *actions[3]; + + if ( fromState->fromStateAction != 0 ) + actions[n++] = fromState->fromStateAction; + if ( trans->action != 0 ) + actions[n++] = trans->action; + if ( trans->targ != 0 && trans->targ->toStateAction != 0 ) + actions[n++] = trans->targ->toStateAction; + + if ( n > 0 ) + out << " / "; + + /* Loop the existing actions and write out what's there. */ + for ( int a = 0; a < n; a++ ) { + for ( GenActionTable::Iter actIt = actions[a]->key.first(); actIt.lte(); actIt++ ) { + GenAction *action = actIt->value; + out << action->nameOrLoc(); + if ( a < n-1 || !actIt.last() ) + out << ", "; + } + } + return out; +} + +std::ostream &GraphvizDotGen::ACTION( RedAction *action ) +{ + /* The action. */ + out << " / "; + for ( GenActionTable::Iter actIt = action->key.first(); actIt.lte(); actIt++ ) { + GenAction *action = actIt->value; + if ( action->name != 0 ) + out << action->name; + else + out << action->loc.line << ":" << action->loc.col; + if ( !actIt.last() ) + out << ", "; + } + return out; +} + +std::ostream &GraphvizDotGen::ONCHAR( Key lowKey, Key highKey ) +{ + GenCondSpace *condSpace; + if ( lowKey > keyOps->maxKey && (condSpace=findCondSpace(lowKey, highKey) ) ) { + Key values = ( lowKey - condSpace->baseKey ) / keyOps->alphSize(); + + lowKey = keyOps->minKey + + (lowKey - condSpace->baseKey - keyOps->alphSize() * values.getVal()); + highKey = keyOps->minKey + + (highKey - condSpace->baseKey - keyOps->alphSize() * values.getVal()); + KEY( lowKey ); + if ( lowKey != highKey ) { + out << ".."; + KEY( highKey ); + } + out << "("; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + bool set = values & (1 << csi.pos()); + if ( !set ) + out << "!"; + out << (*csi)->nameOrLoc(); + if ( !csi.last() ) + out << ", "; + } + out << ")"; + } + else { + /* Output the key. Possibly a range. */ + KEY( lowKey ); + if ( highKey != lowKey ) { + out << ".."; + KEY( highKey ); + } + } + return out; +} + +void GraphvizDotGen::writeTransList( RedStateAp *state ) +{ + /* Build the set of unique transitions out of this state. */ + RedTransSet stTransSet; + for ( RedTransList::Iter tel = state->outRange; tel.lte(); tel++ ) { + /* If we haven't seen the transitions before, the move forward + * emitting all the transitions on the same character. */ + if ( stTransSet.insert( tel->value ) ) { + /* Write out the from and to states. */ + out << "\t" << state->id << " -> "; + + if ( tel->value->targ == 0 ) + out << "err_" << state->id; + else + out << tel->value->targ->id; + + /* Begin the label. */ + out << " [ label = \""; + ONCHAR( tel->lowKey, tel->highKey ); + + /* Walk the transition list, finding the same. */ + for ( RedTransList::Iter mtel = tel.next(); mtel.lte(); mtel++ ) { + if ( mtel->value == tel->value ) { + out << ", "; + ONCHAR( mtel->lowKey, mtel->highKey ); + } + } + + /* Write the action and close the transition. */ + TRANS_ACTION( state, tel->value ); + out << "\" ];\n"; + } + } + + /* Write the default transition. */ + if ( state->defTrans != 0 ) { + /* Write out the from and to states. */ + out << "\t" << state->id << " -> "; + + if ( state->defTrans->targ == 0 ) + out << "err_" << state->id; + else + out << state->defTrans->targ->id; + + /* Begin the label. */ + out << " [ label = \"DEF"; + + /* Write the action and close the transition. */ + TRANS_ACTION( state, state->defTrans ); + out << "\" ];\n"; + } +} + +void GraphvizDotGen::writeDotFile( ) +{ + out << + "digraph " << fsmName << " {\n" + " rankdir=LR;\n"; + + /* Define the psuedo states. Transitions will be done after the states + * have been defined as either final or not final. */ + out << " node [ shape = point ];\n"; + + if ( redFsm->startState != 0 ) + out << " ENTRY;\n"; + + /* Psuedo states for entry points in the entry map. */ + for ( EntryIdVect::Iter en = entryPointIds; en.lte(); en++ ) { + RedStateAp *state = allStates + *en; + out << " en_" << state->id << ";\n"; + } + + /* Psuedo states for final states with eof actions. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 && st->eofTrans->action != 0 ) + out << " eof_" << st->id << ";\n"; + if ( st->eofAction != 0 ) + out << " eof_" << st->id << ";\n"; + } + + out << " node [ shape = circle, height = 0.2 ];\n"; + + /* Psuedo states for states whose default actions go to error. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + bool needsErr = false; + if ( st->defTrans != 0 && st->defTrans->targ == 0 ) + needsErr = true; + else { + for ( RedTransList::Iter tel = st->outRange; tel.lte(); tel++ ) { + if ( tel->value->targ == 0 ) { + needsErr = true; + break; + } + } + } + + if ( needsErr ) + out << " err_" << st->id << " [ label=\"\"];\n"; + } + + /* Attributes common to all nodes, plus double circle for final states. */ + out << " node [ fixedsize = true, height = 0.65, shape = doublecircle ];\n"; + + /* List Final states. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->isFinal ) + out << " " << st->id << ";\n"; + } + + /* List transitions. */ + out << " node [ shape = circle ];\n"; + + /* Walk the states. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + writeTransList( st ); + + /* Transitions into the start state. */ + if ( redFsm->startState != 0 ) + out << " ENTRY -> " << redFsm->startState->id << " [ label = \"IN\" ];\n"; + + /* Transitions into the entry points. */ + for ( EntryIdVect::Iter en = entryPointIds; en.lte(); en++ ) { + RedStateAp *state = allStates + *en; + char *name = entryPointNames[en.pos()]; + out << " en_" << state->id << " -> " << state->id << + " [ label = \"" << name << "\" ];\n"; + } + + /* Out action transitions. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 && st->eofTrans->action != 0 ) { + out << " " << st->id << " -> eof_" << + st->id << " [ label = \"EOF"; + ACTION( st->eofTrans->action ) << "\" ];\n"; + } + if ( st->eofAction != 0 ) { + out << " " << st->id << " -> eof_" << + st->id << " [ label = \"EOF"; + ACTION( st->eofAction ) << "\" ];\n"; + } + } + + out << + "}\n"; +} + +void GraphvizDotGen::finishRagelDef() +{ + /* For dot file generation we want to pick default transitions. */ + redFsm->chooseDefaultSpan(); +} diff --git a/ragel/dotcodegen.h b/ragel/dotcodegen.h new file mode 100644 index 0000000..54e09f3 --- /dev/null +++ b/ragel/dotcodegen.h @@ -0,0 +1,48 @@ +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _GVDOTGEN_H +#define _GVDOTGEN_H + +#include <iostream> +#include "gendata.h" + +class GraphvizDotGen : public CodeGenData +{ +public: + GraphvizDotGen( ostream &out ) : CodeGenData(out) { } + + /* Print an fsm to out stream. */ + void writeTransList( RedStateAp *state ); + void writeDotFile( ); + + virtual void finishRagelDef(); + virtual void writeStatement( InputLoc &, int, char ** ); + +private: + /* Writing labels and actions. */ + std::ostream &ONCHAR( Key lowKey, Key highKey ); + std::ostream &TRANS_ACTION( RedStateAp *fromState, RedTransAp *trans ); + std::ostream &ACTION( RedAction *action ); + std::ostream &KEY( Key key ); +}; + +#endif diff --git a/ragel/fsmap.cpp b/ragel/fsmap.cpp index 551aea0..12d9e56 100644 --- a/ragel/fsmap.cpp +++ b/ragel/fsmap.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2002-2004 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002-2004 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -122,6 +122,12 @@ void FsmAp::startFsmPrior( int ordering, PriorDesc *prior ) if ( trans->toState != 0 ) trans->priorTable.setPrior( ordering, prior ); } + + /* If the new start state is final then set the out priority. This follows + * the same convention as setting start action in the out action table of + * a final start state. */ + if ( startState->stateBits & STB_ISFINAL ) + startState->outPriorTable.setPrior( ordering, prior ); } /* Set the priority of all transitions in a graph. Walks all transition lists @@ -180,6 +186,12 @@ void FsmAp::startFsmAction( int ordering, Action *action ) if ( trans->toState != 0 ) trans->actionTable.setAction( ordering, action ); } + + /* If start state is final then add the action to the out action table. + * This means that when the null string is accepted the start action will + * not be bypassed. */ + if ( startState->stateBits & STB_ISFINAL ) + startState->outActionTable.setAction( ordering, action ); } /* Set functions to execute on all transitions. Walks the out lists of all @@ -290,6 +302,18 @@ void FsmAp::fillGaps( StateAp *state ) } } +void FsmAp::setErrorActions( StateAp *state, const ActionTable &other ) +{ + /* Fill any gaps in the out list with an error transition. */ + fillGaps( state ); + + /* Set error transitions in the transitions that go to error. */ + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) { + if ( trans->toState == 0 ) + trans->actionTable.setActions( other ); + } +} + void FsmAp::setErrorAction( StateAp *state, int ordering, Action *action ) { /* Fill any gaps in the out list with an error transition. */ @@ -320,6 +344,13 @@ void FsmAp::setErrorTarget( StateAp *state, StateAp *target, int *orderings, } } +void FsmAp::transferOutActions( StateAp *state ) +{ + for ( ActionTable::Iter act = state->outActionTable; act.lte(); act++ ) + state->eofActionTable.setAction( act->key, act->value ); + state->outActionTable.empty(); +} + void FsmAp::transferErrorActions( StateAp *state, int transferPoint ) { for ( int i = 0; i < state->errActionTable.length(); ) { @@ -327,6 +358,8 @@ void FsmAp::transferErrorActions( StateAp *state, int transferPoint ) if ( act->transferPoint == transferPoint ) { /* Transfer the error action and remove it. */ setErrorAction( state, act->ordering, act->action ); + if ( ! state->isFinState() ) + state->eofActionTable.setAction( act->ordering, act->action ); state->errActionTable.vremove( i ); } else { @@ -637,14 +670,14 @@ void FsmAp::verifyStates() { for ( StateList::Iter state = stateList; state.lte(); state++ ) { /* Non final states should not have leaving data. */ - if ( ! (state->stateBits & SB_ISFINAL) ) { + if ( ! (state->stateBits & STB_ISFINAL) ) { assert( state->outActionTable.length() == 0 ); assert( state->outCondSet.length() == 0 ); assert( state->outPriorTable.length() == 0 ); } /* Data used in algorithms should be cleared. */ - assert( (state->stateBits & SB_BOTH) == 0 ); + assert( (state->stateBits & STB_BOTH) == 0 ); assert( state->foreignInTrans > 0 ); } } @@ -756,7 +789,7 @@ int FsmAp::compareStateData( const StateAp *state1, const StateAp *state2 ) return cmpRes; /* Test out condition sets. */ - cmpRes = CmpActionSet::compare( state1->outCondSet, + cmpRes = CmpOutCondSet::compare( state1->outCondSet, state2->outCondSet ); if ( cmpRes != 0 ) return cmpRes; @@ -802,8 +835,15 @@ CondSpace *FsmAp::addCondSpace( const CondSet &condSet ) { CondSpace *condSpace = condData->condSpaceMap.find( condSet ); if ( condSpace == 0 ) { - Key baseKey = condData->nextCondKey; - condData->nextCondKey += (1 << condSet.length() ) * keyOps->alphSize(); + /* Do we have enough keyspace left? */ + Size availableSpace = condData->lastCondKey.availableSpace(); + Size neededSpace = (1 << condSet.length() ) * keyOps->alphSize(); + if ( neededSpace > availableSpace ) + throw FsmConstructFail( FsmConstructFail::CondNoKeySpace ); + + Key baseKey = condData->lastCondKey; + baseKey.increment(); + condData->lastCondKey += (1 << condSet.length() ) * keyOps->alphSize(); condSpace = new CondSpace( condSet ); condSpace->baseKey = baseKey; @@ -820,21 +860,21 @@ CondSpace *FsmAp::addCondSpace( const CondSet &condSet ) return condSpace; } -void FsmAp::startFsmCondition( Action *condAction ) +void FsmAp::startFsmCondition( Action *condAction, bool sense ) { /* Make sure the start state has no other entry points. */ isolateStartState(); - embedCondition( startState, condAction ); + embedCondition( startState, condAction, sense ); } -void FsmAp::allTransCondition( Action *condAction ) +void FsmAp::allTransCondition( Action *condAction, bool sense ) { for ( StateList::Iter state = stateList; state.lte(); state++ ) - embedCondition( state, condAction ); + embedCondition( state, condAction, sense ); } -void FsmAp::leaveFsmCondition( Action *condAction ) +void FsmAp::leaveFsmCondition( Action *condAction, bool sense ) { for ( StateSet::Iter state = finStateSet; state.lte(); state++ ) - (*state)->outCondSet.insert( condAction ); + (*state)->outCondSet.insert( OutCond( condAction, sense ) ); } diff --git a/ragel/fsmattach.cpp b/ragel/fsmattach.cpp index 6a90df6..b0911f3 100644 --- a/ragel/fsmattach.cpp +++ b/ragel/fsmattach.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2001 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -183,7 +183,7 @@ void FsmAp::detachState( StateAp *state ) state->outList.abandon(); /* Unset final stateness before detaching from graph. */ - if ( state->stateBits & SB_ISFINAL ) + if ( state->stateBits & STB_ISFINAL ) finStateSet.remove( state ); } diff --git a/ragel/fsmbase.cpp b/ragel/fsmbase.cpp index 16841d0..87c7f0e 100644 --- a/ragel/fsmbase.cpp +++ b/ragel/fsmbase.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2001 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -46,6 +46,7 @@ FsmAp::FsmAp() : /* No start state. */ startState(0), + errState(0), /* Misfit accounting is a switch, turned on only at specific times. It * controls what happens when states have no way in from the outside @@ -65,6 +66,7 @@ FsmAp::FsmAp( const FsmAp &graph ) * pointers will be resolved later. */ entryPoints(graph.entryPoints), startState(graph.startState), + errState(0), /* Will be filled by copy. */ finStateSet(), @@ -96,6 +98,10 @@ FsmAp::FsmAp( const FsmAp &graph ) trans->toState = 0; attachTrans( state, toState, trans ); } + + /* Fix the eofTarg, if set. */ + if ( state->eofTarget != 0 ) + state->eofTarget = state->eofTarget->alg.stateMap; } /* Fix the state pointers in the entry points array. */ @@ -138,10 +144,10 @@ FsmAp::~FsmAp() void FsmAp::setFinState( StateAp *state ) { /* Is it already a fin state. */ - if ( state->stateBits & SB_ISFINAL ) + if ( state->stateBits & STB_ISFINAL ) return; - state->stateBits |= SB_ISFINAL; + state->stateBits |= STB_ISFINAL; finStateSet.insert( state ); } @@ -150,14 +156,14 @@ void FsmAp::setFinState( StateAp *state ) void FsmAp::unsetFinState( StateAp *state ) { /* Is it already a non-final state? */ - if ( ! (state->stateBits & SB_ISFINAL) ) + if ( ! (state->stateBits & STB_ISFINAL) ) return; /* When a state looses its final state status it must relinquish all the * properties that are allowed only for final states. */ clearOutData( state ); - state->stateBits &= ~ SB_ISFINAL; + state->stateBits &= ~ STB_ISFINAL; finStateSet.remove( state ); } @@ -340,12 +346,12 @@ void FsmAp::epsilonTrans( int id ) void FsmAp::markReachableFromHere( StateAp *state ) { /* Base case: return; */ - if ( state->stateBits & SB_ISMARKED ) + if ( state->stateBits & STB_ISMARKED ) return; /* Set this state as processed. We are going to visit all states that this * state has a transition to. */ - state->stateBits |= SB_ISMARKED; + state->stateBits |= STB_ISMARKED; /* Recurse on all out transitions. */ for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) { @@ -357,12 +363,12 @@ void FsmAp::markReachableFromHere( StateAp *state ) void FsmAp::markReachableFromHereStopFinal( StateAp *state ) { /* Base case: return; */ - if ( state->stateBits & SB_ISMARKED ) + if ( state->stateBits & STB_ISMARKED ) return; /* Set this state as processed. We are going to visit all states that this * state has a transition to. */ - state->stateBits |= SB_ISMARKED; + state->stateBits |= STB_ISMARKED; /* Recurse on all out transitions. */ for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) { @@ -377,12 +383,12 @@ void FsmAp::markReachableFromHereStopFinal( StateAp *state ) void FsmAp::markReachableFromHereReverse( StateAp *state ) { /* Base case: return; */ - if ( state->stateBits & SB_ISMARKED ) + if ( state->stateBits & STB_ISMARKED ) return; /* Set this state as processed. We are going to visit all states with * transitions into this state. */ - state->stateBits |= SB_ISMARKED; + state->stateBits |= STB_ISMARKED; /* Recurse on all items in transitions. */ for ( TransInList::Iter trans = state->inList; trans.lte(); trans++ ) @@ -414,19 +420,11 @@ void FsmAp::copyInEntryPoints( FsmAp *other ) entryPoints.insertMulti( en->key, en->value ); } -void FsmAp::setStateNumbers() -{ - int curNum = 0; - StateList::Iter state = stateList; - for ( ; state.lte(); state++ ) - state->alg.stateNum = curNum++; -} - void FsmAp::unsetAllFinStates() { for ( StateSet::Iter st = finStateSet; st.lte(); st++ ) - (*st)->stateBits &= ~ SB_ISFINAL; + (*st)->stateBits &= ~ STB_ISFINAL; finStateSet.empty(); } @@ -462,8 +460,8 @@ void FsmAp::verifyReachability() /* Check that everything got marked. */ for ( StateList::Iter st = stateList; st.lte(); st++ ) { /* Assert it got marked and then clear the mark. */ - assert( st->stateBits & SB_ISMARKED ); - st->stateBits &= ~ SB_ISMARKED; + assert( st->stateBits & STB_ISMARKED ); + st->stateBits &= ~ STB_ISMARKED; } } @@ -474,12 +472,131 @@ void FsmAp::verifyNoDeadEndStates() markReachableFromHereReverse( *pst ); /* Start state gets honorary marking. Must be done AFTER recursive call. */ - startState->stateBits |= SB_ISMARKED; + startState->stateBits |= STB_ISMARKED; /* Make sure everything got marked. */ for ( StateList::Iter st = stateList; st.lte(); st++ ) { /* Assert the state got marked and unmark it. */ - assert( st->stateBits & SB_ISMARKED ); - st->stateBits &= ~ SB_ISMARKED; + assert( st->stateBits & STB_ISMARKED ); + st->stateBits &= ~ STB_ISMARKED; + } +} + +void FsmAp::depthFirstOrdering( StateAp *state ) +{ + /* Nothing to do if the state is already on the list. */ + if ( state->stateBits & STB_ONLIST ) + return; + + /* Doing depth first, put state on the list. */ + state->stateBits |= STB_ONLIST; + stateList.append( state ); + + /* Recurse on everything ranges. */ + for ( TransList::Iter tel = state->outList; tel.lte(); tel++ ) { + if ( tel->toState != 0 ) + depthFirstOrdering( tel->toState ); + } +} + +/* Ordering states by transition connections. */ +void FsmAp::depthFirstOrdering() +{ + /* Init on state list flags. */ + for ( StateList::Iter st = stateList; st.lte(); st++ ) + st->stateBits &= ~STB_ONLIST; + + /* Clear out the state list, we will rebuild it. */ + int stateListLen = stateList.length(); + stateList.abandon(); + + /* Add back to the state list from the start state and all other entry + * points. */ + if ( errState != 0 ) + depthFirstOrdering( errState ); + depthFirstOrdering( startState ); + for ( EntryMap::Iter en = entryPoints; en.lte(); en++ ) + depthFirstOrdering( en->value ); + + /* Make sure we put everything back on. */ + assert( stateListLen == stateList.length() ); +} + +/* Stable sort the states by final state status. */ +void FsmAp::sortStatesByFinal() +{ + /* Move forward through the list and throw final states onto the end. */ + StateAp *state = 0; + StateAp *next = stateList.head; + StateAp *last = stateList.tail; + while ( state != last ) { + /* Move forward and load up the next. */ + state = next; + next = state->next; + + /* Throw to the end? */ + if ( state->isFinState() ) { + stateList.detach( state ); + stateList.append( state ); + } + } +} + +void FsmAp::setStateNumbers( int base ) +{ + for ( StateList::Iter state = stateList; state.lte(); state++ ) + state->alg.stateNum = base++; +} + + +bool FsmAp::checkErrTrans( StateAp *state, TransAp *trans ) +{ + /* Might go directly to error state. */ + if ( trans->toState == 0 ) + return true; + + if ( trans->prev == 0 ) { + /* If this is the first transition. */ + if ( keyOps->minKey < trans->lowKey ) + return true; + } + else { + /* Not the first transition. Compare against the prev. */ + TransAp *prev = trans->prev; + Key nextKey = prev->highKey; + nextKey.increment(); + if ( nextKey < trans->lowKey ) + return true; + } + return false; +} + +bool FsmAp::checkErrTransFinish( StateAp *state ) +{ + /* Check if there are any ranges already. */ + if ( state->outList.length() == 0 ) + return true; + else { + /* Get the last and check for a gap on the end. */ + TransAp *last = state->outList.tail; + if ( last->highKey < keyOps->maxKey ) + return true; + } + return 0; +} + +bool FsmAp::hasErrorTrans() +{ + bool result; + for ( StateList::Iter st = stateList; st.lte(); st++ ) { + for ( TransList::Iter tr = st->outList; tr.lte(); tr++ ) { + result = checkErrTrans( st, tr ); + if ( result ) + return true; + } + result = checkErrTransFinish( st ); + if ( result ) + return true; } + return false; } diff --git a/ragel/fsmgraph.cpp b/ragel/fsmgraph.cpp index 41c4b44..fecc4a5 100644 --- a/ragel/fsmgraph.cpp +++ b/ragel/fsmgraph.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2001, 2002, 2006 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001, 2002, 2006 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -293,7 +293,7 @@ void FsmAp::optionalRepeatOp( int times ) /* Make a duplicate for concating and set the fin bits to graph 2 so we * can pick out it's final states after the optional style concat. */ FsmAp *dup = new FsmAp( *copyFrom ); - dup->setFinBits( SB_GRAPH2 ); + dup->setFinBits( STB_GRAPH2 ); doConcat( dup, &lastFinSet, true ); /* Clear the last final state set and make the new one by taking only @@ -303,9 +303,9 @@ void FsmAp::optionalRepeatOp( int times ) /* If the state came from graph 2, add it to the last set and clear * the bits. */ StateAp *fs = finStateSet[i]; - if ( fs->stateBits & SB_GRAPH2 ) { + if ( fs->stateBits & STB_GRAPH2 ) { lastFinSet.insert( fs ); - fs->stateBits &= ~SB_GRAPH2; + fs->stateBits &= ~STB_GRAPH2; } } } @@ -455,8 +455,8 @@ void FsmAp::intersectOp( FsmAp *other ) other->setMisfitAccounting( true ); /* Set the fin bits on this and other to want each other. */ - setFinBits( SB_GRAPH1 ); - other->setFinBits( SB_GRAPH2 ); + setFinBits( STB_GRAPH1 ); + other->setFinBits( STB_GRAPH2 ); /* Call worker Or routine. */ doOr( other ); @@ -481,7 +481,7 @@ void FsmAp::subtractOp( FsmAp *other ) other->setMisfitAccounting( true ); /* Set the fin bits of other to be killers. */ - other->setFinBits( SB_GRAPH1 ); + other->setFinBits( STB_GRAPH1 ); /* Call worker Or routine. */ doOr( other ); @@ -725,7 +725,7 @@ void FsmAp::joinOp( int startId, int finalId, FsmAp **others, int numOthers ) /* Invoke the relinquish final callback on any states that did not get * final state status back. */ for ( StateSet::Iter st = finStateSetCopy; st.lte(); st++ ) { - if ( !((*st)->stateBits & SB_ISFINAL) ) + if ( !((*st)->stateBits & STB_ISFINAL) ) clearOutData( *st ); } @@ -818,14 +818,14 @@ void FsmAp::unsetKilledFinals() for ( int s = 0; s < fin.length(); s++ ) { /* Check for killing bit. */ StateAp *state = fin.data[s]; - if ( state->stateBits & SB_GRAPH1 ) { + if ( state->stateBits & STB_GRAPH1 ) { /* One final state is a killer, set to non-final. */ unsetFinState( state ); } /* Clear all killing bits. Non final states should never have had those * state bits set in the first place. */ - state->stateBits &= ~SB_GRAPH1; + state->stateBits &= ~STB_GRAPH1; } } @@ -838,8 +838,8 @@ void FsmAp::unsetIncompleteFinals() for ( int s = 0; s < fin.length(); s++ ) { /* Check for one set but not the other. */ StateAp *state = fin.data[s]; - if ( state->stateBits & SB_BOTH && - (state->stateBits & SB_BOTH) != SB_BOTH ) + if ( state->stateBits & STB_BOTH && + (state->stateBits & STB_BOTH) != STB_BOTH ) { /* One state wants the other but it is not there. */ unsetFinState( state ); @@ -847,7 +847,7 @@ void FsmAp::unsetIncompleteFinals() /* Clear wanting bits. Non final states should never have had those * state bits set in the first place. */ - state->stateBits &= ~SB_BOTH; + state->stateBits &= ~STB_BOTH; } } @@ -959,6 +959,7 @@ void FsmAp::findCondExpInTrans( ExpansionList &expansionList, StateAp *state, Key lowKey, Key highKey, CondSpace *fromCondSpace, CondSpace *toCondSpace, long fromVals, LongVect &toValsList ) { + /* Make condition-space low and high keys for searching. */ TransAp searchTrans; searchTrans.lowKey = fromCondSpace->baseKey + fromVals * keyOps->alphSize() + (lowKey - keyOps->minKey); @@ -969,7 +970,14 @@ void FsmAp::findCondExpInTrans( ExpansionList &expansionList, StateAp *state, PairIter<TransAp> pairIter( state->outList.head, &searchTrans ); for ( ; !pairIter.end(); pairIter++ ) { if ( pairIter.userState == RangeOverlap ) { - Expansion *expansion = new Expansion( lowKey, highKey ); + /* Need to make character-space low and high keys from the range + * overlap for the expansion object. */ + Key expLowKey = pairIter.s1Tel.lowKey - fromCondSpace->baseKey - fromVals * + keyOps->alphSize() + keyOps->minKey; + Key expHighKey = pairIter.s1Tel.highKey - fromCondSpace->baseKey - fromVals * + keyOps->alphSize() + keyOps->minKey; + + Expansion *expansion = new Expansion( expLowKey, expHighKey ); expansion->fromTrans = new TransAp(*pairIter.s1Tel.trans); expansion->fromTrans->fromState = 0; expansion->fromTrans->toState = pairIter.s1Tel.trans->toState; @@ -1170,7 +1178,7 @@ void FsmAp::mergeStateConds( StateAp *destState, StateAp *srcState ) } /* A state merge which represents the drawing in of leaving transitions. If - * there is any out data then we duplicate the souce state, transfer the out + * there is any out data then we duplicate the source state, transfer the out * data, then merge in the state. The new state will be reaped because it will * not be given any in transitions. */ void FsmAp::mergeStatesLeaving( MergeData &md, StateAp *destState, StateAp *srcState ) @@ -1182,8 +1190,8 @@ void FsmAp::mergeStatesLeaving( MergeData &md, StateAp *destState, StateAp *srcS mergeStates( md, ssMutable, srcState ); transferOutData( ssMutable, destState ); - for ( ActionSet::Iter cond = destState->outCondSet; cond.lte(); cond++ ) - embedCondition( md, ssMutable, *cond ); + for ( OutCondSet::Iter cond = destState->outCondSet; cond.lte(); cond++ ) + embedCondition( md, ssMutable, cond->action, cond->sense ); mergeStates( md, destState, ssMutable ); } @@ -1220,7 +1228,7 @@ void FsmAp::mergeStates( MergeData &md, StateAp *destState, StateAp *srcState ) expList2.empty(); /* Get its bits and final state status. */ - destState->stateBits |= ( srcState->stateBits & ~SB_ISFINAL ); + destState->stateBits |= ( srcState->stateBits & ~STB_ISFINAL ); if ( srcState->isFinState() ) setFinState( destState ); @@ -1237,7 +1245,7 @@ void FsmAp::mergeStates( MergeData &md, StateAp *destState, StateAp *srcState ) destState->fromStateActionTable.setActions( ActionTable( srcState->fromStateActionTable ) ); destState->outActionTable.setActions( ActionTable( srcState->outActionTable ) ); - destState->outCondSet.insert( ActionSet( srcState->outCondSet ) ); + destState->outCondSet.insert( OutCondSet( srcState->outCondSet ) ); destState->errActionTable.setActions( ErrActionTable( srcState->errActionTable ) ); destState->eofActionTable.setActions( ActionTable( srcState->eofActionTable ) ); } @@ -1283,7 +1291,7 @@ void FsmAp::fillInStates( MergeData &md ) } void FsmAp::findEmbedExpansions( ExpansionList &expansionList, - StateAp *destState, Action *condAction ) + StateAp *destState, Action *condAction, bool sense ) { StateCondList destList; PairIter<TransAp, StateCond> transCond( destState->outList.head, @@ -1309,7 +1317,7 @@ void FsmAp::findEmbedExpansions( ExpansionList &expansionList, expansion->fromCondSpace = 0; expansion->fromVals = 0; expansion->toCondSpace = newStateCond->condSpace; - expansion->toValsList.append( 1 ); + expansion->toValsList.append( sense?1:0 ); #ifdef LOG_CONDS logNewExpansion( expansion ); #endif @@ -1347,7 +1355,7 @@ void FsmAp::findEmbedExpansions( ExpansionList &expansionList, long targVals = basicVals; Action **cim = mergedCS.find( condAction ); long bitPos = (cim - mergedCS.data); - targVals |= 1 << bitPos; + targVals |= (sense?1:0) << bitPos; LongVect expandToVals( targVals ); findCondExpInTrans( expansionList, destState, @@ -1369,7 +1377,7 @@ void FsmAp::findEmbedExpansions( ExpansionList &expansionList, destState->stateCondList.transfer( destList ); } -void FsmAp::embedCondition( StateAp *state, Action *condAction ) +void FsmAp::embedCondition( StateAp *state, Action *condAction, bool sense ) { MergeData md; ExpansionList expList; @@ -1378,7 +1386,7 @@ void FsmAp::embedCondition( StateAp *state, Action *condAction ) setMisfitAccounting( true ); /* Worker. */ - embedCondition( md, state, condAction ); + embedCondition( md, state, condAction, sense ); /* Fill in any states that were newed up as combinations of others. */ fillInStates( md ); @@ -1388,12 +1396,39 @@ void FsmAp::embedCondition( StateAp *state, Action *condAction ) setMisfitAccounting( false ); } -void FsmAp::embedCondition( MergeData &md, StateAp *state, Action *condAction ) +void FsmAp::embedCondition( MergeData &md, StateAp *state, Action *condAction, bool sense ) { ExpansionList expList; - findEmbedExpansions( expList, state, condAction ); + findEmbedExpansions( expList, state, condAction, sense ); doExpand( md, state, expList ); doRemove( md, state, expList ); expList.empty(); } + +/* Check if a machine defines a single character. This is useful in validating + * ranges and machines to export. */ +bool FsmAp::checkSingleCharMachine() +{ + /* Must have two states. */ + if ( stateList.length() != 2 ) + return false; + /* The start state cannot be final. */ + if ( startState->isFinState() ) + return false; + /* There should be only one final state. */ + if ( finStateSet.length() != 1 ) + return false; + /* The final state cannot have any transitions out. */ + if ( finStateSet[0]->outList.length() != 0 ) + return false; + /* The start state should have only one transition out. */ + if ( startState->outList.length() != 1 ) + return false; + /* The singe transition out of the start state should not be a range. */ + TransAp *startTrans = startState->outList.head; + if ( startTrans->lowKey != startTrans->highKey ) + return false; + return true; +} + diff --git a/ragel/fsmgraph.h b/ragel/fsmgraph.h index 1a8e80c..1b7d6f5 100644 --- a/ragel/fsmgraph.h +++ b/ragel/fsmgraph.h @@ -1,5 +1,5 @@ /* - * Copyright 2001-2004 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -22,7 +22,10 @@ #ifndef _FSMGRAPH_H #define _FSMGRAPH_H +#include "config.h" #include <assert.h> +#include <iostream> +#include <string> #include "common.h" #include "vector.h" #include "bstset.h" @@ -35,21 +38,26 @@ #include "sbsttable.h" #include "avlset.h" #include "avlmap.h" +#include "ragel.h" //#define LOG_CONDS /* Flags that control merging. */ -#define SB_GRAPH1 0x01 -#define SB_GRAPH2 0x02 -#define SB_BOTH 0x03 -#define SB_ISFINAL 0x04 -#define SB_ISMARKED 0x08 +#define STB_GRAPH1 0x01 +#define STB_GRAPH2 0x02 +#define STB_BOTH 0x03 +#define STB_ISFINAL 0x04 +#define STB_ISMARKED 0x08 +#define STB_ONLIST 0x10 + +using std::ostream; struct TransAp; struct StateAp; struct FsmAp; struct Action; struct LongestMatchPart; +struct LengthDef; /* State list element for unambiguous access to list element. */ struct FsmListEl @@ -77,6 +85,94 @@ extern KeyOps *keyOps; /* Transistion Action Element. */ typedef SBstMapEl< int, Action* > ActionTableEl; +/* Nodes in the tree that use this action. */ +struct NameInst; +struct InlineList; +typedef Vector<NameInst*> ActionRefs; + +/* Element in list of actions. Contains the string for the code to exectute. */ +struct Action +: + public DListEl<Action>, + public AvlTreeEl<Action> +{ +public: + + Action( const InputLoc &loc, const char *name, InlineList *inlineList, int condId ) + : + loc(loc), + name(name), + inlineList(inlineList), + actionId(-1), + numTransRefs(0), + numToStateRefs(0), + numFromStateRefs(0), + numEofRefs(0), + numCondRefs(0), + anyCall(false), + isLmAction(false), + condId(condId) + { + } + + /* Key for action dictionary. */ + const char *getKey() const { return name; } + + /* Data collected during parse. */ + InputLoc loc; + const char *name; + InlineList *inlineList; + int actionId; + + void actionName( ostream &out ) + { + if ( name != 0 ) + out << name; + else + out << loc.line << ":" << loc.col; + } + + /* Places in the input text that reference the action. */ + ActionRefs actionRefs; + + /* Number of references in the final machine. */ + int numRefs() + { return numTransRefs + numToStateRefs + numFromStateRefs + numEofRefs; } + int numTransRefs; + int numToStateRefs; + int numFromStateRefs; + int numEofRefs; + int numCondRefs; + bool anyCall; + + bool isLmAction; + int condId; +}; + +struct CmpCondId +{ + static inline int compare( const Action *cond1, const Action *cond2 ) + { + if ( cond1->condId < cond2->condId ) + return -1; + else if ( cond1->condId > cond2->condId ) + return 1; + return 0; + } +}; + +/* A list of actions. */ +typedef DList<Action> ActionList; +typedef AvlTree<Action, char *, CmpStr> ActionDict; + +/* Structure for reverse action mapping. */ +struct RevActionMapEl +{ + char *name; + InputLoc location; +}; + + /* Transition Action Table. */ struct ActionTable : public SBstMap< int, Action*, CmpOrd<int> > @@ -285,10 +381,8 @@ struct TransAp highKey(other.highKey), fromState(0), toState(0), actionTable(other.actionTable), - priorTable(other.priorTable) - { - assert( lmActionTable.length() == 0 && other.lmActionTable.length() == 0 ); - } + priorTable(other.priorTable), + lmActionTable(other.lmActionTable) {} Key lowKey, highKey; StateAp *fromState; @@ -442,9 +536,40 @@ typedef BstSet<int> EntryIdSet; /* Set of longest match items that may be active in a given state. */ typedef BstSet<LongestMatchPart*> LmItemSet; +/* A Conditions which is to be + * transfered on pending out transitions. */ +struct OutCond +{ + OutCond( Action *action, bool sense ) + : action(action), sense(sense) {} + + Action *action; + bool sense; +}; + +struct CmpOutCond +{ + static int compare( const OutCond &outCond1, const OutCond &outCond2 ) + { + if ( outCond1.action < outCond2.action ) + return -1; + else if ( outCond1.action > outCond2.action ) + return 1; + else if ( outCond1.sense < outCond2.sense ) + return -1; + else if ( outCond1.sense > outCond2.sense ) + return 1; + return 0; + } +}; + +/* Set of conditions to be transfered to on pending out transitions. */ +typedef SBstSet< OutCond, CmpOutCond > OutCondSet; +typedef CmpSTable< OutCond, CmpOutCond > CmpOutCondSet; + /* Conditions. */ -typedef BstSet< Action*, CmpOrd<Action*> > CondSet; -typedef CmpTable< Action*, CmpOrd<Action*> > CmpCondSet; +typedef BstSet< Action*, CmpCondId > CondSet; +typedef CmpTable< Action*, CmpCondId > CmpCondSet; struct CondSpace : public AvlTreeEl<CondSpace> @@ -516,16 +641,28 @@ struct Removal struct CondData { - CondData() : nextCondKey(0) {} + CondData() : lastCondKey(0) {} /* Condition info. */ - Key nextCondKey; + Key lastCondKey; CondSpaceMap condSpaceMap; }; extern CondData *condData; +struct FsmConstructFail +{ + enum Reason + { + CondNoKeySpace + }; + + FsmConstructFail( Reason reason ) + : reason(reason) {} + Reason reason; +}; + /* State class that implements actions and priorities. */ struct StateAp { @@ -534,7 +671,7 @@ struct StateAp ~StateAp(); /* Is the state final? */ - bool isFinState() { return stateBits & SB_ISFINAL; } + bool isFinState() { return stateBits & STB_ISFINAL; } /* Out transition list and the pointer for the default out trans. */ TransList outList; @@ -542,6 +679,10 @@ struct StateAp /* In transition Lists. */ TransInList inList; + /* Set only during scanner construction when actions are added. NFA to DFA + * code can ignore this. */ + StateAp *eofTarget; + /* Entry points into the state. */ EntryIdSet entryIds; @@ -616,7 +757,7 @@ struct StateAp ActionTable outActionTable; /* Conditions to add to any future transiions that leave via this sttate. */ - ActionSet outCondSet; + OutCondSet outCondSet; /* Error action tables. */ ErrActionTable errActionTable; @@ -970,6 +1111,11 @@ struct FsmAp /* The start state. */ StateAp *startState; + /* Error state, possibly created only when the final machine has been + * created and the XML machine is about to be written. No transitions + * point to this state. */ + StateAp *errState; + /* The set of final states. */ StateSet finStateSet; @@ -987,7 +1133,9 @@ struct FsmAp void leaveFsmPrior( int ordering, PriorDesc *prior ); /* Action setting support. */ + void transferOutActions( StateAp *state ); void transferErrorActions( StateAp *state, int transferPoint ); + void setErrorActions( StateAp *state, const ActionTable &other ); void setErrorAction( StateAp *state, int ordering, Action *action ); /* Fill all spaces in a transition list with an error transition. */ @@ -1008,13 +1156,13 @@ struct FsmAp CondSpace *addCondSpace( const CondSet &condSet ); void findEmbedExpansions( ExpansionList &expansionList, - StateAp *destState, Action *condAction ); - void embedCondition( MergeData &md, StateAp *state, Action *condAction ); - void embedCondition( StateAp *state, Action *condAction ); + StateAp *destState, Action *condAction, bool sense ); + void embedCondition( MergeData &md, StateAp *state, Action *condAction, bool sense ); + void embedCondition( StateAp *state, Action *condAction, bool sense ); - void startFsmCondition( Action *condAction ); - void allTransCondition( Action *condAction ); - void leaveFsmCondition( Action *condAction ); + void startFsmCondition( Action *condAction, bool sense ); + void allTransCondition( Action *condAction, bool sense ); + void leaveFsmCondition( Action *condAction, bool sense ); /* Set error actions to execute. */ void startErrorAction( int ordering, Action *action, int transferPoint ); @@ -1261,8 +1409,13 @@ struct FsmAp * copied into this machine. */ void copyInEntryPoints( FsmAp *other ); - /* Set State numbers starting at 0. */ - void setStateNumbers(); + /* Ordering states. */ + void depthFirstOrdering( StateAp *state ); + void depthFirstOrdering(); + void sortStatesByFinal(); + + /* Set sqequential state numbers starting at 0. */ + void setStateNumbers( int base ); /* Unset all final states. */ void unsetAllFinStates(); @@ -1363,7 +1516,16 @@ struct FsmAp /* Merge neighboring transitions go to the same state and have the same * transitions data. */ void compressTransitions(); -}; + /* Returns true if there is a transtion (either explicit or by a gap) to + * the error state. */ + bool checkErrTrans( StateAp *state, TransAp *trans ); + bool checkErrTransFinish( StateAp *state ); + bool hasErrorTrans(); + + /* Check if a machine defines a single character. This is useful in + * validating ranges and machines to export. */ + bool checkSingleCharMachine( ); +}; -#endif /* _FSMGRAPH_H */ +#endif diff --git a/ragel/fsmmin.cpp b/ragel/fsmmin.cpp index c57de6f..42ebfaa 100644 --- a/ragel/fsmmin.cpp +++ b/ragel/fsmmin.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -407,7 +407,7 @@ bool FsmAp::markRound( MarkIndex &markIndex ) void FsmAp::minimizeStable() { /* Set the state numbers. */ - setStateNumbers(); + setStateNumbers( 0 ); /* This keeps track of which pairs have been marked. */ MarkIndex markIndex( stateList.length() ); @@ -509,8 +509,8 @@ void FsmAp::removeUnreachableStates() while ( state ) { StateAp *next = state->next; - if ( state->stateBits & SB_ISMARKED ) - state->stateBits &= ~ SB_ISMARKED; + if ( state->stateBits & STB_ISMARKED ) + state->stateBits &= ~ STB_ISMARKED; else { detachState( state ); stateList.detach( state ); @@ -573,7 +573,7 @@ void FsmAp::removeDeadEndStates() * recursive call on all the final states so that it does not cause the * start state in transitions to be skipped when the start state is * visited by the traversal. */ - startState->stateBits |= SB_ISMARKED; + startState->stateBits |= STB_ISMARKED; /* Delete all states that are not marked * and unmark the ones that are marked. */ @@ -581,8 +581,8 @@ void FsmAp::removeDeadEndStates() while ( state != 0 ) { StateAp *next = state->next; - if ( state->stateBits & SB_ISMARKED ) - state->stateBits &= ~ SB_ISMARKED; + if ( state->stateBits & STB_ISMARKED ) + state->stateBits &= ~ STB_ISMARKED; else { detachState( state ); stateList.detach( state ); diff --git a/ragel/fsmstate.cpp b/ragel/fsmstate.cpp index 4322c10..63c48e9 100644 --- a/ragel/fsmstate.cpp +++ b/ragel/fsmstate.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2002 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2002 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -76,6 +76,9 @@ StateAp::StateAp() outList(), inList(), + /* No EOF target. */ + eofTarget(0), + /* No entry points, or epsilon trans. */ entryIds(), epsilonTrans(), @@ -115,6 +118,10 @@ StateAp::StateAp(const StateAp &other) outList(), inList(), + /* Set this using the original state's eofTarget. It will get mapped back + * to the new machine in the Fsm copy constructor. */ + eofTarget(other.eofTarget), + /* Duplicate the entry id set and epsilon transitions. These * are sets of integers and as such need no fixing. */ entryIds(other.entryIds), @@ -149,6 +156,7 @@ StateAp::StateAp(const StateAp &other) /* Dupicate and store the orginal target in the transition. This will * be corrected once all the states have been created. */ TransAp *newTrans = new TransAp(*trans); + assert( trans->lmActionTable.length() == 0 ); newTrans->toState = trans->toState; outList.append( newTrans ); } @@ -163,16 +171,16 @@ StateAp::~StateAp() } /* Compare two states using pointers to the states. With the approximate - * compare the idea is that if the compare finds them the same, they can + * compare, the idea is that if the compare finds them the same, they can * immediately be merged. */ -int ApproxCompare::compare( const StateAp *state1 , const StateAp *state2 ) +int ApproxCompare::compare( const StateAp *state1, const StateAp *state2 ) { int compareRes; /* Test final state status. */ - if ( (state1->stateBits & SB_ISFINAL) && !(state2->stateBits & SB_ISFINAL) ) + if ( (state1->stateBits & STB_ISFINAL) && !(state2->stateBits & STB_ISFINAL) ) return -1; - else if ( !(state1->stateBits & SB_ISFINAL) && (state2->stateBits & SB_ISFINAL) ) + else if ( !(state1->stateBits & STB_ISFINAL) && (state2->stateBits & STB_ISFINAL) ) return 1; /* Test epsilon transition sets. */ @@ -216,19 +224,25 @@ int ApproxCompare::compare( const StateAp *state1 , const StateAp *state2 ) } } + /* Check EOF targets. */ + if ( state1->eofTarget < state2->eofTarget ) + return -1; + else if ( state1->eofTarget > state2->eofTarget ) + return 1; + /* Got through the entire state comparison, deem them equal. */ return 0; } -/* Compare class for the sort that does the intial partition of compaction. */ +/* Compare class used in the initial partition. */ int InitPartitionCompare::compare( const StateAp *state1 , const StateAp *state2 ) { int compareRes; /* Test final state status. */ - if ( (state1->stateBits & SB_ISFINAL) && !(state2->stateBits & SB_ISFINAL) ) + if ( (state1->stateBits & STB_ISFINAL) && !(state2->stateBits & STB_ISFINAL) ) return -1; - else if ( !(state1->stateBits & SB_ISFINAL) && (state2->stateBits & SB_ISFINAL) ) + else if ( !(state1->stateBits & STB_ISFINAL) && (state2->stateBits & STB_ISFINAL) ) return 1; /* Test epsilon transition sets. */ @@ -334,6 +348,19 @@ int PartitionCompare::compare( const StateAp *state1, const StateAp *state2 ) } } + /* Test eof targets. */ + if ( state1->eofTarget == 0 && state2->eofTarget != 0 ) + return -1; + else if ( state1->eofTarget != 0 && state2->eofTarget == 0 ) + return 1; + else if ( state1->eofTarget != 0 ) { + /* Both eof targets are set. */ + compareRes = CmpOrd< MinPartition* >::compare( + state1->eofTarget->alg.partition, state2->eofTarget->alg.partition ); + if ( compareRes != 0 ) + return compareRes; + } + return 0; } diff --git a/ragel/gendata.cpp b/ragel/gendata.cpp new file mode 100644 index 0000000..8c57e2c --- /dev/null +++ b/ragel/gendata.cpp @@ -0,0 +1,1016 @@ +/* + * Copyright 2005-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "gendata.h" +#include "ragel.h" +#include <iostream> + +/* + * Code generators. + */ + +#include "cstable.h" +#include "csftable.h" +#include "csflat.h" +#include "csfflat.h" +#include "csgoto.h" +#include "csfgoto.h" +#include "csipgoto.h" +#include "cssplit.h" + +#include "cdtable.h" +#include "cdftable.h" +#include "cdflat.h" +#include "cdfflat.h" +#include "cdgoto.h" +#include "cdfgoto.h" +#include "cdipgoto.h" +#include "cdsplit.h" + +#include "dotcodegen.h" + +#include "javacodegen.h" + +#include "rubytable.h" +#include "rubyftable.h" +#include "rubyflat.h" +#include "rubyfflat.h" +#include "rbxgoto.h" + +string itoa( int i ) +{ + char buf[16]; + sprintf( buf, "%i", i ); + return buf; +} + +using std::cout; +using std::cerr; +using std::endl; + +/* Invoked by the parser when a ragel definition is opened. */ +CodeGenData *dotMakeCodeGen( const char *sourceFileName, const char *fsmName, ostream &out ) +{ + CodeGenData *codeGen = new GraphvizDotGen(out); + + codeGen->sourceFileName = sourceFileName; + codeGen->fsmName = fsmName; + + /* For normal code generation we want a transition on every character so we never + * end up in an undefined state. For graphviz this just clutters the + * drawing so we turn it off. */ + codeGen->wantComplete = false; + + return codeGen; +} + +/* Invoked by the parser when a ragel definition is opened. */ +CodeGenData *cdMakeCodeGen( const char *sourceFileName, const char *fsmName, ostream &out ) +{ + CodeGenData *codeGen = 0; + switch ( hostLang->lang ) { + case HostLang::C: + switch ( codeStyle ) { + case GenTables: + codeGen = new CTabCodeGen(out); + break; + case GenFTables: + codeGen = new CFTabCodeGen(out); + break; + case GenFlat: + codeGen = new CFlatCodeGen(out); + break; + case GenFFlat: + codeGen = new CFFlatCodeGen(out); + break; + case GenGoto: + codeGen = new CGotoCodeGen(out); + break; + case GenFGoto: + codeGen = new CFGotoCodeGen(out); + break; + case GenIpGoto: + codeGen = new CIpGotoCodeGen(out); + break; + case GenSplit: + codeGen = new CSplitCodeGen(out); + break; + } + break; + + case HostLang::D: + switch ( codeStyle ) { + case GenTables: + codeGen = new DTabCodeGen(out); + break; + case GenFTables: + codeGen = new DFTabCodeGen(out); + break; + case GenFlat: + codeGen = new DFlatCodeGen(out); + break; + case GenFFlat: + codeGen = new DFFlatCodeGen(out); + break; + case GenGoto: + codeGen = new DGotoCodeGen(out); + break; + case GenFGoto: + codeGen = new DFGotoCodeGen(out); + break; + case GenIpGoto: + codeGen = new DIpGotoCodeGen(out); + break; + case GenSplit: + codeGen = new DSplitCodeGen(out); + break; + } + break; + + default: break; + } + + codeGen->sourceFileName = sourceFileName; + codeGen->fsmName = fsmName; + + return codeGen; +} + +/* Invoked by the parser when a ragel definition is opened. */ +CodeGenData *javaMakeCodeGen( const char *sourceFileName, const char *fsmName, ostream &out ) +{ + CodeGenData *codeGen = new JavaTabCodeGen(out); + + codeGen->sourceFileName = sourceFileName; + codeGen->fsmName = fsmName; + + return codeGen; +} + +/* Invoked by the parser when a ragel definition is opened. */ +CodeGenData *rubyMakeCodeGen( const char *sourceFileName, const char *fsmName, ostream &out ) +{ + CodeGenData *codeGen = 0; + switch ( codeStyle ) { + case GenTables: + codeGen = new RubyTabCodeGen(out); + break; + case GenFTables: + codeGen = new RubyFTabCodeGen(out); + break; + case GenFlat: + codeGen = new RubyFlatCodeGen(out); + break; + case GenFFlat: + codeGen = new RubyFFlatCodeGen(out); + break; + case GenGoto: + if ( rubyImpl == Rubinius ) { + codeGen = new RbxGotoCodeGen(out); + } else { + cerr << "Goto style is still _very_ experimental " + "and only supported using Rubinius.\n" + "You may want to enable the --rbx flag " + " to give it a try.\n"; + exit(1); + } + break; + default: + cout << "Invalid code style\n"; + exit(1); + break; + } + codeGen->sourceFileName = sourceFileName; + codeGen->fsmName = fsmName; + + return codeGen; +} + +/* Invoked by the parser when a ragel definition is opened. */ +CodeGenData *csharpMakeCodeGen( const char *sourceFileName, const char *fsmName, ostream &out ) +{ + CodeGenData *codeGen = 0; + + switch ( codeStyle ) { + case GenTables: + codeGen = new CSharpTabCodeGen(out); + break; + case GenFTables: + codeGen = new CSharpFTabCodeGen(out); + break; + case GenFlat: + codeGen = new CSharpFlatCodeGen(out); + break; + case GenFFlat: + codeGen = new CSharpFFlatCodeGen(out); + break; + case GenGoto: + codeGen = new CSharpGotoCodeGen(out); + break; + case GenFGoto: + codeGen = new CSharpFGotoCodeGen(out); + break; + case GenIpGoto: + codeGen = new CSharpIpGotoCodeGen(out); + break; + case GenSplit: + codeGen = new CSharpSplitCodeGen(out); + break; + } + + codeGen->sourceFileName = sourceFileName; + codeGen->fsmName = fsmName; + + return codeGen; +} + + +CodeGenData *makeCodeGen( const char *sourceFileName, const char *fsmName, ostream &out ) +{ + CodeGenData *cgd = 0; + if ( generateDot ) + cgd = dotMakeCodeGen( sourceFileName, fsmName, out ); + else if ( hostLang == &hostLangC ) + cgd = cdMakeCodeGen( sourceFileName, fsmName, out ); + else if ( hostLang == &hostLangD ) + cgd = cdMakeCodeGen( sourceFileName, fsmName, out ); + else if ( hostLang == &hostLangJava ) + cgd = javaMakeCodeGen( sourceFileName, fsmName, out ); + else if ( hostLang == &hostLangRuby ) + cgd = rubyMakeCodeGen( sourceFileName, fsmName, out ); + else if ( hostLang == &hostLangCSharp ) + cgd = csharpMakeCodeGen( sourceFileName, fsmName, out ); + return cgd; +} + +void lineDirective( ostream &out, const char *fileName, int line ) +{ + if ( !generateDot ) { + if ( hostLang == &hostLangC ) + cdLineDirective( out, fileName, line ); + else if ( hostLang == &hostLangD ) + cdLineDirective( out, fileName, line ); + else if ( hostLang == &hostLangJava ) + javaLineDirective( out, fileName, line ); + else if ( hostLang == &hostLangRuby ) + rubyLineDirective( out, fileName, line ); + else if ( hostLang == &hostLangCSharp ) + csharpLineDirective( out, fileName, line ); + } +} + +void genLineDirective( ostream &out ) +{ + std::streambuf *sbuf = out.rdbuf(); + output_filter *filter = static_cast<output_filter*>(sbuf); + lineDirective( out, filter->fileName, filter->line + 1 ); +} + + +/* Total error count. */ +/* int gblErrorCount = 0; */ + +CodeGenData::CodeGenData( ostream &out ) +: + sourceFileName(0), + fsmName(0), + out(out), + redFsm(0), + allActions(0), + allActionTables(0), + allConditions(0), + allCondSpaces(0), + allStates(0), + nameIndex(0), + startState(-1), + errState(-1), + getKeyExpr(0), + accessExpr(0), + prePushExpr(0), + postPopExpr(0), + pExpr(0), + peExpr(0), + eofExpr(0), + csExpr(0), + topExpr(0), + stackExpr(0), + actExpr(0), + tokstartExpr(0), + tokendExpr(0), + dataExpr(0), + wantComplete(true), + hasLongestMatch(false), + noEnd(false), + noPrefix(false), + noFinal(false), + noError(false), + noCS(false) +{} + + +void CodeGenData::createMachine() +{ + redFsm = new RedFsmAp(); +} + +void CodeGenData::initActionList( unsigned long length ) +{ + allActions = new GenAction[length]; + for ( unsigned long a = 0; a < length; a++ ) + actionList.append( allActions+a ); +} + +void CodeGenData::newAction( int anum, const char *name, + const InputLoc &loc, GenInlineList *inlineList ) +{ + allActions[anum].actionId = anum; + allActions[anum].name = name; + allActions[anum].loc = loc; + allActions[anum].inlineList = inlineList; +} + +void CodeGenData::initActionTableList( unsigned long length ) +{ + allActionTables = new RedAction[length]; +} + +void CodeGenData::initStateList( unsigned long length ) +{ + allStates = new RedStateAp[length]; + for ( unsigned long s = 0; s < length; s++ ) + redFsm->stateList.append( allStates+s ); + + /* We get the start state as an offset, set the pointer now. */ + if ( startState >= 0 ) + redFsm->startState = allStates + startState; + if ( errState >= 0 ) + redFsm->errState = allStates + errState; + for ( EntryIdVect::Iter en = entryPointIds; en.lte(); en++ ) + redFsm->entryPoints.insert( allStates + *en ); + + /* The nextStateId is no longer used to assign state ids (they come in set + * from the frontend now), however generation code still depends on it. + * Should eventually remove this variable. */ + redFsm->nextStateId = redFsm->stateList.length(); +} + +void CodeGenData::setStartState( unsigned long startState ) +{ + this->startState = startState; +} + +void CodeGenData::setErrorState( unsigned long errState ) +{ + this->errState = errState; +} + +void CodeGenData::addEntryPoint( char *name, unsigned long entryState ) +{ + entryPointIds.append( entryState ); + entryPointNames.append( name ); +} + +void CodeGenData::initTransList( int snum, unsigned long length ) +{ + /* Could preallocate the out range to save time growing it. For now do + * nothing. */ +} + +void CodeGenData::newTrans( int snum, int tnum, Key lowKey, + Key highKey, long targ, long action ) +{ + /* Get the current state and range. */ + RedStateAp *curState = allStates + snum; + RedTransList &destRange = curState->outRange; + + if ( curState == redFsm->errState ) + return; + + /* Make the new transitions. */ + RedStateAp *targState = targ >= 0 ? (allStates + targ) : + wantComplete ? redFsm->getErrorState() : 0; + RedAction *actionTable = action >= 0 ? (allActionTables + action) : 0; + RedTransAp *trans = redFsm->allocateTrans( targState, actionTable ); + RedTransEl transEl( lowKey, highKey, trans ); + + if ( wantComplete ) { + /* If the machine is to be complete then we need to fill any gaps with + * the error transitions. */ + if ( destRange.length() == 0 ) { + /* Range is currently empty. */ + if ( keyOps->minKey < lowKey ) { + /* The first range doesn't start at the low end. */ + Key fillHighKey = lowKey; + fillHighKey.decrement(); + + /* Create the filler with the state's error transition. */ + RedTransEl newTel( keyOps->minKey, fillHighKey, redFsm->getErrorTrans() ); + destRange.append( newTel ); + } + } + else { + /* The range list is not empty, get the the last range. */ + RedTransEl *last = &destRange[destRange.length()-1]; + Key nextKey = last->highKey; + nextKey.increment(); + if ( nextKey < lowKey ) { + /* There is a gap to fill. Make the high key. */ + Key fillHighKey = lowKey; + fillHighKey.decrement(); + + /* Create the filler with the state's error transtion. */ + RedTransEl newTel( nextKey, fillHighKey, redFsm->getErrorTrans() ); + destRange.append( newTel ); + } + } + } + + /* Filler taken care of. Append the range. */ + destRange.append( RedTransEl( lowKey, highKey, trans ) ); +} + +void CodeGenData::finishTransList( int snum ) +{ + /* Get the current state and range. */ + RedStateAp *curState = allStates + snum; + RedTransList &destRange = curState->outRange; + + if ( curState == redFsm->errState ) + return; + + /* If building a complete machine we may need filler on the end. */ + if ( wantComplete ) { + /* Check if there are any ranges already. */ + if ( destRange.length() == 0 ) { + /* Fill with the whole alphabet. */ + /* Add the range on the lower and upper bound. */ + RedTransEl newTel( keyOps->minKey, keyOps->maxKey, redFsm->getErrorTrans() ); + destRange.append( newTel ); + } + else { + /* Get the last and check for a gap on the end. */ + RedTransEl *last = &destRange[destRange.length()-1]; + if ( last->highKey < keyOps->maxKey ) { + /* Make the high key. */ + Key fillLowKey = last->highKey; + fillLowKey.increment(); + + /* Create the new range with the error trans and append it. */ + RedTransEl newTel( fillLowKey, keyOps->maxKey, redFsm->getErrorTrans() ); + destRange.append( newTel ); + } + } + } +} + +void CodeGenData::setId( int snum, int id ) +{ + RedStateAp *curState = allStates + snum; + curState->id = id; +} + +void CodeGenData::setFinal( int snum ) +{ + RedStateAp *curState = allStates + snum; + curState->isFinal = true; +} + + +void CodeGenData::setStateActions( int snum, long toStateAction, + long fromStateAction, long eofAction ) +{ + RedStateAp *curState = allStates + snum; + if ( toStateAction >= 0 ) + curState->toStateAction = allActionTables + toStateAction; + if ( fromStateAction >= 0 ) + curState->fromStateAction = allActionTables + fromStateAction; + if ( eofAction >= 0 ) + curState->eofAction = allActionTables + eofAction; +} + +void CodeGenData::setEofTrans( int snum, long eofTarget, long actId ) +{ + RedStateAp *curState = allStates + snum; + RedStateAp *targState = allStates + eofTarget; + RedAction *eofAct = allActionTables + actId; + curState->eofTrans = redFsm->allocateTrans( targState, eofAct ); +} + +void CodeGenData::resolveTargetStates( GenInlineList *inlineList ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Goto: case GenInlineItem::Call: + case GenInlineItem::Next: case GenInlineItem::Entry: + item->targState = allStates + item->targId; + break; + default: + break; + } + + if ( item->children != 0 ) + resolveTargetStates( item->children ); + } +} + +void CodeGenData::closeMachine() +{ + for ( GenActionList::Iter a = actionList; a.lte(); a++ ) + resolveTargetStates( a->inlineList ); + + /* Note that even if we want a complete graph we do not give the error + * state a default transition. All machines break out of the processing + * loop when in the error state. */ + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + for ( GenStateCondList::Iter sci = st->stateCondList; sci.lte(); sci++ ) + st->stateCondVect.append( sci ); + } +} + + +bool CodeGenData::setAlphType( const char *data ) +{ + HostType *alphType = findAlphTypeInternal( data ); + if ( alphType == 0 ) + return false; + + thisKeyOps.setAlphType( alphType ); + return true; +} + +void CodeGenData::initCondSpaceList( ulong length ) +{ + allCondSpaces = new GenCondSpace[length]; + for ( ulong c = 0; c < length; c++ ) + condSpaceList.append( allCondSpaces + c ); +} + +void CodeGenData::newCondSpace( int cnum, int condSpaceId, Key baseKey ) +{ + GenCondSpace *cond = allCondSpaces + cnum; + cond->condSpaceId = condSpaceId; + cond->baseKey = baseKey; +} + +void CodeGenData::condSpaceItem( int cnum, long condActionId ) +{ + GenCondSpace *cond = allCondSpaces + cnum; + cond->condSet.append( allActions + condActionId ); +} + +void CodeGenData::initStateCondList( int snum, ulong length ) +{ + /* Could preallocate these, as we could with transitions. */ +} + +void CodeGenData::addStateCond( int snum, Key lowKey, Key highKey, long condNum ) +{ + RedStateAp *curState = allStates + snum; + + /* Create the new state condition. */ + GenStateCond *stateCond = new GenStateCond; + stateCond->lowKey = lowKey; + stateCond->highKey = highKey; + + /* Assign it a cond space. */ + GenCondSpace *condSpace = allCondSpaces + condNum; + stateCond->condSpace = condSpace; + + curState->stateCondList.append( stateCond ); +} + + +GenCondSpace *CodeGenData::findCondSpace( Key lowKey, Key highKey ) +{ + for ( CondSpaceList::Iter cs = condSpaceList; cs.lte(); cs++ ) { + Key csHighKey = cs->baseKey; + csHighKey += keyOps->alphSize() * (1 << cs->condSet.length()); + + if ( lowKey >= cs->baseKey && highKey <= csHighKey ) + return cs; + } + return 0; +} + +Condition *CodeGenData::findCondition( Key key ) +{ + for ( ConditionList::Iter cond = conditionList; cond.lte(); cond++ ) { + Key upperKey = cond->baseKey + (1 << cond->condSet.length()); + if ( cond->baseKey <= key && key <= upperKey ) + return cond; + } + return 0; +} + +Key CodeGenData::findMaxKey() +{ + Key maxKey = keyOps->maxKey; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + assert( st->outSingle.length() == 0 ); + assert( st->defTrans == 0 ); + + long rangeLen = st->outRange.length(); + if ( rangeLen > 0 ) { + Key highKey = st->outRange[rangeLen-1].highKey; + if ( highKey > maxKey ) + maxKey = highKey; + } + } + return maxKey; +} + +void CodeGenData::findFinalActionRefs() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Rerence count out of single transitions. */ + for ( RedTransList::Iter rtel = st->outSingle; rtel.lte(); rtel++ ) { + if ( rtel->value->action != 0 ) { + rtel->value->action->numTransRefs += 1; + for ( GenActionTable::Iter item = rtel->value->action->key; item.lte(); item++ ) + item->value->numTransRefs += 1; + } + } + + /* Reference count out of range transitions. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + if ( rtel->value->action != 0 ) { + rtel->value->action->numTransRefs += 1; + for ( GenActionTable::Iter item = rtel->value->action->key; item.lte(); item++ ) + item->value->numTransRefs += 1; + } + } + + /* Reference count default transition. */ + if ( st->defTrans != 0 && st->defTrans->action != 0 ) { + st->defTrans->action->numTransRefs += 1; + for ( GenActionTable::Iter item = st->defTrans->action->key; item.lte(); item++ ) + item->value->numTransRefs += 1; + } + + /* Reference count eof transitions. */ + if ( st->eofTrans != 0 && st->eofTrans->action != 0 ) { + st->eofTrans->action->numTransRefs += 1; + for ( GenActionTable::Iter item = st->eofTrans->action->key; item.lte(); item++ ) + item->value->numTransRefs += 1; + } + + /* Reference count to state actions. */ + if ( st->toStateAction != 0 ) { + st->toStateAction->numToStateRefs += 1; + for ( GenActionTable::Iter item = st->toStateAction->key; item.lte(); item++ ) + item->value->numToStateRefs += 1; + } + + /* Reference count from state actions. */ + if ( st->fromStateAction != 0 ) { + st->fromStateAction->numFromStateRefs += 1; + for ( GenActionTable::Iter item = st->fromStateAction->key; item.lte(); item++ ) + item->value->numFromStateRefs += 1; + } + + /* Reference count EOF actions. */ + if ( st->eofAction != 0 ) { + st->eofAction->numEofRefs += 1; + for ( GenActionTable::Iter item = st->eofAction->key; item.lte(); item++ ) + item->value->numEofRefs += 1; + } + } +} + +void CodeGenData::analyzeAction( GenAction *act, GenInlineList *inlineList ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + /* Only consider actions that are referenced. */ + if ( act->numRefs() > 0 ) { + if ( item->type == GenInlineItem::Goto || item->type == GenInlineItem::GotoExpr ) + redFsm->bAnyActionGotos = true; + else if ( item->type == GenInlineItem::Call || item->type == GenInlineItem::CallExpr ) + redFsm->bAnyActionCalls = true; + else if ( item->type == GenInlineItem::Ret ) + redFsm->bAnyActionRets = true; + } + + /* Check for various things in regular actions. */ + if ( act->numTransRefs > 0 || act->numToStateRefs > 0 || act->numFromStateRefs > 0 ) { + /* Any returns in regular actions? */ + if ( item->type == GenInlineItem::Ret ) + redFsm->bAnyRegActionRets = true; + + /* Any next statements in the regular actions? */ + if ( item->type == GenInlineItem::Next || item->type == GenInlineItem::NextExpr ) + redFsm->bAnyRegNextStmt = true; + + /* Any by value control in regular actions? */ + if ( item->type == GenInlineItem::CallExpr || item->type == GenInlineItem::GotoExpr ) + redFsm->bAnyRegActionByValControl = true; + + /* Any references to the current state in regular actions? */ + if ( item->type == GenInlineItem::Curs ) + redFsm->bAnyRegCurStateRef = true; + + if ( item->type == GenInlineItem::Break ) + redFsm->bAnyRegBreak = true; + } + + if ( item->children != 0 ) + analyzeAction( act, item->children ); + } +} + +void CodeGenData::analyzeActionList( RedAction *redAct, GenInlineList *inlineList ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + /* Any next statements in the action table? */ + if ( item->type == GenInlineItem::Next || item->type == GenInlineItem::NextExpr ) + redAct->bAnyNextStmt = true; + + /* Any references to the current state. */ + if ( item->type == GenInlineItem::Curs ) + redAct->bAnyCurStateRef = true; + + if ( item->type == GenInlineItem::Break ) + redAct->bAnyBreakStmt = true; + + if ( item->children != 0 ) + analyzeActionList( redAct, item->children ); + } +} + +/* Assign ids to referenced actions. */ +void CodeGenData::assignActionIds() +{ + int nextActionId = 0; + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Only ever interested in referenced actions. */ + if ( act->numRefs() > 0 ) + act->actionId = nextActionId++; + } +} + +void CodeGenData::setValueLimits() +{ + redFsm->maxSingleLen = 0; + redFsm->maxRangeLen = 0; + redFsm->maxKeyOffset = 0; + redFsm->maxIndexOffset = 0; + redFsm->maxActListId = 0; + redFsm->maxActionLoc = 0; + redFsm->maxActArrItem = 0; + redFsm->maxSpan = 0; + redFsm->maxCondSpan = 0; + redFsm->maxFlatIndexOffset = 0; + redFsm->maxCondOffset = 0; + redFsm->maxCondLen = 0; + redFsm->maxCondSpaceId = 0; + redFsm->maxCondIndexOffset = 0; + + /* In both of these cases the 0 index is reserved for no value, so the max + * is one more than it would be if they started at 0. */ + redFsm->maxIndex = redFsm->transSet.length(); + redFsm->maxCond = condSpaceList.length(); + + /* The nextStateId - 1 is the last state id assigned. */ + redFsm->maxState = redFsm->nextStateId - 1; + + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + if ( csi->condSpaceId > redFsm->maxCondSpaceId ) + redFsm->maxCondSpaceId = csi->condSpaceId; + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Maximum cond length. */ + if ( st->stateCondList.length() > redFsm->maxCondLen ) + redFsm->maxCondLen = st->stateCondList.length(); + + /* Maximum single length. */ + if ( st->outSingle.length() > redFsm->maxSingleLen ) + redFsm->maxSingleLen = st->outSingle.length(); + + /* Maximum range length. */ + if ( st->outRange.length() > redFsm->maxRangeLen ) + redFsm->maxRangeLen = st->outRange.length(); + + /* The key offset index offset for the state after last is not used, skip it.. */ + if ( ! st.last() ) { + redFsm->maxCondOffset += st->stateCondList.length(); + redFsm->maxKeyOffset += st->outSingle.length() + st->outRange.length()*2; + redFsm->maxIndexOffset += st->outSingle.length() + st->outRange.length() + 2; + } + + /* Max cond span. */ + if ( st->condList != 0 ) { + unsigned long long span = keyOps->span( st->condLowKey, st->condHighKey ); + if ( span > redFsm->maxCondSpan ) + redFsm->maxCondSpan = span; + } + + /* Max key span. */ + if ( st->transList != 0 ) { + unsigned long long span = keyOps->span( st->lowKey, st->highKey ); + if ( span > redFsm->maxSpan ) + redFsm->maxSpan = span; + } + + /* Max cond index offset. */ + if ( ! st.last() ) { + if ( st->condList != 0 ) + redFsm->maxCondIndexOffset += keyOps->span( st->condLowKey, st->condHighKey ); + } + + /* Max flat index offset. */ + if ( ! st.last() ) { + if ( st->transList != 0 ) + redFsm->maxFlatIndexOffset += keyOps->span( st->lowKey, st->highKey ); + redFsm->maxFlatIndexOffset += 1; + } + } + + for ( GenActionTableMap::Iter at = redFsm->actionMap; at.lte(); at++ ) { + /* Maximum id of action lists. */ + if ( at->actListId+1 > redFsm->maxActListId ) + redFsm->maxActListId = at->actListId+1; + + /* Maximum location of items in action array. */ + if ( at->location+1 > redFsm->maxActionLoc ) + redFsm->maxActionLoc = at->location+1; + + /* Maximum values going into the action array. */ + if ( at->key.length() > redFsm->maxActArrItem ) + redFsm->maxActArrItem = at->key.length(); + for ( GenActionTable::Iter item = at->key; item.lte(); item++ ) { + if ( item->value->actionId > redFsm->maxActArrItem ) + redFsm->maxActArrItem = item->value->actionId; + } + } +} + + + +/* Gather various info on the machine. */ +void CodeGenData::analyzeMachine() +{ + /* Find the true count of action references. */ + findFinalActionRefs(); + + /* Check if there are any calls in action code. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Record the occurrence of various kinds of actions. */ + if ( act->numToStateRefs > 0 ) + redFsm->bAnyToStateActions = true; + if ( act->numFromStateRefs > 0 ) + redFsm->bAnyFromStateActions = true; + if ( act->numEofRefs > 0 ) + redFsm->bAnyEofActions = true; + if ( act->numTransRefs > 0 ) + redFsm->bAnyRegActions = true; + + /* Recurse through the action's parse tree looking for various things. */ + analyzeAction( act, act->inlineList ); + } + + /* Analyze reduced action lists. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + for ( GenActionTable::Iter act = redAct->key; act.lte(); act++ ) + analyzeActionList( redAct, act->value->inlineList ); + } + + /* Find states that have transitions with actions that have next + * statements. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Check any actions out of outSinge. */ + for ( RedTransList::Iter rtel = st->outSingle; rtel.lte(); rtel++ ) { + if ( rtel->value->action != 0 && rtel->value->action->anyCurStateRef() ) + st->bAnyRegCurStateRef = true; + } + + /* Check any actions out of outRange. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + if ( rtel->value->action != 0 && rtel->value->action->anyCurStateRef() ) + st->bAnyRegCurStateRef = true; + } + + /* Check any action out of default. */ + if ( st->defTrans != 0 && st->defTrans->action != 0 && + st->defTrans->action->anyCurStateRef() ) + st->bAnyRegCurStateRef = true; + + if ( st->stateCondList.length() > 0 ) + redFsm->bAnyConditions = true; + + if ( st->eofTrans != 0 ) + redFsm->bAnyEofTrans = true; + } + + /* Assign ids to actions that are referenced. */ + assignActionIds(); + + /* Set the maximums of various values used for deciding types. */ + setValueLimits(); +} + +void CodeGenData::write_option_error( InputLoc &loc, char *arg ) +{ + source_warning(loc) << "unrecognized write option \"" << arg << "\"" << endl; +} + +void CodeGenData::writeStatement( InputLoc &loc, int nargs, char **args ) +{ + /* FIXME: This should be moved to the virtual functions in the code + * generators. + * + * Force a newline. */ + out << '\n'; + genLineDirective( out ); + + if ( strcmp( args[0], "data" ) == 0 ) { + for ( int i = 1; i < nargs; i++ ) { + if ( strcmp( args[i], "noerror" ) == 0 ) + noError = true; + else if ( strcmp( args[i], "noprefix" ) == 0 ) + noPrefix = true; + else if ( strcmp( args[i], "nofinal" ) == 0 ) + noFinal = true; + else + write_option_error( loc, args[i] ); + } + writeData(); + } + else if ( strcmp( args[0], "init" ) == 0 ) { + for ( int i = 1; i < nargs; i++ ) { + if ( strcmp( args[i], "nocs" ) == 0 ) + noCS = true; + else + write_option_error( loc, args[i] ); + } + writeInit(); + } + else if ( strcmp( args[0], "exec" ) == 0 ) { + for ( int i = 1; i < nargs; i++ ) { + if ( strcmp( args[i], "noend" ) == 0 ) + noEnd = true; + else + write_option_error( loc, args[i] ); + } + writeExec(); + } + else if ( strcmp( args[0], "exports" ) == 0 ) { + for ( int i = 1; i < nargs; i++ ) + write_option_error( loc, args[i] ); + writeExports(); + } + else if ( strcmp( args[0], "start" ) == 0 ) { + for ( int i = 1; i < nargs; i++ ) + write_option_error( loc, args[i] ); + writeStart(); + } + else if ( strcmp( args[0], "first_final" ) == 0 ) { + for ( int i = 1; i < nargs; i++ ) + write_option_error( loc, args[i] ); + writeFirstFinal(); + } + else if ( strcmp( args[0], "error" ) == 0 ) { + for ( int i = 1; i < nargs; i++ ) + write_option_error( loc, args[i] ); + writeError(); + } + else { + /* EMIT An error here. */ + source_error(loc) << "unrecognized write command \"" << + args[0] << "\"" << endl; + } +} + +ostream &CodeGenData::source_warning( const InputLoc &loc ) +{ + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": warning: "; + return cerr; +} + +ostream &CodeGenData::source_error( const InputLoc &loc ) +{ + gblErrorCount += 1; + assert( sourceFileName != 0 ); + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": "; + return cerr; +} + + diff --git a/ragel/gendata.h b/ragel/gendata.h new file mode 100644 index 0000000..4d21820 --- /dev/null +++ b/ragel/gendata.h @@ -0,0 +1,186 @@ +/* + * Copyright 2005-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _GENDATA_H +#define _GENDATA_H + +#include <iostream> +#include "config.h" +#include "redfsm.h" +#include "common.h" + +using std::ostream; + +extern bool generateDot; + +struct NameInst; +typedef DList<GenAction> GenActionList; + +typedef unsigned long ulong; + +extern int gblErrorCount; + +struct CodeGenData; + +typedef AvlMap<char *, CodeGenData*, CmpStr> CodeGenMap; +typedef AvlMapEl<char *, CodeGenData*> CodeGenMapEl; + +void cdLineDirective( ostream &out, const char *fileName, int line ); +void javaLineDirective( ostream &out, const char *fileName, int line ); +void rubyLineDirective( ostream &out, const char *fileName, int line ); +void csharpLineDirective( ostream &out, const char *fileName, int line ); +void genLineDirective( ostream &out ); +void lineDirective( ostream &out, const char *fileName, int line ); + +string itoa( int i ); + +/*********************************/ + +struct CodeGenData +{ + /* + * The interface to the code generator. + */ + virtual void finishRagelDef() {} + + /* These are invoked by the corresponding write statements. */ + virtual void writeData() {}; + virtual void writeInit() {}; + virtual void writeExec() {}; + virtual void writeExports() {}; + virtual void writeStart() {}; + virtual void writeFirstFinal() {}; + virtual void writeError() {}; + + /* This can also be overwridden to modify the processing of write + * statements. */ + virtual void writeStatement( InputLoc &loc, int nargs, char **args ); + + /********************/ + + CodeGenData( ostream &out ); + virtual ~CodeGenData() {} + + /* + * Collecting the machine. + */ + + const char *sourceFileName; + const char *fsmName; + ostream &out; + RedFsmAp *redFsm; + GenAction *allActions; + RedAction *allActionTables; + Condition *allConditions; + GenCondSpace *allCondSpaces; + RedStateAp *allStates; + NameInst **nameIndex; + int startState; + int errState; + GenActionList actionList; + ConditionList conditionList; + CondSpaceList condSpaceList; + GenInlineList *getKeyExpr; + GenInlineList *accessExpr; + GenInlineList *prePushExpr; + GenInlineList *postPopExpr; + + /* Overriding variables. */ + GenInlineList *pExpr; + GenInlineList *peExpr; + GenInlineList *eofExpr; + GenInlineList *csExpr; + GenInlineList *topExpr; + GenInlineList *stackExpr; + GenInlineList *actExpr; + GenInlineList *tokstartExpr; + GenInlineList *tokendExpr; + GenInlineList *dataExpr; + + KeyOps thisKeyOps; + bool wantComplete; + EntryIdVect entryPointIds; + EntryNameVect entryPointNames; + bool hasLongestMatch; + ExportList exportList; + + /* Write options. */ + bool noEnd; + bool noPrefix; + bool noFinal; + bool noError; + bool noCS; + + void createMachine(); + void initActionList( unsigned long length ); + void newAction( int anum, const char *name, const InputLoc &loc, GenInlineList *inlineList ); + void initActionTableList( unsigned long length ); + void initStateList( unsigned long length ); + void setStartState( unsigned long startState ); + void setErrorState( unsigned long errState ); + void addEntryPoint( char *name, unsigned long entryState ); + void setId( int snum, int id ); + void setFinal( int snum ); + void initTransList( int snum, unsigned long length ); + void newTrans( int snum, int tnum, Key lowKey, Key highKey, + long targ, long act ); + void finishTransList( int snum ); + void setStateActions( int snum, long toStateAction, + long fromStateAction, long eofAction ); + void setEofTrans( int snum, long targ, long eofAction ); + void setForcedErrorState() + { redFsm->forcedErrorState = true; } + + + void initCondSpaceList( ulong length ); + void condSpaceItem( int cnum, long condActionId ); + void newCondSpace( int cnum, int condSpaceId, Key baseKey ); + + void initStateCondList( int snum, ulong length ); + void addStateCond( int snum, Key lowKey, Key highKey, long condNum ); + + GenCondSpace *findCondSpace( Key lowKey, Key highKey ); + Condition *findCondition( Key key ); + + bool setAlphType( const char *data ); + + void resolveTargetStates( GenInlineList *inlineList ); + Key findMaxKey(); + + /* Gather various info on the machine. */ + void analyzeActionList( RedAction *redAct, GenInlineList *inlineList ); + void analyzeAction( GenAction *act, GenInlineList *inlineList ); + void findFinalActionRefs(); + void analyzeMachine(); + + void closeMachine(); + void setValueLimits(); + void assignActionIds(); + + ostream &source_warning( const InputLoc &loc ); + ostream &source_error( const InputLoc &loc ); + void write_option_error( InputLoc &loc, char *arg ); +}; + +CodeGenData *makeCodeGen( const char *sourceFileName, + const char *fsmName, ostream &out ); + +#endif diff --git a/ragel/inputdata.cpp b/ragel/inputdata.cpp new file mode 100644 index 0000000..ac837d5 --- /dev/null +++ b/ragel/inputdata.cpp @@ -0,0 +1,244 @@ +/* + * Copyright 2008 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "common.h" +#include "inputdata.h" +#include "parsedata.h" +#include "rlparse.h" +#include <iostream> +#include "dotcodegen.h" + +using std::cout; +using std::cerr; +using std::endl; +using std::ios; + +/* Invoked by the parser when the root element is opened. */ +void InputData::cdDefaultFileName( const char *inputFile ) +{ + /* If the output format is code and no output file name is given, then + * make a default. */ + if ( outputFileName == 0 ) { + const char *ext = findFileExtension( inputFile ); + if ( ext != 0 && strcmp( ext, ".rh" ) == 0 ) + outputFileName = fileNameFromStem( inputFile, ".h" ); + else { + const char *defExtension = 0; + switch ( hostLang->lang ) { + case HostLang::C: defExtension = ".c"; break; + case HostLang::D: defExtension = ".d"; break; + default: break; + } + outputFileName = fileNameFromStem( inputFile, defExtension ); + } + } +} + +/* Invoked by the parser when the root element is opened. */ +void InputData::javaDefaultFileName( const char *inputFile ) +{ + /* If the output format is code and no output file name is given, then + * make a default. */ + if ( outputFileName == 0 ) + outputFileName = fileNameFromStem( inputFile, ".java" ); +} + +/* Invoked by the parser when the root element is opened. */ +void InputData::rubyDefaultFileName( const char *inputFile ) +{ + /* If the output format is code and no output file name is given, then + * make a default. */ + if ( outputFileName == 0 ) + outputFileName = fileNameFromStem( inputFile, ".rb" ); +} + +/* Invoked by the parser when the root element is opened. */ +void InputData::csharpDefaultFileName( const char *inputFile ) +{ + /* If the output format is code and no output file name is given, then + * make a default. */ + if ( outputFileName == 0 ) { + const char *ext = findFileExtension( inputFile ); + if ( ext != 0 && strcmp( ext, ".rh" ) == 0 ) + outputFileName = fileNameFromStem( inputFile, ".h" ); + else + outputFileName = fileNameFromStem( inputFile, ".cs" ); + } +} + +void InputData::makeOutputStream() +{ + if ( ! generateDot && ! generateXML ) { + switch ( hostLang->lang ) { + case HostLang::C: + case HostLang::D: + cdDefaultFileName( inputFileName ); + break; + case HostLang::Java: + javaDefaultFileName( inputFileName ); + break; + case HostLang::Ruby: + rubyDefaultFileName( inputFileName ); + break; + case HostLang::CSharp: + csharpDefaultFileName( inputFileName ); + break; + } + } + + /* Make sure we are not writing to the same file as the input file. */ + if ( outputFileName != 0 ) { + if ( strcmp( inputFileName, outputFileName ) == 0 ) { + error() << "output file \"" << outputFileName << + "\" is the same as the input file" << endl; + } + + /* Create the filter on the output and open it. */ + outFilter = new output_filter( outputFileName ); + + /* Open the output stream, attaching it to the filter. */ + outStream = new ostream( outFilter ); + } + else { + /* Writing out ot std out. */ + outStream = &cout; + } +} + +void InputData::openOutput() +{ + if ( outFilter != 0 ) { + outFilter->open( outputFileName, ios::out|ios::trunc ); + if ( !outFilter->is_open() ) { + error() << "error opening " << outputFileName << " for writing" << endl; + exit(1); + } + } +} + +void InputData::prepareMachineGen() +{ + if ( generateDot ) { + /* Locate a machine spec to generate dot output for. We can only emit. + * Dot takes one graph at a time. */ + if ( machineSpec != 0 ) { + /* Machine specified. */ + ParserDictEl *pdEl = parserDict.find( machineSpec ); + if ( pdEl == 0 ) + error() << "could not locate machine specified with -S and/or -M" << endp; + dotGenParser = pdEl->value; + } + else { + /* No machine spec given, just use the first one. */ + if ( parserList.length() == 0 ) + error() << "no machine specification to generate graphviz output" << endp; + + dotGenParser = parserList.head; + } + + GraphDictEl *gdEl = 0; + + if ( machineName != 0 ) { + gdEl = dotGenParser->pd->graphDict.find( machineName ); + if ( gdEl == 0 ) + error() << "machine definition/instantiation not found" << endp; + } + else { + /* We are using the whole machine spec. Need to make sure there + * are instances in the spec. */ + if ( dotGenParser->pd->instanceList.length() == 0 ) + error() << "no machine instantiations to generate graphviz output" << endp; + } + + dotGenParser->pd->prepareMachineGen( gdEl ); + } + else { + /* No machine spec or machine name given. Generate everything. */ + for ( ParserDict::Iter parser = parserDict; parser.lte(); parser++ ) { + ParseData *pd = parser->value->pd; + if ( pd->instanceList.length() > 0 ) + pd->prepareMachineGen( 0 ); + } + } +} + +void InputData::generateReduced() +{ + if ( generateDot ) + dotGenParser->pd->generateReduced( *this ); + else { + for ( ParserDict::Iter parser = parserDict; parser.lte(); parser++ ) { + ParseData *pd = parser->value->pd; + if ( pd->instanceList.length() > 0 ) + pd->generateReduced( *this ); + } + } +} + +/* Send eof to all parsers. */ +void InputData::terminateAllParsers( ) +{ + /* FIXME: a proper token is needed here. Suppose we should use the + * location of EOF in the last file that the parser was referenced in. */ + InputLoc loc; + loc.fileName = "<EOF>"; + loc.line = 0; + loc.col = 0; + for ( ParserDict::Iter pdel = parserDict; pdel.lte(); pdel++ ) + pdel->value->token( loc, Parser_tk_eof, 0, 0 ); +} + +void InputData::verifyWritesHaveData() +{ + if ( !generateXML && !generateDot ) { + for ( InputItemList::Iter ii = inputItems; ii.lte(); ii++ ) { + if ( ii->type == InputItem::Write ) { + if ( ii->pd->cgd == 0 ) + error( ii->loc ) << "no machine instantiations to write" << endl; + } + } + } +} + +void InputData::writeOutput() +{ + if ( generateXML ) + writeXML( *outStream ); + else if ( generateDot ) + static_cast<GraphvizDotGen*>(dotGenParser->pd->cgd)->writeDotFile(); + else { + for ( InputItemList::Iter ii = inputItems; ii.lte(); ii++ ) { + if ( ii->type == InputItem::Write ) { + CodeGenData *cgd = ii->pd->cgd; + ::keyOps = &cgd->thisKeyOps; + + cgd->writeStatement( ii->loc, ii->writeArgs.length()-1, ii->writeArgs.data ); + } + else { + *outStream << '\n'; + lineDirective( *outStream, inputFileName, ii->loc.line ); + *outStream << ii->data.str(); + } + } + } +} + diff --git a/ragel/inputdata.h b/ragel/inputdata.h new file mode 100644 index 0000000..045be56 --- /dev/null +++ b/ragel/inputdata.h @@ -0,0 +1,104 @@ +/* + * Copyright 2008 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _INPUT_DATA +#define _INPUT_DATA + +#include "gendata.h" +#include <iostream> +#include <sstream> + +struct Parser; +struct ParseData; + +struct InputItem +{ + enum Type { + HostData, + Write, + }; + + Type type; + std::ostringstream data; + std::string name; + ParseData *pd; + Vector<char *> writeArgs; + + InputLoc loc; + + InputItem *prev, *next; +}; + +struct Parser; + +typedef AvlMap<const char*, Parser*, CmpStr> ParserDict; +typedef AvlMapEl<const char*, Parser*> ParserDictEl; +typedef DList<Parser> ParserList; +typedef DList<InputItem> InputItemList; +typedef Vector<const char *> ArgsVector; + +struct InputData +{ + InputData() : + inputFileName(0), + outputFileName(0), + inStream(0), + outStream(0), + outFilter(0), + dotGenParser(0) + {} + + /* The name of the root section, this does not change during an include. */ + const char *inputFileName; + const char *outputFileName; + + /* Io globals. */ + std::istream *inStream; + std::ostream *outStream; + output_filter *outFilter; + + Parser *dotGenParser; + + ParserDict parserDict; + ParserList parserList; + InputItemList inputItems; + + ArgsVector includePaths; + + void verifyWritesHaveData(); + + void writeOutput(); + void makeOutputStream(); + void openOutput(); + void generateReduced(); + void prepareMachineGen(); + void terminateAllParsers(); + + void cdDefaultFileName( const char *inputFile ); + void javaDefaultFileName( const char *inputFile ); + void rubyDefaultFileName( const char *inputFile ); + void csharpDefaultFileName( const char *inputFile ); + + void writeLanguage( std::ostream &out ); + void writeXML( std::ostream &out ); +}; + +#endif diff --git a/ragel/javacodegen.cpp b/ragel/javacodegen.cpp new file mode 100644 index 0000000..adff67e --- /dev/null +++ b/ragel/javacodegen.cpp @@ -0,0 +1,1683 @@ +/* + * Copyright 2006-2007 Adrian Thurston <thurston@complang.org> + * 2007 Colin Fleming <colin.fleming@caverock.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "javacodegen.h" +#include "redfsm.h" +#include "gendata.h" +#include <iomanip> +#include <sstream> + +/* Integer array line length. */ +#define IALL 12 + +/* Static array initialization item count + * (should be multiple of IALL). */ +#define SAIIC 8184 + +#define _resume 1 +#define _again 2 +#define _eof_trans 3 +#define _test_eof 4 +#define _out 5 + +using std::setw; +using std::ios; +using std::ostringstream; +using std::string; +using std::cerr; + +using std::istream; +using std::ifstream; +using std::ostream; +using std::ios; +using std::cin; +using std::cout; +using std::cerr; +using std::endl; + +void javaLineDirective( ostream &out, const char *fileName, int line ) +{ + /* Write the preprocessor line info for to the input file. */ + out << "// line " << line << " \""; + for ( const char *pc = fileName; *pc != 0; pc++ ) { + if ( *pc == '\\' ) + out << "\\\\"; + else + out << *pc; + } + out << "\"\n"; +} + +void JavaTabCodeGen::genLineDirective( ostream &out ) +{ + std::streambuf *sbuf = out.rdbuf(); + output_filter *filter = static_cast<output_filter*>(sbuf); + javaLineDirective( out, filter->fileName, filter->line + 1 ); +} + +void JavaTabCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{" << vCS() << " = " << gotoDest << "; _goto_targ = " << _again << "; " << + CTRL_FLOW() << "continue _goto;}"; +} + +void JavaTabCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << "); _goto_targ = " << _again << "; " << CTRL_FLOW() << "continue _goto;}"; +} + +void JavaTabCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = " << + callDest << "; _goto_targ = " << _again << "; " << CTRL_FLOW() << "continue _goto;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void JavaTabCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish ); + ret << "); _goto_targ = " << _again << "; " << CTRL_FLOW() << "continue _goto;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void JavaTabCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << TOP() << "];"; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false ); + ret << "}"; + } + + ret << "_goto_targ = " << _again << "; " << CTRL_FLOW() << "continue _goto;}"; +} + +void JavaTabCodeGen::BREAK( ostream &ret, int targState ) +{ + ret << "{ " << P() << " += 1; _goto_targ = " << _out << "; " << + CTRL_FLOW() << " continue _goto;}"; +} + +void JavaTabCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void JavaTabCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + +void JavaTabCodeGen::EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ) +{ + /* The parser gives fexec two children. The double brackets are for D + * code. If the inline list is a single word it will get interpreted as a + * C-style cast by the D compiler. */ + ret << "{" << P() << " = (("; + INLINE_LIST( ret, item->children, targState, inFinish ); + ret << "))-1;}"; +} + +/* Write out an inline tree structure. Walks the list and possibly calls out + * to virtual functions than handle language specific items in the tree. */ +void JavaTabCodeGen::INLINE_LIST( ostream &ret, GenInlineList *inlineList, + int targState, bool inFinish ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Text: + ret << item->data; + break; + case GenInlineItem::Goto: + GOTO( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Call: + CALL( ret, item->targState->id, targState, inFinish ); + break; + case GenInlineItem::Next: + NEXT( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Ret: + RET( ret, inFinish ); + break; + case GenInlineItem::PChar: + ret << P(); + break; + case GenInlineItem::Char: + ret << GET_KEY(); + break; + case GenInlineItem::Hold: + ret << P() << "--;"; + break; + case GenInlineItem::Exec: + EXEC( ret, item, targState, inFinish ); + break; + case GenInlineItem::Curs: + ret << "(_ps)"; + break; + case GenInlineItem::Targs: + ret << "(" << vCS() << ")"; + break; + case GenInlineItem::Entry: + ret << item->targState->id; + break; + case GenInlineItem::GotoExpr: + GOTO_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::CallExpr: + CALL_EXPR( ret, item, targState, inFinish ); + break; + case GenInlineItem::NextExpr: + NEXT_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::LmSwitch: + LM_SWITCH( ret, item, targState, inFinish ); + break; + case GenInlineItem::LmSetActId: + SET_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokEnd: + SET_TOKEND( ret, item ); + break; + case GenInlineItem::LmGetTokEnd: + GET_TOKEND( ret, item ); + break; + case GenInlineItem::LmInitTokStart: + INIT_TOKSTART( ret, item ); + break; + case GenInlineItem::LmInitAct: + INIT_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokStart: + SET_TOKSTART( ret, item ); + break; + case GenInlineItem::SubAction: + SUB_ACTION( ret, item, targState, inFinish ); + break; + case GenInlineItem::Break: + BREAK( ret, targState ); + break; + } + } +} + +string JavaTabCodeGen::DATA_PREFIX() +{ + if ( !noPrefix ) + return FSM_NAME() + "_"; + return ""; +} + +/* Emit the alphabet data type. */ +string JavaTabCodeGen::ALPH_TYPE() +{ + string ret = keyOps->alphType->data1; + if ( keyOps->alphType->data2 != 0 ) { + ret += " "; + ret += + keyOps->alphType->data2; + } + return ret; +} + +/* Emit the alphabet data type. */ +string JavaTabCodeGen::WIDE_ALPH_TYPE() +{ + string ret; + if ( redFsm->maxKey <= keyOps->maxKey ) + ret = ALPH_TYPE(); + else { + long long maxKeyVal = redFsm->maxKey.getLongLong(); + HostType *wideType = keyOps->typeSubsumes( keyOps->isSigned, maxKeyVal ); + assert( wideType != 0 ); + + ret = wideType->data1; + if ( wideType->data2 != 0 ) { + ret += " "; + ret += wideType->data2; + } + } + return ret; +} + + + +void JavaTabCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << GET_KEY() << ";\n" + " _keys = " << CO() << "[" << vCS() << "]*2\n;" + " _klen = " << CL() << "[" << vCS() << "];\n" + " if ( _klen > 0 ) {\n" + " int _lower = _keys\n;" + " int _mid;\n" + " int _upper = _keys + (_klen<<1) - 2;\n" + " while (true) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = _lower + (((_upper-_lower) >> 1) & ~1);\n" + " if ( " << GET_WIDE_KEY() << " < " << CK() << "[_mid] )\n" + " _upper = _mid - 2;\n" + " else if ( " << GET_WIDE_KEY() << " > " << CK() << "[_mid+1] )\n" + " _lower = _mid + 2;\n" + " else {\n" + " switch ( " << C() << "[" << CO() << "[" << vCS() << "]" + " + ((_mid - _keys)>>1)] ) {\n" + ; + + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " case " << condSpace->condSpaceId << ": {\n"; + out << TABS(2) << "_widec = " << KEY(condSpace->baseKey) << + " + (" << GET_KEY() << " - " << KEY(keyOps->minKey) << ");\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(2) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } + + out << + " break;\n" + " }\n"; + } + + out << + " }\n" + " break;\n" + " }\n" + " }\n" + " }\n" + "\n"; +} + + +void JavaTabCodeGen::LOCATE_TRANS() +{ + out << + " _match: do {\n" + " _keys = " << KO() << "[" << vCS() << "]" << ";\n" + " _trans = " << IO() << "[" << vCS() << "];\n" + " _klen = " << SL() << "[" << vCS() << "];\n" + " if ( _klen > 0 ) {\n" + " int _lower = _keys;\n" + " int _mid;\n" + " int _upper = _keys + _klen - 1;\n" + " while (true) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = _lower + ((_upper-_lower) >> 1);\n" + " if ( " << GET_WIDE_KEY() << " < " << K() << "[_mid] )\n" + " _upper = _mid - 1;\n" + " else if ( " << GET_WIDE_KEY() << " > " << K() << "[_mid] )\n" + " _lower = _mid + 1;\n" + " else {\n" + " _trans += (_mid - _keys);\n" + " break _match;\n" + " }\n" + " }\n" + " _keys += _klen;\n" + " _trans += _klen;\n" + " }\n" + "\n" + " _klen = " << RL() << "[" << vCS() << "];\n" + " if ( _klen > 0 ) {\n" + " int _lower = _keys;\n" + " int _mid;\n" + " int _upper = _keys + (_klen<<1) - 2;\n" + " while (true) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = _lower + (((_upper-_lower) >> 1) & ~1);\n" + " if ( " << GET_WIDE_KEY() << " < " << K() << "[_mid] )\n" + " _upper = _mid - 2;\n" + " else if ( " << GET_WIDE_KEY() << " > " << K() << "[_mid+1] )\n" + " _lower = _mid + 2;\n" + " else {\n" + " _trans += ((_mid - _keys)>>1);\n" + " break _match;\n" + " }\n" + " }\n" + " _trans += _klen;\n" + " }\n" + " } while (false);\n" + "\n"; +} + +/* Determine if we should use indicies or not. */ +void JavaTabCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActionLoc) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActionLoc) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +int JavaTabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + return act; +} + +int JavaTabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + return act; +} + +int JavaTabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + return act; +} + + +int JavaTabCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + return act; +} + +std::ostream &JavaTabCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &JavaTabCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &JavaTabCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, true ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &JavaTabCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &JavaTabCodeGen::COND_OFFSETS() +{ + int curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + ARRAY_ITEM( INT(curKeyOffset), st.last() ); + + /* Move the key offset ahead. */ + curKeyOffset += st->stateCondList.length(); + } + return out; +} + +std::ostream &JavaTabCodeGen::KEY_OFFSETS() +{ + int curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + ARRAY_ITEM( INT(curKeyOffset), st.last() ); + + /* Move the key offset ahead. */ + curKeyOffset += st->outSingle.length() + st->outRange.length()*2; + } + return out; +} + + +std::ostream &JavaTabCodeGen::INDEX_OFFSETS() +{ + int curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + ARRAY_ITEM( INT(curIndOffset), st.last() ); + + /* Move the index offset ahead. */ + curIndOffset += st->outSingle.length() + st->outRange.length(); + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + return out; +} + +std::ostream &JavaTabCodeGen::COND_LENS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + ARRAY_ITEM( INT(st->stateCondList.length()), st.last() ); + } + return out; +} + + +std::ostream &JavaTabCodeGen::SINGLE_LENS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + ARRAY_ITEM( INT(st->outSingle.length()), st.last() ); + } + return out; +} + +std::ostream &JavaTabCodeGen::RANGE_LENS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit length of range index. */ + ARRAY_ITEM( INT(st->outRange.length()), st.last() ); + } + return out; +} + +std::ostream &JavaTabCodeGen::TO_STATE_ACTIONS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT(TO_STATE_ACTION(st)), st.last() ); + } + return out; +} + +std::ostream &JavaTabCodeGen::FROM_STATE_ACTIONS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT(FROM_STATE_ACTION(st)), st.last() ); + } + return out; +} + +std::ostream &JavaTabCodeGen::EOF_ACTIONS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT(EOF_ACTION(st)), st.last() ); + } + return out; +} + +std::ostream &JavaTabCodeGen::EOF_TRANS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + + /* Write any eof action. */ + ARRAY_ITEM( INT(trans), st.last() ); + } + return out; +} + + +std::ostream &JavaTabCodeGen::COND_KEYS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Lower key. */ + ARRAY_ITEM( KEY( sc->lowKey ), false ); + ARRAY_ITEM( KEY( sc->highKey ), false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), true ); + return out; +} + +std::ostream &JavaTabCodeGen::COND_SPACES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Cond Space id. */ + ARRAY_ITEM( KEY( sc->condSpace->condSpaceId ), false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), true ); + return out; +} + +std::ostream &JavaTabCodeGen::KEYS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + ARRAY_ITEM( KEY( stel->lowKey ), false ); + } + + /* Loop the state's transitions. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + /* Lower key. */ + ARRAY_ITEM( KEY( rtel->lowKey ), false ); + + /* Upper key. */ + ARRAY_ITEM( KEY( rtel->highKey ), false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), true ); + return out; +} + +std::ostream &JavaTabCodeGen::INDICIES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + ARRAY_ITEM( KEY( stel->value->id ), false ); + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + ARRAY_ITEM( KEY( rtel->value->id ), false ); + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + ARRAY_ITEM( KEY( st->defTrans->id ), false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), true ); + return out; +} + +std::ostream &JavaTabCodeGen::TRANS_TARGS() +{ + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + ARRAY_ITEM( KEY( trans->targ->id ), false ); + totalTrans++; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + ARRAY_ITEM( KEY( trans->targ->id ), false ); + totalTrans++; + } + + /* The state's default target state. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + ARRAY_ITEM( KEY( trans->targ->id ), false ); + totalTrans++; + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + trans->pos = totalTrans++; + ARRAY_ITEM( KEY( trans->targ->id ), false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), true ); + return out; +} + + +std::ostream &JavaTabCodeGen::TRANS_ACTIONS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), false ); + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), false ); + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), false ); + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), true ); + return out; +} + +std::ostream &JavaTabCodeGen::TRANS_TARGS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Save the position. Needed for eofTargs. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + ARRAY_ITEM( INT(trans->targ->id), ( t >= redFsm->transSet.length()-1 ) ); + } + delete[] transPtrs; + return out; +} + + +std::ostream &JavaTabCodeGen::TRANS_ACTIONS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), ( t >= redFsm->transSet.length()-1 ) ); + } + delete[] transPtrs; + return out; +} + +void JavaTabCodeGen::writeExports() +{ + if ( exportList.length() > 0 ) { + for ( ExportList::Iter ex = exportList; ex.lte(); ex++ ) { + STATIC_VAR( ALPH_TYPE(), DATA_PREFIX() + "ex_" + ex->name ) + << " = " << KEY(ex->key) << ";\n"; + } + out << "\n"; + } +} + +void JavaTabCodeGen::writeStart() +{ + out << START_STATE_ID(); +} + +void JavaTabCodeGen::writeFirstFinal() +{ + out << FIRST_FINAL_STATE(); +} + +void JavaTabCodeGen::writeError() +{ + out << ERROR_STATE(); +} + +void JavaTabCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + "\n"; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->startState != 0 ) + STATIC_VAR( "int", START() ) << " = " << START_STATE_ID() << ";\n"; + + if ( !noFinal ) + STATIC_VAR( "int" , FIRST_FINAL() ) << " = " << FIRST_FINAL_STATE() << ";\n"; + + if ( !noError ) + STATIC_VAR( "int", ERROR() ) << " = " << ERROR_STATE() << ";\n"; + + out << "\n"; + + if ( entryPointNames.length() > 0 ) { + for ( EntryNameVect::Iter en = entryPointNames; en.lte(); en++ ) { + STATIC_VAR( "int", DATA_PREFIX() + "en_" + *en ) << + " = " << entryPointIds[en.pos()] << ";\n"; + } + out << "\n"; + } +} + +void JavaTabCodeGen::writeExec() +{ + out << + " {\n" + " int _klen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << + ";\n" + " int _trans = 0;\n"; + + if ( redFsm->anyConditions() ) + out << " int _widec;\n"; + + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() || + redFsm->anyFromStateActions() ) + { + out << + " int _acts;\n" + " int _nacts;\n"; + } + + out << + " int _keys;\n" + " int _goto_targ = 0;\n" + "\n"; + + out << + " _goto: while (true) {\n" + " switch ( _goto_targ ) {\n" + " case 0:\n"; + + if ( !noEnd ) { + out << + " if ( " << P() << " == " << PE() << " ) {\n" + " _goto_targ = " << _test_eof << ";\n" + " continue _goto;\n" + " }\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " ) {\n" + " _goto_targ = " << _out << ";\n" + " continue _goto;\n" + " }\n"; + } + + out << "case " << _resume << ":\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << FSA() << "[" << vCS() << "]" << ";\n" + " _nacts = " << CAST("int") << " " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + FROM_STATE_ACTION_SWITCH() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( useIndicies ) + out << " _trans = " << I() << "[_trans];\n"; + + if ( redFsm->anyEofTrans() ) + out << "case " << _eof_trans << ":\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] != 0 ) {\n" + " _acts = " << TA() << "[_trans]" << ";\n" + " _nacts = " << CAST("int") << " " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 )\n {\n" + " switch ( " << A() << "[_acts++] )\n" + " {\n"; + ACTION_SWITCH() << + " }\n" + " }\n" + " }\n" + "\n"; + } + + out << "case " << _again << ":\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << TSA() << "[" << vCS() << "]" << ";\n" + " _nacts = " << CAST("int") << " " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + TO_STATE_ACTION_SWITCH() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " ) {\n" + " _goto_targ = " << _out << ";\n" + " continue _goto;\n" + " }\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " ) {\n" + " _goto_targ = " << _resume << ";\n" + " continue _goto;\n" + " }\n"; + } + else { + out << + " " << P() << " += 1;\n" + " _goto_targ = " << _resume << ";\n" + " continue _goto;\n"; + } + + out << "case " << _test_eof << ":\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " _goto_targ = " << _eof_trans << ";\n" + " continue _goto;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " int __acts = " << EA() << "[" << vCS() << "]" << ";\n" + " int __nacts = " << CAST("int") << " " << A() << "[__acts++];\n" + " while ( __nacts-- > 0 ) {\n" + " switch ( " << A() << "[__acts++] ) {\n"; + EOF_ACTION_SWITCH() << + " }\n" + " }\n"; + } + + out << + " }\n" + "\n"; + } + + out << "case " << _out << ":\n"; + + /* The switch and goto loop. */ + out << " }\n"; + out << " break; }\n"; + + /* The execute block. */ + out << " }\n"; +} + +std::ostream &JavaTabCodeGen::OPEN_ARRAY( string type, string name ) +{ + array_type = type; + array_name = name; + item_count = 0; + div_count = 1; + + out << "private static " << type << "[] init_" << name << "_0()\n" + "{\n\t" + "return new " << type << " [] {\n\t"; + return out; +} + +std::ostream &JavaTabCodeGen::ARRAY_ITEM( string item, bool last ) +{ + item_count++; + + out << setw(5) << setiosflags(ios::right) << item; + + if ( !last ) { + if ( item_count % SAIIC == 0 ) { + out << "\n\t};\n};\n" + "private static "<< array_type << "[] init_" << + array_name << "_" << div_count << "()\n" + "{\n\t" + "return new " << array_type << " [] {\n\t"; + div_count++; + } else if (item_count % IALL == 0) { + out << ",\n\t"; + } else { + out << ","; + } + } + return out; +} + +std::ostream &JavaTabCodeGen::CLOSE_ARRAY() +{ + out << "\n\t};\n}\n\n"; + + if (item_count < SAIIC) { + out << "private static final " << array_type << " " << array_name << + "[] = init_" << array_name << "_0();\n\n"; + } else { + out << "private static final " << array_type << " [] combine_" << array_name + << "() {\n\t" + << array_type << " [] combined = new " << array_type << + " [ " << item_count << " ];\n\t"; + int block = 0; + int full_blocks = item_count / SAIIC; + for (;block < full_blocks; ++block) { + out << "System.arraycopy ( init_" << array_name << "_" << block << + "(), 0, combined, " << SAIIC * block << ", " << SAIIC << " );\n\t"; + } + if ( (item_count % SAIIC) > 0 ) { + out << "System.arraycopy ( init_" << array_name << "_" << block << + "(), 0, combined, " << SAIIC * block << ", " << + (item_count % SAIIC) << " );\n\t"; + } + out << "return combined;\n}\n"; + out << "private static final " << array_type << " [] " << array_name << + " = combine_" << array_name << "();"; + } + return out; +} + + +std::ostream &JavaTabCodeGen::STATIC_VAR( string type, string name ) +{ + out << "static final " << type << " " << name; + return out; +} + +string JavaTabCodeGen::ARR_OFF( string ptr, string offset ) +{ + return ptr + " + " + offset; +} + +string JavaTabCodeGen::CAST( string type ) +{ + return "(" + type + ")"; +} + +string JavaTabCodeGen::NULL_ITEM() +{ + /* In java we use integers instead of pointers. */ + return "-1"; +} + +string JavaTabCodeGen::GET_KEY() +{ + ostringstream ret; + if ( getKeyExpr != 0 ) { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, getKeyExpr, 0, false ); + ret << ")"; + } + else { + /* Expression for retrieving the key, use simple dereference. */ + ret << DATA() << "[" << P() << "]"; + } + return ret.str(); +} + +string JavaTabCodeGen::CTRL_FLOW() +{ + return "if (true) "; +} + +unsigned int JavaTabCodeGen::arrayTypeSize( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + return arrayType->size; +} + +string JavaTabCodeGen::ARRAY_TYPE( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + + string ret = arrayType->data1; + if ( arrayType->data2 != 0 ) { + ret += " "; + ret += arrayType->data2; + } + return ret; +} + + +/* Write out the fsm name. */ +string JavaTabCodeGen::FSM_NAME() +{ + return fsmName; +} + +/* Emit the offset of the start state as a decimal integer. */ +string JavaTabCodeGen::START_STATE_ID() +{ + ostringstream ret; + ret << redFsm->startState->id; + return ret.str(); +}; + +/* Write out the array of actions. */ +std::ostream &JavaTabCodeGen::ACTIONS_ARRAY() +{ + ARRAY_ITEM( INT(0), false ); + for ( GenActionTableMap::Iter act = redFsm->actionMap; act.lte(); act++ ) { + /* Write out the length, which will never be the last character. */ + ARRAY_ITEM( INT(act->key.length()), false ); + + for ( GenActionTable::Iter item = act->key; item.lte(); item++ ) + ARRAY_ITEM( INT(item->value->actionId), (act.last() && item.last()) ); + } + return out; +} + + +string JavaTabCodeGen::ACCESS() +{ + ostringstream ret; + if ( accessExpr != 0 ) + INLINE_LIST( ret, accessExpr, 0, false ); + return ret.str(); +} + +string JavaTabCodeGen::P() +{ + ostringstream ret; + if ( pExpr == 0 ) + ret << "p"; + else { + ret << "("; + INLINE_LIST( ret, pExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::PE() +{ + ostringstream ret; + if ( peExpr == 0 ) + ret << "pe"; + else { + ret << "("; + INLINE_LIST( ret, peExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::vEOF() +{ + ostringstream ret; + if ( eofExpr == 0 ) + ret << "eof"; + else { + ret << "("; + INLINE_LIST( ret, eofExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::vCS() +{ + ostringstream ret; + if ( csExpr == 0 ) + ret << ACCESS() << "cs"; + else { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, csExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::TOP() +{ + ostringstream ret; + if ( topExpr == 0 ) + ret << ACCESS() + "top"; + else { + ret << "("; + INLINE_LIST( ret, topExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::STACK() +{ + ostringstream ret; + if ( stackExpr == 0 ) + ret << ACCESS() + "stack"; + else { + ret << "("; + INLINE_LIST( ret, stackExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::ACT() +{ + ostringstream ret; + if ( actExpr == 0 ) + ret << ACCESS() + "act"; + else { + ret << "("; + INLINE_LIST( ret, actExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::TOKSTART() +{ + ostringstream ret; + if ( tokstartExpr == 0 ) + ret << ACCESS() + "ts"; + else { + ret << "("; + INLINE_LIST( ret, tokstartExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::TOKEND() +{ + ostringstream ret; + if ( tokendExpr == 0 ) + ret << ACCESS() + "te"; + else { + ret << "("; + INLINE_LIST( ret, tokendExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::DATA() +{ + ostringstream ret; + if ( dataExpr == 0 ) + ret << ACCESS() + "data"; + else { + ret << "("; + INLINE_LIST( ret, dataExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + + +string JavaTabCodeGen::GET_WIDE_KEY() +{ + if ( redFsm->anyConditions() ) + return "_widec"; + else + return GET_KEY(); +} + +string JavaTabCodeGen::GET_WIDE_KEY( RedStateAp *state ) +{ + if ( state->stateCondList.length() > 0 ) + return "_widec"; + else + return GET_KEY(); +} + +/* Write out level number of tabs. Makes the nested binary search nice + * looking. */ +string JavaTabCodeGen::TABS( int level ) +{ + string result; + while ( level-- > 0 ) + result += "\t"; + return result; +} + +string JavaTabCodeGen::KEY( Key key ) +{ + ostringstream ret; + if ( keyOps->isSigned || !hostLang->explicitUnsigned ) + ret << key.getVal(); + else + ret << (unsigned long) key.getVal(); + return ret.str(); +} + +string JavaTabCodeGen::INT( int i ) +{ + ostringstream ret; + ret << i; + return ret.str(); +} + +void JavaTabCodeGen::LM_SWITCH( ostream &ret, GenInlineItem *item, + int targState, int inFinish ) +{ + ret << + " switch( " << ACT() << " ) {\n"; + + for ( GenInlineList::Iter lma = *item->children; lma.lte(); lma++ ) { + /* Write the case label, the action and the case break. */ + if ( lma->lmId < 0 ) + ret << " default:\n"; + else + ret << " case " << lma->lmId << ":\n"; + + /* Write the block and close it off. */ + ret << " {"; + INLINE_LIST( ret, lma->children, targState, inFinish ); + ret << "}\n"; + + ret << " break;\n"; + } + + ret << + " }\n" + "\t"; +} + +void JavaTabCodeGen::SET_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = " << item->lmId << ";"; +} + +void JavaTabCodeGen::SET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + /* The tokend action sets tokend. */ + ret << TOKEND() << " = " << P(); + if ( item->offset != 0 ) + out << "+" << item->offset; + out << ";"; +} + +void JavaTabCodeGen::GET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + ret << TOKEND(); +} + +void JavaTabCodeGen::INIT_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << NULL_ITEM() << ";"; +} + +void JavaTabCodeGen::INIT_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = 0;"; +} + +void JavaTabCodeGen::SET_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << P() << ";"; +} + +void JavaTabCodeGen::SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish ) +{ + if ( item->children->length() > 0 ) { + /* Write the block and close it off. */ + ret << "{"; + INLINE_LIST( ret, item->children, targState, inFinish ); + ret << "}"; + } +} + +void JavaTabCodeGen::ACTION( ostream &ret, GenAction *action, int targState, bool inFinish ) +{ + /* Write the preprocessor line info for going into the source file. */ + javaLineDirective( ret, action->loc.fileName, action->loc.line ); + + /* Write the block and close it off. */ + ret << "\t{"; + INLINE_LIST( ret, action->inlineList, targState, inFinish ); + ret << "}\n"; +} + +void JavaTabCodeGen::CONDITION( ostream &ret, GenAction *condition ) +{ + ret << "\n"; + javaLineDirective( ret, condition->loc.fileName, condition->loc.line ); + INLINE_LIST( ret, condition->inlineList, 0, false ); +} + +string JavaTabCodeGen::ERROR_STATE() +{ + ostringstream ret; + if ( redFsm->errState != 0 ) + ret << redFsm->errState->id; + else + ret << "-1"; + return ret.str(); +} + +string JavaTabCodeGen::FIRST_FINAL_STATE() +{ + ostringstream ret; + if ( redFsm->firstFinState != 0 ) + ret << redFsm->firstFinState->id; + else + ret << redFsm->nextStateId; + return ret.str(); +} + +void JavaTabCodeGen::writeInit() +{ + out << " {\n"; + + if ( !noCS ) + out << "\t" << vCS() << " = " << START() << ";\n"; + + /* If there are any calls, then the stack top needs initialization. */ + if ( redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "\t" << TOP() << " = 0;\n"; + + if ( hasLongestMatch ) { + out << + " " << TOKSTART() << " = " << NULL_ITEM() << ";\n" + " " << TOKEND() << " = " << NULL_ITEM() << ";\n" + " " << ACT() << " = 0;\n"; + } + out << " }\n"; +} + +void JavaTabCodeGen::finishRagelDef() +{ + /* The frontend will do this for us, but it may be a good idea to force it + * if the intermediate file is edited. */ + redFsm->sortByStateId(); + + /* Choose default transitions and the single transition. */ + redFsm->chooseDefaultSpan(); + + /* Maybe do flat expand, otherwise choose single. */ + redFsm->chooseSingle(); + + /* If any errors have occured in the input file then don't write anything. */ + if ( gblErrorCount > 0 ) + return; + + /* Anlayze Machine will find the final action reference counts, among + * other things. We will use these in reporting the usage + * of fsm directives in action code. */ + analyzeMachine(); + + /* Determine if we should use indicies. */ + calcIndexSize(); +} + +ostream &JavaTabCodeGen::source_warning( const InputLoc &loc ) +{ + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": warning: "; + return cerr; +} + +ostream &JavaTabCodeGen::source_error( const InputLoc &loc ) +{ + gblErrorCount += 1; + assert( sourceFileName != 0 ); + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": "; + return cerr; +} + + diff --git a/ragel/javacodegen.h b/ragel/javacodegen.h new file mode 100644 index 0000000..06914c9 --- /dev/null +++ b/ragel/javacodegen.h @@ -0,0 +1,197 @@ +/* + * Copyright 2006-2007 Adrian Thurston <thurston@complang.org> + * 2007 Colin Fleming <colin.fleming@caverock.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _JAVACODEGEN_H +#define _JAVACODEGEN_H + +#include <iostream> +#include <string> +#include <stdio.h> +#include "common.h" +#include "gendata.h" + +using std::string; +using std::ostream; + +/* + * JavaTabCodeGen + */ +struct JavaTabCodeGen : public CodeGenData +{ + JavaTabCodeGen( ostream &out ) : + CodeGenData(out) {} + + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + std::ostream &COND_KEYS(); + std::ostream &COND_SPACES(); + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &COND_OFFSETS(); + std::ostream &KEY_OFFSETS(); + std::ostream &INDEX_OFFSETS(); + std::ostream &COND_LENS(); + std::ostream &SINGLE_LENS(); + std::ostream &RANGE_LENS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + std::ostream &TRANS_TARGS_WI(); + std::ostream &TRANS_ACTIONS_WI(); + + void BREAK( ostream &ret, int targState ); + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void RET( ostream &ret, bool inFinish ); + + void COND_TRANSLATE(); + void LOCATE_TRANS(); + + virtual void writeExec(); + virtual void writeData(); + virtual void writeInit(); + virtual void writeExports(); + virtual void writeStart(); + virtual void writeFirstFinal(); + virtual void writeError(); + virtual void finishRagelDef(); + + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + + int TO_STATE_ACTION( RedStateAp *state ); + int FROM_STATE_ACTION( RedStateAp *state ); + int EOF_ACTION( RedStateAp *state ); + int TRANS_ACTION( RedTransAp *trans ); + + /* Determine if we should use indicies. */ + void calcIndexSize(); + +private: + string array_type; + string array_name; + int item_count; + int div_count; + +public: + + virtual string NULL_ITEM(); + virtual ostream &OPEN_ARRAY( string type, string name ); + virtual ostream &ARRAY_ITEM( string item, bool last ); + virtual ostream &CLOSE_ARRAY(); + virtual ostream &STATIC_VAR( string type, string name ); + virtual string ARR_OFF( string ptr, string offset ); + virtual string CAST( string type ); + virtual string GET_KEY(); + virtual string CTRL_FLOW(); + + string FSM_NAME(); + string START_STATE_ID(); + ostream &ACTIONS_ARRAY(); + string GET_WIDE_KEY(); + string GET_WIDE_KEY( RedStateAp *state ); + string TABS( int level ); + string KEY( Key key ); + string INT( int i ); + void ACTION( ostream &ret, GenAction *action, int targState, bool inFinish ); + void CONDITION( ostream &ret, GenAction *condition ); + string ALPH_TYPE(); + string WIDE_ALPH_TYPE(); + string ARRAY_TYPE( unsigned long maxVal ); + + string ACCESS(); + + string P(); + string PE(); + string vEOF(); + + string vCS(); + string STACK(); + string TOP(); + string TOKSTART(); + string TOKEND(); + string ACT(); + string DATA(); + + string DATA_PREFIX(); + string PM() { return "_" + DATA_PREFIX() + "partition_map"; } + string C() { return "_" + DATA_PREFIX() + "cond_spaces"; } + string CK() { return "_" + DATA_PREFIX() + "cond_keys"; } + string K() { return "_" + DATA_PREFIX() + "trans_keys"; } + string I() { return "_" + DATA_PREFIX() + "indicies"; } + string CO() { return "_" + DATA_PREFIX() + "cond_offsets"; } + string KO() { return "_" + DATA_PREFIX() + "key_offsets"; } + string IO() { return "_" + DATA_PREFIX() + "index_offsets"; } + string CL() { return "_" + DATA_PREFIX() + "cond_lengths"; } + string SL() { return "_" + DATA_PREFIX() + "single_lengths"; } + string RL() { return "_" + DATA_PREFIX() + "range_lengths"; } + string A() { return "_" + DATA_PREFIX() + "actions"; } + string TA() { return "_" + DATA_PREFIX() + "trans_actions"; } + string TT() { return "_" + DATA_PREFIX() + "trans_targs"; } + string TSA() { return "_" + DATA_PREFIX() + "to_state_actions"; } + string FSA() { return "_" + DATA_PREFIX() + "from_state_actions"; } + string EA() { return "_" + DATA_PREFIX() + "eof_actions"; } + string ET() { return "_" + DATA_PREFIX() + "eof_trans"; } + string SP() { return "_" + DATA_PREFIX() + "key_spans"; } + string CSP() { return "_" + DATA_PREFIX() + "cond_key_spans"; } + string START() { return DATA_PREFIX() + "start"; } + string ERROR() { return DATA_PREFIX() + "error"; } + string FIRST_FINAL() { return DATA_PREFIX() + "first_final"; } + string CTXDATA() { return DATA_PREFIX() + "ctxdata"; } + + void INLINE_LIST( ostream &ret, GenInlineList *inlineList, int targState, bool inFinish ); + void EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void EXECTE( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void LM_SWITCH( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void SET_ACT( ostream &ret, GenInlineItem *item ); + void INIT_TOKSTART( ostream &ret, GenInlineItem *item ); + void INIT_ACT( ostream &ret, GenInlineItem *item ); + void SET_TOKSTART( ostream &ret, GenInlineItem *item ); + void SET_TOKEND( ostream &ret, GenInlineItem *item ); + void GET_TOKEND( ostream &ret, GenInlineItem *item ); + void SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish ); + + string ERROR_STATE(); + string FIRST_FINAL_STATE(); + + ostream &source_warning(const InputLoc &loc); + ostream &source_error(const InputLoc &loc); + + unsigned int arrayTypeSize( unsigned long maxVal ); + + bool outLabelUsed; + bool againLabelUsed; + bool useIndicies; + + void genLineDirective( ostream &out ); +}; + +#endif diff --git a/ragel/main.cpp b/ragel/main.cpp index 41d6e6a..2b54b51 100644 --- a/ragel/main.cpp +++ b/ragel/main.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2001-2005 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -26,16 +26,35 @@ #include <fstream> #include <unistd.h> #include <sstream> +#include <unistd.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <fcntl.h> +#include <errno.h> + +#ifdef _WIN32 +#include <windows.h> +#include <psapi.h> +#include <time.h> +#include <io.h> +#include <process.h> + +#if _MSC_VER +#define S_IRUSR _S_IREAD +#define S_IWUSR _S_IWRITE +#endif +#endif /* Parsing. */ #include "ragel.h" +#include "rlscan.h" /* Parameters and output. */ #include "pcheck.h" #include "vector.h" #include "version.h" - -#include "common.cpp" +#include "common.h" +#include "inputdata.h" using std::istream; using std::ostream; @@ -45,21 +64,32 @@ using std::cin; using std::cout; using std::cerr; using std::endl; - -/* Io globals. */ -istream *inStream = 0; -ostream *outStream = 0; -char *outputFileName = 0; +using std::ios; +using std::streamsize; /* Controls minimization. */ MinimizeLevel minimizeLevel = MinimizePartition2; MinimizeOpt minimizeOpt = MinimizeMostOps; /* Graphviz dot file generation. */ -char *machineSpec = 0, *machineName = 0; +const char *machineSpec = 0, *machineName = 0; bool machineSpecFound = false; +bool wantDupsRemoved = true; bool printStatistics = false; +bool generateXML = false; +bool generateDot = false; + +/* Target language and output style. */ +CodeStyle codeStyle = GenTables; + +int numSplitPartitions = 0; +bool noLineDirectives = false; + +bool displayPrintables = false; + +/* Target ruby impl */ +RubyImplEnum rubyImpl = MRI; /* Print a summary of the options. */ void usage() @@ -71,58 +101,107 @@ void usage() " -v, --version Print version information and exit\n" " -o <file> Write output to <file>\n" " -s Print some statistics on stderr\n" +" -d Do not remove duplicates from action lists\n" +" -I <dir> Add <dir> to the list of directories to search\n" +" for included an imported files\n" +"error reporting format:\n" +" --error-format=gnu file:line:column: message (default)\n" +" --error-format=msvc file(line,column): message\n" "fsm minimization:\n" " -n Do not perform minimization\n" " -m Minimize at the end of the compilation\n" " -l Minimize after most operations (default)\n" " -e Minimize after every operation\n" -"machine selection:\n" -" -S <spec> FSM specification to output for -V\n" -" -M <machine> Machine definition/instantiation to output for -V\n" +"visualization:\n" +" -x Run the frontend only: emit XML intermediate format\n" +" -V Generate a dot file for Graphviz\n" +" -p Display printable characters on labels\n" +" -S <spec> FSM specification to output (for graphviz output)\n" +" -M <machine> Machine definition/instantiation to output (for graphviz output)\n" "host language:\n" " -C The host language is C, C++, Obj-C or Obj-C++ (default)\n" " -D The host language is D\n" " -J The host language is Java\n" +" -R The host language is Ruby\n" +" -A The host language is C#\n" +"line direcives: (C/D/C#)\n" +" -L Inhibit writing of #line directives\n" +"code style: (C/D/Java/Ruby/C#)\n" +" -T0 Table driven FSM (default)\n" +"code style: (C/D/Ruby/C#)\n" +" -T1 Faster table driven FSM\n" +" -F0 Flat table driven FSM\n" +" -F1 Faster flat table-driven FSM\n" +"code style: (C/D/C#)\n" +" -G0 Goto-driven FSM\n" +" -G1 Faster goto-driven FSM\n" +"code style: (C/D)\n" +" -G2 Really fast goto-driven FSM\n" +" -P<N> N-Way Split really fast goto-driven FSM\n" ; + + exit(0); } -/* Print version information. */ +/* Print version information and exit. */ void version() { cout << "Ragel State Machine Compiler version " VERSION << " " PUBDATE << endl << - "Copyright (c) 2001-2006 by Adrian Thurston" << endl; + "Copyright (c) 2001-2009 by Adrian Thurston" << endl; + exit(0); } -/* Global parse data pointer. */ -//extern InputData *id; +/* Error reporting format. */ +ErrorFormat errorFormat = ErrorFormatGNU; + +InputLoc makeInputLoc( const char *fileName, int line, int col) +{ + InputLoc loc = { fileName, line, col }; + return loc; +} + +ostream &operator<<( ostream &out, const InputLoc &loc ) +{ + assert( loc.fileName != 0 ); + switch ( errorFormat ) { + case ErrorFormatMSVC: + out << loc.fileName << "(" << loc.line; + if ( loc.col ) + out << "," << loc.col; + out << ")"; + break; + + default: + out << loc.fileName << ":" << loc.line; + if ( loc.col ) + out << ":" << loc.col; + break; + } + return out; +} /* Total error count. */ int gblErrorCount = 0; -/* Print the opening to a program error, then return the error stream. */ -ostream &error() +/* Print the opening to a warning in the input, then return the error ostream. */ +ostream &warning( const InputLoc &loc ) { - /* Keep the error count. */ -// if ( id != 0 && id->pd != 0 ) -// id->pd->errorCount += 1; - gblErrorCount += 1; - - cerr << PROGNAME ": "; + cerr << loc << ": warning: "; return cerr; } -/* Print the opening to a warning, then return the error ostream. */ -ostream &warning( ) +/* Print the opening to a program error, then return the error stream. */ +ostream &error() { -// cerr << id->fileName << ": warning: "; + gblErrorCount += 1; + cerr << PROGNAME ": "; return cerr; } -/* Print the opening to a warning in the input, then return the error ostream. */ -ostream &warning( const InputLoc &loc ) +ostream &error( const InputLoc &loc ) { -// cerr << id->fileName << ":" << loc.line << ":" << -// loc.col << ": warning: "; + gblErrorCount += 1; + cerr << loc << ": "; return cerr; } @@ -136,28 +215,41 @@ void escapeLineDirectivePath( std::ostream &out, char *path ) } } -/* Main, process args and call yyparse to start scanning input. */ -int main(int argc, char **argv) +void processArgs( int argc, const char **argv, InputData &id ) { - ParamCheck pc("o:nmleabjkS:M:CDJvHh?-:s", argc, argv); - char *inputFileName = 0; + ParamCheck pc("xo:dnmleabjkS:M:I:CDJRAvHh?-:sT:F:G:P:LpV", argc, argv); + + /* FIXME: Need to check code styles VS langauge. */ while ( pc.check() ) { switch ( pc.state ) { case ParamCheck::match: switch ( pc.parameter ) { + case 'V': + generateDot = true; + break; + + case 'x': + generateXML = true; + break; + /* Output. */ case 'o': - if ( *pc.parameterArg == 0 ) + if ( *pc.paramArg == 0 ) error() << "a zero length output file name was given" << endl; - else if ( outputFileName != 0 ) + else if ( id.outputFileName != 0 ) error() << "more than one output file name was given" << endl; else { /* Ok, remember the output file name. */ - outputFileName = pc.parameterArg; + id.outputFileName = pc.paramArg; } break; + /* Flag for turning off duplicate action removal. */ + case 'd': + wantDupsRemoved = false; + break; + /* Minimization, mostly hidden options. */ case 'n': minimizeOpt = MinimizeNone; @@ -186,65 +278,142 @@ int main(int argc, char **argv) /* Machine spec. */ case 'S': - if ( *pc.parameterArg == 0 ) + if ( *pc.paramArg == 0 ) error() << "please specify an argument to -S" << endl; else if ( machineSpec != 0 ) error() << "more than one -S argument was given" << endl; else { /* Ok, remember the path to the machine to generate. */ - machineSpec = pc.parameterArg; + machineSpec = pc.paramArg; } break; /* Machine path. */ case 'M': - if ( *pc.parameterArg == 0 ) + if ( *pc.paramArg == 0 ) error() << "please specify an argument to -M" << endl; else if ( machineName != 0 ) error() << "more than one -M argument was given" << endl; else { /* Ok, remember the machine name to generate. */ - machineName = pc.parameterArg; + machineName = pc.paramArg; + } + break; + + case 'I': + if ( *pc.paramArg == 0 ) + error() << "please specify an argument to -I" << endl; + else { + id.includePaths.append( pc.paramArg ); } break; /* Host language types. */ case 'C': - hostLangType = CCode; hostLang = &hostLangC; break; case 'D': - hostLangType = DCode; hostLang = &hostLangD; break; case 'J': - hostLangType = JavaCode; hostLang = &hostLangJava; break; + case 'R': + hostLang = &hostLangRuby; + break; + case 'A': + hostLang = &hostLangCSharp; + break; /* Version and help. */ case 'v': version(); - exit(0); + break; case 'H': case 'h': case '?': usage(); - exit(0); + break; case 's': printStatistics = true; break; - case '-': - if ( strcasecmp(pc.parameterArg, "help") == 0 ) { + case '-': { + char *arg = strdup( pc.paramArg ); + char *eq = strchr( arg, '=' ); + + if ( eq != 0 ) + *eq++ = 0; + + if ( strcmp( arg, "help" ) == 0 ) usage(); - exit(0); - } - else if ( strcasecmp(pc.parameterArg, "version") == 0 ) { + else if ( strcmp( arg, "version" ) == 0 ) version(); - exit(0); + else if ( strcmp( arg, "error-format" ) == 0 ) { + if ( eq == 0 ) + error() << "expecting '=value' for error-format" << endl; + else if ( strcmp( eq, "gnu" ) == 0 ) + errorFormat = ErrorFormatGNU; + else if ( strcmp( eq, "msvc" ) == 0 ) + errorFormat = ErrorFormatMSVC; + else + error() << "invalid value for error-format" << endl; } + else if ( strcmp( arg, "rbx" ) == 0 ) + rubyImpl = Rubinius; else { - error() << "--" << pc.parameterArg << + error() << "--" << pc.paramArg << " is an invalid argument" << endl; } + free( arg ); + break; + } + + /* Passthrough args. */ + case 'T': + if ( pc.paramArg[0] == '0' ) + codeStyle = GenTables; + else if ( pc.paramArg[0] == '1' ) + codeStyle = GenFTables; + else { + error() << "-T" << pc.paramArg[0] << + " is an invalid argument" << endl; + exit(1); + } + break; + case 'F': + if ( pc.paramArg[0] == '0' ) + codeStyle = GenFlat; + else if ( pc.paramArg[0] == '1' ) + codeStyle = GenFFlat; + else { + error() << "-F" << pc.paramArg[0] << + " is an invalid argument" << endl; + exit(1); + } + break; + case 'G': + if ( pc.paramArg[0] == '0' ) + codeStyle = GenGoto; + else if ( pc.paramArg[0] == '1' ) + codeStyle = GenFGoto; + else if ( pc.paramArg[0] == '2' ) + codeStyle = GenIpGoto; + else { + error() << "-G" << pc.paramArg[0] << + " is an invalid argument" << endl; + exit(1); + } + break; + case 'P': + codeStyle = GenSplit; + numSplitPartitions = atoi( pc.paramArg ); + break; + + case 'p': + displayPrintables = true; + break; + + case 'L': + noLineDirectives = true; + break; } break; @@ -256,84 +425,134 @@ int main(int argc, char **argv) /* It is interpreted as an input file. */ if ( *pc.curArg == 0 ) error() << "a zero length input file name was given" << endl; - else if ( inputFileName != 0 ) + else if ( id.inputFileName != 0 ) error() << "more than one input file name was given" << endl; else { /* OK, Remember the filename. */ - inputFileName = pc.curArg; + id.inputFileName = pc.curArg; } break; } } +} + +void process( InputData &id ) +{ + /* Open the input file for reading. */ + assert( id.inputFileName != 0 ); + ifstream *inFile = new ifstream( id.inputFileName ); + if ( ! inFile->is_open() ) + error() << "could not open " << id.inputFileName << " for reading" << endp; + + /* Used for just a few things. */ + std::ostringstream hostData; + + /* Make the first input item. */ + InputItem *firstInputItem = new InputItem; + firstInputItem->type = InputItem::HostData; + firstInputItem->loc.fileName = id.inputFileName; + firstInputItem->loc.line = 1; + firstInputItem->loc.col = 1; + id.inputItems.append( firstInputItem ); + + Scanner scanner( id, id.inputFileName, *inFile, 0, 0, 0, false ); + scanner.do_scan(); - /* Bail on above errors. */ + /* Finished, final check for errors.. */ if ( gblErrorCount > 0 ) exit(1); - /* Make sure we are not writing to the same file as the input file. */ - if ( inputFileName != 0 && outputFileName != 0 && - strcmp( inputFileName, outputFileName ) == 0 ) - { - error() << "output file \"" << outputFileName << - "\" is the same as the input file" << endl; - } + /* Now send EOF to all parsers. */ + id.terminateAllParsers(); - /* Open the input file for reading. */ - if ( inputFileName != 0 ) { - /* Open the input file for reading. */ - ifstream *inFile = new ifstream( inputFileName ); - inStream = inFile; - if ( ! inFile->is_open() ) - error() << "could not open " << inputFileName << " for reading" << endl; - } - else { - inputFileName = "<stdin>"; - inStream = &cin; - } + /* Bail on above error. */ + if ( gblErrorCount > 0 ) + exit(1); + /* Locate the backend program */ + /* Compiles machines. */ + id.prepareMachineGen(); - /* Bail on above errors. */ if ( gblErrorCount > 0 ) exit(1); - std::ostringstream outputBuffer; - outStream = &outputBuffer; + id.makeOutputStream(); - if ( machineSpec == 0 && machineName == 0 ) - *outStream << "<host line=\"1\" col=\"1\">"; + /* Generates the reduced machine, which we use to write output. */ + if ( !generateXML ) { + id.generateReduced(); - scan( inputFileName, *inStream ); + if ( gblErrorCount > 0 ) + exit(1); + } - /* Finished, final check for errors.. */ + id.verifyWritesHaveData(); if ( gblErrorCount > 0 ) - return 1; - - /* Now send EOF to all parsers. */ - terminateAllParsers(); + exit(1); - /* Finished, final check for errors.. */ - if ( gblErrorCount > 0 ) - return 1; + /* + * From this point on we should not be reporting any errors. + */ - if ( machineSpec == 0 && machineName == 0 ) - *outStream << "</host>\n"; + id.openOutput(); + id.writeOutput(); - checkMachines(); + /* Close the input and the intermediate file. */ + delete inFile; + /* If writing to a file, delete the ostream, causing it to flush. + * Standard out is flushed automatically. */ + if ( id.outputFileName != 0 ) { + delete id.outStream; + delete id.outFilter; + } + + assert( gblErrorCount == 0 ); +} + +char *makeIntermedTemplate( const char *baseFileName ) +{ + char *result = 0; + const char *templ = "ragel-XXXXXX.xml"; + const char *lastSlash = strrchr( baseFileName, '/' ); + if ( lastSlash == 0 ) { + result = new char[strlen(templ)+1]; + strcpy( result, templ ); + } + else { + int baseLen = lastSlash - baseFileName + 1; + result = new char[baseLen + strlen(templ) + 1]; + memcpy( result, baseFileName, baseLen ); + strcpy( result+baseLen, templ ); + } + return result; +}; + +/* Main, process args and call yyparse to start scanning input. */ +int main( int argc, const char **argv ) +{ + InputData id; + + processArgs( argc, argv, id ); + + /* Require an input file. If we use standard in then we won't have a file + * name on which to base the output. */ + if ( id.inputFileName == 0 ) + error() << "no input file given" << endl; + + /* Bail on argument processing errors. */ if ( gblErrorCount > 0 ) - return 1; - - ostream *outputFile = 0; - if ( outputFileName != 0 ) - outputFile = new ofstream( outputFileName ); - else - outputFile = &cout; - - /* Write the machines, then the surrounding code. */ - writeMachines( *outputFile, outputBuffer.str(), inputFileName ); - - if ( outputFileName != 0 ) - delete outputFile; + exit(1); + + /* Make sure we are not writing to the same file as the input file. */ + if ( id.inputFileName != 0 && id.outputFileName != 0 && + strcmp( id.inputFileName, id.outputFileName ) == 0 ) + { + error() << "output file \"" << id.outputFileName << + "\" is the same as the input file" << endp; + } + + process( id ); return 0; } diff --git a/ragel/parsedata.cpp b/ragel/parsedata.cpp index 3209e28..8ce89db 100644 --- a/ragel/parsedata.cpp +++ b/ragel/parsedata.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2001-2006 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001-2008 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -31,12 +31,14 @@ #include "parsetree.h" #include "mergesort.h" #include "xmlcodegen.h" +#include "version.h" +#include "inputdata.h" using namespace std; -char machineMain[] = "main"; +char mainMachine[] = "main"; -void Token::set( char *str, int len ) +void Token::set( const char *str, int len ) { length = len; data = new char[len+1]; @@ -60,7 +62,7 @@ void Token::append( const Token &other ) void afterOpMinimize( FsmAp *fsm, bool lastInSeq ) { /* Switch on the prefered minimization algorithm. */ - if ( minimizeOpt == MinimizeEveryOp || minimizeOpt == MinimizeMostOps && lastInSeq ) { + if ( minimizeOpt == MinimizeEveryOp || ( minimizeOpt == MinimizeMostOps && lastInSeq ) ) { /* First clean up the graph. FsmAp operations may leave these * lying around. There should be no dead end states. The subtract * intersection operators are the only places where they may be @@ -107,13 +109,13 @@ Key makeFsmKeyHex( char *str, const InputLoc &loc, ParseData *pd ) unsigned long ul = strtoul( str, 0, 16 ); - if ( errno == ERANGE || unusedBits && ul >> (size * 8) ) { + if ( errno == ERANGE || ( unusedBits && ul >> (size * 8) ) ) { error(loc) << "literal " << str << " overflows the alphabet type" << endl; ul = 1 << (size * 8); } if ( unusedBits && keyOps->alphType->isSigned && ul >> (size * 8 - 1) ) - ul |= (0xffffffff >> (size*8 ) ) << (size*8); + ul |= ( -1L >> (size*8) ) << (size*8); return Key( (long)ul ); } @@ -129,12 +131,12 @@ Key makeFsmKeyDec( char *str, const InputLoc &loc, ParseData *pd ) long long ll = strtoll( str, 0, 10 ); /* Check for underflow. */ - if ( errno == ERANGE && ll < 0 || ll < minVal) { + if ( ( errno == ERANGE && ll < 0 ) || ll < minVal) { error(loc) << "literal " << str << " underflows the alphabet type" << endl; ll = minVal; } /* Check for overflow. */ - else if ( errno == ERANGE && ll > 0 || ll > maxVal ) { + else if ( ( errno == ERANGE && ll > 0 ) || ll > maxVal ) { error(loc) << "literal " << str << " overflows the alphabet type" << endl; ll = maxVal; } @@ -415,7 +417,7 @@ bool NameInst::anyRefsRec() /* Initialize the structure that will collect info during the parse of a * machine. */ -ParseData::ParseData( char *fileName, char *sectionName, +ParseData::ParseData( const char *fileName, char *sectionName, const InputLoc §ionLoc ) : sectionGraph(0), @@ -424,22 +426,35 @@ ParseData::ParseData( char *fileName, char *sectionName, /* 0 is reserved for global error actions. */ nextLocalErrKey(1), nextNameId(0), + nextCondId(0), alphTypeSet(false), getKeyExpr(0), accessExpr(0), - curStateExpr(0), + prePushExpr(0), + postPopExpr(0), + pExpr(0), + peExpr(0), + eofExpr(0), + csExpr(0), + topExpr(0), + stackExpr(0), + actExpr(0), + tokstartExpr(0), + tokendExpr(0), + dataExpr(0), lowerNum(0), upperNum(0), fileName(fileName), sectionName(sectionName), sectionLoc(sectionLoc), - errorCount(0), curActionOrd(0), curPriorOrd(0), rootName(0), + exportsRootName(0), nextEpsilonResolvedLink(0), nextLongestMatchId(1), - lmRequiresErrorState(false) + lmRequiresErrorState(false), + cgd(0) { /* Initialize the dictionary of graphs. This is our symbol table. The * initialization needs to be done on construction which happens at the @@ -458,7 +473,7 @@ ParseData::~ParseData() /* Make a name id in the current name instantiation scope if it is not * already there. */ -NameInst *ParseData::addNameInst( const InputLoc &loc, char *data, bool isLabel ) +NameInst *ParseData::addNameInst( const InputLoc &loc, const char *data, bool isLabel ) { /* Create the name instantitaion object and insert it. */ NameInst *newNameInst = new NameInst( loc, curNameInst, data, nextNameId++, isLabel ); @@ -474,6 +489,12 @@ void ParseData::initNameWalk() curNameChild = 0; } +void ParseData::initExportsNameWalk() +{ + curNameInst = exportsRootName; + curNameChild = 0; +} + /* Goes into the next child scope. The number of the child is already set up. * We need this for the syncronous name tree and parse tree walk to work * properly. It is reset on entry into a scope and advanced on poping of a @@ -532,11 +553,12 @@ void ParseData::unsetObsoleteEntries( FsmAp *graph ) if ( name->numUses == name->numRefs ) { assert( graph->entryPoints.find( name->id ) != 0 ); graph->unsetEntry( name->id ); + assert( graph->entryPoints.find( name->id ) == 0 ); } } } -NameSet ParseData::resolvePart( NameInst *refFrom, char *data, bool recLabelsOnly ) +NameSet ParseData::resolvePart( NameInst *refFrom, const char *data, bool recLabelsOnly ) { /* Queue needed for breadth-first search, load it with the start node. */ NameInstList nameQueue; @@ -715,21 +737,25 @@ void ParseData::resolveNameRefs( InlineList *inlineList, Action *action ) /* Resolve, pass action for local search. */ NameInst *target = resolveStateRef( *item->nameRef, item->loc, action ); - /* Check if the target goes into a longest match. */ - NameInst *search = target->parent; - while ( search != 0 ) { - if ( search->isLongestMatch ) { - error(item->loc) << "cannot enter inside a longest " - "match construction as an entry point" << endl; - break; + /* Name lookup error reporting is handled by resolveStateRef. */ + if ( target != 0 ) { + /* Check if the target goes into a longest match. */ + NameInst *search = target->parent; + while ( search != 0 ) { + if ( search->isLongestMatch ) { + error(item->loc) << "cannot enter inside a longest " + "match construction as an entry point" << endl; + break; + } + search = search->parent; } - search = search->parent; - } - /* Note the reference in the name. This will cause the entry - * point to survive to the end of the graph generating walk. */ - if ( target != 0 ) + /* Record the reference in the name. This will cause the + * entry point to survive to the end of the graph + * generating walk. */ target->numRefs += 1; + } + item->nameTarg = target; break; } @@ -766,18 +792,18 @@ void ParseData::fillNameIndex( NameInst *from ) fillNameIndex( *name ); } -void ParseData::makeRootName() +void ParseData::makeRootNames() { /* Create the root name. */ rootName = new NameInst( InputLoc(), 0, 0, nextNameId++, false ); + exportsRootName = new NameInst( InputLoc(), 0, 0, nextNameId++, false ); } /* Build the name tree and supporting data structures. */ void ParseData::makeNameTree( GraphDictEl *dictEl ) { /* Set up curNameInst for the walk. */ - curNameInst = rootName; - curNameChild = 0; + initNameWalk(); if ( dictEl != 0 ) { /* A start location has been specified. */ @@ -795,14 +821,16 @@ void ParseData::makeNameTree( GraphDictEl *dictEl ) nameIndex = new NameInst*[nextNameId]; memset( nameIndex, 0, sizeof(NameInst*)*nextNameId ); fillNameIndex( rootName ); + fillNameIndex( exportsRootName ); } -void ParseData::createBuiltin( char *name, BuiltinMachine builtin ) + +void ParseData::createBuiltin( const char *name, BuiltinMachine builtin ) { Expression *expression = new Expression( builtin ); Join *join = new Join( expression ); - JoinOrLm *joinOrLm = new JoinOrLm( join ); - VarDef *varDef = new VarDef( name, joinOrLm ); + MachineDef *machineDef = new MachineDef( join ); + VarDef *varDef = new VarDef( name, machineDef ); GraphDictEl *graphDictEl = new GraphDictEl( name, varDef ); graphDict.insert( graphDictEl ); } @@ -830,40 +858,51 @@ void ParseData::initGraphDict( ) } /* Set the alphabet type. If the types are not valid returns false. */ -bool ParseData::setAlphType( char *s1, char *s2 ) +bool ParseData::setAlphType( const InputLoc &loc, char *s1, char *s2 ) { - bool valid = false; - for ( int i = 0; i < hostLang->numHostTypes; i++ ) { - if ( strcmp( s1, hostLang->hostTypes[i].data1 ) == 0 && - hostLang->hostTypes[i].data2 != 0 && - strcmp( s2, hostLang->hostTypes[i].data2 ) == 0 ) - { - valid = true; - userAlphType = hostLang->hostTypes + i; - break; - } - } - + alphTypeLoc = loc; + userAlphType = findAlphType( s1, s2 ); alphTypeSet = true; - return valid; + return userAlphType != 0; } /* Set the alphabet type. If the types are not valid returns false. */ -bool ParseData::setAlphType( char *s1 ) +bool ParseData::setAlphType( const InputLoc &loc, char *s1 ) { - bool valid = false; - for ( int i = 0; i < hostLang->numHostTypes; i++ ) { - if ( strcmp( s1, hostLang->hostTypes[i].data1 ) == 0 && - hostLang->hostTypes[i].data2 == 0 ) - { - valid = true; - userAlphType = hostLang->hostTypes + i; - break; - } - } - + alphTypeLoc = loc; + userAlphType = findAlphType( s1 ); alphTypeSet = true; - return valid; + return userAlphType != 0; +} + +bool ParseData::setVariable( char *var, InlineList *inlineList ) +{ + bool set = true; + + if ( strcmp( var, "p" ) == 0 ) + pExpr = inlineList; + else if ( strcmp( var, "pe" ) == 0 ) + peExpr = inlineList; + else if ( strcmp( var, "eof" ) == 0 ) + eofExpr = inlineList; + else if ( strcmp( var, "cs" ) == 0 ) + csExpr = inlineList; + else if ( strcmp( var, "data" ) == 0 ) + dataExpr = inlineList; + else if ( strcmp( var, "top" ) == 0 ) + topExpr = inlineList; + else if ( strcmp( var, "stack" ) == 0 ) + stackExpr = inlineList; + else if ( strcmp( var, "act" ) == 0 ) + actExpr = inlineList; + else if ( strcmp( var, "ts" ) == 0 ) + tokstartExpr = inlineList; + else if ( strcmp( var, "te" ) == 0 ) + tokendExpr = inlineList; + else + set = false; + + return set; } /* Initialize the key operators object that will be referenced by all fsms @@ -880,8 +919,7 @@ void ParseData::initKeyOps( ) thisKeyOps.maxKey = makeFsmKeyNum( upperNum, rangeHighLoc, this ); } - thisCondData.nextCondKey = thisKeyOps.maxKey; - thisCondData.nextCondKey.increment(); + thisCondData.lastCondKey = thisKeyOps.maxKey; } void ParseData::printNameInst( NameInst *nameInst, int level ) @@ -890,7 +928,8 @@ void ParseData::printNameInst( NameInst *nameInst, int level ) cerr << " "; cerr << (nameInst->name != 0 ? nameInst->name : "<ANON>") << " id: " << nameInst->id << - " refs: " << nameInst->numRefs << endl; + " refs: " << nameInst->numRefs << + " uses: " << nameInst->numUses << endl; for ( NameVect::Iter name = nameInst->childVect; name.lte(); name++ ) printNameInst( *name, level+1 ); } @@ -927,13 +966,14 @@ void ParseData::removeActionDups( FsmAp *graph ) } } -Action *ParseData::newAction( char *name, InlineList *inlineList ) +Action *ParseData::newAction( const char *name, InlineList *inlineList ) { InputLoc loc; loc.line = 1; loc.col = 1; + loc.fileName = "NONE"; - Action *action = new Action( loc, name, inlineList ); + Action *action = new Action( loc, name, inlineList, nextCondId++ ); action->actionRefs.append( rootName ); actionList.append( action ); return action; @@ -957,13 +997,13 @@ void ParseData::initLongestMatchData() /* The setTokStart action sets tokstart. */ InlineList *il5 = new InlineList; il5->append( new InlineItem( InputLoc(), InlineItem::LmSetTokStart ) ); - setTokStart = newAction( "tokstart", il5 ); + setTokStart = newAction( "ts", il5 ); setTokStart->isLmAction = true; /* The setTokEnd action sets tokend. */ InlineList *il3 = new InlineList; il3->append( new InlineItem( InputLoc(), InlineItem::LmSetTokEnd ) ); - setTokEnd = newAction( "tokend", il3 ); + setTokEnd = newAction( "te", il3 ); setTokEnd->isLmAction = true; /* The action will also need an ordering: ahead of all user action @@ -1030,11 +1070,16 @@ FsmAp *ParseData::makeInstance( GraphDictEl *gdNode ) * All state construction is now complete. */ + /* Transfer actions from the out action tables to eof action tables. */ + for ( StateSet::Iter state = graph->finStateSet; state.lte(); state++ ) + graph->transferOutActions( *state ); + /* Transfer global error actions. */ for ( StateList::Iter state = graph->stateList; state.lte(); state++ ) graph->transferErrorActions( state, 0 ); - removeActionDups( graph ); + if ( ::wantDupsRemoved ) + removeActionDups( graph ); /* Remove unreachable states. There should be no dead end states. The * subtract and intersection operators are the only places where they may @@ -1083,7 +1128,7 @@ void ParseData::printNameTree() /* Show that the name index is correct. */ for ( int ni = 0; ni < nextNameId; ni++ ) { cerr << ni << ": "; - char *name = nameIndex[ni]->name; + const char *name = nameIndex[ni]->name; cerr << ( name != 0 ? name : "<ANON>" ) << endl; } } @@ -1102,6 +1147,8 @@ FsmAp *ParseData::makeSpecific( GraphDictEl *gdNode ) * is okay since generating part of the graph is usually only done when * inspecting the compiled machine. */ + /* Same story for extern entry point references. */ + /* Flag this case so that the XML code generator is aware that we haven't * looked up name references in actions. It can then avoid segfaulting. */ generatingSectionSubset = true; @@ -1126,6 +1173,10 @@ FsmAp *ParseData::makeAll() /* Resolve action code name references. */ resolveActionNameRefs(); + /* Force name references to the top level instantiations. */ + for ( NameVect::Iter inst = rootName->childVect; inst.lte(); inst++ ) + (*inst)->numRefs += 1; + FsmAp *mainGraph = 0; FsmAp **graphs = new FsmAp*[instanceList.length()]; int numOthers = 0; @@ -1133,23 +1184,19 @@ FsmAp *ParseData::makeAll() /* Make all the instantiations, we know that main exists in this list. */ initNameWalk(); for ( GraphList::Iter glel = instanceList; glel.lte(); glel++ ) { - if ( strcmp( glel->key, machineMain ) == 0 ) { + if ( strcmp( glel->key, mainMachine ) == 0 ) { /* Main graph is always instantiated. */ mainGraph = makeInstance( glel ); } else { - /* Check to see if the instance is ever referenced. */ - NameInst *nameInst = nextNameScope(); - if ( nameInst->anyRefsRec() ) - graphs[numOthers++] = makeInstance( glel ); - else { - /* Need to walk over the name tree item. */ - NameFrame nameFrame = enterNameScope( true, 1 ); - popNameScope( nameFrame ); - } + /* Instantiate and store in others array. */ + graphs[numOthers++] = makeInstance( glel ); } } + if ( mainGraph == 0 ) + mainGraph = graphs[--numOthers]; + if ( numOthers > 0 ) { /* Add all the other graphs into main. */ mainGraph->globOp( graphs, numOthers ); @@ -1199,31 +1246,9 @@ void ParseData::checkInlineList( Action *act, InlineList *inlineList ) /* EOF checks. */ if ( act->numEofRefs > 0 ) { switch ( item->type ) { - case InlineItem::PChar: - error(item->loc) << "pointer to current element does not exist in " - "EOF action code" << endl; - break; - case InlineItem::Char: - error(item->loc) << "current element does not exist in " - "EOF action code" << endl; - break; - case InlineItem::Hold: - error(item->loc) << "changing the current element not possible in " - "EOF action code" << endl; - break; - case InlineItem::Exec: - error(item->loc) << "changing the current element not possible in " - "EOF action code" << endl; - break; - case InlineItem::Goto: case InlineItem::Call: - case InlineItem::Next: case InlineItem::GotoExpr: - case InlineItem::CallExpr: case InlineItem::NextExpr: - case InlineItem::Ret: - error(item->loc) << "changing the current state not possible in " - "EOF action code" << endl; - break; - default: - break; + /* Currently no checks. */ + default: + break; } } @@ -1287,11 +1312,83 @@ void ParseData::analyzeGraph( FsmAp *graph ) checkAction( act ); } +void ParseData::makeExportsNameTree() +{ + /* Make a name tree for the exports. */ + initExportsNameWalk(); + + /* First make the name tree. */ + for ( GraphDict::Iter gdel = graphDict; gdel.lte(); gdel++ ) { + if ( gdel->value->isExport ) { + /* Recurse on the instance. */ + gdel->value->makeNameTree( gdel->loc, this ); + } + } +} + +void ParseData::makeExports() +{ + makeExportsNameTree(); + + /* Resove name references in the tree. */ + initExportsNameWalk(); + for ( GraphDict::Iter gdel = graphDict; gdel.lte(); gdel++ ) { + if ( gdel->value->isExport ) + gdel->value->resolveNameRefs( this ); + } + + /* Make all the instantiations, we know that main exists in this list. */ + initExportsNameWalk(); + for ( GraphDict::Iter gdel = graphDict; gdel.lte(); gdel++ ) { + /* Check if this var def is an export. */ + if ( gdel->value->isExport ) { + /* Build the graph from a walk of the parse tree. */ + FsmAp *graph = gdel->value->walk( this ); + + /* Build the graph from a walk of the parse tree. */ + if ( !graph->checkSingleCharMachine() ) { + error(gdel->loc) << "bad export machine, must define " + "a single character" << endl; + } + else { + /* Safe to extract the key and declare the export. */ + Key exportKey = graph->startState->outList.head->lowKey; + exportList.append( new Export( gdel->value->name, exportKey ) ); + } + } + } + +} + +/* Construct the machine and catch failures which can occur during + * construction. */ void ParseData::prepareMachineGen( GraphDictEl *graphDictEl ) { + try { + /* This machine construction can fail. */ + prepareMachineGenTBWrapped( graphDictEl ); + } + catch ( FsmConstructFail fail ) { + switch ( fail.reason ) { + case FsmConstructFail::CondNoKeySpace: { + InputLoc &loc = alphTypeSet ? alphTypeLoc : sectionLoc; + error(loc) << "sorry, no more characters are " + "available in the alphabet space" << endl; + error(loc) << " for conditions, please use a " + "smaller alphtype or reduce" << endl; + error(loc) << " the span of characters on which " + "conditions are embedded" << endl; + break; + } + } + } +} + +void ParseData::prepareMachineGenTBWrapped( GraphDictEl *graphDictEl ) +{ beginProcessing(); initKeyOps(); - makeRootName(); + makeRootNames(); initLongestMatchData(); /* Make the graph, do minimization. */ @@ -1299,6 +1396,9 @@ void ParseData::prepareMachineGen( GraphDictEl *graphDictEl ) sectionGraph = makeAll(); else sectionGraph = makeSpecific( graphDictEl ); + + /* Compute exports from the export definitions. */ + makeExports(); /* If any errors have occured in the input file then don't write anything. */ if ( gblErrorCount > 0 ) @@ -1308,17 +1408,37 @@ void ParseData::prepareMachineGen( GraphDictEl *graphDictEl ) /* Depends on the graph analysis. */ setLongestMatchData( sectionGraph ); + + /* Decide if an error state is necessary. + * 1. There is an error transition + * 2. There is a gap in the transitions + * 3. The longest match operator requires it. */ + if ( lmRequiresErrorState || sectionGraph->hasErrorTrans() ) + sectionGraph->errState = sectionGraph->addState(); + + /* State numbers need to be assigned such that all final states have a + * larger state id number than all non-final states. This enables the + * first_final mechanism to function correctly. We also want states to be + * ordered in a predictable fashion. So we first apply a depth-first + * search, then do a stable sort by final state status, then assign + * numbers. */ + + sectionGraph->depthFirstOrdering(); + sectionGraph->sortStatesByFinal(); + sectionGraph->setStateNumbers( 0 ); } -void ParseData::generateXML( ostream &out ) +void ParseData::generateReduced( InputData &inputData ) { beginProcessing(); + cgd = makeCodeGen( inputData.inputFileName, sectionName, *inputData.outStream ); + /* Make the generator. */ - XMLCodeGen codeGen( sectionName, this, sectionGraph, out ); + BackendGen backendGen( sectionName, this, sectionGraph, cgd ); /* Write out with it. */ - codeGen.writeXML(); + backendGen.makeBackend(); if ( printStatistics ) { cerr << "fsm name : " << sectionName << endl; @@ -1327,106 +1447,20 @@ void ParseData::generateXML( ostream &out ) } } -/* Send eof to all parsers. */ -void terminateAllParsers( ) -{ - /* FIXME: a proper token is needed here. Suppose we should use the - * location of EOF in the last file that the parser was referenced in. */ - InputLoc loc; - loc.fileName = "<EOF>"; - loc.line = 0; - loc.col = 0; - for ( ParserDict::Iter pdel = parserDict; pdel.lte(); pdel++ ) - pdel->value->token( loc, _eof, 0, 0 ); -} - -void checkMachines( ) -{ - for ( ParserDict::Iter parser = parserDict; parser.lte(); parser++ ) { - ParseData *pd = parser->value->pd; - if ( pd->instanceList.length() > 0 ) { - /* There must be a main graph defined. */ - /* No machine name. Need to have a main. Make sure it was given. */ - GraphDictEl *mainEl = pd->graphDict.find( machineMain ); - if ( mainEl == 0 ) { - error(pd->sectionLoc) << "main graph not defined in \"" << - pd->sectionName << "\"" << endl; - } - } - } -} - -void writeLanguage( std::ostream &out ) +void ParseData::generateXML( ostream &out ) { - out << " lang=\""; - switch ( hostLangType ) { - case CCode: out << "C"; break; - case DCode: out << "D"; break; - case JavaCode: out << "Java"; break; - } - out << "\""; - -} + beginProcessing(); -void writeMachines( std::ostream &out, std::string hostData, char *inputFileName ) -{ - if ( machineSpec == 0 && machineName == 0 ) { - /* No machine spec or machine name given. Generate everything. */ - for ( ParserDict::Iter parser = parserDict; parser.lte(); parser++ ) { - ParseData *pd = parser->value->pd; - if ( pd->instanceList.length() > 0 ) - pd->prepareMachineGen( 0 ); - } + /* Make the generator. */ + XMLCodeGen codeGen( sectionName, this, sectionGraph, out ); - if ( gblErrorCount == 0 ) { - out << "<ragel filename=\"" << inputFileName << "\""; - writeLanguage( out ); - out << ">\n"; - for ( ParserDict::Iter parser = parserDict; parser.lte(); parser++ ) { - ParseData *pd = parser->value->pd; - if ( pd->instanceList.length() > 0 ) - pd->generateXML( out ); - } - out << hostData; - out << "</ragel>\n"; - } - } - else if ( parserDict.length() > 0 ) { - /* There is either a machine spec or machine name given. */ - ParseData *parseData = 0; - GraphDictEl *graphDictEl = 0; - - /* Traverse the sections, break out when we find a section/machine - * that matches the one specified. */ - for ( ParserDict::Iter parser = parserDict; parser.lte(); parser++ ) { - ParseData *checkPd = parser->value->pd; - if ( machineSpec == 0 || strcmp( checkPd->sectionName, machineSpec ) == 0 ) { - GraphDictEl *checkGdEl = 0; - if ( machineName == 0 || (checkGdEl = - checkPd->graphDict.find( machineName )) != 0 ) - { - /* Have a machine spec and/or machine name that matches - * the -M/-S options. */ - parseData = checkPd; - graphDictEl = checkGdEl; - break; - } - } - } + /* Write out with it. */ + codeGen.writeXML(); - if ( parseData == 0 ) - error() << "could not locate machine specified with -S and/or -M" << endl; - else { - /* Section/Machine to emit was found. Prepare and emit it. */ - parseData->prepareMachineGen( graphDictEl ); - if ( gblErrorCount == 0 ) { - out << "<ragel filename=\"" << inputFileName << "\""; - writeLanguage( out ); - out << ">\n"; - parseData->generateXML( out ); - out << hostData; - out << "</ragel>\n"; - } - } + if ( printStatistics ) { + cerr << "fsm name : " << sectionName << endl; + cerr << "num states: " << sectionGraph->stateList.length() << endl; + cerr << endl; } } + diff --git a/ragel/parsedata.h b/ragel/parsedata.h index a856257..3e3bfa6 100644 --- a/ragel/parsedata.h +++ b/ragel/parsedata.h @@ -1,5 +1,5 @@ /* - * Copyright 2001-2006 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001-2008 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -24,6 +24,7 @@ #include <iostream> #include <limits.h> +#include <sstream> #include "avlmap.h" #include "bstmap.h" #include "vector.h" @@ -37,77 +38,6 @@ /* Forwards. */ using std::ostream; -/* Nodes in the tree that use this action. */ -typedef Vector<NameInst*> ActionRefs; - -/* Element in list of actions. Contains the string for the code to exectute. */ -struct Action -: - public DListEl<Action>, - public AvlTreeEl<Action> -{ -public: - - Action( const InputLoc &loc, char *name, InlineList *inlineList ) - : - loc(loc), - name(name), - inlineList(inlineList), - actionId(-1), - numTransRefs(0), - numToStateRefs(0), - numFromStateRefs(0), - numEofRefs(0), - numCondRefs(0), - anyCall(false), - isLmAction(false) - { - } - - /* Key for action dictionary. */ - char *getKey() const { return name; } - - /* Data collected during parse. */ - InputLoc loc; - char *name; - InlineList *inlineList; - int actionId; - - void actionName( ostream &out ) - { - if ( name != 0 ) - out << name; - else - out << loc.line << ":" << loc.col; - } - - /* Places in the input text that reference the action. */ - ActionRefs actionRefs; - - /* Number of references in the final machine. */ - bool numRefs() - { return numTransRefs + numToStateRefs + numFromStateRefs + numEofRefs; } - int numTransRefs; - int numToStateRefs; - int numFromStateRefs; - int numEofRefs; - int numCondRefs; - bool anyCall; - - bool isLmAction; -}; - -/* A list of actions. */ -typedef DList<Action> ActionList; -typedef AvlTree<Action, char *, CmpStr> ActionDict; - -/* Structure for reverse action mapping. */ -struct RevActionMapEl -{ - char *name; - InputLoc location; -}; - struct VarDef; struct Join; struct Expression; @@ -124,22 +54,25 @@ struct ReItem; struct ReOrBlock; struct ReOrItem; struct LongestMatch; +struct InputData; +struct CodeGenData; typedef DList<LongestMatch> LmList; + /* Graph dictionary. */ struct GraphDictEl : public AvlTreeEl<GraphDictEl>, public DListEl<GraphDictEl> { - GraphDictEl( char *k ) + GraphDictEl( const char *k ) : key(k), value(0), isInstance(false) { } - GraphDictEl( char *k, VarDef *value ) + GraphDictEl( const char *k, VarDef *value ) : key(k), value(value), isInstance(false) { } const char *getKey() { return key; } - char *key; + const char *key; VarDef *value; bool isInstance; @@ -147,7 +80,7 @@ struct GraphDictEl InputLoc loc; }; -typedef AvlTree<GraphDictEl, char*, CmpStr> GraphDict; +typedef AvlTree<GraphDictEl, const char*, CmpStr> GraphDict; typedef DList<GraphDictEl> GraphList; /* Priority name dictionary. */ @@ -155,19 +88,19 @@ typedef AvlMapEl<char*, int> PriorDictEl; typedef AvlMap<char*, int, CmpStr> PriorDict; /* Local error name dictionary. */ -typedef AvlMapEl<char*, int> LocalErrDictEl; -typedef AvlMap<char*, int, CmpStr> LocalErrDict; +typedef AvlMapEl<const char*, int> LocalErrDictEl; +typedef AvlMap<const char*, int, CmpStr> LocalErrDict; /* Tree of instantiated names. */ -typedef BstMapEl<char*, NameInst*> NameMapEl; -typedef BstMap<char*, NameInst*, CmpStr> NameMap; +typedef BstMapEl<const char*, NameInst*> NameMapEl; +typedef BstMap<const char*, NameInst*, CmpStr> NameMap; typedef Vector<NameInst*> NameVect; typedef BstSet<NameInst*> NameSet; /* Node in the tree of instantiated names. */ struct NameInst { - NameInst( const InputLoc &loc, NameInst *parent, char *name, int id, bool isLabel ) : + NameInst( const InputLoc &loc, NameInst *parent, const char *name, int id, bool isLabel ) : loc(loc), parent(parent), name(name), id(id), isLabel(isLabel), isLongestMatch(false), numRefs(0), numUses(0), start(0), final(0) {} @@ -177,7 +110,7 @@ struct NameInst * fully qulified names. */ NameInst *parent; - char *name; + const char *name; int id; bool isLabel; bool isLongestMatch; @@ -218,13 +151,24 @@ struct NameFrame NameInst *prevLocalScope; }; +struct LengthDef +{ + LengthDef( char *name ) + : name(name) {} + + char *name; + LengthDef *prev, *next; +}; + +typedef DList<LengthDef> LengthDefList; + /* Class to collect information about the machine during the * parse of input. */ struct ParseData { /* Create a new parse data object. This is done at the beginning of every * fsm specification. */ - ParseData( char *fileName, char *sectionName, const InputLoc §ionLoc ); + ParseData( const char *fileName, char *sectionName, const InputLoc §ionLoc ); ~ParseData(); /* @@ -233,13 +177,14 @@ struct ParseData /* Initialize a graph dict with the basic fsms. */ void initGraphDict(); - void createBuiltin( char *name, BuiltinMachine builtin ); + void createBuiltin( const char *name, BuiltinMachine builtin ); /* Make a name id in the current name instantiation scope if it is not * already there. */ - NameInst *addNameInst( const InputLoc &loc, char *data, bool isLabel ); - void makeRootName(); + NameInst *addNameInst( const InputLoc &loc, const char *data, bool isLabel ); + void makeRootNames(); void makeNameTree( GraphDictEl *gdNode ); + void makeExportsNameTree(); void fillNameIndex( NameInst *from ); void printNameTree(); @@ -248,7 +193,7 @@ struct ParseData void unsetObsoleteEntries( FsmAp *graph ); /* Resove name references in action code and epsilon transitions. */ - NameSet resolvePart( NameInst *refFrom, char *data, bool recLabelsOnly ); + NameSet resolvePart( NameInst *refFrom, const char *data, bool recLabelsOnly ); void resolveFrom( NameSet &result, NameInst *refFrom, const NameRef &nameRef, int namePos ); NameInst *resolveStateRef( const NameRef &nameRef, InputLoc &loc, Action *action ); @@ -256,8 +201,11 @@ struct ParseData void resolveActionNameRefs(); /* Set the alphabet type. If type types are not valid returns false. */ - bool setAlphType( char *s1, char *s2 ); - bool setAlphType( char *s1 ); + bool setAlphType( const InputLoc &loc, char *s1, char *s2 ); + bool setAlphType( const InputLoc &loc, char *s1 ); + + /* Override one of the variables ragel uses. */ + bool setVariable( char *var, InlineList *inlineList ); /* Unique actions. */ void removeDups( ActionTable &actionTable ); @@ -277,9 +225,12 @@ struct ParseData void analyzeAction( Action *action, InlineList *inlineList ); void analyzeGraph( FsmAp *graph ); + void makeExports(); void prepareMachineGen( GraphDictEl *graphDictEl ); + void prepareMachineGenTBWrapped( GraphDictEl *graphDictEl ); void generateXML( ostream &out ); + void generateReduced( InputData &inputData ); FsmAp *sectionGraph; bool generatingSectionSubset; @@ -308,7 +259,7 @@ struct ParseData ActionList actionList; /* The id of the next priority name and label. */ - int nextPriorKey, nextLocalErrKey, nextNameId; + int nextPriorKey, nextLocalErrKey, nextNameId, nextCondId; /* The default priority number key for a machine. This is active during * the parse of the rhs of a machine assignment. */ @@ -319,11 +270,27 @@ struct ParseData /* Alphabet type. */ HostType *userAlphType; bool alphTypeSet; + InputLoc alphTypeLoc; /* Element type and get key expression. */ InlineList *getKeyExpr; InlineList *accessExpr; - InlineList *curStateExpr; + + /* Stack management */ + InlineList *prePushExpr; + InlineList *postPopExpr; + + /* Overriding variables. */ + InlineList *pExpr; + InlineList *peExpr; + InlineList *eofExpr; + InlineList *csExpr; + InlineList *topExpr; + InlineList *stackExpr; + InlineList *actExpr; + InlineList *tokstartExpr; + InlineList *tokendExpr; + InlineList *dataExpr; /* The alphabet range. */ char *lowerNum, *upperNum; @@ -331,19 +298,20 @@ struct ParseData InputLoc rangeLowLoc, rangeHighLoc; /* The name of the file the fsm is from, and the spec name. */ - char *fileName; + const char *fileName; char *sectionName; InputLoc sectionLoc; - /* Number of errors encountered parsing the fsm spec. */ - int errorCount; - /* Counting the action and priority ordering. */ int curActionOrd; int curPriorOrd; - /* Root of the name tree. */ + /* Root of the name tree. One root is for the instantiated machines. The + * other root is for exported definitions. */ NameInst *rootName; + NameInst *exportsRootName; + + /* Name tree walking. */ NameInst *curNameInst; int curNameChild; @@ -360,6 +328,7 @@ struct ParseData void initLongestMatchData(); void setLongestMatchData( FsmAp *graph ); void initNameWalk(); + void initExportsNameWalk(); NameInst *nextNameScope() { return curNameInst->childVect[curNameChild]; } NameFrame enterNameScope( bool isLocal, int numScopes ); void popNameScope( const NameFrame &frame ); @@ -375,7 +344,7 @@ struct ParseData /* List of all longest match parse tree items. */ LmList lmList; - Action *newAction( char *name, InlineList *inlineList ); + Action *newAction( const char *name, InlineList *inlineList ); Action *initTokStart; int initTokStartOrd; @@ -397,6 +366,11 @@ struct ParseData CondData thisCondData; KeyOps thisKeyOps; + + ExportList exportList; + LengthDefList lengthDefList; + + CodeGenData *cgd; }; void afterOpMinimize( FsmAp *fsm, bool lastInSeq = true ); @@ -413,51 +387,5 @@ FsmAp *dotStarFsm( ParseData *pd ); void errorStateLabels( const NameSet &locations ); -/* Data used by the parser specific to the current file. Supports the include - * system, since a new parser is executed for each included file. */ -struct InputData -{ - InputData( char *fileName, char *includeSpec, char *includeTo ) : - pd(0), sectionName(0), defaultParseData(0), - first_line(1), first_column(1), - last_line(1), last_column(0), - fileName(fileName), includeSpec(includeSpec), - includeTo(includeTo), active(true) - {} - - /* For collecting a name references. */ - NameRef nameRef; - NameRefList nameRefList; - - /* The parse data. For each fsm spec, the parser collects things that it parses - * in data structures in here. */ - ParseData *pd; - - char *sectionName; - ParseData *defaultParseData; - - int first_line; - int first_column; - int last_line; - int last_column; - - char *fileName; - - /* If this is an included file, this contains the specification to search - * for. IncludeTo will contain the spec name that does the includng. */ - char *includeSpec; - char *includeTo; - - bool active; - InputLoc sectionLoc; -}; - -struct Parser; - -typedef AvlMap<char*, Parser *, CmpStr> ParserDict; -typedef AvlMapEl<char*, Parser *> ParserDictEl; - -extern ParserDict parserDict; - -#endif /* _PARSEDATA_H */ +#endif diff --git a/ragel/parsetree.cpp b/ragel/parsetree.cpp index 11c58fa..3245a54 100644 --- a/ragel/parsetree.cpp +++ b/ragel/parsetree.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2001-2006 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -37,28 +37,28 @@ ostream &operator<<( ostream &out, const NameInst &nameInst ); /* Convert the literal string which comes in from the scanner into an array of * characters with escapes and options interpreted. Also null terminates the * string. Though this null termination should not be relied on for - * interpreting literals in the parser because the string may contain a - * literal string with \0 */ -void Token::prepareLitString( Token &result, bool &caseInsensitive ) + * interpreting literals in the parser because the string may contain \0 */ +char *prepareLitString( const InputLoc &loc, const char *data, long length, + long &resLen, bool &caseInsensitive ) { - result.data = new char[this->length+1]; + char *resData = new char[length+1]; caseInsensitive = false; - char *src = this->data + 1; - char *end = this->data + this->length - 1; + const char *src = data + 1; + const char *end = data + length - 1; while ( *end != '\'' && *end != '\"' ) { if ( *end == 'i' ) caseInsensitive = true; else { - error( this->loc ) << "literal string '" << *end << + error( loc ) << "literal string '" << *end << "' option not supported" << endl; } end -= 1; } - char *dest = result.data; - int len = 0; + char *dest = resData; + long len = 0; while ( src != end ) { if ( *src == '\\' ) { switch ( src[1] ) { @@ -79,10 +79,11 @@ void Token::prepareLitString( Token &result, bool &caseInsensitive ) dest[len++] = *src++; } } - result.length = len; - result.data[result.length] = 0; -} + resLen = len; + resData[resLen] = 0; + return resData; +} FsmAp *VarDef::walk( ParseData *pd ) { @@ -90,7 +91,7 @@ FsmAp *VarDef::walk( ParseData *pd ) NameFrame nameFrame = pd->enterNameScope( true, 1 ); /* Recurse on the expression. */ - FsmAp *rtnVal = joinOrLm->walk( pd ); + FsmAp *rtnVal = machineDef->walk( pd ); /* Do the tranfer of local error actions. */ LocalErrDictEl *localErrDictEl = pd->localErrDict.find( name ); @@ -102,7 +103,7 @@ FsmAp *VarDef::walk( ParseData *pd ) /* If the expression below is a join operation with multiple expressions * then it just had epsilon transisions resolved. If it is a join * with only a single expression then run the epsilon op now. */ - if ( joinOrLm->type == JoinOrLm::JoinType && joinOrLm->join->exprList.length() == 1 ) + if ( machineDef->type == MachineDef::JoinType && machineDef->join->exprList.length() == 1 ) rtnVal->epsilonOp(); /* We can now unset entry points that are not longer used. */ @@ -112,7 +113,7 @@ FsmAp *VarDef::walk( ParseData *pd ) * the graph. */ if ( pd->curNameInst->numRefs > 0 ) rtnVal->setEntry( pd->curNameInst->id, rtnVal->startState ); - + /* Pop the name scope. */ pd->popNameScope( nameFrame ); return rtnVal; @@ -124,11 +125,11 @@ void VarDef::makeNameTree( const InputLoc &loc, ParseData *pd ) NameInst *prevNameInst = pd->curNameInst; pd->curNameInst = pd->addNameInst( loc, name, false ); - if ( joinOrLm->type == JoinOrLm::LongestMatchType ) + if ( machineDef->type == MachineDef::LongestMatchType ) pd->curNameInst->isLongestMatch = true; /* Recurse. */ - joinOrLm->makeNameTree( pd ); + machineDef->makeNameTree( pd ); /* The name scope ends, pop the name instantiation. */ pd->curNameInst = prevNameInst; @@ -140,7 +141,7 @@ void VarDef::resolveNameRefs( ParseData *pd ) NameFrame nameFrame = pd->enterNameScope( true, 1 ); /* Recurse. */ - joinOrLm->resolveNameRefs( pd ); + machineDef->resolveNameRefs( pd ); /* The name scope ends, pop the name instantiation. */ pd->popNameScope( nameFrame ); @@ -162,9 +163,9 @@ InputLoc LongestMatchPart::getLoc() */ Action *LongestMatch::newAction( ParseData *pd, const InputLoc &loc, - char *name, InlineList *inlineList ) + const char *name, InlineList *inlineList ) { - Action *action = new Action( loc, name, inlineList ); + Action *action = new Action( loc, name, inlineList, pd->nextCondId++ ); action->actionRefs.append( pd->curNameInst ); pd->actionList.append( action ); action->isLmAction = true; @@ -178,13 +179,15 @@ void LongestMatch::makeActions( ParseData *pd ) /* For each part create actions for setting the match type. We need * to do this so that the actions will go into the actionIndex. */ InlineList *inlineList = new InlineList; - inlineList->append( new InlineItem( lmi->getLoc(), this, lmi, InlineItem::LmSetActId ) ); + inlineList->append( new InlineItem( lmi->getLoc(), this, lmi, + InlineItem::LmSetActId ) ); char *actName = new char[50]; sprintf( actName, "store%i", lmi->longestMatchId ); lmi->setActId = newAction( pd, lmi->getLoc(), actName, inlineList ); } - /* Make actions that execute the user action and restart on the last character. */ + /* Make actions that execute the user action and restart on the last + * character. */ for ( LmPartList::Iter lmi = *longestMatchList; lmi.lte(); lmi++ ) { /* For each part create actions for setting the match type. We need * to do this so that the actions will go into the actionIndex. */ @@ -192,7 +195,7 @@ void LongestMatch::makeActions( ParseData *pd ) inlineList->append( new InlineItem( lmi->getLoc(), this, lmi, InlineItem::LmOnLast ) ); char *actName = new char[50]; - sprintf( actName, "imm%i", lmi->longestMatchId ); + sprintf( actName, "last%i", lmi->longestMatchId ); lmi->actOnLast = newAction( pd, lmi->getLoc(), actName, inlineList ); } @@ -206,7 +209,7 @@ void LongestMatch::makeActions( ParseData *pd ) inlineList->append( new InlineItem( lmi->getLoc(), this, lmi, InlineItem::LmOnNext ) ); char *actName = new char[50]; - sprintf( actName, "lagh%i", lmi->longestMatchId ); + sprintf( actName, "next%i", lmi->longestMatchId ); lmi->actOnNext = newAction( pd, lmi->getLoc(), actName, inlineList ); } @@ -226,11 +229,12 @@ void LongestMatch::makeActions( ParseData *pd ) InputLoc loc; loc.line = 1; loc.col = 1; + loc.fileName = "NONE"; /* Create the error action. */ InlineList *il6 = new InlineList; il6->append( new InlineItem( loc, this, 0, InlineItem::LmSwitch ) ); - lmActSelect = newAction( pd, loc, "lagsel", il6 ); + lmActSelect = newAction( pd, loc, "switch", il6 ); } void LongestMatch::findName( ParseData *pd ) @@ -292,13 +296,13 @@ void LongestMatch::restart( FsmAp *graph, TransAp *trans ) graph->attachTrans( fromState, graph->startState, trans ); } -void LongestMatch::runLonestMatch( ParseData *pd, FsmAp *graph ) +void LongestMatch::runLongestMatch( ParseData *pd, FsmAp *graph ) { graph->markReachableFromHereStopFinal( graph->startState ); for ( StateList::Iter ms = graph->stateList; ms.lte(); ms++ ) { - if ( ms->stateBits & SB_ISMARKED ) { + if ( ms->stateBits & STB_ISMARKED ) { ms->lmItemSet.insert( 0 ); - ms->stateBits &= ~ SB_ISMARKED; + ms->stateBits &= ~ STB_ISMARKED; } } @@ -313,16 +317,16 @@ void LongestMatch::runLonestMatch( ParseData *pd, FsmAp *graph ) StateAp *toState = trans->toState; assert( toState ); - /* Check if there are transitions out, this may be a very - * close approximation? Out transitions going nowhere? - * FIXME: Check. */ + /* Can only optimize this if there are no transitions out. + * Note there can be out transitions going nowhere with + * actions and they too must inhibit this optimization. */ if ( toState->outList.length() > 0 ) { /* Fill the item sets. */ graph->markReachableFromHereStopFinal( toState ); for ( StateList::Iter ms = graph->stateList; ms.lte(); ms++ ) { - if ( ms->stateBits & SB_ISMARKED ) { + if ( ms->stateBits & STB_ISMARKED ) { ms->lmItemSet.insert( lmAct->value ); - ms->stateBits &= ~ SB_ISMARKED; + ms->stateBits &= ~ STB_ISMARKED; } } } @@ -338,14 +342,15 @@ void LongestMatch::runLonestMatch( ParseData *pd, FsmAp *graph ) int maxItemSetLength = 0; graph->markReachableFromHereStopFinal( graph->startState ); for ( StateList::Iter ms = graph->stateList; ms.lte(); ms++ ) { - if ( ms->stateBits & SB_ISMARKED ) { + if ( ms->stateBits & STB_ISMARKED ) { if ( ms->lmItemSet.length() > maxItemSetLength ) maxItemSetLength = ms->lmItemSet.length(); - ms->stateBits &= ~ SB_ISMARKED; + ms->stateBits &= ~ STB_ISMARKED; } } /* The actions executed on starting to match a token. */ + graph->isolateStartState(); graph->startState->toStateActionTable.setAction( pd->initTokStartOrd, pd->initTokStart ); graph->startState->fromStateActionTable.setAction( pd->setTokStartOrd, pd->setTokStart ); if ( maxItemSetLength > 1 ) { @@ -373,12 +378,18 @@ void LongestMatch::runLonestMatch( ParseData *pd, FsmAp *graph ) StateAp *toState = trans->toState; assert( toState ); - /* Check if there are transitions out, this may be a very - * close approximation? Out transitions going nowhere? - * FIXME: Check. */ + /* Can only optimize this if there are no transitions out. + * Note there can be out transitions going nowhere with + * actions and they too must inhibit this optimization. */ if ( toState->outList.length() == 0 ) { /* Can execute the immediate action for the longest match - * part. Redirect the action to the start state. */ + * part. Redirect the action to the start state. + * + * NOTE: When we need to inhibit on_last due to leaving + * actions the above test suffices. If the state has out + * actions then it will fail because the out action will + * have been transferred to an error transition, which + * makes the outlist non-empty. */ trans->actionTable.setAction( lmAct->key, lmAct->value->actOnLast ); restartTrans.append( trans ); @@ -393,12 +404,12 @@ void LongestMatch::runLonestMatch( ParseData *pd, FsmAp *graph ) maxItemSetLength = 0; graph->markReachableFromHereStopFinal( toState ); for ( StateList::Iter ms = graph->stateList; ms.lte(); ms++ ) { - if ( ms->stateBits & SB_ISMARKED ) { + if ( ms->stateBits & STB_ISMARKED ) { if ( ms->lmItemSet.length() > 0 && !ms->isFinState() ) nonFinalNonEmptyItemSet = true; if ( ms->lmItemSet.length() > maxItemSetLength ) maxItemSetLength = ms->lmItemSet.length(); - ms->stateBits &= ~ SB_ISMARKED; + ms->stateBits &= ~ STB_ISMARKED; } } @@ -437,14 +448,20 @@ void LongestMatch::runLonestMatch( ParseData *pd, FsmAp *graph ) * the last character of the token was one back and restart. */ graph->setErrorTarget( st, graph->startState, &lmErrActionOrd, &st->lmItemSet[0]->actOnNext, 1 ); + st->eofActionTable.setAction( lmErrActionOrd, + st->lmItemSet[0]->actOnNext ); + st->eofTarget = graph->startState; } else { graph->setErrorTarget( st, graph->startState, &lmErrActionOrd, &st->lmItemSet[0]->actLagBehind, 1 ); + st->eofActionTable.setAction( lmErrActionOrd, + st->lmItemSet[0]->actLagBehind ); + st->eofTarget = graph->startState; } } else if ( st->lmItemSet.length() > 1 ) { - /* Need to use the select. Take note of the which items the select + /* Need to use the select. Take note of which items the select * is needed for so only the necessary actions are included. */ for ( LmItemSet::Iter plmi = st->lmItemSet; plmi.lte(); plmi++ ) { if ( *plmi != 0 ) @@ -453,6 +470,8 @@ void LongestMatch::runLonestMatch( ParseData *pd, FsmAp *graph ) /* On error, execute the action select and go to the start state. */ graph->setErrorTarget( st, graph->startState, &lmErrActionOrd, &lmActSelect, 1 ); + st->eofActionTable.setAction( lmErrActionOrd, lmActSelect ); + st->eofTarget = graph->startState; } } @@ -460,6 +479,14 @@ void LongestMatch::runLonestMatch( ParseData *pd, FsmAp *graph ) graph->setFinState( graph->startState ); } +void LongestMatch::transferScannerLeavingActions( FsmAp *graph ) +{ + for ( StateList::Iter st = graph->stateList; st.lte(); st++ ) { + if ( st->outActionTable.length() > 0 ) + graph->setErrorActions( st, st->outActionTable ); + } +} + FsmAp *LongestMatch::walk( ParseData *pd ) { /* The longest match has it's own name scope. */ @@ -474,6 +501,13 @@ FsmAp *LongestMatch::walk( ParseData *pd ) parts[i]->longMatchAction( pd->curActionOrd++, lmi ); } + /* Before we union the patterns we need to deal with leaving actions. They + * are transfered to error transitions out of the final states (like local + * error actions) and to eof actions. In the scanner we need to forbid + * on_last for any final state that has an leaving action. */ + for ( int i = 0; i < longestMatchList->length(); i++ ) + transferScannerLeavingActions( parts[i] ); + /* Union machines one and up with machine zero. The grammar dictates that * there will always be at least one part. */ FsmAp *rtnVal = parts[0]; @@ -482,7 +516,7 @@ FsmAp *LongestMatch::walk( ParseData *pd ) afterOpMinimize( rtnVal ); } - runLonestMatch( pd, rtnVal ); + runLongestMatch( pd, rtnVal ); /* Pop the name scope. */ pd->popNameScope( nameFrame ); @@ -491,7 +525,7 @@ FsmAp *LongestMatch::walk( ParseData *pd ) return rtnVal; } -FsmAp *JoinOrLm::walk( ParseData *pd ) +FsmAp *MachineDef::walk( ParseData *pd ) { FsmAp *rtnVal = 0; switch ( type ) { @@ -501,11 +535,16 @@ FsmAp *JoinOrLm::walk( ParseData *pd ) case LongestMatchType: rtnVal = longestMatch->walk( pd ); break; + case LengthDefType: + condData->lastCondKey.increment(); + rtnVal = new FsmAp(); + rtnVal->concatFsm( condData->lastCondKey ); + break; } return rtnVal; } -void JoinOrLm::makeNameTree( ParseData *pd ) +void MachineDef::makeNameTree( ParseData *pd ) { switch ( type ) { case JoinType: @@ -514,10 +553,12 @@ void JoinOrLm::makeNameTree( ParseData *pd ) case LongestMatchType: longestMatch->makeNameTree( pd ); break; + case LengthDefType: + break; } } -void JoinOrLm::resolveNameRefs( ParseData *pd ) +void MachineDef::resolveNameRefs( ParseData *pd ) { switch ( type ) { case JoinType: @@ -526,6 +567,8 @@ void JoinOrLm::resolveNameRefs( ParseData *pd ) case LongestMatchType: longestMatch->resolveNameRefs( pd ); break; + case LengthDefType: + break; } } @@ -638,7 +681,7 @@ void Join::resolveNameRefs( ParseData *pd ) pd->curNameInst->start = resolved[0]; if ( resolved.length() > 1 ) { /* Complain about the multiple references. */ - error(loc) << "multiple start labels" << endl; + error(loc) << "join operation has multiple start labels" << endl; errorStateLabels( resolved ); } } @@ -649,9 +692,8 @@ void Join::resolveNameRefs( ParseData *pd ) pd->curNameInst->start->numRefs += 1; } else { - /* No start label. Complain and recover by adding a label to the - * adding one. Recover ignoring the problem. */ - error(loc) << "no start label" << endl; + /* No start label. */ + error(loc) << "join operation has no start label" << endl; } /* Recurse into all expressions in the list. */ @@ -831,8 +873,8 @@ FsmAp *Term::walk( ParseData *pd, bool lastInSeq ) priorDescs[0].priority = 0; rtnVal->allTransPrior( pd->curPriorOrd++, &priorDescs[0] ); - /* The start transitions right machine get the higher priority. - * Use the same unique key. */ + /* The start transitions of the right machine gets the higher + * priority. Use the same unique key. */ priorDescs[1].key = priorDescs[0].key; priorDescs[1].priority = 1; rhs->startFsmPrior( pd->curPriorOrd++, &priorDescs[1] ); @@ -862,6 +904,14 @@ FsmAp *Term::walk( ParseData *pd, bool lastInSeq ) priorDescs[1].priority = 1; rhs->finishFsmPrior( pd->curPriorOrd++, &priorDescs[1] ); + /* If the right machine's start state is final we need to guard + * against the left machine persisting by moving through the empty + * string. */ + if ( rhs->startState->isFinState() ) { + rhs->startState->outPriorTable.setPrior( + pd->curPriorOrd++, &priorDescs[1] ); + } + /* Perform concatenation. */ rtnVal->concatOp( rhs ); afterOpMinimize( rtnVal, lastInSeq ); @@ -880,14 +930,14 @@ FsmAp *Term::walk( ParseData *pd, bool lastInSeq ) priorDescs[0].priority = 1; rtnVal->allTransPrior( pd->curPriorOrd++, &priorDescs[0] ); - /* The right machine gets the lower priority. Since - * startTransPrior might unnecessarily increase the number of - * states during the state machine construction process (due to - * isolation), we use allTransPrior instead, which has the same - * effect. */ + /* The right machine gets the lower priority. We cannot use + * allTransPrior here in case the start state of the right machine + * is final. It would allow the right machine thread to run along + * with the left if just passing through the start state. Using + * startFsmPrior prevents this. */ priorDescs[1].key = priorDescs[0].key; priorDescs[1].priority = 0; - rhs->allTransPrior( pd->curPriorOrd++, &priorDescs[1] ); + rhs->startFsmPrior( pd->curPriorOrd++, &priorDescs[1] ); /* Perform concatenation. */ rtnVal->concatOp( rhs ); @@ -1119,14 +1169,14 @@ void FactorWithAug::assignConditions( FsmAp *graph ) switch ( conditions[i].type ) { /* Transition actions. */ case at_start: - graph->startFsmCondition( conditions[i].action ); + graph->startFsmCondition( conditions[i].action, conditions[i].sense ); afterOpMinimize( graph ); break; case at_all: - graph->allTransCondition( conditions[i].action ); + graph->allTransCondition( conditions[i].action, conditions[i].sense ); break; case at_leave: - graph->leaveFsmCondition( conditions[i].action ); + graph->leaveFsmCondition( conditions[i].action, conditions[i].sense ); break; default: break; @@ -1172,8 +1222,10 @@ FsmAp *FactorWithAug::walk( ParseData *pd ) actionOrd[i] = pd->curActionOrd++; } + /* Embed conditions. */ assignConditions( rtnVal ); + /* Embed actions. */ assignActions( pd, rtnVal , actionOrd ); /* Make the array of priority orderings. Orderings are local to this walk @@ -1344,6 +1396,7 @@ FsmAp *FactorWithRep::walk( ParseData *pd ) if ( retFsm->startState->isFinState() ) { warning(loc) << "applying kleene star to a machine that " "accepts zero length word" << endl; + retFsm->unsetFinState( retFsm->startState ); } /* Shift over the start action orders then do the kleene star. */ @@ -1396,7 +1449,7 @@ FsmAp *FactorWithRep::walk( ParseData *pd ) retFsm = factorWithRep->walk( pd ); if ( retFsm->startState->isFinState() ) { warning(loc) << "applying plus operator to a machine that " - "accpets zero length word" << endl; + "accepts zero length word" << endl; } /* Need a duplicated for the star end. */ @@ -1700,7 +1753,7 @@ Factor::~Factor() delete reItem; break; case RegExprType: - delete regExp; + delete regExpr; break; case ReferenceType: break; @@ -1728,7 +1781,7 @@ FsmAp *Factor::walk( ParseData *pd ) rtnVal = reItem->walk( pd, 0 ); break; case RegExprType: - rtnVal = regExp->walk( pd, 0 ); + rtnVal = regExpr->walk( pd, 0 ); break; case ReferenceType: rtnVal = varDef->walk( pd ); @@ -1791,43 +1844,19 @@ Range::~Range() delete upperLit; } -bool Range::verifyRangeFsm( FsmAp *rangeEnd ) -{ - /* Must have two states. */ - if ( rangeEnd->stateList.length() != 2 ) - return false; - /* The start state cannot be final. */ - if ( rangeEnd->startState->isFinState() ) - return false; - /* There should be only one final state. */ - if ( rangeEnd->finStateSet.length() != 1 ) - return false; - /* The final state cannot have any transitions out. */ - if ( rangeEnd->finStateSet[0]->outList.length() != 0 ) - return false; - /* The start state should have only one transition out. */ - if ( rangeEnd->startState->outList.length() != 1 ) - return false; - /* The singe transition out of the start state should not be a range. */ - TransAp *startTrans = rangeEnd->startState->outList.head; - if ( startTrans->lowKey != startTrans->highKey ) - return false; - return true; -} - /* Evaluate a range. Gets the lower an upper key and makes an fsm range. */ FsmAp *Range::walk( ParseData *pd ) { /* Construct and verify the suitability of the lower end of the range. */ FsmAp *lowerFsm = lowerLit->walk( pd ); - if ( !verifyRangeFsm( lowerFsm ) ) { + if ( !lowerFsm->checkSingleCharMachine() ) { error(lowerLit->token.loc) << "bad range lower end, must be a single character" << endl; } /* Construct and verify the upper end. */ FsmAp *upperFsm = upperLit->walk( pd ); - if ( !verifyRangeFsm( upperFsm ) ) { + if ( !upperFsm->checkSingleCharMachine() ) { error(upperLit->token.loc) << "bad range upper end, must be a single character" << endl; } @@ -1868,19 +1897,20 @@ FsmAp *Literal::walk( ParseData *pd ) } case LitString: { /* Make the array of keys in int format. */ - Token interp; + long length; bool caseInsensitive; - token.prepareLitString( interp, caseInsensitive ); - Key *arr = new Key[interp.length]; - makeFsmKeyArray( arr, interp.data, interp.length, pd ); + char *data = prepareLitString( token.loc, token.data, token.length, + length, caseInsensitive ); + Key *arr = new Key[length]; + makeFsmKeyArray( arr, data, length, pd ); /* Make the new machine. */ rtnVal = new FsmAp(); if ( caseInsensitive ) - rtnVal->concatFsmCI( arr, interp.length ); + rtnVal->concatFsmCI( arr, length ); else - rtnVal->concatFsm( arr, interp.length ); - delete[] interp.data; + rtnVal->concatFsm( arr, length ); + delete[] data; delete[] arr; break; }} @@ -1892,7 +1922,7 @@ RegExpr::~RegExpr() { switch ( type ) { case RecurseItem: - delete regExp; + delete regExpr; delete item; break; case Empty: @@ -1911,18 +1941,14 @@ FsmAp *RegExpr::walk( ParseData *pd, RegExpr *rootRegex ) switch ( type ) { case RecurseItem: { /* Walk both items. */ - FsmAp *fsm1 = regExp->walk( pd, rootRegex ); + rtnVal = regExpr->walk( pd, rootRegex ); FsmAp *fsm2 = item->walk( pd, rootRegex ); - if ( fsm1 == 0 ) - rtnVal = fsm2; - else { - fsm1->concatOp( fsm2 ); - rtnVal = fsm1; - } + rtnVal->concatOp( fsm2 ); break; } case Empty: { - rtnVal = 0; + rtnVal = new FsmAp(); + rtnVal->lambdaFsm(); break; } } @@ -1972,6 +1998,10 @@ FsmAp *ReItem::walk( ParseData *pd, RegExpr *rootRegex ) case OrBlock: { /* Get the or block and minmize it. */ rtnVal = orBlock->walk( pd, rootRegex ); + if ( rtnVal == 0 ) { + rtnVal = new FsmAp(); + rtnVal->lambdaFsm(); + } rtnVal->minimizePartition2(); break; } @@ -1992,7 +2022,7 @@ FsmAp *ReItem::walk( ParseData *pd, RegExpr *rootRegex ) if ( star ) { if ( rtnVal->startState->isFinState() ) { warning(loc) << "applying kleene star to a machine that " - "accpets zero length word" << endl; + "accepts zero length word" << endl; } rtnVal->starOp(); diff --git a/ragel/parsetree.h b/ragel/parsetree.h index c340171..dd38678 100644 --- a/ragel/parsetree.h +++ b/ragel/parsetree.h @@ -1,5 +1,5 @@ /* - * Copyright 2001-2006 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -22,6 +22,7 @@ #ifndef _PARSETREE_H #define _PARSETREE_H +#include "ragel.h" #include "avlmap.h" #include "bstmap.h" #include "vector.h" @@ -50,13 +51,6 @@ enum BuiltinMachine BT_Empty }; -/* Location in an input file. */ -struct InputLoc -{ - char *fileName; - int line; - int col; -}; struct ParseData; @@ -72,11 +66,12 @@ struct FactorWithNeg; struct Factor; struct Expression; struct Join; -struct JoinOrLm; +struct MachineDef; struct LongestMatch; struct LongestMatchPart; struct LmPartList; struct Range; +struct LengthDef; /* Type of augmentation. Describes locations in the machine. */ enum AugType @@ -187,17 +182,30 @@ struct ParserAction Action *action; }; +struct ConditionTest +{ + ConditionTest( const InputLoc &loc, AugType type, Action *action, bool sense ) : + loc(loc), type(type), action(action), sense(sense) { } + + InputLoc loc; + AugType type; + Action *action; + bool sense; +}; + struct Token { char *data; int length; InputLoc loc; - void prepareLitString( Token &result, bool &caseInsensitive ); void append( const Token &other ); - void set( char *str, int len ); + void set( const char *str, int len ); }; +char *prepareLitString( const InputLoc &loc, const char *src, long length, + long &resLen, bool &caseInsensitive ); + /* Store the value and type of a priority augmentation. */ struct PriorityAug { @@ -214,16 +222,17 @@ struct PriorityAug */ struct VarDef { - VarDef( char *name, JoinOrLm *joinOrLm ) - : name(name), joinOrLm(joinOrLm) { } + VarDef( const char *name, MachineDef *machineDef ) + : name(name), machineDef(machineDef), isExport(false) { } /* Parse tree traversal. */ FsmAp *walk( ParseData *pd ); void makeNameTree( const InputLoc &loc, ParseData *pd ); void resolveNameRefs( ParseData *pd ); - char *name; - JoinOrLm *joinOrLm; + const char *name; + MachineDef *machineDef; + bool isExport; }; @@ -292,8 +301,9 @@ struct LongestMatch FsmAp *walk( ParseData *pd ); void makeNameTree( ParseData *pd ); void resolveNameRefs( ParseData *pd ); - void runLonestMatch( ParseData *pd, FsmAp *graph ); - Action *newAction( ParseData *pd, const InputLoc &loc, char *name, + void transferScannerLeavingActions( FsmAp *graph ); + void runLongestMatch( ParseData *pd, FsmAp *graph ); + Action *newAction( ParseData *pd, const InputLoc &loc, const char *name, InlineList *inlineList ); void makeActions( ParseData *pd ); void findName( ParseData *pd ); @@ -301,7 +311,7 @@ struct LongestMatch InputLoc loc; LmPartList *longestMatchList; - char *name; + const char *name; Action *lmActSelect; bool lmSwitchHandlesError; @@ -313,17 +323,20 @@ struct LongestMatch /* List of Expressions. */ typedef DList<Expression> ExprList; -struct JoinOrLm +struct MachineDef { enum Type { JoinType, - LongestMatchType + LongestMatchType, + LengthDefType }; - JoinOrLm( Join *join ) : - join(join), type(JoinType) {} - JoinOrLm( LongestMatch *longestMatch ) : - longestMatch(longestMatch), type(LongestMatchType) {} + MachineDef( Join *join ) + : join(join), longestMatch(0), lengthDef(0), type(JoinType) {} + MachineDef( LongestMatch *longestMatch ) + : join(0), longestMatch(longestMatch), lengthDef(0), type(LongestMatchType) {} + MachineDef( LengthDef *lengthDef ) + : join(0), longestMatch(0), lengthDef(lengthDef), type(LengthDefType) {} FsmAp *walk( ParseData *pd ); void makeNameTree( ParseData *pd ); @@ -331,6 +344,7 @@ struct JoinOrLm Join *join; LongestMatch *longestMatch; + LengthDef *lengthDef; Type type; }; @@ -459,7 +473,7 @@ struct FactorWithAug PriorDesc *priorDescs; Vector<Label> labels; Vector<EpsilonLink> epsilonLinks; - Vector<ParserAction> conditions; + Vector<ConditionTest> conditions; FactorWithRep *factorWithRep; }; @@ -486,8 +500,8 @@ struct FactorWithRep factorWithNeg(0), lowerRep(lowerRep), upperRep(upperRep), type(type) { } - FactorWithRep( const InputLoc &loc, FactorWithNeg *factorWithNeg ) - : loc(loc), factorWithNeg(factorWithNeg), type(FactorWithNegType) { } + FactorWithRep( FactorWithNeg *factorWithNeg ) + : factorWithNeg(factorWithNeg), type(FactorWithNegType) { } ~FactorWithRep(); @@ -518,8 +532,8 @@ struct FactorWithNeg FactorWithNeg( const InputLoc &loc, FactorWithNeg *factorWithNeg, Type type) : loc(loc), factorWithNeg(factorWithNeg), factor(0), type(type) { } - FactorWithNeg( const InputLoc &loc, Factor *factor ) : - loc(loc), factorWithNeg(0), factor(factor), type(FactorType) { } + FactorWithNeg( Factor *factor ) : + factorWithNeg(0), factor(factor), type(FactorType) { } ~FactorWithNeg(); @@ -563,8 +577,8 @@ struct Factor reItem(reItem), type(OrExprType) { } /* Construct with a regular expression. */ - Factor( RegExpr *regExp ) : - regExp(regExp), type(RegExprType) { } + Factor( RegExpr *regExpr ) : + regExpr(regExpr), type(RegExprType) { } /* Construct with a reference to a var def. */ Factor( const InputLoc &loc, VarDef *varDef ) : @@ -590,7 +604,7 @@ struct Factor Literal *literal; Range *range; ReItem *reItem; - RegExpr *regExp; + RegExpr *regExpr; VarDef *varDef; Join *join; LongestMatch *longestMatch; @@ -606,7 +620,6 @@ struct Range ~Range(); FsmAp *walk( ParseData *pd ); - bool verifyRangeFsm( FsmAp *rangeEnd ); Literal *lowerLit; Literal *upperLit; @@ -634,14 +647,14 @@ struct RegExpr /* Constructors. */ RegExpr() : type(Empty), caseInsensitive(false) { } - RegExpr(RegExpr *regExp, ReItem *item) : - regExp(regExp), item(item), + RegExpr(RegExpr *regExpr, ReItem *item) : + regExpr(regExpr), item(item), type(RecurseItem), caseInsensitive(false) { } ~RegExpr(); FsmAp *walk( ParseData *pd, RegExpr *rootRegex ); - RegExpr *regExp; + RegExpr *regExpr; ReItem *item; RegExpType type; bool caseInsensitive; @@ -756,6 +769,4 @@ struct InlineItem * ptreetypes, which should be just typedef forwards. */ struct InlineList : public DList<InlineItem> { }; - - -#endif /* _PARSETREE_H */ +#endif diff --git a/ragel/pcheck.h b/ragel/pcheck.h new file mode 100644 index 0000000..0e388f8 --- /dev/null +++ b/ragel/pcheck.h @@ -0,0 +1,48 @@ +/* + * Copyright 2001, 2002 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _PCHECK_H +#define _PCHECK_H + +class ParamCheck +{ +public: + ParamCheck( const char *paramSpec, int argc, const char **argv); + + bool check(); + + const char *paramArg; /* The argument to the parameter. */ + char parameter; /* The parameter matched. */ + enum { match, invalid, noparam } state; + + const char *argOffset; /* If we are reading params inside an + * arg this points to the offset. */ + + const char *curArg; /* Pointer to the current arg. */ + int iCurArg; /* Index to the current arg. */ + +private: + const char *paramSpec; /* Parameter spec supplied by the coder. */ + int argc; /* Arguement data from the command line. */ + const char **argv; +}; + +#endif diff --git a/ragel/ragel.h b/ragel/ragel.h index 58f8a88..2f4d6ff 100644 --- a/ragel/ragel.h +++ b/ragel/ragel.h @@ -1,5 +1,5 @@ /* - * Copyright 2001-2003 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -26,10 +26,25 @@ #include <iostream> #include <fstream> #include <string> +#include "vector.h" #include "config.h" +#include "common.h" #define PROGNAME "ragel" +/* Target output style. */ +enum CodeStyle +{ + GenTables, + GenFTables, + GenFlat, + GenFFlat, + GenGoto, + GenFGoto, + GenIpGoto, + GenSplit +}; + /* To what degree are machine minimized. */ enum MinimizeLevel { MinimizeApprox, @@ -45,36 +60,61 @@ enum MinimizeOpt { MinimizeEveryOp }; - -/* IO filenames and stream. */ -extern char *outputFileName; -extern std::istream *inStream; -extern std::ostream *outStream; +/* Target implementation */ +enum RubyImplEnum +{ + MRI, + Rubinius +}; /* Options. */ extern MinimizeLevel minimizeLevel; extern MinimizeOpt minimizeOpt; -extern char *machineSpec, *machineName; +extern const char *machineSpec, *machineName; extern bool printStatistics; +extern bool wantDupsRemoved; +extern bool generateDot; +extern bool generateXML; +extern RubyImplEnum rubyImpl; +/* Error reporting format. */ +enum ErrorFormat { + ErrorFormatGNU, + ErrorFormatMSVC, +}; + +extern ErrorFormat errorFormat; extern int gblErrorCount; -extern char machineMain[]; +extern char mainMachine[]; + +InputLoc makeInputLoc( const char *fileName, int line = 0, int col = 0 ); +std::ostream &operator<<( std::ostream &out, const InputLoc &loc ); /* Error reporting. */ -struct InputLoc; std::ostream &error(); std::ostream &error( const InputLoc &loc ); -std::ostream &warning( ); std::ostream &warning( const InputLoc &loc ); -void scan( char *fileName, std::istream &input ); -void terminateAllParsers( ); -void checkMachines( ); -void writeMachines( std::ostream &out, std::string hostData, char *inputFileName ); -void xmlEscapeHost( std::ostream &out, char *data, int len ); +struct XmlParser; + +void xmlEscapeHost( std::ostream &out, char *data, long len ); + +extern CodeStyle codeStyle; + +/* IO filenames and stream. */ +extern bool displayPrintables; +extern int gblErrorCount; + +/* Options. */ +extern int numSplitPartitions; +extern bool noLineDirectives; + +std::ostream &error(); +/* Target language and output style. */ +extern CodeStyle codeStyle; -/* Size of the include stack. */ -#define INCLUDE_STACK_SIZE 32 +extern int numSplitPartitions; +extern bool noLineDirectives; -#endif /* _RAGEL_H */ +#endif diff --git a/ragel/rbxgoto.cpp b/ragel/rbxgoto.cpp new file mode 100644 index 0000000..c54cb00 --- /dev/null +++ b/ragel/rbxgoto.cpp @@ -0,0 +1,831 @@ +/* + * Copyright 2007 Victor Hugo Borja <vic@rubyforge.org> + * 2006-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <stdio.h> +#include <string> + +#include "rbxgoto.h" +#include "ragel.h" +#include "redfsm.h" +#include "bstmap.h" +#include "gendata.h" + +using std::ostream; +using std::string; + +inline string label(string a, int i) +{ + return a + itoa(i); +} + +ostream &RbxGotoCodeGen::rbxLabel(ostream &out, string label) +{ + out << "Rubinius.asm { @labels[:_" << FSM_NAME() << "_" << label << "].set! }\n"; + return out; +} + +ostream &RbxGotoCodeGen::rbxGoto(ostream &out, string label) +{ + out << "Rubinius.asm { goto @labels[:_" << FSM_NAME() << "_" << label << "] }\n"; + return out; +} + +/* Emit the goto to take for a given transition. */ +std::ostream &RbxGotoCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + out << TABS(level); + return rbxGoto(out, label("tr",trans->id)); +} + +std::ostream &RbxGotoCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\twhen " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RbxGotoCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\twhen " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RbxGotoCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\twhen " << act->actionId << " then\n"; + ACTION( out, act, 0, true ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RbxGotoCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\twhen " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +void RbxGotoCodeGen::GOTO_HEADER( RedStateAp *state ) +{ + /* Label the state. */ + out << "when " << state->id << " then\n"; +} + + +void RbxGotoCodeGen::emitSingleSwitch( RedStateAp *state ) +{ + /* Load up the singles. */ + int numSingles = state->outSingle.length(); + RedTransEl *data = state->outSingle.data; + + if ( numSingles == 1 ) { + /* If there is a single single key then write it out as an if. */ + out << "\tif " << GET_WIDE_KEY(state) << " == " << + KEY(data[0].lowKey) << " \n\t\t"; + + /* Virtual function for writing the target of the transition. */ + TRANS_GOTO(data[0].value, 0) << "\n"; + + out << "end\n"; + } + else if ( numSingles > 1 ) { + /* Write out single keys in a switch if there is more than one. */ + out << "\tcase " << GET_WIDE_KEY(state) << "\n"; + + /* Write out the single indicies. */ + for ( int j = 0; j < numSingles; j++ ) { + out << "\t\twhen " << KEY(data[j].lowKey) << " then\n"; + TRANS_GOTO(data[j].value, 0) << "\n"; + } + + /* Close off the transition switch. */ + out << "\tend\n"; + } +} + +void RbxGotoCodeGen::emitRangeBSearch( RedStateAp *state, int level, int low, int high ) +{ + /* Get the mid position, staying on the lower end of the range. */ + int mid = (low + high) >> 1; + RedTransEl *data = state->outRange.data; + + /* Determine if we need to look higher or lower. */ + bool anyLower = mid > low; + bool anyHigher = mid < high; + + /* Determine if the keys at mid are the limits of the alphabet. */ + bool limitLow = data[mid].lowKey == keyOps->minKey; + bool limitHigh = data[mid].highKey == keyOps->maxKey; + + if ( anyLower && anyHigher ) { + /* Can go lower and higher than mid. */ + out << TABS(level) << "if " << GET_WIDE_KEY(state) << " < " << + KEY(data[mid].lowKey) << " \n"; + emitRangeBSearch( state, level+1, low, mid-1 ); + out << TABS(level) << "elsif " << GET_WIDE_KEY(state) << " > " << + KEY(data[mid].highKey) << " \n"; + emitRangeBSearch( state, level+1, mid+1, high ); + out << TABS(level) << "else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + out << TABS(level) << "end\n"; + } + else if ( anyLower && !anyHigher ) { + /* Can go lower than mid but not higher. */ + out << TABS(level) << "if " << GET_WIDE_KEY(state) << " < " << + KEY(data[mid].lowKey) << " then\n"; + emitRangeBSearch( state, level+1, low, mid-1 ); + + /* if the higher is the highest in the alphabet then there is no + * sense testing it. */ + if ( limitHigh ) { + out << TABS(level) << "else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + out << TABS(level) << "elsif" << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + out << TABS(level) << "end\n"; + } + else if ( !anyLower && anyHigher ) { + /* Can go higher than mid but not lower. */ + out << TABS(level) << "if " << GET_WIDE_KEY(state) << " > " << + KEY(data[mid].highKey) << " \n"; + emitRangeBSearch( state, level+1, mid+1, high ); + + /* If the lower end is the lowest in the alphabet then there is no + * sense testing it. */ + if ( limitLow ) { + out << TABS(level) << "else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + out << TABS(level) << "elsif " << GET_WIDE_KEY(state) << " >= " << + KEY(data[mid].lowKey) << " then\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + out << TABS(level) << "end\n"; + } + else { + /* Cannot go higher or lower than mid. It's mid or bust. What + * tests to do depends on limits of alphabet. */ + if ( !limitLow && !limitHigh ) { + out << TABS(level) << "if " << KEY(data[mid].lowKey) << " <= " << + GET_WIDE_KEY(state) << " && " << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " \n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + out << TABS(level) << "end\n"; + } + else if ( limitLow && !limitHigh ) { + out << TABS(level) << "if " << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " \n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + out << TABS(level) << "end\n"; + } + else if ( !limitLow && limitHigh ) { + out << TABS(level) << "if " << KEY(data[mid].lowKey) << " <= " << + GET_WIDE_KEY(state) << " \n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + out << TABS(level) << "end\n"; + } + else { + /* Both high and low are at the limit. No tests to do. */ + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + } +} + +void RbxGotoCodeGen::STATE_GOTO_ERROR() +{ + /* Label the state and bail immediately. */ + outLabelUsed = true; + RedStateAp *state = redFsm->errState; + out << "when " << state->id << " then\n"; + rbxGoto(out << " ", "_out") << "\n"; +} + +void RbxGotoCodeGen::COND_TRANSLATE( GenStateCond *stateCond, int level ) +{ + GenCondSpace *condSpace = stateCond->condSpace; + out << TABS(level) << "_widec = " << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << ");\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(level) << "if "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << "\n _widec += " << condValOffset << ";\n end"; + } +} + +void RbxGotoCodeGen::emitCondBSearch( RedStateAp *state, int level, int low, int high ) +{ + /* Get the mid position, staying on the lower end of the range. */ + int mid = (low + high) >> 1; + GenStateCond **data = state->stateCondVect.data; + + /* Determine if we need to look higher or lower. */ + bool anyLower = mid > low; + bool anyHigher = mid < high; + + /* Determine if the keys at mid are the limits of the alphabet. */ + bool limitLow = data[mid]->lowKey == keyOps->minKey; + bool limitHigh = data[mid]->highKey == keyOps->maxKey; + + if ( anyLower && anyHigher ) { + /* Can go lower and higher than mid. */ + out << TABS(level) << "if " << GET_KEY() << " < " << + KEY(data[mid]->lowKey) << " \n"; + emitCondBSearch( state, level+1, low, mid-1 ); + out << TABS(level) << "elsif " << GET_KEY() << " > " << + KEY(data[mid]->highKey) << " \n"; + emitCondBSearch( state, level+1, mid+1, high ); + out << TABS(level) << "else\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "end\n"; + } + else if ( anyLower && !anyHigher ) { + /* Can go lower than mid but not higher. */ + out << TABS(level) << "if " << GET_KEY() << " < " << + KEY(data[mid]->lowKey) << " \n"; + emitCondBSearch( state, level+1, low, mid-1 ); + + /* if the higher is the highest in the alphabet then there is no + * sense testing it. */ + if ( limitHigh ) { + out << TABS(level) << "else\n"; + COND_TRANSLATE(data[mid], level+1); + } + else { + out << TABS(level) << "elsif " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " then\n"; + COND_TRANSLATE(data[mid], level+1); + } + out << TABS(level) << "end\n"; + + } + else if ( !anyLower && anyHigher ) { + /* Can go higher than mid but not lower. */ + out << TABS(level) << "if " << GET_KEY() << " > " << + KEY(data[mid]->highKey) << " \n"; + emitCondBSearch( state, level+1, mid+1, high ); + + /* If the lower end is the lowest in the alphabet then there is no + * sense testing it. */ + if ( limitLow ) { + out << TABS(level) << "else\n"; + COND_TRANSLATE(data[mid], level+1); + } + else { + out << TABS(level) << "elsif " << GET_KEY() << " >= " << + KEY(data[mid]->lowKey) << " then\n"; + COND_TRANSLATE(data[mid], level+1); + } + out << TABS(level) << "end\n"; + } + else { + /* Cannot go higher or lower than mid. It's mid or bust. What + * tests to do depends on limits of alphabet. */ + if ( !limitLow && !limitHigh ) { + out << TABS(level) << "if " << KEY(data[mid]->lowKey) << " <= " << + GET_KEY() << " && " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " then\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "end\n"; + } + else if ( limitLow && !limitHigh ) { + out << TABS(level) << "if " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " then\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "end\n"; + } + else if ( !limitLow && limitHigh ) { + out << TABS(level) << "if " << KEY(data[mid]->lowKey) << " <= " << + GET_KEY() << " then\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "end\n"; + } + else { + /* Both high and low are at the limit. No tests to do. */ + COND_TRANSLATE(data[mid], level); + } + } +} + +std::ostream &RbxGotoCodeGen::STATE_GOTOS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st == redFsm->errState ) + STATE_GOTO_ERROR(); + else { + /* Writing code above state gotos. */ + GOTO_HEADER( st ); + + if ( st->stateCondVect.length() > 0 ) { + out << " _widec = " << GET_KEY() << ";\n"; + emitCondBSearch( st, 1, 0, st->stateCondVect.length() - 1 ); + } + + /* Try singles. */ + if ( st->outSingle.length() > 0 ) + emitSingleSwitch( st ); + + /* Default case is to binary search for the ranges, if that fails then */ + if ( st->outRange.length() > 0 ) + emitRangeBSearch( st, 1, 0, st->outRange.length() - 1 ); + + /* Write the default transition. */ + TRANS_GOTO( st->defTrans, 1 ) << "\n"; + } + } + return out; +} + +std::ostream &RbxGotoCodeGen::TRANSITIONS() +{ + /* Emit any transitions that have functions and that go to + * this state. */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* Write the label for the transition so it can be jumped to. */ + rbxLabel(out << " ", label("tr", trans->id)) << "\n"; + + /* Destination state. */ + if ( trans->action != 0 && trans->action->anyCurStateRef() ) + out << "_ps = " << vCS() << "'n"; + out << vCS() << " = " << trans->targ->id << "\n"; + + if ( trans->action != 0 ) { + /* Write out the transition func. */ + rbxGoto(out, label("f", trans->action->actListId)) << "\n"; + } + else { + /* No code to execute, just loop around. */ + rbxGoto(out, "_again") << "\n"; + } + } + return out; +} + +std::ostream &RbxGotoCodeGen::EXEC_FUNCS() +{ + /* Make labels that set acts and jump to execFuncs. Loop func indicies. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + rbxLabel(out, label("f", redAct->actListId)) << "\n" << + "_acts = " << itoa( redAct->location+1 ) << "\n"; + rbxGoto(out, "execFuncs") << "\n"; + } + } + + rbxLabel(out, "execFuncs") << + "\n" + " _nacts = " << A() << "[_acts]\n" + " _acts += 1\n" + " while ( _nacts > 0 ) \n" + " _nacts -= 1\n" + " _acts += 1\n" + " case ( "<< A() << "[_acts-1] ) \n"; + ACTION_SWITCH(); + out << + " end\n" + " end \n"; + rbxGoto(out, "_again"); + return out; +} + +int RbxGotoCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + return act; +} + +int RbxGotoCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + return act; +} + +int RbxGotoCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + return act; +} + +std::ostream &RbxGotoCodeGen::TO_STATE_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = TO_STATE_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &RbxGotoCodeGen::FROM_STATE_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = FROM_STATE_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &RbxGotoCodeGen::EOF_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = EOF_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &RbxGotoCodeGen::FINISH_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* States that are final and have an out action need a case. */ + if ( st->eofAction != 0 ) { + /* Write the case label. */ + out << "\t\twhen " << st->id << " then\n"; + + /* Write the goto func. */ + rbxGoto(out, label("f", st->eofAction->actListId)) << "\n"; + } + } + + return out; +} + +void RbxGotoCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "begin\n" << vCS() << " = " << gotoDest << " "; + rbxGoto(ret, "_again") << + "\nend\n"; +} + +void RbxGotoCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "begin\n" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ")"; + rbxGoto(ret, "_again") << + "\nend\n"; +} + +void RbxGotoCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void RbxGotoCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void RbxGotoCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void RbxGotoCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + +void RbxGotoCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "begin\n" + << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = " << + callDest << "; "; + rbxGoto(ret, "_again") << + "\nend\n"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void RbxGotoCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "begin\n" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish ); + ret << "); "; + rbxGoto(ret, "_again") << + "\nend\n"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void RbxGotoCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "begin\n" << vCS() << " = " << STACK() << "[--" << TOP() << "]; " ; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false ); + ret << "}"; + } + + rbxGoto(ret, "_again") << + "\nend\n"; +} + +void RbxGotoCodeGen::BREAK( ostream &ret, int targState ) +{ + outLabelUsed = true; + + out << + " begin\n" + " " << P() << " += 1\n" + " " << rbxGoto(ret, "_out") << "\n" + " end\n"; +} + +void RbxGotoCodeGen::writeData() +{ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void RbxGotoCodeGen::writeExec() +{ + outLabelUsed = false; + + out << " begin\n"; + + out << " Rubinius.asm { @labels = Hash.new { |h,k| h[k] = new_label } }\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = 0;\n"; + + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + { + out << " _acts, _nacts = nil\n"; + } + + if ( redFsm->anyConditions() ) + out << " _widec = nil\n"; + + out << "\n"; + + if ( !noEnd ) { + outLabelUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n"; + rbxGoto(out << " ", "_out") << "\n" << + " end\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n"; + rbxGoto(out << " ", "_out") << "\n" << + " end\n"; + } + + rbxLabel(out, "_resume") << "\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + + " _acts = " << ARR_OFF( A(), FSA() + "[" + vCS() + "]" ) << ";\n" + " _nacts = " << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + FROM_STATE_ACTION_SWITCH(); + out << + " }\n" + " }\n" + "\n"; + } + + out << + " case ( " << vCS() << " )\n"; + STATE_GOTOS(); + out << + " end # case\n" + "\n"; + TRANSITIONS() << + "\n"; + + if ( redFsm->anyRegActions() ) + EXEC_FUNCS() << "\n"; + + + rbxLabel(out, "_again") << "\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << ARR_OFF( A(), TSA() + "[" + vCS() + "]" ) << ";\n" + " _nacts = " << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + TO_STATE_ACTION_SWITCH(); + out << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n"; + rbxGoto(out << " ", "_out") << "\n" << + " end" << "\n"; + } + + if ( !noEnd ) { + out << " " << P() << " += 1\n" + " if ( " << P() << " != " << PE() << " )\n"; + rbxGoto(out << " ", "_resume") << "\n" << + " end" << "\n"; + } + else { + out << + " " << P() << " += 1;\n"; + rbxGoto(out << " ", "_resume") << "\n"; + } + + if ( outLabelUsed ) + rbxLabel(out, "_out") << "\n"; + + out << " end\n"; +} + +void RbxGotoCodeGen::writeEOF() +{ + if ( redFsm->anyEofActions() ) { + out << + " {\n" + " _acts = " << + ARR_OFF( A(), EA() + "[" + vCS() + "]" ) << ";\n" + " " << " _nacts = " << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + EOF_ACTION_SWITCH(); + out << + " }\n" + " }\n" + " }\n" + "\n"; + } +} + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ diff --git a/ragel/rbxgoto.h b/ragel/rbxgoto.h new file mode 100644 index 0000000..291b656 --- /dev/null +++ b/ragel/rbxgoto.h @@ -0,0 +1,97 @@ +/* + * Copyright 2007 Victor Hugo Borja <vic@rubyforge.org> + * 2006-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RBX_GOTOCODEGEN_H +#define _RBX_GOTOCODEGEN_H + +#include <iostream> +#include <string> +#include "rubycodegen.h" + +using std::string; + +class RbxGotoCodeGen : public RubyCodeGen +{ +public: + RbxGotoCodeGen( ostream &out ) : RubyCodeGen(out) {} + virtual ~RbxGotoCodeGen() {} + + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + std::ostream &STATE_GOTOS(); + std::ostream &TRANSITIONS(); + std::ostream &EXEC_FUNCS(); + std::ostream &FINISH_CASES(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState ); + + int TO_STATE_ACTION( RedStateAp *state ); + int FROM_STATE_ACTION( RedStateAp *state ); + int EOF_ACTION( RedStateAp *state ); + + void COND_TRANSLATE( GenStateCond *stateCond, int level ); + void emitCondBSearch( RedStateAp *state, int level, int low, int high ); + void STATE_CONDS( RedStateAp *state, bool genDefault ); + + virtual std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + + void emitSingleSwitch( RedStateAp *state ); + void emitRangeBSearch( RedStateAp *state, int level, int low, int high ); + + /* Called from STATE_GOTOS just before writing the gotos */ + virtual void GOTO_HEADER( RedStateAp *state ); + virtual void STATE_GOTO_ERROR(); + + virtual void writeData(); + virtual void writeEOF(); + virtual void writeExec(); + + + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + +private: + ostream &rbxGoto(ostream &out, string label); + ostream &rbxLabel(ostream &out, string label); +}; + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ + +#endif diff --git a/ragel/redfsm.cpp b/ragel/redfsm.cpp new file mode 100644 index 0000000..9a58752 --- /dev/null +++ b/ragel/redfsm.cpp @@ -0,0 +1,583 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "redfsm.h" +#include "avlmap.h" +#include "mergesort.h" +#include <iostream> +#include <sstream> + +using std::ostringstream; + +string GenAction::nameOrLoc() +{ + if ( name != 0 ) + return string(name); + else { + ostringstream ret; + ret << loc.line << ":" << loc.col; + return ret.str(); + } +} + +RedFsmAp::RedFsmAp() +: + forcedErrorState(false), + nextActionId(0), + nextTransId(0), + startState(0), + errState(0), + errTrans(0), + firstFinState(0), + numFinStates(0), + bAnyToStateActions(false), + bAnyFromStateActions(false), + bAnyRegActions(false), + bAnyEofActions(false), + bAnyEofTrans(false), + bAnyActionGotos(false), + bAnyActionCalls(false), + bAnyActionRets(false), + bAnyRegActionRets(false), + bAnyRegActionByValControl(false), + bAnyRegNextStmt(false), + bAnyRegCurStateRef(false), + bAnyRegBreak(false), + bAnyConditions(false) +{ +} + +/* Does the machine have any actions. */ +bool RedFsmAp::anyActions() +{ + return actionMap.length() > 0; +} + +void RedFsmAp::depthFirstOrdering( RedStateAp *state ) +{ + /* Nothing to do if the state is already on the list. */ + if ( state->onStateList ) + return; + + /* Doing depth first, put state on the list. */ + state->onStateList = true; + stateList.append( state ); + + /* At this point transitions should only be in ranges. */ + assert( state->outSingle.length() == 0 ); + assert( state->defTrans == 0 ); + + /* Recurse on everything ranges. */ + for ( RedTransList::Iter rtel = state->outRange; rtel.lte(); rtel++ ) { + if ( rtel->value->targ != 0 ) + depthFirstOrdering( rtel->value->targ ); + } +} + +/* Ordering states by transition connections. */ +void RedFsmAp::depthFirstOrdering() +{ + /* Init on state list flags. */ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) + st->onStateList = false; + + /* Clear out the state list, we will rebuild it. */ + int stateListLen = stateList.length(); + stateList.abandon(); + + /* Add back to the state list from the start state and all other entry + * points. */ + if ( startState != 0 ) + depthFirstOrdering( startState ); + for ( RedStateSet::Iter en = entryPoints; en.lte(); en++ ) + depthFirstOrdering( *en ); + if ( forcedErrorState ) + depthFirstOrdering( errState ); + + /* Make sure we put everything back on. */ + assert( stateListLen == stateList.length() ); +} + +/* Assign state ids by appearance in the state list. */ +void RedFsmAp::sequentialStateIds() +{ + /* Table based machines depend on the state numbers starting at zero. */ + nextStateId = 0; + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) + st->id = nextStateId++; +} + +/* Stable sort the states by final state status. */ +void RedFsmAp::sortStatesByFinal() +{ + /* Move forward through the list and throw final states onto the end. */ + RedStateAp *state = 0; + RedStateAp *next = stateList.head; + RedStateAp *last = stateList.tail; + while ( state != last ) { + /* Move forward and load up the next. */ + state = next; + next = state->next; + + /* Throw to the end? */ + if ( state->isFinal ) { + stateList.detach( state ); + stateList.append( state ); + } + } +} + +/* Assign state ids by final state state status. */ +void RedFsmAp::sortStateIdsByFinal() +{ + /* Table based machines depend on this starting at zero. */ + nextStateId = 0; + + /* First pass to assign non final ids. */ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + if ( ! st->isFinal ) + st->id = nextStateId++; + } + + /* Second pass to assign final ids. */ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + if ( st->isFinal ) + st->id = nextStateId++; + } +} + +struct CmpStateById +{ + static int compare( RedStateAp *st1, RedStateAp *st2 ) + { + if ( st1->id < st2->id ) + return -1; + else if ( st1->id > st2->id ) + return 1; + else + return 0; + } +}; + +void RedFsmAp::sortByStateId() +{ + /* Make the array. */ + int pos = 0; + RedStateAp **ptrList = new RedStateAp*[stateList.length()]; + for ( RedStateList::Iter st = stateList; st.lte(); st++, pos++ ) + ptrList[pos] = st; + + MergeSort<RedStateAp*, CmpStateById> mergeSort; + mergeSort.sort( ptrList, stateList.length() ); + + stateList.abandon(); + for ( int st = 0; st < pos; st++ ) + stateList.append( ptrList[st] ); + + delete[] ptrList; +} + +/* Find the final state with the lowest id. */ +void RedFsmAp::findFirstFinState() +{ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + if ( st->isFinal && (firstFinState == 0 || st->id < firstFinState->id) ) + firstFinState = st; + } +} + +void RedFsmAp::assignActionLocs() +{ + int nextLocation = 0; + for ( GenActionTableMap::Iter act = actionMap; act.lte(); act++ ) { + /* Store the loc, skip over the array and a null terminator. */ + act->location = nextLocation; + nextLocation += act->key.length() + 1; + } +} + +/* Check if we can extend the current range by displacing any ranges + * ahead to the singles. */ +bool RedFsmAp::canExtend( const RedTransList &list, int pos ) +{ + /* Get the transition that we want to extend. */ + RedTransAp *extendTrans = list[pos].value; + + /* Look ahead in the transition list. */ + for ( int next = pos + 1; next < list.length(); pos++, next++ ) { + /* If they are not continuous then cannot extend. */ + Key nextKey = list[next].lowKey; + nextKey.decrement(); + if ( list[pos].highKey != nextKey ) + break; + + /* Check for the extenstion property. */ + if ( extendTrans == list[next].value ) + return true; + + /* If the span of the next element is more than one, then don't keep + * checking, it won't be moved to single. */ + unsigned long long nextSpan = keyOps->span( list[next].lowKey, list[next].highKey ); + if ( nextSpan > 1 ) + break; + } + return false; +} + +/* Move ranges to the singles list. */ +void RedFsmAp::moveTransToSingle( RedStateAp *state ) +{ + RedTransList &range = state->outRange; + RedTransList &single = state->outSingle; + for ( int rpos = 0; rpos < range.length(); ) { + /* Check if this is a range we can extend. */ + if ( canExtend( range, rpos ) ) { + /* Transfer singles over. */ + while ( range[rpos].value != range[rpos+1].value ) { + /* Transfer the range to single. */ + single.append( range[rpos+1] ); + range.remove( rpos+1 ); + } + + /* Extend. */ + range[rpos].highKey = range[rpos+1].highKey; + range.remove( rpos+1 ); + } + /* Maybe move it to the singles. */ + else if ( keyOps->span( range[rpos].lowKey, range[rpos].highKey ) == 1 ) { + single.append( range[rpos] ); + range.remove( rpos ); + } + else { + /* Keeping it in the ranges. */ + rpos += 1; + } + } +} + +/* Look through ranges and choose suitable single character transitions. */ +void RedFsmAp::chooseSingle() +{ + /* Loop the states. */ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + /* Rewrite the transition list taking out the suitable single + * transtions. */ + moveTransToSingle( st ); + } +} + +void RedFsmAp::makeFlat() +{ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + if ( st->stateCondList.length() == 0 ) { + st->condLowKey = 0; + st->condHighKey = 0; + } + else { + st->condLowKey = st->stateCondList.head->lowKey; + st->condHighKey = st->stateCondList.tail->highKey; + + unsigned long long span = keyOps->span( st->condLowKey, st->condHighKey ); + st->condList = new GenCondSpace*[ span ]; + memset( st->condList, 0, sizeof(GenCondSpace*)*span ); + + for ( GenStateCondList::Iter sci = st->stateCondList; sci.lte(); sci++ ) { + unsigned long long base, trSpan; + base = keyOps->span( st->condLowKey, sci->lowKey )-1; + trSpan = keyOps->span( sci->lowKey, sci->highKey ); + for ( unsigned long long pos = 0; pos < trSpan; pos++ ) + st->condList[base+pos] = sci->condSpace; + } + } + + if ( st->outRange.length() == 0 ) { + st->lowKey = st->highKey = 0; + st->transList = 0; + } + else { + st->lowKey = st->outRange[0].lowKey; + st->highKey = st->outRange[st->outRange.length()-1].highKey; + unsigned long long span = keyOps->span( st->lowKey, st->highKey ); + st->transList = new RedTransAp*[ span ]; + memset( st->transList, 0, sizeof(RedTransAp*)*span ); + + for ( RedTransList::Iter trans = st->outRange; trans.lte(); trans++ ) { + unsigned long long base, trSpan; + base = keyOps->span( st->lowKey, trans->lowKey )-1; + trSpan = keyOps->span( trans->lowKey, trans->highKey ); + for ( unsigned long long pos = 0; pos < trSpan; pos++ ) + st->transList[base+pos] = trans->value; + } + + /* Fill in the gaps with the default transition. */ + for ( unsigned long long pos = 0; pos < span; pos++ ) { + if ( st->transList[pos] == 0 ) + st->transList[pos] = st->defTrans; + } + } + } +} + + +/* A default transition has been picked, move it from the outRange to the + * default pointer. */ +void RedFsmAp::moveToDefault( RedTransAp *defTrans, RedStateAp *state ) +{ + /* Rewrite the outRange, omitting any ranges that use + * the picked default. */ + RedTransList outRange; + for ( RedTransList::Iter rtel = state->outRange; rtel.lte(); rtel++ ) { + /* If it does not take the default, copy it over. */ + if ( rtel->value != defTrans ) + outRange.append( *rtel ); + } + + /* Save off the range we just created into the state's range. */ + state->outRange.transfer( outRange ); + + /* Store the default. */ + state->defTrans = defTrans; +} + +bool RedFsmAp::alphabetCovered( RedTransList &outRange ) +{ + /* Cannot cover without any out ranges. */ + if ( outRange.length() == 0 ) + return false; + + /* If the first range doesn't start at the the lower bound then the + * alphabet is not covered. */ + RedTransList::Iter rtel = outRange; + if ( keyOps->minKey < rtel->lowKey ) + return false; + + /* Check that every range is next to the previous one. */ + rtel.increment(); + for ( ; rtel.lte(); rtel++ ) { + Key highKey = rtel[-1].highKey; + highKey.increment(); + if ( highKey != rtel->lowKey ) + return false; + } + + /* The last must extend to the upper bound. */ + RedTransEl *last = &outRange[outRange.length()-1]; + if ( last->highKey < keyOps->maxKey ) + return false; + + return true; +} + +RedTransAp *RedFsmAp::chooseDefaultSpan( RedStateAp *state ) +{ + /* Make a set of transitions from the outRange. */ + RedTransSet stateTransSet; + for ( RedTransList::Iter rtel = state->outRange; rtel.lte(); rtel++ ) + stateTransSet.insert( rtel->value ); + + /* For each transition in the find how many alphabet characters the + * transition spans. */ + unsigned long long *span = new unsigned long long[stateTransSet.length()]; + memset( span, 0, sizeof(unsigned long long) * stateTransSet.length() ); + for ( RedTransList::Iter rtel = state->outRange; rtel.lte(); rtel++ ) { + /* Lookup the transition in the set. */ + RedTransAp **inSet = stateTransSet.find( rtel->value ); + int pos = inSet - stateTransSet.data; + span[pos] += keyOps->span( rtel->lowKey, rtel->highKey ); + } + + /* Find the max span, choose it for making the default. */ + RedTransAp *maxTrans = 0; + unsigned long long maxSpan = 0; + for ( RedTransSet::Iter rtel = stateTransSet; rtel.lte(); rtel++ ) { + if ( span[rtel.pos()] > maxSpan ) { + maxSpan = span[rtel.pos()]; + maxTrans = *rtel; + } + } + + delete[] span; + return maxTrans; +} + +/* Pick default transitions from ranges for the states. */ +void RedFsmAp::chooseDefaultSpan() +{ + /* Loop the states. */ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + /* Only pick a default transition if the alphabet is covered. This + * avoids any transitions in the out range that go to error and avoids + * the need for an ERR state. */ + if ( alphabetCovered( st->outRange ) ) { + /* Pick a default transition by largest span. */ + RedTransAp *defTrans = chooseDefaultSpan( st ); + + /* Rewrite the transition list taking out the transition we picked + * as the default and store the default. */ + moveToDefault( defTrans, st ); + } + } +} + +RedTransAp *RedFsmAp::chooseDefaultGoto( RedStateAp *state ) +{ + /* Make a set of transitions from the outRange. */ + RedTransSet stateTransSet; + for ( RedTransList::Iter rtel = state->outRange; rtel.lte(); rtel++ ) { + if ( rtel->value->targ == state->next ) + return rtel->value; + } + return 0; +} + +void RedFsmAp::chooseDefaultGoto() +{ + /* Loop the states. */ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + /* Pick a default transition. */ + RedTransAp *defTrans = chooseDefaultGoto( st ); + if ( defTrans == 0 ) + defTrans = chooseDefaultSpan( st ); + + /* Rewrite the transition list taking out the transition we picked + * as the default and store the default. */ + moveToDefault( defTrans, st ); + } +} + +RedTransAp *RedFsmAp::chooseDefaultNumRanges( RedStateAp *state ) +{ + /* Make a set of transitions from the outRange. */ + RedTransSet stateTransSet; + for ( RedTransList::Iter rtel = state->outRange; rtel.lte(); rtel++ ) + stateTransSet.insert( rtel->value ); + + /* For each transition in the find how many ranges use the transition. */ + int *numRanges = new int[stateTransSet.length()]; + memset( numRanges, 0, sizeof(int) * stateTransSet.length() ); + for ( RedTransList::Iter rtel = state->outRange; rtel.lte(); rtel++ ) { + /* Lookup the transition in the set. */ + RedTransAp **inSet = stateTransSet.find( rtel->value ); + numRanges[inSet - stateTransSet.data] += 1; + } + + /* Find the max number of ranges. */ + RedTransAp *maxTrans = 0; + int maxNumRanges = 0; + for ( RedTransSet::Iter rtel = stateTransSet; rtel.lte(); rtel++ ) { + if ( numRanges[rtel.pos()] > maxNumRanges ) { + maxNumRanges = numRanges[rtel.pos()]; + maxTrans = *rtel; + } + } + + delete[] numRanges; + return maxTrans; +} + +void RedFsmAp::chooseDefaultNumRanges() +{ + /* Loop the states. */ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + /* Pick a default transition. */ + RedTransAp *defTrans = chooseDefaultNumRanges( st ); + + /* Rewrite the transition list taking out the transition we picked + * as the default and store the default. */ + moveToDefault( defTrans, st ); + } +} + +RedTransAp *RedFsmAp::getErrorTrans( ) +{ + /* If the error trans has not been made aready, make it. */ + if ( errTrans == 0 ) { + /* This insert should always succeed since no transition created by + * the user can point to the error state. */ + errTrans = new RedTransAp( getErrorState(), 0, nextTransId++ ); + RedTransAp *inRes = transSet.insert( errTrans ); + assert( inRes != 0 ); + } + return errTrans; +} + +RedStateAp *RedFsmAp::getErrorState() +{ + /* Something went wrong. An error state is needed but one was not supplied + * by the frontend. */ + assert( errState != 0 ); + return errState; +} + + +RedTransAp *RedFsmAp::allocateTrans( RedStateAp *targ, RedAction *action ) +{ + /* Create a reduced trans and look for it in the transiton set. */ + RedTransAp redTrans( targ, action, 0 ); + RedTransAp *inDict = transSet.find( &redTrans ); + if ( inDict == 0 ) { + inDict = new RedTransAp( targ, action, nextTransId++ ); + transSet.insert( inDict ); + } + return inDict; +} + +void RedFsmAp::partitionFsm( int nparts ) +{ + /* At this point the states are ordered by a depth-first traversal. We + * will allocate to partitions based on this ordering. */ + this->nParts = nparts; + int partSize = stateList.length() / nparts; + int remainder = stateList.length() % nparts; + int numInPart = partSize; + int partition = 0; + if ( remainder-- > 0 ) + numInPart += 1; + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + st->partition = partition; + + numInPart -= 1; + if ( numInPart == 0 ) { + partition += 1; + numInPart = partSize; + if ( remainder-- > 0 ) + numInPart += 1; + } + } +} + +void RedFsmAp::setInTrans() +{ + /* First pass counts the number of transitions. */ + for ( TransApSet::Iter trans = transSet; trans.lte(); trans++ ) + trans->targ->numInTrans += 1; + + /* Pass over states to allocate the needed memory. Reset the counts so we + * can use them as the current size. */ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + st->inTrans = new RedTransAp*[st->numInTrans]; + st->numInTrans = 0; + } + + /* Second pass over transitions copies pointers into the in trans list. */ + for ( TransApSet::Iter trans = transSet; trans.lte(); trans++ ) + trans->targ->inTrans[trans->targ->numInTrans++] = trans; +} diff --git a/ragel/redfsm.h b/ragel/redfsm.h new file mode 100644 index 0000000..2e7ad7c --- /dev/null +++ b/ragel/redfsm.h @@ -0,0 +1,530 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _REDFSM_H +#define _REDFSM_H + +#include <assert.h> +#include <string.h> +#include <string> +#include "config.h" +#include "common.h" +#include "vector.h" +#include "dlist.h" +#include "compare.h" +#include "bstmap.h" +#include "bstset.h" +#include "avlmap.h" +#include "avltree.h" +#include "avlbasic.h" +#include "mergesort.h" +#include "sbstmap.h" +#include "sbstset.h" +#include "sbsttable.h" + + +#define TRANS_ERR_TRANS 0 +#define STATE_ERR_STATE 0 +#define FUNC_NO_FUNC 0 + +using std::string; + +struct RedStateAp; +struct GenInlineList; +struct GenAction; + +/* + * Inline code tree + */ +struct GenInlineItem +{ + enum Type + { + Text, Goto, Call, Next, GotoExpr, CallExpr, NextExpr, Ret, + PChar, Char, Hold, Exec, Curs, Targs, Entry, + LmSwitch, LmSetActId, LmSetTokEnd, LmGetTokEnd, LmInitTokStart, + LmInitAct, LmSetTokStart, SubAction, Break + }; + + GenInlineItem( const InputLoc &loc, Type type ) : + loc(loc), data(0), targId(0), targState(0), + lmId(0), children(0), offset(0), + type(type) { } + + InputLoc loc; + char *data; + int targId; + RedStateAp *targState; + int lmId; + GenInlineList *children; + int offset; + Type type; + + GenInlineItem *prev, *next; +}; + +/* Normally this would be atypedef, but that would entail including DList from + * ptreetypes, which should be just typedef forwards. */ +struct GenInlineList : public DList<GenInlineItem> { }; + +/* Element in list of actions. Contains the string for the code to exectute. */ +struct GenAction +: + public DListEl<GenAction> +{ + GenAction( ) + : + name(0), + inlineList(0), + actionId(0), + numTransRefs(0), + numToStateRefs(0), + numFromStateRefs(0), + numEofRefs(0) + { + } + + /* Data collected during parse. */ + InputLoc loc; + const char *name; + GenInlineList *inlineList; + int actionId; + + string nameOrLoc(); + + /* Number of references in the final machine. */ + int numRefs() + { return numTransRefs + numToStateRefs + numFromStateRefs + numEofRefs; } + int numTransRefs; + int numToStateRefs; + int numFromStateRefs; + int numEofRefs; +}; + + +/* Forwards. */ +struct RedStateAp; +struct StateAp; + +/* Transistion GenAction Element. */ +typedef SBstMapEl< int, GenAction* > GenActionTableEl; + +/* Transition GenAction Table. */ +struct GenActionTable + : public SBstMap< int, GenAction*, CmpOrd<int> > +{ + void setAction( int ordering, GenAction *action ); + void setActions( int *orderings, GenAction **actions, int nActs ); + void setActions( const GenActionTable &other ); +}; + +/* Compare of a whole action table element (key & value). */ +struct CmpGenActionTableEl +{ + static int compare( const GenActionTableEl &action1, + const GenActionTableEl &action2 ) + { + if ( action1.key < action2.key ) + return -1; + else if ( action1.key > action2.key ) + return 1; + else if ( action1.value < action2.value ) + return -1; + else if ( action1.value > action2.value ) + return 1; + return 0; + } +}; + +/* Compare for GenActionTable. */ +typedef CmpSTable< GenActionTableEl, CmpGenActionTableEl > CmpGenActionTable; + +/* Set of states. */ +typedef BstSet<RedStateAp*> RedStateSet; +typedef BstSet<int> IntSet; + +/* Reduced action. */ +struct RedAction +: + public AvlTreeEl<RedAction> +{ + RedAction( ) + : + key(), + eofRefs(0), + numTransRefs(0), + numToStateRefs(0), + numFromStateRefs(0), + numEofRefs(0), + bAnyNextStmt(false), + bAnyCurStateRef(false), + bAnyBreakStmt(false) + { } + + const GenActionTable &getKey() + { return key; } + + GenActionTable key; + int actListId; + int location; + IntSet *eofRefs; + + /* Number of references in the final machine. */ + int numRefs() + { return numTransRefs + numToStateRefs + numFromStateRefs + numEofRefs; } + int numTransRefs; + int numToStateRefs; + int numFromStateRefs; + int numEofRefs; + + bool anyNextStmt() { return bAnyNextStmt; } + bool anyCurStateRef() { return bAnyCurStateRef; } + bool anyBreakStmt() { return bAnyBreakStmt; } + + bool bAnyNextStmt; + bool bAnyCurStateRef; + bool bAnyBreakStmt; +}; +typedef AvlTree<RedAction, GenActionTable, CmpGenActionTable> GenActionTableMap; + +/* Reduced transition. */ +struct RedTransAp +: + public AvlTreeEl<RedTransAp> +{ + RedTransAp( RedStateAp *targ, RedAction *action, int id ) + : targ(targ), action(action), id(id), pos(-1), labelNeeded(true) { } + + RedStateAp *targ; + RedAction *action; + int id; + int pos; + bool partitionBoundary; + bool labelNeeded; +}; + +/* Compare of transitions for the final reduction of transitions. Comparison + * is on target and the pointer to the shared action table. It is assumed that + * when this is used the action tables have been reduced. */ +struct CmpRedTransAp +{ + static int compare( const RedTransAp &t1, const RedTransAp &t2 ) + { + if ( t1.targ < t2.targ ) + return -1; + else if ( t1.targ > t2.targ ) + return 1; + else if ( t1.action < t2.action ) + return -1; + else if ( t1.action > t2.action ) + return 1; + else + return 0; + } +}; + +typedef AvlBasic<RedTransAp, CmpRedTransAp> TransApSet; + +/* Element in out range. */ +struct RedTransEl +{ + /* Constructors. */ + RedTransEl( Key lowKey, Key highKey, RedTransAp *value ) + : lowKey(lowKey), highKey(highKey), value(value) { } + + Key lowKey, highKey; + RedTransAp *value; +}; + +typedef Vector<RedTransEl> RedTransList; +typedef Vector<RedStateAp*> RedStateVect; + +typedef BstMapEl<RedStateAp*, unsigned long long> RedSpanMapEl; +typedef BstMap<RedStateAp*, unsigned long long> RedSpanMap; + +/* Compare used by span map sort. Reverse sorts by the span. */ +struct CmpRedSpanMapEl +{ + static int compare( const RedSpanMapEl &smel1, const RedSpanMapEl &smel2 ) + { + if ( smel1.value > smel2.value ) + return -1; + else if ( smel1.value < smel2.value ) + return 1; + else + return 0; + } +}; + +/* Sorting state-span map entries by span. */ +typedef MergeSort<RedSpanMapEl, CmpRedSpanMapEl> RedSpanMapSort; + +/* Set of entry ids that go into this state. */ +typedef Vector<int> EntryIdVect; +typedef Vector<char*> EntryNameVect; + +typedef Vector< GenAction* > GenCondSet; + +struct Condition +{ + Condition( ) + : key(0), baseKey(0) {} + + Key key; + Key baseKey; + GenCondSet condSet; + + Condition *next, *prev; +}; +typedef DList<Condition> ConditionList; + +struct GenCondSpace +{ + Key baseKey; + GenCondSet condSet; + int condSpaceId; + + GenCondSpace *next, *prev; +}; +typedef DList<GenCondSpace> CondSpaceList; + +struct GenStateCond +{ + Key lowKey; + Key highKey; + + GenCondSpace *condSpace; + + GenStateCond *prev, *next; +}; +typedef DList<GenStateCond> GenStateCondList; +typedef Vector<GenStateCond*> StateCondVect; + +/* Reduced state. */ +struct RedStateAp +{ + RedStateAp() + : + defTrans(0), + condList(0), + transList(0), + isFinal(false), + labelNeeded(false), + outNeeded(false), + onStateList(false), + toStateAction(0), + fromStateAction(0), + eofAction(0), + eofTrans(0), + id(0), + bAnyRegCurStateRef(false), + partitionBoundary(false), + inTrans(0), + numInTrans(0) + { } + + /* Transitions out. */ + RedTransList outSingle; + RedTransList outRange; + RedTransAp *defTrans; + + /* For flat conditions. */ + Key condLowKey, condHighKey; + GenCondSpace **condList; + + /* For flat keys. */ + Key lowKey, highKey; + RedTransAp **transList; + + /* The list of states that transitions from this state go to. */ + RedStateVect targStates; + + bool isFinal; + bool labelNeeded; + bool outNeeded; + bool onStateList; + RedAction *toStateAction; + RedAction *fromStateAction; + RedAction *eofAction; + RedTransAp *eofTrans; + int id; + GenStateCondList stateCondList; + StateCondVect stateCondVect; + + /* Pointers for the list of states. */ + RedStateAp *prev, *next; + + bool anyRegCurStateRef() { return bAnyRegCurStateRef; } + bool bAnyRegCurStateRef; + + int partition; + bool partitionBoundary; + + RedTransAp **inTrans; + int numInTrans; +}; + +/* List of states. */ +typedef DList<RedStateAp> RedStateList; + +/* Set of reduced transitons. Comparison is by pointer. */ +typedef BstSet< RedTransAp*, CmpOrd<RedTransAp*> > RedTransSet; + +/* Next version of the fsm machine. */ +struct RedFsmAp +{ + RedFsmAp(); + + bool forcedErrorState; + + int nextActionId; + int nextTransId; + + /* Next State Id doubles as the total number of state ids. */ + int nextStateId; + + TransApSet transSet; + GenActionTableMap actionMap; + RedStateList stateList; + RedStateSet entryPoints; + RedStateAp *startState; + RedStateAp *errState; + RedTransAp *errTrans; + RedTransAp *errActionTrans; + RedStateAp *firstFinState; + int numFinStates; + int nParts; + + bool bAnyToStateActions; + bool bAnyFromStateActions; + bool bAnyRegActions; + bool bAnyEofActions; + bool bAnyEofTrans; + bool bAnyActionGotos; + bool bAnyActionCalls; + bool bAnyActionRets; + bool bAnyRegActionRets; + bool bAnyRegActionByValControl; + bool bAnyRegNextStmt; + bool bAnyRegCurStateRef; + bool bAnyRegBreak; + bool bAnyConditions; + + int maxState; + int maxSingleLen; + int maxRangeLen; + int maxKeyOffset; + int maxIndexOffset; + int maxIndex; + int maxActListId; + int maxActionLoc; + int maxActArrItem; + unsigned long long maxSpan; + unsigned long long maxCondSpan; + int maxFlatIndexOffset; + Key maxKey; + int maxCondOffset; + int maxCondLen; + int maxCondSpaceId; + int maxCondIndexOffset; + int maxCond; + + bool anyActions(); + bool anyToStateActions() { return bAnyToStateActions; } + bool anyFromStateActions() { return bAnyFromStateActions; } + bool anyRegActions() { return bAnyRegActions; } + bool anyEofActions() { return bAnyEofActions; } + bool anyEofTrans() { return bAnyEofTrans; } + bool anyActionGotos() { return bAnyActionGotos; } + bool anyActionCalls() { return bAnyActionCalls; } + bool anyActionRets() { return bAnyActionRets; } + bool anyRegActionRets() { return bAnyRegActionRets; } + bool anyRegActionByValControl() { return bAnyRegActionByValControl; } + bool anyRegNextStmt() { return bAnyRegNextStmt; } + bool anyRegCurStateRef() { return bAnyRegCurStateRef; } + bool anyRegBreak() { return bAnyRegBreak; } + bool anyConditions() { return bAnyConditions; } + + + /* Is is it possible to extend a range by bumping ranges that span only + * one character to the singles array. */ + bool canExtend( const RedTransList &list, int pos ); + + /* Pick single transitions from the ranges. */ + void moveTransToSingle( RedStateAp *state ); + void chooseSingle(); + + void makeFlat(); + + /* Move a selected transition from ranges to default. */ + void moveToDefault( RedTransAp *defTrans, RedStateAp *state ); + + /* Pick a default transition by largest span. */ + RedTransAp *chooseDefaultSpan( RedStateAp *state ); + void chooseDefaultSpan(); + + /* Pick a default transition by most number of ranges. */ + RedTransAp *chooseDefaultNumRanges( RedStateAp *state ); + void chooseDefaultNumRanges(); + + /* Pick a default transition tailored towards goto driven machine. */ + RedTransAp *chooseDefaultGoto( RedStateAp *state ); + void chooseDefaultGoto(); + + /* Ordering states by transition connections. */ + void optimizeStateOrdering( RedStateAp *state ); + void optimizeStateOrdering(); + + /* Ordering states by transition connections. */ + void depthFirstOrdering( RedStateAp *state ); + void depthFirstOrdering(); + + /* Set state ids. */ + void sequentialStateIds(); + void sortStateIdsByFinal(); + + /* Arrange states in by final id. This is a stable sort. */ + void sortStatesByFinal(); + + /* Sorting states by id. */ + void sortByStateId(); + + /* Locating the first final state. This is the final state with the lowest + * id. */ + void findFirstFinState(); + + void assignActionLocs(); + + RedTransAp *getErrorTrans(); + RedStateAp *getErrorState(); + + /* Is every char in the alphabet covered? */ + bool alphabetCovered( RedTransList &outRange ); + + RedTransAp *allocateTrans( RedStateAp *targState, RedAction *actionTable ); + + void partitionFsm( int nParts ); + + void setInTrans(); +}; + + +#endif diff --git a/ragel/rlparse.cpp b/ragel/rlparse.cpp new file mode 100644 index 0000000..ec83836 --- /dev/null +++ b/ragel/rlparse.cpp @@ -0,0 +1,6235 @@ +/* Automatically generated by Kelbt from "rlparse.kl". + * + * Parts of this file are copied from Kelbt source covered by the GNU + * GPL. As a special exception, you may use the parts of this file copied + * from Kelbt source without restriction. The remainder is derived from + * "rlparse.kl" and inherits the copyright status of that file. + */ + +#line 1 "rlparse.kl" +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "rlparse.h" +#include "ragel.h" +#include <iostream> +#include <errno.h> +#include <stdlib.h> + +using std::cout; +using std::cerr; +using std::endl; + +#line 102 "rlparse.kh" +#line 105 "rlparse.kh" +#line 140 "rlparse.kh" +#line 1444 "rlparse.kl" + + +#line 48 "rlparse.cpp" +struct Parser_Lel_action_ref +{ +#line 755 "rlparse.kl" + + Action *action; + + +#line 54 "rlparse.cpp" +}; + +struct Parser_Lel_aug_type +{ +#line 546 "rlparse.kl" + + InputLoc loc; + AugType augType; + + +#line 65 "rlparse.cpp" +}; + +struct Parser_Lel_expression +{ +#line 338 "rlparse.kl" + + Expression *expression; + + +#line 75 "rlparse.cpp" +}; + +struct Parser_Lel_factor +{ +#line 975 "rlparse.kl" + + Factor *factor; + + +#line 85 "rlparse.cpp" +}; + +struct Parser_Lel_factor_rep_num +{ +#line 929 "rlparse.kl" + + int rep; + + +#line 95 "rlparse.cpp" +}; + +struct Parser_Lel_factor_with_aug +{ +#line 452 "rlparse.kl" + + FactorWithAug *factorWithAug; + + +#line 105 "rlparse.cpp" +}; + +struct Parser_Lel_factor_with_ep +{ +#line 436 "rlparse.kl" + + FactorWithAug *factorWithAug; + + +#line 115 "rlparse.cpp" +}; + +struct Parser_Lel_factor_with_label +{ +#line 420 "rlparse.kl" + + FactorWithAug *factorWithAug; + + +#line 125 "rlparse.cpp" +}; + +struct Parser_Lel_factor_with_neg +{ +#line 939 "rlparse.kl" + + FactorWithNeg *factorWithNeg; + + +#line 135 "rlparse.cpp" +}; + +struct Parser_Lel_factor_with_rep +{ +#line 868 "rlparse.kl" + + FactorWithRep *factorWithRep; + + +#line 145 "rlparse.cpp" +}; + +struct Parser_Lel_inline_item +{ +#line 1234 "rlparse.kl" + + InlineItem *inlineItem; + + +#line 155 "rlparse.cpp" +}; + +struct Parser_Lel_inline_list +{ +#line 1213 "rlparse.kl" + + InlineList *inlineList; + + +#line 165 "rlparse.cpp" +}; + +struct Parser_Lel_join +{ +#line 321 "rlparse.kl" + + Join *join; + + +#line 175 "rlparse.cpp" +}; + +struct Parser_Lel_join_or_lm +{ +#line 229 "rlparse.kl" + + MachineDef *machineDef; + + +#line 185 "rlparse.cpp" +}; + +struct Parser_Lel_lm_part_list +{ +#line 253 "rlparse.kl" + + LmPartList *lmPartList; + + +#line 195 "rlparse.cpp" +}; + +struct Parser_Lel_local_err_name +{ +#line 856 "rlparse.kl" + + int error_name; + + +#line 205 "rlparse.cpp" +}; + +struct Parser_Lel_longest_match_part +{ +#line 277 "rlparse.kl" + + LongestMatchPart *lmPart; + + +#line 215 "rlparse.cpp" +}; + +struct Parser_Lel_opt_export +{ +#line 95 "rlparse.kl" + + bool isSet; + + +#line 225 "rlparse.cpp" +}; + +struct Parser_Lel_opt_lm_part_action +{ +#line 294 "rlparse.kl" + + Action *action; + + +#line 235 "rlparse.cpp" +}; + +struct Parser_Lel_priority_aug +{ +#line 803 "rlparse.kl" + + int priorityNum; + + +#line 245 "rlparse.cpp" +}; + +struct Parser_Lel_priority_name +{ +#line 788 "rlparse.kl" + + int priorityName; + + +#line 255 "rlparse.cpp" +}; + +struct Parser_Lel_range_lit +{ +#line 1042 "rlparse.kl" + + Literal *literal; + + +#line 265 "rlparse.cpp" +}; + +struct Parser_Lel_regular_expr +{ +#line 1079 "rlparse.kl" + + RegExpr *regExpr; + + +#line 275 "rlparse.cpp" +}; + +struct Parser_Lel_regular_expr_char +{ +#line 1131 "rlparse.kl" + + ReItem *reItem; + + +#line 285 "rlparse.cpp" +}; + +struct Parser_Lel_regular_expr_item +{ +#line 1114 "rlparse.kl" + + ReItem *reItem; + + +#line 295 "rlparse.cpp" +}; + +struct Parser_Lel_regular_expr_or_char +{ +#line 1188 "rlparse.kl" + + ReOrItem *reOrItem; + + +#line 305 "rlparse.cpp" +}; + +struct Parser_Lel_regular_expr_or_data +{ +#line 1155 "rlparse.kl" + + ReOrBlock *reOrBlock; + + +#line 315 "rlparse.cpp" +}; + +struct Parser_Lel_term +{ +#line 389 "rlparse.kl" + + Term *term; + + +#line 325 "rlparse.cpp" +}; + +struct Parser_Lel_term_short +{ +#line 368 "rlparse.kl" + + Term *term; + + +#line 335 "rlparse.cpp" +}; + +struct Parser_Lel_token_type +{ +#line 146 "rlparse.kl" + + Token token; + + +#line 345 "rlparse.cpp" +}; + +union Parser_UserData +{ + struct Parser_Lel_action_ref action_ref; + struct Parser_Lel_aug_type aug_type; + struct Parser_Lel_expression expression; + struct Parser_Lel_factor factor; + struct Parser_Lel_factor_rep_num factor_rep_num; + struct Parser_Lel_factor_with_aug factor_with_aug; + struct Parser_Lel_factor_with_ep factor_with_ep; + struct Parser_Lel_factor_with_label factor_with_label; + struct Parser_Lel_factor_with_neg factor_with_neg; + struct Parser_Lel_factor_with_rep factor_with_rep; + struct Parser_Lel_inline_item inline_item; + struct Parser_Lel_inline_list inline_list; + struct Parser_Lel_join join; + struct Parser_Lel_join_or_lm join_or_lm; + struct Parser_Lel_lm_part_list lm_part_list; + struct Parser_Lel_local_err_name local_err_name; + struct Parser_Lel_longest_match_part longest_match_part; + struct Parser_Lel_opt_export opt_export; + struct Parser_Lel_opt_lm_part_action opt_lm_part_action; + struct Parser_Lel_priority_aug priority_aug; + struct Parser_Lel_priority_name priority_name; + struct Parser_Lel_range_lit range_lit; + struct Parser_Lel_regular_expr regular_expr; + struct Parser_Lel_regular_expr_char regular_expr_char; + struct Parser_Lel_regular_expr_item regular_expr_item; + struct Parser_Lel_regular_expr_or_char regular_expr_or_char; + struct Parser_Lel_regular_expr_or_data regular_expr_or_data; + struct Parser_Lel_term term; + struct Parser_Lel_term_short term_short; + struct Parser_Lel_token_type token_type; + struct Token token; +}; + +struct Parser_LangEl +{ + char *file; + int line; + int type; + int reduction; + int state; + int causeReduce; + union Parser_UserData user; + unsigned int retry; + struct Parser_LangEl *next, *child, *prev; +}; + +struct Parser_Block +{ + struct Parser_LangEl data[8128]; + struct Parser_Block *next; +}; + +#line 404 "rlparse.cpp" +unsigned int Parser_startState = 0; + +short Parser_indicies[] = { + 152, -1, -1, 152, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 152, 152, 152, 152, -1, + -1, -1, -1, -1, -1, -1, -1, 152, + 152, 152, 152, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 152, + 152, 152, 1, 0, 404, 154, -1, -1, + 154, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 154, 154, 154, 154, -1, -1, -1, -1, + -1, -1, -1, -1, 154, 154, 154, 154, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 154, 154, 150, -1, + -1, 2, 161, -1, -1, 151, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 4, 5, 6, + 7, -1, -1, -1, -1, -1, -1, -1, + -1, 158, 11, 12, 13, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 9, 8, -1, -1, -1, -1, 153, + 392, 393, 394, 395, 396, 397, 398, 399, + 400, 401, 402, 403, -1, 10, 3, 165, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 26, 14, 15, 17, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 326, 328, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 16, 364, 364, 364, -1, 364, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 364, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 364, -1, -1, -1, 364, + 364, -1, -1, -1, -1, -1, -1, -1, + -1, 364, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 364, 364, + 364, 364, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 364, 364, -1, -1, -1, 364, 364, + 364, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 19, 364, 364, 364, + -1, 364, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 364, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 364, -1, -1, + -1, 364, 364, -1, -1, -1, -1, -1, + -1, -1, -1, 364, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 364, 364, 364, 364, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 364, 364, -1, -1, -1, + 364, 364, 364, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 24, 174, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 174, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 23, 25, -1, -1, -1, -1, 159, + 20, 21, 22, 27, 168, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 28, 17, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 326, 328, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 29, 327, 376, + 377, 378, -1, 375, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 170, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 374, + -1, -1, -1, 372, 373, -1, -1, -1, + -1, -1, -1, -1, -1, 379, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 368, 369, 370, 371, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 380, 381, -1, + -1, -1, 382, 383, 30, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 365, -1, + 367, -1, 363, 366, 342, 342, 342, -1, + 342, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 342, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 342, -1, -1, 342, -1, -1, -1, + 342, 342, -1, -1, -1, -1, -1, -1, + -1, -1, 342, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 342, + 342, 342, 342, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 342, + 342, 342, 342, 342, 342, 342, 342, 342, + 342, 342, 342, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 31, 342, + 342, 342, -1, 342, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 342, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 342, -1, -1, 342, + -1, -1, -1, 342, 342, -1, -1, -1, + -1, -1, -1, -1, -1, 342, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 342, 342, 342, 342, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 342, 342, 342, 342, 342, 342, + 342, 342, 342, 342, 342, 342, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 32, 155, 33, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 173, 376, + 377, 378, -1, 375, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 171, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 374, + -1, -1, -1, 372, 373, -1, -1, -1, + -1, -1, -1, -1, -1, 379, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 368, 369, 370, 371, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 380, 381, -1, + -1, -1, 382, 383, 30, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 365, -1, + 367, -1, 363, 366, 154, -1, -1, 154, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 154, 154, 154, 154, -1, -1, + -1, -1, -1, -1, -1, -1, 154, 154, + 154, 154, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 154, 154, + -1, -1, -1, 34, 35, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 36, 342, 342, 342, + -1, 342, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 342, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 342, -1, -1, 342, -1, -1, + -1, 342, 342, -1, -1, -1, -1, -1, + -1, -1, -1, 342, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 342, 342, 342, 342, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 342, 342, 342, 342, 342, 342, 342, 342, + 342, 342, 342, 342, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 37, + 167, 169, 38, 348, 349, 350, -1, 346, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 347, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 156, -1, -1, 374, -1, -1, -1, 372, + 373, -1, -1, -1, -1, -1, -1, -1, + -1, 351, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 368, 369, + 370, 371, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 45, 40, + 39, 380, 381, 41, 43, 44, 382, 383, + 30, 42, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 341, 345, 343, 344, 352, 348, 349, 350, + -1, 346, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 347, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 157, -1, -1, 374, -1, -1, + -1, 372, 373, -1, -1, -1, -1, -1, + -1, -1, -1, 351, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 368, 369, 370, 371, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 45, 40, 39, 380, 381, 41, 43, 44, + 382, 383, 30, 42, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 341, 345, 343, 344, 352, 364, + 364, 364, -1, 364, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 364, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 364, + -1, -1, -1, 364, 364, -1, -1, -1, + -1, -1, -1, -1, -1, 364, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 364, 364, 364, 364, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 364, 364, -1, + -1, -1, 364, 364, 364, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 46, 162, -1, -1, 161, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 4, + 5, 6, 7, -1, -1, -1, -1, -1, + -1, -1, -1, 158, 11, 12, 13, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 9, 8, -1, -1, -1, + -1, 153, 392, 393, 394, 395, 396, 397, + 398, 399, 400, 401, 402, 403, -1, 10, + 3, 55, -1, -1, -1, -1, -1, -1, + 63, -1, -1, -1, -1, 17, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 56, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 50, 57, -1, -1, 326, 328, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 61, 59, 60, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 47, -1, + 58, -1, -1, -1, -1, -1, -1, -1, + 48, 191, 49, 198, 52, -1, 53, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 54, -1, -1, -1, 308, 312, -1, + -1, 62, 55, -1, -1, -1, -1, -1, + -1, 63, -1, -1, -1, -1, 17, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 56, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 50, 57, -1, -1, 326, 328, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 66, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 61, 59, 60, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 65, + 64, 58, -1, -1, -1, -1, -1, -1, + -1, 48, 191, 49, 198, 52, -1, 53, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 54, -1, -1, -1, 308, 312, + -1, -1, 62, 348, 349, 350, -1, 346, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 347, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 166, -1, -1, 374, -1, -1, -1, 372, + 373, -1, -1, -1, -1, -1, -1, -1, + -1, 351, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 368, 369, + 370, 371, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 45, 40, + 39, 380, 381, 41, 43, 44, 382, 383, + 30, 42, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 341, 345, 343, 344, 352, 389, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 388, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 67, + -1, -1, -1, -1, 68, 353, 364, 364, + 364, -1, 364, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 364, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 364, -1, + -1, -1, 364, 364, -1, -1, -1, -1, + -1, -1, -1, -1, 364, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 364, 364, 364, 364, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 364, 364, -1, -1, + -1, 364, 364, 364, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 69, + 71, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 389, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 388, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 70, -1, -1, -1, -1, 68, 75, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 389, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 388, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 74, -1, + -1, -1, -1, 68, 73, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 389, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 388, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 72, -1, -1, -1, + -1, 68, 361, 362, 376, 377, 378, -1, + 375, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 172, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 374, -1, -1, -1, + 372, 373, -1, -1, -1, -1, -1, -1, + -1, -1, 379, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 368, + 369, 370, 371, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 380, 381, -1, -1, -1, 382, + 383, 30, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 365, -1, 367, -1, 363, + 366, 81, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 163, 83, -1, -1, 186, -1, -1, 186, + 84, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 186, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 186, 82, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 186, -1, + -1, -1, -1, 85, 55, -1, -1, -1, + -1, 192, -1, 63, 192, -1, -1, 192, + 18, 86, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 192, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 56, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 192, 192, + -1, -1, -1, 50, 57, -1, -1, 326, + 328, -1, 87, 88, 89, -1, 192, -1, + -1, -1, -1, 192, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 61, 59, + 60, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 58, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 193, 52, + -1, 53, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 54, -1, -1, -1, + 308, 312, -1, -1, 62, 315, -1, -1, + 315, 315, 315, -1, 315, 315, 315, 315, + 315, 315, 315, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 77, 315, + 315, -1, 315, 315, 315, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 315, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 315, + 315, -1, -1, -1, 315, 315, -1, -1, + 315, 315, -1, 315, 315, 315, 315, 315, + 315, -1, -1, -1, 315, 315, 315, 315, + 315, 315, 315, 315, 315, 315, 315, 315, + 315, 315, 315, 315, 315, 315, 315, 315, + 315, 315, 315, 315, 315, 315, 315, 315, + 315, 315, 315, 315, 315, 315, 315, 315, + 315, 315, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 315, 315, 315, 200, -1, + -1, -1, -1, 200, -1, 200, 200, -1, + -1, 200, 200, 200, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 200, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 200, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 200, 200, -1, -1, -1, 200, 200, -1, + -1, 200, 200, -1, 200, 200, 200, 90, + 200, -1, -1, -1, -1, 200, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 200, 200, 200, 202, -1, -1, 100, 99, + 202, -1, 202, 202, -1, -1, 202, 202, + 202, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 202, 102, -1, + 101, -1, 98, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 202, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 202, 202, -1, + -1, -1, 202, 202, -1, -1, 202, 202, + -1, 202, 202, 202, 202, 202, -1, -1, + -1, -1, 202, 219, 221, 223, 103, 264, + 268, 270, 272, 266, 274, 276, 280, 282, + 284, 278, 286, 252, 256, 258, 260, 254, + 262, 228, 232, 234, 236, 230, 238, 240, + 244, 246, 248, 242, 250, 202, 202, 202, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 226, 225, 227, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 91, -1, -1, 92, 93, 94, 95, + 96, 97, 214, -1, -1, 214, 214, 214, + -1, 214, 214, 300, 303, 214, 214, 214, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 214, 214, -1, 214, + 302, 214, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 214, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 104, 214, -1, -1, + -1, 214, 214, -1, -1, 214, 214, -1, + 214, 214, 214, 214, 214, 301, -1, -1, + -1, 214, 214, 214, 214, 214, 214, 214, + 214, 214, 214, 214, 214, 214, 214, 214, + 214, 214, 214, 214, 214, 214, 214, 214, + 214, 214, 214, 214, 214, 214, 214, 214, + 214, 214, 214, 214, 214, 214, 214, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 214, 214, 214, 55, -1, -1, -1, -1, + -1, -1, 63, -1, -1, -1, -1, 17, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 56, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 316, 57, -1, -1, 326, 328, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 61, 59, 60, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 58, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 310, + 312, -1, -1, 62, 55, -1, -1, -1, + -1, -1, -1, 63, -1, -1, -1, -1, + 17, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 56, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 316, 57, -1, -1, 326, + 328, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 61, 59, + 60, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 58, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 311, 312, -1, -1, 62, 313, -1, -1, + 313, 313, 313, -1, 313, 313, 313, 313, + 313, 313, 313, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 313, + 313, -1, 313, 313, 313, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 313, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 313, + 313, -1, -1, -1, 313, 313, -1, -1, + 313, 313, 322, 313, 313, 313, 313, 313, + 313, -1, -1, -1, 313, 313, 313, 313, + 313, 313, 313, 313, 313, 313, 313, 313, + 313, 313, 313, 313, 313, 313, 313, 313, + 313, 313, 313, 313, 313, 313, 313, 313, + 313, 313, 313, 313, 313, 313, 313, 313, + 313, 313, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 313, 313, 313, 314, -1, + -1, 314, 314, 314, -1, 314, 314, 314, + 314, 314, 314, 314, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 314, 314, -1, 314, 314, 314, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 314, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 314, 314, -1, -1, -1, 314, 314, -1, + -1, 314, 314, 324, 314, 314, 314, 314, + 314, 314, -1, -1, -1, 314, 314, 314, + 314, 314, 314, 314, 314, 314, 314, 314, + 314, 314, 314, 314, 314, 314, 314, 314, + 314, 314, 314, 314, 314, 314, 314, 314, + 314, 314, 314, 314, 314, 314, 314, 314, + 314, 314, 314, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 314, 314, 314, 338, + -1, -1, -1, 338, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 79, 338, -1, -1, -1, 338, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 80, 330, 330, 330, -1, 330, + -1, -1, 330, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 78, 105, 55, -1, -1, -1, -1, -1, + -1, 63, -1, -1, -1, -1, 17, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 56, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 50, 57, -1, -1, 326, 328, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 61, 59, 60, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 76, + -1, 58, -1, -1, -1, -1, -1, -1, + -1, 48, 191, 49, 198, 52, -1, 53, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 54, -1, -1, -1, 308, 312, + -1, -1, 62, 164, 81, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 175, 55, -1, -1, -1, + -1, -1, -1, 63, -1, -1, -1, -1, + 17, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 56, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 51, 57, -1, -1, 326, + 328, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 61, 59, + 60, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 4, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 158, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 180, -1, 179, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 160, 108, + -1, 107, -1, 58, -1, -1, 106, 178, + -1, -1, -1, 48, 191, 49, 198, 52, + -1, 53, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 54, -1, -1, -1, + 308, 312, -1, -1, 62, 384, 391, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 109, 376, 377, 378, + -1, 375, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 354, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 374, -1, -1, + -1, 372, 373, -1, -1, -1, -1, -1, + -1, -1, -1, 379, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 368, 369, 370, 371, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 380, 381, -1, -1, -1, + 382, 383, 30, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 365, -1, 367, -1, + 363, 366, 355, 364, 364, 364, -1, 364, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 364, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 364, -1, -1, -1, 364, + 364, -1, -1, -1, -1, -1, -1, -1, + -1, 364, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 364, 364, + 364, 364, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 364, 364, -1, -1, -1, 364, 364, + 364, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 110, 359, 364, 364, + 364, -1, 364, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 364, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 364, -1, + -1, -1, 364, 364, -1, -1, -1, -1, + -1, -1, -1, -1, 364, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 364, 364, 364, 364, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 364, 364, -1, -1, + -1, 364, 364, 364, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 111, + 357, 364, 364, 364, -1, 364, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 364, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 364, -1, -1, -1, 364, 364, -1, + -1, -1, -1, -1, -1, -1, -1, 364, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 364, 364, 364, 364, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 364, + 364, -1, -1, -1, 364, 364, 364, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 112, 321, -1, -1, 81, 55, + -1, -1, -1, -1, -1, -1, 63, -1, + -1, -1, -1, 17, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 56, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 50, 57, + -1, -1, 326, 328, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 61, 59, 60, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 58, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 199, 52, -1, 53, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 54, + -1, -1, -1, 308, 312, -1, -1, 62, + 319, 114, 115, -1, 335, -1, -1, 336, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 329, + 113, 317, -1, -1, -1, 116, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 337, + 318, -1, -1, -1, 116, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 337, 55, + -1, -1, -1, -1, -1, -1, 63, -1, + -1, -1, -1, 17, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 56, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 50, 57, + -1, -1, 326, 328, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 61, 59, 60, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 58, -1, + -1, -1, -1, -1, -1, -1, 117, 191, + 49, 198, 52, -1, 53, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 54, + -1, -1, -1, 308, 312, -1, -1, 62, + 55, -1, -1, -1, -1, -1, -1, 63, + -1, -1, -1, -1, 17, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 56, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 50, + 57, -1, -1, 326, 328, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 61, 59, 60, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 58, + -1, -1, -1, -1, -1, -1, -1, -1, + 187, 49, 198, 52, -1, 53, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 54, -1, -1, -1, 308, 312, -1, -1, + 62, 55, -1, -1, -1, -1, -1, -1, + 63, -1, -1, -1, -1, 17, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 56, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 50, 57, -1, -1, 326, 328, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 61, 59, 60, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 58, -1, -1, -1, -1, -1, -1, -1, + -1, 188, 49, 198, 52, -1, 53, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 54, -1, -1, -1, 308, 312, -1, + -1, 62, 55, -1, -1, -1, -1, -1, + -1, 63, -1, -1, -1, -1, 17, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 56, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 50, 57, -1, -1, 326, 328, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 61, 59, 60, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 58, -1, -1, -1, -1, -1, -1, + -1, -1, 189, 49, 198, 52, -1, 53, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 54, -1, -1, -1, 308, 312, + -1, -1, 62, 55, -1, -1, -1, -1, + -1, -1, 63, -1, -1, -1, -1, 17, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 56, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 50, 57, -1, -1, 326, 328, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 61, 59, 60, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 58, -1, -1, -1, -1, -1, + -1, -1, -1, 190, 49, 198, 52, -1, + 53, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 54, -1, -1, -1, 308, + 312, -1, -1, 62, 55, -1, -1, -1, + -1, -1, -1, 63, -1, -1, -1, -1, + 17, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 56, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 50, 57, -1, -1, 326, + 328, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 61, 59, + 60, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 58, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 194, 52, + -1, 53, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 54, -1, -1, -1, + 308, 312, -1, -1, 62, 55, -1, -1, + -1, -1, -1, -1, 63, -1, -1, -1, + -1, 17, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 56, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 50, 57, -1, -1, + 326, 328, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 61, + 59, 60, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 58, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 195, + 52, -1, 53, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 54, -1, -1, + -1, 308, 312, -1, -1, 62, 55, -1, + -1, -1, -1, -1, -1, 63, -1, -1, + -1, -1, 17, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 56, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 50, 57, -1, + -1, 326, 328, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 61, 59, 60, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 58, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 196, 52, -1, 53, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 54, -1, + -1, -1, 308, 312, -1, -1, 62, 55, + -1, -1, -1, -1, -1, -1, 63, -1, + -1, -1, -1, 17, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 56, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 50, 57, + -1, -1, 326, 328, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 61, 59, 60, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 58, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 197, 52, -1, 53, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 54, + -1, -1, -1, 308, 312, -1, -1, 62, + 386, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 201, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 118, 119, -1, + -1, 121, -1, 122, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 120, -1, -1, -1, -1, 292, -1, + -1, -1, 296, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 203, 290, -1, -1, + -1, -1, -1, -1, -1, -1, 204, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 288, 295, 123, -1, -1, -1, -1, -1, + -1, 124, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 120, -1, -1, -1, + -1, 292, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 206, + 290, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 288, 124, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 120, + -1, -1, -1, -1, 292, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 208, 290, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 288, 124, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 120, -1, -1, -1, -1, 292, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 209, 290, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 288, 124, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 120, -1, -1, + -1, -1, 292, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 210, 290, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 288, 124, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 120, -1, -1, -1, -1, 292, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 211, 290, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 288, + 125, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 120, -1, -1, -1, -1, + 292, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 212, 290, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 288, 215, -1, -1, 215, -1, + 215, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 215, -1, + -1, -1, -1, 215, -1, -1, -1, 215, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 261, 273, 285, + 237, 249, 216, -1, -1, 216, -1, 216, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 216, -1, -1, + -1, -1, 216, -1, -1, -1, 216, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 224, -1, 259, 271, 283, 235, + 247, 217, -1, -1, 217, -1, 217, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 217, -1, -1, -1, + -1, 217, -1, -1, -1, 217, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 222, -1, 257, 269, 281, 233, 245, + 218, -1, -1, 218, -1, 218, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 218, -1, -1, -1, -1, + 218, -1, -1, -1, 218, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 220, -1, 253, 265, 277, 229, 241, 255, + 267, 279, 231, 243, 263, 275, 287, 239, + 251, 127, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 309, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 126, 17, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 323, -1, + -1, 326, 328, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 325, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 320, 55, + -1, -1, -1, -1, -1, -1, 63, -1, + 131, -1, -1, 17, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 56, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 51, 57, + -1, -1, 326, 328, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 61, 59, 60, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 4, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 158, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 180, -1, 179, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 160, 108, -1, 107, -1, 58, -1, + -1, -1, 177, -1, -1, -1, 48, 191, + 49, 198, 52, -1, 53, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 54, + -1, -1, -1, 308, 312, -1, -1, 62, + 81, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 184, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 120, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 130, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 129, -1, 183, 165, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 128, 387, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 387, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 132, + 376, 377, 378, -1, 375, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 356, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 374, -1, -1, -1, 372, 373, -1, -1, + -1, -1, -1, -1, -1, -1, 379, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 368, 369, 370, 371, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 380, 381, + -1, -1, -1, 382, 383, 30, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 365, + -1, 367, -1, 363, 366, 376, 377, 378, + -1, 375, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 360, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 374, -1, -1, + -1, 372, 373, -1, -1, -1, -1, -1, + -1, -1, -1, 379, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 368, 369, 370, 371, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 380, 381, -1, -1, -1, + 382, 383, 30, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 365, -1, 367, -1, + 363, 366, 376, 377, 378, -1, 375, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 358, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 374, -1, -1, -1, 372, 373, + -1, -1, -1, -1, -1, -1, -1, -1, + 379, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 368, 369, 370, + 371, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 380, 381, -1, -1, -1, 382, 383, 30, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 365, -1, 367, -1, 363, 366, 332, + 332, 332, -1, 332, 331, -1, 332, 338, + -1, -1, -1, 338, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 133, 338, -1, -1, -1, 338, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 134, 339, -1, -1, 135, 339, + 83, -1, -1, 185, -1, -1, 185, 84, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 185, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 185, 82, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 185, -1, -1, + -1, -1, 85, 391, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 136, 139, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 137, -1, -1, -1, -1, + -1, -1, -1, -1, 138, 342, 342, 342, + -1, 342, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 342, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 342, -1, -1, 342, -1, -1, + -1, 342, 342, -1, -1, -1, -1, -1, + -1, -1, -1, 342, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 342, 342, 342, 342, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 342, 342, 342, 342, 342, 342, 342, 342, + 342, 342, 342, 342, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 140, + 297, 298, 124, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 120, -1, -1, + -1, -1, 292, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 207, 290, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 288, 292, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 138, + 142, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 141, -1, 138, 144, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 304, 309, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 143, 35, 181, + 124, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 120, -1, -1, -1, -1, + 292, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 182, 290, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 288, 176, 390, 333, -1, -1, + -1, 116, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 337, 334, -1, -1, -1, + 116, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 337, 340, 385, -1, -1, -1, + -1, 385, -1, 385, 385, -1, -1, 385, + 385, 385, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 385, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 385, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 385, 385, + -1, -1, -1, 385, 385, -1, -1, 385, + 385, -1, 385, 385, 385, 385, 385, -1, + -1, 132, -1, 385, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 385, 385, + 385, 145, 289, 291, -1, -1, 294, 348, + 349, 350, -1, 346, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 347, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 293, -1, -1, 374, + -1, -1, -1, 372, 373, -1, -1, -1, + -1, -1, -1, -1, -1, 351, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 368, 369, 370, 371, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 45, 40, 39, 380, 381, 41, + 43, 44, 382, 383, 30, 42, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 341, 345, 343, 344, + 352, 146, 291, -1, -1, 299, 305, 306, + -1, -1, -1, -1, -1, -1, 309, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 147, 121, + -1, 122, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 296, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 148, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 295, + 124, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 120, -1, -1, -1, -1, + 292, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 149, 290, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 288, 307, 205, 213, +}; + +unsigned short Parser_keys[] = { + 128, 227, 225, 225, 128, 228, 128, 244, + 128, 245, 128, 128, 128, 128, 45, 248, + 40, 249, 40, 249, 128, 250, 123, 128, + 123, 123, 123, 123, 128, 128, 123, 123, + 59, 128, 45, 248, 132, 132, 40, 292, + 40, 242, 40, 242, 59, 59, 128, 187, + 40, 292, 125, 228, 61, 141, 40, 242, + 59, 59, 59, 59, 40, 40, 40, 289, + 40, 289, 40, 249, 125, 244, 33, 281, + 33, 281, 40, 289, 128, 295, 59, 59, + 40, 249, 42, 295, 42, 295, 42, 295, + 59, 59, 59, 59, 40, 292, 44, 59, + 38, 144, 33, 281, 33, 201, 33, 181, + 33, 271, 33, 201, 33, 281, 33, 281, + 33, 201, 33, 201, 182, 279, 182, 279, + 179, 280, 134, 134, 33, 281, 59, 59, + 44, 59, 33, 281, 41, 41, 128, 294, + 40, 292, 59, 59, 40, 249, 59, 59, + 40, 249, 59, 59, 40, 249, 41, 44, + 33, 281, 179, 283, 182, 284, 182, 284, + 33, 281, 33, 281, 33, 281, 33, 281, + 33, 281, 33, 281, 33, 281, 33, 281, + 33, 281, 128, 293, 40, 275, 33, 274, + 40, 274, 40, 274, 40, 274, 40, 274, + 40, 274, 40, 206, 40, 206, 40, 206, + 40, 206, 202, 206, 202, 206, 44, 276, + 45, 281, 33, 281, 44, 255, 128, 245, + 41, 142, 40, 292, 40, 292, 40, 292, + 179, 186, 182, 279, 182, 279, 182, 186, + 38, 144, 128, 294, 128, 274, 40, 242, + 132, 132, 132, 132, 40, 274, 128, 274, + 128, 274, 44, 125, 132, 276, 61, 61, + 59, 59, 40, 274, 124, 124, 128, 128, + 182, 284, 182, 284, 186, 186, 33, 181, + 44, 44, 41, 41, 41, 44, 40, 289, + 44, 44, 41, 44, 125, 125, 125, 276, + 43, 275, 40, 274, 125, 125, 41, 41, + 41, 41, 0, 0 +}; + +unsigned int Parser_offsets[] = { + 0, 100, 101, 202, 319, 437, 438, 439, + 643, 853, 1063, 1186, 1192, 1193, 1194, 1195, + 1196, 1266, 1470, 1471, 1724, 1927, 2130, 2131, + 2191, 2444, 2548, 2629, 2832, 2833, 2834, 2835, + 3085, 3335, 3545, 3665, 3914, 4163, 4413, 4581, + 4582, 4792, 5046, 5300, 5554, 5555, 5556, 5809, + 5825, 5932, 6181, 6350, 6499, 6738, 6907, 7156, + 7405, 7574, 7743, 7841, 7939, 8041, 8042, 8291, + 8292, 8308, 8557, 8558, 8725, 8978, 8979, 9189, + 9190, 9400, 9401, 9611, 9615, 9864, 9969, 10072, + 10175, 10424, 10673, 10922, 11171, 11420, 11669, 11918, + 12167, 12416, 12582, 12818, 13060, 13295, 13530, 13765, + 14000, 14235, 14402, 14569, 14736, 14903, 14908, 14913, + 15146, 15383, 15632, 15844, 15962, 16064, 16317, 16570, + 16823, 16831, 16929, 17027, 17032, 17139, 17306, 17453, + 17656, 17657, 17658, 17893, 18040, 18187, 18269, 18414, + 18415, 18416, 18651, 18652, 18653, 18756, 18859, 18860, + 19009, 19010, 19011, 19015, 19265, 19266, 19270, 19271, + 19423, 19656, 19891, 19892, 19893, 19894 +}; + +unsigned short Parser_targs[] = { + 1, 2, 3, 4, 5, 6, 7, 8, + 9, 10, 11, 12, 13, 14, 15, 16, + 17, 18, 18, 19, 20, 21, 22, 23, + 24, 25, 26, 27, 28, 29, 30, 31, + 32, 33, 34, 35, 36, 37, 38, 39, + 40, 41, 42, 43, 44, 45, 46, 47, + 48, 49, 50, 50, 51, 52, 53, 54, + 55, 56, 57, 58, 59, 60, 61, 62, + 63, 64, 65, 66, 67, 68, 69, 70, + 71, 72, 73, 74, 75, 76, 77, 78, + 79, 80, 81, 82, 83, 84, 85, 86, + 87, 88, 89, 90, 91, 92, 93, 94, + 95, 96, 97, 98, 99, 100, 101, 102, + 103, 104, 105, 106, 107, 108, 109, 110, + 111, 112, 113, 114, 115, 116, 117, 118, + 119, 120, 121, 122, 123, 124, 125, 126, + 127, 128, 129, 130, 131, 132, 133, 134, + 135, 136, 137, 138, 139, 140, 141, 142, + 143, 144, 145, 146, 147, 148, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149 +}; + +unsigned int Parser_actInds[] = { + 0, 2, 4, 6, 8, 10, 12, 14, + 16, 18, 20, 22, 24, 26, 28, 30, + 32, 34, 36, 39, 41, 43, 45, 47, + 49, 51, 53, 55, 57, 59, 61, 63, + 65, 67, 69, 71, 73, 75, 77, 79, + 81, 83, 85, 87, 89, 91, 93, 95, + 97, 99, 101, 103, 106, 108, 110, 112, + 114, 116, 118, 120, 122, 124, 126, 128, + 130, 132, 134, 136, 138, 140, 142, 144, + 146, 148, 150, 152, 154, 156, 158, 160, + 162, 164, 166, 168, 170, 172, 174, 176, + 178, 180, 182, 184, 186, 188, 190, 192, + 194, 196, 198, 200, 202, 204, 206, 208, + 210, 213, 215, 217, 219, 221, 223, 225, + 227, 229, 231, 233, 235, 237, 239, 241, + 243, 245, 247, 249, 251, 253, 255, 257, + 259, 261, 263, 265, 267, 269, 271, 273, + 275, 277, 279, 281, 283, 285, 287, 289, + 291, 293, 295, 297, 299, 301, 303, 305, + 307, 309, 311, 313, 315, 317, 319, 321, + 323, 325, 327, 329, 331, 333, 335, 337, + 339, 341, 343, 345, 347, 349, 351, 353, + 355, 357, 359, 361, 363, 365, 367, 369, + 371, 373, 375, 377, 379, 381, 383, 385, + 387, 389, 391, 393, 395, 397, 399, 401, + 403, 405, 407, 409, 411, 413, 415, 417, + 419, 421, 423, 425, 427, 429, 431, 433, + 435, 437, 439, 441, 443, 445, 447, 449, + 451, 453, 455, 457, 459, 461, 463, 465, + 467, 469, 471, 473, 475, 477, 479, 481, + 483, 485, 487, 489, 491, 493, 495, 497, + 499, 501, 503, 505, 507, 509, 511, 513, + 515, 517, 519, 521, 523, 525, 527, 529, + 531, 533, 535, 537, 539, 541, 543, 545, + 547, 549, 551, 553, 555, 557, 559, 561, + 563, 565, 567, 569, 571, 573, 575, 577, + 579, 581, 583, 585, 587, 589, 591, 593, + 595, 597, 599, 601, 603, 605, 607, 609, + 611, 613, 615, 617, 619, 621, 623, 625, + 627, 629, 631, 633, 635, 637, 639, 641, + 643, 645, 647, 649, 651, 653, 655, 657, + 659, 661, 663, 665, 667, 669, 671, 673, + 675, 677, 679, 681, 683, 685, 687, 689, + 691, 693, 695, 697, 699, 701, 703, 705, + 707, 709, 711, 713, 715, 717, 719, 721, + 723, 725, 727, 729, 731, 733, 735, 737, + 739, 741, 743, 745, 747, 749, 751, 753, + 755, 757, 759, 761, 763, 765, 767, 769, + 771, 773, 775, 777, 779, 781, 783, 785, + 787, 789, 791, 793, 795, 797, 799, 801, + 803, 805, 807, 809, 811 +}; + +unsigned int Parser_actions[] = { + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 214, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 90, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 302, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 2, + 0, 7, 0, 10, 0, 15, 0, 18, + 0, 71, 0, 75, 0, 79, 0, 83, + 0, 86, 0, 87, 0, 90, 0, 95, + 0, 99, 0, 103, 0, 107, 0, 111, + 0, 115, 0, 119, 0, 123, 0, 127, + 0, 131, 0, 135, 0, 139, 0, 142, + 0, 146, 0, 151, 0, 155, 0, 159, + 0, 163, 0, 167, 0, 171, 0, 175, + 0, 179, 0, 182, 0, 186, 0, 190, + 0, 195, 0, 199, 0, 203, 0, 207, + 0, 211, 0, 214, 0, 219, 0, 223, + 0, 227, 0, 231, 0, 235, 0, 239, + 0, 243, 0, 246, 0, 251, 0, 254, + 0, 259, 0, 263, 0, 267, 0, 271, + 0, 275, 0, 279, 0, 283, 0, 287, + 0, 291, 0, 295, 0, 299, 0, 302, + 0, 306, 0, 310, 0, 314, 0, 318, + 0, 323, 0, 327, 0, 331, 0, 335, + 0, 339, 0, 343, 0, 347, 0, 351, + 0, 355, 0, 359, 0, 363, 0, 367, + 0, 371, 0, 375, 0, 379, 0, 383, + 0, 387, 0, 391, 0, 395, 0, 399, + 0, 403, 0, 407, 0, 411, 0, 415, + 0, 419, 0, 423, 0, 427, 0, 431, + 0, 435, 0, 439, 0, 443, 0, 447, + 0, 451, 0, 455, 0, 459, 0, 463, + 0, 467, 0, 471, 0, 475, 0, 479, + 0, 483, 0, 487, 0, 491, 0, 495, + 0, 499, 0, 503, 0, 507, 0, 511, + 0, 515, 0, 519, 0, 523, 0, 527, + 0, 531, 0, 535, 0, 539, 0, 543, + 0, 547, 0, 551, 0, 555, 0, 559, + 0, 563, 0, 567, 0, 571, 0, 575, + 0, 579, 0, 583, 0, 587, 0, 591, + 0, 595, 0, 599, 0, 603, 0, 607, + 0, 610, 0, 611, 0, 615, 0, 618, + 0, 623, 0, 627, 0, 631, 0, 635, + 0, 638, 0, 643, 0, 647, 0, 651, + 0, 655, 0, 659, 0, 663, 0, 667, + 0, 671, 0, 675, 0, 679, 0, 683, + 0, 687, 0, 691, 0, 694, 0, 698, + 0, 702, 0, 703, 0, 707, 0, 711, + 0, 715, 0, 719, 0, 723, 0, 726, + 0, 727, 0, 730, 0, 731, 0, 735, + 0, 739, 0, 743, 0, 747, 0, 750, + 0, 755, 0, 758, 0, 763, 0, 767, + 0, 771, 0, 775, 0, 779, 0, 782, + 0, 786, 0, 791, 0, 795, 0, 798, + 0, 803, 0, 807, 0, 811, 0, 815, + 0, 819, 0, 823, 0, 827, 0, 831, + 0, 835, 0, 839, 0, 843, 0, 847, + 0, 851, 0, 855, 0, 859, 0, 863, + 0, 867, 0, 871, 0, 875, 0, 879, + 0, 883, 0, 886, 0, 891, 0, 895, + 0, 899, 0, 903, 0, 907, 0, 911, + 0, 915, 0, 919, 0, 923, 0, 927, + 0, 931, 0, 935, 0, 939, 0, 943, + 0, 947, 0, 951, 0, 955, 0, 959, + 0, 963, 0, 967, 0, 970, 0, 974, + 0, 978, 0, 983, 0, 986, 0, 991, + 0, 995, 0, 23, 0, 27, 0, 31, + 0, 35, 0, 39, 0, 43, 0, 47, + 0, 51, 0, 55, 0, 59, 0, 63, + 0, 67, 0, 1, 0 +}; + +int Parser_commitLen[] = { + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, 2 +}; + +char Parser_prodLengths[] = { + 1, 3, 0, 2, 0, 1, 1, 1, + 1, 1, 1, 1, 1, 1, 1, 1, + 1, 3, 4, 4, 1, 1, 0, 4, + 5, 5, 1, 5, 4, 3, 4, 3, + 3, 5, 2, 0, 1, 4, 2, 1, + 1, 1, 3, 2, 1, 0, 3, 1, + 3, 3, 3, 3, 1, 1, 2, 3, + 3, 3, 3, 1, 3, 1, 3, 1, + 3, 3, 7, 3, 4, 3, 3, 3, + 3, 3, 7, 1, 1, 1, 1, 1, + 1, 2, 1, 2, 1, 2, 1, 1, + 1, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 3, 1, + 1, 3, 1, 1, 1, 2, 2, 1, + 2, 2, 2, 2, 4, 5, 5, 6, + 1, 1, 2, 2, 1, 1, 1, 1, + 3, 3, 3, 3, 3, 1, 1, 1, + 2, 1, 2, 0, 2, 1, 3, 3, + 1, 1, 2, 0, 1, 3, 2, 0, + 1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 2, 3, 3, 4, 3, 4, + 3, 4, 2, 2, 2, 0, 1, 1, + 1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, 1, 1, 1, 1, + 1, 4, 2, 0, 2, 1, 0, 3, + 1, 1 +}; + +unsigned short Parser_prodLhsIds[] = { + 227, 226, 226, 228, 228, 229, 229, 229, + 229, 229, 229, 229, 229, 229, 229, 229, + 229, 241, 239, 240, 243, 244, 244, 238, + 230, 231, 245, 232, 233, 233, 234, 235, + 236, 237, 250, 250, 247, 247, 251, 251, + 252, 252, 252, 253, 253, 253, 246, 246, + 256, 256, 256, 256, 256, 257, 258, 258, + 258, 258, 258, 258, 259, 259, 260, 260, + 262, 262, 262, 262, 262, 262, 262, 262, + 262, 262, 262, 262, 263, 263, 263, 263, + 266, 266, 266, 266, 266, 266, 266, 266, + 266, 267, 267, 267, 267, 267, 267, 267, + 267, 267, 267, 267, 267, 268, 268, 268, + 268, 268, 268, 268, 268, 268, 268, 268, + 268, 269, 269, 269, 269, 269, 269, 269, + 269, 269, 269, 269, 269, 270, 270, 270, + 270, 270, 270, 270, 270, 270, 270, 270, + 270, 271, 271, 271, 271, 271, 271, 271, + 271, 271, 271, 271, 271, 254, 254, 254, + 274, 255, 265, 264, 275, 275, 275, 272, + 273, 273, 273, 273, 273, 273, 273, 273, + 273, 276, 277, 277, 277, 278, 278, 278, + 278, 278, 278, 278, 278, 281, 281, 248, + 248, 248, 280, 280, 282, 282, 283, 283, + 283, 283, 279, 279, 284, 284, 242, 242, + 285, 285, 285, 288, 288, 288, 288, 288, + 288, 286, 286, 286, 286, 286, 286, 286, + 286, 286, 286, 286, 249, 249, 291, 291, + 291, 287, 287, 287, 287, 287, 287, 287, + 292, 292, 292, 292, 292, 289, 289, 289, + 289, 289, 261, 293, 290, 295, 295, 294, + 294, 296 +}; + +const char *Parser_prodNames[] = { + "start-1", + "section_list-1", + "section_list-2", + "statement_list-1", + "statement_list-2", + "statement-1", + "statement-2", + "statement-3", + "statement-4", + "statement-5", + "statement-6", + "statement-7", + "statement-8", + "statement-9", + "statement-10", + "statement-11", + "statement-12", + "length_spec-1", + "pre_push_spec-1", + "post_pop_spec-1", + "export_open-1", + "opt_export-1", + "opt_export-2", + "export_block-1", + "assignment-1", + "instantiation-1", + "machine_name-1", + "action_spec-1", + "alphtype_spec-1", + "alphtype_spec-2", + "range_spec-1", + "getkey_spec-1", + "access_spec-1", + "variable_spec-1", + "opt_whitespace-1", + "opt_whitespace-2", + "join_or_lm-1", + "join_or_lm-2", + "lm_part_list-1", + "lm_part_list-2", + "longest_match_part-1", + "longest_match_part-2", + "longest_match_part-3", + "opt_lm_part_action-1", + "opt_lm_part_action-2", + "opt_lm_part_action-3", + "join-1", + "join-2", + "expression-1", + "expression-2", + "expression-3", + "expression-4", + "expression-5", + "term_short-1", + "term-1", + "term-2", + "term-3", + "term-4", + "term-5", + "term-6", + "factor_with_label-1", + "factor_with_label-2", + "factor_with_ep-1", + "factor_with_ep-2", + "factor_with_aug-1", + "factor_with_aug-2", + "factor_with_aug-3", + "factor_with_aug-4", + "factor_with_aug-5", + "factor_with_aug-6", + "factor_with_aug-7", + "factor_with_aug-8", + "factor_with_aug-9", + "factor_with_aug-10", + "factor_with_aug-11", + "factor_with_aug-12", + "aug_type_base-1", + "aug_type_base-2", + "aug_type_base-3", + "aug_type_base-4", + "aug_type_cond-1", + "aug_type_cond-2", + "aug_type_cond-3", + "aug_type_cond-4", + "aug_type_cond-5", + "aug_type_cond-6", + "aug_type_cond-7", + "aug_type_cond-8", + "aug_type_cond-9", + "aug_type_to_state-1", + "aug_type_to_state-2", + "aug_type_to_state-3", + "aug_type_to_state-4", + "aug_type_to_state-5", + "aug_type_to_state-6", + "aug_type_to_state-7", + "aug_type_to_state-8", + "aug_type_to_state-9", + "aug_type_to_state-10", + "aug_type_to_state-11", + "aug_type_to_state-12", + "aug_type_from_state-1", + "aug_type_from_state-2", + "aug_type_from_state-3", + "aug_type_from_state-4", + "aug_type_from_state-5", + "aug_type_from_state-6", + "aug_type_from_state-7", + "aug_type_from_state-8", + "aug_type_from_state-9", + "aug_type_from_state-10", + "aug_type_from_state-11", + "aug_type_from_state-12", + "aug_type_eof-1", + "aug_type_eof-2", + "aug_type_eof-3", + "aug_type_eof-4", + "aug_type_eof-5", + "aug_type_eof-6", + "aug_type_eof-7", + "aug_type_eof-8", + "aug_type_eof-9", + "aug_type_eof-10", + "aug_type_eof-11", + "aug_type_eof-12", + "aug_type_gbl_error-1", + "aug_type_gbl_error-2", + "aug_type_gbl_error-3", + "aug_type_gbl_error-4", + "aug_type_gbl_error-5", + "aug_type_gbl_error-6", + "aug_type_gbl_error-7", + "aug_type_gbl_error-8", + "aug_type_gbl_error-9", + "aug_type_gbl_error-10", + "aug_type_gbl_error-11", + "aug_type_gbl_error-12", + "aug_type_local_error-1", + "aug_type_local_error-2", + "aug_type_local_error-3", + "aug_type_local_error-4", + "aug_type_local_error-5", + "aug_type_local_error-6", + "aug_type_local_error-7", + "aug_type_local_error-8", + "aug_type_local_error-9", + "aug_type_local_error-10", + "aug_type_local_error-11", + "aug_type_local_error-12", + "action_embed-1", + "action_embed-2", + "action_embed-3", + "action_embed_word-1", + "action_embed_block-1", + "priority_name-1", + "priority_aug-1", + "priority_aug_num-1", + "priority_aug_num-2", + "priority_aug_num-3", + "local_err_name-1", + "factor_with_rep-1", + "factor_with_rep-2", + "factor_with_rep-3", + "factor_with_rep-4", + "factor_with_rep-5", + "factor_with_rep-6", + "factor_with_rep-7", + "factor_with_rep-8", + "factor_with_rep-9", + "factor_rep_num-1", + "factor_with_neg-1", + "factor_with_neg-2", + "factor_with_neg-3", + "factor-1", + "factor-2", + "factor-3", + "factor-4", + "factor-5", + "factor-6", + "factor-7", + "factor-8", + "range_lit-1", + "range_lit-2", + "alphabet_num-1", + "alphabet_num-2", + "alphabet_num-3", + "regular_expr-1", + "regular_expr-2", + "regular_expr_item-1", + "regular_expr_item-2", + "regular_expr_char-1", + "regular_expr_char-2", + "regular_expr_char-3", + "regular_expr_char-4", + "regular_expr_or_data-1", + "regular_expr_or_data-2", + "regular_expr_or_char-1", + "regular_expr_or_char-2", + "inline_block-1", + "inline_block-2", + "inline_block_item-1", + "inline_block_item-2", + "inline_block_item-3", + "inline_block_symbol-1", + "inline_block_symbol-2", + "inline_block_symbol-3", + "inline_block_symbol-4", + "inline_block_symbol-5", + "inline_block_symbol-6", + "inline_block_interpret-1", + "inline_block_interpret-2", + "inline_block_interpret-3", + "inline_block_interpret-4", + "inline_block_interpret-5", + "inline_block_interpret-6", + "inline_block_interpret-7", + "inline_block_interpret-8", + "inline_block_interpret-9", + "inline_block_interpret-10", + "inline_block_interpret-11", + "inline_expr-1", + "inline_expr-2", + "inline_expr_item-1", + "inline_expr_item-2", + "inline_expr_item-3", + "inline_expr_any-1", + "inline_expr_any-2", + "inline_expr_any-3", + "inline_expr_any-4", + "inline_expr_any-5", + "inline_expr_any-6", + "inline_expr_any-7", + "inline_expr_symbol-1", + "inline_expr_symbol-2", + "inline_expr_symbol-3", + "inline_expr_symbol-4", + "inline_expr_symbol-5", + "inline_expr_interpret-1", + "inline_expr_interpret-2", + "inline_expr_interpret-3", + "inline_expr_interpret-4", + "inline_expr_interpret-5", + "local_state_ref-1", + "no_name_sep-1", + "state_ref-1", + "opt_name_sep-1", + "opt_name_sep-2", + "state_ref_names-1", + "state_ref_names-2", + "_start-1" +}; + +const char *Parser_lelNames[] = { + "D-0", + "D-1", + "D-2", + "D-3", + "D-4", + "D-5", + "D-6", + "D-7", + "D-8", + "D-9", + "D-10", + "D-11", + "D-12", + "D-13", + "D-14", + "D-15", + "D-16", + "D-17", + "D-18", + "D-19", + "D-20", + "D-21", + "D-22", + "D-23", + "D-24", + "D-25", + "D-26", + "D-27", + "D-28", + "D-29", + "D-30", + "D-31", + "D-32", + "!", + "\"", + "#", + "$", + "%", + "&", + "'", + "(", + ")", + "*", + "+", + ",", + "-", + ".", + "/", + "0", + "1", + "2", + "3", + "4", + "5", + "6", + "7", + "8", + "9", + ":", + ";", + "<", + "=", + ">", + "?", + "@", + "A", + "B", + "C", + "D", + "E", + "F", + "G", + "H", + "I", + "J", + "K", + "L", + "M", + "N", + "O", + "P", + "Q", + "R", + "S", + "T", + "U", + "V", + "W", + "X", + "Y", + "Z", + "[", + "\\", + "]", + "^", + "_", + "`", + "a", + "b", + "c", + "d", + "e", + "f", + "g", + "h", + "i", + "j", + "k", + "l", + "m", + "n", + "o", + "p", + "q", + "r", + "s", + "t", + "u", + "v", + "w", + "x", + "y", + "z", + "{", + "|", + "}", + "~", + "D-127", + "TK_Word", + "TK_Literal", + "TK_Number", + "TK_EndSection", + "TK_UInt", + "TK_Hex", + "TK_DotDot", + "TK_ColonGt", + "TK_ColonGtGt", + "TK_LtColon", + "TK_Arrow", + "TK_DoubleArrow", + "TK_StarStar", + "TK_ColonEquals", + "TK_NameSep", + "TK_BarStar", + "TK_DashDash", + "TK_StartCond", + "TK_AllCond", + "TK_LeavingCond", + "TK_Middle", + "TK_StartGblError", + "TK_AllGblError", + "TK_FinalGblError", + "TK_NotFinalGblError", + "TK_NotStartGblError", + "TK_MiddleGblError", + "TK_StartLocalError", + "TK_AllLocalError", + "TK_FinalLocalError", + "TK_NotFinalLocalError", + "TK_NotStartLocalError", + "TK_MiddleLocalError", + "TK_StartEOF", + "TK_AllEOF", + "TK_FinalEOF", + "TK_NotFinalEOF", + "TK_NotStartEOF", + "TK_MiddleEOF", + "TK_StartToState", + "TK_AllToState", + "TK_FinalToState", + "TK_NotFinalToState", + "TK_NotStartToState", + "TK_MiddleToState", + "TK_StartFromState", + "TK_AllFromState", + "TK_FinalFromState", + "TK_NotFinalFromState", + "TK_NotStartFromState", + "TK_MiddleFromState", + "RE_Slash", + "RE_SqOpen", + "RE_SqOpenNeg", + "RE_SqClose", + "RE_Dot", + "RE_Star", + "RE_Dash", + "RE_Char", + "IL_WhiteSpace", + "IL_Comment", + "IL_Literal", + "IL_Symbol", + "KW_Machine", + "KW_Include", + "KW_Import", + "KW_Write", + "KW_Action", + "KW_AlphType", + "KW_Range", + "KW_GetKey", + "KW_InWhen", + "KW_When", + "KW_OutWhen", + "KW_Eof", + "KW_Err", + "KW_Lerr", + "KW_To", + "KW_From", + "KW_Export", + "KW_PrePush", + "KW_PostPop", + "KW_Length", + "KW_Break", + "KW_Exec", + "KW_Hold", + "KW_PChar", + "KW_Char", + "KW_Goto", + "KW_Call", + "KW_Ret", + "KW_CurState", + "KW_TargState", + "KW_Entry", + "KW_Next", + "KW_Variable", + "KW_Access", + "Parser_tk_eof", + "section_list", + "start", + "statement_list", + "statement", + "assignment", + "instantiation", + "action_spec", + "alphtype_spec", + "range_spec", + "getkey_spec", + "access_spec", + "variable_spec", + "export_block", + "pre_push_spec", + "post_pop_spec", + "length_spec", + "inline_block", + "export_open", + "opt_export", + "machine_name", + "join", + "join_or_lm", + "alphabet_num", + "inline_expr", + "opt_whitespace", + "lm_part_list", + "longest_match_part", + "opt_lm_part_action", + "action_embed", + "action_embed_block", + "expression", + "term_short", + "term", + "factor_with_label", + "factor_with_ep", + "local_state_ref", + "factor_with_aug", + "aug_type_base", + "priority_aug", + "priority_name", + "aug_type_cond", + "aug_type_to_state", + "aug_type_from_state", + "aug_type_eof", + "aug_type_gbl_error", + "aug_type_local_error", + "local_err_name", + "factor_with_rep", + "action_embed_word", + "priority_aug_num", + "factor_rep_num", + "factor_with_neg", + "factor", + "regular_expr_or_data", + "regular_expr", + "range_lit", + "regular_expr_item", + "regular_expr_char", + "regular_expr_or_char", + "inline_block_item", + "inline_block_interpret", + "inline_expr_any", + "inline_block_symbol", + "inline_expr_interpret", + "state_ref", + "inline_expr_item", + "inline_expr_symbol", + "no_name_sep", + "state_ref_names", + "opt_name_sep", + "_start" +}; + +#line 1449 "rlparse.kl" + + +void Parser::init() +{ + #line 3855 "rlparse.cpp" + curs = Parser_startState; + pool = 0; + block = (struct Parser_Block*) malloc( sizeof(struct Parser_Block) ); + block->next = 0; + freshEl = block->data; + #ifdef KELBT_LOG_ACTIONS + cerr << "allocating 8128 LangEls" << endl; + #endif + stackTop = freshEl; + stackTop->type = 0; + stackTop->state = -1; + stackTop->next = 0; + stackTop->child = 0; + stackTop->causeReduce = 0; + freshPos = 1; + lastFinal = stackTop; + numRetry = 0; + numNodes = 0; + errCount = 0; +#line 1454 "rlparse.kl" +} + +int Parser::parseLangEl( int type, const Token *token ) +{ + #line 3880 "rlparse.cpp" +#define reject() induceReject = 1 + + int pos, targState; + unsigned int *action; + int rhsLen; + struct Parser_LangEl *rhs[32]; + struct Parser_LangEl *lel = 0; + struct Parser_LangEl *input = 0; + struct Parser_LangEl *queue = 0; + char induceReject; + + if ( curs < 0 ) + return 0; + + if ( pool == 0 ) { + if ( freshPos == 8128 ) { + struct Parser_Block* newBlock = (struct Parser_Block*) malloc( sizeof(struct Parser_Block) ); + newBlock->next = block; + block = newBlock; + freshEl = newBlock->data; + #ifdef KELBT_LOG_ACTIONS + cerr << "allocating 8128 LangEls" << endl; + #endif + freshPos = 0; + } + queue = freshEl + freshPos++; + } + else { + queue = pool; + pool = pool->next; + } + numNodes += 1; + + queue->type = type; + queue->user.token = *token; + queue->next = 0; + queue->retry = 0; + queue->child = 0; + queue->causeReduce = 0; + +again: + if ( input == 0 ) { + if ( queue == 0 ) + goto _out; + + input = queue; + queue = queue->next; + input->next = 0; + } + + lel = input; + if ( lel->type < Parser_keys[curs<<1] || lel->type > Parser_keys[(curs<<1)+1] ) + goto parseError; + + pos = Parser_indicies[Parser_offsets[curs] + (lel->type - Parser_keys[curs<<1])]; + if ( pos < 0 ) + goto parseError; + + induceReject = 0; + targState = Parser_targs[pos]; + action = Parser_actions + Parser_actInds[pos]; + if ( lel->retry & 0x0000ffff ) + action += (lel->retry & 0x0000ffff); + + if ( *action & 0x1 ) { + #ifdef KELBT_LOG_ACTIONS + cerr << "shifted: " << Parser_lelNames[lel->type]; + #endif + input = input->next; + lel->state = curs; + lel->next = stackTop; + stackTop = lel; + + if ( action[1] == 0 ) + lel->retry &= 0xffff0000; + else { + lel->retry += 1; + numRetry += 1; + #ifdef KELBT_LOG_ACTIONS + cerr << " retry: " << stackTop; + #endif + } + #ifdef KELBT_LOG_ACTIONS + cerr << endl; + #endif + } + + if ( Parser_commitLen[pos] != 0 ) { + struct Parser_LangEl *commitHead = stackTop, *lel; + int sp = 0, doExec = 0; + #ifdef KELBT_LOG_ACTIONS + cerr << "commit encountered, executing final actions" << endl; + #endif + if ( Parser_commitLen[pos] < 0 ) + commitHead = commitHead->next; + + lel = commitHead; + +commit_head: + if ( lel == lastFinal ) { + doExec = 1; + goto commit_base; + } + + if ( lel->next != 0 ) { + sp += 1; + lel->next->prev = lel; + lel = lel->next; + lel->retry = 0; + goto commit_head; + } + +commit_reverse: + + if ( lel->child != 0 ) { + sp += 1; + lel->child->prev = lel; + lel = lel->child; + lel->retry = 1; + goto commit_head; + } + +commit_upwards: + + if ( doExec ) { + if ( lel->type < 226 ) { + } + else { + struct Parser_LangEl *redLel = lel; + if ( redLel->child != 0 ) { + int r = Parser_prodLengths[redLel->reduction] - 1; + struct Parser_LangEl *rhsEl = redLel->child; + while ( rhsEl != 0 ) { + rhs[r--] = rhsEl; + rhsEl = rhsEl->next; + } + } +switch ( lel->reduction ) { +case 17: { +#line 61 "rlparse.kl" + + LengthDef *lengthDef = new LengthDef( (&rhs[1]->user.token)->data ); + pd->lengthDefList.append( lengthDef ); + + /* Generic creation of machine for instantiation and assignment. */ + MachineDef *machineDef = new MachineDef( lengthDef ); + tryMachineDef( (&rhs[1]->user.token)->loc, (&rhs[1]->user.token)->data, machineDef, false ); + + +#line 4028 "rlparse.cpp" +} break; +case 18: { +#line 72 "rlparse.kl" + + if ( pd->prePushExpr != 0 ) { + /* Recover by just ignoring the duplicate. */ + error((&rhs[1]->user.token)->loc) << "pre_push code already defined" << endl; + } + + pd->prePushExpr = (&rhs[2]->user.inline_list)->inlineList; + + +#line 4041 "rlparse.cpp" +} break; +case 19: { +#line 84 "rlparse.kl" + + if ( pd->postPopExpr != 0 ) { + /* Recover by just ignoring the duplicate. */ + error((&rhs[1]->user.token)->loc) << "post_pop code already defined" << endl; + } + + pd->postPopExpr = (&rhs[2]->user.inline_list)->inlineList; + + +#line 4054 "rlparse.cpp" +} break; +case 20: { +#line 95 "rlparse.kl" + + exportContext.append( true ); + + +#line 4062 "rlparse.cpp" +} break; +case 21: { +#line 104 "rlparse.kl" + (&redLel->user.opt_export)->isSet = true; + +#line 4068 "rlparse.cpp" +} break; +case 22: { +#line 105 "rlparse.kl" + (&redLel->user.opt_export)->isSet = false; + +#line 4074 "rlparse.cpp" +} break; +case 23: { +#line 108 "rlparse.kl" + + exportContext.remove( exportContext.length()-1 ); + + +#line 4082 "rlparse.cpp" +} break; +case 24: { +#line 113 "rlparse.kl" + + /* Main machine must be an instance. */ + bool isInstance = false; + if ( strcmp((&rhs[1]->user.token_type)->token.data, mainMachine) == 0 ) { + warning((&rhs[1]->user.token_type)->token.loc) << + "main machine will be implicitly instantiated" << endl; + isInstance = true; + } + + /* Generic creation of machine for instantiation and assignment. */ + MachineDef *machineDef = new MachineDef( (&rhs[3]->user.join)->join ); + tryMachineDef( (&rhs[1]->user.token_type)->token.loc, (&rhs[1]->user.token_type)->token.data, machineDef, isInstance ); + + if ( (&rhs[0]->user.opt_export)->isSet ) + exportContext.remove( exportContext.length()-1 ); + + (&rhs[3]->user.join)->join->loc = (&rhs[2]->user.token)->loc; + + +#line 4105 "rlparse.cpp" +} break; +case 25: { +#line 133 "rlparse.kl" + + /* Generic creation of machine for instantiation and assignment. */ + tryMachineDef( (&rhs[1]->user.token_type)->token.loc, (&rhs[1]->user.token_type)->token.data, (&rhs[3]->user.join_or_lm)->machineDef, true ); + + if ( (&rhs[0]->user.opt_export)->isSet ) + exportContext.remove( exportContext.length()-1 ); + + /* Pass a location to join_or_lm */ + if ( (&rhs[3]->user.join_or_lm)->machineDef->join != 0 ) + (&rhs[3]->user.join_or_lm)->machineDef->join->loc = (&rhs[2]->user.token)->loc; + + +#line 4121 "rlparse.cpp" +} break; +case 26: { +#line 153 "rlparse.kl" + + /* Make/get the priority key. The name may have already been referenced + * and therefore exist. */ + PriorDictEl *priorDictEl; + if ( pd->priorDict.insert( (&rhs[0]->user.token)->data, pd->nextPriorKey, &priorDictEl ) ) + pd->nextPriorKey += 1; + pd->curDefPriorKey = priorDictEl->value; + + /* Make/get the local error key. */ + LocalErrDictEl *localErrDictEl; + if ( pd->localErrDict.insert( (&rhs[0]->user.token)->data, pd->nextLocalErrKey, &localErrDictEl ) ) + pd->nextLocalErrKey += 1; + pd->curDefLocalErrKey = localErrDictEl->value; + + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + + +#line 4142 "rlparse.cpp" +} break; +case 27: { +#line 171 "rlparse.kl" + + if ( pd->actionDict.find( (&rhs[1]->user.token)->data ) ) { + /* Recover by just ignoring the duplicate. */ + error((&rhs[1]->user.token)->loc) << "action \"" << (&rhs[1]->user.token)->data << "\" already defined" << endl; + } + else { + //cerr << "NEW ACTION " << $2->data << " " << $4->inlineList << endl; + /* Add the action to the list of actions. */ + Action *newAction = new Action( (&rhs[2]->user.token)->loc, (&rhs[1]->user.token)->data, + (&rhs[3]->user.inline_list)->inlineList, pd->nextCondId++ ); + + /* Insert to list and dict. */ + pd->actionList.append( newAction ); + pd->actionDict.insert( newAction ); + } + + +#line 4163 "rlparse.cpp" +} break; +case 28: { +#line 191 "rlparse.kl" + + if ( ! pd->setAlphType( (&rhs[0]->user.token)->loc, (&rhs[1]->user.token)->data, (&rhs[2]->user.token)->data ) ) { + // Recover by ignoring the alphtype statement. + error((&rhs[1]->user.token)->loc) << "\"" << (&rhs[1]->user.token)->data << + " " << (&rhs[2]->user.token)->data << "\" is not a valid alphabet type" << endl; + } + + +#line 4175 "rlparse.cpp" +} break; +case 29: { +#line 200 "rlparse.kl" + + if ( ! pd->setAlphType( (&rhs[0]->user.token)->loc, (&rhs[1]->user.token)->data ) ) { + // Recover by ignoring the alphtype statement. + error((&rhs[1]->user.token)->loc) << "\"" << (&rhs[1]->user.token)->data << + "\" is not a valid alphabet type" << endl; + } + + +#line 4187 "rlparse.cpp" +} break; +case 30: { +#line 210 "rlparse.kl" + + // Save the upper and lower ends of the range and emit the line number. + pd->lowerNum = (&rhs[1]->user.token_type)->token.data; + pd->upperNum = (&rhs[2]->user.token_type)->token.data; + pd->rangeLowLoc = (&rhs[1]->user.token_type)->token.loc; + pd->rangeHighLoc = (&rhs[2]->user.token_type)->token.loc; + + +#line 4199 "rlparse.cpp" +} break; +case 31: { +#line 219 "rlparse.kl" + + pd->getKeyExpr = (&rhs[1]->user.inline_list)->inlineList; + + +#line 4207 "rlparse.cpp" +} break; +case 32: { +#line 224 "rlparse.kl" + + pd->accessExpr = (&rhs[1]->user.inline_list)->inlineList; + + +#line 4215 "rlparse.cpp" +} break; +case 33: { +#line 229 "rlparse.kl" + + /* FIXME: Need to implement the rest of this. */ + bool wasSet = pd->setVariable( (&rhs[2]->user.token)->data, (&rhs[3]->user.inline_list)->inlineList ); + if ( !wasSet ) + error((&rhs[2]->user.token)->loc) << "bad variable name" << endl; + + +#line 4226 "rlparse.cpp" +} break; +case 36: { +#line 249 "rlparse.kl" + + (&redLel->user.join_or_lm)->machineDef = new MachineDef( (&rhs[0]->user.join)->join ); + + +#line 4234 "rlparse.cpp" +} break; +case 37: { +#line 253 "rlparse.kl" + + /* Create a new factor going to a longest match structure. Record + * in the parse data that we have a longest match. */ + LongestMatch *lm = new LongestMatch( (&rhs[0]->user.token)->loc, (&rhs[1]->user.lm_part_list)->lmPartList ); + pd->lmList.append( lm ); + for ( LmPartList::Iter lmp = *((&rhs[1]->user.lm_part_list)->lmPartList); lmp.lte(); lmp++ ) + lmp->longestMatch = lm; + (&redLel->user.join_or_lm)->machineDef = new MachineDef( lm ); + + +#line 4248 "rlparse.cpp" +} break; +case 38: { +#line 270 "rlparse.kl" + + if ( (&rhs[1]->user.longest_match_part)->lmPart != 0 ) + (&rhs[0]->user.lm_part_list)->lmPartList->append( (&rhs[1]->user.longest_match_part)->lmPart ); + (&redLel->user.lm_part_list)->lmPartList = (&rhs[0]->user.lm_part_list)->lmPartList; + + +#line 4258 "rlparse.cpp" +} break; +case 39: { +#line 277 "rlparse.kl" + + /* Create a new list with the part. */ + (&redLel->user.lm_part_list)->lmPartList = new LmPartList; + if ( (&rhs[0]->user.longest_match_part)->lmPart != 0 ) + (&redLel->user.lm_part_list)->lmPartList->append( (&rhs[0]->user.longest_match_part)->lmPart ); + + +#line 4269 "rlparse.cpp" +} break; +case 40: { +#line 290 "rlparse.kl" + (&redLel->user.longest_match_part)->lmPart = 0; + +#line 4275 "rlparse.cpp" +} break; +case 41: { +#line 292 "rlparse.kl" + (&redLel->user.longest_match_part)->lmPart = 0; + +#line 4281 "rlparse.cpp" +} break; +case 42: { +#line 294 "rlparse.kl" + + (&redLel->user.longest_match_part)->lmPart = 0; + Action *action = (&rhs[1]->user.opt_lm_part_action)->action; + if ( action != 0 ) + action->isLmAction = true; + (&redLel->user.longest_match_part)->lmPart = new LongestMatchPart( (&rhs[0]->user.join)->join, action, + (&rhs[2]->user.token)->loc, pd->nextLongestMatchId++ ); + + /* Provide a location to join. Unfortunately We don't + * have the start of the join as in other occurances. Use the end. */ + (&rhs[0]->user.join)->join->loc = (&rhs[2]->user.token)->loc; + + +#line 4298 "rlparse.cpp" +} break; +case 43: { +#line 313 "rlparse.kl" + + (&redLel->user.opt_lm_part_action)->action = (&rhs[1]->user.action_ref)->action; + + +#line 4306 "rlparse.cpp" +} break; +case 44: { +#line 317 "rlparse.kl" + + (&redLel->user.opt_lm_part_action)->action = (&rhs[0]->user.action_ref)->action; + + +#line 4314 "rlparse.cpp" +} break; +case 45: { +#line 321 "rlparse.kl" + + (&redLel->user.opt_lm_part_action)->action = 0; + + +#line 4322 "rlparse.cpp" +} break; +case 46: { +#line 332 "rlparse.kl" + + /* Append the expression to the list and return it. */ + (&rhs[0]->user.join)->join->exprList.append( (&rhs[2]->user.expression)->expression ); + (&redLel->user.join)->join = (&rhs[0]->user.join)->join; + + +#line 4332 "rlparse.cpp" +} break; +case 47: { +#line 338 "rlparse.kl" + + (&redLel->user.join)->join = new Join( (&rhs[0]->user.expression)->expression ); + + +#line 4340 "rlparse.cpp" +} break; +case 48: { +#line 348 "rlparse.kl" + + (&redLel->user.expression)->expression = new Expression( (&rhs[0]->user.expression)->expression, + (&rhs[2]->user.term_short)->term, Expression::OrType ); + + +#line 4349 "rlparse.cpp" +} break; +case 49: { +#line 353 "rlparse.kl" + + (&redLel->user.expression)->expression = new Expression( (&rhs[0]->user.expression)->expression, + (&rhs[2]->user.term_short)->term, Expression::IntersectType ); + + +#line 4358 "rlparse.cpp" +} break; +case 50: { +#line 358 "rlparse.kl" + + (&redLel->user.expression)->expression = new Expression( (&rhs[0]->user.expression)->expression, + (&rhs[2]->user.term_short)->term, Expression::SubtractType ); + + +#line 4367 "rlparse.cpp" +} break; +case 51: { +#line 363 "rlparse.kl" + + (&redLel->user.expression)->expression = new Expression( (&rhs[0]->user.expression)->expression, + (&rhs[2]->user.term_short)->term, Expression::StrongSubtractType ); + + +#line 4376 "rlparse.cpp" +} break; +case 52: { +#line 368 "rlparse.kl" + + (&redLel->user.expression)->expression = new Expression( (&rhs[0]->user.term_short)->term ); + + +#line 4384 "rlparse.cpp" +} break; +case 53: { +#line 389 "rlparse.kl" + + (&redLel->user.term_short)->term = (&rhs[0]->user.term)->term; + + +#line 4392 "rlparse.cpp" +} break; +case 54: { +#line 399 "rlparse.kl" + + (&redLel->user.term)->term = new Term( (&rhs[0]->user.term)->term, (&rhs[1]->user.factor_with_label)->factorWithAug ); + + +#line 4400 "rlparse.cpp" +} break; +case 55: { +#line 403 "rlparse.kl" + + (&redLel->user.term)->term = new Term( (&rhs[0]->user.term)->term, (&rhs[2]->user.factor_with_label)->factorWithAug ); + + +#line 4408 "rlparse.cpp" +} break; +case 56: { +#line 407 "rlparse.kl" + + (&redLel->user.term)->term = new Term( (&rhs[0]->user.term)->term, (&rhs[2]->user.factor_with_label)->factorWithAug, Term::RightStartType ); + + +#line 4416 "rlparse.cpp" +} break; +case 57: { +#line 411 "rlparse.kl" + + (&redLel->user.term)->term = new Term( (&rhs[0]->user.term)->term, (&rhs[2]->user.factor_with_label)->factorWithAug, Term::RightFinishType ); + + +#line 4424 "rlparse.cpp" +} break; +case 58: { +#line 415 "rlparse.kl" + + (&redLel->user.term)->term = new Term( (&rhs[0]->user.term)->term, + (&rhs[2]->user.factor_with_label)->factorWithAug, Term::LeftType ); + + +#line 4433 "rlparse.cpp" +} break; +case 59: { +#line 420 "rlparse.kl" + + (&redLel->user.term)->term = new Term( (&rhs[0]->user.factor_with_label)->factorWithAug ); + + +#line 4441 "rlparse.cpp" +} break; +case 60: { +#line 430 "rlparse.kl" + + /* Add the label to the list and pass the factor up. */ + (&rhs[2]->user.factor_with_label)->factorWithAug->labels.prepend( Label((&rhs[0]->user.token)->loc, (&rhs[0]->user.token)->data) ); + (&redLel->user.factor_with_label)->factorWithAug = (&rhs[2]->user.factor_with_label)->factorWithAug; + + +#line 4451 "rlparse.cpp" +} break; +case 61: { +#line 436 "rlparse.kl" + + (&redLel->user.factor_with_label)->factorWithAug = (&rhs[0]->user.factor_with_ep)->factorWithAug; + + +#line 4459 "rlparse.cpp" +} break; +case 62: { +#line 446 "rlparse.kl" + + /* Add the target to the list and return the factor object. */ + (&rhs[0]->user.factor_with_ep)->factorWithAug->epsilonLinks.append( EpsilonLink( (&rhs[1]->user.token)->loc, nameRef ) ); + (&redLel->user.factor_with_ep)->factorWithAug = (&rhs[0]->user.factor_with_ep)->factorWithAug; + + +#line 4469 "rlparse.cpp" +} break; +case 63: { +#line 452 "rlparse.kl" + + (&redLel->user.factor_with_ep)->factorWithAug = (&rhs[0]->user.factor_with_aug)->factorWithAug; + + +#line 4477 "rlparse.cpp" +} break; +case 64: { +#line 462 "rlparse.kl" + + /* Append the action to the factorWithAug, record the refernce from + * factorWithAug to the action and pass up the factorWithAug. */ + (&rhs[0]->user.factor_with_aug)->factorWithAug->actions.append( + ParserAction( (&rhs[1]->user.aug_type)->loc, (&rhs[1]->user.aug_type)->augType, 0, (&rhs[2]->user.action_ref)->action ) ); + (&redLel->user.factor_with_aug)->factorWithAug = (&rhs[0]->user.factor_with_aug)->factorWithAug; + + +#line 4489 "rlparse.cpp" +} break; +case 65: { +#line 470 "rlparse.kl" + + /* Append the named priority to the factorWithAug and pass it up. */ + (&rhs[0]->user.factor_with_aug)->factorWithAug->priorityAugs.append( + PriorityAug( (&rhs[1]->user.aug_type)->augType, pd->curDefPriorKey, (&rhs[2]->user.priority_aug)->priorityNum ) ); + (&redLel->user.factor_with_aug)->factorWithAug = (&rhs[0]->user.factor_with_aug)->factorWithAug; + + +#line 4500 "rlparse.cpp" +} break; +case 66: { +#line 477 "rlparse.kl" + + /* Append the priority using a default name. */ + (&rhs[0]->user.factor_with_aug)->factorWithAug->priorityAugs.append( + PriorityAug( (&rhs[1]->user.aug_type)->augType, (&rhs[3]->user.priority_name)->priorityName, (&rhs[5]->user.priority_aug)->priorityNum ) ); + (&redLel->user.factor_with_aug)->factorWithAug = (&rhs[0]->user.factor_with_aug)->factorWithAug; + + +#line 4511 "rlparse.cpp" +} break; +case 67: { +#line 484 "rlparse.kl" + + (&rhs[0]->user.factor_with_aug)->factorWithAug->conditions.append( ConditionTest( (&rhs[1]->user.aug_type)->loc, + (&rhs[1]->user.aug_type)->augType, (&rhs[2]->user.action_ref)->action, true ) ); + (&redLel->user.factor_with_aug)->factorWithAug = (&rhs[0]->user.factor_with_aug)->factorWithAug; + + +#line 4521 "rlparse.cpp" +} break; +case 68: { +#line 490 "rlparse.kl" + + (&rhs[0]->user.factor_with_aug)->factorWithAug->conditions.append( ConditionTest( (&rhs[1]->user.aug_type)->loc, + (&rhs[1]->user.aug_type)->augType, (&rhs[3]->user.action_ref)->action, false ) ); + (&redLel->user.factor_with_aug)->factorWithAug = (&rhs[0]->user.factor_with_aug)->factorWithAug; + + +#line 4531 "rlparse.cpp" +} break; +case 69: { +#line 496 "rlparse.kl" + + /* Append the action, pass it up. */ + (&rhs[0]->user.factor_with_aug)->factorWithAug->actions.append( ParserAction( (&rhs[1]->user.aug_type)->loc, + (&rhs[1]->user.aug_type)->augType, 0, (&rhs[2]->user.action_ref)->action ) ); + (&redLel->user.factor_with_aug)->factorWithAug = (&rhs[0]->user.factor_with_aug)->factorWithAug; + + +#line 4542 "rlparse.cpp" +} break; +case 70: { +#line 503 "rlparse.kl" + + /* Append the action, pass it up. */ + (&rhs[0]->user.factor_with_aug)->factorWithAug->actions.append( ParserAction( (&rhs[1]->user.aug_type)->loc, + (&rhs[1]->user.aug_type)->augType, 0, (&rhs[2]->user.action_ref)->action ) ); + (&redLel->user.factor_with_aug)->factorWithAug = (&rhs[0]->user.factor_with_aug)->factorWithAug; + + +#line 4553 "rlparse.cpp" +} break; +case 71: { +#line 510 "rlparse.kl" + + /* Append the action, pass it up. */ + (&rhs[0]->user.factor_with_aug)->factorWithAug->actions.append( ParserAction( (&rhs[1]->user.aug_type)->loc, + (&rhs[1]->user.aug_type)->augType, 0, (&rhs[2]->user.action_ref)->action ) ); + (&redLel->user.factor_with_aug)->factorWithAug = (&rhs[0]->user.factor_with_aug)->factorWithAug; + + +#line 4564 "rlparse.cpp" +} break; +case 72: { +#line 517 "rlparse.kl" + + /* Append the action to the factorWithAug, record the refernce from + * factorWithAug to the action and pass up the factorWithAug. */ + (&rhs[0]->user.factor_with_aug)->factorWithAug->actions.append( ParserAction( (&rhs[1]->user.aug_type)->loc, + (&rhs[1]->user.aug_type)->augType, pd->curDefLocalErrKey, (&rhs[2]->user.action_ref)->action ) ); + (&redLel->user.factor_with_aug)->factorWithAug = (&rhs[0]->user.factor_with_aug)->factorWithAug; + + +#line 4576 "rlparse.cpp" +} break; +case 73: { +#line 525 "rlparse.kl" + + /* Append the action to the factorWithAug, record the refernce from + * factorWithAug to the action and pass up the factorWithAug. */ + (&rhs[0]->user.factor_with_aug)->factorWithAug->actions.append( ParserAction( (&rhs[1]->user.aug_type)->loc, + (&rhs[1]->user.aug_type)->augType, pd->curDefLocalErrKey, (&rhs[2]->user.action_ref)->action ) ); + (&redLel->user.factor_with_aug)->factorWithAug = (&rhs[0]->user.factor_with_aug)->factorWithAug; + + +#line 4588 "rlparse.cpp" +} break; +case 74: { +#line 533 "rlparse.kl" + + /* Append the action to the factorWithAug, record the refernce from + * factorWithAug to the action and pass up the factorWithAug. */ + (&rhs[0]->user.factor_with_aug)->factorWithAug->actions.append( ParserAction( (&rhs[1]->user.aug_type)->loc, + (&rhs[1]->user.aug_type)->augType, (&rhs[3]->user.local_err_name)->error_name, (&rhs[5]->user.action_ref)->action ) ); + (&redLel->user.factor_with_aug)->factorWithAug = (&rhs[0]->user.factor_with_aug)->factorWithAug; + + +#line 4600 "rlparse.cpp" +} break; +case 75: { +#line 541 "rlparse.kl" + + (&redLel->user.factor_with_aug)->factorWithAug = new FactorWithAug( (&rhs[0]->user.factor_with_rep)->factorWithRep ); + + +#line 4608 "rlparse.cpp" +} break; +case 76: { +#line 554 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_finish; + +#line 4614 "rlparse.cpp" +} break; +case 77: { +#line 555 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_leave; + +#line 4620 "rlparse.cpp" +} break; +case 78: { +#line 556 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_all; + +#line 4626 "rlparse.cpp" +} break; +case 79: { +#line 557 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_start; + +#line 4632 "rlparse.cpp" +} break; +case 80: { +#line 562 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_start; + +#line 4638 "rlparse.cpp" +} break; +case 81: { +#line 563 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_start; + +#line 4644 "rlparse.cpp" +} break; +case 82: { +#line 564 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_all; + +#line 4650 "rlparse.cpp" +} break; +case 83: { +#line 565 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_all; + +#line 4656 "rlparse.cpp" +} break; +case 84: { +#line 566 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_leave; + +#line 4662 "rlparse.cpp" +} break; +case 85: { +#line 567 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_leave; + +#line 4668 "rlparse.cpp" +} break; +case 86: { +#line 568 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_all; + +#line 4674 "rlparse.cpp" +} break; +case 87: { +#line 569 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_start; + +#line 4680 "rlparse.cpp" +} break; +case 88: { +#line 570 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_leave; + +#line 4686 "rlparse.cpp" +} break; +case 89: { +#line 579 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_start_to_state; + +#line 4692 "rlparse.cpp" +} break; +case 90: { +#line 581 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_start_to_state; + +#line 4698 "rlparse.cpp" +} break; +case 91: { +#line 584 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_start_to_state; + +#line 4704 "rlparse.cpp" +} break; +case 92: { +#line 586 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_start_to_state; + +#line 4710 "rlparse.cpp" +} break; +case 93: { +#line 589 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_all_to_state; + +#line 4716 "rlparse.cpp" +} break; +case 94: { +#line 591 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_all_to_state; + +#line 4722 "rlparse.cpp" +} break; +case 95: { +#line 594 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_final_to_state; + +#line 4728 "rlparse.cpp" +} break; +case 96: { +#line 596 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_final_to_state; + +#line 4734 "rlparse.cpp" +} break; +case 97: { +#line 599 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_final_to_state; + +#line 4740 "rlparse.cpp" +} break; +case 98: { +#line 601 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_final_to_state; + +#line 4746 "rlparse.cpp" +} break; +case 99: { +#line 604 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_middle_to_state; + +#line 4752 "rlparse.cpp" +} break; +case 100: { +#line 606 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_middle_to_state; + +#line 4758 "rlparse.cpp" +} break; +case 101: { +#line 615 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_start_from_state; + +#line 4764 "rlparse.cpp" +} break; +case 102: { +#line 617 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_start_from_state; + +#line 4770 "rlparse.cpp" +} break; +case 103: { +#line 620 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_start_from_state; + +#line 4776 "rlparse.cpp" +} break; +case 104: { +#line 622 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_start_from_state; + +#line 4782 "rlparse.cpp" +} break; +case 105: { +#line 625 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_all_from_state; + +#line 4788 "rlparse.cpp" +} break; +case 106: { +#line 627 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_all_from_state; + +#line 4794 "rlparse.cpp" +} break; +case 107: { +#line 630 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_final_from_state; + +#line 4800 "rlparse.cpp" +} break; +case 108: { +#line 632 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_final_from_state; + +#line 4806 "rlparse.cpp" +} break; +case 109: { +#line 635 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_final_from_state; + +#line 4812 "rlparse.cpp" +} break; +case 110: { +#line 637 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_final_from_state; + +#line 4818 "rlparse.cpp" +} break; +case 111: { +#line 640 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_middle_from_state; + +#line 4824 "rlparse.cpp" +} break; +case 112: { +#line 642 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_middle_from_state; + +#line 4830 "rlparse.cpp" +} break; +case 113: { +#line 651 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_start_eof; + +#line 4836 "rlparse.cpp" +} break; +case 114: { +#line 653 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_start_eof; + +#line 4842 "rlparse.cpp" +} break; +case 115: { +#line 656 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_start_eof; + +#line 4848 "rlparse.cpp" +} break; +case 116: { +#line 658 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_start_eof; + +#line 4854 "rlparse.cpp" +} break; +case 117: { +#line 661 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_all_eof; + +#line 4860 "rlparse.cpp" +} break; +case 118: { +#line 663 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_all_eof; + +#line 4866 "rlparse.cpp" +} break; +case 119: { +#line 666 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_final_eof; + +#line 4872 "rlparse.cpp" +} break; +case 120: { +#line 668 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_final_eof; + +#line 4878 "rlparse.cpp" +} break; +case 121: { +#line 671 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_final_eof; + +#line 4884 "rlparse.cpp" +} break; +case 122: { +#line 673 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_final_eof; + +#line 4890 "rlparse.cpp" +} break; +case 123: { +#line 676 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_middle_eof; + +#line 4896 "rlparse.cpp" +} break; +case 124: { +#line 678 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_middle_eof; + +#line 4902 "rlparse.cpp" +} break; +case 125: { +#line 687 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_start_gbl_error; + +#line 4908 "rlparse.cpp" +} break; +case 126: { +#line 689 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_start_gbl_error; + +#line 4914 "rlparse.cpp" +} break; +case 127: { +#line 692 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_start_gbl_error; + +#line 4920 "rlparse.cpp" +} break; +case 128: { +#line 694 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_start_gbl_error; + +#line 4926 "rlparse.cpp" +} break; +case 129: { +#line 697 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_all_gbl_error; + +#line 4932 "rlparse.cpp" +} break; +case 130: { +#line 699 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_all_gbl_error; + +#line 4938 "rlparse.cpp" +} break; +case 131: { +#line 702 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_final_gbl_error; + +#line 4944 "rlparse.cpp" +} break; +case 132: { +#line 704 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_final_gbl_error; + +#line 4950 "rlparse.cpp" +} break; +case 133: { +#line 707 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_final_gbl_error; + +#line 4956 "rlparse.cpp" +} break; +case 134: { +#line 709 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_final_gbl_error; + +#line 4962 "rlparse.cpp" +} break; +case 135: { +#line 712 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_middle_gbl_error; + +#line 4968 "rlparse.cpp" +} break; +case 136: { +#line 714 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_middle_gbl_error; + +#line 4974 "rlparse.cpp" +} break; +case 137: { +#line 724 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_start_local_error; + +#line 4980 "rlparse.cpp" +} break; +case 138: { +#line 726 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_start_local_error; + +#line 4986 "rlparse.cpp" +} break; +case 139: { +#line 729 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_start_local_error; + +#line 4992 "rlparse.cpp" +} break; +case 140: { +#line 731 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_start_local_error; + +#line 4998 "rlparse.cpp" +} break; +case 141: { +#line 734 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_all_local_error; + +#line 5004 "rlparse.cpp" +} break; +case 142: { +#line 736 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_all_local_error; + +#line 5010 "rlparse.cpp" +} break; +case 143: { +#line 739 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_final_local_error; + +#line 5016 "rlparse.cpp" +} break; +case 144: { +#line 741 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_final_local_error; + +#line 5022 "rlparse.cpp" +} break; +case 145: { +#line 744 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_final_local_error; + +#line 5028 "rlparse.cpp" +} break; +case 146: { +#line 746 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_not_final_local_error; + +#line 5034 "rlparse.cpp" +} break; +case 147: { +#line 749 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_middle_local_error; + +#line 5040 "rlparse.cpp" +} break; +case 148: { +#line 751 "rlparse.kl" + (&redLel->user.aug_type)->loc = (&rhs[0]->user.token)->loc; (&redLel->user.aug_type)->augType = at_middle_local_error; + +#line 5046 "rlparse.cpp" +} break; +case 149: { +#line 764 "rlparse.kl" + (&redLel->user.action_ref)->action = (&rhs[0]->user.action_ref)->action; + +#line 5052 "rlparse.cpp" +} break; +case 150: { +#line 765 "rlparse.kl" + (&redLel->user.action_ref)->action = (&rhs[1]->user.action_ref)->action; + +#line 5058 "rlparse.cpp" +} break; +case 151: { +#line 766 "rlparse.kl" + (&redLel->user.action_ref)->action = (&rhs[0]->user.action_ref)->action; + +#line 5064 "rlparse.cpp" +} break; +case 152: { +#line 771 "rlparse.kl" + + /* Set the name in the actionDict. */ + Action *action = pd->actionDict.find( (&rhs[0]->user.token)->data ); + if ( action != 0 ) { + /* Pass up the action element */ + (&redLel->user.action_ref)->action = action; + } + else { + /* Will recover by returning null as the action. */ + error((&rhs[0]->user.token)->loc) << "action lookup of \"" << (&rhs[0]->user.token)->data << "\" failed" << endl; + (&redLel->user.action_ref)->action = 0; + } + + +#line 5082 "rlparse.cpp" +} break; +case 153: { +#line 788 "rlparse.kl" + + /* Create the action, add it to the list and pass up. */ + Action *newAction = new Action( (&rhs[0]->user.token)->loc, 0, (&rhs[1]->user.inline_list)->inlineList, pd->nextCondId++ ); + pd->actionList.append( newAction ); + (&redLel->user.action_ref)->action = newAction; + + +#line 5093 "rlparse.cpp" +} break; +case 154: { +#line 803 "rlparse.kl" + + // Lookup/create the priority key. + PriorDictEl *priorDictEl; + if ( pd->priorDict.insert( (&rhs[0]->user.token)->data, pd->nextPriorKey, &priorDictEl ) ) + pd->nextPriorKey += 1; + + // Use the inserted/found priority key. + (&redLel->user.priority_name)->priorityName = priorDictEl->value; + + +#line 5107 "rlparse.cpp" +} break; +case 155: { +#line 820 "rlparse.kl" + + // Convert the priority number to a long. Check for overflow. + errno = 0; + //cerr << "PRIOR AUG: " << $1->token.data << endl; + long aug = strtol( (&rhs[0]->user.token_type)->token.data, 0, 10 ); + if ( errno == ERANGE && aug == LONG_MAX ) { + /* Priority number too large. Recover by setting the priority to 0. */ + error((&rhs[0]->user.token_type)->token.loc) << "priority number " << (&rhs[0]->user.token_type)->token.data << + " overflows" << endl; + (&redLel->user.priority_aug)->priorityNum = 0; + } + else if ( errno == ERANGE && aug == LONG_MIN ) { + /* Priority number too large in the neg. Recover by using 0. */ + error((&rhs[0]->user.token_type)->token.loc) << "priority number " << (&rhs[0]->user.token_type)->token.data << + " underflows" << endl; + (&redLel->user.priority_aug)->priorityNum = 0; + } + else { + /* No overflow or underflow. */ + (&redLel->user.priority_aug)->priorityNum = aug; + } + + +#line 5134 "rlparse.cpp" +} break; +case 156: { +#line 846 "rlparse.kl" + + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + + +#line 5142 "rlparse.cpp" +} break; +case 157: { +#line 850 "rlparse.kl" + + (&redLel->user.token_type)->token.set( "+", 1 ); + (&redLel->user.token_type)->token.loc = (&rhs[0]->user.token)->loc; + (&redLel->user.token_type)->token.append( *(&rhs[1]->user.token) ); + + +#line 5152 "rlparse.cpp" +} break; +case 158: { +#line 856 "rlparse.kl" + + (&redLel->user.token_type)->token.set( "-", 1 ); + (&redLel->user.token_type)->token.loc = (&rhs[0]->user.token)->loc; + (&redLel->user.token_type)->token.append( *(&rhs[1]->user.token) ); + + +#line 5162 "rlparse.cpp" +} break; +case 159: { +#line 868 "rlparse.kl" + + /* Lookup/create the priority key. */ + LocalErrDictEl *localErrDictEl; + if ( pd->localErrDict.insert( (&rhs[0]->user.token)->data, pd->nextLocalErrKey, &localErrDictEl ) ) + pd->nextLocalErrKey += 1; + + /* Use the inserted/found priority key. */ + (&redLel->user.local_err_name)->error_name = localErrDictEl->value; + + +#line 5176 "rlparse.cpp" +} break; +case 160: { +#line 889 "rlparse.kl" + + (&redLel->user.factor_with_rep)->factorWithRep = new FactorWithRep( (&rhs[1]->user.token)->loc, (&rhs[0]->user.factor_with_rep)->factorWithRep, + 0, 0, FactorWithRep::StarType ); + + +#line 5185 "rlparse.cpp" +} break; +case 161: { +#line 894 "rlparse.kl" + + (&redLel->user.factor_with_rep)->factorWithRep = new FactorWithRep( (&rhs[1]->user.token)->loc, (&rhs[0]->user.factor_with_rep)->factorWithRep, + 0, 0, FactorWithRep::StarStarType ); + + +#line 5194 "rlparse.cpp" +} break; +case 162: { +#line 899 "rlparse.kl" + + (&redLel->user.factor_with_rep)->factorWithRep = new FactorWithRep( (&rhs[1]->user.token)->loc, (&rhs[0]->user.factor_with_rep)->factorWithRep, + 0, 0, FactorWithRep::OptionalType ); + + +#line 5203 "rlparse.cpp" +} break; +case 163: { +#line 904 "rlparse.kl" + + (&redLel->user.factor_with_rep)->factorWithRep = new FactorWithRep( (&rhs[1]->user.token)->loc, (&rhs[0]->user.factor_with_rep)->factorWithRep, + 0, 0, FactorWithRep::PlusType ); + + +#line 5212 "rlparse.cpp" +} break; +case 164: { +#line 909 "rlparse.kl" + + (&redLel->user.factor_with_rep)->factorWithRep = new FactorWithRep( (&rhs[1]->user.token)->loc, (&rhs[0]->user.factor_with_rep)->factorWithRep, + (&rhs[2]->user.factor_rep_num)->rep, 0, FactorWithRep::ExactType ); + + +#line 5221 "rlparse.cpp" +} break; +case 165: { +#line 914 "rlparse.kl" + + (&redLel->user.factor_with_rep)->factorWithRep = new FactorWithRep( (&rhs[1]->user.token)->loc, (&rhs[0]->user.factor_with_rep)->factorWithRep, + 0, (&rhs[3]->user.factor_rep_num)->rep, FactorWithRep::MaxType ); + + +#line 5230 "rlparse.cpp" +} break; +case 166: { +#line 919 "rlparse.kl" + + (&redLel->user.factor_with_rep)->factorWithRep = new FactorWithRep( (&rhs[1]->user.token)->loc, (&rhs[0]->user.factor_with_rep)->factorWithRep, + (&rhs[2]->user.factor_rep_num)->rep, 0, FactorWithRep::MinType ); + + +#line 5239 "rlparse.cpp" +} break; +case 167: { +#line 924 "rlparse.kl" + + (&redLel->user.factor_with_rep)->factorWithRep = new FactorWithRep( (&rhs[1]->user.token)->loc, (&rhs[0]->user.factor_with_rep)->factorWithRep, + (&rhs[2]->user.factor_rep_num)->rep, (&rhs[4]->user.factor_rep_num)->rep, FactorWithRep::RangeType ); + + +#line 5248 "rlparse.cpp" +} break; +case 168: { +#line 929 "rlparse.kl" + + (&redLel->user.factor_with_rep)->factorWithRep = new FactorWithRep( (&rhs[0]->user.factor_with_neg)->factorWithNeg ); + + +#line 5256 "rlparse.cpp" +} break; +case 169: { +#line 939 "rlparse.kl" + + // Convert the priority number to a long. Check for overflow. + errno = 0; + long rep = strtol( (&rhs[0]->user.token)->data, 0, 10 ); + if ( errno == ERANGE && rep == LONG_MAX ) { + // Repetition too large. Recover by returing repetition 1. */ + error((&rhs[0]->user.token)->loc) << "repetition number " << (&rhs[0]->user.token)->data << " overflows" << endl; + (&redLel->user.factor_rep_num)->rep = 1; + } + else { + // Cannot be negative, so no overflow. + (&redLel->user.factor_rep_num)->rep = rep; + } + + +#line 5275 "rlparse.cpp" +} break; +case 170: { +#line 965 "rlparse.kl" + + (&redLel->user.factor_with_neg)->factorWithNeg = new FactorWithNeg( (&rhs[0]->user.token)->loc, + (&rhs[1]->user.factor_with_neg)->factorWithNeg, FactorWithNeg::NegateType ); + + +#line 5284 "rlparse.cpp" +} break; +case 171: { +#line 970 "rlparse.kl" + + (&redLel->user.factor_with_neg)->factorWithNeg = new FactorWithNeg( (&rhs[0]->user.token)->loc, + (&rhs[1]->user.factor_with_neg)->factorWithNeg, FactorWithNeg::CharNegateType ); + + +#line 5293 "rlparse.cpp" +} break; +case 172: { +#line 975 "rlparse.kl" + + (&redLel->user.factor_with_neg)->factorWithNeg = new FactorWithNeg( (&rhs[0]->user.factor)->factor ); + + +#line 5301 "rlparse.cpp" +} break; +case 173: { +#line 985 "rlparse.kl" + + /* Create a new factor node going to a concat literal. */ + (&redLel->user.factor)->factor = new Factor( new Literal( *(&rhs[0]->user.token), Literal::LitString ) ); + + +#line 5310 "rlparse.cpp" +} break; +case 174: { +#line 990 "rlparse.kl" + + /* Create a new factor node going to a literal number. */ + (&redLel->user.factor)->factor = new Factor( new Literal( (&rhs[0]->user.token_type)->token, Literal::Number ) ); + + +#line 5319 "rlparse.cpp" +} break; +case 175: { +#line 995 "rlparse.kl" + + /* Find the named graph. */ + GraphDictEl *gdNode = pd->graphDict.find( (&rhs[0]->user.token)->data ); + if ( gdNode == 0 ) { + /* Recover by returning null as the factor node. */ + error((&rhs[0]->user.token)->loc) << "graph lookup of \"" << (&rhs[0]->user.token)->data << "\" failed" << endl; + (&redLel->user.factor)->factor = 0; + } + else if ( gdNode->isInstance ) { + /* Recover by retuning null as the factor node. */ + error((&rhs[0]->user.token)->loc) << "references to graph instantiations not allowed " + "in expressions" << endl; + (&redLel->user.factor)->factor = 0; + } + else { + /* Create a factor node that is a lookup of an expression. */ + (&redLel->user.factor)->factor = new Factor( (&rhs[0]->user.token)->loc, gdNode->value ); + } + + +#line 5343 "rlparse.cpp" +} break; +case 176: { +#line 1015 "rlparse.kl" + + /* Create a new factor node going to an OR expression. */ + (&redLel->user.factor)->factor = new Factor( new ReItem( (&rhs[0]->user.token)->loc, (&rhs[1]->user.regular_expr_or_data)->reOrBlock, ReItem::OrBlock ) ); + + +#line 5352 "rlparse.cpp" +} break; +case 177: { +#line 1020 "rlparse.kl" + + /* Create a new factor node going to a negated OR expression. */ + (&redLel->user.factor)->factor = new Factor( new ReItem( (&rhs[0]->user.token)->loc, (&rhs[1]->user.regular_expr_or_data)->reOrBlock, ReItem::NegOrBlock ) ); + + +#line 5361 "rlparse.cpp" +} break; +case 178: { +#line 1025 "rlparse.kl" + + if ( (&rhs[2]->user.token)->length > 1 ) { + for ( char *p = (&rhs[2]->user.token)->data; *p != 0; p++ ) { + if ( *p == 'i' ) + (&rhs[1]->user.regular_expr)->regExpr->caseInsensitive = true; + } + } + + /* Create a new factor node going to a regular exp. */ + (&redLel->user.factor)->factor = new Factor( (&rhs[1]->user.regular_expr)->regExpr ); + + +#line 5377 "rlparse.cpp" +} break; +case 179: { +#line 1037 "rlparse.kl" + + /* Create a new factor node going to a range. */ + (&redLel->user.factor)->factor = new Factor( new Range( (&rhs[0]->user.range_lit)->literal, (&rhs[2]->user.range_lit)->literal ) ); + + +#line 5386 "rlparse.cpp" +} break; +case 180: { +#line 1042 "rlparse.kl" + + /* Create a new factor going to a parenthesized join. */ + (&redLel->user.factor)->factor = new Factor( (&rhs[1]->user.join)->join ); + (&rhs[1]->user.join)->join->loc = (&rhs[0]->user.token)->loc; + + +#line 5396 "rlparse.cpp" +} break; +case 181: { +#line 1055 "rlparse.kl" + + /* Range literas must have only one char. We restrict this in the parse tree. */ + (&redLel->user.range_lit)->literal = new Literal( *(&rhs[0]->user.token), Literal::LitString ); + + +#line 5405 "rlparse.cpp" +} break; +case 182: { +#line 1060 "rlparse.kl" + + /* Create a new literal number. */ + (&redLel->user.range_lit)->literal = new Literal( (&rhs[0]->user.token_type)->token, Literal::Number ); + + +#line 5414 "rlparse.cpp" +} break; +case 183: { +#line 1069 "rlparse.kl" + + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + + +#line 5422 "rlparse.cpp" +} break; +case 184: { +#line 1073 "rlparse.kl" + + (&redLel->user.token_type)->token.set( "-", 1 ); + (&redLel->user.token_type)->token.loc = (&rhs[0]->user.token)->loc; + (&redLel->user.token_type)->token.append( *(&rhs[1]->user.token) ); + + +#line 5432 "rlparse.cpp" +} break; +case 185: { +#line 1079 "rlparse.kl" + + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + + +#line 5440 "rlparse.cpp" +} break; +case 186: { +#line 1094 "rlparse.kl" + + /* An optimization to lessen the tree size. If a non-starred char is + * directly under the left side on the right and the right side is + * another non-starred char then paste them together and return the + * left side. Otherwise just put the two under a new reg exp node. */ + if ( (&rhs[1]->user.regular_expr_item)->reItem->type == ReItem::Data && !(&rhs[1]->user.regular_expr_item)->reItem->star && + (&rhs[0]->user.regular_expr)->regExpr->type == RegExpr::RecurseItem && + (&rhs[0]->user.regular_expr)->regExpr->item->type == ReItem::Data && !(&rhs[0]->user.regular_expr)->regExpr->item->star ) + { + /* Append the right side to the right side of the left and toss the + * right side. */ + (&rhs[0]->user.regular_expr)->regExpr->item->token.append( (&rhs[1]->user.regular_expr_item)->reItem->token ); + delete (&rhs[1]->user.regular_expr_item)->reItem; + (&redLel->user.regular_expr)->regExpr = (&rhs[0]->user.regular_expr)->regExpr; + } + else { + (&redLel->user.regular_expr)->regExpr = new RegExpr( (&rhs[0]->user.regular_expr)->regExpr, (&rhs[1]->user.regular_expr_item)->reItem ); + } + + +#line 5464 "rlparse.cpp" +} break; +case 187: { +#line 1114 "rlparse.kl" + + /* Can't optimize the tree. */ + (&redLel->user.regular_expr)->regExpr = new RegExpr(); + + +#line 5473 "rlparse.cpp" +} break; +case 188: { +#line 1126 "rlparse.kl" + + (&rhs[0]->user.regular_expr_char)->reItem->star = true; + (&redLel->user.regular_expr_item)->reItem = (&rhs[0]->user.regular_expr_char)->reItem; + + +#line 5482 "rlparse.cpp" +} break; +case 189: { +#line 1131 "rlparse.kl" + + (&redLel->user.regular_expr_item)->reItem = (&rhs[0]->user.regular_expr_char)->reItem; + + +#line 5490 "rlparse.cpp" +} break; +case 190: { +#line 1143 "rlparse.kl" + + (&redLel->user.regular_expr_char)->reItem = new ReItem( (&rhs[0]->user.token)->loc, (&rhs[1]->user.regular_expr_or_data)->reOrBlock, ReItem::OrBlock ); + + +#line 5498 "rlparse.cpp" +} break; +case 191: { +#line 1147 "rlparse.kl" + + (&redLel->user.regular_expr_char)->reItem = new ReItem( (&rhs[0]->user.token)->loc, (&rhs[1]->user.regular_expr_or_data)->reOrBlock, ReItem::NegOrBlock ); + + +#line 5506 "rlparse.cpp" +} break; +case 192: { +#line 1151 "rlparse.kl" + + (&redLel->user.regular_expr_char)->reItem = new ReItem( (&rhs[0]->user.token)->loc, ReItem::Dot ); + + +#line 5514 "rlparse.cpp" +} break; +case 193: { +#line 1155 "rlparse.kl" + + (&redLel->user.regular_expr_char)->reItem = new ReItem( (&rhs[0]->user.token)->loc, *(&rhs[0]->user.token) ); + + +#line 5522 "rlparse.cpp" +} break; +case 194: { +#line 1167 "rlparse.kl" + + /* An optimization to lessen the tree size. If an or char is directly + * under the left side on the right and the right side is another or + * char then paste them together and return the left side. Otherwise + * just put the two under a new or data node. */ + if ( (&rhs[1]->user.regular_expr_or_char)->reOrItem->type == ReOrItem::Data && + (&rhs[0]->user.regular_expr_or_data)->reOrBlock->type == ReOrBlock::RecurseItem && + (&rhs[0]->user.regular_expr_or_data)->reOrBlock->item->type == ReOrItem::Data ) + { + /* Append the right side to right side of the left and toss the + * right side. */ + (&rhs[0]->user.regular_expr_or_data)->reOrBlock->item->token.append( (&rhs[1]->user.regular_expr_or_char)->reOrItem->token ); + delete (&rhs[1]->user.regular_expr_or_char)->reOrItem; + (&redLel->user.regular_expr_or_data)->reOrBlock = (&rhs[0]->user.regular_expr_or_data)->reOrBlock; + } + else { + /* Can't optimize, put the left and right under a new node. */ + (&redLel->user.regular_expr_or_data)->reOrBlock = new ReOrBlock( (&rhs[0]->user.regular_expr_or_data)->reOrBlock, (&rhs[1]->user.regular_expr_or_char)->reOrItem ); + } + + +#line 5547 "rlparse.cpp" +} break; +case 195: { +#line 1188 "rlparse.kl" + + (&redLel->user.regular_expr_or_data)->reOrBlock = new ReOrBlock(); + + +#line 5555 "rlparse.cpp" +} break; +case 196: { +#line 1200 "rlparse.kl" + + (&redLel->user.regular_expr_or_char)->reOrItem = new ReOrItem( (&rhs[0]->user.token)->loc, *(&rhs[0]->user.token) ); + + +#line 5563 "rlparse.cpp" +} break; +case 197: { +#line 1204 "rlparse.kl" + + (&redLel->user.regular_expr_or_char)->reOrItem = new ReOrItem( (&rhs[1]->user.token)->loc, (&rhs[0]->user.token)->data[0], (&rhs[2]->user.token)->data[0] ); + + +#line 5571 "rlparse.cpp" +} break; +case 198: { +#line 1221 "rlparse.kl" + + /* Append the item to the list, return the list. */ + (&redLel->user.inline_list)->inlineList = (&rhs[0]->user.inline_list)->inlineList; + (&redLel->user.inline_list)->inlineList->append( (&rhs[1]->user.inline_item)->inlineItem ); + + +#line 5581 "rlparse.cpp" +} break; +case 199: { +#line 1228 "rlparse.kl" + + /* Start with empty list. */ + (&redLel->user.inline_list)->inlineList = new InlineList; + + +#line 5590 "rlparse.cpp" +} break; +case 200: { +#line 1243 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token_type)->token.loc, (&rhs[0]->user.token_type)->token.data, InlineItem::Text ); + + +#line 5598 "rlparse.cpp" +} break; +case 201: { +#line 1249 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token_type)->token.loc, (&rhs[0]->user.token_type)->token.data, InlineItem::Text ); + + +#line 5606 "rlparse.cpp" +} break; +case 202: { +#line 1255 "rlparse.kl" + + /* Pass the inline item up. */ + (&redLel->user.inline_item)->inlineItem = (&rhs[0]->user.inline_item)->inlineItem; + + +#line 5615 "rlparse.cpp" +} break; +case 203: { +#line 1262 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 5621 "rlparse.cpp" +} break; +case 204: { +#line 1263 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 5627 "rlparse.cpp" +} break; +case 205: { +#line 1264 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 5633 "rlparse.cpp" +} break; +case 206: { +#line 1265 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 5639 "rlparse.cpp" +} break; +case 207: { +#line 1266 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 5645 "rlparse.cpp" +} break; +case 208: { +#line 1267 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 5651 "rlparse.cpp" +} break; +case 209: { +#line 1271 "rlparse.kl" + + /* Pass up interpreted items of inline expressions. */ + (&redLel->user.inline_item)->inlineItem = (&rhs[0]->user.inline_item)->inlineItem; + + +#line 5660 "rlparse.cpp" +} break; +case 210: { +#line 1276 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token)->loc, InlineItem::Hold ); + + +#line 5668 "rlparse.cpp" +} break; +case 211: { +#line 1280 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token)->loc, InlineItem::Exec ); + (&redLel->user.inline_item)->inlineItem->children = (&rhs[1]->user.inline_list)->inlineList; + + +#line 5677 "rlparse.cpp" +} break; +case 212: { +#line 1285 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token)->loc, + new NameRef(nameRef), InlineItem::Goto ); + + +#line 5686 "rlparse.cpp" +} break; +case 213: { +#line 1290 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token)->loc, InlineItem::GotoExpr ); + (&redLel->user.inline_item)->inlineItem->children = (&rhs[2]->user.inline_list)->inlineList; + + +#line 5695 "rlparse.cpp" +} break; +case 214: { +#line 1295 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token)->loc, new NameRef(nameRef), InlineItem::Next ); + + +#line 5703 "rlparse.cpp" +} break; +case 215: { +#line 1299 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token)->loc, InlineItem::NextExpr ); + (&redLel->user.inline_item)->inlineItem->children = (&rhs[2]->user.inline_list)->inlineList; + + +#line 5712 "rlparse.cpp" +} break; +case 216: { +#line 1304 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token)->loc, new NameRef(nameRef), InlineItem::Call ); + + +#line 5720 "rlparse.cpp" +} break; +case 217: { +#line 1308 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token)->loc, InlineItem::CallExpr ); + (&redLel->user.inline_item)->inlineItem->children = (&rhs[2]->user.inline_list)->inlineList; + + +#line 5729 "rlparse.cpp" +} break; +case 218: { +#line 1313 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token)->loc, InlineItem::Ret ); + + +#line 5737 "rlparse.cpp" +} break; +case 219: { +#line 1317 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token)->loc, InlineItem::Break ); + + +#line 5745 "rlparse.cpp" +} break; +case 220: { +#line 1325 "rlparse.kl" + + (&redLel->user.inline_list)->inlineList = (&rhs[0]->user.inline_list)->inlineList; + (&redLel->user.inline_list)->inlineList->append( (&rhs[1]->user.inline_item)->inlineItem ); + + +#line 5754 "rlparse.cpp" +} break; +case 221: { +#line 1330 "rlparse.kl" + + /* Init the list used for this expr. */ + (&redLel->user.inline_list)->inlineList = new InlineList; + + +#line 5763 "rlparse.cpp" +} break; +case 222: { +#line 1339 "rlparse.kl" + + /* Return a text segment. */ + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token_type)->token.loc, (&rhs[0]->user.token_type)->token.data, InlineItem::Text ); + + +#line 5772 "rlparse.cpp" +} break; +case 223: { +#line 1345 "rlparse.kl" + + /* Return a text segment, must heap alloc the text. */ + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token_type)->token.loc, (&rhs[0]->user.token_type)->token.data, InlineItem::Text ); + + +#line 5781 "rlparse.cpp" +} break; +case 224: { +#line 1351 "rlparse.kl" + + /* Pass the inline item up. */ + (&redLel->user.inline_item)->inlineItem = (&rhs[0]->user.inline_item)->inlineItem; + + +#line 5790 "rlparse.cpp" +} break; +case 237: { +#line 1381 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token)->loc, InlineItem::PChar ); + + +#line 5798 "rlparse.cpp" +} break; +case 238: { +#line 1386 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token)->loc, InlineItem::Char ); + + +#line 5806 "rlparse.cpp" +} break; +case 239: { +#line 1391 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token)->loc, InlineItem::Curs ); + + +#line 5814 "rlparse.cpp" +} break; +case 240: { +#line 1396 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token)->loc, InlineItem::Targs ); + + +#line 5822 "rlparse.cpp" +} break; +case 241: { +#line 1401 "rlparse.kl" + + (&redLel->user.inline_item)->inlineItem = new InlineItem( (&rhs[0]->user.token)->loc, + new NameRef(nameRef), InlineItem::Entry ); + + +#line 5831 "rlparse.cpp" +} break; +case 243: { +#line 1412 "rlparse.kl" + + nameRef.empty(); + + +#line 5839 "rlparse.cpp" +} break; +case 245: { +#line 1422 "rlparse.kl" + + /* Insert an initial null pointer val to indicate the existence of the + * initial name seperator. */ + nameRef.setAs( 0 ); + + +#line 5849 "rlparse.cpp" +} break; +case 246: { +#line 1428 "rlparse.kl" + + nameRef.empty(); + + +#line 5857 "rlparse.cpp" +} break; +case 247: { +#line 1435 "rlparse.kl" + + nameRef.append( (&rhs[2]->user.token)->data ); + + +#line 5865 "rlparse.cpp" +} break; +case 248: { +#line 1440 "rlparse.kl" + + nameRef.append( (&rhs[0]->user.token)->data ); + + +#line 5873 "rlparse.cpp" +} break; +} + } + + if ( lel->child != 0 ) { + struct Parser_LangEl *first = lel->child; + struct Parser_LangEl *child = lel->child; + lel->child = 0; + while ( 1 ) { + if ( child->type < 226 ) { + } + else { + } + numNodes -= 1; + if ( child->next == 0 ) + break; + child = child->next; + } + child->next = pool; + pool = first; + } + } + +commit_base: + if ( sp > 0 ) { + sp -= 1; + if ( lel->retry == 0 ) { + lel = lel->prev; + goto commit_reverse; + } + else { + lel->retry = 0; + lel = lel->prev; + goto commit_upwards; + } + } + lel->retry = 0; + + lastFinal = lel; + numRetry = 0; + } + + if ( *action & 0x2 ) { + int reduction = *action >> 2; + struct Parser_LangEl *redLel; + + if ( input != 0 ) + input->causeReduce += 1; + + if ( pool == 0 ) { + if ( freshPos == 8128 ) { + struct Parser_Block* newBlock = (struct Parser_Block*) malloc( sizeof(struct Parser_Block) ); + newBlock->next = block; + block = newBlock; + freshEl = newBlock->data; + #ifdef KELBT_LOG_ACTIONS + cerr << "allocating 8128 LangEls" << endl; + #endif + freshPos = 0; + } + redLel = freshEl + freshPos++; + } + else { + redLel = pool; + pool = pool->next; + } + numNodes += 1; + + redLel->type = Parser_prodLhsIds[reduction]; + redLel->reduction = reduction; + redLel->child = 0; + redLel->next = 0; + redLel->retry = (lel->retry << 16); + redLel->causeReduce = 0; + lel->retry &= 0xffff0000; + + rhsLen = Parser_prodLengths[reduction]; + if ( rhsLen > 0 ) { + int r; + for ( r = rhsLen-1; r > 0; r-- ) { + rhs[r] = stackTop; + stackTop = stackTop->next; + } + rhs[0] = stackTop; + stackTop = stackTop->next; + rhs[0]->next = 0; + } +switch ( reduction ) { +case 225: { +#line 1358 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 5966 "rlparse.cpp" +} break; +case 226: { +#line 1359 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 5972 "rlparse.cpp" +} break; +case 227: { +#line 1360 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 5978 "rlparse.cpp" +} break; +case 228: { +#line 1361 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 5984 "rlparse.cpp" +} break; +case 229: { +#line 1362 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 5990 "rlparse.cpp" +} break; +case 230: { +#line 1363 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 5996 "rlparse.cpp" +} break; +case 231: { +#line 1364 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 6002 "rlparse.cpp" +} break; +case 232: { +#line 1371 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 6008 "rlparse.cpp" +} break; +case 233: { +#line 1372 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 6014 "rlparse.cpp" +} break; +case 234: { +#line 1373 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 6020 "rlparse.cpp" +} break; +case 235: { +#line 1374 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 6026 "rlparse.cpp" +} break; +case 236: { +#line 1375 "rlparse.kl" + (&redLel->user.token_type)->token = *(&rhs[0]->user.token); + +#line 6032 "rlparse.cpp" +} break; +} + #ifdef KELBT_LOG_ACTIONS + cerr << "reduced: " + << Parser_prodNames[reduction] + << " rhsLen: " << rhsLen; + #endif + if ( action[1] == 0 ) + redLel->retry = 0; + else { + redLel->retry += 0x10000; + numRetry += 1; + #ifdef KELBT_LOG_ACTIONS + cerr << " retry: " << redLel; + #endif + } + + #ifdef KELBT_LOG_ACTIONS + cerr << endl; + #endif + + if ( rhsLen == 0 ) { + redLel->file = lel->file; + redLel->line = lel->line; + targState = curs; + } + else { + redLel->child = rhs[rhsLen-1]; + redLel->file = rhs[0]->file; + redLel->line = rhs[0]->line; + targState = rhs[0]->state; + } + + if ( induceReject ) { + #ifdef KELBT_LOG_ACTIONS + cerr << "error induced during reduction of " << + Parser_lelNames[redLel->type] << endl; + #endif + redLel->state = curs; + redLel->next = stackTop; + stackTop = redLel; + curs = targState; + goto parseError; + } + else { + redLel->next = input; + input = redLel; + } + } + + + curs = targState; + goto again; + +parseError: + #ifdef KELBT_LOG_BACKTRACK + cerr << "hit error" << endl; + #endif + if ( numRetry > 0 ) { + struct Parser_LangEl *redLel; + + if ( input != 0 ) { + redLel = input; + goto have_undo_element; + } + + while ( 1 ) { + redLel = stackTop; + if ( stackTop->type < 226 ) { + #ifdef KELBT_LOG_BACKTRACK + cerr << "backing up over terminal: " << + Parser_lelNames[stackTop->type] << endl; + #endif + stackTop = stackTop->next; + redLel->next = input; + input = redLel; + } + else { + #ifdef KELBT_LOG_BACKTRACK + cerr << "backing up over non-terminal: " << + Parser_lelNames[stackTop->type] << endl; + #endif + stackTop = stackTop->next; + struct Parser_LangEl *first = redLel->child; + if ( first == 0 ) + rhsLen = 0; + else { + rhsLen = 1; + while ( first->next != 0 ) { + first = first->next; + rhsLen += 1; + } + first->next = stackTop; + stackTop = redLel->child; + + struct Parser_LangEl *rhsEl = stackTop; + int p = rhsLen; + while ( p > 0 ) { + rhs[--p] = rhsEl; + rhsEl = rhsEl->next; + } + } + redLel->next = pool; + pool = redLel; + numNodes -= 1; + + if ( input != 0 ) + input->causeReduce -= 1; + } + +have_undo_element: + if ( redLel->retry == 0 ) { + if ( input != 0 && input->causeReduce == 0 ) { + #ifdef KELBT_LOG_BACKTRACK + cerr << "pushing back: " << Parser_lelNames[input->type] << endl; + #endif + input->next = queue; + queue = input; + input = 0; + } + } + else { + #ifdef KELBT_LOG_BACKTRACK + cerr << "found retry targ: " << redLel << endl; + #endif + numRetry -= 1; + #ifdef KELBT_LOG_BACKTRACK + cerr << "found retry: " << redLel << endl; + #endif + if ( redLel->retry & 0x0000ffff ) + curs = input->state; + else { + input->retry = redLel->retry >> 16; + if ( stackTop->state < 0 ) + curs = Parser_startState; + else { + curs = Parser_targs[(int)Parser_indicies[Parser_offsets[stackTop->state] + (stackTop->type - Parser_keys[stackTop->state<<1])]]; + } + } + goto again; + } + } + } + curs = -1; + errCount += 1; +_out: {} +#line 1459 "rlparse.kl" + return errCount == 0 ? 0 : -1; +} + +void Parser::tryMachineDef( InputLoc &loc, char *name, + MachineDef *machineDef, bool isInstance ) +{ + GraphDictEl *newEl = pd->graphDict.insert( name ); + if ( newEl != 0 ) { + /* New element in the dict, all good. */ + newEl->value = new VarDef( name, machineDef ); + newEl->isInstance = isInstance; + newEl->loc = loc; + newEl->value->isExport = exportContext[exportContext.length()-1]; + + /* It it is an instance, put on the instance list. */ + if ( isInstance ) + pd->instanceList.append( newEl ); + } + else { + // Recover by ignoring the duplicate. + error(loc) << "fsm \"" << name << "\" previously defined" << endl; + } +} + +ostream &Parser::parse_error( int tokId, Token &token ) +{ + /* Maintain the error count. */ + gblErrorCount += 1; + + cerr << token.loc << ": "; + cerr << "at token "; + if ( tokId < 128 ) + cerr << "\"" << Parser_lelNames[tokId] << "\""; + else + cerr << Parser_lelNames[tokId]; + if ( token.data != 0 ) + cerr << " with data \"" << token.data << "\""; + cerr << ": "; + + return cerr; +} + +int Parser::token( InputLoc &loc, int tokId, char *tokstart, int toklen ) +{ + Token token; + token.data = tokstart; + token.length = toklen; + token.loc = loc; + int res = parseLangEl( tokId, &token ); + if ( res < 0 ) { + parse_error(tokId, token) << "parse error" << endl; + exit(1); + } + return res; +} diff --git a/ragel/rlparse.h b/ragel/rlparse.h new file mode 100644 index 0000000..e3f7c3c --- /dev/null +++ b/ragel/rlparse.h @@ -0,0 +1,210 @@ +/* Automatically generated by Kelbt from "rlparse.kh". + * + * Parts of this file are copied from Kelbt source covered by the GNU + * GPL. As a special exception, you may use the parts of this file copied + * from Kelbt source without restriction. The remainder is derived from + * "rlparse.kh" and inherits the copyright status of that file. + */ + +#line 1 "rlparse.kh" +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RLPARSE_H +#define _RLPARSE_H + +#include <iostream> +#include "avltree.h" +#include "parsedata.h" + + +/* Import scanner tokens. */ +#define IMP_Word 128 +#define IMP_Literal 129 +#define IMP_UInt 130 +#define IMP_Define 131 + +/* This is used for tracking the include files/machine pairs. */ +struct IncludeHistoryItem +{ + IncludeHistoryItem( const char *fileName, const char *sectionName ) + : fileName(fileName), sectionName(sectionName) {} + + const char *fileName; + const char *sectionName; +}; + +typedef Vector<IncludeHistoryItem> IncludeHistory; + +struct Parser +{ +#line 102 "rlparse.kh" + + + #line 63 "rlparse.h" + struct Parser_Block *block; + struct Parser_LangEl *freshEl; + int freshPos; + struct Parser_LangEl *pool; + int numRetry; + int numNodes; + struct Parser_LangEl *stackTop; + struct Parser_LangEl *lastFinal; + int errCount; + int curs; +#line 105 "rlparse.kh" + + void init(); + int parseLangEl( int type, const Token *token ); + + Parser( const char *fileName, char *sectionName, InputLoc §ionLoc ) + : sectionName(sectionName) + { + pd = new ParseData( fileName, sectionName, sectionLoc ); + exportContext.append( false ); + includeHistory.append( IncludeHistoryItem( + fileName, sectionName ) ); + } + + int token( InputLoc &loc, int tokId, char *tokstart, int toklen ); + void tryMachineDef( InputLoc &loc, char *name, + MachineDef *machineDef, bool isInstance ); + + /* Report an error encountered by the parser. */ + ostream &parse_error( int tokId, Token &token ); + + ParseData *pd; + + /* The name of the root section, this does not change during an include. */ + char *sectionName; + + NameRef nameRef; + NameRefList nameRefList; + + Vector<bool> exportContext; + IncludeHistory includeHistory; + + Parser *prev, *next; +}; + +#line 109 "rlparse.h" +#define TK_Word 128 +#define TK_Literal 129 +#define TK_Number 130 +#define TK_EndSection 131 +#define TK_UInt 132 +#define TK_Hex 133 +#define TK_DotDot 134 +#define TK_ColonGt 135 +#define TK_ColonGtGt 136 +#define TK_LtColon 137 +#define TK_Arrow 138 +#define TK_DoubleArrow 139 +#define TK_StarStar 140 +#define TK_ColonEquals 141 +#define TK_NameSep 142 +#define TK_BarStar 143 +#define TK_DashDash 144 +#define TK_StartCond 145 +#define TK_AllCond 146 +#define TK_LeavingCond 147 +#define TK_Middle 148 +#define TK_StartGblError 149 +#define TK_AllGblError 150 +#define TK_FinalGblError 151 +#define TK_NotFinalGblError 152 +#define TK_NotStartGblError 153 +#define TK_MiddleGblError 154 +#define TK_StartLocalError 155 +#define TK_AllLocalError 156 +#define TK_FinalLocalError 157 +#define TK_NotFinalLocalError 158 +#define TK_NotStartLocalError 159 +#define TK_MiddleLocalError 160 +#define TK_StartEOF 161 +#define TK_AllEOF 162 +#define TK_FinalEOF 163 +#define TK_NotFinalEOF 164 +#define TK_NotStartEOF 165 +#define TK_MiddleEOF 166 +#define TK_StartToState 167 +#define TK_AllToState 168 +#define TK_FinalToState 169 +#define TK_NotFinalToState 170 +#define TK_NotStartToState 171 +#define TK_MiddleToState 172 +#define TK_StartFromState 173 +#define TK_AllFromState 174 +#define TK_FinalFromState 175 +#define TK_NotFinalFromState 176 +#define TK_NotStartFromState 177 +#define TK_MiddleFromState 178 +#define RE_Slash 179 +#define RE_SqOpen 180 +#define RE_SqOpenNeg 181 +#define RE_SqClose 182 +#define RE_Dot 183 +#define RE_Star 184 +#define RE_Dash 185 +#define RE_Char 186 +#define IL_WhiteSpace 187 +#define IL_Comment 188 +#define IL_Literal 189 +#define IL_Symbol 190 +#define KW_Machine 191 +#define KW_Include 192 +#define KW_Import 193 +#define KW_Write 194 +#define KW_Action 195 +#define KW_AlphType 196 +#define KW_Range 197 +#define KW_GetKey 198 +#define KW_InWhen 199 +#define KW_When 200 +#define KW_OutWhen 201 +#define KW_Eof 202 +#define KW_Err 203 +#define KW_Lerr 204 +#define KW_To 205 +#define KW_From 206 +#define KW_Export 207 +#define KW_PrePush 208 +#define KW_PostPop 209 +#define KW_Length 210 +#define KW_Break 211 +#define KW_Exec 212 +#define KW_Hold 213 +#define KW_PChar 214 +#define KW_Char 215 +#define KW_Goto 216 +#define KW_Call 217 +#define KW_Ret 218 +#define KW_CurState 219 +#define KW_TargState 220 +#define KW_Entry 221 +#define KW_Next 222 +#define KW_Variable 223 +#define KW_Access 224 +#define Parser_tk_eof 225 + +#line 140 "rlparse.kh" + +#endif diff --git a/ragel/rlparse.kh b/ragel/rlparse.kh index 5d7b404..899bbbc 100644 --- a/ragel/rlparse.kh +++ b/ragel/rlparse.kh @@ -1,5 +1,5 @@ /* - * Copyright 2006 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -19,38 +19,47 @@ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ -#ifndef RLPARSE_H -#define RLPARSE_H +#ifndef _RLPARSE_H +#define _RLPARSE_H #include <iostream> #include "avltree.h" #include "parsedata.h" -extern char *lelNames[]; -struct LangEl; +/* Import scanner tokens. */ +#define IMP_Word 128 +#define IMP_Literal 129 +#define IMP_UInt 130 +#define IMP_Define 131 -struct Parser +/* This is used for tracking the include files/machine pairs. */ +struct IncludeHistoryItem { - %%{ - parser Parser; + IncludeHistoryItem( const char *fileName, const char *sectionName ) + : fileName(fileName), sectionName(sectionName) {} - # These must be declared first and in this order. Ragel currently cannot - # import kelbt keywords for use in machines, so in the scanner - # rely on knowing the values that kelbt will assign to these. - token KW_Machine, KW_Include, KW_Write, TK_Word, TK_Literal; + const char *fileName; + const char *sectionName; +}; - token TK_Number, TK_Inline, TK_Reference, TK_ColonEquals, TK_EndSection; +typedef Vector<IncludeHistoryItem> IncludeHistory; + +struct Parser +{ +%%{ + parser Parser; # General tokens. - token TK_UInt, TK_Hex, TK_Word, TK_Literal, TK_BaseClause, - TK_DotDot, TK_ColonGt, TK_ColonGtGt, TK_LtColon, TK_Arrow, - TK_DoubleArrow, TK_StarStar, TK_ColonEquals, TK_NameSep, TK_BarStar, - TK_DashDash; + token TK_Word, TK_Literal, TK_Number, TK_EndSection, TK_UInt, TK_Hex, + TK_Word, TK_Literal, TK_DotDot, TK_ColonGt, TK_ColonGtGt, TK_LtColon, + TK_Arrow, TK_DoubleArrow, TK_StarStar, TK_ColonEquals, TK_NameSep, + TK_BarStar, TK_DashDash; # Conditions. token TK_StartCond, TK_AllCond, TK_LeavingCond; + # State embedding actions. token TK_Middle; # Global error actions. @@ -81,34 +90,37 @@ struct Parser token IL_WhiteSpace, IL_Comment, IL_Literal, IL_Symbol; # Keywords. - token KW_Action, KW_AlphType, KW_Range, KW_GetKey, KW_Include, KW_Write, - KW_Machine, KW_When, KW_Eof, KW_Err, KW_Lerr, KW_To, KW_From; + token KW_Machine, KW_Include, KW_Import, KW_Write, KW_Action, KW_AlphType, + KW_Range, KW_GetKey, KW_Include, KW_Write, KW_Machine, KW_InWhen, + KW_When, KW_OutWhen, KW_Eof, KW_Err, KW_Lerr, KW_To, KW_From, + KW_Export, KW_PrePush, KW_PostPop, KW_Length; # Specials in code blocks. token KW_Break, KW_Exec, KW_Hold, KW_PChar, KW_Char, KW_Goto, KW_Call, KW_Ret, KW_CurState, KW_TargState, KW_Entry, KW_Next, KW_Exec, KW_Variable, KW_Access; +}%% - # Special token for terminating semi-terminated code blocks. Needed because - # semi is sent as a token in the code block rather than as a generic - # symbol. - token TK_Semi; + %% write instance_data; - interface; - }%% + void init(); + int parseLangEl( int type, const Token *token ); - Parser( char *fileName, char *sectionName, InputLoc §ionLoc ) + Parser( const char *fileName, char *sectionName, InputLoc §ionLoc ) : sectionName(sectionName) { pd = new ParseData( fileName, sectionName, sectionLoc ); + exportContext.append( false ); + includeHistory.append( IncludeHistoryItem( + fileName, sectionName ) ); } int token( InputLoc &loc, int tokId, char *tokstart, int toklen ); void tryMachineDef( InputLoc &loc, char *name, - JoinOrLm *joinOrLm, bool isInstance ); + MachineDef *machineDef, bool isInstance ); /* Report an error encountered by the parser. */ - ostream &parser_error( int tokId, Token &token ); + ostream &parse_error( int tokId, Token &token ); ParseData *pd; @@ -117,6 +129,13 @@ struct Parser NameRef nameRef; NameRefList nameRefList; + + Vector<bool> exportContext; + IncludeHistory includeHistory; + + Parser *prev, *next; }; +%% write token_defs; + #endif diff --git a/ragel/rlparse.kl b/ragel/rlparse.kl index b39fa5c..36b8777 100644 --- a/ragel/rlparse.kl +++ b/ragel/rlparse.kl @@ -1,5 +1,5 @@ /* - * Copyright 2006 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -22,20 +22,23 @@ #include "rlparse.h" #include "ragel.h" #include <iostream> +#include <errno.h> +#include <stdlib.h> using std::cout; using std::cerr; using std::endl; -ParserDict parserDict; - %%{ parser Parser; include "rlparse.kh"; -start: statement_list; +start: section_list; + +section_list: section_list statement_list TK_EndSection; +section_list: ; statement_list: statement_list statement; statement_list: ; @@ -48,29 +51,95 @@ statement: range_spec commit; statement: getkey_spec commit; statement: access_spec commit; statement: variable_spec commit; +statement: export_block commit; +statement: pre_push_spec commit; +statement: post_pop_spec commit; +statement: length_spec commit; + +length_spec: + KW_Length TK_Word ';' + final { + LengthDef *lengthDef = new LengthDef( $2->data ); + pd->lengthDefList.append( lengthDef ); + + /* Generic creation of machine for instantiation and assignment. */ + MachineDef *machineDef = new MachineDef( lengthDef ); + tryMachineDef( $2->loc, $2->data, machineDef, false ); + }; + +pre_push_spec: + KW_PrePush '{' inline_block '}' + final { + if ( pd->prePushExpr != 0 ) { + /* Recover by just ignoring the duplicate. */ + error($2->loc) << "pre_push code already defined" << endl; + } + + pd->prePushExpr = $3->inlineList; + }; + + +post_pop_spec: + KW_PostPop '{' inline_block '}' + final { + if ( pd->postPopExpr != 0 ) { + /* Recover by just ignoring the duplicate. */ + error($2->loc) << "post_pop code already defined" << endl; + } + + pd->postPopExpr = $3->inlineList; + }; -# We use end section tokens to draw firm boundaries between sections. -statement: TK_EndSection; + +export_open: KW_Export + final { + exportContext.append( true ); + }; + +nonterm opt_export +{ + bool isSet; +}; + +opt_export: export_open final { $$->isSet = true; }; +opt_export: final { $$->isSet = false; }; + +export_block: export_open '{' statement_list '}' + final { + exportContext.remove( exportContext.length()-1 ); + }; assignment: - machine_name '=' join ';' final { + opt_export machine_name '=' join ';' final { /* Main machine must be an instance. */ bool isInstance = false; - if ( strcmp($1->token.data, machineMain) == 0 ) { - warning($1->token.loc) << + if ( strcmp($2->token.data, mainMachine) == 0 ) { + warning($2->token.loc) << "main machine will be implicitly instantiated" << endl; isInstance = true; } /* Generic creation of machine for instantiation and assignment. */ - JoinOrLm *joinOrLm = new JoinOrLm( $3->join ); - tryMachineDef( $1->token.loc, $1->token.data, joinOrLm, isInstance ); + MachineDef *machineDef = new MachineDef( $4->join ); + tryMachineDef( $2->token.loc, $2->token.data, machineDef, isInstance ); + + if ( $1->isSet ) + exportContext.remove( exportContext.length()-1 ); + + $4->join->loc = $3->loc; }; instantiation: - machine_name TK_ColonEquals join_or_lm ';' final { + opt_export machine_name TK_ColonEquals join_or_lm ';' final { /* Generic creation of machine for instantiation and assignment. */ - tryMachineDef( $1->token.loc, $1->token.data, $3->joinOrLm, true ); + tryMachineDef( $2->token.loc, $2->token.data, $4->machineDef, true ); + + if ( $1->isSet ) + exportContext.remove( exportContext.length()-1 ); + + /* Pass a location to join_or_lm */ + if ( $4->machineDef->join != 0 ) + $4->machineDef->join->loc = $3->loc; }; type token_type @@ -82,8 +151,6 @@ nonterm machine_name uses token_type; machine_name: TK_Word final { - //cerr << "parser: machine name" << endl; - /* Make/get the priority key. The name may have already been referenced * and therefore exist. */ PriorDictEl *priorDictEl; @@ -109,7 +176,8 @@ action_spec: else { //cerr << "NEW ACTION " << $2->data << " " << $4->inlineList << endl; /* Add the action to the list of actions. */ - Action *newAction = new Action( $3->loc, $2->data, $4->inlineList ); + Action *newAction = new Action( $3->loc, $2->data, + $4->inlineList, pd->nextCondId++ ); /* Insert to list and dict. */ pd->actionList.append( newAction ); @@ -121,7 +189,7 @@ action_spec: # semi-colon. alphtype_spec: KW_AlphType TK_Word TK_Word ';' final { - if ( ! pd->setAlphType( $2->data, $3->data ) ) { + if ( ! pd->setAlphType( $1->loc, $2->data, $3->data ) ) { // Recover by ignoring the alphtype statement. error($2->loc) << "\"" << $2->data << " " << $3->data << "\" is not a valid alphabet type" << endl; @@ -130,7 +198,7 @@ alphtype_spec: alphtype_spec: KW_AlphType TK_Word ';' final { - if ( ! pd->setAlphType( $2->data ) ) { + if ( ! pd->setAlphType( $1->loc, $2->data ) ) { // Recover by ignoring the alphtype statement. error($2->loc) << "\"" << $2->data << "\" is not a valid alphabet type" << endl; @@ -160,11 +228,9 @@ access_spec: variable_spec: KW_Variable opt_whitespace TK_Word inline_expr ';' final { /* FIXME: Need to implement the rest of this. */ - if ( strcmp( $3->data, "curstate" ) == 0 ) - pd->curStateExpr = $4->inlineList; - else { - error($3->loc) << "sorry, unimplementd" << endl; - } + bool wasSet = pd->setVariable( $3->data, $4->inlineList ); + if ( !wasSet ) + error($3->loc) << "bad variable name" << endl; }; opt_whitespace: opt_whitespace IL_WhiteSpace; @@ -176,12 +242,12 @@ opt_whitespace: ; nonterm join_or_lm { - JoinOrLm *joinOrLm; + MachineDef *machineDef; }; join_or_lm: join final { - $$->joinOrLm = new JoinOrLm( $1->join ); + $$->machineDef = new MachineDef( $1->join ); }; join_or_lm: TK_BarStar lm_part_list '*' '|' final { @@ -191,7 +257,7 @@ join_or_lm: pd->lmList.append( lm ); for ( LmPartList::Iter lmp = *($2->lmPartList); lmp.lte(); lmp++ ) lmp->longestMatch = lm; - $$->joinOrLm = new JoinOrLm( lm ); + $$->machineDef = new MachineDef( lm ); }; nonterm lm_part_list @@ -200,13 +266,15 @@ nonterm lm_part_list }; lm_part_list: - lm_part_list longest_match_part final { + lm_part_list longest_match_part + final { if ( $2->lmPart != 0 ) $1->lmPartList->append( $2->lmPart ); $$->lmPartList = $1->lmPartList; }; lm_part_list: - longest_match_part final { + longest_match_part + final { /* Create a new list with the part. */ $$->lmPartList = new LmPartList; if ( $1->lmPart != 0 ) @@ -230,6 +298,10 @@ longest_match_part: action->isLmAction = true; $$->lmPart = new LongestMatchPart( $1->join, action, $3->loc, pd->nextLongestMatchId++ ); + + /* Provide a location to join. Unfortunately We don't + * have the start of the join as in other occurances. Use the end. */ + $1->join->loc = $3->loc; }; nonterm opt_lm_part_action @@ -273,30 +345,51 @@ nonterm expression }; expression: - expression '|' term final { + expression '|' term_short final { $$->expression = new Expression( $1->expression, $3->term, Expression::OrType ); }; expression: - expression '&' term final { + expression '&' term_short final { $$->expression = new Expression( $1->expression, $3->term, Expression::IntersectType ); }; expression: - expression pri(1) '-' term final { + expression '-' term_short final { $$->expression = new Expression( $1->expression, $3->term, Expression::SubtractType ); }; expression: - expression TK_DashDash term final { + expression TK_DashDash term_short final { $$->expression = new Expression( $1->expression, $3->term, Expression::StrongSubtractType ); }; expression: - term final { + term_short final { $$->expression = new Expression( $1->term ); }; +# This is where we resolve the ambiguity involving -. By default ragel tries to +# do a longest match, which gives precedence to a concatenation because it is +# innermost. What we need is to force term into a shortest match so that when - +# is seen it doesn't try to extend term with a concatenation, but ends term and +# goes for a subtraction. +# +# The shortest tag overrides the default longest match action ordering strategy +# and instead forces a shortest match stragegy. The wrap the term production in +# a new nonterminal 'term_short' to guarantee the shortest match behaviour. + +shortest term_short; +nonterm term_short +{ + Term *term; +}; + +term_short: + term final { + $$->term = $1->term; + }; + nonterm term { Term *term; @@ -304,7 +397,6 @@ nonterm term term: term factor_with_label final { - /* FIXME: Need to reject this if of the form (term . -num). */ $$->term = new Term( $1->term, $2->factorWithAug ); }; term: @@ -390,8 +482,14 @@ factor_with_aug: }; factor_with_aug: factor_with_aug aug_type_cond action_embed final { - $1->factorWithAug->conditions.append( ParserAction( $2->loc, - $2->augType, 0, $3->action ) ); + $1->factorWithAug->conditions.append( ConditionTest( $2->loc, + $2->augType, $3->action, true ) ); + $$->factorWithAug = $1->factorWithAug; + }; +factor_with_aug: + factor_with_aug aug_type_cond '!' action_embed final { + $1->factorWithAug->conditions.append( ConditionTest( $2->loc, + $2->augType, $4->action, false ) ); $$->factorWithAug = $1->factorWithAug; }; factor_with_aug: @@ -468,6 +566,8 @@ aug_type_cond: '$' KW_When final { $$->loc = $1->loc; $$->augType = at_all; }; aug_type_cond: TK_LeavingCond final { $$->loc = $1->loc; $$->augType = at_leave; }; aug_type_cond: '%' KW_When final { $$->loc = $1->loc; $$->augType = at_leave; }; aug_type_cond: KW_When final { $$->loc = $1->loc; $$->augType = at_all; }; +aug_type_cond: KW_InWhen final { $$->loc = $1->loc; $$->augType = at_start; }; +aug_type_cond: KW_OutWhen final { $$->loc = $1->loc; $$->augType = at_leave; }; # # To state actions. @@ -662,6 +762,7 @@ type action_ref nonterm action_embed uses action_ref; action_embed: action_embed_word final { $$->action = $1->action; }; +action_embed: '(' action_embed_word ')' final { $$->action = $2->action; }; action_embed: action_embed_block final { $$->action = $1->action; }; nonterm action_embed_word uses action_ref; @@ -686,7 +787,7 @@ nonterm action_embed_block uses action_ref; action_embed_block: '{' inline_block '}' final { /* Create the action, add it to the list and pass up. */ - Action *newAction = new Action( $1->loc, 0, $2->inlineList ); + Action *newAction = new Action( $1->loc, 0, $2->inlineList, pd->nextCondId++ ); pd->actionList.append( newAction ); $$->action = newAction; }; @@ -720,7 +821,7 @@ priority_aug: // Convert the priority number to a long. Check for overflow. errno = 0; //cerr << "PRIOR AUG: " << $1->token.data << endl; - int aug = strtol( $1->token.data, 0, 10 ); + long aug = strtol( $1->token.data, 0, 10 ); if ( errno == ERANGE && aug == LONG_MAX ) { /* Priority number too large. Recover by setting the priority to 0. */ error($1->token.loc) << "priority number " << $1->token.data << @@ -826,8 +927,7 @@ factor_with_rep: }; factor_with_rep: factor_with_neg final { - $$->factorWithRep = new FactorWithRep( - $1->factorWithNeg->loc, $1->factorWithNeg ); + $$->factorWithRep = new FactorWithRep( $1->factorWithNeg ); }; nonterm factor_rep_num @@ -839,7 +939,7 @@ factor_rep_num: TK_UInt final { // Convert the priority number to a long. Check for overflow. errno = 0; - int rep = strtol( $1->data, 0, 10 ); + long rep = strtol( $1->data, 0, 10 ); if ( errno == ERANGE && rep == LONG_MAX ) { // Repetition too large. Recover by returing repetition 1. */ error($1->loc) << "repetition number " << $1->data << " overflows" << endl; @@ -873,7 +973,7 @@ factor_with_neg: }; factor_with_neg: factor final { - $$->factorWithNeg = new FactorWithNeg( $1->factor->loc, $1->factor ); + $$->factorWithNeg = new FactorWithNeg( $1->factor ); }; nonterm factor @@ -942,6 +1042,7 @@ factor: '(' join ')' final { /* Create a new factor going to a parenthesized join. */ $$->factor = new Factor( $2->join ); + $2->join->loc = $1->loc; }; nonterm range_lit @@ -1342,15 +1443,32 @@ state_ref_names: }%% +%%{ + write types; + write data; +}%% + +void Parser::init() +{ + %% write init; +} + +int Parser::parseLangEl( int type, const Token *token ) +{ + %% write exec; + return errCount == 0 ? 0 : -1; +} + void Parser::tryMachineDef( InputLoc &loc, char *name, - JoinOrLm *joinOrLm, bool isInstance ) + MachineDef *machineDef, bool isInstance ) { GraphDictEl *newEl = pd->graphDict.insert( name ); if ( newEl != 0 ) { /* New element in the dict, all good. */ - newEl->value = new VarDef( name, joinOrLm ); + newEl->value = new VarDef( name, machineDef ); newEl->isInstance = isInstance; newEl->loc = loc; + newEl->value->isExport = exportContext[exportContext.length()-1]; /* It it is an instance, put on the instance list. */ if ( isInstance ) @@ -1362,24 +1480,17 @@ void Parser::tryMachineDef( InputLoc &loc, char *name, } } -ostream &error( const InputLoc &loc ) -{ - gblErrorCount += 1; - assert( loc.fileName != 0 ); - cerr << loc.fileName << ":" << loc.line << ": "; - return cerr; -} - -ostream &Parser::parser_error( int tokId, Token &token ) +ostream &Parser::parse_error( int tokId, Token &token ) { + /* Maintain the error count. */ gblErrorCount += 1; - cerr << token.loc.fileName << ":" << token.loc.line << ": "; + cerr << token.loc << ": "; cerr << "at token "; if ( tokId < 128 ) - cerr << "\"" << lelNames[tokId] << "\""; + cerr << "\"" << Parser_lelNames[tokId] << "\""; else - cerr << lelNames[tokId]; + cerr << Parser_lelNames[tokId]; if ( token.data != 0 ) cerr << " with data \"" << token.data << "\""; cerr << ": "; @@ -1393,9 +1504,9 @@ int Parser::token( InputLoc &loc, int tokId, char *tokstart, int toklen ) token.data = tokstart; token.length = toklen; token.loc = loc; - int res = parseLangEl( tokId, token ); + int res = parseLangEl( tokId, &token ); if ( res < 0 ) { - parser_error(tokId, token) << "parse error" << endl; + parse_error(tokId, token) << "parse error" << endl; exit(1); } return res; diff --git a/ragel/rlscan.cpp b/ragel/rlscan.cpp new file mode 100644 index 0000000..7f27795 --- /dev/null +++ b/ragel/rlscan.cpp @@ -0,0 +1,7218 @@ + +#line 1 "rlscan.rl" +/* + * Copyright 2006-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <iostream> +#include <fstream> +#include <string.h> + +#include "ragel.h" +#include "rlscan.h" +#include "inputdata.h" + +//#define LOG_TOKENS + +using std::ifstream; +using std::istream; +using std::ostream; +using std::cout; +using std::cerr; +using std::endl; + +enum InlineBlockType +{ + CurlyDelimited, + SemiTerminated +}; + +#ifdef _WIN32 +#define PATH_SEP '\\' +#else +#define PATH_SEP '/' +#endif + + +/* + * The Scanner for Importing + */ + + +#line 124 "rlscan.rl" + + + +#line 64 "rlscan.cpp" +static const int inline_token_scan_start = 2; +static const int inline_token_scan_first_final = 2; +static const int inline_token_scan_error = -1; + +static const int inline_token_scan_en_main = 2; + + +#line 127 "rlscan.rl" + +void Scanner::flushImport() +{ + int *p = token_data; + int *pe = token_data + cur_token; + int *eof = 0; + + +#line 81 "rlscan.cpp" + { + tok_cs = inline_token_scan_start; + tok_ts = 0; + tok_te = 0; + tok_act = 0; + } + +#line 89 "rlscan.cpp" + { + if ( p == pe ) + goto _test_eof; + switch ( tok_cs ) + { +tr0: +#line 122 "rlscan.rl" + {{p = (( tok_te))-1;}} + goto st2; +tr1: +#line 108 "rlscan.rl" + { tok_te = p+1;{ + int base = tok_ts - token_data; + int nameOff = 0; + int litOff = 2; + + directToParser( inclToParser, fileName, line, column, TK_Word, + token_strings[base+nameOff], token_lens[base+nameOff] ); + directToParser( inclToParser, fileName, line, column, '=', 0, 0 ); + directToParser( inclToParser, fileName, line, column, TK_Literal, + token_strings[base+litOff], token_lens[base+litOff] ); + directToParser( inclToParser, fileName, line, column, ';', 0, 0 ); + }} + goto st2; +tr2: +#line 80 "rlscan.rl" + { tok_te = p+1;{ + int base = tok_ts - token_data; + int nameOff = 0; + int numOff = 2; + + directToParser( inclToParser, fileName, line, column, TK_Word, + token_strings[base+nameOff], token_lens[base+nameOff] ); + directToParser( inclToParser, fileName, line, column, '=', 0, 0 ); + directToParser( inclToParser, fileName, line, column, TK_UInt, + token_strings[base+numOff], token_lens[base+numOff] ); + directToParser( inclToParser, fileName, line, column, ';', 0, 0 ); + }} + goto st2; +tr3: +#line 94 "rlscan.rl" + { tok_te = p+1;{ + int base = tok_ts - token_data; + int nameOff = 1; + int litOff = 2; + + directToParser( inclToParser, fileName, line, column, TK_Word, + token_strings[base+nameOff], token_lens[base+nameOff] ); + directToParser( inclToParser, fileName, line, column, '=', 0, 0 ); + directToParser( inclToParser, fileName, line, column, TK_Literal, + token_strings[base+litOff], token_lens[base+litOff] ); + directToParser( inclToParser, fileName, line, column, ';', 0, 0 ); + }} + goto st2; +tr4: +#line 66 "rlscan.rl" + { tok_te = p+1;{ + int base = tok_ts - token_data; + int nameOff = 1; + int numOff = 2; + + directToParser( inclToParser, fileName, line, column, TK_Word, + token_strings[base+nameOff], token_lens[base+nameOff] ); + directToParser( inclToParser, fileName, line, column, '=', 0, 0 ); + directToParser( inclToParser, fileName, line, column, TK_UInt, + token_strings[base+numOff], token_lens[base+numOff] ); + directToParser( inclToParser, fileName, line, column, ';', 0, 0 ); + }} + goto st2; +tr5: +#line 122 "rlscan.rl" + { tok_te = p+1;} + goto st2; +tr8: +#line 122 "rlscan.rl" + { tok_te = p;p--;} + goto st2; +st2: +#line 1 "rlscan.rl" + { tok_ts = 0;} + if ( ++p == pe ) + goto _test_eof2; +case 2: +#line 1 "rlscan.rl" + { tok_ts = p;} +#line 175 "rlscan.cpp" + switch( (*p) ) { + case 128: goto tr6; + case 131: goto tr7; + } + goto tr5; +tr6: +#line 1 "rlscan.rl" + { tok_te = p+1;} + goto st3; +st3: + if ( ++p == pe ) + goto _test_eof3; +case 3: +#line 189 "rlscan.cpp" + if ( (*p) == 61 ) + goto st0; + goto tr8; +st0: + if ( ++p == pe ) + goto _test_eof0; +case 0: + switch( (*p) ) { + case 129: goto tr1; + case 130: goto tr2; + } + goto tr0; +tr7: +#line 1 "rlscan.rl" + { tok_te = p+1;} + goto st4; +st4: + if ( ++p == pe ) + goto _test_eof4; +case 4: +#line 210 "rlscan.cpp" + if ( (*p) == 128 ) + goto st1; + goto tr8; +st1: + if ( ++p == pe ) + goto _test_eof1; +case 1: + switch( (*p) ) { + case 129: goto tr3; + case 130: goto tr4; + } + goto tr0; + } + _test_eof2: tok_cs = 2; goto _test_eof; + _test_eof3: tok_cs = 3; goto _test_eof; + _test_eof0: tok_cs = 0; goto _test_eof; + _test_eof4: tok_cs = 4; goto _test_eof; + _test_eof1: tok_cs = 1; goto _test_eof; + + _test_eof: {} + if ( p == eof ) + { + switch ( tok_cs ) { + case 3: goto tr8; + case 0: goto tr0; + case 4: goto tr8; + case 1: goto tr0; + } + } + + } + +#line 138 "rlscan.rl" + + + if ( tok_ts == 0 ) + cur_token = 0; + else { + cur_token = pe - tok_ts; + int ts_offset = tok_ts - token_data; + memmove( token_data, token_data+ts_offset, cur_token*sizeof(token_data[0]) ); + memmove( token_strings, token_strings+ts_offset, cur_token*sizeof(token_strings[0]) ); + memmove( token_lens, token_lens+ts_offset, cur_token*sizeof(token_lens[0]) ); + } +} + +void Scanner::directToParser( Parser *toParser, const char *tokFileName, int tokLine, + int tokColumn, int type, char *tokdata, int toklen ) +{ + InputLoc loc; + + #ifdef LOG_TOKENS + cerr << "scanner:" << tokLine << ":" << tokColumn << + ": sending token to the parser " << Parser_lelNames[type]; + cerr << " " << toklen; + if ( tokdata != 0 ) + cerr << " " << tokdata; + cerr << endl; + #endif + + loc.fileName = tokFileName; + loc.line = tokLine; + loc.col = tokColumn; + + toParser->token( loc, type, tokdata, toklen ); +} + +void Scanner::importToken( int token, char *start, char *end ) +{ + if ( cur_token == max_tokens ) + flushImport(); + + token_data[cur_token] = token; + if ( start == 0 ) { + token_strings[cur_token] = 0; + token_lens[cur_token] = 0; + } + else { + int toklen = end-start; + token_lens[cur_token] = toklen; + token_strings[cur_token] = new char[toklen+1]; + memcpy( token_strings[cur_token], start, toklen ); + token_strings[cur_token][toklen] = 0; + } + cur_token++; +} + +void Scanner::pass( int token, char *start, char *end ) +{ + if ( importMachines ) + importToken( token, start, end ); + pass(); +} + +void Scanner::pass() +{ + updateCol(); + + /* If no errors and we are at the bottom of the include stack (the + * source file listed on the command line) then write out the data. */ + if ( includeDepth == 0 && machineSpec == 0 && machineName == 0 ) + id.inputItems.tail->data.write( ts, te-ts ); +} + +/* + * The scanner for processing sections, includes, imports, etc. + */ + + +#line 320 "rlscan.cpp" +static const int section_parse_start = 10; +static const int section_parse_first_final = 10; +static const int section_parse_error = 0; + +static const int section_parse_en_main = 10; + + +#line 217 "rlscan.rl" + + + +void Scanner::init( ) +{ + +#line 335 "rlscan.cpp" + { + cs = section_parse_start; + } + +#line 223 "rlscan.rl" +} + +bool Scanner::active() +{ + if ( ignoreSection ) + return false; + + if ( parser == 0 && ! parserExistsError ) { + scan_error() << "this specification has no name, nor does any previous" + " specification" << endl; + parserExistsError = true; + } + + if ( parser == 0 ) + return false; + + return true; +} + +ostream &Scanner::scan_error() +{ + /* Maintain the error count. */ + gblErrorCount += 1; + cerr << makeInputLoc( fileName, line, column ) << ": "; + return cerr; +} + +/* An approximate check for duplicate includes. Due to aliasing of files it's + * possible for duplicates to creep in. */ +bool Scanner::duplicateInclude( char *inclFileName, char *inclSectionName ) +{ + for ( IncludeHistory::Iter hi = parser->includeHistory; hi.lte(); hi++ ) { + if ( strcmp( hi->fileName, inclFileName ) == 0 && + strcmp( hi->sectionName, inclSectionName ) == 0 ) + { + return true; + } + } + return false; +} + +void Scanner::updateCol() +{ + char *from = lastnl; + if ( from == 0 ) + from = ts; + //cerr << "adding " << te - from << " to column" << endl; + column += te - from; + lastnl = 0; +} + +void Scanner::handleMachine() +{ + /* Assign a name to the machine. */ + char *machine = word; + + if ( !importMachines && inclSectionTarg == 0 ) { + ignoreSection = false; + + ParserDictEl *pdEl = id.parserDict.find( machine ); + if ( pdEl == 0 ) { + pdEl = new ParserDictEl( machine ); + pdEl->value = new Parser( fileName, machine, sectionLoc ); + pdEl->value->init(); + id.parserDict.insert( pdEl ); + id.parserList.append( pdEl->value ); + } + + parser = pdEl->value; + } + else if ( !importMachines && strcmp( inclSectionTarg, machine ) == 0 ) { + /* found include target */ + ignoreSection = false; + parser = inclToParser; + } + else { + /* ignoring section */ + ignoreSection = true; + parser = 0; + } +} + +void Scanner::handleInclude() +{ + if ( active() ) { + char *inclSectionName = word; + char **includeChecks = 0; + + /* Implement defaults for the input file and section name. */ + if ( inclSectionName == 0 ) + inclSectionName = parser->sectionName; + + if ( lit != 0 ) + includeChecks = makeIncludePathChecks( fileName, lit, lit_len ); + else { + char *test = new char[strlen(fileName)+1]; + strcpy( test, fileName ); + + includeChecks = new char*[2]; + + includeChecks[0] = test; + includeChecks[1] = 0; + } + + long found = 0; + ifstream *inFile = tryOpenInclude( includeChecks, found ); + if ( inFile == 0 ) { + scan_error() << "include: failed to locate file" << endl; + char **tried = includeChecks; + while ( *tried != 0 ) + scan_error() << "include: attempted: \"" << *tried++ << '\"' << endl; + } + else { + /* Don't include anything that's already been included. */ + if ( !duplicateInclude( includeChecks[found], inclSectionName ) ) { + parser->includeHistory.append( IncludeHistoryItem( + includeChecks[found], inclSectionName ) ); + + Scanner scanner( id, includeChecks[found], *inFile, parser, + inclSectionName, includeDepth+1, false ); + scanner.do_scan( ); + delete inFile; + } + } + } +} + +void Scanner::handleImport() +{ + if ( active() ) { + char **importChecks = makeIncludePathChecks( fileName, lit, lit_len ); + + /* Open the input file for reading. */ + long found = 0; + ifstream *inFile = tryOpenInclude( importChecks, found ); + if ( inFile == 0 ) { + scan_error() << "import: could not open import file " << + "for reading" << endl; + char **tried = importChecks; + while ( *tried != 0 ) + scan_error() << "import: attempted: \"" << *tried++ << '\"' << endl; + } + + Scanner scanner( id, importChecks[found], *inFile, parser, + 0, includeDepth+1, true ); + scanner.do_scan( ); + scanner.importToken( 0, 0, 0 ); + scanner.flushImport(); + delete inFile; + } +} + + +#line 460 "rlscan.rl" + + +void Scanner::token( int type, char c ) +{ + token( type, &c, &c + 1 ); +} + +void Scanner::token( int type ) +{ + token( type, 0, 0 ); +} + +void Scanner::token( int type, char *start, char *end ) +{ + char *tokdata = 0; + int toklen = 0; + if ( start != 0 ) { + toklen = end-start; + tokdata = new char[toklen+1]; + memcpy( tokdata, start, toklen ); + tokdata[toklen] = 0; + } + + processToken( type, tokdata, toklen ); +} + +void Scanner::processToken( int type, char *tokdata, int toklen ) +{ + int *p, *pe, *eof; + + if ( type < 0 ) + p = pe = eof = 0; + else { + p = &type; + pe = &type + 1; + eof = 0; + } + + +#line 534 "rlscan.cpp" + { + if ( p == pe ) + goto _test_eof; + switch ( cs ) + { +tr2: +#line 390 "rlscan.rl" + { handleMachine(); } + goto st10; +tr6: +#line 391 "rlscan.rl" + { handleInclude(); } + goto st10; +tr10: +#line 392 "rlscan.rl" + { handleImport(); } + goto st10; +tr13: +#line 432 "rlscan.rl" + { + if ( active() && machineSpec == 0 && machineName == 0 ) + id.inputItems.tail->writeArgs.append( 0 ); + } + goto st10; +tr14: +#line 443 "rlscan.rl" + { + /* Send the token off to the parser. */ + if ( active() ) + directToParser( parser, fileName, line, column, type, tokdata, toklen ); + } + goto st10; +st10: + if ( ++p == pe ) + goto _test_eof10; +case 10: +#line 571 "rlscan.cpp" + switch( (*p) ) { + case 191: goto st1; + case 192: goto st3; + case 193: goto st6; + case 194: goto tr18; + } + goto tr14; +st1: + if ( ++p == pe ) + goto _test_eof1; +case 1: + if ( (*p) == 128 ) + goto tr1; + goto tr0; +tr0: +#line 385 "rlscan.rl" + { scan_error() << "bad machine statement" << endl; } + goto st0; +tr3: +#line 386 "rlscan.rl" + { scan_error() << "bad include statement" << endl; } + goto st0; +tr8: +#line 387 "rlscan.rl" + { scan_error() << "bad import statement" << endl; } + goto st0; +tr11: +#line 388 "rlscan.rl" + { scan_error() << "bad write statement" << endl; } + goto st0; +#line 602 "rlscan.cpp" +st0: +cs = 0; + goto _out; +tr1: +#line 382 "rlscan.rl" + { word = tokdata; word_len = toklen; } + goto st2; +st2: + if ( ++p == pe ) + goto _test_eof2; +case 2: +#line 614 "rlscan.cpp" + if ( (*p) == 59 ) + goto tr2; + goto tr0; +st3: + if ( ++p == pe ) + goto _test_eof3; +case 3: + switch( (*p) ) { + case 128: goto tr4; + case 129: goto tr5; + } + goto tr3; +tr4: +#line 381 "rlscan.rl" + { word = lit = 0; word_len = lit_len = 0; } +#line 382 "rlscan.rl" + { word = tokdata; word_len = toklen; } + goto st4; +st4: + if ( ++p == pe ) + goto _test_eof4; +case 4: +#line 637 "rlscan.cpp" + switch( (*p) ) { + case 59: goto tr6; + case 129: goto tr7; + } + goto tr3; +tr5: +#line 381 "rlscan.rl" + { word = lit = 0; word_len = lit_len = 0; } +#line 383 "rlscan.rl" + { lit = tokdata; lit_len = toklen; } + goto st5; +tr7: +#line 383 "rlscan.rl" + { lit = tokdata; lit_len = toklen; } + goto st5; +st5: + if ( ++p == pe ) + goto _test_eof5; +case 5: +#line 657 "rlscan.cpp" + if ( (*p) == 59 ) + goto tr6; + goto tr3; +st6: + if ( ++p == pe ) + goto _test_eof6; +case 6: + if ( (*p) == 129 ) + goto tr9; + goto tr8; +tr9: +#line 383 "rlscan.rl" + { lit = tokdata; lit_len = toklen; } + goto st7; +st7: + if ( ++p == pe ) + goto _test_eof7; +case 7: +#line 676 "rlscan.cpp" + if ( (*p) == 59 ) + goto tr10; + goto tr8; +tr18: +#line 412 "rlscan.rl" + { + if ( active() && machineSpec == 0 && machineName == 0 ) { + InputItem *inputItem = new InputItem; + inputItem->type = InputItem::Write; + inputItem->loc.fileName = fileName; + inputItem->loc.line = line; + inputItem->loc.col = column; + inputItem->name = parser->sectionName; + inputItem->pd = parser->pd; + id.inputItems.append( inputItem ); + } + } + goto st8; +st8: + if ( ++p == pe ) + goto _test_eof8; +case 8: +#line 699 "rlscan.cpp" + if ( (*p) == 128 ) + goto tr12; + goto tr11; +tr12: +#line 426 "rlscan.rl" + { + if ( active() && machineSpec == 0 && machineName == 0 ) + id.inputItems.tail->writeArgs.append( strdup(tokdata) ); + } + goto st9; +st9: + if ( ++p == pe ) + goto _test_eof9; +case 9: +#line 714 "rlscan.cpp" + switch( (*p) ) { + case 59: goto tr13; + case 128: goto tr12; + } + goto tr11; + } + _test_eof10: cs = 10; goto _test_eof; + _test_eof1: cs = 1; goto _test_eof; + _test_eof2: cs = 2; goto _test_eof; + _test_eof3: cs = 3; goto _test_eof; + _test_eof4: cs = 4; goto _test_eof; + _test_eof5: cs = 5; goto _test_eof; + _test_eof6: cs = 6; goto _test_eof; + _test_eof7: cs = 7; goto _test_eof; + _test_eof8: cs = 8; goto _test_eof; + _test_eof9: cs = 9; goto _test_eof; + + _test_eof: {} + if ( p == eof ) + { + switch ( cs ) { + case 1: + case 2: +#line 385 "rlscan.rl" + { scan_error() << "bad machine statement" << endl; } + break; + case 3: + case 4: + case 5: +#line 386 "rlscan.rl" + { scan_error() << "bad include statement" << endl; } + break; + case 6: + case 7: +#line 387 "rlscan.rl" + { scan_error() << "bad import statement" << endl; } + break; + case 8: + case 9: +#line 388 "rlscan.rl" + { scan_error() << "bad write statement" << endl; } + break; +#line 757 "rlscan.cpp" + } + } + + _out: {} + } + +#line 501 "rlscan.rl" + + + updateCol(); + + /* Record the last token for use in controlling the scan of subsequent + * tokens. */ + lastToken = type; +} + +void Scanner::startSection( ) +{ + parserExistsError = false; + + sectionLoc.fileName = fileName; + sectionLoc.line = line; + sectionLoc.col = column; +} + +void Scanner::endSection( ) +{ + /* Execute the eof actions for the section parser. */ + processToken( -1, 0, 0 ); + + /* Close off the section with the parser. */ + if ( active() ) { + InputLoc loc; + loc.fileName = fileName; + loc.line = line; + loc.col = column; + + parser->token( loc, TK_EndSection, 0, 0 ); + } + + if ( includeDepth == 0 ) { + if ( machineSpec == 0 && machineName == 0 ) { + /* The end section may include a newline on the end, so + * we use the last line, which will count the newline. */ + InputItem *inputItem = new InputItem; + inputItem->type = InputItem::HostData; + inputItem->loc.line = line; + inputItem->loc.col = column; + id.inputItems.append( inputItem ); + } + } +} + +bool isAbsolutePath( const char *path ) +{ +#ifdef _WIN32 + return isalpha( path[0] ) && path[1] == ':' && path[2] == '\\'; +#else + return path[0] == '/'; +#endif +} + +char **Scanner::makeIncludePathChecks( const char *thisFileName, + const char *fileName, int fnlen ) +{ + char **checks = 0; + long nextCheck = 0; + long length = 0; + bool caseInsensitive = false; + char *data = prepareLitString( InputLoc(), fileName, fnlen, + length, caseInsensitive ); + + /* Absolute path? */ + if ( isAbsolutePath( data ) ) { + checks = new char*[2]; + checks[nextCheck++] = data; + } + else { + checks = new char*[2 + id.includePaths.length()]; + + /* Search from the the location of the current file. */ + const char *lastSlash = strrchr( thisFileName, PATH_SEP ); + if ( lastSlash == 0 ) + checks[nextCheck++] = data; + else { + long givenPathLen = (lastSlash - thisFileName) + 1; + long checklen = givenPathLen + length; + char *check = new char[checklen+1]; + memcpy( check, thisFileName, givenPathLen ); + memcpy( check+givenPathLen, data, length ); + check[checklen] = 0; + checks[nextCheck++] = check; + } + + /* Search from the include paths given on the command line. */ + for ( ArgsVector::Iter incp = id.includePaths; incp.lte(); incp++ ) { + long pathLen = strlen( *incp ); + long checkLen = pathLen + 1 + length; + char *check = new char[checkLen+1]; + memcpy( check, *incp, pathLen ); + check[pathLen] = PATH_SEP; + memcpy( check+pathLen+1, data, length ); + check[checkLen] = 0; + checks[nextCheck++] = check; + } + } + + checks[nextCheck] = 0; + return checks; +} + +ifstream *Scanner::tryOpenInclude( char **pathChecks, long &found ) +{ + char **check = pathChecks; + ifstream *inFile = new ifstream; + + while ( *check != 0 ) { + inFile->open( *check ); + if ( inFile->is_open() ) { + found = check - pathChecks; + return inFile; + } + check += 1; + } + + found = -1; + delete inFile; + return 0; +} + + +#line 1162 "rlscan.rl" + + + +#line 893 "rlscan.cpp" +static const int rlscan_start = 38; +static const int rlscan_first_final = 38; +static const int rlscan_error = 0; + +static const int rlscan_en_inline_code_ruby = 52; +static const int rlscan_en_inline_code = 95; +static const int rlscan_en_or_literal = 137; +static const int rlscan_en_ragel_re_literal = 139; +static const int rlscan_en_write_statement = 143; +static const int rlscan_en_parser_def = 146; +static const int rlscan_en_main_ruby = 253; +static const int rlscan_en_main = 38; + + +#line 1165 "rlscan.rl" + +void Scanner::do_scan() +{ + int bufsize = 8; + char *buf = new char[bufsize]; + int cs, act, have = 0; + int top; + + /* The stack is two deep, one level for going into ragel defs from the main + * machines which process outside code, and another for going into or literals + * from either a ragel spec, or a regular expression. */ + int stack[2]; + int curly_count = 0; + bool execute = true; + bool singleLineSpec = false; + InlineBlockType inlineBlockType = CurlyDelimited; + + /* Init the section parser and the character scanner. */ + init(); + +#line 929 "rlscan.cpp" + { + cs = rlscan_start; + top = 0; + ts = 0; + te = 0; + act = 0; + } + +#line 1185 "rlscan.rl" + + /* Set up the start state. FIXME: After 5.20 is released the nocs write + * init option should be used, the main machine eliminated and this statement moved + * above the write init. */ + if ( hostLang->lang == HostLang::Ruby ) + cs = rlscan_en_main_ruby; + else + cs = rlscan_en_main; + + while ( execute ) { + char *p = buf + have; + int space = bufsize - have; + + if ( space == 0 ) { + /* We filled up the buffer trying to scan a token. Grow it. */ + bufsize = bufsize * 2; + char *newbuf = new char[bufsize]; + + /* Recompute p and space. */ + p = newbuf + have; + space = bufsize - have; + + /* Patch up pointers possibly in use. */ + if ( ts != 0 ) + ts = newbuf + ( ts - buf ); + te = newbuf + ( te - buf ); + + /* Copy the new buffer in. */ + memcpy( newbuf, buf, have ); + delete[] buf; + buf = newbuf; + } + + input.read( p, space ); + int len = input.gcount(); + char *pe = p + len; + + /* If we see eof then append the eof var. */ + char *eof = 0; + if ( len == 0 ) { + eof = pe; + execute = false; + } + + +#line 984 "rlscan.cpp" + { + if ( p == pe ) + goto _test_eof; + goto _resume; + +_again: + switch ( cs ) { + case 38: goto st38; + case 39: goto st39; + case 40: goto st40; + case 1: goto st1; + case 2: goto st2; + case 41: goto st41; + case 42: goto st42; + case 43: goto st43; + case 3: goto st3; + case 4: goto st4; + case 44: goto st44; + case 5: goto st5; + case 6: goto st6; + case 7: goto st7; + case 45: goto st45; + case 46: goto st46; + case 47: goto st47; + case 48: goto st48; + case 49: goto st49; + case 50: goto st50; + case 51: goto st51; + case 52: goto st52; + case 53: goto st53; + case 54: goto st54; + case 8: goto st8; + case 9: goto st9; + case 55: goto st55; + case 10: goto st10; + case 56: goto st56; + case 11: goto st11; + case 12: goto st12; + case 57: goto st57; + case 13: goto st13; + case 14: goto st14; + case 58: goto st58; + case 59: goto st59; + case 15: goto st15; + case 60: goto st60; + case 61: goto st61; + case 62: goto st62; + case 63: goto st63; + case 64: goto st64; + case 65: goto st65; + case 66: goto st66; + case 67: goto st67; + case 68: goto st68; + case 69: goto st69; + case 70: goto st70; + case 71: goto st71; + case 72: goto st72; + case 73: goto st73; + case 74: goto st74; + case 75: goto st75; + case 76: goto st76; + case 77: goto st77; + case 78: goto st78; + case 79: goto st79; + case 80: goto st80; + case 81: goto st81; + case 82: goto st82; + case 83: goto st83; + case 84: goto st84; + case 85: goto st85; + case 86: goto st86; + case 87: goto st87; + case 88: goto st88; + case 89: goto st89; + case 90: goto st90; + case 91: goto st91; + case 92: goto st92; + case 93: goto st93; + case 94: goto st94; + case 95: goto st95; + case 96: goto st96; + case 97: goto st97; + case 16: goto st16; + case 17: goto st17; + case 98: goto st98; + case 18: goto st18; + case 19: goto st19; + case 99: goto st99; + case 20: goto st20; + case 21: goto st21; + case 22: goto st22; + case 100: goto st100; + case 101: goto st101; + case 23: goto st23; + case 102: goto st102; + case 103: goto st103; + case 104: goto st104; + case 105: goto st105; + case 106: goto st106; + case 107: goto st107; + case 108: goto st108; + case 109: goto st109; + case 110: goto st110; + case 111: goto st111; + case 112: goto st112; + case 113: goto st113; + case 114: goto st114; + case 115: goto st115; + case 116: goto st116; + case 117: goto st117; + case 118: goto st118; + case 119: goto st119; + case 120: goto st120; + case 121: goto st121; + case 122: goto st122; + case 123: goto st123; + case 124: goto st124; + case 125: goto st125; + case 126: goto st126; + case 127: goto st127; + case 128: goto st128; + case 129: goto st129; + case 130: goto st130; + case 131: goto st131; + case 132: goto st132; + case 133: goto st133; + case 134: goto st134; + case 135: goto st135; + case 136: goto st136; + case 137: goto st137; + case 138: goto st138; + case 139: goto st139; + case 140: goto st140; + case 141: goto st141; + case 142: goto st142; + case 143: goto st143; + case 0: goto st0; + case 144: goto st144; + case 145: goto st145; + case 146: goto st146; + case 147: goto st147; + case 148: goto st148; + case 24: goto st24; + case 149: goto st149; + case 25: goto st25; + case 150: goto st150; + case 26: goto st26; + case 151: goto st151; + case 152: goto st152; + case 153: goto st153; + case 27: goto st27; + case 28: goto st28; + case 154: goto st154; + case 155: goto st155; + case 156: goto st156; + case 157: goto st157; + case 158: goto st158; + case 29: goto st29; + case 159: goto st159; + case 160: goto st160; + case 161: goto st161; + case 162: goto st162; + case 163: goto st163; + case 164: goto st164; + case 165: goto st165; + case 166: goto st166; + case 167: goto st167; + case 168: goto st168; + case 169: goto st169; + case 170: goto st170; + case 171: goto st171; + case 172: goto st172; + case 173: goto st173; + case 174: goto st174; + case 175: goto st175; + case 176: goto st176; + case 177: goto st177; + case 178: goto st178; + case 179: goto st179; + case 180: goto st180; + case 181: goto st181; + case 182: goto st182; + case 183: goto st183; + case 184: goto st184; + case 185: goto st185; + case 186: goto st186; + case 187: goto st187; + case 188: goto st188; + case 189: goto st189; + case 190: goto st190; + case 191: goto st191; + case 192: goto st192; + case 193: goto st193; + case 194: goto st194; + case 195: goto st195; + case 196: goto st196; + case 197: goto st197; + case 198: goto st198; + case 199: goto st199; + case 200: goto st200; + case 201: goto st201; + case 202: goto st202; + case 203: goto st203; + case 204: goto st204; + case 205: goto st205; + case 206: goto st206; + case 207: goto st207; + case 208: goto st208; + case 209: goto st209; + case 210: goto st210; + case 211: goto st211; + case 212: goto st212; + case 213: goto st213; + case 214: goto st214; + case 215: goto st215; + case 216: goto st216; + case 217: goto st217; + case 218: goto st218; + case 219: goto st219; + case 220: goto st220; + case 221: goto st221; + case 222: goto st222; + case 223: goto st223; + case 224: goto st224; + case 225: goto st225; + case 226: goto st226; + case 227: goto st227; + case 228: goto st228; + case 229: goto st229; + case 230: goto st230; + case 231: goto st231; + case 232: goto st232; + case 233: goto st233; + case 234: goto st234; + case 235: goto st235; + case 236: goto st236; + case 237: goto st237; + case 238: goto st238; + case 239: goto st239; + case 240: goto st240; + case 241: goto st241; + case 242: goto st242; + case 243: goto st243; + case 244: goto st244; + case 245: goto st245; + case 246: goto st246; + case 247: goto st247; + case 248: goto st248; + case 249: goto st249; + case 250: goto st250; + case 251: goto st251; + case 252: goto st252; + case 30: goto st30; + case 253: goto st253; + case 254: goto st254; + case 255: goto st255; + case 31: goto st31; + case 32: goto st32; + case 256: goto st256; + case 33: goto st33; + case 257: goto st257; + case 258: goto st258; + case 259: goto st259; + case 34: goto st34; + case 35: goto st35; + case 260: goto st260; + case 36: goto st36; + case 37: goto st37; + case 261: goto st261; + case 262: goto st262; + default: break; + } + + if ( ++p == pe ) + goto _test_eof; +_resume: + switch ( cs ) + { +tr0: +#line 1160 "rlscan.rl" + {{p = ((te))-1;}{ pass( *ts, 0, 0 ); }} + goto st38; +tr3: +#line 1144 "rlscan.rl" + {te = p+1;{ pass( IMP_Literal, ts, te ); }} + goto st38; +tr11: +#line 1143 "rlscan.rl" + {te = p+1;{ pass(); }} + goto st38; +tr13: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } +#line 1143 "rlscan.rl" + {te = p+1;{ pass(); }} + goto st38; +tr71: +#line 1160 "rlscan.rl" + {te = p+1;{ pass( *ts, 0, 0 ); }} + goto st38; +tr72: +#line 1159 "rlscan.rl" + {te = p+1;} + goto st38; +tr82: +#line 1158 "rlscan.rl" + {te = p;p--;{ pass(); }} + goto st38; +tr83: +#line 1160 "rlscan.rl" + {te = p;p--;{ pass( *ts, 0, 0 ); }} + goto st38; +tr85: +#line 1152 "rlscan.rl" + {te = p;p--;{ + updateCol(); + singleLineSpec = true; + startSection(); + {stack[top++] = 38; goto st146;} + }} + goto st38; +tr86: +#line 1146 "rlscan.rl" + {te = p+1;{ + updateCol(); + singleLineSpec = false; + startSection(); + {stack[top++] = 38; goto st146;} + }} + goto st38; +tr87: +#line 1142 "rlscan.rl" + {te = p;p--;{ pass( IMP_UInt, ts, te ); }} + goto st38; +tr88: +#line 1 "rlscan.rl" + { switch( act ) { + case 176: + {{p = ((te))-1;} pass( IMP_Define, 0, 0 ); } + break; + case 177: + {{p = ((te))-1;} pass( IMP_Word, ts, te ); } + break; + } + } + goto st38; +tr89: +#line 1141 "rlscan.rl" + {te = p;p--;{ pass( IMP_Word, ts, te ); }} + goto st38; +st38: +#line 1 "rlscan.rl" + {ts = 0;} + if ( ++p == pe ) + goto _test_eof38; +case 38: +#line 1 "rlscan.rl" + {ts = p;} +#line 1347 "rlscan.cpp" + switch( (*p) ) { + case 0: goto tr72; + case 9: goto st39; + case 10: goto tr74; + case 32: goto st39; + case 34: goto tr75; + case 37: goto st41; + case 39: goto tr77; + case 47: goto tr78; + case 95: goto tr80; + case 100: goto st47; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st45; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr80; + } else + goto tr80; + goto tr71; +tr74: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st39; +st39: + if ( ++p == pe ) + goto _test_eof39; +case 39: +#line 1381 "rlscan.cpp" + switch( (*p) ) { + case 9: goto st39; + case 10: goto tr74; + case 32: goto st39; + } + goto tr82; +tr75: +#line 1 "rlscan.rl" + {te = p+1;} + goto st40; +st40: + if ( ++p == pe ) + goto _test_eof40; +case 40: +#line 1396 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr2; + case 34: goto tr3; + case 92: goto st2; + } + goto st1; +tr2: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st1; +st1: + if ( ++p == pe ) + goto _test_eof1; +case 1: +#line 1415 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr2; + case 34: goto tr3; + case 92: goto st2; + } + goto st1; +st2: + if ( ++p == pe ) + goto _test_eof2; +case 2: + if ( (*p) == 10 ) + goto tr2; + goto st1; +st41: + if ( ++p == pe ) + goto _test_eof41; +case 41: + if ( (*p) == 37 ) + goto st42; + goto tr83; +st42: + if ( ++p == pe ) + goto _test_eof42; +case 42: + if ( (*p) == 123 ) + goto tr86; + goto tr85; +tr77: +#line 1 "rlscan.rl" + {te = p+1;} + goto st43; +st43: + if ( ++p == pe ) + goto _test_eof43; +case 43: +#line 1451 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr6; + case 39: goto tr3; + case 92: goto st4; + } + goto st3; +tr6: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st3; +st3: + if ( ++p == pe ) + goto _test_eof3; +case 3: +#line 1470 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr6; + case 39: goto tr3; + case 92: goto st4; + } + goto st3; +st4: + if ( ++p == pe ) + goto _test_eof4; +case 4: + if ( (*p) == 10 ) + goto tr6; + goto st3; +tr78: +#line 1 "rlscan.rl" + {te = p+1;} + goto st44; +st44: + if ( ++p == pe ) + goto _test_eof44; +case 44: +#line 1492 "rlscan.cpp" + switch( (*p) ) { + case 42: goto st5; + case 47: goto st7; + } + goto tr83; +tr9: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st5; +st5: + if ( ++p == pe ) + goto _test_eof5; +case 5: +#line 1510 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr9; + case 42: goto st6; + } + goto st5; +st6: + if ( ++p == pe ) + goto _test_eof6; +case 6: + switch( (*p) ) { + case 10: goto tr9; + case 42: goto st6; + case 47: goto tr11; + } + goto st5; +st7: + if ( ++p == pe ) + goto _test_eof7; +case 7: + if ( (*p) == 10 ) + goto tr13; + goto st7; +st45: + if ( ++p == pe ) + goto _test_eof45; +case 45: + if ( 48 <= (*p) && (*p) <= 57 ) + goto st45; + goto tr87; +tr80: +#line 1 "rlscan.rl" + {te = p+1;} +#line 1141 "rlscan.rl" + {act = 177;} + goto st46; +tr94: +#line 1 "rlscan.rl" + {te = p+1;} +#line 1140 "rlscan.rl" + {act = 176;} + goto st46; +st46: + if ( ++p == pe ) + goto _test_eof46; +case 46: +#line 1556 "rlscan.cpp" + if ( (*p) == 95 ) + goto tr80; + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr80; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr80; + } else + goto tr80; + goto tr88; +st47: + if ( ++p == pe ) + goto _test_eof47; +case 47: + switch( (*p) ) { + case 95: goto tr80; + case 101: goto st48; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr80; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr80; + } else + goto tr80; + goto tr89; +st48: + if ( ++p == pe ) + goto _test_eof48; +case 48: + switch( (*p) ) { + case 95: goto tr80; + case 102: goto st49; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr80; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr80; + } else + goto tr80; + goto tr89; +st49: + if ( ++p == pe ) + goto _test_eof49; +case 49: + switch( (*p) ) { + case 95: goto tr80; + case 105: goto st50; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr80; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr80; + } else + goto tr80; + goto tr89; +st50: + if ( ++p == pe ) + goto _test_eof50; +case 50: + switch( (*p) ) { + case 95: goto tr80; + case 110: goto st51; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr80; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr80; + } else + goto tr80; + goto tr89; +st51: + if ( ++p == pe ) + goto _test_eof51; +case 51: + switch( (*p) ) { + case 95: goto tr80; + case 101: goto tr94; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr80; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr80; + } else + goto tr80; + goto tr89; +tr14: +#line 759 "rlscan.rl" + {{p = ((te))-1;}{ token( IL_Symbol, ts, te ); }} + goto st52; +tr17: +#line 705 "rlscan.rl" + {te = p+1;{ token( IL_Literal, ts, te ); }} + goto st52; +tr20: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } +#line 712 "rlscan.rl" + {te = p+1;{ token( IL_Comment, ts, te ); }} + goto st52; +tr27: +#line 701 "rlscan.rl" + {{p = ((te))-1;}{ token( TK_UInt, ts, te ); }} + goto st52; +tr95: +#line 759 "rlscan.rl" + {te = p+1;{ token( IL_Symbol, ts, te ); }} + goto st52; +tr96: +#line 754 "rlscan.rl" + {te = p+1;{ + scan_error() << "unterminated code block" << endl; + }} + goto st52; +tr102: +#line 734 "rlscan.rl" + {te = p+1;{ token( *ts, ts, te ); }} + goto st52; +tr103: +#line 729 "rlscan.rl" + {te = p+1;{ + whitespaceOn = true; + token( *ts, ts, te ); + }} + goto st52; +tr108: +#line 722 "rlscan.rl" + {te = p+1;{ + whitespaceOn = true; + token( *ts, ts, te ); + if ( inlineBlockType == SemiTerminated ) + {cs = stack[--top];goto _again;} + }} + goto st52; +tr111: +#line 736 "rlscan.rl" + {te = p+1;{ + token( IL_Symbol, ts, te ); + curly_count += 1; + }} + goto st52; +tr112: +#line 741 "rlscan.rl" + {te = p+1;{ + if ( --curly_count == 0 && inlineBlockType == CurlyDelimited ) { + /* Inline code block ends. */ + token( '}' ); + {cs = stack[--top];goto _again;} + } + else { + /* Either a semi terminated inline block or only the closing + * brace of some inner scope, not the block's closing brace. */ + token( IL_Symbol, ts, te ); + } + }} + goto st52; +tr113: +#line 707 "rlscan.rl" + {te = p;p--;{ + if ( whitespaceOn ) + token( IL_WhiteSpace, ts, te ); + }} + goto st52; +tr114: +#line 759 "rlscan.rl" + {te = p;p--;{ token( IL_Symbol, ts, te ); }} + goto st52; +tr115: +#line 701 "rlscan.rl" + {te = p;p--;{ token( TK_UInt, ts, te ); }} + goto st52; +tr117: +#line 702 "rlscan.rl" + {te = p;p--;{ token( TK_Hex, ts, te ); }} + goto st52; +tr118: +#line 714 "rlscan.rl" + {te = p+1;{ token( TK_NameSep, ts, te ); }} + goto st52; +tr119: +#line 1 "rlscan.rl" + { switch( act ) { + case 1: + {{p = ((te))-1;} token( KW_PChar ); } + break; + case 3: + {{p = ((te))-1;} token( KW_CurState ); } + break; + case 4: + {{p = ((te))-1;} token( KW_TargState ); } + break; + case 5: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Entry ); + } + break; + case 6: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Hold ); + } + break; + case 7: + {{p = ((te))-1;} token( KW_Exec, 0, 0 ); } + break; + case 8: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Goto ); + } + break; + case 9: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Next ); + } + break; + case 10: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Call ); + } + break; + case 11: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Ret ); + } + break; + case 12: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Break ); + } + break; + case 13: + {{p = ((te))-1;} token( TK_Word, ts, te ); } + break; + } + } + goto st52; +tr120: +#line 699 "rlscan.rl" + {te = p;p--;{ token( TK_Word, ts, te ); }} + goto st52; +tr134: +#line 664 "rlscan.rl" + {te = p;p--;{ token( KW_Char ); }} + goto st52; +st52: +#line 1 "rlscan.rl" + {ts = 0;} + if ( ++p == pe ) + goto _test_eof52; +case 52: +#line 1 "rlscan.rl" + {ts = p;} +#line 1829 "rlscan.cpp" + switch( (*p) ) { + case 0: goto tr96; + case 9: goto st53; + case 10: goto tr98; + case 32: goto st53; + case 34: goto tr99; + case 35: goto tr100; + case 39: goto tr101; + case 40: goto tr102; + case 44: goto tr102; + case 47: goto tr104; + case 48: goto tr105; + case 58: goto st61; + case 59: goto tr108; + case 95: goto tr109; + case 102: goto st63; + case 123: goto tr111; + case 125: goto tr112; + } + if ( (*p) < 49 ) { + if ( 41 <= (*p) && (*p) <= 42 ) + goto tr103; + } else if ( (*p) > 57 ) { + if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else if ( (*p) >= 65 ) + goto tr109; + } else + goto st59; + goto tr95; +tr98: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st53; +st53: + if ( ++p == pe ) + goto _test_eof53; +case 53: +#line 1873 "rlscan.cpp" + switch( (*p) ) { + case 9: goto st53; + case 10: goto tr98; + case 32: goto st53; + } + goto tr113; +tr99: +#line 1 "rlscan.rl" + {te = p+1;} + goto st54; +st54: + if ( ++p == pe ) + goto _test_eof54; +case 54: +#line 1888 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr16; + case 34: goto tr17; + case 92: goto st9; + } + goto st8; +tr16: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st8; +st8: + if ( ++p == pe ) + goto _test_eof8; +case 8: +#line 1907 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr16; + case 34: goto tr17; + case 92: goto st9; + } + goto st8; +st9: + if ( ++p == pe ) + goto _test_eof9; +case 9: + if ( (*p) == 10 ) + goto tr16; + goto st8; +tr100: +#line 1 "rlscan.rl" + {te = p+1;} + goto st55; +st55: + if ( ++p == pe ) + goto _test_eof55; +case 55: +#line 1929 "rlscan.cpp" + if ( (*p) == 10 ) + goto tr20; + goto st10; +st10: + if ( ++p == pe ) + goto _test_eof10; +case 10: + if ( (*p) == 10 ) + goto tr20; + goto st10; +tr101: +#line 1 "rlscan.rl" + {te = p+1;} + goto st56; +st56: + if ( ++p == pe ) + goto _test_eof56; +case 56: +#line 1948 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr22; + case 39: goto tr17; + case 92: goto st12; + } + goto st11; +tr22: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st11; +st11: + if ( ++p == pe ) + goto _test_eof11; +case 11: +#line 1967 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr22; + case 39: goto tr17; + case 92: goto st12; + } + goto st11; +st12: + if ( ++p == pe ) + goto _test_eof12; +case 12: + if ( (*p) == 10 ) + goto tr22; + goto st11; +tr104: +#line 1 "rlscan.rl" + {te = p+1;} + goto st57; +st57: + if ( ++p == pe ) + goto _test_eof57; +case 57: +#line 1989 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr25; + case 47: goto tr17; + case 92: goto st14; + } + goto st13; +tr25: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st13; +st13: + if ( ++p == pe ) + goto _test_eof13; +case 13: +#line 2008 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr25; + case 47: goto tr17; + case 92: goto st14; + } + goto st13; +st14: + if ( ++p == pe ) + goto _test_eof14; +case 14: + if ( (*p) == 10 ) + goto tr25; + goto st13; +tr105: +#line 1 "rlscan.rl" + {te = p+1;} + goto st58; +st58: + if ( ++p == pe ) + goto _test_eof58; +case 58: +#line 2030 "rlscan.cpp" + if ( (*p) == 120 ) + goto st15; + if ( 48 <= (*p) && (*p) <= 57 ) + goto st59; + goto tr115; +st59: + if ( ++p == pe ) + goto _test_eof59; +case 59: + if ( 48 <= (*p) && (*p) <= 57 ) + goto st59; + goto tr115; +st15: + if ( ++p == pe ) + goto _test_eof15; +case 15: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st60; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st60; + } else + goto st60; + goto tr27; +st60: + if ( ++p == pe ) + goto _test_eof60; +case 60: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st60; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st60; + } else + goto st60; + goto tr117; +st61: + if ( ++p == pe ) + goto _test_eof61; +case 61: + if ( (*p) == 58 ) + goto tr118; + goto tr114; +tr109: +#line 1 "rlscan.rl" + {te = p+1;} +#line 699 "rlscan.rl" + {act = 13;} + goto st62; +tr133: +#line 1 "rlscan.rl" + {te = p+1;} +#line 694 "rlscan.rl" + {act = 12;} + goto st62; +tr138: +#line 1 "rlscan.rl" + {te = p+1;} +#line 686 "rlscan.rl" + {act = 10;} + goto st62; +tr140: +#line 1 "rlscan.rl" + {te = p+1;} +#line 665 "rlscan.rl" + {act = 3;} + goto st62; +tr145: +#line 1 "rlscan.rl" + {te = p+1;} +#line 667 "rlscan.rl" + {act = 5;} + goto st62; +tr147: +#line 1 "rlscan.rl" + {te = p+1;} +#line 677 "rlscan.rl" + {act = 7;} + goto st62; +tr150: +#line 1 "rlscan.rl" + {te = p+1;} +#line 678 "rlscan.rl" + {act = 8;} + goto st62; +tr153: +#line 1 "rlscan.rl" + {te = p+1;} +#line 673 "rlscan.rl" + {act = 6;} + goto st62; +tr156: +#line 1 "rlscan.rl" + {te = p+1;} +#line 682 "rlscan.rl" + {act = 9;} + goto st62; +tr157: +#line 1 "rlscan.rl" + {te = p+1;} +#line 663 "rlscan.rl" + {act = 1;} + goto st62; +tr159: +#line 1 "rlscan.rl" + {te = p+1;} +#line 690 "rlscan.rl" + {act = 11;} + goto st62; +tr163: +#line 1 "rlscan.rl" + {te = p+1;} +#line 666 "rlscan.rl" + {act = 4;} + goto st62; +st62: + if ( ++p == pe ) + goto _test_eof62; +case 62: +#line 2152 "rlscan.cpp" + if ( (*p) == 95 ) + goto tr109; + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr119; +st63: + if ( ++p == pe ) + goto _test_eof63; +case 63: + switch( (*p) ) { + case 95: goto tr109; + case 98: goto st64; + case 99: goto st68; + case 101: goto st73; + case 103: goto st79; + case 104: goto st82; + case 110: goto st85; + case 112: goto st88; + case 114: goto st89; + case 116: goto st91; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st64: + if ( ++p == pe ) + goto _test_eof64; +case 64: + switch( (*p) ) { + case 95: goto tr109; + case 114: goto st65; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st65: + if ( ++p == pe ) + goto _test_eof65; +case 65: + switch( (*p) ) { + case 95: goto tr109; + case 101: goto st66; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st66: + if ( ++p == pe ) + goto _test_eof66; +case 66: + switch( (*p) ) { + case 95: goto tr109; + case 97: goto st67; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st67: + if ( ++p == pe ) + goto _test_eof67; +case 67: + switch( (*p) ) { + case 95: goto tr109; + case 107: goto tr133; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st68: + if ( ++p == pe ) + goto _test_eof68; +case 68: + switch( (*p) ) { + case 95: goto tr109; + case 97: goto st69; + case 117: goto st71; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr134; +st69: + if ( ++p == pe ) + goto _test_eof69; +case 69: + switch( (*p) ) { + case 95: goto tr109; + case 108: goto st70; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st70: + if ( ++p == pe ) + goto _test_eof70; +case 70: + switch( (*p) ) { + case 95: goto tr109; + case 108: goto tr138; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st71: + if ( ++p == pe ) + goto _test_eof71; +case 71: + switch( (*p) ) { + case 95: goto tr109; + case 114: goto st72; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st72: + if ( ++p == pe ) + goto _test_eof72; +case 72: + switch( (*p) ) { + case 95: goto tr109; + case 115: goto tr140; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st73: + if ( ++p == pe ) + goto _test_eof73; +case 73: + switch( (*p) ) { + case 95: goto tr109; + case 110: goto st74; + case 120: goto st77; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st74: + if ( ++p == pe ) + goto _test_eof74; +case 74: + switch( (*p) ) { + case 95: goto tr109; + case 116: goto st75; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st75: + if ( ++p == pe ) + goto _test_eof75; +case 75: + switch( (*p) ) { + case 95: goto tr109; + case 114: goto st76; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st76: + if ( ++p == pe ) + goto _test_eof76; +case 76: + switch( (*p) ) { + case 95: goto tr109; + case 121: goto tr145; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st77: + if ( ++p == pe ) + goto _test_eof77; +case 77: + switch( (*p) ) { + case 95: goto tr109; + case 101: goto st78; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st78: + if ( ++p == pe ) + goto _test_eof78; +case 78: + switch( (*p) ) { + case 95: goto tr109; + case 99: goto tr147; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st79: + if ( ++p == pe ) + goto _test_eof79; +case 79: + switch( (*p) ) { + case 95: goto tr109; + case 111: goto st80; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st80: + if ( ++p == pe ) + goto _test_eof80; +case 80: + switch( (*p) ) { + case 95: goto tr109; + case 116: goto st81; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st81: + if ( ++p == pe ) + goto _test_eof81; +case 81: + switch( (*p) ) { + case 95: goto tr109; + case 111: goto tr150; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st82: + if ( ++p == pe ) + goto _test_eof82; +case 82: + switch( (*p) ) { + case 95: goto tr109; + case 111: goto st83; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st83: + if ( ++p == pe ) + goto _test_eof83; +case 83: + switch( (*p) ) { + case 95: goto tr109; + case 108: goto st84; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st84: + if ( ++p == pe ) + goto _test_eof84; +case 84: + switch( (*p) ) { + case 95: goto tr109; + case 100: goto tr153; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st85: + if ( ++p == pe ) + goto _test_eof85; +case 85: + switch( (*p) ) { + case 95: goto tr109; + case 101: goto st86; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st86: + if ( ++p == pe ) + goto _test_eof86; +case 86: + switch( (*p) ) { + case 95: goto tr109; + case 120: goto st87; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st87: + if ( ++p == pe ) + goto _test_eof87; +case 87: + switch( (*p) ) { + case 95: goto tr109; + case 116: goto tr156; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st88: + if ( ++p == pe ) + goto _test_eof88; +case 88: + switch( (*p) ) { + case 95: goto tr109; + case 99: goto tr157; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st89: + if ( ++p == pe ) + goto _test_eof89; +case 89: + switch( (*p) ) { + case 95: goto tr109; + case 101: goto st90; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st90: + if ( ++p == pe ) + goto _test_eof90; +case 90: + switch( (*p) ) { + case 95: goto tr109; + case 116: goto tr159; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st91: + if ( ++p == pe ) + goto _test_eof91; +case 91: + switch( (*p) ) { + case 95: goto tr109; + case 97: goto st92; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st92: + if ( ++p == pe ) + goto _test_eof92; +case 92: + switch( (*p) ) { + case 95: goto tr109; + case 114: goto st93; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st93: + if ( ++p == pe ) + goto _test_eof93; +case 93: + switch( (*p) ) { + case 95: goto tr109; + case 103: goto st94; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st94: + if ( ++p == pe ) + goto _test_eof94; +case 94: + switch( (*p) ) { + case 95: goto tr109; + case 115: goto tr163; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +tr29: +#line 862 "rlscan.rl" + {{p = ((te))-1;}{ token( IL_Symbol, ts, te ); }} + goto st95; +tr32: +#line 808 "rlscan.rl" + {te = p+1;{ token( IL_Literal, ts, te ); }} + goto st95; +tr40: +#line 815 "rlscan.rl" + {te = p+1;{ token( IL_Comment, ts, te ); }} + goto st95; +tr42: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } +#line 815 "rlscan.rl" + {te = p+1;{ token( IL_Comment, ts, te ); }} + goto st95; +tr43: +#line 804 "rlscan.rl" + {{p = ((te))-1;}{ token( TK_UInt, ts, te ); }} + goto st95; +tr164: +#line 862 "rlscan.rl" + {te = p+1;{ token( IL_Symbol, ts, te ); }} + goto st95; +tr165: +#line 857 "rlscan.rl" + {te = p+1;{ + scan_error() << "unterminated code block" << endl; + }} + goto st95; +tr170: +#line 837 "rlscan.rl" + {te = p+1;{ token( *ts, ts, te ); }} + goto st95; +tr171: +#line 832 "rlscan.rl" + {te = p+1;{ + whitespaceOn = true; + token( *ts, ts, te ); + }} + goto st95; +tr176: +#line 825 "rlscan.rl" + {te = p+1;{ + whitespaceOn = true; + token( *ts, ts, te ); + if ( inlineBlockType == SemiTerminated ) + {cs = stack[--top];goto _again;} + }} + goto st95; +tr179: +#line 839 "rlscan.rl" + {te = p+1;{ + token( IL_Symbol, ts, te ); + curly_count += 1; + }} + goto st95; +tr180: +#line 844 "rlscan.rl" + {te = p+1;{ + if ( --curly_count == 0 && inlineBlockType == CurlyDelimited ) { + /* Inline code block ends. */ + token( '}' ); + {cs = stack[--top];goto _again;} + } + else { + /* Either a semi terminated inline block or only the closing + * brace of some inner scope, not the block's closing brace. */ + token( IL_Symbol, ts, te ); + } + }} + goto st95; +tr181: +#line 810 "rlscan.rl" + {te = p;p--;{ + if ( whitespaceOn ) + token( IL_WhiteSpace, ts, te ); + }} + goto st95; +tr182: +#line 862 "rlscan.rl" + {te = p;p--;{ token( IL_Symbol, ts, te ); }} + goto st95; +tr183: +#line 804 "rlscan.rl" + {te = p;p--;{ token( TK_UInt, ts, te ); }} + goto st95; +tr185: +#line 805 "rlscan.rl" + {te = p;p--;{ token( TK_Hex, ts, te ); }} + goto st95; +tr186: +#line 817 "rlscan.rl" + {te = p+1;{ token( TK_NameSep, ts, te ); }} + goto st95; +tr187: +#line 1 "rlscan.rl" + { switch( act ) { + case 27: + {{p = ((te))-1;} token( KW_PChar ); } + break; + case 29: + {{p = ((te))-1;} token( KW_CurState ); } + break; + case 30: + {{p = ((te))-1;} token( KW_TargState ); } + break; + case 31: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Entry ); + } + break; + case 32: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Hold ); + } + break; + case 33: + {{p = ((te))-1;} token( KW_Exec, 0, 0 ); } + break; + case 34: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Goto ); + } + break; + case 35: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Next ); + } + break; + case 36: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Call ); + } + break; + case 37: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Ret ); + } + break; + case 38: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Break ); + } + break; + case 39: + {{p = ((te))-1;} token( TK_Word, ts, te ); } + break; + } + } + goto st95; +tr188: +#line 802 "rlscan.rl" + {te = p;p--;{ token( TK_Word, ts, te ); }} + goto st95; +tr202: +#line 767 "rlscan.rl" + {te = p;p--;{ token( KW_Char ); }} + goto st95; +st95: +#line 1 "rlscan.rl" + {ts = 0;} + if ( ++p == pe ) + goto _test_eof95; +case 95: +#line 1 "rlscan.rl" + {ts = p;} +#line 2898 "rlscan.cpp" + switch( (*p) ) { + case 0: goto tr165; + case 9: goto st96; + case 10: goto tr167; + case 32: goto st96; + case 34: goto tr168; + case 39: goto tr169; + case 40: goto tr170; + case 44: goto tr170; + case 47: goto tr172; + case 48: goto tr173; + case 58: goto st103; + case 59: goto tr176; + case 95: goto tr177; + case 102: goto st105; + case 123: goto tr179; + case 125: goto tr180; + } + if ( (*p) < 49 ) { + if ( 41 <= (*p) && (*p) <= 42 ) + goto tr171; + } else if ( (*p) > 57 ) { + if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else if ( (*p) >= 65 ) + goto tr177; + } else + goto st101; + goto tr164; +tr167: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st96; +st96: + if ( ++p == pe ) + goto _test_eof96; +case 96: +#line 2941 "rlscan.cpp" + switch( (*p) ) { + case 9: goto st96; + case 10: goto tr167; + case 32: goto st96; + } + goto tr181; +tr168: +#line 1 "rlscan.rl" + {te = p+1;} + goto st97; +st97: + if ( ++p == pe ) + goto _test_eof97; +case 97: +#line 2956 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr31; + case 34: goto tr32; + case 92: goto st17; + } + goto st16; +tr31: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st16; +st16: + if ( ++p == pe ) + goto _test_eof16; +case 16: +#line 2975 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr31; + case 34: goto tr32; + case 92: goto st17; + } + goto st16; +st17: + if ( ++p == pe ) + goto _test_eof17; +case 17: + if ( (*p) == 10 ) + goto tr31; + goto st16; +tr169: +#line 1 "rlscan.rl" + {te = p+1;} + goto st98; +st98: + if ( ++p == pe ) + goto _test_eof98; +case 98: +#line 2997 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr35; + case 39: goto tr32; + case 92: goto st19; + } + goto st18; +tr35: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st18; +st18: + if ( ++p == pe ) + goto _test_eof18; +case 18: +#line 3016 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr35; + case 39: goto tr32; + case 92: goto st19; + } + goto st18; +st19: + if ( ++p == pe ) + goto _test_eof19; +case 19: + if ( (*p) == 10 ) + goto tr35; + goto st18; +tr172: +#line 1 "rlscan.rl" + {te = p+1;} + goto st99; +st99: + if ( ++p == pe ) + goto _test_eof99; +case 99: +#line 3038 "rlscan.cpp" + switch( (*p) ) { + case 42: goto st20; + case 47: goto st22; + } + goto tr182; +tr38: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st20; +st20: + if ( ++p == pe ) + goto _test_eof20; +case 20: +#line 3056 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr38; + case 42: goto st21; + } + goto st20; +st21: + if ( ++p == pe ) + goto _test_eof21; +case 21: + switch( (*p) ) { + case 10: goto tr38; + case 42: goto st21; + case 47: goto tr40; + } + goto st20; +st22: + if ( ++p == pe ) + goto _test_eof22; +case 22: + if ( (*p) == 10 ) + goto tr42; + goto st22; +tr173: +#line 1 "rlscan.rl" + {te = p+1;} + goto st100; +st100: + if ( ++p == pe ) + goto _test_eof100; +case 100: +#line 3087 "rlscan.cpp" + if ( (*p) == 120 ) + goto st23; + if ( 48 <= (*p) && (*p) <= 57 ) + goto st101; + goto tr183; +st101: + if ( ++p == pe ) + goto _test_eof101; +case 101: + if ( 48 <= (*p) && (*p) <= 57 ) + goto st101; + goto tr183; +st23: + if ( ++p == pe ) + goto _test_eof23; +case 23: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st102; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st102; + } else + goto st102; + goto tr43; +st102: + if ( ++p == pe ) + goto _test_eof102; +case 102: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st102; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st102; + } else + goto st102; + goto tr185; +st103: + if ( ++p == pe ) + goto _test_eof103; +case 103: + if ( (*p) == 58 ) + goto tr186; + goto tr182; +tr177: +#line 1 "rlscan.rl" + {te = p+1;} +#line 802 "rlscan.rl" + {act = 39;} + goto st104; +tr201: +#line 1 "rlscan.rl" + {te = p+1;} +#line 797 "rlscan.rl" + {act = 38;} + goto st104; +tr206: +#line 1 "rlscan.rl" + {te = p+1;} +#line 789 "rlscan.rl" + {act = 36;} + goto st104; +tr208: +#line 1 "rlscan.rl" + {te = p+1;} +#line 768 "rlscan.rl" + {act = 29;} + goto st104; +tr213: +#line 1 "rlscan.rl" + {te = p+1;} +#line 770 "rlscan.rl" + {act = 31;} + goto st104; +tr215: +#line 1 "rlscan.rl" + {te = p+1;} +#line 780 "rlscan.rl" + {act = 33;} + goto st104; +tr218: +#line 1 "rlscan.rl" + {te = p+1;} +#line 781 "rlscan.rl" + {act = 34;} + goto st104; +tr221: +#line 1 "rlscan.rl" + {te = p+1;} +#line 776 "rlscan.rl" + {act = 32;} + goto st104; +tr224: +#line 1 "rlscan.rl" + {te = p+1;} +#line 785 "rlscan.rl" + {act = 35;} + goto st104; +tr225: +#line 1 "rlscan.rl" + {te = p+1;} +#line 766 "rlscan.rl" + {act = 27;} + goto st104; +tr227: +#line 1 "rlscan.rl" + {te = p+1;} +#line 793 "rlscan.rl" + {act = 37;} + goto st104; +tr231: +#line 1 "rlscan.rl" + {te = p+1;} +#line 769 "rlscan.rl" + {act = 30;} + goto st104; +st104: + if ( ++p == pe ) + goto _test_eof104; +case 104: +#line 3209 "rlscan.cpp" + if ( (*p) == 95 ) + goto tr177; + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr187; +st105: + if ( ++p == pe ) + goto _test_eof105; +case 105: + switch( (*p) ) { + case 95: goto tr177; + case 98: goto st106; + case 99: goto st110; + case 101: goto st115; + case 103: goto st121; + case 104: goto st124; + case 110: goto st127; + case 112: goto st130; + case 114: goto st131; + case 116: goto st133; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st106: + if ( ++p == pe ) + goto _test_eof106; +case 106: + switch( (*p) ) { + case 95: goto tr177; + case 114: goto st107; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st107: + if ( ++p == pe ) + goto _test_eof107; +case 107: + switch( (*p) ) { + case 95: goto tr177; + case 101: goto st108; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st108: + if ( ++p == pe ) + goto _test_eof108; +case 108: + switch( (*p) ) { + case 95: goto tr177; + case 97: goto st109; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st109: + if ( ++p == pe ) + goto _test_eof109; +case 109: + switch( (*p) ) { + case 95: goto tr177; + case 107: goto tr201; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st110: + if ( ++p == pe ) + goto _test_eof110; +case 110: + switch( (*p) ) { + case 95: goto tr177; + case 97: goto st111; + case 117: goto st113; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr202; +st111: + if ( ++p == pe ) + goto _test_eof111; +case 111: + switch( (*p) ) { + case 95: goto tr177; + case 108: goto st112; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st112: + if ( ++p == pe ) + goto _test_eof112; +case 112: + switch( (*p) ) { + case 95: goto tr177; + case 108: goto tr206; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st113: + if ( ++p == pe ) + goto _test_eof113; +case 113: + switch( (*p) ) { + case 95: goto tr177; + case 114: goto st114; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st114: + if ( ++p == pe ) + goto _test_eof114; +case 114: + switch( (*p) ) { + case 95: goto tr177; + case 115: goto tr208; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st115: + if ( ++p == pe ) + goto _test_eof115; +case 115: + switch( (*p) ) { + case 95: goto tr177; + case 110: goto st116; + case 120: goto st119; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st116: + if ( ++p == pe ) + goto _test_eof116; +case 116: + switch( (*p) ) { + case 95: goto tr177; + case 116: goto st117; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st117: + if ( ++p == pe ) + goto _test_eof117; +case 117: + switch( (*p) ) { + case 95: goto tr177; + case 114: goto st118; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st118: + if ( ++p == pe ) + goto _test_eof118; +case 118: + switch( (*p) ) { + case 95: goto tr177; + case 121: goto tr213; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st119: + if ( ++p == pe ) + goto _test_eof119; +case 119: + switch( (*p) ) { + case 95: goto tr177; + case 101: goto st120; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st120: + if ( ++p == pe ) + goto _test_eof120; +case 120: + switch( (*p) ) { + case 95: goto tr177; + case 99: goto tr215; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st121: + if ( ++p == pe ) + goto _test_eof121; +case 121: + switch( (*p) ) { + case 95: goto tr177; + case 111: goto st122; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st122: + if ( ++p == pe ) + goto _test_eof122; +case 122: + switch( (*p) ) { + case 95: goto tr177; + case 116: goto st123; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st123: + if ( ++p == pe ) + goto _test_eof123; +case 123: + switch( (*p) ) { + case 95: goto tr177; + case 111: goto tr218; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st124: + if ( ++p == pe ) + goto _test_eof124; +case 124: + switch( (*p) ) { + case 95: goto tr177; + case 111: goto st125; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st125: + if ( ++p == pe ) + goto _test_eof125; +case 125: + switch( (*p) ) { + case 95: goto tr177; + case 108: goto st126; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st126: + if ( ++p == pe ) + goto _test_eof126; +case 126: + switch( (*p) ) { + case 95: goto tr177; + case 100: goto tr221; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st127: + if ( ++p == pe ) + goto _test_eof127; +case 127: + switch( (*p) ) { + case 95: goto tr177; + case 101: goto st128; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st128: + if ( ++p == pe ) + goto _test_eof128; +case 128: + switch( (*p) ) { + case 95: goto tr177; + case 120: goto st129; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st129: + if ( ++p == pe ) + goto _test_eof129; +case 129: + switch( (*p) ) { + case 95: goto tr177; + case 116: goto tr224; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st130: + if ( ++p == pe ) + goto _test_eof130; +case 130: + switch( (*p) ) { + case 95: goto tr177; + case 99: goto tr225; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st131: + if ( ++p == pe ) + goto _test_eof131; +case 131: + switch( (*p) ) { + case 95: goto tr177; + case 101: goto st132; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st132: + if ( ++p == pe ) + goto _test_eof132; +case 132: + switch( (*p) ) { + case 95: goto tr177; + case 116: goto tr227; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st133: + if ( ++p == pe ) + goto _test_eof133; +case 133: + switch( (*p) ) { + case 95: goto tr177; + case 97: goto st134; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st134: + if ( ++p == pe ) + goto _test_eof134; +case 134: + switch( (*p) ) { + case 95: goto tr177; + case 114: goto st135; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st135: + if ( ++p == pe ) + goto _test_eof135; +case 135: + switch( (*p) ) { + case 95: goto tr177; + case 103: goto st136; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st136: + if ( ++p == pe ) + goto _test_eof136; +case 136: + switch( (*p) ) { + case 95: goto tr177; + case 115: goto tr231; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +tr232: +#line 889 "rlscan.rl" + {te = p+1;{ token( RE_Char, ts, te ); }} + goto st137; +tr233: +#line 884 "rlscan.rl" + {te = p+1;{ + scan_error() << "unterminated OR literal" << endl; + }} + goto st137; +tr234: +#line 879 "rlscan.rl" + {te = p+1;{ token( RE_Dash, 0, 0 ); }} + goto st137; +tr236: +#line 882 "rlscan.rl" + {te = p+1;{ token( RE_SqClose ); {cs = stack[--top];goto _again;} }} + goto st137; +tr237: +#line 889 "rlscan.rl" + {te = p;p--;{ token( RE_Char, ts, te ); }} + goto st137; +tr238: +#line 876 "rlscan.rl" + {te = p+1;{ token( RE_Char, ts+1, te ); }} + goto st137; +tr239: +#line 875 "rlscan.rl" + {te = p+1;{ updateCol(); }} + goto st137; +tr240: +#line 867 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\0' ); }} + goto st137; +tr241: +#line 868 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\a' ); }} + goto st137; +tr242: +#line 869 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\b' ); }} + goto st137; +tr243: +#line 873 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\f' ); }} + goto st137; +tr244: +#line 871 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\n' ); }} + goto st137; +tr245: +#line 874 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\r' ); }} + goto st137; +tr246: +#line 870 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\t' ); }} + goto st137; +tr247: +#line 872 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\v' ); }} + goto st137; +st137: +#line 1 "rlscan.rl" + {ts = 0;} + if ( ++p == pe ) + goto _test_eof137; +case 137: +#line 1 "rlscan.rl" + {ts = p;} +#line 3845 "rlscan.cpp" + switch( (*p) ) { + case 0: goto tr233; + case 45: goto tr234; + case 92: goto st138; + case 93: goto tr236; + } + goto tr232; +st138: + if ( ++p == pe ) + goto _test_eof138; +case 138: + switch( (*p) ) { + case 10: goto tr239; + case 48: goto tr240; + case 97: goto tr241; + case 98: goto tr242; + case 102: goto tr243; + case 110: goto tr244; + case 114: goto tr245; + case 116: goto tr246; + case 118: goto tr247; + } + goto tr238; +tr248: +#line 924 "rlscan.rl" + {te = p+1;{ token( RE_Char, ts, te ); }} + goto st139; +tr249: +#line 919 "rlscan.rl" + {te = p+1;{ + scan_error() << "unterminated regular expression" << endl; + }} + goto st139; +tr250: +#line 914 "rlscan.rl" + {te = p+1;{ token( RE_Star ); }} + goto st139; +tr251: +#line 913 "rlscan.rl" + {te = p+1;{ token( RE_Dot ); }} + goto st139; +tr255: +#line 907 "rlscan.rl" + {te = p;p--;{ + token( RE_Slash, ts, te ); + {goto st146;} + }} + goto st139; +tr256: +#line 907 "rlscan.rl" + {te = p+1;{ + token( RE_Slash, ts, te ); + {goto st146;} + }} + goto st139; +tr257: +#line 916 "rlscan.rl" + {te = p;p--;{ token( RE_SqOpen ); {stack[top++] = 139; goto st137;} }} + goto st139; +tr258: +#line 917 "rlscan.rl" + {te = p+1;{ token( RE_SqOpenNeg ); {stack[top++] = 139; goto st137;} }} + goto st139; +tr259: +#line 924 "rlscan.rl" + {te = p;p--;{ token( RE_Char, ts, te ); }} + goto st139; +tr260: +#line 904 "rlscan.rl" + {te = p+1;{ token( RE_Char, ts+1, te ); }} + goto st139; +tr261: +#line 903 "rlscan.rl" + {te = p+1;{ updateCol(); }} + goto st139; +tr262: +#line 895 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\0' ); }} + goto st139; +tr263: +#line 896 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\a' ); }} + goto st139; +tr264: +#line 897 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\b' ); }} + goto st139; +tr265: +#line 901 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\f' ); }} + goto st139; +tr266: +#line 899 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\n' ); }} + goto st139; +tr267: +#line 902 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\r' ); }} + goto st139; +tr268: +#line 898 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\t' ); }} + goto st139; +tr269: +#line 900 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\v' ); }} + goto st139; +st139: +#line 1 "rlscan.rl" + {ts = 0;} + if ( ++p == pe ) + goto _test_eof139; +case 139: +#line 1 "rlscan.rl" + {ts = p;} +#line 3961 "rlscan.cpp" + switch( (*p) ) { + case 0: goto tr249; + case 42: goto tr250; + case 46: goto tr251; + case 47: goto st140; + case 91: goto st141; + case 92: goto st142; + } + goto tr248; +st140: + if ( ++p == pe ) + goto _test_eof140; +case 140: + if ( (*p) == 105 ) + goto tr256; + goto tr255; +st141: + if ( ++p == pe ) + goto _test_eof141; +case 141: + if ( (*p) == 94 ) + goto tr258; + goto tr257; +st142: + if ( ++p == pe ) + goto _test_eof142; +case 142: + switch( (*p) ) { + case 10: goto tr261; + case 48: goto tr262; + case 97: goto tr263; + case 98: goto tr264; + case 102: goto tr265; + case 110: goto tr266; + case 114: goto tr267; + case 116: goto tr268; + case 118: goto tr269; + } + goto tr260; +tr270: +#line 933 "rlscan.rl" + {te = p+1;{ + scan_error() << "unterminated write statement" << endl; + }} + goto st143; +tr273: +#line 931 "rlscan.rl" + {te = p+1;{ token( ';' ); {goto st146;} }} + goto st143; +tr275: +#line 930 "rlscan.rl" + {te = p;p--;{ updateCol(); }} + goto st143; +tr276: +#line 929 "rlscan.rl" + {te = p;p--;{ token( TK_Word, ts, te ); }} + goto st143; +st143: +#line 1 "rlscan.rl" + {ts = 0;} + if ( ++p == pe ) + goto _test_eof143; +case 143: +#line 1 "rlscan.rl" + {ts = p;} +#line 4027 "rlscan.cpp" + switch( (*p) ) { + case 0: goto tr270; + case 32: goto st144; + case 59: goto tr273; + case 95: goto st145; + } + if ( (*p) < 65 ) { + if ( 9 <= (*p) && (*p) <= 10 ) + goto st144; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto st145; + } else + goto st145; + goto st0; +st0: +cs = 0; + goto _out; +st144: + if ( ++p == pe ) + goto _test_eof144; +case 144: + if ( (*p) == 32 ) + goto st144; + if ( 9 <= (*p) && (*p) <= 10 ) + goto st144; + goto tr275; +st145: + if ( ++p == pe ) + goto _test_eof145; +case 145: + if ( (*p) == 95 ) + goto st145; + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st145; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto st145; + } else + goto st145; + goto tr276; +tr45: +#line 1110 "rlscan.rl" + {{p = ((te))-1;}{ token( *ts ); }} + goto st146; +tr51: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } +#line 1007 "rlscan.rl" + {te = p+1;{ updateCol(); }} + goto st146; +tr55: +#line 994 "rlscan.rl" + {{p = ((te))-1;}{ token( TK_UInt, ts, te ); }} + goto st146; +tr57: +#line 1075 "rlscan.rl" + {te = p+1;{ + updateCol(); + endSection(); + {cs = stack[--top];goto _again;} + }} + goto st146; +tr277: +#line 1110 "rlscan.rl" + {te = p+1;{ token( *ts ); }} + goto st146; +tr278: +#line 1106 "rlscan.rl" + {te = p+1;{ + scan_error() << "unterminated ragel section" << endl; + }} + goto st146; +tr280: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } +#line 1084 "rlscan.rl" + {te = p+1;{ + updateCol(); + if ( singleLineSpec ) { + endSection(); + {cs = stack[--top];goto _again;} + } + }} + goto st146; +tr289: +#line 1004 "rlscan.rl" + {te = p+1;{ token( RE_Slash ); {goto st139;} }} + goto st146; +tr311: +#line 1092 "rlscan.rl" + {te = p+1;{ + if ( lastToken == KW_Export || lastToken == KW_Entry ) + token( '{' ); + else { + token( '{' ); + curly_count = 1; + inlineBlockType = CurlyDelimited; + if ( hostLang->lang == HostLang::Ruby ) + {stack[top++] = 146; goto st52;} + else + {stack[top++] = 146; goto st95;} + } + }} + goto st146; +tr314: +#line 1081 "rlscan.rl" + {te = p;p--;{ updateCol(); }} + goto st146; +tr315: +#line 1110 "rlscan.rl" + {te = p;p--;{ token( *ts ); }} + goto st146; +tr316: +#line 999 "rlscan.rl" + {te = p;p--;{ token( TK_Literal, ts, te ); }} + goto st146; +tr317: +#line 999 "rlscan.rl" + {te = p+1;{ token( TK_Literal, ts, te ); }} + goto st146; +tr318: +#line 1037 "rlscan.rl" + {te = p+1;{ token( TK_AllGblError ); }} + goto st146; +tr319: +#line 1021 "rlscan.rl" + {te = p+1;{ token( TK_AllFromState ); }} + goto st146; +tr320: +#line 1029 "rlscan.rl" + {te = p+1;{ token( TK_AllEOF ); }} + goto st146; +tr321: +#line 1056 "rlscan.rl" + {te = p+1;{ token( TK_AllCond ); }} + goto st146; +tr322: +#line 1045 "rlscan.rl" + {te = p+1;{ token( TK_AllLocalError ); }} + goto st146; +tr323: +#line 1013 "rlscan.rl" + {te = p+1;{ token( TK_AllToState ); }} + goto st146; +tr324: +#line 1038 "rlscan.rl" + {te = p+1;{ token( TK_FinalGblError ); }} + goto st146; +tr325: +#line 1022 "rlscan.rl" + {te = p+1;{ token( TK_FinalFromState ); }} + goto st146; +tr326: +#line 1030 "rlscan.rl" + {te = p+1;{ token( TK_FinalEOF ); }} + goto st146; +tr327: +#line 1057 "rlscan.rl" + {te = p+1;{ token( TK_LeavingCond ); }} + goto st146; +tr328: +#line 1046 "rlscan.rl" + {te = p+1;{ token( TK_FinalLocalError ); }} + goto st146; +tr329: +#line 1014 "rlscan.rl" + {te = p+1;{ token( TK_FinalToState ); }} + goto st146; +tr330: +#line 1060 "rlscan.rl" + {te = p+1;{ token( TK_StarStar ); }} + goto st146; +tr331: +#line 1061 "rlscan.rl" + {te = p+1;{ token( TK_DashDash ); }} + goto st146; +tr332: +#line 1062 "rlscan.rl" + {te = p+1;{ token( TK_Arrow ); }} + goto st146; +tr333: +#line 1059 "rlscan.rl" + {te = p+1;{ token( TK_DotDot ); }} + goto st146; +tr334: +#line 994 "rlscan.rl" + {te = p;p--;{ token( TK_UInt, ts, te ); }} + goto st146; +tr336: +#line 995 "rlscan.rl" + {te = p;p--;{ token( TK_Hex, ts, te ); }} + goto st146; +tr337: +#line 1073 "rlscan.rl" + {te = p+1;{ token( TK_NameSep, ts, te ); }} + goto st146; +tr338: +#line 1009 "rlscan.rl" + {te = p+1;{ token( TK_ColonEquals ); }} + goto st146; +tr340: +#line 1065 "rlscan.rl" + {te = p;p--;{ token( TK_ColonGt ); }} + goto st146; +tr341: +#line 1066 "rlscan.rl" + {te = p+1;{ token( TK_ColonGtGt ); }} + goto st146; +tr342: +#line 1039 "rlscan.rl" + {te = p+1;{ token( TK_NotStartGblError ); }} + goto st146; +tr343: +#line 1023 "rlscan.rl" + {te = p+1;{ token( TK_NotStartFromState ); }} + goto st146; +tr344: +#line 1031 "rlscan.rl" + {te = p+1;{ token( TK_NotStartEOF ); }} + goto st146; +tr345: +#line 1067 "rlscan.rl" + {te = p+1;{ token( TK_LtColon ); }} + goto st146; +tr347: +#line 1047 "rlscan.rl" + {te = p+1;{ token( TK_NotStartLocalError ); }} + goto st146; +tr348: +#line 1015 "rlscan.rl" + {te = p+1;{ token( TK_NotStartToState ); }} + goto st146; +tr349: +#line 1052 "rlscan.rl" + {te = p;p--;{ token( TK_Middle ); }} + goto st146; +tr350: +#line 1041 "rlscan.rl" + {te = p+1;{ token( TK_MiddleGblError ); }} + goto st146; +tr351: +#line 1025 "rlscan.rl" + {te = p+1;{ token( TK_MiddleFromState ); }} + goto st146; +tr352: +#line 1033 "rlscan.rl" + {te = p+1;{ token( TK_MiddleEOF ); }} + goto st146; +tr353: +#line 1049 "rlscan.rl" + {te = p+1;{ token( TK_MiddleLocalError ); }} + goto st146; +tr354: +#line 1017 "rlscan.rl" + {te = p+1;{ token( TK_MiddleToState ); }} + goto st146; +tr355: +#line 1063 "rlscan.rl" + {te = p+1;{ token( TK_DoubleArrow ); }} + goto st146; +tr356: +#line 1036 "rlscan.rl" + {te = p+1;{ token( TK_StartGblError ); }} + goto st146; +tr357: +#line 1020 "rlscan.rl" + {te = p+1;{ token( TK_StartFromState ); }} + goto st146; +tr358: +#line 1028 "rlscan.rl" + {te = p+1;{ token( TK_StartEOF ); }} + goto st146; +tr359: +#line 1055 "rlscan.rl" + {te = p+1;{ token( TK_StartCond ); }} + goto st146; +tr360: +#line 1044 "rlscan.rl" + {te = p+1;{ token( TK_StartLocalError ); }} + goto st146; +tr361: +#line 1012 "rlscan.rl" + {te = p+1;{ token( TK_StartToState ); }} + goto st146; +tr362: +#line 1040 "rlscan.rl" + {te = p+1;{ token( TK_NotFinalGblError ); }} + goto st146; +tr363: +#line 1024 "rlscan.rl" + {te = p+1;{ token( TK_NotFinalFromState ); }} + goto st146; +tr364: +#line 1032 "rlscan.rl" + {te = p+1;{ token( TK_NotFinalEOF ); }} + goto st146; +tr365: +#line 1048 "rlscan.rl" + {te = p+1;{ token( TK_NotFinalLocalError ); }} + goto st146; +tr366: +#line 1016 "rlscan.rl" + {te = p+1;{ token( TK_NotFinalToState ); }} + goto st146; +tr367: +#line 1 "rlscan.rl" + { switch( act ) { + case 88: + {{p = ((te))-1;} token( KW_Machine ); } + break; + case 89: + {{p = ((te))-1;} token( KW_Include ); } + break; + case 90: + {{p = ((te))-1;} token( KW_Import ); } + break; + case 91: + {{p = ((te))-1;} + token( KW_Write ); + {goto st143;} + } + break; + case 92: + {{p = ((te))-1;} token( KW_Action ); } + break; + case 93: + {{p = ((te))-1;} token( KW_AlphType ); } + break; + case 94: + {{p = ((te))-1;} token( KW_PrePush ); } + break; + case 95: + {{p = ((te))-1;} token( KW_PostPop ); } + break; + case 96: + {{p = ((te))-1;} + token( KW_GetKey ); + inlineBlockType = SemiTerminated; + if ( hostLang->lang == HostLang::Ruby ) + {stack[top++] = 146; goto st52;} + else + {stack[top++] = 146; goto st95;} + } + break; + case 97: + {{p = ((te))-1;} + token( KW_Access ); + inlineBlockType = SemiTerminated; + if ( hostLang->lang == HostLang::Ruby ) + {stack[top++] = 146; goto st52;} + else + {stack[top++] = 146; goto st95;} + } + break; + case 98: + {{p = ((te))-1;} + token( KW_Variable ); + inlineBlockType = SemiTerminated; + if ( hostLang->lang == HostLang::Ruby ) + {stack[top++] = 146; goto st52;} + else + {stack[top++] = 146; goto st95;} + } + break; + case 99: + {{p = ((te))-1;} token( KW_When ); } + break; + case 100: + {{p = ((te))-1;} token( KW_InWhen ); } + break; + case 101: + {{p = ((te))-1;} token( KW_OutWhen ); } + break; + case 102: + {{p = ((te))-1;} token( KW_Eof ); } + break; + case 103: + {{p = ((te))-1;} token( KW_Err ); } + break; + case 104: + {{p = ((te))-1;} token( KW_Lerr ); } + break; + case 105: + {{p = ((te))-1;} token( KW_To ); } + break; + case 106: + {{p = ((te))-1;} token( KW_From ); } + break; + case 107: + {{p = ((te))-1;} token( KW_Export ); } + break; + case 108: + {{p = ((te))-1;} token( TK_Word, ts, te ); } + break; + } + } + goto st146; +tr368: +#line 1001 "rlscan.rl" + {te = p;p--;{ token( RE_SqOpen ); {stack[top++] = 146; goto st137;} }} + goto st146; +tr369: +#line 1002 "rlscan.rl" + {te = p+1;{ token( RE_SqOpenNeg ); {stack[top++] = 146; goto st137;} }} + goto st146; +tr370: +#line 991 "rlscan.rl" + {te = p;p--;{ token( TK_Word, ts, te ); }} + goto st146; +tr461: +#line 1070 "rlscan.rl" + {te = p+1;{ token( TK_BarStar ); }} + goto st146; +st146: +#line 1 "rlscan.rl" + {ts = 0;} + if ( ++p == pe ) + goto _test_eof146; +case 146: +#line 1 "rlscan.rl" + {ts = p;} +#line 4459 "rlscan.cpp" + switch( (*p) ) { + case 0: goto tr278; + case 9: goto st147; + case 10: goto tr280; + case 13: goto st147; + case 32: goto st147; + case 34: goto tr281; + case 35: goto tr282; + case 36: goto st151; + case 37: goto st152; + case 39: goto tr285; + case 42: goto st154; + case 45: goto st155; + case 46: goto st156; + case 47: goto tr289; + case 48: goto tr290; + case 58: goto st160; + case 60: goto st162; + case 61: goto st164; + case 62: goto st165; + case 64: goto st166; + case 91: goto st168; + case 95: goto tr297; + case 97: goto st169; + case 101: goto st183; + case 102: goto st190; + case 103: goto st193; + case 105: goto st198; + case 108: goto st211; + case 109: goto st214; + case 111: goto st220; + case 112: goto st226; + case 116: goto st237; + case 118: goto st238; + case 119: goto st245; + case 123: goto tr311; + case 124: goto st251; + case 125: goto tr313; + } + if ( (*p) < 65 ) { + if ( 49 <= (*p) && (*p) <= 57 ) + goto st158; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr277; +st147: + if ( ++p == pe ) + goto _test_eof147; +case 147: + switch( (*p) ) { + case 9: goto st147; + case 13: goto st147; + case 32: goto st147; + } + goto tr314; +tr281: +#line 1 "rlscan.rl" + {te = p+1;} + goto st148; +st148: + if ( ++p == pe ) + goto _test_eof148; +case 148: +#line 4526 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr47; + case 34: goto st149; + case 92: goto st25; + } + goto st24; +tr47: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st24; +st24: + if ( ++p == pe ) + goto _test_eof24; +case 24: +#line 4545 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr47; + case 34: goto st149; + case 92: goto st25; + } + goto st24; +st149: + if ( ++p == pe ) + goto _test_eof149; +case 149: + if ( (*p) == 105 ) + goto tr317; + goto tr316; +st25: + if ( ++p == pe ) + goto _test_eof25; +case 25: + if ( (*p) == 10 ) + goto tr47; + goto st24; +tr282: +#line 1 "rlscan.rl" + {te = p+1;} + goto st150; +st150: + if ( ++p == pe ) + goto _test_eof150; +case 150: +#line 4574 "rlscan.cpp" + if ( (*p) == 10 ) + goto tr51; + goto st26; +st26: + if ( ++p == pe ) + goto _test_eof26; +case 26: + if ( (*p) == 10 ) + goto tr51; + goto st26; +st151: + if ( ++p == pe ) + goto _test_eof151; +case 151: + switch( (*p) ) { + case 33: goto tr318; + case 42: goto tr319; + case 47: goto tr320; + case 63: goto tr321; + case 94: goto tr322; + case 126: goto tr323; + } + goto tr315; +st152: + if ( ++p == pe ) + goto _test_eof152; +case 152: + switch( (*p) ) { + case 33: goto tr324; + case 42: goto tr325; + case 47: goto tr326; + case 63: goto tr327; + case 94: goto tr328; + case 126: goto tr329; + } + goto tr315; +tr285: +#line 1 "rlscan.rl" + {te = p+1;} + goto st153; +st153: + if ( ++p == pe ) + goto _test_eof153; +case 153: +#line 4619 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr53; + case 39: goto st149; + case 92: goto st28; + } + goto st27; +tr53: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st27; +st27: + if ( ++p == pe ) + goto _test_eof27; +case 27: +#line 4638 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr53; + case 39: goto st149; + case 92: goto st28; + } + goto st27; +st28: + if ( ++p == pe ) + goto _test_eof28; +case 28: + if ( (*p) == 10 ) + goto tr53; + goto st27; +st154: + if ( ++p == pe ) + goto _test_eof154; +case 154: + if ( (*p) == 42 ) + goto tr330; + goto tr315; +st155: + if ( ++p == pe ) + goto _test_eof155; +case 155: + switch( (*p) ) { + case 45: goto tr331; + case 62: goto tr332; + } + goto tr315; +st156: + if ( ++p == pe ) + goto _test_eof156; +case 156: + if ( (*p) == 46 ) + goto tr333; + goto tr315; +tr290: +#line 1 "rlscan.rl" + {te = p+1;} + goto st157; +st157: + if ( ++p == pe ) + goto _test_eof157; +case 157: +#line 4683 "rlscan.cpp" + if ( (*p) == 120 ) + goto st29; + if ( 48 <= (*p) && (*p) <= 57 ) + goto st158; + goto tr334; +st158: + if ( ++p == pe ) + goto _test_eof158; +case 158: + if ( 48 <= (*p) && (*p) <= 57 ) + goto st158; + goto tr334; +st29: + if ( ++p == pe ) + goto _test_eof29; +case 29: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st159; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st159; + } else + goto st159; + goto tr55; +st159: + if ( ++p == pe ) + goto _test_eof159; +case 159: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st159; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st159; + } else + goto st159; + goto tr336; +st160: + if ( ++p == pe ) + goto _test_eof160; +case 160: + switch( (*p) ) { + case 58: goto tr337; + case 61: goto tr338; + case 62: goto st161; + } + goto tr315; +st161: + if ( ++p == pe ) + goto _test_eof161; +case 161: + if ( (*p) == 62 ) + goto tr341; + goto tr340; +st162: + if ( ++p == pe ) + goto _test_eof162; +case 162: + switch( (*p) ) { + case 33: goto tr342; + case 42: goto tr343; + case 47: goto tr344; + case 58: goto tr345; + case 62: goto st163; + case 94: goto tr347; + case 126: goto tr348; + } + goto tr315; +st163: + if ( ++p == pe ) + goto _test_eof163; +case 163: + switch( (*p) ) { + case 33: goto tr350; + case 42: goto tr351; + case 47: goto tr352; + case 94: goto tr353; + case 126: goto tr354; + } + goto tr349; +st164: + if ( ++p == pe ) + goto _test_eof164; +case 164: + if ( (*p) == 62 ) + goto tr355; + goto tr315; +st165: + if ( ++p == pe ) + goto _test_eof165; +case 165: + switch( (*p) ) { + case 33: goto tr356; + case 42: goto tr357; + case 47: goto tr358; + case 63: goto tr359; + case 94: goto tr360; + case 126: goto tr361; + } + goto tr315; +st166: + if ( ++p == pe ) + goto _test_eof166; +case 166: + switch( (*p) ) { + case 33: goto tr362; + case 42: goto tr363; + case 47: goto tr364; + case 94: goto tr365; + case 126: goto tr366; + } + goto tr315; +tr297: +#line 1 "rlscan.rl" + {te = p+1;} +#line 991 "rlscan.rl" + {act = 108;} + goto st167; +tr377: +#line 1 "rlscan.rl" + {te = p+1;} +#line 964 "rlscan.rl" + {act = 97;} + goto st167; +tr380: +#line 1 "rlscan.rl" + {te = p+1;} +#line 948 "rlscan.rl" + {act = 92;} + goto st167; +tr386: +#line 1 "rlscan.rl" + {te = p+1;} +#line 949 "rlscan.rl" + {act = 93;} + goto st167; +tr390: +#line 1 "rlscan.rl" + {te = p+1;} +#line 983 "rlscan.rl" + {act = 102;} + goto st167; +tr391: +#line 1 "rlscan.rl" + {te = p+1;} +#line 984 "rlscan.rl" + {act = 103;} + goto st167; +tr395: +#line 1 "rlscan.rl" + {te = p+1;} +#line 988 "rlscan.rl" + {act = 107;} + goto st167; +tr398: +#line 1 "rlscan.rl" + {te = p+1;} +#line 987 "rlscan.rl" + {act = 106;} + goto st167; +tr403: +#line 1 "rlscan.rl" + {te = p+1;} +#line 956 "rlscan.rl" + {act = 96;} + goto st167; +tr409: +#line 1 "rlscan.rl" + {te = p+1;} +#line 943 "rlscan.rl" + {act = 90;} + goto st167; +tr415: +#line 1 "rlscan.rl" + {te = p+1;} +#line 942 "rlscan.rl" + {act = 89;} + goto st167; +tr418: +#line 1 "rlscan.rl" + {te = p+1;} +#line 981 "rlscan.rl" + {act = 100;} + goto st167; +tr421: +#line 1 "rlscan.rl" + {te = p+1;} +#line 985 "rlscan.rl" + {act = 104;} + goto st167; +tr427: +#line 1 "rlscan.rl" + {te = p+1;} +#line 941 "rlscan.rl" + {act = 88;} + goto st167; +tr433: +#line 1 "rlscan.rl" + {te = p+1;} +#line 982 "rlscan.rl" + {act = 101;} + goto st167; +tr440: +#line 1 "rlscan.rl" + {te = p+1;} +#line 951 "rlscan.rl" + {act = 95;} + goto st167; +tr445: +#line 1 "rlscan.rl" + {te = p+1;} +#line 950 "rlscan.rl" + {act = 94;} + goto st167; +tr446: +#line 1 "rlscan.rl" + {te = p+1;} +#line 986 "rlscan.rl" + {act = 105;} + goto st167; +tr453: +#line 1 "rlscan.rl" + {te = p+1;} +#line 972 "rlscan.rl" + {act = 98;} + goto st167; +tr457: +#line 1 "rlscan.rl" + {te = p+1;} +#line 980 "rlscan.rl" + {act = 99;} + goto st167; +tr460: +#line 1 "rlscan.rl" + {te = p+1;} +#line 944 "rlscan.rl" + {act = 91;} + goto st167; +st167: + if ( ++p == pe ) + goto _test_eof167; +case 167: +#line 4927 "rlscan.cpp" + if ( (*p) == 95 ) + goto tr297; + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr367; +st168: + if ( ++p == pe ) + goto _test_eof168; +case 168: + if ( (*p) == 94 ) + goto tr369; + goto tr368; +st169: + if ( ++p == pe ) + goto _test_eof169; +case 169: + switch( (*p) ) { + case 95: goto tr297; + case 99: goto st170; + case 108: goto st177; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st170: + if ( ++p == pe ) + goto _test_eof170; +case 170: + switch( (*p) ) { + case 95: goto tr297; + case 99: goto st171; + case 116: goto st174; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st171: + if ( ++p == pe ) + goto _test_eof171; +case 171: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto st172; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st172: + if ( ++p == pe ) + goto _test_eof172; +case 172: + switch( (*p) ) { + case 95: goto tr297; + case 115: goto st173; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st173: + if ( ++p == pe ) + goto _test_eof173; +case 173: + switch( (*p) ) { + case 95: goto tr297; + case 115: goto tr377; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st174: + if ( ++p == pe ) + goto _test_eof174; +case 174: + switch( (*p) ) { + case 95: goto tr297; + case 105: goto st175; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st175: + if ( ++p == pe ) + goto _test_eof175; +case 175: + switch( (*p) ) { + case 95: goto tr297; + case 111: goto st176; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st176: + if ( ++p == pe ) + goto _test_eof176; +case 176: + switch( (*p) ) { + case 95: goto tr297; + case 110: goto tr380; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st177: + if ( ++p == pe ) + goto _test_eof177; +case 177: + switch( (*p) ) { + case 95: goto tr297; + case 112: goto st178; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st178: + if ( ++p == pe ) + goto _test_eof178; +case 178: + switch( (*p) ) { + case 95: goto tr297; + case 104: goto st179; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st179: + if ( ++p == pe ) + goto _test_eof179; +case 179: + switch( (*p) ) { + case 95: goto tr297; + case 116: goto st180; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st180: + if ( ++p == pe ) + goto _test_eof180; +case 180: + switch( (*p) ) { + case 95: goto tr297; + case 121: goto st181; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st181: + if ( ++p == pe ) + goto _test_eof181; +case 181: + switch( (*p) ) { + case 95: goto tr297; + case 112: goto st182; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st182: + if ( ++p == pe ) + goto _test_eof182; +case 182: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto tr386; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st183: + if ( ++p == pe ) + goto _test_eof183; +case 183: + switch( (*p) ) { + case 95: goto tr297; + case 111: goto st184; + case 114: goto st185; + case 120: goto st186; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st184: + if ( ++p == pe ) + goto _test_eof184; +case 184: + switch( (*p) ) { + case 95: goto tr297; + case 102: goto tr390; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st185: + if ( ++p == pe ) + goto _test_eof185; +case 185: + switch( (*p) ) { + case 95: goto tr297; + case 114: goto tr391; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st186: + if ( ++p == pe ) + goto _test_eof186; +case 186: + switch( (*p) ) { + case 95: goto tr297; + case 112: goto st187; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st187: + if ( ++p == pe ) + goto _test_eof187; +case 187: + switch( (*p) ) { + case 95: goto tr297; + case 111: goto st188; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st188: + if ( ++p == pe ) + goto _test_eof188; +case 188: + switch( (*p) ) { + case 95: goto tr297; + case 114: goto st189; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st189: + if ( ++p == pe ) + goto _test_eof189; +case 189: + switch( (*p) ) { + case 95: goto tr297; + case 116: goto tr395; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st190: + if ( ++p == pe ) + goto _test_eof190; +case 190: + switch( (*p) ) { + case 95: goto tr297; + case 114: goto st191; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st191: + if ( ++p == pe ) + goto _test_eof191; +case 191: + switch( (*p) ) { + case 95: goto tr297; + case 111: goto st192; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st192: + if ( ++p == pe ) + goto _test_eof192; +case 192: + switch( (*p) ) { + case 95: goto tr297; + case 109: goto tr398; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st193: + if ( ++p == pe ) + goto _test_eof193; +case 193: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto st194; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st194: + if ( ++p == pe ) + goto _test_eof194; +case 194: + switch( (*p) ) { + case 95: goto tr297; + case 116: goto st195; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st195: + if ( ++p == pe ) + goto _test_eof195; +case 195: + switch( (*p) ) { + case 95: goto tr297; + case 107: goto st196; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st196: + if ( ++p == pe ) + goto _test_eof196; +case 196: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto st197; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st197: + if ( ++p == pe ) + goto _test_eof197; +case 197: + switch( (*p) ) { + case 95: goto tr297; + case 121: goto tr403; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st198: + if ( ++p == pe ) + goto _test_eof198; +case 198: + switch( (*p) ) { + case 95: goto tr297; + case 109: goto st199; + case 110: goto st203; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st199: + if ( ++p == pe ) + goto _test_eof199; +case 199: + switch( (*p) ) { + case 95: goto tr297; + case 112: goto st200; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st200: + if ( ++p == pe ) + goto _test_eof200; +case 200: + switch( (*p) ) { + case 95: goto tr297; + case 111: goto st201; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st201: + if ( ++p == pe ) + goto _test_eof201; +case 201: + switch( (*p) ) { + case 95: goto tr297; + case 114: goto st202; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st202: + if ( ++p == pe ) + goto _test_eof202; +case 202: + switch( (*p) ) { + case 95: goto tr297; + case 116: goto tr409; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st203: + if ( ++p == pe ) + goto _test_eof203; +case 203: + switch( (*p) ) { + case 95: goto tr297; + case 99: goto st204; + case 119: goto st208; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st204: + if ( ++p == pe ) + goto _test_eof204; +case 204: + switch( (*p) ) { + case 95: goto tr297; + case 108: goto st205; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st205: + if ( ++p == pe ) + goto _test_eof205; +case 205: + switch( (*p) ) { + case 95: goto tr297; + case 117: goto st206; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st206: + if ( ++p == pe ) + goto _test_eof206; +case 206: + switch( (*p) ) { + case 95: goto tr297; + case 100: goto st207; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st207: + if ( ++p == pe ) + goto _test_eof207; +case 207: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto tr415; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st208: + if ( ++p == pe ) + goto _test_eof208; +case 208: + switch( (*p) ) { + case 95: goto tr297; + case 104: goto st209; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st209: + if ( ++p == pe ) + goto _test_eof209; +case 209: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto st210; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st210: + if ( ++p == pe ) + goto _test_eof210; +case 210: + switch( (*p) ) { + case 95: goto tr297; + case 110: goto tr418; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st211: + if ( ++p == pe ) + goto _test_eof211; +case 211: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto st212; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st212: + if ( ++p == pe ) + goto _test_eof212; +case 212: + switch( (*p) ) { + case 95: goto tr297; + case 114: goto st213; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st213: + if ( ++p == pe ) + goto _test_eof213; +case 213: + switch( (*p) ) { + case 95: goto tr297; + case 114: goto tr421; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st214: + if ( ++p == pe ) + goto _test_eof214; +case 214: + switch( (*p) ) { + case 95: goto tr297; + case 97: goto st215; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st215: + if ( ++p == pe ) + goto _test_eof215; +case 215: + switch( (*p) ) { + case 95: goto tr297; + case 99: goto st216; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st216: + if ( ++p == pe ) + goto _test_eof216; +case 216: + switch( (*p) ) { + case 95: goto tr297; + case 104: goto st217; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st217: + if ( ++p == pe ) + goto _test_eof217; +case 217: + switch( (*p) ) { + case 95: goto tr297; + case 105: goto st218; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st218: + if ( ++p == pe ) + goto _test_eof218; +case 218: + switch( (*p) ) { + case 95: goto tr297; + case 110: goto st219; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st219: + if ( ++p == pe ) + goto _test_eof219; +case 219: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto tr427; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st220: + if ( ++p == pe ) + goto _test_eof220; +case 220: + switch( (*p) ) { + case 95: goto tr297; + case 117: goto st221; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st221: + if ( ++p == pe ) + goto _test_eof221; +case 221: + switch( (*p) ) { + case 95: goto tr297; + case 116: goto st222; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st222: + if ( ++p == pe ) + goto _test_eof222; +case 222: + switch( (*p) ) { + case 95: goto tr297; + case 119: goto st223; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st223: + if ( ++p == pe ) + goto _test_eof223; +case 223: + switch( (*p) ) { + case 95: goto tr297; + case 104: goto st224; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st224: + if ( ++p == pe ) + goto _test_eof224; +case 224: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto st225; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st225: + if ( ++p == pe ) + goto _test_eof225; +case 225: + switch( (*p) ) { + case 95: goto tr297; + case 110: goto tr433; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st226: + if ( ++p == pe ) + goto _test_eof226; +case 226: + switch( (*p) ) { + case 95: goto tr297; + case 111: goto st227; + case 114: goto st232; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st227: + if ( ++p == pe ) + goto _test_eof227; +case 227: + switch( (*p) ) { + case 95: goto tr297; + case 115: goto st228; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st228: + if ( ++p == pe ) + goto _test_eof228; +case 228: + switch( (*p) ) { + case 95: goto tr297; + case 116: goto st229; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st229: + if ( ++p == pe ) + goto _test_eof229; +case 229: + switch( (*p) ) { + case 95: goto tr297; + case 112: goto st230; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st230: + if ( ++p == pe ) + goto _test_eof230; +case 230: + switch( (*p) ) { + case 95: goto tr297; + case 111: goto st231; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st231: + if ( ++p == pe ) + goto _test_eof231; +case 231: + switch( (*p) ) { + case 95: goto tr297; + case 112: goto tr440; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st232: + if ( ++p == pe ) + goto _test_eof232; +case 232: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto st233; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st233: + if ( ++p == pe ) + goto _test_eof233; +case 233: + switch( (*p) ) { + case 95: goto tr297; + case 112: goto st234; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st234: + if ( ++p == pe ) + goto _test_eof234; +case 234: + switch( (*p) ) { + case 95: goto tr297; + case 117: goto st235; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st235: + if ( ++p == pe ) + goto _test_eof235; +case 235: + switch( (*p) ) { + case 95: goto tr297; + case 115: goto st236; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st236: + if ( ++p == pe ) + goto _test_eof236; +case 236: + switch( (*p) ) { + case 95: goto tr297; + case 104: goto tr445; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st237: + if ( ++p == pe ) + goto _test_eof237; +case 237: + switch( (*p) ) { + case 95: goto tr297; + case 111: goto tr446; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st238: + if ( ++p == pe ) + goto _test_eof238; +case 238: + switch( (*p) ) { + case 95: goto tr297; + case 97: goto st239; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st239: + if ( ++p == pe ) + goto _test_eof239; +case 239: + switch( (*p) ) { + case 95: goto tr297; + case 114: goto st240; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st240: + if ( ++p == pe ) + goto _test_eof240; +case 240: + switch( (*p) ) { + case 95: goto tr297; + case 105: goto st241; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st241: + if ( ++p == pe ) + goto _test_eof241; +case 241: + switch( (*p) ) { + case 95: goto tr297; + case 97: goto st242; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st242: + if ( ++p == pe ) + goto _test_eof242; +case 242: + switch( (*p) ) { + case 95: goto tr297; + case 98: goto st243; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st243: + if ( ++p == pe ) + goto _test_eof243; +case 243: + switch( (*p) ) { + case 95: goto tr297; + case 108: goto st244; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st244: + if ( ++p == pe ) + goto _test_eof244; +case 244: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto tr453; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st245: + if ( ++p == pe ) + goto _test_eof245; +case 245: + switch( (*p) ) { + case 95: goto tr297; + case 104: goto st246; + case 114: goto st248; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st246: + if ( ++p == pe ) + goto _test_eof246; +case 246: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto st247; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st247: + if ( ++p == pe ) + goto _test_eof247; +case 247: + switch( (*p) ) { + case 95: goto tr297; + case 110: goto tr457; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st248: + if ( ++p == pe ) + goto _test_eof248; +case 248: + switch( (*p) ) { + case 95: goto tr297; + case 105: goto st249; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st249: + if ( ++p == pe ) + goto _test_eof249; +case 249: + switch( (*p) ) { + case 95: goto tr297; + case 116: goto st250; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st250: + if ( ++p == pe ) + goto _test_eof250; +case 250: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto tr460; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st251: + if ( ++p == pe ) + goto _test_eof251; +case 251: + if ( (*p) == 42 ) + goto tr461; + goto tr315; +tr313: +#line 1 "rlscan.rl" + {te = p+1;} + goto st252; +st252: + if ( ++p == pe ) + goto _test_eof252; +case 252: +#line 6363 "rlscan.cpp" + if ( (*p) == 37 ) + goto st30; + goto tr315; +st30: + if ( ++p == pe ) + goto _test_eof30; +case 30: + if ( (*p) == 37 ) + goto tr57; + goto tr45; +tr58: +#line 1135 "rlscan.rl" + {{p = ((te))-1;}{ pass( *ts, 0, 0 ); }} + goto st253; +tr61: +#line 1119 "rlscan.rl" + {te = p+1;{ pass( IMP_Literal, ts, te ); }} + goto st253; +tr64: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } +#line 1117 "rlscan.rl" + {te = p+1;{ pass(); }} + goto st253; +tr463: +#line 1135 "rlscan.rl" + {te = p+1;{ pass( *ts, 0, 0 ); }} + goto st253; +tr464: +#line 1134 "rlscan.rl" + {te = p+1;} + goto st253; +tr474: +#line 1133 "rlscan.rl" + {te = p;p--;{ pass(); }} + goto st253; +tr475: +#line 1135 "rlscan.rl" + {te = p;p--;{ pass( *ts, 0, 0 ); }} + goto st253; +tr477: +#line 1127 "rlscan.rl" + {te = p;p--;{ + updateCol(); + singleLineSpec = true; + startSection(); + {stack[top++] = 253; goto st146;} + }} + goto st253; +tr478: +#line 1121 "rlscan.rl" + {te = p+1;{ + updateCol(); + singleLineSpec = false; + startSection(); + {stack[top++] = 253; goto st146;} + }} + goto st253; +tr479: +#line 1116 "rlscan.rl" + {te = p;p--;{ pass( IMP_UInt, ts, te ); }} + goto st253; +tr480: +#line 1115 "rlscan.rl" + {te = p;p--;{ pass( IMP_Word, ts, te ); }} + goto st253; +st253: +#line 1 "rlscan.rl" + {ts = 0;} + if ( ++p == pe ) + goto _test_eof253; +case 253: +#line 1 "rlscan.rl" + {ts = p;} +#line 6442 "rlscan.cpp" + switch( (*p) ) { + case 0: goto tr464; + case 9: goto st254; + case 10: goto tr466; + case 32: goto st254; + case 34: goto tr467; + case 35: goto tr468; + case 37: goto st257; + case 39: goto tr470; + case 47: goto tr471; + case 95: goto st262; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st261; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto st262; + } else + goto st262; + goto tr463; +tr466: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st254; +st254: + if ( ++p == pe ) + goto _test_eof254; +case 254: +#line 6476 "rlscan.cpp" + switch( (*p) ) { + case 9: goto st254; + case 10: goto tr466; + case 32: goto st254; + } + goto tr474; +tr467: +#line 1 "rlscan.rl" + {te = p+1;} + goto st255; +st255: + if ( ++p == pe ) + goto _test_eof255; +case 255: +#line 6491 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr60; + case 34: goto tr61; + case 92: goto st32; + } + goto st31; +tr60: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st31; +st31: + if ( ++p == pe ) + goto _test_eof31; +case 31: +#line 6510 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr60; + case 34: goto tr61; + case 92: goto st32; + } + goto st31; +st32: + if ( ++p == pe ) + goto _test_eof32; +case 32: + if ( (*p) == 10 ) + goto tr60; + goto st31; +tr468: +#line 1 "rlscan.rl" + {te = p+1;} + goto st256; +st256: + if ( ++p == pe ) + goto _test_eof256; +case 256: +#line 6532 "rlscan.cpp" + if ( (*p) == 10 ) + goto tr64; + goto st33; +st33: + if ( ++p == pe ) + goto _test_eof33; +case 33: + if ( (*p) == 10 ) + goto tr64; + goto st33; +st257: + if ( ++p == pe ) + goto _test_eof257; +case 257: + if ( (*p) == 37 ) + goto st258; + goto tr475; +st258: + if ( ++p == pe ) + goto _test_eof258; +case 258: + if ( (*p) == 123 ) + goto tr478; + goto tr477; +tr470: +#line 1 "rlscan.rl" + {te = p+1;} + goto st259; +st259: + if ( ++p == pe ) + goto _test_eof259; +case 259: +#line 6565 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr66; + case 39: goto tr61; + case 92: goto st35; + } + goto st34; +tr66: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st34; +st34: + if ( ++p == pe ) + goto _test_eof34; +case 34: +#line 6584 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr66; + case 39: goto tr61; + case 92: goto st35; + } + goto st34; +st35: + if ( ++p == pe ) + goto _test_eof35; +case 35: + if ( (*p) == 10 ) + goto tr66; + goto st34; +tr471: +#line 1 "rlscan.rl" + {te = p+1;} + goto st260; +st260: + if ( ++p == pe ) + goto _test_eof260; +case 260: +#line 6606 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr69; + case 47: goto tr61; + case 92: goto st37; + } + goto st36; +tr69: +#line 630 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st36; +st36: + if ( ++p == pe ) + goto _test_eof36; +case 36: +#line 6625 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr69; + case 47: goto tr61; + case 92: goto st37; + } + goto st36; +st37: + if ( ++p == pe ) + goto _test_eof37; +case 37: + if ( (*p) == 10 ) + goto tr69; + goto st36; +st261: + if ( ++p == pe ) + goto _test_eof261; +case 261: + if ( 48 <= (*p) && (*p) <= 57 ) + goto st261; + goto tr479; +st262: + if ( ++p == pe ) + goto _test_eof262; +case 262: + if ( (*p) == 95 ) + goto st262; + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st262; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto st262; + } else + goto st262; + goto tr480; + } + _test_eof38: cs = 38; goto _test_eof; + _test_eof39: cs = 39; goto _test_eof; + _test_eof40: cs = 40; goto _test_eof; + _test_eof1: cs = 1; goto _test_eof; + _test_eof2: cs = 2; goto _test_eof; + _test_eof41: cs = 41; goto _test_eof; + _test_eof42: cs = 42; goto _test_eof; + _test_eof43: cs = 43; goto _test_eof; + _test_eof3: cs = 3; goto _test_eof; + _test_eof4: cs = 4; goto _test_eof; + _test_eof44: cs = 44; goto _test_eof; + _test_eof5: cs = 5; goto _test_eof; + _test_eof6: cs = 6; goto _test_eof; + _test_eof7: cs = 7; goto _test_eof; + _test_eof45: cs = 45; goto _test_eof; + _test_eof46: cs = 46; goto _test_eof; + _test_eof47: cs = 47; goto _test_eof; + _test_eof48: cs = 48; goto _test_eof; + _test_eof49: cs = 49; goto _test_eof; + _test_eof50: cs = 50; goto _test_eof; + _test_eof51: cs = 51; goto _test_eof; + _test_eof52: cs = 52; goto _test_eof; + _test_eof53: cs = 53; goto _test_eof; + _test_eof54: cs = 54; goto _test_eof; + _test_eof8: cs = 8; goto _test_eof; + _test_eof9: cs = 9; goto _test_eof; + _test_eof55: cs = 55; goto _test_eof; + _test_eof10: cs = 10; goto _test_eof; + _test_eof56: cs = 56; goto _test_eof; + _test_eof11: cs = 11; goto _test_eof; + _test_eof12: cs = 12; goto _test_eof; + _test_eof57: cs = 57; goto _test_eof; + _test_eof13: cs = 13; goto _test_eof; + _test_eof14: cs = 14; goto _test_eof; + _test_eof58: cs = 58; goto _test_eof; + _test_eof59: cs = 59; goto _test_eof; + _test_eof15: cs = 15; goto _test_eof; + _test_eof60: cs = 60; goto _test_eof; + _test_eof61: cs = 61; goto _test_eof; + _test_eof62: cs = 62; goto _test_eof; + _test_eof63: cs = 63; goto _test_eof; + _test_eof64: cs = 64; goto _test_eof; + _test_eof65: cs = 65; goto _test_eof; + _test_eof66: cs = 66; goto _test_eof; + _test_eof67: cs = 67; goto _test_eof; + _test_eof68: cs = 68; goto _test_eof; + _test_eof69: cs = 69; goto _test_eof; + _test_eof70: cs = 70; goto _test_eof; + _test_eof71: cs = 71; goto _test_eof; + _test_eof72: cs = 72; goto _test_eof; + _test_eof73: cs = 73; goto _test_eof; + _test_eof74: cs = 74; goto _test_eof; + _test_eof75: cs = 75; goto _test_eof; + _test_eof76: cs = 76; goto _test_eof; + _test_eof77: cs = 77; goto _test_eof; + _test_eof78: cs = 78; goto _test_eof; + _test_eof79: cs = 79; goto _test_eof; + _test_eof80: cs = 80; goto _test_eof; + _test_eof81: cs = 81; goto _test_eof; + _test_eof82: cs = 82; goto _test_eof; + _test_eof83: cs = 83; goto _test_eof; + _test_eof84: cs = 84; goto _test_eof; + _test_eof85: cs = 85; goto _test_eof; + _test_eof86: cs = 86; goto _test_eof; + _test_eof87: cs = 87; goto _test_eof; + _test_eof88: cs = 88; goto _test_eof; + _test_eof89: cs = 89; goto _test_eof; + _test_eof90: cs = 90; goto _test_eof; + _test_eof91: cs = 91; goto _test_eof; + _test_eof92: cs = 92; goto _test_eof; + _test_eof93: cs = 93; goto _test_eof; + _test_eof94: cs = 94; goto _test_eof; + _test_eof95: cs = 95; goto _test_eof; + _test_eof96: cs = 96; goto _test_eof; + _test_eof97: cs = 97; goto _test_eof; + _test_eof16: cs = 16; goto _test_eof; + _test_eof17: cs = 17; goto _test_eof; + _test_eof98: cs = 98; goto _test_eof; + _test_eof18: cs = 18; goto _test_eof; + _test_eof19: cs = 19; goto _test_eof; + _test_eof99: cs = 99; goto _test_eof; + _test_eof20: cs = 20; goto _test_eof; + _test_eof21: cs = 21; goto _test_eof; + _test_eof22: cs = 22; goto _test_eof; + _test_eof100: cs = 100; goto _test_eof; + _test_eof101: cs = 101; goto _test_eof; + _test_eof23: cs = 23; goto _test_eof; + _test_eof102: cs = 102; goto _test_eof; + _test_eof103: cs = 103; goto _test_eof; + _test_eof104: cs = 104; goto _test_eof; + _test_eof105: cs = 105; goto _test_eof; + _test_eof106: cs = 106; goto _test_eof; + _test_eof107: cs = 107; goto _test_eof; + _test_eof108: cs = 108; goto _test_eof; + _test_eof109: cs = 109; goto _test_eof; + _test_eof110: cs = 110; goto _test_eof; + _test_eof111: cs = 111; goto _test_eof; + _test_eof112: cs = 112; goto _test_eof; + _test_eof113: cs = 113; goto _test_eof; + _test_eof114: cs = 114; goto _test_eof; + _test_eof115: cs = 115; goto _test_eof; + _test_eof116: cs = 116; goto _test_eof; + _test_eof117: cs = 117; goto _test_eof; + _test_eof118: cs = 118; goto _test_eof; + _test_eof119: cs = 119; goto _test_eof; + _test_eof120: cs = 120; goto _test_eof; + _test_eof121: cs = 121; goto _test_eof; + _test_eof122: cs = 122; goto _test_eof; + _test_eof123: cs = 123; goto _test_eof; + _test_eof124: cs = 124; goto _test_eof; + _test_eof125: cs = 125; goto _test_eof; + _test_eof126: cs = 126; goto _test_eof; + _test_eof127: cs = 127; goto _test_eof; + _test_eof128: cs = 128; goto _test_eof; + _test_eof129: cs = 129; goto _test_eof; + _test_eof130: cs = 130; goto _test_eof; + _test_eof131: cs = 131; goto _test_eof; + _test_eof132: cs = 132; goto _test_eof; + _test_eof133: cs = 133; goto _test_eof; + _test_eof134: cs = 134; goto _test_eof; + _test_eof135: cs = 135; goto _test_eof; + _test_eof136: cs = 136; goto _test_eof; + _test_eof137: cs = 137; goto _test_eof; + _test_eof138: cs = 138; goto _test_eof; + _test_eof139: cs = 139; goto _test_eof; + _test_eof140: cs = 140; goto _test_eof; + _test_eof141: cs = 141; goto _test_eof; + _test_eof142: cs = 142; goto _test_eof; + _test_eof143: cs = 143; goto _test_eof; + _test_eof144: cs = 144; goto _test_eof; + _test_eof145: cs = 145; goto _test_eof; + _test_eof146: cs = 146; goto _test_eof; + _test_eof147: cs = 147; goto _test_eof; + _test_eof148: cs = 148; goto _test_eof; + _test_eof24: cs = 24; goto _test_eof; + _test_eof149: cs = 149; goto _test_eof; + _test_eof25: cs = 25; goto _test_eof; + _test_eof150: cs = 150; goto _test_eof; + _test_eof26: cs = 26; goto _test_eof; + _test_eof151: cs = 151; goto _test_eof; + _test_eof152: cs = 152; goto _test_eof; + _test_eof153: cs = 153; goto _test_eof; + _test_eof27: cs = 27; goto _test_eof; + _test_eof28: cs = 28; goto _test_eof; + _test_eof154: cs = 154; goto _test_eof; + _test_eof155: cs = 155; goto _test_eof; + _test_eof156: cs = 156; goto _test_eof; + _test_eof157: cs = 157; goto _test_eof; + _test_eof158: cs = 158; goto _test_eof; + _test_eof29: cs = 29; goto _test_eof; + _test_eof159: cs = 159; goto _test_eof; + _test_eof160: cs = 160; goto _test_eof; + _test_eof161: cs = 161; goto _test_eof; + _test_eof162: cs = 162; goto _test_eof; + _test_eof163: cs = 163; goto _test_eof; + _test_eof164: cs = 164; goto _test_eof; + _test_eof165: cs = 165; goto _test_eof; + _test_eof166: cs = 166; goto _test_eof; + _test_eof167: cs = 167; goto _test_eof; + _test_eof168: cs = 168; goto _test_eof; + _test_eof169: cs = 169; goto _test_eof; + _test_eof170: cs = 170; goto _test_eof; + _test_eof171: cs = 171; goto _test_eof; + _test_eof172: cs = 172; goto _test_eof; + _test_eof173: cs = 173; goto _test_eof; + _test_eof174: cs = 174; goto _test_eof; + _test_eof175: cs = 175; goto _test_eof; + _test_eof176: cs = 176; goto _test_eof; + _test_eof177: cs = 177; goto _test_eof; + _test_eof178: cs = 178; goto _test_eof; + _test_eof179: cs = 179; goto _test_eof; + _test_eof180: cs = 180; goto _test_eof; + _test_eof181: cs = 181; goto _test_eof; + _test_eof182: cs = 182; goto _test_eof; + _test_eof183: cs = 183; goto _test_eof; + _test_eof184: cs = 184; goto _test_eof; + _test_eof185: cs = 185; goto _test_eof; + _test_eof186: cs = 186; goto _test_eof; + _test_eof187: cs = 187; goto _test_eof; + _test_eof188: cs = 188; goto _test_eof; + _test_eof189: cs = 189; goto _test_eof; + _test_eof190: cs = 190; goto _test_eof; + _test_eof191: cs = 191; goto _test_eof; + _test_eof192: cs = 192; goto _test_eof; + _test_eof193: cs = 193; goto _test_eof; + _test_eof194: cs = 194; goto _test_eof; + _test_eof195: cs = 195; goto _test_eof; + _test_eof196: cs = 196; goto _test_eof; + _test_eof197: cs = 197; goto _test_eof; + _test_eof198: cs = 198; goto _test_eof; + _test_eof199: cs = 199; goto _test_eof; + _test_eof200: cs = 200; goto _test_eof; + _test_eof201: cs = 201; goto _test_eof; + _test_eof202: cs = 202; goto _test_eof; + _test_eof203: cs = 203; goto _test_eof; + _test_eof204: cs = 204; goto _test_eof; + _test_eof205: cs = 205; goto _test_eof; + _test_eof206: cs = 206; goto _test_eof; + _test_eof207: cs = 207; goto _test_eof; + _test_eof208: cs = 208; goto _test_eof; + _test_eof209: cs = 209; goto _test_eof; + _test_eof210: cs = 210; goto _test_eof; + _test_eof211: cs = 211; goto _test_eof; + _test_eof212: cs = 212; goto _test_eof; + _test_eof213: cs = 213; goto _test_eof; + _test_eof214: cs = 214; goto _test_eof; + _test_eof215: cs = 215; goto _test_eof; + _test_eof216: cs = 216; goto _test_eof; + _test_eof217: cs = 217; goto _test_eof; + _test_eof218: cs = 218; goto _test_eof; + _test_eof219: cs = 219; goto _test_eof; + _test_eof220: cs = 220; goto _test_eof; + _test_eof221: cs = 221; goto _test_eof; + _test_eof222: cs = 222; goto _test_eof; + _test_eof223: cs = 223; goto _test_eof; + _test_eof224: cs = 224; goto _test_eof; + _test_eof225: cs = 225; goto _test_eof; + _test_eof226: cs = 226; goto _test_eof; + _test_eof227: cs = 227; goto _test_eof; + _test_eof228: cs = 228; goto _test_eof; + _test_eof229: cs = 229; goto _test_eof; + _test_eof230: cs = 230; goto _test_eof; + _test_eof231: cs = 231; goto _test_eof; + _test_eof232: cs = 232; goto _test_eof; + _test_eof233: cs = 233; goto _test_eof; + _test_eof234: cs = 234; goto _test_eof; + _test_eof235: cs = 235; goto _test_eof; + _test_eof236: cs = 236; goto _test_eof; + _test_eof237: cs = 237; goto _test_eof; + _test_eof238: cs = 238; goto _test_eof; + _test_eof239: cs = 239; goto _test_eof; + _test_eof240: cs = 240; goto _test_eof; + _test_eof241: cs = 241; goto _test_eof; + _test_eof242: cs = 242; goto _test_eof; + _test_eof243: cs = 243; goto _test_eof; + _test_eof244: cs = 244; goto _test_eof; + _test_eof245: cs = 245; goto _test_eof; + _test_eof246: cs = 246; goto _test_eof; + _test_eof247: cs = 247; goto _test_eof; + _test_eof248: cs = 248; goto _test_eof; + _test_eof249: cs = 249; goto _test_eof; + _test_eof250: cs = 250; goto _test_eof; + _test_eof251: cs = 251; goto _test_eof; + _test_eof252: cs = 252; goto _test_eof; + _test_eof30: cs = 30; goto _test_eof; + _test_eof253: cs = 253; goto _test_eof; + _test_eof254: cs = 254; goto _test_eof; + _test_eof255: cs = 255; goto _test_eof; + _test_eof31: cs = 31; goto _test_eof; + _test_eof32: cs = 32; goto _test_eof; + _test_eof256: cs = 256; goto _test_eof; + _test_eof33: cs = 33; goto _test_eof; + _test_eof257: cs = 257; goto _test_eof; + _test_eof258: cs = 258; goto _test_eof; + _test_eof259: cs = 259; goto _test_eof; + _test_eof34: cs = 34; goto _test_eof; + _test_eof35: cs = 35; goto _test_eof; + _test_eof260: cs = 260; goto _test_eof; + _test_eof36: cs = 36; goto _test_eof; + _test_eof37: cs = 37; goto _test_eof; + _test_eof261: cs = 261; goto _test_eof; + _test_eof262: cs = 262; goto _test_eof; + + _test_eof: {} + if ( p == eof ) + { + switch ( cs ) { + case 39: goto tr82; + case 40: goto tr83; + case 1: goto tr0; + case 2: goto tr0; + case 41: goto tr83; + case 42: goto tr85; + case 43: goto tr83; + case 3: goto tr0; + case 4: goto tr0; + case 44: goto tr83; + case 5: goto tr0; + case 6: goto tr0; + case 7: goto tr0; + case 45: goto tr87; + case 46: goto tr88; + case 47: goto tr89; + case 48: goto tr89; + case 49: goto tr89; + case 50: goto tr89; + case 51: goto tr89; + case 53: goto tr113; + case 54: goto tr114; + case 8: goto tr14; + case 9: goto tr14; + case 55: goto tr114; + case 10: goto tr14; + case 56: goto tr114; + case 11: goto tr14; + case 12: goto tr14; + case 57: goto tr114; + case 13: goto tr14; + case 14: goto tr14; + case 58: goto tr115; + case 59: goto tr115; + case 15: goto tr27; + case 60: goto tr117; + case 61: goto tr114; + case 62: goto tr119; + case 63: goto tr120; + case 64: goto tr120; + case 65: goto tr120; + case 66: goto tr120; + case 67: goto tr120; + case 68: goto tr134; + case 69: goto tr120; + case 70: goto tr120; + case 71: goto tr120; + case 72: goto tr120; + case 73: goto tr120; + case 74: goto tr120; + case 75: goto tr120; + case 76: goto tr120; + case 77: goto tr120; + case 78: goto tr120; + case 79: goto tr120; + case 80: goto tr120; + case 81: goto tr120; + case 82: goto tr120; + case 83: goto tr120; + case 84: goto tr120; + case 85: goto tr120; + case 86: goto tr120; + case 87: goto tr120; + case 88: goto tr120; + case 89: goto tr120; + case 90: goto tr120; + case 91: goto tr120; + case 92: goto tr120; + case 93: goto tr120; + case 94: goto tr120; + case 96: goto tr181; + case 97: goto tr182; + case 16: goto tr29; + case 17: goto tr29; + case 98: goto tr182; + case 18: goto tr29; + case 19: goto tr29; + case 99: goto tr182; + case 20: goto tr29; + case 21: goto tr29; + case 22: goto tr29; + case 100: goto tr183; + case 101: goto tr183; + case 23: goto tr43; + case 102: goto tr185; + case 103: goto tr182; + case 104: goto tr187; + case 105: goto tr188; + case 106: goto tr188; + case 107: goto tr188; + case 108: goto tr188; + case 109: goto tr188; + case 110: goto tr202; + case 111: goto tr188; + case 112: goto tr188; + case 113: goto tr188; + case 114: goto tr188; + case 115: goto tr188; + case 116: goto tr188; + case 117: goto tr188; + case 118: goto tr188; + case 119: goto tr188; + case 120: goto tr188; + case 121: goto tr188; + case 122: goto tr188; + case 123: goto tr188; + case 124: goto tr188; + case 125: goto tr188; + case 126: goto tr188; + case 127: goto tr188; + case 128: goto tr188; + case 129: goto tr188; + case 130: goto tr188; + case 131: goto tr188; + case 132: goto tr188; + case 133: goto tr188; + case 134: goto tr188; + case 135: goto tr188; + case 136: goto tr188; + case 138: goto tr237; + case 140: goto tr255; + case 141: goto tr257; + case 142: goto tr259; + case 144: goto tr275; + case 145: goto tr276; + case 147: goto tr314; + case 148: goto tr315; + case 24: goto tr45; + case 149: goto tr316; + case 25: goto tr45; + case 150: goto tr315; + case 26: goto tr45; + case 151: goto tr315; + case 152: goto tr315; + case 153: goto tr315; + case 27: goto tr45; + case 28: goto tr45; + case 154: goto tr315; + case 155: goto tr315; + case 156: goto tr315; + case 157: goto tr334; + case 158: goto tr334; + case 29: goto tr55; + case 159: goto tr336; + case 160: goto tr315; + case 161: goto tr340; + case 162: goto tr315; + case 163: goto tr349; + case 164: goto tr315; + case 165: goto tr315; + case 166: goto tr315; + case 167: goto tr367; + case 168: goto tr368; + case 169: goto tr370; + case 170: goto tr370; + case 171: goto tr370; + case 172: goto tr370; + case 173: goto tr370; + case 174: goto tr370; + case 175: goto tr370; + case 176: goto tr370; + case 177: goto tr370; + case 178: goto tr370; + case 179: goto tr370; + case 180: goto tr370; + case 181: goto tr370; + case 182: goto tr370; + case 183: goto tr370; + case 184: goto tr370; + case 185: goto tr370; + case 186: goto tr370; + case 187: goto tr370; + case 188: goto tr370; + case 189: goto tr370; + case 190: goto tr370; + case 191: goto tr370; + case 192: goto tr370; + case 193: goto tr370; + case 194: goto tr370; + case 195: goto tr370; + case 196: goto tr370; + case 197: goto tr370; + case 198: goto tr370; + case 199: goto tr370; + case 200: goto tr370; + case 201: goto tr370; + case 202: goto tr370; + case 203: goto tr370; + case 204: goto tr370; + case 205: goto tr370; + case 206: goto tr370; + case 207: goto tr370; + case 208: goto tr370; + case 209: goto tr370; + case 210: goto tr370; + case 211: goto tr370; + case 212: goto tr370; + case 213: goto tr370; + case 214: goto tr370; + case 215: goto tr370; + case 216: goto tr370; + case 217: goto tr370; + case 218: goto tr370; + case 219: goto tr370; + case 220: goto tr370; + case 221: goto tr370; + case 222: goto tr370; + case 223: goto tr370; + case 224: goto tr370; + case 225: goto tr370; + case 226: goto tr370; + case 227: goto tr370; + case 228: goto tr370; + case 229: goto tr370; + case 230: goto tr370; + case 231: goto tr370; + case 232: goto tr370; + case 233: goto tr370; + case 234: goto tr370; + case 235: goto tr370; + case 236: goto tr370; + case 237: goto tr370; + case 238: goto tr370; + case 239: goto tr370; + case 240: goto tr370; + case 241: goto tr370; + case 242: goto tr370; + case 243: goto tr370; + case 244: goto tr370; + case 245: goto tr370; + case 246: goto tr370; + case 247: goto tr370; + case 248: goto tr370; + case 249: goto tr370; + case 250: goto tr370; + case 251: goto tr315; + case 252: goto tr315; + case 30: goto tr45; + case 254: goto tr474; + case 255: goto tr475; + case 31: goto tr58; + case 32: goto tr58; + case 256: goto tr475; + case 33: goto tr58; + case 257: goto tr475; + case 258: goto tr477; + case 259: goto tr475; + case 34: goto tr58; + case 35: goto tr58; + case 260: goto tr475; + case 36: goto tr58; + case 37: goto tr58; + case 261: goto tr479; + case 262: goto tr480; + } + } + + _out: {} + } + +#line 1230 "rlscan.rl" + + /* Check if we failed. */ + if ( cs == rlscan_error ) { + /* Machine failed before finding a token. I'm not yet sure if this + * is reachable. */ + scan_error() << "scanner error" << endl; + exit(1); + } + + /* Decide if we need to preserve anything. */ + char *preserve = ts; + + /* Now set up the prefix. */ + if ( preserve == 0 ) + have = 0; + else { + /* There is data that needs to be shifted over. */ + have = pe - preserve; + memmove( buf, preserve, have ); + unsigned int shiftback = preserve - buf; + if ( ts != 0 ) + ts -= shiftback; + te -= shiftback; + + preserve = buf; + } + } + + delete[] buf; +} diff --git a/ragel/rlscan.h b/ragel/rlscan.h new file mode 100644 index 0000000..b56bd60 --- /dev/null +++ b/ragel/rlscan.h @@ -0,0 +1,132 @@ +/* + * Copyright 2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RLSCAN_H +#define _RLSCAN_H + +#include <iostream> +#include "rlscan.h" +#include "vector.h" +#include "rlparse.h" +#include "parsedata.h" +#include "avltree.h" +#include "vector.h" + +using std::istream; +using std::ostream; + +extern char *Parser_lelNames[]; + +struct Scanner +{ + Scanner( InputData &id, const char *fileName, istream &input, + Parser *inclToParser, char *inclSectionTarg, + int includeDepth, bool importMachines ) + : + id(id), fileName(fileName), + input(input), + inclToParser(inclToParser), + inclSectionTarg(inclSectionTarg), + includeDepth(includeDepth), + importMachines(importMachines), + cur_token(0), + line(1), column(1), lastnl(0), + parser(0), ignoreSection(false), + parserExistsError(false), + whitespaceOn(true), + lastToken(0) + {} + + bool duplicateInclude( char *inclFileName, char *inclSectionName ); + + /* Make a list of places to look for an included file. */ + char **makeIncludePathChecks( const char *curFileName, const char *fileName, int len ); + std::ifstream *tryOpenInclude( char **pathChecks, long &found ); + + void handleMachine(); + void handleInclude(); + void handleImport(); + + void init(); + void token( int type, char *start, char *end ); + void token( int type, char c ); + void token( int type ); + void processToken( int type, char *tokdata, int toklen ); + void directToParser( Parser *toParser, const char *tokFileName, int tokLine, + int tokColumn, int type, char *tokdata, int toklen ); + void flushImport( ); + void importToken( int type, char *start, char *end ); + void pass( int token, char *start, char *end ); + void pass(); + void updateCol(); + void startSection(); + void endSection(); + void do_scan(); + bool active(); + ostream &scan_error(); + + InputData &id; + const char *fileName; + istream &input; + Parser *inclToParser; + char *inclSectionTarg; + int includeDepth; + bool importMachines; + + /* For import parsing. */ + int tok_cs, tok_act; + int *tok_ts, *tok_te; + int cur_token; + static const int max_tokens = 32; + int token_data[max_tokens]; + char *token_strings[max_tokens]; + int token_lens[max_tokens]; + + /* For section processing. */ + int cs; + char *word, *lit; + int word_len, lit_len; + + /* For character scanning. */ + int line; + InputLoc sectionLoc; + char *ts, *te; + int column; + char *lastnl; + + /* Set by machine statements, these persist from section to section + * allowing for unnamed sections. */ + Parser *parser; + bool ignoreSection; + + /* This is set if ragel has already emitted an error stating that + * no section name has been seen and thus no parser exists. */ + bool parserExistsError; + + /* This is for inline code. By default it is on. It goes off for + * statements and values in inline blocks which are parsed. */ + bool whitespaceOn; + + /* Keeps a record of the previous token sent to the section parser. */ + int lastToken; +}; + +#endif diff --git a/ragel/rlscan.rl b/ragel/rlscan.rl index b1396f1..68c9c22 100644 --- a/ragel/rlscan.rl +++ b/ragel/rlscan.rl @@ -1,5 +1,5 @@ /* - * Copyright 2006 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2006-2007 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -24,11 +24,10 @@ #include <string.h> #include "ragel.h" -#include "rlparse.h" -#include "parsedata.h" -#include "avltree.h" -#include "vector.h" +#include "rlscan.h" +#include "inputdata.h" +//#define LOG_TOKENS using std::ifstream; using std::istream; @@ -37,96 +36,179 @@ using std::cout; using std::cerr; using std::endl; -/* This is used for tracking the current stack of include file/machine pairs. It is - * is used to detect and recursive include structure. */ -struct IncludeStackItem -{ - IncludeStackItem( char *fileName, char *sectionName ) - : fileName(fileName), sectionName(sectionName) {} - - char *fileName; - char *sectionName; -}; - -typedef Vector<IncludeStackItem> IncludeStack; -IncludeStack includeStack; - enum InlineBlockType { CurlyDelimited, SemiTerminated }; -struct Scanner +#ifdef _WIN32 +#define PATH_SEP '\\' +#else +#define PATH_SEP '/' +#endif + + +/* + * The Scanner for Importing + */ + +%%{ + machine inline_token_scan; + alphtype int; + access tok_; + + # Import scanner tokens. + import "rlparse.h"; + + main := |* + # Define of number. + IMP_Define IMP_Word IMP_UInt => { + int base = tok_ts - token_data; + int nameOff = 1; + int numOff = 2; + + directToParser( inclToParser, fileName, line, column, TK_Word, + token_strings[base+nameOff], token_lens[base+nameOff] ); + directToParser( inclToParser, fileName, line, column, '=', 0, 0 ); + directToParser( inclToParser, fileName, line, column, TK_UInt, + token_strings[base+numOff], token_lens[base+numOff] ); + directToParser( inclToParser, fileName, line, column, ';', 0, 0 ); + }; + + # Assignment of number. + IMP_Word '=' IMP_UInt => { + int base = tok_ts - token_data; + int nameOff = 0; + int numOff = 2; + + directToParser( inclToParser, fileName, line, column, TK_Word, + token_strings[base+nameOff], token_lens[base+nameOff] ); + directToParser( inclToParser, fileName, line, column, '=', 0, 0 ); + directToParser( inclToParser, fileName, line, column, TK_UInt, + token_strings[base+numOff], token_lens[base+numOff] ); + directToParser( inclToParser, fileName, line, column, ';', 0, 0 ); + }; + + # Define of literal. + IMP_Define IMP_Word IMP_Literal => { + int base = tok_ts - token_data; + int nameOff = 1; + int litOff = 2; + + directToParser( inclToParser, fileName, line, column, TK_Word, + token_strings[base+nameOff], token_lens[base+nameOff] ); + directToParser( inclToParser, fileName, line, column, '=', 0, 0 ); + directToParser( inclToParser, fileName, line, column, TK_Literal, + token_strings[base+litOff], token_lens[base+litOff] ); + directToParser( inclToParser, fileName, line, column, ';', 0, 0 ); + }; + + # Assignment of literal. + IMP_Word '=' IMP_Literal => { + int base = tok_ts - token_data; + int nameOff = 0; + int litOff = 2; + + directToParser( inclToParser, fileName, line, column, TK_Word, + token_strings[base+nameOff], token_lens[base+nameOff] ); + directToParser( inclToParser, fileName, line, column, '=', 0, 0 ); + directToParser( inclToParser, fileName, line, column, TK_Literal, + token_strings[base+litOff], token_lens[base+litOff] ); + directToParser( inclToParser, fileName, line, column, ';', 0, 0 ); + }; + + # Catch everything else. + any; + *|; +}%% + +%% write data; + +void Scanner::flushImport() { - Scanner( char *fileName, istream &input, - Parser *inclToParser, char *inclSectionTarg, - int include_depth ) - : - fileName(fileName), input(input), - inclToParser(inclToParser), - inclSectionTarg(inclSectionTarg), - include_depth(include_depth), - line(1), column(1), lastnl(0), - parser(0), active(false), - parserExistsError(false), ragelDefOpen(false), - whitespaceOn(true) - {} - - bool recursiveInclude( IncludeStack &includeStack, - char *inclFileName, char *inclSectionName ); - - char *prepareFileName( char *fileName, int len ) - { - bool caseInsensitive; - Token tokenFnStr, tokenRes; - tokenFnStr.data = fileName; - tokenFnStr.length = len; - tokenFnStr.prepareLitString( tokenRes, caseInsensitive ); - return tokenRes.data; + int *p = token_data; + int *pe = token_data + cur_token; + int *eof = 0; + + %%{ + machine inline_token_scan; + write init; + write exec; + }%% + + if ( tok_ts == 0 ) + cur_token = 0; + else { + cur_token = pe - tok_ts; + int ts_offset = tok_ts - token_data; + memmove( token_data, token_data+ts_offset, cur_token*sizeof(token_data[0]) ); + memmove( token_strings, token_strings+ts_offset, cur_token*sizeof(token_strings[0]) ); + memmove( token_lens, token_lens+ts_offset, cur_token*sizeof(token_lens[0]) ); } +} - void init(); - void token( int type, char *start, char *end ); - void token( int type, char *string ); - void token( int type ); - void updateCol(); - void startSection(); - void endSection(); - void openRagelDef(); - void do_scan(); - bool parserExists(); - ostream &error(); - - char *fileName; - istream &input; - Parser *inclToParser; - char *inclSectionTarg; - int include_depth; - - int cs; - int line; - char *word, *lit; - int word_len, lit_len; - InputLoc sectionLoc; - char *tokstart, *tokend; - int column; - char *lastnl; - - /* Set by machine statements, these persist from section to section - * allowing for unnamed sections. */ - Parser *parser; - bool active; - - /* This is set if ragel has already emitted an error stating that - * no section name has been seen and thus no parser exists. */ - bool parserExistsError; - bool ragelDefOpen; - - /* This is for inline code. By default it is on. It goes off for - * statements and values in inline blocks which are parsed. */ - bool whitespaceOn; -}; +void Scanner::directToParser( Parser *toParser, const char *tokFileName, int tokLine, + int tokColumn, int type, char *tokdata, int toklen ) +{ + InputLoc loc; + + #ifdef LOG_TOKENS + cerr << "scanner:" << tokLine << ":" << tokColumn << + ": sending token to the parser " << Parser_lelNames[type]; + cerr << " " << toklen; + if ( tokdata != 0 ) + cerr << " " << tokdata; + cerr << endl; + #endif + + loc.fileName = tokFileName; + loc.line = tokLine; + loc.col = tokColumn; + + toParser->token( loc, type, tokdata, toklen ); +} + +void Scanner::importToken( int token, char *start, char *end ) +{ + if ( cur_token == max_tokens ) + flushImport(); + + token_data[cur_token] = token; + if ( start == 0 ) { + token_strings[cur_token] = 0; + token_lens[cur_token] = 0; + } + else { + int toklen = end-start; + token_lens[cur_token] = toklen; + token_strings[cur_token] = new char[toklen+1]; + memcpy( token_strings[cur_token], start, toklen ); + token_strings[cur_token][toklen] = 0; + } + cur_token++; +} + +void Scanner::pass( int token, char *start, char *end ) +{ + if ( importMachines ) + importToken( token, start, end ); + pass(); +} + +void Scanner::pass() +{ + updateCol(); + + /* If no errors and we are at the bottom of the include stack (the + * source file listed on the command line) then write out the data. */ + if ( includeDepth == 0 && machineSpec == 0 && machineName == 0 ) + id.inputItems.tail->data.write( ts, te-ts ); +} + +/* + * The scanner for processing sections, includes, imports, etc. + */ %%{ machine section_parse; @@ -134,38 +216,44 @@ struct Scanner write data; }%% + void Scanner::init( ) { %% write init; } -bool Scanner::parserExists() +bool Scanner::active() { - if ( parser != 0 ) - return true; + if ( ignoreSection ) + return false; - if ( ! parserExistsError ) { - error() << "include: there is no previous specification name" << endl; + if ( parser == 0 && ! parserExistsError ) { + scan_error() << "this specification has no name, nor does any previous" + " specification" << endl; parserExistsError = true; } - return false; + + if ( parser == 0 ) + return false; + + return true; } -ostream &Scanner::error() +ostream &Scanner::scan_error() { /* Maintain the error count. */ gblErrorCount += 1; - - cerr << fileName << ":" << line << ":" << column << ": "; + cerr << makeInputLoc( fileName, line, column ) << ": "; return cerr; } -bool Scanner::recursiveInclude( IncludeStack &includeStack, - char *inclFileName, char *inclSectionName ) +/* An approximate check for duplicate includes. Due to aliasing of files it's + * possible for duplicates to creep in. */ +bool Scanner::duplicateInclude( char *inclFileName, char *inclSectionName ) { - for ( IncludeStack::Iter si = includeStack; si.lte(); si++ ) { - if ( strcmp( si->fileName, inclFileName ) == 0 && - strcmp( si->sectionName, inclSectionName ) == 0 ) + for ( IncludeHistory::Iter hi = parser->includeHistory; hi.lte(); hi++ ) { + if ( strcmp( hi->fileName, inclFileName ) == 0 && + strcmp( hi->sectionName, inclSectionName ) == 0 ) { return true; } @@ -177,123 +265,135 @@ void Scanner::updateCol() { char *from = lastnl; if ( from == 0 ) - from = tokstart; - //cerr << "adding " << tokend - from << " to column" << endl; - column += tokend - from; + from = ts; + //cerr << "adding " << te - from << " to column" << endl; + column += te - from; lastnl = 0; } -void Scanner::token( int type, char *string ) +void Scanner::handleMachine() { - token( type, string, string + strlen(string) ); -} + /* Assign a name to the machine. */ + char *machine = word; + + if ( !importMachines && inclSectionTarg == 0 ) { + ignoreSection = false; + + ParserDictEl *pdEl = id.parserDict.find( machine ); + if ( pdEl == 0 ) { + pdEl = new ParserDictEl( machine ); + pdEl->value = new Parser( fileName, machine, sectionLoc ); + pdEl->value->init(); + id.parserDict.insert( pdEl ); + id.parserList.append( pdEl->value ); + } -void Scanner::token( int type ) -{ - token( type, 0, 0 ); + parser = pdEl->value; + } + else if ( !importMachines && strcmp( inclSectionTarg, machine ) == 0 ) { + /* found include target */ + ignoreSection = false; + parser = inclToParser; + } + else { + /* ignoring section */ + ignoreSection = true; + parser = 0; + } } -%%{ - machine section_parse; - - # This relies on the the kelbt implementation and the order - # that tokens are declared. - KW_Machine = 128; - KW_Include = 129; - KW_Write = 130; - TK_Word = 131; - TK_Literal = 132; - - action clear_words { word = lit = 0; word_len = lit_len = 0; } - action store_word { word = tokdata; word_len = toklen; } - action store_lit { lit = tokdata; lit_len = toklen; } - - action mach_err { error() << "bad machine statement" << endl; } - action incl_err { error() << "bad include statement" << endl; } - action write_err { error() << "bad write statement" << endl; } +void Scanner::handleInclude() +{ + if ( active() ) { + char *inclSectionName = word; + char **includeChecks = 0; - action handle_machine - { - /* Assign a name to the machine. */ - char *machine = word; - //cerr << "scanner: machine statement: " << machine << endl; + /* Implement defaults for the input file and section name. */ + if ( inclSectionName == 0 ) + inclSectionName = parser->sectionName; - if ( inclSectionTarg == 0 ) { - active = true; + if ( lit != 0 ) + includeChecks = makeIncludePathChecks( fileName, lit, lit_len ); + else { + char *test = new char[strlen(fileName)+1]; + strcpy( test, fileName ); - ParserDictEl *pdEl = parserDict.find( machine ); - if ( pdEl != 0 ) { - //cerr << "scanner: using existing parser" << endl; - } - else { - //cerr << "scanner: creating a new parser" << endl; - pdEl = new ParserDictEl( machine ); - pdEl->value = new Parser( fileName, machine, sectionLoc ); - pdEl->value->init(); - parserDict.insert( pdEl ); - } + includeChecks = new char*[2]; - parser = pdEl->value; + includeChecks[0] = test; + includeChecks[1] = 0; } - else if ( strcmp( inclSectionTarg, machine ) == 0 ) { - //cerr << "scanner: found include target" << endl; - active = true; - parser = inclToParser; + + long found = 0; + ifstream *inFile = tryOpenInclude( includeChecks, found ); + if ( inFile == 0 ) { + scan_error() << "include: failed to locate file" << endl; + char **tried = includeChecks; + while ( *tried != 0 ) + scan_error() << "include: attempted: \"" << *tried++ << '\"' << endl; } else { - //cerr << "scanner: ignoring section" << endl; - active = false; - parser = 0; + /* Don't include anything that's already been included. */ + if ( !duplicateInclude( includeChecks[found], inclSectionName ) ) { + parser->includeHistory.append( IncludeHistoryItem( + includeChecks[found], inclSectionName ) ); + + Scanner scanner( id, includeChecks[found], *inFile, parser, + inclSectionName, includeDepth+1, false ); + scanner.do_scan( ); + delete inFile; + } } } +} - machine_stmt = - ( KW_Machine TK_Word @store_word ';' ) @handle_machine - <>err mach_err <>eof mach_err; +void Scanner::handleImport() +{ + if ( active() ) { + char **importChecks = makeIncludePathChecks( fileName, lit, lit_len ); + + /* Open the input file for reading. */ + long found = 0; + ifstream *inFile = tryOpenInclude( importChecks, found ); + if ( inFile == 0 ) { + scan_error() << "import: could not open import file " << + "for reading" << endl; + char **tried = importChecks; + while ( *tried != 0 ) + scan_error() << "import: attempted: \"" << *tried++ << '\"' << endl; + } - action handle_include - { - if ( active && parserExists() ) { - char *inclSectionName = word; - char *inclFileName = 0; + Scanner scanner( id, importChecks[found], *inFile, parser, + 0, includeDepth+1, true ); + scanner.do_scan( ); + scanner.importToken( 0, 0, 0 ); + scanner.flushImport(); + delete inFile; + } +} - /* Implement defaults for the input file and section name. */ - if ( inclSectionName == 0 ) - inclSectionName = parser->sectionName; +%%{ + machine section_parse; - if ( lit != 0 ) - inclFileName = prepareFileName( lit, lit_len ); - else - inclFileName = fileName; + # Need the defines representing tokens. + import "rlparse.h"; - /* Open the file and process it. */ - //cerr << "scanner: include: " << inclSectionName << " " << inclFileName << endl; + action clear_words { word = lit = 0; word_len = lit_len = 0; } + action store_word { word = tokdata; word_len = toklen; } + action store_lit { lit = tokdata; lit_len = toklen; } - /* Check for a recursive include structure. Add the current file/section - * name then check if what we are including is already in the stack. */ - includeStack.append( IncludeStackItem( fileName, parser->sectionName ) ); + action mach_err { scan_error() << "bad machine statement" << endl; } + action incl_err { scan_error() << "bad include statement" << endl; } + action import_err { scan_error() << "bad import statement" << endl; } + action write_err { scan_error() << "bad write statement" << endl; } - if ( recursiveInclude( includeStack, inclFileName, inclSectionName ) ) - error() << "include: this is a recursive include operation" << endl; - else { - /* Open the input file for reading. */ - ifstream *inFile = new ifstream( inclFileName ); - if ( ! inFile->is_open() ) { - error() << "include: could not open " << - inclFileName << " for reading" << endl; - } - - Scanner scanner( inclFileName, *inFile, parser, - inclSectionName, include_depth+1 ); - scanner.init(); - scanner.do_scan( ); - delete inFile; - } + action handle_machine { handleMachine(); } + action handle_include { handleInclude(); } + action handle_import { handleImport(); } - /* Remove the last element (len-1) */ - includeStack.remove( -1 ); - } - } + machine_stmt = + ( KW_Machine TK_Word @store_word ';' ) @handle_machine + <>err mach_err <>eof mach_err; include_names = ( TK_Word @store_word ( TK_Literal @store_lit )? | @@ -304,75 +404,75 @@ void Scanner::token( int type ) ( KW_Include include_names ';' ) @handle_include <>err incl_err <>eof incl_err; + import_stmt = + ( KW_Import TK_Literal @store_lit ';' ) @handle_import + <>err import_err <>eof import_err; + action write_command { - if ( active ) { - openRagelDef(); - if ( strcmp( tokdata, "data" ) != 0 && - strcmp( tokdata, "init" ) != 0 && - strcmp( tokdata, "exec" ) != 0 && - strcmp( tokdata, "eof" ) != 0 ) - { - error() << "unknown write command" << endl; - } - *outStream << " <write what=\"" << tokdata << "\">"; + if ( active() && machineSpec == 0 && machineName == 0 ) { + InputItem *inputItem = new InputItem; + inputItem->type = InputItem::Write; + inputItem->loc.fileName = fileName; + inputItem->loc.line = line; + inputItem->loc.col = column; + inputItem->name = parser->sectionName; + inputItem->pd = parser->pd; + id.inputItems.append( inputItem ); } } - action write_option + action write_arg { - if ( active ) - *outStream << "<option>" << tokdata << "</option>"; + if ( active() && machineSpec == 0 && machineName == 0 ) + id.inputItems.tail->writeArgs.append( strdup(tokdata) ); } + action write_close { - if ( active ) - *outStream << "</write>\n"; + if ( active() && machineSpec == 0 && machineName == 0 ) + id.inputItems.tail->writeArgs.append( 0 ); } write_stmt = - ( KW_Write TK_Word @write_command - ( TK_Word @write_option )* ';' @write_close ) + ( KW_Write @write_command + ( TK_Word @write_arg )+ ';' @write_close ) <>err write_err <>eof write_err; action handle_token { /* Send the token off to the parser. */ - if ( active && parserExists() ) { - InputLoc loc; - - //cerr << "scanner:" << line << ":" << column << - // ": sending token to the parser " << lelNames[*p]; - //if ( tokdata != 0 ) - // cerr << " " << tokdata; - //cerr << endl; - - loc.fileName = fileName; - loc.line = line; - loc.col = column; - - parser->token( loc, type, tokdata, toklen ); - } + if ( active() ) + directToParser( parser, fileName, line, column, type, tokdata, toklen ); } # Catch everything else. - everything_else = ^( KW_Machine | KW_Include | KW_Write ) @handle_token; + everything_else = + ^( KW_Machine | KW_Include | KW_Import | KW_Write ) @handle_token; main := ( machine_stmt | include_stmt | + import_stmt | write_stmt | everything_else )*; }%% +void Scanner::token( int type, char c ) +{ + token( type, &c, &c + 1 ); +} + +void Scanner::token( int type ) +{ + token( type, 0, 0 ); +} + void Scanner::token( int type, char *start, char *end ) { char *tokdata = 0; int toklen = 0; - int *p = &type; - int *pe = &type + 1; - if ( start != 0 ) { toklen = end-start; tokdata = new char[toklen+1]; @@ -380,67 +480,145 @@ void Scanner::token( int type, char *start, char *end ) tokdata[toklen] = 0; } + processToken( type, tokdata, toklen ); +} + +void Scanner::processToken( int type, char *tokdata, int toklen ) +{ + int *p, *pe, *eof; + + if ( type < 0 ) + p = pe = eof = 0; + else { + p = &type; + pe = &type + 1; + eof = 0; + } + %%{ machine section_parse; write exec; }%% updateCol(); + + /* Record the last token for use in controlling the scan of subsequent + * tokens. */ + lastToken = type; } void Scanner::startSection( ) { parserExistsError = false; - if ( include_depth == 0 ) { - if ( machineSpec == 0 && machineName == 0 ) - *outStream << "</host>\n"; - ragelDefOpen = false; - } - sectionLoc.fileName = fileName; sectionLoc.line = line; - sectionLoc.col = 0; -} - -void Scanner::openRagelDef() -{ - if ( ! ragelDefOpen ) { - ragelDefOpen = true; - *outStream << "<ragel_def name=\"" << parser->sectionName << "\">\n"; - } + sectionLoc.col = column; } void Scanner::endSection( ) { /* Execute the eof actions for the section parser. */ - %%{ - machine section_parse; - write eof; - }%% + processToken( -1, 0, 0 ); /* Close off the section with the parser. */ - if ( active && parserExists() ) { + if ( active() ) { InputLoc loc; loc.fileName = fileName; loc.line = line; - loc.col = 0; + loc.col = column; parser->token( loc, TK_EndSection, 0, 0 ); } - if ( include_depth == 0 ) { - if ( ragelDefOpen ) { - *outStream << "</ragel_def>\n"; - ragelDefOpen = false; - } - + if ( includeDepth == 0 ) { if ( machineSpec == 0 && machineName == 0 ) { /* The end section may include a newline on the end, so * we use the last line, which will count the newline. */ - *outStream << "<host line=\"" << line << "\">"; + InputItem *inputItem = new InputItem; + inputItem->type = InputItem::HostData; + inputItem->loc.line = line; + inputItem->loc.col = column; + id.inputItems.append( inputItem ); + } + } +} + +bool isAbsolutePath( const char *path ) +{ +#ifdef _WIN32 + return isalpha( path[0] ) && path[1] == ':' && path[2] == '\\'; +#else + return path[0] == '/'; +#endif +} + +char **Scanner::makeIncludePathChecks( const char *thisFileName, + const char *fileName, int fnlen ) +{ + char **checks = 0; + long nextCheck = 0; + long length = 0; + bool caseInsensitive = false; + char *data = prepareLitString( InputLoc(), fileName, fnlen, + length, caseInsensitive ); + + /* Absolute path? */ + if ( isAbsolutePath( data ) ) { + checks = new char*[2]; + checks[nextCheck++] = data; + } + else { + checks = new char*[2 + id.includePaths.length()]; + + /* Search from the the location of the current file. */ + const char *lastSlash = strrchr( thisFileName, PATH_SEP ); + if ( lastSlash == 0 ) + checks[nextCheck++] = data; + else { + long givenPathLen = (lastSlash - thisFileName) + 1; + long checklen = givenPathLen + length; + char *check = new char[checklen+1]; + memcpy( check, thisFileName, givenPathLen ); + memcpy( check+givenPathLen, data, length ); + check[checklen] = 0; + checks[nextCheck++] = check; + } + + /* Search from the include paths given on the command line. */ + for ( ArgsVector::Iter incp = id.includePaths; incp.lte(); incp++ ) { + long pathLen = strlen( *incp ); + long checkLen = pathLen + 1 + length; + char *check = new char[checkLen+1]; + memcpy( check, *incp, pathLen ); + check[pathLen] = PATH_SEP; + memcpy( check+pathLen+1, data, length ); + check[checkLen] = 0; + checks[nextCheck++] = check; } } + + checks[nextCheck] = 0; + return checks; +} + +ifstream *Scanner::tryOpenInclude( char **pathChecks, long &found ) +{ + char **check = pathChecks; + ifstream *inFile = new ifstream; + + while ( *check != 0 ) { + inFile->open( *check ); + if ( inFile->is_open() ) { + found = check - pathChecks; + return inFile; + } + check += 1; + } + + found = -1; + delete inFile; + return 0; } %%{ @@ -469,16 +647,120 @@ void Scanner::endSection( ) c_cpp_comment = c_comment | cpp_comment; - # These literal forms are common to C-like host code and ragel. + ruby_comment = '#' [^\n]* NL; + + # These literal forms are common to host code and ragel. s_literal = "'" ([^'\\] | NL | '\\' (any | NL))* "'"; d_literal = '"' ([^"\\] | NL | '\\' (any | NL))* '"'; + host_re_literal = '/' ([^/\\] | NL | '\\' (any | NL))* '/'; whitespace = [ \t] | NL; pound_comment = '#' [^\n]* NL; - # An inline block of code. This is specified as a scanned, but is sent to - # the parser as one long block. The inline_block pointer is used to handle - # the preservation of the data. + # An inline block of code for Ruby. + inline_code_ruby := |* + # Inline expression keywords. + "fpc" => { token( KW_PChar ); }; + "fc" => { token( KW_Char ); }; + "fcurs" => { token( KW_CurState ); }; + "ftargs" => { token( KW_TargState ); }; + "fentry" => { + whitespaceOn = false; + token( KW_Entry ); + }; + + # Inline statement keywords. + "fhold" => { + whitespaceOn = false; + token( KW_Hold ); + }; + "fexec" => { token( KW_Exec, 0, 0 ); }; + "fgoto" => { + whitespaceOn = false; + token( KW_Goto ); + }; + "fnext" => { + whitespaceOn = false; + token( KW_Next ); + }; + "fcall" => { + whitespaceOn = false; + token( KW_Call ); + }; + "fret" => { + whitespaceOn = false; + token( KW_Ret ); + }; + "fbreak" => { + whitespaceOn = false; + token( KW_Break ); + }; + + ident => { token( TK_Word, ts, te ); }; + + number => { token( TK_UInt, ts, te ); }; + hex_number => { token( TK_Hex, ts, te ); }; + + ( s_literal | d_literal | host_re_literal ) + => { token( IL_Literal, ts, te ); }; + + whitespace+ => { + if ( whitespaceOn ) + token( IL_WhiteSpace, ts, te ); + }; + + ruby_comment => { token( IL_Comment, ts, te ); }; + + "::" => { token( TK_NameSep, ts, te ); }; + + # Some symbols need to go to the parser as with their cardinal value as + # the token type (as opposed to being sent as anonymous symbols) + # because they are part of the sequences which we interpret. The * ) ; + # symbols cause whitespace parsing to come back on. This gets turned + # off by some keywords. + + ";" => { + whitespaceOn = true; + token( *ts, ts, te ); + if ( inlineBlockType == SemiTerminated ) + fret; + }; + + [*)] => { + whitespaceOn = true; + token( *ts, ts, te ); + }; + + [,(] => { token( *ts, ts, te ); }; + + '{' => { + token( IL_Symbol, ts, te ); + curly_count += 1; + }; + + '}' => { + if ( --curly_count == 0 && inlineBlockType == CurlyDelimited ) { + /* Inline code block ends. */ + token( '}' ); + fret; + } + else { + /* Either a semi terminated inline block or only the closing + * brace of some inner scope, not the block's closing brace. */ + token( IL_Symbol, ts, te ); + } + }; + + EOF => { + scan_error() << "unterminated code block" << endl; + }; + + # Send every other character as a symbol. + any => { token( IL_Symbol, ts, te ); }; + *|; + + + # An inline block of code for languages other than Ruby. inline_code := |* # Inline expression keywords. "fpc" => { token( KW_PChar ); }; @@ -517,21 +799,22 @@ void Scanner::endSection( ) token( KW_Break ); }; - ident => { token( TK_Word, tokstart, tokend ); }; + ident => { token( TK_Word, ts, te ); }; - number => { token( TK_UInt, tokstart, tokend ); }; - hex_number => { token( TK_Hex, tokstart, tokend ); }; + number => { token( TK_UInt, ts, te ); }; + hex_number => { token( TK_Hex, ts, te ); }; ( s_literal | d_literal ) - => { token( IL_Literal, tokstart, tokend ); }; + => { token( IL_Literal, ts, te ); }; whitespace+ => { if ( whitespaceOn ) - token( IL_WhiteSpace, tokstart, tokend ); + token( IL_WhiteSpace, ts, te ); }; - c_cpp_comment => { token( IL_Comment, tokstart, tokend ); }; - "::" => { token( TK_NameSep, tokstart, tokend ); }; + c_cpp_comment => { token( IL_Comment, ts, te ); }; + + "::" => { token( TK_NameSep, ts, te ); }; # Some symbols need to go to the parser as with their cardinal value as # the token type (as opposed to being sent as anonymous symbols) @@ -541,20 +824,20 @@ void Scanner::endSection( ) ";" => { whitespaceOn = true; - token( *tokstart, tokstart, tokend ); + token( *ts, ts, te ); if ( inlineBlockType == SemiTerminated ) - fgoto parser_def; + fret; }; [*)] => { whitespaceOn = true; - token( *tokstart, tokstart, tokend ); + token( *ts, ts, te ); }; - [,(] => { token( *tokstart, tokstart, tokend ); }; + [,(] => { token( *ts, ts, te ); }; '{' => { - token( IL_Symbol, tokstart, tokend ); + token( IL_Symbol, ts, te ); curly_count += 1; }; @@ -562,31 +845,35 @@ void Scanner::endSection( ) if ( --curly_count == 0 && inlineBlockType == CurlyDelimited ) { /* Inline code block ends. */ token( '}' ); - fgoto parser_def; + fret; } else { /* Either a semi terminated inline block or only the closing * brace of some inner scope, not the block's closing brace. */ - token( IL_Symbol, tokstart, tokend ); + token( IL_Symbol, ts, te ); } }; + EOF => { + scan_error() << "unterminated code block" << endl; + }; + # Send every other character as a symbol. - any => { token( IL_Symbol, tokstart, tokend ); }; + any => { token( IL_Symbol, ts, te ); }; *|; or_literal := |* # Escape sequences in OR expressions. - '\\0' => { token( RE_Char, "\0" ); }; - '\\a' => { token( RE_Char, "\a" ); }; - '\\b' => { token( RE_Char, "\b" ); }; - '\\t' => { token( RE_Char, "\t" ); }; - '\\n' => { token( RE_Char, "\n" ); }; - '\\v' => { token( RE_Char, "\v" ); }; - '\\f' => { token( RE_Char, "\f" ); }; - '\\r' => { token( RE_Char, "\r" ); }; + '\\0' => { token( RE_Char, '\0' ); }; + '\\a' => { token( RE_Char, '\a' ); }; + '\\b' => { token( RE_Char, '\b' ); }; + '\\t' => { token( RE_Char, '\t' ); }; + '\\n' => { token( RE_Char, '\n' ); }; + '\\v' => { token( RE_Char, '\v' ); }; + '\\f' => { token( RE_Char, '\f' ); }; + '\\r' => { token( RE_Char, '\r' ); }; '\\\n' => { updateCol(); }; - '\\' any => { token( RE_Char, tokstart+1, tokend ); }; + '\\' any => { token( RE_Char, ts+1, te ); }; # Range dash in an OR expression. '-' => { token( RE_Dash, 0, 0 ); }; @@ -594,26 +881,31 @@ void Scanner::endSection( ) # Terminate an OR expression. ']' => { token( RE_SqClose ); fret; }; + EOF => { + scan_error() << "unterminated OR literal" << endl; + }; + # Characters in an OR expression. - [^\]] => { token( RE_Char, tokstart, tokend ); }; + [^\]] => { token( RE_Char, ts, te ); }; + *|; - re_literal := |* + ragel_re_literal := |* # Escape sequences in regular expressions. - '\\0' => { token( RE_Char, "\0" ); }; - '\\a' => { token( RE_Char, "\a" ); }; - '\\b' => { token( RE_Char, "\b" ); }; - '\\t' => { token( RE_Char, "\t" ); }; - '\\n' => { token( RE_Char, "\n" ); }; - '\\v' => { token( RE_Char, "\v" ); }; - '\\f' => { token( RE_Char, "\f" ); }; - '\\r' => { token( RE_Char, "\r" ); }; + '\\0' => { token( RE_Char, '\0' ); }; + '\\a' => { token( RE_Char, '\a' ); }; + '\\b' => { token( RE_Char, '\b' ); }; + '\\t' => { token( RE_Char, '\t' ); }; + '\\n' => { token( RE_Char, '\n' ); }; + '\\v' => { token( RE_Char, '\v' ); }; + '\\f' => { token( RE_Char, '\f' ); }; + '\\r' => { token( RE_Char, '\r' ); }; '\\\n' => { updateCol(); }; - '\\' any => { token( RE_Char, tokstart+1, tokend ); }; + '\\' any => { token( RE_Char, ts+1, te ); }; # Terminate an OR expression. '/' [i]? => { - token( RE_Slash, tokstart, tokend ); + token( RE_Slash, ts, te ); fgoto parser_def; }; @@ -624,64 +916,92 @@ void Scanner::endSection( ) '[' => { token( RE_SqOpen ); fcall or_literal; }; '[^' => { token( RE_SqOpenNeg ); fcall or_literal; }; + EOF => { + scan_error() << "unterminated regular expression" << endl; + }; + # Characters in an OR expression. - [^\/] => { token( RE_Char, tokstart, tokend ); }; + [^\/] => { token( RE_Char, ts, te ); }; *|; + # We need a separate token space here to avoid the ragel keywords. write_statement := |* - ident => { token( TK_Word, tokstart, tokend ); } ; + ident => { token( TK_Word, ts, te ); } ; [ \t\n]+ => { updateCol(); }; ';' => { token( ';' ); fgoto parser_def; }; + + EOF => { + scan_error() << "unterminated write statement" << endl; + }; *|; # Parser definitions. parser_def := |* + #'length_cond' => { token( KW_Length ); }; 'machine' => { token( KW_Machine ); }; 'include' => { token( KW_Include ); }; + 'import' => { token( KW_Import ); }; 'write' => { token( KW_Write ); fgoto write_statement; }; 'action' => { token( KW_Action ); }; 'alphtype' => { token( KW_AlphType ); }; - 'range' => { token( KW_Range ); }; + 'prepush' => { token( KW_PrePush ); }; + 'postpop' => { token( KW_PostPop ); }; + + # FIXME: Enable this post 5.17. + # 'range' => { token( KW_Range ); }; + 'getkey' => { token( KW_GetKey ); inlineBlockType = SemiTerminated; - fgoto inline_code; + if ( hostLang->lang == HostLang::Ruby ) + fcall inline_code_ruby; + else + fcall inline_code; }; 'access' => { token( KW_Access ); inlineBlockType = SemiTerminated; - fgoto inline_code; + if ( hostLang->lang == HostLang::Ruby ) + fcall inline_code_ruby; + else + fcall inline_code; }; 'variable' => { token( KW_Variable ); inlineBlockType = SemiTerminated; - fgoto inline_code; + if ( hostLang->lang == HostLang::Ruby ) + fcall inline_code_ruby; + else + fcall inline_code; }; 'when' => { token( KW_When ); }; + 'inwhen' => { token( KW_InWhen ); }; + 'outwhen' => { token( KW_OutWhen ); }; 'eof' => { token( KW_Eof ); }; 'err' => { token( KW_Err ); }; 'lerr' => { token( KW_Lerr ); }; 'to' => { token( KW_To ); }; 'from' => { token( KW_From ); }; + 'export' => { token( KW_Export ); }; # Identifiers. - ident => { token( TK_Word, tokstart, tokend ); } ; + ident => { token( TK_Word, ts, te ); } ; # Numbers - number => { token( TK_UInt, tokstart, tokend ); }; - hex_number => { token( TK_Hex, tokstart, tokend ); }; + number => { token( TK_UInt, ts, te ); }; + hex_number => { token( TK_Hex, ts, te ); }; # Literals, with optionals. ( s_literal | d_literal ) [i]? - => { token( TK_Literal, tokstart, tokend ); }; + => { token( TK_Literal, ts, te ); }; '[' => { token( RE_SqOpen ); fcall or_literal; }; '[^' => { token( RE_SqOpenNeg ); fcall or_literal; }; - '/' => { token( RE_Slash ); fgoto re_literal; }; + '/' => { token( RE_Slash ); fgoto ragel_re_literal; }; # Ignore. pound_comment => { updateCol(); }; @@ -749,71 +1069,96 @@ void Scanner::endSection( ) # Opening of longest match. "|*" => { token( TK_BarStar ); }; + # Separater for name references. + "::" => { token( TK_NameSep, ts, te ); }; + '}%%' => { - /* In order to generate anything we must be in the top level file - * and the current spec must be active and there must not have been - * any parse errors. */ updateCol(); endSection(); - fgoto main; + fret; }; - [ \t]+ => { updateCol(); }; + [ \t\r]+ => { updateCol(); }; # If we are in a single line machine then newline may end the spec. NL => { updateCol(); if ( singleLineSpec ) { - /* In order to generate anything we must be in the top level file - * and the current spec must be active and there must not have been - * any parse errors. */ endSection(); - fgoto main; + fret; } }; '{' => { - token( '{' ); - curly_count = 1; - inlineBlockType = CurlyDelimited; - fgoto inline_code; + if ( lastToken == KW_Export || lastToken == KW_Entry ) + token( '{' ); + else { + token( '{' ); + curly_count = 1; + inlineBlockType = CurlyDelimited; + if ( hostLang->lang == HostLang::Ruby ) + fcall inline_code_ruby; + else + fcall inline_code; + } + }; + + EOF => { + scan_error() << "unterminated ragel section" << endl; }; - any => { token( *tokstart ); } ; + any => { token( *ts ); } ; *|; - action pass { - updateCol(); + # Outside code scanner. These tokens get passed through. + main_ruby := |* + ident => { pass( IMP_Word, ts, te ); }; + number => { pass( IMP_UInt, ts, te ); }; + ruby_comment => { pass(); }; + ( s_literal | d_literal | host_re_literal ) + => { pass( IMP_Literal, ts, te ); }; - /* If no errors and we are at the bottom of the include stack (the - * source file listed on the command line) then write out the data. */ - if ( include_depth == 0 && machineSpec == 0 && machineName == 0 ) - xmlEscapeHost( *outStream, tokstart, tokend-tokstart ); - } + '%%{' => { + updateCol(); + singleLineSpec = false; + startSection(); + fcall parser_def; + }; + '%%' => { + updateCol(); + singleLineSpec = true; + startSection(); + fcall parser_def; + }; + whitespace+ => { pass(); }; + EOF; + any => { pass( *ts, 0, 0 ); }; + *|; # Outside code scanner. These tokens get passed through. main := |* - ident => pass; - number => pass; - c_cpp_comment => pass; - s_literal | d_literal => pass; + 'define' => { pass( IMP_Define, 0, 0 ); }; + ident => { pass( IMP_Word, ts, te ); }; + number => { pass( IMP_UInt, ts, te ); }; + c_cpp_comment => { pass(); }; + ( s_literal | d_literal ) => { pass( IMP_Literal, ts, te ); }; + '%%{' => { updateCol(); singleLineSpec = false; startSection(); - fgoto parser_def; + fcall parser_def; }; '%%' => { updateCol(); singleLineSpec = true; startSection(); - fgoto parser_def; + fcall parser_def; }; - whitespace+ => pass; + whitespace+ => { pass(); }; EOF; - any => pass; + any => { pass( *ts, 0, 0 ); }; *|; - }%% %% write data; @@ -822,16 +1167,30 @@ void Scanner::do_scan() { int bufsize = 8; char *buf = new char[bufsize]; - const char last_char = 0; int cs, act, have = 0; - int top, stack[1]; + int top; + + /* The stack is two deep, one level for going into ragel defs from the main + * machines which process outside code, and another for going into or literals + * from either a ragel spec, or a regular expression. */ + int stack[2]; int curly_count = 0; bool execute = true; bool singleLineSpec = false; - InlineBlockType inlineBlockType; + InlineBlockType inlineBlockType = CurlyDelimited; + /* Init the section parser and the character scanner. */ + init(); %% write init; + /* Set up the start state. FIXME: After 5.20 is released the nocs write + * init option should be used, the main machine eliminated and this statement moved + * above the write init. */ + if ( hostLang->lang == HostLang::Ruby ) + cs = rlscan_en_main_ruby; + else + cs = rlscan_en_main; + while ( execute ) { char *p = buf + have; int space = bufsize - have; @@ -840,16 +1199,15 @@ void Scanner::do_scan() /* We filled up the buffer trying to scan a token. Grow it. */ bufsize = bufsize * 2; char *newbuf = new char[bufsize]; - //cerr << "FULL BUFFER, NEW SIZE: " << bufsize << endl; /* Recompute p and space. */ p = newbuf + have; space = bufsize - have; /* Patch up pointers possibly in use. */ - if ( tokstart != 0 ) - tokstart = newbuf + ( tokstart - buf ); - tokend = newbuf + ( tokend - buf ); + if ( ts != 0 ) + ts = newbuf + ( ts - buf ); + te = newbuf + ( te - buf ); /* Copy the new buffer in. */ memcpy( newbuf, buf, have ); @@ -859,25 +1217,27 @@ void Scanner::do_scan() input.read( p, space ); int len = input.gcount(); + char *pe = p + len; - /* If we see eof then append the EOF char. */ + /* If we see eof then append the eof var. */ + char *eof = 0; if ( len == 0 ) { - p[0] = last_char, len = 1; + eof = pe; execute = false; } - char *pe = p + len; %% write exec; /* Check if we failed. */ if ( cs == rlscan_error ) { - /* Machine failed before finding a token. */ - //cerr << "PARSE ERROR" << endl; + /* Machine failed before finding a token. I'm not yet sure if this + * is reachable. */ + scan_error() << "scanner error" << endl; exit(1); } /* Decide if we need to preserve anything. */ - char *preserve = tokstart; + char *preserve = ts; /* Now set up the prefix. */ if ( preserve == 0 ) @@ -887,9 +1247,9 @@ void Scanner::do_scan() have = pe - preserve; memmove( buf, preserve, have ); unsigned int shiftback = preserve - buf; - if ( tokstart != 0 ) - tokstart -= shiftback; - tokend -= shiftback; + if ( ts != 0 ) + ts -= shiftback; + te -= shiftback; preserve = buf; } @@ -897,11 +1257,3 @@ void Scanner::do_scan() delete[] buf; } - -void scan( char *fileName, istream &input ) -{ - Scanner scanner( fileName, input, 0, 0, 0 ); - scanner.init(); - scanner.do_scan(); -} - diff --git a/ragel/rubycodegen.cpp b/ragel/rubycodegen.cpp new file mode 100644 index 0000000..5117823 --- /dev/null +++ b/ragel/rubycodegen.cpp @@ -0,0 +1,826 @@ +/* + * 2007 Victor Hugo Borja <vic@rubyforge.org> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <iomanip> +#include <sstream> +#include "redfsm.h" +#include "gendata.h" +#include "ragel.h" +#include "rubycodegen.h" +#include "pcheck.h" +#include "vector.h" +#include "version.h" +#include "common.h" + +#include "ragel.h" +#include "rubytable.h" +#include "rubyftable.h" +#include "rubyflat.h" +#include "rubyfflat.h" +#include "rbxgoto.h" + +using std::ostream; +using std::ostringstream; +using std::string; +using std::cerr; +using std::endl; +using std::istream; +using std::ifstream; +using std::ostream; +using std::ios; +using std::cin; +using std::cout; +using std::cerr; +using std::endl; + +/* Target ruby impl */ + +/* Target language and output style. */ +extern CodeStyle codeStyle; + +extern int numSplitPartitions; + +/* + * Callbacks invoked by the XML data parser. + */ + + +void rubyLineDirective( ostream &out, const char *fileName, int line ) +{ + /* Write a comment containing line info. */ + out << "# line " << line << " \""; + for ( const char *pc = fileName; *pc != 0; pc++ ) { + if ( *pc == '\\' ) + out << "\\\\"; + else + out << *pc; + } + out << "\"\n"; +} + +void RubyCodeGen::genLineDirective( ostream &out ) +{ + std::streambuf *sbuf = out.rdbuf(); + output_filter *filter = static_cast<output_filter*>(sbuf); + rubyLineDirective( out, filter->fileName, filter->line + 1 ); +} + +string RubyCodeGen::DATA_PREFIX() +{ + if ( !noPrefix ) + return FSM_NAME() + "_"; + return ""; +} + +std::ostream &RubyCodeGen::STATIC_VAR( string type, string name ) +{ + out << + "class << self\n" + " attr_accessor :" << name << "\n" + "end\n" + "self." << name; + return out; +} + + +std::ostream &RubyCodeGen::OPEN_ARRAY( string type, string name ) +{ + out << + "class << self\n" + " attr_accessor :" << name << "\n" + " private :" << name << ", :" << name << "=\n" + "end\n" + "self." << name << " = [\n"; + return out; +} + +std::ostream &RubyCodeGen::CLOSE_ARRAY() +{ + out << "]\n"; + return out; +} + + +string RubyCodeGen::ARR_OFF( string ptr, string offset ) +{ + return ptr + "[" + offset + "]"; +} + +string RubyCodeGen::NULL_ITEM() +{ + return "nil"; +} + + +string RubyCodeGen::P() +{ + ostringstream ret; + if ( pExpr == 0 ) + ret << "p"; + else { + //ret << "("; + INLINE_LIST( ret, pExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::PE() +{ + ostringstream ret; + if ( peExpr == 0 ) + ret << "pe"; + else { + //ret << "("; + INLINE_LIST( ret, peExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::vEOF() +{ + ostringstream ret; + if ( eofExpr == 0 ) + ret << "eof"; + else { + //ret << "("; + INLINE_LIST( ret, eofExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::vCS() +{ + ostringstream ret; + if ( csExpr == 0 ) + ret << ACCESS() << "cs"; + else { + //ret << "("; + INLINE_LIST( ret, csExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::TOP() +{ + ostringstream ret; + if ( topExpr == 0 ) + ret << ACCESS() + "top"; + else { + //ret << "("; + INLINE_LIST( ret, topExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::STACK() +{ + ostringstream ret; + if ( stackExpr == 0 ) + ret << ACCESS() + "stack"; + else { + //ret << "("; + INLINE_LIST( ret, stackExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::ACT() +{ + ostringstream ret; + if ( actExpr == 0 ) + ret << ACCESS() + "act"; + else { + //ret << "("; + INLINE_LIST( ret, actExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::TOKSTART() +{ + ostringstream ret; + if ( tokstartExpr == 0 ) + ret << ACCESS() + "ts"; + else { + //ret << "("; + INLINE_LIST( ret, tokstartExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::TOKEND() +{ + ostringstream ret; + if ( tokendExpr == 0 ) + ret << ACCESS() + "te"; + else { + //ret << "("; + INLINE_LIST( ret, tokendExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::DATA() +{ + ostringstream ret; + if ( dataExpr == 0 ) + ret << ACCESS() + "data"; + else { + //ret << "("; + INLINE_LIST( ret, dataExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +/* Write out the fsm name. */ +string RubyCodeGen::FSM_NAME() +{ + return fsmName; +} + + +void RubyCodeGen::ACTION( ostream &ret, GenAction *action, int targState, bool inFinish ) +{ + /* Write the preprocessor line info for going into the source file. */ + rubyLineDirective( ret, action->loc.fileName, action->loc.line ); + + /* Write the block and close it off. */ + ret << " begin\n"; + INLINE_LIST( ret, action->inlineList, targState, inFinish ); + ret << " end\n"; +} + + + +string RubyCodeGen::GET_WIDE_KEY() +{ + if ( redFsm->anyConditions() ) + return "_widec"; + else + return GET_KEY(); +} + +string RubyCodeGen::GET_WIDE_KEY( RedStateAp *state ) +{ + if ( state->stateCondList.length() > 0 ) + return "_widec"; + else + return GET_KEY(); +} + +string RubyCodeGen::GET_KEY() +{ + ostringstream ret; + if ( getKeyExpr != 0 ) { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, getKeyExpr, 0, false ); + ret << ")"; + } + else { + /* Expression for retrieving the key, use simple dereference. */ + ret << DATA() << "[" << P() << "]"; + } + return ret.str(); +} + +string RubyCodeGen::KEY( Key key ) +{ + ostringstream ret; + if ( keyOps->isSigned || !hostLang->explicitUnsigned ) + ret << key.getVal(); + else + ret << (unsigned long) key.getVal(); + return ret.str(); +} + + +/* Write out level number of tabs. Makes the nested binary search nice + * looking. */ +string RubyCodeGen::TABS( int level ) +{ + string result; + while ( level-- > 0 ) + result += "\t"; + return result; +} + +string RubyCodeGen::INT( int i ) +{ + ostringstream ret; + ret << i; + return ret.str(); +} + +void RubyCodeGen::CONDITION( ostream &ret, GenAction *condition ) +{ + ret << "\n"; + rubyLineDirective( ret, condition->loc.fileName, condition->loc.line ); + INLINE_LIST( ret, condition->inlineList, 0, false ); +} + +/* Emit the alphabet data type. */ +string RubyCodeGen::ALPH_TYPE() +{ + string ret = keyOps->alphType->data1; + if ( keyOps->alphType->data2 != 0 ) { + ret += " "; + ret += + keyOps->alphType->data2; + } + return ret; +} + +/* Emit the alphabet data type. */ +string RubyCodeGen::WIDE_ALPH_TYPE() +{ + string ret; + if ( redFsm->maxKey <= keyOps->maxKey ) + ret = ALPH_TYPE(); + else { + long long maxKeyVal = redFsm->maxKey.getLongLong(); + HostType *wideType = keyOps->typeSubsumes( keyOps->isSigned, maxKeyVal ); + assert( wideType != 0 ); + + ret = wideType->data1; + if ( wideType->data2 != 0 ) { + ret += " "; + ret += wideType->data2; + } + } + return ret; +} + + +string RubyCodeGen::ARRAY_TYPE( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + + string ret = arrayType->data1; + if ( arrayType->data2 != 0 ) { + ret += " "; + ret += arrayType->data2; + } + return ret; +} + +/* Write out the array of actions. */ +std::ostream &RubyCodeGen::ACTIONS_ARRAY() +{ + START_ARRAY_LINE(); + int totalActions = 0; + ARRAY_ITEM( INT(0), ++totalActions, false ); + for ( GenActionTableMap::Iter act = redFsm->actionMap; act.lte(); act++ ) { + /* Write out the length, which will never be the last character. */ + ARRAY_ITEM( INT(act->key.length()), ++totalActions, false ); + + for ( GenActionTable::Iter item = act->key; item.lte(); item++ ) { + ARRAY_ITEM( INT(item->value->actionId), ++totalActions, (act.last() && item.last()) ); + } + } + END_ARRAY_LINE(); + return out; +} + +void RubyCodeGen::STATE_IDS() +{ + if ( redFsm->startState != 0 ) + STATIC_VAR( "int", START() ) << " = " << START_STATE_ID() << ";\n"; + + if ( !noFinal ) + STATIC_VAR( "int" , FIRST_FINAL() ) << " = " << FIRST_FINAL_STATE() << ";\n"; + + if ( !noError ) + STATIC_VAR( "int", ERROR() ) << " = " << ERROR_STATE() << ";\n"; + + out << "\n"; + + if ( entryPointNames.length() > 0 ) { + for ( EntryNameVect::Iter en = entryPointNames; en.lte(); en++ ) { + STATIC_VAR( "int", DATA_PREFIX() + "en_" + *en ) << + " = " << entryPointIds[en.pos()] << ";\n"; + } + out << "\n"; + } +} + +std::ostream &RubyCodeGen::START_ARRAY_LINE() +{ + out << "\t"; + return out; +} + +std::ostream &RubyCodeGen::ARRAY_ITEM( string item, int count, bool last ) +{ + out << item; + if ( !last ) + { + out << ", "; + if ( count % IALL == 0 ) + { + END_ARRAY_LINE(); + START_ARRAY_LINE(); + } + } + return out; +} + +std::ostream &RubyCodeGen::END_ARRAY_LINE() +{ + out << "\n"; + return out; +} + +/* Emit the offset of the start state as a decimal integer. */ +string RubyCodeGen::START_STATE_ID() +{ + ostringstream ret; + ret << redFsm->startState->id; + return ret.str(); +}; + +string RubyCodeGen::ERROR_STATE() +{ + ostringstream ret; + if ( redFsm->errState != 0 ) + ret << redFsm->errState->id; + else + ret << "-1"; + return ret.str(); +} + +string RubyCodeGen::FIRST_FINAL_STATE() +{ + ostringstream ret; + if ( redFsm->firstFinState != 0 ) + ret << redFsm->firstFinState->id; + else + ret << redFsm->nextStateId; + return ret.str(); +} + +string RubyCodeGen::ACCESS() +{ + ostringstream ret; + if ( accessExpr != 0 ) + INLINE_LIST( ret, accessExpr, 0, false ); + return ret.str(); +} + +/* Write out an inline tree structure. Walks the list and possibly calls out + * to virtual functions than handle language specific items in the tree. */ +void RubyCodeGen::INLINE_LIST( ostream &ret, GenInlineList *inlineList, + int targState, bool inFinish ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Text: + ret << item->data; + break; + case GenInlineItem::Goto: + GOTO( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Call: + CALL( ret, item->targState->id, targState, inFinish ); + break; + case GenInlineItem::Next: + NEXT( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Ret: + RET( ret, inFinish ); + break; + case GenInlineItem::PChar: + ret << P(); + break; + case GenInlineItem::Char: + ret << GET_KEY(); + break; + case GenInlineItem::Hold: + ret << P() << " = " << P() << " - 1;"; + break; + case GenInlineItem::Exec: + EXEC( ret, item, targState, inFinish ); + break; + case GenInlineItem::Curs: + ret << "(_ps)"; + break; + case GenInlineItem::Targs: + ret << "(" << vCS() << ")"; + break; + case GenInlineItem::Entry: + ret << item->targState->id; + break; + case GenInlineItem::GotoExpr: + GOTO_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::CallExpr: + CALL_EXPR( ret, item, targState, inFinish ); + break; + case GenInlineItem::NextExpr: + NEXT_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::LmSwitch: + LM_SWITCH( ret, item, targState, inFinish ); + break; + case GenInlineItem::LmSetActId: + SET_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokEnd: + SET_TOKEND( ret, item ); + break; + case GenInlineItem::LmGetTokEnd: + GET_TOKEND( ret, item ); + break; + case GenInlineItem::LmInitTokStart: + INIT_TOKSTART( ret, item ); + break; + case GenInlineItem::LmInitAct: + INIT_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokStart: + SET_TOKSTART( ret, item ); + break; + case GenInlineItem::SubAction: + SUB_ACTION( ret, item, targState, inFinish ); + break; + case GenInlineItem::Break: + BREAK( ret, targState ); + break; + } + } +} + + +void RubyCodeGen::EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ) +{ + /* The parser gives fexec two children. The double brackets are for D + * code. If the inline list is a single word it will get interpreted as a + * C-style cast by the D compiler. */ + ret << " begin " << P() << " = (("; + INLINE_LIST( ret, item->children, targState, inFinish ); + ret << "))-1; end\n"; +} + +void RubyCodeGen::LM_SWITCH( ostream &ret, GenInlineItem *item, + int targState, int inFinish ) +{ + ret << + " case " << ACT() << "\n"; + + for ( GenInlineList::Iter lma = *item->children; lma.lte(); lma++ ) { + /* Write the case label, the action and the case break. */ + if ( lma->lmId < 0 ) + ret << " else\n"; + else + ret << " when " << lma->lmId << " then\n"; + + + /* Write the block and close it off. */ + ret << " begin"; + INLINE_LIST( ret, lma->children, targState, inFinish ); + ret << "end\n"; + } + + ret << "end \n\t"; +} + +void RubyCodeGen::SET_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = " << item->lmId << ";"; +} + +void RubyCodeGen::INIT_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << NULL_ITEM() << ";"; +} + +void RubyCodeGen::INIT_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = 0\n"; +} + +void RubyCodeGen::SET_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << P() << "\n"; +} + +void RubyCodeGen::SET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + /* The tokend action sets tokend. */ + ret << TOKEND() << " = " << P(); + if ( item->offset != 0 ) + out << "+" << item->offset; + out << "\n"; +} + +void RubyCodeGen::GET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + ret << TOKEND(); +} + +void RubyCodeGen::SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish ) +{ + if ( item->children->length() > 0 ) { + /* Write the block and close it off. */ + ret << " begin "; + INLINE_LIST( ret, item->children, targState, inFinish ); + ret << " end\n"; + } +} + +int RubyCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + return act; +} + +ostream &RubyCodeGen::source_warning( const InputLoc &loc ) +{ + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": warning: "; + return cerr; +} + +ostream &RubyCodeGen::source_error( const InputLoc &loc ) +{ + gblErrorCount += 1; + assert( sourceFileName != 0 ); + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": "; + return cerr; +} + +void RubyCodeGen::finishRagelDef() +{ + if ( codeStyle == GenGoto || codeStyle == GenFGoto || + codeStyle == GenIpGoto || codeStyle == GenSplit ) + { + /* For directly executable machines there is no required state + * ordering. Choose a depth-first ordering to increase the + * potential for fall-throughs. */ + redFsm->depthFirstOrdering(); + } + else { + /* The frontend will do this for us, but it may be a good idea to + * force it if the intermediate file is edited. */ + redFsm->sortByStateId(); + } + + /* Choose default transitions and the single transition. */ + redFsm->chooseDefaultSpan(); + + /* Maybe do flat expand, otherwise choose single. */ + if ( codeStyle == GenFlat || codeStyle == GenFFlat ) + redFsm->makeFlat(); + else + redFsm->chooseSingle(); + + /* If any errors have occured in the input file then don't write anything. */ + if ( gblErrorCount > 0 ) + return; + + if ( codeStyle == GenSplit ) + redFsm->partitionFsm( numSplitPartitions ); + + if ( codeStyle == GenIpGoto || codeStyle == GenSplit ) + redFsm->setInTrans(); + + /* Anlayze Machine will find the final action reference counts, among + * other things. We will use these in reporting the usage + * of fsm directives in action code. */ + analyzeMachine(); + + /* Determine if we should use indicies. */ + calcIndexSize(); +} + + +/* Determine if we should use indicies or not. */ +void RubyCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActionLoc) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActionLoc) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +unsigned int RubyCodeGen::arrayTypeSize( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + return arrayType->size; +} + + +void RubyCodeGen::writeInit() +{ + out << "begin\n"; + + out << " " << P() << " ||= 0\n"; + + if ( !noEnd ) + out << " " << PE() << " ||= " << DATA() << ".length\n"; + + if ( !noCS ) + out << " " << vCS() << " = " << START() << "\n"; + + /* If there are any calls, then the stack top needs initialization. */ + if ( redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << " " << TOP() << " = 0\n"; + + if ( hasLongestMatch ) { + out << + " " << TOKSTART() << " = " << NULL_ITEM() << "\n" + " " << TOKEND() << " = " << NULL_ITEM() << "\n" + " " << ACT() << " = 0\n"; + } + + out << "end\n"; +} + +void RubyCodeGen::writeExports() +{ + if ( exportList.length() > 0 ) { + for ( ExportList::Iter ex = exportList; ex.lte(); ex++ ) { + STATIC_VAR( ALPH_TYPE(), DATA_PREFIX() + "ex_" + ex->name ) + << " = " << KEY(ex->key) << "\n"; + } + out << "\n"; + } +} + +void RubyCodeGen::writeStart() +{ + out << START_STATE_ID(); +} + +void RubyCodeGen::writeFirstFinal() +{ + out << FIRST_FINAL_STATE(); +} + +void RubyCodeGen::writeError() +{ + out << ERROR_STATE(); +} + + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ diff --git a/ragel/rubycodegen.h b/ragel/rubycodegen.h new file mode 100644 index 0000000..4e3a30d --- /dev/null +++ b/ragel/rubycodegen.h @@ -0,0 +1,174 @@ +/* + * 2007 Victor Hugo Borja <vic@rubyforge.org> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RUBY_CODEGEN_H +#define _RUBY_CODEGEN_H + +#include "common.h" +#include "gendata.h" + +/* Integer array line length. */ +#define IALL 8 + + +class RubyCodeGen : public CodeGenData +{ +public: + RubyCodeGen( ostream &out ) : CodeGenData(out) { } + virtual ~RubyCodeGen() {} +protected: + ostream &START_ARRAY_LINE(); + ostream &ARRAY_ITEM( string item, int count, bool last ); + ostream &END_ARRAY_LINE(); + + + string FSM_NAME(); + + string START_STATE_ID(); + string ERROR_STATE(); + string FIRST_FINAL_STATE(); + void INLINE_LIST(ostream &ret, GenInlineList *inlineList, int targState, bool inFinish); + string ACCESS(); + + void ACTION( ostream &ret, GenAction *action, int targState, bool inFinish ); + string GET_KEY(); + string GET_WIDE_KEY(); + string GET_WIDE_KEY( RedStateAp *state ); + string KEY( Key key ); + string TABS( int level ); + string INT( int i ); + void CONDITION( ostream &ret, GenAction *condition ); + string ALPH_TYPE(); + string WIDE_ALPH_TYPE(); + string ARRAY_TYPE( unsigned long maxVal ); + ostream &ACTIONS_ARRAY(); + void STATE_IDS(); + + + string DATA_PREFIX(); + string PM() { return "_" + DATA_PREFIX() + "partition_map"; } + string C() { return "_" + DATA_PREFIX() + "cond_spaces"; } + string CK() { return "_" + DATA_PREFIX() + "cond_keys"; } + string K() { return "_" + DATA_PREFIX() + "trans_keys"; } + string I() { return "_" + DATA_PREFIX() + "indicies"; } + string CO() { return "_" + DATA_PREFIX() + "cond_offsets"; } + string KO() { return "_" + DATA_PREFIX() + "key_offsets"; } + string IO() { return "_" + DATA_PREFIX() + "index_offsets"; } + string CL() { return "_" + DATA_PREFIX() + "cond_lengths"; } + string SL() { return "_" + DATA_PREFIX() + "single_lengths"; } + string RL() { return "_" + DATA_PREFIX() + "range_lengths"; } + string A() { return "_" + DATA_PREFIX() + "actions"; } + string TA() { return "_" + DATA_PREFIX() + "trans_actions"; } + string TT() { return "_" + DATA_PREFIX() + "trans_targs"; } + string TSA() { return "_" + DATA_PREFIX() + "to_state_actions"; } + string FSA() { return "_" + DATA_PREFIX() + "from_state_actions"; } + string EA() { return "_" + DATA_PREFIX() + "eof_actions"; } + string ET() { return "_" + DATA_PREFIX() + "eof_trans"; } + string SP() { return "_" + DATA_PREFIX() + "key_spans"; } + string CSP() { return "_" + DATA_PREFIX() + "cond_key_spans"; } + string START() { return DATA_PREFIX() + "start"; } + string ERROR() { return DATA_PREFIX() + "error"; } + string FIRST_FINAL() { return DATA_PREFIX() + "first_final"; } + string CTXDATA() { return DATA_PREFIX() + "ctxdata"; } + +public: + string NULL_ITEM(); + ostream &OPEN_ARRAY( string type, string name ); + ostream &CLOSE_ARRAY(); + ostream &STATIC_VAR( string type, string name ); + string ARR_OFF( string ptr, string offset ); + + string P(); + string PE(); + string vEOF(); + + string vCS(); + string TOP(); + string STACK(); + string ACT(); + string TOKSTART(); + string TOKEND(); + string DATA(); + + + void finishRagelDef(); + unsigned int arrayTypeSize( unsigned long maxVal ); + +protected: + virtual void writeExports(); + virtual void writeInit(); + virtual void writeStart(); + virtual void writeFirstFinal(); + virtual void writeError(); + + /* Determine if we should use indicies. */ + virtual void calcIndexSize(); + + virtual void BREAK( ostream &ret, int targState ) = 0; + virtual void GOTO( ostream &ret, int gotoDest, bool inFinish ) = 0; + virtual void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) = 0; + virtual void CALL( ostream &ret, int callDest, int targState, bool inFinish ) = 0; + virtual void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) = 0; + virtual void RET( ostream &ret, bool inFinish ) = 0; + + + virtual void NEXT( ostream &ret, int nextDest, bool inFinish ) = 0; + virtual void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) = 0; + + virtual int TO_STATE_ACTION( RedStateAp *state ) = 0; + virtual int FROM_STATE_ACTION( RedStateAp *state ) = 0; + virtual int EOF_ACTION( RedStateAp *state ) = 0; + + virtual int TRANS_ACTION( RedTransAp *trans ); + + void EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void LM_SWITCH( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void SET_ACT( ostream &ret, GenInlineItem *item ); + void INIT_TOKSTART( ostream &ret, GenInlineItem *item ); + void INIT_ACT( ostream &ret, GenInlineItem *item ); + void SET_TOKSTART( ostream &ret, GenInlineItem *item ); + void SET_TOKEND( ostream &ret, GenInlineItem *item ); + void GET_TOKEND( ostream &ret, GenInlineItem *item ); + void SUB_ACTION( ostream &ret, GenInlineItem *item, int targState, bool inFinish ); + +protected: + ostream &source_warning(const InputLoc &loc); + ostream &source_error(const InputLoc &loc); + + + /* fields */ + bool outLabelUsed; + bool againLabelUsed; + bool useIndicies; + + void genLineDirective( ostream &out ); +}; + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ + +#endif diff --git a/ragel/rubyfflat.cpp b/ragel/rubyfflat.cpp new file mode 100644 index 0000000..53c7896 --- /dev/null +++ b/ragel/rubyfflat.cpp @@ -0,0 +1,488 @@ +/* + * 2007 Victor Hugo Borja <vic@rubyforge.org> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "rubyfflat.h" + +void RubyFFlatCodeGen::GOTO( ostream &out, int gotoDest, bool inFinish ) +{ + out << + " begin\n" + " " << vCS() << " = " << gotoDest << "\n" + " _goto_level = _again\n" + " next\n" + " end\n"; +} + +void RubyFFlatCodeGen::GOTO_EXPR( ostream &out, GenInlineItem *ilItem, bool inFinish ) +{ + out << + " begin\n" + " " << vCS() << " = ("; + INLINE_LIST( out, ilItem->children, 0, inFinish ); + out << ")\n"; + out << + " _goto_level = _again\n" + " next\n" + " end\n"; +} + +void RubyFFlatCodeGen::CALL( ostream &out, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, prePushExpr, 0, false ); + } + + out << + " begin\n" + " " << STACK() << "[" << TOP() << "] = " << vCS() << "\n" + " " << TOP() << "+= 1\n" + " " << vCS() << " = " << callDest << "\n" + " _goto_level = _again\n" + " next\n" + " end\n"; + + if ( prePushExpr != 0 ) + out << "end\n"; +} + +void RubyFFlatCodeGen::CALL_EXPR(ostream &out, GenInlineItem *ilItem, + int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, prePushExpr, 0, false ); + } + + out << + " begin\n" + " " << STACK() << "[" << TOP() << "] = " << vCS() << "\n" + " " << TOP() << " += 1\n" + " " << vCS() << " = ("; + INLINE_LIST( out, ilItem->children, targState, inFinish ); + out << ")\n"; + + out << + " _goto_level = _again\n" + " next\n" + " end\n"; + + if ( prePushExpr != 0 ) + out << "end\n"; +} + +void RubyFFlatCodeGen::RET( ostream &out, bool inFinish ) +{ + out << + " begin\n" + " " << TOP() << " -= 1\n" + " " << vCS() << " = " << STACK() << "[" << TOP() << "]\n"; + + if ( postPopExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, postPopExpr, 0, false ); + out << "end\n"; + } + + out << + " _goto_level = _again\n" + " next\n" + " end\n"; +} + +void RubyFFlatCodeGen::BREAK( ostream &out, int targState ) +{ + out << + " begin\n" + " " << P() << " += 1\n" + " _goto_level = _out\n" + " next\n" + " end\n"; +} + + +int RubyFFlatCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + return act; +} + +int RubyFFlatCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + return act; +} + +int RubyFFlatCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + return act; +} + +/* Write out the function for a transition. */ +int RubyFFlatCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + int action = 0; + if ( trans->action != 0 ) + action = trans->action->actListId+1; + return action; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &RubyFFlatCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\twhen " << redAct->actListId+1 << " then\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &RubyFFlatCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\twhen " << redAct->actListId+1 << " then\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RubyFFlatCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\twhen " << redAct->actListId+1 << " then\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true ); + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &RubyFFlatCodeGen::ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* Write the entry label. */ + out << "\twhen " << redAct->actListId+1 << " then\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + } + } + + genLineDirective( out ); + return out; +} + +void RubyFFlatCodeGen::writeData() +{ + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpan), CSP() ); + COND_KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCond), C() ); + CONDS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondIndexOffset), CO() ); + COND_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSpan), SP() ); + KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxFlatIndexOffset), IO() ); + FLAT_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void RubyFFlatCodeGen::writeExec() +{ + out << + "begin\n" + " testEof = false\n" + " _slen, _trans, _keys, _inds"; + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + if ( redFsm->anyConditions() ) + out << ", _cond, _conds, _widec"; + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + out << ", _acts, _nacts"; + + out << " = nil\n"; + + out << + " _goto_level = 0\n" + " _resume = 10\n" + " _eof_trans = 15\n" + " _again = 20\n" + " _test_eof = 30\n" + " _out = 40\n"; + + out << + " while true\n" + " if _goto_level <= 0\n"; + + if ( !noEnd ) { + out << + " if " << P() << " == " << PE() << "\n" + " _goto_level = _test_eof\n" + " next\n" + " end\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if " << vCS() << " == " << redFsm->errState->id << "\n" + " _goto_level = _out\n" + " next\n" + " end\n"; + } + + /* The resume label. */ + out << + " end\n" + " if _goto_level <= _resume\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " case " << FSA() << "[" << vCS() << "] \n"; + FROM_STATE_ACTION_SWITCH() << + " end\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( redFsm->anyEofTrans() ) { + out << + " end\n" + " if _goto_level <= _eof_trans\n"; + } + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << "\n"; + + out << " " << vCS() << " = " << TT() << "[_trans]\n"; + + if ( redFsm->anyRegActions() ) { + /* break _again */ + out << + " if " << TA() << "[_trans] != 0\n" + " case " << TA() << "[_trans]" << "\n"; + ACTION_SWITCH() << + " end\n" + " end\n"; + } + + /* The again label. */ + out << + " end\n" + " if _goto_level <= _again\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " case " << TSA() << "[" << vCS() << "] \n"; + TO_STATE_ACTION_SWITCH() << + " end\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if " << vCS() << " == " << redFsm->errState->id << "\n" + " _goto_level = _out\n" + " next\n" + " end\n"; + } + + out << " " << P() << " += 1\n"; + + if ( !noEnd ) { + out << + " if " << P() << " != " << PE() << "\n" + " _goto_level = _resume\n" + " next\n" + " end\n"; + } + else { + out << + " _goto_level = _resume\n" + " next\n"; + } + + /* The test eof label. */ + out << + " end\n" + " if _goto_level <= _test_eof\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " == " << vEOF() << "\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if " << ET() << "[" << vCS() << "] > 0\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " _goto_level = _eof_trans\n" + " next;\n" + " end\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " case " << EA() << "[" << vCS() << "]\n"; + EOF_ACTION_SWITCH() << + " end\n"; + } + + out << + " end\n" + "\n"; + } + + out << + " end\n" + " if _goto_level <= _out\n" + " break\n" + " end\n" + "end\n"; + + /* Wrapping the execute block. */ + out << " end\n"; +} + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ + diff --git a/ragel/rubyfflat.h b/ragel/rubyfflat.h new file mode 100644 index 0000000..4ac412f --- /dev/null +++ b/ragel/rubyfflat.h @@ -0,0 +1,65 @@ +/* + * 2007 Victor Hugo Borja <vic@rubyforge.org> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RUBY_FFLATCODEGEN_H +#define _RUBY_FFLATCODEGEN_H + +#include <iostream> +#include "rubyflat.h" + +class RubyFFlatCodeGen : public RubyFlatCodeGen +{ +public: + RubyFFlatCodeGen( ostream &out ) : + RubyFlatCodeGen(out) {} +protected: + + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + void GOTO( ostream &out, int gotoDest, bool inFinish ); + void GOTO_EXPR( ostream &out, GenInlineItem *ilItem, bool inFinish ); + void CALL( ostream &out, int callDest, int targState, bool inFinish ); + void CALL_EXPR(ostream &out, GenInlineItem *ilItem, int targState, bool inFinish ); + void RET( ostream &out, bool inFinish ); + void BREAK( ostream &out, int targState ); + + virtual int TO_STATE_ACTION( RedStateAp *state ); + virtual int FROM_STATE_ACTION( RedStateAp *state ); + virtual int EOF_ACTION( RedStateAp *state ); + virtual int TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); +}; + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ + +#endif diff --git a/ragel/rubyflat.cpp b/ragel/rubyflat.cpp new file mode 100644 index 0000000..6cf7e45 --- /dev/null +++ b/ragel/rubyflat.cpp @@ -0,0 +1,871 @@ +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + * Copyright 2007 Victor Hugo Borja <vic@rubyforge.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "rubyflat.h" +#include "ragel.h" +#include "redfsm.h" +#include "gendata.h" + +using std::ostream; +using std::string; + +std::ostream &RubyFlatCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\twhen " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RubyFlatCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\twhen " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RubyFlatCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\twhen " << act->actionId << " then\n"; + ACTION( out, act, 0, true ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RubyFlatCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\twhen " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &RubyFlatCodeGen::KEYS() +{ + START_ARRAY_LINE(); + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit just low key and high key. */ + ARRAY_ITEM( KEY( st->lowKey ), ++totalTrans, false ); + ARRAY_ITEM( KEY( st->highKey ), ++totalTrans, false ); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT( 0 ), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::INDICIES() +{ + int totalTrans = 0; + START_ARRAY_LINE(); + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->transList != 0 ) { + /* Walk the singles. */ + unsigned long long span = keyOps->span( st->lowKey, st->highKey ); + for ( unsigned long long pos = 0; pos < span; pos++ ) { + ARRAY_ITEM( KEY( st->transList[pos]->id ), ++totalTrans, false ); + } + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) + ARRAY_ITEM( KEY( st->defTrans->id ), ++totalTrans, false ); + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT( 0 ), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::FLAT_INDEX_OFFSET() +{ + START_ARRAY_LINE(); + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + ARRAY_ITEM( INT( curIndOffset ), ++totalStateNum, st.last() ); + /* Move the index offset ahead. */ + if ( st->transList != 0 ) + curIndOffset += keyOps->span( st->lowKey, st->highKey ); + + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::KEY_SPANS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + unsigned long long span = 0; + if ( st->transList != 0 ) + span = keyOps->span( st->lowKey, st->highKey ); + ARRAY_ITEM( INT( span ), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::TO_STATE_ACTIONS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT( TO_STATE_ACTION(st) ), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::FROM_STATE_ACTIONS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT( FROM_STATE_ACTION(st) ), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::EOF_ACTIONS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT( EOF_ACTION(st) ), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::EOF_TRANS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + + /* Write any eof action. */ + ARRAY_ITEM( INT(trans), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::TRANS_TARGS() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + START_ARRAY_LINE(); + + int totalStates = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Save the position. Needed for eofTargs. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + ARRAY_ITEM( INT( trans->targ->id ), ++totalStates, t >= redFsm->transSet.length()-1 ); + } + END_ARRAY_LINE(); + delete[] transPtrs; + return out; +} + + +std::ostream &RubyFlatCodeGen::TRANS_ACTIONS() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + START_ARRAY_LINE(); + int totalAct = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + ARRAY_ITEM( INT( TRANS_ACTION( trans ) ), ++totalAct, t >= redFsm->transSet.length()-1 ); + } + END_ARRAY_LINE(); + delete[] transPtrs; + return out; +} + + +void RubyFlatCodeGen::LOCATE_TRANS() +{ + out << + " _keys = " << vCS() << " << 1\n" + " _inds = " << IO() << "[" << vCS() << "]\n" + " _slen = " << SP() << "[" << vCS() << "]\n" + " _trans = if ( _slen > 0 && \n" + " " << K() << "[_keys] <= " << GET_WIDE_KEY() << " && \n" + " " << GET_WIDE_KEY() << " <= " << K() << "[_keys + 1] \n" + " ) then\n" + " " << I() << "[ _inds + " << GET_WIDE_KEY() << " - " << K() << "[_keys] ] \n" + " else \n" + " " << I() << "[ _inds + _slen ]\n" + " end\n" + ""; + +} + +std::ostream &RubyFlatCodeGen::COND_INDEX_OFFSET() +{ + START_ARRAY_LINE(); + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + ARRAY_ITEM( INT( curIndOffset ), ++totalStateNum, st.last() ); + /* Move the index offset ahead. */ + if ( st->condList != 0 ) + curIndOffset += keyOps->span( st->condLowKey, st->condHighKey ); + } + END_ARRAY_LINE(); + return out; +} + +void RubyFlatCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << GET_KEY() << "\n" + " _keys = " << vCS() << " << 1\n" + " _conds = " << CO() << "[" << vCS() << "]\n" + " _slen = " << CSP() << "[" << vCS() << "]\n" + " _cond = if ( _slen > 0 && \n" + " " << CK() << "[_keys] <= " << GET_WIDE_KEY() << " &&\n" + " " << GET_WIDE_KEY() << " <= " << CK() << "[_keys + 1]\n" + " ) then \n" + " " << C() << "[ _conds + " << GET_WIDE_KEY() << " - " << CK() << "[_keys]" << " ]\n" + " else\n" + " 0\n" + " end\n"; + out << + " case _cond \n"; + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " when " << condSpace->condSpaceId + 1 << " then\n"; + out << TABS(2) << "_widec = " << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "))\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(2) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << + " ) then \n" << + TABS(3) << " _widec += " << condValOffset << "\n" + "end\n"; + } + } + + out << + " end # _cond switch \n"; +} + +std::ostream &RubyFlatCodeGen::CONDS() +{ + int totalTrans = 0; + START_ARRAY_LINE(); + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->condList != 0 ) { + /* Walk the singles. */ + unsigned long long span = keyOps->span( st->condLowKey, st->condHighKey ); + for ( unsigned long long pos = 0; pos < span; pos++ ) { + if ( st->condList[pos] != 0 ) + ARRAY_ITEM( INT( st->condList[pos]->condSpaceId + 1 ), ++totalTrans, false ); + else + ARRAY_ITEM( INT( 0 ), ++totalTrans, false ); + } + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT( 0 ), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::COND_KEYS() +{ + START_ARRAY_LINE(); + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit just cond low key and cond high key. */ + ARRAY_ITEM( KEY( st->condLowKey ), ++totalTrans, false ); + ARRAY_ITEM( KEY( st->condHighKey ), ++totalTrans, false ); + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT( 0 ), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::COND_KEY_SPANS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + unsigned long long span = 0; + if ( st->condList != 0 ) + span = keyOps->span( st->condLowKey, st->condHighKey ); + ARRAY_ITEM( INT( span ), ++totalStateNum, false ); + } + END_ARRAY_LINE(); + return out; +} + + +void RubyFlatCodeGen::GOTO( ostream &out, int gotoDest, bool inFinish ) +{ + out << + " begin\n" + " " << vCS() << " = " << gotoDest << "\n" + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; +} + +void RubyFlatCodeGen::CALL( ostream &out, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, prePushExpr, 0, false ); + } + + out << + " begin\n" + " " << STACK() << "[" << TOP() << "] = " << vCS() << "\n" + " " << TOP() << "+= 1\n" + " " << vCS() << " = " << callDest << "\n" + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; + + if ( prePushExpr != 0 ) + out << "end\n"; +} + +void RubyFlatCodeGen::CALL_EXPR(ostream &out, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, prePushExpr, 0, false ); + } + + out << + " begin\n" + " " << STACK() << "[" << TOP() << "] = " << vCS() << "\n" + " " << TOP() << " += 1\n" + " " << vCS() << " = ("; + INLINE_LIST( out, ilItem->children, targState, inFinish ); + out << ")\n"; + + out << + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; + + if ( prePushExpr != 0 ) + out << "end\n"; +} + +void RubyFlatCodeGen::RET( ostream &out, bool inFinish ) +{ + out << + " begin\n" + " " << TOP() << " -= 1\n" + " " << vCS() << " = " << STACK() << "[" << TOP() << "]\n"; + + if ( postPopExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, postPopExpr, 0, false ); + out << "end\n"; + } + + out << + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; +} + +void RubyFlatCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void RubyFlatCodeGen::GOTO_EXPR( ostream &out, GenInlineItem *ilItem, bool inFinish ) +{ + out << + " begin\n" + " " << vCS() << " = ("; + INLINE_LIST( out, ilItem->children, 0, inFinish ); + out << ")\n"; + out << + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; +} + +void RubyFlatCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + + +void RubyFlatCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void RubyFlatCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void RubyFlatCodeGen::BREAK( ostream &out, int targState ) +{ + out << + " begin\n" + " " << P() << " += 1\n" + " _trigger_goto = true\n" + " _goto_level = _out\n" + " break\n" + " end\n"; +} + +int RubyFlatCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + return act; +} + +int RubyFlatCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + return act; +} + +int RubyFlatCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + return act; +} + +int RubyFlatCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + return act; +} + +void RubyFlatCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpan), CSP() ); + COND_KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCond), C() ); + CONDS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondIndexOffset), CO() ); + COND_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSpan), SP() ); + KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxFlatIndexOffset), IO() ); + FLAT_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void RubyFlatCodeGen::writeExec() +{ + out << + "begin # ragel flat\n" + " testEof = false\n" + " _slen, _trans, _keys, _inds"; + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + if ( redFsm->anyConditions() ) + out << ", _cond, _conds, _widec"; + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + out << ", _acts, _nacts"; + + out << " = nil\n"; + + out << + " _goto_level = 0\n" + " _resume = 10\n" + " _eof_trans = 15\n" + " _again = 20\n" + " _test_eof = 30\n" + " _out = 40\n"; + + out << + " while true\n" + " _trigger_goto = false\n" + " if _goto_level <= 0\n"; + + if ( !noEnd ) { + out << + " if " << P() << " == " << PE() << "\n" + " _goto_level = _test_eof\n" + " next\n" + " end\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if " << vCS() << " == " << redFsm->errState->id << "\n" + " _goto_level = _out\n" + " next\n" + " end\n"; + } + + /* The resume label. */ + out << + " end\n" + " if _goto_level <= _resume\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << FSA() << "[" << vCS() << "]\n" + " _nacts = " << A() << "[_acts]\n" + " _acts += 1\n" + " while _nacts > 0\n" + " _nacts -= 1\n" + " _acts += 1\n" + " case " << A() << "[_acts - 1]\n"; + FROM_STATE_ACTION_SWITCH(); + out << + " end # from state action switch\n" + " end\n" + " if _trigger_goto\n" + " next\n" + " end\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( redFsm->anyEofTrans() ) { + out << + " end\n" + " if _goto_level <= _eof_trans\n"; + } + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << "\n"; + + out << " " << vCS() << " = " << TT() << "[_trans]\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if " << TA() << "[_trans] != 0\n" + " _acts = " << TA() << "[_trans]\n" + " _nacts = " << A() << "[_acts]\n" + " _acts += 1\n" + " while _nacts > 0\n" + " _nacts -= 1\n" + " _acts += 1\n" + " case " << A() << "[_acts - 1]\n"; + ACTION_SWITCH(); + out << + " end # action switch\n" + " end\n" + " end\n" + " if _trigger_goto\n" + " next\n" + " end\n"; + } + + /* The again label. */ + out << + " end\n" + " if _goto_level <= _again\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << TSA() << "[" << vCS() << "]\n" + " _nacts = " << A() << "[_acts]\n" + " _acts += 1\n" + " while _nacts > 0\n" + " _nacts -= 1\n" + " _acts += 1\n" + " case " << A() << "[_acts - 1]\n"; + TO_STATE_ACTION_SWITCH() << + " end # to state action switch\n" + " end\n" + " if _trigger_goto\n" + " next\n" + " end\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if " << vCS() << " == " << redFsm->errState->id << "\n" + " _goto_level = _out\n" + " next\n" + " end\n"; + } + + out << " " << P() << " += 1\n"; + + if ( !noEnd ) { + out << + " if " << P() << " != " << PE() << "\n" + " _goto_level = _resume\n" + " next\n" + " end\n"; + } + else { + out << + " _goto_level = _resume\n" + " next\n"; + } + + /* The test_eof label. */ + out << + " end\n" + " if _goto_level <= _test_eof\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " == " << vEOF() << "\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if " << ET() << "[" << vCS() << "] > 0\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " _goto_level = _eof_trans\n" + " next;\n" + " end\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " begin\n" + " __acts = " << EA() << "[" << vCS() << "]\n" + " __nacts = " << A() << "[__acts]\n" << + " __acts += 1\n" + " while ( __nacts > 0 ) \n" + " __nacts -= 1\n" + " __acts += 1\n" + " case ( "<< A() << "[__acts-1] ) \n"; + EOF_ACTION_SWITCH() << + " end\n" + " end\n" + " if _trigger_goto\n" + " next\n" + " end\n" + " end\n"; + } + + out << + " end\n"; + } + + out << + " end\n" + " if _goto_level <= _out\n" + " break\n" + " end\n"; + + /* The loop for faking goto. */ + out << + " end\n"; + + /* Wrapping the execute block. */ + out << + " end\n"; +} + + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ diff --git a/ragel/rubyflat.h b/ragel/rubyflat.h new file mode 100644 index 0000000..5136791 --- /dev/null +++ b/ragel/rubyflat.h @@ -0,0 +1,99 @@ +/* + * 2007 Victor Hugo Borja <vic@rubyforge.org> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RUBY_FLATCODEGEN_H +#define _RUBY_FLATCODEGEN_H + +#include <iostream> +#include "rubycodegen.h" + +using std::string; +using std::ostream; + + +/* + * FlatCodeGen + */ +class RubyFlatCodeGen : public RubyCodeGen +{ +public: + RubyFlatCodeGen( ostream &out ) : + RubyCodeGen(out) {}; + virtual ~RubyFlatCodeGen() {} +protected: + + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &FLAT_INDEX_OFFSET(); + std::ostream &KEY_SPANS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + void LOCATE_TRANS(); + + std::ostream &COND_INDEX_OFFSET(); + void COND_TRANSLATE(); + std::ostream &CONDS(); + std::ostream &COND_KEYS(); + std::ostream &COND_KEY_SPANS(); + + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState ); + + + virtual int TO_STATE_ACTION( RedStateAp *state ); + virtual int FROM_STATE_ACTION( RedStateAp *state ); + virtual int EOF_ACTION( RedStateAp *state ); + virtual int TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); + +}; + +#endif + + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ diff --git a/ragel/rubyftable.cpp b/ragel/rubyftable.cpp new file mode 100644 index 0000000..b90ffa7 --- /dev/null +++ b/ragel/rubyftable.cpp @@ -0,0 +1,563 @@ +/* + * 2007 Victor Hugo Borja <vic@rubyforge.org> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <iomanip> +#include <sstream> +#include "redfsm.h" +#include "gendata.h" +#include "ragel.h" +#include "rubyftable.h" + +using std::ostream; +using std::ostringstream; +using std::string; +using std::cerr; +using std::endl; + +void RubyFTabCodeGen::GOTO( ostream &out, int gotoDest, bool inFinish ) +{ + out << + " begin\n" + " " << vCS() << " = " << gotoDest << "\n" + " _goto_level = _again\n" + " next\n" + " end\n"; +} + +void RubyFTabCodeGen::GOTO_EXPR( ostream &out, GenInlineItem *ilItem, bool inFinish ) +{ + out << + " begin\n" + " " << vCS() << " = ("; + INLINE_LIST( out, ilItem->children, 0, inFinish ); + out << ")\n"; + out << + " _goto_level = _again\n" + " next\n" + " end\n"; +} + +void RubyFTabCodeGen::CALL( ostream &out, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, prePushExpr, 0, false ); + } + + out << + " begin\n" + " " << STACK() << "[" << TOP() << "] = " << vCS() << "\n" + " " << TOP() << "+= 1\n" + " " << vCS() << " = " << callDest << "\n" + " _goto_level = _again\n" + " next\n" + " end\n"; + + if ( prePushExpr != 0 ) + out << "end\n"; +} + +void RubyFTabCodeGen::CALL_EXPR(ostream &out, GenInlineItem *ilItem, + int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, prePushExpr, 0, false ); + } + + out << + " begin\n" + " " << STACK() << "[" << TOP() << "] = " << vCS() << "\n" + " " << TOP() << " += 1\n" + " " << vCS() << " = ("; + INLINE_LIST( out, ilItem->children, targState, inFinish ); + out << ")\n"; + + out << + " _goto_level = _again\n" + " next\n" + " end\n"; + + if ( prePushExpr != 0 ) + out << "end\n"; +} + +void RubyFTabCodeGen::RET( ostream &out, bool inFinish ) +{ + out << + " begin\n" + " " << TOP() << " -= 1\n" + " " << vCS() << " = " << STACK() << "[" << TOP() << "]\n"; + + if ( postPopExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, postPopExpr, 0, false ); + out << "end\n"; + } + + out << + " _goto_level = _again\n" + " next\n" + " end\n"; +} + +void RubyFTabCodeGen::BREAK( ostream &out, int targState ) +{ + out << + " begin\n" + " " << P() << " += 1\n" + " _goto_level = _out\n" + " next\n" + " end\n"; +} + + +std::ostream &RubyFTabCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\twhen " << redAct->actListId+1 << " then\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &RubyFTabCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\twhen " << redAct->actListId+1 << " then\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RubyFTabCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\twhen " << redAct->actListId+1 << " then\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true ); + + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &RubyFTabCodeGen::ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* Write the entry label. */ + out << "\twhen " << redAct->actListId+1 << " then\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + } + } + + genLineDirective( out ); + return out; +} + + +int RubyFTabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + return act; +} + +int RubyFTabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + return act; +} + +int RubyFTabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + return act; +} + + +/* Write out the function for a transition. */ +int RubyFTabCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + int action = 0; + if ( trans->action != 0 ) + action = trans->action->actListId+1; + return action; +} + +void RubyFTabCodeGen::writeData() +{ + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + "\n"; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void RubyFTabCodeGen::writeExec() +{ + out << + "begin\n" + " testEof = false\n" + " _klen, _trans, _keys"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + if ( redFsm->anyConditions() ) + out << ", _widec"; + + out << " = nil\n"; + + out << + " _goto_level = 0\n" + " _resume = 10\n" + " _eof_trans = 15\n" + " _again = 20\n" + " _test_eof = 30\n" + " _out = 40\n"; + + out << + " while true\n" + " if _goto_level <= 0\n"; + + if ( !noEnd ) { + out << + " if " << P() << " == " << PE() << "\n" + " _goto_level = _test_eof\n" + " next\n" + " end\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if " << vCS() << " == " << redFsm->errState->id << "\n" + " _goto_level = _out\n" + " next\n" + " end\n"; + } + + /* The resume label. */ + out << + " end\n" + " if _goto_level <= _resume\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " case " << FSA() << "[" << vCS() << "] \n"; + FROM_STATE_ACTION_SWITCH() << + " end # from state action switch \n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( useIndicies ) + out << " _trans = " << I() << "[_trans];\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " end\n" + " if _goto_level <= _eof_trans\n"; + } + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if " << TA() << "[_trans] != 0\n" + "\n" + " case " << TA() << "[_trans] \n"; + ACTION_SWITCH() << + " end # action switch \n" + " end\n" + "\n"; + } + + /* The again label. */ + out << + " end\n" + " if _goto_level <= _again\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " case " << TSA() << "[" << vCS() << "] \n"; + TO_STATE_ACTION_SWITCH() << + " end\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if " << vCS() << " == " << redFsm->errState->id << "\n" + " _goto_level = _out\n" + " next\n" + " end\n"; + } + + out << " " << P() << " += 1\n"; + + if ( !noEnd ) { + out << + " if " << P() << " != " << PE() << "\n" + " _goto_level = _resume\n" + " next\n" + " end\n"; + } + else { + out << + " _goto_level = _resume\n" + " next\n"; + } + + /* The test eof label. */ + out << + " end\n" + " if _goto_level <= _test_eof\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " == " << vEOF() << "\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if " << ET() << "[" << vCS() << "] > 0\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " _goto_level = _eof_trans\n" + " next;\n" + " end\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " begin\n" + " case ( " << EA() << "[" << vCS() << "] )\n"; + EOF_ACTION_SWITCH() << + " end\n" + " end\n"; + } + + out << + " end\n" + "\n"; + } + + out << + " end\n" + " if _goto_level <= _out\n" + " break\n" + " end\n" + "end\n"; + + /* Wrapping the execute block. */ + out << " end\n"; +} + + +void RubyFTabCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActListId) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActListId) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ diff --git a/ragel/rubyftable.h b/ragel/rubyftable.h new file mode 100644 index 0000000..91d7fe5 --- /dev/null +++ b/ragel/rubyftable.h @@ -0,0 +1,64 @@ +/* + * 2007 Victor Hugo Borja <vic@rubyforge.org> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RUBY_FTABCODEGEN_H +#define _RUBY_FTABCODEGEN_H + +#include "rubytable.h" + +class RubyFTabCodeGen : public RubyTabCodeGen +{ +public: + RubyFTabCodeGen( ostream &out ): RubyTabCodeGen(out) {} +protected: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + void GOTO( ostream &out, int gotoDest, bool inFinish ); + void GOTO_EXPR( ostream &out, GenInlineItem *ilItem, bool inFinish ); + void CALL( ostream &out, int callDest, int targState, bool inFinish ); + void CALL_EXPR(ostream &out, GenInlineItem *ilItem, int targState, bool inFinish ); + void RET( ostream &out, bool inFinish ); + void BREAK( ostream &out, int targState ); + + int TO_STATE_ACTION( RedStateAp *state ); + int FROM_STATE_ACTION( RedStateAp *state ); + int EOF_ACTION( RedStateAp *state ); + virtual int TRANS_ACTION( RedTransAp *trans ); + + void writeData(); + void writeExec(); + void calcIndexSize(); +}; + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ + +#endif + diff --git a/ragel/rubytable.cpp b/ragel/rubytable.cpp new file mode 100644 index 0000000..eb5dfd5 --- /dev/null +++ b/ragel/rubytable.cpp @@ -0,0 +1,1033 @@ +/* + * Copyright 2007 Victor Hugo Borja <vic@rubyforge.org> + * 2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <iomanip> +#include <sstream> +#include "redfsm.h" +#include "gendata.h" +#include "ragel.h" +#include "rubytable.h" + +using std::ostream; +using std::ostringstream; +using std::string; +using std::cerr; +using std::endl; + + + +void RubyTabCodeGen::GOTO( ostream &out, int gotoDest, bool inFinish ) +{ + out << + " begin\n" + " " << vCS() << " = " << gotoDest << "\n" + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; +} + +void RubyTabCodeGen::GOTO_EXPR( ostream &out, GenInlineItem *ilItem, bool inFinish ) +{ + out << + " begin\n" + " " << vCS() << " = ("; + INLINE_LIST( out, ilItem->children, 0, inFinish ); + out << ")\n"; + out << + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; +} + +void RubyTabCodeGen::CALL( ostream &out, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, prePushExpr, 0, false ); + } + + out << + " begin\n" + " " << STACK() << "[" << TOP() << "] = " << vCS() << "\n" + " " << TOP() << "+= 1\n" + " " << vCS() << " = " << callDest << "\n" + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; + + if ( prePushExpr != 0 ) + out << "end\n"; +} + +void RubyTabCodeGen::CALL_EXPR(ostream &out, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, prePushExpr, 0, false ); + } + + out << + " begin\n" + " " << STACK() << "[" << TOP() << "] = " << vCS() << "\n" + " " << TOP() << " += 1\n" + " " << vCS() << " = ("; + INLINE_LIST( out, ilItem->children, targState, inFinish ); + out << ")\n"; + + out << + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; + + if ( prePushExpr != 0 ) + out << "end\n"; +} + +void RubyTabCodeGen::RET( ostream &out, bool inFinish ) +{ + out << + " begin\n" + " " << TOP() << " -= 1\n" + " " << vCS() << " = " << STACK() << "[" << TOP() << "]\n"; + + if ( postPopExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, postPopExpr, 0, false ); + out << "end\n"; + } + + out << + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; +} + +void RubyTabCodeGen::BREAK( ostream &out, int targState ) +{ + out << + " begin\n" + " " << P() << " += 1\n" + " _trigger_goto = true\n" + " _goto_level = _out\n" + " break\n" + " end\n"; +} + +void RubyTabCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << GET_KEY() << "\n" + " _keys = " << CO() << "[" << vCS() << "]*2\n" + " _klen = " << CL() << "[" << vCS() << "]\n" + " if _klen > 0\n" + " _lower = _keys\n" + " _upper = _keys + (_klen<<1) - 2\n" + " loop do\n" + " break if _upper < _lower\n" + " _mid = _lower + (((_upper-_lower) >> 1) & ~1)\n" + " if " << GET_WIDE_KEY() << " < " << CK() << "[_mid]\n" + " _upper = _mid - 2\n" + " elsif " << GET_WIDE_KEY() << " > " << CK() << "[_mid+1]\n" + " _lower = _mid + 2\n" + " else\n" + " case " << C() << "[" << CO() << "[" << vCS() << "]" + " + ((_mid - _keys)>>1)]\n"; + + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " when " << condSpace->condSpaceId << " then" ; + out << " _widec = " << KEY(condSpace->baseKey) << + "+ (" << GET_KEY() << " - " << KEY(keyOps->minKey) << ")\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " _widec += " << condValOffset << " if ( "; + CONDITION( out, *csi ); + out << " )\n"; + } + } + + out << + " end # case\n" + " end\n" + " end # loop\n" + " end\n"; +} + + +void RubyTabCodeGen::LOCATE_TRANS() +{ + out << + " _keys = " << KO() << "[" << vCS() << "]\n" + " _trans = " << IO() << "[" << vCS() << "]\n" + " _klen = " << SL() << "[" << vCS() << "]\n" + " _break_match = false\n" + " \n" + " begin\n" + " if _klen > 0\n" + " _lower = _keys\n" + " _upper = _keys + _klen - 1\n" + "\n" + " loop do\n" + " break if _upper < _lower\n" + " _mid = _lower + ( (_upper - _lower) >> 1 )\n" + "\n" + " if " << GET_WIDE_KEY() << " < " << K() << "[_mid]\n" + " _upper = _mid - 1\n" + " elsif " << GET_WIDE_KEY() << " > " << K() << "[_mid]\n" + " _lower = _mid + 1\n" + " else\n" + " _trans += (_mid - _keys)\n" + " _break_match = true\n" + " break\n" + " end\n" + " end # loop\n" + " break if _break_match\n" + " _keys += _klen\n" + " _trans += _klen\n" + " end" + "\n" + " _klen = " << RL() << "[" << vCS() << "]\n" + " if _klen > 0\n" + " _lower = _keys\n" + " _upper = _keys + (_klen << 1) - 2\n" + " loop do\n" + " break if _upper < _lower\n" + " _mid = _lower + (((_upper-_lower) >> 1) & ~1)\n" + " if " << GET_WIDE_KEY() << " < " << K() << "[_mid]\n" + " _upper = _mid - 2\n" + " elsif " << GET_WIDE_KEY() << " > " << K() << "[_mid+1]\n" + " _lower = _mid + 2\n" + " else\n" + " _trans += ((_mid - _keys) >> 1)\n" + " _break_match = true\n" + " break\n" + " end\n" + " end # loop\n" + " break if _break_match\n" + " _trans += _klen\n" + " end\n" + " end while false\n"; +} + +void RubyTabCodeGen::writeExec() +{ + out << + "begin\n" + " _klen, _trans, _keys"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + if ( redFsm->anyConditions() ) + out << ", _widec"; + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + out << ", _acts, _nacts"; + + out << " = nil\n"; + + out << + " _goto_level = 0\n" + " _resume = 10\n" + " _eof_trans = 15\n" + " _again = 20\n" + " _test_eof = 30\n" + " _out = 40\n"; + + out << + " while true\n" + " _trigger_goto = false\n" + " if _goto_level <= 0\n"; + + if ( !noEnd ) { + out << + " if " << P() << " == " << PE() << "\n" + " _goto_level = _test_eof\n" + " next\n" + " end\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if " << vCS() << " == " << redFsm->errState->id << "\n" + " _goto_level = _out\n" + " next\n" + " end\n"; + } + + /* The resume label. */ + out << + " end\n" + " if _goto_level <= _resume\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << FSA() << "[" << vCS() << "]\n" + " _nacts = " << A() << "[_acts]\n" + " _acts += 1\n" + " while _nacts > 0\n" + " _nacts -= 1\n" + " _acts += 1\n" + " case " << A() << "[_acts - 1]\n"; + FROM_STATE_ACTION_SWITCH(); + out << + " end # from state action switch\n" + " end\n" + " if _trigger_goto\n" + " next\n" + " end\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( useIndicies ) + out << " _trans = " << I() << "[_trans]\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " end\n" + " if _goto_level <= _eof_trans\n"; + } + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << "\n"; + + out << " " << vCS() << " = " << TT() << "[_trans]\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if " << TA() << "[_trans] != 0\n" + " _acts = " << TA() << "[_trans]\n" + " _nacts = " << A() << "[_acts]\n" + " _acts += 1\n" + " while _nacts > 0\n" + " _nacts -= 1\n" + " _acts += 1\n" + " case " << A() << "[_acts - 1]\n"; + ACTION_SWITCH(); + out << + " end # action switch\n" + " end\n" + " end\n" + " if _trigger_goto\n" + " next\n" + " end\n"; + } + + /* The again label. */ + out << + " end\n" + " if _goto_level <= _again\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << TSA() << "[" << vCS() << "]\n" + " _nacts = " << A() << "[_acts]\n" + " _acts += 1\n" + " while _nacts > 0\n" + " _nacts -= 1\n" + " _acts += 1\n" + " case " << A() << "[_acts - 1]\n"; + TO_STATE_ACTION_SWITCH(); + out << + " end # to state action switch\n" + " end\n" + " if _trigger_goto\n" + " next\n" + " end\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if " << vCS() << " == " << redFsm->errState->id << "\n" + " _goto_level = _out\n" + " next\n" + " end\n"; + } + + out << " " << P() << " += 1\n"; + + if ( !noEnd ) { + out << + " if " << P() << " != " << PE() << "\n" + " _goto_level = _resume\n" + " next\n" + " end\n"; + } + else { + out << + " _goto_level = _resume\n" + " next\n"; + } + + /* The test_eof label. */ + out << + " end\n" + " if _goto_level <= _test_eof\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " == " << vEOF() << "\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if " << ET() << "[" << vCS() << "] > 0\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " _goto_level = _eof_trans\n" + " next;\n" + " end\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " __acts = " << EA() << "[" << vCS() << "]\n" + " __nacts = " << " " << A() << "[__acts]\n" + " __acts += 1\n" + " while __nacts > 0\n" + " __nacts -= 1\n" + " __acts += 1\n" + " case " << A() << "[__acts - 1]\n"; + EOF_ACTION_SWITCH() << + " end # eof action switch\n" + " end\n" + " if _trigger_goto\n" + " next\n" + " end\n"; + } + + out << + "end\n"; + } + + out << + " end\n" + " if _goto_level <= _out\n" + " break\n" + " end\n"; + + /* The loop for next. */ + out << + " end\n"; + + /* Wrapping the execute block. */ + out << + " end\n"; +} + +std::ostream &RubyTabCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action */ + out << " when " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &RubyTabCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "when " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RubyTabCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "when " << act->actionId << " then\n"; + ACTION( out, act, 0, true ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RubyTabCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "when " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + + +void RubyTabCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void RubyTabCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + + +int RubyTabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + return act; +} + +int RubyTabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + return act; +} + +int RubyTabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + return act; +} + + +std::ostream &RubyTabCodeGen::COND_OFFSETS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0, curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + ARRAY_ITEM( INT(curKeyOffset), ++totalStateNum, st.last() ); + + /* Move the key offset ahead. */ + curKeyOffset += st->stateCondList.length(); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::KEY_OFFSETS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0, curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + ARRAY_ITEM( INT(curKeyOffset), ++totalStateNum, st.last() ); + + /* Move the key offset ahead. */ + curKeyOffset += st->outSingle.length() + st->outRange.length()*2; + } + END_ARRAY_LINE(); + return out; +} + + +std::ostream &RubyTabCodeGen::INDEX_OFFSETS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + ARRAY_ITEM( INT(curIndOffset), ++totalStateNum, st.last() ); + + /* Move the index offset ahead. */ + curIndOffset += st->outSingle.length() + st->outRange.length(); + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::COND_LENS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + ARRAY_ITEM( INT(st->stateCondList.length()), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + + +std::ostream &RubyTabCodeGen::SINGLE_LENS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + ARRAY_ITEM( INT(st->outSingle.length()), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::RANGE_LENS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit length of range index. */ + ARRAY_ITEM( INT(st->outRange.length()), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::TO_STATE_ACTIONS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT(TO_STATE_ACTION(st)), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::FROM_STATE_ACTIONS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT(FROM_STATE_ACTION(st)), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::EOF_ACTIONS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT(EOF_ACTION(st)), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::EOF_TRANS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + + /* Write any eof action. */ + ARRAY_ITEM( INT(trans), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::COND_KEYS() +{ + START_ARRAY_LINE(); + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Lower key. */ + ARRAY_ITEM( KEY( sc->lowKey ), ++totalTrans, false ); + ARRAY_ITEM( KEY( sc->highKey ), ++totalTrans, false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::COND_SPACES() +{ + START_ARRAY_LINE(); + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Cond Space id. */ + ARRAY_ITEM( KEY( sc->condSpace->condSpaceId ), ++totalTrans, false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::KEYS() +{ + START_ARRAY_LINE(); + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + ARRAY_ITEM( KEY( stel->lowKey ), ++totalTrans, false ); + } + + /* Loop the state's transitions. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + /* Lower key. */ + ARRAY_ITEM( KEY( rtel->lowKey ), ++totalTrans, false ); + + /* Upper key. */ + ARRAY_ITEM( KEY( rtel->highKey ), ++totalTrans, false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::INDICIES() +{ + int totalTrans = 0; + START_ARRAY_LINE(); + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + ARRAY_ITEM( KEY( stel->value->id ), ++totalTrans, false ); + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + ARRAY_ITEM( KEY( rtel->value->id ), ++totalTrans, false ); + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + ARRAY_ITEM( KEY( st->defTrans->id ), ++totalTrans, false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::TRANS_TARGS() +{ + int totalTrans = 0; + START_ARRAY_LINE(); + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + ARRAY_ITEM( KEY( trans->targ->id ), ++totalTrans, false ); + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + ARRAY_ITEM( KEY( trans->targ->id ), ++totalTrans, false ); + } + + /* The state's default target state. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + ARRAY_ITEM( KEY( trans->targ->id ), ++totalTrans, false ); + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + trans->pos = totalTrans; + ARRAY_ITEM( KEY( trans->targ->id ), ++totalTrans, false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + + +std::ostream &RubyTabCodeGen::TRANS_ACTIONS() +{ + int totalTrans = 0; + START_ARRAY_LINE(); + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), ++totalTrans, false ); + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), ++totalTrans, false ); + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), ++totalTrans, false ); + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), ++totalTrans, false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::TRANS_TARGS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + START_ARRAY_LINE(); + int totalStates = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Save the position. Needed for eofTargs. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + ARRAY_ITEM( INT(trans->targ->id), ++totalStates, ( t >= redFsm->transSet.length()-1 ) ); + } + END_ARRAY_LINE(); + delete[] transPtrs; + return out; +} + + +std::ostream &RubyTabCodeGen::TRANS_ACTIONS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + START_ARRAY_LINE(); + int totalAct = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), ++totalAct, + ( t >= redFsm->transSet.length()-1 ) ); + } + END_ARRAY_LINE(); + delete[] transPtrs; + return out; +} + + +void RubyTabCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + "\n"; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +/* + Local Variables: + mode: c++ + indent-tabs-mode: 1 + c-file-style: "bsd" + End: + */ diff --git a/ragel/rubytable.h b/ragel/rubytable.h new file mode 100644 index 0000000..76b847f --- /dev/null +++ b/ragel/rubytable.h @@ -0,0 +1,124 @@ +/* + * Copyright 2007 Victor Hugo Borja <vic@rubyforge.org> + * 2006-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RUBY_TABCODEGEN_H +#define _RUBY_TABCODEGEN_H + +#include <iostream> +#include <string> +#include <stdio.h> +#include "common.h" +#include "gendata.h" +#include "rubycodegen.h" + + +using std::string; +using std::ostream; + +/* + * RubyCodeGen + */ +class RubyTabCodeGen : public RubyCodeGen +{ +public: + RubyTabCodeGen( ostream &out ) : + RubyCodeGen(out) {} + virtual ~RubyTabCodeGen() {} + +public: + void BREAK( ostream &ret, int targState ); + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void RET( ostream &ret, bool inFinish ); + + void COND_TRANSLATE(); + void LOCATE_TRANS(); + + virtual void writeExec(); + virtual void writeData(); + + protected: + virtual std::ostream &TO_STATE_ACTION_SWITCH(); + virtual std::ostream &FROM_STATE_ACTION_SWITCH(); + virtual std::ostream &EOF_ACTION_SWITCH(); + virtual std::ostream &ACTION_SWITCH(); + + std::ostream &COND_KEYS(); + std::ostream &COND_SPACES(); + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &COND_OFFSETS(); + std::ostream &KEY_OFFSETS(); + std::ostream &INDEX_OFFSETS(); + std::ostream &COND_LENS(); + std::ostream &SINGLE_LENS(); + std::ostream &RANGE_LENS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + std::ostream &TRANS_TARGS_WI(); + std::ostream &TRANS_ACTIONS_WI(); + + + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + + virtual int TO_STATE_ACTION( RedStateAp *state ); + virtual int FROM_STATE_ACTION( RedStateAp *state ); + virtual int EOF_ACTION( RedStateAp *state ); + +private: + string array_type; + string array_name; + +public: + + void EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void EXECTE( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void LM_SWITCH( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void SET_ACT( ostream &ret, GenInlineItem *item ); + void INIT_TOKSTART( ostream &ret, GenInlineItem *item ); + void INIT_ACT( ostream &ret, GenInlineItem *item ); + void SET_TOKSTART( ostream &ret, GenInlineItem *item ); + void SET_TOKEND( ostream &ret, GenInlineItem *item ); + void GET_TOKEND( ostream &ret, GenInlineItem *item ); + void SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish ); + + +}; + + +#endif + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ diff --git a/ragel/xmlcodegen.cpp b/ragel/xmlcodegen.cpp index bc9c155..9a9c388 100644 --- a/ragel/xmlcodegen.cpp +++ b/ragel/xmlcodegen.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2005, 2006 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2005-2007 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -24,21 +24,71 @@ #include "xmlcodegen.h" #include "parsedata.h" #include "fsmgraph.h" +#include "gendata.h" +#include "inputdata.h" #include <string.h> +#include "rlparse.h" +#include "version.h" using namespace std; -XMLCodeGen::XMLCodeGen( char *fsmName, ParseData *pd, FsmAp *fsm, - std::ostream &out ) +GenBase::GenBase( char *fsmName, ParseData *pd, FsmAp *fsm ) : fsmName(fsmName), pd(pd), fsm(fsm), - out(out), nextActionTableId(0) { } +void GenBase::appendTrans( TransListVect &outList, Key lowKey, + Key highKey, TransAp *trans ) +{ + if ( trans->toState != 0 || trans->actionTable.length() > 0 ) + outList.append( TransEl( lowKey, highKey, trans ) ); +} + +void GenBase::reduceActionTables() +{ + /* Reduce the actions tables to a set. */ + for ( StateList::Iter st = fsm->stateList; st.lte(); st++ ) { + RedActionTable *actionTable = 0; + + /* Reduce To State Actions. */ + if ( st->toStateActionTable.length() > 0 ) { + if ( actionTableMap.insert( st->toStateActionTable, &actionTable ) ) + actionTable->id = nextActionTableId++; + } + + /* Reduce From State Actions. */ + if ( st->fromStateActionTable.length() > 0 ) { + if ( actionTableMap.insert( st->fromStateActionTable, &actionTable ) ) + actionTable->id = nextActionTableId++; + } + + /* Reduce EOF actions. */ + if ( st->eofActionTable.length() > 0 ) { + if ( actionTableMap.insert( st->eofActionTable, &actionTable ) ) + actionTable->id = nextActionTableId++; + } + + /* Loop the transitions and reduce their actions. */ + for ( TransList::Iter trans = st->outList; trans.lte(); trans++ ) { + if ( trans->actionTable.length() > 0 ) { + if ( actionTableMap.insert( trans->actionTable, &actionTable ) ) + actionTable->id = nextActionTableId++; + } + } + } +} + +XMLCodeGen::XMLCodeGen( char *fsmName, ParseData *pd, FsmAp *fsm, std::ostream &out ) +: + GenBase(fsmName, pd, fsm), + out(out) +{ +} + void XMLCodeGen::writeActionList() { @@ -82,47 +132,6 @@ void XMLCodeGen::writeActionTableList() delete[] tables; } -void XMLCodeGen::reduceActionTables() -{ - /* Reduce the actions tables to a set. */ - for ( StateList::Iter st = fsm->stateList; st.lte(); st++ ) { - RedActionTable *actionTable = 0; - - /* Reduce To State Actions. */ - if ( st->toStateActionTable.length() > 0 ) { - if ( actionTableMap.insert( st->toStateActionTable, &actionTable ) ) - actionTable->id = nextActionTableId++; - } - - /* Reduce From State Actions. */ - if ( st->fromStateActionTable.length() > 0 ) { - if ( actionTableMap.insert( st->fromStateActionTable, &actionTable ) ) - actionTable->id = nextActionTableId++; - } - - /* Reduce EOF actions. */ - if ( st->eofActionTable.length() > 0 ) { - if ( actionTableMap.insert( st->eofActionTable, &actionTable ) ) - actionTable->id = nextActionTableId++; - } - - /* Loop the transitions and reduce their actions. */ - for ( TransList::Iter trans = st->outList; trans.lte(); trans++ ) { - if ( trans->actionTable.length() > 0 ) { - if ( actionTableMap.insert( trans->actionTable, &actionTable ) ) - actionTable->id = nextActionTableId++; - } - } - } -} - -void XMLCodeGen::appendTrans( TransListVect &outList, Key lowKey, - Key highKey, TransAp *trans ) -{ - if ( trans->toState != 0 || trans->actionTable.length() > 0 ) - outList.append( TransEl( lowKey, highKey, trans ) ); -} - void XMLCodeGen::writeKey( Key key ) { if ( keyOps->isSigned ) @@ -175,26 +184,24 @@ void XMLCodeGen::writeTransList( StateAp *state ) out << " </trans_list>\n"; } -void XMLCodeGen::writeLmSwitch( InlineItem *item ) +void XMLCodeGen::writeEofTrans( StateAp *state ) { - LongestMatch *longestMatch = item->longestMatch; - - out << "<lm_switch"; - if ( longestMatch->lmSwitchHandlesError ) - out << " handles_error=\"t\""; - out << ">\n"; + RedActionTable *eofActions = 0; + if ( state->eofActionTable.length() > 0 ) + eofActions = actionTableMap.find( state->eofActionTable ); - for ( LmPartList::Iter lmi = *longestMatch->longestMatchList; lmi.lte(); lmi++ ) { - if ( lmi->inLmSelect && lmi->action != 0 ) { - /* Open the action. Write it with the context that sets up _p - * when doing control flow changes from inside the machine. */ - out << " <sub_action id=\"" << lmi->longestMatchId << "\">"; - writeInlineList( lmi->action->inlineList, item ); - out << "</sub_action>\n"; - } - } + /* The <eof_t> is used when there is an eof target, otherwise the eof + * action goes into state actions. */ + if ( state->eofTarget != 0 ) { + out << " <eof_t>" << state->eofTarget->alg.stateNum; - out << " </lm_switch><exec><get_tokend></get_tokend></exec>"; + if ( eofActions != 0 ) + out << " " << eofActions->id; + else + out << " x"; + + out << "</eof_t>" << endl; + } } void XMLCodeGen::writeText( InlineItem *item ) @@ -206,91 +213,7 @@ void XMLCodeGen::writeText( InlineItem *item ) out << "</text>"; } -void XMLCodeGen::writeCtrlFlow( InlineItem *item, InlineItem *context ) -{ - if ( context != 0 ) { - out << "<sub_action>"; - - switch ( context->type ) { - case InlineItem::LmOnLast: - out << "<exec><get_tokend></get_tokend></exec>"; - break; - case InlineItem::LmOnNext: - out << "<exec><get_tokend></get_tokend></exec>"; - break; - case InlineItem::LmOnLagBehind: - out << "<exec><get_tokend></get_tokend></exec>"; - break; - case InlineItem::LmSwitch: - out << "<exec><get_tokend></get_tokend></exec>"; - break; - default: break; - } - } - - switch ( item->type ) { - case InlineItem::Goto: - writeGoto( item, context ); - break; - case InlineItem::GotoExpr: - writeGotoExpr( item, context ); - break; - case InlineItem::Call: - writeCall( item, context ); - break; - case InlineItem::CallExpr: - writeCallExpr( item, context ); - break; - case InlineItem::Next: - writeNext( item, context ); - break; - case InlineItem::NextExpr: - writeNextExpr( item, context ); - break; - case InlineItem::Break: - out << "<break></break>"; - break; - case InlineItem::Ret: - out << "<ret></ret>"; - break; - default: break; - } - - if ( context != 0 ) - out << "</sub_action>"; -} - -void XMLCodeGen::writePtrMod( InlineItem *item, InlineItem *context ) -{ - if ( context != 0 && ( context->type == InlineItem::LmOnNext || - context->type == InlineItem::LmOnLagBehind || - context->type == InlineItem::LmSwitch ) ) - { - switch ( item->type ) { - case InlineItem::Hold: - out << "<holdte></holdte>"; - break; - case InlineItem::Exec: - writeActionExecTE( item ); - break; - default: break; - } - } - else { - switch ( item->type ) { - case InlineItem::Hold: - out << "<hold></hold>"; - break; - case InlineItem::Exec: - writeActionExec( item ); - break; - default: break; - } - } -} - - -void XMLCodeGen::writeGoto( InlineItem *item, InlineItem *context ) +void XMLCodeGen::writeGoto( InlineItem *item ) { if ( pd->generatingSectionSubset ) out << "<goto>-1</goto>"; @@ -300,7 +223,7 @@ void XMLCodeGen::writeGoto( InlineItem *item, InlineItem *context ) } } -void XMLCodeGen::writeCall( InlineItem *item, InlineItem *context ) +void XMLCodeGen::writeCall( InlineItem *item ) { if ( pd->generatingSectionSubset ) out << "<call>-1</call>"; @@ -310,7 +233,7 @@ void XMLCodeGen::writeCall( InlineItem *item, InlineItem *context ) } } -void XMLCodeGen::writeNext( InlineItem *item, InlineItem *context ) +void XMLCodeGen::writeNext( InlineItem *item ) { if ( pd->generatingSectionSubset ) out << "<next>-1</next>"; @@ -320,28 +243,28 @@ void XMLCodeGen::writeNext( InlineItem *item, InlineItem *context ) } } -void XMLCodeGen::writeGotoExpr( InlineItem *item, InlineItem *context ) +void XMLCodeGen::writeGotoExpr( InlineItem *item ) { out << "<goto_expr>"; - writeInlineList( item->children, 0 ); + writeInlineList( item->children ); out << "</goto_expr>"; } -void XMLCodeGen::writeCallExpr( InlineItem *item, InlineItem *context ) +void XMLCodeGen::writeCallExpr( InlineItem *item ) { out << "<call_expr>"; - writeInlineList( item->children, 0 ); + writeInlineList( item->children ); out << "</call_expr>"; } -void XMLCodeGen::writeNextExpr( InlineItem *item, InlineItem *context ) +void XMLCodeGen::writeNextExpr( InlineItem *item ) { out << "<next_expr>"; - writeInlineList( item->children, 0 ); + writeInlineList( item->children ); out << "</next_expr>"; } -void XMLCodeGen::writeEntry( InlineItem * item ) +void XMLCodeGen::writeEntry( InlineItem *item ) { if ( pd->generatingSectionSubset ) out << "<entry>-1</entry>"; @@ -354,62 +277,118 @@ void XMLCodeGen::writeEntry( InlineItem * item ) void XMLCodeGen::writeActionExec( InlineItem *item ) { out << "<exec>"; - writeInlineList( item->children, 0 ); + writeInlineList( item->children ); out << "</exec>"; } -void XMLCodeGen::writeActionExecTE( InlineItem *item ) -{ - out << "<execte>"; - writeInlineList( item->children, 0 ); - out << "</execte>"; -} - void XMLCodeGen::writeLmOnLast( InlineItem *item ) { out << "<set_tokend>1</set_tokend>"; + if ( item->longestMatchPart->action != 0 ) { out << "<sub_action>"; - writeInlineList( item->longestMatchPart->action->inlineList, item ); + writeInlineList( item->longestMatchPart->action->inlineList ); out << "</sub_action>"; } - out << "<exec><get_tokend></get_tokend></exec>"; } void XMLCodeGen::writeLmOnNext( InlineItem *item ) { out << "<set_tokend>0</set_tokend>"; + out << "<hold></hold>"; + if ( item->longestMatchPart->action != 0 ) { out << "<sub_action>"; - writeInlineList( item->longestMatchPart->action->inlineList, item ); + writeInlineList( item->longestMatchPart->action->inlineList ); out << "</sub_action>"; } - out << "<exec><get_tokend></get_tokend></exec>"; } void XMLCodeGen::writeLmOnLagBehind( InlineItem *item ) { + out << "<exec><get_tokend></get_tokend></exec>"; + if ( item->longestMatchPart->action != 0 ) { out << "<sub_action>"; - writeInlineList( item->longestMatchPart->action->inlineList, item ); + writeInlineList( item->longestMatchPart->action->inlineList ); out << "</sub_action>"; } - out << "<exec><get_tokend></get_tokend></exec>"; } +void XMLCodeGen::writeLmSwitch( InlineItem *item ) +{ + LongestMatch *longestMatch = item->longestMatch; + out << "<lm_switch>\n"; + + /* We can't put the <exec> here because we may need to handle the error + * case and in that case p should not be changed. Instead use a default + * label in the switch to adjust p when user actions are not set. An id of + * -1 indicates the default. */ + + if ( longestMatch->lmSwitchHandlesError ) { + /* If the switch handles error then we should have also forced the + * error state. */ + assert( fsm->errState != 0 ); + + out << " <sub_action id=\"0\">"; + out << "<goto>" << fsm->errState->alg.stateNum << "</goto>"; + out << "</sub_action>\n"; + } + + bool needDefault = false; + for ( LmPartList::Iter lmi = *longestMatch->longestMatchList; lmi.lte(); lmi++ ) { + if ( lmi->inLmSelect ) { + if ( lmi->action == 0 ) + needDefault = true; + else { + /* Open the action. Write it with the context that sets up _p + * when doing control flow changes from inside the machine. */ + out << " <sub_action id=\"" << lmi->longestMatchId << "\">"; + out << "<exec><get_tokend></get_tokend></exec>"; + writeInlineList( lmi->action->inlineList ); + out << "</sub_action>\n"; + } + } + } + + if ( needDefault ) { + out << " <sub_action id=\"-1\"><exec><get_tokend>" + "</get_tokend></exec></sub_action>\n"; + } -void XMLCodeGen::writeInlineList( InlineList *inlineList, InlineItem *context ) + out << " </lm_switch>"; +} + +void XMLCodeGen::writeInlineList( InlineList *inlineList ) { for ( InlineList::Iter item = *inlineList; item.lte(); item++ ) { switch ( item->type ) { case InlineItem::Text: writeText( item ); break; - case InlineItem::Goto: case InlineItem::GotoExpr: - case InlineItem::Call: case InlineItem::CallExpr: - case InlineItem::Next: case InlineItem::NextExpr: - case InlineItem::Break: case InlineItem::Ret: - writeCtrlFlow( item, context ); + case InlineItem::Goto: + writeGoto( item ); + break; + case InlineItem::GotoExpr: + writeGotoExpr( item ); + break; + case InlineItem::Call: + writeCall( item ); + break; + case InlineItem::CallExpr: + writeCallExpr( item ); + break; + case InlineItem::Next: + writeNext( item ); + break; + case InlineItem::NextExpr: + writeNextExpr( item ); + break; + case InlineItem::Break: + out << "<break></break>"; + break; + case InlineItem::Ret: + out << "<ret></ret>"; break; case InlineItem::PChar: out << "<pchar></pchar>"; @@ -428,13 +407,12 @@ void XMLCodeGen::writeInlineList( InlineList *inlineList, InlineItem *context ) break; case InlineItem::Hold: + out << "<hold></hold>"; + break; case InlineItem::Exec: - writePtrMod( item, context ); + writeActionExec( item ); break; - case InlineItem::LmSwitch: - writeLmSwitch( item ); - break; case InlineItem::LmSetActId: out << "<set_act>" << item->longestMatchPart->longestMatchId << @@ -443,6 +421,7 @@ void XMLCodeGen::writeInlineList( InlineList *inlineList, InlineItem *context ) case InlineItem::LmSetTokEnd: out << "<set_tokend>1</set_tokend>"; break; + case InlineItem::LmOnLast: writeLmOnLast( item ); break; @@ -452,6 +431,10 @@ void XMLCodeGen::writeInlineList( InlineList *inlineList, InlineItem *context ) case InlineItem::LmOnLagBehind: writeLmOnLagBehind( item ); break; + case InlineItem::LmSwitch: + writeLmSwitch( item ); + break; + case InlineItem::LmInitAct: out << "<init_act></init_act>"; break; @@ -465,17 +448,278 @@ void XMLCodeGen::writeInlineList( InlineList *inlineList, InlineItem *context ) } } +BackendGen::BackendGen( char *fsmName, ParseData *pd, FsmAp *fsm, CodeGenData *cgd ) +: + GenBase(fsmName, pd, fsm), + cgd(cgd) +{ +} + + +void BackendGen::makeText( GenInlineList *outList, InlineItem *item ) +{ + GenInlineItem *inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::Text ); + inlineItem->data = item->data; + + outList->append( inlineItem ); +} + +void BackendGen::makeTargetItem( GenInlineList *outList, NameInst *nameTarg, + GenInlineItem::Type type ) +{ + long targetState; + if ( pd->generatingSectionSubset ) + targetState = -1; + else { + EntryMapEl *targ = fsm->entryPoints.find( nameTarg->id ); + targetState = targ->value->alg.stateNum; + } + + /* Make the item. */ + GenInlineItem *inlineItem = new GenInlineItem( InputLoc(), type ); + inlineItem->targId = targetState; + outList->append( inlineItem ); +} + +/* Make a sublist item with a given type. */ +void BackendGen::makeSubList( GenInlineList *outList, + InlineList *inlineList, GenInlineItem::Type type ) +{ + /* Fill the sub list. */ + GenInlineList *subList = new GenInlineList; + makeGenInlineList( subList, inlineList ); + + /* Make the item. */ + GenInlineItem *inlineItem = new GenInlineItem( InputLoc(), type ); + inlineItem->children = subList; + outList->append( inlineItem ); +} + +void BackendGen::makeLmOnLast( GenInlineList *outList, InlineItem *item ) +{ + makeSetTokend( outList, 1 ); + + if ( item->longestMatchPart->action != 0 ) { + makeSubList( outList, + item->longestMatchPart->action->inlineList, + GenInlineItem::SubAction ); + } +} + +void BackendGen::makeLmOnNext( GenInlineList *outList, InlineItem *item ) +{ + makeSetTokend( outList, 0 ); + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::Hold ) ); + + if ( item->longestMatchPart->action != 0 ) { + makeSubList( outList, + item->longestMatchPart->action->inlineList, + GenInlineItem::SubAction ); + } +} + +void BackendGen::makeExecGetTokend( GenInlineList *outList ) +{ + /* Make the Exec item. */ + GenInlineItem *execItem = new GenInlineItem( InputLoc(), GenInlineItem::Exec ); + execItem->children = new GenInlineList; + + /* Make the GetTokEnd */ + GenInlineItem *getTokend = new GenInlineItem( InputLoc(), GenInlineItem::LmGetTokEnd ); + execItem->children->append( getTokend ); + + outList->append( execItem ); +} + +void BackendGen::makeLmOnLagBehind( GenInlineList *outList, InlineItem *item ) +{ + /* Jump to the tokend. */ + makeExecGetTokend( outList ); + + if ( item->longestMatchPart->action != 0 ) { + makeSubList( outList, + item->longestMatchPart->action->inlineList, + GenInlineItem::SubAction ); + } +} + +void BackendGen::makeLmSwitch( GenInlineList *outList, InlineItem *item ) +{ + GenInlineItem *lmSwitch = new GenInlineItem( InputLoc(), GenInlineItem::LmSwitch ); + GenInlineList *lmList = lmSwitch->children = new GenInlineList; + LongestMatch *longestMatch = item->longestMatch; + + /* We can't put the <exec> here because we may need to handle the error + * case and in that case p should not be changed. Instead use a default + * label in the switch to adjust p when user actions are not set. An id of + * -1 indicates the default. */ + + if ( longestMatch->lmSwitchHandlesError ) { + /* If the switch handles error then we should have also forced the + * error state. */ + assert( fsm->errState != 0 ); + + GenInlineItem *errCase = new GenInlineItem( InputLoc(), GenInlineItem::SubAction ); + errCase->lmId = 0; + errCase->children = new GenInlineList; + + /* Make the item. */ + GenInlineItem *gotoItem = new GenInlineItem( InputLoc(), GenInlineItem::Goto ); + gotoItem->targId = fsm->errState->alg.stateNum; + errCase->children->append( gotoItem ); + + lmList->append( errCase ); + } + + bool needDefault = false; + for ( LmPartList::Iter lmi = *longestMatch->longestMatchList; lmi.lte(); lmi++ ) { + if ( lmi->inLmSelect ) { + if ( lmi->action == 0 ) + needDefault = true; + else { + /* Open the action. Write it with the context that sets up _p + * when doing control flow changes from inside the machine. */ + GenInlineItem *lmCase = new GenInlineItem( InputLoc(), + GenInlineItem::SubAction ); + lmCase->lmId = lmi->longestMatchId; + lmCase->children = new GenInlineList; + + makeExecGetTokend( lmCase->children ); + makeGenInlineList( lmCase->children, lmi->action->inlineList ); + + lmList->append( lmCase ); + } + } + } + + if ( needDefault ) { + GenInlineItem *defCase = new GenInlineItem( InputLoc(), + GenInlineItem::SubAction ); + defCase->lmId = -1; + defCase->children = new GenInlineList; + + makeExecGetTokend( defCase->children ); + + lmList->append( defCase ); + } + + outList->append( lmSwitch ); +} + +void BackendGen::makeSetTokend( GenInlineList *outList, long offset ) +{ + GenInlineItem *inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::LmSetTokEnd ); + inlineItem->offset = offset; + outList->append( inlineItem ); +} + +void BackendGen::makeSetAct( GenInlineList *outList, long lmId ) +{ + GenInlineItem *inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::LmSetActId ); + inlineItem->lmId = lmId; + outList->append( inlineItem ); +} + +void BackendGen::makeGenInlineList( GenInlineList *outList, InlineList *inList ) +{ + for ( InlineList::Iter item = *inList; item.lte(); item++ ) { + switch ( item->type ) { + case InlineItem::Text: + makeText( outList, item ); + break; + case InlineItem::Goto: + makeTargetItem( outList, item->nameTarg, GenInlineItem::Goto ); + break; + case InlineItem::GotoExpr: + makeSubList( outList, item->children, GenInlineItem::GotoExpr ); + break; + case InlineItem::Call: + makeTargetItem( outList, item->nameTarg, GenInlineItem::Call ); + break; + case InlineItem::CallExpr: + makeSubList( outList, item->children, GenInlineItem::CallExpr ); + break; + case InlineItem::Next: + makeTargetItem( outList, item->nameTarg, GenInlineItem::Next ); + break; + case InlineItem::NextExpr: + makeSubList( outList, item->children, GenInlineItem::NextExpr ); + break; + case InlineItem::Break: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::Break ) ); + break; + case InlineItem::Ret: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::Ret ) ); + break; + case InlineItem::PChar: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::PChar ) ); + break; + case InlineItem::Char: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::Char ) ); + break; + case InlineItem::Curs: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::Curs ) ); + break; + case InlineItem::Targs: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::Targs ) ); + break; + case InlineItem::Entry: + makeTargetItem( outList, item->nameTarg, GenInlineItem::Entry ); + break; + + case InlineItem::Hold: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::Hold ) ); + break; + case InlineItem::Exec: + makeSubList( outList, item->children, GenInlineItem::Exec ); + break; + + case InlineItem::LmSetActId: + makeSetAct( outList, item->longestMatchPart->longestMatchId ); + break; + case InlineItem::LmSetTokEnd: + makeSetTokend( outList, 1 ); + break; + + case InlineItem::LmOnLast: + makeLmOnLast( outList, item ); + break; + case InlineItem::LmOnNext: + makeLmOnNext( outList, item ); + break; + case InlineItem::LmOnLagBehind: + makeLmOnLagBehind( outList, item ); + break; + case InlineItem::LmSwitch: + makeLmSwitch( outList, item ); + break; + + case InlineItem::LmInitAct: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::LmInitAct ) ); + break; + case InlineItem::LmInitTokStart: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::LmInitTokStart ) ); + break; + case InlineItem::LmSetTokStart: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::LmSetTokStart ) ); + cgd->hasLongestMatch = true; + break; + } + } +} + + void XMLCodeGen::writeAction( Action *action ) { out << " <action id=\"" << action->actionId << "\""; if ( action->name != 0 ) out << " name=\"" << action->name << "\""; out << " line=\"" << action->loc.line << "\" col=\"" << action->loc.col << "\">"; - writeInlineList( action->inlineList, 0 ); + writeInlineList( action->inlineList ); out << "</action>\n"; } -void xmlEscapeHost( std::ostream &out, char *data, int len ) +void xmlEscapeHost( std::ostream &out, char *data, long len ) { char *end = data + len; while ( data != end ) { @@ -499,8 +743,10 @@ void XMLCodeGen::writeStateActions( StateAp *state ) if ( state->fromStateActionTable.length() > 0 ) fromStateActions = actionTableMap.find( state->fromStateActionTable ); + /* EOF actions go out here only if the state has no eof target. If it has + * an eof target then an eof transition will be used instead. */ RedActionTable *eofActions = 0; - if ( state->eofActionTable.length() > 0 ) + if ( state->eofTarget == 0 && state->eofActionTable.length() > 0 ) eofActions = actionTableMap.find( state->eofActionTable ); if ( toStateActions != 0 || fromStateActions != 0 || eofActions != 0 ) { @@ -518,7 +764,9 @@ void XMLCodeGen::writeStateActions( StateAp *state ) if ( eofActions != 0 ) out << " " << eofActions->id; else - out << " x"; out << "</state_actions>\n"; + out << " x"; + + out << "</state_actions>\n"; } } @@ -550,6 +798,7 @@ void XMLCodeGen::writeStateList() out << ">\n"; writeStateActions( st ); + writeEofTrans( st ); writeStateConditions( st ); writeTransList( st ); @@ -557,11 +806,26 @@ void XMLCodeGen::writeStateList() if ( !st.last() ) out << "\n"; - } out << " </state_list>\n"; } +bool XMLCodeGen::writeNameInst( NameInst *nameInst ) +{ + bool written = false; + if ( nameInst->parent != 0 ) + written = writeNameInst( nameInst->parent ); + + if ( nameInst->name != 0 ) { + if ( written ) + out << '_'; + out << nameInst->name; + written = true; + } + + return written; +} + void XMLCodeGen::writeEntryPoints() { /* List of entry points other than start state. */ @@ -574,8 +838,9 @@ void XMLCodeGen::writeEntryPoints() /* Get the name instantiation from nameIndex. */ NameInst *nameInst = pd->nameIndex[en->key]; StateAp *state = en->value; - out << " <entry name=\"" << nameInst->name << "\">" << - state->alg.stateNum << "</entry>\n"; + out << " <entry name=\""; + writeNameInst( nameInst ); + out << "\">" << state->alg.stateNum << "</entry>\n"; } out << " </entry_points>\n"; } @@ -583,8 +848,6 @@ void XMLCodeGen::writeEntryPoints() void XMLCodeGen::writeMachine() { - fsm->setStateNumbers(); - /* Open the machine. */ out << " <machine>\n"; @@ -598,6 +861,12 @@ void XMLCodeGen::writeMachine() /* Start state. */ out << " <start_state>" << fsm->startState->alg.stateNum << "</start_state>\n"; + + /* Error state. */ + if ( fsm->errState != 0 ) { + out << " <error_state>" << fsm->errState->alg.stateNum << + "</error_state>\n"; + } writeEntryPoints(); writeStateList(); @@ -605,32 +874,6 @@ void XMLCodeGen::writeMachine() out << " </machine>\n"; } -void XMLCodeGen::writeAlphType() -{ - out << " <alphtype>" << - (keyOps->alphType - hostLang->hostTypes) << "</alphtype>\n"; -} - -void XMLCodeGen::writeGetKeyExpr() -{ - out << " <getkey>"; - writeInlineList( pd->getKeyExpr, 0 ); - out << "</getkey>\n"; -} - -void XMLCodeGen::writeAccessExpr() -{ - out << " <access>"; - writeInlineList( pd->accessExpr, 0 ); - out << "</access>\n"; -} - -void XMLCodeGen::writeCurStateExpr() -{ - out << " <curstate>"; - writeInlineList( pd->curStateExpr, 0 ); - out << "</curstate>\n"; -} void XMLCodeGen::writeConditions() { @@ -652,20 +895,120 @@ void XMLCodeGen::writeConditions() } } +void XMLCodeGen::writeExports() +{ + if ( pd->exportList.length() > 0 ) { + out << " <exports>\n"; + for ( ExportList::Iter exp = pd->exportList; exp.lte(); exp++ ) { + out << " <ex name=\"" << exp->name << "\">"; + writeKey( exp->key ); + out << "</ex>\n"; + } + out << " </exports>\n"; + } +} + void XMLCodeGen::writeXML() { /* Open the definition. */ out << "<ragel_def name=\"" << fsmName << "\">\n"; - writeAlphType(); + + /* Alphabet type. */ + out << " <alphtype>" << keyOps->alphType->internalName << "</alphtype>\n"; - if ( pd->getKeyExpr != 0 ) - writeGetKeyExpr(); + /* Getkey expression. */ + if ( pd->getKeyExpr != 0 ) { + out << " <getkey>"; + writeInlineList( pd->getKeyExpr ); + out << "</getkey>\n"; + } + + /* Access expression. */ + if ( pd->accessExpr != 0 ) { + out << " <access>"; + writeInlineList( pd->accessExpr ); + out << "</access>\n"; + } - if ( pd->accessExpr != 0 ) - writeAccessExpr(); + /* PrePush expression. */ + if ( pd->prePushExpr != 0 ) { + out << " <prepush>"; + writeInlineList( pd->prePushExpr ); + out << "</prepush>\n"; + } + + /* PostPop expression. */ + if ( pd->postPopExpr != 0 ) { + out << " <postpop>"; + writeInlineList( pd->postPopExpr ); + out << "</postpop>\n"; + } - if ( pd->curStateExpr != 0 ) - writeCurStateExpr(); + /* + * Variable expressions. + */ + + if ( pd->pExpr != 0 ) { + out << " <p_expr>"; + writeInlineList( pd->pExpr ); + out << "</p_expr>\n"; + } + + if ( pd->peExpr != 0 ) { + out << " <pe_expr>"; + writeInlineList( pd->peExpr ); + out << "</pe_expr>\n"; + } + + if ( pd->eofExpr != 0 ) { + out << " <eof_expr>"; + writeInlineList( pd->eofExpr ); + out << "</eof_expr>\n"; + } + + if ( pd->csExpr != 0 ) { + out << " <cs_expr>"; + writeInlineList( pd->csExpr ); + out << "</cs_expr>\n"; + } + + if ( pd->topExpr != 0 ) { + out << " <top_expr>"; + writeInlineList( pd->topExpr ); + out << "</top_expr>\n"; + } + + if ( pd->stackExpr != 0 ) { + out << " <stack_expr>"; + writeInlineList( pd->stackExpr ); + out << "</stack_expr>\n"; + } + + if ( pd->actExpr != 0 ) { + out << " <act_expr>"; + writeInlineList( pd->actExpr ); + out << "</act_expr>\n"; + } + + if ( pd->tokstartExpr != 0 ) { + out << " <tokstart_expr>"; + writeInlineList( pd->tokstartExpr ); + out << "</tokstart_expr>\n"; + } + + if ( pd->tokendExpr != 0 ) { + out << " <tokend_expr>"; + writeInlineList( pd->tokendExpr ); + out << "</tokend_expr>\n"; + } + + if ( pd->dataExpr != 0 ) { + out << " <data_expr>"; + writeInlineList( pd->dataExpr ); + out << "</data_expr>\n"; + } + + writeExports(); writeMachine(); @@ -673,3 +1016,411 @@ void XMLCodeGen::writeXML() "</ragel_def>\n"; } +void BackendGen::makeExports() +{ + for ( ExportList::Iter exp = pd->exportList; exp.lte(); exp++ ) + cgd->exportList.append( new Export( exp->name, exp->key ) ); +} + +void BackendGen::makeAction( Action *action ) +{ + GenInlineList *genList = new GenInlineList; + makeGenInlineList( genList, action->inlineList ); + + cgd->newAction( curAction++, action->name, action->loc, genList ); +} + + +void BackendGen::makeActionList() +{ + /* Determine which actions to write. */ + int nextActionId = 0; + for ( ActionList::Iter act = pd->actionList; act.lte(); act++ ) { + if ( act->numRefs() > 0 || act->numCondRefs > 0 ) + act->actionId = nextActionId++; + } + + /* Write the list. */ + cgd->initActionList( nextActionId ); + curAction = 0; + + for ( ActionList::Iter act = pd->actionList; act.lte(); act++ ) { + if ( act->actionId >= 0 ) + makeAction( act ); + } +} + +void BackendGen::makeActionTableList() +{ + /* Must first order the action tables based on their id. */ + int numTables = nextActionTableId; + RedActionTable **tables = new RedActionTable*[numTables]; + for ( ActionTableMap::Iter at = actionTableMap; at.lte(); at++ ) + tables[at->id] = at; + + cgd->initActionTableList( numTables ); + curActionTable = 0; + + for ( int t = 0; t < numTables; t++ ) { + long length = tables[t]->key.length(); + + /* Collect the action table. */ + RedAction *redAct = cgd->allActionTables + curActionTable; + redAct->actListId = curActionTable; + redAct->key.setAsNew( length ); + + for ( ActionTable::Iter atel = tables[t]->key; atel.lte(); atel++ ) { + redAct->key[atel.pos()].key = 0; + redAct->key[atel.pos()].value = cgd->allActions + + atel->value->actionId; + } + + /* Insert into the action table map. */ + cgd->redFsm->actionMap.insert( redAct ); + + curActionTable += 1; + } + + delete[] tables; +} + +void BackendGen::makeConditions() +{ + if ( condData->condSpaceMap.length() > 0 ) { + long nextCondSpaceId = 0; + for ( CondSpaceMap::Iter cs = condData->condSpaceMap; cs.lte(); cs++ ) + cs->condSpaceId = nextCondSpaceId++; + + long listLength = condData->condSpaceMap.length(); + cgd->initCondSpaceList( listLength ); + curCondSpace = 0; + + for ( CondSpaceMap::Iter cs = condData->condSpaceMap; cs.lte(); cs++ ) { + long id = cs->condSpaceId; + cgd->newCondSpace( curCondSpace, id, cs->baseKey ); + for ( CondSet::Iter csi = cs->condSet; csi.lte(); csi++ ) + cgd->condSpaceItem( curCondSpace, (*csi)->actionId ); + curCondSpace += 1; + } + } +} + +bool BackendGen::makeNameInst( std::string &res, NameInst *nameInst ) +{ + bool written = false; + if ( nameInst->parent != 0 ) + written = makeNameInst( res, nameInst->parent ); + + if ( nameInst->name != 0 ) { + if ( written ) + res += '_'; + res += nameInst->name; + written = true; + } + + return written; +} + +void BackendGen::makeEntryPoints() +{ + /* List of entry points other than start state. */ + if ( fsm->entryPoints.length() > 0 || pd->lmRequiresErrorState ) { + if ( pd->lmRequiresErrorState ) + cgd->setForcedErrorState(); + + for ( EntryMap::Iter en = fsm->entryPoints; en.lte(); en++ ) { + /* Get the name instantiation from nameIndex. */ + NameInst *nameInst = pd->nameIndex[en->key]; + std::string name; + makeNameInst( name, nameInst ); + StateAp *state = en->value; + cgd->addEntryPoint( strdup(name.c_str()), state->alg.stateNum ); + } + } +} + +void BackendGen::makeStateActions( StateAp *state ) +{ + RedActionTable *toStateActions = 0; + if ( state->toStateActionTable.length() > 0 ) + toStateActions = actionTableMap.find( state->toStateActionTable ); + + RedActionTable *fromStateActions = 0; + if ( state->fromStateActionTable.length() > 0 ) + fromStateActions = actionTableMap.find( state->fromStateActionTable ); + + /* EOF actions go out here only if the state has no eof target. If it has + * an eof target then an eof transition will be used instead. */ + RedActionTable *eofActions = 0; + if ( state->eofTarget == 0 && state->eofActionTable.length() > 0 ) + eofActions = actionTableMap.find( state->eofActionTable ); + + if ( toStateActions != 0 || fromStateActions != 0 || eofActions != 0 ) { + long to = -1; + if ( toStateActions != 0 ) + to = toStateActions->id; + + long from = -1; + if ( fromStateActions != 0 ) + from = fromStateActions->id; + + long eof = -1; + if ( eofActions != 0 ) + eof = eofActions->id; + + cgd->setStateActions( curState, to, from, eof ); + } +} + +void BackendGen::makeEofTrans( StateAp *state ) +{ + RedActionTable *eofActions = 0; + if ( state->eofActionTable.length() > 0 ) + eofActions = actionTableMap.find( state->eofActionTable ); + + /* The EOF trans is used when there is an eof target, otherwise the eof + * action goes into state actions. */ + if ( state->eofTarget != 0 ) { + long targ = state->eofTarget->alg.stateNum; + long action = -1; + if ( eofActions != 0 ) + action = eofActions->id; + + cgd->setEofTrans( curState, targ, action ); + } +} + +void BackendGen::makeStateConditions( StateAp *state ) +{ + if ( state->stateCondList.length() > 0 ) { + long length = state->stateCondList.length(); + cgd->initStateCondList( curState, length ); + curStateCond = 0; + + for ( StateCondList::Iter scdi = state->stateCondList; scdi.lte(); scdi++ ) { + cgd->addStateCond( curState, scdi->lowKey, scdi->highKey, + scdi->condSpace->condSpaceId ); + } + } +} + +void BackendGen::makeTrans( Key lowKey, Key highKey, TransAp *trans ) +{ + /* First reduce the action. */ + RedActionTable *actionTable = 0; + if ( trans->actionTable.length() > 0 ) + actionTable = actionTableMap.find( trans->actionTable ); + + long targ = -1; + if ( trans->toState != 0 ) + targ = trans->toState->alg.stateNum; + + long action = -1; + if ( actionTable != 0 ) + action = actionTable->id; + + cgd->newTrans( curState, curTrans++, lowKey, highKey, targ, action ); +} + +void BackendGen::makeTransList( StateAp *state ) +{ + TransListVect outList; + + /* If there is only are no ranges the task is simple. */ + if ( state->outList.length() > 0 ) { + /* Loop each source range. */ + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) { + /* Reduce the transition. If it reduced to anything then add it. */ + appendTrans( outList, trans->lowKey, trans->highKey, trans ); + } + } + + cgd->initTransList( curState, outList.length() ); + curTrans = 0; + + for ( TransListVect::Iter tvi = outList; tvi.lte(); tvi++ ) + makeTrans( tvi->lowKey, tvi->highKey, tvi->value ); + + cgd->finishTransList( curState ); +} + + +void BackendGen::makeStateList() +{ + /* Write the list of states. */ + long length = fsm->stateList.length(); + cgd->initStateList( length ); + curState = 0; + for ( StateList::Iter st = fsm->stateList; st.lte(); st++ ) { + makeStateActions( st ); + makeEofTrans( st ); + makeStateConditions( st ); + makeTransList( st ); + + long id = st->alg.stateNum; + cgd->setId( curState, id ); + + if ( st->isFinState() ) + cgd->setFinal( curState ); + + curState += 1; + } +} + + +void BackendGen::makeMachine() +{ + cgd->createMachine(); + + /* Action tables. */ + reduceActionTables(); + + makeActionList(); + makeActionTableList(); + makeConditions(); + + /* Start State. */ + cgd->setStartState( fsm->startState->alg.stateNum ); + + /* Error state. */ + if ( fsm->errState != 0 ) + cgd->setErrorState( fsm->errState->alg.stateNum ); + + makeEntryPoints(); + makeStateList(); + + cgd->closeMachine(); +} + +void BackendGen::close_ragel_def() +{ + /* Do this before distributing transitions out to singles and defaults + * makes life easier. */ + cgd->redFsm->maxKey = cgd->findMaxKey(); + + cgd->redFsm->assignActionLocs(); + + /* Find the first final state (The final state with the lowest id). */ + cgd->redFsm->findFirstFinState(); + + /* Call the user's callback. */ + cgd->finishRagelDef(); +} + + +void BackendGen::makeBackend() +{ + /* Alphabet type. */ + cgd->setAlphType( keyOps->alphType->internalName ); + + /* Getkey expression. */ + if ( pd->getKeyExpr != 0 ) { + cgd->getKeyExpr = new GenInlineList; + makeGenInlineList( cgd->getKeyExpr, pd->getKeyExpr ); + } + + /* Access expression. */ + if ( pd->accessExpr != 0 ) { + cgd->accessExpr = new GenInlineList; + makeGenInlineList( cgd->accessExpr, pd->accessExpr ); + } + + /* PrePush expression. */ + if ( pd->prePushExpr != 0 ) { + cgd->prePushExpr = new GenInlineList; + makeGenInlineList( cgd->prePushExpr, pd->prePushExpr ); + } + + /* PostPop expression. */ + if ( pd->postPopExpr != 0 ) { + cgd->postPopExpr = new GenInlineList; + makeGenInlineList( cgd->postPopExpr, pd->postPopExpr ); + } + + /* + * Variable expressions. + */ + + if ( pd->pExpr != 0 ) { + cgd->pExpr = new GenInlineList; + makeGenInlineList( cgd->pExpr, pd->pExpr ); + } + + if ( pd->peExpr != 0 ) { + cgd->peExpr = new GenInlineList; + makeGenInlineList( cgd->peExpr, pd->peExpr ); + } + + if ( pd->eofExpr != 0 ) { + cgd->eofExpr = new GenInlineList; + makeGenInlineList( cgd->eofExpr, pd->eofExpr ); + } + + if ( pd->csExpr != 0 ) { + cgd->csExpr = new GenInlineList; + makeGenInlineList( cgd->csExpr, pd->csExpr ); + } + + if ( pd->topExpr != 0 ) { + cgd->topExpr = new GenInlineList; + makeGenInlineList( cgd->topExpr, pd->topExpr ); + } + + if ( pd->stackExpr != 0 ) { + cgd->stackExpr = new GenInlineList; + makeGenInlineList( cgd->stackExpr, pd->stackExpr ); + } + + if ( pd->actExpr != 0 ) { + cgd->actExpr = new GenInlineList; + makeGenInlineList( cgd->actExpr, pd->actExpr ); + } + + if ( pd->tokstartExpr != 0 ) { + cgd->tokstartExpr = new GenInlineList; + makeGenInlineList( cgd->tokstartExpr, pd->tokstartExpr ); + } + + if ( pd->tokendExpr != 0 ) { + cgd->tokendExpr = new GenInlineList; + makeGenInlineList( cgd->tokendExpr, pd->tokendExpr ); + } + + if ( pd->dataExpr != 0 ) { + cgd->dataExpr = new GenInlineList; + makeGenInlineList( cgd->dataExpr, pd->dataExpr ); + } + + makeExports(); + makeMachine(); + + close_ragel_def(); +} + +void InputData::writeLanguage( std::ostream &out ) +{ + out << " lang=\""; + switch ( hostLang->lang ) { + case HostLang::C: out << "C"; break; + case HostLang::D: out << "D"; break; + case HostLang::Java: out << "Java"; break; + case HostLang::Ruby: out << "Ruby"; break; + case HostLang::CSharp: out << "C#"; break; + } + out << "\""; +} + +void InputData::writeXML( std::ostream &out ) +{ + out << "<ragel version=\"" VERSION "\" filename=\"" << inputFileName << "\""; + writeLanguage( out ); + out << ">\n"; + + for ( ParserDict::Iter parser = parserDict; parser.lte(); parser++ ) { + ParseData *pd = parser->value->pd; + if ( pd->instanceList.length() > 0 ) + pd->generateXML( *outStream ); + } + + out << "</ragel>\n"; +} diff --git a/ragel/xmlcodegen.h b/ragel/xmlcodegen.h index ab08bc2..d20d6ea 100644 --- a/ragel/xmlcodegen.h +++ b/ragel/xmlcodegen.h @@ -1,5 +1,5 @@ /* - * Copyright 2005, 2006 Adrian Thurston <thurston@cs.queensu.ca> + * Copyright 2005-2007 Adrian Thurston <thurston@complang.org> */ /* This file is part of Ragel. @@ -19,18 +19,21 @@ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ -#ifndef _XMLDOTGEN_H -#define _XMLDOTGEN_H +#ifndef _XMLCODEGEN_H +#define _XMLCODEGEN_H #include <iostream> #include "avltree.h" #include "fsmgraph.h" #include "parsedata.h" +#include "redfsm.h" /* Forwards. */ struct TransAp; struct FsmAp; struct ParseData; +struct GenInlineList; +struct CodeGenData; struct RedActionTable : @@ -76,60 +79,114 @@ struct NextRedTrans } }; -class XMLCodeGen +struct GenBase +{ + GenBase( char *fsmName, ParseData *pd, FsmAp *fsm ); + + void appendTrans( TransListVect &outList, Key lowKey, Key highKey, TransAp *trans ); + void reduceActionTables(); + + char *fsmName; + ParseData *pd; + FsmAp *fsm; + + ActionTableMap actionTableMap; + int nextActionTableId; +}; + +class XMLCodeGen : protected GenBase { public: XMLCodeGen( char *fsmName, ParseData *pd, FsmAp *fsm, std::ostream &out ); + void writeXML( ); private: - void appendTrans( TransListVect &outList, Key lowKey, Key highKey, TransAp *trans ); void writeStateActions( StateAp *state ); void writeStateList(); void writeStateConditions( StateAp *state ); void writeKey( Key key ); void writeText( InlineItem *item ); - void writeCtrlFlow( InlineItem *item, InlineItem *context ); - void writePtrMod( InlineItem *item, InlineItem *context ); - void writeGoto( InlineItem *item, InlineItem *context ); - void writeGotoExpr( InlineItem *item, InlineItem *context ); - void writeCall( InlineItem *item, InlineItem *context ); - void writeCallExpr( InlineItem *item, InlineItem *context ); - void writeNext( InlineItem *item, InlineItem *context ); - void writeNextExpr( InlineItem *item, InlineItem *context ); + void writeGoto( InlineItem *item ); + void writeGotoExpr( InlineItem *item ); + void writeCall( InlineItem *item ); + void writeCallExpr( InlineItem *item ); + void writeNext( InlineItem *item ); + void writeNextExpr( InlineItem *item ); void writeEntry( InlineItem *item ); - void writeLmSetActId( InlineItem *item ); void writeLmOnLast( InlineItem *item ); void writeLmOnNext( InlineItem *item ); void writeLmOnLagBehind( InlineItem *item ); + void writeExports(); + bool writeNameInst( NameInst *nameInst ); void writeEntryPoints(); - void writeGetKeyExpr(); - void writeAccessExpr(); - void writeCurStateExpr(); void writeConditions(); - void writeInlineList( InlineList *inlineList, InlineItem *context ); - void writeAlphType(); + void writeInlineList( InlineList *inlineList ); void writeActionList(); void writeActionTableList(); void reduceTrans( TransAp *trans ); - void reduceActionTables(); void writeTransList( StateAp *state ); + void writeEofTrans( StateAp *state ); void writeTrans( Key lowKey, Key highKey, TransAp *defTrans ); void writeAction( Action *action ); void writeLmSwitch( InlineItem *item ); void writeMachine(); void writeActionExec( InlineItem *item ); - void writeActionExecTE( InlineItem *item ); - char *fsmName; - ParseData *pd; - FsmAp *fsm; std::ostream &out; - ActionTableMap actionTableMap; - int nextActionTableId; }; +class BackendGen : protected GenBase +{ +public: + BackendGen( char *fsmName, ParseData *pd, FsmAp *fsm, CodeGenData *cgd ); + void makeBackend( ); + +private: + void makeGenInlineList( GenInlineList *outList, InlineList *inList ); + void makeKey( GenInlineList *outList, Key key ); + void makeText( GenInlineList *outList, InlineItem *item ); + void makeLmOnLast( GenInlineList *outList, InlineItem *item ); + void makeLmOnNext( GenInlineList *outList, InlineItem *item ); + void makeLmOnLagBehind( GenInlineList *outList, InlineItem *item ); + void makeActionExec( GenInlineList *outList, InlineItem *item ); + void makeLmSwitch( GenInlineList *outList, InlineItem *item ); + void makeSetTokend( GenInlineList *outList, long offset ); + void makeSetAct( GenInlineList *outList, long lmId ); + void makeSubList( GenInlineList *outList, InlineList *inlineList, + GenInlineItem::Type type ); + void makeTargetItem( GenInlineList *outList, NameInst *nameTarg, GenInlineItem::Type type ); + void makeExecGetTokend( GenInlineList *outList ); + void makeExports(); + void makeMachine(); + void makeActionList(); + void makeAction( Action *action ); + void makeActionTableList(); + void makeConditions(); + void makeEntryPoints(); + bool makeNameInst( std::string &out, NameInst *nameInst ); + void makeStateList(); + + void makeStateActions( StateAp *state ); + void makeEofTrans( StateAp *state ); + void makeStateConditions( StateAp *state ); + void makeTransList( StateAp *state ); + void makeTrans( Key lowKey, Key highKey, TransAp *trans ); + + void close_ragel_def(); + + CodeGenData *cgd; + + /* Collected during parsing. */ + int curAction; + int curActionTable; + int curTrans; + int curState; + int curCondSpace; + int curStateCond; + +}; -#endif /* _XMLDOTGEN_H */ +#endif diff --git a/ragel/xmlparse.kh b/ragel/xmlparse.kh new file mode 100644 index 0000000..47b2758 --- /dev/null +++ b/ragel/xmlparse.kh @@ -0,0 +1,209 @@ +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ +#ifndef _XMLPARSE_H +#define _XMLPARSE_H + +#include "vector.h" +#include "gendata.h" +#include "buffer.h" +#include <iostream> + +using std::istream; +using std::ostream; + +#define XML_BUFSIZE 4096 + +struct AttrMarker +{ + char *id; + int idLen; + char *value; + int valueLen; +}; + +struct Attribute +{ + char *id; + char *value; +}; + +typedef Vector<AttrMarker> AttrMkList; +typedef Vector<Attribute> AttrList; +struct XMLTagHashPair; + +struct XMLTag +{ + enum TagType { Open, Close }; + + XMLTag( XMLTagHashPair *tagId, TagType type ) : + tagId(tagId), type(type), + content(0), attrList(0) {} + + Attribute *findAttr( const char *id ) + { + if ( attrList != 0 ) { + for ( AttrList::Iter attr = *attrList; attr.lte(); attr++ ) { + if ( strcmp( id, attr->id ) == 0 ) + return attr; + } + } + return 0; + } + + XMLTagHashPair *tagId; + TagType type; + + /* Content is associtated with closing tags. */ + char *content; + + /* Attribute lists are associated with opening tags. */ + AttrList *attrList; +}; + + +struct XMLTagHashPair +{ + const char *name; + int id; +}; + +struct Token; + +struct GenInlineItem; +struct GenInlineList; + +struct LmSwitchVect; +struct LmSwitchAction; + +struct XmlScanner +{ + XmlScanner( const char *fileName, istream &input ); + + int scan(); + void adjustAttrPointers( int distance ); + std::ostream &error(); + + const char *fileName; + istream &input; + + /* Scanner State. */ + int cs, act, have, curline, curcol; + char *ts, *te; + char *p, *pe; + int done; + + /* Token data */ + char *data; + int data_len; + int value; + AttrMkList attrMkList; + Buffer buffer; + char *tag_id_start; + int tag_id_len; + int token_col, token_line; + + char buf[XML_BUFSIZE]; +}; + + +struct XmlParser +{ + %%{ + parser XmlParser; + + token TAG_unknown, TAG_ragel, TAG_ragel_def, TAG_host, TAG_state_list, + TAG_state, TAG_trans_list, TAG_t, TAG_machine, TAG_start_state, + TAG_error_state, TAG_action_list, TAG_action_table_list, + TAG_action, TAG_action_table, TAG_alphtype, TAG_element, + TAG_getkey, TAG_state_actions, TAG_entry_points, TAG_sub_action, + TAG_cond_space_list, TAG_cond_space, TAG_cond_list, TAG_c, + TAG_exports, TAG_ex; + + # Inline block tokens. + token TAG_text, TAG_goto, TAG_call, TAG_next, TAG_goto_expr, + TAG_call_expr, TAG_next_expr, TAG_ret, TAG_pchar, TAG_char, + TAG_hold, TAG_exec, TAG_curs, TAG_targs, TAG_entry, TAG_data, + TAG_lm_switch, TAG_init_act, TAG_set_act, TAG_set_tokend, + TAG_get_tokend, TAG_init_tokstart, TAG_set_tokstart; + + token TAG_write, TAG_access, TAG_break, TAG_arg, TAG_cs_expr; + + token TAG_p_expr, TAG_pe_expr, TAG_eof_expr, TAG_cs_expr, TAG_top_expr, + TAG_stack_expr, TAG_act_expr, TAG_tokstart_expr, TAG_tokend_expr, + TAG_data_expr, TAG_prepush, TAG_postpop, TAG_eof_t; + }%% + + %% write instance_data; + + void init(); + int parseLangEl( int type, const Token *token ); + + XmlParser( const char *sourceFileName, const char *xmlFileName, bool outputActive, bool wantComplete ) : + sourceFileName(sourceFileName), + fileName(xmlFileName), + outStream(0), + outputActive(outputActive), + wantComplete(wantComplete), + cgd(0) { } + + int token( int tokenId, Token &token ); + int token( int tokenId, int col, int line ); + int token( XMLTag *tag, int col, int line ); + + void openOutput(); + + /* Report an error encountered by the parser. */ + ostream &warning( const InputLoc &loc ); + ostream &error(); + ostream &error( const InputLoc &loc ); + ostream &parser_error( int tokId, Token &token ); + ostream &source_error( const InputLoc &loc ); + + /* The name of the root section, this does not change during an include. */ + const char *sourceFileName; + const char *fileName; + ostream *outStream; + bool outputActive; + bool wantComplete; + + /* Collected during parsing. */ + char *attrKey; + char *attrValue; + int curAction; + int curActionTable; + int curTrans; + int curState; + int curCondSpace; + int curStateCond; + + CodeGenData *cgd; + CodeGenMap codeGenMap; + + Vector <char*> writeOptions; +}; + +%% write token_defs; + +int xml_parse( std::istream &input, const char *fileName, + bool outputActive, bool wantComplete, + XmlScanner &scanner, XmlParser &parser ); + +#endif diff --git a/ragel/xmlparse.kl b/ragel/xmlparse.kl new file mode 100644 index 0000000..2dc572d --- /dev/null +++ b/ragel/xmlparse.kl @@ -0,0 +1,1006 @@ +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "xmlparse.h" +#include "common.h" +#include "gendata.h" +#include "version.h" +#include <iostream> +#include <stdlib.h> + +using std::cout; +using std::ostream; +using std::istream; +using std::cerr; +using std::endl; + +Key readKey( char *td, char **end ); +long readOffsetPtr( char *td, char **end ); +unsigned long readLength( char *td ); + +struct Token +{ + XMLTag *tag; + InputLoc loc; +}; + +%%{ + +parser XmlParser; + +include "xmlparse.kh"; + +start: tag_ragel; +start: + final { + /* If we get no input the assumption is that the frontend died and + * emitted an error. This forces the backend to return a non-zero + * exit status, but does not print an error. */ + gblErrorCount += 1; + }; + +tag_ragel: tag_ragel_head ragel_def_list host_or_write_list '/' TAG_ragel; + +tag_ragel_head: TAG_ragel + final { + /* Check version used to generated the intermediate file. */ + Attribute *versionAttr = $1->tag->findAttr( "version" ); + if ( versionAttr == 0 ) + error($1->loc) << "tag <ragel> requires a version attribute" << endp; + if ( strcmp( versionAttr->value, VERSION ) != 0 ) + error($1->loc) << "version mismatch between frontend and backend" << endp; + + /* Check for file name attribute. */ + Attribute *fileNameAttr = $1->tag->findAttr( "filename" ); + if ( fileNameAttr == 0 ) + error($1->loc) << "tag <ragel> requires a filename attribute" << endp; + sourceFileName = fileNameAttr->value; + + /* Check for language attribute. */ + Attribute *langAttr = $1->tag->findAttr( "lang" ); + if ( langAttr == 0 ) + error($1->loc) << "tag <ragel> requires a lang attribute" << endp; + + if ( generateDot ) + outStream = dotOpenOutput( sourceFileName ); + else if ( strcmp( langAttr->value, "C" ) == 0 ) { + hostLang = &hostLangC; + outStream = cdOpenOutput( sourceFileName ); + } + else if ( strcmp( langAttr->value, "D" ) == 0 ) { + hostLang = &hostLangD; + outStream = cdOpenOutput( sourceFileName ); + } + else if ( strcmp( langAttr->value, "Java" ) == 0 ) { + hostLang = &hostLangJava; + outStream = javaOpenOutput( sourceFileName ); + } + else if ( strcmp( langAttr->value, "Ruby" ) == 0 ) { + hostLang = &hostLangRuby; + outStream = rubyOpenOutput( sourceFileName ); + } + else if ( strcmp( langAttr->value, "C#" ) == 0 ) { + hostLang = &hostLangCSharp; + outStream = csharpOpenOutput( sourceFileName ); + } + else { + error($1->loc) << "expecting lang attribute to be " + "one of C, D, Java, Ruby or C#" << endp; + } + }; + +ragel_def_list: ragel_def_list ragel_def; +ragel_def_list: ; + +host_or_write_list: host_or_write_list host_or_write; +host_or_write_list: ; + +host_or_write: tag_host; +host_or_write: tag_write; + +tag_host: + TAG_host '/' TAG_host + final { + Attribute *lineAttr = $1->tag->findAttr( "line" ); + if ( lineAttr == 0 ) + error($1->loc) << "tag <host> requires a line attribute" << endp; + else { + int line = atoi( lineAttr->value ); + if ( outputActive ) + lineDirective( *outStream, sourceFileName, line ); + } + + if ( outputActive ) + *outStream << $3->tag->content; + }; + +ragel_def: + tag_ragel_def_head ragel_def_item_list '/' TAG_ragel_def + final { + /* Do this before distributing transitions out to singles and defaults + * makes life easier. */ + cgd->redFsm->maxKey = cgd->findMaxKey(); + + cgd->redFsm->assignActionLocs(); + + /* Find the first final state (The final state with the lowest id). */ + cgd->redFsm->findFirstFinState(); + + /* Call the user's callback. */ + cgd->finishRagelDef(); + }; + +tag_ragel_def_head: TAG_ragel_def + final { + char *fsmName = 0; + Attribute *nameAttr = $1->tag->findAttr( "name" ); + if ( nameAttr != 0 ) { + fsmName = nameAttr->value; + + CodeGenMapEl *mapEl = codeGenMap.find( fsmName ); + if ( mapEl != 0 ) + cgd = mapEl->value; + else { + cgd = makeCodeGen( sourceFileName, fsmName, *outStream, wantComplete ); + codeGenMap.insert( fsmName, cgd ); + } + } + else { + cgd = makeCodeGen( sourceFileName, fsmName, + *outStream, wantComplete ); + } + + ::keyOps = &cgd->thisKeyOps; + }; + +ragel_def_item_list: ragel_def_item_list ragel_def_item; +ragel_def_item_list: ; + +ragel_def_item: tag_alph_type; +ragel_def_item: tag_getkey_expr; +ragel_def_item: tag_access_expr; +ragel_def_item: tag_prepush_expr; +ragel_def_item: tag_postpop_expr; +ragel_def_item: tag_export_list; +ragel_def_item: tag_machine; +ragel_def_item: tag_p_expr; +ragel_def_item: tag_pe_expr; +ragel_def_item: tag_eof_expr; +ragel_def_item: tag_cs_expr; +ragel_def_item: tag_top_expr; +ragel_def_item: tag_stack_expr; +ragel_def_item: tag_act_expr; +ragel_def_item: tag_tokstart_expr; +ragel_def_item: tag_tokend_expr; +ragel_def_item: tag_data_expr; + +tag_export_list: TAG_exports export_list '/' TAG_exports; + +export_list: export_list tag_export; +export_list: ; + +tag_export: TAG_ex '/' TAG_ex + final { + Attribute *nameAttr = $1->tag->findAttr( "name" ); + if ( nameAttr == 0 ) + error($1->loc) << "tag <ex> requires a name attribute" << endp; + else { + char *td = $3->tag->content; + Key exportKey = readKey( td, &td ); + cgd->exportList.append( new Export( nameAttr->value, exportKey ) ); + } + }; + +tag_alph_type: TAG_alphtype '/' TAG_alphtype + final { + if ( ! cgd->setAlphType( $3->tag->content ) ) + error($1->loc) << "tag <alphtype> specifies unknown alphabet type" << endp; + }; + +tag_getkey_expr: TAG_getkey inline_list '/' TAG_getkey + final { + cgd->getKeyExpr = $2->inlineList; + }; + +tag_access_expr: TAG_access inline_list '/' TAG_access + final { + cgd->accessExpr = $2->inlineList; + }; + +tag_prepush_expr: TAG_prepush inline_list '/' TAG_prepush + final { + cgd->prePushExpr = $2->inlineList; + }; + +tag_postpop_expr: TAG_postpop inline_list '/' TAG_postpop + final { + cgd->postPopExpr = $2->inlineList; + }; + +tag_p_expr: TAG_p_expr inline_list '/' TAG_p_expr + final { cgd->pExpr = $2->inlineList; }; +tag_pe_expr: TAG_pe_expr inline_list '/' TAG_pe_expr + final { cgd->peExpr = $2->inlineList; }; +tag_eof_expr: TAG_eof_expr inline_list '/' TAG_eof_expr + final { cgd->eofExpr = $2->inlineList; }; +tag_cs_expr: TAG_cs_expr inline_list '/' TAG_cs_expr + final { cgd->csExpr = $2->inlineList; }; +tag_top_expr: TAG_top_expr inline_list '/' TAG_top_expr + final { cgd->topExpr = $2->inlineList; }; +tag_stack_expr: TAG_stack_expr inline_list '/' TAG_stack_expr + final { cgd->stackExpr = $2->inlineList; }; +tag_act_expr: TAG_act_expr inline_list '/' TAG_act_expr + final { cgd->actExpr = $2->inlineList; }; +tag_tokstart_expr: TAG_tokstart_expr inline_list '/' TAG_tokstart_expr + final { cgd->tokstartExpr = $2->inlineList; }; +tag_tokend_expr: TAG_tokend_expr inline_list '/' TAG_tokend_expr + final { cgd->tokendExpr = $2->inlineList; }; +tag_data_expr: TAG_data_expr inline_list '/' TAG_data_expr + final { cgd->dataExpr = $2->inlineList; }; + + +tag_write: tag_write_head write_option_list '/' TAG_write + final { + /* Terminate the options list and call the write statement handler. */ + writeOptions.append(0); + cgd->writeStatement( $1->loc, writeOptions.length()-1, writeOptions.data ); + + /* Clear the options in prep for the next write statement. */ + writeOptions.empty(); + }; + +nonterm tag_write_head +{ + InputLoc loc; +}; + +tag_write_head: TAG_write + final { + Attribute *nameAttr = $1->tag->findAttr( "def_name" ); + Attribute *lineAttr = $1->tag->findAttr( "line" ); + Attribute *colAttr = $1->tag->findAttr( "col" ); + + if ( nameAttr == 0 ) + error($1->loc) << "tag <write> requires a def_name attribute" << endp; + if ( lineAttr == 0 ) + error($1->loc) << "tag <write> requires a line attribute" << endp; + if ( colAttr == 0 ) + error($1->loc) << "tag <write> requires a col attribute" << endp; + + if ( nameAttr != 0 && lineAttr != 0 && colAttr != 0 ) { + $$->loc.line = atoi(lineAttr->value); + $$->loc.col = atoi(colAttr->value); + + CodeGenMapEl *mapEl = codeGenMap.find( nameAttr->value ); + if ( mapEl == 0 ) { + source_error($$->loc) << "write statement given " + "but there are no machine instantiations" << endp; + } + else { + cgd = mapEl->value; + ::keyOps = &cgd->thisKeyOps; + } + } + }; + + +write_option_list: write_option_list tag_arg; +write_option_list: ; + +nonterm tag_arg +{ + char *option; +}; + +tag_arg: TAG_arg '/' TAG_arg + final { + writeOptions.append( $3->tag->content ); + }; + +tag_machine: tag_machine_head machine_item_list '/' TAG_machine + final { + cgd->closeMachine(); + }; + +tag_machine_head: TAG_machine + final { + cgd->createMachine(); + }; + +machine_item_list: machine_item_list machine_item; +machine_item_list: ; + +machine_item: tag_start_state; +machine_item: tag_error_state; +machine_item: tag_entry_points; +machine_item: tag_state_list; +machine_item: tag_action_list; +machine_item: tag_action_table_list; +machine_item: tag_cond_space_list; + +# +# States. +# + +tag_start_state: TAG_start_state '/' TAG_start_state + final { + unsigned long startState = strtoul( $3->tag->content, 0, 10 ); + cgd->setStartState( startState ); + }; + +tag_error_state: TAG_error_state '/' TAG_error_state + final { + unsigned long errorState = strtoul( $3->tag->content, 0, 10 ); + cgd->setErrorState( errorState ); + }; + +tag_entry_points: TAG_entry_points entry_point_list '/' TAG_entry_points + final { + Attribute *errorAttr = $1->tag->findAttr( "error" ); + if ( errorAttr != 0 ) + cgd->setForcedErrorState(); + }; + +entry_point_list: entry_point_list tag_entry; +entry_point_list: ; + +tag_entry: TAG_entry '/' TAG_entry + final { + Attribute *nameAttr = $1->tag->findAttr( "name" ); + if ( nameAttr == 0 ) { + error($1->loc) << "tag <entry_points>::<entry> " + "requires a name attribute" << endp; + } + else { + char *data = $3->tag->content; + unsigned long entry = strtoul( data, &data, 10 ); + cgd->addEntryPoint( nameAttr->value, entry ); + } + }; + +tag_state_list: tag_state_list_head state_list '/' TAG_state_list; + +tag_state_list_head: TAG_state_list + final { + Attribute *lengthAttr = $1->tag->findAttr( "length" ); + if ( lengthAttr == 0 ) + error($1->loc) << "tag <state_list> requires a length attribute" << endp; + else { + unsigned long length = strtoul( lengthAttr->value, 0, 10 ); + cgd->initStateList( length ); + curState = 0; + } + }; + +state_list: state_list tag_state; +state_list: ; + +tag_state: TAG_state state_item_list '/' TAG_state + final { + Attribute *idAttr = $1->tag->findAttr( "id" ); + if ( idAttr == 0 ) + error($1->loc) << "tag <state> requires an id attribute" << endp; + else { + int id = atoi( idAttr->value ); + cgd->setId( curState, id ); + } + + Attribute *lengthAttr = $1->tag->findAttr( "final" ); + if ( lengthAttr != 0 ) + cgd->setFinal( curState ); + curState += 1; + }; + +state_item_list: state_item_list state_item; +state_item_list: ; + +state_item: tag_state_actions; +state_item: tag_eof_t; +state_item: tag_state_cond_list; +state_item: tag_trans_list; + +tag_state_actions: TAG_state_actions '/' TAG_state_actions + final { + char *ad = $3->tag->content; + + long toStateAction = readOffsetPtr( ad, &ad ); + long fromStateAction = readOffsetPtr( ad, &ad ); + long eofAction = readOffsetPtr( ad, &ad ); + + cgd->setStateActions( curState, toStateAction, + fromStateAction, eofAction ); + }; + +tag_eof_t: TAG_eof_t '/' TAG_eof_t + final { + char *et = $3->tag->content; + long targ = readOffsetPtr( et, &et ); + long eofAction = readOffsetPtr( et, &et ); + + cgd->setEofTrans( curState, targ, eofAction ); + }; + +tag_state_cond_list: tag_state_cond_list_head state_cond_list '/' TAG_cond_list; + +tag_state_cond_list_head: TAG_cond_list + final { + Attribute *lengthAttr = $1->tag->findAttr( "length" ); + if ( lengthAttr == 0 ) + error($1->loc) << "tag <cond_list> requires a length attribute" << endp; + else { + ulong length = readLength( lengthAttr->value ); + cgd->initStateCondList( curState, length ); + curStateCond = 0; + } + }; + +state_cond_list: state_cond_list state_cond; +state_cond_list: ; + +state_cond: TAG_c '/' TAG_c + final { + char *td = $3->tag->content; + Key lowKey = readKey( td, &td ); + Key highKey = readKey( td, &td ); + long condId = readOffsetPtr( td, &td ); + cgd->addStateCond( curState, lowKey, highKey, condId ); + }; + +tag_trans_list: tag_trans_list_head trans_list '/' TAG_trans_list + final { + cgd->finishTransList( curState ); + }; + +tag_trans_list_head: TAG_trans_list + final { + Attribute *lengthAttr = $1->tag->findAttr( "length" ); + if ( lengthAttr == 0 ) + error($1->loc) << "tag <trans_list> requires a length attribute" << endp; + else { + unsigned long length = strtoul( lengthAttr->value, 0, 10 ); + cgd->initTransList( curState, length ); + curTrans = 0; + } + }; + +trans_list: trans_list tag_trans; +trans_list: ; + +tag_trans: TAG_t '/' TAG_t + final { + char *td = $3->tag->content; + Key lowKey = readKey( td, &td ); + Key highKey = readKey( td, &td ); + long targ = readOffsetPtr( td, &td ); + long action = readOffsetPtr( td, &td ); + + cgd->newTrans( curState, curTrans++, lowKey, highKey, targ, action ); + }; + +# +# Action Lists. +# + +tag_action_list: tag_action_list_head action_list '/' TAG_action_list; + +tag_action_list_head: TAG_action_list + final { + Attribute *lengthAttr = $1->tag->findAttr( "length" ); + if ( lengthAttr == 0 ) + error($1->loc) << "tag <action_list> requires a length attribute" << endp; + else { + unsigned long length = strtoul( lengthAttr->value, 0, 10 ); + cgd->initActionList( length ); + curAction = 0; + } + }; + +action_list: action_list tag_action; +action_list: ; + +# +# Actions. +# + +tag_action: TAG_action inline_list '/' TAG_action + final { + Attribute *lineAttr = $1->tag->findAttr( "line" ); + Attribute *colAttr = $1->tag->findAttr( "col" ); + Attribute *nameAttr = $1->tag->findAttr( "name" ); + if ( lineAttr == 0 || colAttr == 0) + error($1->loc) << "tag <action> requires a line and col attributes" << endp; + else { + unsigned long line = strtoul( lineAttr->value, 0, 10 ); + unsigned long col = strtoul( colAttr->value, 0, 10 ); + + char *name = 0; + if ( nameAttr != 0 ) + name = nameAttr->value; + + cgd->newAction( curAction++, name, line, col, $2->inlineList ); + } + }; + +nonterm inline_list +{ + GenInlineList *inlineList; +}; + + +inline_list: inline_list inline_item + final { + /* Append the item to the list, return the list. */ + $1->inlineList->append( $2->inlineItem ); + $$->inlineList = $1->inlineList; + }; + +inline_list: + final { + /* Start with empty list. */ + $$->inlineList = new GenInlineList; + }; + +nonterm inline_item_type +{ + GenInlineItem *inlineItem; +}; + +nonterm inline_item uses inline_item_type; + +inline_item: tag_text final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_goto final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_call final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_next final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_goto_expr final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_call_expr final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_next_expr final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_ret final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_break final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_pchar final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_char final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_hold final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_exec final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_curs final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_targs final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_il_entry final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_init_tokstart final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_init_act final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_get_tokend final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_set_tokstart final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_set_tokend final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_set_act final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_sub_action final { $$->inlineItem = $1->inlineItem; }; +inline_item: tag_lm_switch final { $$->inlineItem = $1->inlineItem; }; + +nonterm tag_text uses inline_item_type; +nonterm tag_goto uses inline_item_type; +nonterm tag_call uses inline_item_type; +nonterm tag_next uses inline_item_type; +nonterm tag_goto_expr uses inline_item_type; +nonterm tag_call_expr uses inline_item_type; +nonterm tag_next_expr uses inline_item_type; +nonterm tag_ret uses inline_item_type; +nonterm tag_break uses inline_item_type; +nonterm tag_pchar uses inline_item_type; +nonterm tag_char uses inline_item_type; +nonterm tag_hold uses inline_item_type; +nonterm tag_exec uses inline_item_type; +nonterm tag_curs uses inline_item_type; +nonterm tag_targs uses inline_item_type; +nonterm tag_il_entry uses inline_item_type; +nonterm tag_init_tokstart uses inline_item_type; +nonterm tag_init_act uses inline_item_type; +nonterm tag_get_tokend uses inline_item_type; +nonterm tag_set_tokstart uses inline_item_type; +nonterm tag_set_tokend uses inline_item_type; +nonterm tag_set_act uses inline_item_type; +nonterm tag_sub_action uses inline_item_type; +nonterm tag_lm_switch uses inline_item_type; + +tag_text: TAG_text '/' TAG_text + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::Text ); + $$->inlineItem->data = $3->tag->content; + }; + +tag_goto: TAG_goto '/' TAG_goto + final { + int targ = strtol( $3->tag->content, 0, 10 ); + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::Goto ); + $$->inlineItem->targId = targ; + }; + +tag_call: TAG_call '/' TAG_call + final { + int targ = strtol( $3->tag->content, 0, 10 ); + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::Call ); + $$->inlineItem->targId = targ; + }; + +tag_next: TAG_next '/' TAG_next + final { + int targ = strtol( $3->tag->content, 0, 10 ); + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::Next ); + $$->inlineItem->targId = targ; + }; + +tag_goto_expr: TAG_goto_expr inline_list '/' TAG_goto_expr + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::GotoExpr ); + $$->inlineItem->children = $2->inlineList; + }; + +tag_call_expr: TAG_call_expr inline_list '/' TAG_call_expr + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::CallExpr ); + $$->inlineItem->children = $2->inlineList; + }; + +tag_next_expr: TAG_next_expr inline_list '/' TAG_next_expr + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::NextExpr ); + $$->inlineItem->children = $2->inlineList; + }; + +tag_ret: TAG_ret '/' TAG_ret + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::Ret ); + }; + +tag_break: TAG_break '/' TAG_break + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::Break ); + }; + +tag_pchar: TAG_pchar '/' TAG_pchar + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::PChar ); + }; + +tag_char: TAG_char '/' TAG_char + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::Char ); + }; + +tag_hold: TAG_hold '/' TAG_hold + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::Hold ); + }; + +tag_exec: TAG_exec inline_list '/' TAG_exec + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::Exec ); + $$->inlineItem->children = $2->inlineList; + }; + +tag_curs: TAG_curs '/' TAG_curs + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::Curs ); + }; + +tag_targs: TAG_targs '/' TAG_targs + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::Targs ); + }; + +tag_il_entry: TAG_entry '/' TAG_entry + final { + int targ = strtol( $3->tag->content, 0, 10 ); + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::Entry ); + $$->inlineItem->targId = targ; + }; + +tag_init_tokstart: TAG_init_tokstart '/' TAG_init_tokstart + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::LmInitTokStart ); + }; + +tag_init_act: TAG_init_act '/' TAG_init_act + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::LmInitAct ); + }; + +tag_get_tokend: TAG_get_tokend '/' TAG_get_tokend + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::LmGetTokEnd ); + }; + +tag_set_tokstart: TAG_set_tokstart '/' TAG_set_tokstart + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::LmSetTokStart ); + cgd->hasLongestMatch = true; + }; + +tag_set_tokend: TAG_set_tokend '/' TAG_set_tokend + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::LmSetTokEnd ); + $$->inlineItem->offset = strtol( $3->tag->content, 0, 10 ); + }; + +tag_set_act: TAG_set_act '/' TAG_set_act + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::LmSetActId ); + $$->inlineItem->lmId = strtol( $3->tag->content, 0, 10 ); + }; + +tag_sub_action: TAG_sub_action inline_list '/' TAG_sub_action + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::SubAction ); + $$->inlineItem->children = $2->inlineList; + }; + +# Action switches. +tag_lm_switch: TAG_lm_switch lm_action_list '/' TAG_lm_switch + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::LmSwitch ); + $$->inlineItem->children = $2->inlineList; + }; + +nonterm lm_action_list +{ + GenInlineList *inlineList; +}; + +lm_action_list: lm_action_list tag_inline_action + final { + $$->inlineList = $1->inlineList; + $$->inlineList->append( $2->inlineItem ); + }; +lm_action_list: + final { + $$->inlineList = new GenInlineList; + }; + +nonterm tag_inline_action uses inline_item_type; + +tag_inline_action: TAG_sub_action inline_list '/' TAG_sub_action + final { + $$->inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::SubAction ); + $$->inlineItem->children = $2->inlineList; + + Attribute *idAttr = $1->tag->findAttr( "id" ); + if ( idAttr != 0 ) { + unsigned long id = strtoul( idAttr->value, 0, 10 ); + $$->inlineItem->lmId = id; + } + }; + +# +# Lists of Actions. +# + +tag_action_table_list: + tag_action_table_list_head action_table_list '/' TAG_action_table_list; + +tag_action_table_list_head: TAG_action_table_list + final { + Attribute *lengthAttr = $1->tag->findAttr( "length" ); + if ( lengthAttr == 0 ) { + error($1->loc) << "tag <action_table_list> requires " + "a length attribute" << endp; + } + else { + unsigned long length = strtoul( lengthAttr->value, 0, 10 ); + cgd->initActionTableList( length ); + curActionTable = 0; + } + }; + +action_table_list: action_table_list tag_action_table; +action_table_list: ; + +tag_action_table: TAG_action_table '/' TAG_action_table + final { + /* Find the length of the action table. */ + Attribute *lengthAttr = $1->tag->findAttr( "length" ); + if ( lengthAttr == 0 ) + error($1->loc) << "tag <at> requires a length attribute" << endp; + else { + unsigned long length = strtoul( lengthAttr->value, 0, 10 ); + + /* Collect the action table. */ + RedAction *redAct = cgd->allActionTables + curActionTable; + redAct->actListId = curActionTable; + redAct->key.setAsNew( length ); + char *ptr = $3->tag->content; + int pos = 0; + while ( *ptr != 0 ) { + unsigned long actionId = strtoul( ptr, &ptr, 10 ); + redAct->key[pos].key = 0; + redAct->key[pos].value = cgd->allActions+actionId; + pos += 1; + } + + /* Insert into the action table map. */ + cgd->redFsm->actionMap.insert( redAct ); + } + + curActionTable += 1; + }; + +# +# Conditions. +# + +tag_cond_space_list: tag_cond_space_list_head cond_space_list '/' TAG_cond_space_list; + +tag_cond_space_list_head: TAG_cond_space_list + final { + Attribute *lengthAttr = $1->tag->findAttr( "length" ); + if ( lengthAttr == 0 ) { + error($1->loc) << "tag <cond_space_list> " + "requires a length attribute" << endp; + } + else { + ulong length = readLength( lengthAttr->value ); + cgd->initCondSpaceList( length ); + curCondSpace = 0; + } + }; + +cond_space_list: cond_space_list tag_cond_space; +cond_space_list: tag_cond_space; + +tag_cond_space: TAG_cond_space '/' TAG_cond_space + final { + Attribute *lengthAttr = $1->tag->findAttr( "length" ); + Attribute *idAttr = $1->tag->findAttr( "id" ); + if ( lengthAttr == 0 ) + error($1->loc) << "tag <cond_space> requires a length attribute" << endp; + else { + if ( lengthAttr == 0 ) + error($1->loc) << "tag <cond_space> requires an id attribute" << endp; + else { + unsigned long condSpaceId = strtoul( idAttr->value, 0, 10 ); + ulong length = readLength( lengthAttr->value ); + + char *td = $3->tag->content; + Key baseKey = readKey( td, &td ); + + cgd->newCondSpace( curCondSpace, condSpaceId, baseKey ); + for ( ulong a = 0; a < length; a++ ) { + long actionOffset = readOffsetPtr( td, &td ); + cgd->condSpaceItem( curCondSpace, actionOffset ); + } + curCondSpace += 1; + } + } + }; + +}%% + +%%{ + write types; + write data; +}%% + +void XmlParser::init() +{ + %% write init; +} + +int XmlParser::parseLangEl( int type, const Token *token ) +{ + %% write exec; + return errCount == 0 ? 0 : -1; +} + + +unsigned long readLength( char *td ) +{ + return strtoul( td, 0, 10 ); +} + +Key readKey( char *td, char **end ) +{ + if ( keyOps->isSigned ) + return Key( strtol( td, end, 10 ) ); + else + return Key( strtoul( td, end, 10 ) ); +} + +long readOffsetPtr( char *td, char **end ) +{ + while ( *td == ' ' || *td == '\t' ) + td++; + + if ( *td == 'x' ) { + if ( end != 0 ) + *end = td + 1; + return -1; + } + + return strtol( td, end, 10 ); +} + +ostream &XmlParser::warning( const InputLoc &loc ) +{ + cerr << fileName << ":" << loc.line << ":" << loc.col << ": warning: "; + return cerr; +} + +ostream &XmlParser::error( const InputLoc &loc ) +{ + gblErrorCount += 1; + assert( fileName != 0 ); + cerr << fileName << ":" << loc.line << ":" << loc.col << ": "; + return cerr; +} + + +ostream &XmlParser::parser_error( int tokId, Token &token ) +{ + gblErrorCount += 1; + assert( fileName != 0 ); + cerr << fileName << ":" << token.loc.line << ":" << token.loc.col; + if ( token.tag != 0 ) { + if ( token.tag->tagId == 0 ) + cerr << ": at unknown tag"; + else + cerr << ": at tag <" << token.tag->tagId->name << ">"; + } + cerr << ": "; + + return cerr; +} + +ostream &XmlParser::source_error( const InputLoc &loc ) +{ + gblErrorCount += 1; + assert( sourceFileName != 0 ); + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": "; + return cerr; +} + + +int XmlParser::token( int tokenId, Token &tok ) +{ + int res = parseLangEl( tokenId, &tok ); + if ( res < 0 ) + parser_error( tokenId, tok ) << "parse error" << endp; + return res; +} + +int XmlParser::token( int tokenId, int col, int line ) +{ + Token tok; + tok.loc.col = col; + tok.loc.line = line; + tok.tag = 0; + return token( tokenId, tok ); +} + +int XmlParser::token( XMLTag *tag, int col, int line ) +{ + Token tok; + tok.loc.col = col; + tok.loc.line = line; + tok.tag = tag; + + if ( tag->type == XMLTag::Close ) { + int res = token( '/', tok ); + if ( res < 0 ) + return res; + } + + tok.tag = tag; + return token( tag->tagId != 0 ? tag->tagId->id : TAG_unknown, tok ); +} diff --git a/ragel/xmlscan.rl b/ragel/xmlscan.rl new file mode 100644 index 0000000..e217183 --- /dev/null +++ b/ragel/xmlscan.rl @@ -0,0 +1,313 @@ +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ +#include <iostream> +#include <string.h> +#include "vector.h" +#include "xmlparse.h" + +using std::istream; +using std::cout; +using std::cerr; +using std::endl; + +%%{ + machine XmlScanner; + write data; +}%% + +class Perfect_Hash +{ +private: + static inline unsigned int hash (const char *str, unsigned int len); + +public: + static struct XMLTagHashPair *in_word_set (const char *str, unsigned int len); +}; + +XmlScanner::XmlScanner( const char *fileName, istream &input ) : + fileName(fileName), + input(input), + curline(1), + curcol(1), + p(0), pe(0), + done(false), + data(0), data_len(0), + value(0) +{ + %%{ + machine XmlScanner; + write init; + }%% +} + +#define TK_NO_TOKEN (-1) +#define TK_ERR 1 +#define TK_SPACE 2 +#define TK_EOF 3 +#define TK_OpenTag 4 +#define TK_CloseTag 5 + +#define ret_tok( _tok ) token = (_tok); data = ts + +void XmlScanner::adjustAttrPointers( int distance ) +{ + for ( AttrMkList::Iter attr = attrMkList; attr.lte(); attr++ ) { + attr->id -= distance; + attr->value -= distance; + } +} + +/* There is no claim that this is a proper XML parser, but it is good + * enough for our purposes. */ +%%{ + machine XmlScanner; + + action colup { curcol++; } + action start_tok { token_col = curcol; token_line = curline; } + NL = '\n' @{ curcol = 0; curline++; }; + + WS = [\r\t ] | NL; + id = [_a-zA-Z][_a-zA-Z0-9]*; + literal = '"' ( [^"] | NL )* '"'; + + # Attribute identifiers. + action start_attr_id { attr_id_start = p; } + action leave_attr_id { attr_id_len = p - attr_id_start; } + + attr_id = id >start_attr_id %leave_attr_id; + + # Attribute values + action start_attr_value { attr_value_start = p; } + action leave_attr_value + { + attr_value_len = p - attr_value_start; + + AttrMarker newAttr; + newAttr.id = attr_id_start; + newAttr.idLen = attr_id_len; + newAttr.value = attr_value_start; + newAttr.valueLen = attr_value_len; + attrMkList.append( newAttr ); + } + + attr_value = literal >start_attr_value %leave_attr_value; + + # Attribute list. + attribute = attr_id WS* '=' WS* attr_value WS*; + + # Tag identifiers. + action tag_id_start { tag_id_start = p; } + action leave_tag_id { tag_id_len = p - tag_id_start; } + + tag_id = id >tag_id_start %leave_tag_id; + + main := |* + # Tags + ( '<' WS* tag_id ( WS+ attribute* )? '>' ) >start_tok $colup + => { ret_tok( TK_OpenTag ); fbreak; }; + + ( '<' WS* '/' WS* tag_id WS* '>' ) >start_tok $colup + => { ret_tok( TK_CloseTag ); fbreak; }; + + # Data in between tags. + ( [^<&\0] | NL ) $colup + => { buffer.append( *p ); }; + + # Specials. + "&" $colup + => { buffer.append( '&' ); }; + "<" $colup + => { buffer.append( '<' ); }; + ">" $colup + => { buffer.append( '>' ); }; + + # EOF + 0 >start_tok => { ret_tok( TK_EOF ); fbreak; }; + + *|; +}%% + +int XmlScanner::scan( ) +{ + int token = TK_NO_TOKEN; + int space = 0, readlen = 0; + char *attr_id_start = 0; + char *attr_value_start = 0; + int attr_id_len = 0; + int attr_value_len = 0; + + attrMkList.empty(); + buffer.clear(); + + while ( 1 ) { + if ( p == pe ) { + //printf("scanner: need more data\n"); + + if ( ts == 0 ) + have = 0; + else { + /* There is data that needs to be shifted over. */ + //printf("scanner: buffer broken mid token\n"); + have = pe - ts; + memmove( buf, ts, have ); + + int distance = ts - buf; + te -= distance; + tag_id_start -= distance; + attr_id_start -= distance; + attr_value_start -= distance; + adjustAttrPointers( distance ); + ts = buf; + } + + p = buf + have; + space = XML_BUFSIZE - have; + + if ( space == 0 ) { + /* We filled up the buffer trying to scan a token. */ + return TK_SPACE; + } + + if ( done ) { + //printf("scanner: end of file\n"); + p[0] = 0; + readlen = 1; + } + else { + input.read( p, space ); + readlen = input.gcount(); + if ( input.eof() ) { + //printf("scanner: setting done flag\n"); + done = 1; + } + } + + pe = p + readlen; + } + + %% write exec; + + if ( cs == XmlScanner_error ) + return TK_ERR; + + if ( token != TK_NO_TOKEN ) { + data_len = p - data; + return token; + } + } +} + +int xml_parse( std::istream &input, const char *fileName, + bool outputActive, bool wantComplete, + XmlScanner &scanner, XmlParser &parser ) +{ + while ( 1 ) { + int token = scanner.scan(); + if ( token == TK_NO_TOKEN ) { + cerr << "xmlscan: interal error: scanner returned NO_TOKEN" << endl; + exit(1); + } + else if ( token == TK_EOF ) { + parser.token( XmlParser_tk_eof, scanner.token_col, scanner.token_line ); + break; + } + else if ( token == TK_ERR ) { + scanner.error() << "scanner error" << endl; + break; + } + else if ( token == TK_SPACE ) { + scanner.error() << "scanner is out of buffer space" << endl; + break; + } + else { + /* All other tokens are either open or close tags. */ + XMLTagHashPair *tagId = Perfect_Hash::in_word_set( + scanner.tag_id_start, scanner.tag_id_len ); + + XMLTag *tag = new XMLTag( tagId, token == TK_OpenTag ? + XMLTag::Open : XMLTag::Close ); + + if ( tagId != 0 ) { + /* Get attributes for open tags. */ + if ( token == TK_OpenTag && scanner.attrMkList.length() > 0 ) { + tag->attrList = new AttrList; + for ( AttrMkList::Iter attr = scanner.attrMkList; + attr.lte(); attr++ ) + { + Attribute newAttr; + newAttr.id = new char[attr->idLen+1]; + memcpy( newAttr.id, attr->id, attr->idLen ); + newAttr.id[attr->idLen] = 0; + + /* Exclude the surrounding quotes. */ + newAttr.value = new char[attr->valueLen-1]; + memcpy( newAttr.value, attr->value+1, attr->valueLen-2 ); + newAttr.value[attr->valueLen-2] = 0; + + tag->attrList->append( newAttr ); + } + } + + /* Get content for closing tags. */ + if ( token == TK_CloseTag ) { + switch ( tagId->id ) { + case TAG_host: case TAG_arg: + case TAG_t: case TAG_alphtype: + case TAG_text: case TAG_goto: + case TAG_call: case TAG_next: + case TAG_entry: case TAG_set_tokend: + case TAG_set_act: case TAG_start_state: + case TAG_error_state: case TAG_state_actions: + case TAG_action_table: case TAG_cond_space: + case TAG_c: case TAG_ex: case TAG_eof_t: + tag->content = new char[scanner.buffer.length+1]; + memcpy( tag->content, scanner.buffer.data, + scanner.buffer.length ); + tag->content[scanner.buffer.length] = 0; + break; + } + } + } + + #if 0 + cerr << "parser_driver: " << (tag->type == XMLTag::Open ? "open" : "close") << + ": " << (tag->tagId != 0 ? tag->tagId->name : "<unknown>") << endl; + if ( tag->attrList != 0 ) { + for ( AttrList::Iter attr = *tag->attrList; attr.lte(); attr++ ) + cerr << " " << attr->id << ": " << attr->value << endl; + } + if ( tag->content != 0 ) + cerr << " content: " << tag->content << endl; + #endif + + parser.token( tag, scanner.token_col, scanner.token_line ); + } + } + + return 0; +} + +std::ostream &XmlScanner::error() +{ + gblErrorCount += 1; + cerr << fileName << ":" << curline << ":" << curcol << ": "; + return cerr; +} diff --git a/ragel/xmltags.gperf b/ragel/xmltags.gperf new file mode 100644 index 0000000..48ae352 --- /dev/null +++ b/ragel/xmltags.gperf @@ -0,0 +1,94 @@ +/* + * Copyright 2005 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + + +%{ +#include <string.h> +#include "xmlparse.h" +%} +%compare-strncmp +struct XMLTagHashPair; +%% +ragel, TAG_ragel +ragel_def, TAG_ragel_def +host, TAG_host +state_list, TAG_state_list +state, TAG_state +trans_list, TAG_trans_list +t, TAG_t +machine, TAG_machine +start_state, TAG_start_state +error_state, TAG_error_state +action_list, TAG_action_list +action, TAG_action +action_table_list, TAG_action_table_list +action_table, TAG_action_table +alphtype, TAG_alphtype +getkey, TAG_getkey +state_actions, TAG_state_actions +entry_points, TAG_entry_points +text, TAG_text +goto, TAG_goto +call, TAG_call +next, TAG_next +goto_expr, TAG_goto_expr +call_expr, TAG_call_expr +next_expr, TAG_next_expr +ret, TAG_ret +pchar, TAG_pchar +char, TAG_char +hold, TAG_hold +exec, TAG_exec +curs, TAG_curs +targs, TAG_targs +entry, TAG_entry +data, TAG_data +lm_switch, TAG_lm_switch +sub_action, TAG_sub_action +init_act, TAG_init_act +set_act, TAG_set_act +get_tokend, TAG_get_tokend +set_tokend, TAG_set_tokend +init_tokstart, TAG_init_tokstart +set_tokstart, TAG_set_tokstart +write, TAG_write +access, TAG_access +break, TAG_break +arg, TAG_arg +cond_space_list, TAG_cond_space_list +cond_space, TAG_cond_space +cond_list, TAG_cond_list +c, TAG_c +exports, TAG_exports +ex, TAG_ex +p_expr, TAG_p_expr +pe_expr, TAG_pe_expr +eof_expr, TAG_eof_expr +cs_expr, TAG_cs_expr +top_expr, TAG_top_expr +stack_expr, TAG_stack_expr +act_expr, TAG_act_expr +tokstart_expr, TAG_tokstart_expr +tokend_expr, TAG_tokend_expr +data_expr, TAG_data_expr +prepush, TAG_prepush +postpop, TAG_postpop +eof_t, TAG_eof_t diff --git a/test/Makefile.am b/test/Makefile.am new file mode 100644 index 0000000..f49c9c1 --- /dev/null +++ b/test/Makefile.am @@ -0,0 +1,45 @@ +# +# Copyright 2002-2009 Adrian Thurston <thurston@complang.org> +# + +# This file is part of Ragel. +# +# Ragel is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# Ragel is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with Ragel; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + + +TESTS = runtests + +EXTRA_DIST = \ + atoi1.rl clang2.rl cond7.rl element3.rl erract8.rl forder3.rl java1.rl \ + range.rl scan3.rl xml.rl atoi2.rl clang3.rl cppscan1.rl eofact.rl \ + erract9.rl gotocallret1.rl java2.rl recdescent1.rl scan4.rl atoi3.rl \ + clang4.rl cppscan2.rl erract1.rl export1.rl gotocallret2.rl keller1.rl \ + recdescent2.rl stateact1.rl awkemu.rl cond1.rl cppscan3.rl erract2.rl \ + export2.rl high1.rl lmgoto.rl recdescent3.rl statechart1.rl builtin.rl \ + cond2.rl cppscan4.rl erract3.rl export3.rl high2.rl mailbox1.rl \ + repetition.rl strings1.rl call1.rl cond3.rl cppscan5.rl erract4.rl \ + export4.rl high3.rl mailbox2.rl rlscan.rl strings2.rl call2.rl cond4.rl \ + cppscan6.rl erract5.rl fnext1.rl import1.rl mailbox3.rl ruby1.rl \ + tokstart1.rl call3.rl cond5.rl element1.rl erract6.rl forder1.rl \ + include1.rl minimize1.rl scan1.rl union.rl clang1.rl cond6.rl \ + element2.rl erract7.rl forder2.rl include2.rl patact.rl scan2.rl \ + xmlcommon.rl langtrans_c.sh langtrans_csharp.sh langtrans_d.sh \ + langtrans_java.sh langtrans_ruby.sh checkeofact.txl \ + langtrans_csharp.txl langtrans_c.txl langtrans_d.txl langtrans_java.txl \ + langtrans_ruby.txl testcase.txl cppscan1.h eofact.h mailbox1.h strings2.h + +CLEANFILES = \ + *.c *.cpp *.m *.d *.java *.bin *.class *.exp \ + *.out *_c.rl *_d.rl *_java.rl *_ruby.rl *_csharp.rl *.cs *.exe diff --git a/test/atoi1.rl b/test/atoi1.rl index 60fc959..ea706c8 100644 --- a/test/atoi1.rl +++ b/test/atoi1.rl @@ -19,7 +19,7 @@ neg = false; } action add_digit { - val = val * 10 + (fc - '0'); + val = val * 10 + (fc - 48); } action finish { diff --git a/test/atoi2.rl b/test/atoi2.rl index e16380a..923e88f 100644 --- a/test/atoi2.rl +++ b/test/atoi2.rl @@ -20,7 +20,7 @@ neg = false; } action add_digit { - val = val * 10 + (fc - '0'); + val = val * 10 + (fc - 48); } action finish { diff --git a/test/atoi3.rl b/test/atoi3.rl new file mode 100644 index 0000000..fcd4a41 --- /dev/null +++ b/test/atoi3.rl @@ -0,0 +1,75 @@ +# +# @LANG: ruby +# + +%%{ + machine atoi3; + action begin { + neg = false; + val = 0; + } + action see_neg { + neg = true; + } + action add_digit { + val = val * 10 + (fc - "0"[0]); + } + action finish { + val = -1 * val if neg + } + action print { + puts val; + } + atoi = (('-' @ see_neg | '+') ? (digit @ add_digit) +) > begin % finish; + main := atoi '\n' @ print; +}%% + +%% write data; + +def run_machine( data ) + p = 0; + pe = data.length + cs = 0 + val = 0; + neg = false; + + %% write init; + %% write exec; + if cs >= atoi3_first_final + puts "ACCEPT" + else + puts "FAIL" + end +end + +inp = [ + "1\n", + "12\n", + "222222\n", + "+2123\n", + "213 3213\n", + "-12321\n", + "--123\n", + "-99\n", + " -3000\n", +] + +inp.each { |str| run_machine(str) } + +=begin _____OUTPUT_____ +1 +ACCEPT +12 +ACCEPT +222222 +ACCEPT +2123 +ACCEPT +FAIL +-12321 +ACCEPT +FAIL +-99 +ACCEPT +FAIL +=end _____OUTPUT_____ diff --git a/test/awkemu.rl b/test/awkemu.rl index a988651..343f3e6 100644 --- a/test/awkemu.rl +++ b/test/awkemu.rl @@ -27,7 +27,7 @@ struct awkemu %%{ machine awkemu; - variable curstate fsm->cs; + variable cs fsm->cs; # Starts a line. Will initialize all the data necessary for capturing the line. action startline { @@ -113,8 +113,6 @@ void awkemu_execute( struct awkemu *fsm, const char *_data, int _len ) int awkemu_finish( struct awkemu *fsm ) { - %% write eof; - if ( fsm->cs == awkemu_error ) return -1; if ( fsm->cs >= awkemu_first_final ) diff --git a/test/builtin.rl b/test/builtin.rl index 626927b..816b441 100644 --- a/test/builtin.rl +++ b/test/builtin.rl @@ -17,7 +17,7 @@ struct builtin %%{ machine builtin; alphtype unsigned int; - variable curstate fsm->cs; + variable cs fsm->cs; main := ( any @{alph("any");} | diff --git a/test/call1.rl b/test/call1.rl index 24a362f..ddd552a 100644 --- a/test/call1.rl +++ b/test/call1.rl @@ -56,8 +56,6 @@ void test_execute( struct test *fsm, const char *data, int len ) int test_finish( struct test *fsm ) { - %% write eof; - if ( fsm->cs == test_error ) return -1; if ( fsm->cs >= test_first_final ) diff --git a/test/call2.rl b/test/call2.rl index c447496..a553855 100644 --- a/test/call2.rl +++ b/test/call2.rl @@ -72,8 +72,6 @@ void CallTest::execute( const char *data, int len ) int CallTest::finish( ) { - %% write eof; - if ( this->cs == CallTest_error ) return -1; if ( this->cs >= CallTest_first_final ) @@ -83,7 +81,7 @@ int CallTest::finish( ) #define BUFSIZE 1024 -void test( char *buf ) +void test( const char *buf ) { CallTest test; diff --git a/test/call3.rl b/test/call3.rl index c253b24..b19b96c 100644 --- a/test/call3.rl +++ b/test/call3.rl @@ -79,7 +79,6 @@ int num = 0; - (int) finish; { - %% write eof; if ( cs == CallTest_error ) return -1; return ( cs >= CallTest_first_final ) ? 1 : 0; diff --git a/test/checkeofact.txl b/test/checkeofact.txl new file mode 100644 index 0000000..8189013 --- /dev/null +++ b/test/checkeofact.txl @@ -0,0 +1,95 @@ +include "testcase.txl" + +define program + [lang_indep] + | 'yes + | 'no +end define + +rule findEof1 + match [machine_expr_item] + '>/ +end rule + +rule findEof2 + match [machine_expr_item] + '</ +end rule + +rule findEof3 + match [machine_expr_item] + '$/ +end rule + +rule findEof4 + match [machine_expr_item] + '%/ +end rule + +rule findEof5 + match [machine_expr_item] + '@/ +end rule + +rule findEof6 + match [machine_expr_item] + '<>/ +end rule + +rule findEof7 + match [repeat machine_expr_item] + '> 'eof _ [repeat machine_expr_item] +end rule + +rule findEof8 + match [repeat machine_expr_item] + '< 'eof _ [repeat machine_expr_item] +end rule + +rule findEof9 + match [repeat machine_expr_item] + '$ 'eof _ [repeat machine_expr_item] +end rule + +rule findEof10 + match [repeat machine_expr_item] + '% 'eof _ [repeat machine_expr_item] +end rule + +rule findEof11 + match [repeat machine_expr_item] + '@ 'eof _ [repeat machine_expr_item] +end rule + +rule findEof12 + match [repeat machine_expr_item] + '<> 'eof _ [repeat machine_expr_item] +end rule + +rule findScanner + match [machine_expr_item] + '|* _ [repeat scanner_item] '*| +end rule + +function findEof P [program] + replace [program] + _ [program] + where + P + [findEof1] [findEof2] [findEof3] + [findEof4] [findEof5] [findEof6] + [findEof7] [findEof8] [findEof9] + [findEof10] [findEof11] [findEof12] + [findScanner] + by + 'yes +end function + +function main + replace [program] + P [program] + construct NewP [program] + 'no + by + NewP [findEof P] +end function diff --git a/test/clang1.rl b/test/clang1.rl index ade8f9a..85532c6 100644 --- a/test/clang1.rl +++ b/test/clang1.rl @@ -161,6 +161,7 @@ void test( char *buf ) { int len = strlen( buf ); char *p = buf, *pe = buf + len; + char *eof = pe; char identBuf[IDENT_BUFLEN+1]; int identLen; int curLine; @@ -171,7 +172,6 @@ void test( char *buf ) %% write init; %% write exec; - %% write eof; if ( cs >= clang_first_final ) printf("ACCEPT\n"); diff --git a/test/clang2.rl b/test/clang2.rl index fcb6ba8..3b22e4b 100644 --- a/test/clang2.rl +++ b/test/clang2.rl @@ -162,13 +162,13 @@ { const char *p = data; const char *pe = data + len; + const char *eof = pe; %% write exec; } - (int) finish; { - %% write eof; if ( cs == Clang_error ) return -1; if ( cs >= Clang_first_final ) diff --git a/test/clang3.rl b/test/clang3.rl index d950eb3..82f5eed 100644 --- a/test/clang3.rl +++ b/test/clang3.rl @@ -159,6 +159,7 @@ class CLang { char *p = _data; char *pe = _data + _len; + char *eof = pe; %% write exec; } @@ -168,7 +169,6 @@ class CLang // accepting state. int finish( ) { - %% write eof; if ( cs == error ) return -1; if ( cs >= first_final ) diff --git a/test/clang4.rl b/test/clang4.rl new file mode 100644 index 0000000..c3bb399 --- /dev/null +++ b/test/clang4.rl @@ -0,0 +1,188 @@ +/* + * @LANG: indep + * @NEEDS_EOF: yes + */ + +char array[32]; +int pos; +int line; +%% +pos = 0; +line = 1; +%%{ + machine clang; + + # Function to buffer a character. + action bufChar { array[pos] = fc; pos = pos + 1; } + + # Function to clear the buffer. + action clearBuf { pos = 0; } + + # Functions to dump tokens as they are matched. + action ident { + prints "ident("; + printi line; + prints ","; + printi pos; + prints "): "; + printb array; + prints "\n"; + } + action literal { + prints "literal("; + printi line; + prints ","; + printi pos; + prints "): "; + printb array; + prints "\n"; + } + action float { + prints "float("; + printi line; + prints ","; + printi pos; + prints "): "; + printb array; + prints "\n"; + } + action integer { + prints "int("; + printi line; + prints ","; + printi pos; + prints "): "; + printb array; + prints "\n"; + } + action hex { + prints "hex("; + printi line; + prints ","; + printi pos; + prints "): "; + printb array; + prints "\n"; + } + action symbol { + prints "symbol("; + printi line; + prints ","; + printi pos; + prints "): "; + printb array; + prints "\n"; + } + + # Alpha numberic characters or underscore. + alnumu = alnum | '_'; + + # Alpha charactres or underscore. + alphau = alpha | '_'; + + # Symbols. Upon entering clear the buffer. On all transitions + # buffer a character. Upon leaving dump the symbol. + symbol = ( punct - [_'"] ) >clearBuf $bufChar %symbol; + + # Identifier. Upon entering clear the buffer. On all transitions + # buffer a character. Upon leaving, dump the identifier. + ident = (alphau . alnumu*) >clearBuf $bufChar %ident; + + # Match single characters inside literal strings. Or match + # an escape sequence. Buffers the charater matched. + sliteralChar = + ( extend - ['\\] ) @bufChar | + ( '\\' . extend @bufChar ); + dliteralChar = + ( extend - ["\\] ) @bufChar | + ( '\\' . extend @bufChar ); + + # Single quote and double quota literals. At the start clear + # the buffer. Upon leaving dump the literal. + sliteral = ('\'' @clearBuf . sliteralChar* . '\'' ) %literal; + dliteral = ('"' @clearBuf . dliteralChar* . '"' ) %literal; + literal = sliteral | dliteral; + + # Whitespace is standard ws, newlines and control codes. + whitespace = any - 33 .. 126; + + # Describe both c style comments and c++ style comments. The + # priority bump on tne terminator of the comments brings us + # out of the extend* which matches everything. + ccComment = '//' . extend* $0 . '\n' @1; + cComment = '/!' . extend* $0 . '!/' @1; + + # Match an integer. We don't bother clearing the buf or filling it. + # The float machine overlaps with int and it will do it. + integer = digit+ %integer; + + # Match a float. Upon entering the machine clear the buf, buffer + # characters on every trans and dump the float upon leaving. + float = ( digit+ . '.' . digit+ ) >clearBuf $bufChar %float; + + # Match a hex. Upon entering the hex part, clear the buf, buffer characters + # on every trans and dump the hex on leaving transitions. + hex = '0x' . xdigit+ >clearBuf $bufChar %hex; + + # Or together all the lanuage elements. + fin = ( ccComment | + cComment | + symbol | + ident | + literal | + whitespace | + integer | + float | + hex ); + + # Star the language elements. It is critical in this type of application + # that we decrease the priority of out transitions before doing so. This + # is so that when we see 'aa' we stay in the fin machine to match an ident + # of length two and not wrap around to the front to match two idents of + # length one. + clang_main = ( fin $1 %0 )*; + + # This machine matches everything, taking note of newlines. + newline = ( any | '\n' @{ line = line + 1; } )*; + + # The final fsm is the lexer intersected with the newline machine which + # will count lines for us. Since the newline machine accepts everything, + # the strings accepted is goverened by the clang_main machine, onto which + # the newline machine overlays line counting. + main := clang_main & newline; +}%% +/* _____INPUT_____ +"999 0xaAFF99 99.99 /!\n!/ 'lksdj' //\n\"\n\nliteral\n\n\n\"0x00aba foobardd.ddsf 0x0.9\n" +"wordwithnum00asdf\n000wordfollowsnum,makes new symbol\n\nfinishing early /! unfinished ...\n" +_____INPUT_____ */ +/* _____OUTPUT_____ +int(1,3): 999 +hex(1,6): aAFF99 +float(1,5): 99.99 +literal(2,5): lksdj +literal(8,12): + +literal + + + +hex(8,5): 00aba +ident(8,8): foobardd +symbol(8,1): . +ident(8,4): ddsf +hex(8,1): 0 +symbol(8,1): . +int(8,1): 9 +ACCEPT +ident(1,17): wordwithnum00asdf +int(2,3): 000 +ident(2,14): wordfollowsnum +symbol(2,1): , +ident(2,5): makes +ident(2,3): new +ident(2,6): symbol +ident(4,9): finishing +ident(4,5): early +FAIL +_____OUTPUT_____ */ + diff --git a/test/cond1.rl b/test/cond1.rl index 620ea5e..7c3ffff 100644 --- a/test/cond1.rl +++ b/test/cond1.rl @@ -1,5 +1,6 @@ /* * @LANG: indep + * @ALLOW_GENFLAGS: -T0 -T1 -G0 -G1 -G2 */ bool i; bool j; @@ -16,9 +17,9 @@ bool k; action two { prints " two\n";} action three { prints " three\n";} - action seti { if ( fc == '0' ) i = false; else i = true; } - action setj { if ( fc == '0' ) j = false; else j = true; } - action setk { if ( fc == '0' ) k = false; else k = true; } + action seti { if ( fc == 48 ) i = false; else i = true; } + action setj { if ( fc == 48 ) j = false; else j = true; } + action setk { if ( fc == 48 ) k = false; else k = true; } action break {fbreak;} diff --git a/test/cond2.rl b/test/cond2.rl index 7593a3f..7e49ab8 100644 --- a/test/cond2.rl +++ b/test/cond2.rl @@ -25,11 +25,11 @@ using std::endl; %% write data noerror; -void test( int i, int j, char *str ) +void test( int i, int j, const char *str ) { int cs = foo_start; - char *p = str; - char *pe = str + strlen( str ); + const char *p = str; + const char *pe = str + strlen( str ); cout << "run:" << endl; %% write exec; diff --git a/test/cond3.rl b/test/cond3.rl index 1847727..80904b5 100644 --- a/test/cond3.rl +++ b/test/cond3.rl @@ -20,12 +20,12 @@ using std::endl; %% write data noerror; -void test( char *str ) +void test( const char *str ) { int cs = foo_start; int c = 0; - char *p = str; - char *pe = str + strlen( str ); + const char *p = str; + const char *pe = str + strlen( str ); cout << "run:" << endl; %% write exec; diff --git a/test/cond5.rl b/test/cond5.rl index 57e3c85..b6ab4ae 100644 --- a/test/cond5.rl +++ b/test/cond5.rl @@ -12,12 +12,12 @@ using std::endl; write data noerror; }%% -void test( char *str ) +void test( const char *str ) { int cs = foo_start; int c = 0; - char *p = str; - char *pe = str + strlen( str ); + const char *p = str; + const char *pe = str + strlen( str ); char last = '0'; cout << "run:"; diff --git a/test/cond6.rl b/test/cond6.rl index 25bf45d..ede9ed8 100644 --- a/test/cond6.rl +++ b/test/cond6.rl @@ -14,11 +14,11 @@ using std::endl; write data noerror; }%% -void test( char *str ) +void test( const char *str ) { int cs = cond_start, n = 0; - char *p = str; - char *pe = str + strlen( str ); + const char *p = str; + const char *pe = str + strlen( str ); %%{ comment = '(' @{n=0;} diff --git a/test/cond7.rl b/test/cond7.rl new file mode 100644 index 0000000..3951af0 --- /dev/null +++ b/test/cond7.rl @@ -0,0 +1,82 @@ +/* + * @LANG: indep + */ +int i; +int c; +%% + +%%{ + machine foo; + + action testi {i > 0} + action inc { + i = i - 1; + c = fc; + prints "item: "; + printi c; + prints "\n"; + } + + count = [0-9] @{ + i = fc - '0'; + prints "count: "; + printi i; + prints "\n"; + }; + + sub = + count # record the number of digits + ( digit when testi @inc )* outwhen !testi; + + main := sub sub '\n'; +}%% + +/* _____INPUT_____ +"00\n" +"019\n" +"190\n" +"1719\n" +"1040000\n" +"104000a\n" +"104000\n" +_____INPUT_____ */ +/* _____OUTPUT_____ +count: 0 +count: 0 +ACCEPT +count: 0 +count: 1 +item: 57 +ACCEPT +count: 1 +item: 57 +count: 0 +ACCEPT +count: 1 +item: 55 +count: 1 +item: 57 +ACCEPT +count: 1 +item: 48 +count: 4 +item: 48 +item: 48 +item: 48 +item: 48 +ACCEPT +count: 1 +item: 48 +count: 4 +item: 48 +item: 48 +item: 48 +FAIL +count: 1 +item: 48 +count: 4 +item: 48 +item: 48 +item: 48 +FAIL +_____OUTPUT_____ */ diff --git a/test/cppscan1.h b/test/cppscan1.h index 4497cd2..346dd9b 100644 --- a/test/cppscan1.h +++ b/test/cppscan1.h @@ -2,6 +2,8 @@ #define _CPPSCAN1_H #include <iostream> +#include <cstdlib> +#include <cstring> using namespace std; @@ -98,7 +100,7 @@ struct Scanner // the data, the machine is in the error state and can never accept, 0 if // the machine is in a non-accepting state and 1 if the machine is in an // accepting state. - int execute( char *data, int len ); + int execute( const char *data, int len ); // Indicate that there is no more data. Returns -1 if the machine finishes // in the error state and does not accept, 0 if the machine finishes diff --git a/test/cppscan1.rl b/test/cppscan1.rl index 623af28..92869f7 100644 --- a/test/cppscan1.rl +++ b/test/cppscan1.rl @@ -1,6 +1,7 @@ /* * @LANG: c++ - * @ALLOW_GENFLAGS: -T0 -T1 -F0 -F1 -G0 -G1 -G2 -P + * + * Test works with split code gen. */ #include "cppscan1.h" @@ -130,11 +131,12 @@ void Scanner::init( ) %% write init; } -int Scanner::execute( char *data, int len ) +int Scanner::execute( const char *data, int len ) { Scanner *fsm = this; - char *p = data; - char *pe = data + len; + const char *p = data; + const char *pe = data + len; + const char *eof = pe; %% write exec; if ( cs == Scanner_error ) @@ -146,7 +148,6 @@ int Scanner::execute( char *data, int len ) int Scanner::finish( ) { - %% write eof; if ( cs == Scanner_error ) return -1; if ( cs >= Scanner_first_final ) @@ -189,7 +190,7 @@ void Buffer::upAllocate( int len ) allocated = len; } -void test( char *buf ) +void test( const char *buf ) { Scanner scanner(cout); scanner.init(); diff --git a/test/cppscan2.rl b/test/cppscan2.rl index 07fc01b..a609bba 100644 --- a/test/cppscan2.rl +++ b/test/cppscan2.rl @@ -3,6 +3,7 @@ */ #include <iostream> +#include <string.h> using namespace std; #define TK_Dlit 192 @@ -39,8 +40,9 @@ using namespace std; #define BUFSIZE 4096 int tok; -char buf[BUFSIZE], *tokstart, *tokend; -void token( char *data, int len ); +char buf[BUFSIZE]; +const char *ts, *te; +void token( const char *data, int len ); bool discard = false; struct Scanner @@ -56,7 +58,7 @@ struct Scanner // the data, the machine is in the error state and can never accept, 0 if // the machine is in a non-accepting state and 1 if the machine is in an // accepting state. - int execute( char *data, int len ); + int execute( const char *data, int len ); // Indicate that there is no more data. Returns -1 if the machine finishes // in the error state and does not accept, 0 if the machine finishes @@ -141,7 +143,7 @@ struct Scanner action onError { if ( tok != 0 ) { - char *rst_data; + const char *rst_data; if ( tok == TK_Comment || tok == TK_Whitespace ) { /* Reset comment status, don't send. */ @@ -153,19 +155,19 @@ struct Scanner } else { /* Send the token. */ - token( tokstart, tokend - tokstart + 1 ); + token( ts, te - ts + 1 ); /* Restart right after the token. */ - rst_data = tokend+1; + rst_data = te+1; } - tokstart = 0; + ts = 0; fexec rst_data; fgoto main; } } - main := tokens >{tokstart=fpc;} @{tokend=fpc;} $!onError; + main := tokens >{ts=fpc;} @{te=fpc;} $!onError; }%% %% write data; @@ -173,17 +175,18 @@ struct Scanner int Scanner::init( ) { tok = 0; - tokstart = 0; - tokend = 0; + ts = 0; + te = 0; %% write init; return 1; } -int Scanner::execute( char *data, int len ) +int Scanner::execute( const char *data, int len ) { - char *p = data; - char *pe = data + len; + const char *p = data; + const char *pe = data + len; + const char *eof = pe; %% write exec; @@ -196,7 +199,6 @@ int Scanner::execute( char *data, int len ) int Scanner::finish( ) { - %% write eof; if ( cs == Scanner_error ) return -1; if ( cs >= Scanner_first_final ) @@ -205,7 +207,7 @@ int Scanner::finish( ) } -void token( char *data, int len ) +void token( const char *data, int len ) { cout << "<" << tok << "> "; for ( int i = 0; i < len; i++ ) @@ -213,14 +215,14 @@ void token( char *data, int len ) cout << '\n'; } -void test( char * data ) +void test( const char * data ) { Scanner scanner; scanner.init(); scanner.execute( data, strlen(data) ); scanner.finish(); if ( tok != 0 && tok != TK_Comment && tok != TK_Whitespace ) - token( tokstart, tokend - tokstart + 1 ); + token( ts, te - ts + 1 ); } int main() diff --git a/test/cppscan3.rl b/test/cppscan3.rl index afe832d..67b8624 100644 --- a/test/cppscan3.rl +++ b/test/cppscan3.rl @@ -44,13 +44,13 @@ char buf[BUFSIZE]; struct Scanner { int cs, act; - char *tokstart, *tokend; + const char *ts, *te; void token( int tok ); void run(); void init( ); - void execute( char *data, int len ); + void execute( const char *data, int len ); int finish( ); }; @@ -114,7 +114,7 @@ struct Scanner '...' => { token( TK_DotDotDot );}; # Single char symbols. - ( punct - [_"'] ) => { token( tokstart[0] );}; + ( punct - [_"'] ) => { token( ts[0] );}; action comment { token( TK_Comment ); @@ -137,17 +137,19 @@ void Scanner::init( ) /* Returns the count of bytes still in the buffer * (shifted to the biginning) */ -void Scanner::execute( char *data, int len ) +void Scanner::execute( const char *data, int len ) { - char *p = data; - char *pe = data + len; + const char *p = data; + const char *pe = data + len; + const char *eof = pe; %% write exec; + + cout << "P: " << (p - data) << endl; } int Scanner::finish( ) { - %% write eof; if ( cs == Scanner_error ) return -1; if ( cs >= Scanner_first_final ) @@ -158,15 +160,15 @@ int Scanner::finish( ) void Scanner::token( int tok ) { - const char *data = tokstart; - int len = tokend - tokstart; + const char *data = ts; + int len = te - ts; cout << "<" << tok << "> "; for ( int i = 0; i < len; i++ ) cout << data[i]; cout << '\n'; } -void test( char *buf ) +void test( const char *buf ) { int len = strlen( buf ); std::ios::sync_with_stdio(false); @@ -192,8 +194,7 @@ int main() "44. 44\n" "44 . 44\n" "44.44\n" - "_hithere22\n" - "\n" + "_hithere22" ); test( @@ -207,7 +208,7 @@ int main() "0x98\n" "0x\n" "//\n" - "/* * */\n" + "/* * */" ); test( @@ -245,6 +246,7 @@ int main() <241> <195> _hithere22 +P: 51 <193> '\'' <192> "\n\d'\"" <241> @@ -277,5 +279,7 @@ int main() <242> // <242> /* * */ +P: 55 +P: 1 PARSE ERROR #endif diff --git a/test/cppscan4.rl b/test/cppscan4.rl index fa7499f..42a97f1 100644 --- a/test/cppscan4.rl +++ b/test/cppscan4.rl @@ -192,6 +192,7 @@ class Scanner { char *p = _data; char *pe = _data + _len; + char *eof = null; %% write exec; @@ -208,8 +209,6 @@ class Scanner // accepting state. int finish( ) { - %% write eof; - if ( cs == error ) return -1; if ( cs >= first_final ) diff --git a/test/cppscan5.rl b/test/cppscan5.rl index 3c0035b..057725a 100644 --- a/test/cppscan5.rl +++ b/test/cppscan5.rl @@ -43,12 +43,12 @@ static const int TK_Comment = 242; class Scanner { int cs, act; - char *tokstart, tokend; + char *ts, te; void token( int tok ) { - char *data = tokstart; - int len = tokend - tokstart; + char *data = ts; + int len = te - ts; printf( "<%i> ", tok ); for ( int i = 0; i < len; i++ ) printf( "%c", data[i] ); @@ -116,7 +116,7 @@ class Scanner '...' => { token( TK_DotDotDot );}; # Single char symbols. - ( punct - [_"'] ) => { token( tokstart[0] );}; + ( punct - [_"'] ) => { token( ts[0] );}; action comment { token( TK_Comment ); @@ -142,6 +142,7 @@ class Scanner { char *p = data; char *pe = data + len; + char *eof = pe; %% write exec; } @@ -152,8 +153,6 @@ class Scanner // accepting state. int finish( ) { - %% write eof; - if ( cs == error ) return -1; if ( cs >= first_final ) @@ -187,8 +186,7 @@ int main() "44. 44\n" "44 . 44\n" "44.44\n" - "_hithere22\n" - "\n" + "_hithere22" ); test( @@ -202,7 +200,7 @@ int main() "0x98\n" "0x\n" "//\n" - "/* * */\n" + "/* * */" ); test( diff --git a/test/cppscan6.rl b/test/cppscan6.rl new file mode 100644 index 0000000..ed82fe6 --- /dev/null +++ b/test/cppscan6.rl @@ -0,0 +1,358 @@ +/* + * @LANG: indep + * + * const char *data = ts; + * int len = te - ts; + * cout << "<" << tok << "> "; + * for ( int i = 0; i < len; i++ ) + * cout << data[i]; + * cout << '\n'; + */ +ptr ts; +ptr te; +int act; +int token; +%% +%%{ + machine scanner; + + action comment { + token = 242; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + } + + + main := |* + + # Single and double literals. + ( 'L'? "'" ( [^'\\\n] | '\\' any )* "'" ) + => { + token = 193; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + ( 'L'? '"' ( [^"\\\n] | '\\' any )* '"' ) + => { + token = 192; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + + # Identifiers + ( [a-zA-Z_] [a-zA-Z0-9_]* ) + =>{ + token = 195; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + + # Floating literals. + fract_const = digit* '.' digit+ | digit+ '.'; + exponent = [eE] [+\-]? digit+; + float_suffix = [flFL]; + + ( fract_const exponent? float_suffix? | + digit+ exponent float_suffix? ) + => { + token = 194; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + + # Integer decimal. Leading part buffered by float. + ( ( '0' | [1-9] [0-9]* ) [ulUL]? ) + => { + token = 218; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + + # Integer octal. Leading part buffered by float. + ( '0' [0-9]+ [ulUL]? ) + => { + token = 219; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + + # Integer hex. Leading 0 buffered by float. + ( '0' ( 'x' [0-9a-fA-F]+ [ulUL]? ) ) + => { + token = 220; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + + # Only buffer the second item, first buffered by symbol. + '::' => { + token = 197; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '==' => { + token = 223; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '!=' => { + token = 224; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '&&' => { + token = 225; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '||' => { + token = 226; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '*=' => { + token = 227; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '/=' => { + token = 228; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '%=' => { + token = 229; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '+=' => { + token = 230; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '-=' => { + token = 231; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '&=' => { + token = 232; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '^=' => { + token = 233; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '|=' => { + token = 234; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '++' => { + token = 212; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '--' => { + token = 213; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '->' => { + token = 211; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '->*' => { + token = 214; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + '.*' => { + token = 215; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + + # Three char compounds, first item already buffered. + '...' => { + token = 240; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + + # Single char symbols. + ( punct - [_"'] ) => { + token = first_token_char; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + + # Comments and whitespace. + '/!' ( any* $0 '!/' @1 ) => comment; + '//' ( any* $0 '\n' @1 ) => comment; + ( any - 33..126 )+ => { + token = 241; + prints "<"; + printi token; + prints "> "; + print_token; + prints "\n"; + }; + *|; +}%% +/* _____INPUT_____ +"\"\\\"hi\" /!\n!/\n44 .44\n44. 44\n44 . 44\n44.44\n_hithere22" +"'\\''\"\\n\\d'\\\"\"\nhi\n99\n.99\n99e-4\n->*\n||\n0x98\n0x\n//\n/! * !/" +"'\n'\n" +_____INPUT_____ */ +/* _____OUTPUT_____ +<192> "\"hi" +<241> +<242> /! +!/ +<241> + +<218> 44 +<241> +<194> .44 +<241> + +<194> 44. +<241> +<218> 44 +<241> + +<218> 44 +<241> +<46> . +<241> +<218> 44 +<241> + +<194> 44.44 +<241> + +<195> _hithere22 +ACCEPT +<193> '\'' +<192> "\n\d'\"" +<241> + +<195> hi +<241> + +<218> 99 +<241> + +<194> .99 +<241> + +<194> 99e-4 +<241> + +<214> ->* +<241> + +<226> || +<241> + +<220> 0x98 +<241> + +<218> 0 +<195> x +<241> + +<242> // + +<242> /! * !/ +ACCEPT +FAIL +_____OUTPUT_____ */ diff --git a/test/element1.rl b/test/element1.rl index 3113058..0795778 100644 --- a/test/element1.rl +++ b/test/element1.rl @@ -8,7 +8,7 @@ using namespace std; struct LangEl { int key; - char *name; + const char *name; }; struct Fsm @@ -39,6 +39,7 @@ struct Fsm alphtype int; getkey fpc->key; + variable eof eof_marker; action a1 {} action a2 {} @@ -57,10 +58,11 @@ int Fsm::init( ) return 0; } -int Fsm::execute( LangEl *_data, int _len ) +int Fsm::execute( LangEl *data, int len ) { - LangEl *p = _data; - LangEl *pe = _data+_len; + LangEl *p = data; + LangEl *pe = data + len; + LangEl *eof_marker = pe; %% write exec; if ( cs == Fsm_error ) @@ -72,8 +74,6 @@ int Fsm::execute( LangEl *_data, int _len ) int Fsm::finish( ) { - %% write eof; - if ( cs == Fsm_error ) return -1; if ( cs >= Fsm_first_final ) diff --git a/test/element2.rl b/test/element2.rl index 55f7610..7aa6217 100644 --- a/test/element2.rl +++ b/test/element2.rl @@ -19,7 +19,7 @@ struct fsm machine fsm; alphtype int; getkey fpc->key; - variable curstate fsm->cs; + variable cs fsm->cs; action a1 {} action a2 {} @@ -41,14 +41,13 @@ void fsm_execute( struct fsm *fsm, struct LangEl *_data, int _len ) { struct LangEl *p = _data; struct LangEl *pe = _data+_len; + struct LangEl *eof = pe; %% write exec; } int fsm_finish( struct fsm *fsm ) { - %% write eof; - if ( fsm->cs == fsm_error ) return -1; if ( fsm->cs >= fsm_first_final ) diff --git a/test/element3.rl b/test/element3.rl index 773e801..66435f4 100644 --- a/test/element3.rl +++ b/test/element3.rl @@ -66,6 +66,7 @@ struct LangEl { struct LangEl *p = _data; struct LangEl *pe = _data + _len; + struct LangEl *eof = pe; %% write exec; if ( self->cs == Fsm_error ) @@ -75,7 +76,6 @@ struct LangEl - (int) finish; { - %% write eof; if ( self->cs == Fsm_error ) return -1; return ( self->cs >= Fsm_first_final ) ? 1 : 0; diff --git a/test/eofact.rl b/test/eofact.rl index 890b73c..eeb91b8 100644 --- a/test/eofact.rl +++ b/test/eofact.rl @@ -1,6 +1,7 @@ /* * @LANG: indep - * @ALLOW_GENFLAGS: -T0 -T1 -F0 -F1 -G0 -G1 -G2 -P + * + * Test works with split code gen. */ %% %%{ diff --git a/test/erract1.rl b/test/erract1.rl index ee0237d..d5c01ea 100644 --- a/test/erract1.rl +++ b/test/erract1.rl @@ -65,6 +65,7 @@ int ErrAct::execute( const char *_data, int _len ) { const char *p = _data; const char *pe = _data+_len; + const char *eof = pe; %% write exec; if ( cs == ErrAct_error ) @@ -76,7 +77,6 @@ int ErrAct::execute( const char *_data, int _len ) int ErrAct::finish( ) { - %% write eof; if ( cs == ErrAct_error ) return -1; if ( cs >= ErrAct_first_final ) @@ -86,7 +86,7 @@ int ErrAct::finish( ) #define BUFSIZE 1024 -void test( char *buf ) +void test( const char *buf ) { ErrAct errAct; errAct.init(); diff --git a/test/erract2.rl b/test/erract2.rl index a4d0ef9..b1fbfbf 100644 --- a/test/erract2.rl +++ b/test/erract2.rl @@ -40,33 +40,45 @@ _____INPUT_____ */ /* _____OUTPUT_____ +err_start eof_start +err_all eof_all FAIL +err_all +err_middle eof_all eof_middle FAIL err_start err_all FAIL +err_all +err_middle eof_all eof_middle FAIL err_all err_middle FAIL +err_all +err_middle eof_all eof_middle FAIL err_all err_middle FAIL +err_all +err_middle eof_all eof_middle FAIL err_all err_middle FAIL +err_all +err_out eof_all eof_out FAIL diff --git a/test/erract3.rl b/test/erract3.rl index 5490b67..adfe76c 100644 --- a/test/erract3.rl +++ b/test/erract3.rl @@ -12,7 +12,7 @@ struct erract %%{ machine erract; - variable curstate fsm->cs; + variable cs fsm->cs; # The data that is to go into the fsm structure. action hello_fails { printf("hello fails\n");} @@ -33,13 +33,12 @@ void erract_execute( struct erract *fsm, const char *_data, int _len ) { const char *p = _data; const char *pe = _data+_len; + const char *eof = pe; %% write exec; } int erract_finish( struct erract *fsm ) { - %% write eof; - if ( fsm->cs == erract_error ) return -1; else if ( fsm->cs >= erract_first_final ) diff --git a/test/erract4.rl b/test/erract4.rl index 1a753ef..bef1139 100644 --- a/test/erract4.rl +++ b/test/erract4.rl @@ -57,12 +57,12 @@ { const char *p = _data; const char *pe = _data + _len; + const char *eof = pe; %% write exec; } - (int) finish; { - %% write eof; if ( cs == ErrAct_error ) return -1; else if ( cs >= ErrAct_first_final ) diff --git a/test/erract5.rl b/test/erract5.rl index 73edec8..0ea6e9a 100644 --- a/test/erract5.rl +++ b/test/erract5.rl @@ -69,12 +69,12 @@ { const char *p = _data; const char *pe = _data + _len; + const char *eof = pe; %% write exec; } - (int) finish; { - %% write eof; if ( cs == ErrAct_error ) return -1; else if ( cs >= ErrAct_first_final ) diff --git a/test/erract6.rl b/test/erract6.rl new file mode 100644 index 0000000..688042f --- /dev/null +++ b/test/erract6.rl @@ -0,0 +1,82 @@ +/* + * @LANG: c + */ + +/* + * Test of a transition going to the error state. + */ + +#include <stdio.h> +#define BUFSIZE 2048 + +struct errintrans +{ + int cs; +}; + +%%{ + machine errintrans; + variable cs fsm->cs; + + char = any - (digit | '\n'); + line = char* "\n"; + main := line+; +}%% + +%% write data; + +void errintrans_init( struct errintrans *fsm ) +{ + %% write init; +} + +void errintrans_execute( struct errintrans *fsm, const char *_data, int _len ) +{ + const char *p = _data; + const char *pe = _data+_len; + + %% write exec; +} + +int errintrans_finish( struct errintrans *fsm ) +{ + if ( fsm->cs == errintrans_error ) + return -1; + if ( fsm->cs >= errintrans_first_final ) + return 1; + return 0; +} + + +struct errintrans fsm; +#include <string.h> + +void test( char *buf ) +{ + int len = strlen( buf ); + errintrans_init( &fsm ); + errintrans_execute( &fsm, buf, len ); + if ( errintrans_finish( &fsm ) > 0 ) + printf("ACCEPT\n"); + else + printf("FAIL\n"); +} + + +int main() +{ + test( + "good, does not have numbers\n" + ); + + test( + "bad, has numbers 666\n" + ); + + return 0; +} + +#ifdef _____OUTPUT_____ +ACCEPT +FAIL +#endif diff --git a/test/erract7.rl b/test/erract7.rl new file mode 100644 index 0000000..040ad73 --- /dev/null +++ b/test/erract7.rl @@ -0,0 +1,42 @@ +/* + * @LANG: c + */ + +#include <stdio.h> +#include <string.h> + +%%{ + machine foo; + + action on_char { printf("char: %c\n", *p); } + action on_err { printf("err: %c\n", *p); } + action to_state { printf("to state: %c\n", *p); } + + main := 'heXXX' $on_char $err(on_err) $to(to_state); +}%% + +%% write data; + +int main() +{ + int cs; + char *p = "hello", *pe = p + strlen(p); + char *eof = pe; + %%{ + write init; + write exec; + }%% + + printf( "rest: %s\n", p ); + + return 0; +} + +#ifdef _____OUTPUT_____ +char: h +to state: h +char: e +to state: e +err: l +rest: llo +#endif diff --git a/test/erract8.rl b/test/erract8.rl new file mode 100644 index 0000000..7926186 --- /dev/null +++ b/test/erract8.rl @@ -0,0 +1,44 @@ +/* + * @LANG: java + */ + +class erract8 +{ + %%{ + machine erract8; + + action on_char { System.out.println("char: " + data[p]); } + action on_err { System.out.println("err: " + data[p]); } + action to_state { System.out.println("to state: " + data[p]); } + + main := 'heXXX' $on_char $err(on_err) $to(to_state); + }%% + + %% write data; + + static void test( char data[] ) + { + int cs, p = 0, pe = data.length; + int eof = pe; + int top; + + %% write init; + %% write exec; + + System.out.println("rest: " + data[p] + data[p+1] + data[p+2]); + } + + public static void main( String args[] ) + { + test( "hello".toCharArray() ); + } +} + +/* _____OUTPUT_____ +char: h +to state: h +char: e +to state: e +err: l +rest: llo +*/ diff --git a/test/erract9.rl b/test/erract9.rl new file mode 100644 index 0000000..ccd848a --- /dev/null +++ b/test/erract9.rl @@ -0,0 +1,43 @@ +# +# @LANG: ruby +# +# Test the host language scanning for ruby. +# + +%%{ + machine erract9; + + action on_char { print("char: ", data[p..p], "\n"); } + action on_err { print("err: ", data[p..p], "\n"); } + action to_state { print("to state: " , data[p..p], "\n"); } + + main := 'heXXX' $on_char $err(on_err) $to(to_state); +}%% + +%% write data; + +def run_machine( data ) + p = 0; + pe = data.length + cs = 0 + + %% write init; + %% write exec; + + print("rest: " , data[p..p+2], "\n") +end + +inp = [ + "hello\n", +] + +inp.each { |str| run_machine(str) } + +=begin _____OUTPUT_____ +char: h +to state: h +char: e +to state: e +err: l +rest: llo +=end _____OUTPUT_____ diff --git a/test/export1.rl b/test/export1.rl new file mode 100644 index 0000000..fe96141 --- /dev/null +++ b/test/export1.rl @@ -0,0 +1,59 @@ +/* + * @LANG: c + */ + +#include <stdio.h> +#include <string.h> + +%%{ + machine test; + + export c1 = 'c'; + export c2 = 'z'; + export c3 = 't'; + + commands := ( + c1 . digit* '\n' @{ printf( "c1\n" );} | + c2 . alpha* '\n' @{ printf( "c2\n" );}| + c3 . '.'* '\n' @{ printf( "c3\n" );} + )*; + + some_other := any*; +}%% + +%% write exports; +%% write data; + +int test( const char *data, int len ) +{ + int cs = test_en_commands; + const char *p = data, *pe = data + len; + + %% write init nocs; + %% write exec; + + if ( cs >= test_first_final ) + printf("ACCEPT\n"); + else + printf("ERROR\n"); + return 0; +} + +char data[] = { + test_ex_c1, '1', '2', '\n', + test_ex_c2, 'a', 'b', '\n', + test_ex_c3, '.', '.', '\n', 0 +}; + +int main() +{ + test( data, strlen( data ) ); + return 0; +} + +#ifdef _____OUTPUT_____ +c1 +c2 +c3 +ACCEPT +#endif diff --git a/test/export2.rl b/test/export2.rl new file mode 100644 index 0000000..881a4c9 --- /dev/null +++ b/test/export2.rl @@ -0,0 +1,57 @@ +/* + * @LANG: java + */ + +class export2 +{ + %%{ + machine test; + + export c1 = 'c'; + export c2 = 'z'; + export c3 = 't'; + + commands := ( + c1 . digit* '\n' @{ System.out.println( "c1" );} | + c2 . alpha* '\n' @{ System.out.println( "c2" );}| + c3 . '.'* '\n' @{ System.out.println( "c3" );} + )*; + + other := any*; + }%% + + %% write exports; + %% write data; + + static void test( char data[] ) + { + int cs = test_en_commands, p = 0, pe = data.length; + int top; + + %% write init nocs; + %% write exec; + + if ( cs >= test_first_final ) + System.out.println( "ACCEPT" ); + else + System.out.println( "FAIL" ); + } + + public static void main( String args[] ) + { + char data[] = { + test_ex_c1, '1', '2', '\n', + test_ex_c2, 'a', 'b', '\n', + test_ex_c3, '.', '.', '\n', + }; + test( data ); + } +} + + +/* _____OUTPUT_____ +c1 +c2 +c3 +ACCEPT +*/ diff --git a/test/export3.rl b/test/export3.rl new file mode 100644 index 0000000..dbf74b8 --- /dev/null +++ b/test/export3.rl @@ -0,0 +1,53 @@ +# +# @LANG: ruby +# + +%%{ + machine test; + + export c1 = 'c'; + export c2 = 'z'; + export c3 = 't'; + + commands := ( + c1 . digit* '\n' @{ puts "c1"; } | + c2 . alpha* '\n' @{ puts "c2"; }| + c3 . '.'* '\n' @{ puts "c3"; } + )*; + + other := any*; +}%% + +%% write exports; +%% write data; + +def run_machine( data ) + p = 0; + pe = data.length + cs = test_en_commands + val = 0; + neg = false; + + %% write init nocs; + %% write exec; + if cs >= test_first_final + puts "ACCEPT" + else + puts "FAIL" + end +end + +inp = [ + test_ex_c1, ?1, ?2, ?\n, + test_ex_c2, ?a, ?b, ?\n, + test_ex_c3, ?., ?., ?\n +] + +run_machine( inp ); + +=begin _____OUTPUT_____ +c1 +c2 +c3 +ACCEPT +=end _____OUTPUT_____ diff --git a/test/export4.rl b/test/export4.rl new file mode 100644 index 0000000..94d50e4 --- /dev/null +++ b/test/export4.rl @@ -0,0 +1,59 @@ +/* + * @LANG: d + */ + +import std.c.stdio; +import std.string; + +%%{ + machine test; + + export c1 = 'c'; + export c2 = 'z'; + export c3 = 't'; + + commands := ( + c1 . digit* '\n' @{ printf( "c1\n" );} | + c2 . alpha* '\n' @{ printf( "c2\n" );}| + c3 . '.'* '\n' @{ printf( "c3\n" );} + )*; + + some_other := any*; +}%% + +%% write exports; +%% write data; + +int test( char data[] ) +{ + int cs = test_en_commands; + char *p = data.ptr, pe = data.ptr + data.length; + + %% write init nocs; + %% write exec; + + if ( cs >= test_first_final ) + printf("ACCEPT\n"); + else + printf("ERROR\n"); + return 0; +} + +char data[] = [ + test_ex_c1, '1', '2', '\n', + test_ex_c2, 'a', 'b', '\n', + test_ex_c3, '.', '.', '\n' +]; + +int main() +{ + test( data ); + return 0; +} + +/+ _____OUTPUT_____ +c1 +c2 +c3 +ACCEPT +++++++++++++++++++/ diff --git a/test/fnext1.rl b/test/fnext1.rl new file mode 100644 index 0000000..a2925eb --- /dev/null +++ b/test/fnext1.rl @@ -0,0 +1,81 @@ +/* + * @LANG: c + * + * Tests fnext in combination with fbreak. + */ + +#include <string.h> +#include <stdio.h> + +char comm; +int top; +int stack [32]; + +%%{ + machine fnext; + action break {fbreak;} + + main := 'h' @{ /*h*/ fnext e; fbreak; }; + e := 'e' @{ /*e*/ fnext l; } @{ fbreak; }; + l := 'll' @{ /*ll*/ fnext o; } ${ fbreak; }; + o := |* 'o' { /*o*/ fnext nl; fbreak; }; *|; + nl := '\n' @{ /*nl*/ fbreak; printf("ACCEPT\n"); }; +}%% + +int cs; +char *ts, *te; +int act; + +%% write data; + +void init() +{ + %% write init; +} + +void exec( char *data, int len ) +{ + char *p = data; + char *pe = data + len; + + while ( cs != fnext_error && p < pe ) { + printf( "%c\n", *p ); + %% write exec; + } +} + +void finish( ) +{ + if ( cs >= fnext_first_final ) + printf( "ACCEPT\n" ); + else + printf( "FAIL\n" ); +} + +char *inp[] = { + "hello\n" +}; + +int inplen = 1; + +int main( ) +{ + int i; + for ( i = 0; i < inplen; i++ ) { + init(); + exec( inp[i], strlen(inp[i]) ); + finish(); + } + return 0; +} + +#ifdef _____OUTPUT_____ +h +e +l +l +o + + +ACCEPT +#endif diff --git a/test/forder1.rl b/test/forder1.rl index a6366cb..9873af4 100644 --- a/test/forder1.rl +++ b/test/forder1.rl @@ -12,7 +12,7 @@ struct forder %%{ machine forder; - variable curstate fsm->cs; + variable cs fsm->cs; second = 'b' >{printf("enter b1\n");} @@ -44,8 +44,6 @@ void forder_execute( struct forder *fsm, const char *_data, int _len ) int forder_finish( struct forder *fsm ) { - %% write eof; - if ( fsm->cs == forder_error ) return -1; if ( fsm->cs >= forder_first_final ) diff --git a/test/forder2.rl b/test/forder2.rl index 9592179..d92f888 100644 --- a/test/forder2.rl +++ b/test/forder2.rl @@ -17,7 +17,7 @@ struct forder %%{ machine forder; - variable curstate fsm->cs; + variable cs fsm->cs; inner = 'inner' >{printf("enter inner\n");} @@ -51,8 +51,6 @@ void forder_execute( struct forder *fsm, const char *_data, int _len ) int forder_finish( struct forder *fsm ) { - %% write eof; - if ( fsm->cs == forder_error ) return -1; if ( fsm->cs >= forder_first_final ) diff --git a/test/forder3.rl b/test/forder3.rl index 7a659bb..5cb3725 100644 --- a/test/forder3.rl +++ b/test/forder3.rl @@ -12,7 +12,7 @@ struct forder %%{ machine forder; - variable curstate fsm->cs; + variable cs fsm->cs; m1 = ( "" %{printf("enter m1 aa\n");} | 'aa'* >{printf("enter m1 aa\n");} %{printf("leave m1 aa\n");} ) @@ -23,7 +23,7 @@ struct forder main := ( m1 %{printf("accept m1\n");} | "" %{printf("enter m2\n");} | - m2 >{printf("enter m2\n");} %{printf("accpet m2\n");} + m2 >{printf("enter m2\n");} %{printf("accept m2\n");} ) . '\n'; }%% @@ -44,8 +44,6 @@ void forder_execute( struct forder *fsm, const char *_data, int _len ) int forder_finish( struct forder *fsm ) { - %% write eof; - if ( fsm->cs == forder_error ) return -1; if ( fsm->cs >= forder_first_final ) @@ -85,20 +83,23 @@ through m1 b accept m1 ACCEPT enter m2 -accpet m2 +enter m2 +accept m2 ACCEPT enter m1 aa +enter m1 aa leave m1 aa through m1 b enter m2 accept m1 ACCEPT enter m1 aa +enter m1 aa leave m1 aa through m1 b enter m2 accept m1 -accpet m2 +accept m2 ACCEPT enter m1 aa enter m2 diff --git a/test/gotocallret1.rl b/test/gotocallret1.rl index 54626dd..8b294dd 100644 --- a/test/gotocallret1.rl +++ b/test/gotocallret1.rl @@ -106,6 +106,7 @@ correct command ACCEPT correct command ACCEPT +error: failed to recover FAIL error: garbling line error: failed to recover diff --git a/test/gotocallret2.rl b/test/gotocallret2.rl index 4a3bc0e..5b4f740 100644 --- a/test/gotocallret2.rl +++ b/test/gotocallret2.rl @@ -5,8 +5,8 @@ char comm; int top; int stack[32]; -ptr tokstart; -ptr tokend; +ptr ts; +ptr te; int act; int val; %% diff --git a/test/high1.rl b/test/high1.rl index 2ad1b60..9179c89 100644 --- a/test/high1.rl +++ b/test/high1.rl @@ -19,7 +19,7 @@ struct high %%{ machine high; - variable curstate fsm->cs; + variable cs fsm->cs; # We Want the header portion. alphtype unsigned int; @@ -49,8 +49,6 @@ void high_execute( struct high *fsm, const unsigned int *_data, int _len ) int high_finish( struct high *fsm ) { - %% write eof; - if ( fsm->cs == high_error ) return -1; if ( fsm->cs >= high_first_final ) @@ -74,7 +72,7 @@ struct tokenizer %%{ machine tokenizer; - variable curstate fsm->cs; + variable cs fsm->cs; action bufdigit { if ( numlen < 8 ) @@ -119,14 +117,13 @@ void tokenizer_execute( struct tokenizer *fsm, const char *_data, int _len ) { const char *p = _data; const char *pe = _data+_len; + const char *eof = pe; %% write exec; } int tokenizer_finish( struct tokenizer *fsm ) { - %% write eof; - if ( fsm->cs == tokenizer_error ) return -1; if ( fsm->cs >= tokenizer_first_final ) diff --git a/test/high2.rl b/test/high2.rl index 1aeb9b3..53f1d80 100644 --- a/test/high2.rl +++ b/test/high2.rl @@ -67,7 +67,6 @@ int Fsm::execute( const unsigned char *_data, int _len ) int Fsm::finish() { - %% write eof; if ( cs == Fsm_error ) return -1; if ( cs >= Fsm_first_final ) diff --git a/test/high3.rl b/test/high3.rl index 03d2a74..d915bff 100644 --- a/test/high3.rl +++ b/test/high3.rl @@ -67,7 +67,6 @@ - (int) finish; { - %% write eof; if ( cs == Fsm_error ) return -1; else if ( cs >= Fsm_first_final ) diff --git a/test/import1.rl b/test/import1.rl new file mode 100644 index 0000000..e721f56 --- /dev/null +++ b/test/import1.rl @@ -0,0 +1,73 @@ +/* + * @LANG: c + */ + +#include <stdio.h> + +char *foo = "foo"; + +char b = 98; +char a = 97; +char r = 114; + +#define SP 32 +#define NL '\n' + +%%{ + machine tmp; + import "import1.rl"; + + foobar = + foo @{printf("foo\n"); } | + b a r @{printf("bar\n");}; + + main := ( foobar SP foobar NL )*; +}%% + +%% write data; + +int cs; + +void exec_str( char *p, int len ) +{ + char *pe = p + len; + %% write exec; +} + +void exec_c( char c ) +{ + exec_str( &c, 1 ); +} + +int main() +{ + %% write init; + + exec_str( foo, 3 ); + exec_c( SP ); + exec_c( b ); + exec_c( a ); + exec_c( r ); + exec_c( NL ); + + exec_c( b ); + exec_c( a ); + exec_c( r ); + exec_c( SP ); + exec_str( foo, 3 ); + exec_c( NL ); + + if ( cs < tmp_first_final ) + printf("FAIL\n"); + else + printf("ACCEPT\n"); + + return 0; +} +#ifdef _____OUTPUT_____ +foo +bar +bar +foo +ACCEPT +#endif diff --git a/test/java1.rl b/test/java1.rl index 128386f..461d177 100644 --- a/test/java1.rl +++ b/test/java1.rl @@ -1,6 +1,5 @@ /* * @LANG: java - * @ALLOW_GENFLAGS: -T0 */ class java1 diff --git a/test/java2.rl b/test/java2.rl index 61d9ac9..f308902 100644 --- a/test/java2.rl +++ b/test/java2.rl @@ -1,6 +1,5 @@ /* * @LANG: java - * @ALLOW_GENFLAGS: -T0 */ class java2 diff --git a/test/keller1.rl b/test/keller1.rl index 94d25b7..646cec9 100644 --- a/test/keller1.rl +++ b/test/keller1.rl @@ -777,7 +777,6 @@ int Parser::execute( LangEl *_data, int _len ) int Parser::finish( ) { - %% write eof; if ( cs == Parser_error ) return -1; if ( cs >= Parser_first_final ) diff --git a/test/langtrans_c.sh b/test/langtrans_c.sh index 7d9cf41..2523b0a 100755 --- a/test/langtrans_c.sh +++ b/test/langtrans_c.sh @@ -5,17 +5,28 @@ file=$1 [ -f $file ] || exit 1 -# Get the amchine name. +# Get the machine name. machine=`sed -n 's/^[\t ]*machine[\t ]*\([a-zA-Z_0-9]*\)[\t ]*;[\t ]*$/\1/p' $file` -# Make a temporary version of the test case the C language translations. +# Make a temporary version of the test case using the C language translations. sed -n '/\/\*/,/\*\//d;p' $file | txl -q stdin langtrans_c.txl > $file.pr +needs_eof=`sed '/@NEEDS_EOF/s/^.*$/yes/p;d' $file` +if [ "$needs_eof" != 'yes' ]; then + needs_eof=`sed -n '/\/\*/,/\*\//d;p' $file | txl -q stdin checkeofact.txl` +fi + # Begin writing out the test case. cat << EOF /* * @LANG: c * @GENERATED: yes +EOF + +grep '@ALLOW_GENFLAGS:' $file +grep '@ALLOW_MINFLAGS:' $file + +cat << EOF */ #include <string.h> #include <stdio.h> @@ -35,7 +46,7 @@ void init() { EOF -sed -n '1,/^%%$/d; /^%%{$/q; {s/^/\t/;p}' $file.pr +sed -n '0,/^%%$/d; /^%%{$/q; {s/^/\t/;p}' $file.pr cat << EOF %% write init; @@ -45,12 +56,16 @@ void exec( char *data, int len ) { char *p = data; char *pe = data + len; +EOF + +[ "$needs_eof" = "yes" ] && echo "char *eof = pe;" + +cat << EOF %% write exec; } void finish( ) { - %% write eof; if ( cs >= ${machine}_first_final ) printf( "ACCEPT\\n" ); else @@ -59,7 +74,7 @@ void finish( ) EOF # Write out the test data. -sed -n '1,/\/\* _____INPUT_____/d; /_____INPUT_____ \*\//q; p;' $file | awk ' +sed -n '0,/\/\* _____INPUT_____/d; /_____INPUT_____ \*\//q; p;' $file | awk ' BEGIN { print "char *inp[] = {" } @@ -89,7 +104,7 @@ int main( ) EOF # Write out the expected output. -sed -n '1,/\/\* _____OUTPUT_____/d; /_____OUTPUT_____ \*\//q; p;' $file +sed -n '0,/\/\* _____OUTPUT_____/d; /_____OUTPUT_____ \*\//q; p;' $file echo "#endif" # Don't need this language-specific file anymore. diff --git a/test/langtrans_c.txl b/test/langtrans_c.txl index 831350c..e80e508 100644 --- a/test/langtrans_c.txl +++ b/test/langtrans_c.txl @@ -74,6 +74,12 @@ redefine al_host_block | '{ [NL] [IN] [c_statements] [EX] '} [NL] end define +redefine cond_action_stmt + 'action [id] '{ [al_expr] '} [NL] + | 'action [id] '{ [c_expr] '} [NL] +end redefine + + rule boolTypes replace [al_type_decl] 'bool @@ -116,6 +122,13 @@ function alStmtToC1 AlStmt [action_lang_stmt] Result end function +function alTermToC + replace [al_term] + 'first_token_char + by + 'ts '[0] +end function + function alExprExtendToC AlExprExtend [repeat al_expr_extend] deconstruct AlExprExtend Op [al_expr_op] Term [al_term] Rest [repeat al_expr_extend] @@ -123,7 +136,7 @@ function alExprExtendToC AlExprExtend [repeat al_expr_extend] _ [alExprExtendToC Rest] replace [repeat c_expr_extend] by - Op Term RestC + Op Term [alTermToC] RestC end function function alExprToC AlExpr [al_expr] @@ -132,7 +145,7 @@ function alExprToC AlExpr [al_expr] construct CExprExtend [repeat c_expr_extend] _ [alExprExtendToC AlExprExtend] construct Result [opt c_expr] - ALTerm CExprExtend + ALTerm [alTermToC] CExprExtend replace [opt c_expr] by Result [boolVals1] [boolVals2] @@ -206,6 +219,22 @@ function alStmtToC4b AlStmt [action_lang_stmt] 'fputs '( String , 'stdout '); end function +function alStmtToC4c AlStmt [action_lang_stmt] + deconstruct AlStmt + 'printb Id [id] '; + replace [repeat c_lang_stmt] + by + 'fwrite '( Id ', '1 ', 'pos ', 'stdout '); +end function + +function alStmtToC4d AlStmt [action_lang_stmt] + deconstruct AlStmt + 'print_token '; + replace [repeat c_lang_stmt] + by + 'fwrite '( 'ts ', '1 ', 'te '- 'ts ', 'stdout '); +end function + function alStmtToC5 AlStmt [action_lang_stmt] deconstruct AlStmt '{ AlSubStmts [repeat action_lang_stmt] '} @@ -234,6 +263,8 @@ function alToC AlStmts [repeat action_lang_stmt] [alStmtToC3 FirstStmt] [alStmtToC4a FirstStmt] [alStmtToC4b FirstStmt] + [alStmtToC4c FirstStmt] + [alStmtToC4d FirstStmt] [alStmtToC5 FirstStmt] [alStmtToC6 FirstStmt] construct RestC [repeat c_lang_stmt] @@ -252,6 +283,17 @@ rule actionTransC '{ CStmts '} end rule +rule condTransC + replace [cond_action_stmt] + 'action Id [id] '{ AlExpr [al_expr] '} + construct OptCExpr [opt c_expr] + _ [alExprToC AlExpr] + deconstruct OptCExpr + CExpr [c_expr] + by + 'action Id '{ CExpr '} +end rule + function langTransC replace [program] Definitions [repeat action_lang_stmt] @@ -266,7 +308,7 @@ function langTransC CDefinitions '%% CInitializations - RagelDef [actionTransC] + RagelDef [actionTransC] [condTransC] end function function main diff --git a/test/langtrans_csharp.sh b/test/langtrans_csharp.sh new file mode 100755 index 0000000..3980f48 --- /dev/null +++ b/test/langtrans_csharp.sh @@ -0,0 +1,109 @@ +#!/bin/bash +# + +file=$1 + +[ -f $file ] || exit 1 +root=${file%.rl} +class=${root}_csharp + +# Make a temporary version of the test case using the Java language translations. +sed -n '/\/\*/,/\*\//d;p' $file | txl -q stdin langtrans_csharp.txl - $class > $file.pr + +# Begin writing out the test case. +cat << EOF +/* + * @LANG: csharp + * @GENERATED: yes +EOF + +grep '@ALLOW_GENFLAGS:' $file | sed 's/-G2//g' +grep '@ALLOW_MINFLAGS:' $file + +cat << EOF + */ +using System; +// Disables lots of warnings that appear in the test suite +#pragma warning disable 0168, 0169, 0219, 0162, 0414 +namespace Test { +class $class +{ +EOF + +# Write the data declarations +sed -n '/^%%$/q;{s/^/\t/;p}' $file.pr + +# Write out the machine specification. +sed -n '/^%%{$/,/^}%%/{s/^/\t/;p}' $file.pr + +# Write out the init and execute routines. +cat << EOF + + int cs; + %% write data; + + void init() + { +EOF + +sed -n '0,/^%%$/d; /^%%{$/q; {s/^/\t\t/;p}' $file.pr + +cat << EOF + %% write init; + } + + void exec( char[] data, int len ) + { + int p = 0; + int pe = len; + int eof = len; + string _s; + %% write exec; + } + + void finish( ) + { + if ( cs >= ${class}_first_final ) + Console.WriteLine( "ACCEPT" ); + else + Console.WriteLine( "FAIL" ); + } + +EOF + +# Write out the test data. +sed -n '0,/\/\* _____INPUT_____/d; /_____INPUT_____ \*\//q; p;' $file | awk ' +BEGIN { + print " static readonly string[] inp = {" +} +{ + print " " $0 "," +} +END { + print " };" + print "" + print " static readonly int inplen = " NR ";" +}' + + +# Write out the main routine. +cat << EOF + + public static void Main (string[] args) + { + $class machine = new $class(); + for ( int i = 0; i < inplen; i++ ) { + machine.init(); + machine.exec( inp[i].ToCharArray(), inp[i].Length ); + machine.finish(); + } + } +} +} +EOF + +# Write out the expected output. +sed -n '/\/\* _____OUTPUT_____/,/_____OUTPUT_____ \*\//p;' $file + +# Don't need this language-specific file anymore. +rm $file.pr diff --git a/test/langtrans_csharp.txl b/test/langtrans_csharp.txl new file mode 100644 index 0000000..cf4a2c1 --- /dev/null +++ b/test/langtrans_csharp.txl @@ -0,0 +1,342 @@ +include "testcase.txl" + +keys + 'bool 'new +end keys + + +define csharp_statements + [repeat csharp_lang_stmt] +end define + +define csharp_lang_stmt + [al_ragel_stmt] + | [csharp_variable_decl] + | [csharp_expr_stmt] + | [csharp_if_stmt] + | [EX] '{ [IN] [NL] [csharp_statements] [EX] '} [IN] [NL] +end define + +define csharp_variable_decl + [csharp_type_decl] [opt union] [id] '; [NL] +end define + +define csharp_type_decl + [al_type_decl] + | 'bool + | 'String +end define + +define csharp_expr_stmt + [csharp_expr] '; [NL] +end define + +define csharp_expr + [csharp_term] [repeat csharp_expr_extend] +end define + +define csharp_expr_extend + [al_expr_op] [csharp_term] +end define + +define csharp_term + [al_term] + | [id] [repeat csharp_dot_id] + | [id] [repeat csharp_dot_id] '( [csharp_args] ') + | 'new [csharp_type_decl] [union] + | 'new [csharp_type_decl] '( [csharp_args] ') +end define + +define csharp_dot_id + '. [id] +end define + +define csharp_args + [list csharp_expr] +end define + +define csharp_sign + '- | '+ +end define + +define csharp_if_stmt + 'if '( [csharp_expr] ') [NL] [IN] + [csharp_lang_stmt] [EX] + [opt csharp_else] +end define + +define csharp_else + 'else [NL] [IN] + [csharp_lang_stmt] [EX] +end define + +define csharp_lang + [csharp_statements] + '%% [NL] + [csharp_statements] + [ragel_def] +end define + +define program + [lang_indep] + | [csharp_lang] +end define + +redefine al_host_block + '{ [NL] [IN] [al_statements] [EX] '} [NL] + | '{ [NL] [IN] [csharp_statements] [EX] '} [NL] +end define + +redefine cond_action_stmt + 'action [id] '{ [al_expr] '} [NL] + | 'action [id] '{ [csharp_expr] '} [NL] +end redefine + + +function clearUnion Type [csharp_type_decl] Id [id] + replace [opt union] + Union [union] + import ArrayInits [csharp_statements] + Stmts [repeat csharp_lang_stmt] + export ArrayInits + Id '= 'new Type Union '; Stmts + by + '[] +end function + +rule ptrTypes + replace [al_type_decl] + 'ptr + by + 'int +end rule + +function alStmtToCSharp1 AlStmt [action_lang_stmt] + deconstruct AlStmt + VarDecl [al_variable_decl] + deconstruct VarDecl + Type [al_type_decl] Id [id] OptUnion [opt union] '; + construct CSharpType [csharp_type_decl] + Type + construct Result [csharp_variable_decl] + CSharpType [ptrTypes] OptUnion [clearUnion CSharpType Id] Id '; + replace [repeat csharp_lang_stmt] + by + Result +end function + +function alTermToCSharp + replace [al_term] + 'first_token_char + by + 'data '[ts] +end function + +function alExprExtendToCSharp AlExprExtend [repeat al_expr_extend] + deconstruct AlExprExtend + Op [al_expr_op] Term [al_term] Rest [repeat al_expr_extend] + construct CSharpRest [repeat csharp_expr_extend] + _ [alExprExtendToCSharp Rest] + replace [repeat csharp_expr_extend] + by + Op Term [alTermToCSharp] CSharpRest +end function + +function alExprToCSharp AlExpr [al_expr] + deconstruct AlExpr + ALTerm [al_term] AlExprExtend [repeat al_expr_extend] + construct CSharpExprExtend [repeat csharp_expr_extend] + _ [alExprExtendToCSharp AlExprExtend] + construct Result [opt csharp_expr] + ALTerm [alTermToCSharp] CSharpExprExtend + replace [opt csharp_expr] + by + Result +end function + +function alStmtToCSharp2 AlStmt [action_lang_stmt] + deconstruct AlStmt + AlExpr [al_expr] '; + construct OptCSharpExpr [opt csharp_expr] + _ [alExprToCSharp AlExpr] + deconstruct OptCSharpExpr + CSharpExpr [csharp_expr] + replace [repeat csharp_lang_stmt] + by + CSharpExpr '; +end function + +function alOptElseCSharp AlOptElse [opt al_else] + deconstruct AlOptElse + 'else + AlSubStmt [action_lang_stmt] + construct AlSubStmts [repeat action_lang_stmt] + AlSubStmt + construct CSharpSubStmts [repeat csharp_lang_stmt] + _ [alToCSharp AlSubStmts] + deconstruct CSharpSubStmts + CSharpSubStmt [csharp_lang_stmt] + replace [opt csharp_else] + by + 'else + CSharpSubStmt +end function + +function alStmtToCSharp3 AlStmt [action_lang_stmt] + deconstruct AlStmt + 'if '( AlExpr [al_expr] ') + AlSubStmt [action_lang_stmt] + AlOptElse [opt al_else] + construct OptCSharpExpr [opt csharp_expr] + _ [alExprToCSharp AlExpr] + deconstruct OptCSharpExpr + CSharpExpr [csharp_expr] + construct AlSubStmts [repeat action_lang_stmt] + AlSubStmt + construct CSharpSubStmts [repeat csharp_lang_stmt] + _ [alToCSharp AlSubStmts] + deconstruct CSharpSubStmts + CSharpSubStmt [csharp_lang_stmt] + construct OptCSharpElse [opt csharp_else] + _ [alOptElseCSharp AlOptElse] + replace [repeat csharp_lang_stmt] + by + 'if '( CSharpExpr ') + CSharpSubStmt + OptCSharpElse +end function + +function alStmtToCSharp4a AlStmt [action_lang_stmt] + deconstruct AlStmt + 'printi Id [id] '; + replace [repeat csharp_lang_stmt] + by + 'Console '. 'Write '( Id '); +end function + +function alStmtToCSharp4b AlStmt [action_lang_stmt] + deconstruct AlStmt + 'prints String [stringlit] '; + replace [repeat csharp_lang_stmt] + by + 'Console '. 'Write '( String '); +end function + +function alStmtToCSharp4c AlStmt [action_lang_stmt] + deconstruct AlStmt + 'printb Id [id] '; + replace [repeat csharp_lang_stmt] + by + '_s '= 'new 'String '( Id ', '0 ', 'pos ') '; + 'Console '. 'Write '( '_s '); +end function + +function alStmtToCSharp4d AlStmt [action_lang_stmt] + deconstruct AlStmt + 'print_token '; + replace [repeat csharp_lang_stmt] + by + '_s '= 'new 'String '( 'data ', 'ts ', 'te '- 'ts ') '; + 'Console '. 'Write '( '_s '); +end function + +function alStmtToCSharp5 AlStmt [action_lang_stmt] + deconstruct AlStmt + '{ AlSubStmts [repeat action_lang_stmt] '} + construct CSharpSubStmts [repeat csharp_lang_stmt] + _ [alToCSharp AlSubStmts] + replace [repeat csharp_lang_stmt] + by + '{ CSharpSubStmts '} +end function + +function alStmtToCSharp6 AlStmt [action_lang_stmt] + deconstruct AlStmt + RagelStmt [al_ragel_stmt] + replace [repeat csharp_lang_stmt] + by + RagelStmt +end function + + +function alToCSharp AlStmts [repeat action_lang_stmt] + deconstruct AlStmts + FirstStmt [action_lang_stmt] Rest [repeat action_lang_stmt] + construct CSharpFirst [repeat csharp_lang_stmt] + _ + [alStmtToCSharp1 FirstStmt] + [alStmtToCSharp2 FirstStmt] + [alStmtToCSharp3 FirstStmt] + [alStmtToCSharp4a FirstStmt] + [alStmtToCSharp4b FirstStmt] + [alStmtToCSharp4c FirstStmt] + [alStmtToCSharp4d FirstStmt] + [alStmtToCSharp5 FirstStmt] + [alStmtToCSharp6 FirstStmt] + construct CSharpRest [repeat csharp_lang_stmt] + _ [alToCSharp Rest] + replace [repeat csharp_lang_stmt] + by + CSharpFirst [. CSharpRest] +end function + +rule actionTransCSharp + replace [al_host_block] + '{ AlStmts [repeat action_lang_stmt] '} + construct CSharpStmts [repeat csharp_lang_stmt] + _ [alToCSharp AlStmts] + by + '{ CSharpStmts '} +end rule + +rule condTransCSharp + replace [cond_action_stmt] + 'action Id [id] '{ AlExpr [al_expr] '} + construct OptCSharpExpr [opt csharp_expr] + _ [alExprToCSharp AlExpr] + deconstruct OptCSharpExpr + CSharpExpr [csharp_expr] + by + 'action Id '{ CSharpExpr '} +end rule + +rule machineName + replace $ [machine_stmt] + 'machine _ [id] '; + import TXLargs [repeat stringlit] + Arg1 [stringlit] _ [repeat stringlit] + construct ClassName [id] + _ [unquote Arg1] + by + 'machine ClassName '; +end rule + +function langTransCSharp + replace [program] + Definitions [repeat action_lang_stmt] + '%% + Initializations [repeat action_lang_stmt] + RagelDef [ragel_def] + construct CSharpDefinitions [repeat csharp_lang_stmt] + _ [alToCSharp Definitions] + construct CSharpInitializations [repeat csharp_lang_stmt] + _ [alToCSharp Initializations] + construct NewRagelDef [ragel_def] + RagelDef [actionTransCSharp] [condTransCSharp] [machineName] + import ArrayInits [csharp_statements] + ArrayInitStmts [repeat csharp_lang_stmt] + by + CSharpDefinitions + '%% + ArrayInitStmts [. CSharpInitializations] + NewRagelDef +end function + +function main + replace [program] + P [program] + export ArrayInits [csharp_statements] + _ + by + P [langTransCSharp] +end function diff --git a/test/langtrans_d.sh b/test/langtrans_d.sh index 117e50a..764afd7 100755 --- a/test/langtrans_d.sh +++ b/test/langtrans_d.sh @@ -16,6 +16,12 @@ cat << EOF /* * @LANG: d * @GENERATED: yes +EOF + +grep '@ALLOW_GENFLAGS:' $file +grep '@ALLOW_MINFLAGS:' $file + +cat << EOF */ import std.stdio; import std.string; @@ -38,22 +44,24 @@ cat << EOF { EOF -sed -n '1,/^%%$/d; /^%%{$/q; {s/^/\t\t/;p}' $file.pr +sed -n '0,/^%%$/d; /^%%{$/q; {s/^/\t\t/;p}' $file.pr cat << EOF %% write init; } - void exec( char *data, int len ) + void exec( char data[] ) { - char *p = data; - char *pe = data + len; + char *p = data.ptr; + char *pe = data.ptr + data.length; + char *eof = pe; + char _s[]; + %% write exec; } void finish( ) { - %% write eof; if ( cs >= ${machine}_first_final ) writefln( "ACCEPT" ); else @@ -63,7 +71,7 @@ cat << EOF EOF # Write out the test data. -sed -n '1,/\/\* _____INPUT_____/d; /_____INPUT_____ \*\//q; p;' $file | awk ' +sed -n '0,/\/\* _____INPUT_____/d; /_____INPUT_____ \*\//q; p;' $file | awk ' BEGIN { print " char[][] inp = [" } @@ -86,7 +94,7 @@ int main( ) int i; for ( i = 0; i < m.inplen; i++ ) { m.init(); - m.exec( m.inp[i], m.inp[i].length ); + m.exec( m.inp[i] ); m.finish(); } return 0; @@ -95,7 +103,7 @@ int main( ) EOF # Write out the expected output. -sed -n '1,/\/\* _____OUTPUT_____/d; /_____OUTPUT_____ \*\//q; p;' $file +sed -n '0,/\/\* _____OUTPUT_____/d; /_____OUTPUT_____ \*\//q; p;' $file echo "*/" # Don't need this language-specific file anymore. diff --git a/test/langtrans_d.txl b/test/langtrans_d.txl index a9151f9..3d8b8a7 100644 --- a/test/langtrans_d.txl +++ b/test/langtrans_d.txl @@ -95,6 +95,13 @@ function alStmtToD1 AlStmt [action_lang_stmt] Result end function +function alTermToD + replace [al_term] + 'first_token_char + by + 'ts '[0] +end function + function alExprExtendToD AlExprExtend [repeat al_expr_extend] deconstruct AlExprExtend Op [al_expr_op] Term [al_term] Rest [repeat al_expr_extend] @@ -102,7 +109,7 @@ function alExprExtendToD AlExprExtend [repeat al_expr_extend] _ [alExprExtendToD Rest] replace [repeat d_expr_extend] by - Op Term DRest + Op Term [alTermToD] DRest end function function alExprToD AlExpr [al_expr] @@ -111,7 +118,7 @@ function alExprToD AlExpr [al_expr] construct DExprExtend [repeat d_expr_extend] _ [alExprExtendToD AlExprExtend] construct Result [opt d_expr] - ALTerm DExprExtend + ALTerm [alTermToD] DExprExtend replace [opt d_expr] by Result @@ -174,7 +181,7 @@ function alStmtToD4a AlStmt [action_lang_stmt] 'printi Id [id] '; replace [repeat d_lang_stmt] by - 'writef '( '"%d" ', Id '); + 'writef '( '"%d" ', Id ') '; end function function alStmtToD4b AlStmt [action_lang_stmt] @@ -182,7 +189,25 @@ function alStmtToD4b AlStmt [action_lang_stmt] 'prints String [stringlit] '; replace [repeat d_lang_stmt] by - 'writef '( '"%s" ', String '); + 'writef '( '"%s" ', String ') '; +end function + +function alStmtToD4c AlStmt [action_lang_stmt] + deconstruct AlStmt + 'printb Id [id] '; + replace [repeat d_lang_stmt] + by + '_s '= Id '[0..pos] '; + 'writef '( '"%s" ', '_s ') '; +end function + +function alStmtToD4d AlStmt [action_lang_stmt] + deconstruct AlStmt + 'print_token '; + replace [repeat d_lang_stmt] + by + '_s '= ts '[0..(te-ts)] '; + 'writef '( '"%s" ', '_s ') '; end function function alStmtToD5 AlStmt [action_lang_stmt] @@ -213,6 +238,8 @@ function alToD AlStmts [repeat action_lang_stmt] [alStmtToD3 FirstStmt] [alStmtToD4a FirstStmt] [alStmtToD4b FirstStmt] + [alStmtToD4c FirstStmt] + [alStmtToD4d FirstStmt] [alStmtToD5 FirstStmt] [alStmtToD6 FirstStmt] construct DRest [repeat d_lang_stmt] diff --git a/test/langtrans_java.sh b/test/langtrans_java.sh index 65b6184..69a6f90 100755 --- a/test/langtrans_java.sh +++ b/test/langtrans_java.sh @@ -7,15 +7,20 @@ file=$1 root=${file%.rl} class=${root}_java -# Make a temporary version of the test case the Java language translations. +# Make a temporary version of the test case using the Java language translations. sed -n '/\/\*/,/\*\//d;p' $file | txl -q stdin langtrans_java.txl - $class > $file.pr # Begin writing out the test case. cat << EOF /* * @LANG: java - * @ALLOW_GENFLAGS: -T0 * @GENERATED: yes +EOF + +grep '@ALLOW_GENFLAGS:' $file +grep '@ALLOW_MINFLAGS:' $file + +cat << EOF */ class $class @@ -38,7 +43,7 @@ cat << EOF { EOF -sed -n '1,/^%%$/d; /^%%{$/q; {s/^/\t\t/;p}' $file.pr +sed -n '0,/^%%$/d; /^%%{$/q; {s/^/\t\t/;p}' $file.pr cat << EOF %% write init; @@ -48,12 +53,13 @@ cat << EOF { int p = 0; int pe = len; + int eof = len; + String _s; %% write exec; } void finish( ) { - %% write eof; if ( cs >= ${class}_first_final ) System.out.println( "ACCEPT" ); else @@ -63,7 +69,7 @@ cat << EOF EOF # Write out the test data. -sed -n '1,/\/\* _____INPUT_____/d; /_____INPUT_____ \*\//q; p;' $file | awk ' +sed -n '0,/\/\* _____INPUT_____/d; /_____INPUT_____ \*\//q; p;' $file | awk ' BEGIN { print " static final String inp[] = {" } diff --git a/test/langtrans_java.txl b/test/langtrans_java.txl index 3f1755d..8016ab2 100644 --- a/test/langtrans_java.txl +++ b/test/langtrans_java.txl @@ -24,6 +24,7 @@ end define define java_type_decl [al_type_decl] | 'boolean + | 'String end define define java_expr_stmt @@ -43,6 +44,7 @@ define java_term | [id] [repeat java_dot_id] | [id] [repeat java_dot_id] '( [java_args] ') | 'new [java_type_decl] [union] + | 'new [java_type_decl] '( [java_args] ') end define define java_dot_id @@ -85,6 +87,12 @@ redefine al_host_block | '{ [NL] [IN] [java_statements] [EX] '} [NL] end define +redefine cond_action_stmt + 'action [id] '{ [al_expr] '} [NL] + | 'action [id] '{ [java_expr] '} [NL] +end redefine + + function clearUnion Type [java_type_decl] Id [id] replace [opt union] Union [union] @@ -124,6 +132,13 @@ function alStmtToJava1 AlStmt [action_lang_stmt] Result end function +function alTermToJava + replace [al_term] + 'first_token_char + by + 'data '[ts] +end function + function alExprExtendToJava AlExprExtend [repeat al_expr_extend] deconstruct AlExprExtend Op [al_expr_op] Term [al_term] Rest [repeat al_expr_extend] @@ -131,7 +146,7 @@ function alExprExtendToJava AlExprExtend [repeat al_expr_extend] _ [alExprExtendToJava Rest] replace [repeat java_expr_extend] by - Op Term JavaRest + Op Term [alTermToJava] JavaRest end function function alExprToJava AlExpr [al_expr] @@ -140,7 +155,7 @@ function alExprToJava AlExpr [al_expr] construct JavaExprExtend [repeat java_expr_extend] _ [alExprExtendToJava AlExprExtend] construct Result [opt java_expr] - ALTerm JavaExprExtend + ALTerm [alTermToJava] JavaExprExtend replace [opt java_expr] by Result @@ -214,6 +229,24 @@ function alStmtToJava4b AlStmt [action_lang_stmt] 'System '. 'out '. 'print '( String '); end function +function alStmtToJava4c AlStmt [action_lang_stmt] + deconstruct AlStmt + 'printb Id [id] '; + replace [repeat java_lang_stmt] + by + '_s '= 'new 'String '( Id ', '0 ', 'pos ') '; + 'System '. 'out '. 'print '( '_s '); +end function + +function alStmtToJava4d AlStmt [action_lang_stmt] + deconstruct AlStmt + 'print_token '; + replace [repeat java_lang_stmt] + by + '_s '= 'new 'String '( 'data ', 'ts ', 'te '- 'ts ') '; + 'System '. 'out '. 'print '( '_s '); +end function + function alStmtToJava5 AlStmt [action_lang_stmt] deconstruct AlStmt '{ AlSubStmts [repeat action_lang_stmt] '} @@ -243,6 +276,8 @@ function alToJava AlStmts [repeat action_lang_stmt] [alStmtToJava3 FirstStmt] [alStmtToJava4a FirstStmt] [alStmtToJava4b FirstStmt] + [alStmtToJava4c FirstStmt] + [alStmtToJava4d FirstStmt] [alStmtToJava5 FirstStmt] [alStmtToJava6 FirstStmt] construct JavaRest [repeat java_lang_stmt] @@ -261,6 +296,17 @@ rule actionTransJava '{ JavaStmts '} end rule +rule condTransJava + replace [cond_action_stmt] + 'action Id [id] '{ AlExpr [al_expr] '} + construct OptJavaExpr [opt java_expr] + _ [alExprToJava AlExpr] + deconstruct OptJavaExpr + JavaExpr [java_expr] + by + 'action Id '{ JavaExpr '} +end rule + rule machineName replace $ [machine_stmt] 'machine _ [id] '; @@ -283,7 +329,7 @@ function langTransJava construct JavaInitializations [repeat java_lang_stmt] _ [alToJava Initializations] construct NewRagelDef [ragel_def] - RagelDef [actionTransJava] [machineName] + RagelDef [actionTransJava] [condTransJava] [machineName] import ArrayInits [java_statements] ArrayInitStmts [repeat java_lang_stmt] by diff --git a/test/langtrans_ruby.sh b/test/langtrans_ruby.sh new file mode 100755 index 0000000..355d8d4 --- /dev/null +++ b/test/langtrans_ruby.sh @@ -0,0 +1,96 @@ +#!/bin/bash +# + +file=$1 + +[ -f $file ] || exit 1 + +# Get the machine name. +machine=`sed -n 's/^[\t ]*machine[\t ]*\([a-zA-Z_0-9]*\)[\t ]*;[\t ]*$/\1/p' \ + $file | tr '[A-Z]' '[a-z]'` + +# Make a temporary version of the test case using the Ruby language translations. +sed -n '/\/\*/,/\*\//d;p' $file | txl -q stdin langtrans_ruby.txl > $file.pr + +# Begin writing out the test case. +cat << EOF +# +# @LANG: ruby +# @GENERATED: yes +EOF + +grep '@ALLOW_GENFLAGS:' $file | sed 's/^ *\*/#/' | sed 's/-G.//g' +grep '@ALLOW_MINFLAGS:' $file | sed 's/^ *\*/#/' + +cat << EOF +# + +EOF + +# Write out the machine specification. +sed -n '/^%%{$/,/^}%%/{s/^/\t/;p}' $file.pr + +# Write out the init and execute routines. +cat << EOF + + %% write data; + + def run_machine( data ) + p = 0 + pe = data.length + eof = data.length + cs = 0; +EOF + +# Write the data declarations +sed -n '/^%%$/q;{s/^/\t/;p}' $file.pr + +# Write the data initializations +sed -n '0,/^%%$/d; /^%%{$/q; {s/^/\t\t/;p}' $file.pr + +cat << EOF + + %% write init; + %% write exec; + if cs >= ${machine}_first_final + puts "ACCEPT" + else + puts "FAIL" + end + end + +EOF + +# Write out the test data. +sed -n '0,/\/\* _____INPUT_____/d; /_____INPUT_____ \*\//q; p;' $file | awk ' +BEGIN { + print " inp = [" +} +{ + print " " $0 "," +} +END { + print " ]" + print "" + print " inplen = " NR ";" +}' + + +# Write out the main routine. +cat << EOF + + inp.each { |str| + run_machine(str.unpack("c*")) + } + +EOF + +# Write out the expected output. +echo "=begin _____OUTPUT_____" + +sed -n '/\/\* _____OUTPUT_____/,/_____OUTPUT_____ \*\//{/_____OUTPUT_____/d;p;};' $file + +echo "=end _____OUTPUT_____" + +# Don't need this language-specific file anymore. +rm $file.pr diff --git a/test/langtrans_ruby.txl b/test/langtrans_ruby.txl new file mode 100644 index 0000000..265426f --- /dev/null +++ b/test/langtrans_ruby.txl @@ -0,0 +1,376 @@ +include "testcase.txl" + +keys + 'boolean 'new +end keys + + +define ruby_statements + [repeat ruby_lang_stmt] +end define + +define ruby_lang_stmt + [al_ragel_stmt] + | [ruby_expr_stmt] + | [ruby_if_stmt] + | [EX] 'do [IN] [NL] [ruby_statements] [EX] 'end [IN] [NL] +end define + +define ruby_type_decl + [al_type_decl] + | 'boolean +end define + +define ruby_expr_stmt + [ruby_expr] '; [NL] +end define + +define ruby_expr + [ruby_term] [repeat ruby_expr_extend] +end define + +define ruby_expr_extend + [al_expr_op] [ruby_term] +end define + +define ruby_term + [al_term] + | [stringlit] [union] + | [id] [repeat ruby_dot_id] + | [SPOFF] [id] [repeat ruby_dot_id] '( [SPON] [ruby_args] ') + | [union] +end define + +define ruby_dot_id + '. [id] +end define + +define ruby_args + [list ruby_expr] +end define + +define ruby_sign + '- | '+ +end define + +define ruby_if_stmt + 'if [ruby_expr] [NL] [IN] + [ruby_statements] [EX] + [opt ruby_else] + 'end [NL] +end define + +define ruby_else + 'else [NL] [IN] + [ruby_statements] [EX] +end define + +define ruby_lang + [ruby_statements] + '%% [NL] + [ruby_statements] + [ragel_def] +end define + +define program + [lang_indep] + | [ruby_lang] +end define + +redefine al_host_block + '{ [NL] [IN] [al_statements] [EX] '} [NL] + | '{ [NL] [IN] [ruby_statements] [EX] '} [NL] +end define + +redefine cond_action_stmt + 'action [id] '{ [al_expr] '} [NL] + | 'action [id] '{ [ruby_expr] '} [NL] +end redefine + +function initDecl1 VarDecl [al_variable_decl] + deconstruct VarDecl + 'bool Id [id] '; + replace [repeat ruby_lang_stmt] + by + Id '= 'false '; +end function + +function initDecl2 VarDecl [al_variable_decl] + deconstruct VarDecl + 'char Id [id] '; + replace [repeat ruby_lang_stmt] + by + Id '= ''c' '; +end function + +function initDecl3 VarDecl [al_variable_decl] + deconstruct VarDecl + 'int Id [id] '; + replace [repeat ruby_lang_stmt] + by + Id '= '0 '; +end function + +function initDecl4 VarDecl [al_variable_decl] + deconstruct VarDecl + 'ptr Id [id] '; + replace [repeat ruby_lang_stmt] + by + Id '= '-1 '; +end function + +function initDecl5 VarDecl [al_variable_decl] + deconstruct VarDecl + Type [al_type_decl] Id [id] Union [union] '; + replace [repeat ruby_lang_stmt] + by + Id '= '[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0] '; +end function + + +function alStmtToRuby1 AlStmt [action_lang_stmt] + deconstruct AlStmt + VarDecl [al_variable_decl] + deconstruct VarDecl + Type [al_type_decl] Id [id] OptUnion [opt union] '; + replace [repeat ruby_lang_stmt] + by + _ [initDecl1 VarDecl] [initDecl2 VarDecl] + [initDecl3 VarDecl] [initDecl4 VarDecl] + [initDecl5 VarDecl] +end function + +function alTermToRuby + replace [al_term] + 'first_token_char + by + 'data '[ts] +end function + +function alExprExtendToRuby AlExprExtend [repeat al_expr_extend] + deconstruct AlExprExtend + Op [al_expr_op] Term [al_term] Rest [repeat al_expr_extend] + construct RubyRest [repeat ruby_expr_extend] + _ [alExprExtendToRuby Rest] + replace [repeat ruby_expr_extend] + by + Op Term [alTermToRuby] RubyRest +end function + +% Note: this doesn't go into the ( al_expr ) form of al_term. +function alExprToRuby AlExpr [al_expr] + deconstruct AlExpr + ALTerm [al_term] AlExprExtend [repeat al_expr_extend] + construct RubyExprExtend [repeat ruby_expr_extend] + _ [alExprExtendToRuby AlExprExtend] + construct Result [opt ruby_expr] + ALTerm [alTermToRuby] RubyExprExtend + replace [opt ruby_expr] + by + Result +end function + +function alStmtToRuby2 AlStmt [action_lang_stmt] + deconstruct AlStmt + AlExpr [al_expr] '; + construct OptRubyExpr [opt ruby_expr] + _ [alExprToRuby AlExpr] + deconstruct OptRubyExpr + RubyExpr [ruby_expr] + replace [repeat ruby_lang_stmt] + by + RubyExpr '; +end function + +function liftBlock + replace [repeat ruby_lang_stmt] + 'do Block [repeat ruby_lang_stmt] 'end + by + Block +end function + +function alOptElseRuby AlOptElse [opt al_else] + deconstruct AlOptElse + 'else + AlSubStmt [action_lang_stmt] + construct AlSubStmts [repeat action_lang_stmt] + AlSubStmt + construct RubySubStmts [repeat ruby_lang_stmt] + _ [alToRuby AlSubStmts] + deconstruct RubySubStmts + RubySubStmt [ruby_lang_stmt] + replace [opt ruby_else] + by + 'else + RubySubStmts [liftBlock] +end function + +function alStmtToRuby3 AlStmt [action_lang_stmt] + deconstruct AlStmt + 'if '( AlExpr [al_expr] ') + AlSubStmt [action_lang_stmt] + AlOptElse [opt al_else] + construct OptRubyExpr [opt ruby_expr] + _ [alExprToRuby AlExpr] + deconstruct OptRubyExpr + RubyExpr [ruby_expr] + construct AlSubStmts [repeat action_lang_stmt] + AlSubStmt + construct RubySubStmts [repeat ruby_lang_stmt] + _ [alToRuby AlSubStmts] + construct OptRubyElse [opt ruby_else] + _ [alOptElseRuby AlOptElse] + replace [repeat ruby_lang_stmt] + by + 'if RubyExpr + RubySubStmts [liftBlock] + OptRubyElse + 'end +end function + +function alStmtToRuby4a AlStmt [action_lang_stmt] + deconstruct AlStmt + 'printi Id [id] '; + replace [repeat ruby_lang_stmt] + by + 'print '( Id ') '; +end function + +function alStmtToRuby4b AlStmt [action_lang_stmt] + deconstruct AlStmt + 'prints String [stringlit] '; + replace [repeat ruby_lang_stmt] + by + 'print '( String ') '; +end function + +function alStmtToRuby4c AlStmt [action_lang_stmt] + deconstruct AlStmt + 'printb Id [id] '; + replace [repeat ruby_lang_stmt] + by + '_a = Id '[0..pos-1] '; + 'print '( '_a '. 'pack '( '"c*" ') ') '; +end function + +function alStmtToRuby4d AlStmt [action_lang_stmt] + deconstruct AlStmt + 'print_token '; + replace [repeat ruby_lang_stmt] + by + '_m = 'data '[ts..te-1] '; + 'print '( '_m '. 'pack '( '"c*" ') ') '; +end function + +function alStmtToRuby5 AlStmt [action_lang_stmt] + deconstruct AlStmt + '{ AlSubStmts [repeat action_lang_stmt] '} + construct RubySubStmts [repeat ruby_lang_stmt] + _ [alToRuby AlSubStmts] + replace [repeat ruby_lang_stmt] + by + 'do RubySubStmts 'end +end function + +function alStmtToRuby6 AlStmt [action_lang_stmt] + deconstruct AlStmt + RagelStmt [al_ragel_stmt] + replace [repeat ruby_lang_stmt] + by + RagelStmt +end function + +rule fixCharLit + replace $ [al_term] + CharLit [charlit] + construct BaseId [id] + 'id + construct Id [id] + BaseId [unquote CharLit] + construct EmptyString [stringlit] + '"" + construct Repl [stringlit] + EmptyString [quote Id] + by + Repl '[0] +end rule + + +function alToRuby AlStmts [repeat action_lang_stmt] + deconstruct AlStmts + FirstStmt [action_lang_stmt] Rest [repeat action_lang_stmt] + construct RubyFirst [repeat ruby_lang_stmt] + _ + [alStmtToRuby1 FirstStmt] + [alStmtToRuby2 FirstStmt] + [alStmtToRuby3 FirstStmt] + [alStmtToRuby4a FirstStmt] + [alStmtToRuby4b FirstStmt] + [alStmtToRuby4c FirstStmt] + [alStmtToRuby4d FirstStmt] + [alStmtToRuby5 FirstStmt] + [alStmtToRuby6 FirstStmt] + [fixCharLit] + construct RubyRest [repeat ruby_lang_stmt] + _ [alToRuby Rest] + replace [repeat ruby_lang_stmt] + by + RubyFirst [. RubyRest] +end function + +rule actionTransRuby + replace [al_host_block] + '{ AlStmts [repeat action_lang_stmt] '} + construct RubyStmts [repeat ruby_lang_stmt] + _ [alToRuby AlStmts] + by + '{ RubyStmts '} +end rule + +rule condTransRuby + replace [cond_action_stmt] + 'action Id [id] '{ AlExpr [al_expr] '} + construct OptRubyExpr [opt ruby_expr] + _ [alExprToRuby AlExpr] + deconstruct OptRubyExpr + RubyExpr [ruby_expr] + by + 'action Id '{ RubyExpr '} +end rule + +rule lowercaseMachine + replace $ [machine_stmt] + 'machine Id [id] '; + by + 'machine Id [tolower] '; +end rule + +function langTransRuby + replace [program] + Definitions [repeat action_lang_stmt] + '%% + Initializations [repeat action_lang_stmt] + RagelDef [ragel_def] + construct RubyDefinitions [repeat ruby_lang_stmt] + _ [alToRuby Definitions] + construct RubyInitializations [repeat ruby_lang_stmt] + _ [alToRuby Initializations] + construct NewRagelDef [ragel_def] + RagelDef [actionTransRuby] [condTransRuby] [lowercaseMachine] + import ArrayInits [ruby_statements] + ArrayInitStmts [repeat ruby_lang_stmt] + by + RubyDefinitions + '%% + ArrayInitStmts [. RubyInitializations] + NewRagelDef +end function + +function main + replace [program] + P [program] + export ArrayInits [ruby_statements] + _ + by + P [langTransRuby] +end function diff --git a/test/lmgoto.rl b/test/lmgoto.rl index 96c4392..e8e82a8 100644 --- a/test/lmgoto.rl +++ b/test/lmgoto.rl @@ -40,11 +40,11 @@ using namespace std; struct Scanner { int cs, act; - char *tokstart, *tokend; + const char *ts, *te; bool isCxx; void token( int tok ); - void run( char *buf ); + void run( const char *buf ); }; @@ -57,8 +57,8 @@ struct Scanner if ( ! isCxx ) fgoto main; else { - cout << "comm char: " << tokstart[0] << endl; - cout << "comm char: " << tokstart[1] << endl; + cout << "comm char: " << ts[0] << endl; + cout << "comm char: " << ts[1] << endl; } }; @@ -66,11 +66,11 @@ struct Scanner if ( isCxx ) fgoto main; else - cout << "comm char: " << tokstart[0] << endl; + cout << "comm char: " << ts[0] << endl; }; any { - cout << "comm char: " << tokstart[0] << endl; + cout << "comm char: " << ts[0] << endl; }; *|; @@ -124,7 +124,7 @@ struct Scanner '...' { token( TK_DotDotDot );}; # Single char symbols. - ( punct - [_"'] ) { token( tokstart[0] );}; + ( punct - [_"'] ) { token( ts[0] );}; # Comments and whitespace. Handle these outside of the machine so that se # don't end up buffering the comments. @@ -140,8 +140,8 @@ struct Scanner void Scanner::token( int tok ) { - const char *data = tokstart; - int len = tokend - tokstart; + const char *data = ts; + int len = te - ts; cout << "<" << tok << "> "; if ( data != 0 ) { for ( int i = 0; i < len; i++ ) @@ -150,19 +150,19 @@ void Scanner::token( int tok ) cout << '\n'; } -void Scanner::run( char *buf ) +void Scanner::run( const char *buf ) { int len = strlen( buf ); %% write init; - char *p = buf; - char *pe = buf + len; + const char *p = buf; + const char *pe = buf + len; + const char *eof = pe; %% write exec; if ( cs == Scanner_error ) { /* Machine failed before finding a token. */ cout << "PARSE ERROR" << endl; } - %% write eof; } int main() @@ -170,7 +170,7 @@ int main() Scanner scanner; scanner.run( "//hello*/\n" - "/*hi there*/ hello 0x88\n" + "/*hi there*/ hello 0x88" ); return 0; } diff --git a/test/mailbox1.h b/test/mailbox1.h index bf9a87e..e7cd37c 100644 --- a/test/mailbox1.h +++ b/test/mailbox1.h @@ -21,7 +21,7 @@ struct MBox // the data, the machine is in the error state and can never accept, 0 if // the machine is in a non-accepting state and 1 if the machine is in an // accepting state. - void execute( char *data, int len ); + void execute( const char *data, int len ); // Indicate that there is no more data. Returns -1 if the machine finishes // in the error state and does not accept, 0 if the machine finishes diff --git a/test/mailbox1.rl b/test/mailbox1.rl index 89e8775..ea23173 100644 --- a/test/mailbox1.rl +++ b/test/mailbox1.rl @@ -1,7 +1,8 @@ /* * @LANG: c++ * @CFLAGS: -I../aapl - * @ALLOW_GENFLAGS: -T0 -T1 -F0 -F1 -G0 -G1 -G2 -P + * + * Test works with split code gen. */ /* @@ -98,11 +99,11 @@ void MBox::init( ) %% write init; } -void MBox::execute( char *data, int len ) +void MBox::execute( const char *data, int len ) { MBox *fsm = this; - char *p = data; - char *pe = data + len; + const char *p = data; + const char *pe = data + len; %%{ access fsm->; write exec; @@ -120,7 +121,7 @@ int MBox::finish( ) MBox mbox; -void test( char *buf ) +void test( const char *buf ) { int len = strlen( buf ); mbox.init(); diff --git a/test/mailbox2.rl b/test/mailbox2.rl index d84696d..bbaf820 100644 --- a/test/mailbox2.rl +++ b/test/mailbox2.rl @@ -121,14 +121,14 @@ using std::endl; #define BUFSIZE 8192 -void test( char *buf ) +void test( const char *buf ) { int cs, len = strlen( buf ); - char *preserve = 0, *ws = 0; + const char *preserve = 0, *ws = 0; %% write init; - char *p = buf; - char *pe = p + len; + const char *p = buf; + const char *pe = p + len; %% write exec; if ( cs == mailbox_error ) @@ -142,7 +142,7 @@ int main() { test( "From user@host.com Wed Nov 28 13:30:05 2001\n" - "From: \"Adrian D. Thurston\" <thurston@cs.queensu.ca>\n" + "From: \"Adrian D. Thurston\" <thurston@complang.org>\n" "Subject: the squirrel has landed\n" "\n" "Message goes here. \n" @@ -162,7 +162,7 @@ int main() #ifdef _____OUTPUT_____ ------ FROM - "Adrian D. Thurston" | thurston@cs.queensu.ca + "Adrian D. Thurston" | thurston@complang.org SUBJECT the squirrel has landed ------ diff --git a/test/mailbox3.rl b/test/mailbox3.rl index e8089bb..8039f80 100644 --- a/test/mailbox3.rl +++ b/test/mailbox3.rl @@ -129,15 +129,15 @@ using std::endl; #define BUFSIZE 8192 -void test( char *buf ) +void test( const char *buf ) { int cs, len = strlen( buf ); - char *preserve = 0, *ws = 0; + const char *preserve = 0, *ws = 0; int hlen = 0; %% write init; - char *p = buf; - char *pe = p + len; + const char *p = buf; + const char *pe = p + len; %% write exec; if ( cs < mailbox_first_final ) { @@ -150,7 +150,7 @@ int main() { test( "From user@host.com Wed Nov 28 13:30:05 2001\n" - "From: \"Adrian D. Thurston\" <thurston@cs.queensu.ca>\n" + "From: \"Adrian D. Thurston\" <thurston@complang.org>\n" "Subject: the squirrel has landed\n" "\n" "Message goes here. \n" @@ -168,7 +168,7 @@ int main() test( "From user@host.com Wed Nov 28 13:30:05 2001\n" "To: \"(kill 2)\" some guy <sg@net.com>\n" - "From: \"Adrian D. Thurston this name is far too long\" <thurston@cs.queensu.ca>\n" + "From: \"Adrian D. Thurston this name is far too long\" <thurston@complang.org>\n" "Subject: the squirrel has landed\n" "\n" "From user2@host2.com Wed Nov 28 13:30:05 2001\n" @@ -204,7 +204,7 @@ int main() #ifdef _____OUTPUT_____ ------ FROM - "Adrian D. Thurston" | thurston@cs.queensu.ca + "Adrian D. Thurston" | thurston@complang.org SUBJECT the squirrel has landed ------ diff --git a/test/minimize1.rl b/test/minimize1.rl index d7c6ef4..c550ebb 100644 --- a/test/minimize1.rl +++ b/test/minimize1.rl @@ -12,7 +12,7 @@ struct min %%{ machine min; - variable curstate fsm->cs; + variable cs fsm->cs; action a_or_b { printf("a or b\n"); } @@ -39,8 +39,6 @@ void min_execute( struct min *fsm, const char *_data, int _len ) int min_finish( struct min *fsm ) { - %% write eof; - if ( fsm->cs == min_error ) return -1; if ( fsm->cs >= min_first_final ) diff --git a/test/patact.rl b/test/patact.rl index c15d93d..864299d 100644 --- a/test/patact.rl +++ b/test/patact.rl @@ -5,8 +5,8 @@ char comm; int top; int stack[32]; -ptr tokstart; -ptr tokend; +ptr ts; +ptr te; int act; int val; %% @@ -20,15 +20,19 @@ int val; *|; exec_test := |* - [a-z]+ => { prints "word (w/lbh)\n"; fexec tokend-1; fgoto other; }; + [a-z]+ => { prints "word (w/lbh)\n"; fexec te-1; fgoto other; }; [a-z]+ ' foil' => { prints "word (c/lbh)\n"; }; [\n ] => { prints "space\n"; }; '22' => { prints "num (w/switch)\n"; }; - [0-9]+ => { prints "num (w/switch)\n"; fexec tokend-1; fgoto other;}; + [0-9]+ => { prints "num (w/switch)\n"; fexec te-1; fgoto other;}; [0-9]+ ' foil' => {prints "num (c/switch)\n"; }; '!';# => { prints "immdiate\n"; fgoto exec_test; }; *|; + semi := |* + ';' => { prints "in semi\n"; fgoto main; }; + *|; + main := |* [a-z]+ => { prints "word (w/lbh)\n"; fhold; fgoto other; }; [a-z]+ ' foil' => { prints "word (c/lbh)\n"; }; @@ -36,6 +40,7 @@ int val; '22' => { prints "num (w/switch)\n"; }; [0-9]+ => { prints "num (w/switch)\n"; fhold; fgoto other;}; [0-9]+ ' foil' => {prints "num (c/switch)\n"; }; + ';' => { prints "going to semi\n"; fhold; fgoto semi;}; '!' => { prints "immdiate\n"; fgoto exec_test; }; *|; }%% @@ -46,6 +51,7 @@ int val; "!abcd foix\n" "!abcd\nanother\n" "!123 foix\n" +";" _____INPUT_____ */ /* _____OUTPUT_____ word (w/lbh) @@ -87,5 +93,8 @@ space word space ACCEPT +going to semi +in semi +ACCEPT _____OUTPUT_____ */ diff --git a/test/range.rl b/test/range.rl index 34bc430..43e6214 100644 --- a/test/range.rl +++ b/test/range.rl @@ -11,8 +11,8 @@ struct range }; %%{ - machine range_fsm; - variable curstate fsm->cs; + machine range; + variable cs fsm->cs; main := ( 'a' .. 'c' | 'c' .. 'e' | 'm' .. 'n' | 'a' .. 'z' ) '\n'; }%% @@ -34,11 +34,9 @@ void range_execute( struct range *fsm, const char *_data, int _len ) int range_finish( struct range *fsm ) { - %% write eof; - - if ( fsm->cs == range_fsm_error ) + if ( fsm->cs == range_error ) return -1; - if ( fsm->cs >= range_fsm_first_final ) + if ( fsm->cs >= range_first_final ) return 1; return 0; } diff --git a/test/recdescent1.rl b/test/recdescent1.rl new file mode 100644 index 0000000..1ffca28 --- /dev/null +++ b/test/recdescent1.rl @@ -0,0 +1,128 @@ +/* + * @LANG: c + * Test growable stack. + */ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +%%{ + machine recdescent; + + prepush { + if ( top == stack_size ) { + printf( "growing stack\n" ); + stack_size = top * 2; + stack = (int*)realloc( stack, sizeof(int)*stack_size ); + } + } + + postpop { + if ( stack_size > (top * 4) ) { + stack_size = top * 2; + stack = (int*)realloc( stack, sizeof(int)*stack_size ); + printf( "shrinking stack\n" ); + } + } + + action item_start { item = p; } + + action item_finish + { + printf( "item: " ); + fwrite( item, 1, p-item, stdout ); + printf( "\n" ); + } + + action call_main + { + printf( "calling main\n" ); + fcall main; + } + + action return_main + { + if ( top == 0 ) { + printf( "STRAY CLOSE\n" ); + fbreak; + } + + printf( "returning from main\n" ); + fhold; + fret; + } + + id = [a-zA-Z_]+; + number = [0-9]+; + ws = [ \t\n]+; + + main := ( + ws | + ( number | id ) >item_start %item_finish | + + '{' @call_main '}' | + + '}' @return_main + )**; +}%% + +%% write data; + +void test( char *buf ) +{ + int cs; + int *stack; + int top, stack_size; + char *p, *pe, *eof, *item = 0; + + int len = strlen( buf ); + + %% write init; + + stack_size = 1; + stack = (int*)malloc( sizeof(int) * stack_size ); + + p = buf; + pe = buf + len; + eof = pe; + + %% write exec; + + if ( cs == recdescent_error ) { + /* Machine failed before finding a token. */ + printf( "PARSE ERROR\n" ); + } +} + +int main() +{ + test( "88 foo { 99 {{{{}}}}{ } }"); + test( "76 } sadf"); + return 0; +} + +#ifdef _____OUTPUT_____ +item: 88 +item: foo +calling main +item: 99 +calling main +growing stack +calling main +growing stack +calling main +calling main +growing stack +returning from main +returning from main +returning from main +returning from main +shrinking stack +calling main +returning from main +returning from main +shrinking stack +item: 76 +STRAY CLOSE +#endif diff --git a/test/recdescent2.rl b/test/recdescent2.rl new file mode 100644 index 0000000..59c4586 --- /dev/null +++ b/test/recdescent2.rl @@ -0,0 +1,116 @@ +/* + * @LANG: java + */ + +class recdescent2 +{ + %%{ + machine recdescent; + + prepush { + if ( top == stack_size ) { + System.out.print( "growing stack\n" ); + stack_size = top * 2; + // Don't actually bother to resize here, but we do print messages. + //stack = (int*)realloc( stack, sizeof(int)*stack_size ); + } + } + + postpop { + if ( stack_size > (top * 4) ) { + stack_size = top * 2; + // Don't actually bother to resize here, but we do print messages. + //stack = (int*)realloc( stack, sizeof(int)*stack_size ); + System.out.print( "shrinking stack\n" ); + } + } + + action item_start { item = p; } + + action item_finish + { + String item_data = new String ( data, item, p-item ); + System.out.print( "item: " ); + System.out.print( item_data ); + System.out.print( "\n" ); + } + + action call_main + { + System.out.print( "calling main\n" ); + fcall main; + } + + action return_main + { + if ( top == 0 ) { + System.out.print( "STRAY CLOSE\n" ); + fbreak; + } + + System.out.print( "returning from main\n" ); + fhold; + fret; + } + + id = [a-zA-Z_]+; + number = [0-9]+; + ws = [ \t\n]+; + + main := ( + ws | + ( number | id ) >item_start %item_finish | + + '{' @call_main '}' | + + '}' @return_main + )**; + }%% + + %% write data; + + static void test( char data[] ) + { + int cs, p = 0, pe = data.length, eof = data.length, item = 0; + int stack[] = new int[1024]; + int stack_size = 1; + int top; + + %% write init; + %% write exec; + + if ( cs == recdescent_error ) + System.out.println( "SCANNER ERROR" ); + } + + public static void main( String args[] ) + { + test( "88 foo { 99 {{{{}}}}{ } }".toCharArray() ); + test( "76 } sadf".toCharArray() ); + } +} + +/* _____OUTPUT_____ +item: 88 +item: foo +calling main +item: 99 +calling main +growing stack +calling main +growing stack +calling main +calling main +growing stack +returning from main +returning from main +returning from main +returning from main +shrinking stack +calling main +returning from main +returning from main +shrinking stack +item: 76 +STRAY CLOSE +*/ diff --git a/test/recdescent3.rl b/test/recdescent3.rl new file mode 100644 index 0000000..1216b43 --- /dev/null +++ b/test/recdescent3.rl @@ -0,0 +1,117 @@ +# +# @LANG: ruby +# + +%%{ + machine recdescent3; + + prepush { + if top == stack_size + print( "growing stack\n" ); + stack_size = top * 2; + # Don't actually bother to resize here, but we do print messages. + # stack = (int*)realloc( stack, sizeof(int)*stack_size ); + end + } + + postpop { + if stack_size > (top * 4) + print( "shrinking stack\n" ); + stack_size = top * 2; + # Don't actually bother to resize here, but we do print messages. + # stack = (int*)realloc( stack, sizeof(int)*stack_size ); + end + } + + action item_start { item = p; } + + action item_finish + { + print( "item: " ); + print( data[item..p-1] ); + print( "\n" ); + } + + action call_main + { + print( "calling main\n" ); + fcall main; + } + + action return_main + { + if top == 0 + print( "STRAY CLOSE\n" ); + fbreak; + end + + print( "returning from main\n" ); + fhold; + fret; + } + + id = [a-zA-Z_]+; + number = [0-9]+; + ws = [ \t\n]+; + + main := ( + ws | + ( number | id ) >item_start %item_finish | + + '{' @call_main '}' | + + '}' @return_main + )**; +}%% + +%% write data; + +def run_machine( data ) + item = 0; + p = 0; + pe = data.length; + eof = pe; + cs = 0; + stack = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ]; + stack_size = 1; + top = 0; + + %% write init; + %% write exec; + + if cs == recdescent3_error + puts "SCANNER_ERROR" + end +end + +inp = [ + "88 foo { 99 {{{{}}}}{ } }", + "76 } sadf" +] + +inp.each { |str| run_machine(str) } + +=begin _____OUTPUT_____ +item: 88 +item: foo +calling main +item: 99 +calling main +growing stack +calling main +growing stack +calling main +calling main +growing stack +returning from main +returning from main +returning from main +returning from main +shrinking stack +calling main +returning from main +returning from main +shrinking stack +item: 76 +STRAY CLOSE +=end _____OUTPUT_____ diff --git a/test/repetition.rl b/test/repetition.rl index 23638b3..328cfa9 100644 --- a/test/repetition.rl +++ b/test/repetition.rl @@ -7,6 +7,7 @@ #include <iostream> #include <stdlib.h> #include <stdio.h> +#include <string.h> using namespace std; @@ -62,7 +63,6 @@ int Rep::execute( const char *_data, int _len ) int Rep::finish( ) { - %% write eof; if ( cs == Rep_error ) return -1; if ( cs >= Rep_first_final ) @@ -70,7 +70,7 @@ int Rep::finish( ) return 0; } -void test( char *buf ) +void test( const char *buf ) { Rep rep; int len = strlen( buf ); diff --git a/test/rlscan.rl b/test/rlscan.rl index 943c4f5..448b979 100644 --- a/test/rlscan.rl +++ b/test/rlscan.rl @@ -2,7 +2,8 @@ * Lexes Ragel input files. * * @LANG: c++ - * @ALLOW_GENFLAGS: -T0 -T1 -F0 -F1 -G0 -G1 -G2 -P + * + * Test works with split code gen. */ #include <iostream> @@ -12,7 +13,7 @@ using namespace std; -void escapeXML( char *data ) +void escapeXML( const char *data ) { while ( *data != 0 ) { switch ( *data ) { @@ -35,9 +36,9 @@ void escapeXML( char c ) } } -void escapeXML( char *data, int len ) +void escapeXML( const char *data, int len ) { - for ( char *end = data + len; data != end; data++ ) { + for ( const char *end = data + len; data != end; data++ ) { switch ( *data ) { case '<': cout << "<"; break; case '>': cout << ">"; break; @@ -47,7 +48,7 @@ void escapeXML( char *data, int len ) } } -inline void write( char *data ) +inline void write( const char *data ) { cout << data; } @@ -57,7 +58,7 @@ inline void write( char c ) cout << c; } -inline void write( char *data, int len ) +inline void write( const char *data, int len ) { cout.write( data, len ); } @@ -80,7 +81,7 @@ inline void write( char *data, int len ) @{ fret; }; action emit { - escapeXML( tokstart, tokend-tokstart ); + escapeXML( ts, te-ts ); } # @@ -112,7 +113,7 @@ inline void write( char *data, int len ) } }; - default => { escapeXML( *tokstart ); }; + default => { escapeXML( *ts ); }; *|; # @@ -137,21 +138,21 @@ inline void write( char *data, int len ) # Word word { write( "<word>" ); - write( tokstart, tokend-tokstart ); + write( ts, te-ts ); write( "</word>\n" ); }; # Decimal integer. integer { write( "<int>" ); - write( tokstart, tokend-tokstart ); + write( ts, te-ts ); write( "</int>\n" ); }; # Hexidecimal integer. hex { write( "<hex>" ); - write( tokstart, tokend-tokstart ); + write( ts, te-ts ); write( "</hex>\n" ); }; @@ -161,28 +162,28 @@ inline void write( char *data, int len ) # Single literal string. "'" ( [^'\\] | /\\./ )* "'" { write( "<single_lit>" ); - escapeXML( tokstart, tokend-tokstart ); + escapeXML( ts, te-ts ); write( "</single_lit>\n" ); }; # Double literal string. '"' ( [^"\\] | /\\./ )* '"' { write( "<double_lit>" ); - escapeXML( tokstart, tokend-tokstart ); + escapeXML( ts, te-ts ); write( "</double_lit>\n" ); }; # Or literal. '[' ( [^\]\\] | /\\./ )* ']' { write( "<or_lit>" ); - escapeXML( tokstart, tokend-tokstart ); + escapeXML( ts, te-ts ); write( "</or_lit>\n" ); }; # Regex Literal. '/' ( [^/\\] | /\\./ ) * '/' { write( "<re_lit>" ); - escapeXML( tokstart, tokend-tokstart ); + escapeXML( ts, te-ts ); write( "</re_lit>\n" ); }; @@ -212,7 +213,7 @@ inline void write( char *data, int len ) '"' ( [^"\\] | /\\./ )* '"' => emit; '/*' { - escapeXML( tokstart, tokend-tokstart ); + escapeXML( ts, te-ts ); fcall c_comment; }; @@ -231,7 +232,7 @@ inline void write( char *data, int len ) }; default { - escapeXML( *tokstart ); + escapeXML( *ts ); }; # EOF. @@ -241,12 +242,12 @@ inline void write( char *data, int len ) %% write data nofinal; -void test( char *data ) +void test( const char *data ) { std::ios::sync_with_stdio(false); int cs, act; - char *tokstart, *tokend; + const char *ts, *te; int stack[1], top; bool single_line = false; @@ -255,8 +256,9 @@ void test( char *data ) %% write init; /* Read in a block. */ - char *p = data; - char *pe = data + strlen( data ); + const char *p = data; + const char *pe = data + strlen( data ); + const char *eof = pe; %% write exec; if ( cs == RagelScan_error ) { diff --git a/test/ruby1.rl b/test/ruby1.rl new file mode 100644 index 0000000..e2f4bc9 --- /dev/null +++ b/test/ruby1.rl @@ -0,0 +1,56 @@ +# +# @LANG: ruby +# +# Test the host language scanning for ruby. +# + +# %%{ +a = 1 +b = /%%\{\}/; + +%%{ + machine ruby1; + + main := lower+ digit+ '\n' @{ + + # } + c = 1 + d = /\}/ + puts "NL" + }; +}%% + +# %%{ +e = 1 +f = /%%\{\}/; + +%% write data; + +# %%{ +g = 1 +h = /%%\{\}/; + +def run_machine( data ) + p = 0; + pe = data.length + cs = 0 + + %% write init; + %% write exec; + if cs >= ruby1_first_final + puts "ACCEPT" + else + puts "FAIL" + end +end + +inp = [ + "abc1231\n", +] + +inp.each { |str| run_machine(str) } + +=begin _____OUTPUT_____ +NL +ACCEPT +=end _____OUTPUT_____ diff --git a/test/runtests.in b/test/runtests.in new file mode 100755 index 0000000..d331f77 --- /dev/null +++ b/test/runtests.in @@ -0,0 +1,313 @@ +#!/bin/bash + +# +# Copyright 2006-2009 Adrian Thurston <thurston@complang.org> +# + +# This file is part of Ragel. +# +# Ragel is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# Ragel is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with Ragel; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + +while getopts "gcnmleT:F:G:P:CDJRA" opt; do + case $opt in + T|F|G|P) + genflags="$genflags -$opt$OPTARG" + options="$options -$opt$OPTARG" + ;; + n|m|l|e) + minflags="$minflags -$opt" + options="$options -$opt" + ;; + c) + compile_only="true" + options="$options -$opt" + ;; + g) + allow_generated="true" + ;; + C|D|J|R|A) + langflags="$langflags -$opt" + ;; + esac +done + +[ -z "$minflags" ] && minflags="-n -m -l -e" +[ -z "$genflags" ] && genflags="-T0 -T1 -F0 -F1 -G0 -G1 -G2" +[ -z "$langflags" ] && langflags="-C -D -J -R -A" + +shift $((OPTIND - 1)); + +[ -z "$*" ] && set -- *.rl + +config=../ragel/config.h +ragel=../ragel/ragel + +cxx_compiler="@CXX@" +c_compiler="@CC@" +objc_compiler="@GOBJC@" +d_compiler="@GDC@" +java_compiler="@JAVAC@" +txl_engine="@TXL@" +ruby_engine="@RUBY@" +csharp_compiler="@GMCS@" + +function test_error +{ + exit 1; +} + +# split_objs="" +# if test $split_iters != "$gen_opt"; then +# n=0; +# while test $n -lt $split_iters; do +# part_root=${root}_`awk 'BEGIN { +# width = 0; +# high = '$split_iters' - 1; +# while ( high > 0 ) { +# width = width + 1; +# high = int(high / 10); +# } +# suffFormat = "%" width "." width "d\n"; +# printf( suffFormat, '$n' ); +# exit 0; +# }'` +# part_src=${part_root}.c +# part_bin=${part_root}.o +# echo "$compiler -c $cflags -o $part_bin $part_src" +# if ! $compiler -c $cflags -o $part_bin $part_src; then +# test_error; +# fi +# split_objs="$split_objs $part_bin" +# n=$((n+1)) +# done +# fi + +function run_test() +{ + echo "$ragel $lang_opt $min_opt $gen_opt -o $code_src $test_case" + if ! $ragel $lang_opt $min_opt $gen_opt -o $code_src $test_case; then + test_error; + fi + + out_args="" + [ $lang != java ] && out_args="-o ${binary}"; + [ $lang == csharp ] && out_args="-out:${binary}"; + + # Ruby doesn't need to be compiled. + if [ $lang != ruby ]; then + echo "$compiler ${cflags} ${out_args} ${code_src}" + if ! $compiler ${cflags} ${out_args} ${code_src}; then + test_error; + fi + fi + + if [ "$compile_only" != "true" ]; then + echo -n "running $root ... "; + + exec_cmd=./$binary + [ $lang = java ] && exec_cmd="java ${root}" + [ $lang = ruby ] && exec_cmd="ruby ${code_src}" + [ $lang = csharp ] && [ "$csharp_compiler" = gmcs ] && exec_cmd="mono ${exec_cmd}" + + $exec_cmd 2>&1 > $output; + if diff --strip-trailing-cr $expected_out $output > /dev/null; then + echo "passed"; + else + echo "FAILED"; + test_error; + fi; + fi +} + +for test_case; do + root=${test_case%.rl}; + + if ! [ -f "$test_case" ]; then + echo "runtests: not a file: $test_case"; >&2 + exit 1; + fi + + # Check if we should ignore the test case + ignore=`sed '/@IGNORE:/s/^.*: *//p;d' $test_case` + if [ "$ignore" = yes ]; then + continue; + fi + + # If the generated flag is given make sure that the test case is generated. + is_generated=`sed '/@GENERATED:/s/^.*: *//p;d' $test_case` + if [ "$is_generated" = yes ] && [ "$allow_generated" != true ]; then + continue; + fi + + expected_out=$root.exp; + sed '1,/_____OUTPUT_____/d;$d' $test_case > $expected_out + + lang=`sed '/@LANG:/s/^.*: *//p;d' $test_case` + if [ -z "$lang" ]; then + echo "$test_case: language unset"; >&2 + exit 1; + fi + + case $lang in + c++) + lang_opt=-C; + code_suffix=cpp; + compiler=$cxx_compiler; + cflags="-pedantic -ansi -Wall -O3" + ;; + d) + lang_opt=-D; + code_suffix=d; + compiler=$d_compiler; + cflags="-Wall -O3" + ;; + c) + lang_opt=-C; + code_suffix=c; + compiler=$c_compiler; + cflags="-pedantic -ansi -Wall -O3" + ;; + obj-c) + lang_opt=-C; + code_suffix=m; + compiler=$objc_compiler + cflags="-Wall -O3 -fno-strict-aliasing -lobjc" + ;; + java) + lang_opt=-J; + code_suffix=java; + compiler=$java_compiler + cflags="" + ;; + ruby) + lang_opt=-R; + code_suffix=rb; + compiler=$ruby_engine + cflags="" + ;; + csharp) + lang_opt="-A"; + code_suffix=cs; + compiler=$csharp_compiler + cflags="" + ;; + indep) + lang_opt=""; + + # If we have no txl engine then skip this test. + [ -z "$txl_engine" ] && continue + for lang in c d java ruby csharp; do + case $lang in + c) lf="-C";; + d) lf="-D";; + java) lf="-J";; + ruby) lf="-R";; + csharp) lf="-A";; + esac + + echo "$langflags" | grep -e $lf >/dev/null || continue + + targ=${root}_$lang.rl + echo "./langtrans_$lang.sh $test_case > $targ" + if ! ./langtrans_$lang.sh $test_case > $targ; then + test_error + fi + echo "./runtests -g $options $targ" + if ! ./runtests -g $options $targ; then + test_error + fi + done + continue; + ;; + *) + echo "$test_case: unknown language type $lang" >&2 + exit 1; + ;; + esac + + # Make sure that we are interested in the host language. + echo "$langflags" | grep -e $lang_opt >/dev/null || continue + + code_src=$root.$code_suffix; + binary=$root.bin; + output=$root.out; + + # If we have no compiler for the source program then skip it. + [ -z "$compiler" ] && continue + + additional_cflags=`sed '/@CFLAGS:/s/^.*: *//p;d' $test_case` + [ -n "$additional_cflags" ] && cflags="$cflags $additional_cflags" + + allow_minflags=`sed '/@ALLOW_MINFLAGS:/s/^.*: *//p;d' $test_case` + [ -z "$allow_minflags" ] && allow_minflags="-n -m -l -e" + + case $lang in + c|c++|d) + # Using genflags, get the allowed gen flags from the test case. If the + # test case doesn't specify assume that all gen flags are allowed. + allow_genflags=`sed '/@ALLOW_GENFLAGS:/s/^.*: *//p;d' $test_case` + [ -z "$allow_genflags" ] && allow_genflags="-T0 -T1 -F0 -F1 -G0 -G1 -G2" + + for min_opt in $minflags; do + echo "$allow_minflags" | grep -e $min_opt >/dev/null || continue + for gen_opt in $genflags; do + echo "$allow_genflags" | grep -e $gen_opt >/dev/null || continue + run_test + done + done + ;; + + java) + # Not interested in gen opt. + gen_opt="" + for min_opt in $minflags; do + echo "$allow_minflags" | grep -e $min_opt >/dev/null || continue + run_test + done + ;; + + ruby) + # Using genflags, get the allowed gen flags from the test case. If the + # test case doesn't specify assume that all gen flags are allowed. + allow_genflags=`sed '/@ALLOW_GENFLAGS:/s/^.*: *//p;d' $test_case` + [ -z "$allow_genflags" ] && allow_genflags="-T0 -T1 -F0 -F1" + + for min_opt in $minflags; do + echo "$allow_minflags" | grep -e $min_opt >/dev/null || continue + + for gen_opt in $genflags; do + echo "$allow_genflags" | grep -e $gen_opt >/dev/null || continue + run_test + done + done + ;; + + csharp) + # Using genflags, get the allowed gen flags from the test case. If the + # test case doesn't specify assume that all gen flags are allowed. + allow_genflags=`sed '/@ALLOW_GENFLAGS:/s/^.*: *//p;d' $test_case` + [ -z "$allow_genflags" ] && allow_genflags="-T0 -T1 -F0 -F1 -G0 -G1" + + for min_opt in $minflags; do + echo "$allow_minflags" | grep -e $min_opt >/dev/null || continue + for gen_opt in $genflags; do + echo "$allow_genflags" | grep -e $gen_opt >/dev/null || continue + run_test + done + done + ;; + esac + +done diff --git a/test/scan1.rl b/test/scan1.rl new file mode 100644 index 0000000..df0971a --- /dev/null +++ b/test/scan1.rl @@ -0,0 +1,64 @@ +/* + * @LANG: indep + */ +ptr ts; +ptr te; +int act; +int token; +%% +%%{ + machine scanner; + + # Warning: changing the patterns or the input string will affect the + # coverage of the scanner action types. + main := |* + 'a' => { + prints "on last "; + if ( p+1 == te ) + prints "yes"; + prints "\n"; + }; + + 'b'+ => { + prints "on next "; + if ( p+1 == te ) + prints "yes"; + prints "\n"; + }; + + 'c1' 'dxxx'? => { + prints "on lag "; + if ( p+1 == te ) + prints "yes"; + prints "\n"; + }; + + 'd1' => { + prints "lm switch1 "; + if ( p+1 == te ) + prints "yes"; + prints "\n"; + }; + 'd2' => { + prints "lm switch2 "; + if ( p+1 == te ) + prints "yes"; + prints "\n"; + }; + + [d0-9]+ '.'; + + '\n'; + *|; +}%% +/* _____INPUT_____ +"abbc1d1d2\n" +_____INPUT_____ */ +/* _____OUTPUT_____ +on last yes +on next yes +on lag yes +lm switch1 yes +lm switch2 yes +ACCEPT +_____OUTPUT_____ */ diff --git a/test/scan2.rl b/test/scan2.rl new file mode 100644 index 0000000..a1ae959 --- /dev/null +++ b/test/scan2.rl @@ -0,0 +1,34 @@ +/* + * @LANG: indep + */ +ptr ts; +ptr te; +int act; +int token; +%% +%%{ + machine scanner; + + # Warning: changing the patterns or the input string will affect the + # coverage of the scanner action types. + main := |* + 'a' => { + prints "pat1\n"; + }; + + [ab]+ . 'c' => { + prints "pat2\n"; + }; + + any => { + prints "any\n"; + }; + *|; +}%% +/* _____INPUT_____ +"a" +_____INPUT_____ */ +/* _____OUTPUT_____ +pat1 +ACCEPT +_____OUTPUT_____ */ diff --git a/test/scan3.rl b/test/scan3.rl new file mode 100644 index 0000000..ca1a136 --- /dev/null +++ b/test/scan3.rl @@ -0,0 +1,32 @@ +/* + * @LANG: indep + */ +ptr ts; +ptr te; +int act; +int token; +%% +%%{ + machine scanner; + + # Warning: changing the patterns or the input string will affect the + # coverage of the scanner action types. + main := |* + 'a' => { + prints "pat1\n"; + }; + 'b' => { + prints "pat2\n"; + }; + [ab] any* => { + prints "pat3\n"; + }; + *|; +}%% +/* _____INPUT_____ +"ab89" +_____INPUT_____ */ +/* _____OUTPUT_____ +pat3 +ACCEPT +_____OUTPUT_____ */ diff --git a/test/scan4.rl b/test/scan4.rl new file mode 100644 index 0000000..12d4d4c --- /dev/null +++ b/test/scan4.rl @@ -0,0 +1,33 @@ +/* + * @LANG: indep + */ +ptr ts; +ptr te; +int act; +int token; +%% +%%{ + machine scanner; + + # Warning: changing the patterns or the input string will affect the + # coverage of the scanner action types. + main := |* + 'a' => { + prints "pat1\n"; + }; + + [ab]+ . 'c' => { + prints "pat2\n"; + }; + + any; + *|; +}%% +/* _____INPUT_____ +"ba a" +_____INPUT_____ */ +/* _____OUTPUT_____ +pat1 +pat1 +ACCEPT +_____OUTPUT_____ */ diff --git a/test/statechart1.rl b/test/statechart1.rl index 9f1ce49..884f3f0 100644 --- a/test/statechart1.rl +++ b/test/statechart1.rl @@ -16,7 +16,7 @@ struct state_chart %%{ machine state_chart; - variable curstate fsm->cs; + variable cs fsm->cs; action a { printf("a"); } action b { printf("b"); } @@ -67,8 +67,6 @@ void state_chart_execute( struct state_chart *fsm, const char *_data, int _len ) int state_chart_finish( struct state_chart *fsm ) { - %% write eof; - if ( fsm->cs == state_chart_error ) return -1; if ( fsm->cs >= state_chart_first_final ) diff --git a/test/strings1.rl b/test/strings1.rl index d156da1..0d5eea8 100644 --- a/test/strings1.rl +++ b/test/strings1.rl @@ -12,7 +12,7 @@ struct strs %%{ machine strs; - variable curstate fsm->cs; + variable cs fsm->cs; main := "__gmon_start__\n" | @@ -150,8 +150,6 @@ void strs_execute( struct strs *fsm, const char *_data, int _len ) int strs_finish( struct strs *fsm ) { - %% write eof; - if ( fsm->cs == strs_error ) return -1; if ( fsm->cs >= strs_first_final ) diff --git a/test/strings2.rl b/test/strings2.rl index ad4dd40..edad63b 100644 --- a/test/strings2.rl +++ b/test/strings2.rl @@ -1,7 +1,9 @@ /* * @LANG: c - * @ALLOW_GENFLAGS: -T0 -T1 -F0 -F1 -P + * @ALLOW_GENFLAGS: -T0 -T1 -F0 -F1 * @ALLOW_MINFLAGS: -n -m -l + * + * Test works with split code gen. */ #include <string.h> @@ -11,7 +13,7 @@ %%{ machine strs; - variable curstate fsm->cs; + variable cs fsm->cs; main := "/lib/ld-linux.so.2\n" | @@ -1317,8 +1319,6 @@ void test( char *buf ) %% write init; %% write exec; - %% write eof; - if ( fsm->cs >= strs_first_final ) printf("ACCEPT\n"); else diff --git a/test/testcase.txl b/test/testcase.txl index cd02bb8..3369c15 100644 --- a/test/testcase.txl +++ b/test/testcase.txl @@ -15,8 +15,10 @@ end compounds keys 'int 'bool 'true 'false 'char 'ptr - 'if 'else 'printi 'prints + 'if 'else 'printi 'prints 'printb 'print_token 'fc 'fpc 'fbreak 'fgoto 'fcall 'fret 'fhold 'fexec + 'machine 'alphtype 'action + 'first_token_char end keys define lang_indep @@ -38,6 +40,7 @@ end define define statement [machine_stmt] + | [alphtype_stmt] | [action_stmt] | [cond_action_stmt] | [machine_def] @@ -48,6 +51,10 @@ define machine_stmt 'machine [id] '; [NL] end define +define alphtype_stmt + 'alphtype [repeat id] '; [NL] +end define + define action_stmt 'action [id] [al_host_block] end define @@ -71,10 +78,11 @@ end define define al_print_stmt [print_cmd] [al_expr] '; [NL] + | 'print_token '; [NL] end define define print_cmd - 'printi | 'prints + 'printi | 'prints | 'printb end define define al_variable_decl @@ -102,18 +110,24 @@ define al_expr_extend end define define al_expr_op - '= | '+ | '- | '* | '/ | '== | '<= | '>= + '= | '+ | '- | '* | '/ | '== | '<= | '>= | '< | '> end define define al_term + [al_term_base] [opt union] +end define + +define al_term_base [id] + | [SPOFF] [id] '( [SPON] [al_expr] ') | [opt al_sign] [number] - | [stringlit] - | [charlit] + | [stringlit] + | [charlit] | 'fc | 'true | 'false | '( [al_expr] ') + | 'first_token_char end define define al_sign diff --git a/test/tokstart1.rl b/test/tokstart1.rl index b6df225..e8c1552 100644 --- a/test/tokstart1.rl +++ b/test/tokstart1.rl @@ -11,7 +11,7 @@ extern char buf[]; struct Scanner { int cs, act; - char *tokstart, *tokend; + char *ts, *te; // Initialize the machine. Invokes any init statement blocks. Returns 0 // if the machine begins in a non-accepting state and 1 if the machine @@ -40,7 +40,7 @@ struct Scanner cout << (int)fc; else cout << fc; - cout << " tokstart = " << ( tokstart == 0 ? -1 : tokstart-buf ) << endl; + cout << " ts = " << ( ts == 0 ? -1 : ts-buf ) << endl; } action from_act { cout << "from: fc = "; @@ -48,7 +48,7 @@ struct Scanner cout << (int)fc; else cout << fc; - cout << " tokstart = " << ( tokstart == 0 ? -1 : tokstart-buf ) << endl; + cout << " ts = " << ( ts == 0 ? -1 : ts-buf ) << endl; } c_comm := ( any* $0 '*/' @1 @{ fgoto main; } ) $~to_act $*from_act; @@ -72,7 +72,7 @@ struct Scanner digit+ exponent float_suffix? ) $~ to_act $* from_act; # Integer decimal. Leading part buffered by float. - ( ( '0' | [1-9] [0-9]* ) [ulUL]{0,3} ) $~ to_act $* from_act + ( ( '0' | [1-9] [0-9]* ) [ulUL]{0,3} ) $~ to_act $* from_act; # Integer octal. Leading part buffered by float. ( '0' [0-9]+ [ulUL]{0,2} ) $~ to_act $* from_act; @@ -106,20 +106,15 @@ int Scanner::execute( char *data, int len ) { char *p = data; char *pe = data + len; + char *eof = pe; %% write exec; - int have = 0; - if ( tokstart != 0 ) { - have = pe - tokstart; - memmove( data, tokstart, have ); - } - return have; + return 0; } int Scanner::finish( ) { - %% write eof; if ( cs == Scanner_error ) return -1; if ( cs >= Scanner_first_final ) @@ -155,87 +150,89 @@ int main() } #ifdef _____OUTPUT_____ -from: fc = a tokstart = 0 -to: fc = a tokstart = 0 -from: fc = tokstart = 0 -to: fc = a tokstart = -1 -from: fc = tokstart = 1 -to: fc = tokstart = 1 -from: fc = b tokstart = 1 -to: fc = tokstart = -1 -from: fc = b tokstart = 2 -to: fc = b tokstart = 2 -from: fc = tokstart = 2 -to: fc = b tokstart = -1 -from: fc = tokstart = 3 -to: fc = tokstart = 3 -from: fc = 0 tokstart = 3 -to: fc = tokstart = -1 -from: fc = 0 tokstart = 4 -to: fc = 0 tokstart = 4 -from: fc = . tokstart = 4 -to: fc = . tokstart = 4 -from: fc = 9 tokstart = 4 -to: fc = 9 tokstart = 4 -from: fc = 8 tokstart = 4 -to: fc = 8 tokstart = 4 -from: fc = tokstart = 4 -to: fc = 8 tokstart = -1 -from: fc = tokstart = 8 -to: fc = tokstart = 8 -from: fc = / tokstart = 8 -to: fc = tokstart = -1 -from: fc = / tokstart = 9 -to: fc = / tokstart = 9 -from: fc = * tokstart = 9 -to: fc = * tokstart = -1 +from: fc = a ts = 0 +to: fc = a ts = 0 +from: fc = ts = 0 +to: fc = a ts = -1 +from: fc = ts = 1 +to: fc = ts = 1 +from: fc = b ts = 1 +to: fc = ts = -1 +from: fc = b ts = 2 +to: fc = b ts = 2 +from: fc = ts = 2 +to: fc = b ts = -1 +from: fc = ts = 3 +to: fc = ts = 3 +from: fc = 0 ts = 3 +to: fc = ts = -1 +from: fc = 0 ts = 4 +to: fc = 0 ts = 4 +from: fc = . ts = 4 +to: fc = . ts = 4 +from: fc = 9 ts = 4 +to: fc = 9 ts = 4 +from: fc = 8 ts = 4 +to: fc = 8 ts = 4 +from: fc = ts = 4 +to: fc = 8 ts = -1 +from: fc = ts = 8 +to: fc = ts = 8 +from: fc = / ts = 8 +to: fc = ts = -1 +from: fc = / ts = 9 +to: fc = / ts = 9 +from: fc = * ts = 9 +to: fc = * ts = -1 from: fc = - tokstart = -1 + ts = -1 to: fc = - tokstart = -1 -from: fc = 9 tokstart = -1 -to: fc = 9 tokstart = -1 -from: fc = tokstart = -1 -to: fc = tokstart = -1 -from: fc = * tokstart = -1 -to: fc = * tokstart = -1 -from: fc = / tokstart = -1 -to: fc = / tokstart = -1 -from: fc = 39 tokstart = 16 -to: fc = 39 tokstart = 16 -from: fc = \ tokstart = 16 -to: fc = \ tokstart = 16 -from: fc = 39 tokstart = 16 -to: fc = 39 tokstart = 16 -from: fc = 39 tokstart = 16 -to: fc = 39 tokstart = -1 -from: fc = / tokstart = 20 -to: fc = / tokstart = 20 -from: fc = / tokstart = 20 -to: fc = / tokstart = -1 -from: fc = h tokstart = -1 -to: fc = h tokstart = -1 -from: fc = i tokstart = -1 -to: fc = i tokstart = -1 + ts = -1 +from: fc = 9 ts = -1 +to: fc = 9 ts = -1 +from: fc = ts = -1 +to: fc = ts = -1 +from: fc = * ts = -1 +to: fc = * ts = -1 +from: fc = / ts = -1 +to: fc = / ts = -1 +from: fc = 39 ts = 16 +to: fc = 39 ts = 16 +from: fc = \ ts = 16 +to: fc = \ ts = 16 +from: fc = 39 ts = 16 +to: fc = 39 ts = 16 +from: fc = 39 ts = 16 +to: fc = 39 ts = -1 +from: fc = / ts = 20 +to: fc = / ts = 20 +from: fc = / ts = 20 +to: fc = / ts = -1 +from: fc = h ts = -1 +to: fc = h ts = -1 +from: fc = i ts = -1 +to: fc = i ts = -1 from: fc = - tokstart = -1 + ts = -1 to: fc = - tokstart = -1 -from: fc = t tokstart = 25 -to: fc = t tokstart = 25 -from: fc = h tokstart = 25 -to: fc = h tokstart = 25 -from: fc = e tokstart = 25 -to: fc = e tokstart = 25 -from: fc = r tokstart = 25 -to: fc = r tokstart = 25 -from: fc = e tokstart = 25 -to: fc = e tokstart = 25 + ts = -1 +from: fc = t ts = 25 +to: fc = t ts = 25 +from: fc = h ts = 25 +to: fc = h ts = 25 +from: fc = e ts = 25 +to: fc = e ts = 25 +from: fc = r ts = 25 +to: fc = r ts = 25 +from: fc = e ts = 25 +to: fc = e ts = 25 from: fc = - tokstart = 25 -to: fc = e tokstart = -1 + ts = 25 +to: fc = e ts = -1 from: fc = - tokstart = 30 + ts = 30 +to: fc = + ts = 30 to: fc = - tokstart = 30 + ts = -1 #endif diff --git a/test/union.rl b/test/union.rl index 6ad7e9a..41e24bf 100644 --- a/test/union.rl +++ b/test/union.rl @@ -6,6 +6,7 @@ #include <iostream> #include <stdlib.h> #include <stdio.h> +#include <string.h> using namespace std; @@ -100,13 +101,13 @@ void Concurrent::execute( const char *data, int len ) { const char *p = data; const char *pe = data + len; + const char *eof = pe; %% write exec; } int Concurrent::finish( ) { - %% write eof; if ( cs == Concurrent_error ) return -1; if ( cs >= Concurrent_first_final ) @@ -114,7 +115,7 @@ int Concurrent::finish( ) return 0; } -void test( char *buf ) +void test( const char *buf ) { Concurrent concurrent; concurrent.init(); diff --git a/test/xml.rl b/test/xml.rl index ca13f43..3a76400 100644 --- a/test/xml.rl +++ b/test/xml.rl @@ -92,7 +92,6 @@ struct XML int XML::finish( ) { - %% write eof; if ( cs == XML_error ) return -1; if ( cs >= XML_first_final ) |