diff options
Diffstat (limited to 'NOTES-btyacc-Changes')
| -rw-r--r-- | NOTES-btyacc-Changes | 385 |
1 files changed, 385 insertions, 0 deletions
diff --git a/NOTES-btyacc-Changes b/NOTES-btyacc-Changes new file mode 100644 index 0000000..f074907 --- /dev/null +++ b/NOTES-btyacc-Changes @@ -0,0 +1,385 @@ +Tom Shields, March 17, 2014 + +PARKING LOT ISSUES: +------------------- +- verify debian packaging still works? + +- there are no #line directives in y.tab.i, other than those that come + from the input file and the skeleton file; to fix this, would need to + count output lines in externs_file and add 'write_externs_lineno()' + similar to 'write_code_lineno()' +- if there are no defined symbols, the .tab.h file isn't empty (weird case, + may not be worth fixing) +- consider: treat []-actions identical to {}-actions if not processing a + backtracking parser (avoids test case error)? + + +BTYACC CHANGES CURRENTLY DEFERRED, BY FILE: +------------------------------------------- + +push.skel +- skeleton for a 'push' parser +- needs to be upgraded match the structure of yaccpar.skel + +defs.h +- adopt '%include' changes +- adopt '%define'/'%ifdef'/'%endif' +- adopt -E flag to print preprocessed grammar to stdout + +error.c +- adopt '%include' changes +- NOTE: there is a btyacc change that might be worth adopting in byacc + [FileError() refactoring to eliminate duplicated code in most of the + error message functions] + +main.c +- adopt '%define' changes +- adopt '-DNAME' command line option to define preprocessor variable NAME +- adopt -E flag to print preprocessed grammar to stdout +- adopt '-S skeleton_file' command line option to select an alternate parser + skeleton file +- the skeleton file named by the -S flag is used as provided to open the + file; consider a change to this behavior to check whether the named file + has a path prefix, and if not, look in 'installation' directory if the + file is not found in the working directory + +output.c +- adopt '%include' changes + +reader.c +- adopt '%include' changes +- adopt '%define'/'%ifdef'/'%endif' changes +- adopt -E flag to print preprocessed grammar to stdout +- NOTE: there is a btyacc change that might be worth adopting in byacc + [copy_string() & copy_comment() refactoring to eliminate duplicated + code in copy_text() and copy_union()] + +warshall.c +- NOTE: there is a btyacc change that might be worth adopting in byacc + [shifting 'mask' incrementally rather than literal '1' by a variable + amount each time thru the loop] + + +================================================================================ + + +new files: +---------- + +skel2c +- modified from btyacc distribution: don't generate #include defs.h +- extended syntax recognized to include '%% insert VERSION here', generating + the defines for YYMAJOR, YYMINOR and YYPATCH at that point +- made generated tables type 'const char *const' to match skelton.c from + byacc-20130925 baseline +- added code to append text for write_section() to end of generated skeleton.c +- remove conversion of tab to \t in generated skeleton.c +- extended syntax recognized to include '%%ifdef', '%%ifndef', '%%else' and + '%%endif'; used in yaccpar.skel to bracket code that is specific to + backtracking + +yaccpar.skel.old +- created from skeleton.c in byacc-20140101 baseline; use of this skeleton + will create a version of skeleton.c that is close to that in the + byacc-20140101 baseline +- eliminated 'body_3' and 'trailer_2' skeleton segments - no need to generate + yyerror() invocation dynamically; YYERROR_CALL() is already generated + earlier, and so can be used in the skeleton to simplify +- added 'const' to types in '%% tables' section to match what skel2c, + start_int_table() and state_str_table() generate +- added a few cosmetic changes (e.g., added some additional comments, + reworded debugging output to match yaccpar.skel, changed yygrowstack() + to return YYENOMEM for 'out of memory' error, rather than -1, to match + yaccpar.skel; changed yyparse() return value from 1 to 2 for the + 'out of memory' error to match yaccpar.skel) +- added '#ifndef'/'#endif' around '#define YYINITSTACKSIZE 200' to allow + the value to be changed at compile time +- changed 'printf(' to 'fprintf(stderr, '; added stack depth (yydepth) to + debugging output from yaccpar.skel +- use 'YYINT' rather than 'short' for integer table types + +yaccpar.skel +- renamed from btyaccpa.ske, merged with btyacc-c.ske +- modified from btyacc distribution to match the latest byacc-20140101 + skeleton structure & data structures +- make local functions static +- change "virtual memory exceeded" to "memory exhausted" for bison + compatibility +- change debug output generation from printf/puts/putc onto stdout to use + fprintf/fputs/fputc onto stderr; include + stack depth and whether or not in trial parsing +- changed types of generated string tables to be 'const pointer to const char' +- check all malloc()/realloc() return values, ensure return value of + yyparse() = 2 if parsing failed due to memory exhaustion +- change YYDBPR() macro to YYSTYPE_TOSTRING(); define semantics as delivering + a char* value representing a semantic value (e.g., yylval or yyval, or the + contents of an entry on the semantic stack); additional parameter passed: + grammar symbol # (to assist interpretation of semantic value) +- change YYPOSN to YYLTYPE and yyposn to yylloc (position corresponding to + yylval) for bison compatibility; add yyloc (corresponding to yyval) +- move default definition of YYLTYPE into output.c, generating a typedef +- add '#if defined(YYLTYPE) || defined(YYLTYPE_IS_DECLARED)'/'#endif' around + all lines specific to position processing +- add '#if defined(YYDESTRUCT_CALL)'/'#endif' around all lines specific to + semantic & position stack processing to reclaim memory associated with + discarded symbols +- add '%%ifdef YYBTYACC'/'%%endif' around all lines specific to backtrack + parsing; converted by skel2c into '#if defined(YYBTYACC)'/'#endif' +- distinguish between "yacc stack overflow" and "memory exhausted" situations +- consolidated termination cleanup code; introduced yyreturn, set to 2 after + labels yyoverflow/yyenomem, set to 1 after label yyabort, set to 0 after + label yyaccept; all termination cases jump to label yyreturn, which does + any cleanup then returns yyreturn value +- replaced YYDELETEVAL & YYDELETEPOSN user-supplied macro capability by + implementation of byacc-generated yydestruct() as defined by bison + compatible %destructor mechanism +- moved invocation of 'YYREDUCEPOSNFUNC' macro to immediately prior to, rather + than after, execution of final rule action (so that, at some future + date, implementation extensions can be added to enable custom calculation + of locations associated with non-terminals within rule actions); deleted + unnecessary flag 'reduce_posn'; deleted 'YYCALLREDUCEPOSN' macro; deleted + C++ variant of 'YYREDUCEPOSNFUNC' invocation +- adopt approach similar to bison for default computation of yyloc; change + macro 'YYREDUCEPOSNFUNC' name to 'YYLLOC_DEFAULT' for bison compatibility; + added 'yyerror_loc_range[2]' to hold start & end locations for error + situations that pop the stack +- use 'YYINT' rather than 'short' for integer table types, and for indexing + parser tables + +readskel.c +http://www.verisign.com/index.html- replaced error() with fprintf() + +mstring.h +- moved contents of mstring.h to defs.h - mstring.h is obsolete + +mstring.c +- replaced include of mstring.h with defs.h +- changed 'START' to 'HEAD' to remove conflict with 'START' used for + the start symbol defined in defs.h + +modified byacc files: +--------------------- + +skeleton.c +- skeleton.c is now generated from the appropriate skeleton file by 'skel2c' + +configure.in +- added configuration for --enable-btyacc option; if 'yes' add '-DYYBTYACC' + to DEFINES in makefile.in; --enable-btyacc defaults to 'no' +- added configuration for --with-max-table-size option; if present, + overrides the value of MAXTABLE defined in defs.h +- regenerate configure using autoconf + +makefile.in +- added mstring.c to C_FILES +- added mstring$o to OBJS +- added @DEFINES@ as value of DEFINES make variable +- added new make variable SKELETON with value 'yaccpar.skel' +- added rule to generate skeleton.c from $(SKELETON), depending on skel2c + and makefile +- added rm -f skeleton.c distclean rule +- moved dependency on makefile from only main$o & skeleton$o to $(OBJS), + since if ./configure is run changing, for example, from --enable-btyacc + to --disable-btyacc, all files must be recompiled to ensure a clean + executable +- add @MAXTABLE@ for optional '-DMAXTABLE=nnn' if configured using + --with-max-table-size=nnn +- changed 'cd test && rn 0f test-*'to 'rm -f $(testdir)/test-*' + +test/run_test.sh +- ??? + +test/run_make.sh +- ??? + +defs.h +- moved contents of mstring.h to defs.h - mstring.h is obsolete +- added <limits.h> to get the various system defined machine limits; + changed definitions of MAXCHAR, MAXSHORT, MINSHORT & BITS_PER_WORD to use + defines from <limits.h>; changed definitions of BIT and SETBIT to use + value of BITS_PER_WORD +- added typedef for __compar_fn_t, conditioned on _COMPAR_FN_T being + undefined (at least for Mac OSX environment) +- adopt new symbol class values ACTION and ARGUMENT +- adopt changes/additions used by inherited attribute processing +- clean up locations of extern function definitions to match where they + actually live in source files +- adopt error functions from inherited attribute processing; added new error + functions +- added keyword code LOCATIONS for %locations +- added keyword code DESTRUCTOR for %destructor +- added extern decl for 'int locations'; true if %locations present +- added extern decl for 'int backtrack'; initialized to 0 (= false), set to + 1 (= true) if -B flag is present +- added extern decl for 'int destructor'; true if at least one %destructor + present in grammar spec file +- define 'YYINT' as the smallest C type that can be used to address a + table of size 'MAXTABLE'; define 'YYINT' based on the value of + 'MAXTABLE' using the standard system type size definitions from <limits.h>; + define 'MAXYYINT' and 'MINYYINT' accordingly +- change 'Value_t' and 'Index_t' to 'YYINT' from 'short' +- allow 'MAXTABLE' to be defined by '-DMAXTABLE=nnn' at compile-time + +closure.c +- changed print_closure(), print_EFF() and print_first_derives() to 'static'; + added fwd declarations +- changed 'short' to 'Value_t' (in some instances, 'Value_t' was already + used for variables/parameters that were related to variables/parameters + declared as 'short' + +error.c +- adopt error functions from inherited attribute processing; added a few + additional inherited attribute error functions + +graph.c +- changed 'short' to 'Value_t' (in some instances, 'Value_t' was already + used for variables/parameters that were related to variables/parameters + declared as 'short' + +lalr.c +- changed MAXSHORT to MAXYYINT + +lr0.c +- changed MAXSHORT to MAXYYINT +- changed 'short' to 'Value_t' (in some instances, 'Value_t' was already + used for variables/parameters that were related to variables/parameters + declared as 'short' + +main.c +- changed 'short' to 'Value_t' (in some instances, 'Value_t' was already + used for variables/parameters that were related to variables/parameters + declared as 'short' + +mkpar.c +- backtracking attempts to resolve shift/reduce and reduce/reduce conflicts + +output.c +- generate prefix & YYPREFIX defines into externs file (-i, .tab.i) and + code file (-r, .code.c); generate into output file (.tab.c) only if not + using -r option; eliminates doubled output of prefix aliases if -r with + no -i in y.tab.c and y.code.c or if -r & -i in y.tab.i and y.code.c +- changed types of generated string tables to be 'const pointer to const char' +- adopt backtracking as an alternative in cases where otherwise we have a + conflict in the parsing actions (3, rather than 2, choices) +- wrap defines file with (where "yy" is value of 'symbol_prefix') + #ifndef __yy_defines_h_ + #define _yy_defines_h_ + + <defines> + + #endif +- avoid writing %%xdecls skeleton section twice if -r used +- eliminated 'body_3' and 'trailer_2' skeleton segments - no need to generate + yyerror() invocation dynamically; YYERROR_CALL() is already generated + earlier, and can be used in the pareser skeleton +- if -P flag (pure_parser), add yylloc as 2nd parameter to yylex() + (declaration & call) +- change YYPOSN to YYLTYPE and yyposn to yylloc (position corresponding to + yylval) for bison compatibility; add yyloc (corresponding to yyval) +- generate yylloc parameters for yylex & yyerror if %locations present +- add location as 1st parameter to declaraion & invocation of yyerror() if + %locations present +- output backtrack parsing tables if -B flag is present +- added generation of yystos[] with output_accessing_symbols() to allow + translation from a parser internal state number to the corresponding + grammar symbol number [0 .. nsyms) of the accessing symbol of that parser + state; used in the generated code for YYDESTRUCT_CALL() & + YYSTYPE_TOSTRING() to enable the correct semantic value union tag to be + determined when executing the implementation of YYDESTRUCT_CALL() or + YYSTYPE_TOSTRING() (similar to yystos[] in bison) +- added to output_prefix(): yystos; yycindex & yyctable if compiling + backtracking; yyloc & yylloc if %locations used +- extended yyname[] to include all grammar symbols, not just the terminal + symbols: '$end', 'error', '$accept', all non-terminals, including internally + generated non-terminals for embedded actions in rules, and 'illegal-symbol' + (which bison spells '$undefined'); '$end' already defined as a symbol 0, + rathern than adding 'end-of-file' as the name of symbol 0; added + 'illegal-symbol' from byacc-20140101 (NOTE: the comment in the code that + says byacc does not predefine '$end' and '$error' is incorrect; however, + both bison and byacc spell '$error' as 'error') +- added generation of #define YYTRANSLATE() from byacc-20140101, but changed + the definition for the undefined symbol case because it is no longer in + yyname[YYMAXTOKEN+1] but rather occurs after the last non-terminal symbol; + added #define YYUNDFTOKEN to contain the index in yyname of 'illegal-symbol' +- generate YYLTYPE in output_ltype() as a struct like for bison rather than + using #define in yaccpar.skel +- added 'write_code_lineno' invocation at start of 'output_prefix' +- added 'write_code_lineno' invocation at start of 'output_pure_parser' +- added 'write_code_lineno' invocation prior to generation of #include + for externs file +- added 'write_code_lineno' invocation after 1st 'write_section(fp, xdecls)' +- added '++outline;' prior to output of '#define YYTRANSLATE' - this was + actually causing almost all of the invocations of 'write_code_lineno' to + put out the correct #line directive +- corrected 'write_code_lineno' - the line number in a #line directive is + the number of the next line, not the number of the #line line +- changed MAXSHORT to MAXYYINT; changed 'high' local static from 'int' to + 'long' so that it can get higher than 'MAXYYINT' without machine-dependent + behavior; changed related formats from '%d' to '%ld' +- generate 'YYINT' rather than 'short' for integer table types +- generate YYDESTRUCT_DECL & YYDESTRUCT_CALL macros, similar to YYERROR_DECL + and YYERROR_CALL macros, that can be redefined by user, if desired, to add + additional parameters to yydestruct() (and even change the 'yydestruct' + function name) +- if at least one %destructor present, generate yydestruct(); 1st parameter + is a string indicating the context in which yydestruct() is invoked + (e.g., discarding input token, discarding state on stack, cleanup when + aborting); 2nd parameter is the internal grammar symbol number [0..nsyms) + of the accessing symbol of the parser state on the top of the stack; 3rd + parameter is a pointer to the semantic value to be reclaimed associated + with the grammar symbol in the 2nd parameter; if %locations is defined, + the 4th parameter is a pointer to the position value to be reclaimed + associated with the grammar symbol in the 2nd parameter + +reader.c +- adopt []-actions, similar to {}-actions; {}-actions are only executed when + not in trial mode, but []-actions are executed regardless of mode +- adopt new symbol class values ACTION and ARGUMENT +- adopt inherited attributes (syntax resembles arguments to non-terminal + symbols) +- adopt keyword table lookup from btyacc, modified to handle equivalence + of '-' and '_' in spelling of keywords +- adopt refactoring of tag table creation into cache_tag() for use in + multiple locations +- added new error functions in place of btyacc's generic error() function +- changed '0' to 'NULL' for pointer initialization +- reworked for-loop at end of get_line (part of DEFERRED '%ifdef/%endif' change) +- added %locations directive for bison compatibility to enable position + processing +- added decl for 'int locations'; true if %locations present +- added decl 'int backtrack'; initialized to 0 (= false), set to + 1 (= true) if -B flag is present +- process %locations if present, set location = 1 +- only process []-actions and only generate 'if (!yytrial)' prefix for + {}-actions if backtracking is enabled +- add decl for 'int destructor'; true if at least one %destructor is present +- add %destructor directive to enable semantic & position stack processing to + reclaim memory associated with discarded symbols +- process bison compatible %destructor (set destructor = 1); support @$ in + %destructor code to reference the position value if %locations is defined +- changed 'short' to 'Value_t' (in some instances, 'Value_t' was already + used for variables/parameters that were related to variables/parameters + declared as 'short' +- if %locations present, support @N and @$ syntax as for bison to reference + the locations associated with the N-th rhs symbol and the lhs symbol, + respectively + +symtab.c +- initialize fields added to 'struct bucket' for non-terminal symbol + inherited attributes + +verbose.c +- for parse states with conflicts, the contents of the y.output file include + the trial shift and/or trial reduce actions +- added output to the end of the verbose report showing the correspondance + between grammar symbol #, internal parser symbol #, and grammar symbol name +- changed 'short' to 'Value_t' (in some instances, 'Value_t' was already + used for variables/parameters that were related to variables/parameters + declared as 'short' + +yacc.1 +- added options 'P', 'V', 'y' and '-o output_file' to the yacc command + synopsis (already covered in the description section) +- added options 'B', 'D' and 'L' to the yacc command synopsis; added text in + the description section +- added %locations description to the extensions section |