diff options
Diffstat (limited to 'doc/flex.info-1')
| -rw-r--r-- | doc/flex.info-1 | 351 |
1 files changed, 81 insertions, 270 deletions
diff --git a/doc/flex.info-1 b/doc/flex.info-1 index 25f61b3..cda615c 100644 --- a/doc/flex.info-1 +++ b/doc/flex.info-1 @@ -1,4 +1,4 @@ -This is flex.info, produced by makeinfo version 4.8 from flex.texi. +This is flex.info, produced by makeinfo version 4.13 from flex.texi. INFO-DIR-SECTION Programming START-INFO-DIR-ENTRY @@ -8,8 +8,8 @@ END-INFO-DIR-ENTRY The flex manual is placed under the same licensing conditions as the rest of flex: - Copyright (C) 2001, 2002, 2003, 2004, 2005, 2006, 2007 The Flex -Project. + Copyright (C) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2012 The +Flex Project. Copyright (C) 1990, 1997 The Regents of the University of California. All rights reserved. @@ -51,8 +51,8 @@ This manual describes `flex', a tool for generating programs that perform pattern-matching on text. The manual includes both tutorial and reference sections. - This edition of `The flex Manual' documents `flex' version 2.5.35. -It was last updated on 10 September 2007. + This edition of `The flex Manual' documents `flex' version 2.5.37. +It was last updated on 22 July 2012. This manual was written by Vern Paxson, Will Estes and John Millaway. @@ -265,8 +265,8 @@ File: flex.info, Node: Copyright, Next: Reporting Bugs, Prev: Top, Up: Top The flex manual is placed under the same licensing conditions as the rest of flex: - Copyright (C) 2001, 2002, 2003, 2004, 2005, 2006, 2007 The Flex -Project. + Copyright (C) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2012 The +Flex Project. Copyright (C) 1990, 1997 The Regents of the University of California. All rights reserved. @@ -338,7 +338,6 @@ First some simple examples to get the flavor of how one uses `flex'. encounters the string `username' will replace it with the user's login name: - %% username printf( "%s", getlogin() ); @@ -351,7 +350,6 @@ rules. Here's another simple example: - int num_lines = 0, num_chars = 0; %% @@ -359,7 +357,8 @@ rules. . ++num_chars; %% - main() + + int main() { yylex(); printf( "# of lines = %d, # of chars = %d\n", @@ -377,12 +376,11 @@ other than a newline (indicated by the `.' regular expression). A somewhat more complicated example: - /* scanner for a toy Pascal-like language */ %{ /* need this for the call to atof() below */ - #include math.h> + #include <math.h> %} DIGIT [0-9] @@ -416,9 +414,7 @@ other than a newline (indicated by the `.' regular expression). %% - main( argc, argv ) - int argc; - char **argv; + int main( int argc, char **argv ) { ++argv, --argc; /* skip over program name */ if ( argc > 0 ) @@ -445,7 +441,6 @@ File: flex.info, Node: Format, Next: Patterns, Prev: Simple Examples, Up: To The `flex' input file consists of three sections, separated by a line containing only `%%'. - definitions %% rules @@ -471,7 +466,6 @@ definitions to simplify the scanner specification, and declarations of Name definitions have the form: - name definition The `name' is a word beginning with a letter or an underscore (`_') @@ -481,7 +475,6 @@ following the name and continuing to the end of the line. The definition can subsequently be referred to using `{name}', which will expand to `(definition)'. For example, - DIGIT [0-9] ID [a-z][a-z0-9]* @@ -489,12 +482,10 @@ expand to `(definition)'. For example, digit, and `ID' to be a regular expression which matches a letter followed by zero-or-more letters-or-digits. A subsequent reference to - {DIGIT}+"."{DIGIT}* is identical to - ([0-9])+"."([0-9])* and matches one-or-more digits followed by a `.' followed by @@ -514,7 +505,6 @@ want certain preprocessor macros to be defined or certain files to be included before the generated code. The single characters, `{' and `}' are used to delimit the `%top' block, as show in the example below: - %top{ /* This code goes at the "top" of the generated file. */ #include <stdint.h> @@ -536,7 +526,6 @@ File: flex.info, Node: Rules Section, Next: User Code Section, Prev: Definiti The "rules" section of the `flex' input contains a series of rules of the form: - pattern action where the pattern must be unindented and the action must begin on @@ -592,7 +581,6 @@ new line, with one or more whitespace characters before the initial All the comments in the following example are valid: - %{ /* code block */ %} @@ -620,7 +608,7 @@ File: flex.info, Node: Patterns, Next: Matching, Prev: Format, Up: Top 6 Patterns ********** -The patterns in the input (see *Note Rules Section::) are written using +The patterns in the input (see *note Rules Section::) are written using an extended set of regular expressions. These are: `x' @@ -705,7 +693,6 @@ an extended set of regular expressions. These are: The following are all valid: - (?:foo) same as (foo) (?i:ab7) same as ([aA][bB]7) (?-i:ab) same as (ab) @@ -756,7 +743,7 @@ an extended set of regular expressions. These are: yourself, or explicitly use `r/\r\n' for `r$'. `<s>r' - an `r', but only in start condition `s' (see *Note Start + an `r', but only in start condition `s' (see *note Start Conditions:: for discussion of start conditions). `<s1,s2,s3>r' @@ -782,12 +769,10 @@ Those grouped together have equal precedence (see special note on the precedence of the repeat operator, `{}', under the documentation for the `--posix' POSIX compliance option). For example, - foo|bar* is the same as - (foo)|(ba(r*)) since the `*' operator has higher precedence than concatenation, and @@ -796,13 +781,11 @@ matches _either_ the string `foo' _or_ the string `ba' followed by zero-or-more `r''s. To match `foo' or zero-or-more repetitions of the string `bar', use: - foo|(bar)* And to match a sequence of zero or more repetitions of `foo' and `bar': - (foo|bar)* In addition to characters and ranges of characters, character classes @@ -811,7 +794,6 @@ enclosed inside `[': and `:]' delimiters (which themselves must appear between the `[' and `]' of the character class. Other elements may occur inside the character class, too). The valid expressions are: - [:alnum:] [:alpha:] [:blank:] [:cntrl:] [:digit:] [:graph:] [:lower:] [:print:] [:punct:] @@ -825,7 +807,6 @@ any alphabetic or numeric character. Some systems don't provide For example, the following character classes are all equivalent: - [[:alnum:]] [[:alpha:][:digit:]] [[:alpha:][0-9]] @@ -871,7 +852,6 @@ not be desirable. Flex allows negation of character class expressions by prepending `^' to the POSIX character class name. - [:^alnum:] [:^alpha:] [:^blank:] [:^cntrl:] [:^digit:] [:^graph:] [:^lower:] [:^print:] [:^punct:] @@ -906,7 +886,6 @@ not be desirable. * The following are invalid: - foo/bar$ <sc1>foo<sc2>bar @@ -915,7 +894,6 @@ not be desirable. * The following will result in `$' or `^' being treated as a normal character: - foo|(bar$) foo|^bar @@ -923,7 +901,6 @@ not be desirable. `bar'-followed-by-a-newline, the following could be used (the special `|' action is explained below, *note Actions::): - foo | bar$ /* action goes here */ @@ -954,7 +931,6 @@ Actions::), and then the remaining input is scanned for another match. character in the input is considered matched and copied to the standard output. Thus, the simplest valid `flex' input is: - %% which generates a scanner that simply copies its input (one @@ -979,7 +955,6 @@ your heart's content, and calls to `unput()' do not destroy `yytext' (*note Actions::). Furthermore, existing `lex' programs sometimes access `yytext' externally using declarations of the form: - extern char yytext[]; This definition is erroneous when used with `%pointer', but correct @@ -1013,7 +988,6 @@ action is empty, then when the pattern is matched the input token is simply discarded. For example, here is the specification for a program which deletes all occurrences of `zap me' from its input: - %% "zap me" @@ -1023,7 +997,6 @@ output since they will be matched by the default rule. Here is a program which compresses multiple blanks and tabs down to a single blank, and throws away whitespace found at the end of a line: - %% [ \t]+ putchar( ' ' ); [ \t]+$ /* ignore this token */ @@ -1066,14 +1039,13 @@ within an action: `REJECT' directs the scanner to proceed on to the "second best" rule which matched the input (or a prefix of the input). The rule is chosen - as described above in *Note Matching::, and `yytext' and `yyleng' + as described above in *note Matching::, and `yytext' and `yyleng' set up appropriately. It may either be one which matched as much text as the originally chosen rule but came later in the `flex' input file, or one which matched less text. For example, the following will both count the words in the input and call the routine `special()' whenever `frob' is seen: - int word_count = 0; %% @@ -1087,7 +1059,6 @@ within an action: example, when the following scanner scans the token `abcd', it will write `abcdabcaba' to the output: - %% a | ab | @@ -1115,7 +1086,6 @@ within an action: `mega-kludge' the following will write `mega-mega-kludge' to the output: - %% mega- ECHO; yymore(); kludge ECHO; @@ -1137,7 +1107,6 @@ scanner looks for the next match. `yytext' and `yyleng' are adjusted appropriately (e.g., `yyleng' will now be equal to `n'). For example, on the input `foobar' the following will write out `foobarbar': - %% foobar ECHO; yyless(3); [a-z]+ ECHO; @@ -1154,7 +1123,6 @@ input file, not from other source files. will be the next character scanned. The following action will take the current token and cause it to be rescanned enclosed in parentheses. - { int i; /* Copy yytext because unput() trashes yytext */ @@ -1184,7 +1152,6 @@ input stream with an end-of-file. `input()' reads the next character from the input stream. For example, the following is one way to eat up C comments: - %% "/*" { register int c; @@ -1215,11 +1182,11 @@ example, the following is one way to eat up C comments: instead referred to as yyinput(), in order to avoid a name clash with the `C++' stream by the name of `input'.) - `YY_FLUSH_BUFFER()' flushes the scanner's internal buffer so that -the next time the scanner attempts to match a token, it will first -refill the buffer using `YY_INPUT()' (*note Generated Scanner::). This -action is a special case of the more general `yy_flush_buffer()' -function, described below (*note Multiple Input Buffers::) + `YY_FLUSH_BUFFER;' flushes the scanner's internal buffer so that the +next time the scanner attempts to match a token, it will first refill +the buffer using `YY_INPUT()' (*note Generated Scanner::). This action +is a special case of the more general `yy_flush_buffer;' function, +described below (*note Multiple Input Buffers::) `yyterminate()' can be used in lieu of a return statement in an action. It terminates the scanner and returns a 0 to the scanner's @@ -1238,7 +1205,6 @@ scanning routine `yylex()', a number of tables used by it for matching tokens, and a number of auxiliary routines and macros. By default, `yylex()' is declared as follows: - int yylex() { ... various definitions and the actions in here ... @@ -1248,7 +1214,6 @@ tokens, and a number of auxiliary routines and macros. By default, `int yylex( void )'.) This definition may be changed by defining the `YY_DECL' macro. For example, you could use: - #define YY_DECL float lexscan( a, b ) float a, b; to give the scanning routine the name `lexscan', returning a float, @@ -1303,7 +1268,6 @@ constant `YY_NULL' (0 on Unix systems) to indicate `EOF'. The default Here is a sample definition of `YY_INPUT' (in the definitions section of the input file): - %{ #define YY_INPUT(buf,result,max_size) \ { \ @@ -1329,7 +1293,7 @@ though `yywrap()' returned 1), or you must link with `-lfl' to obtain the default version of the routine, which always returns 1. For scanning from in-memory buffers (e.g., scanning strings), see -*Note Scanning Strings::. *Note Multiple Input Buffers::. +*note Scanning Strings::. *Note Multiple Input Buffers::. The scanner writes its `ECHO' output to the `yyout' global (default, `stdout'), which may be redefined by the user simply by assigning it to @@ -1345,7 +1309,6 @@ File: flex.info, Node: Start Conditions, Next: Multiple Input Buffers, Prev: rule whose pattern is prefixed with `<sc>' will only be active when the scanner is in the "start condition" named `sc'. For example, - <STRING>[^"]* { /* eat up the string body ... */ ... } @@ -1353,7 +1316,6 @@ scanner is in the "start condition" named `sc'. For example, will be active only when the scanner is in the `STRING' start condition, and - <INITIAL,STRING,QUOTE>\. { /* handle an escape ... */ ... } @@ -1381,7 +1343,6 @@ the rest (e.g., comments). is still a little vague, here's a simple example illustrating the connection between the two. The set of rules: - %s example %% @@ -1391,7 +1352,6 @@ connection between the two. The set of rules: is equivalent to - %x example %% @@ -1411,7 +1371,6 @@ condition. every start condition. Thus, the above example could also have been written: - %x example %% @@ -1422,7 +1381,6 @@ written: The default rule (to `ECHO' any unmatched character) remains active in start conditions. It is equivalent to: - <*>.|\n ECHO; `BEGIN(0)' returns to the original state where only the rules with @@ -1436,7 +1394,6 @@ of the rules section. For example, the following will cause the scanner to enter the `SPECIAL' start condition whenever `yylex()' is called and the global variable `enter_special' is true: - int enter_special; %x SPECIAL @@ -1454,7 +1411,6 @@ default it will treat it as three tokens, the integer `123', a dot the line by the string `expect-floats' it will treat it as a single token, the floating-point number `123.456': - %{ #include <math.h> %} @@ -1463,7 +1419,7 @@ token, the floating-point number `123.456': %% expect-floats BEGIN(expect); - <expect>[0-9]+@samp{.}[0-9]+ { + <expect>[0-9]+.[0-9]+ { printf( "found a float, = %f\n", atof( yytext ) ); } @@ -1486,7 +1442,6 @@ token, the floating-point number `123.456': Here is a scanner which recognizes (and discards) C comments while maintaining a count of the current input line. - %x comment %% int line_num = 1; @@ -1507,7 +1462,6 @@ a big win. be stored as such. Thus, the above could be extended in the following fashion: - %x comment foo %% int line_num = 1; @@ -1534,7 +1488,6 @@ fashion: integer-valued `YY_START' macro. For example, the above assignments to `comment_caller' could instead be written - comment_caller = YY_START; Flex provides `YYSTATE' as an alias for `YY_START' (since that is @@ -1549,7 +1502,6 @@ option-header::. *Note option-prefix::. using exclusive start conditions, including expanded escape sequences (but not including checking for a string that's too long): - %x str %% @@ -1610,7 +1562,6 @@ whole bunch of rules all preceded by the same start condition(s). Flex makes this a little easier and cleaner by introducing a notion of start condition "scope". A start condition scope is begun with: - <SCs>{ where `SCs' is a list of one or more start conditions. Inside the @@ -1618,7 +1569,6 @@ start condition scope, every rule automatically has the prefix `SCs>' applied to it, until a `}' which matches the initial `{'. So, for example, - <ESC>{ "\\n" return '\n'; "\\r" return '\r'; @@ -1628,7 +1578,6 @@ example, is equivalent to: - <ESC>"\\n" return '\n'; <ESC>"\\r" return '\r'; <ESC>"\\f" return '\f'; @@ -1744,7 +1693,6 @@ which expands include files (the `<<EOF>>' feature is discussed below). This first example uses yypush_buffer_state and yypop_buffer_state. Flex maintains the stack internally. - /* the "incl" state is used for picking up the name * of an include file */ @@ -1780,7 +1728,6 @@ Flex maintains the stack internally. example did, but manages its own input buffer stack manually (instead of letting flex do it). - /* the "incl" state is used for picking up the name * of an include file */ @@ -1899,13 +1846,11 @@ rule is given, it applies to _all_ start conditions which do not already have <<EOF>> actions. To specify an <<EOF>> rule for only the initial start condition, use: - <INITIAL><<EOF>> These rules are useful for catching things like unclosed comments. An example: - %x quote %% @@ -1936,7 +1881,6 @@ number of the matched rule (rules are numbered starting with 1). Suppose you want to profile how often each of your rules is matched. The following would do the trick: - #define YY_USER_ACTION ++ctr[yy_act] where `ctr' is an array to hold the counts for the different rules. @@ -1944,7 +1888,6 @@ Note that the macro `YY_NUM_RULES' gives the total number of rules (including the default rule), even if you use `-s)', so a correct declaration for `ctr' is: - int ctr[YY_NUM_RULES]; The macro `YY_USER_INIT' may be defined to provide an action which @@ -1956,7 +1899,7 @@ call a routine to read in a data table or open a logging file. control whether the current buffer is considered "interactive". An interactive buffer is processed more slowly, but must be used when the scanner's input source is indeed interactive to avoid problems due to -waiting to fill buffers (see the discussion of the `-I' flag in *Note +waiting to fill buffers (see the discussion of the `-I' flag in *note Scanner Options::). A non-zero value in the macro invocation marks the buffer as interactive, a zero value as non-interactive. Note that use of this macro overrides `%option always-interactive' or `%option @@ -2057,7 +2000,6 @@ input. This file is then included in the `flex' scanner. For example, if one of the tokens is `TOK_NUMBER', part of the scanner might look like: - %{ #include "y.tab.h" %} @@ -2088,7 +2030,6 @@ Scanner Options::. Even though there are many scanner options, a typical scanner might only specify the following options: - %option 8bit reentrant bison-bridge %option warn nodefault %option yylineno @@ -2124,7 +2065,6 @@ suppress the appearance of unneeded routines in the generated scanner. Each of the following, if unset (e.g., `%option nounput'), results in the corresponding routine not appearing in the generated scanner: - input, unput yy_push_state, yy_pop_state, yy_top_state yy_scan_buffer, yy_scan_bytes, yy_scan_string @@ -2199,7 +2139,7 @@ File: flex.info, Node: Options Affecting Scanner Behavior, Next: Code-Level An case of letters given in the `flex' input patterns will be ignored, and tokens in the input will be matched regardless of case. The matched text given in `yytext' will have the preserved case (i.e., - it will not be folded). For tricky behavior, see *Note case and + it will not be folded). For tricky behavior, see *note case and character ranges::. `-l, --lex-compat, `%option lex-compat'' @@ -2208,7 +2148,7 @@ File: flex.info, Node: Options Affecting Scanner Behavior, Next: Code-Level An Use of this option costs a considerable amount of performance, and it cannot be used with the `--c++', `--full', `--fast', `-Cf', or `-CF' options. For details on the compatibilities it provides, see - *Note Lex and Posix::. This option also results in the name + *note Lex and Posix::. This option also results in the name `YY_FLEX_LEX_COMPAT' being `#define''d in the generated scanner. `-B, --batch, `%option batch'' @@ -2377,7 +2317,7 @@ File: flex.info, Node: Code-Level And API Options, Next: Options for Scanner S located with respect to either the original `flex' input file (if the errors are due to code in the input file), or `lex.yy.c' (if the errors are `flex''s fault - you should report these sorts of - errors to the email address given in *Note Reporting Bugs::). + errors to the email address given in *note Reporting Bugs::). `-R, --reentrant, `%option reentrant'' instructs flex to generate a reentrant C scanner. The generated @@ -2410,7 +2350,6 @@ File: flex.info, Node: Code-Level And API Options, Next: Options for Scanner S output file from `lex.yy.c' to `lex.foo.c'. Here is a partial list of the names affected: - yy_create_buffer yy_delete_buffer yy_flex_debug @@ -2544,7 +2483,6 @@ File: flex.info, Node: Options for Scanner Speed and Size, Next: Debugging Opt trade off faster-executing scanners at the cost of larger tables with the following generally being true: - slowest & smallest -Cem -Cm @@ -2575,7 +2513,6 @@ File: flex.info, Node: Options for Scanner Speed and Size, Next: Debugging Opt general, if the pattern set contains both _keywords_ and a catch-all, _identifier_ rule, such as in the set: - "case" return TOK_CASE; "switch" return TOK_SWITCH; ... @@ -2613,7 +2550,6 @@ File: flex.info, Node: Debugging Options, Next: Miscellaneous Options, Prev: non-zero (which is the default), the scanner will write to `stderr' a line of the form: - -accepting rule at line 53 ("the matched text") The line number refers to the location of the rule in the file @@ -2696,7 +2632,6 @@ rules. Aside from the effects on scanner speed of the table compression `-C' options outlined above, there are a number of options/actions which degrade performance. These are, from most expensive to least: - REJECT arbitrary trailing context @@ -2707,7 +2642,7 @@ which degrade performance. These are, from most expensive to least: %option interactive %option always-interactive - @samp{^} beginning-of-line operator + ^ beginning-of-line operator yymore() with the first two all being quite expensive and the last two being @@ -2727,7 +2662,6 @@ newlines. In general, you should avoid rules such as `[^f]+', which match very long tokens, including newlines, and may possibly match your entire file! A better approach is to separate `[^f]+' into two rules: - %option yylineno %% [^f\n]+ @@ -2740,14 +2674,12 @@ amount of work for a complicated scanner. In principal, one begins by using the `-b' flag to generate a `lex.backup' file. For example, on the input: - %% foo return TOK_KEYWORD; foobar return TOK_KEYWORD; the file looks like: - State #6 is non-accepting - associated rule line numbers: 2 3 @@ -2792,7 +2724,6 @@ scanners. The way to remove the backing up is to add "error" rules: - %% foo return TOK_KEYWORD; foobar return TOK_KEYWORD; @@ -2807,7 +2738,6 @@ scanners. Eliminating backing up among a list of keywords can also be done using a "catch-all" rule: - %% foo return TOK_KEYWORD; foobar return TOK_KEYWORD; @@ -2831,20 +2761,17 @@ backing up. Leaving just one means you gain nothing. parts do not have a fixed length) entails almost the same performance loss as `REJECT' (i.e., substantial). So when possible a rule like: - %% mouse|rat/(cat|dog) run(); is better written: - %% mouse/cat|dog run(); rat/cat|dog run(); or as - %% mouse|rat/cat run(); mouse|rat/dog run(); @@ -2860,7 +2787,6 @@ long tokens the processing of most input characters takes place in the additional work of setting up the scanning environment (e.g., `yytext') for the action. Recall the scanner for C comments: - %x comment %% int line_num = 1; @@ -2874,7 +2800,6 @@ for the action. Recall the scanner for C comments: This could be sped up by writing it as: - %x comment %% int line_num = 1; @@ -2900,7 +2825,6 @@ through a file containing identifiers and keywords, one per line and with no other extraneous characters, and recognize all the keywords. A natural first approach is: - %% asm | auto | @@ -2913,7 +2837,6 @@ natural first approach is: To eliminate the back-tracking, introduce a catch-all rule: - %% asm | auto | @@ -2929,7 +2852,6 @@ natural first approach is: we can reduce the total number of matches by a half by merging in the recognition of newlines with that of the other tokens: - %% asm\n | auto\n | @@ -2954,7 +2876,6 @@ newlines, or, since we never expect to encounter such an input and therefore don't how it's classified, we can introduce one more catch-all rule, this one which doesn't include a newline: - %% asm\n | auto\n | @@ -2975,7 +2896,7 @@ a token contains multiple `NUL's. It's best to write rules which match _short_ amounts of text if it's anticipated that the text will often include `NUL's. - Another final note regarding performance: as mentioned in *Note + Another final note regarding performance: as mentioned in *note Matching::, dynamically resizing `yytext' to accommodate huge tokens is a slow process because it presently requires that the (huge) token be rescanned from the beginning. Thus if performance is vital, you should @@ -3077,7 +2998,7 @@ scanner: reads up to `max_size' characters into `buf' and returns the number of characters read. To indicate end-of-input, return 0 characters. Note that `interactive' scanners (see the `-B' and - `-I' flags in *Note Scanner Options::) define the macro + `-I' flags in *note Scanner Options::) define the macro `YY_INTERACTIVE'. If you redefine `LexerInput()' and need to take different actions depending on whether or not the scanner might be scanning an interactive input source, you can test for the @@ -3094,7 +3015,7 @@ scanner: Note that a `yyFlexLexer' object contains its _entire_ scanning state. Thus you can use such objects to create reentrant scanners, but -see also *Note Reentrant::. You can instantiate multiple instances of +see also *note Reentrant::. You can instantiate multiple instances of the same `yyFlexLexer' class, and you can also combine multiple C++ scanner classes together in the same program using the `-P' option discussed above. @@ -3104,13 +3025,16 @@ scanner classes; you must use `%pointer' (the default). Here is an example of a simple C++ scanner: - - // An example of using the flex C++ scanner class. + // An example of using the flex C++ scanner class. %{ + #include <iostream> + using namespace std; int mylineno = 0; %} + %option noyywrap + string \"[^\n"]+\" ws [ \t]+ @@ -3134,7 +3058,7 @@ scanner classes; you must use `%pointer' (the default). if(c == '\n') ++mylineno; - else if(c == @samp{*}) + else if(c == '*') { if((c = yyinput()) == '/') break; @@ -3144,23 +3068,23 @@ scanner classes; you must use `%pointer' (the default). } } - {number} cout "number " YYText() '\n'; + {number} cout << "number " << YYText() << '\n'; \n mylineno++; - {name} cout "name " YYText() '\n'; + {name} cout << "name " << YYText() << '\n'; - {string} cout "string " YYText() '\n'; + {string} cout << "string " << YYText() << '\n'; %% int main( int /* argc */, char** /* argv */ ) - { - @code{flex}Lexer* lexer = new yyFlexLexer; + { + FlexLexer* lexer = new yyFlexLexer; while(lexer->yylex() != 0) ; return 0; - } + } If you want to create multiple (different) lexer classes, you use the `-P' flag (or the `prefix=' option) to rename each `yyFlexLexer' to @@ -3168,7 +3092,6 @@ some other `xxFlexLexer'. You then can include `<FlexLexer.h>' in your other sources once per lexer class, first renaming `yyFlexLexer' as follows: - #undef yyFlexLexer #define yyFlexLexer xxFlexLexer #include <FlexLexer.h> @@ -3212,7 +3135,6 @@ However, there are other uses for a reentrant scanner. For example, you could scan two or more files simultaneously to implement a `diff' at the token level (i.e., instead of at the character level): - /* Example of maintaining more than one active scanner. */ do { @@ -3233,7 +3155,6 @@ buffer states. *Note Multiple Input Buffers::.) The following crude scanner supports the `eval' command by invoking another instance of itself. - /* Example of recursive invocation. */ %option reentrant @@ -3286,7 +3207,6 @@ File: flex.info, Node: Reentrant Example, Next: Reentrant Detail, Prev: Reent ====================== First, an example of a reentrant scanner: - /* This scanner prints "//" comments. */ %option reentrant stack noyywrap @@ -3359,7 +3279,6 @@ have an argument, `yyscanner' , that is not present in a non-reentrant scanner. Here are the declarations of `yy_push_state' and `yy_pop_state' in the reentrant scanner: - static void yy_push_state ( int new_state , yyscan_t yyscanner ) ; static void yy_pop_state ( yyscan_t yyscanner ) ; @@ -3370,7 +3289,7 @@ argument list, it is always of type `yyscan_t' (which is typedef'd to `void *') and it is always named `yyscanner'. As you may have guessed, `yyscanner' is a pointer to an opaque data structure encapsulating the current state of the scanner. For a list of function declarations, see -*Note Reentrant Functions::. Note that preprocessor macros, such as +*note Reentrant Functions::. Note that preprocessor macros, such as `BEGIN', `ECHO', and `REJECT', do not take this additional argument. @@ -3392,7 +3311,6 @@ tell you this so you don't expect to link to these variables externally. Currently, each macro expands to a member of an internal struct, e.g., - #define yytext (((struct yyguts_t*)yyscanner)->yytext_r) One important thing to remember about `yytext' and friends is that @@ -3410,7 +3328,6 @@ File: flex.info, Node: Init and Destroy Functions, Next: Accessor Methods, Pr `yylex_init' and `yylex_destroy' must be called before and after `yylex', respectively. - int yylex_init ( yyscan_t * ptr_yy_globals ) ; int yylex_init_extra ( YY_EXTRA_TYPE user_defined, yyscan_t * ptr_yy_globals ) ; int yylex ( yyscan_t yyscanner ) ; @@ -3451,7 +3368,6 @@ and non-reentrant) may be restarted by calling `yyrestart'. Below is an example of a program that creates a scanner, uses it, then destroys it when done: - int main () { yyscan_t scanner; @@ -3459,7 +3375,7 @@ then destroys it when done: yylex_init(&scanner); - while ((tok=yylex()) > 0) + while ((tok=yylex(scanner)) > 0) printf("tok=%d yytext=%s\n", tok, yyget_text(scanner)); yylex_destroy(scanner); @@ -3484,7 +3400,6 @@ Instead, you must access `flex' values using accessor methods (get/set functions). Each accessor method is named `yyget_NAME' or `yyset_NAME', where `NAME' is the name of the `flex' variable you want. For example: - /* Set the last character of yytext to NULL. */ void chop ( yyscan_t scanner ) { @@ -3494,7 +3409,6 @@ where `NAME' is the name of the `flex' variable you want. For example: The above code may be called from within an action like this: - %% .+\n { chop( yyscanner );} @@ -3522,7 +3436,6 @@ accessible through the accessor methods `yyget_extra' and `yyset_extra' from outside the scanner, and through the shortcut macro `yyextra' from within the scanner itself. They are defined as follows: - #define YY_EXTRA_TYPE void* YY_EXTRA_TYPE yyget_extra ( yyscan_t scanner ); void yyset_extra ( YY_EXTRA_TYPE arbitrary_data , yyscan_t scanner); @@ -3536,7 +3449,6 @@ the scanner itself. redefine this type using `%option extra-type="your_type"' in the scanner: - /* An example of overriding YY_EXTRA_TYPE. */ %{ #include <sys/stat.h> @@ -3574,7 +3486,6 @@ File: flex.info, Node: About yyscan_t, Prev: Extra Data, Up: Reentrant Detail `yyscan_t' is defined as: - typedef void* yyscan_t; It is initialized by `yylex_init()' to point to an internal @@ -3589,7 +3500,6 @@ File: flex.info, Node: Reentrant Functions, Prev: Reentrant Detail, Up: Reent The following Functions are available in a reentrant scanner: - char *yyget_text ( yyscan_t scanner ); int yyget_leng ( yyscan_t scanner ); FILE *yyget_in ( yyscan_t scanner ); @@ -3610,7 +3520,6 @@ intentional. The following Macro shortcuts are available in actions in a reentrant scanner: - yytext yyleng yyin @@ -3627,7 +3536,6 @@ to maintain the line count independently of `flex'. The following functions and macros are made available when `%option bison-bridge' (`--bison-bridge') is specified: - YYSTYPE * yyget_lval ( yyscan_t scanner ); void yyset_lval ( YYSTYPE * yylvalp , yyscan_t scanner ); yylval @@ -3635,7 +3543,6 @@ bison-bridge' (`--bison-bridge') is specified: The following functions and macros are made available when `%option bison-locations' (`--bison-locations') is specified: - YYLTYPE *yyget_lloc ( yyscan_t scanner ); void yyset_lloc ( YYLTYPE * yyllocp , yyscan_t scanner ); yylloc @@ -3695,12 +3602,10 @@ exceptions: handler which long-jumps out of the scanner, and the scanner is subsequently called again, you may get the following message: - - fatal @code{flex} scanner internal error--end of buffer missed + fatal flex scanner internal error--end of buffer missed To reenter the scanner, first use: - yyrestart( yyin ); Note that this call will throw away any buffered input; usually @@ -3709,7 +3614,7 @@ exceptions: * Also note that `flex' C++ scanner classes _are_ reentrant, so if using C++ is an option for you, you should use them instead. - *Note Cxx::, and *Note Reentrant:: for details. + *Note Cxx::, and *note Reentrant:: for details. * `output()' is not supported. Output from the ECHO macro is done to the file-pointer `yyout' (default `stdout)'. @@ -3722,7 +3627,6 @@ exceptions: * When definitions are expanded, `flex' encloses them in parentheses. With `lex', the following: - NAME [A-Z][A-Z0-9]* %% foo{NAME}? printf( "Found it\n" ); @@ -3749,7 +3653,6 @@ exceptions: * Some implementations of `lex' allow a rule's action to begin on a separate line, if the rule's pattern has trailing whitespace: - %% foo|bar<space here> { foobar_action();} @@ -3765,8 +3668,8 @@ exceptions: `-l' option does away with this incompatibility. * The precedence of the `{,}' (numeric range) operator is different. - The AT&T and POSIX specifications of `lex' interpret `abc{1,3}' - as match one, two, or three occurrences of `abc'", whereas `flex' + The AT&T and POSIX specifications of `lex' interpret `abc{1,3}' as + match one, two, or three occurrences of `abc'", whereas `flex' interprets it as "match `ab' followed by one, two, or three occurrences of `c'". The `-l' and `--posix' options do away with this incompatibility. @@ -3840,12 +3743,10 @@ specification: with `flex' you can put multiple actions on the same line, separated with semi-colons, while with `lex', the following: - foo handle_foo(); ++num_foos_seen; is (rather surprisingly) truncated to - foo handle_foo(); `flex' does not truncate the action. Actions that are not enclosed @@ -3897,7 +3798,7 @@ all memory when you call `yylex_destroy' *Note faq-memory-leak::. memory management API. 64kb for the REJECT state. This will only be allocated if you use REJECT. - The size is the large enough to hold the same number of states as + The size is large enough to hold the same number of states as characters in the input buffer. If you override the size of the input buffer (via `YY_BUF_SIZE'), then you automatically override the size of this buffer as well. @@ -3966,7 +3867,6 @@ telling flex that you will provide your own implementations. 2. Provide your own implementation of the following functions: (1) - // For a non-reentrant scanner void * yyalloc (size_t bytes); void * yyrealloc (void * ptr, size_t bytes); @@ -3984,7 +3884,6 @@ collection. In order to make this example interesting, we will use a reentrant scanner, passing a pointer to the custom allocator through `yyextra'. - %{ #include "some_allocator.h" %} @@ -4078,7 +3977,6 @@ File: flex.info, Node: Creating Serialized Tables, Next: Loading and Unloading You may create a scanner with serialized tables by specifying: - %option tables-file=FILE or --tables-file=FILE @@ -4097,7 +3995,6 @@ the serialized tables into one file, and flex will find the correct set of tables, using the scanner prefix as part of the lookup key. An example follows: - $ flex --tables-file --prefix=cpp cpp.l $ flex --tables-file --prefix=c c.l $ cat lex.cpp.tables lex.c.tables > all.tables @@ -4158,7 +4055,6 @@ This section defines the file format of serialized `flex' tables. specified, where each set corresponds to a given scanner. Scanners are indexed by name, as described below. The file format is as follows: - TABLE SET 1 +-------------------------------+ Header | uint32 th_magic; | @@ -4171,8 +4067,8 @@ indexed by name, as described below. The file format is as follows: +-------------------------------+ Table 1 | uint16 td_id; | | uint16 td_flags; | - | uint32 td_lolen; | | uint32 td_hilen; | + | uint32 td_lolen; | | void td_data[]; | | uint8 td_pad64[]; | +-------------------------------+ @@ -4303,12 +4199,6 @@ Fields of a table: elements or between structs. The type of each member is determined by the `YYTD_DATA*' bits. -`td_lolen' - Specifies the number of elements in the lowest dimension array. If - this is a one-dimensional array, then it is simply the number of - elements in this array. The element size is determined by the - `td_flags' field. - `td_hilen' If `td_hilen' is non-zero, then the data is a two-dimensional array. Otherwise, the data is a one-dimensional array. `td_hilen' @@ -4324,11 +4214,17 @@ Fields of a table: simply skipped. Flex does not currently generate tables of zero length. +`td_lolen' + Specifies the number of elements in the lowest dimension array. If + this is a one-dimensional array, then it is simply the number of + elements in this array. The element size is determined by the + `td_flags' field. + `td_data[]' The table data. This array may be a one- or two-dimensional array, of type `int8', `int16', `int32', `struct yy_trans_info', or `struct yy_trans_info*', depending upon the values in the - `td_flags', `td_lolen', and `td_hilen' fields. + `td_flags', `td_hilen', and `td_lolen' fields. `td_pad64[]' Zero or more NULL bytes, padding the entire table to the next @@ -4348,7 +4244,6 @@ The following is a list of `flex' diagnostic messages: cannot be matched because it comes after an identifier "catch-all" rule: - [a-z]+ got_identifier(); foo got_foo(); @@ -4384,7 +4279,7 @@ The following is a list of `flex' diagnostic messages: specification includes recognizing the 8-bit character `'x'' and you did not specify the -8 flag, and your scanner defaulted to 7-bit because you used the `-Cf' or `-CF' table compression - options. See the discussion of the `-7' flag, *Note Scanner + options. See the discussion of the `-7' flag, *note Scanner Options::, for details. * `flex scanner push-back overflow'. you used `unput()' to push back @@ -4402,7 +4297,6 @@ The following is a list of `flex' diagnostic messages: occur in a scanner which is reentered after a long-jump has jumped out (or over) the scanner's activation frame. Before reentering the scanner, use: - yyrestart( yyin ); or, as noted above, switch to using the C++ scanner class. @@ -4430,7 +4324,6 @@ special `|' action can result in _fixed_ trailing context being turned into the more expensive _variable_ trailing context. For example, in the following: - %% abc | xyz/def @@ -4647,7 +4540,6 @@ Does flex support recursive pattern definitions? e.g., - %% block "{"({block}|{statement})*"}" @@ -4689,7 +4581,6 @@ section *Note Matching::. around this behavior by expanding your short rule to match more text, then put back the extra: - data_.* yyless( 5 ); BEGIN BLOCKIDSTATE; Another fix would be to make the second rule active only during the @@ -4711,7 +4602,6 @@ My actions are executing out of order or sometimes not at all. Most likely, you have (in error) placed the opening `{' of the action block on a different line than the rule, e.g., - ^(foo|bar) { <<<--- WRONG! @@ -4721,7 +4611,6 @@ block on a different line than the rule, e.g., rule begin on the same line as does the rule. You need instead to write your rules as follows: - ^(foo|bar) { // CORRECT! } @@ -4794,7 +4683,6 @@ input. Then you redefine `YY_INPUT' to call your own routine which, if it sees an `EOF', returns the magic character first (and remembers to return a real `EOF' next time it's called). Then you could write: - <COMMENT>(.|\n)*{EOF_CHAR} /* saw comment at EOF */ @@ -4807,7 +4695,6 @@ You can do this as follows. Suppose you have a start condition `A', and after exhausting all of the possible matches in `<A>', you want to try matches in `<INITIAL>'. Then you could use the following: - %x A %% <A>rule_that_is_long ...; REJECT; @@ -4885,22 +4772,18 @@ How can I match C-style comments? You might be tempted to try something like this: - "/*".*"*/" // WRONG! or, worse, this: - "/*"(.|\n)"*/" // WRONG! The above rules will eat too much input, and blow up on things like: - /* a comment */ do_my_thing( "oops */" ); Here is one way which allows you to track line information: - <INITIAL>{ "/*" BEGIN(IN_COMMENT); } @@ -4992,7 +4875,6 @@ How can I use more than 8192 rules? you need more than 8192 rules in your scanner, you'll have to recompile `flex' with the following changes in `flexdef.h': - < #define YY_TRAILING_MASK 0x2000 < #define YY_TRAILING_HEAD_MASK 0x4000 -- @@ -5008,7 +4890,6 @@ number of rules is the best way to solve your problem. With luck, you should be able to increase the definitions in flexdef.h for: - #define JAMSTATE -32766 /* marks a reference to the state that always jams */ #define MAXIMUM_MNS 31999 #define BAD_SUBSCRIPT -32767 @@ -5039,7 +4920,6 @@ You can specify an initial action by defining the macro `YY_USER_INIT' (though note that `yyout' may not be available at the time this macro is executed). Or you can add to the beginning of your rules section: - %% /* Must be indented! */ static int did_init = 0; @@ -5085,7 +4965,6 @@ I get an error about undefined yywrap(). You must supply a `yywrap()' function of your own, or link to `libfl.a' (which provides one), or use - %option noyywrap in your source to say you don't want a `yywrap()' function. @@ -5110,7 +4989,6 @@ the parser. However, you can do this using multiple input buffers. - %% macro/[a-z]+ { /* Saw the macro "macro" followed by extra stuff. */ @@ -5163,7 +5041,6 @@ One way to assign precedence, is to place the more specific rules first. If two rules would match the same input (same sequence of characters) then the first rule listed in the `flex' input wins, e.g., - %% foo[a-zA-Z_]+ return FOO_ID; bar[a-zA-Z_]+ return BAR_ID; @@ -5194,7 +5071,7 @@ Is there a way to make flex treat NULL like a regular character? ================================================================ Yes, `\0' and `\x00' should both do the trick. Perhaps you have an -ancient version of `flex'. The latest release is version 2.5.35. +ancient version of `flex'. The latest release is version 2.5.37. File: flex.info, Node: Whenever flex can not match the input it says "flex scanner jammed"., Next: Why doesn't flex have non-greedy operators like perl does?, Prev: Is there a way to make flex treat NULL like a regular character?, Up: FAQ @@ -5204,7 +5081,6 @@ Whenever flex can not match the input it says "flex scanner jammed". You need to add a rule that matches the otherwise-unmatched text, e.g., - %option yylineno %% [[a bunch of rules here]] @@ -5261,7 +5137,6 @@ matter how many times you call `yylex()'. If you want to reclaim the memory when you are completely done scanning, then you might try this: - /* For non-reentrant C scanner only. */ yy_delete_buffer(YY_CURRENT_BUFFER); yy_init = 1; @@ -5276,7 +5151,6 @@ File: flex.info, Node: How do I track the byte offset for lseek()?, Next: How How do I track the byte offset for lseek()? =========================================== - > We thought that it would be possible to have this number through the > evaluation of the following expression: > @@ -5297,7 +5171,6 @@ already read from the current buffer. matched since starting to scan. This can be done by using `YY_USER_ACTION'. For example, - #define YY_USER_ACTION num_chars += yyleng; (You need to be careful to update your bookkeeping if you use @@ -5333,7 +5206,6 @@ other patterns? the phrase "endskip". The following will _NOT_ work correctly (do you see why not?) - /* INCORRECT SCANNER */ %x SKIP %% @@ -5345,14 +5217,12 @@ see why not?) The problem is that the pattern .* will eat up the word "endskip." The simplest (but slow) fix is: - <SKIP>"endskip" BEGIN(INITIAL); <SKIP>. ; The fix involves making the second rule match more, without making it match "endskip" plus something else. So for example: - <SKIP>"endskip" BEGIN(INITIAL); <SKIP>[^e]+ ; <SKIP>. ;/* so you eat up e's, too */ @@ -5363,7 +5233,6 @@ File: flex.info, Node: deleteme00, Next: Are certain equivalent patterns faste deleteme00 ========== - QUESTION: When was flex born? @@ -5378,7 +5247,6 @@ File: flex.info, Node: Are certain equivalent patterns faster than others?, Ne Are certain equivalent patterns faster than others? =================================================== - To: Adoram Rogel <adoram@orna.hybridge.com> Subject: Re: Flex 2.5.2 performance questions In-reply-to: Your message of Wed, 18 Sep 96 11:12:17 EDT. @@ -5456,7 +5324,6 @@ File: flex.info, Node: Is backing up a big deal?, Next: Can I fake multi-byte Is backing up a big deal? ========================= - To: Adoram Rogel <adoram@hybridge.com> Subject: Re: Flex 2.5.2 performance questions In-reply-to: Your message of Thu, 19 Sep 96 10:16:04 EDT. @@ -5509,7 +5376,6 @@ File: flex.info, Node: Can I fake multi-byte character support?, Next: deletem Can I fake multi-byte character support? ======================================== - To: Heeman_Lee@hp.com Subject: Re: flex - multi-byte support? In-reply-to: Your message of Thu, 03 Oct 1996 17:24:04 PDT. @@ -5548,7 +5414,6 @@ File: flex.info, Node: deleteme01, Next: Can you discuss some flex internals?, deleteme01 ========== - To: moleary@primus.com Subject: Re: Flex / Unicode compatibility question In-reply-to: Your message of Tue, 22 Oct 1996 10:15:42 PDT. @@ -5575,7 +5440,6 @@ File: flex.info, Node: Can you discuss some flex internals?, Next: unput() mes Can you discuss some flex internals? ==================================== - To: Johan Linde <jl@theophys.kth.se> Subject: Re: translation of flex In-reply-to: Your message of Sun, 10 Nov 1996 09:16:36 PST. @@ -5631,7 +5495,6 @@ File: flex.info, Node: unput() messes up yy_at_bol, Next: The | operator is no unput() messes up yy_at_bol =========================== - To: Xinying Li <xli@npac.syr.edu> Subject: Re: FLEX ? In-reply-to: Your message of Wed, 13 Nov 1996 17:28:38 PST. @@ -5662,7 +5525,6 @@ File: flex.info, Node: The | operator is not doing what I want, Next: Why can' The | operator is not doing what I want ======================================= - To: Alain.ISSARD@st.com Subject: Re: Start condition with FLEX In-reply-to: Your message of Mon, 18 Nov 1996 09:45:02 PST. @@ -5703,7 +5565,6 @@ File: flex.info, Node: Why can't flex understand this variable trailing context Why can't flex understand this variable trailing context pattern? ================================================================= - To: Gregory Margo <gmargo@newton.vip.best.com> Subject: Re: flex-2.5.3 bug report In-reply-to: Your message of Sat, 23 Nov 1996 16:50:09 PST. @@ -5729,7 +5590,6 @@ File: flex.info, Node: The ^ operator isn't working, Next: Trailing context is The ^ operator isn't working ============================ - To: Thomas Hadig <hadig@toots.physik.rwth-aachen.de> Subject: Re: Flex Bug ? In-reply-to: Your message of Tue, 26 Nov 1996 14:35:01 PST. @@ -5766,7 +5626,6 @@ File: flex.info, Node: Trailing context is getting confused with trailing optio Trailing context is getting confused with trailing optional patterns ==================================================================== - To: Adoram Rogel <adoram@hybridge.com> Subject: Re: Flex 2.5.4 BOF ??? In-reply-to: Your message of Tue, 26 Nov 1996 16:10:41 PST. @@ -5797,7 +5656,6 @@ File: flex.info, Node: Is flex GNU or not?, Next: ERASEME53, Prev: Trailing c Is flex GNU or not? =================== - To: Cameron MacKinnon <mackin@interlog.com> Subject: Re: Flex documentation bug In-reply-to: Your message of Mon, 02 Dec 1996 00:07:08 PST. @@ -5838,7 +5696,6 @@ File: flex.info, Node: ERASEME53, Next: I need to scan if-then-else blocks and ERASEME53 ========= - To: tsv@cs.UManitoba.CA Subject: Re: Flex (reg).. In-reply-to: Your message of Thu, 06 Mar 1997 23:50:16 PST. @@ -5864,7 +5721,6 @@ File: flex.info, Node: I need to scan if-then-else blocks and while loops, Nex I need to scan if-then-else blocks and while loops ================================================== - To: "Mike Stolnicki" <mstolnic@ford.com> Subject: Re: FLEX help In-reply-to: Your message of Fri, 30 May 1997 13:33:27 PDT. @@ -5893,7 +5749,6 @@ File: flex.info, Node: ERASEME55, Next: ERASEME56, Prev: I need to scan if-th ERASEME55 ========= - To: Colin Paul Adams <colin@colina.demon.co.uk> Subject: Re: Flex C++ classes and Bison In-reply-to: Your message of 09 Aug 1997 17:11:41 PDT. @@ -5923,7 +5778,6 @@ File: flex.info, Node: ERASEME56, Next: ERASEME57, Prev: ERASEME55, Up: FAQ ERASEME56 ========= - To: Mikael.Latvala@lmf.ericsson.se Subject: Re: Possible mistake in Flex v2.5 document In-reply-to: Your message of Fri, 05 Sep 1997 16:07:24 PDT. @@ -5960,7 +5814,6 @@ File: flex.info, Node: ERASEME57, Next: Is there a repository for flex scanner ERASEME57 ========= - To: "Marty Leisner" <leisner@sdsp.mc.xerox.com> Subject: Re: flex limitations In-reply-to: Your message of Sat, 06 Sep 1997 11:27:21 PDT. @@ -6032,7 +5885,6 @@ File: flex.info, Node: unnamed-faq-62, Next: unnamed-faq-63, Prev: I get an e unnamed-faq-62 ============== - To: Georg.Rehm@CL-KI.Uni-Osnabrueck.DE Subject: Re: Flex maximums In-reply-to: Your message of Mon, 17 Nov 1997 17:16:06 PST. @@ -6066,7 +5918,6 @@ File: flex.info, Node: unnamed-faq-63, Next: unnamed-faq-64, Prev: unnamed-fa unnamed-faq-63 ============== - To: jimmey@lexis-nexis.com (Jimmey Todd) Subject: Re: FLEX question regarding istream vs ifstream In-reply-to: Your message of Mon, 08 Dec 1997 15:54:15 PST. @@ -6099,7 +5950,6 @@ File: flex.info, Node: unnamed-faq-64, Next: unnamed-faq-65, Prev: unnamed-fa unnamed-faq-64 ============== - To: Enda Fadian <fadiane@piercom.ie> Subject: Re: Question related to Flex man page? In-reply-to: Your message of Tue, 16 Dec 1997 15:17:34 PST. @@ -6129,7 +5979,6 @@ File: flex.info, Node: unnamed-faq-65, Next: unnamed-faq-66, Prev: unnamed-fa unnamed-faq-65 ============== - To: hassan@larc.info.uqam.ca (Hassan Alaoui) Subject: Re: Need urgent Help In-reply-to: Your message of Sat, 20 Dec 1997 19:38:19 PST. @@ -6156,7 +6005,6 @@ File: flex.info, Node: unnamed-faq-66, Next: unnamed-faq-67, Prev: unnamed-fa unnamed-faq-66 ============== - To: mc0307@mclink.it Cc: gnu@prep.ai.mit.edu Subject: Re: [mc0307@mclink.it: Help request] @@ -6187,7 +6035,6 @@ File: flex.info, Node: unnamed-faq-67, Next: unnamed-faq-68, Prev: unnamed-fa unnamed-faq-67 ============== - To: hassan@larc.info.uqam.ca (Hassan Alaoui) Subject: Re: Thanks In-reply-to: Your message of Mon, 22 Dec 1997 16:06:35 PST. @@ -6213,7 +6060,6 @@ File: flex.info, Node: unnamed-faq-68, Next: unnamed-faq-69, Prev: unnamed-fa unnamed-faq-68 ============== - To: "Bart Niswonger" <NISWONGR@almaden.ibm.com> Subject: Re: flex 2.5: c++ scanners & start conditions In-reply-to: Your message of Tue, 06 Jan 1998 10:34:21 PST. @@ -6241,7 +6087,6 @@ File: flex.info, Node: unnamed-faq-69, Next: unnamed-faq-70, Prev: unnamed-fa unnamed-faq-69 ============== - To: "Boris Zinin" <boris@ippe.rssi.ru> Subject: Re: current position in flex buffer In-reply-to: Your message of Mon, 12 Jan 1998 18:58:23 PST. @@ -6264,7 +6109,6 @@ File: flex.info, Node: unnamed-faq-70, Next: unnamed-faq-71, Prev: unnamed-fa unnamed-faq-70 ============== - To: Bik.Dhaliwal@bis.org Subject: Re: Flex question In-reply-to: Your message of Mon, 26 Jan 1998 13:05:35 PST. @@ -6288,7 +6132,6 @@ File: flex.info, Node: unnamed-faq-71, Next: unnamed-faq-72, Prev: unnamed-fa unnamed-faq-71 ============== - To: Vladimir Alexiev <vladimir@cs.ualberta.ca> Subject: Re: flex: how to control start condition from parser? In-reply-to: Your message of Mon, 26 Jan 1998 05:50:16 PST. @@ -6318,7 +6161,6 @@ File: flex.info, Node: unnamed-faq-72, Next: unnamed-faq-73, Prev: unnamed-fa unnamed-faq-72 ============== - To: Barbara Denny <denny@3com.com> Subject: Re: freebsd flex bug? In-reply-to: Your message of Fri, 30 Jan 1998 12:00:43 PST. @@ -6354,7 +6196,6 @@ File: flex.info, Node: unnamed-faq-73, Next: unnamed-faq-74, Prev: unnamed-fa unnamed-faq-73 ============== - To: Maurice Petrie <mpetrie@infoscigroup.com> Subject: Re: Lost flex .l file In-reply-to: Your message of Mon, 02 Feb 1998 14:10:01 PST. @@ -6381,7 +6222,6 @@ File: flex.info, Node: unnamed-faq-74, Next: unnamed-faq-75, Prev: unnamed-fa unnamed-faq-74 ============== - To: jimmey@lexis-nexis.com (Jimmey Todd) Subject: Re: Flex performance question In-reply-to: Your message of Thu, 19 Feb 1998 11:01:17 PST. @@ -6410,7 +6250,6 @@ File: flex.info, Node: unnamed-faq-75, Next: unnamed-faq-76, Prev: unnamed-fa unnamed-faq-75 ============== - To: jimmey@lexis-nexis.com (Jimmey Todd) Subject: Re: Flex performance question In-reply-to: Your message of Thu, 19 Feb 1998 11:01:17 PST. @@ -6449,7 +6288,6 @@ File: flex.info, Node: unnamed-faq-76, Next: unnamed-faq-77, Prev: unnamed-fa unnamed-faq-76 ============== - To: "Frescatore, David (CRD, TAD)" <frescatore@exc01crdge.crd.ge.com> Subject: Re: FLEX 2.5 & THE YEAR 2000 In-reply-to: Your message of Wed, 03 Jun 1998 11:26:22 PDT. @@ -6471,7 +6309,6 @@ File: flex.info, Node: unnamed-faq-77, Next: unnamed-faq-78, Prev: unnamed-fa unnamed-faq-77 ============== - To: "Hans Dermot Doran" <htd@ibhdoran.com> Subject: Re: flex problem In-reply-to: Your message of Wed, 15 Jul 1998 21:30:13 PDT. @@ -6495,7 +6332,6 @@ File: flex.info, Node: unnamed-faq-78, Next: unnamed-faq-79, Prev: unnamed-fa unnamed-faq-78 ============== - To: soumen@almaden.ibm.com Subject: Re: Flex++ 2.5.3 instance member vs. static member In-reply-to: Your message of Mon, 27 Jul 1998 02:10:04 PDT. @@ -6528,7 +6364,6 @@ File: flex.info, Node: unnamed-faq-79, Next: unnamed-faq-80, Prev: unnamed-fa unnamed-faq-79 ============== - To: Adoram Rogel <adoram@hybridge.com> Subject: Re: More than 32K states change hangs In-reply-to: Your message of Tue, 04 Aug 1998 16:55:39 PDT. @@ -6576,7 +6411,6 @@ File: flex.info, Node: unnamed-faq-80, Next: unnamed-faq-81, Prev: unnamed-fa unnamed-faq-80 ============== - To: "Schmackpfeffer, Craig" <Craig.Schmackpfeffer@usa.xerox.com> Subject: Re: flex output for static code portion In-reply-to: Your message of Tue, 11 Aug 1998 11:55:30 PDT. @@ -6608,7 +6442,6 @@ File: flex.info, Node: unnamed-faq-81, Next: unnamed-faq-82, Prev: unnamed-fa unnamed-faq-81 ============== - Received: from 131.173.17.11 (131.173.17.11 [131.173.17.11]) by ee.lbl.gov (8.9.1/8.9.1) with ESMTP id AAA03838 for <vern@ee.lbl.gov>; Thu, 20 Aug 1998 00:47:57 -0700 (PDT) @@ -6664,7 +6497,6 @@ File: flex.info, Node: unnamed-faq-82, Next: unnamed-faq-83, Prev: unnamed-fa unnamed-faq-82 ============== - To: Georg.Rehm@CL-KI.Uni-Osnabrueck.DE Subject: Re: "flex scanner push-back overflow" In-reply-to: Your message of Thu, 20 Aug 1998 09:47:54 PDT. @@ -6693,7 +6525,6 @@ File: flex.info, Node: unnamed-faq-83, Next: unnamed-faq-84, Prev: unnamed-fa unnamed-faq-83 ============== - To: Jan Kort <jan@research.techforce.nl> Subject: Re: Flex In-reply-to: Your message of Fri, 04 Sep 1998 12:18:43 +0200. @@ -6740,7 +6571,6 @@ File: flex.info, Node: unnamed-faq-84, Next: unnamed-faq-85, Prev: unnamed-fa unnamed-faq-84 ============== - To: Patrick Krusenotto <krusenot@mac-info-link.de> Subject: Re: Problems with restarting flex-2.5.2-generated scanner In-reply-to: Your message of Thu, 24 Sep 1998 10:14:07 PDT. @@ -6767,7 +6597,6 @@ File: flex.info, Node: unnamed-faq-85, Next: unnamed-faq-86, Prev: unnamed-fa unnamed-faq-85 ============== - To: Henric Jungheim <junghelh@pe-nelson.com> Subject: Re: flex 2.5.4a In-reply-to: Your message of Tue, 27 Oct 1998 16:41:42 PST. @@ -6801,7 +6630,6 @@ File: flex.info, Node: unnamed-faq-86, Next: unnamed-faq-87, Prev: unnamed-fa unnamed-faq-86 ============== - To: "Repko, Billy D" <billy.d.repko@intel.com> Subject: Re: Compiling scanners In-reply-to: Your message of Wed, 13 Jan 1999 10:52:47 PST. @@ -6834,7 +6662,6 @@ File: flex.info, Node: unnamed-faq-87, Next: unnamed-faq-88, Prev: unnamed-fa unnamed-faq-87 ============== - To: Erick Branderhorst <Erick.Branderhorst@asml.nl> Subject: Re: flex input buffer In-reply-to: Your message of Tue, 09 Feb 1999 13:53:46 PST. @@ -6855,7 +6682,6 @@ File: flex.info, Node: unnamed-faq-88, Next: unnamed-faq-90, Prev: unnamed-fa unnamed-faq-88 ============== - To: "Guido Minnen" <guidomi@cogs.susx.ac.uk> Subject: Re: Flex error message In-reply-to: Your message of Wed, 24 Feb 1999 15:31:46 PST. @@ -6883,7 +6709,6 @@ File: flex.info, Node: unnamed-faq-90, Next: unnamed-faq-91, Prev: unnamed-fa unnamed-faq-90 ============== - To: "Dmitriy Goldobin" <gold@ems.chel.su> Subject: Re: FLEX trouble In-reply-to: Your message of Mon, 31 May 1999 18:44:49 PDT. @@ -6919,7 +6744,6 @@ File: flex.info, Node: unnamed-faq-91, Next: unnamed-faq-92, Prev: unnamed-fa unnamed-faq-91 ============== - Received: from mc-qout4.whowhere.com (mc-qout4.whowhere.com [209.185.123.18]) by ee.lbl.gov (8.9.3/8.9.3) with SMTP id IAA05100 for <vern@ee.lbl.gov>; Tue, 15 Jun 1999 08:56:06 -0700 (PDT) @@ -6988,7 +6812,6 @@ File: flex.info, Node: unnamed-faq-92, Next: unnamed-faq-93, Prev: unnamed-fa unnamed-faq-92 ============== - To: neko@my-deja.com Subject: Re: A question on flex C++ scanner In-reply-to: Your message of Tue, 15 Jun 1999 08:55:43 PDT. @@ -7008,7 +6831,6 @@ File: flex.info, Node: unnamed-faq-93, Next: unnamed-faq-94, Prev: unnamed-fa unnamed-faq-93 ============== - To: "Stones, Darren" <Darren.Stones@nectech.co.uk> Subject: Re: You're the man to see? In-reply-to: Your message of Wed, 23 Jun 1999 11:10:29 PDT. @@ -7033,7 +6855,6 @@ File: flex.info, Node: unnamed-faq-94, Next: unnamed-faq-95, Prev: unnamed-fa unnamed-faq-94 ============== - To: Petr Danecek <petr@ics.cas.cz> Subject: Re: flex - question In-reply-to: Your message of Mon, 28 Jun 1999 19:21:41 PDT. @@ -7070,7 +6891,6 @@ File: flex.info, Node: unnamed-faq-95, Next: unnamed-faq-96, Prev: unnamed-fa unnamed-faq-95 ============== - To: Tielman Koekemoer <tielman@spi.co.za> Subject: Re: Please help. In-reply-to: Your message of Thu, 08 Jul 1999 13:20:37 PDT. @@ -7112,7 +6932,6 @@ File: flex.info, Node: unnamed-faq-96, Next: unnamed-faq-97, Prev: unnamed-fa unnamed-faq-96 ============== - To: Tielman Koekemoer <tielman@spi.co.za> Subject: Re: Please help. In-reply-to: Your message of Fri, 09 Jul 1999 09:16:14 PDT. @@ -7140,7 +6959,6 @@ File: flex.info, Node: unnamed-faq-97, Next: unnamed-faq-98, Prev: unnamed-fa unnamed-faq-97 ============== - To: Sumanth Kamenani <skamenan@crl.nmsu.edu> Subject: Re: Error In-reply-to: Your message of Mon, 19 Jul 1999 23:08:41 PDT. @@ -7161,7 +6979,6 @@ File: flex.info, Node: unnamed-faq-98, Next: unnamed-faq-99, Prev: unnamed-fa unnamed-faq-98 ============== - To: daniel@synchrods.synchrods.COM (Daniel Senderowicz) Subject: Re: lex In-reply-to: Your message of Mon, 22 Nov 1999 11:19:04 PST. @@ -7185,7 +7002,6 @@ File: flex.info, Node: unnamed-faq-99, Next: unnamed-faq-100, Prev: unnamed-f unnamed-faq-99 ============== - To: archow@hss.hns.com Subject: Re: Regarding distribution of flex and yacc based grammars In-reply-to: Your message of Sun, 19 Dec 1999 17:50:24 +0530. @@ -7214,7 +7030,6 @@ File: flex.info, Node: unnamed-faq-100, Next: unnamed-faq-101, Prev: unnamed- unnamed-faq-100 =============== - To: Martin Gallwey <gallweym@hyperion.moe.ul.ie> Subject: Re: Flex, and self referencing rules In-reply-to: Your message of Sun, 20 Feb 2000 01:01:21 PST. @@ -7237,7 +7052,6 @@ File: flex.info, Node: unnamed-faq-101, Next: What is the difference between Y unnamed-faq-101 =============== - To: slg3@lehigh.edu (SAMUEL L. GULDEN) Subject: Re: Flex problem In-reply-to: Your message of Thu, 02 Mar 2000 12:29:04 PST. @@ -7274,7 +7088,6 @@ pass extra params when it calls yylex() from the parser. YY_DECL is the Flex declaration of yylex. The default is similar to this: - #define int yy_lex () @@ -7294,7 +7107,7 @@ How do I access the values set in a Flex action from within a Bison action? =========================================================================== With $1, $2, $3, etc. These are called "Semantic Values" in the Bison -manual. See *Note Top: (bison)Top. +manual. See *note Top: (bison)Top. File: flex.info, Node: Appendices, Next: Indices, Prev: FAQ, Up: Top @@ -7330,7 +7143,6 @@ following Makefile does not explicitly instruct `make' how to build `foo.c' from `foo.l'. Instead, it relies on the implicit rules of the `make' program to build the intermediate file, `scan.c': - # Basic Makefile -- relies on implicit rules # Creates "myprogram" from "scan.l" and "myprogram.c" # @@ -7342,7 +7154,6 @@ following Makefile does not explicitly instruct `make' how to build may have to explicitly instruct `make' how to build your scanner. The following is an example of a Makefile containing explicit rules: - # Basic Makefile -- provides explicit rules # Creates "myprogram" from "scan.l" and "myprogram.c" # @@ -7373,7 +7184,6 @@ with. Since a `flex' scanner will typically include a header file header file is generated BEFORE the scanner is compiled. We handle this case in the following example: - # Makefile example -- scanner and parser. # Creates "myprogram" from "scan.l", "parse.y", and "myprogram.c" # @@ -7389,7 +7199,6 @@ case in the following example: In the above example, notice the line, - scan.o: scan.l parse.c , which lists the file `parse.c' (the generated parser) as a @@ -7398,7 +7207,7 @@ before the scanner is compiled, and the above line seems to do the trick. Feel free to experiment with your specific implementation of `make'. - For more details on writing Makefiles, see *Note Top: (make)Top. + For more details on writing Makefiles, see *note Top: (make)Top. ---------- Footnotes ---------- @@ -7421,7 +7230,7 @@ This section describes the `flex' features useful when integrating `flex' with `GNU bison'(1). Skip this section if you are not using `bison' with your scanner. Here we discuss only the `flex' half of the `flex' and `bison' pair. We do not discuss `bison' in any detail. For -more information about generating `bison' parsers, see *Note Top: +more information about generating `bison' parsers, see *note Top: (bison)Top. A compatible `bison' scanner is generated by declaring `%option @@ -7434,13 +7243,11 @@ bison-functions::. The declaration of yylex becomes, - int yylex ( YYSTYPE * lvalp, yyscan_t scanner ); If `%option bison-locations' is specified, then the declaration becomes, - int yylex ( YYSTYPE * lvalp, YYLTYPE * llocp, yyscan_t scanner ); Note that the macros `yylval' and `yylloc' evaluate to pointers. @@ -7448,7 +7255,6 @@ Support for `yylloc' is optional in `bison', so it is optional in `flex' as well. The following is an example of a `flex' scanner that is compatible with `bison'. - /* Scanner for "C" assignment statements... sort of. */ %{ #include "y.tab.h" /* Generated by bison. */ @@ -7468,7 +7274,6 @@ as we would any other variable. The data type of `yylval' is generated by `bison', and included in the file `y.tab.h'. Here is the corresponding `bison' parser: - /* Parser to convert "C" assignments to lisp. */ %{ /* Pass the argument to yyparse through to yylex. */ @@ -7560,10 +7365,9 @@ C99 hexadecimal constant `0[xX][[:xdigit:]]+' C99 octal constant - `0[0123456]*' + `0[01234567]*' C99 floating point constant - {dseq} ([[:digit:]]+) {dseq_opt} ([[:digit:]]*) {frac} (({dseq_opt}"."{dseq})|{dseq}".") @@ -7591,7 +7395,6 @@ A.4.2 Identifiers ----------------- C99 Identifier - ucn ((\\u([[:xdigit:]]{4}))|(\\U([[:xdigit:]]{8}))) nondigit [_[:alpha:]] c99_id ([_[:alpha:]]|{ucn})([_[:alnum:]]|{ucn})* @@ -7602,7 +7405,6 @@ C99 Identifier with the addition of the `$' character. UTF-8 Encoded Unicode Code Point - [\x09\x0A\x0D\x20-\x7E]|[\xC2-\xDF][\x80-\xBF]|\xE0[\xA0-\xBF][\x80-\xBF]|[\xE1-\xEC\xEE\xEF]([\x80-\xBF]{2})|\xED[\x80-\x9F][\x80-\xBF]|\xF0[\x90-\xBF]([\x80-\xBF]{2})|[\xF1-\xF3]([\x80-\xBF]{3})|\xF4[\x80-\x8F]([\x80-\xBF]{2}) @@ -7628,7 +7430,6 @@ C99 Comment appear at sane intervals. This is also more efficient when used with automatic line number processing. *Note option-yylineno::. - <INITIAL>{ "/*" BEGIN(COMMENT); } @@ -7647,23 +7448,33 @@ A.4.4 Addresses --------------- IPv4 Address - `(([[:digit:]]{1,3}"."){3}([[:digit:]]{1,3}))' + dec-octet [0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5] + IPv4address {dec-octet}\.{dec-octet}\.{dec-octet}\.{dec-octet} IPv6 Address - - hex4 ([[:xdigit:]]{1,4}) - hexseq ({hex4}(:{hex4}*)) - hexpart ({hexseq}|({hexseq}::({hexseq}?))|::{hexseq}) - IPv6address ({hexpart}(":"{IPv4address})?) - - See RFC2373 for details. + h16 [0-9A-Fa-f]{1,4} + ls32 {h16}:{h16}|{IPv4address} + IPv6address ({h16}:){6}{ls32}| + ::({h16}:){5}{ls32}| + ({h16})?::({h16}:){4}{ls32}| + (({h16}:){0,1}{h16})?::({h16}:){3}{ls32}| + (({h16}:){0,2}{h16})?::({h16}:){2}{ls32}| + (({h16}:){0,3}{h16})?::{h16}:{ls32}| + (({h16}:){0,4}{h16})?::{ls32}| + (({h16}:){0,5}{h16})?::{h16}| + (({h16}:){0,6}{h16})?:: + + See RFC 2373 (http://www.ietf.org/rfc/rfc2373.txt) for details. + Note that you have to fold the definition of `IPv6address' into one + line and that it also matches the "unspecified address" "::". URI `(([^:/?#]+):)?("//"([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))?' This pattern is nearly useless, since it allows just about any character to appear in a URI, including spaces and control - characters. See RFC2396 for details. + characters. See RFC 2396 (http://www.ietf.org/rfc/rfc2396.txt) + for details. |