summaryrefslogtreecommitdiff
path: root/libs/spirit/doc/html/spirit/lex/abstracts/lexer_semantic_actions.html
blob: cb68385bd1137da720e1387d16d8a4721e146520 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
<title>Lexer Semantic Actions</title>
<link rel="stylesheet" href="../../../../../../../doc/src/boostbook.css" type="text/css">
<meta name="generator" content="DocBook XSL Stylesheets V1.76.1">
<link rel="home" href="../../../index.html" title="Spirit 2.5.2">
<link rel="up" href="../abstracts.html" title="Abstracts">
<link rel="prev" href="lexer_tokenizing.html" title="Tokenizing Input Data">
<link rel="next" href="lexer_static_model.html" title="The Static Lexer Model">
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
<table cellpadding="2" width="100%"><tr>
<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../boost.png"></td>
<td align="center"><a href="../../../../../../../index.html">Home</a></td>
<td align="center"><a href="../../../../../../../libs/libraries.htm">Libraries</a></td>
<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
<td align="center"><a href="../../../../../../../more/index.htm">More</a></td>
</tr></table>
<hr>
<div class="spirit-nav">
<a accesskey="p" href="lexer_tokenizing.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../abstracts.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="lexer_static_model.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a>
</div>
<div class="section">
<div class="titlepage"><div><div><h4 class="title">
<a name="spirit.lex.abstracts.lexer_semantic_actions"></a><a class="link" href="lexer_semantic_actions.html" title="Lexer Semantic Actions">Lexer
        Semantic Actions</a>
</h4></div></div></div>
<p>
          The main task of a lexer normally is to recognize tokens in the input.
          Traditionally this has been complemented with the possibility to execute
          arbitrary code whenever a certain token has been detected. <span class="emphasis"><em>Spirit.Lex</em></span>
          has been designed to support this mode of operation as well. We borrow
          from the concept of semantic actions for parsers (<span class="emphasis"><em>Spirit.Qi</em></span>)
          and generators (<span class="emphasis"><em>Spirit.Karma</em></span>). Lexer semantic actions
          may be attached to any token definition. These are C++ functions or function
          objects that are called whenever a token definition successfully recognizes
          a portion of the input. Say you have a token definition <code class="computeroutput"><span class="identifier">D</span></code>,
          and a C++ function <code class="computeroutput"><span class="identifier">f</span></code>, you
          can make the lexer call <code class="computeroutput"><span class="identifier">f</span></code>
          whenever it matches an input by attaching <code class="computeroutput"><span class="identifier">f</span></code>:
        </p>
<pre class="programlisting"><span class="identifier">D</span><span class="special">[</span><span class="identifier">f</span><span class="special">]</span>
</pre>
<p>
          The expression above links <code class="computeroutput"><span class="identifier">f</span></code>
          to the token definition, <code class="computeroutput"><span class="identifier">D</span></code>.
          The required prototype of <code class="computeroutput"><span class="identifier">f</span></code>
          is:
        </p>
<pre class="programlisting"><span class="keyword">void</span> <span class="identifier">f</span> <span class="special">(</span><span class="identifier">Iterator</span><span class="special">&amp;</span> <span class="identifier">start</span><span class="special">,</span> <span class="identifier">Iterator</span><span class="special">&amp;</span> <span class="identifier">end</span><span class="special">,</span> <span class="identifier">pass_flag</span><span class="special">&amp;</span> <span class="identifier">matched</span><span class="special">,</span> <span class="identifier">Idtype</span><span class="special">&amp;</span> <span class="identifier">id</span><span class="special">,</span> <span class="identifier">Context</span><span class="special">&amp;</span> <span class="identifier">ctx</span><span class="special">);</span>
</pre>
<div class="variablelist">
<p class="title"><b>where:</b></p>
<dl>
<dt><span class="term"><code class="computeroutput"><span class="identifier">Iterator</span><span class="special">&amp;</span>
            <span class="identifier">start</span></code></span></dt>
<dd><p>
                This is the iterator pointing to the begin of the matched range in
                the underlying input sequence. The type of the iterator is the same
                as specified while defining the type of the <code class="computeroutput"><span class="identifier">lexertl</span><span class="special">::</span><span class="identifier">actor_lexer</span><span class="special">&lt;...&gt;</span></code> (its first template parameter).
                The semantic action is allowed to change the value of this iterator
                influencing, the matched input sequence.
              </p></dd>
<dt><span class="term"><code class="computeroutput"><span class="identifier">Iterator</span><span class="special">&amp;</span>
            <span class="identifier">end</span></code></span></dt>
<dd><p>
                This is the iterator pointing to the end of the matched range in
                the underlying input sequence. The type of the iterator is the same
                as specified while defining the type of the <code class="computeroutput"><span class="identifier">lexertl</span><span class="special">::</span><span class="identifier">actor_lexer</span><span class="special">&lt;...&gt;</span></code> (its first template parameter).
                The semantic action is allowed to change the value of this iterator
                influencing, the matched input sequence.
              </p></dd>
<dt><span class="term"><code class="computeroutput"><span class="identifier">pass_flag</span><span class="special">&amp;</span>
            <span class="identifier">matched</span></code></span></dt>
<dd><p>
                This value is pre/initialized to <code class="computeroutput"><span class="identifier">pass_normal</span></code>.
                If the semantic action sets it to <code class="computeroutput"><span class="identifier">pass_fail</span></code>
                this behaves as if the token has not been matched in the first place.
                If the semantic action sets this to <code class="computeroutput"><span class="identifier">pass_ignore</span></code>
                the lexer ignores the current token and tries to match a next token
                from the input.
              </p></dd>
<dt><span class="term"><code class="computeroutput"><span class="identifier">Idtype</span><span class="special">&amp;</span>
            <span class="identifier">id</span></code></span></dt>
<dd><p>
                This is the token id of the type Idtype (most of the time this will
                be a <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">size_t</span></code>) for the matched token.
                The semantic action is allowed to change the value of this token
                id, influencing the if of the created token.
              </p></dd>
<dt><span class="term"><code class="computeroutput"><span class="identifier">Context</span><span class="special">&amp;</span>
            <span class="identifier">ctx</span></code></span></dt>
<dd><p>
                This is a reference to a lexer specific, unspecified type, providing
                the context for the current lexer state. It can be used to access
                different internal data items and is needed for lexer state control
                from inside a semantic action.
              </p></dd>
</dl>
</div>
<p>
          When using a C++ function as the semantic action the following prototypes
          are allowed as well:
        </p>
<pre class="programlisting"><span class="keyword">void</span> <span class="identifier">f</span> <span class="special">(</span><span class="identifier">Iterator</span><span class="special">&amp;</span> <span class="identifier">start</span><span class="special">,</span> <span class="identifier">Iterator</span><span class="special">&amp;</span> <span class="identifier">end</span><span class="special">,</span> <span class="identifier">pass_flag</span><span class="special">&amp;</span> <span class="identifier">matched</span><span class="special">,</span> <span class="identifier">Idtype</span><span class="special">&amp;</span> <span class="identifier">id</span><span class="special">);</span>
<span class="keyword">void</span> <span class="identifier">f</span> <span class="special">(</span><span class="identifier">Iterator</span><span class="special">&amp;</span> <span class="identifier">start</span><span class="special">,</span> <span class="identifier">Iterator</span><span class="special">&amp;</span> <span class="identifier">end</span><span class="special">,</span> <span class="identifier">pass_flag</span><span class="special">&amp;</span> <span class="identifier">matched</span><span class="special">);</span>
<span class="keyword">void</span> <span class="identifier">f</span> <span class="special">(</span><span class="identifier">Iterator</span><span class="special">&amp;</span> <span class="identifier">start</span><span class="special">,</span> <span class="identifier">Iterator</span><span class="special">&amp;</span> <span class="identifier">end</span><span class="special">);</span>
<span class="keyword">void</span> <span class="identifier">f</span> <span class="special">();</span>
</pre>
<div class="important"><table border="0" summary="Important">
<tr>
<td rowspan="2" align="center" valign="top" width="25"><img alt="[Important]" src="../../../images/important.png"></td>
<th align="left">Important</th>
</tr>
<tr><td align="left" valign="top"><p>
            In order to use lexer semantic actions you need to use type <code class="computeroutput"><span class="identifier">lexertl</span><span class="special">::</span><span class="identifier">actor_lexer</span><span class="special">&lt;&gt;</span></code>
            as your lexer class (instead of the type <code class="computeroutput"><span class="identifier">lexertl</span><span class="special">::</span><span class="identifier">lexer</span><span class="special">&lt;&gt;</span></code> as described in earlier examples).
          </p></td></tr>
</table></div>
<h6>
<a name="spirit.lex.abstracts.lexer_semantic_actions.h0"></a>
          <span><a name="spirit.lex.abstracts.lexer_semantic_actions.the_context_of_a_lexer_semantic_action"></a></span><a class="link" href="lexer_semantic_actions.html#spirit.lex.abstracts.lexer_semantic_actions.the_context_of_a_lexer_semantic_action">The
          context of a lexer semantic action</a>
        </h6>
<p>
          The last parameter passed to any lexer semantic action is a reference to
          an unspecified type (see the <code class="computeroutput"><span class="identifier">Context</span></code>
          type in the table above). This type is unspecified because it depends on
          the token type returned by the lexer. It is implemented in the internals
          of the iterator type exposed by the lexer. Nevertheless, any context type
          is expected to expose a couple of functions allowing to influence the behavior
          of the lexer. The following table gives an overview and a short description
          of the available functionality.
        </p>
<div class="table">
<a name="spirit.lex.abstracts.lexer_semantic_actions.functions_exposed_by_any_context_passed_to_a_lexer_semantic_action"></a><p class="title"><b>Table&#160;8.&#160;Functions exposed by any context passed to a lexer semantic action</b></p>
<div class="table-contents"><table class="table" summary="Functions exposed by any context passed to a lexer semantic action">
<colgroup>
<col>
<col>
</colgroup>
<thead><tr>
<th>
                  <p>
                    Name
                  </p>
                </th>
<th>
                  <p>
                    Description
                  </p>
                </th>
</tr></thead>
<tbody>
<tr>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">Iterator</span> <span class="keyword">const</span><span class="special">&amp;</span> <span class="identifier">get_eoi</span><span class="special">()</span> <span class="keyword">const</span></code>
                  </p>
                </td>
<td>
                  <p>
                    The function <code class="computeroutput"><span class="identifier">get_eoi</span><span class="special">()</span></code> may be used by to access the
                    end iterator of the input stream the lexer has been initialized
                    with
                  </p>
                </td>
</tr>
<tr>
<td>
                  <p>
                    <code class="computeroutput"><span class="keyword">void</span> <span class="identifier">more</span><span class="special">()</span></code>
                  </p>
                </td>
<td>
                  <p>
                    The function <code class="computeroutput"><span class="identifier">more</span><span class="special">()</span></code> tells the lexer that the next
                    time it matches a rule, the corresponding token should be appended
                    onto the current token value rather than replacing it.
                  </p>
                </td>
</tr>
<tr>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">Iterator</span> <span class="keyword">const</span><span class="special">&amp;</span> <span class="identifier">less</span><span class="special">(</span><span class="identifier">Iterator</span>
                    <span class="keyword">const</span><span class="special">&amp;</span>
                    <span class="identifier">it</span><span class="special">,</span>
                    <span class="keyword">int</span> <span class="identifier">n</span><span class="special">)</span></code>
                  </p>
                </td>
<td>
                  <p>
                    The function <code class="computeroutput"><span class="identifier">less</span><span class="special">()</span></code> returns an iterator positioned
                    to the nth input character beyond the current token start iterator
                    (i.e. by passing the return value to the parameter <code class="computeroutput"><span class="identifier">end</span></code> it is possible to return
                    all but the first n characters of the current token back to the
                    input stream.
                  </p>
                </td>
</tr>
<tr>
<td>
                  <p>
                    <code class="computeroutput"><span class="keyword">bool</span> <span class="identifier">lookahead</span><span class="special">(</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">size_t</span>
                    <span class="identifier">id</span><span class="special">)</span></code>
                  </p>
                </td>
<td>
                  <p>
                    The function <code class="computeroutput"><span class="identifier">lookahead</span><span class="special">()</span></code> can be used to implement lookahead
                    for lexer engines not supporting constructs like flex' <code class="computeroutput"><span class="identifier">a</span><span class="special">/</span><span class="identifier">b</span></code> (match <code class="computeroutput"><span class="identifier">a</span></code>,
                    but only when followed by <code class="computeroutput"><span class="identifier">b</span></code>).
                    It invokes the lexer on the input following the current token
                    without actually moving forward in the input stream. The function
                    returns whether the lexer was able to match a token with the
                    given token-id <code class="computeroutput"><span class="identifier">id</span></code>.
                  </p>
                </td>
</tr>
<tr>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">size_t</span> <span class="identifier">get_state</span><span class="special">()</span> <span class="keyword">const</span></code>
                    and <code class="computeroutput"><span class="keyword">void</span> <span class="identifier">set_state</span><span class="special">(</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">size_t</span>
                    <span class="identifier">state</span><span class="special">)</span></code>
                  </p>
                </td>
<td>
                  <p>
                    The functions <code class="computeroutput"><span class="identifier">get_state</span><span class="special">()</span></code> and <code class="computeroutput"><span class="identifier">set_state</span><span class="special">()</span></code> may be used to introspect and
                    change the current lexer state.
                  </p>
                </td>
</tr>
<tr>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">token_value_type</span> <span class="identifier">get_value</span><span class="special">()</span>
                    <span class="keyword">const</span></code> and <code class="computeroutput"><span class="keyword">void</span> <span class="identifier">set_value</span><span class="special">(</span><span class="identifier">Value</span>
                    <span class="keyword">const</span><span class="special">&amp;)</span></code>
                  </p>
                </td>
<td>
                  <p>
                    The functions <code class="computeroutput"><span class="identifier">get_value</span><span class="special">()</span></code> and <code class="computeroutput"><span class="identifier">set_value</span><span class="special">()</span></code> may be used to introspect and
                    change the current token value.
                  </p>
                </td>
</tr>
</tbody>
</table></div>
</div>
<br class="table-break"><h6>
<a name="spirit.lex.abstracts.lexer_semantic_actions.h1"></a>
          <span><a name="spirit.lex.abstracts.lexer_semantic_actions.lexer_semantic_actions_using_phoenix"></a></span><a class="link" href="lexer_semantic_actions.html#spirit.lex.abstracts.lexer_semantic_actions.lexer_semantic_actions_using_phoenix">Lexer
          Semantic Actions Using Phoenix</a>
        </h6>
<p>
          Even if it is possible to write your own function object implementations
          (i.e. using Boost.Lambda or Boost.Bind), the preferred way of defining
          lexer semantic actions is to use <a href="../../../../../../../libs/phoenix/doc/html/index.html" target="_top">Boost.Phoenix</a>.
          In this case you can access the parameters described above by using the
          predefined <a href="http://boost-spirit.com" target="_top">Spirit</a> placeholders:
        </p>
<div class="table">
<a name="spirit.lex.abstracts.lexer_semantic_actions.predefined_phoenix_placeholders_for_lexer_semantic_actions"></a><p class="title"><b>Table&#160;9.&#160;Predefined Phoenix placeholders for lexer semantic actions</b></p>
<div class="table-contents"><table class="table" summary="Predefined Phoenix placeholders for lexer semantic actions">
<colgroup>
<col>
<col>
</colgroup>
<thead><tr>
<th>
                  <p>
                    Placeholder
                  </p>
                </th>
<th>
                  <p>
                    Description
                  </p>
                </th>
</tr></thead>
<tbody>
<tr>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">_start</span></code>
                  </p>
                </td>
<td>
                  <p>
                    Refers to the iterator pointing to the beginning of the matched
                    input sequence. Any modifications to this iterator value will
                    be reflected in the generated token.
                  </p>
                </td>
</tr>
<tr>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">_end</span></code>
                  </p>
                </td>
<td>
                  <p>
                    Refers to the iterator pointing past the end of the matched input
                    sequence. Any modifications to this iterator value will be reflected
                    in the generated token.
                  </p>
                </td>
</tr>
<tr>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">_pass</span></code>
                  </p>
                </td>
<td>
                  <p>
                    References the value signaling the outcome of the semantic action.
                    This is pre-initialized to <code class="computeroutput"><span class="identifier">lex</span><span class="special">::</span><span class="identifier">pass_flags</span><span class="special">::</span><span class="identifier">pass_normal</span></code>.
                    If this is set to <code class="computeroutput"><span class="identifier">lex</span><span class="special">::</span><span class="identifier">pass_flags</span><span class="special">::</span><span class="identifier">pass_fail</span></code>,
                    the lexer will behave as if no token has been matched, if is
                    set to <code class="computeroutput"><span class="identifier">lex</span><span class="special">::</span><span class="identifier">pass_flags</span><span class="special">::</span><span class="identifier">pass_ignore</span></code>, the lexer will
                    ignore the current match and proceed trying to match tokens from
                    the input.
                  </p>
                </td>
</tr>
<tr>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">_tokenid</span></code>
                  </p>
                </td>
<td>
                  <p>
                    Refers to the token id of the token to be generated. Any modifications
                    to this value will be reflected in the generated token.
                  </p>
                </td>
</tr>
<tr>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">_val</span></code>
                  </p>
                </td>
<td>
                  <p>
                    Refers to the value the next token will be initialized from.
                    Any modifications to this value will be reflected in the generated
                    token.
                  </p>
                </td>
</tr>
<tr>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">_state</span></code>
                  </p>
                </td>
<td>
                  <p>
                    Refers to the lexer state the input has been match in. Any modifications
                    to this value will be reflected in the lexer itself (the next
                    match will start in the new state). The currently generated token
                    is not affected by changes to this variable.
                  </p>
                </td>
</tr>
<tr>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">_eoi</span></code>
                  </p>
                </td>
<td>
                  <p>
                    References the end iterator of the overall lexer input. This
                    value cannot be changed.
                  </p>
                </td>
</tr>
</tbody>
</table></div>
</div>
<br class="table-break"><p>
          The context object passed as the last parameter to any lexer semantic action
          is not directly accessible while using <a href="../../../../../../../libs/phoenix/doc/html/index.html" target="_top">Boost.Phoenix</a>
          expressions. We rather provide predefined Phoenix functions allowing to
          invoke the different support functions as mentioned above. The following
          table lists the available support functions and describes their functionality:
        </p>
<div class="table">
<a name="spirit.lex.abstracts.lexer_semantic_actions.support_functions_usable_from_phoenix_expressions_inside_lexer_semantic_actions"></a><p class="title"><b>Table&#160;10.&#160;Support functions usable from Phoenix expressions inside lexer semantic
          actions</b></p>
<div class="table-contents"><table class="table" summary="Support functions usable from Phoenix expressions inside lexer semantic
          actions">
<colgroup>
<col>
<col>
<col>
</colgroup>
<thead><tr>
<th>
                  <p>
                    Plain function
                  </p>
                </th>
<th>
                  <p>
                    Phoenix function
                  </p>
                </th>
<th>
                  <p>
                    Description
                  </p>
                </th>
</tr></thead>
<tbody>
<tr>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">ctx</span><span class="special">.</span><span class="identifier">more</span><span class="special">()</span></code>
                  </p>
                </td>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">more</span><span class="special">()</span></code>
                  </p>
                </td>
<td>
                  <p>
                    The function <code class="computeroutput"><span class="identifier">more</span><span class="special">()</span></code> tells the lexer that the next
                    time it matches a rule, the corresponding token should be appended
                    onto the current token value rather than replacing it.
                  </p>
                </td>
</tr>
<tr>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">ctx</span><span class="special">.</span><span class="identifier">less</span><span class="special">()</span></code>
                  </p>
                </td>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">less</span><span class="special">(</span><span class="identifier">n</span><span class="special">)</span></code>
                  </p>
                </td>
<td>
                  <p>
                    The function <code class="computeroutput"><span class="identifier">less</span><span class="special">()</span></code> takes a single integer parameter
                    <code class="computeroutput"><span class="identifier">n</span></code> and returns
                    an iterator positioned to the nth input character beyond the
                    current token start iterator (i.e. by assigning the return value
                    to the placeholder <code class="computeroutput"><span class="identifier">_end</span></code>
                    it is possible to return all but the first <code class="computeroutput"><span class="identifier">n</span></code>
                    characters of the current token back to the input stream.
                  </p>
                </td>
</tr>
<tr>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">ctx</span><span class="special">.</span><span class="identifier">lookahead</span><span class="special">()</span></code>
                  </p>
                </td>
<td>
                  <p>
                    <code class="computeroutput"><span class="identifier">lookahead</span><span class="special">(</span><span class="identifier">std</span><span class="special">::</span><span class="identifier">size_t</span><span class="special">)</span></code>
                    or <code class="computeroutput"><span class="identifier">lookahead</span><span class="special">(</span><span class="identifier">token_def</span><span class="special">)</span></code>
                  </p>
                </td>
<td>
                  <p>
                    The function <code class="computeroutput"><span class="identifier">lookahead</span><span class="special">()</span></code> takes a single parameter specifying
                    the token to match in the input. The function can be used for
                    instance to implement lookahead for lexer engines not supporting
                    constructs like flex' <code class="computeroutput"><span class="identifier">a</span><span class="special">/</span><span class="identifier">b</span></code>
                    (match <code class="computeroutput"><span class="identifier">a</span></code>, but
                    only when followed by <code class="computeroutput"><span class="identifier">b</span></code>).
                    It invokes the lexer on the input following the current token
                    without actually moving forward in the input stream. The function
                    returns whether the lexer was able to match the specified token.
                  </p>
                </td>
</tr>
</tbody>
</table></div>
</div>
<br class="table-break">
</div>
<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
<td align="left"></td>
<td align="right"><div class="copyright-footer">Copyright &#169; 2001-2011 Joel de Guzman, Hartmut Kaiser<p>
        Distributed under the Boost Software License, Version 1.0. (See accompanying
        file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
      </p>
</div></td>
</tr></table>
<hr>
<div class="spirit-nav">
<a accesskey="p" href="lexer_tokenizing.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../abstracts.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="lexer_static_model.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a>
</div>
</body>
</html>