1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
|
How to build
============
Dependencies
------------
To build re2c from a release tarball one needs only a C++ compiler and Python 3
(for build-time scripts and for testing). Building on Windows additionally
requires either CMake or a Mingw build environment.
To develop re2c one also needs CMake or Autotools (both build systems are
maintained). Other dependencies include Bison (to rebuild parsers), Python 3
Docutils (to rebuild documentation) and Sphinx (to rebuild the website).
Benchmarks also use Google-benchmark, re2 and Java. A few helper scripts are
written in Haskell, but they are not essential for re2c development.
re2c is a self-hosting lexer generator, meaning that parts of its source code
are written in re2c (namely, all the source files that have a *.re* extension).
By default re2c builds from pre-generated bootstrap files. To build from source
*.re* files, it is necessary to reconfigure the build and provide a path to an
existing re2c binary that will be used for bootstrap (see the details below).
Build (Autotools)
-----------------
If you are building from a release tarball, the configure script is already
there. Otherwise it is necessary to generate it:
```
$ autoreconf -i -W all
```
An in-tree build (objects are generated side by side with sources):
```
$ ./configure && make && make install
```
An out-of-tree build (objects are generated in a separate directory):
```
$ mkdir .build && cd .build && ../configure && make && make install
```
The configure script has many options (to see them all, run
`configure --help`). The options specific to re2c are the following:
* `--enable-debug`
Enable debug checks in the re2c source code. Also enables debug options.
* `--enable-golang`
Build re2go (identical to `re2c --lang go`). This is on by default.
* `--enable-rust`
Build re2rust (identical to `re2c --lang rust`). This is on by default.
* `--enable-lexers`
Enable regeneration of lexers from *.re* files (as opposed to using
pre-generated bootstrap files). This requires setting `RE2C_FOR_BUILD` to
an existing re2c executable path.
* `--enable-parsers`
Enable regeneration of parsers from *.ypp* files with bison (as opposed to
using pre-generated bootstrap files).
* `--enable-docs`
Enable regeneration of documentation (requires Python 3 Docutils).
* `--enable-benchmarks`
Build benchmarks (requires Google benchmarks library).
* `--enable-benchmarks-regenerate`
Regenerate C code for Ragel and Kleenex benchmarks (this will download and
build Ragel and Kleenex). re2c benchmarks are always regenerated.
* `--enable-libs`
Build the experimental libre2c library that provides POSIX
`regcomp`/`regexec`/`regfree` interface to re2c.
Cross-compile re2c for Windows (for some Mingw versions you might have to use
`-std=gnu++11` compiler option):
```
$ ./configure --host i686-w64-ming32 \
LDFLAGS="-static -static-libstdc++ -static-libgcc"
```
There is a bunch of build scripts for specialized builds with Asan, Ubsan,
GLIBCXX_DEBUG, etc. in the ``build`` subdirectory.
Build (CMake)
-------------
re2c includes CMake presets for Linux, MacOS, and Windows. Use `cmake
--list-presets` to see them.
* The `-fast` presets have minimal dependencies and build re2c using
pre-generated manpages, lexers, and parsers.
* The `-full` presets have additional prerequisites (Docutils, Bison, and a
pre-existing re2c binary) and re-generate the manpages, lexers, and parsers.
If you just want to build re2c, use the `-release-ootree-fast` preset. The
other presets are primarily of interest to developers.
For example, to build the `windows-msvc-release-ootree-fast` preset:
* CD to the root of the re2c folder (the folder containing this README).
* Configure: `cmake --preset=windows-msvc-release-ootree-fast`
* Build: `cmake --build --preset=windows-msvc-release-ootree-fast`
* The binary will be in the `.build/<preset-name>` folder.
CMake supports different build modes, which can be further customized with
individual options. By default re2c builds with `-O2 -g` and does not set any
particular mode to allow the user complete freedom in overriding the build
options. To enable a build mode at configure phase, use
`-DCMAKE_BUILD_TYPE=<mode>` option (this works with single-config generators,
such as Makefiles or Ninja). At build phase use `--config <mode>` option.
The following modes are supported:
* `Release`
This is an optimized `-O3 -DNDEBUG` build without debug symbols, asserts
and debug checks.
* `Debug`
This is a non-optimized `-g -DRE2C_DEBUG` build with debug symbols, asserts,
and debug checks.
* `RelWithDebInfo`
This is an optimized `-O2 -g -DNDEBUG` build with debug symbols, but
without asserts and debug checks.
* `MinSizeRel`
This is an optimized for size `-Os -DNDEBUG` build without debug symbols,
asserts and debug checks.
* `Asan`
This is a non-optimized `-g -fsanitize=address` build that instruments the
code with Address Sanitizer to check for memory errors.
* `Ubsan`
This is a non-optimized `-g -fsanitize=undefined` build that instruments
the code with Undefined Behavior Sanitizer to check for undefined behavior
errors.
A simple build (objects are generated in a separate build directory):
```
$ mkdir .build && cd .build && cmake .. && cmake --build .
```
CMake supports a lot of configuration options. Here is the list of options
specific to re2c:
* `-DRE2C_BUILD_RE2GO=yes`
Build re2go executable (an alias to `re2c --lang go`). Enabled by default.
* `-DRE2C_BUILD_RE2RUST=yes`
Build re2rust executable (an alias to `re2c --lang rust`). Enabled by
default.
* `-DRE2C_REBUILD_LEXERS=yes`
Enable regeneration of lexers from *.re* files (as opposed to using
pre-generated bootstrap files). This requires setting `-DRE2C_FOR_BUILD` to
an existing re2c executable path.
* `-DRE2C_REBUILD_PARSERS=yes`
Enable regeneration of parsers from *.ypp* files with bison (as opposed to
using pre-generated bootstrap files).
* `-DRE2C_REBUILD_DOCS=yes`
Enable regeneration of documentation (requires Python 3 Docutils).
* `-DRE2C_BUILD_BENCHMARKS=yes`
Build benchmarks (requires Google benchmarks library).
* `-DRE2C_REGEN_BENCHMARKS=yes`
Regenerate C code for Ragel and Kleenex benchmarks (this will download and
build Ragel and Kleenex). re2c benchmarks are always regenerated.
* `-DRE2C_BUILD_LIBS=yes`
Build the experimental libre2c library that provides POSIX
`regcomp`/`regexec`/`regfree` interface to re2c.
Cross-compile re2c for Windows using Mingw:
```
$ cmake . -DCMAKE_TOOLCHAIN_FILE=cmake/Toolchain-cross-mingw32-linux.cmake \
-DCMAKE_EXE_LINKER_FLAGS="-static -static-libstdc++ -static-libgcc" \
-DBUILD_SHARED_LIBS=no
```
There is a bunch of build scripts for specialized builds with Asan, Ubsan,
GLIBCXX_DEBUG, etc. in the ``build`` subdirectory.
Test
----
re2c has a main test suite and a number of unit tests. Run them all:
```
$ make check
```
Run only the main test suite and watch the progress dumped on stdout:
```
$ python3 run_tests.py -j<N>
```
Run the main test suite with `--skeleton` re2c option:
```
$ python3 run_tests.py --skeleton
```
Run the test suite under Valgrind (takes some time and memory):
```
$ python3 run_tests.py --valgrind
```
Test Mingw builds with Wine:
```
$ make wtests
```
Check the distribution (works with Autotools):
```
$ make distcheck
```
re2c provides a helper script `build/__alltest.sh` that builds and tests various
re2c build flavours with Asan, Ubsan, GLIBCXX_DEBUG, etc. There is a couple of
fuzz-testing Haskell scripts in the *fuzz* subdirectory; they are based on the
QuickCheck library and can be easily modified to fuzz-test various aspects of
re2c by comparing current re2c version against older versions or against other
regular expression libraries.
|
