1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
|
How to build
============
Dependencies
------------
To build re2c from a release tarball one needs only a C++ compiler and
optionally Python 3 to run the tests.
To develop re2c one also needs CMake or Autotools (both build systems are
maintained). Other dependencies include Bison (to rebuild parsers), rst2man (to
rebuild documentation) and Sphinx (to rebuild the website). Benchmarks also use
google-benchmark, re2 and Java. A few helper scripts are written in Haskell, but
they are not essential for re2c development.
re2c is a self-hosting lexer generator, meaning that parts of its source code
are written in re2c (namely, all the source files that have a *.re* extension).
By default re2c builds from pre-generated bootstrap files. To build from source
*.re* files, it is necessary to reconfigure the build and provide a path to an
existing re2c binary that will be used for bootstrap (see the details below).
Build (Autotools)
-----------------
If you are building from a release tarball, the configure script is already
there. Otherwise it is necessary to generate it:
```
$ autoreconf -i -W all
```
An in-tree build (objects are generated side by side with sources):
```
$ ./configure && make && make install
```
An out-of-tree build (objects are generated in a separate directory):
```
$ mkdir .build && cd .build && ../configure && make && make install
```
The configure script has many options (to see them all, run
`configure --help`). The options specific to re2c are the following:
* `--enable-debug`
Enable debug checks in the re2c source code. Also enables debug options.
* `--enable-golang`
Build re2go (identical to `re2c --lang go`). This is on by default.
* `--enable-rust`
Build re2rust (identical to `re2c --lang rust`). This is on by default.
* `--enable-lexers`
Enable regeneration of *.re* files (as opposed to using bootstrap files).
This requires setting `RE2C_FOR_BUILD` to an existing re2c executable
path.
* `--enable-docs`
Enable regeneration of documentation (requires rst2man).
* `--enable-benchmarks`
Build benchmarks (requires google benchmarks library).
* `--enable-benchmarks-regenerate`
Regenerate C code for Ragel and Kleenex benchmarks (this will download and
build Ragel and Kleenex). re2c benchmarks are always regenerated.
* `--enable-libs`
Build the experimental libre2c library that provides POSIX
`regcomp`/`regexec`/`regfree` interface to re2c.
Cross-compile re2c for Windows (for some Mingw versions you might have to use
`-std=gnu++11` compiler option):
```
$ ./configure --host i686-w64-ming32 \
LDFLAGS="-static -static-libstdc++ -static-libgcc"
```
There is a bunch of build scripts for specialized builds with Asan, Ubsan,
GLIBCXX_DEBUG, etc. in the ``build`` subdirectory.
Build (CMake)
-------------
CMake supports different build modes, which can be further customized with
individual options. By default re2c builds with `-O2 -g` and does not set any
particular mode to allow the user complete freedom in overriding the build
options. To enable a build mode at configure phase, use
`-DCMAKE_BUILD_TYPE=<mode>` option (this works with single-config generators,
such as Makefiles or Ninja). At build phase use `--config <mode>` option.
The following modes are supported:
* `Release`
This is an optimized `-O3 -DNDEBUG` build without debug symbols, asserts
and debug checks.
* `Debug`
This is a non-optimized `-g -DRE2C_DEBUG` build with debug symbols, asserts,
and debug checks.
* `RelWithDebInfo`
This is an optimized `-O2 -g -DNDEBUG` build with debug symbols, but
without asserts and debug checks.
* `MinSizeRel`
This is an optimized for size `-Os -DNDEBUG` build without debug symbols,
asserts and debug checks.
* `Asan`
This is a non-optimized `-g -fsanitize=address` build that instruments the
code with Address Sanitizer to check for memory errors.
* `Ubsan`
This is a non-optimized `-g -fsanitize=undefined` build that instruments
the code with Undefined Behavior Sanitizer to check for undefined behavior
errors.
A simple build (objects are generated in a separate build directory):
```
$ mkdir .build && cd .build && cmake .. && cmake --build .
```
CMake supports a lot of configuration options. Here is the list of options
specific to re2c:
* `-DRE2C_BUILD_RE2GO=yes`
Build re2go executable (an alias to `re2c --lang go`). Enabled by default.
* `-DRE2C_BUILD_RE2RUST=yes`
Build re2rust executable (an alias to `re2c --lang rust`). Enabled by
default.
* `-DRE2C_REBUILD_LEXERS=yes`
Enable regeneration of *.re* files (as opposed to using bootstrap files).
This requires setting `-DRE2C_FOR_BUILD` to an existing re2c executable
path.
* `-DRE2C_REBUILD_DOCS=yes`
Enable regeneration of documentation (requires rst2man).
* `-DRE2C_BUILD_BENCHMARKS=yes`
Build benchmarks (requires google benchmarks library).
* `-DRE2C_REGEN_BENCHMARKS=yes`
Regenerate C code for Ragel and Kleenex benchmarks (this will download and
build Ragel and Kleenex). re2c benchmarks are always regenerated.
* `-DRE2C_BUILD_LIBS=yes`
Build the experimental libre2c library that provides POSIX
`regcomp`/`regexec`/`regfree` interface to re2c.
Cross-compile re2c for Windows using Mingw:
```
$ cmake . -DCMAKE_TOOLCHAIN_FILE=cmake/Toolchain-cross-mingw32-linux.cmake \
-DCMAKE_EXE_LINKER_FLAGS="-static -static-libstdc++ -static-libgcc" \
-DBUILD_SHARED_LIBS=no
```
There is a bunch of build scripts for specialized builds with Asan, Ubsan,
GLIBCXX_DEBUG, etc. in the ``build`` subdirectory.
Test
----
re2c has a main test suite and a number of unit tests. Run them all:
```
$ make check
```
Run only the main test suite and watch the progress dumped on stdout:
```
$ python3 run_tests.py -j<N>
```
Run the main test suite with `--skeleton` re2c option:
```
$ python3 run_tests.py --skeleton
```
Run the test suite under Valgrind (takes some time and memory):
```
$ python3 run_tests.py --valgrind
```
Test Mingw builds with Wine:
```
$ make wtests
```
Check the distribution (works with Autotools):
```
$ make distcheck
```
re2c provides a helper script `build/__alltest.sh` that builds and tests various
re2c build flavours with Asan, Ubsan, GLIBCXX_DEBUG, etc. There is a couple of
fuzz-testing Haskell scripts in the *fuzz* subdirectory; they are based on the
QuickCheck library and can be easily modified to fuzz-test various aspects of
re2c by comparing current re2c version against older versions or against other
regular expression libraries.
|
