1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
|
Rules for the Berkeley DB and Berkeley DB-XML test suites
1. Test Naming
The primary script for running Berkeley DB scripts is named
'test.tcl'. The primary script for running DB-XML is named
'xmltest.tcl'.
Tests are named with a (prefix, test number) combination. The
prefix indicates the type of test (lock, log, xml, etc.). The
prefix 'test' is used for plain vanilla DB testing. Test numbers
are 3 digits long, starting with 001.
Procedures common to a group of tests, or to all tests, are placed
in files named 'xxxutils.tcl'. At the moment, we have the following
utilities files:
testutils.tcl Utilities common to all DB tests
reputils.tcl Utilities for replication testing.
siutils.tcl Utilities for secondary index testing.
xmlutils.tcl Utilities for XML testing.
2. Internal test structure
Each line within a test should be no more than 80 characters long.
Each test starts with a section like the following:
# See the file LICENSE for redistribution information.
#
# Copyright (c) 1996-2009 Oracle. All rights reserved.
#
# $Id$
#
# TEST test001
# TEST Small keys/data
# TEST Put/get per key
# TEST Dump file
# TEST Close, reopen
# TEST Dump file
# TEST
# TEST Use the first 10,000 entries from the dictionary.
# TEST Insert each with self as key and data; retrieve each.
# TEST After all are entered, retrieve all; compare output to original.
# TEST Close file, reopen, do retrieve and re-verify.
First we refer to the license and assert copyright, then comes the CVS
header string. The section of lines beginning # TEST is used to
automatically maintain the TESTS file, a listing of all tests and
what they do. Use this section to briefly describe the test's purpose
and structure.
Next comes the main procedure of the test, which has the same name
as the tcl file. The test should be liberally commented, and also
should use 'puts' to send messages to the output file.
Sections of a test are identified with letters: test001.a, test001.b,
test001.c.
Here's some typical output:
puts "Test$tnum: $method ($args) $nentries equal key/data pairs"
puts "\tTest$tnum.a: put/get loop"
puts "\tTest$tnum.b: dump file"
puts "\tTest$tnum.c: close, open, and dump file"
puts "\tTest$tnum.d: close, open, and dump file in reverse direction"
The reporting of the current value of the args is particularly
useful, allowing us to say at a glance that "testxxx is failing in
btree" or whatever. Each line of output must begin with the test name.
We use this to separate expected informational output from errors.
Ancillary procedures follow the main procedure. Procedures used
by more than one test should go into the appropriate XXXutils.tcl
file.
3. Reporting failures
Failures in tests are reported with a message starting with the
prefix "FAIL:". Failures in tests are usually caught with the
error_check_good and error_check_bad routines to compare an
actual return value to an expected return value. These routines
take care of putting the "FAIL:" prefix on the message.
4. Running tests
Any single test can be run from the tclsh prompt by typing the
name of the test. If it's a test from the 'testxxx' group, you
should also specify the method you'd like to test:
log001
test001 btree
To run one of the 'testxxx' tests for all methods, use the
run_test procedure:
run_test test001
Any group of tests (the subsystems lock, log, test, etc.) can be
run by typing
r $sub
where sub is the name of the subsystem.
For any of the following methods
run_method
run_secmethod
run_secenv
run_reptest
run_repmethod
run_envmethod
run_recd
you can type
run (suffix method start stop).
For example, to run test010 through test020 in btree using
run_method:
run method btree 10 20
Or the same tests in repmethod:
run repmethod btree 10 20
Notice the missing underbar.
If you omit the start and stop numbers, you'll get all the tests:
run method btree
run_recd is a special case, in that it runs the recdxxx tests;
all the others run the testxxx tests.
To run the standard test suite, type run_std at the tclsh prompt.
To run all the tests, type run_all.
If you are running run_std or run_all, you may use the run_parallel
interface to speed things up or to test under conditions of high
system load. Run_parallel creates a list of all tests in the run,
reorders the tests randomly, then runs the tests in a number of
parallel processes. To run run_std in five processes type
run_parallel 5 run_std
|
