summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorCasper <casperneo@uchicago.edu>2020-11-24 11:05:39 -0800
committerGitHub <noreply@github.com>2020-11-24 14:05:39 -0500
commit338393f854eb5ba24761a22cd9316ff5cee4eab0 (patch)
treecf15ca53761215e34ad53c47fe066aaab16c64e6
parentc27bc2d76f42613ff4f9d86ca050a30fae83e85e (diff)
downloadflatbuffers-338393f854eb5ba24761a22cd9316ff5cee4eab0.tar.gz
flatbuffers-338393f854eb5ba24761a22cd9316ff5cee4eab0.tar.bz2
flatbuffers-338393f854eb5ba24761a22cd9316ff5cee4eab0.zip
Documentation updates for Optional Scalars (#6014) (#6270)
* Documentation updates for Optional Scalars * Updated Support * Reword stuff * s/NULL/null Co-authored-by: Casper Neo <cneo@google.com>
-rw-r--r--docs/source/Schemas.md102
-rw-r--r--docs/source/Support.md59
-rw-r--r--docs/source/Tutorial.md17
3 files changed, 100 insertions, 78 deletions
diff --git a/docs/source/Schemas.md b/docs/source/Schemas.md
index bd119b0b..06c0f5c2 100644
--- a/docs/source/Schemas.md
+++ b/docs/source/Schemas.md
@@ -39,16 +39,16 @@ first:
### Tables
-Tables are the main way of defining objects in FlatBuffers, and consist
-of a name (here `Monster`) and a list of fields. Each field has a name,
-a type, and optionally a default value (if omitted, it defaults to `0` /
-`NULL`).
+Tables are the main way of defining objects in FlatBuffers, and consist of a
+name (here `Monster`) and a list of fields. Each field has a name, a type, and
+optionally a default value. If the default value is not specified in the schema,
+it will be `0` for scalar types, or `null` for other types. Some languages
+support setting a scalar's default to `null`. This makes the scalar optional.
-Each field is optional: It does not have to appear in the wire
-representation, and you can choose to omit fields for each individual
-object. As a result, you have the flexibility to add fields without fear of
-bloating your data. This design is also FlatBuffer's mechanism for forward
-and backwards compatibility. Note that:
+Fields do not have to appear in the wire representation, and you can choose
+to omit fields when constructing an object. You have the flexibility to add
+fields without fear of bloating your data. This design is also FlatBuffer's
+mechanism for forward and backwards compatibility. Note that:
- You can add new fields in the schema ONLY at the end of a table
definition. Older data will still
@@ -135,24 +135,35 @@ Both representations are binary equivalent.
Arrays are currently only supported in a `struct`.
-### (Default) Values
+### Default, Optional and Required Values
-Values are a sequence of digits. Values may be optionally followed by a decimal
-point (`.`) and more digits, for float constants, or optionally prefixed by
-a `-`. Floats may also be in scientific notation; optionally ending with an `e`
-or `E`, followed by a `+` or `-` and more digits.
+There are three, mutually exclusive, reactions to the non-presence of a table's
+field in the binary data:
+
+1. Default valued fields will return the default value (as defined in the schema).
+2. Optional valued fields will return some form of `null` depending on the
+ local language. (In a sense, `null` is the default value).
+3. Required fields will cause an error. Flatbuffer verifiers would
+ consider the whole buffer invalid. See the `required` tag below.
+
+When writing a schema, values are a sequence of digits. Values may be optionally
+followed by a decimal point (`.`) and more digits, for float constants, or
+optionally prefixed by a `-`. Floats may also be in scientific notation;
+optionally ending with an `e` or `E`, followed by a `+` or `-` and more digits.
+Values can also be the keyword `null`.
Only scalar values can have defaults, non-scalar (string/vector/table) fields
-default to `NULL` when not present.
+default to `null` when not present.
You generally do not want to change default values after they're initially
defined. Fields that have the default value are not actually stored in the
-serialized data (see also Gotchas below) but are generated in code,
-so when you change the default, you'd
-now get a different value than from code generated from an older version of
-the schema. There are situations, however, where this may be
-desirable, especially if you can ensure a simultaneous rebuild of
-all code.
+serialized data (see also Gotchas below). Values explicitly written by code
+generated by the old schema old version, if they happen to be the default, will
+be read as a different value by code generated with the new schema. This is
+slightly less bad when converting an optional scalar into a default valued
+scalar since non-presence would not be overloaded with a previous default value.
+There are situations, however, where this may be desirable, especially if you
+can ensure a simultaneous rebuild of all code.
### Enums
@@ -325,18 +336,16 @@ Current understood attributes:
deprecate a field that was previous required, old code may fail to validate
new data (when using the optional verifier).
- `required` (on a non-scalar table field): this field must always be set.
- By default, all fields are optional, i.e. may be left out. This is
+ By default, fields do not need to be present in the binary. This is
desirable, as it helps with forwards/backwards compatibility, and
- flexibility of data structures. It is also a burden on the reading code,
- since for non-scalar fields it requires you to check against NULL and
- take appropriate action. By specifying this field, you force code that
- constructs FlatBuffers to ensure this field is initialized, so the reading
- code may access it directly, without checking for NULL. If the constructing
- code does not initialize this field, they will get an assert, and also
- the verifier will fail on buffers that have missing required fields. Note
- that if you add this attribute to an existing field, this will only be
- valid if existing data always contains this field / existing code always
- writes this field.
+ flexibility of data structures. By specifying this attribute, you make non-
+ presence in an error for both reader and writer. The reading code may access
+ the field directly, without checking for null. If the constructing code does
+ not initialize this field, they will get an assert, and also the verifier
+ will fail on buffers that have missing required fields. Both adding and
+ removing this attribute may be forwards/backwards incompatible as readers
+ will be unable read old or new data, respectively, unless the data happens to
+ always have the field set.
- `force_align: size` (on a struct): force the alignment of this struct
to be something higher than what it is naturally aligned to. Causes
these structs to be aligned to that amount inside a buffer, IF that
@@ -486,7 +495,7 @@ of related data structures is a union. Unions do have a cost however,
so an alternative to a union is to have a single table that has
all the fields of all the data structures you are trying to
represent, if they are relatively similar / share many fields.
-Again, this is efficient because optional fields are cheap.
+Again, this is efficient because non-present fields are cheap.
FlatBuffers supports the full range of integer sizes, so try to pick
the smallest size needed, rather than defaulting to int/long.
@@ -611,22 +620,25 @@ Most serialization formats (e.g. JSON or Protocol Buffers) make it very
explicit in the format whether a field is present in an object or not,
allowing you to use this as "extra" information.
-In FlatBuffers, this also holds for everything except scalar values.
-
-FlatBuffers by default will not write fields that are equal to the default
-value (for scalars), sometimes resulting in a significant space savings.
+FlatBuffers will not write fields that are equal to their default value,
+sometimes resulting in significant space savings. However, this also means we
+cannot disambiguate the meaning of non-presence as "written default value" or
+"not written at all". This only applies to scalar fields since only they support
+default values. Unless otherwise specified, their default is 0.
-However, this also means testing whether a field is "present" is somewhat
-meaningless, since it does not tell you if the field was actually written by
-calling `add_field` style calls, unless you're only interested in this
-information for non-default values.
+If you care about the presence of scalars, most languages support "optional
+scalars." You can set `null` as the default value in the schema. `null` is a
+value that's outside of all types, so we will always write if `add_field` is
+called. The generated field accessor should use the local language's canonical
+optional type.
Some `FlatBufferBuilder` implementations have an option called `force_defaults`
-that circumvents this behavior, and writes fields even if they are equal to
-the default. You can then use `IsFieldPresent` to query this.
+that circumvents this "not writing defaults" behavior you can then use
+`IsFieldPresent` to query presence.
Another option that works in all languages is to wrap a scalar field in a
-struct. This way it will return null if it is not present. The cool thing
-is that structs don't take up any more space than the scalar they represent.
+struct. This way it will return null if it is not present. This will be slightly
+less ergonomic but structs don't take up any more space than the scalar they
+represent.
[Interface Definition Language]: https://en.wikipedia.org/wiki/Interface_description_language
diff --git a/docs/source/Support.md b/docs/source/Support.md
index ab3b4bdf..4cac209b 100644
--- a/docs/source/Support.md
+++ b/docs/source/Support.md
@@ -18,31 +18,40 @@ In general:
NOTE: this table is a start, it needs to be extended.
-Feature | C++ | Java | C# | Go | Python | JS | TS | C | PHP | Dart | Lobster | Rust | Swift
------------------------------- | ------ | ------ | ------ | ------ | ------ | --------- | --------- | ------ | --- | ------- | ------- | ------- | ------
-Codegen for all basic features | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | WiP | Yes | Yes | Yes | Yes
-JSON parsing | Yes | No | No | No | No | No | No | Yes | No | No | Yes | No | No
-Simple mutation | Yes | Yes | Yes | Yes | No | No | No | No | No | No | No | No | Yes
-Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No | No | No
-Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No | No | No
-Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | Yes | Yes | Yes | Yes
-Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | No | No | Yes | No
-Performance: | Superb | Great | Great | Great | Ok | ? | ? | Superb | ? | ? | Great | Superb | Great
-Platform: Windows | VS2010 | Yes | Yes | ? | ? | ? | Yes | VS2010 | ? | Yes | Yes | Yes | No
-Platform: Linux | GCC282 | Yes | ? | Yes | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes
-Platform: OS X | Xcode4 | ? | ? | ? | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes
-Platform: Android | NDK10d | Yes | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | No
-Platform: iOS | ? | ? | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | Yes
-Engine: Unity | ? | ? | Yes | ? | ? | ? | ? | ? | ? | ? | No | ? | No
-Primary authors (github) | aard* | aard* | ev*/js*| rw | rw | evanw/ev* | kr* | mik* | ch* | dnfield | aard* | rw | mi*/mz*
+Feature | C++ | Java | C# | Go | Python | JS | TS | C | PHP | Dart | Lobster | Rust | Swift
+------------------------------ | ------ | ----- | -------- | ----- | ------ | ----- | --- | ------ | --- | ------- | ------- | ------ | ------
+Codegen for all basic features | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | WiP | Yes | Yes | Yes | Yes
+JSON parsing | Yes | No | No | No | No | No | No | Yes | No | No | Yes | No | No
+Simple mutation | Yes | Yes | Yes | Yes | No | No | No | No | No | No | No | No | Yes
+Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No | No | No
+Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No | No | No
+Native Object API | Yes | No | Yes | Yes | Yes | Yes | Yes | No | No | No | No | No | No
+Optional Scalars | Yes | Yes | Yes | No | No | Yes | Yes | Yes | No | No | Yes | Yes | Yes
+Flexbuffers | Yes | Yes | ? | ? | ? | ? | ? | ? | ? | ? | ? | Yes | ?
+Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | Yes | Yes | Yes | Yes
+Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | No | No | Yes | No
+Performance: | Superb | Great | Great | Great | Ok | ? | ? | Superb | ? | ? | Great | Superb | Great
+Platform: Windows | VS2010 | Yes | Yes | ? | ? | ? | Yes | VS2010 | ? | Yes | Yes | Yes | No
+Platform: Linux | GCC282 | Yes | ? | Yes | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes
+Platform: OS X | Xcode4 | ? | ? | ? | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes
+Platform: Android | NDK10d | Yes | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | No
+Platform: iOS | ? | ? | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | Yes
+Engine: Unity | ? | ? | Yes | ? | ? | ? | ? | ? | ? | ? | No | ? | No
+Primary authors (github) | aard | aard | ev/js/df | rw | rw | ew/ev | kr | mik | ch | df | aard | rw/cn | mi/mz
- * aard = aardappel (previously: gwvo)
- * ev = evolutional
- * js = jonsimantov
- * mik = mikkelfj
- * ch = chobie
- * kr = krojew
- * mi = mustiikhalil
- * mz = mzaks
+Above | Github username
+----- | -----------------------------
+aard | aardappel (previously: gwvo)
+ch | chobie
+cn | caspern
+df | dnfield
+ev | evolutional
+ew | evanw
+js | jonsimantov
+kr | krojew
+mi | mustiikhalil
+mik | mikkelfj
+mz | mzaks
+rw | rw
<br>
diff --git a/docs/source/Tutorial.md b/docs/source/Tutorial.md
index 72bd494d..419ea09d 100644
--- a/docs/source/Tutorial.md
+++ b/docs/source/Tutorial.md
@@ -227,13 +227,14 @@ less memory and have faster lookup.
The `Monster` table is the main object in our FlatBuffer. This will be used as
the template to store our `orc` monster. We specify some default values for
-fields, such as `mana:short = 150`. All unspecified fields will default to `0`
-or `NULL`. Another thing to note is the line
-`friendly:bool = false (deprecated);`. Since you cannot delete fields from a
-`table` (to support backwards compatability), you can set fields as
-`deprecated`, which will prevent the generation of accessors for this field in
-the generated code. Be careful when using `deprecated`, however, as it may break
-legacy code that used this accessor.
+fields, such as `mana:short = 150`. If unspecified, scalar fields (like `int`,
+`uint`, or `float`) will be given a default of `0` while strings and tables will
+be given a default of `null`. Another thing to note is the line `friendly:bool =
+false (deprecated);`. Since you cannot delete fields from a `table` (to support
+backwards compatability), you can set fields as `deprecated`, which will prevent
+the generation of accessors for this field in the generated code. Be careful
+when using `deprecated`, however, as it may break legacy code that used this
+accessor.
The `Weapon` table is a sub-table used within our FlatBuffer. It is
used twice: once within the `Monster` table and once within the `Equipment`
@@ -2702,7 +2703,7 @@ To access sub-objects, in the case of our `pos`, which is a `Vec3`:
`x`, `y`, and `z` will contain `1.0`, `2.0`, and `3.0`, respectively.
-*Note: Had we not set `pos` during serialization, it would be a `NULL`-value.*
+*Note: Had we not set `pos` during serialization, it would be a `null`-value.*
Similarly, we can access elements of the inventory `vector` by indexing it. You
can also iterate over the length of the array/vector representing the