diff options
author | Casper <casperneo@uchicago.edu> | 2020-11-24 11:05:39 -0800 |
---|---|---|
committer | GitHub <noreply@github.com> | 2020-11-24 14:05:39 -0500 |
commit | 338393f854eb5ba24761a22cd9316ff5cee4eab0 (patch) | |
tree | cf15ca53761215e34ad53c47fe066aaab16c64e6 | |
parent | c27bc2d76f42613ff4f9d86ca050a30fae83e85e (diff) | |
download | flatbuffers-338393f854eb5ba24761a22cd9316ff5cee4eab0.tar.gz flatbuffers-338393f854eb5ba24761a22cd9316ff5cee4eab0.tar.bz2 flatbuffers-338393f854eb5ba24761a22cd9316ff5cee4eab0.zip |
Documentation updates for Optional Scalars (#6014) (#6270)
* Documentation updates for Optional Scalars
* Updated Support
* Reword stuff
* s/NULL/null
Co-authored-by: Casper Neo <cneo@google.com>
-rw-r--r-- | docs/source/Schemas.md | 102 | ||||
-rw-r--r-- | docs/source/Support.md | 59 | ||||
-rw-r--r-- | docs/source/Tutorial.md | 17 |
3 files changed, 100 insertions, 78 deletions
diff --git a/docs/source/Schemas.md b/docs/source/Schemas.md index bd119b0b..06c0f5c2 100644 --- a/docs/source/Schemas.md +++ b/docs/source/Schemas.md @@ -39,16 +39,16 @@ first: ### Tables -Tables are the main way of defining objects in FlatBuffers, and consist -of a name (here `Monster`) and a list of fields. Each field has a name, -a type, and optionally a default value (if omitted, it defaults to `0` / -`NULL`). +Tables are the main way of defining objects in FlatBuffers, and consist of a +name (here `Monster`) and a list of fields. Each field has a name, a type, and +optionally a default value. If the default value is not specified in the schema, +it will be `0` for scalar types, or `null` for other types. Some languages +support setting a scalar's default to `null`. This makes the scalar optional. -Each field is optional: It does not have to appear in the wire -representation, and you can choose to omit fields for each individual -object. As a result, you have the flexibility to add fields without fear of -bloating your data. This design is also FlatBuffer's mechanism for forward -and backwards compatibility. Note that: +Fields do not have to appear in the wire representation, and you can choose +to omit fields when constructing an object. You have the flexibility to add +fields without fear of bloating your data. This design is also FlatBuffer's +mechanism for forward and backwards compatibility. Note that: - You can add new fields in the schema ONLY at the end of a table definition. Older data will still @@ -135,24 +135,35 @@ Both representations are binary equivalent. Arrays are currently only supported in a `struct`. -### (Default) Values +### Default, Optional and Required Values -Values are a sequence of digits. Values may be optionally followed by a decimal -point (`.`) and more digits, for float constants, or optionally prefixed by -a `-`. Floats may also be in scientific notation; optionally ending with an `e` -or `E`, followed by a `+` or `-` and more digits. +There are three, mutually exclusive, reactions to the non-presence of a table's +field in the binary data: + +1. Default valued fields will return the default value (as defined in the schema). +2. Optional valued fields will return some form of `null` depending on the + local language. (In a sense, `null` is the default value). +3. Required fields will cause an error. Flatbuffer verifiers would + consider the whole buffer invalid. See the `required` tag below. + +When writing a schema, values are a sequence of digits. Values may be optionally +followed by a decimal point (`.`) and more digits, for float constants, or +optionally prefixed by a `-`. Floats may also be in scientific notation; +optionally ending with an `e` or `E`, followed by a `+` or `-` and more digits. +Values can also be the keyword `null`. Only scalar values can have defaults, non-scalar (string/vector/table) fields -default to `NULL` when not present. +default to `null` when not present. You generally do not want to change default values after they're initially defined. Fields that have the default value are not actually stored in the -serialized data (see also Gotchas below) but are generated in code, -so when you change the default, you'd -now get a different value than from code generated from an older version of -the schema. There are situations, however, where this may be -desirable, especially if you can ensure a simultaneous rebuild of -all code. +serialized data (see also Gotchas below). Values explicitly written by code +generated by the old schema old version, if they happen to be the default, will +be read as a different value by code generated with the new schema. This is +slightly less bad when converting an optional scalar into a default valued +scalar since non-presence would not be overloaded with a previous default value. +There are situations, however, where this may be desirable, especially if you +can ensure a simultaneous rebuild of all code. ### Enums @@ -325,18 +336,16 @@ Current understood attributes: deprecate a field that was previous required, old code may fail to validate new data (when using the optional verifier). - `required` (on a non-scalar table field): this field must always be set. - By default, all fields are optional, i.e. may be left out. This is + By default, fields do not need to be present in the binary. This is desirable, as it helps with forwards/backwards compatibility, and - flexibility of data structures. It is also a burden on the reading code, - since for non-scalar fields it requires you to check against NULL and - take appropriate action. By specifying this field, you force code that - constructs FlatBuffers to ensure this field is initialized, so the reading - code may access it directly, without checking for NULL. If the constructing - code does not initialize this field, they will get an assert, and also - the verifier will fail on buffers that have missing required fields. Note - that if you add this attribute to an existing field, this will only be - valid if existing data always contains this field / existing code always - writes this field. + flexibility of data structures. By specifying this attribute, you make non- + presence in an error for both reader and writer. The reading code may access + the field directly, without checking for null. If the constructing code does + not initialize this field, they will get an assert, and also the verifier + will fail on buffers that have missing required fields. Both adding and + removing this attribute may be forwards/backwards incompatible as readers + will be unable read old or new data, respectively, unless the data happens to + always have the field set. - `force_align: size` (on a struct): force the alignment of this struct to be something higher than what it is naturally aligned to. Causes these structs to be aligned to that amount inside a buffer, IF that @@ -486,7 +495,7 @@ of related data structures is a union. Unions do have a cost however, so an alternative to a union is to have a single table that has all the fields of all the data structures you are trying to represent, if they are relatively similar / share many fields. -Again, this is efficient because optional fields are cheap. +Again, this is efficient because non-present fields are cheap. FlatBuffers supports the full range of integer sizes, so try to pick the smallest size needed, rather than defaulting to int/long. @@ -611,22 +620,25 @@ Most serialization formats (e.g. JSON or Protocol Buffers) make it very explicit in the format whether a field is present in an object or not, allowing you to use this as "extra" information. -In FlatBuffers, this also holds for everything except scalar values. - -FlatBuffers by default will not write fields that are equal to the default -value (for scalars), sometimes resulting in a significant space savings. +FlatBuffers will not write fields that are equal to their default value, +sometimes resulting in significant space savings. However, this also means we +cannot disambiguate the meaning of non-presence as "written default value" or +"not written at all". This only applies to scalar fields since only they support +default values. Unless otherwise specified, their default is 0. -However, this also means testing whether a field is "present" is somewhat -meaningless, since it does not tell you if the field was actually written by -calling `add_field` style calls, unless you're only interested in this -information for non-default values. +If you care about the presence of scalars, most languages support "optional +scalars." You can set `null` as the default value in the schema. `null` is a +value that's outside of all types, so we will always write if `add_field` is +called. The generated field accessor should use the local language's canonical +optional type. Some `FlatBufferBuilder` implementations have an option called `force_defaults` -that circumvents this behavior, and writes fields even if they are equal to -the default. You can then use `IsFieldPresent` to query this. +that circumvents this "not writing defaults" behavior you can then use +`IsFieldPresent` to query presence. Another option that works in all languages is to wrap a scalar field in a -struct. This way it will return null if it is not present. The cool thing -is that structs don't take up any more space than the scalar they represent. +struct. This way it will return null if it is not present. This will be slightly +less ergonomic but structs don't take up any more space than the scalar they +represent. [Interface Definition Language]: https://en.wikipedia.org/wiki/Interface_description_language diff --git a/docs/source/Support.md b/docs/source/Support.md index ab3b4bdf..4cac209b 100644 --- a/docs/source/Support.md +++ b/docs/source/Support.md @@ -18,31 +18,40 @@ In general: NOTE: this table is a start, it needs to be extended. -Feature | C++ | Java | C# | Go | Python | JS | TS | C | PHP | Dart | Lobster | Rust | Swift ------------------------------- | ------ | ------ | ------ | ------ | ------ | --------- | --------- | ------ | --- | ------- | ------- | ------- | ------ -Codegen for all basic features | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | WiP | Yes | Yes | Yes | Yes -JSON parsing | Yes | No | No | No | No | No | No | Yes | No | No | Yes | No | No -Simple mutation | Yes | Yes | Yes | Yes | No | No | No | No | No | No | No | No | Yes -Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No | No | No -Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No | No | No -Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | Yes | Yes | Yes | Yes -Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | No | No | Yes | No -Performance: | Superb | Great | Great | Great | Ok | ? | ? | Superb | ? | ? | Great | Superb | Great -Platform: Windows | VS2010 | Yes | Yes | ? | ? | ? | Yes | VS2010 | ? | Yes | Yes | Yes | No -Platform: Linux | GCC282 | Yes | ? | Yes | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes -Platform: OS X | Xcode4 | ? | ? | ? | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes -Platform: Android | NDK10d | Yes | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | No -Platform: iOS | ? | ? | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | Yes -Engine: Unity | ? | ? | Yes | ? | ? | ? | ? | ? | ? | ? | No | ? | No -Primary authors (github) | aard* | aard* | ev*/js*| rw | rw | evanw/ev* | kr* | mik* | ch* | dnfield | aard* | rw | mi*/mz* +Feature | C++ | Java | C# | Go | Python | JS | TS | C | PHP | Dart | Lobster | Rust | Swift +------------------------------ | ------ | ----- | -------- | ----- | ------ | ----- | --- | ------ | --- | ------- | ------- | ------ | ------ +Codegen for all basic features | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | WiP | Yes | Yes | Yes | Yes +JSON parsing | Yes | No | No | No | No | No | No | Yes | No | No | Yes | No | No +Simple mutation | Yes | Yes | Yes | Yes | No | No | No | No | No | No | No | No | Yes +Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No | No | No +Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No | No | No +Native Object API | Yes | No | Yes | Yes | Yes | Yes | Yes | No | No | No | No | No | No +Optional Scalars | Yes | Yes | Yes | No | No | Yes | Yes | Yes | No | No | Yes | Yes | Yes +Flexbuffers | Yes | Yes | ? | ? | ? | ? | ? | ? | ? | ? | ? | Yes | ? +Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | Yes | Yes | Yes | Yes +Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | No | No | Yes | No +Performance: | Superb | Great | Great | Great | Ok | ? | ? | Superb | ? | ? | Great | Superb | Great +Platform: Windows | VS2010 | Yes | Yes | ? | ? | ? | Yes | VS2010 | ? | Yes | Yes | Yes | No +Platform: Linux | GCC282 | Yes | ? | Yes | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes +Platform: OS X | Xcode4 | ? | ? | ? | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes +Platform: Android | NDK10d | Yes | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | No +Platform: iOS | ? | ? | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | Yes +Engine: Unity | ? | ? | Yes | ? | ? | ? | ? | ? | ? | ? | No | ? | No +Primary authors (github) | aard | aard | ev/js/df | rw | rw | ew/ev | kr | mik | ch | df | aard | rw/cn | mi/mz - * aard = aardappel (previously: gwvo) - * ev = evolutional - * js = jonsimantov - * mik = mikkelfj - * ch = chobie - * kr = krojew - * mi = mustiikhalil - * mz = mzaks +Above | Github username +----- | ----------------------------- +aard | aardappel (previously: gwvo) +ch | chobie +cn | caspern +df | dnfield +ev | evolutional +ew | evanw +js | jonsimantov +kr | krojew +mi | mustiikhalil +mik | mikkelfj +mz | mzaks +rw | rw <br> diff --git a/docs/source/Tutorial.md b/docs/source/Tutorial.md index 72bd494d..419ea09d 100644 --- a/docs/source/Tutorial.md +++ b/docs/source/Tutorial.md @@ -227,13 +227,14 @@ less memory and have faster lookup. The `Monster` table is the main object in our FlatBuffer. This will be used as the template to store our `orc` monster. We specify some default values for -fields, such as `mana:short = 150`. All unspecified fields will default to `0` -or `NULL`. Another thing to note is the line -`friendly:bool = false (deprecated);`. Since you cannot delete fields from a -`table` (to support backwards compatability), you can set fields as -`deprecated`, which will prevent the generation of accessors for this field in -the generated code. Be careful when using `deprecated`, however, as it may break -legacy code that used this accessor. +fields, such as `mana:short = 150`. If unspecified, scalar fields (like `int`, +`uint`, or `float`) will be given a default of `0` while strings and tables will +be given a default of `null`. Another thing to note is the line `friendly:bool = +false (deprecated);`. Since you cannot delete fields from a `table` (to support +backwards compatability), you can set fields as `deprecated`, which will prevent +the generation of accessors for this field in the generated code. Be careful +when using `deprecated`, however, as it may break legacy code that used this +accessor. The `Weapon` table is a sub-table used within our FlatBuffer. It is used twice: once within the `Monster` table and once within the `Equipment` @@ -2702,7 +2703,7 @@ To access sub-objects, in the case of our `pos`, which is a `Vec3`: `x`, `y`, and `z` will contain `1.0`, `2.0`, and `3.0`, respectively. -*Note: Had we not set `pos` during serialization, it would be a `NULL`-value.* +*Note: Had we not set `pos` during serialization, it would be a `null`-value.* Similarly, we can access elements of the inventory `vector` by indexing it. You can also iterate over the length of the array/vector representing the |