diff options
Diffstat (limited to 'nnpackage/spec/20_model_and_operators.md')
-rw-r--r-- | nnpackage/spec/20_model_and_operators.md | 90 |
1 files changed, 90 insertions, 0 deletions
diff --git a/nnpackage/spec/20_model_and_operators.md b/nnpackage/spec/20_model_and_operators.md new file mode 100644 index 000000000..fa4131645 --- /dev/null +++ b/nnpackage/spec/20_model_and_operators.md @@ -0,0 +1,90 @@ +# Model + +## Serialization Format + +`nnpackage` uses flatbuffers to store model. + +Rationale: + +1. `flatbuffers` is: + +- space-efficient +- explicit-schema based +- royalty-free license open-source library +- header-only solution (unless we use flatbuffer's reflection) +- proven solution (used by TensorFlow-Lite) + +2. We've checked other solutions: +- [`bjson (binary JSON)`](http://bjson.org/) +- `protocol buffers` + +## Baseline Schema + +`nnpackage` schema is based on tensorflow-lite schema. + +Rationale: + +- Fundamentally, `nnpackage` and `TFLite` have same aim: +Running pre-trained models on a device, which has relatively low computing power and memory. +TFLite's solution is acceptable, we don't need to create same thing again. +- We can use several infra-structures and tools from TFLite. + +## Extensions + +`nnpackage` model has some extensions that are different or missing from TFLite. + +### Multiple Layout + +`nnpackage` can support multiple layouts. + +1. The layout is presented using `DataFormat` enumeration. + +`DataFormat` must be one of the enumeration defined in `nnpackage_schema.fbs`. + +For example, `CHANNELS_FIRST` or `CHANNELS_LAST` can be used. + +``` + // For 2D data, NHWC(batch, height, width, channels) + // For 3D data, NDHWC(batch, depth, height, width, channels) + CHANNELS_LAST = 0, + // For 2D data, NCHW(batch, channels, height, width) + // For 3D data, NCDHW(batch, channels, depth, height, width) + CHANNELS_FIRST = 1, +``` + +2. `DataFormat` must be same within a submodel. + +Rationale: + +- frequent switching between different layout degrades the performance + +Under this assumption, We expect to + +- simplify the runtime implementation +- accelerate the performance +- reduce the memory usage + +### Unspecified Dimension + +`nnpackage` represents unspecified dimension with `-1`. + +Rationale: + +1. It should be `int` since dimension is int type flatbuffer schema. Thus '?' cannot be used. +2. `0` is also a candidate, which is used for Android NN API. +However, we would like to reserve `0` because `0` could be a valid dimension for a certain +operator (e.g. `tflite.slice`). + +## Operator Reference + +All operators use same semantics of tensorflow lite operators. +Refer tensorflow lite source code to understand what inputs, outputs and attributes +are required and how they are interpretered. + +## Schema Source + +nnpackage supports two kinds of models: `tflite` and `circle` + +- For tflite, see `schema.fbs` from tensorflow lite source. + +- For circle, see [`../schema/circle_schema.fbs`](../schema/circle_schema.fbs). |