summaryrefslogtreecommitdiff
path: root/nnpackage/spec/20_model_and_operators.md
diff options
context:
space:
mode:
Diffstat (limited to 'nnpackage/spec/20_model_and_operators.md')
-rw-r--r--nnpackage/spec/20_model_and_operators.md90
1 files changed, 90 insertions, 0 deletions
diff --git a/nnpackage/spec/20_model_and_operators.md b/nnpackage/spec/20_model_and_operators.md
new file mode 100644
index 000000000..fa4131645
--- /dev/null
+++ b/nnpackage/spec/20_model_and_operators.md
@@ -0,0 +1,90 @@
+# Model
+
+## Serialization Format
+
+`nnpackage` uses flatbuffers to store model.
+
+Rationale:
+
+1. `flatbuffers` is:
+
+- space-efficient
+- explicit-schema based
+- royalty-free license open-source library
+- header-only solution (unless we use flatbuffer's reflection)
+- proven solution (used by TensorFlow-Lite)
+
+2. We've checked other solutions:
+- [`bjson (binary JSON)`](http://bjson.org/)
+- `protocol buffers`
+
+## Baseline Schema
+
+`nnpackage` schema is based on tensorflow-lite schema.
+
+Rationale:
+
+- Fundamentally, `nnpackage` and `TFLite` have same aim:
+Running pre-trained models on a device, which has relatively low computing power and memory.
+TFLite's solution is acceptable, we don't need to create same thing again.
+- We can use several infra-structures and tools from TFLite.
+
+## Extensions
+
+`nnpackage` model has some extensions that are different or missing from TFLite.
+
+### Multiple Layout
+
+`nnpackage` can support multiple layouts.
+
+1. The layout is presented using `DataFormat` enumeration.
+
+`DataFormat` must be one of the enumeration defined in `nnpackage_schema.fbs`.
+
+For example, `CHANNELS_FIRST` or `CHANNELS_LAST` can be used.
+
+```
+ // For 2D data, NHWC(batch, height, width, channels)
+ // For 3D data, NDHWC(batch, depth, height, width, channels)
+ CHANNELS_LAST = 0,
+ // For 2D data, NCHW(batch, channels, height, width)
+ // For 3D data, NCDHW(batch, channels, depth, height, width)
+ CHANNELS_FIRST = 1,
+```
+
+2. `DataFormat` must be same within a submodel.
+
+Rationale:
+
+- frequent switching between different layout degrades the performance
+
+Under this assumption, We expect to
+
+- simplify the runtime implementation
+- accelerate the performance
+- reduce the memory usage
+
+### Unspecified Dimension
+
+`nnpackage` represents unspecified dimension with `-1`.
+
+Rationale:
+
+1. It should be `int` since dimension is int type flatbuffer schema. Thus '?' cannot be used.
+2. `0` is also a candidate, which is used for Android NN API.
+However, we would like to reserve `0` because `0` could be a valid dimension for a certain
+operator (e.g. `tflite.slice`).
+
+## Operator Reference
+
+All operators use same semantics of tensorflow lite operators.
+Refer tensorflow lite source code to understand what inputs, outputs and attributes
+are required and how they are interpretered.
+
+## Schema Source
+
+nnpackage supports two kinds of models: `tflite` and `circle`
+
+- For tflite, see `schema.fbs` from tensorflow lite source.
+
+- For circle, see [`../schema/circle_schema.fbs`](../schema/circle_schema.fbs).