1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
|
# Model
## Serialization Format
`nnpackage` uses flatbuffers to store model.
Rationale:
1. `flatbuffers` is:
- space-efficient
- explicit-schema based
- royalty-free license open-source library
- header-only solution (unless we use flatbuffer's reflection)
- proven solution (used by TensorFlow-Lite)
2. We've checked other solutions:
- [`bjson (binary JSON)`](http://bjson.org/)
- `protocol buffers`
## Baseline Schema
`nnpackage` schema is based on tensorflow-lite schema.
Rationale:
- Fundamentally, `nnpackage` and `TFLite` have same aim:
Running pre-trained models on a device, which has relatively low computing power and memory.
TFLite's solution is acceptable, we don't need to create same thing again.
- We can use several infra-structures and tools from TFLite.
## Extensions
`nnpackage` model has some extensions that are different or missing from TFLite.
### Multiple Layout
`nnpackage` can support multiple layouts.
1. The layout is presented using `DataFormat` enumeration.
`DataFormat` must be one of the enumeration defined in `nnpackage_schema.fbs`.
For example, `CHANNELS_FIRST` or `CHANNELS_LAST` can be used.
```
// For 2D data, NHWC(batch, height, width, channels)
// For 3D data, NDHWC(batch, depth, height, width, channels)
CHANNELS_LAST = 0,
// For 2D data, NCHW(batch, channels, height, width)
// For 3D data, NCDHW(batch, channels, depth, height, width)
CHANNELS_FIRST = 1,
```
2. `DataFormat` must be same within a submodel.
Rationale:
- frequent switching between different layout degrades the performance
Under this assumption, We expect to
- simplify the runtime implementation
- accelerate the performance
- reduce the memory usage
### Unspecified Dimension
`nnpackage` represents unspecified dimension with `-1`.
Rationale:
1. It should be `int` since dimension is int type flatbuffer schema. Thus '?' cannot be used.
2. `0` is also a candidate, which is used for Android NN API.
However, we would like to reserve `0` because `0` could be a valid dimension for a certain
operator (e.g. `tflite.slice`).
## Operator Reference
All operators use same semantics of tensorflow lite operators.
Refer tensorflow lite source code to understand what inputs, outputs and attributes
are required and how they are interpretered.
## Schema Source
nnpackage supports two kinds of models: `tflite` and `circle`
- For tflite, see `schema.fbs` from tensorflow lite source.
- For circle, see [`../schema/circle_schema.fbs`](../schema/circle_schema.fbs).
|
