diff options
Diffstat (limited to 'docs/nncc/project/detailed_level_design.md')
| -rw-r--r-- | docs/nncc/project/detailed_level_design.md | 329 |
1 files changed, 329 insertions, 0 deletions
diff --git a/docs/nncc/project/detailed_level_design.md b/docs/nncc/project/detailed_level_design.md new file mode 100644 index 000000000..50fb8fa13 --- /dev/null +++ b/docs/nncc/project/detailed_level_design.md @@ -0,0 +1,329 @@ +# SW Detailed Level Design + +**Revision history** + +| Ver. | Date | Contents | Author | Approver | +| ---- | ---------- | ----------------- | ----------------- | ------------ | +| 0.1 | 2018.06.20 | Initial version | Vostokov Sergey | Sung-Jae Lee | +| 0.2 | 2018.06.21 | SE member review | Alexey Kondrashov | | +| 1.0 | 2018.06.22 | Final DR1 version | Vostokov Sergey | Sung-Jae Lee | + +**Terminology and Abbreviation** + +| | | +| ------------ | ------------------------------------------------------------- | +| OS | Operating System | +| OS API | Application interface of OS | +| HW | Hardware | +| SW | Software | +| NN | Neural Network | +| NN model | Neural network model (Instance of NN built with ML framework) | +| NN compiler | The compiler for neural network | +| ML framework | The machine learning framework | +| TF/TF Lite | Tensorflow/Tensorflow Lite ML framework | +| IR | Intermediate representation | +| CI/CI system | Continuous integration system | +| UI | The user interface | +| GUI | The graphical user interface | +| CLI | The command-line interface | + +**References** + +\[1\] Vostokov Sergey, [SW Requirements Specification](requirements_specification.md) + +\[2\] Vostokov Sergey, [SW High-Level Design](high_level_design.md) + +## Overview + +### Scope + +The main goal of the project is to develop a compiler for neural +networks to produce executable artefact for specified SW and HW +platform. + +The development scope includes the following components: + + - Develop importer module to parse, verify and represent NN model for + further optimization and compilation + - Develop code emitters to produce executable binary for CPU and GPU + + +**2018 year goals:** + + - Support TensorFlow Lite NN model format + - Support Caffe NN model format + - Support Caffe2 NN model format (Optional) + - Support compilation of MobileNet NN + - Support compilation of Inception v3 NN + - Support ARM CPU + - Support ARM GPU (Mali) + - Support Tizen OS + - Support SmartMachine OS +(Optional) + +| Product | Target Model Name | Comment | +| ------------------- | ------------------------------ | ---------------- | +| Tizen phone | Tizen TM2 | Reference device | +| Tizen device | Odroid XU4 | Reference board | +| SmartMachine target | Microvision mv8890, exynos8890 | Reference device | + +Table 1-1. Target Model + +### Design Consideration + +Deep learning software demands reliability and performance. The common +approach which comes from the history is to develop a SW framework +(machine learning framework) which would compute each step of the neural +network inference process using supported hardware. This approach is +used in many popular solutions like Google Tensorflow/Tensorflow Lite, +Caffe/2, etc. Traditionally, neural network developers build a +computation graph and then an appropriate machine learning framework +interprets it. The latest discoveries in AI field show that the +node-visitor method of execution is inefficient. As a result, a second +approach has been worked out by the industry, which is a neural network +compiler that executes code more efficiently. + +This document presents the design of the *nncc*, a neural network +compiler collection. The design should provide the easiest way to extend +the functionality of the *nncc* by adding new modules with the following +features: + + - Support neural networks produced by various machine learning + frameworks; + - Produce an artefact taking advantages of various hardware + including specialized processors like NPU; + - Apply new domain specific optimization techniques over given NN. + +Non-functional requirements to the developed software are well-described +in the SW Requirements Specification, such requirements are not shown +here to avoid duplication. + +### Constraints + +See constraints in SW Requirements Specification. + + +<table> +<colgroup> +<col style="width: 24%" /> +<col style="width: 64%" /> +<col style="width: 10%" /> +</colgroup> +<thead> +<tr class="header"> +<th>Item</th> +<th>Assumptions, Dependencies and the Constraints</th> +<th>Reference</th> +</tr> +</thead> +<tbody> +<tr class="odd"> +<td>Tizen SW Platform</td> +<td><dl> +<dt>The following items should be provided:</dt> +<dd><ul> +<li>Tizen API</li> +<li>Tizen kernel</li> +<li>Tizen FW</li> +<li>Tizen SDK</li> +<li>Tizen naming convention</li> +</ul> +</dd> +</dl></td> +<td>- <a href="www.tizen.org" class="uri">www.tizen.org</a> <br>- <a href="wiki.tizen.org" class="uri">wiki.tizen.org</a> <br>- <a href="developer.tizen.org" class="uri">developer.tizen.org</a></td> +</tr> +<tr class="even"> +<td>SmartMachine OS Platform</td> +<td><dl> +<dt>The following items should be provided:</dt> +<dd><ul> +<li>SmartMachine API</li> +<li>SmartMachine kernel</li> +<li>SmartMachine FW</li> +<li>SmartMachine SDK</li> +<li>SmartMachine naming convention</li> +</ul> +</dd> +</dl></td> +<td>- <a href="http://suprem.sec.samsung.net/confluence/pages/viewpage.action?pageId=81833987">Platform confluence</a> <br>- <a href="https://github.sec.samsung.net/RS7-SmartMachine">Github</a> <br>- <a href="http://suprem.sec.samsung.net/confluence/display/ASEC/Adaptive+AUTOSAR">Functional Safety confluence</a></td> +</tr> +<tr class="odd"> +<td>Host OS</td> +<td>Linux-based OS (Ubuntu, Archlinux, etc)</td> +<td>- <a href="https://www.ubuntu.com/">Ubuntu site</a> <br>- <a href="https://www.archlinux.org/">Archlinux site</a></td> +</tr> +<tr class="even"> +<td>Tizen target HW</td> +<td>The reference device should be provided: Tizen TM2</td> +<td></td> +</tr> +<tr class="odd"> +<td>SmartMachine target HW</td> +<td>The reference device should be provided</td> +<td></td> +</tr> +</tbody> +</table> +Table 1-2. Assumptions, Dependecies and the Constraints</caption> + +## SW Detailed Structure Design + +### SW Block Structure + +Top-Level Components of the nncc descriped in HLD. More detailed +structure and class diagram will be available after development +completion. + +### SW Block Feature + +1. Initialization: configure all internal modules (see + [{Initialization} Detailed Design](#initialization-detailed-design)) +2. Frontend: Import NN model (see [{Import NN model} Detailed + Design](#import-nn-model-detailed-design)) + - *Caffe frontend*: includes the parser of Caffe NN model format, + verifier to ensure that parsed data is valid and consentient, + and Caffe-specific IR converter + - *Caffe2 frontend*: includes the parser of Caffe2 NN model + format, verifier to ensure that parsed data is valid and + consentient, and Caffe2-specific IR converter to Model IR + - *Tensorflow Lite frontend*: includes the parser of Tensorflow NN + model format with automatic version recognition feature, + verifier to ensure that parsed data is valid and consentient, + and Tensorflow Lite-specific IR converter to Model IR +3. Backend: Generate the code (see [{Generate the code} Detailed + Design](#generate-the-code-detailed-design)) + - *Interpreter:* As it was described in SW High-Level Document + imported NN model may proceed through three step of Intermediate + representation: Model IR, Coarse-Grained IR, Fine-Grained IR. + The Interpreter backend uses each this IR to do inference of + given NN model. As the output, the user gets the resulting + calculation of all NN ops included into original computation + graph. + - *Binary*:This type refers to generating binary code that can be + executed on the target device. NN compiler can generate code + that is either executed solely on CPU or takes advantage of the + GPU when possible if the corresponding target was specified. The + user may want to incorporate 3rd party libraries included into + target firmware or delivered with the application package. In + this case, the compiler prepares the data following EABI + convention and embeds an invocation of high-level functions by + appropriate symbol. + - *Soft*: Resulting program is a generated source code in + high-level programming language C or C++. Here there are two + options: the first one is to generate the source code that does + not depend on libraries outside of itself, with the exception of + system libraries. The second one is to include the code to + invoke high-level functions from 3rd party libraries. For + example, it may be an invocation of matrix multiplication from + GEMM library. + +## SW Detailed Operation Design + +### {Initialization} Detailed Design + +#### Major Function + +To provide a valid configuration session for all modules of *nncc* using +user input from the command line/config file/environment variables. + +#### Operation Sequence + +Initialization of the *nncc* includes command line option processing, +configuration of its subsystems as well as any error checking possible +at this stage. It consists of the following steps: + +1. Collect all command line options and verify their format for + validity (no syntax errors etc.) + +2. Check for validity and then process general options + +3. Load subsystem modules + +4. For each one of them: + + - Configure + - Pass command line options + - Check command line options for validity (for example, check + that every required option is present) + +At the end of this process each subsystem is configured and has access +to all data needed for its operation. + +### {Import NN model} Detailed Design + +#### Major Function + +To convert given NN model from framework-specific IR to Model IR for +further processing. + +#### Operation Sequence + +As you may see on the diagram, neural network import is the main +function of the compiler front-end part. The result of this operation is +a computation graph which is presented as Model IR. + + + +The import process consists of three parts: + +1. NN model parsing +2. Verification of the result from the previous step +3. Converting the model to the Model IR + +During the first step, file or files containing the model are read and +represented in some format specific to each NN framework. + +Verification step is included to ensure that: + + - None of the files constituting the model are damaged + - Model format corresponds to the specified one + - Version of the model format corresponds to the specified one + +The most important step is accurately converting the model from the +framework-specific representation to the Model IR. This conversion +includes: + + - *Translation of the NN model computation graph to the Model IR + computation graph.* During the translation new nodes may be + introduced - for example, a high-level NN operation may be split + into a few smaller ones. + - *NN model parameter layout conversion.* The way parameters (also + known as weights) of a model are layed out in each specific NN + framework may differ, and it is necessary to convert such layout + into a unified format. + - *NN operation parameter conversion.* Each NN operation has a set + of its own parameters describing the way this operation should be + performed, and these parameters also differ between frameworks. + +Resulting Model IR is equivalent to the initial NN model in terms of how +NN model inputs would be transformed into its outputs if all the +operations in the Model IR were executed. + +### {Generate the code} Detailed Design + +Development in progress. Will be described on Completion DR. + +## Interface Design + +Development in progress. Will be described on DR2. + +## SW Code Structure + +| Directory | Description | +| ------------------------ | -------------------------------------------------------------------- | +| / | source codes of the build system, main README file | +| /contrib | Incubating projects | +| /doc | Contains the documentation of the project | +| /doc/project | Contains project management documents (SRS, SDD, STD, HLD, DLD, etc) | +| /libs | Contains the source of the libraries which are used by the nncc | +| /libs/core | Contains the source code of the core library of nncc | +| /libs/frontend | Contains the source code of supported frontend's plugins | +| /libs/frontend/caffe | The source code for the Caffe frontend | +| /libs/frontend/caffe2 | The source code for the Caffe2 frontend | +| /libs/frontend/tflite | The source code for the Tensorflow Lite frontend | +| /libs/backend | Contains the source code of supported backend’ plugins | +| /libs/backend/cpu | Contains the source code of CPU backend | +| /libs/backend/gpu | Contains the source code of GPU backend | +| /libs/backend/3rd\_party | Contains the source code of backend to utilize 3rd party libraries | +| /scripts | Various scripts for building and testing the nncc | +| /tools | The source code of the executables | |