diff options
Diffstat (limited to 'docs/nnfw/project/2019_requirement_specification.md')
-rw-r--r-- | docs/nnfw/project/2019_requirement_specification.md | 131 |
1 files changed, 131 insertions, 0 deletions
diff --git a/docs/nnfw/project/2019_requirement_specification.md b/docs/nnfw/project/2019_requirement_specification.md new file mode 100644 index 000000000..937530766 --- /dev/null +++ b/docs/nnfw/project/2019_requirement_specification.md @@ -0,0 +1,131 @@ +# Software Requirement Specification + +## Background +Artificial intelligence (AI) techniques are getting popular and utilized in various products and +services. While the cloud-based AI techniques have been used to perform compute/memory intensive +inferences because of the powerful servers on cloud, on-device AI technologies are recently drawing +attention from the mobile industry for response time reduction, privacy protection, and +connection-less AI service. Big mobile players, such as Google, Apple, and Huawei, are investing +their research effort on the on-device AI technologies and already announced hardware and software +on-device AI solutions. Samsung is not leading this trend currently, but since on-device AI area is +just started and still in the initial state, there are still opportunities and possibilities to +reduce the gap between pioneer companies and Samsung. We believe on-device AI will become a key +differentiator for mobile phone, TV, and other home appliances, and thus developing on-device AI +software stack is of paramount importance in order to take leadership in the on-device AI +technology. + +Although the vision of on-device AI is promising, enabling on-device AI involves unique technical +challenges compared to traditional cloud-based approach. This is because on-device AI tries to +conduct inference tasks solely on device without connecting to cloud resources. Specifically, +hardware resources on device, such as processor performance, memory capacity, and power budget, are +very scarce and limit the compute capability, which is typically required to execute complicated +neural network (NN) models. For example, in one product requirement, a mobile device should consume +less than 1.2W and could use at most 2W only for 10 minutes due to thermal issue. Next, on-device +AI software stack needs to support diverse device environments, since embedded platforms may consist +of heterogeneous compute devices, such as CPU, GPU, DSP, or neural processing unit (NPU), and use +different OS platforms, such as Tizen, Android, or various embedded Linux. + +To tackle the challenges above and to have the leadership on on-device AI technology, this project +aims at developing a neural network inference framework specialized and optimized for on-device AI. + + +## Product Context + +This project _nnfw_ aims at providing a high-performance, on-device neural network (NN) inference +framework that performs inference of a given NN model on processors, such as CPU, GPU, or NPU, in +the target platform, such as Tizen and Smart Machine Platform (SMP). + +### Expected Value + +We expect the following would be possible with _nnfw_: + +- To improve user experience by reducing the service response time. +- To provide AI services without network connection while achieving similar performance. +- To protect personal information and company confidential by limiting data transfer to the network. + + +### Success Criteria + +The goals of this project are: + +- Supports mixed acceleration using CPU and GPU. + + for operator coverage flexibility. + + for flexible utilization of computing resource on device. +- Define Common IR and Runtime APIs and perform successful inference using it. + + for _nncc_ integration as a frontend, and _nnfw_ itself to concentrate on the backend. +- Support of user implemented kernel extension for custom operator. +- Construction of SW infrastructure to support SR NPU. + + +### Target + +_nnfw_ targets following platforms and target devices: + +- Odroid-XU4 running Tizen 5.5 (Primary) +- A variety of Android based mobile phones (Secondary) + + +### Product Roadmap + +- March: Set up milestones, tasks, workgroups, initial code structure, and build/test + infrastructure. +- May: Tizen M1 release / Execute InceptionV3 with static scheduling by mixing CPU and GPU on + Odroid-XU4. +- August: Perform inference using NN Package and Runtime API. +- October: Tizen M2 release / Completed neural network acceleration SW stack integrated with NN + Compiler. +- December: Release NPU SDK v1.0 pre-alpha. + + +## Requirements + +### Functionality Requirements + +_nnfw_ has the following functionality requirements: + +1. CPU/GPU mixed acceleration + - Description + + Run the model using a mixture of CPU and GPU + - Validation + + Run the InceptionV3 model by selecting CPU or GPU for individual operators. + + Confirm execution results against ground truth(for example, the result of using CPU or GPU + alone). +1. Support its own input format + - Description + + Define and support its own input format to ensure file format independence + + Define and implement of Common IR + - Validation + + Read and execute the input model described by Common IR. + + Confirm execution results against ground truth(for example, the result of using NNAPI). +1. Support its own API + - Description + + Define and implement its own API to replace current NNAPI. + - Validation + + Perform unit tests and integration tests for individual APIs. +1. Custom operator support + - Description + + Defines a specification that describes a custom operation, and provides a mechanism for + writing and installing the kernel implementation needed to run it at runtime. + - Validation + + Load and execute the input format that contains the model using the custom op and the kernel + implementation of the custom op. + + Confirm execution results against ground truth. +1. Prepare SW infrastructure for NPU support + - Description + + The runtime must be able to read and process the model information developed for the purpose + of execution in NPU. + - Validation + + Read the model developed for the NPU and run it on the NPU. + + Confirm execution results against ground truth. + + +### Non-Functionality Requirements + +1. Optimizing mixed acceleration performance + - Description + + Ensure performance above the individual acceleration averages for mixed acceleration using + CPU and GPU. + - Validation + + Measure the inference time for mixed accelerations for the target model. + + Compare the result to the average value of the CPU or GPU alone acceleration time for the + same model. |