Khronos releases NNEF 1.0 standard for optimized deployment of trained neural networks

Khronos releases NNEF 1.0 standard for optimized deployment of trained neural networks


US: The Khronos Group, an open consortium of leading hardware and software companies creating advanced acceleration standards, announces the ratification and the public release of the NNEF™ 1.0 (Neural Network Exchange Format) specification. After gathering feedback from the industry review of the provisional specification, Khronos releases NNEF 1.0 as a stable, flexible, and extensible open standard for hardware manufacturers to reliably deploy optimized, accelerated neural network inferencing onto diverse edge devices. Together with this release, an ecosystem of tools is now also available on GitHub, including an NNEF parser and converters from Tensorflow and Caffe. Importers into popular inferencing environments, including Android’s Neural Network API (NNAPI) and Khronos’ OpenVX™, are also being developed.

NNEF was created to reduce industry fragmentation by facilitating the exchange of neural networks among training frameworks and inference engines, increasing the freedom for developers to mix and match the inferencing and training solutions of their choice. The open, royalty-free standard, which is the result of collaboration between industry-leading members of the Khronos NNEF Working Group, provides roadmap stability that hardware and software companies can rely on for product deployment while maintaining the flexibility to respond to the needs of the rapidly-evolving machine learning industry.

NNEF accommodates a wide range of use-cases and network types. Its base format is designed to be both humanly readable and easy to parse and optimize, and it includes the flexible representation of operator precisions and compound operations to facilitate optimal mapping to diverse accelerator architectures. NNEF extensions allow for human-editing of large networks and handling of specific issues, such as custom data formats for trained network weights.

“Khronos recognized a growing format logjam for companies deploying trained neural networks onto edge devices. We set out to build the first open standard solution for engineers to optimize and deploy trained networks onto diverse inference engines. Core NNEF 1.0 will enable cutting-edge solutions today and also flexibly evolve through its extension mechanisms,” said Peter McGuinness, Khronos NNEF working group chair. “In December 2017, we released the developer preview of NNEF and made an open call for industry feedback. Community response has been tremendous, confirming the demand for this standard and enabling us to achieve a responsive and complete NNEF 1.0 specification.”

A complete workflow from training through optimization to deployment is possible using NNEF as a standardized transfer format. At launch, the standard will be supported by two open source Tensorflow converters, both for network descriptions based on protobuf and python code, and a converter for Caffe. A Caffe2 open-source importer/exporter that is in development by Au-Zone Technologies will be available in Q3 2018. Various tools from member companies, such as Almotive and AMD, are in development, including an importer to Android NNAPI by National Tsing-Hua University of Taiwan.

The NNEF working group is committed to enabling and encouraging reliable interchange with the rapidly growing number of training frameworks, including Torch, Chainer, Theano, PyTorch, and MXNet with open source importers and exporters. Additional open-source tools, including an NNEF syntax parser and validator, are available now for the easy creation of importers into custom, mobile, and embedded edge-inferencing environments, such as Apple’s Core ML and Khronos’ OpenVX standard, for vision and inferencing runtime acceleration.