In case of CIFG LSTM, input layer norm weights are not used in the
computation.
Bug: 129126572
Test: mma
Change-Id: I57bc578606132a2c44c71ab63dd7645dcc001302
Merged-In: I57bc578606132a2c44c71ab63dd7645dcc001302
(cherry picked from commit 0480af9aa8)
This CL adapts the VTS code to the corresponding changes made in the NN
utility library.
Bug: 128319484
Test: mma
Test: atest VtsHalNeuralnetworksV1_0Target
Test: atest VtsHalNeuralnetworksV1_1Target
Test: atest VtsHalNeuralnetworksV1_2Target
Change-Id: I470e8228cde2b75620ad851e4fe408f8e8329e7c
Merged-In: I470e8228cde2b75620ad851e4fe408f8e8329e7c
(cherry picked from commit 102e0442d8)
This CL adapts the VTS code to the corresponding changes made in the NN
utility library.
Bug: 119570067
Test: mma
Test: atest VtsHalNeuralnetworksV1_0Target
Test: atest VtsHalNeuralnetworksV1_1Target
Test: atest VtsHalNeuralnetworksV1_2Target
Change-Id: I7cbc1d7025c0352aa1ed29d71dc84c2fcfc20a4f
Merged-In: I7cbc1d7025c0352aa1ed29d71dc84c2fcfc20a4f
(cherry picked from commit e68668f65b)
- Instead of isCachingSupport returning a single boolean, switch to
getNumberOfCacheFilesNeeded returning the number of cache files. This
is to support use cases when driver needs more than one cache file for
each type, or when driver does not need data cache.
- Instead of a separate saveToCache, pass cache info along with
prepareModel_1_2 to save into cache as well as perform compilation.
This is to avoid a potential additional copy of cache files.
Bug: 123780248
Test: VtsHalNeuralnetworksV1_xTargetTest with 1.2 sample driver
Test: VtsHalNeuralnetworksV1_xTargetTest with a test driver that can
read and write cache entries
Change-Id: I921b7b8ccc3c66af19f6589f7213c6870d6f07bf
Merged-In: I921b7b8ccc3c66af19f6589f7213c6870d6f07bf
(cherry picked from commit b61ba1ed0b)
Also updates documentation for this op and UnidirectionalSequenceLSTM
op.
Bug: 123644584
Test: in ag/6758764
Change-Id: I72d029fef6d890eb1771c21814b028b09af280c7
Merged-In: I72d029fef6d890eb1771c21814b028b09af280c7
(cherry picked from commit f404a1e894)
Performance information in Capabilities is used by the runtime when
it selects the appropriate processor to distribute work to. Prior to
this CL, Capabilities can only distinguish between float and non-float
data types -- so, for example, float16 and float32 performance is
considered to be the same, and performance for all non-float data types is
considered to be the same.
Bug: 124041010
Test: NeuralNetworksTest_static
Test: VtsHalNeuralnetworksV1_2TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.2::IDevice/sample-all
Change-Id: I83fb5920c1c75afbd7750d793a0b8f3e72a0552c
Merged-In: I83fb5920c1c75afbd7750d793a0b8f3e72a0552c
(cherry picked from commit 632b4bd9b0)
The filter tensor is not always operand 1 if there are multiple
operations in the model.
Test: 1.2 VTS tests with sample driver
Change-Id: I8925dfd18072ae5579657e8aa13128e7d9f58324
Merged-In: I8925dfd18072ae5579657e8aa13128e7d9f58324
(cherry picked from commit 8804423ca1)
This is needed to be able to test DEQUANTIZE after adding
TENSOR_QUANT8_SYMM support.
Test: NeuralNetworksTest_static
Test: VtsHalNeuralnetworksV1_2TargetTest
Change-Id: Iba9b286df70919d7b67cd77c91e625a044bd686c
Merged-In: Iba9b286df70919d7b67cd77c91e625a044bd686c
(cherry picked from commit bf26a9e3d7)
This CL creates a new suite of tests to enable presubmit tests:
* PresubmitHalNeuralnetworksV1_0TargetTest
* PresubmitHalNeuralnetworksV1_1TargetTest
* PresubmitHalNeuralnetworksV1_2TargetTest
These tests are the same as the VTS tests, with the exception that they
will skip running all tests (but still pass) if the service cannot be
found and its name starts with "service-".
This change does not affect the existing NNAPI VTS tests.
Test: mma
Test: atest
Bug: 124040554
Change-Id: I36a38b66b21fd51d0ca381bb4e05a39266dd353f
(cherry picked from commit ed68233697)
Argument-dependent lookup will only work for operator>> if the operator
is in one of the argument's namespaces. This caused the enumerations to
pretty-print for V1_0, but not for V1_1 or V1_2. This change ensures the
V1_0 namespace is used.
Test: mma
Test: atest VtsHalNeuralnetworksV1_0TargetTest (verified the test output "OFFLINE" for DeviceStatus and "DEVICE_UNAVAILABLE" for ErrorStatus instead of raw byte value representation)
Test: atest VtsHalNeuralnetworksV1_1TargetTest (verified the test output "OFFLINE" for DeviceStatus and "DEVICE_UNAVAILABLE" for ErrorStatus instead of raw byte value representation)
Test: atest VtsHalNeuralnetworksV1_2TargetTest (verified ran and passed tests)
Fixes: 124316129
Change-Id: I764a46e2d87615b1f3da0ab0e6edb134bb533887
(cherry picked from commit 42a35bee10)
The tmp directory is only removed when the driver reports caching not
supported, otherwise it is kept for debugging purpose.
Test: 1.2 VTS tests with sample driver
Change-Id: I5969beb1ec365c992765f40d7693630606f16e18
Merged-In: I5969beb1ec365c992765f40d7693630606f16e18
(cherry picked from commit 350d91b1df)
Test was failing with segmentation fault (and crashing whole VTS) when
device was not available.
Change-Id: Id0f28d061dc5858fa00ef1bac5f7aa467d860864
Merged-In: Id0f28d061dc5858fa00ef1bac5f7aa467d860864
(cherry picked from commit 70d25b813e)
* Adds a specification of invalid scale and zero point for TENSOR_BOOL8.
This fixes vts failures for comparison ops.
* Removes (FUNDAMENTAL_MIN - 1) from invalid OperationTypes.
FUNDAMENTAL_MIN is equal to 0 and resulting -1 was statically casted to
uint32_t and passed 4294967295 as an invalid OperationType. However, our
validateOperation function interpreted this ID as an extension ID and
didn't fail.
* Adds mutateOperationOperandTypeSkip for QUANTIZE and DEQUANTIZE.
* Adds removeOperandSkip for BIDIRECTIONAL_SEQUENCE_RNN.
Fix: 121130841
Fix: 123247345
Test: VtsHalNeuralnetworksV1_2TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.2::IDevice/sample-all
Change-Id: Iefb502c6b9301d5470eb4cdaa46d398f1a0e512a
Merged-In: Iefb502c6b9301d5470eb4cdaa46d398f1a0e512a
(cherry picked from commit 923b8c5842)
This is a follow-up to change Ia9b99015eec7a48bbf969cbe503862271f09adca
Bug: 118605927
Test: mma
Change-Id: I7ddafca04bce6fd37a9c0877270cee325111d833
Merged-In: I7ddafca04bce6fd37a9c0877270cee325111d833
(cherry picked from commit 1ffe69a8e9)
Add the following tests for compilation caching:
- validation tests
- Test isCachingSupported
- Test prepareModelFromCache with invalid numFd and invalid access mode
- Test saveToCache with invalid numFd, invalid access mode,
invalid file size, and invalid fd offset
- execution test
- Save a mobilenet model to cache and then retrieve and run accuracy
evaluation.
- The same test but the file offsets for prepareModelFromCache is not at zero.
- security test
- CompilationCachingSecurityTest.CorruptedSecuritySensitiveCache
Randomly flip one bit of security-sensitive cache.
- CompilationCachingSecurityTest.WrongLengthSecuritySensitiveCache
Randomly append bytes to security-sensitive cache.
- CompilationCachingSecurityTest.WrongToken
Randomly flip one bit of cache token.
Bug: 119616526
Test: VtsHalNeuralnetworksV1_xTargetTest with 1.2 sample driver
Test: VtsHalNeuralnetworksV1_xTargetTest with a test driver that can
read and write cache entries
Change-Id: Iae9211cb28ce972b29572dfedd45d1ade4dfdaf5
Merged-In: Iae9211cb28ce972b29572dfedd45d1ade4dfdaf5
(cherry picked from commit 3405878e5e)
Machine Learning is a fast moving domain: new operations and data types
are introduced at a staggering speed. The Android API, on the other
hand, evolves on a yearly cycle. Many application developers, OEMs, and
SoC vendors would like to be able to define new operations at a faster
cycle.
In OC-MR1, NNAPI provided a simple mechanism to mitigate this problem:
two OEM data types and an OEM operation. The downside of this mechanism is that
it is simplistic: There’s no guarantee of consistency between vendors.
Examples of features that are required by the first parties' use cases
that we would like to be able to define outside of the normal Android
release cycle:
- New data types:
- Sparse tensor (used by some speech generation models).
- 16 bit int tensor (needed by OCR).
- 16 bit float tensor.
- New operations:
- 16 bit quantized LSTM (needed by OCR).
- Basic primitives like sqrt and floor.
- Logical operations.
- Complex neural network layers.
- Control flow.
- Enhancement to existing operations:
- Concatenate supporting different scales/zeroPoints for arguments.
- High-dimensional tensors.
- Ordering of dimensions.
We are going to provide support for two types of extensions:
- NNAPI Platform extension. This is functionality that will become part of
future releases of NNAPI. These extensions provide generic,
non-vendor-specific functionality. Only the Android team can define new
platform extensions. These extensions will be defined in the master
branch of AOSP. Each extension comes with:
- Documentation defining the extension,
- A header file for the new constants,
- A parameter validation library to be used by drivers, and
- Validation tests akin to the CTS and VTS tests.
- Vendor extension. A vendor-specific extension is an alternative to OEM
operation and data types. Its usage will be limited only to first party apps
preinstalled on the /vendor or /odm partition.
Each vendor will be identified by a specific value to
prevent collisions between multiple IPs found on the same SoC. This
effectively creates a vendor-specific namespace. These identifiers are
assigned by Google.
This change only defines the new interface. The implementation follows
in changes Ie4e2530e4c81fabe4eb59b7a6ba4a3b4bb483bd1,
I02aa12f4a8444012ddf3b20c2bfbba6a21ce9ce9, and
Icf59ed04e602aa7a730eb1eb45e5f6b1204fafb3.
Bug: 118605927
Test: mma
Change-Id: Ia9b99015eec7a48bbf969cbe503862271f09adca
Merged-In: Ia9b99015eec7a48bbf969cbe503862271f09adca
(cherry picked from commit 9212018558)
Test dynamic output shape with generated models when
- Dimensions of output operands are fully specified
- Dimensions of output operands are unspecified with sufficient buffer
- Dimensions of output operands are unspecified with insufficient buffer
Test: VTS on 1.2 sample driver
Change-Id: I4d26395ce443687ccbd47445b36e3356d70035cc
Merged-In: I4d26395ce443687ccbd47445b36e3356d70035cc
(cherry picked from commit 929fd21e06)
- Instead of reporting PASS for unsupported tests, use GTEST_SKIP to
skip the tests at runtime.
Bug: 113356629
Test: mm
Test: VTS tests on HVX driver
Change-Id: I6a870b61809e58490e66dd4ea36ddeb64fc68a07
Merged-In: I6a870b61809e58490e66dd4ea36ddeb64fc68a07
(cherry picked from commit bb685a4a97)
FastMessageQueue is a Treble-compliant data structure that enables fast
communication between two processes. The FMQ object itself is an atomic
circular buffer that is optionally synchronized with a futex. However,
FMQ has no notion of ownership or lifetime across processes, so it must
be paired with higher-level constructs to manage the lifetime and
ownership.
The NNAPI is introducing the notion of an "Execution Burst" object (or
more simply a "Burst" object), which is similar to an
ANeuralNetworksExecution, but is intended to be reused across multiple
executions and has lower IPC overheads. It achieves this low IPC
overhead by replacing HIDL HwBinder calls with FMQ messages.
Specifically, it replaces IPreparedModel::executeSynchronously's call
from the client into the service with fmq_sync<FmqRequestDatum> (an FMQ
channel used to pass a serialized Request object) and it replaces
the return from the service into the client with
fmq_sync<FmqResultDatum> (an FMQ channel used to return serialized
result status and OutputShapes information).
Each channel is a unidirectional flow of information with exactly one
producer and exactly one consumer. The channels are created by the NN
runtime and passed to the service via
IPreparedModel::configureExecutionBurst.
This CL tests the Burst in both the execution path and validation path
in the Vendor Test Suite (VTS) in neuralnetworks/1.*/vts/functional/.
The VTS binary--VtsHalNeuralnetworksV1_2TargetTest--can be built and run
as any previous version could.
Bug: 119570067
Test: mma
Test: VtsHalNeuralnetworksV1_2TargetTest
Change-Id: I3a36484eff9565c2d028c07c099804a0289f294a
Merged-In: I3a36484eff9565c2d028c07c099804a0289f294a
(cherry picked from commit 814d8372f3)
Enable VTS unit test for dynamic output shape deduction.
Only test dynamic output shape for V1_2::IDevice with V1_2::Model.
Bug: 73506513
Test: VtsHalNeuralnetworksV1_xTargetTest with 1.2 sample driver
Change-Id: I4134e1ec54a15554eb8533134897279651b57da3
Merged-In: I4134e1ec54a15554eb8533134897279651b57da3
(cherry picked from commit a316581b21)
FastMessageQueue is a Treble-compliant data structure that enables fast
communication between two processes. The FMQ object itself is an atomic
circular buffer that is optionally synchronized with a futex. However,
FMQ has no notion of ownership or lifetime across processes, so it must
be paired with higher-level constructs to manage the lifetime and
ownership.
The NNAPI is introducing the notion of an "Execution Burst" object (or
more simply a "Burst" object), which is similar to an
ANeuralNetworksExecution, but is intended to be reused across multiple
executions and has lower IPC overheads. It achieves this low IPC
overhead by replacing HIDL HwBinder calls with FMQ messages.
Specifically, it replaces IPreparedModel::executeSynchronously's call
from the client into the service with fmq_sync<FmqRequestDatum> (an FMQ
channel used to pass a serialized Request object) and it replaces
the return from the service into the client with
fmq_sync<FmqResultDatum> (an FMQ channel used to return serialized
result status and OutputShapes information).
Each channel is a unidirectional flow of information with exactly one
producer and exactly one consumer. The channels are created by the NN
runtime and passed to the service via
IPreparedModel::configureExecutionBurst.
This CL defines the FmqRequestDatum and FmqResultDatum types in
types.hal. IBurstContext.hal defines IBurstContext, a HIDL object used
by the service to manage the resources of a Burst. IBurstCallback.hal
defines IBurstCallback, a HIDL callback object that can be used to
retrieve the handle to a resource the service has either not yet seen or
has evicted from its cache. Finally, IPreparedModel.hal is extended with
IPreparedModel::configureExecutionBurst to create the burst object.
Bug: 119570067
Test: mma
Change-Id: I333da70201531b1396efc714d096c277e8e1d47b
Merged-In: I333da70201531b1396efc714d096c277e8e1d47b
(cherry picked from commit 7e91e24fe1)
