Commit graph

6 commits

Author SHA1 Message Date
Michael Butler
7ed6135471 NN validation tests
This CL adds validation tests for all of the existing generated models.
The strategy of this CL is this: given a valid model or request, make a
single change to invalidate the model or request, then verify that the
vendor service driver catches the inconsistency and returns
INVALID_ARGUMENT.

Bug: 67828197
Test: mma
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Merged-In: I8efcdbdccc77aaf78992e52c1eac5c940fc81a03
Change-Id: I8efcdbdccc77aaf78992e52c1eac5c940fc81a03
(cherry picked from commit f76acd0312)
2018-04-18 19:07:52 +00:00
David Gross
fa89f009f9 Add validation tests for consistency of model inputs and outputs.
Test: VtsHalNeuralnetworksV1_1TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.1::IDevice/sample-all --gtest_filter=Flavor/NeuralnetworksInputsOutputsTest.*
Test: VtsHalNeuralnetworksV1_1TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.1::IDevice/sample-float-fast --gtest_filter=Flavor/NeuralnetworksInputsOutputsTest.*
Test: VtsHalNeuralnetworksV1_1TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.1::IDevice/sample-quant --gtest_filter=Flavor/NeuralnetworksInputsOutputsTest.*

Bug: 67828197
Merged-In: I245227dce095b9cbbb9b527ad99aa71d11f77c4f
Change-Id: I245227dce095b9cbbb9b527ad99aa71d11f77c4f
(cherry picked from commit 7a76d8a7a8)
2018-04-18 19:07:45 +00:00
Pirama Arumuga Nainar
0d8dbeecd6 Disable arm32 asan for VtsHalNeuralnetworksV1_1TargetTest
Bug: http://b/74200014

Building this module with asan triggers an internal error in gold.

Test: mma SANITIZE_TARGET=address in
hardware/interfaces/neuralnetworks/1.1/vts/functional

Merged-In: I3ce5352a6c86c8dca257d35af06989e57f871749
Change-Id: I3ce5352a6c86c8dca257d35af06989e57f871749
(cherry picked from commit cca38c56f4)
2018-04-17 22:30:07 +00:00
Michael Butler
1ae02d63df Relax NeuralNetwork's VTS positive and negative base tests
There are some NN VTS tests that assume a service is able to generate a
model consisting only of a floating point add operation. However, some
drivers do not support floating point operations. This CL relaxes the
test requirements to allow a test to be skipped if the service does not
support floating point add.

Bug: 72764145
Test: mma
Test: VtsHalNeuralnetworksV1_0TargetTest

Merged-In: I6b0644432680fc2f8098b5187795dc2953df03f9
Change-Id: I6b0644432680fc2f8098b5187795dc2953df03f9
(cherry picked from commit 4d5bb1097a)
2018-03-05 13:01:00 -08:00
David Gross
a8cd639dc0 Add Capabilities::relaxedFloat32toFloat16Performance for 1.1.
Test: VtsHalNeuralnetworksV1_0TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.0::IDevice/sample-all
Test: VtsHalNeuralnetworksV1_0TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.0::IDevice/hvx
Test: VtsHalNeuralnetworksV1_1TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.1::IDevice/sample-all
Test: NeuralNetworksTest_static

Bug: 63911257

Merged-In: I040b68d84504180ba3039e4690d0cd43c2c2649f
Change-Id: I040b68d84504180ba3039e4690d0cd43c2c2649f
(cherry picked from commit 4b458caa4e)
2018-03-05 13:00:46 -08:00
Miao Wang
4862d612ad Refactor NN API VTS tests and add v1.1 tests
- Create VtsHalNeuralnetworksTest_utils module to be shared between
    v1.x tests.
  - Split the existing tests into two categories: basic, and generated.
  - Created v1.1 VTS tests ensuring no regression in existing ML models.

Bug: 63911257
Test: mm
Test: NNAPI VTS tests pass on v1.0 and v1.1 sample drivers

Merged-In: Ic77c90a3a5bbd96b0ce2acd03764dde4b3034cc9
Change-Id: Ic77c90a3a5bbd96b0ce2acd03764dde4b3034cc9
(cherry picked from commit a2d04c828e)
2018-03-05 12:59:55 -08:00