Commit graph

76 commits

Author SHA1 Message Date
David Gross
8939b5df12 Document unspecified dimensions.
Test: mm

Bug: 77234888
Bug: 109687058
Change-Id: Ia8928e494d8c712bc9cfb592d67476fde9753f45
2018-06-05 12:51:08 -07:00
Michael Butler
e98352806b NNAPI: sync NDK and HAL documentation
Bug: 72650109
Test: mma
Change-Id: I3e0a3680b89d80be500d8975f587f2d7c33fea10
2018-04-27 14:45:32 -07:00
Mika Raento
95eef46fa2 Merge "Update namespace to match framework/ml/nn" into pi-dev 2018-04-26 09:20:41 +00:00
Mika Raento
7e64e7f924 Update neuralnetworks/*/types.hal to match impl
Updates hardware/interfaces/neuralnetworks/1.(0|1)/types.hal to match
the NeuralNetworks.h header in framework/ml/nn. Only comments have
changed.

Updated using framework/ml/nn/tools/sync_enums_to_hal.py.

Change-Id: I0754868ad8acf6e2e0c5b83661d04682febec9b0
Bug: 77604249
Test: checked changes with git diff
Test: mm in $ANDROID_BUILD_TOP
2018-04-25 12:46:33 +01:00
Michael Butler
3559309ace Merge "NNAPI: Create VTS _relaxed tests for newly added tests" into pi-dev 2018-04-23 22:35:00 +00:00
Michael Butler
f2036fd441 NNAPI: Create VTS _relaxed tests for newly added tests
New tests were added to V1_0 and V1_1 specs, but they did not have a
"_relaxed" test in V1_1.

This CL updates the 1.1 VTS validation as a followup CL.

Bug: 76434855
Test: mma
Test: NeuralNetworksTest_static
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Change-Id: Ic6b3aeb12a8b90e43cf082c78442af7ec7a1018f
2018-04-23 12:51:03 -07:00
Michael Butler
579ef36ee8 Merge "Update NNAPI OWNERS file -- VTS" into pi-dev 2018-04-20 19:52:48 +00:00
Michael Butler
d42a6d4ca8 Update NNAPI OWNERS file -- VTS
Bug: 78353126
Test: Not necessary
Change-Id: Ic7e14202454d81c7ad432c0aca6752be9f58f71f
2018-04-20 19:50:05 +00:00
Mika Raento
de16694a60 Update namespace to match framework/ml/nn
Bug: 72448000
Test: make vts
Change-Id: I3543913561b33e4d1e9d960ec1f81624fb0a1e6b
2018-04-19 14:38:52 +01:00
Michael Butler
23ee97e748 Update Android.bp via update-makefiles.sh
When calling update-makefiles.sh there are
some unrelated changes that were missed in
previous commits.

Bug: 68112778
Test: mma
Change-Id: I22b76f3d5bcd084afd2f06e613d2f9f37c1a0c9c
2018-04-18 19:34:10 +00:00
Michael Butler
2504c2fe4f NNAPI: Add execution preference to prepareModel (HAL)
A model can be prepared in different ways to optimize for different
use-cases. This CL propagates the execution preference across the HAL so
that the NN service can better fit the users needs.

Bug: 77864669
Test: mma
Test: NeuralNetworksTest_static
Test: VtsHalNeuralnetworksV1_1TargetTest
Change-Id: Ib928d510d462f36b6a87d5e81010513db7829fa8
2018-04-12 19:56:14 -07:00
Michael Butler
25f3ad328b Sync NNAPI Operand and Operation documentation fixes
This CL adds the typo fixes, bug fixes, and missing descriptions added
to the NNAPI but not to the NN HAL. This CL also adds description of
implicit padding, which was present but not documented in 1.0.

Bug: 77541934
Bug: 75459529
Test: mma
Change-Id: I7baa9e515057c43e759849284b3bb0d420a71f1b
2018-04-11 13:22:00 -07:00
David Gross
541e24311d Fix documentation.
1. Fix documentation for Operand::numberOfConsumers.
2. Fix typos.

Bug: 69055733
Bug: 76016141
Bug: 74111771

Test: NeuralNetworksTest_static
Test: VtsHalNeuralnetworksV1_0TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.0::IDevice/sample-all
Test: VtsHalNeuralnetworksV1_1TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.1::IDevice/sample-all

Change-Id: Ib3d5bbfa701070597731992c3867f72d7b0280bd
2018-04-10 14:31:54 -07:00
Miao Wang
a82d39102a Fix the spec for TENSOR_QUANT8_ASYMM to match our validation.
- Scale of 0.0 is invalid for quantized tensor.

Bug: 77236592
Test: mma
Change-Id: I3a53d6303d8c964d451e17a3b1671de82d0ff335
2018-04-09 18:31:47 +00:00
TreeHugger Robot
fcf9546ffe Merge "Fix the NNAPI HAL documentation about ADD and MUL" into pi-dev 2018-03-29 22:41:15 +00:00
Michael Butler
f76acd0312 NN validation tests
This CL adds validation tests for all of the existing generated models.
The strategy of this CL is this: given a valid model or request, make a
single change to invalidate the model or request, then verify that the
vendor service driver catches the inconsistency and returns
INVALID_ARGUMENT.

Bug: 67828197
Test: mma
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Change-Id: I8efcdbdccc77aaf78992e52c1eac5c940fc81a03
2018-03-28 11:48:13 -07:00
Miao Wang
f62984027c Fix the NNAPI HAL documentation about ADD and MUL
- ADD and MUL supports QUANT8_ASYMM since OMR1. It was a bug missing
  them in the HAL documentation.
  - Added the updated hash to current.txt for this ABI preserving
  change.

Bug: 75459529
Test: mm
Change-Id: I492a7431c0dbb3dc5967c586d080eb134e380bf2
2018-03-28 11:13:48 -07:00
David Gross
7a76d8a7a8 Add validation tests for consistency of model inputs and outputs.
Test: VtsHalNeuralnetworksV1_1TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.1::IDevice/sample-all --gtest_filter=Flavor/NeuralnetworksInputsOutputsTest.*
Test: VtsHalNeuralnetworksV1_1TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.1::IDevice/sample-float-fast --gtest_filter=Flavor/NeuralnetworksInputsOutputsTest.*
Test: VtsHalNeuralnetworksV1_1TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.1::IDevice/sample-quant --gtest_filter=Flavor/NeuralnetworksInputsOutputsTest.*

Bug: 67828197
Change-Id: I245227dce095b9cbbb9b527ad99aa71d11f77c4f
2018-03-23 10:12:47 -07:00
Nathan Harold
c893754c0d Update Makefiles Missed In Previous Commits
When calling update-makefiles.sh there are
some unrelated changes that were missed in
previous commits.

Bug: 8675309
Test: compilation
Change-Id: I5bf67fbcc809de36bde1869ada7b835566a5198b
2018-03-19 16:37:46 -07:00
Pirama Arumuga Nainar
cca38c56f4 Disable arm32 asan for VtsHalNeuralnetworksV1_1TargetTest
Bug: http://b/74200014

Building this module with asan triggers an internal error in gold.

Test: mma SANITIZE_TARGET=address in
hardware/interfaces/neuralnetworks/1.1/vts/functional

Change-Id: I3ce5352a6c86c8dca257d35af06989e57f871749
2018-03-06 19:30:40 +00:00
Michael Butler
4d5bb1097a Relax NeuralNetwork's VTS positive and negative base tests
There are some NN VTS tests that assume a service is able to generate a
model consisting only of a floating point add operation. However, some
drivers do not support floating point operations. This CL relaxes the
test requirements to allow a test to be skipped if the service does not
support floating point add.

Bug: 72764145
Test: mma
Test: VtsHalNeuralnetworksV1_0TargetTest
Change-Id: I6b0644432680fc2f8098b5187795dc2953df03f9
2018-02-26 15:24:46 -08:00
David Gross
fe606d5aee Merge "Add Capabilities::relaxedFloat32toFloat16Performance for 1.1." 2018-02-26 19:27:13 +00:00
David Gross
4b458caa4e Add Capabilities::relaxedFloat32toFloat16Performance for 1.1.
Test: VtsHalNeuralnetworksV1_0TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.0::IDevice/sample-all
Test: VtsHalNeuralnetworksV1_0TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.0::IDevice/hvx
Test: VtsHalNeuralnetworksV1_1TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.1::IDevice/sample-all
Test: NeuralNetworksTest_static

Bug: 63911257

Change-Id: I040b68d84504180ba3039e4690d0cd43c2c2649f
2018-02-26 19:01:19 +00:00
Miao Wang
842e1a0331 Minor modification to allow relaxed precision test
Bug: 63911257
Test: mm
Test: VTS test pass on walleye
Change-Id: I41d9ea782262a894e7dbd00b782803c40b31c430
2018-02-22 13:40:26 -08:00
Michael Butler
7ffe78ba76 NNAPI narrow evaluation for P -- HAL
We have determined that for Android P it is sufficient to have a
mechanism for a developer to specify on a per-model basis that it is
acceptable for FLOAT32 operations to be carried out as if they were
FLOAT16 operations. This CL extends the model to include information on
these precision requirements.

Bug: 63911257
Test: mm
Test: NeuralNetworksTest
Test: VtsHalNeuralnetworksV1_0TargetTest
Change-Id: I5331e69c05c20c588035897322a6c7dceca374a0
2018-02-20 19:06:37 -08:00
Miao Wang
a2d04c828e Refactor NN API VTS tests and add v1.1 tests
- Create VtsHalNeuralnetworksTest_utils module to be shared between
    v1.x tests.
  - Split the existing tests into two categories: basic, and generated.
  - Created v1.1 VTS tests ensuring no regression in existing ML models.

Bug: 63911257
Test: mm
Test: NNAPI VTS tests pass on v1.0 and v1.1 sample drivers

Change-Id: Ic77c90a3a5bbd96b0ce2acd03764dde4b3034cc9
2018-02-09 18:07:37 -08:00
Michael Butler
3530f9ef0d Add README to NeuralNetworks HAL
Adds README explaining where to find the NeuralNetworks sample driver.

Test: N/A
Change-Id: Ifbbbbb3801176da27fb03bbed0b0d739031a195b
2018-02-01 13:49:49 -08:00
Michael Butler
5c6ee9ecef Create NeuralNetworks HAL v1.1 for new OperationTypes
Test: mm
Change-Id: I08efaba79ec28a2f89e94a84ab88b0fa701b7d98
2018-01-29 22:24:13 -08:00
Steven Moreland
fcca4e23b7 Remove subdirs
Removing whenever I see these in code reviews.

Test: none
Merged-In: I4322f533a837d55618ec2ed2125e8966ace9d61d
Change-Id: I4322f533a837d55618ec2ed2125e8966ace9d61d
2017-11-28 14:22:43 -08:00
Steven Moreland
8db261bc99 Updating makefiles for hidl_interface.
Bug: 64487114
Test: manual
Merged-In: Ie13d9e014cf2b81c18c67f551b4644fb9f0ba812
Change-Id: Ie13d9e014cf2b81c18c67f551b4644fb9f0ba812
2017-11-13 10:00:18 -08:00
Steven Moreland
f390f03aa4 Update for Soong-only makefiles.
Test: pass
Bug: 33420795
Change-Id: Ibec6d1e38939b9e7566a96381439be2aac3bf8ca
2017-10-11 18:30:28 +00:00
I-Jui (Ray) Sung
5bf4edfb9f Refactor test harness for generated VTS tests
Address comments in ag/2926388

Test: VtsHalNeuralnetworksV1_0TargetTest on Angler with sample driver
Bug: 67597726
Change-Id: Ie3118dcbfb11c25dbc84d2becdefbbae868ff28f
2017-10-09 20:57:42 +00:00
Jean-Luc Brouillet
90379def76 Corrected performanceInfo documentation
We'll be using relative numbers, change the doc to reflect that.

Bug: 63905942
Test: Compiled
Change-Id: Iad67621cfc76490c2dc471865a3f82cebf175440
2017-10-05 12:06:36 -07:00
TreeHugger Robot
0c7d0a9d0d Merge "Fix VTS failures on tests with optional inputs" into oc-mr1-dev 2017-10-05 06:15:59 +00:00
I-Jui (Ray) Sung
959cd78e67 Fix VTS failures on tests with optional inputs
RequestArgument for optional inputs should have its .hasNoValue set
and other fields left empty. The test harness was incorrectly ignoring
the hasNoValue field and initializing the offset field even for missing
arguments. That inconsistency was caught by the sample driver and caused
an invalid argument error. With this CL the test harness now populates
requests correctly for tests with intentionally missing inputs, and that
fixes lstm* and lsh_projection.

Test: VtsHalNeuralnetworksV1_0TargetTest on Angler
Bug: 63905942
Change-Id: I8729124b720f0d4163f4b9efd452f82b825b0885
2017-10-04 20:49:57 -07:00
Michael Butler
13cbc53a65 Early test termination for unsupported models
Some service drivers won't be able to support all operations. In those
cases, try to prepare the model anyway. If it works, proceed with the
test; if it fails, terminate the test early.

For example, the hvx nn service does not support svdf, so it should
attempt to prepare the model, but the test should not fail if the hvx
driver reports an error.

Bug: 63905942
Test: mm, expected behavior when running on hvx service
Change-Id: I51aa1911145f0d17590906c21314d2bdc24cffea
2017-10-04 18:14:45 -07:00
Michael Butler
0897ab3aad Only verify output of models a vendor nn service fully supports
Some service drivers won't be able to support all operations. In those
cases, run the test as far as it can go, but ignore the output data.

For example, the hvx nn service does not support svdf, so it should
attempt to prepare and execute the model, but the test should not fail
if the hvx driver reports an error. However, hvx indicates it is able to
run MAX_POOL_2D, so the test should fail if that output is incorrect.

Bug: 63905942
Test: mm
Test: run vts binary on hvx service
Change-Id: I3ebc05836c8e070b66daa2e38b107f96637ca812
2017-10-04 19:31:08 +00:00
TreeHugger Robot
072811f5fb Merge "Fix problem passing null optional operands." into oc-mr1-dev 2017-10-04 01:40:37 +00:00
Jean-Luc Brouillet
578b61c67e Fix problem passing null optional operands.
Allow optional operands to be specified as having no value in the model.
Make sure that both these and model arguements that have no value can
be passed to through the HAL. Change the validation to ensure that all
required ops are there.

Bug: 63905942
Test: Compiled and ran runtime/test and VTS tests.

Change-Id: Ia20954b9712042f5c0571822967521aedb66150b
2017-10-03 23:30:40 +00:00
Michael Butler
cf22a57c1a NNAPI HAL: Change IEvent to explicit callbacks
IEvent was a synchronization primitive which caused some confusion
in the interface. Originally the event object was paired with an
asynchronous task, and the asynchronous task would signal this event
when the corresponding output was ready to be used.

In the case of IDevice::prepareModel, the function call would return an
IPreparedModel object that was not guaranteed to be prepared until the
runtime had returned from waiting on the corresponding event object.
The event object has been changed to two explicit callbacks--
IPreparedModelCallback and IExecutionCallback. Now,
IDevice::prepareModel no longer returns an unfinished IPreparedModel;
instead, it will pass the IPreparedModel object to the runtime through
IPreparedModelCallback::notify. When the runtime retreives the
IPreparedModel object, the asynchronous task has already finished
preparing the model.

The two callbacks are used for different purposes. Each has its own
version of notify to pass the data back to the runtime:
* IPreparedModelCallback::notify(ErrorStatus, IPreparedModel)
* IExecutionCallback::notify(ErrorStatus)

Bug: 63905942
Test: mm, vts, ml/nn/runtime/tests
Change-Id: I0c88cd262ba762e0af15e9da31ebe813a5d150b2
2017-10-03 18:01:29 +00:00
Jean-Luc Brouillet
39ac22e908 Remove operationTuple.
Removed operationTuple from Model in the HAL, as the data type was
redundant information.

Removed	supportedOperationTuples from Capabilities, as real drivers need
more complex restrictions than this provided.  For the OC-MR1, we'll just
rely on getSupportedNodes.

Also removed the unused	cachesCompilation.

Bug: 63905942
Test: Compiled and ran tests.
Change-Id: I15f33d14634f2e1c8d726b1bd01d5b9e123b47ea
2017-09-24 00:44:11 -07:00
Michael Butler
e9a134b868 Merge "Removed WIP tag from Neuralnetworks HAL" into oc-mr1-dev 2017-09-23 03:38:47 +00:00
Michael Butler
ee3906793c Merge "Add negative tests to the Neuralnetworks HAL" into oc-mr1-dev 2017-09-23 01:07:43 +00:00
Michael Butler
926df1e182 Removed WIP tag from Neuralnetworks HAL
Additionally cleaned up some comments and formatting.

Bug: 63905942
Test: not needed
Change-Id: I0306f6d37f3c9ce9b365b5f838c27e81fa001ccd
2017-09-22 15:53:03 -07:00
I-Jui (Ray) Sung
f6b85506fe Downgrade generated test harness for NDK libc++
NDK libc++ does not yet support full C++14. This CL replaced use of
std::get with type as well as auto lambda arguments with equivalent
C++11 constructs in the VTS test harness for NNAPI.

Test: VtsHalNeuralnetworksV1_0TargetTest
Bug: 63905942
Change-Id: If75e7c088e9221a70bcc47bc647e0dd7b045bfa1
2017-09-21 12:46:13 -07:00
Michael Butler
2f499a9ba8 Add negative tests to the Neuralnetworks HAL
This CL adds negative tests to:
* IDevice::getSupportedOperations
* IDevice::prepareModel
* IPreparedModel::execute

It does not make sense to have negative tests for the remaining HAL
entries:
* IDevice::getStatus (all returned values are valid)
* IDevice::getCapabilities (no input or state)
* IEvent::notify (synchronization callback primitive)

Bug: 63905942
Test: mm, running on sample driver has expected output
Change-Id: Ia3cf5ce05dd07ff3f688395b8d0e581851909028
2017-09-20 16:18:55 -07:00
I-Jui (Ray) Sung
7d765bdd0f Support ignored output in test harness
Filtered out outputs that are not needed for correctness verification
during result validation.

Bug: 63905942
Test: VtsHalNeuralnetworksV1_0TargetTest on Angler
Change-Id: Ic967a821c2a60194c9d9f386601a865f5c409a4f
2017-09-13 20:16:47 -07:00
Miao Wang
18622e620a Merge "Add documentation to the Neuralnetworks HIDL HAL." into oc-mr1-dev 2017-09-14 02:26:37 +00:00
Michael Butler
c782110765 Add documentation to the Neuralnetworks HIDL HAL.
Bug: 63905942
Test: not needed
Change-Id: I1582d20e66ef939581d82f21088f0db0bc0f5eb7
2017-09-13 17:56:05 -07:00
Ray Sung
4385047049 Merge "Relax tolerance for FP comparison in test harness" into oc-mr1-dev 2017-09-14 00:53:27 +00:00