Commit graph

444 commits

Author SHA1 Message Date
Michael Butler
61f508e018 Cleanup how transport errors are handled in NN utils
Prior to this change, whenever the NN utility code encountered a HIDL
transport error, the error message would display the file and line
number of the "handleTransportError" function itself. This change
introduces a new macro "HANDLE_TRANSPORT_FAILURE" that handles the
transport error in a similar way but now the error message displays
the file and line number of where the macro is called.

Bug: N/A
Test: mma
Change-Id: I35b34f8f5be52b7fcff0fbb58a37ab2b8c7dd8bb
2020-12-07 22:58:39 +00:00
Michael Butler
32acc06144 Validate during NN conversions by default -- hal
This change renames all `convert` functions to `unvalidatedConvert`.
This change also introduces new `convert` functions that act only on the
types that appear in the NN HIDL methods directly. These new `convert`
functions perform validation. Specifically, if either the source or
destination value is invalid, then the conversion fails.

Bug: 160667419
Test: mma
Test: NeuralNetworksTest_static
Change-Id: I492956ff60ad1466c67893993d28cdd6f3860708
2020-12-03 21:16:07 +00:00
Slava Shklyaev
d594cd0a4d Fix a bug where 1.3 Device ignores 1.3 capabilities
Bug: 170289677
Test: NNT_static --gtest_filter=PartitioningTest.Perf
Change-Id: I476c7ab6ae807b4fd3ed27ec4d63aa6c04d0c7e1
2020-12-01 12:28:25 +00:00
Slava Shklyaev
d4290b8bf8 Replace nn::NativeHandle with nn::SharedHandle
Bug: 160669116
Test: mma
Change-Id: I73b2b93aab6cbf37d3c145e15ee9ae45228954f1
2020-11-16 14:42:28 +00:00
Michael Butler
3670c385c4 Implement NNAPI canonical interfaces
This CL implements the canonical IDevice, IPreparedModel, and IBuffer
interfaces for the 1.0, 1.1, 1.2, and 1.3 NN HIDL HAL interfaces.
Further, it introduces "Resilient" adapter interfaces to automatically
retrieve a handle to a recovered interface object after it has died and
rebooted.

This CL also updates the conversion code from returning nn::Result to
nn::GeneralResult, which includes a ErrorStatus code in the case of an
error.

Finally, this CL introduces a new static library
neuralnetworks_utils_hal_service which consists of a single function
::android::nn::hal::getDevices which can be used by the NNAPI runtime to
retrieve the HIDL services without knowing the underlying HIDL types.

Bug: 160668438
Test: mma
Test: NeuralNetworksTest_static
Change-Id: Iec6ae739df196b4034ffb35ea76781fd541ffec3
2020-11-14 16:21:04 -08:00
Michael Butler
a685c3dbf4 Create conversions to/from NNAPI canonical types
This CL creates the following primary sets of functions:
* V1_X::utils::convert(<canonical_type>) -- Converts a canonical type
  to the corresponding HAL version type.
* nn::convert(<V1_X_HAL_type>) -- Converts a HAL version type to the
  corresponding canonical type.
* neuralnetworks::utils::hasNoPointerData -- Indicates if the object
  contains no pointer-based data that could be relocated to shared
  memory.
* neuralnetworks::utils::flushDataFromPointerToShared -- Relocate
  pointer-based data to shared memory.
* neuralnetworks::utils::unflushDataFromSharedToPointer -- Undoes
  `flushDataFromPointerToShared` on a Request object. More
  specifically, `unflushDataFromSharedToPointer` copies the output
  shared memory data from the transformed Request object back to the
  output pointer-based memory in the original Request object.

It also introduces some other minor utility code, including
makeQuantized8PerformanceConsistentWithP, countNumberOfConsumers,
validate, valid, and validatedConvertToCanonical.

Bug: 160667419
Test: mma
Change-Id: I0732e658c1f4ed40cd122f1ca8581fb40b056757
2020-10-01 11:32:53 -07:00
Xusong Wang
af9ac3b8a0 Suppress gtest error for tests without any instance
NNAPI VTS is parameterized by discovered devices. When there is no
device available, the compilation caching and memory domain tests will
not be instantiated, and will cause a gtest error. This patch suppresses
the error by GTEST_ALLOW_UNINSTANTIATED_PARAMETERIZED_TEST macro.

Bug: 168152007
Test: 1.2/1.3 VTS without NNAPI driver
Change-Id: I715ae6f45df66e4a03c12b5d3f38bcc028a9ed7c
2020-09-11 10:55:36 -07:00
Xusong Wang
aab6678cb4 Merge "Increase NNAPI VTS single-test timeout to 20m." am: e551cc102e
Original change: https://android-review.googlesource.com/c/platform/hardware/interfaces/+/1380711

Change-Id: I17f3ca8aeaa66e2bcc6a1b0613e76230faea2a9a
2020-09-10 01:36:11 +00:00
Xusong Wang
7a8a6dd5af Increase NNAPI VTS single-test timeout to 20m.
SaveToCache_TOCTOU/PrepareFromCache_TOCTOU loops 100 times to generate
and reload compilation cache. Some drivers may hit the 60-second
timeout.

Bug: 160197150
Test: Add a test sleeping for 2min to NNAPI 1.2/1.3 VTS
      and run with vts-tradefed
Change-Id: Id232f4954180d8d7a9b91f8d2e4e6f2ee4640ed7
2020-09-10 00:58:10 +00:00
Xin Li
36dcf1a404 Merge Android R (rvc-dev-plus-aosp-without-vendor@6692709)
Bug: 166295507
Merged-In: I6502829205ede2de914b27e6c2c5c42916af2b39
Change-Id: I7cb06511e43bd1fffd5f80a11dbdf5b1314cfe8e
2020-08-28 13:05:04 -07:00
Sushil Nath
898088df75 Update language to comply with Android's inclusive language guidance
See https://source.android.com/setup/contribute/respectful-code for reference


BUG=162536543

Change-Id: I360c610e89aea020bbeff89d496fa8033d551916
2020-08-18 01:07:26 +00:00
Jim Pollock
9df50c6615 vts: remove deprecated gtest macros
INSTANTIATE_TEST_SUITE_P is the drop-in replacement for
INSTANTIATE_TEST_CASE_P and will enable uprev of gtest.

See
https://github.com/google/googletest/releases/tag/release-1.10.0 for
full context.

BUG=chromium:1091244
Test: mma within hardware/interfaces/neuralnetworks

Change-Id: Ibb44a06f3e9a682847d47f64676b3bb7afdde1d9
2020-08-07 13:20:09 +01:00
Dan Shi
ba4d532fee Suppress gtest error for tests without any instance
Bug: 162052785
Test: m -j vts
Change-Id: I8c1a48e6fbd7c8161137902b5332911fa0d7b8b3
2020-07-28 15:12:32 -07:00
Slava Shklyaev
d23380f0a3 Fix stale NNAPI documentation
The scale and zeroPoint fields are applicable to other types since 1.2.

Also makes some whitespaces changes due to the generated documentation
getting out of sync with the template in frameworks/ml/nn.

Fix: 160406237
Test: generate_api.sh
Test: m
Change-Id: Icf594d40c73ff8c05044c320ac9eb6a9c5a89754
Merged-In: Icf594d40c73ff8c05044c320ac9eb6a9c5a89754
(cherry picked from commit 0d6cefe90b)
2020-07-16 11:49:56 +01:00
Jooyung Han
d610435ac4 update hidl .bp
HIDL libs are not necessarily part of VNDK now. Because some are
used by VNDK libs, they are still VNDK. But rest are now just
vendor-available.

.hidl_for_test files are also removed because they are used to exclude
test-purpose hidl libs from VNDK libs.

Instead, .hidl_for_system_ext files are added to tests/lazy to
distinguish them from others which are installed /system.

Bug: 143933769
Test: update-makefiles.sh && m com.android.vndk.current
Merged-In: Ia81312dda340b6b5cbdd7a3c21e1d323bda39a4a
Change-Id: Ia81312dda340b6b5cbdd7a3c21e1d323bda39a4a
(cherry picked from commit b0907a6bb8)
2020-07-07 16:22:36 +09:00
Jooyung Han
d18cba85fe update hidl .bp
HIDL libs are not necessarily part of VNDK now. Because some are
used by VNDK libs, they are still VNDK. But rest are now just
vendor-available.

.hidl_for_test files are also removed because they are used to exclude
test-purpose hidl libs from VNDK libs.

Instead, .hidl_for_system_ext files are added to tests/lazy to
distinguish them from others which are installed /system.

Bug: 143933769
Test: update-makefiles.sh && m com.android.vndk.current
Merged-In: Ia81312dda340b6b5cbdd7a3c21e1d323bda39a4a
Change-Id: Ia81312dda340b6b5cbdd7a3c21e1d323bda39a4a
(cherry picked from commit b0907a6bb8)
2020-07-02 12:26:37 +09:00
Matej Pfajfar
3468366623 Change 'blacklist' to 'disallow' am: 26f365b810 am: 2d0bac77ab
Original change: https://android-review.googlesource.com/c/platform/hardware/interfaces/+/1351587

Change-Id: I4e7fec2b1823cbb7e974be59804cccc38d13614d
2020-06-29 23:05:00 +00:00
Matej Pfajfar
26f365b810 Change 'blacklist' to 'disallow'
Bug: None
Test: mma
Test: atest
Change-Id: I6d1bc4f1cd95a56287127918f4c8111cafd6e833
Merged-In: I6d1bc4f1cd95a56287127918f4c8111cafd6e833
(cherry picked from commit 1bdd58dcf6)
2020-06-29 13:24:11 +01:00
Xusong Wang
3dca799497 Revert "Allow compilations in TOCTOU tests to fail with GENERAL_FAILURE."
This reverts commit 362dfd64d5.

Bug: 157489048
Test: 1.2/1.3 VTS TOCTOU tests
Change-Id: Ifdf4a8fab129cd90a3be019c3c598d980b785aaa
Merged-In: Ifdf4a8fab129cd90a3be019c3c598d980b785aaa
(cherry picked from commit 81fa7605cf)
2020-06-26 15:26:47 +01:00
Miao Wang
29a1225561 Remove temporary test filters
Bug: 155577050
Test: mm
Change-Id: Ibe8a3ed79f640249ba54f94bdde9649f156383ea
Merged-In: Ibe8a3ed79f640249ba54f94bdde9649f156383ea
(cherry picked from commit eeb0068015)
2020-06-26 15:26:44 +01:00
Xusong Wang
0b617ae022 Allow compilations in TOCTOU tests to fail with GENERAL_FAILURE.
A compilation failure is not related to the security aspect of the
TOCTOU test, but it will skip one iteration of security testing. This CL
allows the compilation to fail with GENERAL_FAILURE in TOCTOU tests, and
issues a retry once it happens to ensure enough test coverage.

Bug: 157489048
Test: 1.2/1.3 VTS
Change-Id: Idc88e0365c5d2799187093b6fd7b4abf8f8b463d
Merged-In: Idc88e0365c5d2799187093b6fd7b4abf8f8b463d
(cherry picked from commit 362dfd64d5)
2020-06-26 15:26:40 +01:00
Xusong Wang
12f483fdfd Merge "Revert "Allow compilations in TOCTOU tests to fail with GENERAL_FAILURE."" into rvc-dev 2020-06-23 00:53:05 +00:00
Xusong Wang
81fa7605cf Revert "Allow compilations in TOCTOU tests to fail with GENERAL_FAILURE."
This reverts commit 362dfd64d5.

Bug: 157489048
Test: 1.2/1.3 VTS TOCTOU tests
Change-Id: Ifdf4a8fab129cd90a3be019c3c598d980b785aaa
2020-06-22 18:36:07 +00:00
Slava Shklyaev
3722d6dcfa Log NNAPI VTS test case to logcat
Sample output:

    I VtsHalNeuralnetworksV1_2TargetTest: [Test Case] TestGenerated/GeneratedTest.Test/nnapi_sample_all_add BEGIN
    ...
    I VtsHalNeuralnetworksV1_2TargetTest: [Test Case] TestGenerated/GeneratedTest.Test/nnapi_sample_all_add END

Bug: 159060565
Test: VtsHalNeuralnetworksV1_0TargetTest (inspected logcat output)
Test: VtsHalNeuralnetworksV1_1TargetTest (inspected logcat output)
Test: VtsHalNeuralnetworksV1_2TargetTest (inspected logcat output)
Test: VtsHalNeuralnetworksV1_3TargetTest (inspected logcat output)
Change-Id: Ic1a0c90e0853ca84dd925d0f7df65e159bb284aa
Merged-In: Ic1a0c90e0853ca84dd925d0f7df65e159bb284aa
(cherry picked from commit 8c8efd68e8)
2020-06-17 13:48:25 +01:00
Slava Shklyaev
a6d65ad073 Merge "Log NNAPI VTS test case to logcat" into rvc-dev 2020-06-17 12:08:45 +00:00
Slava Shklyaev
8c8efd68e8 Log NNAPI VTS test case to logcat
Sample output:

    I VtsHalNeuralnetworksV1_2TargetTest: [Test Case] TestGenerated/GeneratedTest.Test/nnapi_sample_all_add BEGIN
    ...
    I VtsHalNeuralnetworksV1_2TargetTest: [Test Case] TestGenerated/GeneratedTest.Test/nnapi_sample_all_add END

Bug: 159060565
Test: VtsHalNeuralnetworksV1_0TargetTest (inspected logcat output)
Test: VtsHalNeuralnetworksV1_1TargetTest (inspected logcat output)
Test: VtsHalNeuralnetworksV1_2TargetTest (inspected logcat output)
Test: VtsHalNeuralnetworksV1_3TargetTest (inspected logcat output)
Change-Id: Ic1a0c90e0853ca84dd925d0f7df65e159bb284aa
2020-06-17 12:10:00 +01:00
TreeHugger Robot
c55a62bc79 Merge "Remove temporary test filters" into rvc-dev 2020-06-16 23:15:47 +00:00
Miao Wang
eeb0068015 Remove temporary test filters
Bug: 155577050
Test: mm
Change-Id: Ibe8a3ed79f640249ba54f94bdde9649f156383ea
2020-06-16 15:31:24 +00:00
Xusong Wang
362dfd64d5 Allow compilations in TOCTOU tests to fail with GENERAL_FAILURE.
A compilation failure is not related to the security aspect of the
TOCTOU test, but it will skip one iteration of security testing. This CL
allows the compilation to fail with GENERAL_FAILURE in TOCTOU tests, and
issues a retry once it happens to ensure enough test coverage.

Bug: 157489048
Test: 1.2/1.3 VTS
Change-Id: Idc88e0365c5d2799187093b6fd7b4abf8f8b463d
2020-06-15 13:22:27 -07:00
Lev Proleev
d90a5de791 NNAPI: Clarify behaviour of ops that reduce dimensions
Clarify behaviour of the following ops in the case when inputs would
cause an output to have an empty shape:
* MEAN
* ARGMIN/ARGMAX
* STRIDED_SLICE

Bug: 155508675
Bug: 155660285
Bug: 155508675
Bug: 155238914
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I92d1c3866a462a99b4eed7782b72f62bb6a076c6
Merged-In: I92d1c3866a462a99b4eed7782b72f62bb6a076c6
(cherry picked from commit d4bba83f6a)
2020-05-19 13:39:26 -07:00
Lev Proleev
4b2917a5ed NNAPI: Clarify behaviour of REDUCE_* ops
Add a clarification on the behaviour of REDUCE_* ops when all dimensions
of an input tensor are reduced.

Bug: 155508675
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I3a67225c3af80af40a11362ca5d0f55350cbfa05
Merged-In: I3a67225c3af80af40a11362ca5d0f55350cbfa05
(cherry picked from commit eec7d86b44)
2020-05-19 13:38:00 -07:00
Miao Wang
e1c1e7e2f6 Filter out failing NNAPI validation tests
Bug: 155577050
Test: atest VtsHalNeuralnetworksV1_*TargetTest
Change-Id: Id8d497dddd5521670689ff2ea08610c3aaeaf748
Merged-In: Id8d497dddd5521670689ff2ea08610c3aaeaf748
(cherry picked from commit 4429d856cd)
2020-05-19 13:35:55 -07:00
Miao Wang
398adea8b6 Add AndroidTest.xml for NNAPI 1.2 VTS and blacklist failing tests
- Also adds TEST_MAPPING filters here to make sure sample driver tests
  are still being filtered out.

Bug: 155674368
Test: mm
Test: atest VtsHalNeuralnetworksV1_2TargetTest
Change-Id: I5f5d272742d4cdd6a8cc87ec035b5e2508eaa98f
Merged-In: I5f5d272742d4cdd6a8cc87ec035b5e2508eaa98f
(cherry picked from commit f7c55d5f9b)
2020-05-19 13:35:29 -07:00
Lev Proleev
614ba3ef2e Merge "NNAPI: Clarify behaviour of ops that reduce dimensions" into rvc-dev 2020-05-18 15:31:13 +00:00
Lev Proleev
74b04e9649 Merge "NNAPI: Clarify behaviour of REDUCE_* ops" into rvc-dev 2020-05-15 10:28:39 +00:00
Lev Proleev
d4bba83f6a NNAPI: Clarify behaviour of ops that reduce dimensions
Clarify behaviour of the following ops in the case when inputs would
cause an output to have an empty shape:
* MEAN
* ARGMIN/ARGMAX
* STRIDED_SLICE

Bug: 155508675
Bug: 155660285
Bug: 155508675
Bug: 155238914
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I92d1c3866a462a99b4eed7782b72f62bb6a076c6
2020-05-13 21:57:58 +01:00
Lev Proleev
eec7d86b44 NNAPI: Clarify behaviour of REDUCE_* ops
Add a clarification on the behaviour of REDUCE_* ops when all dimensions
of an input tensor are reduced.

Bug: 155508675
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I3a67225c3af80af40a11362ca5d0f55350cbfa05
2020-05-13 15:00:32 +01:00
Miao Wang
4429d856cd Filter out failing NNAPI validation tests
Bug: 155577050
Test: atest VtsHalNeuralnetworksV1_*TargetTest
Change-Id: Id8d497dddd5521670689ff2ea08610c3aaeaf748
2020-05-10 19:38:17 -07:00
Miao Wang
f7c55d5f9b Add AndroidTest.xml for NNAPI 1.2 VTS and blacklist failing tests
- Also adds TEST_MAPPING filters here to make sure sample driver tests
  are still being filtered out.

Bug: 155674368
Test: mm
Test: atest VtsHalNeuralnetworksV1_2TargetTest
Change-Id: I5f5d272742d4cdd6a8cc87ec035b5e2508eaa98f
2020-05-04 20:10:48 -07:00
David Gross
6174f00cc6 More tests for graph validation.
- detect cycle (CycleTest)
- detect bad execution order (mutateExecutionOrderTest)
- detect lifetime inconsistent with whether operand is written (mutateOperandLifeTimeTest)
- detect lifetime inconsistent with Model inputIndexes/outputIndexes (mutateOperandInputOutputTest)
- detect incorrect number of consumers (mutateOperandNumberOfConsumersTest)
- detect operand written multiple times (mutateOperandAddWriterTest)
- detect operand never written (mutateOperationRemoveWriteTest)

Bug: 66478689
Test: VtsHalNeuralnetworksV1_*TargetTest

Change-Id: Id4ba19660bbd31a16f8a675f7b6437f4d779e8da
Merged-In: Id4ba19660bbd31a16f8a675f7b6437f4d779e8da
(cherry picked from commit af51663e99)
2020-05-04 17:29:52 -07:00
David Gross
12ab6c57cc More tests for graph validation.
- detect cycle (CycleTest)
- detect bad execution order (mutateExecutionOrderTest)
- detect lifetime inconsistent with whether operand is written (mutateOperandLifeTimeTest)
- detect lifetime inconsistent with Model inputIndexes/outputIndexes (mutateOperandInputOutputTest)
- detect incorrect number of consumers (mutateOperandNumberOfConsumersTest)
- detect operand written multiple times (mutateOperandAddWriterTest)
- detect operand never written (mutateOperationRemoveWriteTest)

Bug: 66478689
Test: VtsHalNeuralnetworksV1_*TargetTest

Change-Id: Id4ba19660bbd31a16f8a675f7b6437f4d779e8da
Merged-In: Id4ba19660bbd31a16f8a675f7b6437f4d779e8da
(cherry picked from commit af51663e99)
2020-05-02 02:34:30 +00:00
Slava Shklyaev
17a0b4fcdc Merge changes I866fe138,I0096a2fc
* changes:
  Remove BLOB AHWB tests from 1.2 VTS.
  Update loopTimeoutDuration documentation in NNAPI
2020-04-16 12:45:32 +00:00
Miao Wang
6048fd9dd5 Update the specification for the following operations
- RANDOM_MULTINOMIAL
 - L2_NORMALIZATION

Bug: 136279892
Bug: 140177375
Test: mm
Change-Id: Iab38ef29ebf6d1f5c0a408436b3d564e45e537a0
Merged-In: Iab38ef29ebf6d1f5c0a408436b3d564e45e537a0
(cherry picked from commit 077c0339d4)
2020-04-14 19:29:51 +01:00
Xusong Wang
b79c31aba8 Remove BLOB AHWB tests from 1.2 VTS.
Bug: 153282844
Test: 1.2 VTS
Change-Id: I866fe1388a64a36372f90c60fef340d2fe4c8682
Merged-In: I866fe1388a64a36372f90c60fef340d2fe4c8682
(cherry picked from commit b642700c7f)
2020-04-14 16:30:01 +01:00
Xusong Wang
d923a3f06f Merge "Remove BLOB AHWB tests from 1.2 VTS." into rvc-dev 2020-04-10 17:44:43 +00:00
Xusong Wang
b642700c7f Remove BLOB AHWB tests from 1.2 VTS.
Bug: 153282844
Test: 1.2 VTS
Change-Id: I866fe1388a64a36372f90c60fef340d2fe4c8682
2020-04-09 14:26:54 -07:00
Dan Shi
aad51fa000 Rename vts-core to vts
Bug: 151896491
Test: local build
Exempt-From-Owner-Approval: This CL update suite name vts-core to vts as
the suite name is updated. This CL won't change test logic or behavior.

Change-Id: I562b4dc50765e953800a814a8fd84a01c1b9352b
Merged-In: I562b4dc50765e953800a814a8fd84a01c1b9352b
2020-04-07 16:37:40 -07:00
Dan Shi
ba894f81db Rename vts-core to vts
Bug: 151896491
Test: local build
Exempt-From-Owner-Approval: This CL update suite name vts-core to vts as
the suite name is updated. This CL won't change test logic or behavior.

Change-Id: I562b4dc50765e953800a814a8fd84a01c1b9352b
Merged-In: I562b4dc50765e953800a814a8fd84a01c1b9352b
2020-04-07 15:17:02 -07:00
Michael Butler
b1865b6593 Remove extra tests from NNAPI VTS validation tests
Do not run validation on "inputs_as_internal" and
"all_tensors_as_inputs" variants.

Bug: 138149072
Bug: 149840439
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I0699ed6703e48b6c4bc0e7a392b79c12770f04c9
Merged-In: I0699ed6703e48b6c4bc0e7a392b79c12770f04c9
(cherry picked from commit 678a10600c)
2020-03-27 11:44:10 +00:00
Xusong Wang
75e63ad743 Add BLOB AHWB tests in VTS.
Bug: 149847930
Test: 1.3 VTS
Change-Id: I9c795dcb7696c843afd12551927463c5529a4b60
Merged-In: I9c795dcb7696c843afd12551927463c5529a4b60
(cherry picked from commit 41adc5bc11)
2020-03-27 11:44:06 +00:00