Commit graph

726 commits

Author SHA1 Message Date
Dan Shi
e5c9b5e722 Suppress gtest error for tests without any instance am: ba4d532fee am: 73122cd284
Original change: https://android-review.googlesource.com/c/platform/hardware/interfaces/+/1374050

Change-Id: Ie5e956d4613f5e947c36ba4f1048f0c3aa8944ba
2020-07-29 05:45:58 +00:00
Dan Shi
ba4d532fee Suppress gtest error for tests without any instance
Bug: 162052785
Test: m -j vts
Change-Id: I8c1a48e6fbd7c8161137902b5332911fa0d7b8b3
2020-07-28 15:12:32 -07:00
Slava Shklyaev
886f321139 Fix typo in NNAPI documentation
Fix: 160960007
Test: m
Change-Id: I6c1af92f640fde833d2d93f7db4576c92a6f9719
Merged-In: I6c1af92f640fde833d2d93f7db4576c92a6f9719
(cherry picked from commit 3bbc2aedd0)
2020-07-20 13:47:50 +01:00
Slava Shklyaev
456f6d7c91 Fix stale NNAPI documentation
The scale and zeroPoint fields are applicable to other types since 1.2.

Also makes some whitespaces changes due to the generated documentation
getting out of sync with the template in frameworks/ml/nn.

Fix: 160406237
Test: generate_api.sh
Test: m
Change-Id: Icf594d40c73ff8c05044c320ac9eb6a9c5a89754
Merged-In: Icf594d40c73ff8c05044c320ac9eb6a9c5a89754
(cherry picked from commit 0d6cefe90b)
2020-07-20 13:47:27 +01:00
Slava Shklyaev
a2fbeefaa8 Fix typo in NNAPI documentation
Fix: 160960007
Test: m
Change-Id: I6c1af92f640fde833d2d93f7db4576c92a6f9719
Merged-In: I6c1af92f640fde833d2d93f7db4576c92a6f9719
(cherry picked from commit 3bbc2aedd0)
2020-07-16 11:50:23 +01:00
Slava Shklyaev
d23380f0a3 Fix stale NNAPI documentation
The scale and zeroPoint fields are applicable to other types since 1.2.

Also makes some whitespaces changes due to the generated documentation
getting out of sync with the template in frameworks/ml/nn.

Fix: 160406237
Test: generate_api.sh
Test: m
Change-Id: Icf594d40c73ff8c05044c320ac9eb6a9c5a89754
Merged-In: Icf594d40c73ff8c05044c320ac9eb6a9c5a89754
(cherry picked from commit 0d6cefe90b)
2020-07-16 11:49:56 +01:00
Jooyung Han
d610435ac4 update hidl .bp
HIDL libs are not necessarily part of VNDK now. Because some are
used by VNDK libs, they are still VNDK. But rest are now just
vendor-available.

.hidl_for_test files are also removed because they are used to exclude
test-purpose hidl libs from VNDK libs.

Instead, .hidl_for_system_ext files are added to tests/lazy to
distinguish them from others which are installed /system.

Bug: 143933769
Test: update-makefiles.sh && m com.android.vndk.current
Merged-In: Ia81312dda340b6b5cbdd7a3c21e1d323bda39a4a
Change-Id: Ia81312dda340b6b5cbdd7a3c21e1d323bda39a4a
(cherry picked from commit b0907a6bb8)
2020-07-07 16:22:36 +09:00
Treehugger Robot
348df54c43 Merge "Clarify NNAPI control flow shape constraints RESTRICT AUTOMERGE" 2020-07-06 17:49:45 +00:00
Jooyung Han
d18cba85fe update hidl .bp
HIDL libs are not necessarily part of VNDK now. Because some are
used by VNDK libs, they are still VNDK. But rest are now just
vendor-available.

.hidl_for_test files are also removed because they are used to exclude
test-purpose hidl libs from VNDK libs.

Instead, .hidl_for_system_ext files are added to tests/lazy to
distinguish them from others which are installed /system.

Bug: 143933769
Test: update-makefiles.sh && m com.android.vndk.current
Merged-In: Ia81312dda340b6b5cbdd7a3c21e1d323bda39a4a
Change-Id: Ia81312dda340b6b5cbdd7a3c21e1d323bda39a4a
(cherry picked from commit b0907a6bb8)
2020-07-02 12:26:37 +09:00
Matej Pfajfar
3468366623 Change 'blacklist' to 'disallow' am: 26f365b810 am: 2d0bac77ab
Original change: https://android-review.googlesource.com/c/platform/hardware/interfaces/+/1351587

Change-Id: I4e7fec2b1823cbb7e974be59804cccc38d13614d
2020-06-29 23:05:00 +00:00
Slava Shklyaev
dfe1525859 Clarify NNAPI control flow shape constraints RESTRICT AUTOMERGE
This change also removes some non-NN hashes in current.txt that were
unintentionally introduced in https://r.android.com/1178665.

Bug: 156918813
Bug: 158557728
Test: m
Change-Id: I4c2b83e11b27b791b3aa624be68474011b92a269
Merged-In: I4c2b83e11b27b791b3aa624be68474011b92a269
(cherry picked from commit 709842ab98)
2020-06-29 15:31:22 +01:00
Matej Pfajfar
26f365b810 Change 'blacklist' to 'disallow'
Bug: None
Test: mma
Test: atest
Change-Id: I6d1bc4f1cd95a56287127918f4c8111cafd6e833
Merged-In: I6d1bc4f1cd95a56287127918f4c8111cafd6e833
(cherry picked from commit 1bdd58dcf6)
2020-06-29 13:24:11 +01:00
Xusong Wang
3dca799497 Revert "Allow compilations in TOCTOU tests to fail with GENERAL_FAILURE."
This reverts commit 362dfd64d5.

Bug: 157489048
Test: 1.2/1.3 VTS TOCTOU tests
Change-Id: Ifdf4a8fab129cd90a3be019c3c598d980b785aaa
Merged-In: Ifdf4a8fab129cd90a3be019c3c598d980b785aaa
(cherry picked from commit 81fa7605cf)
2020-06-26 15:26:47 +01:00
Miao Wang
29a1225561 Remove temporary test filters
Bug: 155577050
Test: mm
Change-Id: Ibe8a3ed79f640249ba54f94bdde9649f156383ea
Merged-In: Ibe8a3ed79f640249ba54f94bdde9649f156383ea
(cherry picked from commit eeb0068015)
2020-06-26 15:26:44 +01:00
Xusong Wang
0b617ae022 Allow compilations in TOCTOU tests to fail with GENERAL_FAILURE.
A compilation failure is not related to the security aspect of the
TOCTOU test, but it will skip one iteration of security testing. This CL
allows the compilation to fail with GENERAL_FAILURE in TOCTOU tests, and
issues a retry once it happens to ensure enough test coverage.

Bug: 157489048
Test: 1.2/1.3 VTS
Change-Id: Idc88e0365c5d2799187093b6fd7b4abf8f8b463d
Merged-In: Idc88e0365c5d2799187093b6fd7b4abf8f8b463d
(cherry picked from commit 362dfd64d5)
2020-06-26 15:26:40 +01:00
TreeHugger Robot
36e9e40436 Merge "Clarify NNAPI control flow shape constraints" into rvc-dev 2020-06-26 12:07:06 +00:00
Slava Shklyaev
709842ab98 Clarify NNAPI control flow shape constraints
Bug: 156918813
Bug: 158557728
Test: m
Change-Id: I4c2b83e11b27b791b3aa624be68474011b92a269
2020-06-23 13:08:05 +01:00
Xusong Wang
12f483fdfd Merge "Revert "Allow compilations in TOCTOU tests to fail with GENERAL_FAILURE."" into rvc-dev 2020-06-23 00:53:05 +00:00
Xusong Wang
81fa7605cf Revert "Allow compilations in TOCTOU tests to fail with GENERAL_FAILURE."
This reverts commit 362dfd64d5.

Bug: 157489048
Test: 1.2/1.3 VTS TOCTOU tests
Change-Id: Ifdf4a8fab129cd90a3be019c3c598d980b785aaa
2020-06-22 18:36:07 +00:00
Slava Shklyaev
3722d6dcfa Log NNAPI VTS test case to logcat
Sample output:

    I VtsHalNeuralnetworksV1_2TargetTest: [Test Case] TestGenerated/GeneratedTest.Test/nnapi_sample_all_add BEGIN
    ...
    I VtsHalNeuralnetworksV1_2TargetTest: [Test Case] TestGenerated/GeneratedTest.Test/nnapi_sample_all_add END

Bug: 159060565
Test: VtsHalNeuralnetworksV1_0TargetTest (inspected logcat output)
Test: VtsHalNeuralnetworksV1_1TargetTest (inspected logcat output)
Test: VtsHalNeuralnetworksV1_2TargetTest (inspected logcat output)
Test: VtsHalNeuralnetworksV1_3TargetTest (inspected logcat output)
Change-Id: Ic1a0c90e0853ca84dd925d0f7df65e159bb284aa
Merged-In: Ic1a0c90e0853ca84dd925d0f7df65e159bb284aa
(cherry picked from commit 8c8efd68e8)
2020-06-17 13:48:25 +01:00
Slava Shklyaev
a6d65ad073 Merge "Log NNAPI VTS test case to logcat" into rvc-dev 2020-06-17 12:08:45 +00:00
Slava Shklyaev
8c8efd68e8 Log NNAPI VTS test case to logcat
Sample output:

    I VtsHalNeuralnetworksV1_2TargetTest: [Test Case] TestGenerated/GeneratedTest.Test/nnapi_sample_all_add BEGIN
    ...
    I VtsHalNeuralnetworksV1_2TargetTest: [Test Case] TestGenerated/GeneratedTest.Test/nnapi_sample_all_add END

Bug: 159060565
Test: VtsHalNeuralnetworksV1_0TargetTest (inspected logcat output)
Test: VtsHalNeuralnetworksV1_1TargetTest (inspected logcat output)
Test: VtsHalNeuralnetworksV1_2TargetTest (inspected logcat output)
Test: VtsHalNeuralnetworksV1_3TargetTest (inspected logcat output)
Change-Id: Ic1a0c90e0853ca84dd925d0f7df65e159bb284aa
2020-06-17 12:10:00 +01:00
TreeHugger Robot
c55a62bc79 Merge "Remove temporary test filters" into rvc-dev 2020-06-16 23:15:47 +00:00
Miao Wang
eeb0068015 Remove temporary test filters
Bug: 155577050
Test: mm
Change-Id: Ibe8a3ed79f640249ba54f94bdde9649f156383ea
2020-06-16 15:31:24 +00:00
Xusong Wang
362dfd64d5 Allow compilations in TOCTOU tests to fail with GENERAL_FAILURE.
A compilation failure is not related to the security aspect of the
TOCTOU test, but it will skip one iteration of security testing. This CL
allows the compilation to fail with GENERAL_FAILURE in TOCTOU tests, and
issues a retry once it happens to ensure enough test coverage.

Bug: 157489048
Test: 1.2/1.3 VTS
Change-Id: Idc88e0365c5d2799187093b6fd7b4abf8f8b463d
2020-06-15 13:22:27 -07:00
Xusong Wang
91521ab86b Add checks of output shapes when OUTPUT_INSUFFICIENT_SIZE.
Add checks that all returned output dimensions must be at
least as fully specified as the union of the information about the
corresponding operand in the model and in the request.

Bug: 154054474
Test: VTS
Change-Id: I934d084c7665160a98da9828604ce8297fef73b8
Merged-In: I934d084c7665160a98da9828604ce8297fef73b8
(cherry picked from commit d454751e00)
2020-06-02 16:52:56 -07:00
Xusong Wang
d454751e00 Add checks of output shapes when OUTPUT_INSUFFICIENT_SIZE.
Add checks that all returned output dimensions must be at
least as fully specified as the union of the information about the
corresponding operand in the model and in the request.

Bug: 154054474
Test: VTS
Change-Id: I934d084c7665160a98da9828604ce8297fef73b8
2020-05-26 15:08:12 -07:00
Lev Proleev
d90a5de791 NNAPI: Clarify behaviour of ops that reduce dimensions
Clarify behaviour of the following ops in the case when inputs would
cause an output to have an empty shape:
* MEAN
* ARGMIN/ARGMAX
* STRIDED_SLICE

Bug: 155508675
Bug: 155660285
Bug: 155508675
Bug: 155238914
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I92d1c3866a462a99b4eed7782b72f62bb6a076c6
Merged-In: I92d1c3866a462a99b4eed7782b72f62bb6a076c6
(cherry picked from commit d4bba83f6a)
2020-05-19 13:39:26 -07:00
Miao Wang
4749010554 Add AndroidTest.xml for NNAPI 1.3 VTS and temporarily blacklist failing tests
- Also adds TEST_MAPPING filters here to make sure sample driver tests
  are still being filtered out.
  - There may still be failing tests, but should not cause OOM in test harness.

Bug: 156691406
Test: mm
Test: atest VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ie05350d71dede04967d94dad4147bd3ed3435f7d
Merged-In: Ie05350d71dede04967d94dad4147bd3ed3435f7d
(cherry picked from commit 338acf5c15)
2020-05-19 13:38:28 -07:00
Lev Proleev
4b2917a5ed NNAPI: Clarify behaviour of REDUCE_* ops
Add a clarification on the behaviour of REDUCE_* ops when all dimensions
of an input tensor are reduced.

Bug: 155508675
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I3a67225c3af80af40a11362ca5d0f55350cbfa05
Merged-In: I3a67225c3af80af40a11362ca5d0f55350cbfa05
(cherry picked from commit eec7d86b44)
2020-05-19 13:38:00 -07:00
Miao Wang
e1c1e7e2f6 Filter out failing NNAPI validation tests
Bug: 155577050
Test: atest VtsHalNeuralnetworksV1_*TargetTest
Change-Id: Id8d497dddd5521670689ff2ea08610c3aaeaf748
Merged-In: Id8d497dddd5521670689ff2ea08610c3aaeaf748
(cherry picked from commit 4429d856cd)
2020-05-19 13:35:55 -07:00
Miao Wang
f904bb11d2 Add AndroidTest.xml for NNAPI 1.0 VTS and blacklist failing tests
- Also adds TEST_MAPPING filters here to make sure sample driver tests
  are still being filtered out.

Bug: 155577050
Test: mm
Test: atest VtsHalNeuralnetworksV1_0TargetTest
Change-Id: Ic8f556a957e6af340d500e2c4696b41b2a6e8516
Merged-In: Ic8f556a957e6af340d500e2c4696b41b2a6e8516
(cherry picked from commit 8ebe2a04c5)
2020-05-19 13:35:42 -07:00
Miao Wang
398adea8b6 Add AndroidTest.xml for NNAPI 1.2 VTS and blacklist failing tests
- Also adds TEST_MAPPING filters here to make sure sample driver tests
  are still being filtered out.

Bug: 155674368
Test: mm
Test: atest VtsHalNeuralnetworksV1_2TargetTest
Change-Id: I5f5d272742d4cdd6a8cc87ec035b5e2508eaa98f
Merged-In: I5f5d272742d4cdd6a8cc87ec035b5e2508eaa98f
(cherry picked from commit f7c55d5f9b)
2020-05-19 13:35:29 -07:00
Lev Proleev
614ba3ef2e Merge "NNAPI: Clarify behaviour of ops that reduce dimensions" into rvc-dev 2020-05-18 15:31:13 +00:00
Miao Wang
f118c86ba2 Merge "Add AndroidTest.xml for NNAPI 1.3 VTS and temporarily blacklist failing tests" into rvc-dev 2020-05-15 22:33:33 +00:00
Miao Wang
338acf5c15 Add AndroidTest.xml for NNAPI 1.3 VTS and temporarily blacklist failing tests
- Also adds TEST_MAPPING filters here to make sure sample driver tests
  are still being filtered out.
  - There may still be failing tests, but should not cause OOM in test harness.

Bug: 156691406
Test: mm
Test: atest VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ie05350d71dede04967d94dad4147bd3ed3435f7d
2020-05-15 13:46:13 -07:00
Lev Proleev
74b04e9649 Merge "NNAPI: Clarify behaviour of REDUCE_* ops" into rvc-dev 2020-05-15 10:28:39 +00:00
Lev Proleev
d4bba83f6a NNAPI: Clarify behaviour of ops that reduce dimensions
Clarify behaviour of the following ops in the case when inputs would
cause an output to have an empty shape:
* MEAN
* ARGMIN/ARGMAX
* STRIDED_SLICE

Bug: 155508675
Bug: 155660285
Bug: 155508675
Bug: 155238914
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I92d1c3866a462a99b4eed7782b72f62bb6a076c6
2020-05-13 21:57:58 +01:00
Lev Proleev
eec7d86b44 NNAPI: Clarify behaviour of REDUCE_* ops
Add a clarification on the behaviour of REDUCE_* ops when all dimensions
of an input tensor are reduced.

Bug: 155508675
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I3a67225c3af80af40a11362ca5d0f55350cbfa05
2020-05-13 15:00:32 +01:00
Miao Wang
4429d856cd Filter out failing NNAPI validation tests
Bug: 155577050
Test: atest VtsHalNeuralnetworksV1_*TargetTest
Change-Id: Id8d497dddd5521670689ff2ea08610c3aaeaf748
2020-05-10 19:38:17 -07:00
Miao Wang
8ebe2a04c5 Add AndroidTest.xml for NNAPI 1.0 VTS and blacklist failing tests
- Also adds TEST_MAPPING filters here to make sure sample driver tests
  are still being filtered out.

Bug: 155577050
Test: mm
Test: atest VtsHalNeuralnetworksV1_0TargetTest
Change-Id: Ic8f556a957e6af340d500e2c4696b41b2a6e8516
2020-05-05 15:28:57 -07:00
Miao Wang
f7c55d5f9b Add AndroidTest.xml for NNAPI 1.2 VTS and blacklist failing tests
- Also adds TEST_MAPPING filters here to make sure sample driver tests
  are still being filtered out.

Bug: 155674368
Test: mm
Test: atest VtsHalNeuralnetworksV1_2TargetTest
Change-Id: I5f5d272742d4cdd6a8cc87ec035b5e2508eaa98f
2020-05-04 20:10:48 -07:00
David Gross
6174f00cc6 More tests for graph validation.
- detect cycle (CycleTest)
- detect bad execution order (mutateExecutionOrderTest)
- detect lifetime inconsistent with whether operand is written (mutateOperandLifeTimeTest)
- detect lifetime inconsistent with Model inputIndexes/outputIndexes (mutateOperandInputOutputTest)
- detect incorrect number of consumers (mutateOperandNumberOfConsumersTest)
- detect operand written multiple times (mutateOperandAddWriterTest)
- detect operand never written (mutateOperationRemoveWriteTest)

Bug: 66478689
Test: VtsHalNeuralnetworksV1_*TargetTest

Change-Id: Id4ba19660bbd31a16f8a675f7b6437f4d779e8da
Merged-In: Id4ba19660bbd31a16f8a675f7b6437f4d779e8da
(cherry picked from commit af51663e99)
2020-05-04 17:29:52 -07:00
David Gross
12ab6c57cc More tests for graph validation.
- detect cycle (CycleTest)
- detect bad execution order (mutateExecutionOrderTest)
- detect lifetime inconsistent with whether operand is written (mutateOperandLifeTimeTest)
- detect lifetime inconsistent with Model inputIndexes/outputIndexes (mutateOperandInputOutputTest)
- detect incorrect number of consumers (mutateOperandNumberOfConsumersTest)
- detect operand written multiple times (mutateOperandAddWriterTest)
- detect operand never written (mutateOperationRemoveWriteTest)

Bug: 66478689
Test: VtsHalNeuralnetworksV1_*TargetTest

Change-Id: Id4ba19660bbd31a16f8a675f7b6437f4d779e8da
Merged-In: Id4ba19660bbd31a16f8a675f7b6437f4d779e8da
(cherry picked from commit af51663e99)
2020-05-02 02:34:30 +00:00
Slava Shklyaev
af1c77d74f Exclude unused operands from removeOperandTest
Bug: 148208229
Test: TestGenerated/ValidationTest.Test/nnapi_sample*_while*_unused_output*
Change-Id: I39bebefd0e40d370882d614d4beb97ab36c9da8f
Merged-In: I39bebefd0e40d370882d614d4beb97ab36c9da8f
(cherry picked from commit baa4cf02f2)
2020-04-20 13:19:00 +01:00
Slava Shklyaev
baa4cf02f2 Exclude unused operands from removeOperandTest
Bug: 148208229
Test: TestGenerated/ValidationTest.Test/nnapi_sample*_while*_unused_output*
Change-Id: I39bebefd0e40d370882d614d4beb97ab36c9da8f
2020-04-16 15:03:26 +01:00
Slava Shklyaev
17a0b4fcdc Merge changes I866fe138,I0096a2fc
* changes:
  Remove BLOB AHWB tests from 1.2 VTS.
  Update loopTimeoutDuration documentation in NNAPI
2020-04-16 12:45:32 +00:00
Miao Wang
6048fd9dd5 Update the specification for the following operations
- RANDOM_MULTINOMIAL
 - L2_NORMALIZATION

Bug: 136279892
Bug: 140177375
Test: mm
Change-Id: Iab38ef29ebf6d1f5c0a408436b3d564e45e537a0
Merged-In: Iab38ef29ebf6d1f5c0a408436b3d564e45e537a0
(cherry picked from commit 077c0339d4)
2020-04-14 19:29:51 +01:00
Lev Proleev
f0401fe0b0 Clarify supported ranks for new ops in R
Fix: 151108533
Test: mma
Change-Id: I85b66ab5940d09b2786ecb14bff5bc1c1dee894a
Merged-In: I85b66ab5940d09b2786ecb14bff5bc1c1dee894a
(cherry picked from commit f161cc74a1)
2020-04-14 19:29:43 +01:00
Xusong Wang
b79c31aba8 Remove BLOB AHWB tests from 1.2 VTS.
Bug: 153282844
Test: 1.2 VTS
Change-Id: I866fe1388a64a36372f90c60fef340d2fe4c8682
Merged-In: I866fe1388a64a36372f90c60fef340d2fe4c8682
(cherry picked from commit b642700c7f)
2020-04-14 16:30:01 +01:00