A compilation failure is not related to the security aspect of the
TOCTOU test, but it will skip one iteration of security testing. This CL
allows the compilation to fail with GENERAL_FAILURE in TOCTOU tests, and
issues a retry once it happens to ensure enough test coverage.
Bug: 157489048
Test: 1.2/1.3 VTS
Change-Id: Idc88e0365c5d2799187093b6fd7b4abf8f8b463d
Add checks that all returned output dimensions must be at
least as fully specified as the union of the information about the
corresponding operand in the model and in the request.
Bug: 154054474
Test: VTS
Change-Id: I934d084c7665160a98da9828604ce8297fef73b8
Merged-In: I934d084c7665160a98da9828604ce8297fef73b8
(cherry picked from commit d454751e00)
Add checks that all returned output dimensions must be at
least as fully specified as the union of the information about the
corresponding operand in the model and in the request.
Bug: 154054474
Test: VTS
Change-Id: I934d084c7665160a98da9828604ce8297fef73b8
Clarify behaviour of the following ops in the case when inputs would
cause an output to have an empty shape:
* MEAN
* ARGMIN/ARGMAX
* STRIDED_SLICE
Bug: 155508675
Bug: 155660285
Bug: 155508675
Bug: 155238914
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I92d1c3866a462a99b4eed7782b72f62bb6a076c6
Merged-In: I92d1c3866a462a99b4eed7782b72f62bb6a076c6
(cherry picked from commit d4bba83f6a)
- Also adds TEST_MAPPING filters here to make sure sample driver tests
are still being filtered out.
- There may still be failing tests, but should not cause OOM in test harness.
Bug: 156691406
Test: mm
Test: atest VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ie05350d71dede04967d94dad4147bd3ed3435f7d
Merged-In: Ie05350d71dede04967d94dad4147bd3ed3435f7d
(cherry picked from commit 338acf5c15)
Add a clarification on the behaviour of REDUCE_* ops when all dimensions
of an input tensor are reduced.
Bug: 155508675
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I3a67225c3af80af40a11362ca5d0f55350cbfa05
Merged-In: I3a67225c3af80af40a11362ca5d0f55350cbfa05
(cherry picked from commit eec7d86b44)
- Also adds TEST_MAPPING filters here to make sure sample driver tests
are still being filtered out.
Bug: 155577050
Test: mm
Test: atest VtsHalNeuralnetworksV1_0TargetTest
Change-Id: Ic8f556a957e6af340d500e2c4696b41b2a6e8516
Merged-In: Ic8f556a957e6af340d500e2c4696b41b2a6e8516
(cherry picked from commit 8ebe2a04c5)
- Also adds TEST_MAPPING filters here to make sure sample driver tests
are still being filtered out.
Bug: 155674368
Test: mm
Test: atest VtsHalNeuralnetworksV1_2TargetTest
Change-Id: I5f5d272742d4cdd6a8cc87ec035b5e2508eaa98f
Merged-In: I5f5d272742d4cdd6a8cc87ec035b5e2508eaa98f
(cherry picked from commit f7c55d5f9b)
- Also adds TEST_MAPPING filters here to make sure sample driver tests
are still being filtered out.
- There may still be failing tests, but should not cause OOM in test harness.
Bug: 156691406
Test: mm
Test: atest VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ie05350d71dede04967d94dad4147bd3ed3435f7d
Clarify behaviour of the following ops in the case when inputs would
cause an output to have an empty shape:
* MEAN
* ARGMIN/ARGMAX
* STRIDED_SLICE
Bug: 155508675
Bug: 155660285
Bug: 155508675
Bug: 155238914
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I92d1c3866a462a99b4eed7782b72f62bb6a076c6
Add a clarification on the behaviour of REDUCE_* ops when all dimensions
of an input tensor are reduced.
Bug: 155508675
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I3a67225c3af80af40a11362ca5d0f55350cbfa05
- Also adds TEST_MAPPING filters here to make sure sample driver tests
are still being filtered out.
Bug: 155577050
Test: mm
Test: atest VtsHalNeuralnetworksV1_0TargetTest
Change-Id: Ic8f556a957e6af340d500e2c4696b41b2a6e8516
- Also adds TEST_MAPPING filters here to make sure sample driver tests
are still being filtered out.
Bug: 155674368
Test: mm
Test: atest VtsHalNeuralnetworksV1_2TargetTest
Change-Id: I5f5d272742d4cdd6a8cc87ec035b5e2508eaa98f
- detect cycle (CycleTest)
- detect bad execution order (mutateExecutionOrderTest)
- detect lifetime inconsistent with whether operand is written (mutateOperandLifeTimeTest)
- detect lifetime inconsistent with Model inputIndexes/outputIndexes (mutateOperandInputOutputTest)
- detect incorrect number of consumers (mutateOperandNumberOfConsumersTest)
- detect operand written multiple times (mutateOperandAddWriterTest)
- detect operand never written (mutateOperationRemoveWriteTest)
Bug: 66478689
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: Id4ba19660bbd31a16f8a675f7b6437f4d779e8da
Merged-In: Id4ba19660bbd31a16f8a675f7b6437f4d779e8da
(cherry picked from commit af51663e99)
- detect cycle (CycleTest)
- detect bad execution order (mutateExecutionOrderTest)
- detect lifetime inconsistent with whether operand is written (mutateOperandLifeTimeTest)
- detect lifetime inconsistent with Model inputIndexes/outputIndexes (mutateOperandInputOutputTest)
- detect incorrect number of consumers (mutateOperandNumberOfConsumersTest)
- detect operand written multiple times (mutateOperandAddWriterTest)
- detect operand never written (mutateOperationRemoveWriteTest)
Bug: 66478689
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: Id4ba19660bbd31a16f8a675f7b6437f4d779e8da
Merged-In: Id4ba19660bbd31a16f8a675f7b6437f4d779e8da
(cherry picked from commit af51663e99)
Bug: 151896491
Test: local build
Exempt-From-Owner-Approval: This CL update suite name vts-core to vts as
the suite name is updated. This CL won't change test logic or behavior.
Change-Id: I562b4dc50765e953800a814a8fd84a01c1b9352b
Merged-In: I562b4dc50765e953800a814a8fd84a01c1b9352b
Bug: 151896491
Test: local build
Exempt-From-Owner-Approval: This CL update suite name vts-core to vts as
the suite name is updated. This CL won't change test logic or behavior.
Change-Id: I562b4dc50765e953800a814a8fd84a01c1b9352b
Merged-In: I562b4dc50765e953800a814a8fd84a01c1b9352b
Do not run validation on "inputs_as_internal" and
"all_tensors_as_inputs" variants.
Bug: 138149072
Bug: 149840439
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I0699ed6703e48b6c4bc0e7a392b79c12770f04c9
Merged-In: I0699ed6703e48b6c4bc0e7a392b79c12770f04c9
(cherry picked from commit 678a10600c)
Do not run validation on "inputs_as_internal" and
"all_tensors_as_inputs" variants.
Bug: 138149072
Bug: 149840439
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I0699ed6703e48b6c4bc0e7a392b79c12770f04c9
This CL also cleans up the validation framework code.
Bug: 67828197
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I84661fb2b8204148788d10425ca0ac986158b15f
Merged-In: I84661fb2b8204148788d10425ca0ac986158b15f
(cherry picked from commit da1a692880)
This CL also cleans up the validation framework code.
Bug: 67828197
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I84661fb2b8204148788d10425ca0ac986158b15f
The Deadline test was directly loading into the field of a std::optional
instead of emplacing, which is undefined behavior. This has been
corrected to emplace.
Additionally, previously the deadline test always checked the tensor
results of the execution. This is changed to only check the results
when the driver reports that the execution completed successfully.
Finally, this CL merges similar switch case statements.
Bug: 149766387
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I1068fe70e74a12006dea3cdf4ae9da50004ad24b
Merged-In: I1068fe70e74a12006dea3cdf4ae9da50004ad24b
(cherry picked from commit c7327c57f9)
Prior to this CL, the NNAPI QoS deadline is a strict deadline: a task
with a deadline must either complete or abort by the deadline. To
avoid as much overhead as possible, it is requested for Android R to
have a relaxed deadline, where the task *may* be aborted after the
deadline has passed or when the task is estimated to take too long.
Because the deadline is now relaxed, the querying method
IDevice::supportsDeadlines can be removed.
Bug: 149766387
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ib9937b5bb95646c2fd82e1aa17f2b0b639e2c420
Merged-In: Ib9937b5bb95646c2fd82e1aa17f2b0b639e2c420
(cherry picked from commit 9b9a804fe4)
The Deadline test was directly loading into the field of a std::optional
instead of emplacing, which is undefined behavior. This has been
corrected to emplace.
Additionally, previously the deadline test always checked the tensor
results of the execution. This is changed to only check the results
when the driver reports that the execution completed successfully.
Finally, this CL merges similar switch case statements.
Bug: 149766387
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I1068fe70e74a12006dea3cdf4ae9da50004ad24b
Prior to this CL, the NNAPI QoS deadline is a strict deadline: a task
with a deadline must either complete or abort by the deadline. To
avoid as much overhead as possible, it is requested for Android R to
have a relaxed deadline, where the task *may* be aborted after the
deadline has passed or when the task is estimated to take too long.
Because the deadline is now relaxed, the querying method
IDevice::supportsDeadlines can be removed.
Bug: 149766387
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ib9937b5bb95646c2fd82e1aa17f2b0b639e2c420
This CL makes the following three interface changes:
* Removes @1.3::Operand.ExtraParams, because it was the same as
@1.2::Operand.ExtraParams
* Changes int32_t token to uint32_t for IDevice::allocate and for
@1.3::Request.MemoryPool
* Renames OptionalTimePoint::nanoseconds to
OptionalTimePoint::nanosecondsSinceEpoch
This CL also makes test changes in response to the interface changes,
and fixes a minor comment issue in @1.2::types.hal and @1.3::types.hal.
Bug: 148617339
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I95b49ab34b0b79f587dda035ede700b73dc44a38
Merged-In: I95b49ab34b0b79f587dda035ede700b73dc44a38
(cherry picked from commit 6a4172cd3d)
- Remove the DEVICE_UNAVAILABLE check
- Pass deadline down to the driver.
Bug: 148979873
Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: If90a0dce8c8907a2f8f806455074fa27e3f397ef
Merged-In: If90a0dce8c8907a2f8f806455074fa27e3f397ef
(cherry picked from commit 15a25f6ec1)
- Remove the DEVICE_UNAVAILABLE check
- Pass deadline down to the driver.
Bug: 148979873
Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: If90a0dce8c8907a2f8f806455074fa27e3f397ef
prepareModelFromCache_1_3 uses model cache and data cache that
represent a prepared model. Any argument that contributes in a
meaningful way to the prepared model is implicitly included in
this model cache and data cache. For example, "model" and
"executionPreference" appear in prepareModel_1_3 but not in
prepareModelFromCache_1_3 because they are implicitly included
in the model cache and data cache. In a similar way, because
it could affect the resulting model, "priority" should be
removed from prepareModelFromCache_1_3.
Fixes: 148802784
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I518e493ead8aa97220b16370cce8523b425c378c
Merged-In: I518e493ead8aa97220b16370cce8523b425c378c
(cherry picked from commit c90250ff57)
At present, the tests might not actually use _Float16 (the generated
tests that are incorporated into the functional tests are built in
frameworks/ml rather than in hardware/interfaces); but the makefiles
are changed anyway, in case _Float16 is added in the future.
Bug: 138709788
Test: for (walleye, aosp_x86 [on emulator], aosp_x86_64 [on emulator])
Use clang where _Float16 is disabled for x86[_64]
$ cd $ANDROID_BUILD_TOP/neuralnetworks/1.0/vts/functional
inspect build commands for tests as to whether or not
expected options related to redefinition are present
$ cd $ANDROID_BUILD_TOP/neuralnetworks/1.1/vts/functional
inspect build commands for tests as to whether or not
expected options related to redefinition are present
Change-Id: I9225b228947696d249771b3a35b1f82de9c26b6e
(cherry picked from commit 2452ee5a80)
This CL makes the following three interface changes:
* Removes @1.3::Operand.ExtraParams, because it was the same as
@1.2::Operand.ExtraParams
* Changes int32_t token to uint32_t for IDevice::allocate and for
@1.3::Request.MemoryPool
* Renames OptionalTimePoint::nanoseconds to
OptionalTimePoint::nanosecondsSinceEpoch
This CL also makes test changes in response to the interface changes,
and fixes a minor comment issue in @1.2::types.hal and @1.3::types.hal.
Bug: 148617339
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I95b49ab34b0b79f587dda035ede700b73dc44a38
prepareModelFromCache_1_3 uses model cache and data cache that
represent a prepared model. Any argument that contributes in a
meaningful way to the prepared model is implicitly included in
this model cache and data cache. For example, "model" and
"executionPreference" appear in prepareModel_1_3 but not in
prepareModelFromCache_1_3 because they are implicitly included
in the model cache and data cache. In a similar way, because
it could affect the resulting model, "priority" should be
removed from prepareModelFromCache_1_3.
Fixes: 148802784
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I518e493ead8aa97220b16370cce8523b425c378c
At present, the tests might not actually use _Float16 (the generated
tests that are incorporated into the functional tests are built in
frameworks/ml rather than in hardware/interfaces); but the makefiles
are changed anyway, in case _Float16 is added in the future.
Bug: 138709788
Test: for (walleye, aosp_x86 [on emulator], aosp_x86_64 [on emulator])
Use clang where _Float16 is disabled for x86[_64]
$ cd $ANDROID_BUILD_TOP/neuralnetworks/1.0/vts/functional
inspect build commands for tests as to whether or not
expected options related to redefinition are present
$ cd $ANDROID_BUILD_TOP/neuralnetworks/1.1/vts/functional
inspect build commands for tests as to whether or not
expected options related to redefinition are present
Change-Id: I9225b228947696d249771b3a35b1f82de9c26b6e
- Allow IPreparedModel::executeFenced to measure gated durations.
- Allow IPreparedModel::executeFenced to specify an optional deadline
and aa optional post-gate-release maximum duration.
Bug: 142778241
Bug: 141363565
Test: mm
Change-Id: I1687ec5543f93ed9bb5d7fcf14dbe15ed5b5cd0d
Merged-In: I1687ec5543f93ed9bb5d7fcf14dbe15ed5b5cd0d
(cherry picked from commit 3fd70b020e)
- Allow IPreparedModel::executeFenced to measure gated durations.
- Allow IPreparedModel::executeFenced to specify an optional deadline
and aa optional post-gate-release maximum duration.
Bug: 142778241
Bug: 141363565
Test: mm
Change-Id: I1687ec5543f93ed9bb5d7fcf14dbe15ed5b5cd0d
- Skip the test if the driver reject executeFenced.
- remove the -1 fd for hidl_handle as it causes error before reaching
the driver.
Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Idc3f815040efccbbfa95b70d5d437441d0bd8682
Merged-In: Idc3f815040efccbbfa95b70d5d437441d0bd8682
(cherry picked from commit 33173a3e68)
- Add IPreparedModel::dispatchRequest to NNAPI 1.3 HAL
- Add IDispatchExecutionCallback to allow clients query information
related to the actual evaluation.
Bug: 142778241
Test: mm
Change-Id: I87cbb7f2aee87342b0418fce04eb4050e2bc1920
Merged-In: I87cbb7f2aee87342b0418fce04eb4050e2bc1920
(cherry picked from commit 90cf3dd37c)
This CL makes the following changes:
* introduces a new Priority enum
* extends ErrorStatus with new error codes
* adds "supportsDeadline" method to IDevice
* adds priority and deadline arguments to IDevice::prepareModel*
* adds deadline argument to IPreparedModel::execute*
* updates IExecutionCallback with new ErrorStatus
* updates current.txt accordingly
Bug: 136739795
Bug: 142902514
Bug: 145300530
Test: mma
Change-Id: Iaa7877bde1f463635b8bbdb4e8a001d7b79b9c65
Merged-In: Iaa7877bde1f463635b8bbdb4e8a001d7b79b9c65
(cherry picked from commit c2499ecda1)
The types are the same as in 1.2. No changes are expected.
Bug: 136735929
Test: m
Change-Id: I7431d2e9263fafa0e63b8b1b40f6715e3832d17c
Merged-In: I7431d2e9263fafa0e63b8b1b40f6715e3832d17c
(cherry picked from commit 8a179f3499)
- Skip the test if the driver reject executeFenced.
- remove the -1 fd for hidl_handle as it causes error before reaching
the driver.
Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Idc3f815040efccbbfa95b70d5d437441d0bd8682
- Add IPreparedModel::dispatchRequest to NNAPI 1.3 HAL
- Add IDispatchExecutionCallback to allow clients query information
related to the actual evaluation.
Bug: 142778241
Test: mm
Change-Id: I87cbb7f2aee87342b0418fce04eb4050e2bc1920
This CL makes the following changes:
* introduces a new Priority enum
* extends ErrorStatus with new error codes
* adds "supportsDeadline" method to IDevice
* adds priority and deadline arguments to IDevice::prepareModel*
* adds deadline argument to IPreparedModel::execute*
* updates IExecutionCallback with new ErrorStatus
* updates current.txt accordingly
Bug: 136739795
Bug: 142902514
Bug: 145300530
Test: mma
Change-Id: Iaa7877bde1f463635b8bbdb4e8a001d7b79b9c65
- Add and document memory domain HAL APIs.
- Make necessary changes to the existing VTS codes to make them work
with V1_3::Request.
Bug: 141353602
Bug: 141363565
Test: mma
Test: NNT_static
Test: 1.3 VTS
Change-Id: Ia32555d4fef149fad4a79728981c5d9cca675a1a
Merged-In: Ia32555d4fef149fad4a79728981c5d9cca675a1a
(cherry picked from commit 931d5a18bc)
- Add and document memory domain HAL APIs.
- Make necessary changes to the existing VTS codes to make them work
with V1_3::Request.
Bug: 141353602
Bug: 141363565
Test: mma
Test: NNT_static
Test: 1.3 VTS
Change-Id: Ia32555d4fef149fad4a79728981c5d9cca675a1a
The CL changes explicit prefixes for types in types.spec to a macro, so
that the prefixes of tensor types are correct both in NDK and HAL docs.
Bug: 144841609
Test: mma
Change-Id: I6d904bd3a858f555beed9270f141f080f96e429a
Merged-In: I6d904bd3a858f555beed9270f141f080f96e429a
(cherry picked from commit da5079c5cd)
The CL changes explicit prefixes for types in types.spec to a macro, so
that the prefixes of tensor types are correct both in NDK and HAL docs.
Bug: 144841609
Test: mma
Change-Id: I6d904bd3a858f555beed9270f141f080f96e429a
types.hal files are generated from current types.spec.
Bug: 136735770
Test: mma
Change-Id: Ide24606f8b1d5e755a71d46f230409942d8df7e9
Merged-In: Ide24606f8b1d5e755a71d46f230409942d8df7e9
(cherry picked from commit eb7f175613)
Main commit is in frameworks/ml/nn. This one only renames
testModel.hasQuant8AsymmOperands() to
testModel.hasQuant8CoupledOperands().
Bug: 143935412
Bug: 143934188
Bug: 143935353
Bug: 143934467
Bug: 143934184
Test: quantization coupling tests in CTS and VTS
Change-Id: I0f7c331c355fe61ee06605786f09bc1ca7cdef57
Merged-In: I0f7c331c355fe61ee06605786f09bc1ca7cdef57
(cherry picked from commit 6a1d5e405f)
