This CL also cleans up the validation framework code.
Bug: 67828197
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I84661fb2b8204148788d10425ca0ac986158b15f
The Deadline test was directly loading into the field of a std::optional
instead of emplacing, which is undefined behavior. This has been
corrected to emplace.
Additionally, previously the deadline test always checked the tensor
results of the execution. This is changed to only check the results
when the driver reports that the execution completed successfully.
Finally, this CL merges similar switch case statements.
Bug: 149766387
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I1068fe70e74a12006dea3cdf4ae9da50004ad24b
Merged-In: I1068fe70e74a12006dea3cdf4ae9da50004ad24b
(cherry picked from commit c7327c57f9)
Prior to this CL, the NNAPI QoS deadline is a strict deadline: a task
with a deadline must either complete or abort by the deadline. To
avoid as much overhead as possible, it is requested for Android R to
have a relaxed deadline, where the task *may* be aborted after the
deadline has passed or when the task is estimated to take too long.
Because the deadline is now relaxed, the querying method
IDevice::supportsDeadlines can be removed.
Bug: 149766387
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ib9937b5bb95646c2fd82e1aa17f2b0b639e2c420
Merged-In: Ib9937b5bb95646c2fd82e1aa17f2b0b639e2c420
(cherry picked from commit 9b9a804fe4)
The Deadline test was directly loading into the field of a std::optional
instead of emplacing, which is undefined behavior. This has been
corrected to emplace.
Additionally, previously the deadline test always checked the tensor
results of the execution. This is changed to only check the results
when the driver reports that the execution completed successfully.
Finally, this CL merges similar switch case statements.
Bug: 149766387
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I1068fe70e74a12006dea3cdf4ae9da50004ad24b
Prior to this CL, the NNAPI QoS deadline is a strict deadline: a task
with a deadline must either complete or abort by the deadline. To
avoid as much overhead as possible, it is requested for Android R to
have a relaxed deadline, where the task *may* be aborted after the
deadline has passed or when the task is estimated to take too long.
Because the deadline is now relaxed, the querying method
IDevice::supportsDeadlines can be removed.
Bug: 149766387
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ib9937b5bb95646c2fd82e1aa17f2b0b639e2c420
This CL makes the following three interface changes:
* Removes @1.3::Operand.ExtraParams, because it was the same as
@1.2::Operand.ExtraParams
* Changes int32_t token to uint32_t for IDevice::allocate and for
@1.3::Request.MemoryPool
* Renames OptionalTimePoint::nanoseconds to
OptionalTimePoint::nanosecondsSinceEpoch
This CL also makes test changes in response to the interface changes,
and fixes a minor comment issue in @1.2::types.hal and @1.3::types.hal.
Bug: 148617339
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I95b49ab34b0b79f587dda035ede700b73dc44a38
Merged-In: I95b49ab34b0b79f587dda035ede700b73dc44a38
(cherry picked from commit 6a4172cd3d)
- Remove the DEVICE_UNAVAILABLE check
- Pass deadline down to the driver.
Bug: 148979873
Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: If90a0dce8c8907a2f8f806455074fa27e3f397ef
Merged-In: If90a0dce8c8907a2f8f806455074fa27e3f397ef
(cherry picked from commit 15a25f6ec1)
- Remove the DEVICE_UNAVAILABLE check
- Pass deadline down to the driver.
Bug: 148979873
Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: If90a0dce8c8907a2f8f806455074fa27e3f397ef
prepareModelFromCache_1_3 uses model cache and data cache that
represent a prepared model. Any argument that contributes in a
meaningful way to the prepared model is implicitly included in
this model cache and data cache. For example, "model" and
"executionPreference" appear in prepareModel_1_3 but not in
prepareModelFromCache_1_3 because they are implicitly included
in the model cache and data cache. In a similar way, because
it could affect the resulting model, "priority" should be
removed from prepareModelFromCache_1_3.
Fixes: 148802784
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I518e493ead8aa97220b16370cce8523b425c378c
Merged-In: I518e493ead8aa97220b16370cce8523b425c378c
(cherry picked from commit c90250ff57)
At present, the tests might not actually use _Float16 (the generated
tests that are incorporated into the functional tests are built in
frameworks/ml rather than in hardware/interfaces); but the makefiles
are changed anyway, in case _Float16 is added in the future.
Bug: 138709788
Test: for (walleye, aosp_x86 [on emulator], aosp_x86_64 [on emulator])
Use clang where _Float16 is disabled for x86[_64]
$ cd $ANDROID_BUILD_TOP/neuralnetworks/1.0/vts/functional
inspect build commands for tests as to whether or not
expected options related to redefinition are present
$ cd $ANDROID_BUILD_TOP/neuralnetworks/1.1/vts/functional
inspect build commands for tests as to whether or not
expected options related to redefinition are present
Change-Id: I9225b228947696d249771b3a35b1f82de9c26b6e
(cherry picked from commit 2452ee5a80)
This CL makes the following three interface changes:
* Removes @1.3::Operand.ExtraParams, because it was the same as
@1.2::Operand.ExtraParams
* Changes int32_t token to uint32_t for IDevice::allocate and for
@1.3::Request.MemoryPool
* Renames OptionalTimePoint::nanoseconds to
OptionalTimePoint::nanosecondsSinceEpoch
This CL also makes test changes in response to the interface changes,
and fixes a minor comment issue in @1.2::types.hal and @1.3::types.hal.
Bug: 148617339
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I95b49ab34b0b79f587dda035ede700b73dc44a38
prepareModelFromCache_1_3 uses model cache and data cache that
represent a prepared model. Any argument that contributes in a
meaningful way to the prepared model is implicitly included in
this model cache and data cache. For example, "model" and
"executionPreference" appear in prepareModel_1_3 but not in
prepareModelFromCache_1_3 because they are implicitly included
in the model cache and data cache. In a similar way, because
it could affect the resulting model, "priority" should be
removed from prepareModelFromCache_1_3.
Fixes: 148802784
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I518e493ead8aa97220b16370cce8523b425c378c
At present, the tests might not actually use _Float16 (the generated
tests that are incorporated into the functional tests are built in
frameworks/ml rather than in hardware/interfaces); but the makefiles
are changed anyway, in case _Float16 is added in the future.
Bug: 138709788
Test: for (walleye, aosp_x86 [on emulator], aosp_x86_64 [on emulator])
Use clang where _Float16 is disabled for x86[_64]
$ cd $ANDROID_BUILD_TOP/neuralnetworks/1.0/vts/functional
inspect build commands for tests as to whether or not
expected options related to redefinition are present
$ cd $ANDROID_BUILD_TOP/neuralnetworks/1.1/vts/functional
inspect build commands for tests as to whether or not
expected options related to redefinition are present
Change-Id: I9225b228947696d249771b3a35b1f82de9c26b6e
- Allow IPreparedModel::executeFenced to measure gated durations.
- Allow IPreparedModel::executeFenced to specify an optional deadline
and aa optional post-gate-release maximum duration.
Bug: 142778241
Bug: 141363565
Test: mm
Change-Id: I1687ec5543f93ed9bb5d7fcf14dbe15ed5b5cd0d
Merged-In: I1687ec5543f93ed9bb5d7fcf14dbe15ed5b5cd0d
(cherry picked from commit 3fd70b020e)
- Allow IPreparedModel::executeFenced to measure gated durations.
- Allow IPreparedModel::executeFenced to specify an optional deadline
and aa optional post-gate-release maximum duration.
Bug: 142778241
Bug: 141363565
Test: mm
Change-Id: I1687ec5543f93ed9bb5d7fcf14dbe15ed5b5cd0d
- Skip the test if the driver reject executeFenced.
- remove the -1 fd for hidl_handle as it causes error before reaching
the driver.
Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Idc3f815040efccbbfa95b70d5d437441d0bd8682
Merged-In: Idc3f815040efccbbfa95b70d5d437441d0bd8682
(cherry picked from commit 33173a3e68)
- Add IPreparedModel::dispatchRequest to NNAPI 1.3 HAL
- Add IDispatchExecutionCallback to allow clients query information
related to the actual evaluation.
Bug: 142778241
Test: mm
Change-Id: I87cbb7f2aee87342b0418fce04eb4050e2bc1920
Merged-In: I87cbb7f2aee87342b0418fce04eb4050e2bc1920
(cherry picked from commit 90cf3dd37c)
This CL makes the following changes:
* introduces a new Priority enum
* extends ErrorStatus with new error codes
* adds "supportsDeadline" method to IDevice
* adds priority and deadline arguments to IDevice::prepareModel*
* adds deadline argument to IPreparedModel::execute*
* updates IExecutionCallback with new ErrorStatus
* updates current.txt accordingly
Bug: 136739795
Bug: 142902514
Bug: 145300530
Test: mma
Change-Id: Iaa7877bde1f463635b8bbdb4e8a001d7b79b9c65
Merged-In: Iaa7877bde1f463635b8bbdb4e8a001d7b79b9c65
(cherry picked from commit c2499ecda1)
The types are the same as in 1.2. No changes are expected.
Bug: 136735929
Test: m
Change-Id: I7431d2e9263fafa0e63b8b1b40f6715e3832d17c
Merged-In: I7431d2e9263fafa0e63b8b1b40f6715e3832d17c
(cherry picked from commit 8a179f3499)
- Skip the test if the driver reject executeFenced.
- remove the -1 fd for hidl_handle as it causes error before reaching
the driver.
Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Idc3f815040efccbbfa95b70d5d437441d0bd8682
- Add IPreparedModel::dispatchRequest to NNAPI 1.3 HAL
- Add IDispatchExecutionCallback to allow clients query information
related to the actual evaluation.
Bug: 142778241
Test: mm
Change-Id: I87cbb7f2aee87342b0418fce04eb4050e2bc1920
This CL makes the following changes:
* introduces a new Priority enum
* extends ErrorStatus with new error codes
* adds "supportsDeadline" method to IDevice
* adds priority and deadline arguments to IDevice::prepareModel*
* adds deadline argument to IPreparedModel::execute*
* updates IExecutionCallback with new ErrorStatus
* updates current.txt accordingly
Bug: 136739795
Bug: 142902514
Bug: 145300530
Test: mma
Change-Id: Iaa7877bde1f463635b8bbdb4e8a001d7b79b9c65
- Add and document memory domain HAL APIs.
- Make necessary changes to the existing VTS codes to make them work
with V1_3::Request.
Bug: 141353602
Bug: 141363565
Test: mma
Test: NNT_static
Test: 1.3 VTS
Change-Id: Ia32555d4fef149fad4a79728981c5d9cca675a1a
Merged-In: Ia32555d4fef149fad4a79728981c5d9cca675a1a
(cherry picked from commit 931d5a18bc)
- Add and document memory domain HAL APIs.
- Make necessary changes to the existing VTS codes to make them work
with V1_3::Request.
Bug: 141353602
Bug: 141363565
Test: mma
Test: NNT_static
Test: 1.3 VTS
Change-Id: Ia32555d4fef149fad4a79728981c5d9cca675a1a
The CL changes explicit prefixes for types in types.spec to a macro, so
that the prefixes of tensor types are correct both in NDK and HAL docs.
Bug: 144841609
Test: mma
Change-Id: I6d904bd3a858f555beed9270f141f080f96e429a
Merged-In: I6d904bd3a858f555beed9270f141f080f96e429a
(cherry picked from commit da5079c5cd)
The CL changes explicit prefixes for types in types.spec to a macro, so
that the prefixes of tensor types are correct both in NDK and HAL docs.
Bug: 144841609
Test: mma
Change-Id: I6d904bd3a858f555beed9270f141f080f96e429a
types.hal files are generated from current types.spec.
Bug: 136735770
Test: mma
Change-Id: Ide24606f8b1d5e755a71d46f230409942d8df7e9
Merged-In: Ide24606f8b1d5e755a71d46f230409942d8df7e9
(cherry picked from commit eb7f175613)
Main commit is in frameworks/ml/nn. This one only renames
testModel.hasQuant8AsymmOperands() to
testModel.hasQuant8CoupledOperands().
Bug: 143935412
Bug: 143934188
Bug: 143935353
Bug: 143934467
Bug: 143934184
Test: quantization coupling tests in CTS and VTS
Change-Id: I0f7c331c355fe61ee06605786f09bc1ca7cdef57
Merged-In: I0f7c331c355fe61ee06605786f09bc1ca7cdef57
(cherry picked from commit 6a1d5e405f)
Add TENSOR_QUANT8_ASYMM_SIGNED to the list of exceptions when mutating
DEQUANTIZE for validation.
Bug: 143934768
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I1b3b0a362d3949d4e31708388100d4794846ca3a
Merged-In: I1b3b0a362d3949d4e31708388100d4794846ca3a
(cherry picked from commit 2bd0b3339c)
Main commit is in frameworks/ml/nn. This one only renames
testModel.hasQuant8AsymmOperands() to
testModel.hasQuant8CoupledOperands().
Bug: 143935412
Bug: 143934188
Bug: 143935353
Bug: 143934467
Bug: 143934184
Test: quantization coupling tests in CTS and VTS
Change-Id: I0f7c331c355fe61ee06605786f09bc1ca7cdef57
Add TENSOR_QUANT8_ASYMM_SIGNED to the list of exceptions when mutating
DEQUANTIZE for validation.
Bug: 143934768
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I1b3b0a362d3949d4e31708388100d4794846ca3a
Also fix repo hook complaining about comment formatting.
Bug: 143935354
Test: NNTest_static and VTS_1_3 with --gtest_filter="*QuantizationCouplingTest*select*"
Change-Id: I1b0d1e987ebccc6700dd172b4222f9996105513d
Merged-In: I1b0d1e987ebccc6700dd172b4222f9996105513d
(cherry picked from commit 04bf939c7a)
Quantization coupling test is ran only on models that contain only one
operator and at least one of its inputs has a type of
TENSOR_QUANT8_ASYMM. The test verifies that a model with all the
operands converted to TENSOR_QUANT8_ASYMM_SIGNED will produce the same
result (OK/SKIPPED/FAILED).
Bug: 137828994
Test: VtsHalNeuralNetworksV1_3TargetTest --gtest_filter="*QuantizationCouplingTest*"
Change-Id: I5a2e09b2b1bd790e7e37dde133554e516155a34d
Merged-In: I5a2e09b2b1bd790e7e37dde133554e516155a34d
(cherry picked from commit c88f4c60a8)
Also fix repo hook complaining about comment formatting.
Bug: 143935354
Test: NNTest_static and VTS_1_3 with --gtest_filter="*QuantizationCouplingTest*select*"
Change-Id: I1b0d1e987ebccc6700dd172b4222f9996105513d
Quantization coupling test is ran only on models that contain only one
operator and at least one of its inputs has a type of
TENSOR_QUANT8_ASYMM. The test verifies that a model with all the
operands converted to TENSOR_QUANT8_ASYMM_SIGNED will produce the same
result (OK/SKIPPED/FAILED).
Bug: 137828994
Test: VtsHalNeuralNetworksV1_3TargetTest --gtest_filter="*QuantizationCouplingTest*"
Change-Id: I5a2e09b2b1bd790e7e37dde133554e516155a34d
The frameworks/ml/nn CL "Improve performance of Burst executions"
changed the signature of a utility function used by NNAPI's VTS tests.
This CL modifies the test accordingly.
Bug: 132073143
Test: mma
Test: VtsHalNeuralnetworksV1_2TargetTest (with sample-all)
Test: VtsHalNeuralnetworksV1_3TargetTest (with sample-all)
Change-Id: I56c3b04e6fec11d526e2a1ff5b896f31cb12eb5f
Merged-In: I56c3b04e6fec11d526e2a1ff5b896f31cb12eb5f
(cherry picked from commit 5756887a7b)
After this update, types.hal v1.3 no longer refers to API level 30.
Test: none needed
Bug: 140132458
Change-Id: If707dfbcade6be1a0885fc21f6ddf47f3c27b244
Merged-In: If707dfbcade6be1a0885fc21f6ddf47f3c27b244
(cherry picked from commit e98e6793be)
So that it's easier to see what actually has changed in VTS tests for
version 1.3
Bug: 139120468
Test: m
Change-Id: I09797f5f3898501a008186a22dd411b00e9e2c67
Merged-In: I09797f5f3898501a008186a22dd411b00e9e2c67
(cherry picked from commit 3b13b55ac1)
So that it's easier to see what actually has changed in VTS tests for
version 1.3
Bug: 139120468
Test: m
Change-Id: Ief294d21349ca6531595612a16fa3ae3382f83ac
Merged-In: Ief294d21349ca6531595612a16fa3ae3382f83ac
(cherry picked from commit 3b13b55ac1)
