A compilation failure is not related to the security aspect of the
TOCTOU test, but it will skip one iteration of security testing. This CL
allows the compilation to fail with GENERAL_FAILURE in TOCTOU tests, and
issues a retry once it happens to ensure enough test coverage.
Bug: 157489048
Test: 1.2/1.3 VTS
Change-Id: Idc88e0365c5d2799187093b6fd7b4abf8f8b463d
Merged-In: Idc88e0365c5d2799187093b6fd7b4abf8f8b463d
(cherry picked from commit 362dfd64d5)
Clarify behaviour of the following ops in the case when inputs would
cause an output to have an empty shape:
* MEAN
* ARGMIN/ARGMAX
* STRIDED_SLICE
Bug: 155508675
Bug: 155660285
Bug: 155508675
Bug: 155238914
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I92d1c3866a462a99b4eed7782b72f62bb6a076c6
Merged-In: I92d1c3866a462a99b4eed7782b72f62bb6a076c6
(cherry picked from commit d4bba83f6a)
Add a clarification on the behaviour of REDUCE_* ops when all dimensions
of an input tensor are reduced.
Bug: 155508675
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I3a67225c3af80af40a11362ca5d0f55350cbfa05
Merged-In: I3a67225c3af80af40a11362ca5d0f55350cbfa05
(cherry picked from commit eec7d86b44)
- Also adds TEST_MAPPING filters here to make sure sample driver tests
are still being filtered out.
Bug: 155674368
Test: mm
Test: atest VtsHalNeuralnetworksV1_2TargetTest
Change-Id: I5f5d272742d4cdd6a8cc87ec035b5e2508eaa98f
Merged-In: I5f5d272742d4cdd6a8cc87ec035b5e2508eaa98f
(cherry picked from commit f7c55d5f9b)
- detect cycle (CycleTest)
- detect bad execution order (mutateExecutionOrderTest)
- detect lifetime inconsistent with whether operand is written (mutateOperandLifeTimeTest)
- detect lifetime inconsistent with Model inputIndexes/outputIndexes (mutateOperandInputOutputTest)
- detect incorrect number of consumers (mutateOperandNumberOfConsumersTest)
- detect operand written multiple times (mutateOperandAddWriterTest)
- detect operand never written (mutateOperationRemoveWriteTest)
Bug: 66478689
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: Id4ba19660bbd31a16f8a675f7b6437f4d779e8da
Merged-In: Id4ba19660bbd31a16f8a675f7b6437f4d779e8da
(cherry picked from commit af51663e99)
Bug: 151896491
Test: local build
Exempt-From-Owner-Approval: This CL update suite name vts-core to vts as
the suite name is updated. This CL won't change test logic or behavior.
Change-Id: I562b4dc50765e953800a814a8fd84a01c1b9352b
Merged-In: I562b4dc50765e953800a814a8fd84a01c1b9352b
Do not run validation on "inputs_as_internal" and
"all_tensors_as_inputs" variants.
Bug: 138149072
Bug: 149840439
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I0699ed6703e48b6c4bc0e7a392b79c12770f04c9
Merged-In: I0699ed6703e48b6c4bc0e7a392b79c12770f04c9
(cherry picked from commit 678a10600c)
This CL also cleans up the validation framework code.
Bug: 67828197
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I84661fb2b8204148788d10425ca0ac986158b15f
Merged-In: I84661fb2b8204148788d10425ca0ac986158b15f
(cherry picked from commit da1a692880)
This CL makes the following three interface changes:
* Removes @1.3::Operand.ExtraParams, because it was the same as
@1.2::Operand.ExtraParams
* Changes int32_t token to uint32_t for IDevice::allocate and for
@1.3::Request.MemoryPool
* Renames OptionalTimePoint::nanoseconds to
OptionalTimePoint::nanosecondsSinceEpoch
This CL also makes test changes in response to the interface changes,
and fixes a minor comment issue in @1.2::types.hal and @1.3::types.hal.
Bug: 148617339
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I95b49ab34b0b79f587dda035ede700b73dc44a38
Merged-In: I95b49ab34b0b79f587dda035ede700b73dc44a38
(cherry picked from commit 6a4172cd3d)
At present, the tests might not actually use _Float16 (the generated
tests that are incorporated into the functional tests are built in
frameworks/ml rather than in hardware/interfaces); but the makefiles
are changed anyway, in case _Float16 is added in the future.
Bug: 138709788
Test: for (walleye, aosp_x86 [on emulator], aosp_x86_64 [on emulator])
Use clang where _Float16 is disabled for x86[_64]
$ cd $ANDROID_BUILD_TOP/neuralnetworks/1.0/vts/functional
inspect build commands for tests as to whether or not
expected options related to redefinition are present
$ cd $ANDROID_BUILD_TOP/neuralnetworks/1.1/vts/functional
inspect build commands for tests as to whether or not
expected options related to redefinition are present
Change-Id: I9225b228947696d249771b3a35b1f82de9c26b6e
(cherry picked from commit 2452ee5a80)
types.hal files are generated from current types.spec.
Bug: 136735770
Test: mma
Change-Id: Ide24606f8b1d5e755a71d46f230409942d8df7e9
Merged-In: Ide24606f8b1d5e755a71d46f230409942d8df7e9
(cherry picked from commit eb7f175613)
Also fix repo hook complaining about comment formatting.
Bug: 143935354
Test: NNTest_static and VTS_1_3 with --gtest_filter="*QuantizationCouplingTest*select*"
Change-Id: I1b0d1e987ebccc6700dd172b4222f9996105513d
Merged-In: I1b0d1e987ebccc6700dd172b4222f9996105513d
(cherry picked from commit 04bf939c7a)
The frameworks/ml/nn CL "Improve performance of Burst executions"
changed the signature of a utility function used by NNAPI's VTS tests.
This CL modifies the test accordingly.
Bug: 132073143
Test: mma
Test: VtsHalNeuralnetworksV1_2TargetTest (with sample-all)
Test: VtsHalNeuralnetworksV1_3TargetTest (with sample-all)
Change-Id: I56c3b04e6fec11d526e2a1ff5b896f31cb12eb5f
Merged-In: I56c3b04e6fec11d526e2a1ff5b896f31cb12eb5f
(cherry picked from commit 5756887a7b)
After this update, types.hal v1.3 no longer refers to API level 30.
Test: none needed
Bug: 140132458
Change-Id: If707dfbcade6be1a0885fc21f6ddf47f3c27b244
Merged-In: If707dfbcade6be1a0885fc21f6ddf47f3c27b244
(cherry picked from commit e98e6793be)
There are certain pieces of NeuralNetworks.h and of our various *.hal
files that ought to be kept in sync -- most notably the operand type
and operation type definitions and descriptions in our
NeuralNetworks.h and types.hal files. To avoid having to do this
manually, a tool can be employed.
The old solution to this problem is sync_enums_to_hal.py, which parses
the existing NeuralNetworks.h and types.hal files, and then rewrites
the types.hal files: It identifies the operand type and operation type
portions of the files, and in the types.hal files replaces them with
transformed versions of the portions from NeuralNetworks.h. This
approach is brittle -- it is very sensitive to the structure of the
files, as was recognized when this tool was created. Changes to those
files since the script was last updated essentially broke the script,
as noted in http://b/140132458.
The new solution employs a new tool generate_api.py to combine a
single "specification file" with one "template file" per API file
(NeuralNetworks.h or types.hal) to produce that API file. The script
generate_api.sh invokes generate_api.py once per API file, passing
appropriate arguments. See frameworks/ml/nn/tools/api/README.md for
more details.
In the process of combining information from NeuralNetworks.h and
*/types.hal, some formatting, syntactic, and semantic changes have
been made to those files. For example:
- types.hal files no longer refer to API levels or (similarly)
to HAL versions other than earlier HAL versions
Bug: 130169064
Bug: 140132458
Test: cd neuralnetworks ; mma
Change-Id: Iceb38effc2c7e6c32344e49225b85256144c0945
Merged-In: Iceb38effc2c7e6c32344e49225b85256144c0945
(cherry picked from commit 2cae5c8b01)
This CL removes a dependency on the VTS test runner by dynamically
discovering all NN HAL service instances in the gtest binary itself,
and runs through all service instances with parameterized tests.
This CL converts TEST_F cases to TEST_P cases, where the test parameter
is the name of the service instance. For existing TEST_P cases (such as
the generated test cases), the service instance name is made to be the
first test parameter.
This CL enables the NN VTS tests to be more portable, e.g., they can
run directly as a presubmit test.
Fixes: 124540002
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Test: cd $ANDROID_BUILD_TOP/hardware/interfaces/neuralnetworks && atest
Change-Id: I1e301d7c9f9342bb8f35a267bef180f510944b19
Merged-In: I1e301d7c9f9342bb8f35a267bef180f510944b19
(cherry picked from commit 7076f629b7)
This CL additionally adds missing documentation for timing
information when returning results on the resultChannel.
Bug: 133773876
Test: mma
Change-Id: I1eb1affbb4a912d5fdeab012e2be7e7005deb04d
Merged-In: I1eb1affbb4a912d5fdeab012e2be7e7005deb04d
(cherry picked from commit f99bffd08a)
Under certain circumstances, we guarantee that a prepared model can be
executed successfully. In describing those circumstances, we
neglected to specify that operation input operands must have legal
values for the guarantee to hold. For example, the guarantee doesn't
hold if an ADD operation has an activation input that is not one of
the defined values; or if a RESHAPE operation has a shape input in
which two or more components are -1.
This change modifies the guarantee to apply only when operation input
operands have legal values. It also documents this guarantee for
burst execution.
Note that if an operation has an input operand that can be proven to
have an illegal value at preparation time (e.g., a constant value that
is illegal), model preparation might (but is not required to) fail for
that reason.
Bug: 135933040
Test: $ cd neuralnetworks ; mma
Change-Id: I8b421550dd89e4bbbdae899e7cb5e9e88a46d2fb
(cherry picked from commit 48544cc38a)
This CL does additional NNAPI VTS test cleanup, including consolidating
duplicate functionality. Specifically, this CL:
* consolidates the createPreparedModel function, removing the duplicate
* consolidates the std::out ErrorStatus and DeviceStatus code into Utils
* changes non-null constant pointers to constant references
* removes redudant leading namespace specifiers (V1_0::, ::testing, etc.)
* makes the Valdiation tests free functions
* renames device to kDevice and mTestModel to kTestModel
Bug: N/A
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Change-Id: Ic401bb1f1760cc10384ac0d30c0c93409b63a9c7
Merged-In: Ic401bb1f1760cc10384ac0d30c0c93409b63a9c7
(cherry picked from commit e16af0a44b)
