Commit graph

359 commits

Author SHA1 Message Date
Przemyslaw Szczepaniak
6d3cdc322b Add TENSOR_QUANT8_ASYMM_SIGNED support for more ops
Updated:
 BATCH_TO_SPACE_ND
 CHANNEL_SHUFFLE
 DEPTH_TO_SPACE
 GROUPED_CONV_2D
 PAD
 PAD_V2
 QUANTIZE
 RESIZE_BILINEAR
 RESIZE_NEAREST_NEIGHBOR
 SPACE_TO_BATCH_ND
 SPACE_TO_DEPTH
 SPLIT
 TILE
 TOPK_V2
 TRANSPOSE

Bug: 143934582
Bug: 143934585
Bug: 143934628
Bug: 143934630
Bug: 143934721
Bug: 143935039
Bug: 143935052
Bug: 143935113
Bug: 143935115
Bug: 143935141
Bug: 143935353
Bug: 143935355
Bug: 143935392
Bug: 143935394
Bug: 143935413
Test: quantization coupling tests in CTS and VTS
Change-Id: I7e1b65507ea0f7dcdfdb5fd98e7871d84f569ed7
2019-11-27 15:09:09 +00:00
Lev Proleev
5c305e52ef Merge changes from topic "quant_coupling"
* changes:
  Add QUANT8_ASYMM_SIGNED support to SELECT op
  Add quantization coupling test
2019-11-20 10:56:14 +00:00
Dan Shi
4c3a94d48d Merge "Remove VtsHalNeuralnetworksV1_2TargetTest from vts-core suite" am: 86c7aabcc0 am: 572970bcc6
am: 2327f621e7

Change-Id: Ibf66f98dd9d5d82d620f8907629eb56dae65ca81
2019-11-18 13:05:14 -08:00
Lev Proleev
04bf939c7a Add QUANT8_ASYMM_SIGNED support to SELECT op
Also fix repo hook complaining about comment formatting.

Bug: 143935354
Test: NNTest_static and VTS_1_3  with --gtest_filter="*QuantizationCouplingTest*select*"
Change-Id: I1b0d1e987ebccc6700dd172b4222f9996105513d
2019-11-18 17:20:55 +00:00
Dan Shi
07f4235b53 Remove VtsHalNeuralnetworksV1_2TargetTest from vts-core suite
The test takes over 12 hours to run. Remove it from vts-core suite.

Bug: 144527956
Test: None
Change-Id: If37a7b04e6d7b044d1594dcb4df07432236a1cf3
2019-11-18 08:40:19 -08:00
Lev Proleev
66f598e10d Combine test parameters into TestConfig structure
Test: VtsHalNeuralnetworksV1_2TargetTest --gtest_filter="GeneratedTests*"
Change-Id: I928aaa42e4745b4a8e0e461046e9632b052d0135
2019-11-12 15:38:36 +00:00
Michael Butler
648ada5fcf NNAPI VTS update in response to utility function change
The frameworks/ml/nn CL "Improve performance of Burst executions"
changed the signature of a utility function used by NNAPI's VTS tests.
This CL modifies the test accordingly.

Bug: 132073143
Test: mma
Test: VtsHalNeuralnetworksV1_2TargetTest (with sample-all)
Test: VtsHalNeuralnetworksV1_3TargetTest (with sample-all)
Change-Id: I56c3b04e6fec11d526e2a1ff5b896f31cb12eb5f
Merged-In: I56c3b04e6fec11d526e2a1ff5b896f31cb12eb5f
(cherry picked from commit 5756887a7b)
2019-10-28 13:17:08 -07:00
Lev Proleev
e5376fea95 Add template for 1.3 types.hal and regenerate it
After this update, types.hal v1.3 no longer refers to API level 30.

Test: none needed
Bug: 140132458
Change-Id: If707dfbcade6be1a0885fc21f6ddf47f3c27b244
Merged-In: If707dfbcade6be1a0885fc21f6ddf47f3c27b244
(cherry picked from commit e98e6793be)
2019-10-28 13:03:00 +00:00
Lev Proleev
26d1bc8a4e Modify NNAPI VTS tests to run on version 1.3
Bug: 139120468
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I4654dc75c17f8801103015dc1da91663dfa28d52
Merged-In: I4654dc75c17f8801103015dc1da91663dfa28d52
(cherry picked from commit b49dadfb64)
2019-10-17 18:34:51 +01:00
Lev Proleev
ce8c632a09 Create NNAPI HAL v1.3 and add TENSOR_QUANT8_ASYMM_SIGNED OperandType
Bug: 137828494
Bug: 139120468
Bug: 136735770
Test: mma
Change-Id: I28f74e4b364fec1d7431a96cf5687256b3106069
Merged-In: I28f74e4b364fec1d7431a96cf5687256b3106069
(cherry picked from commit 5a7b67ab8f)
2019-10-17 18:34:51 +01:00
Dan Shi
ecbc1ab590 Merge "Add VtsHalLightV2_0TargetTest and VtsHalNeuralnetworksV1_* to vts-core" am: 0e1efc568c am: 7626cc8cd5 am: 86395146bf
am: 9154805dce

Change-Id: Ic87df0df72d87be833bc408af5bc6a9139dcf7f4
2019-10-14 09:12:32 -07:00
Dan Shi
0e1efc568c Merge "Add VtsHalLightV2_0TargetTest and VtsHalNeuralnetworksV1_* to vts-core" 2019-10-14 15:35:48 +00:00
Michael Butler
50d76d6cfe Merge "NNAPI VTS update in response to utility function change" 2019-10-11 17:31:20 +00:00
Colin Cross
552c7a66ea Revert "Revert "Modify NNAPI VTS tests to run on version 1.3" am: 64f9eb4bd5 am: 7973d7dec5 am: 8bde98b027"
This reverts commit 496320b1ec.

Reason for revert: revert of a cherry-pick broke master

Change-Id: I4654dc75c17f8801103015dc1da91663dfa28d52
2019-10-10 23:13:36 +00:00
Lev Proleev
496320b1ec Revert "Modify NNAPI VTS tests to run on version 1.3" am: 64f9eb4bd5 am: 7973d7dec5 am: 8bde98b027
am: edadec3f9e

Change-Id: Idb6d7245c58afd0aba52db67cb1a540604ed8e33
2019-10-10 14:54:42 -07:00
Dan Shi
4856370e54 Add VtsHalLightV2_0TargetTest and VtsHalNeuralnetworksV1_* to vts-core
Bug: None
Test: None
Change-Id: I3fbbba0dc283bec2de025f0b743d97ded5874bf9
2019-10-10 13:45:43 -07:00
Lev Proleev
839ca3c428 Revert "Create NNAPI HAL v1.3 and add TENSOR_QUANT8_ASYMM_SIGNED OperandType"
This reverts commit bd2b4e78ac.

Reason for revert: the topic breaks git_qt-dev-plus-aosp

Change-Id: Ic72dcbb7ac4ef8e6ed597eb778289b39475f03f8
2019-10-10 14:01:05 +00:00
Lev Proleev
64f9eb4bd5 Revert "Modify NNAPI VTS tests to run on version 1.3"
This reverts commit 543606fb92.

Reason for revert: the topic breaks git_qt-dev-plus-aosp

Change-Id: I74859bae687d65d344ed7edc617e1a7d086960a1
2019-10-10 13:54:38 +00:00
Michael Butler
5756887a7b NNAPI VTS update in response to utility function change
The frameworks/ml/nn CL "Improve performance of Burst executions"
changed the signature of a utility function used by NNAPI's VTS tests.
This CL modifies the test accordingly.

Bug: 132073143
Test: mma
Test: VtsHalNeuralnetworksV1_2TargetTest (with sample-all)
Test: VtsHalNeuralnetworksV1_3TargetTest (with sample-all)
Change-Id: I56c3b04e6fec11d526e2a1ff5b896f31cb12eb5f
2019-10-09 11:15:08 -07:00
Lev Proleev
e98e6793be Add template for 1.3 types.hal and regenerate it
After this update, types.hal v1.3 no longer refers to API level 30.

Test: none needed
Bug: 140132458
Change-Id: If707dfbcade6be1a0885fc21f6ddf47f3c27b244
2019-10-09 16:32:11 +01:00
Lev Proleev
543606fb92 Modify NNAPI VTS tests to run on version 1.3
Bug: 139120468
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Id9e4d99852da8a3d5167ab7464c0e71885250501
Merged-In: Id9e4d99852da8a3d5167ab7464c0e71885250501
(cherry picked from commit b49dadfb64)
2019-10-09 14:18:49 +01:00
Lev Proleev
bd2b4e78ac Create NNAPI HAL v1.3 and add TENSOR_QUANT8_ASYMM_SIGNED OperandType
Bug: 137828494
Bug: 139120468
Bug: 136735770
Test: mma
Change-Id: I1f615047d1c0c208a90082ffb6ffc43f252f77b4
Merged-In: I1f615047d1c0c208a90082ffb6ffc43f252f77b4
(cherry picked from commit 5a7b67ab8f)
2019-10-09 14:18:49 +01:00
David Gross
8c0a48bceb Replace sync_enums_to_hal.py with generate_api.{py,sh} and regenerate */types.hal
There are certain pieces of NeuralNetworks.h and of our various *.hal
files that ought to be kept in sync -- most notably the operand type
and operation type definitions and descriptions in our
NeuralNetworks.h and types.hal files.  To avoid having to do this
manually, a tool can be employed.

The old solution to this problem is sync_enums_to_hal.py, which parses
the existing NeuralNetworks.h and types.hal files, and then rewrites
the types.hal files: It identifies the operand type and operation type
portions of the files, and in the types.hal files replaces them with
transformed versions of the portions from NeuralNetworks.h.  This
approach is brittle -- it is very sensitive to the structure of the
files, as was recognized when this tool was created.  Changes to those
files since the script was last updated essentially broke the script,
as noted in http://b/140132458.

The new solution employs a new tool generate_api.py to combine a
single "specification file" with one "template file" per API file
(NeuralNetworks.h or types.hal) to produce that API file.  The script
generate_api.sh invokes generate_api.py once per API file, passing
appropriate arguments.  See frameworks/ml/nn/tools/api/README.md for
more details.

In the process of combining information from NeuralNetworks.h and
*/types.hal, some formatting, syntactic, and semantic changes have
been made to those files.  For example:
- types.hal files no longer refer to API levels or (similarly)
  to HAL versions other than earlier HAL versions

Bug: 130169064
Bug: 140132458

Test: cd neuralnetworks ; mma

Change-Id: Iceb38effc2c7e6c32344e49225b85256144c0945
Merged-In: Iceb38effc2c7e6c32344e49225b85256144c0945
(cherry picked from commit 2cae5c8b01)
2019-10-09 14:18:06 +01:00
Lev Proleev
b58075603a Merge changes from topic "NNAPI v1.3"
* changes:
  Modify NNAPI VTS tests to run on version 1.3
  Copy VTS tests from v1.2 to v1.3
  Create NNAPI HAL v1.3 and add TENSOR_QUANT8_ASYMM_SIGNED OperandType
2019-10-04 15:34:10 +00:00
David Gross
2cae5c8b01 Replace sync_enums_to_hal.py with generate_api.{py,sh} and regenerate */types.hal
There are certain pieces of NeuralNetworks.h and of our various *.hal
files that ought to be kept in sync -- most notably the operand type
and operation type definitions and descriptions in our
NeuralNetworks.h and types.hal files.  To avoid having to do this
manually, a tool can be employed.

The old solution to this problem is sync_enums_to_hal.py, which parses
the existing NeuralNetworks.h and types.hal files, and then rewrites
the types.hal files: It identifies the operand type and operation type
portions of the files, and in the types.hal files replaces them with
transformed versions of the portions from NeuralNetworks.h.  This
approach is brittle -- it is very sensitive to the structure of the
files, as was recognized when this tool was created.  Changes to those
files since the script was last updated essentially broke the script,
as noted in http://b/140132458.

The new solution employs a new tool generate_api.py to combine a
single "specification file" with one "template file" per API file
(NeuralNetworks.h or types.hal) to produce that API file.  The script
generate_api.sh invokes generate_api.py once per API file, passing
appropriate arguments.  See frameworks/ml/nn/tools/api/README.md for
more details.

In the process of combining information from NeuralNetworks.h and
*/types.hal, some formatting, syntactic, and semantic changes have
been made to those files.  For example:
- types.hal files no longer refer to API levels or (similarly)
  to HAL versions other than earlier HAL versions

Bug: 130169064
Bug: 140132458

Test: cd neuralnetworks ; mma

Change-Id: Iceb38effc2c7e6c32344e49225b85256144c0945
2019-10-03 14:14:20 -07:00
Lev Proleev
b49dadfb64 Modify NNAPI VTS tests to run on version 1.3
Bug: 139120468
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Id9e4d99852da8a3d5167ab7464c0e71885250501
2019-10-02 13:45:02 +01:00
Lev Proleev
5a7b67ab8f Create NNAPI HAL v1.3 and add TENSOR_QUANT8_ASYMM_SIGNED OperandType
Bug: 137828494
Bug: 139120468
Bug: 136735770
Test: mma
Change-Id: I1f615047d1c0c208a90082ffb6ffc43f252f77b4
2019-10-02 13:44:52 +01:00
Michael Butler
0763328464 Change NNAPI VTS to use TEST_P to iterate across all service instances
This CL removes a dependency on the VTS test runner by dynamically
discovering all NN HAL service instances in the gtest binary itself,
and runs through all service instances with parameterized tests.

This CL converts TEST_F cases to TEST_P cases, where the test parameter
is the name of the service instance. For existing TEST_P cases (such as
the generated test cases), the service instance name is made to be the
first test parameter.

This CL enables the NN VTS tests to be more portable, e.g., they can
run directly as a presubmit test.

Fixes: 124540002
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Test: cd $ANDROID_BUILD_TOP/hardware/interfaces/neuralnetworks && atest
Change-Id: I1e301d7c9f9342bb8f35a267bef180f510944b19
Merged-In: I1e301d7c9f9342bb8f35a267bef180f510944b19
(cherry picked from commit 7076f629b7)
2019-09-20 11:20:48 -07:00
Michael Butler
b3c8032b12 Specify NNAPI Burst FMQ read/write requirements
This CL additionally adds missing documentation for timing
information when returning results on the resultChannel.

Bug: 133773876
Test: mma
Change-Id: I1eb1affbb4a912d5fdeab012e2be7e7005deb04d
Merged-In: I1eb1affbb4a912d5fdeab012e2be7e7005deb04d
(cherry picked from commit f99bffd08a)
2019-09-19 13:31:01 -07:00
Michael Butler
7b19ebb5c8 Merge "Change NNAPI VTS to use TEST_P to iterate across all service instances" 2019-09-19 17:46:56 +00:00
Michael Butler
7076f629b7 Change NNAPI VTS to use TEST_P to iterate across all service instances
This CL removes a dependency on the VTS test runner by dynamically
discovering all NN HAL service instances in the gtest binary itself,
and runs through all service instances with parameterized tests.

This CL converts TEST_F cases to TEST_P cases, where the test parameter
is the name of the service instance. For existing TEST_P cases (such as
the generated test cases), the service instance name is made to be the
first test parameter.

This CL enables the NN VTS tests to be more portable, e.g., they can
run directly as a presubmit test.

Fixes: 124540002
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Test: cd $ANDROID_BUILD_TOP/hardware/interfaces/neuralnetworks && atest
Change-Id: I1e301d7c9f9342bb8f35a267bef180f510944b19
2019-09-16 16:34:20 -07:00
Michael Butler
f99bffd08a Specify NNAPI Burst FMQ read/write requirements
This CL additionally adds missing documentation for timing
information when returning results on the resultChannel.

Bug: 133773876
Test: mma
Change-Id: I1eb1affbb4a912d5fdeab012e2be7e7005deb04d
2019-09-12 15:58:16 -07:00
David Gross
29d90d1455 "successful preparation means successful execution" guarantee neglects bad operation inputs
Under certain circumstances, we guarantee that a prepared model can be
executed successfully.  In describing those circumstances, we
neglected to specify that operation input operands must have legal
values for the guarantee to hold.  For example, the guarantee doesn't
hold if an ADD operation has an activation input that is not one of
the defined values; or if a RESHAPE operation has a shape input in
which two or more components are -1.

This change modifies the guarantee to apply only when operation input
operands have legal values.  It also documents this guarantee for
burst execution.

Note that if an operation has an input operand that can be proven to
have an illegal value at preparation time (e.g., a constant value that
is illegal), model preparation might (but is not required to) fail for
that reason.

Bug: 135933040

Test: $ cd neuralnetworks ; mma
Change-Id: I8b421550dd89e4bbbdae899e7cb5e9e88a46d2fb
(cherry picked from commit 48544cc38a)
2019-09-12 14:27:35 -07:00
Michael Butler
13b0516e56 Consolidate NNAPI VTS utility code
This CL does additional NNAPI VTS test cleanup, including consolidating
duplicate functionality. Specifically, this CL:
* consolidates the createPreparedModel function, removing the duplicate
* consolidates the std::out ErrorStatus and DeviceStatus code into Utils
* changes non-null constant pointers to constant references
* removes redudant leading namespace specifiers (V1_0::, ::testing, etc.)
* makes the Valdiation tests free functions
* renames device to kDevice and mTestModel to kTestModel

Bug: N/A
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Change-Id: Ic401bb1f1760cc10384ac0d30c0c93409b63a9c7
Merged-In: Ic401bb1f1760cc10384ac0d30c0c93409b63a9c7
(cherry picked from commit e16af0a44b)
2019-09-11 11:04:54 -07:00
David Gross
48544cc38a "successful preparation means successful execution" guarantee neglects bad operation inputs
Under certain circumstances, we guarantee that a prepared model can be
executed successfully.  In describing those circumstances, we
neglected to specify that operation input operands must have legal
values for the guarantee to hold.  For example, the guarantee doesn't
hold if an ADD operation has an activation input that is not one of
the defined values; or if a RESHAPE operation has a shape input in
which two or more components are -1.

This change modifies the guarantee to apply only when operation input
operands have legal values.  It also documents this guarantee for
burst execution.

Note that if an operation has an input operand that can be proven to
have an illegal value at preparation time (e.g., a constant value that
is illegal), model preparation might (but is not required to) fail for
that reason.

Bug: 135933040

Test: $ cd neuralnetworks ; mma
Change-Id: I8b421550dd89e4bbbdae899e7cb5e9e88a46d2fb
2019-09-09 14:35:49 -07:00
Michael Butler
e16af0a44b Consolidate NNAPI VTS utility code
This CL does additional NNAPI VTS test cleanup, including consolidating
duplicate functionality. Specifically, this CL:
* consolidates the createPreparedModel function, removing the duplicate
* consolidates the std::out ErrorStatus and DeviceStatus code into Utils
* changes non-null constant pointers to constant references
* removes redudant leading namespace specifiers (V1_0::, ::testing, etc.)
* makes the Valdiation tests free functions
* renames device to kDevice and mTestModel to kTestModel

Bug: N/A
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Change-Id: Ic401bb1f1760cc10384ac0d30c0c93409b63a9c7
2019-08-30 14:01:38 -07:00
Michael Butler
f952adda07 Cleanup NNAPI VTS tests
am: 62749b917e

Change-Id: Ia71e9765a810b2f07c4dbec5e101e357c64ba91d
2019-08-30 11:17:32 -07:00
Michael Butler
62749b917e Cleanup NNAPI VTS tests
This CL includes the following cleanups:
* namespace compression
* remove "using" from header files
* remove no-op code, default no-op constructors
* clang-formats the code

Bug: N/A
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I023997d8686ca65223858eed3a0881f5444ed2d6
Merged-In: I023997d8686ca65223858eed3a0881f5444ed2d6
(cherry picked from commit bbe5dad266)
2019-08-29 12:55:56 -07:00
David Gross
c9981f9592 Merge "Fix V1_2::IDevice::getType documentation: was @param instead of @return"
am: 15b826ad6a

Change-Id: I81b920f6c0edb6f1709af8d8b609e21d64e5755e
2019-08-29 11:14:21 -07:00
Michael Butler
8bc15b59ae Merge "Cleanup NNAPI VTS tests" 2019-08-29 18:04:01 +00:00
David Gross
15b826ad6a Merge "Fix V1_2::IDevice::getType documentation: was @param instead of @return" 2019-08-29 17:44:04 +00:00
Michael Butler
bbe5dad266 Cleanup NNAPI VTS tests
This CL includes the following cleanups:
* namespace compression
* remove "using" from header files
* remove no-op code, default no-op constructors
* clang-formats the code

Bug: N/A
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I023997d8686ca65223858eed3a0881f5444ed2d6
2019-08-28 17:29:40 -07:00
Michael Butler
52da1b820c NNAPI Combine Compatability test with VTS test
am: 3247f76e35

Change-Id: If447a9db2d0a9d5906396eb5f927a0df13f868ae
2019-08-28 14:12:36 -07:00
Michael Butler
ba2da6c2d3 Change NNAPI VTS to use static library
am: 01ba094edb

Change-Id: Ia6d6fc4f68c77a36c514025af4384bbe18b0db99
2019-08-28 14:12:28 -07:00
Xusong Wang
0ac1896ef5 Merge changes from topic "nnapi_test_dynamic_reg"
am: 6568622209

Change-Id: I298f137a70b07bdb7f9e6f85d8b293ec1bad7c86
2019-08-28 13:21:01 -07:00
Xusong Wang
5fb57da9a2 Update VTS tests with the new test harness.
am: bcaa78293a

Change-Id: I690d0111167d8ec1e7db98021a6f302dfe01f839
2019-08-28 13:20:49 -07:00
David Gross
9dedd439ab Fix V1_2::IDevice::getType documentation: was @param instead of @return
Bug: 124346013

Test: $ cd neuralnetworks/1.2 ; mma
Change-Id: I45d1320a9edae73cde7d9ccaca57b5b9519b0210
Merged-In: I45d1320a9edae73cde7d9ccaca57b5b9519b0210
(cherry picked from commit c7cdd550a0)
2019-08-28 11:40:01 -07:00
Michael Butler
3247f76e35 NNAPI Combine Compatability test with VTS test
Bug: 139889855
Fixes: 140120312
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Change-Id: Ib93d159dc657eaf292d16140ebdd9de3b0f5f1d3
Merged-In: Ib93d159dc657eaf292d16140ebdd9de3b0f5f1d3
(cherry picked from commit 72227f3ef8)
2019-08-28 11:35:29 -07:00
Michael Butler
01ba094edb Change NNAPI VTS to use static library
This CL also adds Compilation Caching tests to the Presubmit tests,
which were missing configuration files before.

Bug: 139889855
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Test: VtsHalNeuralnetworksV1_*CompatV1_*TargetTest (with sample-all)
Change-Id: I65274b5f72def815ff238a8d8df10c9d014afc44
Merged-In: I65274b5f72def815ff238a8d8df10c9d014afc44
(cherry picked from commit f7a8f03e3d)
2019-08-28 11:34:58 -07:00
David Gross
1f75c5cbaf Merge "Fix V1_2::IDevice::getType documentation: was @param instead of @return" 2019-08-28 18:21:32 +00:00