Commit graph

819 commits

Author SHA1 Message Date
Michael Butler
c16a7603fc Merge "NNAPI VTS: use max time point instead of uint64_t max" 2020-02-14 00:26:58 +00:00
Michael Butler
79d6bb1ad2 NNAPI VTS: use max time point instead of uint64_t max
Bug: 149441015
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ic74f83242cbe04a4cec47adbfc51002fdf8bb47d
2020-02-12 19:06:13 -08:00
Michael Butler
c3e1a29a60 NN 1.3 HAL: ANAPIC review follow up
This CL makes the following three interface changes:
* Removes @1.3::Operand.ExtraParams, because it was the same as
  @1.2::Operand.ExtraParams
* Changes int32_t token to uint32_t for IDevice::allocate and for
  @1.3::Request.MemoryPool
* Renames OptionalTimePoint::nanoseconds to
  OptionalTimePoint::nanosecondsSinceEpoch

This CL also makes test changes in response to the interface changes,
and fixes a minor comment issue in @1.2::types.hal and @1.3::types.hal.

Bug: 148617339
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I95b49ab34b0b79f587dda035ede700b73dc44a38
Merged-In: I95b49ab34b0b79f587dda035ede700b73dc44a38
(cherry picked from commit 6a4172cd3d)
2020-02-12 17:52:34 -08:00
Slava Shklyaev
1f98e2e929 Add control flow support to NNAPI VTS tests
See change I98a3edd1.

Bug: 148077633
Bug: 148601177
Bug: 136735929
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Test: VtsHalNeuralnetworksV1_2TargetTest
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I1e436cdba404b68026a45797ac4fb3a34f8be76a
2020-02-12 14:29:27 +00:00
Slava Shklyaev
8a9b306571 Add control flow performance to NNAPI Capabilities
Bug: 139181916
Test: m
Change-Id: I62696a6519191e33568c9664c0047fb57426b99f
2020-02-12 14:29:27 +00:00
Lev Proleev
0b0d69e74f Merge "NNAPI: Add state outputs for {UNI|BI}DIRECTIONAL_SEQUENCE_{RNN|LSTM}" 2020-02-12 11:12:57 +00:00
Miao Wang
aaa3c24f10 Update VTS tests for executeFenced
- Remove the DEVICE_UNAVAILABLE check
 - Pass deadline down to the driver.

Bug: 148979873
Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: If90a0dce8c8907a2f8f806455074fa27e3f397ef
Merged-In: If90a0dce8c8907a2f8f806455074fa27e3f397ef
(cherry picked from commit 15a25f6ec1)
2020-02-11 12:37:36 -08:00
Miao Wang
cc18d3fede Add a VTS test to validate NNAPI driver name string.
Bug: 134755158
Test: VtsHalNeuralnetworksV1_2TargetTest
Change-Id: Iead93f0a251fdbbf94df6dfdf27919d3f6fd71a8
Merged-In: Iead93f0a251fdbbf94df6dfdf27919d3f6fd71a8
(cherry picked from commit 2729d825cb)
2020-02-11 12:37:20 -08:00
Automerger Merge Worker
c231f0be30 Merge "Revert "Remove VtsHalNeuralnetworksV1_2TargetTest from vts-core suite"" am: 20a5d89241 am: 3251b30fb2 am: 38d5f26305
Change-Id: I0db62643a33ad66d76da15455afc7a4e35efd895
2020-02-11 19:27:18 +00:00
Treehugger Robot
20a5d89241 Merge "Revert "Remove VtsHalNeuralnetworksV1_2TargetTest from vts-core suite"" 2020-02-11 18:48:38 +00:00
Lev Proleev
188a8f90f0 NNAPI: Add state outputs for {UNI|BI}DIRECTIONAL_SEQUENCE_{RNN|LSTM}
Fix: 138443991
Test: NNTest_static and VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I0a3af22826d438f6f38de1f64042a50f98265e2d
2020-02-11 11:05:48 +00:00
Dan Shi
6e48bde237 Revert "Remove VtsHalNeuralnetworksV1_2TargetTest from vts-core suite"
This reverts commit 07f4235b53.

Reason for revert: b/144527956

Change-Id: I70c3e99689901556e279200751a90e77fb4fd840
2020-02-11 01:04:07 +00:00
Miao Wang
78743e19b2 Merge "Update VTS tests for executeFenced" 2020-02-10 21:51:45 +00:00
Michael Butler
aba3fc08f2 Merge "NN 1.3 HAL: ANAPIC review follow up" 2020-02-07 02:16:13 +00:00
Miao Wang
15a25f6ec1 Update VTS tests for executeFenced
- Remove the DEVICE_UNAVAILABLE check
 - Pass deadline down to the driver.

Bug: 148979873
Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: If90a0dce8c8907a2f8f806455074fa27e3f397ef
2020-02-06 16:10:54 -08:00
Miao Wang
e3966e907f Merge "Add a VTS test to validate NNAPI driver name string." 2020-02-06 23:09:55 +00:00
Michael Butler
68a868b2ac NNAPI HAL: Remove priority from prepareModelFromCache_1_3
prepareModelFromCache_1_3 uses model cache and data cache that
represent a prepared model. Any argument that contributes in a
meaningful way to the prepared model is implicitly included in
this model cache and data cache. For example, "model" and
"executionPreference" appear in prepareModel_1_3 but not in
prepareModelFromCache_1_3 because they are implicitly included
in the model cache and data cache. In a similar way, because
it could affect the resulting model, "priority" should be
removed from prepareModelFromCache_1_3.

Fixes: 148802784
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I518e493ead8aa97220b16370cce8523b425c378c
Merged-In: I518e493ead8aa97220b16370cce8523b425c378c
(cherry picked from commit c90250ff57)
2020-02-06 11:38:08 +00:00
Lev Proleev
124659e450 NNAPI: Add OptionalTimeoutDuration to 1.3/types.t
To sync 1.3/types.t with 1.3/types.hal

Test: generate_api.sh
Change-Id: I6a6e46ec82b4f25d77f7e1af18cc9d6f9478e5b7
Merged-In: I6a6e46ec82b4f25d77f7e1af18cc9d6f9478e5b7
(cherry picked from commit a9c097abec)
2020-02-06 11:38:06 +00:00
Miao Wang
2729d825cb Add a VTS test to validate NNAPI driver name string.
Bug: 134755158
Test: VtsHalNeuralnetworksV1_2TargetTest
Change-Id: Iead93f0a251fdbbf94df6dfdf27919d3f6fd71a8
2020-02-05 17:24:02 -08:00
David Gross
0db1b54e9a redefine _Float16 as __fp16 for x86[_64] to allow removing workaround from inhouse clang
At present, the tests might not actually use _Float16 (the generated
tests that are incorporated into the functional tests are built in
frameworks/ml rather than in hardware/interfaces); but the makefiles
are changed anyway, in case _Float16 is added in the future.

Bug: 138709788

Test: for (walleye, aosp_x86 [on emulator], aosp_x86_64 [on emulator])
        Use clang where _Float16 is disabled for x86[_64]
        $ cd $ANDROID_BUILD_TOP/neuralnetworks/1.0/vts/functional
        inspect build commands for tests as to whether or not
          expected options related to redefinition are present
        $ cd $ANDROID_BUILD_TOP/neuralnetworks/1.1/vts/functional
        inspect build commands for tests as to whether or not
          expected options related to redefinition are present

Change-Id: I9225b228947696d249771b3a35b1f82de9c26b6e
(cherry picked from commit 2452ee5a80)
2020-02-05 11:09:22 -08:00
David Gross
2ca858a491 Merge "redefine _Float16 as __fp16 for x86[_64] to allow removing workaround from inhouse clang" 2020-02-05 19:01:31 +00:00
Michael Butler
6a4172cd3d NN 1.3 HAL: ANAPIC review follow up
This CL makes the following three interface changes:
* Removes @1.3::Operand.ExtraParams, because it was the same as
  @1.2::Operand.ExtraParams
* Changes int32_t token to uint32_t for IDevice::allocate and for
  @1.3::Request.MemoryPool
* Renames OptionalTimePoint::nanoseconds to
  OptionalTimePoint::nanosecondsSinceEpoch

This CL also makes test changes in response to the interface changes,
and fixes a minor comment issue in @1.2::types.hal and @1.3::types.hal.

Bug: 148617339
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I95b49ab34b0b79f587dda035ede700b73dc44a38
2020-02-05 03:09:13 +00:00
Michael Butler
e8d82de36c Merge "NNAPI HAL: Remove priority from prepareModelFromCache_1_3" 2020-02-04 18:43:04 +00:00
Michael Butler
c90250ff57 NNAPI HAL: Remove priority from prepareModelFromCache_1_3
prepareModelFromCache_1_3 uses model cache and data cache that
represent a prepared model. Any argument that contributes in a
meaningful way to the prepared model is implicitly included in
this model cache and data cache. For example, "model" and
"executionPreference" appear in prepareModel_1_3 but not in
prepareModelFromCache_1_3 because they are implicitly included
in the model cache and data cache. In a similar way, because
it could affect the resulting model, "priority" should be
removed from prepareModelFromCache_1_3.

Fixes: 148802784
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I518e493ead8aa97220b16370cce8523b425c378c
2020-02-03 15:06:00 -08:00
David Gross
2452ee5a80 redefine _Float16 as __fp16 for x86[_64] to allow removing workaround from inhouse clang
At present, the tests might not actually use _Float16 (the generated
tests that are incorporated into the functional tests are built in
frameworks/ml rather than in hardware/interfaces); but the makefiles
are changed anyway, in case _Float16 is added in the future.

Bug: 138709788

Test: for (walleye, aosp_x86 [on emulator], aosp_x86_64 [on emulator])
        Use clang where _Float16 is disabled for x86[_64]
        $ cd $ANDROID_BUILD_TOP/neuralnetworks/1.0/vts/functional
        inspect build commands for tests as to whether or not
          expected options related to redefinition are present
        $ cd $ANDROID_BUILD_TOP/neuralnetworks/1.1/vts/functional
        inspect build commands for tests as to whether or not
          expected options related to redefinition are present

Change-Id: I9225b228947696d249771b3a35b1f82de9c26b6e
2020-02-03 11:55:38 -08:00
Lev Proleev
a9c097abec NNAPI: Add OptionalTimeoutDuration to 1.3/types.t
To sync 1.3/types.t with 1.3/types.hal

Test: generate_api.sh
Change-Id: I6a6e46ec82b4f25d77f7e1af18cc9d6f9478e5b7
2020-01-31 14:20:43 +00:00
Miao Wang
99753d9e90 Update sync fence related API for NNAPI
- Allow IPreparedModel::executeFenced to measure gated durations.
 - Allow IPreparedModel::executeFenced to specify an optional deadline
 and aa optional post-gate-release maximum duration.

Bug: 142778241
Bug: 141363565
Test: mm
Change-Id: I1687ec5543f93ed9bb5d7fcf14dbe15ed5b5cd0d
Merged-In: I1687ec5543f93ed9bb5d7fcf14dbe15ed5b5cd0d
(cherry picked from commit 3fd70b020e)
2020-01-30 15:23:23 -08:00
Miao Wang
3fd70b020e Update sync fence related API for NNAPI
- Allow IPreparedModel::executeFenced to measure gated durations.
 - Allow IPreparedModel::executeFenced to specify an optional deadline
 and aa optional post-gate-release maximum duration.

Bug: 142778241
Bug: 141363565
Test: mm
Change-Id: I1687ec5543f93ed9bb5d7fcf14dbe15ed5b5cd0d
2020-01-28 14:31:57 -08:00
Miao Wang
e51e4a9f26 Fix the NNAPI vts tests about validateExecuteFenced
- Skip the test if the driver reject executeFenced.
  - remove the -1 fd for hidl_handle as it causes error before reaching
  the driver.

Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Idc3f815040efccbbfa95b70d5d437441d0bd8682
Merged-In: Idc3f815040efccbbfa95b70d5d437441d0bd8682
(cherry picked from commit 33173a3e68)
2020-01-27 12:59:07 -08:00
Miao Wang
2b5c4cda77 Add VTS tests for NNAPI IPreparedModel::executeFenced
- Validation tests
  - Generated tests to exercise executeFenced path

Bug: 142778241
Test: mm
Change-Id: I509f0b5713fc86885d597940aae5ade0502c97ad
Merged-In: I509f0b5713fc86885d597940aae5ade0502c97ad
(cherry picked from commit 2c4e02329b)
2020-01-27 12:58:50 -08:00
Miao Wang
12ea0d4ed6 Support sync fence in NNAPI
- Add IPreparedModel::dispatchRequest to NNAPI 1.3 HAL
  - Add IDispatchExecutionCallback to allow clients query information
  related to the actual evaluation.

Bug: 142778241
Test: mm
Change-Id: I87cbb7f2aee87342b0418fce04eb4050e2bc1920
Merged-In: I87cbb7f2aee87342b0418fce04eb4050e2bc1920
(cherry picked from commit 90cf3dd37c)
2020-01-27 12:58:31 -08:00
Lev Proleev
53a51cb352 Add FILL and RANK ops
Bug: 148050168
Bug: 148049333
Test: NNTest_static and VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Iebdfa600d84e31532807740c21d95cae41c76ad5
Merged-In: Iebdfa600d84e31532807740c21d95cae41c76ad5
(cherry picked from commit 8b3f240ea3)
2020-01-27 12:58:14 -08:00
Lev Proleev
503d850369 Add ELU and HARD_SWISH
Bug: 147482068
Bug: 147481241
Test: NNTest_static and VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Iab8da2a666ad9775dfb53d9297e94962fb651353
Merged-In: Iab8da2a666ad9775dfb53d9297e94962fb651353
(cherry picked from commit aee67f83f9)
2020-01-27 12:57:58 -08:00
Lev Proleev
e4f1583337 NNAPI: Regenerate types.hal to sync with types.spec
Bug: 147765446
Test: mma
Change-Id: I2b41bd959b7084ecb89674c659c66ebb158ac77d
Merged-In: I2b41bd959b7084ecb89674c659c66ebb158ac77d
(cherry picked from commit 2d7c489f67)
2020-01-27 12:57:05 -08:00
Michael Butler
95899b3d39 Create VTS tests for QoS in NNAPI
Bug: 136739795
Bug: 142902514
Bug: 145300530
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: If3ab91cfb3158e4c33e809ff3b149dff47cda76f
Merged-In: If3ab91cfb3158e4c33e809ff3b149dff47cda76f
(cherry picked from commit 616701d3cd)
2020-01-27 12:51:42 -08:00
Michael Butler
79a41d77c0 Update NNAPI 1.3 VTS tests with new types
Bug: 136739795
Bug: 142902514
Bug: 145300530
Test: mma
Test: atest VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ie76da9dc9d6993a56bf644cfe20c5f5b421672c9
Merged-In: Ie76da9dc9d6993a56bf644cfe20c5f5b421672c9
(cherry picked from commit 9449a28b2f)
2020-01-27 12:51:34 -08:00
Michael Butler
ed8e77bf12 Add Quality of Service to NNAPI HAL
This CL makes the following changes:
* introduces a new Priority enum
* extends ErrorStatus with new error codes
* adds "supportsDeadline" method to IDevice
* adds priority and deadline arguments to IDevice::prepareModel*
* adds deadline argument to IPreparedModel::execute*
* updates IExecutionCallback with new ErrorStatus
* updates current.txt accordingly

Bug: 136739795
Bug: 142902514
Bug: 145300530
Test: mma
Change-Id: Iaa7877bde1f463635b8bbdb4e8a001d7b79b9c65
Merged-In: Iaa7877bde1f463635b8bbdb4e8a001d7b79b9c65
(cherry picked from commit c2499ecda1)
2020-01-27 12:51:25 -08:00
Slava Shklyaev
f8124a861f Add NNAPI control flow
Bug: 136735929
Bug: 139181916
Test: m
Change-Id: I7a75175f00fc98df626c40ea669021ccd40130e0
Merged-In: I7a75175f00fc98df626c40ea669021ccd40130e0
(cherry picked from commit a785a3faac)
2020-01-27 12:51:09 -08:00
Slava Shklyaev
351ca59582 Remove neuralnetworks@1.3::Model.Extension* in favor of 1.2 counterparts
The types are the same as in 1.2. No changes are expected.

Bug: 136735929
Test: m
Change-Id: I7431d2e9263fafa0e63b8b1b40f6715e3832d17c
Merged-In: I7431d2e9263fafa0e63b8b1b40f6715e3832d17c
(cherry picked from commit 8a179f3499)
2020-01-27 12:50:50 -08:00
Xusong Wang
e9da9852a5 Add memory domain VTS generated tests.
Bug: 141353602
Bug: 141363565
Test: 1.3 VTS
Change-Id: Ifc7eb3fd6f15e28ba403f02bdf66b4568bddcb64
Merged-In: Ifc7eb3fd6f15e28ba403f02bdf66b4568bddcb64
(cherry picked from commit 1f50e54cf8)
2020-01-24 14:25:32 -08:00
Miao Wang
33173a3e68 Fix the NNAPI vts tests about validateExecuteFenced
- Skip the test if the driver reject executeFenced.
  - remove the -1 fd for hidl_handle as it causes error before reaching
  the driver.

Test: mm
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Idc3f815040efccbbfa95b70d5d437441d0bd8682
2020-01-23 15:08:15 -08:00
Miao Wang
6604e51458 Merge changes from topic "nn_sync_fence"
* changes:
  Add VTS tests for NNAPI IPreparedModel::executeFenced
  Support sync fence in NNAPI
2020-01-23 08:01:42 +00:00
Miao Wang
2c4e02329b Add VTS tests for NNAPI IPreparedModel::executeFenced
- Validation tests
  - Generated tests to exercise executeFenced path

Bug: 142778241
Test: mm
Change-Id: I509f0b5713fc86885d597940aae5ade0502c97ad
2020-01-22 18:01:58 -08:00
Miao Wang
90cf3dd37c Support sync fence in NNAPI
- Add IPreparedModel::dispatchRequest to NNAPI 1.3 HAL
  - Add IDispatchExecutionCallback to allow clients query information
  related to the actual evaluation.

Bug: 142778241
Test: mm
Change-Id: I87cbb7f2aee87342b0418fce04eb4050e2bc1920
2020-01-22 09:45:47 -08:00
Lev Proleev
8b3f240ea3 Add FILL and RANK ops
Bug: 148050168
Bug: 148049333
Test: NNTest_static and VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Iebdfa600d84e31532807740c21d95cae41c76ad5
2020-01-22 17:16:43 +00:00
Lev Proleev
aee67f83f9 Add ELU and HARD_SWISH
Bug: 147482068
Bug: 147481241
Test: NNTest_static and VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Iab8da2a666ad9775dfb53d9297e94962fb651353
2020-01-22 17:16:41 +00:00
Lev Proleev
2d7c489f67 NNAPI: Regenerate types.hal to sync with types.spec
Bug: 147765446
Test: mma
Change-Id: I2b41bd959b7084ecb89674c659c66ebb158ac77d
2020-01-22 17:16:07 +00:00
Michael Butler
616701d3cd Create VTS tests for QoS in NNAPI
Bug: 136739795
Bug: 142902514
Bug: 145300530
Test: mma
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: If3ab91cfb3158e4c33e809ff3b149dff47cda76f
2020-01-21 16:02:05 -08:00
Michael Butler
9449a28b2f Update NNAPI 1.3 VTS tests with new types
Bug: 136739795
Bug: 142902514
Bug: 145300530
Test: mma
Test: atest VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ie76da9dc9d6993a56bf644cfe20c5f5b421672c9
2020-01-21 16:02:05 -08:00
Michael Butler
c2499ecda1 Add Quality of Service to NNAPI HAL
This CL makes the following changes:
* introduces a new Priority enum
* extends ErrorStatus with new error codes
* adds "supportsDeadline" method to IDevice
* adds priority and deadline arguments to IDevice::prepareModel*
* adds deadline argument to IPreparedModel::execute*
* updates IExecutionCallback with new ErrorStatus
* updates current.txt accordingly

Bug: 136739795
Bug: 142902514
Bug: 145300530
Test: mma
Change-Id: Iaa7877bde1f463635b8bbdb4e8a001d7b79b9c65
2020-01-21 15:55:49 -08:00
Xusong Wang
b345a4688f Memory Domain HAL: Define HAL APIs.
- Add and document memory domain HAL APIs.
- Make necessary changes to the existing VTS codes to make them work
  with V1_3::Request.

Bug: 141353602
Bug: 141363565
Test: mma
Test: NNT_static
Test: 1.3 VTS
Change-Id: Ia32555d4fef149fad4a79728981c5d9cca675a1a
Merged-In: Ia32555d4fef149fad4a79728981c5d9cca675a1a
(cherry picked from commit 931d5a18bc)
2020-01-21 15:15:39 -08:00
Slava Shklyaev
a785a3faac Add NNAPI control flow
Bug: 136735929
Bug: 139181916
Test: m
Change-Id: I7a75175f00fc98df626c40ea669021ccd40130e0
2020-01-21 13:39:22 +00:00
Slava Shklyaev
8a179f3499 Remove neuralnetworks@1.3::Model.Extension* in favor of 1.2 counterparts
The types are the same as in 1.2. No changes are expected.

Bug: 136735929
Test: m
Change-Id: I7431d2e9263fafa0e63b8b1b40f6715e3832d17c
2020-01-21 11:39:02 +00:00
Xusong Wang
1f50e54cf8 Add memory domain VTS generated tests.
Bug: 141353602
Bug: 141363565
Test: 1.3 VTS
Change-Id: Ifc7eb3fd6f15e28ba403f02bdf66b4568bddcb64
2020-01-16 16:11:38 -08:00
Xusong Wang
931d5a18bc Memory Domain HAL: Define HAL APIs.
- Add and document memory domain HAL APIs.
- Make necessary changes to the existing VTS codes to make them work
  with V1_3::Request.

Bug: 141353602
Bug: 141363565
Test: mma
Test: NNT_static
Test: 1.3 VTS
Change-Id: Ia32555d4fef149fad4a79728981c5d9cca675a1a
2020-01-16 16:10:13 -08:00
Lev Proleev
3c68b3412f Enable QuantizationCouplingTest in NNAPI VTS
Bug: 137828994
Test: NNTest_static and VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I9b4f20ab0287be9cae9fc1b69a8f64cc8f1996b0
Merged-In: I9b4f20ab0287be9cae9fc1b69a8f64cc8f1996b0
(cherry picked from commit baac15d5fd)
2020-01-16 13:22:33 +00:00
Lev Proleev
e4c8f5fe23 Fix quantized LSTM doc
The CL changes explicit prefixes for types in types.spec to a macro, so
that the prefixes of tensor types are correct both in NDK and HAL docs.

Bug: 144841609
Test: mma
Change-Id: I6d904bd3a858f555beed9270f141f080f96e429a
Merged-In: I6d904bd3a858f555beed9270f141f080f96e429a
(cherry picked from commit da5079c5cd)
2020-01-13 09:56:06 +00:00
Lev Proleev
da5079c5cd Fix quantized LSTM doc
The CL changes explicit prefixes for types in types.spec to a macro, so
that the prefixes of tensor types are correct both in NDK and HAL docs.

Bug: 144841609
Test: mma
Change-Id: I6d904bd3a858f555beed9270f141f080f96e429a
2020-01-10 17:01:43 +00:00
TreeHugger Robot
fde81ba688 Merge "Enable QuantizationCouplingTest in NNAPI VTS" 2020-01-10 14:03:33 +00:00
Viet Dang
232cfebd5b Add quantized LSTM op.
Bug: 144841609

Test: NeuralNetworksTest_static

Change-Id: I8aab51695382211e003d8fc442d33bae3006dc13
Merged-In: I8aab51695382211e003d8fc442d33bae3006dc13
(cherry picked from commit c8b0cd33b9)
2020-01-10 11:15:14 +00:00
Lev Proleev
baac15d5fd Enable QuantizationCouplingTest in NNAPI VTS
Bug: 137828994
Test: NNTest_static and VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I9b4f20ab0287be9cae9fc1b69a8f64cc8f1996b0
2020-01-09 16:40:41 +00:00
Viet Dang
c8b0cd33b9 Add quantized LSTM op.
Bug: 144841609

Test: NeuralNetworksTest_static

Change-Id: I8aab51695382211e003d8fc442d33bae3006dc13
2020-01-09 15:20:15 +00:00
Lev Proleev
c9d7f1cdc4 NNAPI: Update docs for ops with signed quantization
types.hal files are generated from current types.spec.

Bug: 136735770
Test: mma
Change-Id: Ide24606f8b1d5e755a71d46f230409942d8df7e9
Merged-In: Ide24606f8b1d5e755a71d46f230409942d8df7e9
(cherry picked from commit eb7f175613)
2020-01-07 13:36:03 +00:00
Lev Proleev
da779f322c Add quant8 signed generated tests
Fixes ValidateModel test for AXIS_ALIGNED_BBOX_TRANSFORM.

Bug: 136735770
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ie2959ba7258d8106d5b3dd36970181519b75e3b3
Merged-In: Ie2959ba7258d8106d5b3dd36970181519b75e3b3
(cherry picked from commit 16e8c9cc02)
2020-01-07 13:36:00 +00:00
Lev Proleev
673fdcf4e6 Add exceptions to the signed quantization coupling tests
Main commit is in frameworks/ml/nn. This one only renames
testModel.hasQuant8AsymmOperands() to
testModel.hasQuant8CoupledOperands().

Bug: 143935412
Bug: 143934188
Bug: 143935353
Bug: 143934467
Bug: 143934184
Test: quantization coupling tests in CTS and VTS
Change-Id: I0f7c331c355fe61ee06605786f09bc1ca7cdef57
Merged-In: I0f7c331c355fe61ee06605786f09bc1ca7cdef57
(cherry picked from commit 6a1d5e405f)
2020-01-07 13:35:58 +00:00
TreeHugger Robot
6744b76ef4 Merge "NNAPI: Update docs for ops with signed quantization" 2020-01-07 09:55:01 +00:00
Lev Proleev
ae643ae705 Add TENSOR_QUANT8_ASYMM_SIGNED support for DEQUANTIZE
Add TENSOR_QUANT8_ASYMM_SIGNED to the list of exceptions when mutating
DEQUANTIZE for validation.

Bug: 143934768
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I1b3b0a362d3949d4e31708388100d4794846ca3a
Merged-In: I1b3b0a362d3949d4e31708388100d4794846ca3a
(cherry picked from commit 2bd0b3339c)
2020-01-06 15:20:16 +00:00
Xusong Wang
d4a060bb29 NN HAL: Upgrade IPreparedModel::executeSynchronously to 1.3.
Bug: 143242728
Test: 1.3 VTS with sample driver
Change-Id: I4b74e8ac031c5d793cf7e5d66190734949367538
Merged-In: I4b74e8ac031c5d793cf7e5d66190734949367538
(cherry picked from commit ebd88ba8d2)
2020-01-06 15:20:14 +00:00
Przemyslaw Szczepaniak
2326dd1a88 Add TENSOR_QUANT8_ASYMM_SIGNED support for ARG(MIN|MAX)
Test: CTS/VTS
Change-Id: I4a90642f4d8500237fd41de76687fbe4c021b652
Merged-In: I4a90642f4d8500237fd41de76687fbe4c021b652
(cherry picked from commit e9f0b92568)
2020-01-06 15:20:10 +00:00
Lev Proleev
eb7f175613 NNAPI: Update docs for ops with signed quantization
types.hal files are generated from current types.spec.

Bug: 136735770
Test: mma
Change-Id: Ide24606f8b1d5e755a71d46f230409942d8df7e9
2020-01-06 12:52:06 +00:00
Lev Proleev
16e8c9cc02 Add quant8 signed generated tests
Fixes ValidateModel test for AXIS_ALIGNED_BBOX_TRANSFORM.

Bug: 136735770
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Ie2959ba7258d8106d5b3dd36970181519b75e3b3
2020-01-03 15:22:33 +00:00
Lev Proleev
6a1d5e405f Add exceptions to the signed quantization coupling tests
Main commit is in frameworks/ml/nn. This one only renames
testModel.hasQuant8AsymmOperands() to
testModel.hasQuant8CoupledOperands().

Bug: 143935412
Bug: 143934188
Bug: 143935353
Bug: 143934467
Bug: 143934184
Test: quantization coupling tests in CTS and VTS
Change-Id: I0f7c331c355fe61ee06605786f09bc1ca7cdef57
2020-01-02 18:34:03 +00:00
TreeHugger Robot
019b0c47e6 Merge "Add TENSOR_QUANT8_ASYMM_SIGNED support for DEQUANTIZE" 2019-12-12 12:15:05 +00:00
Xusong Wang
f4f2a0d6bf Merge "NN HAL: Upgrade IPreparedModel::executeSynchronously to 1.3." 2019-12-11 20:24:12 +00:00
Przemyslaw Szczepaniak
e9f0b92568 Add TENSOR_QUANT8_ASYMM_SIGNED support for ARG(MIN|MAX)
Test: CTS/VTS
Change-Id: I4a90642f4d8500237fd41de76687fbe4c021b652
2019-12-10 13:08:48 +00:00
Lev Proleev
af5fc2374d Fix use-after-free error in NNAPI VTS tests am: 56cda83f8c am: 0854f5f759
am: 5f688eb948

Change-Id: Ibea07442f2d230caef39f6a3f73e3acf40f5cf00
2019-12-09 12:08:38 -08:00
Lev Proleev
56cda83f8c Fix use-after-free error in NNAPI VTS tests
Replace std::initializer_list with std::vector.

Fix: 145705114
Test: VtsHalNeuralnetworksV1_[23]TargetTest
Change-Id: I92cbc56b2b8fbb36b3f3225e7cd0d4b05c84041e
2019-12-09 14:56:09 +00:00
Lev Proleev
2bd0b3339c Add TENSOR_QUANT8_ASYMM_SIGNED support for DEQUANTIZE
Add TENSOR_QUANT8_ASYMM_SIGNED to the list of exceptions when mutating
DEQUANTIZE for validation.

Bug: 143934768
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I1b3b0a362d3949d4e31708388100d4794846ca3a
2019-12-05 16:59:20 +00:00
Lev Proleev
5595da3741 Add TENSOR_QUANT8_ASYMM_SIGNED support for more ops
Updated:
  CONCATENATION
  REDUCE_MIN
  REDUCE_MAX
  STRIDED_SLICE
  RESHAPE
  SQUEEZE
  MEAN

Bug: 143934554
Bug: 143935314
Bug: 143934726
Bug: 143934771
Bug: 143934557
Bug: 143934306
Bug: 143935140
Test: quantization coupling tests in CTS and VTS
Change-Id: Ie73d070fd6f0a27185c1f26561710635b6e03b26
Merged-In: Ie73d070fd6f0a27185c1f26561710635b6e03b26
(cherry picked from commit 91134e7fb9)
2019-12-02 11:40:55 +00:00
Przemyslaw Szczepaniak
90fc2cc193 Add TENSOR_QUANT8_ASYMM_SIGNED support for more ops
Updated:
 BATCH_TO_SPACE_ND
 CHANNEL_SHUFFLE
 DEPTH_TO_SPACE
 GROUPED_CONV_2D
 PAD
 PAD_V2
 QUANTIZE
 RESIZE_BILINEAR
 RESIZE_NEAREST_NEIGHBOR
 SPACE_TO_BATCH_ND
 SPACE_TO_DEPTH
 SPLIT
 TILE
 TOPK_V2
 TRANSPOSE

Bug: 143934582
Bug: 143934585
Bug: 143934628
Bug: 143934630
Bug: 143934721
Bug: 143935039
Bug: 143935052
Bug: 143935113
Bug: 143935115
Bug: 143935141
Bug: 143935353
Bug: 143935355
Bug: 143935392
Bug: 143935394
Bug: 143935413
Test: quantization coupling tests in CTS and VTS
Change-Id: I7e1b65507ea0f7dcdfdb5fd98e7871d84f569ed7
Merged-In: I7e1b65507ea0f7dcdfdb5fd98e7871d84f569ed7
(cherry picked from commit 6d3cdc322b)
2019-12-02 11:40:40 +00:00
Lev Proleev
91134e7fb9 Add TENSOR_QUANT8_ASYMM_SIGNED support for more ops
Updated:
  CONCATENATION
  REDUCE_MIN
  REDUCE_MAX
  STRIDED_SLICE
  RESHAPE
  SQUEEZE
  MEAN

Bug: 143934554
Bug: 143935314
Bug: 143934726
Bug: 143934771
Bug: 143934557
Bug: 143934306
Bug: 143935140
Test: quantization coupling tests in CTS and VTS
Change-Id: Ie73d070fd6f0a27185c1f26561710635b6e03b26
2019-11-28 11:54:29 +00:00
Przemyslaw Szczepaniak
6d3cdc322b Add TENSOR_QUANT8_ASYMM_SIGNED support for more ops
Updated:
 BATCH_TO_SPACE_ND
 CHANNEL_SHUFFLE
 DEPTH_TO_SPACE
 GROUPED_CONV_2D
 PAD
 PAD_V2
 QUANTIZE
 RESIZE_BILINEAR
 RESIZE_NEAREST_NEIGHBOR
 SPACE_TO_BATCH_ND
 SPACE_TO_DEPTH
 SPLIT
 TILE
 TOPK_V2
 TRANSPOSE

Bug: 143934582
Bug: 143934585
Bug: 143934628
Bug: 143934630
Bug: 143934721
Bug: 143935039
Bug: 143935052
Bug: 143935113
Bug: 143935115
Bug: 143935141
Bug: 143935353
Bug: 143935355
Bug: 143935392
Bug: 143935394
Bug: 143935413
Test: quantization coupling tests in CTS and VTS
Change-Id: I7e1b65507ea0f7dcdfdb5fd98e7871d84f569ed7
2019-11-27 15:09:09 +00:00
Lev Proleev
cdb67453de Add TENSOR_QUANT8_ASYMM_SIGNED support for activations
Ops updated: RELU, RELU1, RELU6, TANH, LOGISTIC

Fix: 143933951
Fix: 143934720
Fix: 143933831
Fix: 143934770
Fix: 143934743
Test: quantization coupling tests in CTS and VTS
Change-Id: Id5e7a8c6b30463708bd93dbf6a3f30d05c2bcf40
Merged-In: Id5e7a8c6b30463708bd93dbf6a3f30d05c2bcf40
(cherry picked from commit 3466c78b04)
2019-11-27 14:55:49 +00:00
Lev Proleev
d1b3450b3d Add TENSOR_QUANT8_ASYMM_SIGNED support for pooling ops
* Update AVERAGE_POOL_2D and MAX_POOL_2D

Bug: 143934716
Bug: 143934303
Test: quantization coupling tests in CTS and VTS
Change-Id: I15f36d6f819c92f96a46c40f9cc1491627644278
Merged-In: I15f36d6f819c92f96a46c40f9cc1491627644278
(cherry picked from commit 3f17cc3daa)
2019-11-27 14:55:13 +00:00
Lev Proleev
53c79d179a Add TENSOR_QUANT8_ASYMM_SIGNED support to ADD, MUL and SUB.
Bug: 143934463
Bug: 143934627
Bug: 143935040
Test: NNTest_static and 1.3 VTS
Change-Id: I47ffc2c264390f3c6b98aa9567fb47f2c58333fe
Merged-In: I47ffc2c264390f3c6b98aa9567fb47f2c58333fe
(cherry picked from commit 122bc2c0cc)
2019-11-27 14:54:27 +00:00
Lev Proleev
3466c78b04 Add TENSOR_QUANT8_ASYMM_SIGNED support for activations
Ops updated: RELU, RELU1, RELU6, TANH, LOGISTIC

Change-Id: Id5e7a8c6b30463708bd93dbf6a3f30d05c2bcf40
Fix: 143933951
Fix: 143934720
Fix: 143933831
Fix: 143934770
Fix: 143934743
Test: quantization coupling tests in CTS and VTS
2019-11-21 17:35:30 +00:00
Lev Proleev
3f17cc3daa Add TENSOR_QUANT8_ASYMM_SIGNED support for pooling ops
* Update AVERAGE_POOL_2D and MAX_POOL_2D

Bug: 143934716
Bug: 143934303
Test: quantization coupling tests in CTS and VTS
Change-Id: I15f36d6f819c92f96a46c40f9cc1491627644278
2019-11-21 14:39:40 +00:00
Lev Proleev
3683c785f2 Add QUANT8_ASYMM_SIGNED support to SELECT op
Also fix repo hook complaining about comment formatting.

Bug: 143935354
Test: NNTest_static and VTS_1_3  with --gtest_filter="*QuantizationCouplingTest*select*"
Change-Id: I1b0d1e987ebccc6700dd172b4222f9996105513d
Merged-In: I1b0d1e987ebccc6700dd172b4222f9996105513d
(cherry picked from commit 04bf939c7a)
2019-11-21 12:06:36 +00:00
Lev Proleev
9226c1e910 Add quantization coupling test
Quantization coupling test is ran only on models that contain only one
operator and at least one of its inputs has a type of
TENSOR_QUANT8_ASYMM. The test verifies that a model with all the
operands converted to TENSOR_QUANT8_ASYMM_SIGNED will produce the same
result (OK/SKIPPED/FAILED).

Bug: 137828994
Test: VtsHalNeuralNetworksV1_3TargetTest --gtest_filter="*QuantizationCouplingTest*"
Change-Id: I5a2e09b2b1bd790e7e37dde133554e516155a34d
Merged-In: I5a2e09b2b1bd790e7e37dde133554e516155a34d
(cherry picked from commit c88f4c60a8)
2019-11-21 12:06:34 +00:00
Lev Proleev
0d4ba3f33a Combine test parameters into TestConfig structure
Test: VtsHalNeuralnetworksV1_2TargetTest --gtest_filter="GeneratedTests*"
Change-Id: I928aaa42e4745b4a8e0e461046e9632b052d0135
Merged-In: I928aaa42e4745b4a8e0e461046e9632b052d0135
(cherry picked from commit 66f598e10d)
2019-11-21 12:06:31 +00:00
Lev Proleev
3b60d80ff6 Add NNAPI 1.3 OperationType
* Add 1.3 OperationType.
* Remove "using V1_2::OperationType" from 1.3 VTS tests.
* Update current.txt

Test: mma
Change-Id: Ieaede9b7a6cecb16dbcc753f347f7ff04c009f20
Merged-In: Ieaede9b7a6cecb16dbcc753f347f7ff04c009f20
(cherry picked from commit 8774f10b76)
2019-11-21 12:06:27 +00:00
Lev Proleev
122bc2c0cc Add TENSOR_QUANT8_ASYMM_SIGNED support to ADD, MUL and SUB.
Bug: 143934463
Bug: 143934627
Bug: 143935040
Test: NNTest_static and 1.3 VTS
Change-Id: I47ffc2c264390f3c6b98aa9567fb47f2c58333fe
2019-11-20 11:15:02 +00:00
Lev Proleev
5c305e52ef Merge changes from topic "quant_coupling"
* changes:
  Add QUANT8_ASYMM_SIGNED support to SELECT op
  Add quantization coupling test
2019-11-20 10:56:14 +00:00
Xusong Wang
1b3f426648 NN HAL: Upgrade IPreparedModel::execute to 1.3.
Bug: 143242728
Test: 1.3 VTS with sample driver
Change-Id: I9ca1e28ddc97fe880a72885afe7afb6c93903697
Merged-In: I9ca1e28ddc97fe880a72885afe7afb6c93903697
(cherry picked from commit 62a760c32d)
2019-11-19 12:18:41 -08:00
Xusong Wang
cc47dffa57 NN HAL: Upgrade IPreparedModelCallback::notify to 1.3.
Bug: 143242728
Test: 1.3 VTS with sample driver
Change-Id: I56bc7a2fb179a9576036ad0c2aae0e1f41ec4e2c
Merged-In: I56bc7a2fb179a9576036ad0c2aae0e1f41ec4e2c
(cherry picked from commit 68c3234ad3)
2019-11-19 12:18:33 -08:00
Dan Shi
4c3a94d48d Merge "Remove VtsHalNeuralnetworksV1_2TargetTest from vts-core suite" am: 86c7aabcc0 am: 572970bcc6
am: 2327f621e7

Change-Id: Ibf66f98dd9d5d82d620f8907629eb56dae65ca81
2019-11-18 13:05:14 -08:00
Xusong Wang
ebd88ba8d2 NN HAL: Upgrade IPreparedModel::executeSynchronously to 1.3.
Bug: 143242728
Test: 1.3 VTS with sample driver
Change-Id: I4b74e8ac031c5d793cf7e5d66190734949367538
2019-11-18 11:20:37 -08:00
Lev Proleev
04bf939c7a Add QUANT8_ASYMM_SIGNED support to SELECT op
Also fix repo hook complaining about comment formatting.

Bug: 143935354
Test: NNTest_static and VTS_1_3  with --gtest_filter="*QuantizationCouplingTest*select*"
Change-Id: I1b0d1e987ebccc6700dd172b4222f9996105513d
2019-11-18 17:20:55 +00:00
Lev Proleev
c88f4c60a8 Add quantization coupling test
Quantization coupling test is ran only on models that contain only one
operator and at least one of its inputs has a type of
TENSOR_QUANT8_ASYMM. The test verifies that a model with all the
operands converted to TENSOR_QUANT8_ASYMM_SIGNED will produce the same
result (OK/SKIPPED/FAILED).

Bug: 137828994
Test: VtsHalNeuralNetworksV1_3TargetTest --gtest_filter="*QuantizationCouplingTest*"
Change-Id: I5a2e09b2b1bd790e7e37dde133554e516155a34d
2019-11-18 17:20:55 +00:00
Dan Shi
07f4235b53 Remove VtsHalNeuralnetworksV1_2TargetTest from vts-core suite
The test takes over 12 hours to run. Remove it from vts-core suite.

Bug: 144527956
Test: None
Change-Id: If37a7b04e6d7b044d1594dcb4df07432236a1cf3
2019-11-18 08:40:19 -08:00
Lev Proleev
66f598e10d Combine test parameters into TestConfig structure
Test: VtsHalNeuralnetworksV1_2TargetTest --gtest_filter="GeneratedTests*"
Change-Id: I928aaa42e4745b4a8e0e461046e9632b052d0135
2019-11-12 15:38:36 +00:00
Lev Proleev
8774f10b76 Add NNAPI 1.3 OperationType
* Add 1.3 OperationType.
* Remove "using V1_2::OperationType" from 1.3 VTS tests.
* Update current.txt

Test: mma
Change-Id: Ieaede9b7a6cecb16dbcc753f347f7ff04c009f20
2019-11-11 18:11:50 +00:00
Xusong Wang
62a760c32d NN HAL: Upgrade IPreparedModel::execute to 1.3.
Bug: 143242728
Test: 1.3 VTS with sample driver
Change-Id: I9ca1e28ddc97fe880a72885afe7afb6c93903697
2019-11-07 17:30:56 -08:00
Xusong Wang
68c3234ad3 NN HAL: Upgrade IPreparedModelCallback::notify to 1.3.
Bug: 143242728
Test: 1.3 VTS with sample driver
Change-Id: I56bc7a2fb179a9576036ad0c2aae0e1f41ec4e2c
2019-11-07 17:30:43 -08:00
Michael Butler
b2e17dd9cd Merge "Change NNAPI VTS TEST_MAPPING to restrict testing on sample drivers" am: 3d3c792bb8 am: fb196c4365
am: b64ab1daa4

Change-Id: Ie9ba34ca05702a7a254c6f07db5d975032e3ae86
2019-11-01 12:00:55 -07:00
Michael Butler
3076228e92 Merge "Add VtsHalNeuralnetworksV1_3TargetTest to TEST_MAPPING" am: 70798b6b10 am: ed9ade26de
am: 463118ec7d

Change-Id: Id0fc549c0c3de87c14f54f298c183d6a2c2e471f
2019-10-31 16:18:04 -07:00
Michael Butler
24f83d497b Merge "Simplify NNAPI VTS TEST_MAPPING options" am: 950dcb63cf am: cf57e42449
am: 2951b226a2

Change-Id: I98e6aafcac065e5040652acdf9865733b3898722
2019-10-30 11:26:33 -07:00
Michael Butler
c9c0d5bef9 Change NNAPI VTS TEST_MAPPING to restrict testing on sample drivers
Bug: N/A
Test: cd $TOP/hardware/interfaces/neuralnetworks && atest
Change-Id: If06740718e7cfebbe92058779b19282913cd5fc7
2019-10-29 19:08:38 -07:00
Michael Butler
62b76902fb Add VtsHalNeuralnetworksV1_3TargetTest to TEST_MAPPING
Bug: N/A
Test: cd $TOP/hardware/interfaces/neuralnetworks && atest
Change-Id: Ic903a45fbb6487f2b8b18dfd32d356c987a98b14
2019-10-29 19:08:27 -07:00
Michael Butler
3104464c7f Simplify NNAPI VTS TEST_MAPPING options
Bug: N/A
Test: cd $TOP/hardware/interfaces/neuralnetworks && atest
Change-Id: I1eb0f25fe13c641cc661484021629ecd811ed984
2019-10-29 18:43:17 -07:00
Michael Butler
648ada5fcf NNAPI VTS update in response to utility function change
The frameworks/ml/nn CL "Improve performance of Burst executions"
changed the signature of a utility function used by NNAPI's VTS tests.
This CL modifies the test accordingly.

Bug: 132073143
Test: mma
Test: VtsHalNeuralnetworksV1_2TargetTest (with sample-all)
Test: VtsHalNeuralnetworksV1_3TargetTest (with sample-all)
Change-Id: I56c3b04e6fec11d526e2a1ff5b896f31cb12eb5f
Merged-In: I56c3b04e6fec11d526e2a1ff5b896f31cb12eb5f
(cherry picked from commit 5756887a7b)
2019-10-28 13:17:08 -07:00
Lev Proleev
e5376fea95 Add template for 1.3 types.hal and regenerate it
After this update, types.hal v1.3 no longer refers to API level 30.

Test: none needed
Bug: 140132458
Change-Id: If707dfbcade6be1a0885fc21f6ddf47f3c27b244
Merged-In: If707dfbcade6be1a0885fc21f6ddf47f3c27b244
(cherry picked from commit e98e6793be)
2019-10-28 13:03:00 +00:00
Lev Proleev
26d1bc8a4e Modify NNAPI VTS tests to run on version 1.3
Bug: 139120468
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: I4654dc75c17f8801103015dc1da91663dfa28d52
Merged-In: I4654dc75c17f8801103015dc1da91663dfa28d52
(cherry picked from commit b49dadfb64)
2019-10-17 18:34:51 +01:00
Lev Proleev
13fdfcd44f Copy VTS tests from v1.2 to v1.3
So that it's easier to see what actually has changed in VTS tests for
version 1.3

Bug: 139120468
Test: m
Change-Id: I09797f5f3898501a008186a22dd411b00e9e2c67
Merged-In: I09797f5f3898501a008186a22dd411b00e9e2c67
(cherry picked from commit 3b13b55ac1)
2019-10-17 18:34:51 +01:00
Lev Proleev
ce8c632a09 Create NNAPI HAL v1.3 and add TENSOR_QUANT8_ASYMM_SIGNED OperandType
Bug: 137828494
Bug: 139120468
Bug: 136735770
Test: mma
Change-Id: I28f74e4b364fec1d7431a96cf5687256b3106069
Merged-In: I28f74e4b364fec1d7431a96cf5687256b3106069
(cherry picked from commit 5a7b67ab8f)
2019-10-17 18:34:51 +01:00
Dan Shi
ecbc1ab590 Merge "Add VtsHalLightV2_0TargetTest and VtsHalNeuralnetworksV1_* to vts-core" am: 0e1efc568c am: 7626cc8cd5 am: 86395146bf
am: 9154805dce

Change-Id: Ic87df0df72d87be833bc408af5bc6a9139dcf7f4
2019-10-14 09:12:32 -07:00
Dan Shi
0e1efc568c Merge "Add VtsHalLightV2_0TargetTest and VtsHalNeuralnetworksV1_* to vts-core" 2019-10-14 15:35:48 +00:00
Michael Butler
50d76d6cfe Merge "NNAPI VTS update in response to utility function change" 2019-10-11 17:31:20 +00:00
Colin Cross
552c7a66ea Revert "Revert "Modify NNAPI VTS tests to run on version 1.3" am: 64f9eb4bd5 am: 7973d7dec5 am: 8bde98b027"
This reverts commit 496320b1ec.

Reason for revert: revert of a cherry-pick broke master

Change-Id: I4654dc75c17f8801103015dc1da91663dfa28d52
2019-10-10 23:13:36 +00:00
Colin Cross
b098d21e0c Revert "Revert "Copy VTS tests from v1.2 to v1.3" am: 9613b4649b am: 432f6d1609 am: fe2cd91115"
This reverts commit 1ea542bd1a.

Reason for revert: revert of a cherry-pick broke master

Change-Id: I09797f5f3898501a008186a22dd411b00e9e2c67
2019-10-10 23:13:02 +00:00
Lev Proleev
1ea542bd1a Revert "Copy VTS tests from v1.2 to v1.3" am: 9613b4649b am: 432f6d1609 am: fe2cd91115
am: 42fb310936

Change-Id: Icf12606c6d9d8efae82e35de09581fcdfa24f6f6
2019-10-10 14:54:49 -07:00
Lev Proleev
496320b1ec Revert "Modify NNAPI VTS tests to run on version 1.3" am: 64f9eb4bd5 am: 7973d7dec5 am: 8bde98b027
am: edadec3f9e

Change-Id: Idb6d7245c58afd0aba52db67cb1a540604ed8e33
2019-10-10 14:54:42 -07:00
Dan Shi
4856370e54 Add VtsHalLightV2_0TargetTest and VtsHalNeuralnetworksV1_* to vts-core
Bug: None
Test: None
Change-Id: I3fbbba0dc283bec2de025f0b743d97ded5874bf9
2019-10-10 13:45:43 -07:00
Lev Proleev
839ca3c428 Revert "Create NNAPI HAL v1.3 and add TENSOR_QUANT8_ASYMM_SIGNED OperandType"
This reverts commit bd2b4e78ac.

Reason for revert: the topic breaks git_qt-dev-plus-aosp

Change-Id: Ic72dcbb7ac4ef8e6ed597eb778289b39475f03f8
2019-10-10 14:01:05 +00:00
Lev Proleev
9613b4649b Revert "Copy VTS tests from v1.2 to v1.3"
This reverts commit d0e339e1da.

Reason for revert: the topic breaks git_qt-dev-plus-aosp

Change-Id: I68d31526cc7699b9a6c69a1ccc669fdb7605124e
2019-10-10 13:58:56 +00:00
Lev Proleev
64f9eb4bd5 Revert "Modify NNAPI VTS tests to run on version 1.3"
This reverts commit 543606fb92.

Reason for revert: the topic breaks git_qt-dev-plus-aosp

Change-Id: I74859bae687d65d344ed7edc617e1a7d086960a1
2019-10-10 13:54:38 +00:00
Michael Butler
5756887a7b NNAPI VTS update in response to utility function change
The frameworks/ml/nn CL "Improve performance of Burst executions"
changed the signature of a utility function used by NNAPI's VTS tests.
This CL modifies the test accordingly.

Bug: 132073143
Test: mma
Test: VtsHalNeuralnetworksV1_2TargetTest (with sample-all)
Test: VtsHalNeuralnetworksV1_3TargetTest (with sample-all)
Change-Id: I56c3b04e6fec11d526e2a1ff5b896f31cb12eb5f
2019-10-09 11:15:08 -07:00
Lev Proleev
e98e6793be Add template for 1.3 types.hal and regenerate it
After this update, types.hal v1.3 no longer refers to API level 30.

Test: none needed
Bug: 140132458
Change-Id: If707dfbcade6be1a0885fc21f6ddf47f3c27b244
2019-10-09 16:32:11 +01:00
Lev Proleev
543606fb92 Modify NNAPI VTS tests to run on version 1.3
Bug: 139120468
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Id9e4d99852da8a3d5167ab7464c0e71885250501
Merged-In: Id9e4d99852da8a3d5167ab7464c0e71885250501
(cherry picked from commit b49dadfb64)
2019-10-09 14:18:49 +01:00
Lev Proleev
d0e339e1da Copy VTS tests from v1.2 to v1.3
So that it's easier to see what actually has changed in VTS tests for
version 1.3

Bug: 139120468
Test: m
Change-Id: Ief294d21349ca6531595612a16fa3ae3382f83ac
Merged-In: Ief294d21349ca6531595612a16fa3ae3382f83ac
(cherry picked from commit 3b13b55ac1)
2019-10-09 14:18:49 +01:00
Lev Proleev
bd2b4e78ac Create NNAPI HAL v1.3 and add TENSOR_QUANT8_ASYMM_SIGNED OperandType
Bug: 137828494
Bug: 139120468
Bug: 136735770
Test: mma
Change-Id: I1f615047d1c0c208a90082ffb6ffc43f252f77b4
Merged-In: I1f615047d1c0c208a90082ffb6ffc43f252f77b4
(cherry picked from commit 5a7b67ab8f)
2019-10-09 14:18:49 +01:00
David Gross
8c0a48bceb Replace sync_enums_to_hal.py with generate_api.{py,sh} and regenerate */types.hal
There are certain pieces of NeuralNetworks.h and of our various *.hal
files that ought to be kept in sync -- most notably the operand type
and operation type definitions and descriptions in our
NeuralNetworks.h and types.hal files.  To avoid having to do this
manually, a tool can be employed.

The old solution to this problem is sync_enums_to_hal.py, which parses
the existing NeuralNetworks.h and types.hal files, and then rewrites
the types.hal files: It identifies the operand type and operation type
portions of the files, and in the types.hal files replaces them with
transformed versions of the portions from NeuralNetworks.h.  This
approach is brittle -- it is very sensitive to the structure of the
files, as was recognized when this tool was created.  Changes to those
files since the script was last updated essentially broke the script,
as noted in http://b/140132458.

The new solution employs a new tool generate_api.py to combine a
single "specification file" with one "template file" per API file
(NeuralNetworks.h or types.hal) to produce that API file.  The script
generate_api.sh invokes generate_api.py once per API file, passing
appropriate arguments.  See frameworks/ml/nn/tools/api/README.md for
more details.

In the process of combining information from NeuralNetworks.h and
*/types.hal, some formatting, syntactic, and semantic changes have
been made to those files.  For example:
- types.hal files no longer refer to API levels or (similarly)
  to HAL versions other than earlier HAL versions

Bug: 130169064
Bug: 140132458

Test: cd neuralnetworks ; mma

Change-Id: Iceb38effc2c7e6c32344e49225b85256144c0945
Merged-In: Iceb38effc2c7e6c32344e49225b85256144c0945
(cherry picked from commit 2cae5c8b01)
2019-10-09 14:18:06 +01:00
Lev Proleev
b58075603a Merge changes from topic "NNAPI v1.3"
* changes:
  Modify NNAPI VTS tests to run on version 1.3
  Copy VTS tests from v1.2 to v1.3
  Create NNAPI HAL v1.3 and add TENSOR_QUANT8_ASYMM_SIGNED OperandType
2019-10-04 15:34:10 +00:00
David Gross
2cae5c8b01 Replace sync_enums_to_hal.py with generate_api.{py,sh} and regenerate */types.hal
There are certain pieces of NeuralNetworks.h and of our various *.hal
files that ought to be kept in sync -- most notably the operand type
and operation type definitions and descriptions in our
NeuralNetworks.h and types.hal files.  To avoid having to do this
manually, a tool can be employed.

The old solution to this problem is sync_enums_to_hal.py, which parses
the existing NeuralNetworks.h and types.hal files, and then rewrites
the types.hal files: It identifies the operand type and operation type
portions of the files, and in the types.hal files replaces them with
transformed versions of the portions from NeuralNetworks.h.  This
approach is brittle -- it is very sensitive to the structure of the
files, as was recognized when this tool was created.  Changes to those
files since the script was last updated essentially broke the script,
as noted in http://b/140132458.

The new solution employs a new tool generate_api.py to combine a
single "specification file" with one "template file" per API file
(NeuralNetworks.h or types.hal) to produce that API file.  The script
generate_api.sh invokes generate_api.py once per API file, passing
appropriate arguments.  See frameworks/ml/nn/tools/api/README.md for
more details.

In the process of combining information from NeuralNetworks.h and
*/types.hal, some formatting, syntactic, and semantic changes have
been made to those files.  For example:
- types.hal files no longer refer to API levels or (similarly)
  to HAL versions other than earlier HAL versions

Bug: 130169064
Bug: 140132458

Test: cd neuralnetworks ; mma

Change-Id: Iceb38effc2c7e6c32344e49225b85256144c0945
2019-10-03 14:14:20 -07:00
Lev Proleev
b49dadfb64 Modify NNAPI VTS tests to run on version 1.3
Bug: 139120468
Test: VtsHalNeuralnetworksV1_3TargetTest
Change-Id: Id9e4d99852da8a3d5167ab7464c0e71885250501
2019-10-02 13:45:02 +01:00
Lev Proleev
3b13b55ac1 Copy VTS tests from v1.2 to v1.3
So that it's easier to see what actually has changed in VTS tests for
version 1.3

Bug: 139120468
Test: m
Change-Id: Ief294d21349ca6531595612a16fa3ae3382f83ac
2019-10-02 13:45:02 +01:00
Lev Proleev
5a7b67ab8f Create NNAPI HAL v1.3 and add TENSOR_QUANT8_ASYMM_SIGNED OperandType
Bug: 137828494
Bug: 139120468
Bug: 136735770
Test: mma
Change-Id: I1f615047d1c0c208a90082ffb6ffc43f252f77b4
2019-10-02 13:44:52 +01:00
Michael Butler
0763328464 Change NNAPI VTS to use TEST_P to iterate across all service instances
This CL removes a dependency on the VTS test runner by dynamically
discovering all NN HAL service instances in the gtest binary itself,
and runs through all service instances with parameterized tests.

This CL converts TEST_F cases to TEST_P cases, where the test parameter
is the name of the service instance. For existing TEST_P cases (such as
the generated test cases), the service instance name is made to be the
first test parameter.

This CL enables the NN VTS tests to be more portable, e.g., they can
run directly as a presubmit test.

Fixes: 124540002
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Test: cd $ANDROID_BUILD_TOP/hardware/interfaces/neuralnetworks && atest
Change-Id: I1e301d7c9f9342bb8f35a267bef180f510944b19
Merged-In: I1e301d7c9f9342bb8f35a267bef180f510944b19
(cherry picked from commit 7076f629b7)
2019-09-20 11:20:48 -07:00
Michael Butler
b3c8032b12 Specify NNAPI Burst FMQ read/write requirements
This CL additionally adds missing documentation for timing
information when returning results on the resultChannel.

Bug: 133773876
Test: mma
Change-Id: I1eb1affbb4a912d5fdeab012e2be7e7005deb04d
Merged-In: I1eb1affbb4a912d5fdeab012e2be7e7005deb04d
(cherry picked from commit f99bffd08a)
2019-09-19 13:31:01 -07:00
Michael Butler
7b19ebb5c8 Merge "Change NNAPI VTS to use TEST_P to iterate across all service instances" 2019-09-19 17:46:56 +00:00
Michael Butler
7076f629b7 Change NNAPI VTS to use TEST_P to iterate across all service instances
This CL removes a dependency on the VTS test runner by dynamically
discovering all NN HAL service instances in the gtest binary itself,
and runs through all service instances with parameterized tests.

This CL converts TEST_F cases to TEST_P cases, where the test parameter
is the name of the service instance. For existing TEST_P cases (such as
the generated test cases), the service instance name is made to be the
first test parameter.

This CL enables the NN VTS tests to be more portable, e.g., they can
run directly as a presubmit test.

Fixes: 124540002
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Test: cd $ANDROID_BUILD_TOP/hardware/interfaces/neuralnetworks && atest
Change-Id: I1e301d7c9f9342bb8f35a267bef180f510944b19
2019-09-16 16:34:20 -07:00
Michael Butler
f99bffd08a Specify NNAPI Burst FMQ read/write requirements
This CL additionally adds missing documentation for timing
information when returning results on the resultChannel.

Bug: 133773876
Test: mma
Change-Id: I1eb1affbb4a912d5fdeab012e2be7e7005deb04d
2019-09-12 15:58:16 -07:00
David Gross
29d90d1455 "successful preparation means successful execution" guarantee neglects bad operation inputs
Under certain circumstances, we guarantee that a prepared model can be
executed successfully.  In describing those circumstances, we
neglected to specify that operation input operands must have legal
values for the guarantee to hold.  For example, the guarantee doesn't
hold if an ADD operation has an activation input that is not one of
the defined values; or if a RESHAPE operation has a shape input in
which two or more components are -1.

This change modifies the guarantee to apply only when operation input
operands have legal values.  It also documents this guarantee for
burst execution.

Note that if an operation has an input operand that can be proven to
have an illegal value at preparation time (e.g., a constant value that
is illegal), model preparation might (but is not required to) fail for
that reason.

Bug: 135933040

Test: $ cd neuralnetworks ; mma
Change-Id: I8b421550dd89e4bbbdae899e7cb5e9e88a46d2fb
(cherry picked from commit 48544cc38a)
2019-09-12 14:27:35 -07:00
Michael Butler
13b0516e56 Consolidate NNAPI VTS utility code
This CL does additional NNAPI VTS test cleanup, including consolidating
duplicate functionality. Specifically, this CL:
* consolidates the createPreparedModel function, removing the duplicate
* consolidates the std::out ErrorStatus and DeviceStatus code into Utils
* changes non-null constant pointers to constant references
* removes redudant leading namespace specifiers (V1_0::, ::testing, etc.)
* makes the Valdiation tests free functions
* renames device to kDevice and mTestModel to kTestModel

Bug: N/A
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Change-Id: Ic401bb1f1760cc10384ac0d30c0c93409b63a9c7
Merged-In: Ic401bb1f1760cc10384ac0d30c0c93409b63a9c7
(cherry picked from commit e16af0a44b)
2019-09-11 11:04:54 -07:00
David Gross
48544cc38a "successful preparation means successful execution" guarantee neglects bad operation inputs
Under certain circumstances, we guarantee that a prepared model can be
executed successfully.  In describing those circumstances, we
neglected to specify that operation input operands must have legal
values for the guarantee to hold.  For example, the guarantee doesn't
hold if an ADD operation has an activation input that is not one of
the defined values; or if a RESHAPE operation has a shape input in
which two or more components are -1.

This change modifies the guarantee to apply only when operation input
operands have legal values.  It also documents this guarantee for
burst execution.

Note that if an operation has an input operand that can be proven to
have an illegal value at preparation time (e.g., a constant value that
is illegal), model preparation might (but is not required to) fail for
that reason.

Bug: 135933040

Test: $ cd neuralnetworks ; mma
Change-Id: I8b421550dd89e4bbbdae899e7cb5e9e88a46d2fb
2019-09-09 14:35:49 -07:00
Michael Butler
e16af0a44b Consolidate NNAPI VTS utility code
This CL does additional NNAPI VTS test cleanup, including consolidating
duplicate functionality. Specifically, this CL:
* consolidates the createPreparedModel function, removing the duplicate
* consolidates the std::out ErrorStatus and DeviceStatus code into Utils
* changes non-null constant pointers to constant references
* removes redudant leading namespace specifiers (V1_0::, ::testing, etc.)
* makes the Valdiation tests free functions
* renames device to kDevice and mTestModel to kTestModel

Bug: N/A
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Change-Id: Ic401bb1f1760cc10384ac0d30c0c93409b63a9c7
2019-08-30 14:01:38 -07:00
Michael Butler
f952adda07 Cleanup NNAPI VTS tests
am: 62749b917e

Change-Id: Ia71e9765a810b2f07c4dbec5e101e357c64ba91d
2019-08-30 11:17:32 -07:00
Michael Butler
62749b917e Cleanup NNAPI VTS tests
This CL includes the following cleanups:
* namespace compression
* remove "using" from header files
* remove no-op code, default no-op constructors
* clang-formats the code

Bug: N/A
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I023997d8686ca65223858eed3a0881f5444ed2d6
Merged-In: I023997d8686ca65223858eed3a0881f5444ed2d6
(cherry picked from commit bbe5dad266)
2019-08-29 12:55:56 -07:00
David Gross
c9981f9592 Merge "Fix V1_2::IDevice::getType documentation: was @param instead of @return"
am: 15b826ad6a

Change-Id: I81b920f6c0edb6f1709af8d8b609e21d64e5755e
2019-08-29 11:14:21 -07:00
Michael Butler
8bc15b59ae Merge "Cleanup NNAPI VTS tests" 2019-08-29 18:04:01 +00:00
David Gross
15b826ad6a Merge "Fix V1_2::IDevice::getType documentation: was @param instead of @return" 2019-08-29 17:44:04 +00:00
Michael Butler
bbe5dad266 Cleanup NNAPI VTS tests
This CL includes the following cleanups:
* namespace compression
* remove "using" from header files
* remove no-op code, default no-op constructors
* clang-formats the code

Bug: N/A
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest
Change-Id: I023997d8686ca65223858eed3a0881f5444ed2d6
2019-08-28 17:29:40 -07:00
Michael Butler
52da1b820c NNAPI Combine Compatability test with VTS test
am: 3247f76e35

Change-Id: If447a9db2d0a9d5906396eb5f927a0df13f868ae
2019-08-28 14:12:36 -07:00
Michael Butler
ba2da6c2d3 Change NNAPI VTS to use static library
am: 01ba094edb

Change-Id: Ia6d6fc4f68c77a36c514025af4384bbe18b0db99
2019-08-28 14:12:28 -07:00
Xusong Wang
0ac1896ef5 Merge changes from topic "nnapi_test_dynamic_reg"
am: 6568622209

Change-Id: I298f137a70b07bdb7f9e6f85d8b293ec1bad7c86
2019-08-28 13:21:01 -07:00
Xusong Wang
5fb57da9a2 Update VTS tests with the new test harness.
am: bcaa78293a

Change-Id: I690d0111167d8ec1e7db98021a6f302dfe01f839
2019-08-28 13:20:49 -07:00
David Gross
9dedd439ab Fix V1_2::IDevice::getType documentation: was @param instead of @return
Bug: 124346013

Test: $ cd neuralnetworks/1.2 ; mma
Change-Id: I45d1320a9edae73cde7d9ccaca57b5b9519b0210
Merged-In: I45d1320a9edae73cde7d9ccaca57b5b9519b0210
(cherry picked from commit c7cdd550a0)
2019-08-28 11:40:01 -07:00
Michael Butler
3247f76e35 NNAPI Combine Compatability test with VTS test
Bug: 139889855
Fixes: 140120312
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Change-Id: Ib93d159dc657eaf292d16140ebdd9de3b0f5f1d3
Merged-In: Ib93d159dc657eaf292d16140ebdd9de3b0f5f1d3
(cherry picked from commit 72227f3ef8)
2019-08-28 11:35:29 -07:00
Michael Butler
01ba094edb Change NNAPI VTS to use static library
This CL also adds Compilation Caching tests to the Presubmit tests,
which were missing configuration files before.

Bug: 139889855
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Test: VtsHalNeuralnetworksV1_*CompatV1_*TargetTest (with sample-all)
Change-Id: I65274b5f72def815ff238a8d8df10c9d014afc44
Merged-In: I65274b5f72def815ff238a8d8df10c9d014afc44
(cherry picked from commit f7a8f03e3d)
2019-08-28 11:34:58 -07:00
David Gross
1f75c5cbaf Merge "Fix V1_2::IDevice::getType documentation: was @param instead of @return" 2019-08-28 18:21:32 +00:00
Michael Butler
72227f3ef8 NNAPI Combine Compatability test with VTS test
Bug: 139889855
Fixes: 140120312
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Change-Id: Ib93d159dc657eaf292d16140ebdd9de3b0f5f1d3
2019-08-27 23:33:41 +00:00
Michael Butler
05f50b3d52 Merge "Change NNAPI VTS to use static library" 2019-08-27 23:32:00 +00:00
Xusong Wang
ee80b02ae9 Statically check test enums against HIDL definitions.
Fixes: 123092187
Test: mma all VTS
Test: Attempt to modify test enum value,
      confirm that there is build error with proper error message.

Change-Id: I0129378c87637b33bb145ec93307634273a9d2fd
Merged-In: I0129378c87637b33bb145ec93307634273a9d2fd
(cherry picked from commit cbfa6b998e)
2019-08-27 16:09:30 -07:00
Xusong Wang
bcaa78293a Update VTS tests with the new test harness.
Bug: 120601396
Test: All VTS
Change-Id: I539e75585b2cc01d153565814491361adfa048be
Merged-In: I539e75585b2cc01d153565814491361adfa048be
(cherry picked from commit 9e2b97b83c)
2019-08-27 16:09:12 -07:00
Michael Butler
f7a8f03e3d Change NNAPI VTS to use static library
This CL also adds Compilation Caching tests to the Presubmit tests,
which were missing configuration files before.

Bug: 139889855
Test: mma
Test: VtsHalNeuralnetworksV1_*TargetTest (with sample-all)
Test: VtsHalNeuralnetworksV1_*CompatV1_*TargetTest (with sample-all)
Change-Id: I65274b5f72def815ff238a8d8df10c9d014afc44
2019-08-27 21:59:16 +00:00
David Gross
c7cdd550a0 Fix V1_2::IDevice::getType documentation: was @param instead of @return
Bug: 124346013

Test: $ cd neuralnetworks/1.2 ; mma
Change-Id: I45d1320a9edae73cde7d9ccaca57b5b9519b0210
2019-08-27 12:22:01 -07:00
TreeHugger Robot
c6145e4327 Merge "Statically check test enums against HIDL definitions." 2019-08-27 18:04:32 +00:00
Xusong Wang
323ba2ee8b Modify validateBurstSanitized to take a single request.
This CL is to modify validateBurstSanitized implementation in accordance
with the new test generator and test harness change in the same topic.

Bug: 123092187
Test: 1.2 VTS
Change-Id: Iacfe0e3bba8b4b44b7aee023d1c8129cbd089d4a
Merged-In: Iacfe0e3bba8b4b44b7aee023d1c8129cbd089d4a
(cherry picked from commit 8fc4622bf1)
2019-08-27 10:45:03 -07:00
Xusong Wang
6d0270b056 Modify 1.2 VTS tests to consume test struct directly.
Comparing with v1.1, the converter for 1.2 HIDL model has additional support
for extraParam, dynamic output shape, and zero-sized output.

Modify CompilationCachingTests to use the new test struct.

Bug: 123092187
Bug: 138718240
Test: All VTS
Change-Id: I54ac97f62898e47a338b51cc6d895a0309ab001f
Merged-In: I54ac97f62898e47a338b51cc6d895a0309ab001f
(cherry picked from commit 491b0a8913)
2019-08-27 10:44:29 -07:00
Xusong Wang
0c03b85005 Modify 1.1 VTS tests to consume test struct directly.
This CL is very similar to the 1.0 VTS CL. The only difference is that
1.1 Models have an additional field for relaxed model computation.

Bug: 123092187
Bug: 138718240
Test: All VTS
Change-Id: I9264e5b2468b9c6db47d86683d24f8c2c5ec46aa
Merged-In: I9264e5b2468b9c6db47d86683d24f8c2c5ec46aa
(cherry picked from commit 6aad040278)
2019-08-27 10:43:51 -07:00
Xusong Wang
8487c5e21f Modify 1.0 VTS tests to consume test struct directly.
Implement converter utilities constructing HIDL model and request from
TestModel.

Bug: 123092187
Bug: 138718240
Test: All VTS
Change-Id: I0b26b7f41d31d5e63ed083ab5f6f269a3620f034
Merged-In: I0b26b7f41d31d5e63ed083ab5f6f269a3620f034
(cherry picked from commit 8e8b70c439)
2019-08-27 10:43:45 -07:00
Xusong Wang
ead950d402 Modify 1.2 VTS tests to consume test struct directly.
Comparing with v1.1, the converter for 1.2 HIDL model has additional support
for extraParam, dynamic output shape, and zero-sized output.

Modify CompilationCachingTests to use the new test struct.

Bug: 123092187
Bug: 138718240
Test: All VTS
Change-Id: I54ac97f62898e47a338b51cc6d895a0309ab001f
Merged-In: I54ac97f62898e47a338b51cc6d895a0309ab001f
(cherry picked from commit 491b0a8913)
2019-08-26 15:10:18 -07:00
Xusong Wang
8161196bd5 Modify 1.1 VTS tests to consume test struct directly.
This CL is very similar to the 1.0 VTS CL. The only difference is that
1.1 Models have an additional field for relaxed model computation.

Bug: 123092187
Bug: 138718240
Test: All VTS
Change-Id: I9264e5b2468b9c6db47d86683d24f8c2c5ec46aa
Merged-In: I9264e5b2468b9c6db47d86683d24f8c2c5ec46aa
(cherry picked from commit 6aad040278)
2019-08-26 15:07:50 -07:00
Xusong Wang
7df4306f8c Modify 1.0 VTS tests to consume test struct directly.
Implement converter utilities constructing HIDL model and request from
TestModel.

Bug: 123092187
Bug: 138718240
Test: All VTS
Change-Id: I0b26b7f41d31d5e63ed083ab5f6f269a3620f034
Merged-In: I0b26b7f41d31d5e63ed083ab5f6f269a3620f034
(cherry picked from commit 8e8b70c439)
2019-08-26 15:07:35 -07:00
Xusong Wang
cbfa6b998e Statically check test enums against HIDL definitions.
Fixes: 123092187
Test: mma all VTS
Test: Attempt to modify test enum value,
      confirm that there is build error with proper error message.

Change-Id: I0129378c87637b33bb145ec93307634273a9d2fd
2019-08-26 14:47:30 -07:00
Xusong Wang
9e2b97b83c Update VTS tests with the new test harness.
Bug: 120601396
Test: All VTS
Change-Id: I539e75585b2cc01d153565814491361adfa048be
2019-08-26 14:15:15 -07:00
Xusong Wang
8fc4622bf1 Modify validateBurstSanitized to take a single request.
This CL is to modify validateBurstSanitized implementation in accordance
with the new test generator and test harness change in the same topic.

Bug: 123092187
Test: 1.2 VTS
Change-Id: Iacfe0e3bba8b4b44b7aee023d1c8129cbd089d4a
2019-08-21 17:35:10 -07:00
Xusong Wang
491b0a8913 Modify 1.2 VTS tests to consume test struct directly.
Comparing with v1.1, the converter for 1.2 HIDL model has additional support
for extraParam, dynamic output shape, and zero-sized output.

Modify CompilationCachingTests to use the new test struct.

Bug: 123092187
Bug: 138718240
Test: All VTS
Change-Id: I54ac97f62898e47a338b51cc6d895a0309ab001f
2019-08-21 17:35:05 -07:00
Xusong Wang
6aad040278 Modify 1.1 VTS tests to consume test struct directly.
This CL is very similar to the 1.0 VTS CL. The only difference is that
1.1 Models have an additional field for relaxed model computation.

Bug: 123092187
Bug: 138718240
Test: All VTS
Change-Id: I9264e5b2468b9c6db47d86683d24f8c2c5ec46aa
2019-08-20 15:33:44 -07:00
Xusong Wang
8e8b70c439 Modify 1.0 VTS tests to consume test struct directly.
Implement converter utilities constructing HIDL model and request from
TestModel.

Bug: 123092187
Bug: 138718240
Test: All VTS
Change-Id: I0b26b7f41d31d5e63ed083ab5f6f269a3620f034
2019-08-20 15:33:38 -07:00
Slava Shklyaev
1c4b95fb56 Merge "Refactor generated NNAPI tests (follow-up)" am: 651869e0c6 am: adf64d6cbe am: 6c1d24473f am: 0deb8aa21b
am: 20763ef488

Change-Id: I9e96057d994c322157588b818de58bc09089e796
2019-08-15 13:39:20 -07:00
Slava Shklyaev
adf64d6cbe Merge "Refactor generated NNAPI tests (follow-up)"
am: 651869e0c6

Change-Id: Ia3bdfbfc6abbdc970efcd0cf31df2b58676eebe8
2019-08-15 10:15:52 -07:00
Slava Shklyaev
e2f9d55a26 Merge "Refactor generated NNAPI tests"
am: caedc76b18

Change-Id: I5087c042a51954a7276d7d20167f526884c987c0
2019-08-15 09:14:24 -07:00
Slava Shklyaev
0da5c34a3a Refactor generated NNAPI tests (follow-up)
Bug: 136097638
Test: m
Change-Id: I0688a843ab04caca9d62f95d38179804eb1b0f80
2019-08-07 12:29:38 +01:00
Slava Shklyaev
e8b2446b35 Refactor generated NNAPI tests
See change I2c0366fb87c96851fa6e0f8fe9ceac012d8e3513

Bug: 136097638
Test: m VtsHalNeuralnetworksV1_0TargetTest
Test: m VtsHalNeuralnetworksV1_1TargetTest
Test: m VtsHalNeuralnetworksV1_2TargetTest
Test: m VtsHalNeuralnetworksV1_2CompatV1_0TargetTest
Change-Id: I6fdede028422145d313d46532b5d2154ef0d40bc
Merged-In: I6fdede028422145d313d46532b5d2154ef0d40bc
(cherry picked from commit 2bcfdc82a0)
2019-08-07 12:17:59 +01:00
Michael Butler
8f77ce7442 NNAPI VTS fix from NNAPI CL: Cleanup HalInterfaces.h
am: 3835f61986

Change-Id: If5d04badade5bc72470aafd186468701d21ee2ee
2019-07-26 14:18:35 -07:00
Xusong Wang
6e8f9859c2 Move strided_slice_invalid_output_dims to spec directory.
am: ee7fe25e99

Change-Id: If3ac5efaf33acfe3f8aa3808e631f877216d3ead
2019-07-26 14:18:19 -07:00
Michael Butler
3835f61986 NNAPI VTS fix from NNAPI CL: Cleanup HalInterfaces.h
Prior to this CL, HalInterfaces.h polluted the global namespace through
the "using" declarations. To fix b/72880287, those names were put in a
new ::android::nn::hal namespace. This CL fixes the compilation errors
that occur due to this cleanup.

Bug: 72880287
Test: mma
Change-Id: Idb228e37615af7c0343a09f9a51de378e63bdb5a
Merged-In: Idb228e37615af7c0343a09f9a51de378e63bdb5a
(cherry picked from commit 4745a3d96e)
2019-07-26 13:55:25 +01:00
Xusong Wang
ee7fe25e99 Move strided_slice_invalid_output_dims to spec directory.
Bug: 132155416
Test: 1.2 VTS
Change-Id: I0479b8172c55e5e290f3fc8530a428ac5f54998f
Merged-In: I0479b8172c55e5e290f3fc8530a428ac5f54998f
(cherry picked from commit c99ced161c)
2019-07-26 13:55:24 +01:00
Slava Shklyaev
2bcfdc82a0 Refactor generated NNAPI tests
See change I2c0366fb87c96851fa6e0f8fe9ceac012d8e3513

Bug: 136097638
Test: m VtsHalNeuralnetworksV1_0TargetTest
Test: m VtsHalNeuralnetworksV1_1TargetTest
Test: m VtsHalNeuralnetworksV1_2TargetTest
Test: m VtsHalNeuralnetworksV1_2CompatV1_0TargetTest
Change-Id: I6fdede028422145d313d46532b5d2154ef0d40bc
2019-07-23 15:30:49 +01:00
Michael Butler
734a3020ac Merge "NNAPI VTS fix from NNAPI CL: Cleanup HalInterfaces.h" 2019-07-22 21:06:24 +00:00
Xusong Wang
0d0af2cbe3 Merge "Move strided_slice_invalid_output_dims to spec directory." 2019-07-22 19:47:38 +00:00
Michael Butler
6fd5eaec52 Update NN VTS callback objects
am: 051cf39f99

Change-Id: Ibab8657b851d56af18965766303e0f26bc51bded
2019-07-19 13:25:15 -07:00
Xusong Wang
237bfc9854 Remove compiler flag for dynamic output shape tests.
am: e108bc15e6

Change-Id: Icf4f851b0ed919f7041620d1e7a4b40ccfcbae34
2019-07-19 13:25:00 -07:00
Slava Shklyaev
868ef4ac7a Update paths to NNAPI VTS models
am: cac83eefa5

Change-Id: I72034e2205fb7a6052894271974d3bedcae71f6b
2019-07-19 13:24:44 -07:00
Slava Shklyaev
bc50034109 Refactor NNAPI VTS to remove unreasonable dependence between versions
am: 73ee79dafa

Change-Id: Ie79a5693ceb255d0e44579cc6627254840166d8f
2019-07-19 13:10:50 -07:00
Michael Butler
051cf39f99 Update NN VTS callback objects
The VTS Callback files are a subset of the Callback files in
frameworks/ml/nn/runtime/Callbacks.*. This CL syncs the implementations,
removing the functionality that is not needed in VTS.

Fixes: 132322149
Test: mma
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Test: VtsHalNeuralnetworksV1_2TargetTest
Change-Id: I114ce7f3b6c3d58de0196e9508209614d0a73e11
Merged-In: I114ce7f3b6c3d58de0196e9508209614d0a73e11
(cherry picked from commit 23d0e562e0)
2019-07-19 14:00:40 +01:00
Xusong Wang
e108bc15e6 Remove compiler flag for dynamic output shape tests.
Having separate VTS models and tests for difference HAL version. This
compiler flag is no longer needed.

Bug: 122740334
Test: All VTS 1.x and 1.xCompat1.y
Change-Id: Idbd96a954da9bd5a0e0e66afd9120d84a1efc784
Merged-In: Idbd96a954da9bd5a0e0e66afd9120d84a1efc784
(cherry picked from commit 7763d53545)
2019-07-19 14:00:38 +01:00
Slava Shklyaev
cac83eefa5 Update paths to NNAPI VTS models
See change If3bfbdfa016cd32a39b8a63028481da7318d6e02

Bug: 124040454
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Test: VtsHalNeuralnetworksV1_1CompatV1_0TargetTest
Test: VtsHalNeuralnetworksV1_2TargetTest
Test: VtsHalNeuralnetworksV1_2CompatV1_0TargetTest
Test: VtsHalNeuralnetworksV1_2CompatV1_1TargetTest
Change-Id: I17dd4377ee5110ac10ae8142fa7403329eda067c
Merged-In: I17dd4377ee5110ac10ae8142fa7403329eda067c
(cherry picked from commit 8226861489)
2019-07-19 14:00:35 +01:00
Slava Shklyaev
73ee79dafa Refactor NNAPI VTS to remove unreasonable dependence between versions
To make it easier to create the next version of NNAPI, this change
removes the following nonsensical dependence:
- NNAPI 1.0 VTS depends on NNAPI 1.1 and 1.2
- NNAPI 1.1 VTS depends on NNAPI 1.2

In particular, I made the following changes:
- split GeneratedTestHarness.cpp into three separate implementations,
- created a restricted version of Callbacks.h for 1.0 and 1.1,
- removed the dependency on frameworks/ml/nn/HalInterfaces.h,
- refactored Android.bp files for more autonomy between 1.0, 1.1, and 1.2,
- consolidated some common code into Utils.h,
- created structure for sharing code between VTS versions (VtsHalNeuralNetworksV1_0_utils).

Bug: 74827824
Bug: 124462414
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Test: VtsHalNeuralnetworksV1_1CompatV1_0TargetTest
Test: VtsHalNeuralnetworksV1_2TargetTest
Test: VtsHalNeuralnetworksV1_2CompatV1_0TargetTest
Test: VtsHalNeuralnetworksV1_2CompatV1_1TargetTest
Change-Id: I4243d0b5e574255cef1070850f4d0a284f65f54e
Merged-In: I4243d0b5e574255cef1070850f4d0a284f65f54e
(cherry picked from commit 1d6b465997)
2019-07-19 14:00:29 +01:00
Xusong Wang
c99ced161c Move strided_slice_invalid_output_dims to spec directory.
Bug: 132155416
Test: 1.2 VTS
Change-Id: I0479b8172c55e5e290f3fc8530a428ac5f54998f
2019-07-18 13:31:04 -07:00
Michael Butler
4745a3d96e NNAPI VTS fix from NNAPI CL: Cleanup HalInterfaces.h
Prior to this CL, HalInterfaces.h polluted the global namespace through
the "using" declarations. To fix b/72880287, those names were put in a
new ::android::nn::hal namespace. This CL fixes the compilation errors
that occur due to this cleanup.

Bug: 72880287
Test: mma
Change-Id: Idb228e37615af7c0343a09f9a51de378e63bdb5a
2019-07-17 21:25:37 +00:00
Michael Butler
23d0e562e0 Update NN VTS callback objects
The VTS Callback files are a subset of the Callback files in
frameworks/ml/nn/runtime/Callbacks.*. This CL syncs the implementations,
removing the functionality that is not needed in VTS.

Fixes: 132322149
Test: mma
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Test: VtsHalNeuralnetworksV1_2TargetTest
Change-Id: I114ce7f3b6c3d58de0196e9508209614d0a73e11
2019-07-16 17:13:13 -07:00
Xusong Wang
7763d53545 Remove compiler flag for dynamic output shape tests.
Having separate VTS models and tests for difference HAL version. This
compiler flag is no longer needed.

Bug: 122740334
Test: All VTS 1.x and 1.xCompat1.y
Change-Id: Idbd96a954da9bd5a0e0e66afd9120d84a1efc784
2019-07-11 10:01:41 -07:00
Slava Shklyaev
8226861489 Update paths to NNAPI VTS models
See change If3bfbdfa016cd32a39b8a63028481da7318d6e02

Bug: 124040454
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Test: VtsHalNeuralnetworksV1_1CompatV1_0TargetTest
Test: VtsHalNeuralnetworksV1_2TargetTest
Test: VtsHalNeuralnetworksV1_2CompatV1_0TargetTest
Test: VtsHalNeuralnetworksV1_2CompatV1_1TargetTest
Change-Id: I17dd4377ee5110ac10ae8142fa7403329eda067c
2019-07-03 15:58:08 +01:00
Slava Shklyaev
1d6b465997 Refactor NNAPI VTS to remove unreasonable dependence between versions
To make it easier to create the next version of NNAPI, this change
removes the following nonsensical dependence:
- NNAPI 1.0 VTS depends on NNAPI 1.1 and 1.2
- NNAPI 1.1 VTS depends on NNAPI 1.2

In particular, I made the following changes:
- split GeneratedTestHarness.cpp into three separate implementations,
- created a restricted version of Callbacks.h for 1.0 and 1.1,
- removed the dependency on frameworks/ml/nn/HalInterfaces.h,
- refactored Android.bp files for more autonomy between 1.0, 1.1, and 1.2,
- consolidated some common code into Utils.h,
- created structure for sharing code between VTS versions (VtsHalNeuralNetworksV1_0_utils).

Bug: 74827824
Bug: 124462414
Test: VtsHalNeuralnetworksV1_0TargetTest
Test: VtsHalNeuralnetworksV1_1TargetTest
Test: VtsHalNeuralnetworksV1_1CompatV1_0TargetTest
Test: VtsHalNeuralnetworksV1_2TargetTest
Test: VtsHalNeuralnetworksV1_2CompatV1_0TargetTest
Test: VtsHalNeuralnetworksV1_2CompatV1_1TargetTest
Change-Id: I4243d0b5e574255cef1070850f4d0a284f65f54e
2019-07-03 15:58:08 +01:00
David Gross
491d626c89 Merge "Improve documentation related to execution." am: b575cedd74 am: 3059c66d53 am: 9c59f1d42b
am: 653714d3bb

Change-Id: Ib0a93f03cfa7bf1138fac44930d6eb95a8deb269
2019-06-21 12:28:33 -07:00
David Gross
9c59f1d42b Merge "Improve documentation related to execution." am: b575cedd74
am: 3059c66d53

Change-Id: Ie1b4244f9267a96df67f8f64c42b91f3b455f416
2019-06-21 12:11:13 -07:00
David Gross
b575cedd74 Merge "Improve documentation related to execution." 2019-06-21 18:40:03 +00:00
David Gross
f406bf9b1e Improve documentation related to execution.
Bug: 121347610 document that NNAPI Execution inputs/outputs and HAL Request inputs/outputs must not be modified

Test: cd hardware/interfaces/neuralnetworks/1.0/vts/functional ; mma
Test: cd hardware/interfaces/neuralnetworks/1.2/vts/functional ; mma

Change-Id: Iac71d6d5ad92a90afd1b6babb7cfa128d7484c64
2019-06-21 11:28:19 -07:00
Slava Shklyaev
a29dd23938 Ensure that drivers return an error if STRIDED_SLICE parameters are invalid am: 95a5978983
am: 87dff647e1

Change-Id: I92774e7aeff1376e354d6624d60e1add21810d33
2019-06-20 13:59:21 -07:00
Slava Shklyaev
95a5978983 Ensure that drivers return an error if STRIDED_SLICE parameters are invalid
We don't have a framework for handcrafting tests resulting in execution
failure. To produce this test, I created
strided_slice_invalid_output_dims.mod.py and used our VTS test generator
to create the corresponding example.cpp and model.cpp files. I put those
under the new directory neuralnetworks/1.2/vts/functional/generated to
distinguish them from the streamlined generated tests.

Bug: 79856511
Bug: 132155416
Test: VtsHalNeuralnetworksV1_2TargetTest
Change-Id: Ib331778605c2e3828195e1e7d1651091873985a0
Merged-In: Ib331778605c2e3828195e1e7d1651091873985a0
(cherry picked from commit 1a18c08052)
2019-06-19 11:16:48 +01:00
Lev Proleev
e61b7b98c9 Fix LSTM documentation
The documentation said that cell-to-input weights are required to be
present when input-to-input weights, recurrent-to-input weights and
input gate bias are present. This was incorrect since this weights can
be omitted if peephole connections are not used even if all the other
tensors are present.

Another bug that is fixed in this change is that for output #0 the docs
said "of shape [batch_size, num_units * 4] with CIFG, or [batch_size,
num_units * 3] without CIFG" when in fact it is the opposite, i.e.  "of
shape [batch_size, num_units * 3] with CIFG, or [batch_size, num_units *
4] without CIFG."

Existing CTS/VTS tests expect behaviour described in the fixed documentation.
Existing CPU implementation is also compliant with the fixed documentation.

Fix: 111842951
Test: mma
Change-Id: Id011783e33672ae65dc6fe3784cb26feb832acf9
Merged-In: Id011783e33672ae65dc6fe3784cb26feb832acf9
(cherry picked from commit e0537f09fb)
2019-05-15 14:57:34 +01:00
Slava Shklyaev
1a18c08052 Ensure that drivers return an error if STRIDED_SLICE parameters are invalid
We don't have a framework for handcrafting tests resulting in execution
failure. To produce this test, I created
strided_slice_invalid_output_dims.mod.py and used our VTS test generator
to create the corresponding example.cpp and model.cpp files. I put those
under the new directory neuralnetworks/1.2/vts/functional/generated to
distinguish them from the streamlined generated tests.

Bug: 79856511
Bug: 132155416
Test: VtsHalNeuralnetworksV1_2TargetTest
Change-Id: Ib331778605c2e3828195e1e7d1651091873985a0
2019-05-15 10:36:55 +00:00
Xusong Wang
8fe6f244f0 Merge "Add quant8 variant of VTS CompilationCachingTest." into qt-dev am: c9d90efcc7
am: 5d8fec169e

Change-Id: I71acf1cabd83d0f9e70736990fc9305c5d133429
2019-05-09 20:23:37 -07:00
Xusong Wang
5d8fec169e Merge "Add quant8 variant of VTS CompilationCachingTest." into qt-dev
am: c9d90efcc7

Change-Id: Iccc7cbb84462e07d8f3462d3ef4eb4f3edf59459
2019-05-09 17:42:51 -07:00
Xusong Wang
0e0721f56e Add quant8 variant of VTS CompilationCachingTest.
Prior to this CL, the VTS compilation caching tests are using float32
models. However, there exists several drivers that only support QUANT8
operations, and most of the tests are simply skipped.

This CL makes the CompilationCachingTest fixture parameterized that
every test will run twice, with the first pass running with float32
models and the second pass running with quant8 models.

Bug: 132112463
Test: 1.2 VTS with sample drivers
Test: 1.2 VTS with test drivers that can read and write cache entries
Change-Id: Iaf28065e46c2d90ab8b347b9848be1e6546cce86
Merged-In: Iaf28065e46c2d90ab8b347b9848be1e6546cce86
(cherry picked from commit 35e7599eb2)
2019-05-09 15:45:35 -07:00
Xusong Wang
4f71afc50d Move getSupportedOperations out of saveToCache.
Before this change, getSupportedOperations_1_2 was invoked in
saveModelToCache prior to prepareModel_1_2, which delayed the model
preparation and cache overwriting always ran faster in
SaveToCache_TOCTOU test.

This change increases the chance that the model is overwritten after it
is saved in SaveToCache_TOCTOU test by moving getSupportedOperations_1_2
out of saveToCache.

Bug: 123433989
Test: VtsHalNeuralnetworksV1_xTargetTest with 1.2 sample driver
Test: VtsHalNeuralnetworksV1_xTargetTest with a test driver that can
      read and write cache entries
Test: TOCTOU tests with a sample driver that is deliberately broken

Change-Id: Ieba6a82252623ac429d5fcb6950b99db949f9886
Merged-In: Ieba6a82252623ac429d5fcb6950b99db949f9886
(cherry picked from commit f8c5987bbd)
2019-05-09 15:45:01 -07:00
Xusong Wang
e371f6f85e Test corrupted data cache in CompilationCachingTests.
We only expect the driver to not crash.

Bug: 123433989
Test: VtsHalNeuralnetworksV1_xTargetTest with 1.2 sample driver
Test: VtsHalNeuralnetworksV1_xTargetTest with a test driver that can
      read and write cache entries

Change-Id: Ic9bd7ad6e42d77d505955cb9dda597a39e95cdb6
Merged-In: Ic9bd7ad6e42d77d505955cb9dda597a39e95cdb6
(cherry picked from commit 83ab17f224)
2019-05-09 15:44:46 -07:00
Xusong Wang
7cc0cccf5d Test TOCTOU in VTS CompilationCachingTests.
Two tests are added into VTS:

1. The TOCTOU test, for a fixed number of iterations.

We try to attack the driver by replacing the model cache with another
cache entry from a similar model while the driver is saving to or
preparing from cache, and see if the driver crashes or falsely prepares
an unexpected model (by checking the execution result). Either of the
following results are acceptable

* Fails -> the driver detects this corruption.
* Succeeds and produces correct execution result -> the corruption
happens before saving to cache or after preparing from cache.

Due to the racy nature, this test is probabilistic and we run it several
times.

2. Similar to the TOCTOU test but replace the content between
compile-to-cache and compile-from-cache (once only).

Additionally, remove tmp cache directory if test succeeds.

Bug: 123433989
Test: VtsHalNeuralnetworksV1_xTargetTest with 1.2 sample driver
Test: VtsHalNeuralnetworksV1_xTargetTest with a test driver that can
      read and write cache entries
Test: Check /data/local/tmp and confirm that
      * dirs are preserved on failure
      * dirs are removed on success

Change-Id: Ie04fa905f465c3775979f0ca74359d185dcacea9
Merged-In: Ie04fa905f465c3775979f0ca74359d185dcacea9
(cherry picked from commit a44e130a92)
2019-05-09 15:44:28 -07:00
Xusong Wang
c9d90efcc7 Merge "Add quant8 variant of VTS CompilationCachingTest." into qt-dev 2019-05-09 22:38:54 +00:00
Michael Butler
353a624bd4 Follow up CL to "Add validation tests for NNAPI Burst serialized format"
Bug: 129779280
Bug: 129157135
Test: mma
Test: atest NeuralNetworksTest_static
Test: atest VtsHalNeuralnetworksV1_0TargetTest (with sample-all)
Test: atest VtsHalNeuralnetworksV1_1TargetTest (with sample-all)
Test: atest VtsHalNeuralnetworksV1_2TargetTest (with sample-all)
Change-Id: I6bd088f01c051a76561c0ecf9bbf878bb21c5754
Merged-In: I6bd088f01c051a76561c0ecf9bbf878bb21c5754
(cherry picked from commit 0a1ad962bb)
2019-05-08 22:46:10 +00:00
Michael Butler
d6e38fd71b Add validation tests for NNAPI Burst serialized format
This CL adds the following two types of validation tests on the NNAPI
Burst serialized format:
(1) it directly modifies the serialized data (invalidating it) to ensure
    that vendor  driver services properly validates the serialized
    request
(2) it ensures that vendor driver services properly fail when the result
    channel is not large enough to return the data

This CL additionally includes miscellaneous cleanups:
(1) having a generic "validateEverything" function
(2) moving the "prepareModel" function that's common across
    validateRequest and validateBurst to a common area

Fixes: 129779280
Bug: 129157135
Test: mma
Test: VtsHalNeuralnetworksV1_2TargetTest (with sample-all)
Change-Id: Ib90fe7f662824de17db5a254a8c501855e45f6bd
Merged-In: Ib90fe7f662824de17db5a254a8c501855e45f6bd
(cherry picked from commit 20f28a24e9)
2019-05-08 22:40:22 +00:00
Xusong Wang
35e7599eb2 Add quant8 variant of VTS CompilationCachingTest.
Prior to this CL, the VTS compilation caching tests are using float32
models. However, there exists several drivers that only support QUANT8
operations, and most of the tests are simply skipped.

This CL makes the CompilationCachingTest fixture parameterized that
every test will run twice, with the first pass running with float32
models and the second pass running with quant8 models.

Bug: 132112463
Test: 1.2 VTS with sample drivers
Test: 1.2 VTS with test drivers that can read and write cache entries
Change-Id: Iaf28065e46c2d90ab8b347b9848be1e6546cce86
2019-05-08 15:22:34 -07:00
Michael Butler
7b1c33dd07 Merge "Follow up CL to "Add validation tests for NNAPI Burst serialized format"" into qt-dev am: e84f442c4c
am: 720f869dd9

Change-Id: I2aa81b9466c42b995d0aab27184a42d8939980c5
2019-05-08 15:05:04 -07:00
Michael Butler
720f869dd9 Merge "Follow up CL to "Add validation tests for NNAPI Burst serialized format"" into qt-dev
am: e84f442c4c

Change-Id: I4f40772d30e3b0fa5aeb41edbf9a7d8a26d2ef51
2019-05-08 14:35:43 -07:00
Lev Proleev
41cb1c3265 Merge "Fix LSTM documentation" into qt-dev am: 881e261f0e
am: 9201bd34ca

Change-Id: Ie838e84c562e584fb5588ebc96cb259b439c2c54
2019-05-08 14:23:19 -07:00
Michael Butler
e84f442c4c Merge "Follow up CL to "Add validation tests for NNAPI Burst serialized format"" into qt-dev 2019-05-08 21:04:12 +00:00
Lev Proleev
9201bd34ca Merge "Fix LSTM documentation" into qt-dev
am: 881e261f0e

Change-Id: Iaae38e885aa2f60d9d39030b2db06e7039cc0872
2019-05-08 13:49:14 -07:00
Lev Proleev
e0537f09fb Fix LSTM documentation
The documentation said that cell-to-input weights are required to be
present when input-to-input weights, recurrent-to-input weights and
input gate bias are present. This was incorrect since this weights can
be omitted if peephole connections are not used even if all the other
tensors are present.

Another bug that is fixed in this change is that for output #0 the docs
said "of shape [batch_size, num_units * 4] with CIFG, or [batch_size,
num_units * 3] without CIFG" when in fact it is the opposite, i.e.  "of
shape [batch_size, num_units * 3] with CIFG, or [batch_size, num_units *
4] without CIFG."

Existing CTS/VTS tests expect behaviour described in the fixed documentation.
Existing CPU implementation is also compliant with the fixed documentation.

Fix: 111842951
Test: mma
Change-Id: Id011783e33672ae65dc6fe3784cb26feb832acf9
2019-05-08 17:28:53 +01:00
Slava Shklyaev
cc0b0d4509 Document that quantized PAD behavior is undefined before NNAPI 1.2
We had no tests for quantized PAD in NNAPI 1.1 and think that vendors might have implemented different behaviors.

Bug: 122243484
Test: N/A
Change-Id: Ibfc0801ab746fc271dc5f8efc764b818c6d49df4
Merged-In: Ibfc0801ab746fc271dc5f8efc764b818c6d49df4
(cherry picked from commit b01ce9644e)
2019-05-08 15:34:41 +01:00
Miao Wang
9bfecb5591 Fix the order of width and height for NNAPI RESIZE_BILINEAR and
RESIZE_NEAREST_NEIGHBOR

  - The CPU implementation always had the order of {width, height}.
  - In P, the documentation was incorrectly changed to {height, width}.

Bug: 131623949
Bug: 130035110
Test: mm
Change-Id: I6c79459fa73347fb51fc34a76ad78d5ac207f210
Merged-In: I6c79459fa73347fb51fc34a76ad78d5ac207f210
(cherry picked from commit 286339b4c8)
2019-05-08 15:33:55 +01:00
Michael Butler
0a1ad962bb Follow up CL to "Add validation tests for NNAPI Burst serialized format"
Bug: 129779280
Bug: 129157135
Test: mma
Test: atest NeuralNetworksTest_static
Test: atest VtsHalNeuralnetworksV1_0TargetTest (with sample-all)
Test: atest VtsHalNeuralnetworksV1_1TargetTest (with sample-all)
Test: atest VtsHalNeuralnetworksV1_2TargetTest (with sample-all)
Change-Id: I6bd088f01c051a76561c0ecf9bbf878bb21c5754
2019-05-07 15:45:48 -07:00
Michael Butler
7dc791d1cd Merge "NNAPI: validate that FmqResultDatum padding is 0 -- VTS" into qt-dev am: 2cd02a4121
am: 557b9a9581

Change-Id: Ice778d3f6d5b31bc2d3e7f570b04196973fd2d63
2019-05-07 13:21:10 -07:00
Michael Butler
557b9a9581 Merge "NNAPI: validate that FmqResultDatum padding is 0 -- VTS" into qt-dev
am: 2cd02a4121

Change-Id: I6bbb1c4247ca7279edb35db1565afdb56b75f114
2019-05-07 12:54:12 -07:00
Michael Butler
ddb770f0e4 NNAPI: validate that FmqResultDatum padding is 0 -- VTS
FmqResultDatum::OperandInformation has padding that may not be
initialized by an NN HAL service instance. This CL adds a validation
check to ensure that services are not leaking uninitialized data through
this padding region.

Bug: 131356202
Test: mma
Test: atest VtsHalNeuralnetworksV1_2TargetTest
      (for ValidationTest with sample-all)
Test: altered sample-driver to randomly set a padding byte to 1;
      the new validation test successfully failed the test

Change-Id: I6661945392b3fc773493d8f2f306f29b39e09bab
2019-05-07 18:42:47 +00:00
Slava Shklyaev
796842078d Merge "Document that quantized PAD behavior is undefined before NNAPI 1.2" into qt-dev am: 7af01ada85
am: bb840a986a

Change-Id: Id09e8dc3a6ff6f9496274702be954cfd6209cf6b
2019-05-02 03:36:08 -07:00
Slava Shklyaev
bb840a986a Merge "Document that quantized PAD behavior is undefined before NNAPI 1.2" into qt-dev
am: 7af01ada85

Change-Id: Ie720d5e1010d5dfc40f7b05c212fa1e56a028a96
2019-05-02 03:31:09 -07:00
Michael Butler
b3b579659c Merge "Add validation tests for NNAPI Burst serialized format" into qt-dev am: ef85db2b65
am: 852820c4ed

Change-Id: I82295fb8c25f0a392ea8de449023f4a12368d954
2019-04-30 17:54:44 -07:00
Michael Butler
852820c4ed Merge "Add validation tests for NNAPI Burst serialized format" into qt-dev
am: ef85db2b65

Change-Id: I8c8cdab2b2ea67322601f7d35146430b555e821a
2019-04-30 14:05:38 -07:00
Slava Shklyaev
b01ce9644e Document that quantized PAD behavior is undefined before NNAPI 1.2
We had no tests for quantized PAD in NNAPI 1.1 and think that vendors might have implemented different behaviors.

Bug: 122243484
Test: N/A
Change-Id: Ibfc0801ab746fc271dc5f8efc764b818c6d49df4
2019-04-30 15:27:53 +01:00
Miao Wang
66c1e14e25 Merge "Fix the order of width and height for NNAPI RESIZE_BILINEAR and RESIZE_NEAREST_NEIGHBOR" into qt-dev am: e867be8b40
am: 0221549cfb

Change-Id: I795b47ee8e8a6640dcbfd69ea6a9606c003dffd4
2019-04-29 23:43:50 -07:00
Xusong Wang
35afac9757 Merge "Move getSupportedOperations out of saveToCache." into qt-dev am: 3ca195bf02
am: d49c50c270

Change-Id: I995d3ac2a416fb38483cd931bfd0489f2fa745d8
2019-04-29 23:15:33 -07:00
Miao Wang
0221549cfb Merge "Fix the order of width and height for NNAPI RESIZE_BILINEAR and RESIZE_NEAREST_NEIGHBOR" into qt-dev
am: e867be8b40

Change-Id: Ic98819efbddfba1be56fd4202806470aabdc29cf
2019-04-29 23:07:35 -07:00
Xusong Wang
d49c50c270 Merge "Move getSupportedOperations out of saveToCache." into qt-dev
am: 3ca195bf02

Change-Id: I391d730ee0c7d8965fe4066f303e8348d35013f8
2019-04-29 22:37:05 -07:00
Michael Butler
ef85db2b65 Merge "Add validation tests for NNAPI Burst serialized format" into qt-dev 2019-04-30 03:58:12 +00:00
TreeHugger Robot
e867be8b40 Merge "Fix the order of width and height for NNAPI RESIZE_BILINEAR and RESIZE_NEAREST_NEIGHBOR" into qt-dev 2019-04-30 03:23:31 +00:00
Miao Wang
286339b4c8 Fix the order of width and height for NNAPI RESIZE_BILINEAR and
RESIZE_NEAREST_NEIGHBOR

  - The CPU implementation always had the order of {width, height}.
  - In P, the documentation was incorrectly changed to {height, width}.

Bug: 131623949
Bug: 130035110
Test: mm
Change-Id: I6c79459fa73347fb51fc34a76ad78d5ac207f210
2019-04-30 00:06:17 +00:00
Michael Butler
20f28a24e9 Add validation tests for NNAPI Burst serialized format
This CL adds the following two types of validation tests on the NNAPI
Burst serialized format:
(1) it directly modifies the serialized data (invalidating it) to ensure
    that vendor  driver services properly validates the serialized
    request
(2) it ensures that vendor driver services properly fail when the result
    channel is not large enough to return the data

This CL additionally includes miscellaneous cleanups:
(1) having a generic "validateEverything" function
(2) moving the "prepareModel" function that's common across
    validateRequest and validateBurst to a common area

Fixes: 129779280
Bug: 129157135
Test: mma
Test: VtsHalNeuralnetworksV1_2TargetTest (with sample-all)
Change-Id: Ib90fe7f662824de17db5a254a8c501855e45f6bd
2019-04-29 23:56:48 +00:00
Xusong Wang
f8c5987bbd Move getSupportedOperations out of saveToCache.
Before this change, getSupportedOperations_1_2 was invoked in
saveModelToCache prior to prepareModel_1_2, which delayed the model
preparation and cache overwriting always ran faster in
SaveToCache_TOCTOU test.

This change increases the chance that the model is overwritten after it
is saved in SaveToCache_TOCTOU test by moving getSupportedOperations_1_2
out of saveToCache.

Bug: 123433989
Test: VtsHalNeuralnetworksV1_xTargetTest with 1.2 sample driver
Test: VtsHalNeuralnetworksV1_xTargetTest with a test driver that can
      read and write cache entries
Test: TOCTOU tests with a sample driver that is deliberately broken

Change-Id: Ieba6a82252623ac429d5fcb6950b99db949f9886
2019-04-29 11:47:54 -07:00