Commit graph

8711 commits

Author SHA1 Message Date
Vinh Tran
73b4f2d4e2 Merge changes from topic "rust_aconfig-test-mode" into main
* changes:
  Add rust_test with rust_aconfig_library in test mode
  Fix rust aconfig test
2023-08-17 14:48:37 +00:00
Dennis Shen
f3ef7363a2 Merge "Update codegen to use single channel in sys prop" into main 2023-08-17 14:29:06 +00:00
Dennis Shen
b352b9d33f Update codegen to use single channel in sys prop
Update sys prop name from persist.device_config.<namespace>.<flag> to persist.aconfig_flags.aconfig_flags.<namespace>.<flag>

Bug: b/295379636
Change-Id: I78f00daa574dc016902dce4b9b4ec961f973693c
2023-08-17 13:48:42 +00:00
Treehugger Robot
ef9ba4d22b Merge "Check device's launch API level before using new compresison algo" into main 2023-08-17 00:35:36 +00:00
Kelvin Zhang
8f83000732 Check device's launch API level before using new compresison algo
If a T launch device later changes to LZ4(which is supported since U),
we will fallback to older compression algorithms, because if the full
OTA is applied on an T build, the update_engine on device won't support
lz4.

Bug: 295989519
Test: generate full OTA for T launched devices, make sure lz4 is not
used

Change-Id: I82fdf788e47e2a6daeaa4479bfecf317d8ebb5d5
2023-08-16 13:16:48 -07:00
Zhi Dou
5f81b7dc96 Merge "aconfig: add default-permission argument for create-cache command" into main 2023-08-14 14:12:27 +00:00
Vinh Tran
906405f5c9 Add rust_test with rust_aconfig_library in test mode
Using test mode in aconfig  allows us to override the flag for unit
testing

Test: atest aconfig.test_mode.test.rust
Change-Id: I890fb20fd8cf83e6033defc7093430e3a77e4de4
2023-08-11 21:41:13 +00:00
Vinh Tran
36fcf63c4e Fix rust aconfig test
Test: atest aconfig.test.rust --start-avd
Change-Id: Ife50e8f81999042b8af91828f5682e84fc7b3811
2023-08-11 17:36:15 -04:00
Zhi Dou
24a0b6a20c aconfig: add default-permission argument for create-cache command
Aconfig set flag default permission as READ_WRITE regardless of the
release configuration. This change enables the caller of create-cache to
set the default permission for all flags, if the flag doesn't have a
value.

Test: atest aconfig.test
Bug: 294417368
Change-Id: I1ba19e1ba793cd2ae59923b136b9b50a92315ece
2023-08-10 21:39:59 +00:00
David Iserovich
babdafe97e [config] Dist supports custom super image script
Some boards need to supply a custom super image script, and this
is supported via the BOARD_CUSTOM_BUILD_SUPER_IMAGE variable.

However, this variable is not used by img_from_target_files.py.

Change img_from_target_files.py to call the supplied script.

Bug: None
Change-Id: I6bca937534f58459d3419139ee7a37a10138ab8f
2023-08-10 18:12:44 +00:00
Dennis Shen
908aba1147 Merge "Add rust_test for aconfig" into main 2023-08-10 17:10:29 +00:00
Daniel Zheng
31694e3f83 Merge "Adding option to configure compression level" into main 2023-08-09 20:03:44 +00:00
Zhi Dou
a7200115c5 Add setFlag and resetAll in FeatureFlags test mode
Add methods setFlag and resetAll in FeatureFlags in test mode. For the
injection usecase, user will use the interface FeatureFlags in the code
to control the flags.

Add tests for test mode.

Bug: 280833463
Test: Atest AconfigJavaHostTest --host
Change-Id: Ib59ba35a9011a6400af42fc9c283d37193577997
2023-08-09 01:14:57 +00:00
Vinh Tran
4559309aaf Add rust_test for aconfig
Test: atest aconfig.test.rust
Change-Id: Ic1b201d05fb216c8d881667461e7750949752b4a
2023-08-08 21:25:22 +00:00
Zhi Dou
5aaeee3749 Generate FakeFeatureFlagsImpl in test mode
Before FeatureFlagsImpl will be used as a fake for testing. This change
adds new class FakeFeatureFlagsImpl. The FeatureFlagsImpl will keep the
same as production. FakeFeatureFlagsImpl can be used as fake for
testing.

FakeFeatureFlagsImpl, and FeatureFlagsImpl will be generated in both
test and prod mode. In test mode FeatureFlagsImpl will just be a stub,
and in prod mode FakeFeatureFlagsImpl will just be a stub.

Bug: 280833463
Test: atest aconfig.test
Change-Id: I11c1e716a9ea00d55600e5e9d5fb6442420762e6
2023-08-08 20:35:18 +00:00
Daniel Zheng
74955f8883 Adding option to configure compression level
Adding option to ota_from_target_files to configure compression level.
This option can be configured via the --vabc_compression_param flag.
e.g. --vabc_compression_param=gz,9 specifies gz compression algorithm
using level 9 compression

Test: ota_from_target_files
Change-Id: Ifc851faccbb3fba466d45c9695aaab322a362081
2023-08-08 08:09:15 -07:00
Dennis Song
a380533f13 Remove system_dlkm from _FRAMEWORK_PARTITIONS
system_dlkm partition is related to GKI, so it should be
in the same category as boot image.

Test: merge_target_files \
        --path otatools \
        --framework-target-files framework-target_files.zip \
	--vendor-target-files vendor-target_files.zip \
	--output-target-files merged-target_files.zip
Bug: 294482075
Change-Id: Ie8f60d72f44d31e1bc3b556735b92ed0a2fda662
2023-08-04 07:08:17 +00:00
Daniel Zheng
3bff72fbc6 Adding supported compression methods
Adding in supported compression methods to help

Test: m ota_from_target_files
Change-Id: I1c3101f983eb8b38d43752530bd9f61a85fddba4
2023-08-03 15:32:29 -07:00
Jason Wu
8e1baff3d3 Merge "Revise minSdkVersion to fix failed mainline_modules_bundles-userdebug" into main 2023-08-02 21:15:25 +00:00
James Wu
c5e321a06e Revise minSdkVersion to fix failed mainline_modules_bundles-userdebug
Test: abtd tests
Bug: 293456031
Change-Id: I674c50690bd04b7e4867c4aceb31d5bdcb53be84
2023-08-02 19:25:58 +00:00
Wei Li
fb55377c4b Merge "Fix the calculation of package verification code which should not include algorithm information." into main 2023-08-02 19:10:52 +00:00
Treehugger Robot
092719a471 Merge "Remove OPTIONS.host_tools" into main 2023-08-02 01:52:27 +00:00
Cole Faust
6833d7d47a Remove OPTIONS.host_tools
It appears to be unused.

Test: Presubmits
Change-Id: Idd7f9dda996458b93da77c783a546cdb3d1e0047
2023-08-01 18:00:37 -07:00
Treehugger Robot
d5474ac3da Merge "Add the option for custom_image to be AVB or NONAVB" into main 2023-08-01 19:08:06 +00:00
Kelvin Zhang
5b9ca88501 Support generating secondary OTAs from directories
Test: ota_from_target_files --include_secondary
Change-Id: I1004defdb4fe601a6ec733a7ceb213de7d540174
2023-08-01 10:04:08 -07:00
Kelvin Zhang
456c6d9a67 Adding an option to disable puffdiff
This is handy for quick OTA generation during development.

Change-Id: I878f7cb3479d6338c04aca44d16e48e83270252c
2023-08-01 10:03:54 -07:00
Wei Li
f99db9977c Fix the calculation of package verification code which should not include algorithm information.
Bug: 293304694
Test: atest --host sbom_data_test
Test: build/soong/tests/sbom_test.sh
Change-Id: I94ea42284a9a6b5cc787a3489bfa575aa7663282
2023-07-31 15:09:16 -07:00
Treehugger Robot
9997fdbacf Merge "aconfig: require exactly one bug field" into main 2023-07-28 07:14:42 +00:00
Treehugger Robot
4d37a829dd Merge "aconfig: improve error messages" into main 2023-07-27 18:09:37 +00:00
Treehugger Robot
0c8b7efdcc Merge "Fix test failures in releasetools" into main 2023-07-27 18:07:47 +00:00
Mårten Kongstad
cd414d4c2e aconfig: improve error messages
Propagate anyhow errors to main.rs and commands.rs to improve the error
message. As an example. instead of just "bad flag declaration: exactly
one bug required", aconfig will now print the following.

---- 8< ----
Error: failed to create cache

Caused by:
    0: failed to parse build/make/tools/aconfig/tests/test.aconfig
    1: bad flag declaration: missing description
---- >8 ----

Error messages can be improved further by including additional
information in the protos.rs error cases. This will be handled in a
follow-up CL.

Bug: 290300657
Test: manual: introduce error in an aconfig file and run `m all_aconfig_declarations`
Test: atest aconfig.test
Change-Id: Id278f4877e5794b95913ae8ba0ca3ee211293f38
2023-07-27 15:54:45 +02:00
Mårten Kongstad
6353c6c635 aconfig: require exactly one bug field
Some consumers of `aconfig dump` do not support multiple bugs. At the
same time, we want all flags to be associated with at least one bug.

Teach aconfig to require that the bug field in the flag_declaration and
parsed_flag proto messages appear exactly once.

This change could have been implemented as a change of `repeated` to
`optional` in the proto definition. However, the chosen approach, with a
runtime check, is easier to revert if we want to support multiple bugs
in the future.

Bug: 293156797
Test: m all_aconfig_declarations && printflags
Test: atest aconfig.test aconfig.test.java aconfig.test.cpp
Change-Id: Ib87dac68b392986a8daa64e56cd85477c92fbe83
2023-07-27 09:44:00 +02:00
Ray-cy.lee
e97e0cb16b Add the option for custom_image to be AVB or NONAVB
Check BOARD_AVB_$(call to-upper,$(partition))_KEY_PATH to decide whether custom_image should sign AVB or not. If key path isn't set, the custom image will be excluded from AVB and copied to /IMAGES in target-files directly. This allows vendor to use custom_images flow packing unsigned image.
And to every non-avb custom partition, one image whose name is partition name must be added in its BOARD_<CUSTOM_PARTITION>_IMAGE_LIST.

BOARD_CUSTOMIMAGES_PARTITION_LIST := tvconfig
BOARD_TVCONFIG_IMAGE_LIST := \
    device/xxxx/yyyy/tvconfig.img \
    device/xxxx/yyyy/tvconfig_custom1.img

Test:
1) Build image, target-files, OTA package by m and m dist
2) Sign images by sign_target_files_apk.py

Fix: 285227850
Change-Id: I7477dafe023e4b168f0f08fb7aedd9e511a60e1b
2023-07-26 20:21:29 +08:00
Kelvin Zhang
43df0805c2 Fix test failures in releasetools
new_path used before assignment, use `path` instead.

Bug: 280425458
Change-Id: I55086f92cf109af67032614cefde60e6478bb149
2023-07-24 13:16:03 -07:00
Mårten Kongstad
af3da9d1be aconfig: restructure integration tests
Consolidate integration tests under tests/. Rename build targets to
follow the same pattern.

The important aconfig build targets are:

  - aconfig           : the aconfig binary
  - aconfig.test      : the aconfig unit tests
  - aconfig.test.java : the Java integration tests
  - aconfig.test.cpp  : the C++ integration tests
  - aconfig.test.rust : the Rust integration tests (not implemented yet)

Note: the Rust integration tests are blocked until the build system has
added support for aconfig Rust libraries.

Note: the flags used in the integration are not set correctly. A
follow-up CL will fix this.

Bug: 283911467
Test: atest aconfig.test.java aconfig.test.cpp
Change-Id: Idb8dad4c38a49231c2e89228775bacf065ee1f57
2023-07-24 13:05:43 +02:00
Joe Onorato
ac692c5eae Make aconfig c++ generate only one header and only one .cc file
Bug: 283479529
Test: m aconfig_hello_world_cc && adb push $TOP/out/target/product/panther/system/bin/aconfig_hello_world_cc /system/bin && adb shell aconfig_hello_world_cc
Change-Id: I1890aff70262343e18a62fb2efb2fd0e13a48fb2
2023-07-21 09:12:01 -07:00
Treehugger Robot
08cb169cbe Merge changes from topic "aconfig-improve-dump-formats" into main
* changes:
  aconfig: add 'verbose' dump format
  aconfig: improve the 'text' dump format
  aconfig: remove the 'debug' dump format
2023-07-20 12:35:57 +00:00
Mårten Kongstad
2d5309cab1 aconfig: fix broken auto-generated Rust code
Fix syntax error in auto-generated Rust code that prevents the code from
compiling.

For reference, the error was:

---- 8< ----
error: expected `where`, `{`, `(`, or `;` after struct name, found keyword `impl`
 --> .../src/lib.rs:6:1
  |
6 | impl FlagProvider {
  | ^^^^ expected `where`, `{`, `(`, or `;` after struct name
---- >8 ----

Bug: 292052163
Test: manual: create sample based on auto-generated Rust lib
Change-Id: I4c029c0ee16ea3849e7bbe381717a10402c9cb81
2023-07-20 11:33:19 +02:00
Mårten Kongstad
ea4981407b aconfig: add 'verbose' dump format
Add a new --format=verbose dump output format. This is identical to the
default 'text' format but includes the source files from which the flag
declaration and values were read, and is intended to help debug why a
flag isn't set to some expected value.

Bug: 283910447
Test: atest aconfig.test
Test: printflags --format=verbose # manually inspect output
Change-Id: I03a1c8d940e7a0c6f91e986c5bafa4aa6cd1125a
2023-07-20 11:07:35 +02:00
Mårten Kongstad
3fa2f078ff aconfig: improve the 'text' dump format
Standardize the flag values as '<permission> + <state>', e.g.
'READ_ONLY + ENABLED'.

Bug: 283910447
Test: atest aconfig.test
Test: printflags # manually inspect output
Change-Id: I60f74196816327613bd8165a688f8b36da5bdac7
2023-07-20 09:35:05 +02:00
Mårten Kongstad
9338007cff aconfig: remove the 'debug' dump format
The --format=debug dump output format is largely superseded by the
textproto format now that aconfig doesn't use hand-written wrappers
around the auto-generated proto struct anymore. Remove the format to
reduce the risk of code rot.

Bug: 283910447
Test: atest aconfig.test
Change-Id: I6700fc4eafd3fa3a63952109c0105d34c7ffd98b
2023-07-20 09:25:56 +02:00
Zhi Dou
92cf0ec232 Aconfig: sort the parsed flag order in create-cache
This change will sorted the parsed flags from the declarations. Without
this change the code will expect the passed in declarations sorted.
After this change the code will still guarantee the parsed flags in the
cache is sorted, but it won't expect the passed in declarations are
sorted

Test: atest aconfig.test
Bug: 291926035
Change-Id: I5f0637fe770003224b128591890e04277bc09345
2023-07-20 02:32:00 +00:00
Dennis Shen
3cfbcf5384 aconfig: Rust codegen 2nd iteration
Introduce rust codegen 2nd iteration with unit test support. The design
is described in detail in the design doc (go/aconfig_rust_codegen). The general idea is to generate
different code with the same signature for production and test build
targets, just like java/c/cpp codegen.

We will have a FlagProvider struct that has flag methods implementation.
This flag provider instance can then be used in injection pattern. In
additon, we also generate top level functions that wraps around flag
provider call so it can be used in static function style.

Things to be decided later:

should we just generate one set of code, and use cfg! as compile
time marco to compile the right code for the right targets.

Bug: b/279483360
Test: atest aconfig.test
Change-Id: Ic75cedbd0d27b5242014c3ac7fc80692d2ab4589
2023-07-18 20:14:46 +00:00
Dennis Shen
01332169f3 Merge "Minor c/c++ codegen update" into main 2023-07-17 12:58:23 +00:00
Justin Yun
22ce9479be Disable copying images for target_files.zip
For the OTA build, target_files.zip requires setting fixed timestamp
for images. However, it caused regression on 'adb sync'.
Restoring image build process for target_files.zip instead of copying
them from the PRODUCT_OUT, only the images in target_files.zip set the
fixed timestamp.
By setting COPY_IMAGES_FOR_TARGET_FILES_ZIP flag, we still may copy
the images for target_files.zip for future use.

Bug: 287534409
Test: m; flashall; adb sync
Test: m dist; flash images from target_files.zip and check the timestamp
Change-Id: I4135eb721c33581f191621847e6595833263d707
2023-07-17 07:46:48 +09:00
Dennis Shen
5c2421394c Minor c/c++ codegen update
1, Moved "#include <string>" from exported header to test flag provider
header file.

2, For production target and read only flags, the generated c
api should just return default value instead of calling into c++ api

3, Remove using namespace server_configurable_flags from header, instead
of just having the namespace to be spelled out in each api call. Having
using namespace xxx in header is not a c++ best practice.

4, Replace #ifdef #def #endif with #pragma once

Bug: b/279483801
Test: atest aconfig.test
Change-Id: I3e55a7b14301f3de419795467f33e2dc889d371e
2023-07-14 16:09:35 +00:00
Dennis Shen
17a08eeca0 aconfig: update c/c++ codegen
Two major changes to c/c++ codegen

(1) explicit setter for each flag instead of a generic flag setter
void override_flag(std::string name, bool val) is replaced with

void <flag name>(bool val) for each flag name

This has several advantages:
  (a) generated code is more c++ idomatic
  (b) no longer need to create flag name string constants
  (c) any typo in the code is caught early in the build time

(2) remove flag setter and flag override reset methods/functions when
generating code targets for production. If developers want to update
their main function to take command line arg for flag overrides, they
can use compile time macros to decide if the flag override code should
be included.

Bug: b/279483801
Test: atest aconfig.test
Change-Id: I6141f7f979b32fe0426154d578edeb997ae5ff7c
2023-07-14 11:40:49 +00:00
Dennis Shen
7321f4f6f1 aconfig: add c codegen
c codegen can be done at the same time with cpp codegen, the idea is to
create a c compatible header that defines the flag apis, including flag
override apis for test. then in the corresponding cpp file, the
implementation simply calls into cpp api.

c header supports static method interface, and unit test override, but
it does not support injection pattern compared to cpp header

Bug: b/279483801
Test: atest aconfig.test
Change-Id: Ie62b76d6524e443de5d3c2f9000f7f66623ab571
2023-07-13 13:55:06 +00:00
Tiffany Yang
b8306086d3 Merge "Make common signature system property definitions optional" into main 2023-07-12 18:35:37 +00:00
Cole Faust
f5350e965b Merge "Remove BOARD_BPT* variables and the bpt parition table image" into main 2023-07-12 01:26:51 +00:00
Cole Faust
4f3dc8759f Remove BOARD_BPT* variables and the bpt parition table image
These variables appear to never be set.

Test: Presubmits
Change-Id: If4b89376f73204ad5780dff421f1216da255c3eb
2023-07-11 16:58:06 -07:00
Tiffany Yang
19450e3a0b Make common signature system property definitions optional
This change allows ARC to define ARC-specific system properties in
Android build without changing these property definitions for other
Android builds. Please see go/arc-sigprop-changes and
go/arc-android-sigprop-sync for additional details.

Bug: 195609932
Test: built bertha_x86_64 with forward declarations
Change-Id: I22bd9d60c2491506fe5c633dbbb9e7516f529b35
2023-07-11 22:40:19 +00:00
Kelvin Zhang
f3f6d0569b Merge "Mark extracted files as executable" into main 2023-07-11 17:39:31 +00:00
Elliott Hughes
c1577ff58b Merge "Switch from minigzip to gzip." into main 2023-07-11 15:52:27 +00:00
Kelvin Zhang
4cb28f6f80 Mark extracted files as executable
Python's zipfile doens't restore file permission by default, so we need
to manually restore permission.

Test: th
Bug: 290643514
Change-Id: I89c1e2ee178b534fa7e3f02afd04d170100d37e7
2023-07-10 13:03:07 -07:00
Dennis Shen
e398144643 aconfig: cpp codegen update: Move server_configurable_flags header include
Move server_configurable_flags header include away from exported header,
instead put it in the flag provider headers. Otherwise, it can cause
compilation error if the source code who wants to use the generated c flag lib has
not added dependency on server_configurable_flags yet.

Bug: b/279483801
Test: atest aconfig.test
Change-Id: Ib75877ee88f995caf72b3fd2554c3da195032c14
2023-07-10 18:19:44 +00:00
Mårten Kongstad
206a382d4e aconfig: improve duplicate flags error message: include paths
Improve the error message returned when `aconfig dump` is fed multiple
declarations of the same flag: include the paths to the declaration
files.

In general all error messages from the protos::*::verify_* functions
should include paths to the offending files. This will be handled in a
follow-up CL.

Bug: 290300657
Test: atest aconfig.test
Test: manual: add duplicate flag and run `m all_aconfig_declarations`, inspect error message
Change-Id: I46dc23f7128dd5c68ced9f2e8518cfa89d81c2df
2023-07-07 08:52:52 +02:00
Wenshan Fu
a2d93ae1eb Merge "aconfig: create java libraries for aconfig proto" 2023-07-06 01:51:08 +00:00
Dennis Shen
8d544f7439 cpp codegen redesign, unit test support
cpp codegen iteration 2, based on discussions with three internal teams
that use c++. Refer to the design doc "aconfig c++ codegen" for detailed
design. At a high level, we generate two sets of code artifacts with the
same signatured api: one for production that without any local flag
override capability, one for unit test that allows local flag overrides.
It supports static methods style interface as well as injection pattern.

Refer to the test points in the codegen_cpp.rs for examples of generated
code.

for production target codegen: aconfig create-cpp-lib --cache <cache> --out <out dir>
for test target codegen: aconfig create-cpp-lib --cache <cache> --out
<out dir> --mode test

Bug: b/279483801
Test: atest aconfig.test
Change-Id: I92fefb9623d5435525339a74f57bbd36d0afef08
2023-07-05 14:05:44 +00:00
Wenshan Fu
89467c3109 aconfig: create java libraries for aconfig proto
Host side test infra needs the library to consume the dumped flags info
generated by aconfig.

Test: m libaconfig_java_proto_lite libaconfig_java_proto_full
Bug: 289906970
Change-Id: I9891cb8445b90f7269e0c679ceaae6bc102bbb0c
2023-07-05 13:57:19 +00:00
Wenshan Fu
e0b709b9a6 Merge "Create the TEST_MAPPING file" 2023-07-04 00:44:05 +00:00
Zhi Dou
7734879e69 Merge changes I4bf1d3ba,I33605054
* changes:
  aconfig: Add resetAll method to test mode FeatureFlagsImpl
  aconfig: remove extra new lines and spaces in generated java code
2023-06-30 13:23:28 +00:00
Treehugger Robot
32efc846f5 Merge "Add --classes optional flag in list_files" 2023-06-30 09:36:42 +00:00
Wenshan Fu
13be0a5f98 Create the TEST_MAPPING file
Add FlagAnnotationTests in presubmit to ensure that test filtering rule
always works as expected.

Test: atest --test-mapping
Bug: 277819423
Change-Id: I7cc3a70495f8d62ed7a5b26edc4fd240e73ccf61
2023-06-30 03:28:02 +00:00
Zhi Dou
8d27cc39b1 aconfig: Add resetAll method to test mode FeatureFlagsImpl
Add resetAll method to test mode FeatureFlagsImpl. This method
is used to reset all the flags values to null. It provides convenient
way to the test tools to reset the flags values.

Bug: 280833463
Test: atest aconfig.test
Change-Id: I4bf1d3ba69ee106ef8d0c1cc62c00bbeca1b72aa
2023-06-29 15:15:32 +00:00
Zhi Dou
a41cc5ecf9 aconfig: remove extra new lines and spaces in generated java code
Change the java template to remove the extra new lines and spaces
in generated java code.

Bug: 280833463
Test: atest aconfig.test
Change-Id: I336050540f8a2b0966bf21cf6e034a8fa701458e
2023-06-29 15:05:18 +00:00
Mårten Kongstad
19776d19e6 aconfig: package fields must contain at least one dot char
Introduce a new requirement on package fields: a package must contain at
least one dot character.

Bug: 289336036
Test: atest aconfig.test
Change-Id: Idadcd2a76783a484cc5c6d6e94778c0248fa475f
2023-06-29 10:38:02 +02:00
Mårten Kongstad
c0d618c3a4 aconfig: fix incorrect check in create_device_config_ident
Use is_valid_name_ident to check the flag name, not
is_valid_package_ident.

Bug: 283910447
Test: atest aconfig.test
Change-Id: I3718e7de565b975a1696190e8effbcb077e5fde2
2023-06-29 10:18:20 +02:00
Mårten Kongstad
60ee8a633f aconfig: improve package identifier test readability
Rearrange the tests in test_is_valid_package_ident to make it more
apparent what constitutes a valid package pattern, and what doesn't.

Bug: 283910447
Test: atest aconfig.test
Change-Id: I3d2b87aed008d0ed3a0aa6e483e655178dda7277
2023-06-29 10:03:38 +02:00
Treehugger Robot
0dc5eeaaca Merge "Make test more deterministic by setting search_path to None" 2023-06-28 22:28:28 +00:00
Jooyung Han
c269ec3538 Merge changes from topic "apexd_host"
* changes:
  Use apexd_host for host-side APEX extraction
  Deprecate flattened apexes
2023-06-28 17:36:15 +00:00
Kelvin Zhang
03dc6ee96f Make test more deterministic by setting search_path to None
Test: th
Bug: 2640751
Change-Id: I9779dfa40c1e8319cb67ae0928c977f5a10cda5f
2023-06-28 10:10:39 -07:00
Zhi Dou
514f99b205 Merge "aconfig: add java codegen test mode" 2023-06-28 13:01:01 +00:00
Treehugger Robot
59f99b31eb Merge "Fix test flake" 2023-06-28 01:18:55 +00:00
Elliott Hughes
97ad120adb Switch from minigzip to gzip.
Bug: http://b/288169261
Test: treehugger
Change-Id: I404cbcb850734ed8291c2215a4329fb372806d63
2023-06-27 15:39:05 -07:00
Zhi Dou
8ba6aa71b1 aconfig: add java codegen test mode
Add java codegen test mode. The test mode will generate Flags.java and
FeatureFlagsImpl.java differently.
    * Flags.java will have getter and setter function to switch the
      FeatureFlagsImpl. Flags.java will not initialize the instance
      of FeatureFlagsImpl during initialization, thus it will force the
      user to set up the flag values for the tests.
    * FeatureFlagsImpl removes the dependency on DeviceConfig, and
      allows the caller to set the values of flags.

Command changes
This change adds a new parameter `mode` to `create-java-lib` subcommand.
The default value of `mode` is production, which will generate files for
production usage, and keeps the same behavior as before.

The new `mode` test is added to trigger the test mode. The command is
aconfig create-java-lib --cache=<path_to_cache> --out=<out_path>
--mode=test

Test: atest aconfig.test
Bug: 288632682
Change-Id: I7566464eb762f3107142fe787f56b17f5be631b7
2023-06-27 18:57:35 +00:00
Kelvin Zhang
97a5afe0f3 Fix test flake
Test: th
Bug: 289013326
Change-Id: Ib233978ff861b010bdc3bd1a02ea9baefe6a3489
2023-06-27 10:30:48 -07:00
Jooyung Han
c9542abbbd Use apexd_host for host-side APEX extraction
Host-side simulation of APEX activation is done by a new tool
'apexd_host'. This simplies checkvintf invocation for local builds and
for target-files.

For local builds, checkvintf no londer depends on $OUT/apex, a flattened
view of APEXes. In fact, the build system doesn't need to install
$OUT/apex. They are installed for now only for its side-effect of
installing symbol files. We'd better not rely on $OUT/apex.

For target-files, scanning/activating apexes are extracted and moved to
the new tool. Now check_target_files_vintf is not more efficient because
it doesn't copy .apex files to a temporary directory.

Bug: 260358957
Bug: 288826922
Test: m (running checkvintf) # for local builds
Test: m target-files-package && check_garget_files_vintf target-files.zip
Test: atest releasetools_test
Change-Id: Iba23f429d96f9ec31814196aa14bdbb800649218
2023-06-27 17:00:06 +09:00
Jihoon Kang
c8e1d69687 Add --classes optional flag in list_files
The flag enables list_files to output a list of .class files pattern
that will be generated when compiling the java files in the input
directory. The flag can be used to extract the appropriate class files
from stub jar file provided via dep_api_srcs in per api scope
sdk_library generated java_api_library modules.

Test: m --build-from-text-stub in same topic
Bug: 275570206
Change-Id: I134fcd9781024dcb6781297d02645f5fb9befb18
2023-06-27 04:46:50 +00:00
Jooyung Han
2ac1f2fabc Deprecate flattened apexes
Flattened apexes are deprecated. mege_dexopt does not need to care about
flattened apexes.

Bug: 278826656
Test: m otapackage
Change-Id: I527eed64c5f1a7466e2c48d73360f47f6cfa0911
2023-06-27 13:01:56 +09:00
Xin Li
24c928cda8 Merge "Merge Android 13 QPR3" 2023-06-26 22:29:55 +00:00
Mårten Kongstad
3228b29cac aconfig: add dump --format=textproto
Teach `aconfig dump` to print its as text protobuf.

Bug: 288713733
Test: atest aconfig.test aconfig.test.java
Change-Id: Id31ce53226162a563fbe2d4841a8f6cf96c22436
2023-06-26 10:17:42 +02:00
Mårten Kongstad
37cb74b3e4 Merge "aconfig: add proto bug field" 2023-06-26 06:27:48 +00:00
Joe Onorato
4a80353011 Merge "Rename device_config --> aconfig and definitions --> declarations" am: 9ccb457588 am: 958b906ed8 am: a83f50d161
Original change: https://android-review.googlesource.com/c/platform/build/+/2633872

Change-Id: If9360ebd0acebe85f42f89a268c316d51c765708
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-22 15:36:36 +00:00
Mårten Kongstad
1b8636bd7d aconfig: add proto bug field
Add a `bug` field on the flag_declaration and parsed_flag proto
messages. This field is optional in the sense that it can occur zero or
more times, and aconfig will simply pass any value through.

Bug fields are included in the aconfig dump format, which can be
processed by other tools.

Also unify how protos.rs checks that fields marked 'optional' in the
proto file, but in practice are 'required', are actually set.

Test: atest aconfig.test aconfig.test.java
Bug: 288261336
Change-Id: I93de0005674822c6ff4d699bdc2c6509763a7f7f
2023-06-22 16:50:33 +02:00
Joe Onorato
958b906ed8 Merge "Rename device_config --> aconfig and definitions --> declarations" am: 9ccb457588
Original change: https://android-review.googlesource.com/c/platform/build/+/2633872

Change-Id: I0cb0c21ac2df9a34caf4e82346007674f49c97d0
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-22 14:10:22 +00:00
Joe Onorato
9ccb457588 Merge "Rename device_config --> aconfig and definitions --> declarations" 2023-06-22 13:30:53 +00:00
Treehugger Robot
767b6bc6e8 Merge "Handle symlinks when extracting zipfiles" am: 0e5e1749bb am: f6399f8b17 am: 444bb1c08e
Original change: https://android-review.googlesource.com/c/platform/build/+/2632808

Change-Id: Iaa75915a875a76df5d654cc60001dce4a52ad6d2
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-22 03:59:23 +00:00
Treehugger Robot
f6399f8b17 Merge "Handle symlinks when extracting zipfiles" am: 0e5e1749bb
Original change: https://android-review.googlesource.com/c/platform/build/+/2632808

Change-Id: I4316f46baf8d79390a892c1391b89ea50487e4e6
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-22 02:29:29 +00:00
Treehugger Robot
0e5e1749bb Merge "Handle symlinks when extracting zipfiles" 2023-06-22 01:27:35 +00:00
Joe Onorato
6aa48f8c5f Rename device_config --> aconfig and definitions --> declarations
Bug: 285303012
Test: for x in next trunk trunk_food trunk_staging ; do lunch aosp_panther-$x-eng ; m nothing ; done
Change-Id: I174ce3e609fa9077ea6bc61679ddb83e8efe57c8
2023-06-21 15:18:15 -07:00
Kelvin Zhang
e473ce9e80 Handle symlinks when extracting zipfiles
python3.11's zipfile implementation does not handle symlinks. This
causes important symlinks in ramdisk to be broken, and later causing a
boo failure.

Test: unzip a target files with symlinks, make sure symlinks are created
Bug: 287896098

Change-Id: Ia7d6ac8ffb03807680a36ff648aa11afafb7f481
2023-06-21 13:31:26 -07:00
Mårten Kongstad
2302e3c4be Merge "aconfig: make proto fields optional" am: a3b0027fdb am: 0201a71f0c am: a34196902b
Original change: https://android-review.googlesource.com/c/platform/build/+/2627237

Change-Id: I6c0d6c0195b41a71c5f8cb9701784e55ae2ef9f9
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-20 21:34:06 +00:00
Mårten Kongstad
0201a71f0c Merge "aconfig: make proto fields optional" am: a3b0027fdb
Original change: https://android-review.googlesource.com/c/platform/build/+/2627237

Change-Id: I697af95c091cf919fe0f094c03d83f99834d4ad0
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-20 20:05:02 +00:00
Zhi Dou
5ace4a9cc3 Merge "aconfig: generate Java flag name constants" am: cff292f8b2 am: 1f7e9252ed am: 6b29eed1a1
Original change: https://android-review.googlesource.com/c/platform/build/+/2611170

Change-Id: I72c9c984f1e5f7380e027c6e6ada23c333a296a8
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-20 18:58:44 +00:00
Mårten Kongstad
a3b0027fdb Merge "aconfig: make proto fields optional" 2023-06-20 18:47:08 +00:00
Zhi Dou
1f7e9252ed Merge "aconfig: generate Java flag name constants" am: cff292f8b2
Original change: https://android-review.googlesource.com/c/platform/build/+/2611170

Change-Id: I21da79301c4dc8828c6b4b48923720121dc91342
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-20 17:32:16 +00:00
Treehugger Robot
0531aac817 Merge "aconfig: use proto struct directly" am: 23788f85a4 am: 7639e644a3 am: 6a5ce4a2a9
Original change: https://android-review.googlesource.com/c/platform/build/+/2628052

Change-Id: I92b8b44af3956702740f508119c491222fff659c
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-20 17:28:39 +00:00
Zhi Dou
cff292f8b2 Merge "aconfig: generate Java flag name constants" 2023-06-20 16:53:53 +00:00
Treehugger Robot
7639e644a3 Merge "aconfig: use proto struct directly" am: 23788f85a4
Original change: https://android-review.googlesource.com/c/platform/build/+/2628052

Change-Id: Ie7a63ce89d02bc3fc4fcee4e3080cc86779a167f
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-20 16:04:25 +00:00
Treehugger Robot
23788f85a4 Merge "aconfig: use proto struct directly" 2023-06-20 15:28:58 +00:00
Wei Li
83cdd1f97d Merge "Include static libraries information in Android SBOM." am: c20175155a am: 7f298d8a1a am: 98a187fb69
Original change: https://android-review.googlesource.com/c/platform/build/+/2609475

Change-Id: I16510d59533dc69cbeeeddaefaba91e691631949
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-20 00:19:01 +00:00
Wei Li
7f298d8a1a Merge "Include static libraries information in Android SBOM." am: c20175155a
Original change: https://android-review.googlesource.com/c/platform/build/+/2609475

Change-Id: I342757931802a04dd09d0f50e3eea2219fdd5948
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-19 22:51:32 +00:00
Wei Li
c20175155a Merge "Include static libraries information in Android SBOM." 2023-06-19 22:02:20 +00:00
Treehugger Robot
26fb87e0fd Merge "Fix usage message." am: fb8c86513e am: 9d9e540cc5 am: 8b1c423839
Original change: https://android-review.googlesource.com/c/platform/build/+/2625356

Change-Id: I807ef4d81821548df70ead6fc3db2bde6ce3d75d
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-19 21:19:17 +00:00
Treehugger Robot
0a12cee3aa Merge "Fix godoc typos." am: d999938ab0 am: 6df5275646 am: b89f8537cc
Original change: https://android-review.googlesource.com/c/platform/build/+/2628611

Change-Id: I656792d1b06e40d378b4cbccf39cc1550cf646d5
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-19 21:19:03 +00:00
Treehugger Robot
9d9e540cc5 Merge "Fix usage message." am: fb8c86513e
Original change: https://android-review.googlesource.com/c/platform/build/+/2625356

Change-Id: I7b1b9db8ded7ae73aa94152c66032aebffaa2186
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-19 19:53:45 +00:00
Treehugger Robot
6df5275646 Merge "Fix godoc typos." am: d999938ab0
Original change: https://android-review.googlesource.com/c/platform/build/+/2628611

Change-Id: I7a4b16d885e777143143d8beeffea38ddd77b6e3
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-19 19:53:30 +00:00
Treehugger Robot
fb8c86513e Merge "Fix usage message." 2023-06-19 19:20:15 +00:00
Treehugger Robot
d999938ab0 Merge "Fix godoc typos." 2023-06-19 19:10:47 +00:00
Mårten Kongstad
a2e5ab82c7 aconfig: make proto fields optional
Change all required proto fields to optional. While the proto file is
supposed to be a backwards compatible API, and fields are not supposed
to be deprecated, this commit will allow for that option if needed.

Implementation wise this change doesn't matter much: any parsed data
needs additional verification outside what the protobuf crate's parser
provides anyway, so adding checks to verify that all required fields,
even though marked optional in the proto file, were found is a minor
increase in code complexity.

If in the future a proto field should no longer be used:

  - keep the field in the proto, still marked optional and clearly
    document that it is no longer in use
  - change protos.rs from checking struct.has_field() to explicitly
    dropping any value via struct.clear_field()

Bug: 286337317
Test: atest aconfig.test
Change-Id: Iad1ccfe50ecac286ff7a796aec909bec70b9520d
2023-06-19 16:53:22 +02:00
Mårten Kongstad
a2e152a139 aconfig: generate Java flag name constants
Generate Java constants for use in @FlaggedApi(flag = ...).

Also update the generated Java code to use the constants when reaching
out to DeviceConfig instead of hard-coding (duplicate) strings.

Bug: 285288440
Test: atest aconfig.test aconfig.test.java
Change-Id: I1127cacba650cc7a7896b1533e03631d7f5ec71b
2023-06-19 16:12:01 +02:00
Mårten Kongstad
403658f9cb aconfig: use proto struct directly
Remove the hand-crafted wrappers around the structures auto-generated
from protos/aconfig.proto, and use the auto-generated structs directly
intead. This gets rid of a lot of manual repetition, and its inherent
risk.

Also unify how individual fields read from text proto are verified (e.g.
is the flag.name field a valid identifier).

Also change the intermediate cache format from JSON to binary protobuf.

The concept of a 'cache' as an intermediate internal format to represent
parsed input stays. The command line interface still refers to caches.
At the moment a cache file is identical to a parsed_file protbuf, and
the code exploits this internally.

A couple of points regarding the auto-generated structs:

  - Vectors are named in the singular (e.g. parsed_flags.parsed_flag is
    a Vec<ProtoParsedFlag>) because this improves ergonomics for all
    devs working with aconfig input files

  - The auto-generated structs have fields that are of type Option<T>
    and convenience methods (named the same as the fields) to access T

Test: atest aconfig.test aconfig.test.java
Bug: 283910447
Change-Id: I512820cc4bc6c543dea9f6a4356f863120a10be3
2023-06-19 16:04:32 +02:00
Treehugger Robot
3a2c089415 Merge changes from topic "aconfig-prepare-for-proto-structs" am: a99ac90eb5 am: 36e4f3fb37 am: 643be21ccb
Original change: https://android-review.googlesource.com/c/platform/build/+/2628050

Change-Id: I5c5138bdba4275aee2eab015e3246f64c8715acd
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-19 12:51:37 +00:00
Treehugger Robot
36e4f3fb37 Merge changes from topic "aconfig-prepare-for-proto-structs" am: a99ac90eb5
Original change: https://android-review.googlesource.com/c/platform/build/+/2628050

Change-Id: I11dc893158f85eb104ac8085714079c5ca3ad6ef
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-19 11:28:46 +00:00
Treehugger Robot
a99ac90eb5 Merge changes from topic "aconfig-prepare-for-proto-structs"
* changes:
  aconfig: fix incorrect source path in test cache
  aconfig: reduce number of #[cfg(feature = "cargo")] uses
2023-06-19 10:43:34 +00:00
Treehugger Robot
dab757ddc2 Merge "Handle zip64 extra fields better" am: 57c1aa7b16 am: 5c3a34b358 am: acddbb78de
Original change: https://android-review.googlesource.com/c/platform/build/+/2628431

Change-Id: I40f8a83bcd76d49a4512f3f26079442696e07fc3
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-18 00:45:05 +00:00
Treehugger Robot
5c3a34b358 Merge "Handle zip64 extra fields better" am: 57c1aa7b16
Original change: https://android-review.googlesource.com/c/platform/build/+/2628431

Change-Id: Ieefcc8ee08c2d91f447fc5209967d43db7454c75
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-17 23:04:21 +00:00
Treehugger Robot
57c1aa7b16 Merge "Handle zip64 extra fields better" 2023-06-17 22:12:26 +00:00
Kelvin Zhang
1e774245a4 Handle zip64 extra fields better
Test: check_target_files_signatures
Bug: 283033491
Change-Id: I7da89f8389c09cc99201cff342483c158bd7e9c1
2023-06-17 09:21:46 -07:00
Wei Li
d263695cd4 Include static libraries information in Android SBOM.
Bug: 280852724
Test: CIs
Test: lunch aosp_cf_x86_64_phone-userdebug && m sbom
Change-Id: Ie2365d79ba24910b7ace132b578589be10a17d78
2023-06-16 23:02:09 -07:00
Treehugger Robot
4b6b6f95d2 Merge "Use patch instead of git apply." am: 0d2d11bc00 am: 251eac36ab am: c3afc51e86
Original change: https://android-review.googlesource.com/c/platform/build/+/2627735

Change-Id: I6db10d07636ab82de5b9366f4687b94713472b4c
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-17 04:19:12 +00:00
Treehugger Robot
251eac36ab Merge "Use patch instead of git apply." am: 0d2d11bc00
Original change: https://android-review.googlesource.com/c/platform/build/+/2627735

Change-Id: Ia63a8d9eb8877d763bcfee18beedba41d775c481
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-17 02:49:12 +00:00
Treehugger Robot
0d2d11bc00 Merge "Use patch instead of git apply." 2023-06-17 01:56:52 +00:00
Treehugger Robot
cdfd8e3f4f Merge "aconfig: change java flag method name to camelCase" am: 59d662524c am: bec4ce710f am: 9c7c5dabaa
Original change: https://android-review.googlesource.com/c/platform/build/+/2626776

Change-Id: I5158885f1f8dc25a166c4f06ced7940663fe0042
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-17 00:09:54 +00:00
Treehugger Robot
bec4ce710f Merge "aconfig: change java flag method name to camelCase" am: 59d662524c
Original change: https://android-review.googlesource.com/c/platform/build/+/2626776

Change-Id: I7d5747d6a4a33f6b49e2c13f06bf539c71c324ee
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-16 22:41:03 +00:00
Alex Buynytskyy
bce1a51df8 Use patch instead of git apply.
Bug: 281682520
Test: run locally
Change-Id: Ia84055b7a9074836c6d7b3039ba980e39043a41e
2023-06-16 20:23:36 +00:00
Zhi Dou
af81e20653 aconfig: change java flag method name to camelCase
Before java code will directly use the flag name as the method name.
This change adds funciton to try the best to convert flag name to
camelCase, and then use the camelCase string as the method name in the
generated code.

Bug: 279483816
Test: atest aconfig.test aconfig.test.java
Change-Id: I45fc6df46c9d535cd38a657a41313202f9b660af
2023-06-16 12:40:40 +00:00
Jooyung Han
db9d2f09b7 Merge "Remove --blkid_path argument" am: e4635b3e04 am: 4941f6fd47 am: da705680d7
Original change: https://android-review.googlesource.com/c/platform/build/+/2624574

Change-Id: Idebb626816eebee2f789ec7d372a0a016d192ea9
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-16 08:36:48 +00:00
Jooyung Han
4941f6fd47 Merge "Remove --blkid_path argument" am: e4635b3e04
Original change: https://android-review.googlesource.com/c/platform/build/+/2624574

Change-Id: Icb064093d587479713e7da5bd670d4cb36f977a3
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-16 07:30:28 +00:00
Jooyung Han
e4635b3e04 Merge "Remove --blkid_path argument" 2023-06-16 06:48:57 +00:00
Bob Badour
a0b53ceac2 Fix godoc typos.
Methods got renamed, but the godoc didn't.

Test: m droid dist
Change-Id: Ia1181a2c7a876e54b8cd26ba1c91ce731ba8ac96
2023-06-15 16:26:21 -07:00
Treehugger Robot
d9f460f252 Merge changes I9c1a5346,I0a9d2c58 am: 5c73eb33db am: 8ac8ce2b16 am: 58316e6b49
Original change: https://android-review.googlesource.com/c/platform/build/+/2627031

Change-Id: Ib418962a6efe79c672e571db450d8961532e065c
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-15 18:28:23 +00:00
Treehugger Robot
8ac8ce2b16 Merge changes I9c1a5346,I0a9d2c58 am: 5c73eb33db
Original change: https://android-review.googlesource.com/c/platform/build/+/2627031

Change-Id: Ied665b5cbd2d1b207c184458eef0ef49259df129
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-15 16:36:23 +00:00
Mårten Kongstad
0cd8092376 aconfig: fix incorrect source path in test cache
Bug: 283910447
Test: atest aconfig.test
Change-Id: I9bc34e838a0945891f866e9788bcf9f4f6c23d3b
2023-06-15 11:43:33 +02:00
Mårten Kongstad
f94225266f aconfig: reduce number of #[cfg(feature = "cargo")] uses
Group statements with identical #[cfg(feature = "cargo")] attributes in
the same block. This reduces repetition and makes the code easier to
read and less error prone.

Bug: 284779868
Test: atest aconfig.test
Change-Id: Iebdcd20e7cd22cb641424a1af594f5c9ac57b623
2023-06-15 09:28:49 +02:00
Kelvin Zhang
38d0c373ac Fix python3.11's support for zip64
Bug: 283033491
Test: check_target_files_signatures -v -l
Change-Id: I9c1a5346e3a5f3920242dc9a5268d999f50a4937
2023-06-14 12:54:57 -07:00
Kelvin Zhang
b789e84499 Search for partition maps in IMAGES dir as well
Partition images are allowed to be in either IMAGES/ or RADIO/ dir of a
target_files zip, so when searching for .map files we should look in
both dirs.

Test: th
Bug: 227848550
Change-Id: I0a9d2c582d8f5d570237434902fac012513c9aad
2023-06-14 10:01:21 -07:00
Zhi Dou
aab20107a3 Merge "aconfig: Java codegen iteration 1" am: 06377d79ab am: 678166cbdb am: 1024339fd0
Original change: https://android-review.googlesource.com/c/platform/build/+/2619534

Change-Id: I68a3890cd26caad4f25ce6fa91b9c6a59520c40a
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-14 15:31:47 +00:00
Bob Badour
5a3e4a439a Fix usage message.
Incorrectly had an old dumpresolutions usage.

Test: m droid dist

Change-Id: Icf1a8ef99978c1287d88aae953e4a23f310f900a
2023-06-14 08:05:31 -07:00
Zhi Dou
678166cbdb Merge "aconfig: Java codegen iteration 1" am: 06377d79ab
Original change: https://android-review.googlesource.com/c/platform/build/+/2619534

Change-Id: I3ef56bcda7625dcaf70abb937b1b6503b4e7333d
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-14 14:02:19 +00:00
Zhi Dou
06377d79ab Merge "aconfig: Java codegen iteration 1" 2023-06-14 13:21:16 +00:00
Jooyung Han
629490292a Remove --blkid_path argument
deapexer doesn't need it.

Bug: 279858383
Test: presubmit
Change-Id: If0ec42b5edd4642f07c96ba641030c4dd6fb4660
2023-06-14 15:16:34 +09:00
Treehugger Robot
a114c493ca Merge "aconfig: reject consecutive underscores in identifiers" am: c01b9a2877 am: e6d7954914 am: 71b0098f49
Original change: https://android-review.googlesource.com/c/platform/build/+/2624469

Change-Id: Iaed8a96cf23e18133ec091accb75eb7026c938af
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-14 00:08:45 +00:00
Treehugger Robot
e6d7954914 Merge "aconfig: reject consecutive underscores in identifiers" am: c01b9a2877
Original change: https://android-review.googlesource.com/c/platform/build/+/2624469

Change-Id: I08a32db9a755df12a8a6744a8843dacc9dc74607
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-13 22:40:50 +00:00
Zhi Dou
4655c967e1 aconfig: Java codegen iteration 1
This change includes
    - refactor generated java code to generate
        - Flags.java to support the static API
	- FeatureFlagsImpl.java to support injection API
	- FeatureFlags.java interface

Bug: 279483816
Test: atest aconfig.test aconfig.test.java
Change-Id: If0d4baf317b9174635cd0fff3832ab7091ee52ed
2023-06-13 14:45:53 +00:00
Mårten Kongstad
d18c978f31 aconfig: reject consecutive underscores in identifiers
The Java codegen may translate flag names (snake_case) to Java
camelCase, dropping the underscores. The flags a_b and a__b will
translate to the same camelCase form, which is ambiguous.

Circumvent this problem by disallowing consecutive underscores in flag
names, flag namespaces, and packages.

Bug: 284252015
Test: atest aconfig.test
Change-Id: I2586a38160723c06265a140193da8178655553e4
2023-06-13 13:30:58 +02:00
Dennis Shen
f2e371c32f Merge "aconfig: update cpp codegen to use static methods" am: f1cd83bece am: d57d4fed31 am: c9156e607b
Original change: https://android-review.googlesource.com/c/platform/build/+/2620871

Change-Id: I7b7f141892ed2aa200ac87b3534dbcc771e4586c
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-12 15:11:31 +00:00
Dennis Shen
d57d4fed31 Merge "aconfig: update cpp codegen to use static methods" am: f1cd83bece
Original change: https://android-review.googlesource.com/c/platform/build/+/2620871

Change-Id: I614fac778dd96a3da711ea9cbd78ee8656725cea
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-12 13:43:38 +00:00
Dennis Shen
f1cd83bece Merge "aconfig: update cpp codegen to use static methods" 2023-06-12 12:57:57 +00:00
Dennis Shen
4f78f10015 aconfig: update cpp codegen to use static methods
Update c++ codegen to static methods interface.

Bug: b/279483801
Test: atest aconfig.test
Change-Id: I78da3bbca6240bee660c692807930d00f2242b0a
2023-06-12 12:57:38 +00:00
Joe Onorato
0bbfa29e0c Merge "Define release flags in starlark instead of make" am: 593bef7150 am: 11a113676a am: 0cd220797b
Original change: https://android-review.googlesource.com/c/platform/build/+/2617915

Change-Id: Iea431abef7ea5a3704eab684b056ed9cf303a702
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-10 05:13:42 +00:00
Joe Onorato
11a113676a Merge "Define release flags in starlark instead of make" am: 593bef7150
Original change: https://android-review.googlesource.com/c/platform/build/+/2617915

Change-Id: I97dc13cec1e261baa7efe56d5a8b880f262956a8
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-10 03:49:13 +00:00
Joe Onorato
593bef7150 Merge "Define release flags in starlark instead of make" 2023-06-10 03:21:22 +00:00
Joe Onorato
d47d8e8636 Merge "Fix build breakage" am: ed1e3b50f3 am: d1ac3e6971 am: d433bbae25
Original change: https://android-review.googlesource.com/c/platform/build/+/2618395

Change-Id: I05f346c5292e0c319ea92c5f6c7d9b879de5aa0b
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-09 23:04:37 +00:00
Joe Onorato
d1ac3e6971 Merge "Fix build breakage" am: ed1e3b50f3
Original change: https://android-review.googlesource.com/c/platform/build/+/2618395

Change-Id: I7db1ffc93f713da34a301da04418b7b5a6afd86e
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-09 21:34:34 +00:00
Treehugger Robot
04d9f1e0de Merge "Supply apex_info.pb as parameter to delta_generator if file exists" am: df2af7c840 am: b2dec242f8 am: 8fdc4db774
Original change: https://android-review.googlesource.com/c/platform/build/+/2617618

Change-Id: I4c142f415bbce52f08ab2c6051420ea96356da2e
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-09 21:33:07 +00:00
Joe Onorato
ed1e3b50f3 Merge "Fix build breakage" 2023-06-09 20:57:36 +00:00
Joe Onorato
8b51859317 Fix build breakage
error: build/make/tools/aconfig/Android.bp:69:1: module "aconfig.test.java" variant "android_common": path dependency ":aconfig.test.flags{.srcjar}": unsupported device_config_definitions module reference tag ".srcjar"

Test: m aconfig.test.java
Change-Id: I3df980b06796c90dafee3940fe52809748305950
2023-06-09 13:39:41 -07:00
Cole Faust
386b3746ce Define release flags in starlark instead of make
So that we have a more restricted enviornment for this new configuration
axis that can also be imported into other tools more easily.

Test: Manually (this time also tested setting OUT_DIR outside of the tree)
Change-Id: I01d90e06e45cba756156af16f63e04f575877263
2023-06-09 13:10:05 -07:00
Treehugger Robot
b2dec242f8 Merge "Supply apex_info.pb as parameter to delta_generator if file exists" am: df2af7c840
Original change: https://android-review.googlesource.com/c/platform/build/+/2617618

Change-Id: I9751859fb7114e3cc216860bb2cb951fcefe74fa
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-09 20:04:17 +00:00
Treehugger Robot
df2af7c840 Merge "Supply apex_info.pb as parameter to delta_generator if file exists" 2023-06-09 19:23:31 +00:00
Zhi Dou
f72740212f Merge "aconfig: add Java integration tests" am: 1120cb8aa4 am: e6bcd369a9 am: b6108695f0
Original change: https://android-review.googlesource.com/c/platform/build/+/2607427

Change-Id: I9b91945b743d13363c20097dff41b04e4a1fbb7d
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-09 15:54:06 +00:00
Zhi Dou
e6bcd369a9 Merge "aconfig: add Java integration tests" am: 1120cb8aa4
Original change: https://android-review.googlesource.com/c/platform/build/+/2607427

Change-Id: I8f1a4d78fdcc297f719e10b9364bb16d266727fc
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-09 14:21:44 +00:00
Håkan Kvist
ddb968dce0 Supply apex_info.pb as parameter to delta_generator if file exists
Supply generator with parameter --apex_info_file=META/apex_info.pb
if the file exists.

This ensures that apex_info is included in payload header.

This is identical to the behaviour of brillo_update_payload
which is not being used since:
Invoke delta_generator directly
fcd731e3d6

Issue: 286253576
Test: Manual, confirm that apex_info is included in payload header
Change-Id: Ic096c5f8966beec8686f918aba462c955290a6c5
2023-06-09 12:46:58 +02:00
Mårten Kongstad
9c59c31499 aconfig: add Java integration tests
Add integration tests for Java. This test setup verifies that

  - the build system calls aconfig to generate a Java library
  - the Java test compiles against the auto-generated library
  - the auto-generated code returns expected values

Similar integration tests for C++ and Rust will be added in follow-up
CLs.

Note: the build does not currently support specifying that
tests/*.values should be applied, so the test flags will all be assigned
the defaults. A later CL will fix this.

Bug: b/283911467
Test: atest aconfig.test aconfig.test.java
Change-Id: Ia365e209261f4935a23e2dac9ef0ab5b60f76e52
2023-06-09 09:59:21 +02:00
Treehugger Robot
ecc527eb52 Merge "fixup! Allow ota_from_target_file to work entirely on directories" am: d60dbf9773 am: 980458ae17 am: 08fc8461a1
Original change: https://android-review.googlesource.com/c/platform/build/+/2617616

Change-Id: I77011348d3b32a35217b804feb9f4ea38fcffea9
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-09 00:46:53 +00:00
Treehugger Robot
980458ae17 Merge "fixup! Allow ota_from_target_file to work entirely on directories" am: d60dbf9773
Original change: https://android-review.googlesource.com/c/platform/build/+/2617616

Change-Id: I093dccc54174e68f282d9a0f79e6a5ba7f2555ea
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-08 23:08:30 +00:00
Treehugger Robot
d60dbf9773 Merge "fixup! Allow ota_from_target_file to work entirely on directories" 2023-06-08 22:20:30 +00:00
Mårten Kongstad
35f8c6b0a1 Merge changes from topic "aconfig-rename-namespace-to-package" am: ec5da68dbd am: 5c4398ba52 am: 7446093e9b
Original change: https://android-review.googlesource.com/c/platform/build/+/2617393

Change-Id: Ia6cc72cf50115bac6c5816cd786dde5156e7ba43
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-08 17:39:13 +00:00
Mårten Kongstad
5c4398ba52 Merge changes from topic "aconfig-rename-namespace-to-package" am: ec5da68dbd
Original change: https://android-review.googlesource.com/c/platform/build/+/2617393

Change-Id: Ia24fee81a09ad3dd8bbecabfab38359b1d6cceb5
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-08 16:13:24 +00:00
Mårten Kongstad
ec5da68dbd Merge changes from topic "aconfig-rename-namespace-to-package"
* changes:
  aconfig: include namespace in create-device-config-defaults
  aconfig: improve code diffs in tests
  aconfig: add namespace field to flag_declaration and parsed_flag
  aconfig: allow dots in package fields
  aconfig: rename namespace -> package
2023-06-08 15:12:10 +00:00
Håkan Kvist
b8982fc538 fixup! Allow ota_from_target_file to work entirely on directories
The following log message would always be shown:
WARNING : Cannot find care map file in target_file package

Break out of the care map copying loop as soon a file has been
copied. This ensures that else statement is only executed if no
care map file exists.

Test: Manual. Run ota_from_target_files with target-zip with and
    without care map files.
Change-Id: Ia196aa182ed81f21424317a7005f5634866b4b99
2023-06-08 15:17:57 +02:00
Mårten Kongstad
202102f7be aconfig: include namespace in create-device-config-defaults
Update the output format of create-device-config-defaults to include the
flag's namespace. Also change the delimiters. The new format is

  <namespace>:<package>.<flag-name>=[enabled|disabled|

Bug: 285468565
Test: atest aconfig.test
Change-Id: I9b4ca1611cca8528dc341fc12812b614c86f6c08
2023-06-08 11:28:09 +02:00
Mårten Kongstad
b025507857 aconfig: improve code diffs in tests
Implement a helper function to make it easier for unit tests to diff
(and find the first difference) generated code and expected code.

Bug: 283910447
Test: atest aconfig.test
Change-Id: I460e8fbf05e8f33e8a62ecef67b2d9d77051e876
2023-06-08 11:27:59 +02:00
Mårten Kongstad
066575b95f aconfig: add namespace field to flag_declaration and parsed_flag
Add a new field to the proto messages flag_declaration and parsed_flag.

The new field will be used verbatim as a parameter when calling
DeviceConfig.getBoolean to read the value of a READ_WRITE flag. See the
DeviceConfig API for more info.

Note: not to be confused with the old namespace field, which has been
renamed to package.

Bug: 285211724
Test: atest aconfig.test
Change-Id: I2181be7b5e98fc334e5277fb5f7e386f1fe0b550
2023-06-08 11:27:57 +02:00
Mårten Kongstad
fbd71e2773 aconfig: allow dots in package fields
Allow package fields to include dots.

Update the generated code based on the package name: if the package name
is com.android.example:

  - java: package com.android.example; ...
  - C++: namespace com::android::example { ... }
  - Rust: mod com { mod android { mod example { ... } } }

Also, update examples to use dots in the package fields.

Also, remove unnecessary #include from the auto-generated C++ code: the
header should not include itself.

Bug: 285000854
Test: atest aconfig.test
Change-Id: I8a5352e25c64c34dee0725202a1b7c9957819de8
2023-06-08 11:25:43 +02:00
Devin Moore
d10e1e46b7 Merge "Add new script to analyze static/shared library usage" am: 9bf17fd9dd am: 0d3d349885 am: df53f92fd6
Original change: https://android-review.googlesource.com/c/platform/build/+/2590128

Change-Id: Ic48dc344b6c31db45bd3f72d1d2716c86bbe9921
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-07 22:20:47 +00:00
Devin Moore
0d3d349885 Merge "Add new script to analyze static/shared library usage" am: 9bf17fd9dd
Original change: https://android-review.googlesource.com/c/platform/build/+/2590128

Change-Id: I6fced63d717ea4b5cfff7fff0a4e00e0e3a6deba
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-07 20:55:17 +00:00
Devin Moore
9bf17fd9dd Merge "Add new script to analyze static/shared library usage" 2023-06-07 20:18:09 +00:00
Treehugger Robot
39113c0584 Merge "Resolve test crash in build/make/tools/compliance." am: 69227ff9ae am: 295a2df7cd am: 9c1ac01d12
Original change: https://android-review.googlesource.com/c/platform/build/+/2619070

Change-Id: I391d522aad017d1d72b5274cd643062d00ef84b4
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-07 20:07:44 +00:00
Treehugger Robot
295a2df7cd Merge "Resolve test crash in build/make/tools/compliance." am: 69227ff9ae
Original change: https://android-review.googlesource.com/c/platform/build/+/2619070

Change-Id: I28a58137fe08032a88da937a8bb412eeb0766dc6
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-07 18:19:52 +00:00
Jingwen Chen
376b6b6378 Resolve test crash in build/make/tools/compliance.
Test: m compliance_sbom
Test: go test ./...
Fixes: 285157766
BUG: 282746032
Change-Id: I5cc39f45f7ac3d9c574e2ebaa7fdfda864cb109d
2023-06-07 13:06:13 +00:00
Mårten Kongstad
9fb58965af aconfig: rename namespace -> package
What used to be referred to as a namespace is now called a package.

This CL is a semantic change only.

Bug: 285000854
Test: m nothing
Test: atest aconfig.test
Change-Id: If3fca67c415af75b44f316e16666b97089407069
2023-06-07 14:53:04 +02:00
Treehugger Robot
161b0fab75 Merge changes from topic "aconfig-gen-device-config-files" am: 2bb714ed2b am: 2960cffd7b am: 167b26de05
Original change: https://android-review.googlesource.com/c/platform/build/+/2613433

Change-Id: I22dd2fddc3f0e5ecdc8ac8c8ffb01289d2103f6b
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-07 11:58:12 +00:00
Treehugger Robot
2960cffd7b Merge changes from topic "aconfig-gen-device-config-files" am: 2bb714ed2b
Original change: https://android-review.googlesource.com/c/platform/build/+/2613433

Change-Id: I06f9c490eea3026808db32a80b3f2f2e2909050f
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-07 10:35:09 +00:00
Treehugger Robot
2bb714ed2b Merge changes from topic "aconfig-gen-device-config-files"
* changes:
  aconfig: add create-device-config-sysprops command
  aconfig: add create-device-config-defaults command
  aconfig: add test utilities
  aconfig: cache.rs: remove unnecessary use statements
  aconfig: give commands ownership of all arguments
2023-06-07 09:50:23 +00:00
Treehugger Robot
54eca11796 Merge "Use python based unzip function for portability" am: 3cda7834aa am: 976a36fb8e am: 60b74959a8
Original change: https://android-review.googlesource.com/c/platform/build/+/2615433

Change-Id: I73b5e85885ee22eaa5eb98623715abcaea17489e
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-06 20:29:33 +00:00
Treehugger Robot
e7ae241f2e Merge "Always set a avb salt for hermetic build" am: fff48d788f am: 4c357806a3 am: f9fb104f9e
Original change: https://android-review.googlesource.com/c/platform/build/+/2613794

Change-Id: Ie529f3b071ba7def82cb963ea1a99eb4ead0f334
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-06 20:27:20 +00:00
Treehugger Robot
976a36fb8e Merge "Use python based unzip function for portability" am: 3cda7834aa
Original change: https://android-review.googlesource.com/c/platform/build/+/2615433

Change-Id: Id10787e68ac4a0a6ea0ada47a24f46dd7e128915
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-06 19:08:34 +00:00
Treehugger Robot
4c357806a3 Merge "Always set a avb salt for hermetic build" am: fff48d788f
Original change: https://android-review.googlesource.com/c/platform/build/+/2613794

Change-Id: I3feffd1c65f8e92e7c6a6c6e0481e206b28ec8dd
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-06 19:07:16 +00:00
Kelvin Zhang
7c9205b008 Use python based unzip function for portability
Bug: 283033491
Test: th
Change-Id: Ief86b55c1d4a14220a0fb593c583a721d59cf86c
2023-06-05 15:54:12 -07:00
Devin Moore
bd13e63e99 Add new script to analyze static/shared library usage
Parses module-info.json, gathers stats on how many times each library is
included shared or statically.

Can print a list of libraries that would be a candidate for changing
from static to shared or visa versa.

Test: m
Bug: 280829178
Change-Id: I4bbffbd673ab2e08c69d0ab6e68402be77c9ffbc
2023-06-05 20:23:36 +00:00
Kelvin Zhang
c819b29f46 Always set a avb salt for hermetic build
When building images via `m` , build_image.py is invoked directly
without going through add_img_to_target_files. To ensure images built in
either way are identical, move uuid/salt computation to build_image.py,
so that the same uuid/salt will be used.

Bug: 281960439
Test: m installclean && m && m target-files-dir , maks sure images in
$OUT and $OUT/obj/PACKING/target_files_intermediates are identical

Change-Id: Icdab29df84f5a0ec7c080f99f9fdbdc3c9b10b90
2023-06-05 09:59:54 -07:00
Treehugger Robot
b2989fc109 Merge "Adding option to disable fastboot_info" am: ecf635f507 am: 8f26a9ca1d am: b083f09783
Original change: https://android-review.googlesource.com/c/platform/build/+/2612133

Change-Id: Ic02ad4911e965eb9d07e0882c804f56d945ad6e9
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-03 04:13:22 +00:00
Treehugger Robot
8f26a9ca1d Merge "Adding option to disable fastboot_info" am: ecf635f507
Original change: https://android-review.googlesource.com/c/platform/build/+/2612133

Change-Id: I45872eab4df9b1044ff6a6fb1c582f5cd316a365
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-03 02:42:53 +00:00
Treehugger Robot
ecf635f507 Merge "Adding option to disable fastboot_info" 2023-06-03 01:56:41 +00:00
Dennis Shen
d0799b1a77 Merge "aconfig: update rust code gen to use libflags_rust" am: 3b0ca61d36 am: 8c1cce4e39 am: 1b0d706359
Original change: https://android-review.googlesource.com/c/platform/build/+/2607428

Change-Id: Id767e678c777b383a0143584c4bd9dd55e858abe
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-02 17:41:54 +00:00
Dennis Shen
8c1cce4e39 Merge "aconfig: update rust code gen to use libflags_rust" am: 3b0ca61d36
Original change: https://android-review.googlesource.com/c/platform/build/+/2607428

Change-Id: I5f100c044c906f06714ad1b4acbcbea83048a770
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-02 15:48:31 +00:00
Dennis Shen
3b0ca61d36 Merge "aconfig: update rust code gen to use libflags_rust" 2023-06-02 15:13:40 +00:00
Mårten Kongstad
c31a6ff653 aconfig: add create-device-config-sysprops command
Add a new "create-device-config-sysprops" command that works like
"create-device-config-defaults" but for system properties.

DeviceConfig is a Java service, and will mirror (some of) its values by
setting system properties in the persist.device_config namespace. Native
code will access DeviceConfig (actually, the system properties) via the
server_configurable_flags library.

The new command writes a file that can be appended to /system/build.prop
to pre-populate persist.device_config before DeviceConfig has started.

Like create-device-config-defaults, the new command skips READ_ONLY
flags.

Bug: 285468565
Test: atest aconfig.test
Change-Id: I311c7c5e0b03dc897b09204137d43cc182324717
2023-06-02 16:47:04 +02:00
Mårten Kongstad
f02734e915 aconfig: add create-device-config-defaults command
DeviceConfig is the backend for READ_WRITE flags.

Add a new command "create-device-config-defaults" to create a file that
DeviceConfig will read to pre-populate its data store on first init.

This will be used to quickly switch flag values during CI tests:
rebuilding and reflashing a device would have the same effect, but would
be costlier. This feature is not expected to be used outside CI tests.

Note: because DeviceConfig only works with READ_WRITE flags, the
generated file excludes READ_ONLY flags.

Bug: 285468565
Test: atest aconfig.test
Change-Id: I4caff1a10647b8da0ce4e3615678993a957a92dd
2023-06-02 16:46:28 +02:00
Mårten Kongstad
83a8760bbc aconfig: add test utilities
Create a test utility function to create a Cache from the files in
testdata/*. A follow-up CL will update the unit tests to use this
instead of creating their own caches.

Bug: 283910447
Test: atest aconfig.test
Change-Id: Ice5064eadb0babde5eb38d292330d213ab136d96
2023-06-02 12:54:22 +02:00
Mårten Kongstad
1cd166cd31 aconfig: cache.rs: remove unnecessary use statements
Remove unnecessary use from the cache::test module: they already covered
by `use super:*;`.

Bug: 283910447
Test: atest aconfig.test
Change-Id: I9e03385629f38180c0f92080c7f097a8e0d9ef69
2023-06-02 12:52:48 +02:00
Mårten Kongstad
b27f2ce436 aconfig: give commands ownership of all arguments
Pass the Cache argument to command::create_<lang>_lib functions by value
instead of by reference, to align with other commands.

The intended ownership flow is as follows:

  - main creates objects based on command line arguments
  - main hands commands ownership of the objects
  - command processes the objects
  - command gives main ownership of any generated output
  - main writes the output to file

Rationale: commands.rs is a unit testable version of main, and to the
rest of aconfig, acts as the top level entry point; main.rs exists only
to parse command line arguments and perform I/O.

Bug: 283910447
Test: atest aconfig.test
Change-Id: I1e1dea7da8ecc2bb6e2f7ee4a6df64562c148959
2023-06-02 12:49:26 +02:00
Treehugger Robot
eb6c9b7921 Merge "Open ab_partitions.txt with just read access" am: f9a6f5f06b am: b5535214ad am: 4cd34e5ce6
Original change: https://android-review.googlesource.com/c/platform/build/+/2611511

Change-Id: I263ecf630f7e62caaa6a70e4642f02122dfb64d0
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-02 00:13:28 +00:00
Treehugger Robot
b5535214ad Merge "Open ab_partitions.txt with just read access" am: f9a6f5f06b
Original change: https://android-review.googlesource.com/c/platform/build/+/2611511

Change-Id: I075bf60c1586dd0af5d39cc23db97e9ae5c5532d
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-06-01 22:42:53 +00:00
Daniel Zheng
ee10d07057 Adding option to disable fastboot_info
Fastboot_info can be disabled if use_fastboot_info is set to false.
Adding this flag as fastboot-info.txt is currently broken

Test: m updatepackage -> inspect contents
Bug: 284263071
Change-Id: I3e0ca13968ba9747cc39284ea6798981d22ad5e5
2023-06-01 14:21:58 -07:00
Kelvin Zhang
89b87f6aa9 Open ab_partitions.txt with just read access
We don't actually need write permission, so going with least privilege
principle. We have observed some mysterious permission denied errors on server environments. Without detailed logs or access to the server it's hard to pinpoint what the root cause is. This is an attempt/hypothesis to fix the permission denied error.

Test: th
Bug: 283033491
Change-Id: I52dc360d593aab57c749109994bf3e1e3625d0ce
2023-06-01 17:28:25 +00:00
Dennis Shen
6461673140 aconfig: update rust code gen to use libflags_rust
Bug: b/284096062, b/279483360
Test: atest aconfig.test
Change-Id: I4c96f35807de5cb2a745ba1c653513d9b22a5013
2023-05-31 14:35:59 +00:00
Treehugger Robot
1fe7af4274 Merge "aconfig: restrict valid namespace and flag names" am: cfc5f5e9fd am: 2d8615198c am: 6129693adb
Original change: https://android-review.googlesource.com/c/platform/build/+/2607087

Change-Id: Ic26969bbb8b273ceeabd4004e2b3ac07815fac03
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-30 22:11:32 +00:00
Treehugger Robot
2d8615198c Merge "aconfig: restrict valid namespace and flag names" am: cfc5f5e9fd
Original change: https://android-review.googlesource.com/c/platform/build/+/2607087

Change-Id: Ifb0c77ff4e3c51f8bd0dce787abb022946518c79
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-30 20:42:52 +00:00
Mårten Kongstad
00cf045c85 aconfig: restrict valid namespace and flag names
The namespace and flag names will be used as identifiers in the
auto-generated code. Place restrictions on what constitutes a valid
name.

Valid identifiers are those that match /[a-z][a-z0-9_]/. aconfig
explicitly does not implement any automatic translation to make names
valid identifiers: this sidesteps potential conflicts such as "foo.bar"
and "foo_bar" mapping to the same name if dots were translated to
underscores.

Bug: b/284252015
Test: atest aconfig.test
Change-Id: I38d005a74311e5829e540063404d1565071e6e96
2023-05-26 17:09:10 +02:00
Mårten Kongstad
e144b18ab9 Merge changes from topic "aconfig-create-rust-lib" am: aa977fb083 am: 5c2756c13a am: e6c068798d
Original change: https://android-review.googlesource.com/c/platform/build/+/2602726

Change-Id: I84c5ac6dc692612f1caa9c45fc99be8319287c3f
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-26 09:01:35 +00:00
Mårten Kongstad
5c2756c13a Merge changes from topic "aconfig-create-rust-lib" am: aa977fb083
Original change: https://android-review.googlesource.com/c/platform/build/+/2602726

Change-Id: I2e708156312d2274c430a698b983b5cfe73c20ab
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-26 07:08:20 +00:00
Mårten Kongstad
aa977fb083 Merge changes from topic "aconfig-create-rust-lib"
* changes:
  aconfig: first iteration of Rust codegen
  aconfig: improve dump --format=debug output
2023-05-26 06:00:47 +00:00
Mårten Kongstad
f73b963283 aconfig: first iteration of Rust codegen
Add a new `create-rust-lib` command to generate Rust code. The output is
a src/lib.rs file; the build system is assumed to set the generated
crate's name.

For READ_ONLY flags, the generated code returns a hard-coded true or false.

For READ_WRITE flags, the generated code reaches out to DeviceConfig via
the cc_library server_configurable_flags via the
libprofcollect_libflags_rust Rust bindings. The build system is assumed
to add this to the generated crate's dependencies.

Note: libprofcollect_libflags_rust seems generic enough that it should
be moved to an official Rust wrapper for server_configurable_flags. This
is tracked in b/284096062.

Summary of module the built system is assumed to wrap the auto-generated
code in:

  rust_library {
      name: "lib<namespace>_rs",
      crate_name: "<namespace>_rs",
      edition: "2021",
      clippy_lints: "none",
      no_stdlibs: true,
      lints: "none",
      srcs: ["src/lib.rs"],
      rustlibs: [
          "libprofcollect_libflags_rust",
      ],
  }

Also add a set of test input to be used in the unit tests for a more
coherent test strategy. A follow-up CL will migrate the code in
commands.rs, codegen_java.rs and codegen_cpp.rs.

Bug: 279483360
Bug: 283907905
Test: atest aconfig.test
Test: manual: create cache from files in testdata, create rust lib, add to module template above, verify the module builds
Change-Id: I02606aa3686eda921116e33f7e2df8fd1156a7aa
2023-05-25 16:18:59 +02:00
Treehugger Robot
26fda7e335 Merge "Fix merge_target_files in GRF case" am: b3adc24301 am: 592ad2cb13 am: 7c629dbb53
Original change: https://android-review.googlesource.com/c/platform/build/+/2564310

Change-Id: I5a0f0236e07f50b7d8cfa05b476734ed3f98300a
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-25 13:06:05 +00:00
Treehugger Robot
592ad2cb13 Merge "Fix merge_target_files in GRF case" am: b3adc24301
Original change: https://android-review.googlesource.com/c/platform/build/+/2564310

Change-Id: I564bb6d43bfa5b9cb79463f0ba50038d03ef6abb
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-25 11:37:45 +00:00
Treehugger Robot
b3adc24301 Merge "Fix merge_target_files in GRF case" 2023-05-25 10:46:32 +00:00
Mårten Kongstad
993111f91c aconfig: improve dump --format=debug output
Use Rust's {:#?} formatter for more readable output.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I127f413e3d7aebfba96cad1dd58d9e261dd613a4
2023-05-24 14:56:45 +02:00
Treehugger Robot
4f4da804bf Merge "Unsparse images before generating OTA" am: b4406d06f0 am: 74ad39125c am: 8b55fd52a0
Original change: https://android-review.googlesource.com/c/platform/build/+/2598008

Change-Id: Ib062901d1b91380b98da95f38666c66f1205f587
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-23 23:11:17 +00:00
Treehugger Robot
74ad39125c Merge "Unsparse images before generating OTA" am: b4406d06f0
Original change: https://android-review.googlesource.com/c/platform/build/+/2598008

Change-Id: I40ac20dd7b30d11e1eda77688718566af4c1c416
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-23 21:22:54 +00:00
Treehugger Robot
b4406d06f0 Merge "Unsparse images before generating OTA" 2023-05-23 20:51:10 +00:00
Treehugger Robot
2f727af4b9 Merge "Add 16K artifacts to PREBUILT_IMAGES/ dir of target_files" am: 8485145bd7 am: f6d61bd3c1 am: af0cfd9e98
Original change: https://android-review.googlesource.com/c/platform/build/+/2595883

Change-Id: I4f6bac39abfd1ec19d21b763201770c92a2e0c3f
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-23 04:55:01 +00:00
Treehugger Robot
f6d61bd3c1 Merge "Add 16K artifacts to PREBUILT_IMAGES/ dir of target_files" am: 8485145bd7
Original change: https://android-review.googlesource.com/c/platform/build/+/2595883

Change-Id: I36815cfa877645e242d40a3b4b4cadd0e50a6ef5
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-23 03:30:56 +00:00
Treehugger Robot
8485145bd7 Merge "Add 16K artifacts to PREBUILT_IMAGES/ dir of target_files" 2023-05-23 02:36:17 +00:00
Yihan Dong
355b6b4a52 Merge "Remove GTS-Verifier license check" into udc-dev am: a75b82f9d2
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/+/23343092

Change-Id: I41e817b323c93d103486825200936b41fdfac8d5
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-23 02:23:02 +00:00
Yihan Dong
a75b82f9d2 Merge "Remove GTS-Verifier license check" into udc-dev 2023-05-23 02:00:54 +00:00
Kelvin Zhang
2268091521 Unsparse images before generating OTA
Test: th
Bug: 283172692
Change-Id: Ie6d3dc704fd9a8c107e2888222e4c2bf804dad3e
2023-05-22 10:22:59 -07:00
Treehugger Robot
1ad09e0bfa Merge changes from topic "aconfig-stable-flag-order" am: f469fd6cc6
Original change: https://android-review.googlesource.com/c/platform/build/+/2595206

Change-Id: Ic66ea4047487b6f8c4c51237bf583d035b7032d1
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-22 12:30:11 +00:00
Mårten Kongstad
2f95444281 aconfig: sort items in cache by name
Introduce a builder pattern for constructing a cache from flag
declarations and flag values. Teach the builder to sort the flags by
name as the last step. This will ensure consistent dump output
regardless of the order flags are specified in the input files.

Bug: 279485059
Test: atest aconfig.test
Change-Id: Icdd62f51fa3761a469663f17581a83d9909e9ffe
2023-05-22 10:01:05 +02:00
Mårten Kongstad
af677038b3 aconfig: dump: support multiple caches
Teach `aconfig dump` to read multiple caches at the same time.

A cache represents a single namespace, and the Android build creates
multiple caches for different namespaces. By passing in all those cache
files to `aconfig dump`, aconfig will create a complete overview of all
flags in the Android tree.

Caches are traversed in order of namespace to produce the same output
regardless of the order the cache files are given on the command line.
If two caches use the same namespace, their order with respect to each
other is undefined.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I54c3950bbb7b2be7d96c8928e78ae83aa626c2e2
2023-05-22 09:57:48 +02:00
Kelvin Zhang
decee4adb7 Add 16K artifacts to PREBUILT_IMAGES/ dir of target_files
Test: th
Bug: 283506035
Change-Id: I9be3982b5a4324ec839937d10c33266c7d27c525
2023-05-19 14:31:00 -07:00
Yihan Dong
58c428e336 Remove GTS-Verifier license check
The whole GTS-Verifier will be removed. Before that, remove GTS-V
license check first.

Ignore-AOSP-First: This CL must be submit with other CLs in the topic to avoid breaking the test. This CL will be cherry pick to AOSP after submitting.

Bug: 283384166
Test: m gts
Change-Id: I5bc8c1f4609f55cedd5bfbed014b65de164e2b16
2023-05-19 10:38:55 +00:00
Wei Li
233d5b97f8 Some changes to support SBOM generation for b build unbundled APEXs.
1) Use output file path of installed files in build system since there is no PRODUCT_OUT in Bazel
2) Use CONTAINS to describe the relationship between a APEX and files it contains
3) Generate SBOM of APEXs, which is similar to SBOM of products

Bug: 275472038
Test: CIs
(cherry picked from https://android-review.googlesource.com/q/commit:fd7e6517d345d3b8d4af12dae345434d968b83b9)
Merged-In: I41622366e5e6ed9dc78cca7bc7bb69a1f8f9bd9f
Change-Id: I41622366e5e6ed9dc78cca7bc7bb69a1f8f9bd9f
2023-05-19 00:35:20 +00:00
Daniel Zheng
b87d46cb1a Merge "Add --max_threads to OTA generation" into udc-dev 2023-05-18 19:53:12 +00:00
Treehugger Robot
97f144f4f7 Merge "aconfig: Add first iteration of cpp codegen to aconfig" am: 4266df817d
Original change: https://android-review.googlesource.com/c/platform/build/+/2585679

Change-Id: Ic76387fcc26f48db6417cbf564a450d6acc6a184
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-17 19:56:49 +00:00
Treehugger Robot
4266df817d Merge "aconfig: Add first iteration of cpp codegen to aconfig" 2023-05-17 19:37:28 +00:00
Wei Li
802310c4fd Merge "Some changes to support SBOM generation for b build unbundled APEXs." am: eb035dc89f
Original change: https://android-review.googlesource.com/c/platform/build/+/2585677

Change-Id: I0f302bf5cf4e6313aedc836034ccd9111f4ec534
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-17 19:33:20 +00:00
Baligh Uddin
f04ce4eb36 Merge "Add upstream package of a prebuilt fork package, which will have the package information from the METADATA file." am: 82d450e501
Original change: https://android-review.googlesource.com/c/platform/build/+/2590485

Change-Id: I2b8dace3a3b2f257919c4f2a1ca775d766e0aff9
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-17 19:32:56 +00:00
Wei Li
eb035dc89f Merge "Some changes to support SBOM generation for b build unbundled APEXs." 2023-05-17 19:19:47 +00:00
Satoshi Futenma
6c8f896589 Support custom payload signer in merge_ota
Add options to handle custom payload signer as it is required in
merge_ota.py as well if the original OTA packages are signed by
the signer.

If input is only one OTA, clone apex_info.pb to the target.

Use common.ZipWriteStr instead of zipfile.writestr, this ensures
that the same permission for the entries as done by
ota_from_target_files.

Bug: 282189563
Test: manual. pass single OTA to merge_ota, with same signing
    parameters as originally used. Confirm that output zip is
    binary identical to input.
(cherry picked from https://android-review.googlesource.com/q/commit:1f93ce20c5a5f3e7c2f6b5e1f9fdc4d2572220cc)
Merged-In: I3b926b8cd69bc74dff6ccf8b5ccc72bedffcac6f
Change-Id: I3b926b8cd69bc74dff6ccf8b5ccc72bedffcac6f
2023-05-17 17:22:51 +00:00
Daniel Zheng
0d3c15c2ae Add --max_threads to OTA generation
Adding an option to configure max number of threads used during
ota_from_target_files

Bug: 277794445
Test: tested ota_from_target_files
(cherry picked from https://android-review.googlesource.com/q/commit:a30989bafa820faf24eca7e11e48f8f122d7991f)
Merged-In: I9bae92ee43c21926434ab04563a2b4f0678b73eb
Change-Id: I9bae92ee43c21926434ab04563a2b4f0678b73eb
this patch is necessary for partner to improve OTA generation b/277794445
2023-05-17 16:14:05 +00:00
Dennis Shen
1dc9ad4662 aconfig: Add first iteration of cpp codegen to aconfig
The general idea to reuse java codegen's very neat tiny template idea.
For generated cpp code, it is in the form of a collection of classes
inside a namespace. The reason we choose a collection of classes rather
than a collection of static functions is because gmock test technology
only supports mocking virtual method.

Bug: b/279483801
Test: atest aconfig.test
Change-Id: I9ba00667437ff7c3e147ff2828171fc95528bebf
2023-05-17 14:57:58 +00:00
Wei Li
16e7aa3c2e Add upstream package of a prebuilt fork package, which will have the package information from the METADATA file.
Test: CIs
Test: lunch barbet-user && m sbom
Change-Id: Ic8eb42c369de8c94c7977b9631ff4b9084dfef01
2023-05-17 01:38:05 +00:00
Joe Onorato
6ee16caf99 Merge "Use the namepace as the java package name in aconfig and call the class "Flags"" am: 7833547153
Original change: https://android-review.googlesource.com/c/platform/build/+/2588271

Change-Id: Id42b087eea8e723d2dc66534760d00842263dda6
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-16 23:14:30 +00:00
Treehugger Robot
e1623bb140 Merge "Allow source build to be an extracted target_files" am: fce28c1b97
Original change: https://android-review.googlesource.com/c/platform/build/+/2589846

Change-Id: I0a4ebf623140c02b39c7d3cb2e9bd060b7ea4090
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-16 23:14:19 +00:00
Joe Onorato
7833547153 Merge "Use the namepace as the java package name in aconfig and call the class "Flags"" 2023-05-16 23:13:52 +00:00
Treehugger Robot
fce28c1b97 Merge "Allow source build to be an extracted target_files" 2023-05-16 22:56:29 +00:00