Commit graph

9201 commits

Author SHA1 Message Date
Michael Merg
75d934f261 Set enforce SOONG_GEN_COMPDB=1 when running soong for ide_query
Change-Id: I125a82fb07285bf53e9a6d591dee69d85ee82050
2024-04-18 12:06:31 +00:00
Dennis Shen
e6b424ad4a Merge "aconfig: add support for local override" into main 2024-04-17 19:07:56 +00:00
Mårten Kongstad
9238a3ab76 check-flagged-apis: create list of @FlaggedApi errors
Teach check-flagged-apis to cross-check the data from its three input
sources. This allows the tool to detect

  - @FlaggedApi references to non-existent flags
  - @FlaggedApi APIs present in the build artifacts even though the flag
    is disabled
  - @FlaggedApi APIs not present in the build artifacts even though the
    flag is enabled

By passing in different sources, the tool can detect these errors for
any of the API surfaces (public, @SystemApi(MODULE_LIBRARIES), etc).

Note: the tool assumes that a disabled flag means that the @FlaggedApi
should not be present in the build output. This is currently true, but
won't be once metalava starts reverting @FlaggedApis to their previous
SDK snapshot.

Bug: 334870672
Test: atest --host check-flagged-apis-test
Test: check-flagged-apis --api-signature out/target/product/mainline_x86/obj/ETC/frameworks-base-api-current.txt_intermediates/frameworks-base-api-current.txt --flag-values out/soong/.intermediates/all_aconfig_declarations.pb --api-versions out/dist/data/api-versions.xml
Change-Id: I790234865f831af7d45895def14d1d6740365622
2024-04-17 16:40:03 +02:00
Mårten Kongstad
b673d3bb7d check-flagged-apis: parse API versions XML
Teach check-flagged-apis to parse API versions XML; this represents the
APIs after metalava has processed the source and kept APIs as is, or
reverted them to the previous SDK snapshot, according to their
@FlaggedApi flags.

As with the API signature parser, limit support to fields to keep things
simple; support for classes and methods will be added in later CLs.

Note: `m sdk dist` will generate an API versions XML file.

Bug: 334870672
Test: atest --host check-flagged-apis-test
Test: check-flagged-apis --api-signature out/target/product/mainline_x86/obj/ETC/frameworks-base-api-current.txt_intermediates/frameworks-base-api-current.txt --flag-values out/soong/.intermediates/all_aconfig_declarations.pb --api-versions out/dist/data/api-versions.xml
Change-Id: I779a0d0cdb8a50536d3fc8d517fa38ba4b0dcd1c
2024-04-17 15:27:48 +02:00
Mårten Kongstad
387ff6cfe8 check-flagged-apis: parse flag names and values
Teach check-flagged-apis to parse the parsed_flags protobuf generated by
aconfig.

Note: `m all_aconfig_declarations` generates a protobuf file that
contains all info about all flags.

Bug: 334870672
Test: atest --host check-flagged-apis-test
Test: check-flagged-apis --api-signature out/target/product/mainline_x86/obj/ETC/frameworks-base-api-current.txt_intermediates/frameworks-base-api-current.txt --flag-values out/soong/.intermediates/all_aconfig_declarations.pb
Change-Id: I397b32ae2a373b429ef6ce22e0a06a0f15202b91
2024-04-17 15:27:48 +02:00
Mårten Kongstad
20de405dd5 check-flagged-apis: parse API signature files
Teach check-flagged-apis to extract flagged APIs from API signature files.

To keep things simple, only consider fields for now: support for classes
and methods will be added in a later CL.

Note: `m frameworks-base-api-current.txt` will generate an API signature
file that includes both the platform and mainline APIs.

Bug: 334870672
Test: atest --host check-flagged-apis-test
Test: check-flagged-apis --api-signature out/target/product/mainline_x86/obj/ETC/frameworks-base-api-current.txt_intermediates/frameworks-base-api-current.txt
Change-Id: Ic244b896672569f44af793796189b34c1f9d0c36
2024-04-17 15:27:48 +02:00
Mårten Kongstad
f242ec8989 check-flagged-apis: add unit test infrastructure
Add scaffolding for unit tests.

Bug: 334870672
Test: atest --host check-flagged-apis-test
Change-Id: I5ccf2a6424c19e739923379cdc41c359388484da
2024-04-17 15:27:45 +02:00
Dennis Shen
9f236037a2 aconfig: add support for local override
1, add a new field in storage metadata proto, for each container, there
is a local override file.

2, update write api to expose the api to map a writable file given the
specific path. the cannonical api will require a storage metadata proto,
and then find the right file path and map it.

3, minor update to make the proto lib cc_library instead of
cc_library_static

Bug: b/312444587
Test: atest -c
Change-Id: Iaf0aff44c1ca3ad4bffc5e06bb99bb276b9069c5
2024-04-17 12:25:11 +00:00
Mårten Kongstad
dc3fc2e0bc check-flagged-apis: add Flag value class
Add a value class to represent Flag names. We could use plain Strings
but having a dedicated class (with no overhead compared to String) makes
the intent of the code much clearer.

Bug: 334870672
Test: m check-flagged-apis && check-flagged-apis
Change-Id: Icdd4fb97d3fd49e507b7559504ea173a3dc52dea
2024-04-17 09:54:53 +02:00
Mårten Kongstad
e0179976cd check-flagged-apis: add Symbol value class
check-flagged-apis will read contents from various sources, which use
different formats to represent the same piece information (e.g.
"class#field" or "<class><field>").

Introduce a Symbol value class to represent any API (i.e. a class, field
or method) in a unified format.

Bug: 334870672
Test: m check-flagged-apis && check-flagged-apis
Change-Id: Id9404294a87b23a9d43e5e13ce39ea5a92608e33
2024-04-17 09:54:53 +02:00
Mårten Kongstad
4feb33fefc Merge "check-flagged-apis: add support for command line options" into main 2024-04-17 05:52:09 +00:00
Mårten Kongstad
e2ec38c51b Merge "check-flagged-apis: add new CLI to verify @FlaggedApi APIs" into main 2024-04-17 05:51:48 +00:00
Daniel Zheng
2fc519a568 Merge "ota_from_target_files: read correct api level" into main 2024-04-16 19:32:37 +00:00
Daniel Zheng
dc411eda4c ota_from_target_files: read correct api level
ro.product.first_api_level looks like the value we actually want to read
to since it inherits from $PRODUCT_SHIPPING_API_LEVEL. Other variables
we attempted to look at were giving other values (e.g 202504 for pixel
watch)

Bug: 331202590
Test: th
Change-Id: Ied3da247cba45611132db6c9e3cdefb447891f96
2024-04-16 12:32:05 -07:00
Mårten Kongstad
acfeb11d86 check-flagged-apis: add support for command line options
Use clikt as the command line options parser library.

Bug: 334870672
Test: m check-flagged-apis && check-flagged-apis
Change-Id: I7c406456b00e29293294dcdbef411d2543a1e8d5
2024-04-16 10:32:17 +02:00
Mårten Kongstad
9008724b50 check-flagged-apis: add new CLI to verify @FlaggedApi APIs
Add a new CLI to verify that the build artifacts contain the right set
of @FlaggedApi APIs, based on the value of the corresponding aconfig
flag.

This CLI will act as an end-to-end test of Soong and metalava.

This CL only adds the project scaffolding; later CLs will add the
implementation.

Bug: 334870672
Test: m check-flagged-apis && check-flagged-apis
Change-Id: Ib00653f2a549217da2b0058867c711f35efd5760
2024-04-16 10:32:17 +02:00
Treehugger Robot
8b1e29ae7e Merge "Make output of MakeVbmeta consistent" into main 2024-04-15 22:27:36 +00:00
Ted Bauer
8f9ca8075f Merge "aconfig: switch read api lib to cc_library" into main 2024-04-15 17:22:00 +00:00
Kelvin Zhang
1d9b3daad2 Make output of MakeVbmeta consistent
The order of items in `partitions` map is non-deterministic. To make
sure that builds are hermetic, sort the map before use.

Test: th
Bug: 332550989
Change-Id: I76c7037ef9f03904899339813819fcf985a7c893
2024-04-15 09:45:03 -07:00
Dennis Shen
36661d4cd0 aconfig: switch read api lib to cc_library
Bug: b/321077378
Test: atest -c
Change-Id: I8d64396797b01a8ce2f4bee165596701940836e4
2024-04-12 21:21:44 +00:00
Dennis Shen
7d642b10df Merge "aconfig: add write api cpp test to presubmit" into main 2024-04-12 18:26:55 +00:00
Dennis Shen
e8fa42f45a aconfig: add write api cpp test to presubmit
Bug: b/321077378
Test atest -c

Change-Id: Ib04280f25eb6c4de6ff1523579e6ac96a868ff1d
2024-04-12 16:57:28 +00:00
Dennis Shen
f17dbcd8b7 aconfig: update flag info query api
Currently flag info query api is called get_boolean_flag_attribute, in
this change, we switched it over to a flag value type generic
implementation get_flag_attribute. So in the future we want to add more flag value types, this api can stay the same.

Bug: b/312444587
Test: atest -c
Change-Id: I2b272f3fa3cb1d0edc8b77a44bf37752ffe95925
2024-04-12 16:28:10 +00:00
Nariyasu Heseri
5ffefcc642 Ensure insertion order of Python dict
Add keys to the dict `partitions` in serial to have a deterministic
insertion order.
This guarantees the generated vbmeta.img is always the same on the same
input, fixing image consistency issues.

Reference:
https://android-review.googlesource.com/c/platform/build/+/2585397

Bug: 332550989
Test: Generate vbmeta several times, confirm identical result.
Change-Id: I39da5844045f497fdc89e6477a7e32ddacb70764
2024-04-12 07:57:37 +00:00
Dennis Shen
d74941b42d aconfig: create flag info file write c api
Bug: b/312444587
Test: atest -c
Change-Id: I310e1ed727ced454bec2016afe48f7a29561fac3
2024-04-11 20:23:22 +00:00
Dennis Shen
4647c0e1f0 aconfig: create flag info file write rust api
Bug: b/312444587
Test: atest aconfig_storage_write_api.test; atest
aconfig_storage_write_api.test.rust

Change-Id: Icbedc78ae78b2c7590539f008521c507c2b3df5b
2024-04-11 20:23:00 +00:00
Dennis Shen
d22eee50fa Merge changes from topic "index_instead_of_offset" into main
* changes:
  aconfig: update storage write api
  aconfig: update storage read api
  aconfig: update storage file creation to allow storage files to store flag index instead of byte offset
  aconfig: update storage file to store flag index instead of file byte offset
2024-04-10 18:53:43 +00:00
Daniel Zheng
e836a3a2e4 Merge "zstd negative compression support" into main 2024-04-10 17:56:45 +00:00
Daniel Zheng
d919100382 zstd negative compression support
supporting negative compression levels in ota_from_target_files in the
format of zstd,-$compression_level

Test: ota_from_target_files
Change-Id: If2ca592829f10e7a0226cb41680cb47223d17c71
2024-04-10 10:54:51 -07:00
Dennis Shen
0efe47967b aconfig: update storage write api
Now we are storing flag index instead of flag byte offset in storage
files, need to update storage write api accordingly.

Bug: b/321077378
Test: atest -c
Change-Id: I67e55287a242df0af1925ede886f8c087d343c1f
2024-04-10 16:40:06 +00:00
Dennis Shen
b18227e8d8 aconfig: update storage read api
With the update to store flag index instead of flag byte offset in
storage file. We need to update flag read api accordingly.

Bug: b/321077378
Test: atest -c
Change-Id: Iccbf939fc5b7fe0ce0f2c33f7e3aa403c268120a
2024-04-10 16:39:21 +00:00
Dennis Shen
a8ba164422 aconfig: update storage file creation to allow storage files to store
flag index instead of byte offset

Bug: b/321077378
Test: atest -c
Change-Id: I2fbfa09ac880c172fd8e47866d9ec1449718e0b6
2024-04-10 16:37:23 +00:00
Dennis Shen
702408ecbb aconfig: update storage file to store flag index instead of file byte offset
Previously, in package.map, we store the byte offset of the first flag
in a flag package in a flag value file. In flag.map, we store the within
package flag byte offset. Once we know the total offset, we can locate
the flag in the flag value file. However, this offset may not work for
flag info file. Currently we only have boolean flags. Each boolean flag
value and info takes up 1 byte in flag value and flag info file. So the
byte offsets are the same. But in the future, when we add a new flag
value type, or add more contents to flag info file. Then the byte offset
for flag value file and flag info file may not be same.

Instead, we should store index rather than offset. The package map
should store the index of the first flag in this package. The flag map
stores the within package index. Then flag value read api and flag info
read api can then infer the byte offset independently.

Bug: b/321077378
Test: atest -c
Change-Id: Iad89776d5369715ba2d8faaea18261ac3271f13b
2024-04-10 16:36:46 +00:00
Dennis Shen
9fa78d373c aconfig: return stored flag type enum as a part of flag offset info
Bug: b/321077378
Test: atest -c
Change-Id: I25bad6b226de570eedfef4b450e09be2ac75c5eb
2024-04-10 15:14:14 +00:00
Dennis Shen
ccfb4c284f Merge changes from topic "flag_info_read_api" into main
* changes:
  aconfig: add flag info read c api
  aconfig: add flag info read rust api
2024-04-10 14:33:36 +00:00
Dennis Shen
cf7f5375be Merge "aconfig: add flag info field to proto" into main 2024-04-09 13:26:57 +00:00
Treehugger Robot
cfec7d0f7b Merge changes I1a5615e9,I01a673e7 into main
* changes:
  Remove dependency on non-AB modules
  Make build_image output consistent for partitions w/o build.prop
2024-04-08 19:57:56 +00:00
Kelvin Zhang
7a17cfd81a Remove dependency on non-AB modules
imgdiff is scheduled for removal.

Test: th
Bug: 324360816
Change-Id: I1a5615e9f5f776c16269ac8780187f97edba16f2
2024-04-08 11:08:14 -07:00
Dennis Shen
58ada6f972 aconfig: add flag info read c api
Bug: b/321077378
Test: atest aconfig_storage_read_api.test.cpp
Change-Id: If6c258429b8d8ccc872cac9f72cb533cf454ecbd
2024-04-07 20:36:25 +00:00
Dennis Shen
2ac7a4c2bd aconfig: add flag info read rust api
Bug: b/321077378
Test: atest aconfig_storage_read_api.test; atest
aconfig_storage_read_api.test.rust

Change-Id: I382ac0145c5d91827952b3ddb01cabefd1539854
2024-04-07 20:34:28 +00:00
Dennis Shen
eff5363e2d aconfig: add flag info field to proto
Bug: b/321077378
Test: m
Change-Id: I1b974f83ea49e8d4b19b3392cb7feab3d4b3fcdc
2024-04-05 16:40:43 +00:00
Dennis Shen
acfb82cf73 Merge "aconfig: move create flag info file api to aconfig_storage_write_api" into main 2024-04-05 13:59:45 +00:00
Dennis Shen
455dc608f3 aconfig: move create flag info file api to aconfig_storage_write_api
rust_ffi_static right now will export excessive symbols, this lead to a
duplication of symbols when aconfig links against both
libaconfig_storage_file_cc and libaconfig_storage_read_api_cc. This is
because aconfig_storage_read_api crate depends on aconfig_storage_file
crate. To solve this problem, move create flag info file api to
aconfig_storage_write_api crate which is parallel to
aconfig_storage_read_api crate.

Bug: b/321077378
Test: atest aconfig_storage_file.test; atest
aconfig_storage_read_api.test; atest aconfig_storage_write_api.test

Change-Id: Ibbb50193a2da82d52ccbb4087c8e3fb9f320805f
2024-04-04 21:39:04 +00:00
Zhi Dou
bc71cd03b9 aconfig: remove @com.android.aconfig.annotations.AconfigFlagAccessor from the
exported mode

The exported mode library will be dropped into google3, and there isn't
the library for this annotation.

Test: atest aconfig.test
Bug: n/a
Change-Id: I318d1c0faf232b22f42587b8ea2f976173c8554a
2024-04-04 20:21:50 +00:00
Dennis Shen
83066f6dc0 Merge "aconfig: add create_flag_info cc api" into main 2024-04-04 18:26:12 +00:00
Zhi Dou
93937ac342 Merge "aconfig: prototype AconfigFlagAccessor" into main 2024-04-03 19:17:15 +00:00
Dennis Shen
cf3042db77 Merge "aconfig: add api to create flag info file based on package map and flag map file" into main 2024-04-03 18:31:08 +00:00
Dennis Shen
2922ca9019 aconfig: add create_flag_info cc api
Bug: b/321077378
Test: atest aconfig_storage_file.test
Change-Id: I1d48311cade178f0b047a5b4d4f81c6dac4362c8
2024-04-03 18:09:39 +00:00
Treehugger Robot
6b1292065f Merge "aflags: add aflags.test to presubmit runs" into main 2024-04-03 15:45:35 +00:00
Mårten Kongstad
bf1b2ba6b2 Finalization: update OWNERS
Add new folks working on finalization, remove folks no longer working on
it.

Also, sort the entires alphabetically.

Bug: 323940469
Test: N/A
Change-Id: Ib8f6c3f485a06e3d1269e4142317174ae7ac276b
2024-04-03 17:20:42 +02:00