Commit graph

8191 commits

Author SHA1 Message Date
Treehugger Robot
57c1aa7b16 Merge "Handle zip64 extra fields better" 2023-06-17 22:12:26 +00:00
Kelvin Zhang
1e774245a4 Handle zip64 extra fields better
Test: check_target_files_signatures
Bug: 283033491
Change-Id: I7da89f8389c09cc99201cff342483c158bd7e9c1
2023-06-17 09:21:46 -07:00
Treehugger Robot
0d2d11bc00 Merge "Use patch instead of git apply." 2023-06-17 01:56:52 +00:00
Alex Buynytskyy
bce1a51df8 Use patch instead of git apply.
Bug: 281682520
Test: run locally
Change-Id: Ia84055b7a9074836c6d7b3039ba980e39043a41e
2023-06-16 20:23:36 +00:00
Zhi Dou
af81e20653 aconfig: change java flag method name to camelCase
Before java code will directly use the flag name as the method name.
This change adds funciton to try the best to convert flag name to
camelCase, and then use the camelCase string as the method name in the
generated code.

Bug: 279483816
Test: atest aconfig.test aconfig.test.java
Change-Id: I45fc6df46c9d535cd38a657a41313202f9b660af
2023-06-16 12:40:40 +00:00
Jooyung Han
e4635b3e04 Merge "Remove --blkid_path argument" 2023-06-16 06:48:57 +00:00
Kelvin Zhang
38d0c373ac Fix python3.11's support for zip64
Bug: 283033491
Test: check_target_files_signatures -v -l
Change-Id: I9c1a5346e3a5f3920242dc9a5268d999f50a4937
2023-06-14 12:54:57 -07:00
Kelvin Zhang
b789e84499 Search for partition maps in IMAGES dir as well
Partition images are allowed to be in either IMAGES/ or RADIO/ dir of a
target_files zip, so when searching for .map files we should look in
both dirs.

Test: th
Bug: 227848550
Change-Id: I0a9d2c582d8f5d570237434902fac012513c9aad
2023-06-14 10:01:21 -07:00
Zhi Dou
06377d79ab Merge "aconfig: Java codegen iteration 1" 2023-06-14 13:21:16 +00:00
Jooyung Han
629490292a Remove --blkid_path argument
deapexer doesn't need it.

Bug: 279858383
Test: presubmit
Change-Id: If0ec42b5edd4642f07c96ba641030c4dd6fb4660
2023-06-14 15:16:34 +09:00
Zhi Dou
4655c967e1 aconfig: Java codegen iteration 1
This change includes
    - refactor generated java code to generate
        - Flags.java to support the static API
	- FeatureFlagsImpl.java to support injection API
	- FeatureFlags.java interface

Bug: 279483816
Test: atest aconfig.test aconfig.test.java
Change-Id: If0d4baf317b9174635cd0fff3832ab7091ee52ed
2023-06-13 14:45:53 +00:00
Mårten Kongstad
d18c978f31 aconfig: reject consecutive underscores in identifiers
The Java codegen may translate flag names (snake_case) to Java
camelCase, dropping the underscores. The flags a_b and a__b will
translate to the same camelCase form, which is ambiguous.

Circumvent this problem by disallowing consecutive underscores in flag
names, flag namespaces, and packages.

Bug: 284252015
Test: atest aconfig.test
Change-Id: I2586a38160723c06265a140193da8178655553e4
2023-06-13 13:30:58 +02:00
Dennis Shen
f1cd83bece Merge "aconfig: update cpp codegen to use static methods" 2023-06-12 12:57:57 +00:00
Dennis Shen
4f78f10015 aconfig: update cpp codegen to use static methods
Update c++ codegen to static methods interface.

Bug: b/279483801
Test: atest aconfig.test
Change-Id: I78da3bbca6240bee660c692807930d00f2242b0a
2023-06-12 12:57:38 +00:00
Joe Onorato
593bef7150 Merge "Define release flags in starlark instead of make" 2023-06-10 03:21:22 +00:00
Joe Onorato
ed1e3b50f3 Merge "Fix build breakage" 2023-06-09 20:57:36 +00:00
Joe Onorato
8b51859317 Fix build breakage
error: build/make/tools/aconfig/Android.bp:69:1: module "aconfig.test.java" variant "android_common": path dependency ":aconfig.test.flags{.srcjar}": unsupported device_config_definitions module reference tag ".srcjar"

Test: m aconfig.test.java
Change-Id: I3df980b06796c90dafee3940fe52809748305950
2023-06-09 13:39:41 -07:00
Cole Faust
386b3746ce Define release flags in starlark instead of make
So that we have a more restricted enviornment for this new configuration
axis that can also be imported into other tools more easily.

Test: Manually (this time also tested setting OUT_DIR outside of the tree)
Change-Id: I01d90e06e45cba756156af16f63e04f575877263
2023-06-09 13:10:05 -07:00
Treehugger Robot
df2af7c840 Merge "Supply apex_info.pb as parameter to delta_generator if file exists" 2023-06-09 19:23:31 +00:00
Håkan Kvist
ddb968dce0 Supply apex_info.pb as parameter to delta_generator if file exists
Supply generator with parameter --apex_info_file=META/apex_info.pb
if the file exists.

This ensures that apex_info is included in payload header.

This is identical to the behaviour of brillo_update_payload
which is not being used since:
Invoke delta_generator directly
fcd731e3d6

Issue: 286253576
Test: Manual, confirm that apex_info is included in payload header
Change-Id: Ic096c5f8966beec8686f918aba462c955290a6c5
2023-06-09 12:46:58 +02:00
Mårten Kongstad
9c59c31499 aconfig: add Java integration tests
Add integration tests for Java. This test setup verifies that

  - the build system calls aconfig to generate a Java library
  - the Java test compiles against the auto-generated library
  - the auto-generated code returns expected values

Similar integration tests for C++ and Rust will be added in follow-up
CLs.

Note: the build does not currently support specifying that
tests/*.values should be applied, so the test flags will all be assigned
the defaults. A later CL will fix this.

Bug: b/283911467
Test: atest aconfig.test aconfig.test.java
Change-Id: Ia365e209261f4935a23e2dac9ef0ab5b60f76e52
2023-06-09 09:59:21 +02:00
Treehugger Robot
d60dbf9773 Merge "fixup! Allow ota_from_target_file to work entirely on directories" 2023-06-08 22:20:30 +00:00
Mårten Kongstad
ec5da68dbd Merge changes from topic "aconfig-rename-namespace-to-package"
* changes:
  aconfig: include namespace in create-device-config-defaults
  aconfig: improve code diffs in tests
  aconfig: add namespace field to flag_declaration and parsed_flag
  aconfig: allow dots in package fields
  aconfig: rename namespace -> package
2023-06-08 15:12:10 +00:00
Håkan Kvist
b8982fc538 fixup! Allow ota_from_target_file to work entirely on directories
The following log message would always be shown:
WARNING : Cannot find care map file in target_file package

Break out of the care map copying loop as soon a file has been
copied. This ensures that else statement is only executed if no
care map file exists.

Test: Manual. Run ota_from_target_files with target-zip with and
    without care map files.
Change-Id: Ia196aa182ed81f21424317a7005f5634866b4b99
2023-06-08 15:17:57 +02:00
Mårten Kongstad
202102f7be aconfig: include namespace in create-device-config-defaults
Update the output format of create-device-config-defaults to include the
flag's namespace. Also change the delimiters. The new format is

  <namespace>:<package>.<flag-name>=[enabled|disabled|

Bug: 285468565
Test: atest aconfig.test
Change-Id: I9b4ca1611cca8528dc341fc12812b614c86f6c08
2023-06-08 11:28:09 +02:00
Mårten Kongstad
b025507857 aconfig: improve code diffs in tests
Implement a helper function to make it easier for unit tests to diff
(and find the first difference) generated code and expected code.

Bug: 283910447
Test: atest aconfig.test
Change-Id: I460e8fbf05e8f33e8a62ecef67b2d9d77051e876
2023-06-08 11:27:59 +02:00
Mårten Kongstad
066575b95f aconfig: add namespace field to flag_declaration and parsed_flag
Add a new field to the proto messages flag_declaration and parsed_flag.

The new field will be used verbatim as a parameter when calling
DeviceConfig.getBoolean to read the value of a READ_WRITE flag. See the
DeviceConfig API for more info.

Note: not to be confused with the old namespace field, which has been
renamed to package.

Bug: 285211724
Test: atest aconfig.test
Change-Id: I2181be7b5e98fc334e5277fb5f7e386f1fe0b550
2023-06-08 11:27:57 +02:00
Mårten Kongstad
fbd71e2773 aconfig: allow dots in package fields
Allow package fields to include dots.

Update the generated code based on the package name: if the package name
is com.android.example:

  - java: package com.android.example; ...
  - C++: namespace com::android::example { ... }
  - Rust: mod com { mod android { mod example { ... } } }

Also, update examples to use dots in the package fields.

Also, remove unnecessary #include from the auto-generated C++ code: the
header should not include itself.

Bug: 285000854
Test: atest aconfig.test
Change-Id: I8a5352e25c64c34dee0725202a1b7c9957819de8
2023-06-08 11:25:43 +02:00
Devin Moore
9bf17fd9dd Merge "Add new script to analyze static/shared library usage" 2023-06-07 20:18:09 +00:00
Jingwen Chen
376b6b6378 Resolve test crash in build/make/tools/compliance.
Test: m compliance_sbom
Test: go test ./...
Fixes: 285157766
BUG: 282746032
Change-Id: I5cc39f45f7ac3d9c574e2ebaa7fdfda864cb109d
2023-06-07 13:06:13 +00:00
Mårten Kongstad
9fb58965af aconfig: rename namespace -> package
What used to be referred to as a namespace is now called a package.

This CL is a semantic change only.

Bug: 285000854
Test: m nothing
Test: atest aconfig.test
Change-Id: If3fca67c415af75b44f316e16666b97089407069
2023-06-07 14:53:04 +02:00
Treehugger Robot
2bb714ed2b Merge changes from topic "aconfig-gen-device-config-files"
* changes:
  aconfig: add create-device-config-sysprops command
  aconfig: add create-device-config-defaults command
  aconfig: add test utilities
  aconfig: cache.rs: remove unnecessary use statements
  aconfig: give commands ownership of all arguments
2023-06-07 09:50:23 +00:00
Kelvin Zhang
7c9205b008 Use python based unzip function for portability
Bug: 283033491
Test: th
Change-Id: Ief86b55c1d4a14220a0fb593c583a721d59cf86c
2023-06-05 15:54:12 -07:00
Devin Moore
bd13e63e99 Add new script to analyze static/shared library usage
Parses module-info.json, gathers stats on how many times each library is
included shared or statically.

Can print a list of libraries that would be a candidate for changing
from static to shared or visa versa.

Test: m
Bug: 280829178
Change-Id: I4bbffbd673ab2e08c69d0ab6e68402be77c9ffbc
2023-06-05 20:23:36 +00:00
Kelvin Zhang
c819b29f46 Always set a avb salt for hermetic build
When building images via `m` , build_image.py is invoked directly
without going through add_img_to_target_files. To ensure images built in
either way are identical, move uuid/salt computation to build_image.py,
so that the same uuid/salt will be used.

Bug: 281960439
Test: m installclean && m && m target-files-dir , maks sure images in
$OUT and $OUT/obj/PACKING/target_files_intermediates are identical

Change-Id: Icdab29df84f5a0ec7c080f99f9fdbdc3c9b10b90
2023-06-05 09:59:54 -07:00
Treehugger Robot
ecf635f507 Merge "Adding option to disable fastboot_info" 2023-06-03 01:56:41 +00:00
Dennis Shen
3b0ca61d36 Merge "aconfig: update rust code gen to use libflags_rust" 2023-06-02 15:13:40 +00:00
Mårten Kongstad
c31a6ff653 aconfig: add create-device-config-sysprops command
Add a new "create-device-config-sysprops" command that works like
"create-device-config-defaults" but for system properties.

DeviceConfig is a Java service, and will mirror (some of) its values by
setting system properties in the persist.device_config namespace. Native
code will access DeviceConfig (actually, the system properties) via the
server_configurable_flags library.

The new command writes a file that can be appended to /system/build.prop
to pre-populate persist.device_config before DeviceConfig has started.

Like create-device-config-defaults, the new command skips READ_ONLY
flags.

Bug: 285468565
Test: atest aconfig.test
Change-Id: I311c7c5e0b03dc897b09204137d43cc182324717
2023-06-02 16:47:04 +02:00
Mårten Kongstad
f02734e915 aconfig: add create-device-config-defaults command
DeviceConfig is the backend for READ_WRITE flags.

Add a new command "create-device-config-defaults" to create a file that
DeviceConfig will read to pre-populate its data store on first init.

This will be used to quickly switch flag values during CI tests:
rebuilding and reflashing a device would have the same effect, but would
be costlier. This feature is not expected to be used outside CI tests.

Note: because DeviceConfig only works with READ_WRITE flags, the
generated file excludes READ_ONLY flags.

Bug: 285468565
Test: atest aconfig.test
Change-Id: I4caff1a10647b8da0ce4e3615678993a957a92dd
2023-06-02 16:46:28 +02:00
Mårten Kongstad
83a8760bbc aconfig: add test utilities
Create a test utility function to create a Cache from the files in
testdata/*. A follow-up CL will update the unit tests to use this
instead of creating their own caches.

Bug: 283910447
Test: atest aconfig.test
Change-Id: Ice5064eadb0babde5eb38d292330d213ab136d96
2023-06-02 12:54:22 +02:00
Mårten Kongstad
1cd166cd31 aconfig: cache.rs: remove unnecessary use statements
Remove unnecessary use from the cache::test module: they already covered
by `use super:*;`.

Bug: 283910447
Test: atest aconfig.test
Change-Id: I9e03385629f38180c0f92080c7f097a8e0d9ef69
2023-06-02 12:52:48 +02:00
Mårten Kongstad
b27f2ce436 aconfig: give commands ownership of all arguments
Pass the Cache argument to command::create_<lang>_lib functions by value
instead of by reference, to align with other commands.

The intended ownership flow is as follows:

  - main creates objects based on command line arguments
  - main hands commands ownership of the objects
  - command processes the objects
  - command gives main ownership of any generated output
  - main writes the output to file

Rationale: commands.rs is a unit testable version of main, and to the
rest of aconfig, acts as the top level entry point; main.rs exists only
to parse command line arguments and perform I/O.

Bug: 283910447
Test: atest aconfig.test
Change-Id: I1e1dea7da8ecc2bb6e2f7ee4a6df64562c148959
2023-06-02 12:49:26 +02:00
Daniel Zheng
ee10d07057 Adding option to disable fastboot_info
Fastboot_info can be disabled if use_fastboot_info is set to false.
Adding this flag as fastboot-info.txt is currently broken

Test: m updatepackage -> inspect contents
Bug: 284263071
Change-Id: I3e0ca13968ba9747cc39284ea6798981d22ad5e5
2023-06-01 14:21:58 -07:00
Kelvin Zhang
89b87f6aa9 Open ab_partitions.txt with just read access
We don't actually need write permission, so going with least privilege
principle. We have observed some mysterious permission denied errors on server environments. Without detailed logs or access to the server it's hard to pinpoint what the root cause is. This is an attempt/hypothesis to fix the permission denied error.

Test: th
Bug: 283033491
Change-Id: I52dc360d593aab57c749109994bf3e1e3625d0ce
2023-06-01 17:28:25 +00:00
Dennis Shen
6461673140 aconfig: update rust code gen to use libflags_rust
Bug: b/284096062, b/279483360
Test: atest aconfig.test
Change-Id: I4c96f35807de5cb2a745ba1c653513d9b22a5013
2023-05-31 14:35:59 +00:00
Mårten Kongstad
00cf045c85 aconfig: restrict valid namespace and flag names
The namespace and flag names will be used as identifiers in the
auto-generated code. Place restrictions on what constitutes a valid
name.

Valid identifiers are those that match /[a-z][a-z0-9_]/. aconfig
explicitly does not implement any automatic translation to make names
valid identifiers: this sidesteps potential conflicts such as "foo.bar"
and "foo_bar" mapping to the same name if dots were translated to
underscores.

Bug: b/284252015
Test: atest aconfig.test
Change-Id: I38d005a74311e5829e540063404d1565071e6e96
2023-05-26 17:09:10 +02:00
Mårten Kongstad
aa977fb083 Merge changes from topic "aconfig-create-rust-lib"
* changes:
  aconfig: first iteration of Rust codegen
  aconfig: improve dump --format=debug output
2023-05-26 06:00:47 +00:00
Mårten Kongstad
f73b963283 aconfig: first iteration of Rust codegen
Add a new `create-rust-lib` command to generate Rust code. The output is
a src/lib.rs file; the build system is assumed to set the generated
crate's name.

For READ_ONLY flags, the generated code returns a hard-coded true or false.

For READ_WRITE flags, the generated code reaches out to DeviceConfig via
the cc_library server_configurable_flags via the
libprofcollect_libflags_rust Rust bindings. The build system is assumed
to add this to the generated crate's dependencies.

Note: libprofcollect_libflags_rust seems generic enough that it should
be moved to an official Rust wrapper for server_configurable_flags. This
is tracked in b/284096062.

Summary of module the built system is assumed to wrap the auto-generated
code in:

  rust_library {
      name: "lib<namespace>_rs",
      crate_name: "<namespace>_rs",
      edition: "2021",
      clippy_lints: "none",
      no_stdlibs: true,
      lints: "none",
      srcs: ["src/lib.rs"],
      rustlibs: [
          "libprofcollect_libflags_rust",
      ],
  }

Also add a set of test input to be used in the unit tests for a more
coherent test strategy. A follow-up CL will migrate the code in
commands.rs, codegen_java.rs and codegen_cpp.rs.

Bug: 279483360
Bug: 283907905
Test: atest aconfig.test
Test: manual: create cache from files in testdata, create rust lib, add to module template above, verify the module builds
Change-Id: I02606aa3686eda921116e33f7e2df8fd1156a7aa
2023-05-25 16:18:59 +02:00
Treehugger Robot
b3adc24301 Merge "Fix merge_target_files in GRF case" 2023-05-25 10:46:32 +00:00
Mårten Kongstad
993111f91c aconfig: improve dump --format=debug output
Use Rust's {:#?} formatter for more readable output.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I127f413e3d7aebfba96cad1dd58d9e261dd613a4
2023-05-24 14:56:45 +02:00