Commit graph

93055 commits

Author SHA1 Message Date
Zhi Dou
e381a17cfc Merge "aconfig: update codegen templates" into main 2023-12-20 16:29:44 +00:00
Zhi Dou
ea48335042 aconfig: update error message in modify_parsed_flags_based_on_mode
This commit updates the error message in
modify_parsed_flags_based_on_mode, when there is no certain mode flags
in the corresponding library, and this change remove "." at the end of
the error message to conform to existing standard in this project.

Bug: 311152507
Test: atest aconfig.test
Change-Id: Ib120e7f7252981bbeb6ca1cf785da24f329507b4
2023-12-20 15:42:49 +00:00
Treehugger Robot
e1f89d9336 Merge "aconfig: add missing tests to TEST_MAPPING" into main 2023-12-20 15:35:33 +00:00
Ludovic Barman
d842f5d074 Merge "Remove unused use_protobuf3 flag" into main 2023-12-20 15:27:57 +00:00
Zhi Dou
34785f4261 Merge changes I74045709,I69b9a843 into main
* changes:
  aconfig: modify and filter flags before passing into java codegen
  aconfig: modify_parsed_flags_based_on_mode return Result
2023-12-20 14:00:05 +00:00
Mårten Kongstad
328d700b93 aconfig: remove support for --format=text
This CL is part of a larger effort to simplify maintenance by removing
format aliases: with the new --format option, clients of `aconfig dump`
can specify the format they need themselves.

This CL removes the last of the format aliases.

Bug: 317044914
Test: m
Change-Id: I09b706b9931a85a86c17be359902245b9b53d584
2023-12-20 13:41:07 +01:00
Mårten Kongstad
ce3a299f23 aconfig: remove support for --format=verbose
This CL is part of a larger effort to simplify maintenance by removing
format aliases: with the new --format option, clients of `aconfig dump`
can specify the format they need themselves.

Bug: 317044914
Test: m
Change-Id: I4db31eb3fe03c34118d61a0a1911e12f5049b77d
2023-12-20 13:39:58 +01:00
Mårten Kongstad
3d76d1afe0 aconfig: remove support for --format=bool
This CL is part of a larger effort to simplify maintenance by removing
format aliases: with the new --format option, clients of `aconfig dump`
can specify the format they need themselves.

Bug: 317044914
Test: m
Change-Id: I11efc5d7280402f503ddd05f2dc65bb15ffd8ac7
2023-12-20 13:39:58 +01:00
Mårten Kongstad
ce09eb2f31 aconfig: add missing tests to TEST_MAPPING
A few new aconfig tests have been created. Add them to TEST_MAPPING as
postsubmit tests.

Bug: N/A
Test: atest :all
Change-Id: Ifa4d764d26114367227c9c5622fb31b304a6972c
2023-12-20 10:32:50 +01:00
Håkan Kvist
b77972402e Detect downgrade by checking build time for all partitions
Checking ro.build.date.utc to detect a downgrade is not enough in the
case where several target files are merged, as timestamps can differ
for each partition.

The solution is to check the build timestamp for each partition and
ro.build.date.utc.

With this change ota generation will be aborted during creation
instead of rejected when installing on device.

This also fixes the --override_timestamp so it is possible to generate
a package where timestamps are reversed, as was added in:
3e6161a3b3.

Issue: 315913966
Test: Manual, confirm that ota generation is aborted when timestamp
  on post vendor partitions is newer than on pre.
  Ota created successfully with --override_timestamp parameter and
  installation successful
Test: atest --host releasetools_test
Change-Id: I275e67a3840f4ef2263381c253231068e72f47d2
2023-12-20 08:45:26 +01:00
Joe Onorato
0543464834 Add --dist-one flag to the benchmark script to make it copy a single set of build metrics to a dist dir.
Change-Id: Id392a33c42fed6c5014b2fd5dd69fa80fa0d9f0f
2023-12-20 05:00:04 +00:00
Joe Onorato
01277d4067 Add the rest of the CUJs to the new benchmark script.
Change-Id: Iabf8875f9d77a3d50a37f6ab1230bc9f473a15dd
2023-12-20 04:00:42 +00:00
Treehugger Robot
45e9ce5d49 Merge "aconfig: rename subcommand 'dump' -> 'dump-cache'" into main 2023-12-19 22:00:40 +00:00
Zhi Dou
533421c194 aconfig: update codegen templates
This change removes empty lines and spaces in generated code.

Bug: 311152507
Test: atest aconfig.test
Change-Id: Ib41e633323ffcce348299afa5a09bd0252db1bb9
2023-12-19 21:48:51 +00:00
Mårten Kongstad
c91ebf8153 aconfig: rename subcommand 'dump' -> 'dump-cache'
Rename the 'dump' subcommand to 'dump-cache' for better symmetry with
'create-cache', and to prepare for a potentially upcoming
'dump-storage' subcommand.

Keep 'dump' as an alias for 'dump-cache' for backwards compatibility.

Bug: N/A
Test: m
Change-Id: I521ce507985e4cf450ee1b8517993b798f11430c
2023-12-19 22:06:24 +01:00
Zhi Dou
eeed7990de aconfig: modify and filter flags before passing into java codegen
Before this change java codegen filter flags for exported mode in the
template. This change move the filter process to commands as other
codegen. Thus the codegen code will only generate code based on the
passed in flags.

Bug: 311152507
Test: atest aconfig.test aconfig.test.java AconfigJavaHostTest
Change-Id: I74045709cde19e6c687c3eb0d94050ea40cf5042
2023-12-19 20:32:59 +00:00
Zhi Dou
80040d2d2c aconfig: modify_parsed_flags_based_on_mode return Result
This commit changes modify_parsed_flags_based_on_mode to return Result
instead of vector. This change will have two consequences.
1. Error will be thrown if no flags fit into the current codegen mode
2. Error will be thrown if the flag declaration is empty

Bug: 311152507
Test: atest aconfig.test
Change-Id: I69b9a84312faed9f757bf3974b3cea49c5c5e285
2023-12-19 20:28:23 +00:00
Zhi Dou
8351b78234 Merge "aconfig: move CodegenMode to codegen crate" into main 2023-12-19 20:00:56 +00:00
Zhi Dou
dcfa0400c9 aconfig: move CodegenMode to codegen crate
This change moves CodegenMode to codegen crate.

Bug: 311152507
Test: ateset aconfig.test
Change-Id: I4f64cdbdc3d1b419412012560bfe03344b2b8faf
2023-12-19 18:59:26 +00:00
Dennis Shen
4575b273fe Merge "aconfig: package.map file serialization" into main 2023-12-19 18:57:24 +00:00
Treehugger Robot
8d1e52319e Merge "aconfig: add support for dump --filter='{fully_qualified_name}:...'" into main 2023-12-19 18:56:40 +00:00
Zhi Dou
0f1b9da4d9 Merge "aconfig: add exported mode in rust codegen" into main 2023-12-19 18:32:46 +00:00
Zhi Dou
123ec684da Merge changes Iadea7ba2,Iaedb8a68,I511e05ab into main
* changes:
  aconfig: add c/c++ codegen exported mode integration test
  aconfig: add new testing flag enabled_fixed_ro_exported
  aconfig: add exported mode in c/c++ codegen
2023-12-19 18:32:22 +00:00
Dennis Shen
adc7b732ca aconfig: package.map file serialization
For each container, there will be three storage files: package.map,
flag.map and flags.val. This change adds package.map creation.

package.map is a hash table in file that maps a package name to its
package id and package value offset. This file will be used by flag
storage client lib to locate the fixed offset of a flag in flag value
file. package.map provides the file byte offset to the start of all flag
values for this package. Together with flag value offset within its package
which will be provided by flag.map, we can locate a flag's value in the
value file.

The top level struct for this file is called "PackageTable". The struct
consists of three parts: (1) table header which includes the file
version and other metadata such as number of packages and etc. (2) table
buckets which is an array of u32, each bucket stores the package.map
file offset to the package table node. (3) package table node array.
each node stores package name, package id, package value offset and
offset to the next package table node.

The table uses fixed format serialization. All u32 are encoded using
little endian format as that is the Android format. All strings are UTF8
encoded. This is to ensure cross platform compatibility.

Bug: b/312243587
Test: atest aconfig.test
Change-Id: I1041405db42862573ec320c0e557948732c28eb8
2023-12-19 17:20:27 +00:00
Zhi Dou
e8a4ee7288 aconfig: add exported mode in rust codegen
This commit adds exported mode to rust codegen.

When the codegen mode is exported
1. only flags with exported: true will be generated
2. the generated getter should be in a read_write format and with
   default value as false, regardless the original permission and state
   of the flag

This commit adds integration test for rust codegen exported mode.

Bug: 316357104
Test: atest aconfig.test aconfig.test_mode.test.rust aconfig.exported_mode.test.rust
Change-Id: Ib7dae666e13eb8898289b06d42a4f89326e175c4
2023-12-19 16:58:06 +00:00
Zhi Dou
2909666ae4 aconfig: add c/c++ codegen exported mode integration test
This commit adds c/c++ codegen exported mode integration test

Bug: 316357680
Test: atest aconfig.test.cpp.exported_mode

Change-Id: Iadea7ba2459eb3e80c7ffc7dea83cc9c386798e5
2023-12-19 16:40:53 +00:00
Zhi Dou
b52465da31 aconfig: add new testing flag enabled_fixed_ro_exported
This commit adds a new testing flag enabled_fixed_ro_exported to test
the case of a exported and fixed_read_only flag.

Test: atest aconfig.test aconfig.test.java AconfigJavaHostTest
aconfig.test.cpp aconfig.test.cpp.test_mode aconfig.prod_mode.test.rust
Bug: 316357680

Change-Id: Iaedb8a6875166c6a6d24c7c3deee701a496b4964
2023-12-19 16:40:53 +00:00
Zhi Dou
c1d0a14b69 aconfig: add exported mode in c/c++ codegen
This commit adds exported mode to c/c++ codegen.

When the codegen mode is exported
1. only flags with exported: true will be generated
2. the generated getter should be in a read_write format and with
   default value as false, regardless the original permission and state
   of the flag

This change moves process parsed_flags funciton into codegen crate. The
process function is used to process parsed_flags based on the codegen
mode.

The template has also modified to keep readability.

Bug: 316357680
Test: atest aconfig.test aconfig.test.cpp aconfig.test.cpp.test_mode

Change-Id: I511e05ab93b07a04236055d956d1926f4ed89f36
2023-12-19 16:35:13 +00:00
Treehugger Robot
ba541c7e06 Merge "Rewrite the build system benchmarks to be much simpler and not require bazel." into main 2023-12-19 16:34:35 +00:00
Joe Onorato
88ede35828 Rewrite the build system benchmarks to be much simpler and not require bazel.
Test: ./benchmarks && ./format_benchmarks
Change-Id: I907421ed0c85e961d78342a3e58b2d4ab4aaf6ac
2023-12-19 15:39:56 +00:00
Mårten Kongstad
009a494572 aconfig: add support for dump --filter='{fully_qualified_name}:...'
Add support for filtering by fully qualified flag name. (This should
have been part of aosp/2878034.)

Bug: 315487153
Test: atest aconfig.test
Change-Id: I4cd209d6f6940dcaa138e8e0b8e580cea99c350f
2023-12-19 10:24:35 +01:00
Julien Desprez
2538c86562 Merge "Use sha256 when zipping target_files" into main 2023-12-19 03:00:22 +00:00
Julien Desprez
eebf6dcd1a Use sha256 when zipping target_files
similar to general-tests.mk, this will avoid recomputing sha256 when
uploading target files to RBE.

Test: presubmit
Bug: 316952196
Change-Id: I879214ac79f5c269f337fafcede8513c1e90e60e
2023-12-18 22:39:04 +00:00
Dennis Shen
4dc569ae49 Merge "aconfig: update c/c++ codegen" into main 2023-12-18 13:43:30 +00:00
Dennis Shen
d555bbfa80 aconfig: update c/c++ codegen
1, Current codegen would still include an empty flag value cache even if
there are no read write flags. This is important as the flag value cache
is a std::vector. Thus adding this cache would automatically pull in
libc++ dependency which we should not.

2, There lacks of c/c++ codegen unit tests to lock down the generated
code when a flag package contains only read only flags. It should not
include server_configurable_flags header. And it should not include the
flag cache.

Bug: b/316614694
Test: atest aconfig.test
Change-Id: Iee4366672932ee507694f5fc73b526c66fcf0e9b
2023-12-18 13:43:22 +00:00
Ludovic Barman
8af1ac2a90 Remove unused use_protobuf3 flag
This is a No-op.
Protobuf3 is used regardless of this flag since https://android-review.git.corp.google.com/c/platform/build/soong/+/2817733

Test: m rust

Bug: 308790516
Change-Id: Ib84a01b76a2998249a836fb0278602a5229bd242
2023-12-16 10:56:26 +00:00
Treehugger Robot
1a03031a99 Merge "Raise go.mod version of build/make/tools/rbcrun" into main 2023-12-15 22:52:32 +00:00
Mårten Kongstad
9c80abc3ec Merge changes from topic "aconfig-dump-format-and-filter-args" into main
* changes:
  aconfig: dump --filter: implement predicates
  aconfig: dump --filter: hook up command line args to dump.rs
  aconfig: support custom `dump format` specs
2023-12-15 16:35:35 +00:00
Mårten Kongstad
517ac4801b aconfig: dump --filter: implement predicates
The aconfig dump command can now limit which flags to print by passing
in one or more --filter=<query> commands.

If multiple --filter arguments are provided, flags that match any filter
will be included in the output.

The <query> syntax is <what>:<value>, where <what> is the name of a
ProtoParsedFlag field. Multiple queries can be AND-ed together by
joining them with a plus ('+') character.

Example queries:

  - --filter='is_exported:true' # only show exported flags
  - --filter='permission:READ_ONLY+state:ENABLED' # only show flags that are read-only and enabled
  - --filter='permission:READ_ONLY' --filter='state:ENABLED' # only show flags that are read-only or enabled (or both)

Current limitations that may be addressed in future CLs:

  - No support to invert a query, e.g. "flags *not* in the following
    namespace"

  - No support to quote strings; this makes description matching
    difficult

  - No support to glob strings, only exact matches are considered

  - No support to filter by description, trace or metadata fields

Bug: 315487153
Test: atest aconfig.test
Test: printflags --format="{fully_qualified_name}={state}" --filter=permission:READ_ONLY # manually verify output
Change-Id: Ie1e40fa444cec429e336048439da955f30e22979
2023-12-15 17:30:48 +01:00
Treehugger Robot
ca2175d3af Merge changes from topic "aconfig-codegen-ownership" into main
* changes:
  aconfig: pass ownership of ProtoParsedFlag items to Java codegen
  aconfig: pass ownership of ProtoParsedFlag items to C++ codegen
  aconfig: pass ownership of ProtoParsedFlag items to Rust codegen
2023-12-15 15:24:39 +00:00
Mårten Kongstad
49e4d6e73d aconfig: dump --filter: hook up command line args to dump.rs
This is the first step towards teaching dump to (optionally) filter
which flags to print.

A follow-up CL will implement dump::create_filter_predicate.

Bug: 315487153
Test: atest aconfig.test
Change-Id: Ibe0d4ce6563d3b5718fedd3ebfd45fbf5d935b92
2023-12-15 15:23:31 +01:00
Mårten Kongstad
ee58f98cc0 aconfig: support custom dump format specs
Teach `dump --format=<arg>` to format the output according to a
user-defined format string. The format string now accepts these
arguments:

  - "protobuf": output all data as binary protobuf (as before)
  - "textproto": output all data as text protobuf (as before)
  - any other string: format according to the format spec, see below

Custom format spec: placeholders, enclosed in { and } and named after
the fields of ProtoParsedFlag, will be replaced by the actual values.
All other text is output verbatim. As an example:

  - "{name}={state}" -> "enabled_ro=ENABLED"

Some fields support an alternative formatting via {<field>:<format>}. As
an example:

  - "{name}={state:bool}" -> "enabled_ro=true"

Note that the text replacement does not support escaping { and }. This
means there is no way to print the string "{name}" without expanding it
to the actual flag's name. If needed this feature can be introduced in a
later CL.

For backwards compatibility, the following format strings have special
meaning and will produce an output identically to what it was before
this change:

  - "text"
  - "verbose"
  - "bool"

A follow-up CL will add a new `dump --filter=` argument to limit which
parsed flags are included in the output.

Test: atest
Bug: b/315487153
Change-Id: If7c14b5fb3e7b41ea962425078bd04b4996318f4
2023-12-15 08:31:51 +01:00
Colin Cross
d7eac17f46 Raise go.mod version of build/make/tools/rbcrun
Raising the go.mod version to 1.17 or higher enables module graph
pruning (https://go.dev/ref/mod#graph-pruning), which prevents the
go tools like "go build ./..." from loading unused transitive
dependencies, including ones that are missing from our tree.

Bug: 314133304
Test: prebuilts/build-tools/build-prebuilts.sh
Change-Id: If1cc0fda1dc744e65fc3367b7f44b8c91230e5ce
2023-12-14 12:05:03 -08:00
Dmitrii Merkurev
ecabf893a0 Merge "Revert "Revert "Allow to override fastboot-info.txt for specific..."" into main 2023-12-14 18:57:06 +00:00
Treehugger Robot
9e4744383a Merge "Update go.mod to 1.21" into main 2023-12-14 18:46:11 +00:00
Atneya Nair
a3a6bfb4a0 aconfig: Print flags with namespace, and sort them
For readability/convenience when looking at BRs/dev.

Test: adb shell printflags
Bug: 315402442
Change-Id: Ia8c447d99f6fe0a9ed6a33c4c68f2e97f5310dc0
2023-12-14 18:02:57 +00:00
Jeongik Cha
0be07c2279 Update go.mod to 1.21
because it requires android/soong which uses go 1.21
Bug: 3141333041
Test: go test ./...
Change-Id: I184e6165beafab337b09cd87fd02748e49c65650
2023-12-14 15:37:35 +00:00
Dmitrii Merkurev
901136e9d8 Revert "Revert "Allow to override fastboot-info.txt for specific..."
Revert submission 2871792-revert-2871156-fastboot_info-UVHXWCTZZB

Reason for revert: Some images weren't available in Android.mk

Reverted changes: /q/submissionid:2871792-revert-2871156-fastboot_info-UVHXWCTZZB

Bug: 315866782
Test: confirmed bluejay abtd build has correct fastboot-info.txt

Change-Id: Id75d879119744ab97225e5a551e8d32a1b09a356
Signed-off-by: Dmitrii Merkurev <dimorinny@google.com>
2023-12-14 13:27:38 +00:00
Mårten Kongstad
0c6a2fac77 aconfig: pass ownership of ProtoParsedFlag items to Java codegen
Pass ownership of the ProtoParsedFlag iterator items to the Java
codegen: this removes the need for an explicit lifetime annotation.

Bug: N/A
Test: atest aconfig.test
Change-Id: I78705b65b4cf41c3e23b59afbd122a029ee45df4
2023-12-14 13:56:25 +01:00
Mårten Kongstad
49c49f365a aconfig: pass ownership of ProtoParsedFlag items to C++ codegen
Pass ownership of the ProtoParsedFlag iterator items to the C++ codegen:
this removes the need for an explicit lifetime annotation.

Bug: N/A
Test: atest aconfig.test
Change-Id: Ic6606a9ab01ddcb61aa668d7ac901469b1e25a1f
2023-12-14 13:53:52 +01:00