Commit graph

93014 commits

Author SHA1 Message Date
Mårten Kongstad
c761847832 aconfig: call Vec::retain instead of Vec::retain_mut
Replace a call to Vec::retain_mut with a call to Vec::retain instead:
the predicate used doesn't require its parameter to be a *mutable*
reference.

Bug: N/A
Test: atest aconfig.test
Change-Id: Iacd5f2ebef358ebc92ae6a1a78dd3045e44196f4
2023-12-21 10:28:34 +01:00
Mårten Kongstad
0b2a2a8c19 aconfig: adjust integration tests to correctly set flag values
With the introduction of a root release config, the
com.android.aconfig.test flags now have their values correctly assigned.
Update the integration tests accordingly.

Bug: 317178122
Test: atest :all
Merged-In: I1acdf9442a465f20f71bcda9b15eb69058aa4b3a
Change-Id: I1acdf9442a465f20f71bcda9b15eb69058aa4b3a
2023-12-21 09:00:48 +01:00
Mårten Kongstad
85c9c5b81a Merge changes from topic "aconfig-remove-format-aliases" into main
* changes:
  aconfig: remove support for --format=text
  aconfig: remove support for --format=verbose
  aconfig: remove support for --format=bool
2023-12-21 07:03:38 +00:00
Treehugger Robot
fb5630b393 Merge "Detect downgrade by checking build time for all partitions" into main 2023-12-21 05:14:54 +00:00
Dennis Shen
6376ecee75 Merge "aconfig: add whole table serialization test" into main 2023-12-21 00:45:35 +00:00
Dennis Shen
e6330986d0 aconfig: add whole table serialization test
1, Add whole table serialization test to also lock down how the buckets
are serialized. Before this change, we are only locking down the header
and nodes serialization.

2, Switch over to the absolute offset in the file instead of relative
offset. Before we are using relative offset. For example, if a bucket is
Some(10), it means that it is pointing to 10 bytes over the start of the
node region of the table. Now it will be the absolute byte offset with
respect to the start of the table.

Bug: b/312243587
Test: atest aconfig.test
Change-Id: If7abc8c6b6687c0bc0c40bbfc6afbe0e46ece770
2023-12-20 19:46:03 +00:00
Dennis Shen
bd635ce666 Merge "aconfig: add additional test aconfig to test chaining" into main 2023-12-20 19:45:00 +00:00
Dennis Shen
8bab859f84 aconfig: add additional test aconfig to test chaining
Add an additional package com.android.acondfig.storage.test_4 to test
so that the chaining in package.map can also be tested.

Bug: b/312243587
Test: atest aconfig.test
Change-Id: I8d0fad8941a20c7f6ed60caf529667549abdcdd7
2023-12-20 17:45:00 +00:00
Treehugger Robot
e1cd8461a3 Merge changes Id392a33c,Iabf8875f into main
* changes:
  Add --dist-one flag to the benchmark script to make it copy a single set of build metrics to a dist dir.
  Add the rest of the CUJs to the new benchmark script.
2023-12-20 17:20:13 +00:00
Zhi Dou
9cd08c66a4 Merge "aconfig: update error message in modify_parsed_flags_based_on_mode" into main 2023-12-20 16:46:32 +00:00
Zhi Dou
e381a17cfc Merge "aconfig: update codegen templates" into main 2023-12-20 16:29:44 +00:00
Zhi Dou
ea48335042 aconfig: update error message in modify_parsed_flags_based_on_mode
This commit updates the error message in
modify_parsed_flags_based_on_mode, when there is no certain mode flags
in the corresponding library, and this change remove "." at the end of
the error message to conform to existing standard in this project.

Bug: 311152507
Test: atest aconfig.test
Change-Id: Ib120e7f7252981bbeb6ca1cf785da24f329507b4
2023-12-20 15:42:49 +00:00
Treehugger Robot
e1f89d9336 Merge "aconfig: add missing tests to TEST_MAPPING" into main 2023-12-20 15:35:33 +00:00
Ludovic Barman
d842f5d074 Merge "Remove unused use_protobuf3 flag" into main 2023-12-20 15:27:57 +00:00
Zhi Dou
34785f4261 Merge changes I74045709,I69b9a843 into main
* changes:
  aconfig: modify and filter flags before passing into java codegen
  aconfig: modify_parsed_flags_based_on_mode return Result
2023-12-20 14:00:05 +00:00
Mårten Kongstad
328d700b93 aconfig: remove support for --format=text
This CL is part of a larger effort to simplify maintenance by removing
format aliases: with the new --format option, clients of `aconfig dump`
can specify the format they need themselves.

This CL removes the last of the format aliases.

Bug: 317044914
Test: m
Change-Id: I09b706b9931a85a86c17be359902245b9b53d584
2023-12-20 13:41:07 +01:00
Mårten Kongstad
ce3a299f23 aconfig: remove support for --format=verbose
This CL is part of a larger effort to simplify maintenance by removing
format aliases: with the new --format option, clients of `aconfig dump`
can specify the format they need themselves.

Bug: 317044914
Test: m
Change-Id: I4db31eb3fe03c34118d61a0a1911e12f5049b77d
2023-12-20 13:39:58 +01:00
Mårten Kongstad
3d76d1afe0 aconfig: remove support for --format=bool
This CL is part of a larger effort to simplify maintenance by removing
format aliases: with the new --format option, clients of `aconfig dump`
can specify the format they need themselves.

Bug: 317044914
Test: m
Change-Id: I11efc5d7280402f503ddd05f2dc65bb15ffd8ac7
2023-12-20 13:39:58 +01:00
Mårten Kongstad
ce09eb2f31 aconfig: add missing tests to TEST_MAPPING
A few new aconfig tests have been created. Add them to TEST_MAPPING as
postsubmit tests.

Bug: N/A
Test: atest :all
Change-Id: Ifa4d764d26114367227c9c5622fb31b304a6972c
2023-12-20 10:32:50 +01:00
Håkan Kvist
b77972402e Detect downgrade by checking build time for all partitions
Checking ro.build.date.utc to detect a downgrade is not enough in the
case where several target files are merged, as timestamps can differ
for each partition.

The solution is to check the build timestamp for each partition and
ro.build.date.utc.

With this change ota generation will be aborted during creation
instead of rejected when installing on device.

This also fixes the --override_timestamp so it is possible to generate
a package where timestamps are reversed, as was added in:
3e6161a3b3.

Issue: 315913966
Test: Manual, confirm that ota generation is aborted when timestamp
  on post vendor partitions is newer than on pre.
  Ota created successfully with --override_timestamp parameter and
  installation successful
Test: atest --host releasetools_test
Change-Id: I275e67a3840f4ef2263381c253231068e72f47d2
2023-12-20 08:45:26 +01:00
Joe Onorato
0543464834 Add --dist-one flag to the benchmark script to make it copy a single set of build metrics to a dist dir.
Change-Id: Id392a33c42fed6c5014b2fd5dd69fa80fa0d9f0f
2023-12-20 05:00:04 +00:00
Joe Onorato
01277d4067 Add the rest of the CUJs to the new benchmark script.
Change-Id: Iabf8875f9d77a3d50a37f6ab1230bc9f473a15dd
2023-12-20 04:00:42 +00:00
Treehugger Robot
45e9ce5d49 Merge "aconfig: rename subcommand 'dump' -> 'dump-cache'" into main 2023-12-19 22:00:40 +00:00
Zhi Dou
533421c194 aconfig: update codegen templates
This change removes empty lines and spaces in generated code.

Bug: 311152507
Test: atest aconfig.test
Change-Id: Ib41e633323ffcce348299afa5a09bd0252db1bb9
2023-12-19 21:48:51 +00:00
Mårten Kongstad
c91ebf8153 aconfig: rename subcommand 'dump' -> 'dump-cache'
Rename the 'dump' subcommand to 'dump-cache' for better symmetry with
'create-cache', and to prepare for a potentially upcoming
'dump-storage' subcommand.

Keep 'dump' as an alias for 'dump-cache' for backwards compatibility.

Bug: N/A
Test: m
Change-Id: I521ce507985e4cf450ee1b8517993b798f11430c
2023-12-19 22:06:24 +01:00
Zhi Dou
eeed7990de aconfig: modify and filter flags before passing into java codegen
Before this change java codegen filter flags for exported mode in the
template. This change move the filter process to commands as other
codegen. Thus the codegen code will only generate code based on the
passed in flags.

Bug: 311152507
Test: atest aconfig.test aconfig.test.java AconfigJavaHostTest
Change-Id: I74045709cde19e6c687c3eb0d94050ea40cf5042
2023-12-19 20:32:59 +00:00
Zhi Dou
80040d2d2c aconfig: modify_parsed_flags_based_on_mode return Result
This commit changes modify_parsed_flags_based_on_mode to return Result
instead of vector. This change will have two consequences.
1. Error will be thrown if no flags fit into the current codegen mode
2. Error will be thrown if the flag declaration is empty

Bug: 311152507
Test: atest aconfig.test
Change-Id: I69b9a84312faed9f757bf3974b3cea49c5c5e285
2023-12-19 20:28:23 +00:00
Zhi Dou
8351b78234 Merge "aconfig: move CodegenMode to codegen crate" into main 2023-12-19 20:00:56 +00:00
Zhi Dou
dcfa0400c9 aconfig: move CodegenMode to codegen crate
This change moves CodegenMode to codegen crate.

Bug: 311152507
Test: ateset aconfig.test
Change-Id: I4f64cdbdc3d1b419412012560bfe03344b2b8faf
2023-12-19 18:59:26 +00:00
Dennis Shen
4575b273fe Merge "aconfig: package.map file serialization" into main 2023-12-19 18:57:24 +00:00
Treehugger Robot
8d1e52319e Merge "aconfig: add support for dump --filter='{fully_qualified_name}:...'" into main 2023-12-19 18:56:40 +00:00
Zhi Dou
0f1b9da4d9 Merge "aconfig: add exported mode in rust codegen" into main 2023-12-19 18:32:46 +00:00
Zhi Dou
123ec684da Merge changes Iadea7ba2,Iaedb8a68,I511e05ab into main
* changes:
  aconfig: add c/c++ codegen exported mode integration test
  aconfig: add new testing flag enabled_fixed_ro_exported
  aconfig: add exported mode in c/c++ codegen
2023-12-19 18:32:22 +00:00
Dennis Shen
adc7b732ca aconfig: package.map file serialization
For each container, there will be three storage files: package.map,
flag.map and flags.val. This change adds package.map creation.

package.map is a hash table in file that maps a package name to its
package id and package value offset. This file will be used by flag
storage client lib to locate the fixed offset of a flag in flag value
file. package.map provides the file byte offset to the start of all flag
values for this package. Together with flag value offset within its package
which will be provided by flag.map, we can locate a flag's value in the
value file.

The top level struct for this file is called "PackageTable". The struct
consists of three parts: (1) table header which includes the file
version and other metadata such as number of packages and etc. (2) table
buckets which is an array of u32, each bucket stores the package.map
file offset to the package table node. (3) package table node array.
each node stores package name, package id, package value offset and
offset to the next package table node.

The table uses fixed format serialization. All u32 are encoded using
little endian format as that is the Android format. All strings are UTF8
encoded. This is to ensure cross platform compatibility.

Bug: b/312243587
Test: atest aconfig.test
Change-Id: I1041405db42862573ec320c0e557948732c28eb8
2023-12-19 17:20:27 +00:00
Zhi Dou
e8a4ee7288 aconfig: add exported mode in rust codegen
This commit adds exported mode to rust codegen.

When the codegen mode is exported
1. only flags with exported: true will be generated
2. the generated getter should be in a read_write format and with
   default value as false, regardless the original permission and state
   of the flag

This commit adds integration test for rust codegen exported mode.

Bug: 316357104
Test: atest aconfig.test aconfig.test_mode.test.rust aconfig.exported_mode.test.rust
Change-Id: Ib7dae666e13eb8898289b06d42a4f89326e175c4
2023-12-19 16:58:06 +00:00
Zhi Dou
2909666ae4 aconfig: add c/c++ codegen exported mode integration test
This commit adds c/c++ codegen exported mode integration test

Bug: 316357680
Test: atest aconfig.test.cpp.exported_mode

Change-Id: Iadea7ba2459eb3e80c7ffc7dea83cc9c386798e5
2023-12-19 16:40:53 +00:00
Zhi Dou
b52465da31 aconfig: add new testing flag enabled_fixed_ro_exported
This commit adds a new testing flag enabled_fixed_ro_exported to test
the case of a exported and fixed_read_only flag.

Test: atest aconfig.test aconfig.test.java AconfigJavaHostTest
aconfig.test.cpp aconfig.test.cpp.test_mode aconfig.prod_mode.test.rust
Bug: 316357680

Change-Id: Iaedb8a6875166c6a6d24c7c3deee701a496b4964
2023-12-19 16:40:53 +00:00
Zhi Dou
c1d0a14b69 aconfig: add exported mode in c/c++ codegen
This commit adds exported mode to c/c++ codegen.

When the codegen mode is exported
1. only flags with exported: true will be generated
2. the generated getter should be in a read_write format and with
   default value as false, regardless the original permission and state
   of the flag

This change moves process parsed_flags funciton into codegen crate. The
process function is used to process parsed_flags based on the codegen
mode.

The template has also modified to keep readability.

Bug: 316357680
Test: atest aconfig.test aconfig.test.cpp aconfig.test.cpp.test_mode

Change-Id: I511e05ab93b07a04236055d956d1926f4ed89f36
2023-12-19 16:35:13 +00:00
Treehugger Robot
ba541c7e06 Merge "Rewrite the build system benchmarks to be much simpler and not require bazel." into main 2023-12-19 16:34:35 +00:00
Joe Onorato
88ede35828 Rewrite the build system benchmarks to be much simpler and not require bazel.
Test: ./benchmarks && ./format_benchmarks
Change-Id: I907421ed0c85e961d78342a3e58b2d4ab4aaf6ac
2023-12-19 15:39:56 +00:00
Mårten Kongstad
009a494572 aconfig: add support for dump --filter='{fully_qualified_name}:...'
Add support for filtering by fully qualified flag name. (This should
have been part of aosp/2878034.)

Bug: 315487153
Test: atest aconfig.test
Change-Id: I4cd209d6f6940dcaa138e8e0b8e580cea99c350f
2023-12-19 10:24:35 +01:00
Julien Desprez
2538c86562 Merge "Use sha256 when zipping target_files" into main 2023-12-19 03:00:22 +00:00
Julien Desprez
eebf6dcd1a Use sha256 when zipping target_files
similar to general-tests.mk, this will avoid recomputing sha256 when
uploading target files to RBE.

Test: presubmit
Bug: 316952196
Change-Id: I879214ac79f5c269f337fafcede8513c1e90e60e
2023-12-18 22:39:04 +00:00
Dennis Shen
4dc569ae49 Merge "aconfig: update c/c++ codegen" into main 2023-12-18 13:43:30 +00:00
Dennis Shen
d555bbfa80 aconfig: update c/c++ codegen
1, Current codegen would still include an empty flag value cache even if
there are no read write flags. This is important as the flag value cache
is a std::vector. Thus adding this cache would automatically pull in
libc++ dependency which we should not.

2, There lacks of c/c++ codegen unit tests to lock down the generated
code when a flag package contains only read only flags. It should not
include server_configurable_flags header. And it should not include the
flag cache.

Bug: b/316614694
Test: atest aconfig.test
Change-Id: Iee4366672932ee507694f5fc73b526c66fcf0e9b
2023-12-18 13:43:22 +00:00
Ludovic Barman
8af1ac2a90 Remove unused use_protobuf3 flag
This is a No-op.
Protobuf3 is used regardless of this flag since https://android-review.git.corp.google.com/c/platform/build/soong/+/2817733

Test: m rust

Bug: 308790516
Change-Id: Ib84a01b76a2998249a836fb0278602a5229bd242
2023-12-16 10:56:26 +00:00
Treehugger Robot
1a03031a99 Merge "Raise go.mod version of build/make/tools/rbcrun" into main 2023-12-15 22:52:32 +00:00
Mårten Kongstad
9c80abc3ec Merge changes from topic "aconfig-dump-format-and-filter-args" into main
* changes:
  aconfig: dump --filter: implement predicates
  aconfig: dump --filter: hook up command line args to dump.rs
  aconfig: support custom `dump format` specs
2023-12-15 16:35:35 +00:00
Mårten Kongstad
517ac4801b aconfig: dump --filter: implement predicates
The aconfig dump command can now limit which flags to print by passing
in one or more --filter=<query> commands.

If multiple --filter arguments are provided, flags that match any filter
will be included in the output.

The <query> syntax is <what>:<value>, where <what> is the name of a
ProtoParsedFlag field. Multiple queries can be AND-ed together by
joining them with a plus ('+') character.

Example queries:

  - --filter='is_exported:true' # only show exported flags
  - --filter='permission:READ_ONLY+state:ENABLED' # only show flags that are read-only and enabled
  - --filter='permission:READ_ONLY' --filter='state:ENABLED' # only show flags that are read-only or enabled (or both)

Current limitations that may be addressed in future CLs:

  - No support to invert a query, e.g. "flags *not* in the following
    namespace"

  - No support to quote strings; this makes description matching
    difficult

  - No support to glob strings, only exact matches are considered

  - No support to filter by description, trace or metadata fields

Bug: 315487153
Test: atest aconfig.test
Test: printflags --format="{fully_qualified_name}={state}" --filter=permission:READ_ONLY # manually verify output
Change-Id: Ie1e40fa444cec429e336048439da955f30e22979
2023-12-15 17:30:48 +01:00
Treehugger Robot
ca2175d3af Merge changes from topic "aconfig-codegen-ownership" into main
* changes:
  aconfig: pass ownership of ProtoParsedFlag items to Java codegen
  aconfig: pass ownership of ProtoParsedFlag items to C++ codegen
  aconfig: pass ownership of ProtoParsedFlag items to Rust codegen
2023-12-15 15:24:39 +00:00