Commit graph

8328 commits

Author SHA1 Message Date
Cole Faust
5da2c1142d Merge "Allow importing starlark code in makefiles" am: 99d89154f1 am: 0a9d1d6025 am: 7933d9b60e am: a6ff248861 am: 00c84de792 am: db6793508b
Original change: https://android-review.googlesource.com/c/platform/build/+/2583544

Change-Id: Iaad9a4aea9b4a73b7b968a9a8467e2df2f62f17e
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-11 21:50:34 +00:00
Cole Faust
0a9d1d6025 Merge "Allow importing starlark code in makefiles" am: 99d89154f1
Original change: https://android-review.googlesource.com/c/platform/build/+/2583544

Change-Id: If559b5f4119df6b19c42f5bf60f738ddf4365d8f
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-11 17:14:36 +00:00
Cole Faust
99d89154f1 Merge "Allow importing starlark code in makefiles" 2023-05-11 17:04:46 +00:00
Mårten Kongstad
260a4b13f8 Merge "aconfig: Add codegen for java" am: 7b6aacb055 am: 86d0c527c0 am: ed130f11ed am: 6e6c5ca7b7 am: 6385dedef6 am: eb8862c747
Original change: https://android-review.googlesource.com/c/platform/build/+/2583478

Change-Id: I23aab40e5a91ea5223b539842fb7975187ca9211
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-11 12:03:24 +00:00
Mårten Kongstad
86d0c527c0 Merge "aconfig: Add codegen for java" am: 7b6aacb055
Original change: https://android-review.googlesource.com/c/platform/build/+/2583478

Change-Id: I8bd49c346fdd4f2e30c5033f63783e66bc7d4bb8
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-11 08:24:31 +00:00
Treehugger Robot
b0890111cd Merge changes from topic "aconfig-part-4" am: ef0a2c21dd am: 13456aa63c am: e84e9eaf2f am: 51771a5de5 am: 7a9674fbb9 am: cdf0303f70
Original change: https://android-review.googlesource.com/c/platform/build/+/2584571

Change-Id: I709f816aa0a50561bf166505c92f15fe244b346a
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-11 08:07:52 +00:00
Treehugger Robot
07f3b9f40d Merge "Improve finalization perfomance by building only current version." am: b1a7ba8735 am: 02591d64de am: e4eea917db am: 686ef58cf2 am: ab2f5e357c am: d8c5b1f5fe
Original change: https://android-review.googlesource.com/c/platform/build/+/2584105

Change-Id: Ib524958f0e6d9cc72a8d6d703883e74b5488ab0d
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-11 04:33:27 +00:00
Zhi Dou
eb74489b3a aconfig: Add codegen for java
Add codegen for java skeleton

Bug: 279485059
Test: atest aconfig.test
Change-Id: Ia0481cec9c2e137e88e9a77d1b82412529b64adc
2023-05-11 03:08:53 +00:00
Treehugger Robot
13456aa63c Merge changes from topic "aconfig-part-4" am: ef0a2c21dd
Original change: https://android-review.googlesource.com/c/platform/build/+/2584571

Change-Id: I7bb9c8cfab6084b27ce8e62afabe747bb05e43a0
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-10 19:38:09 +00:00
Treehugger Robot
ef0a2c21dd Merge changes from topic "aconfig-part-4"
* changes:
  aconfig: introduce namespace, rename proto messages
  aconfig: add dump protobuf format
2023-05-10 19:22:00 +00:00
Treehugger Robot
02591d64de Merge "Improve finalization perfomance by building only current version." am: b1a7ba8735
Original change: https://android-review.googlesource.com/c/platform/build/+/2584105

Change-Id: I2386fa0a10a112343d0ba20ac256368bec7dbcc3
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-10 16:05:07 +00:00
Treehugger Robot
b1a7ba8735 Merge "Improve finalization perfomance by building only current version." 2023-05-10 15:19:12 +00:00
Mårten Kongstad
3095078981 aconfig: introduce namespace, rename proto messages
Flags belong to a namespace. Update the proto files to reflect this.

Config files can only refer to a single namespace. Override files can
refer to multiple namespaces; an override directive for a flag in a
different namespace than the one represented by the cache will be
silently ignored.

Rename the proto messages to make it more clear what they are. Propagate
this change through the wrappers in aconfig.rs and the rest of the code.

Also, settle on Tracepoint instead of TracePoint.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I16e69dd14687bc498b2ba89d6a35879459903801
2023-05-10 17:17:20 +02:00
Kelvin Zhang
b865fd956d Merge "Fix non-AB ota generation for directories" am: ea70695d92 am: 06417aaa9e am: ee0f974d2c am: 003de339bf am: 0238a07651 am: a85fee6439
Original change: https://android-review.googlesource.com/c/platform/build/+/2585207

Change-Id: I78889dfc84d86e5205dc515344b83bbb17c5aa9f
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-10 07:57:40 +00:00
Kelvin Zhang
06417aaa9e Merge "Fix non-AB ota generation for directories" am: ea70695d92
Original change: https://android-review.googlesource.com/c/platform/build/+/2585207

Change-Id: I7a6472946360238393a734640e27557f7623446f
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-10 04:46:20 +00:00
Kelvin Zhang
ea70695d92 Merge "Fix non-AB ota generation for directories" 2023-05-10 04:07:09 +00:00
Kelvin Zhang
b036007592 Fix non-AB ota generation for directories
Test: ota_from_target_files -v --block yukawa-target_files-10097950.zip
test.zip

Change-Id: I4291010d66cc28bdb6b6b334cdbed06daf3de837
2023-05-09 20:30:53 -07:00
Alex Buynytskyy
aac2a187d4 Improve finalization perfomance by building only current version.
Bug: 275409981
Test: run locally
Change-Id: If531178a17792166726ff7ed230592c392eeb899
2023-05-10 03:10:32 +00:00
Cole Faust
c63ce1ab58 Allow importing starlark code in makefiles
Adds a new `$(call run-starlark,my/starlark/file.bzl)` function that
will run the starlark file and set all the variables in the
variables_to_export_to_make dictionary as make variables.

Fixes: 280685526
Test: m nothing repeatedly causes no ninja regeneration, but touching all_versions.bzl does. go test, ./out/rbcrun -mode=rbc ./build/make/tests/run.rbc
Change-Id: Ic72e18dd28dba8233ba2dfb658b5d03ccece1bfd
2023-05-09 15:12:58 -07:00
Mårten Kongstad
a102909e09 aconfig: add dump protobuf format
Introduce a new protobuf format to represent the all flags parsed by
aconfig. This data in this new format is similar to that of the internal
cache object, but the protobuf is a public API for other tools to
consume and any changes to the proto spec must be backwards compatible.

When aconfig has matured more, Cache can potentially be rewritten to
work with proto structs directly, removing some of the hand-written
wrapper structs and the logic to convert to and from the proto structs.
At this point, the intermediate json format can be replaced by the
protobuf dump.

Also, teach `aconfig dump` to accept an --out <file> argument (default:
stdout).

Also, teach `aconfig dump` to read more than once cache file.

Note: the new protobuf fields refer to existing fields. It would make
sense to split the .proto file in one for input and one for output
formats, and import the common messages, but the Android build system
and cargo will need different import paths. Keep the definitions in the
same file to circumvent this problem.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I55ee4a52c0fb3369d91d61406867ae03a15805c3
2023-05-09 15:27:42 +02:00
Treehugger Robot
05e6976daa Merge "Remove unneeded rbcrun features" am: 872023376e am: 9840061379 am: 512662b325 am: b03cf2a481 am: 194eb7cf28 am: 1d848a1a07
Original change: https://android-review.googlesource.com/c/platform/build/+/2580473

Change-Id: Ia68b8573aa59fedf64b463ef8becabefb49baa32
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-09 12:42:03 +00:00
Treehugger Robot
9840061379 Merge "Remove unneeded rbcrun features" am: 872023376e
Original change: https://android-review.googlesource.com/c/platform/build/+/2580473

Change-Id: Ia8b8f735b272a8a61eb989ecef819516b3a9f2b7
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-09 08:15:26 +00:00
Treehugger Robot
872023376e Merge "Remove unneeded rbcrun features" 2023-05-09 07:31:40 +00:00
Treehugger Robot
c73583a716 Merge changes from topic "aconfig-part-3" am: d279e5d79c am: 8bad8c04de am: 8830d1f018 am: 5c6e7e5672 am: 757f82e57a am: 83c3586feb
Original change: https://android-review.googlesource.com/c/platform/build/+/2581670

Change-Id: I8543cc2df4214d235c705712797e5430db3b7110
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-09 03:18:44 +00:00
Treehugger Robot
8bad8c04de Merge changes from topic "aconfig-part-3" am: d279e5d79c
Original change: https://android-review.googlesource.com/c/platform/build/+/2581670

Change-Id: I0479ee840934c8195172698248fcb06887072ebd
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-08 18:42:27 +00:00
Mårten Kongstad
2c8f3bdd57 Merge changes from topic "aconfig-part-2" am: 80bff11aef am: 2fc13df5d7 am: 08492141ab am: 0abf095ac5 am: c1397bd11c am: 359f95ab50
Original change: https://android-review.googlesource.com/c/platform/build/+/2578555

Change-Id: I4dfaa141c77973149e0b51563bdae6526c61b32d
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-08 15:56:09 +00:00
Mårten Kongstad
2fc13df5d7 Merge changes from topic "aconfig-part-2" am: 80bff11aef
Original change: https://android-review.googlesource.com/c/platform/build/+/2578555

Change-Id: I9f41660ea3cefad70f405b93587baa005b5b4f6e
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-08 11:40:43 +00:00
Mårten Kongstad
98b0eeb2fc aconfig: simplify argument parsing in main
Improve readability by extracting how multiple input files are
transformed from a clap argument to an Input.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I299179d39eca80e107342f6978984da79ee81e34
2023-05-08 12:47:34 +02:00
Mårten Kongstad
76adff2af0 aconfig: improve flag value tracing
Improve how the flag values (state, permission) are tracked when parsing
config and override files: migrate from a raw string to a proper struct,
as this will make it easier when aconfig will output this data in some
structured format for other tools to consume.

Also, rename Cache.debug to Cache.trace.

Bug: 279485059
Test: atest aconfig.test
Change-Id: Idad57c969515f697e9065429d8a44c38d8a512d2
2023-05-08 12:47:34 +02:00
Mårten Kongstad
80bff11aef Merge changes from topic "aconfig-part-2"
* changes:
  aconfig: change flag values to enabled/disabled enum
  aconfig: add read/write permission
  aconfig: Source: remove unnecessary #[derive(Serialize, Deserialize)]
2023-05-08 10:46:26 +00:00
Treehugger Robot
28e2826d58 Merge "Script fixes and improvements." am: be8badd74e am: 9cc5216ed0 am: 661c11ecbb am: 159c0597b2 am: 4ccc9ab350 am: 5e343c1aa5
Original change: https://android-review.googlesource.com/c/platform/build/+/2581131

Change-Id: I460067bf63799a9993bfec9166ab617e13daa8ec
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-07 05:14:57 +00:00
Treehugger Robot
9cc5216ed0 Merge "Script fixes and improvements." am: be8badd74e
Original change: https://android-review.googlesource.com/c/platform/build/+/2581131

Change-Id: Icf9617f5f6ddf548ce5994f2c8945bcd9c06829c
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-07 01:43:18 +00:00
Alex Buynytskyy
00a88ec3ec Script fixes and improvements.
Fix to mark all versions REL.

More robust patch application. Keep getting these build breakages, not sure why. Looks like git is
confusing one folder for the other?
https://android-build.googleplex.com/builds/submitted/10065142/test_suites_arm64/latest/logs/build_error.log

Bug: 275409981
Test: run locally
Change-Id: I6173b9ecae5e9ab12d9abf2eee3f2136dc353210
2023-05-06 20:55:10 +00:00
Cole Faust
a874f88cee Remove unneeded rbcrun features
- rblf_cli and rblf_env
- -c and -f

This is in preparation for making rbcrun able to function as a more
general purpose starlark interpreter.

Bug: 280685526
Test: go test, ./out/rbc ./build/make/tests/run.rbc, ./build/bazel/ci/rbc_dashboard.py --quick aosp_arm64
Change-Id: Ifff9ce7b4369422f39c5003bb85a168c78bde7cf
2023-05-05 11:52:02 -07:00
Mårten Kongstad
c68c4eabea aconfig: change flag values to enabled/disabled enum
Change the underlying type of a flag's value from bool to an explicit
enum (Disabled, Enabled): this will hopefully reduce future confusion on
how flags are intended to be used.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I9535f9b23baf93ad5916ca06fb7d21277b4573eb
2023-05-05 16:20:09 +02:00
Mårten Kongstad
416330b060 aconfig: add read/write permission
Introduce the concept of flag read/write permissions: a read-only flag
can only have its value set during the build; a writable flag can by
updated in runtime.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I3ec5c9571faa54de5666120ccd60090d3db9e331
2023-05-05 13:45:28 +02:00
Mårten Kongstad
2937566c55 aconfig: Source: remove unnecessary #[derive(Serialize, Deserialize)]
The Source struct is no longer serialized/deserialized. Remove the
unused derives.

Bug: 279485059
Test: atest aconfig.test
Change-Id: Ifd78988ed8134ab43013314b4437e428a8927981
2023-05-05 10:57:19 +02:00
Mårten Kongstad
2cc7d9c93e Merge changes from topic "aconfig-part-1" am: 0b6b92e5a5 am: 787a0d1ffd am: 2ec2f44e46 am: 068a8d6f88 am: dc023f2272 am: 4a67e62aad
Original change: https://android-review.googlesource.com/c/platform/build/+/2578416

Change-Id: Ia1d1906e951b82d3a497507053d5c3d9d80bd4fb
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-05 02:36:06 +00:00
Mårten Kongstad
787a0d1ffd Merge changes from topic "aconfig-part-1" am: 0b6b92e5a5
Original change: https://android-review.googlesource.com/c/platform/build/+/2578416

Change-Id: Icdeb54a877d6e0e7df55321e4ed61d04a9011e91
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-04 21:27:00 +00:00
Mårten Kongstad
0b6b92e5a5 Merge changes from topic "aconfig-part-1"
* changes:
  aconfig: add support for changing flag value based on build
  aconfig: introduce cache
  aconfig: define Aconfig proto
  aconfig: add support for cargo
2023-05-04 20:36:22 +00:00
Mårten Kongstad
09c28d1689 aconfig: add support for changing flag value based on build
Teach aconfig about build IDs (continuously increasing integers). Extend
the aconfig file format to allow flags to say "by default, my value is
X, but starting from build ID A, it's Y, and from build ID B, it's Z".

Bug: 279485059
Test: atest aconfig.test
Change-Id: Idde03dee06f6cb9041c0dd4ca917c8b2f2faafdd
2023-05-04 14:34:55 +02:00
Mårten Kongstad
4d2b4b047b aconfig: introduce cache
Introduce the Cache struct to represent parsed and verified aconfig and
override content. Most commands in aconfig will work of an existing
cache file, eliminating the need to re-read the input every time.

Restructure main.rs to use clap to create a proper command line
interface with support for sub-commands. main.rs is responsible for
parsing the command line, performing disk I/O and calling the correct
subcommand implementation (in commands.rs).

To simplify unit tests, subcommands never perform explicit I/O; instead
they only work with Read and Write traits.

Also add dependencies on clap, serde and serde_json.

Bug: 279485059
Test: atest aconfig.test
Change-Id: Ib6abf2eabd264009804f253874b6fba924fc391b
2023-05-04 10:40:26 +02:00
Automerger Merge Worker
36e9b018a1 Merge "Merge "Support generating partial OTAs from extracted target_files" am: c3216a321b" into stage-aosp-master am: abb6ca7ab3 am: 4cc1379797 am: d506fc5b65 am: 2e2f826087 am: edaf0ce34e
Original change: https://android-review.googlesource.com/c/platform/build/+/2577071

Change-Id: I44da8e2c4a0efd8b409a7af339d65b6a59995323
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-04 08:37:53 +00:00
Mårten Kongstad
bb520729e8 aconfig: define Aconfig proto
Fill in aconfig.proto. Define Aconfig definitions (for introducing flags
and setting their values) and Overrides (for overriding flags regardless
of what their definitions say). More changes to the proto schema are
expected when more of the aconfig project is outlined.

Use proto2 instead of proto3: this will cause the protobuf text parser
to error on missing fields instead of returning a default value which is
ambiguous, especially for booleans. (Also, the text protobuf parser
doesn't provide good error messages: if the input is missing a field,
the error is always "1:1: Message not initialized").

Unfortunately the generated Rust wrappers around the proto structs land
in an external crate, which prevents aconfig from adding new impl
blocks. Circumvent this by converting the data read from proto into
structs defined in the aconfig crate.

Change main.rs to parse (static) text proto.

Also add dependency on anyhow.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I512e3b61ef20e2f55b7699a178d466d2a9a89eac
2023-05-04 10:35:25 +02:00
Treehugger Robot
5e4f59f9ca Merge "Support generating partial OTAs from extracted target_files" am: c3216a321b
Original change: https://android-review.googlesource.com/c/platform/build/+/2577071

Change-Id: I79fa834772dc12fe26678238bcf80ecffd2a34ce
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-04 00:54:58 +00:00
Treehugger Robot
a3ef62b3d3 Merge "Update sdk finalizer for change to api_levels_released_versions" am: 626b012556 am: acbf5e5c45 am: dc0b1da6ae am: ae6b2ffb9b am: 073c40d56b am: 929a4cdbd9
Original change: https://android-review.googlesource.com/c/platform/build/+/2573034

Change-Id: I62dbb2088f144f51a3a73be34fe3a813573d1611
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-04 00:19:51 +00:00
Treehugger Robot
c3216a321b Merge "Support generating partial OTAs from extracted target_files" 2023-05-04 00:05:29 +00:00
Treehugger Robot
acbf5e5c45 Merge "Update sdk finalizer for change to api_levels_released_versions" am: 626b012556
Original change: https://android-review.googlesource.com/c/platform/build/+/2573034

Change-Id: Id8fb00b47f16bcd17f0503e5f4fb733d3c5fc03a
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-03 18:01:54 +00:00
Treehugger Robot
626b012556 Merge "Update sdk finalizer for change to api_levels_released_versions" 2023-05-03 17:24:31 +00:00
Mårten Kongstad
30ef7ddd99 Merge "aconfig: add project scaffolding" am: e7a850e942 am: 7b9a0b649c am: 98d42a6c1d am: a7249193d0 am: 1a2c02b0ab am: d67a3dd168
Original change: https://android-review.googlesource.com/c/platform/build/+/2563154

Change-Id: I200a8c9f16a69113d3268481abec758302493086
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-03 11:01:07 +00:00