Release configs with `aconfig_flags_only: true` can inherit build flag
values, but cannot set them.
Bug: 328495189
Test: manual
Change-Id: I2bdc88761d61fb20f10dc734f2ba87114f51b859
- Automatically determine the top of the workspace.
- build-flag defaults to using `get_build_var` to get product specific
release config maps.
- release-config defaults to using PRODUCT_RELEASE_CONFIG_MAPS but does
not use `get_build_var` unless the argument is given.
Bug: 328495189
Test: manual
Change-Id: I4ba3c5dfab43c4ebc3eeda13318f42e886dada4e
- Parse release_config_map.textproto files only once
- Fix inheritance
- Sort flag artifacts by flag name
- Add --all_make option for testing
- Fix value() parsing in crunch_flags
Bug: 328495189
Test: manual
Change-Id: I577e7fb07171bea9a53d61eaf77ec728b60d7a26
When set, this removes all traces of the flag from the release config.
Bug: 328495189
Test: manual
Change-Id: Iab39d06e134bfc020ba4875c1b8e89c5cdf18924
- crunch-flags automates converting build flags from starlark to protobuf.
- build-flag is used to set, get and trace flag values.
Bug: 328495189
Test: manual
Change-Id: I941a4420a8bdfa2df73d94e52b3f34a6d1ea3278
- Allow each of the formats (json, pb, textproto) to be individually
controlled.
- Include product and release in the name of the aritfact.
Bug: 328495189
Test: manual
Change-Id: Ia08e7224200a48994bea882a61b8199d53576d38
This will allow the `build-flag` command to use release_config_lib code.
Bug: 328495189
Test: manual
Change-Id: If068597b1d68c52c941788931997b7f77c73d2b3
flag_declaration needs to include the namespace, and origin is not
needed.
Bug: 328495189
Test: manual
Change-Id: Icbadbe6d9888323874c07d84ccd6f50c284e1a1c
This is no longer used since the roboleaf cancellation.
Bug: 315353489
Test: m nothing --no-skip-soong-tests
Change-Id: Ie6ee093c2810498306ea4a2288050eed17a35357
A map iteration was causing a non-deterministic order
in output entries. The default behaviour is to preserve
the same order of the input zips, which is necessary
to ensure a deterministic output for deterministic inputs.
Bug: b/322788229
Test: Ran a couple of builds and confirmed no cache misses from
the output of merge_zips.
Change-Id: I3217e6887ab108d213a290b59da5b33d51b8241f
In order to use soongdbg, you must run analysis with GENERATE_SOONG_DEBUG=true set
in the environment.
Test: GENERATE_SOONG_DEBUG=true m nothing ; soongdbg ...
Change-Id: If43676fe2784f05cd87c0ecb4a46ab676b91023f
n2 (a reimplemntation of ninja) adds a check that depfiles only add
dependencies on source files, not generated files, because generated
files are not guaranteed to exist before an action runs if there isn't
an explicit dependency between the two.
This is the case for our glob result files. However, we can rework the
build so that they're directly depended on instead of using a depfile.
Bug: 318434287
Test: rm -rf out && m nothing && m nothing
Change-Id: I513fa5536136e6bf19c347710f0722d696199612
Generate module_info.json for some Soong modules in Soong in order to
pass fewer properties to Kati, which can prevent Kati reanalysis when
some Android.bp changes are made.
Soong modules can export a ModuleInfoJSONProvider containing the
data that should be included in module-info.json. During the androidmk
singleton the providers are collected and written to a single JSON
file. Make then merges the Soong modules into its own modules.
For now, to keep the result as similar as possible to the
module-info.json currently being generated by Make, only modules that
are exported to Make are written to the Soong module-info.json.
Bug: 309006256
Test: Compare module-info.json
Change-Id: I996520eb48e04743d43ac11c9aba0f3ada7745de
This reverts commit dd9529050d.
Reason for revert: There are at least b/315337725 and b/315036698 issues, reverting this cl while I fix them.
Change-Id: I644bbbe594b9dd53a99a3b6513fbcba68afe0b4a
Previously, sbox would follow the symlink and copy the symlink's target
into the sbox, instead of the symlink itself.
camera-its is a genrule that zips up a folder that includes a symlink,
so we want the symlink to still be there instead of a copy of the file
it points to.
Bug: 307824623
Test: ./build/soong/tests/genrule_sandbox_test.py camera-its
Change-Id: Ib81f6fd6745ec2c58580082c73e33760d5051f5e
Go 1.21 modifies the file limit but restores the original when
execing subprocesses if it hasn't be overridden. Call Setrlimit
even if it doesn't appear to be necessary so that the syscall package
considers it set.
Bug: 309895579
Test: m nothing
Change-Id: I4d0b27bac90a2a88bfc68a8491d54dc106e1ec13
This change defaults Java stubs to be generated from API text files
during build. Using the `--build-from-source-stubs` flag, users can
toggle between the feature.
This change relands aosp/2617457.
Test: m nothing && verify ninja path exists between android_stubs_current and android_stubs_current.from-text, and does not exist between android_stubs_current.from-source, m nothing --build-from-source-stub && verify the opposite
Bug: 301522358
Change-Id: Ic129dafc3231e71470af982374739930947b69cb
This allows product config to determine what build flags are present in
a product's build, as well as their value.
As product config moves to a declarative specification, this can be
handled with less unnecessary work.
Bug: b/302593603, b/308849337, b/309477343
Test: manual
Change-Id: Ic0551990864bba6d03b6bf49fcfd4ae2523d9c6d
This allows product config to determine what build flags are present in
a product's build, as well as their value.
As product config moves to a declarative specification, this can be
handled with less unnecessary work.
Bug: b/302593603
Bug: b/308849337
Test: manual
Change-Id: If57622059bb7d1c33df7498321db621c0c2d30c0
Jacoco uses ziptime and then merge_zips. merge_zips is already rewriting
all the zip headers so its trivial to move the ziptime functionality
into merge_zips.
Bug: 308765940
Test: TestMergeZips
Change-Id: I8158c0b99878492233ac01fde3820e1951c9088e
Start a background goroutine at the beginning of soong_build that
captures the CPU usage, heap size, and total system memory every
second. Propagate the values through soong_build_metrics.pb back
to soong_ui, and then into build.trace.gz.
Test: m nothing, examine build.trace.gz
Change-Id: Iad99f8f1f088f4f7f7d5f76566a38c0c4f4d0daa
This reverts commit 182b56b870.
Reason for revert: prime suspect for breaking boot tests b/307495247, b/307411752
Bug:307495247
Change-Id: Iea05703b767d2699ca3cf69377eb44b1d21697ad
This change defaults Java stubs to be generated from API text files
during build. Using the `--build-from-source-stubs` flag, users can
toggle between the feature.
Test: m nothing && verify ninja path exists between android_stubs_current and android_stubs_current.from-text, and does not exist between android_stubs_current.from-source, m nothing --build-from-source-stub && verify the opposite
Bug: 274805756
Change-Id: I28834f92c1b1311e3fe0a71a6ea9e8ec2e278d7e
Cleanspec wasn't working, so let's handle it in Soong
Test: create an invalid file, run m nothing, verify it is removed
Change-Id: I55d3c9344541b8e504887225a660044e14204427
It looks like hardware/qcom/sdm710 no longer exists in the source tree, so this symlink removal is should no longer be needed?
Bug: n/a
Change-Id: I8c9c6c0c0371039f910fc4290f81a692e4cd5cd4
This involves some minor changes to testing infrastructure.
This is a rollforward of aosp/2628496 with a couple of minor changes:
- In ParseBuildFiles, filter out all build files that are kept due
to ShouldKeepExistingBuildFileForDir
- Add some minor test infrastructure for StubbedBuildDefinitions, with a
couple of proof of concept tests used to demonstrate its usage. This
pattern will become immensely more common as we implement allowlist v2
(as we will need to update all tests which today simulate build
definitions that have missing deps)
Bug: 285631638
Fixes: 286545783
Test: bp2build.sh
Test: m nothing
Change-Id: I7c3a03b02098e39dd8e51d327482b440f294478f
The RuleBuilder API would not expand Ninja variables because the
variables would be written verbatim to the sandbox manifest file. This
commit allows a rule to specify that the manifest file should be written
in an un-escaped format so that Ninja variables are expanded before
writing the manifest file.
Bug: 286077158
Test: rust sandboxing topic + go test
Change-Id: I1915431f6e24d04d343dacc213c9079674ec8251
This feature is obsolete.
This makes a large number of codepaths "dead code" (such as
module-specific implementations of ApiBp2build functionality). These
will be deleted in a followup CL.
Bug: 284029211
Test: Presubmits
Change-Id: Ib53b99f1fe8c24380d219caf44e9bb3b96724fa0
writeHeader generates zip64 extras that are correct for the local
header, but incorrect for the central directory header. Strip the
extras again after writeHeader so that the central directory header
extras are recreated correctly.
Test: Zip2Zip64
Bug: 296314205
Change-Id: I1ca6a5745a9f97426df6c111db444facdfa25b2e
Revert submission 2651299-bazel_sandwich
Reason for revert: b/293883239, checking if this breaks the build.
Reverted changes: /q/submissionid:2651299-bazel_sandwich
Change-Id: I5f1d4bb13d21e6599f5c353dcaba2375f5ec234d
The "bazel sandwich" is a mechanism for bazel to depend on make/soong
outputs. The name comes from the fact that bazel is now at the top
and bottom of the build graph. This is intended to allow us to work
on converting the partition builds to bazel while not all of the
dependencies of the partition have been converted.
It works by adding the bazel_sandwich_import_file rule, which emits a
dangling symlink that starts with bazel_sandwich:, and includes
information that the aquery handler in soong reads. The aquery handler
rewrites the symlink so that it points to a file generated by
make/soong, and adds a ninja dependency from the symlink to the file
it's targeting.
This allows us to depend on make-built files from bazel, but notably
it doesn't allow us to depend on analysis-time information from make.
This shouldn't be a problem for the partitions, but limits the use of
the bazel sandwich to similar, less complicated types of builds.
go/roboleaf-bazel-sandwich
Bug: 265127181
Test: m bazel_sandwich
Change-Id: Ic41bae7be0b55f251d04a6a95f846c50ce897adc
kotlinx_coroutines_test and kotlinx_coroutine_android each provide a
META-INF/services/kotlinx.coroutines.CoroutineExceptionHandler with
different contents, and the final contents needs to be the combination
of the two files. Implement service merging in merge_zips when the
-jar argument is provided.
Bug: 290933559
Test: TestMergeZips
Change-Id: I69f80d1265c64c671d308ef4cdccfa1564abe056
This allows us to set product variables as build settings instead
of loading them from a target's provider, which further allows us
to read product config variables in transitions.
Bug: 287539062
Bug: 269577299
Test: Presubmits
Change-Id: I8497703f706162572ceb3486240e1eb02a37f5f6
The only difference between preprocessed + presigned vs just presigned
is that just presigned will zip align the apk, and preprocessed +
presigned will instead check that the apk is already aligned and has
uncompressed jni and dex libs. This should be the default to ensure
we're getting valid apk imports from external sources.
Presigned alone also won't work for apps with target sdk >= 30, because
on those target sdks you have to use apk signature v2, and having the
build system align the zip file will break that signature.
Bug: 185811447
Test: Presubmits
Change-Id: Ie825ab2eab4feeda5c266233a9dca7bbf9559be7
This reverts commit bcfadce598.
Because now test_compiler.py copies clang-dev instead of creating a
symlink.
Bug: 288327832
Test: run test_compiler.py
Change-Id: Ib676a41c152046d710d73fdaac678d8a98de5af2
Also applied this feature to kotlinc since it needs both java and kt files.
Test: local test and presubmits
Bug: 287071909
Change-Id: I28c725b03115aef47055448cb75412980015da15
From the previous change, build.ninja is generated as a symlink of
build.<product>.ninja for any missing use case of build.ninja. As this
was a temporary solution while introducing new changes, removing this
feature and move all build.ninja usage into build.<product>.ninja.
Bug: 277029044
Test: aosp_cf build succeeded
Change-Id: I6e8d2840491654fa4253b1febfaecf0ee9455a2d
This paves the way for some feature work for bp2build, and also shaves
0.25-0.5s from the runtime of bp2build (as tested on my machine).
Test: Treehugger
Bug: 285631638
Change-Id: I841f3f904731d02468a6a68a9535d45e0ea9d7c1
The parsed BUILD files will be scanned for obvious definitions of BUILD
targets which have Android.bp counterparts. In such cases, bp2build will
automatically omit conversion of these defined modules (to prevent
collisions). With this change, we no longer need one-off denylisting of
modules which have BUILD file definitions.
This has a 0.03s to 0.2s slowdown for bp2build with current state. This
impact is identical on a heavier test branch, as well. I also ran an
experiment that applied BUILD scanning to all source BUILD files
(regardless of allowlisting), and this had a 2 second slowdown.
We may want to look into parallelizing or improving the performance of
this evaluation, but it's probably not worth the effort at this time,
since the current performance hit is small.
Test: New integration test
Test: Removed libprotobuf-python from denylist and tested building the
package
Test: Treehugger
Change-Id: Ibde3bab12cd4a8fed642ad46e5344a56953bec91
In non-incremental build, there is no ninja_log. For this case, use
HINT_FROM_SOONG as an alternative solution.
Bug: 273947040
Test: 1.m after removing out/.ninja_log
2.check if non-incremental CI build uses HINT_FROM_SOOONG
3.check if incremental CI build uses NINJA_LOG
4.check if there is no regression in CUJ
Change-Id: I00cd216df096cb2288eeab233729acefb0d1b73c
This allows USE_BAZEL_VERSION to be set for m builds, which will use
Bazelisk for any Bazel invocations during those builds.
This should be used only for manual debugging, typically to either test
new Bazel features, verify Bazel compatibility with Android, or culprit
find new Bazel breakages.
Test: Manually run builds with USE_BAZEL_VERSION, toggled off and on to
ensure the build was rerun. Tested with a broken commit, a working
commit, and 'last_green' special term
Change-Id: I8b475dca5c8d4bd849ee3724a8c3aca9b631bcb8
Current build.ninja does not contain any product name, while other ninja
files (such as combined ninja) do. This change adds product name to the
build.ninja so it can be separated over multiple lunch targets
Bug: 277029044
Test: build succeeded and checked if out/soong/build.ninja has been
renamed
Change-Id: I16dc71f829fd76f01b98da0d509a8e0ef6f62fa9
This fixes an incrementality bug that resulted in superfluous
build.ninja regeneration after a new glob definition is added.
Fixes: 279674820
Test: Treehugger
Test; New integration test in this CL
Change-Id: Ifefe66a0eb1c125e9ad5373d60437a1cb1e6fdec
It also processes bazel metrics, and is thus misnamed.
Bug: 279988780
Test: b build libcore:all (with prints to verify that it's called)
Change-Id: I99828e759872ba2dfb583c20a01b8101395b3558
This is for use in tracking bazel exit codes from b invocations.
Bug: 279754118
Test: b build libcore:all (with prints to verify)
Change-Id: If36b5b73ff93cd15647e5c1c1f676e95137210ff
