Commit graph

7012 commits

Author SHA1 Message Date
Cole Faust
894bb3b530 Add //visibility:any_partition
When we convert the partitions to be built with soong, there will
likely be separate partition modules per product. This means that a
lot of installable modules will have to be visible to a lot of
partitions. To make this easier, add the //visibility:any_partition
visibility spec, which allows the module to be used from any module
of type android_filesystem or android_system_image.

Fixes: 321000103
Test: go test
Change-Id: Iea1f1ab7d88dfdb1fd00f19eb8c9941693a2375f
2024-02-07 12:03:32 -08:00
Jihoon Kang
d2e49a9261 Merge "Introduce make var ANDROID_PUBLIC_EXPORTABLE_STUBS" into main 2024-02-06 21:59:11 +00:00
Jihoon Kang
7cf2c0cce7 Introduce make var ANDROID_PUBLIC_EXPORTABLE_STUBS
ANDROID_PUBLIC_STUBS currently has multiple usages in the build, in
multiple .mk files.

Instead of modifying the current functionality of ANDROID_PUBLIC_STUBS
by replacing its value from "android_stubs_current" to
"android_stubs_current_exportable", this change introduces
ANDROID_PUBLIC_EXPORTABLE_STUBS, which evaluates to
"android_stubs_current_exportable" and used only when generating the
`full_target` in `development/build/Android.mk`.

Test: patch in git_main, lunch aosp_arm-ap31-eng && m sdk dist && inspect android.jar outputs
Bug: 323261972
Change-Id: Ic2b03a5a4afdefb459d89f8104916446599eaf31
2024-02-06 18:51:19 +00:00
Spandan Das
7870d329a5 Merge "Mechanism to select a specific version of java_sdk_library_import" into main 2024-02-06 17:54:58 +00:00
Treehugger Robot
d7e6bd8a04 Merge "Revert "Enforce that soong config variables are identifiers"" into main 2024-02-06 10:50:10 +00:00
Almaz Mingaleev
149b5d003a Revert "Enforce that soong config variables are identifiers"
This reverts commit d7c32db93d.

Reason for revert: breaks builds b/323989166

Change-Id: I6937f2cbecbde39fcfe605937861510329ba44c3
2024-02-06 09:13:50 +00:00
Ke-Yu Lu
196d9493c4 Merge changes from topic "revert-2925209-KVPWEBRQHT" into main
* changes:
  Revert "Convert AFDO mutators to TransitionMutator"
  Revert "Remove fdoProfileMutator"
2024-02-06 05:46:07 +00:00
Spandan Das
23956d12ab Mechanism to select a specific version of java_sdk_library_import
This CL is the java_sdk_library_import equivalent of aosp/2928483.

With trunk stable, we will have multiple apex prebuilts in the tree.
Each apex prebuilt will have its own module sdk. This means that it is
possible to have mutliple versions of `framework-foo` in the tree. This
CL introduces a mechanism to select a specific versioned
java_sdk_library prebuilt.

Implementation details
- Add a `source_module_name` property to java_sdk_library_import. This
  will identify the source equivalent of the jsl in packages/modules.
  This used to be implicit - i.e. the name without the prebuilt_ prefix.
  With multiple prebuilts, this has to become explicit.
- Set appropriate `source_module_name`(s) in the dynamically created
  child modules. e.g. the source_module_name on sdklib.v1.stubs and
  sdklib.v2.stubs will both be sdklib.stubs, assuming
  `source_module_name` on the top-level jsl_import is sdklib
- Add a private Created_by_java_sdk_library_name property to java_import
  and prebuilt_stubs_sources modules. This will be used to idenfity the
  top level java_sdk_library_import that was used to create these child
  modules. This is necessary because java_sdk_library_imoprt is a macro
  that creates 1:many modules. However, to avoid toil, only the
  top-level java_sdk_library_import will be listed in
  `apex_contributions`. This new property will be used for
  source/prebuilt selection in android/prebuuilt.go
- Rename BaseModuleName in commonSdkLibraryAndImportModule to
  RootLibraryName. This is necesssary because the former is now reserved
  to identify the source equivalent of a prebuilt module (maybe we
  should rename it?)

Bug: 322175508

Test: Added a unit test
Change-Id: If6aa6b0e939a07f8263874941413908383c81a78
2024-02-06 04:14:59 +00:00
Ke-Yu Lu
0be9d6034e Revert "Convert AFDO mutators to TransitionMutator"
Revert submission 2925209

Reason for revert: b/323975183

Reverted changes: /q/submissionid:2925209

Change-Id: Ie907d23081817d601a1a2083347b5f683829a36c
2024-02-06 02:15:03 +00:00
Cole Faust
f7757f1246 Merge "Enforce that soong config variables are identifiers" into main 2024-02-06 02:04:29 +00:00
Zi Wang
4f0b9b4289 Merge "Automatically propagate jarjar rules for aconfig libraries" into main 2024-02-06 00:04:07 +00:00
Cole Faust
d7c32db93d Enforce that soong config variables are identifiers
I want to be able to use certain characters to separate a namespace
from a variable, but currently it's possible for soong config namespaces
and variables to have any character in them.

Restrict them to just identifiers, which should cover all of our
existing usages.

Bug: 323382414
Test: Presubmits
Change-Id: I6d55dd3378ac4d257efde450189b81a4aa3932c3
2024-02-05 16:01:18 -08:00
Treehugger Robot
87177f21b4 Merge "Fix non-determinism in all_teams.pb" into main 2024-02-05 23:29:14 +00:00
Colin Cross
6c392749d5 Merge changes Ied8fd7b5,Ib0584545 into main
* changes:
  Remove fdoProfileMutator
  Convert AFDO mutators to TransitionMutator
2024-02-05 23:03:49 +00:00
Joe Onorato
349ae8dd6b Automatically propagate jarjar rules for aconfig libraries
Test: treehugger
Bug: 310504781
(cherry picked from https://googleplex-android-review.googlesource.com/q/commit:97c03a6dc659102ff40793759fb3f0f18164a85b)
Merged-In: I639d12ff33175b7bed7e7d0595a40dd9b0d99367
Change-Id: I639d12ff33175b7bed7e7d0595a40dd9b0d99367
2024-02-05 22:46:00 +00:00
Spandan Das
2268cd8b8b Fix non-determinism in all_teams.pb
this.teams_for_mods uses a dictionary and `range` yields a
non-deterministic ordering. This would cause all_teams.pb to be
non-determinisitic. Since this file is created during Soong analysis, it
would cause the .ninja file to also be non-deterministic.

Use SortedKeys to do the iteration instead.

Test: go build ./android
Test: m nothing a couple of times and checked that the .ninja files are
identical

Change-Id: Ife2d2520d118ef25639f86390912d98b5f057655
2024-02-05 22:11:28 +00:00
Spandan Das
bb0d5866c5 Merge "Sort the inputs to mergedAconfigFilesRule" into main 2024-02-05 21:16:54 +00:00
Colin Cross
6b2b493bf3 Merge "Follow signature update to ExtendMatchingProperties" into main 2024-02-05 21:02:02 +00:00
Colin Cross
2a28cd3c5e Merge changes I64a5f3c9,I1ed0a380 into main
* changes:
  Optimize mutator contexts
  Optimize InstallPath.String()
2024-02-05 20:22:20 +00:00
Spandan Das
4d4edfb8d8 Sort the inputs to mergedAconfigFilesRule
The aconfig files are collected by visiting the direct deps of the
module. VisitDirectDeps does not guarantee ordered traversal. To prevent
non-determinism in the generated ninja file, sort the inputs.

Test: go build ./android
Change-Id: Ic67fc0859bf18de62b6297ed502d1d495cc3a780
2024-02-05 19:34:48 +00:00
Spandan Das
81d95c5824 Do not replace the direct edge between rdeps and java_sdk_library
android/prebuilt.go#isSelected has a special-case inside it to ignore
apex_contributions contents for the top-level java_sdk_library hook.
This was necessary because even though we might want source stubs in
next builds, we still needed the top-level prebuilt hook to be active to
emit the dexpreopt rules to .mk.

This worked fine for rdeps that create a dependency edge on the child
stub modules of java_sdk_library. Notable examples include the full
android api stubs created by f/b/api.go. In postdeps mutator, these
expanded deps get rewritten to source/prebuilt if necesssry.

The exception to this are workflows which depend on the top-level hook
directly via `libs`. We resolve these rdeps to an appropriate provider
during GenerateAndroidBuildActions stage. This meant that rdeps were
getting prebuilt stubs of these even in next builds.

Bug: 323454855

Test: Added a unit test

Test: lunch cf_x86_64_only_phone-next-userdebug
Test: aninja -t query
out/soong/.intermediates/packages/modules/Permission/SafetyCenter/Config/safety-center-config/android_common/javac/safety-center-config.jar
| grep prebilts/module_sdk # empty now

Change-Id: Id91333d88055519f3c58ab40466f9628085f5180
2024-02-03 00:17:16 +00:00
Colin Cross
1e7e0432fa Follow signature update to ExtendMatchingProperties
The order function passed to ExtendMatchingProperites no longer takes
a property, dstValue or srcValue parameter.

Test: builds
Change-Id: I19fc554f705fbf42100f0a38df87e2d1fb2454ec
2024-02-02 15:56:32 -08:00
Colin Cross
984223fd04 Optimize mutator contexts
Mutator contexts are created for every module, and in the case of
transition mutator contexts for every dependency of every module.
Unlike the Blueprint mutator contexts that they wrap, the Soong
mutator contexts can be relatively large.  Add global pools for all
of them that can avoid the repeated allocations.

Test: SOONG_PROFILE_MEM=/tmp/mem.pprof m nothing
Change-Id: I64a5f3c91292cff6352300f99c11ac50c713f96d
2024-02-02 15:56:32 -08:00
Colin Cross
c0e42d5dfc Optimize InstallPath.String()
Every InstallPath will have String() called on it eventually, often
more than once if it is in a slice that is sorted.  Precompute the
full path so it can be returned from InstallPath.String() without
recomputing every time.

Test: paths_test.go
Change-Id: I1ed0a3801806854356865c0a5fc35d5cf6d349fe
2024-02-02 15:56:32 -08:00
Treehugger Robot
73b1b48d9b Merge "Use ordered interface from cmp package" into main 2024-01-30 23:45:39 +00:00
Spandan Das
f3c63e6743 Merge "Propagate profile_guided requirement of imports to top-level apex" into main 2024-01-30 22:50:38 +00:00
Cole Faust
efc70122af Use ordered interface from cmp package
I just learned this exists.

Test: m nothing --no-skip-soong-tests
Change-Id: Ic6f0d11c24b41b394b1760fcca9e0da53243e6d9
2024-01-30 14:42:12 -08:00
Spandan Das
2ea84dd0dc Propagate profile_guided requirement of imports to top-level apex
For prebuilts, the dexpreopt rules of system server jars are now
generated from the context of the top-level prebuilt apex and not in the
context of the shim java_import modules. Since
`dex_preopt.profile_guided` property is defined in java_import, this
needs to be bubbled up to the top-level apex. This will be done using
deapxerInfo. If profile_guided of a transitive java_import is true, the deapexed .prof file will be
set as dexreopter.inputProfilePathOnHost before invoking
dexpreopter.dexpreopt. This ensures that only that java_import undergoes
profile guided dexpreopt, and not every other transitive java_import

Test: go test ./apex -run TestPrebuiltStandaloneSystemserverclasspathFragmentContents
Test: lunch cf_x86_64_only_phone-next-userdebug && m
$ANDROID_PRODUCT_OUT/system/framework/oat/x86_64/apex@com.android.art@javalib@service-art.jar@classes.odex
Test: du -sh
$ANDROID_PRODUCT_OUT/system/framework/oat/x86_64/apex@com.android.art@javalib@service-art.jar@classes.odex
24K

Bug: 308790457
Change-Id: Ibf46ecb400b3f126b243fc8d27b08d9a1aa4cc97
2024-01-30 18:04:52 +00:00
Colin Cross
69a680feec Prevent defaults modules from expanding path property dependencies
Defaults modules should not have dependencies added to them for
path properties, the properties will already have been squashed
into other modules that depend on the defaults modules, and the
path dependencies will be added to those modules.

Fixes: 321056451
Test: TestDefaultsPathProperties
Change-Id: I59049f94b7a0924a7b1d997d15723901b0d522ee
2024-01-29 21:21:30 +00:00
Yi-Yo Chiang
c21f548fd4 Merge "cc: Vendor modules should NOT derive its "SDK" level from VNDK" into main 2024-01-29 03:14:22 +00:00
Colin Cross
a6b407fbf5 Convert AFDO mutators to TransitionMutator
Convert afdoDepsMutator and afdoMutator to a TransitionMutator as a
step towards variants-on-demand.

Bug: 319288033
Test: afdo_test.go
Change-Id: Ib05845455ccf43a07b3915a0d7b0a95896062f13
2024-01-26 11:08:32 -08:00
Treehugger Robot
99ed03f4fe Merge "Add build flag to enable ResourceProcessorBusyBox by default" into main 2024-01-26 19:07:25 +00:00
Ronald Braunstein
92eb74475b Merge "Add rule for proto file so we can write python to use it." into main 2024-01-26 18:56:24 +00:00
Colin Cross
8f1b033cf7 Add build flag to enable ResourceProcessorBusyBox by default
Setting the RELEASE_USE_RESOURCE_PROCESSOR_BY_DEFAULT build flag will
change the default for use_resource_processor from false to true.

Bug: 294256649
Test: builds
Change-Id: If643bf8db9e2e039200f63f4aac226ed5b44d9e0
2024-01-26 09:09:42 -08:00
Ronald Braunstein
1228b0baef Add rule for proto file so we can write python to use it.
I wrote another tool filter and augment the proto files, but these proto
files need to have a python library for that tool.

Test: m filter_teams && out/host/linux-x86/bin/filter_teams  --filter_teams

Change-Id: I6b259ddda2b6c09d55a9c6d445b96e5b77741f09
2024-01-25 21:10:04 -08:00
Spandan Das
f5e03f1c1a Move ApexBundleInfoProvider from apex/ to android/
This provider contains information about the contents of an apex and is
set during the post deps phase of top-level source and prebuilt apexes.

java.dexpreoptDisabled needs to distinguish between platform and apex
variants of a library to prevent creating duplicate ninja rules. It does
so by looking at ApexInfoProvider. This provider gets set on the ctx of
the java_library being dexpreopted.

In case of prebuilts, these rules are being moved to the top-level
prebuilt apexes. Move ApexInfoBundleProvider from apex to android so
that java.dexpreoptDisabled can consume this provider without running
into a golang circular dependency. (java.dexpreoptDisabled will use this
provider in the next CL of this stack).

Test: go build ./android ./apex
Change-Id: I665abd9d0591eeb17b273cc822beb20690e6af09
2024-01-25 22:35:54 +00:00
Treehugger Robot
471e181f92 Merge "Replace goldmont-without-xsaves build with goldmont-without-sha-xsaves" into main 2024-01-25 08:11:40 +00:00
LaMont Jones
b509938e4f Propagate AconfigFiles in ModuleBase.
Aconfig file dependencies are module-independent, and properly part of
ModuleBase.

Bug: b/308625757
Test: manual

Change-Id: I38c5907d1671cc69bb198345201316ae781fdc9f
2024-01-24 23:45:12 +00:00
Colin Cross
c50999eb0a Merge changes from topic "transition-mutator-config" into main
* changes:
  Add ArchModuleContext to TransitionMutator contexts
  Split ArchMutatorContext out of BaseMutatorContext
2024-01-24 20:03:46 +00:00
Colin Cross
4aa3e0ab81 Add ArchModuleContext to TransitionMutator contexts
Converting coverageMutator to a TransitionMutator requires adding
the ctx.Device() and ctx.DeviceConfig() methods.

Bug: 319288033
Test: builds
Change-Id: I697b48eb89bc23800d2d3c62d68358769f0d1075
2024-01-24 10:17:10 -08:00
Colin Cross
1d3d9f13b8 Split ArchMutatorContext out of BaseMutatorContext
Split the context methods that are useful on anything visiting a
module that has arch variants into a separate ArchMutatorContext
for reuse by TranstitionMutators.

Bug: 319288033
Test: builds
Change-Id: Ifdc21983c6c79f22965a49f169812a8cc3ad975b
2024-01-24 10:17:10 -08:00
Satoshi Niwa
b6d818dbf0 Replace goldmont-without-xsaves build with goldmont-without-sha-xsaves
For chromebooks, we need a new build variant whose target arch is
goldmont, but without SHA and XSAVES support (b/314243939#comment21)

Bug: 314243939
Test: Build an image with TARGET_ARCH_VARIANT:=goldmont-without-sha-xsaves
Change-Id: If73660b515b443d5c138ca367fa3d1c6f18485b5
2024-01-24 16:27:56 +09:00
Ronald Braunstein
43f3b601e2 Merge "Add team property to all modules." into main 2024-01-23 16:00:58 +00:00
Yi-Yo Chiang
88960aa827 cc: Vendor modules should NOT derive its "SDK" level from VNDK
Although vendor modules can be built bundled, they are technially not
part of the "platform", as it cannot access platform libraries (system
partition) directly.
They can only link to a restricted set of platform libraries (LLNDK),
and access restricted set of APIs.
We used to derive the LLNDK API level from the VNDK version. However
after VNDK deprecation, there is no "VNDK version" anymore.
Instead we would just derive the value from platform SDK version:
* If building an in-development build, build vendor modules against the
  in-development "current" API level.
* If building a REL / Final build, vendor should target the latest
  stable API.

Bug: 320423828
Test: go test
Test: presubmit
Test: build and boot
Change-Id: I2c5ef6530e9046b2dcc282bc1f020d8a505eab15
2024-01-23 12:43:38 +08:00
LaMont Jones
34314b7d15 Add VisitDirectDepsIgnoreBlueprint
This method allows dependencies on non-Android modules, which it ignores
in strict mode, rather than flagging the dependency as an error.

Bug: none
Test: manual
Change-Id: I9575e46638fa8ffc69c8935b1b65aab37ceab3c3
2024-01-19 20:22:10 +00:00
Cole Faust
b64c070edf Merge "Remove SymlinkOutputs" into main 2024-01-19 19:38:55 +00:00
Paul Duffin
169cffe526 Merge "Pretty print SBox manifest files to make them easier to read" into main 2024-01-19 10:09:11 +00:00
Paul Duffin
4a3a0a5412 Pretty print SBox manifest files to make them easier to read
When debugging problems with, or just modifying a ninja rule that uses
the SBox it is often necessary to read the manifest file. This change
makes them easier to read by pretty printing them onto multiple lines
with a small but clear indent.

This increased the size of the following file by abot 6% from 66K to
70K bytes:
    out/soong/.intermediates/packages/modules/Wifi/framework/framework-wifi.stubs.source.system/android_common/metalava.sbox.textproto

Test: m out/soong/.intermediates/packages/modules/Wifi/framework/framework-wifi.stubs.source.system/android_common/metalava.sbox.textproto
      less out/soong/.intermediates/packages/modules/Wifi/framework/framework-wifi.stubs.source.system/android_common/metalava.sbox.textproto
Change-Id: Icfadeaad1c6334bd0010128f4437669f35822850
2024-01-19 00:57:16 +00:00
Colin Cross
1b1b9a937b Merge changes from topic "soong-tests-presubmit" into main
* changes:
  Add script to run Soong tests with go tools
  Disable TestVariantSingletonModule when go test -short is used
  Fix data race in propagateRROEnforcementMutator
  Fix data race in finder_test.go
  Fix data race in dex_bootjars
  Fix race CommonGlobalCflags when running tests in parallel.
  Fix data race in snapshot singletons when running parallel tests
2024-01-18 23:58:45 +00:00
Ronald Braunstein
73b08ffd0d Add team property to all modules.
This allows vendors (like google) to specify which team owns the test
module and code.

Team is a commonProperty on modules and points to the designate "team"
module.  The DepsMutator adds the dependency on the "team" module and
"GenerateBuildActions" write the team data to intermediate files.

A new singleton rule, all_teams visits all modules and writes out
the proto containing the team for each module.
If a module doesn't have a team, then it finds the package in the
blueprint file and parent directory blueprint files that have a
default_team and uses that team.

Test: m all_teams
Test: go test ./python ./java ./cc ./rust ./android
Test: added team to HelloWorldHostTest and built the new asciiproto target
Test: added package default_team and checkout output proto.
Change-Id: I5c07bf489de460a04fc540f5fff0394f39f574a7
2024-01-18 14:16:06 -08:00