Commit graph

38058 commits

Author SHA1 Message Date
Kiyoung Kim
48f3778cb4 Separate snapshot definition
Current snapshot definition is located in the CC module, so it is
difficult to capture non-CC module (such as prebuilt_etc) to the
snapshot. Separate general snapshot definition from cc so other modules
can also define its own snapshot.

Bug: 192430376
Test: m nothing passed
Change-Id: Ifb69fb3d2ec555b629aa31ec03e7ce5831fd3063
2021-07-19 11:42:32 +09:00
hamzeh
41ad881e16 Refactoring fuzz packaging code
Rust and cc fuzz packaging have common. This change is to put this
common methods and fields into fuzz_common.

Test: make haiku and make haiku-rust ran locally and verify corpus,
dict and config are packaged along with the executable.

Change-Id: I04bf535cd2597ed8b2bd3d8e6ea8c28f38bdc18b
2021-07-16 17:43:33 -07:00
Paul Duffin
393bffee78 Merge "Avoid permitted_packages related build failures in unbundled builds" 2021-07-16 19:06:20 +00:00
Treehugger Robot
b68048e520 Merge "Use note in linker.s to place linker" 2021-07-16 19:02:27 +00:00
Paul Duffin
5c21145915 Revert "Export implementation class jars for java_boot_libs"
This reverts commit 22ff0aaf51.

The workaround to support hidden API generation with prebuilts by
passing the full implementation jars through to the SDK snapshot is no
longer needed as the monolithic hidden API flag generation uses the
prebuilt flag files that are already part of the snapshot.

This change reverts that previous workaround.

Bug: 192868581
Test: - Update the prebuilts
      DIST_DIR=$PWD/dist TARGET_BUILD_VARIANT=userdebug art/build/build-art-module.sh --skip-apex
      packages/modules/ArtPrebuilt/update-art-module-prebuilts.py --local-dist=dist --skip-cls --skip-apex
      - Build hidden API flags to make sure it does not fail
Change-Id: Idb3fde6f7dcb171677316d8794a4af91ede1f7e0
2021-07-16 17:04:21 +01:00
Paul Duffin
3bc8e218f9 Avoid permitted_packages related build failures in unbundled builds
Unbundled builds cause AlwaysUsePrebuiltSdks() to return true which
causes java_sdk_library_import modules to be preferred instead of their
corresponding java_sdk_library module. That causes the
platform-bootclasspath module to depend on prebuilt versions of modules
like framework-statsd which currently do not specify
permitted_packages and so cause a failure.

This is a temporary hack to prevent an unbundled build from checking
for permitted_packages. It can be removed once the prebuilts have been
updated to include the permitted_packages.

Bug: 193095050
Bug: 193889859
Test: m TARGET_BUILD_APPS=Calendar apps_only
      - before the change it failed in Soong.
      - after the change it started failing, later in Make, so the
        Soong problem has been worked around.
      - I reverted all the recent changes to permitted_packages and
        ran this and it still failed so that failure is unrelated to
        these changes.
Change-Id: I213d35437d76d61cf616d1f16dae213b311c6c8e
2021-07-16 17:03:17 +01:00
Ulyana Trafimovich
fe261473b8 Merge "Don't attempt to add stub libraries to class loader context." 2021-07-16 15:45:19 +00:00
Ulya Trafimovich
78645fbba1 Remove OptionalImplicitSdkLibrary() function (it's not used anymore).
Slightly improve documentation for `OptionalSdkLibraryImplementation()`.

Bug: 193425964
Test: m nothing
Change-Id: I3c3f385cf948cdbec54b5a6fb33a4d24d4b54be3
2021-07-16 16:24:31 +01:00
Ivan Lozano
2bd6cce95a Merge "rust: Remove libtest from stdlibs list" 2021-07-16 14:44:25 +00:00
Paul Duffin
da286f4615 Make bootclasspath_fragments always perform hidden API processing
Previously, bootclasspath_fragment modules would only perform hidden
API processing if they provided some stub libraries and fragments. That
was needed because the bootclasspath_fragment modules were added before
Soong supported hidden API processing on all the different modules and
before they all provided the necessary information that hidden API
processing required.

This change stops hidden API being conditional as it is no longer
required as it has been enabled on all existing bootclasspath_fragment
modules.

Bug: 179354495
Test: m nothing
Change-Id: I0cbf11986adff1f2f967b96f86e6bfe0e9b8b1ef
2021-07-16 15:09:59 +01:00
Ulya Trafimovich
840efb6661 Don't attempt to add stub libraries to class loader context.
A Java module may depend on a stub library. In that case an additional
dependency on the implementation library is created, and it is used to
add the implementation library to class loader context. We should not
attempt to add the stubs library as well (previously the attempt to add
it happend after the implemention was added to CLC, to the attempt was
unsuccessful).

Raise an error if someone tries to add the same library with different
build/instal paths.

Also, rename local variable `implicitSdkLib` to `sdkLib` to better
reflect its meaning.

Bug: 193425964

Test: $ lunch aosp_cf_x86_64_phone-userdebug && m && launch_cvd
      $ adb wait-for-device && \
        adb root && \
        adb logcat | \
        grep -E 'ClassLoaderContext [a-z ]+ mismatch' -C1
      # empty output, no errors
Change-Id: I01c1bdd23f9d118d891d0b806e7e3b4d78896a34
2021-07-16 14:44:18 +01:00
Ulyana Trafimovich
f87aae5356 Merge "Fix handling of "prebuilt_" prefix in classLoaderContextForUsesLibDeps." 2021-07-16 12:00:29 +00:00
Ulyana Trafimovich
9cb1577ac7 Merge "Refactor function to reduce nestedness level. No functional changes." 2021-07-16 09:37:00 +00:00
Jingwen Chen
f59a8e117e Fix apex bp2build.
apexBundle was missing InitBazelModule for it to have bazel_module properties.

Test: soong unit test
Fixes: 193866723
Change-Id: I5fb38c8b2e8edaecef44162491d399c0223d3fb1
2021-07-16 09:32:49 +00:00
Jingwen Chen
6dcca41d76 Soong OWNERS: add more folks and categorize by timezones.
Test: N/A
Change-Id: Ifc16654d1a295fe341b982319101be9c8c170113
2021-07-16 07:25:00 +00:00
Colin Cross
51736b82fc Merge "Rename ninja variables with Clang in the name" 2021-07-16 00:28:13 +00:00
Colin Cross
908fbe200e Merge changes I29bf144b,Id2879453
* changes:
  Remove more clang indirection without affecting command lines
  Remove clang indirection without affecting build.ninja
2021-07-16 00:00:22 +00:00
Liz Kammer
c07b16f49c Merge "Add test to reproduce bug in error message." 2021-07-15 20:37:52 +00:00
Ivan Lozano
3ee74c8252 rust: Remove libtest from stdlibs list
libtest does not need to be linked for every module type, so remove it
from the stdlibs list. Instead, link it only when building benchmarks or
tests.

Bug: 193782599
Test: cd external/rust/crates/; mma
Test: Rust tests still run correctly.
Change-Id: I536be8754da0987e09340744d9ebf668b8e734d0
2021-07-15 15:45:31 -04:00
Martin Stjernholm
9fe17bc8d9 Merge "Document how APEXes are identified in the boot jar variables." 2021-07-15 18:57:00 +00:00
Liz Kammer
0fc1e134b9 Add test to reproduce bug in error message.
soong config variable modules allow specifying a non-existent property,
which results in an unhelpful error message when the module type is then
used.

Bug: 171232169
Test: go test soong tests
Change-Id: I6174c0d35a28952157ee925f51d615e2ee735f8a
2021-07-15 12:38:35 -04:00
Paul Duffin
9e3e9c905b Merge "Support fully qualified names in android:"path" properties" 2021-07-15 16:02:34 +00:00
Paul Duffin
d2a23518b4 Merge "Build updatable-bcp-packages.txt from prebuilts in sdk snapshot" 2021-07-15 14:58:12 +00:00
Paul Duffin
81d56dcd3f Merge "Propagate permitted packages to sdk snapshot" 2021-07-15 14:58:03 +00:00
Ulya Trafimovich
e14f80b2d3 Fix handling of "prebuilt_" prefix in classLoaderContextForUsesLibDeps.
Changes:
- Cut off "prebuilt_" prefix from `dep`, not `libName`, because `dep` is
  used to replace library name in `usesLibraryProperties`, and these
  properties are specified without "prebuilt_" prefix.

- Do not remove "prebuilt_" prefix on the return value of
  `ProvidesUsesLib()`: this boils down to `provides_uses_lib` property
  which is specified without the prefix.

Bug: 193425964
Test: $ lunch aosp_cf_x86_64_phone-userdebug && m && launch_cvd
      $ adb wait-for-device && \
        adb root && \
        adb logcat | \
        grep -E 'ClassLoaderContext [a-z ]+ mismatch' -C1
      # empty output, no errors
Change-Id: I6cb65040027023d12ca0bc132d60c2c245174c40
2021-07-15 15:19:26 +01:00
Ulya Trafimovich
2444671b4a Refactor function to reduce nestedness level. No functional changes.
Bug: 193425964
Test: m nothing
Change-Id: Ic2d20eb5317f397919ecb01b64c5dd0da78e4a1a
2021-07-15 15:18:25 +01:00
satayev
1180b1fc4a Merge "Update a comment about PRODUCT_UPDATABLE_BOOT_JARS." 2021-07-15 14:01:38 +00:00
Paul Duffin
630b11e9d4 Build updatable-bcp-packages.txt from prebuilts in sdk snapshot
Previously, the java_import and java_sdk_library_import modules did not
make their permitted_packages available to the rule that generates the
updatable-bcp-packages.txt file. This change corrects that.

Bug: 193763688
Test: m nothing
      - Added unit tests, which all failed and then fixed the tests.
Change-Id: If0706e4551a331b48d383123088e63924dded48b
2021-07-15 14:16:44 +01:00
Paul Duffin
869de147ab Propagate permitted packages to sdk snapshot
Previously, permitted_packages were not copied to the sdk snapshot.
This change corrects that.

Bug: 193763688
Test: m nothing
      - Added unit tests, which all failed and then fixed the tests.
Change-Id: I4560987f746f78c0ae706058195b6db4bea438aa
2021-07-15 14:15:58 +01:00
Liz Kammer
da29c65eb6 Merge "Include bazel-status of build in metrics" 2021-07-15 12:58:04 +00:00
satayev
f208a5766e Update a comment about PRODUCT_UPDATABLE_BOOT_JARS.
Bug: 191127295
Test: n/a
Change-Id: I3d8e8689a79be79714b01cfd8abac34e7d963aee
2021-07-15 13:22:31 +01:00
Treehugger Robot
f6ecb09322 Merge "Update comment text of SrcIsModuleWithTag()" 2021-07-15 11:24:19 +00:00
Ulyana Trafimovich
bd14f9a952 Merge "Drop "prebuilt_" prefix when adding libraries to class loader context." 2021-07-15 10:15:44 +00:00
Ulyana Trafimovich
7beb099546 Merge "Drop strings.HasPrefix when guarding strings.TrimPrefix." 2021-07-15 10:15:26 +00:00
Yi-Yo Chiang
ba9ea32f73 Update comment text of SrcIsModuleWithTag()
The actual behavior of the method is slightly different from its text
right now. Update the text to describe the actual behavior.

Bug: 193228441
Test: m nothing (change is noop)
Change-Id: I755829af6aa1b3f95ac71f33d75751b94e71b54b
2021-07-15 17:18:21 +08:00
Paul Duffin
4e444dab84 Merge "Support generating sdk snapshots that can be selected by Soong config vars" 2021-07-15 07:59:06 +00:00
Ulya Trafimovich
69c1aa94ac Drop "prebuilt_" prefix when adding libraries to class loader context.
This is needed when some source libraries get replaced with prebuilt
variants: CLC should contain one entry for a library (the prebuilt one
or the source one, whichever of them is preferred). Because the prebuilt
module name starts with "prebuilt_" prefix, previously Soong considered
such libraries as two different ones libraries and added both to CLC.

Bug: 193425964

Test: mark "prebuilt_android.net.ipsec.ike" as preferred and `m nothing`
      (before this CL it would fail the build, now it builds fine).

Test: $ lunch aosp_cf_x86_64_phone-userdebug && m && launch_cvd
      $ adb wait-for-device && \
        adb root && \
        adb logcat | \
        grep -E 'ClassLoaderContext [a-z ]+ mismatch' -C 1
      # empty output, no errors
Change-Id: Icc42533d9915060d7fffda12aa93b9d18dc4f83d
2021-07-15 08:30:27 +01:00
Ulya Trafimovich
497a09310b Drop strings.HasPrefix when guarding strings.TrimPrefix.
There is no need to check as `TrimPrefix` does nothing if the prefix
doesn't match (see https://pkg.go.dev/strings#TrimPrefix).

Bug: 193425964
Test: m nothing
Change-Id: Ieb82e72a471800e29eb6bb54308041a9ab4173e5
2021-07-15 08:30:27 +01:00
Colin Cross
9cc59c1cbd Remove clang indirection, affects command line ordering
Move the last clang-specific list into the common global list.  This
affects the command line flag ordering due to the the -fdebug-prefix-map=
flag that is added by an init() function, which will force every native
rule to recompile.

Bug: 68947919
Test: m checkbuild
Change-Id: Ic2509e61e9555e9483b92a18d7e8d9913b7125cc
2021-07-14 20:55:25 -07:00
Colin Cross
0523ba2675 Rename ninja variables with Clang in the name
Rename the ninja variables that have an extraneous Clang in the name,
since there are no longer any non-Clang variables.

Bug: 68947919
Test: no change to command lines used to build mmma bionic
Change-Id: I081d5e1d423a66d55faece6cb1d265cd5e504d5f
2021-07-14 20:55:23 -07:00
Colin Cross
c8bed31f56 Remove more clang indirection without affecting command lines
Squash the contents of some clang-specific ninja variables into
the main ninja variables.

Bug: 68947919
Test: no change to command lines used to build mmma bionic
Change-Id: I29bf144b03307a7893376e68d82f12f5c40e8e95
2021-07-14 20:55:02 -07:00
Colin Cross
33bac24bb9 Remove clang indirection without affecting build.ninja
Remove cflags that are not handled by clang from the global defaults,
and remove calls to ClangFilterUnknownCflags.

Squash lists of clang-specific default flags into the main default
flags lists.

Rename Toolchain.Clang*flags to Toolchain.*flags.

Rename Go variables with Clang in the name that have no non-Clang
equivalent.

Remove unused ninja variables.

Bug: 68947919
Test: no change to build.ninja for aosp_cf_x86_64_phone-userdebug or aosp_crosshatch-userdebug
Change-Id: Id287945315d53e5eaef197adbbb4f1302f2e3680
2021-07-14 20:54:22 -07:00
Treehugger Robot
75d719fdd7 Merge "dexpreopt.config should be created even though unbundled image is built" 2021-07-15 03:41:30 +00:00
Jeongik Cha
4b073cd083 dexpreopt.config should be created even though unbundled image is built
Bug: 188179858
Test: compare dexpreopt_config.zip files from
 1. TARGET_BUILD_UNBUNDLED_IMAGE=true m dexpreopt_config_zip
 2. m dexpreopt_config_zip
(note that m clean should run between steps)

Change-Id: I36a6e8b10b9922cc5522accaf90af1aa05049a86
2021-07-15 09:57:49 +09:00
Paul Duffin
40131a3f9e Support fully qualified names in android:"path" properties
Previously, a module reference in a path property would be parsed into
two parts, the module name and the optional output tag, which defaults
to an empty string if not specified. The output tag would be stored in
a sourceOrOutputDependencyTag which would then be used, along with the
module name to add a dependency on the module.

Later, when the paths were processed the same module reference would be
parsed into the same two parts again and the module name used to find a
matching Module by comparing it against the name returned by either
Module.Name(), ctx.OtherModuleName() or ModuleBase.BaseModuleName().
Once the module had been found then if it supported OutputFilesProducer
then the tag would be passed to its OutputFiles(tag) method. Otherwise,
it would fall back to treating it as SourceFilesProducer.

The problem with that is the module name retrieved from the module in
some way (either directly or through a context name) could be different
to that originally supplied when adding the dependency. e.g.
1. If the original dependency was added onto a source module but there
   existed a suitable and preferred prebuilt module then the dependency
   onto the source module would have been replaced by the prebuilt
   module which has a different name.
2. If the path property included a fully qualified name that included
   a qualifying path then it would not match the name retrieved from
   the module which would not include the qualifying path.

This change circumvents that whole issue by adding the module name that
was originally used to add the dependency into the DependencyTag. Now
the DependencyTag uniquely identifies the original module/outputTag
pair parsed from the module reference. The pathDepsMutator guarantees
that they are unique as it dedups them before adding the dependencies.

It is possible that calling ExtractSource(s)Deps() would add some
duplicate but if they did they would be identical, i.e. the same
sourceOrOutputDependencyTag would be used to add a dependency onto the
exact same module. In that case it would not matter which of the
dependencies was found as it would still return the same module.

Bug: 193228441
Test: m nothing
Change-Id: I661514a2984818e5c26577411cede53eb57bcd02
2021-07-14 23:58:49 +01:00
Paul Duffin
fb9a7f99be Support generating sdk snapshots that can be selected by Soong config vars
Allows the builder of an sdk snapshot to specify a Soong config
variable that can be used to control whether the snapshot's prebuilts
are used or not.

Bug: 193523070
Test: m nothing
Change-Id: Ib09500ba61befc1202dff61dc06847c730ba9253
2021-07-14 23:19:53 +01:00
Liz Kammer
ca9cb2e5bd Include bazel-status of build in metrics
Currently within metrics we cannot distinguish between Bazel-involved
configurations:
  Bazel as Ninja vs Ninja as Ninja builds
  Bazel mixed builds vs non-mixed builds

Add these to the build configuration in order to distinguish between and
track them independently.

Test: go soong tests
Test: USE_BAZEL_ANALYSIS=1 m nothing and verify out/soong_metrics
Bug: 193672397
Change-Id: I94ce56c1a98a76d929598d9fe3070df16ffa1376
2021-07-14 17:48:04 -04:00
Paul Duffin
e88944c51e Merge "Choose prebuilt or source via an Soong config variable" 2021-07-14 20:23:34 +00:00
Paul Duffin
0c52c7ba63 Choose prebuilt or source via an Soong config variable
A simple way to provide fine grained control over the use of a prebuilt
or source module via a Soong config variable.

Bug: 193523070
Test: m nothing
Change-Id: I47ae8ac04fa29156d2e87efd9e60ab995f50ea6d
2021-07-14 20:23:06 +00:00
Treehugger Robot
e04058f291 Merge "Add ClassLoaderContextMap.Dump()" 2021-07-14 12:57:56 +00:00