Commit graph

105 commits

Author SHA1 Message Date
Cole Faust
256cfbee24 Remove starlark_import
This is no longer used since the roboleaf cancellation.

Bug: 315353489
Test: m nothing --no-skip-soong-tests
Change-Id: Ie6ee093c2810498306ea4a2288050eed17a35357
2024-03-07 10:53:41 -08:00
Cole Faust
5a231bd819 Select statements
See the blueprint cl for more information.

Things still to do:
 - Support selecting on product variables and
   variants
 - Test/Support property struct reflection tags
   like arch_variant, path, and variant_prepend
 - Test that selects combine well with existing
   configurability mechanisms like arch:, target:,
   multilib:, python's version:, etc.

Bug: 323382414
Test: go tests
Change-Id: If5d1ea1ad0c4ebabffaea91d62e1a1c6f926a793
2024-03-06 16:03:01 -08:00
Colin Cross
c50999eb0a Merge changes from topic "transition-mutator-config" into main
* changes:
  Add ArchModuleContext to TransitionMutator contexts
  Split ArchMutatorContext out of BaseMutatorContext
2024-01-24 20:03:46 +00:00
Colin Cross
1d3d9f13b8 Split ArchMutatorContext out of BaseMutatorContext
Split the context methods that are useful on anything visiting a
module that has arch variants into a separate ArchMutatorContext
for reuse by TranstitionMutators.

Bug: 319288033
Test: builds
Change-Id: Ifdc21983c6c79f22965a49f169812a8cc3ad975b
2024-01-24 10:17:10 -08:00
Ronald Braunstein
73b08ffd0d Add team property to all modules.
This allows vendors (like google) to specify which team owns the test
module and code.

Team is a commonProperty on modules and points to the designate "team"
module.  The DepsMutator adds the dependency on the "team" module and
"GenerateBuildActions" write the team data to intermediate files.

A new singleton rule, all_teams visits all modules and writes out
the proto containing the team for each module.
If a module doesn't have a team, then it finds the package in the
blueprint file and parent directory blueprint files that have a
default_team and uses that team.

Test: m all_teams
Test: go test ./python ./java ./cc ./rust ./android
Test: added team to HelloWorldHostTest and built the new asciiproto target
Test: added package default_team and checkout output proto.
Change-Id: I5c07bf489de460a04fc540f5fff0394f39f574a7
2024-01-18 14:16:06 -08:00
Colin Cross
3cc3154d33 Remove more bp2build
Bug: 315353489
Test: m blueprint_tests
Change-Id: Ib854fe1a448c258fe086691a6e5ed2d98537f5e4
2024-01-05 09:54:09 -08:00
Colin Cross
138d37dd6c Merge "Support generating module_info.json in Soong" into main 2024-01-03 17:55:47 +00:00
Colin Cross
d6fd013394 Support generating module_info.json in Soong
Generate module_info.json for some Soong modules in Soong in order to
pass fewer properties to Kati, which can prevent Kati reanalysis when
some Android.bp changes are made.

Soong modules can export a ModuleInfoJSONProvider containing the
data that should be included in module-info.json.  During the androidmk
singleton the providers are collected and written to a single JSON
file.  Make then merges the Soong modules into its own modules.

For now, to keep the result as similar as possible to the
module-info.json currently being generated by Make, only modules that
are exported to Make are written to the Soong module-info.json.

Bug: 309006256
Test: Compare module-info.json
Change-Id: I996520eb48e04743d43ac11c9aba0f3ada7745de
2024-01-02 16:03:43 -08:00
Colin Cross
a3759f98a6 Merge "Write raw files to disk instead of the ninja file" into main 2024-01-02 19:44:52 +00:00
Colin Cross
31a674571e Write raw files to disk instead of the ninja file
Writing raw files as rules in the ninja file unnecessarily bloats
the ninja file.  Write files immediately to disk instead to files
based on the hash of the contents, and then emit ninja rules to
copy the files into place during the build.  Delete obsolete files
in a singleton at the end of analysis.

Bug: 306029038
Test: Run: m libc_musl_version.h
           touch build/soong/Android.bp
           m libc_musl_version.h
      libc_musl_version.h/genrule.sbox.textproto is not recopied.
Test: Run: lunch aosp_cf_x86_64_phone-userdebug
           m libc_musl_version.h
	   lunch aosp_x86_64-userdebug
	   m libc_musl_version.h
	   lunch aosp_cf_x86_64_phone-userdebug
	   m libc_musl_version.h
      libc_musl_version.h/genrule.sbox.textproto is recopied but restat prevents rerunning the genrule.
Test: Run: touch out/soong/raw-aosp_cf_x86_64_phone/00/foo
           touch build/soong/Android.bp
	   m nothing
      out/soong/raw-aosp_cf_x86_64_phone/00/foo is removed.
Change-Id: I172869c4d49565504794c051e2e8c1f7cf46486e
2023-12-19 16:33:46 -08:00
LaMont Jones
aa005ae080 move CollectDependencyAconfigFiles to android
This needs to be called by some modules in android.

Bug: 308625757
Test: manual
Change-Id: I389fcfd88a3f4bd85a9218fdd4dd66d8a239bb67
2023-12-19 19:34:00 +00:00
Colin Cross
3c0a83d19f Use generics for providers API
Using generics for the providers API allows a type to be associated
with a ProviderKey, resulting in a type-safe API without that doesn't
require runtime type assertions by every caller.

Unfortunately, Go does not allow generic types in methods, only in
functions [1].  This prevents a type-safe API on ModuleContext, and
requires moving the API to be functions that take a ModuleContext as
a parameter.

This CL creates the new API, but doesn't convert all of the callers.

[1] https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md#no-parameterized-methods)

Bug: 316410648
Test: builds
Change-Id: I3e30d68b966b730efd968166a38a25cc144bd6de
2023-12-14 16:12:20 -08:00
Colin Cross
b63d7b3af7 Remove infrastructure to run bp2build
Bug: 315353489
Test: m blueprint_tests
Change-Id: Idcf6377d389b94c39e4e6ff4b8efa8a9f9e78b17
2023-12-08 13:51:07 -08:00
Colin Cross
69452e14a3 Split *ModuleContext out of module.go
Move EarlyModuleContext, BaseModuleContext and ModuleContext out of
module.go and into early_module_context, base_module_context and
module_context.go respectively.

Test: builds
Change-Id: I52e6eb1589d1478233c1c55d770b395a16eaa1a3
2023-11-15 11:40:49 -08:00
Spandan Das
0d53dd289e Create an apex_contributions module type
This module type does not any build actions, but provides metadata about
the contributions of the mainline module to other mainline modules/
platform. One immediate use case for this metadata is source vs
prebuilts selection

Each mainline module can have more than 1 metadata modules. Static build
flags will be used to select the right one in trunk.bzl, next.bzl, ...

Bug: 308174923
Test: go build ./android
Change-Id: I5ead0c38fe64b78de08db1736d97a8ab7374d6d9
2023-11-03 22:33:02 +00:00
Jiakai Zhang
4d90da29a8 Add a Make variable that overrides configured jar locations.
We had hardcoded logic to override "framework-minus-apex", but we need
to override more jars, so this CL adds a general solution for this.

Bug: 290583827
Test: m nothing
Change-Id: I211ebda7aa2803886ab6e6d081d26327665e49a6
2023-07-14 12:13:30 +01:00
Colin Cross
313d32f730 Merge "Use generics for DepSets" 2023-06-22 20:57:01 +00:00
Colin Cross
c85750bfe3 Use generics for DepSets
Use Go's generics for DepSets so they don't require a type-specific
wrapper and reflection.

Test: depsets_test.go
Change-Id: I22ba0b7d680d37d2cd05230b0f560d166c4dd20b
2023-06-22 08:18:33 -07:00
Chris Parsons
39a169721c Provide reason for unconverted bp2build modules
This also changes the expectation of ConvertWithBp2build. Each
implementation must either create one or more Bazel target modules, or
mark the module as unconvertible (with a specific reason).

Manually verified no runtime hit in AOSP
In AOSP, the metrics file size increases from 252K to 1.6M

This changes some effective module counts in bp2build metrics:
 - Removes "package" modules from the module count list in
metrics, as these will not be converted like regular modules.
 - Counts Handcrafted modules as being "unconverted", as bp2build is not
   responsible for them.

Bug: 285631638
Test: Verified generated BUILD.bazel files are bit-for-bit identical
with this change
Test: Manually verified one case of each implemented reasonType

Change-Id: I308dd451d8f28379b15671dae9f931bd0446f5c1
2023-06-16 13:45:17 +00:00
Liz Kammer
33cc80eb1f Revert "Revert "Restrict plugins to an existing allowlist""
This reverts commit b20a12553e.

Reason for revert: Rollforward with fix

soong-ca-certificates-apex was missing from allowlist

Test: verify fixes CI breakages
(cherry picked from https://googleplex-android-review.googlesource.com/q/commit:dfe4a9e4dd088f42ba5e0a3c4f7777c253d378d3)
Merged-In: I00687bd2962dbdaaf55189ca4aa82183264eb8d7
Change-Id: I00687bd2962dbdaaf55189ca4aa82183264eb8d7
2023-05-18 18:20:28 +00:00
Cole Faust
3e0836e865 Merge "Load starlark files from soong" 2023-04-27 17:15:02 +00:00
Cole Faust
c9508aac4c Load starlark files from soong
There are a number of instances where we are exporting information
from soong to bazel via soong_injection. This could be more bazel-centric
if the information was instead held in bzl files, and both bazel and
soong read it from there.

Add a starlark package that will run
//build/bazel/constants_exported_to_soong.bzl at initialization time,
and then results can be retreived with GetStarlarkValue.

Since changes to the starlark files mean that soong has to rerun,
add them as ninja deps.

Unfortunately, the starlark code has to be run at runtime rather than
pregenerating their results, because tests run from intellij wouldn't
go through any pregeneration steps. This means that starlark is run
multiple times during the build, once per test package and once per
primary builder invocation. (currently 3, could be reduced to 2 if we
made the symlink forest generation into its own standalone tool) The
starlark code we have so far in this cl is very fast, roughly half a
millisecond, so it's not a big deal for now, but something to keep an
eye on as we add more starlark constants.

Bug: 279095899
Test: go test
Change-Id: I1e7ca1df1d8d67333cbfc46e8396e229820e4476
2023-04-26 17:18:19 -07:00
Jooyung Han
a2b57abb4a Remove makefile_goal
This was added to use 'make' target in soong. It was a hacky way and
caused many problems:
- can't track dependency
- doesn't work with sanitizer
- VSDK snapshot doesn't support
- ..

Now references are all gone. Let's remove it.

Bug: n/a
Test: m
Change-Id: I0336a490780f37a4e77d343c61315686e23db643
2023-04-26 16:46:03 +09:00
Anton Hansson
7a444cec60 Add filegroup_test.go to sources
I don't think it was running before.

Test: yes
Change-Id: Ia3bf53ddcc1ddefd7a236122256594ba764ee8e5
2023-03-13 15:24:17 +00:00
Spandan Das
81593891ee Create api_domain module type and its bp2build converter
This module does not have any build actions in Soong yet. It will be
converted to a Bazel target using bp2build, and API export will be run
on the generated Bazel target

Test: go test ./bp2build
Test: TH
Change-Id: I76df32bd9f23b60e746b2700af07668a2ed74844
2022-09-21 19:49:46 +00:00
Liz Kammer
5480f9414a Merge changes I8a99e3d2,I717d7f5e
* changes:
  Remove unused bp2build logic from arch.go
  Split up config.go
2022-08-12 20:39:59 +00:00
Liz Kammer
7ec40ccb15 Split up config.go
Test: go test soong tests
Change-Id: I717d7f5eec4ce33f71c23ab613a26c33b2b8f203
2022-08-01 16:07:30 -04:00
Alexei Nicoara
7579b67ef9 Moving default version outside of apex package
The default version needs to be used by both apex and apk updatable
modules. Moving it to android package so it can be accessed without
circular dependencies.

Test: presubmit
Bug: 231691162

Change-Id: Icafd50d0b065e89a115ae9c9b6ea55adf2730b49
Merged-In: Icafd50d0b065e89a115ae9c9b6ea55adf2730b49
2022-08-01 16:28:48 +01:00
Cole Faust
9a06d25be2 Make filegroup paths relative to module dir in mixed builds
filegroup.srcs must have a .rel() version of their paths
that is relative to the module directory. Previously in
mixed builds, the relative path wasn't filled in.

Bug: 229251008
Bug: 204244290
Test: go tests, and USE_BAZEL_ANALYSIS=1 m apexer with the following cl
Change-Id: I11a2a8b96139e0e7b956fa4c256a9a99c02f88f3
2022-06-08 11:47:24 -07:00
Bob Badour
afd99fd305 Add gen_notice test.
Test: m cts dist reportmissinglicenses
Change-Id: I1d630c3d14d27b7605ab13d204d34b6851a24d33
2022-06-01 17:57:00 -07:00
Bob Badour
eef4c1c563 Add gen_notice module.
Refactor notices to support notices for multiple modules.

Enforce visibility and handle missing dependencies.

Bug: 213388645

Change-Id: Id6a81987f087419ad37d0cce57a71e8a7c4cd6e0
2022-05-18 16:38:19 -07:00
Inseob Kim
4f1f3d97ca Add buildinfo_prop module
buildinfo_prop module is a replacement for build/make/tools/buildinfo.sh
so other images like microdroid can refer to build.prop.

For now, buildinfo_prop only supports a few build.prop properties, and
it's only used in microdroid.

Bug: 189164487
Test: build
Change-Id: I120654ca23a68de414df8da2051c6677afbab441
2022-04-27 14:10:52 +09:00
Sam Delmerico
24c5603815 refactor bp2build module allowlists into struct
To make testing easier, refactor existing module-global variables into a
struct that can be mocked.

Test: build/bazel/bp2build.go
Change-Id: I9d177677644ea743641a745b1839a3a8b29f902a
2022-04-14 13:31:05 +00:00
Sam Delmerico
7f88956c16 refactor Bazel variable export
Most of the variable export code for cc modules can be re-used for
exporting variables for java modules. Refactor this code into a more
composable structure for reuse.

Test: build/bazel/bp2build.sh
Test: manual comparison of
  out/soong/soong_injection/cc_toolchain/constants.bzl
  with previous output
Change-Id: Ie5a6fee08cc888b7dc69c3e324e5c3f8aa269a8f
2022-04-06 14:48:32 +00:00
Chris Parsons
715b08fd82 Add metrics collection to soong_build
Metrics are added to a materialized protobuf soong_build_metrics.pb

Recorded events have scoped names and recorded runtimes.
For example, mixedbuilds includes the following events:

mixedbuilds.prepare
mixedbuilds.analyze.write_files
mixedbuild     (this is a total of all mixedbuild events)

Test: Manually verified materialized protos for mixedbuild, bp2build,
and legacy build.

Change-Id: I2e72d4502c0373bee0279bb1320b2e4dd586129c
2022-03-25 17:19:44 -04:00
Liz Kammer
72beb34609 Add package for printing starlark formatted data
Bug: 216168792
Test: build/bazel/ci/bp2build.sh
Change-Id: I3a06b19396f7ffe1c638042cda7e731dd840f1d6
2022-02-08 17:32:28 -05:00
Colin Cross
4acaea9cbf Revert^2 "Build license metadata files in Soong"
b3264f2090

Change-Id: I5ea5f7e29eb1dbc72b8b30f010006a25f5f72fac
2021-12-10 23:05:02 +00:00
Colin Cross
b3264f2090 Revert "Build license metadata files in Soong"
Revert submission 1899703

Reason for revert: b/210158272
Reverted Changes:
I142c19284:Reuse license metadata files from Soong
Ife89b8f23:Add dependency license annotations
I8e7410837:Build license metadata files in Soong

Change-Id: I48f2bf2e2797ac9d38f7ece35ba616b823035607
2021-12-10 23:02:57 +00:00
Colin Cross
3dd2ff28ed Build license metadata files in Soong
Soong has enough information to build the license metadata files
without resorting to the fixups required in Make.

Bug: 207445310
Test: m checkbuild
Change-Id: I8e74108376162b8fdb87ba098ebe94350aa1f7c4
2021-12-08 15:08:59 -08:00
satayev
0ee2f913ef Test SdkSpecForm.
Bug: 190818041
Test: presubmit
Change-Id: Ib8cd891f03537712d709ed063dd76dee55221118
2021-12-01 17:43:00 +00:00
Colin Cross
6ac9576eef Add tests for ctx.InstallFile
Add tests that cover Soong-only installation as well as installation with
InstallBypassMake.

Bug: 204136549
Test: TestInstall
Test: TestInstallBypassMake
Change-Id: Iac22c9fdf99994e06b419623ee5fa399ef6957fb
2021-11-09 17:25:57 -08:00
Paul Duffin
581f2e5f79 Detect duplicates in sdkRegistry
Bug: 195754365
Test: m nothing
Change-Id: I67c5022b7cc61891fd6b90365f8271d97d7bcd98
2021-09-23 11:37:57 +01:00
Lukacs T. Berki
c6012f36e1 Invoke soong_docs from the bootstrap Ninja file.
This makes soong_ui the only place where soong_build is invoked, thus
greatly simplifying the conceptual model of the build.

It comes with the slight limitation that now soong_docs (and queryview
and the JSON module graph) are not Make targets anymore, but I suppose
that's an acceptable loss.

The only place where someone depended on soong_docs from a Makefile is
removed in a separate change.

Test: Presubmits.
Change-Id: I3f9ac327725c15d84de725d05e3cdde1da3dcbe2
2021-09-08 09:31:52 +02:00
Lukacs T. Berki
3a82169a55 Invoke queryview from the bootstrap Ninja file.
It used to be invoked from out/soong/build.ninja, which required two
soong_build invocations one after the other (ne to generate
out/soong/build.ninja, one to generate the queryview workspace). This
was slower and required some shell-quoted-in-ninja-quoted-in-Go .

Test: Presubmits.
Change-Id: Idda79c067606663b66e9f94626fa24f3b5af4114
2021-09-06 17:10:49 +02:00
Dan Willemsen
4591b6496d Upgrade to golang protobuf api v2
Major operations:
 * Fix the go_package entries to be consistent, as the new tool
   complains when it doesn't have a '/'.
 * Regenerate with the new protoc-gen-go tool.
 * github.com/golang/protobuf -> google.golang.org/protobuf
 * proto.[Un]MarshalText -> prototext.[Un]Marshal

Change-Id: Ie1147bd2457fafb66ba555461b3bf14f0561a25d
2021-07-23 11:12:05 -07:00
Paul Duffin
b0bb376efa Add license modules to the sdk
Adds initial support for adding license modules to the sdk, along with
any referenced license text files. There is a number of minor
improvements to be made but the core of the support is there and it
works for ART sdks.

Basically, this change will automatically add license modules
referenced from any sdk member module as an internal sdk member. An
internal module has an sdk snapshot specific name that will not
conflict with the source module and which cannot be referenced from
outside the sdk snapshot.

Bug: 181569894
Test: m art-module-sdk art-module-host-exports art-module-test-exports
      - diff output before and after this change
        made sure that every prebuilt had a licenses field
        and that the license modules were added
Change-Id: I0c5ccabf58f4ef487e42ef8e61a5b2a74c0e81af
2021-05-11 08:28:49 +01:00
Rupert Shuttleworth
b7e30760af Add various test files as testSrcs, so that they run on CI.
Test: go test
Test: TH (now that these are hooked up to TH)

Change-Id: Ia199794ef1a64074f3f9159f8c160ba3e45fd181
2021-04-21 11:23:57 -04:00
Liz Kammer
620dea6720 Split bazel -path functions and clarify docs
Test: go test
Change-Id: I62f58998fc7d52c67ed5acfdb8230d404b2a5472
2021-04-15 13:00:42 -04:00
Lukacs T. Berki
53b2f36bd6 Delete android/env.go .
Properly plumb the available environment to the configuration instead of
going through the global variable originalEnv.

Test: Presubmit.
Change-Id: Ia1ea753d5e72c02a0dcaa4c0b43bd6e24fc47cec
2021-04-13 08:10:01 +02:00
Jiyong Park
f1691d2a2c Move java.sdkSpec to the android package
... in preparation for making the handling of sdk versions consistent
across java and cc modules.

Bug: 175678607
Test: m
Change-Id: I598f0454bce9b7320621022115412fbe97403945
2021-04-03 08:25:12 +09:00