Commit graph

188 commits

Author SHA1 Message Date
Lukacs T. Berki
9f6c24a887 Rename BuildDir and NinjaBuildDir.
These are just out/ and out/soong/ and the old names were quite
confusing.

Test: Presubmits.
Merged-In: I999ca62162062f27e449f83ffb00fa580d4065b8
Merged-In: I2fcd80a9e376a6842a06aa104a9f6f5c5e16f8c7
Change-Id: Ib481d2aac40df8da1b74174dc8ec576c1cb48be0
2021-08-27 10:08:49 +02:00
Lukacs T. Berki
f7e36d80ff Remove Srcdir() from BootstrapConfig.
It was always ".".

Test: Presubmits.
Change-Id: I5381002b3f2986122f1b335a88119cead0a86d75
2021-08-16 17:05:09 +02:00
Chris Wailes
b2703adce4 Conditionally apply rustdoc flags to third party crates
This CL changes the logic in rust/builder.go so that some rustdoc flags
are only applied to external crates.  This will allow us to since
warnings and deal with soft-failures in external crates while allowing
us to be more strict with our internal Rust code.

Bug: 195136952
Test: m rustdoc
Change-Id: Icdde304bbbb323cae9657e8f842f58ae79e811ce
2021-08-02 11:50:34 -07:00
Colin Cross
528d67e523 Reland "Split the x86 host toolchain into glibc and musl variants"
Split the x86 host toolchain into glibc and musl variants

Create new musl toolchains that are based on the existing glibc
toolchains, and add the necessary flags for musl compiles.

This relands Ifc02f9e5afa61ff758be98b0c962f3a4b53d0546 with changes
for I46672e3a096b6ea94ff4c10e1c31e8fd010a163c.

Bug: 190084016
Change-Id: Iaa9f7a50ff601155ecd73acc5701a2c226be66dc
Test: TestArchMutator
2021-07-23 22:25:36 +00:00
Colin Cross
5d6904e1cf Revert "Split the x86 host toolchain into glibc and musl variants"
This reverts commit 4fb4ef2242.

Reason for revert: crossed with aosp/1775072

Change-Id: Ied922850b810d82627ba4f9ee42f672cfe286c91
2021-07-23 22:20:29 +00:00
Colin Cross
4fb4ef2242 Split the x86 host toolchain into glibc and musl variants
Create new musl toolchains that are based on the existing glibc
toolchains, and add the necessary flags for musl compiles.

Bug: 190084016
Test: TestArchMutator
Change-Id: Ifc02f9e5afa61ff758be98b0c962f3a4b53d0546
2021-07-23 08:57:17 -07:00
Paul Duffin
40131a3f9e Support fully qualified names in android:"path" properties
Previously, a module reference in a path property would be parsed into
two parts, the module name and the optional output tag, which defaults
to an empty string if not specified. The output tag would be stored in
a sourceOrOutputDependencyTag which would then be used, along with the
module name to add a dependency on the module.

Later, when the paths were processed the same module reference would be
parsed into the same two parts again and the module name used to find a
matching Module by comparing it against the name returned by either
Module.Name(), ctx.OtherModuleName() or ModuleBase.BaseModuleName().
Once the module had been found then if it supported OutputFilesProducer
then the tag would be passed to its OutputFiles(tag) method. Otherwise,
it would fall back to treating it as SourceFilesProducer.

The problem with that is the module name retrieved from the module in
some way (either directly or through a context name) could be different
to that originally supplied when adding the dependency. e.g.
1. If the original dependency was added onto a source module but there
   existed a suitable and preferred prebuilt module then the dependency
   onto the source module would have been replaced by the prebuilt
   module which has a different name.
2. If the path property included a fully qualified name that included
   a qualifying path then it would not match the name retrieved from
   the module which would not include the qualifying path.

This change circumvents that whole issue by adding the module name that
was originally used to add the dependency into the DependencyTag. Now
the DependencyTag uniquely identifies the original module/outputTag
pair parsed from the module reference. The pathDepsMutator guarantees
that they are unique as it dedups them before adding the dependencies.

It is possible that calling ExtractSource(s)Deps() would add some
duplicate but if they did they would be identical, i.e. the same
sourceOrOutputDependencyTag would be used to add a dependency onto the
exact same module. In that case it would not matter which of the
dependencies was found as it would still return the same module.

Bug: 193228441
Test: m nothing
Change-Id: I661514a2984818e5c26577411cede53eb57bcd02
2021-07-14 23:58:49 +01:00
Paul Duffin
407501b82c Stop PathForModuleSrc from validating the paths unnecessarily
PathForModuleSrc calls validatePath in order to convert the supplied
path components into a single path. Unfortunately, that corrupts a
fully qualified module name. So, when given "//namespace:module" it
treats it as a path and replaces "//" with "/". That replacement is
done by a call to filepath.Join().

This change simply concatenates the path components together textually,
using the path separator, to avoid the corruption. That ensures that
a fully qualified module name is preserved and processed properly. If
the path components do not contain a module name then expandOneSrcPath
will call pathForModuleSrc which validates the path so it does not open
up a way to create an invalid path as the validation was unnecessary
anyway.

Bug: 193228441
Test: m nothing
Change-Id: I0bb66feac182b77ce96c8d5d5f17e28ea28d75ba
2021-07-12 20:15:06 +01:00
Paul Duffin
d5cf92e298 Support customizing behavior around sourceOrOutputDependencyTag
Previously, modules customized behavior around the handling of
sourceOrOutputDependencyTag by comparing them to android.SourceDepTag
and retrieving the module using something like this:
    ctx.GetDirectDepWithTag(m, android.SourceDepTag)

The problem with that is it does not allow an output tag to be
specified and does not handle fully qualified names properly.

This adds the following:
* IsSourceDepTag and IsSourceDepTagWithOutputTag to check whether a
  blueprint.DependencyTag is a sourceOrOutputDependencyTag. The latter
  also checks that it has the correct output tag.
* GetModuleFromPathDep(ctx, moduleName, outputTag) as a replacement for
  ctx.GetDirectDepWithTag(m, android.SourceDepTag).

Replaces usages of:
* t == SourceDepTag with IsSourceDepTagWithOutputTag(t, "")
* ctx.GetDirectDepWithTag(m, android.SourceDepTag) with
  GetModuleFromPathDep(ctx, m, "")

It also deprecates the following:
* android.SourcDepTag - as a follow up change needs to modify the
  sourceOrOutputDependencyTag will make this useless.
* ExpandSources, ExpandsSources - copies existing deprecated messages
  from the implementation to the interface so that they can be seen
  by users of that interface.

Bug: 193228441
Test: m nothing
Change-Id: I8c397232b8d7dc1f9702c04ad45ea7819d4631ae
2021-07-09 23:54:01 +01:00
Spandan Das
5d1b929f21 Port module_partition logic for RRO from Make to Soong
The default partition for RRO is "product/" in Make, but it was
"system/" in Soong. This CL ports the logic from Make to Soong

To implement this, a new function PathForModuleInPartitionInstall is
created that enables callers to provide the relevant partition

Bug: 158407753
Test: from build/soong, ran go test ./java
Change-Id: I05b02eae7fe57189aaad5109c26cccc5823518ef
2021-06-08 17:53:31 +00:00
Jingwen Chen
78257e5b6b Add a comment to absolutePath on why it's not exported.
Change-Id: Ib6595e8af3d845861d4c5988750e3c43280bb697
Test: TH
2021-05-21 02:37:00 +00:00
Liz Kammer
09f947d67e Dump bazel product config in Soong
Bug: 187862880
Test: build/bazel/ci/bp2build.sh
Test: build/soong/tests/bp2build_bazel_test.sh
Change-Id: I24b09baad973e25bec4476e1ea4a7692b72b7d20
2021-05-19 17:38:56 -04:00
Paul Duffin
ef08185951 Add support for converting OptionalPath to Paths
Appending a Path or Paths to a slice of Paths is simple but appending
an OptionalPath requires conditional logic which makes OptionalPaths
harder to use. This change makes it easy to append the embedded Path,
if any, to a slice of Paths.

Bug: 179354495
Test: m nothing
Change-Id: Ibf80a23043c846162e17c3a98b2590bca653b170
2021-05-13 12:40:08 +01:00
Colin Cross
57892ceafb Merge "Support blueprint_go_binary in PathForModuleSrc" 2021-05-06 23:29:22 +00:00
Colin Cross
0e446159bc Support blueprint_go_binary in PathForModuleSrc
Allow blueprint_go_binary to be used as test data by supporting it in
PathForModuleSrc.

Also make python use GetDirectDepsWithTag instead of VisitDirectDeps
so it doesn't fail on the dependency to a Blueprint (non-Android)
module.

Fixes: 186528269
Test: manual
Change-Id: Ibc1d7caf7bf1fa5db805c3b885a95df4baee0ecf
2021-05-03 13:35:32 -07:00
Inseob Kim
08758f08e9 Add debug ramdisk variant.
A module will be installed to /debug_ramdisk if debug_ramdisk is set to
true.

This is a reland of f84e9c05e2, with a fix
that removes /first_stage_ramdisk.

Bug: 184004542
Test: soong test
Change-Id: I739de63cfec6b0fec5a90f7c4741fc4d884d209c
2021-04-29 22:58:17 +09:00
Inseob Kim
aeb6bad073 Revert^3 "Add debug ramdisk variant"
836a8f3d61

Change-Id: If59529238e26a197bc33b10245b88f538f280ed0
2021-04-22 23:14:58 +00:00
Florian Mayer
836a8f3d61 Revert^2 "Add debug ramdisk variant"
78ea2f5eac

Change-Id: I6a6a2ee82e2807045364bab105ec03a1934b411f
2021-04-22 16:38:47 +00:00
Bowgo Tsai
78ea2f5eac Revert "Add debug ramdisk variant"
This reverts commit f84e9c05e2.

Because this breaks the usage of boot-debug.img and
vendor_boot-debug.img

Bug: 185970130
Bug: 185990198
Test: make bootimage_debug
Change-Id: I7886c971982faae1d5bc34688643de8d94d6e201
2021-04-22 09:49:24 +08:00
Liz Kammer
620dea6720 Split bazel -path functions and clarify docs
Test: go test
Change-Id: I62f58998fc7d52c67ed5acfdb8230d404b2a5472
2021-04-15 13:00:42 -04:00
Treehugger Robot
b06a4bd400 Merge "Transform paths to headers in include dirs to take package boundaries into account." 2021-04-15 05:42:40 +00:00
Inseob Kim
f84e9c05e2 Add debug ramdisk variant
A module will be installed to debug_ramdisk (or
debug_ramdisk/first_stage_ramdisk if recovery as boot is true) if
debug_ramdisk is set to true.

Bug: 184004542
Test: soong test
Change-Id: Ic5a4d27407e506fffa462de2149e0785f11b2ac7
2021-04-15 00:56:30 +00:00
Rupert Shuttleworth
c143cc50dd Transform paths to headers in include dirs to take package boundaries into account.
This allows the following cc_library_static targets to build with bp2build:

- libc_freebsd
- libc_freebsd_large_stack
- libc_openbsd_ndk
- libc_gdtoa  (* but still fails for mixed builds)
- libc_aeabi
- libc_static_dispatch
- libc_dynamic_dispatch

This also allows a number of other cc_library_static targets to progress further in their builds.

Test: Added unit test
Test: bp2build-sync.py write; bazel build //bionic/...

Change-Id: I71742565c16594448a41a6428a5c993171ec4cb4
2021-04-14 17:05:36 -04:00
Colin Cross
66d154e436 Merge "Follow blueprint change to return GlobResult from Glob" 2021-04-13 17:01:39 +00:00
Colin Cross
82ea3fb273 Follow blueprint change to return GlobResult from Glob
Follow I2159cc9d85f388073198eac7456e5bf43e813096 that makes Glob
return a GlobResult.

Bug: 159845846
Test: glob_test.go
Change-Id: Ia771bdbdf1eb668623c4b3f00bf65e0e1e3a55c0
2021-04-09 09:46:18 -07:00
Jingwen Chen
6393098ecf bp2build: cc_library converter for //bionic/libdl:libdl_android
This CL contains the converter for libdl_android, a cc_library that
expands into a cc_shared_library and a regular cc_library.

Test: TH
Test: bp2build; bazel test //build/bazel/tests/...

Change-Id: If70641a538211b0d6b2aac0e4d0d06912318304d
2021-04-09 09:42:53 +00:00
Paul Duffin
afdd4061ee Rename OptionalPath.RelativeToPath -> RelativeToTop
Bug: 183650682
Test: m nothing
Change-Id: I95239c2273e715e8c4c15a4a02967e01f6913234
2021-03-30 20:06:22 +01:00
Paul Duffin
544c368d2e Merge "Add OptionalPath.RelativeToTop" 2021-03-30 16:43:23 +00:00
Jingwen Chen
0702791a99 bp2build: arch-configurable selects for label list attrs.
This CL adds the configurable LabelListAttribute support to bp2build.

Test: go test
Change-Id: I2ef9e385d9cf1b1845988128eca1d8cda1ecb5e8
2021-03-30 13:22:48 +00:00
Paul Duffin
a5b8135bdf Add OptionalPath.RelativeToTop
Needed for some follow up changes.

Bug: 183650682
Test: m nothing
Change-Id: If38dfe8e919094a7365bcee6cb71bfd0409f4454
2021-03-29 17:29:30 +01:00
Martin Stjernholm
32312eb75a Don't panic on "go test" invocations from the command line.
Test: cd build/soong; go test ./android
Bug: 183650682
Change-Id: I2d1bbda21cb262eafc7f7d329206720809399985
2021-03-28 22:06:33 +01:00
Paul Duffin
85d8f0dcd9 Add Path.RelativeToTop
Makes Path.RelativeToTop the source of truth as to what `relative to
top` means.

Uses it in PathRelativeToTop so that it will be tested by the existing
tests that use AssertPathRelativeToTop.

Also adds RelativeToTop() to WritablePaths and Paths.

Bug: 183650682
Test: m droid
Change-Id: I88dc56afd1314e51b4e41795a2448cab4ce8a899
2021-03-25 13:14:37 +00:00
Paul Duffin
74abc5d4ee Remove config from basePath
Bug: 183650682
Test: m droid
Change-Id: I60eb2b8ed0317e874ecdafd5cb37d7c997d41bb8
2021-03-25 09:18:29 +00:00
Paul Duffin
d65c58b204 Add buildDir to WritablePath implementations
First, the buildDir() method was renamed to getBuildDir() to avoid
clashing with the buildDir field.

Then, a buildDir was added to both `OutputPath` and `InstallPath` but
not to `PhonyPath` as it does not contain any path components. Instead
the `PhonyPath.getBuildDir()` was changed to simply return "".

Bug: 183650682
Test: m droid
Change-Id: I12e1854c829b980c5c01205753c62c00dc0a4774
2021-03-25 09:18:29 +00:00
Paul Duffin
580efc8716 Add srcDir to SourcePath
Bug: 183650682
Test: m droid
Change-Id: I08f7dba449538d289a6c4d6b5325c1a833cff834
2021-03-25 09:18:29 +00:00
Jingwen Chen
95c6eb3959 Merge "bp2build: add allowlist for package-level conversions." 2021-03-24 21:38:07 +00:00
Jingwen Chen
12b4c2706d bp2build: add allowlist for package-level conversions.
This CL adds the support for specifying lists of directories in
build/soong/android/bazel.go, which are then written into
out/soong/bp2build/MANIFEST. Using this configuration,
modules/directories can either default to bp2build_available: true or
false, while still retaining the ability to opt-in or out at the module level.

It also ensures that ConvertWithBp2Build returns true iff the module
type has a registered bp2build converter.

Test: go tests
Test: demo.sh full
Test: TreeHugger presubmits for bp2build and mixed builds.

Change-Id: I0e0f6f4b1b2ec045f2f1c338f7084defc5d23a55
2021-03-24 02:27:19 -04:00
Colin Cross
fa65cee27f Fix genrule depending on disabled module with ALLOW_MISSING_DEPENDENCIES=true
If a genrule depends on a module that is disabled, in this case because
it is a device module in a host-only build, it can cause panics when
getPathsFromModuleDep retrieves a nil Path from the disabled module.
Treat disabled modules as missing dependencies.

Test: TestGenruleAllowMissingDependencies
Change-Id: I3c689c6b5505b21eaf7ae7cb93c00f96f438ac17
2021-03-23 16:17:37 -07:00
Liz Kammer
bdc609972c Support autoconverted modules in mixed builds
modules converted with bp2build_available are will also be available to
be used in mixed builds.

Test: build/bazel/scripts/milestone-2/demo.sh full
Test: go tests
Change-Id: I49f16ec3ba5bb11dfed8066af069c27eb04371fb
2021-03-16 15:11:49 -04:00
Petri Gynther
ac22956dcb Rename vendor-ramdisk to vendor_ramdisk
Rename to get consistent ramdisk directory naming in
out/target/product/<name>:
  debug_ramdisk
  ramdisk
  vendor_debug_ramdisk
  vendor_ramdisk

Test: build and inspect out/target/product/<name>
Change-Id: Ib86e6509bd910e8c95bcb6e3f7bb639ebbfe3758
2021-03-03 00:15:54 -08:00
Pedro Loureiro
8a75058d3f Merge "Add support for lint baseline files" 2021-02-18 15:53:13 +00:00
Pedro Loureiro
5d190cc24e Add support for lint baseline files
Test: m droid
Test: go test ^TestJavaLint # (from soong/build/java)

Change-Id: I249a0a0597b0bf8495460ed283b476ad2eb36edc
2021-02-18 11:15:30 +00:00
Treehugger Robot
5e3fa0a9a5 Merge "Clarify ExistentPathForSource docs" 2021-02-17 02:02:30 +00:00
Liz Kammer
ebfcf672ef Omit bazel.LabelList attrs if no path is specified
Test: go test build_conversion_test.go
Change-Id: Ibcd81049788e99422569fe74e5f3834df4a05122
2021-02-16 15:00:05 -05:00
Liz Kammer
7aa52883dd Clarify ExistentPathForSource docs
Specify that these paths are relative to SrcDir, not relative to a
module's local source directory.

Test: n/a
Change-Id: I9a80f19b2eb3a15e6276e98200796b0680af9ee7
2021-02-11 09:16:14 -05:00
Colin Cross
dcf71b299c Convert java.Dependency to JavaInfo provider
Export information about java dependencies through a Provider
instead of accessing the module directly.

Test: java_test.go
Test: no changes to build.ninja
Change-Id: Ifc5d566bf6f6ebc0ad399e948effaa1ef6a22876
2021-02-09 15:36:25 -08:00
Liz Kammer
356f7d45c1 bp2build: convert paths/module refs to Bazel label
This currently expands all globs, still need to support converting glob
syntax.

Test: go build_conversion_test
Test: GENERATE_BAZEL_FILES=true m nothing
Test: m nothing
Bug: 165114590
Change-Id: If7b26e8e663d17566fad9614ca87a8da1f095284
2021-02-04 13:45:56 -05:00
Paul Duffin
0267d49255 Make OutputPath implement objPathProvider
Needed to allow OutputPath to replace some usages of ModuleOutPath.

Bug: 179124768
Test: m droid
Change-Id: I0f746a11c53ed8c9c9193b57afc0b06c0249ff75
2021-02-02 13:33:51 +00:00
Liz Kammer
a830f3abc2 Refactor android/paths.go to allow reuse
Extracts out the relevant pieces of ModuleContext into a minimal
interfaces necessary for path handling for Modules. This will allow
reuse in queryview by allowing it to implement only the parts of the
module interface necessary for path expansion in order to properly
handle path properties.

Test: go test soong tests
Change-Id: Iea39a365e5a2ff55071e435fb64f9e6464bbcafd
2021-01-26 12:51:02 -05:00
Liz Kammer
2dd9ca422d Refactor queryview.
Splits queryview into queryview and bp2build. The latter runs as a
presingleton (could be converted to a singleton). This prevents needing
to run soong_ui a subsequent time to build the query. Queryview remains
as a separate step to prevent increasing runtime due to this generation
every time Soong runs.

Currently this is running as a presingleton as this gives a translation
of Android.bp files after only LoadHooks have run, no mutators.

Test: go tests
Test: m queryview && bazel query --config=queryview //...
Change-Id: If2ba28c7ef60fbd41f43bda6623d41c8c7d23a1b
2021-01-07 14:34:00 -05:00