Revert submission 2651299-bazel_sandwich
Reason for revert: b/293883239, checking if this breaks the build.
Reverted changes: /q/submissionid:2651299-bazel_sandwich
Change-Id: I5f1d4bb13d21e6599f5c353dcaba2375f5ec234d
The "bazel sandwich" is a mechanism for bazel to depend on make/soong
outputs. The name comes from the fact that bazel is now at the top
and bottom of the build graph. This is intended to allow us to work
on converting the partition builds to bazel while not all of the
dependencies of the partition have been converted.
It works by adding the bazel_sandwich_import_file rule, which emits a
dangling symlink that starts with bazel_sandwich:, and includes
information that the aquery handler in soong reads. The aquery handler
rewrites the symlink so that it points to a file generated by
make/soong, and adds a ninja dependency from the symlink to the file
it's targeting.
This allows us to depend on make-built files from bazel, but notably
it doesn't allow us to depend on analysis-time information from make.
This shouldn't be a problem for the partitions, but limits the use of
the bazel sandwich to similar, less complicated types of builds.
go/roboleaf-bazel-sandwich
Bug: 265127181
Test: m bazel_sandwich
Change-Id: Ic41bae7be0b55f251d04a6a95f846c50ce897adc
Use Go's generics for DepSets so they don't require a type-specific
wrapper and reflection.
Test: depsets_test.go
Change-Id: I22ba0b7d680d37d2cd05230b0f560d166c4dd20b
This is to differentiate soong intermediate directories for soong config
modules. This will help incremental build across different
devices.
Test result of building panther, building cheetah, and building panther
again:
Before this change
- build time: 02:57
- # of tasks: 31044
After this change
- build time: 01:48
- # of tasks: 1694
Build time includes build.ninja generating time (which is more than 1
minute), so the overriden artifacts become far fewer.
And "NINJA_ARGS='-n -d explain' m" only gave 4 "command line changed"
nodes.
Bug: 279362051
Test: see above
Change-Id: I4891cbe823ae21628465e5c6eb26a4837ccdd202
This changes bp2build codegen, symlink forest generation, and
soong_build so that they do not rewrite output files if the contents are
unchanged.
Bug: 266983462
Test: m droid
Test: canonical_perf.sh benchmarking
Test: Manually verified that rerunning analysis did not regenerate
out/soong/workspace/prebuilts/sdk/BUILD.bazel unless contents changed
Change-Id: I5ec227df7a32b53c7fa0d741fb1403a51931024b
Perform a single pass over pathComponents in both validatePath and
validateSafePath instead.
Test: presubmits
Change-Id: I1499183e345ca180900c54a18f9dc8bd534f2588
This was always set to "." after aosp/1652613 and
aosp/1796650. While doing some rudimentary performance
profiling of soong, it turned out that SourcePath.String()
was taking 600+MB of memory joining the "." to the rest
of the path. Remove it for performance gains.
Bug: 262629589
Test: Verfied ninja files didn't change with this cl on aosp_arm64-userdebug
Change-Id: Id6b552ab8defe37e0c2b58e813fb1dbae427ae96
Make PackageVarContext implement PathGlobContext by implementing
GlobWithDeps. This will allow calls to ExistentPathForSource
inside a VariableFunc to use optimized glob dependencies instead of
falling back to AddNinjaFileDeps, which is resulting in extra
dependencies from soong_build on top level directories, triggering
extra Soong regenerations.
Remove the fallback path in ExistentPathForSource by making it take
a PathGlobContext, which is now a superset of PathContext.
Rewrite TestNinjaDeps to not rely on the unoptimized glob dependencies
in VariableFuncs and instead call ctx.Config().AddNinjaFileDeps
directly.
Bug: 257079828
Test: test_create_global_include_directory
Change-Id: I48cf189157d78b9252d339dbc9baeb27e4694807
This is a fix to aosp/2260763 and skips the existence check corectly.
Previously, it would not throw an exception during Soong analysis, but
would add the path to a `MissingDeps` object. The ninja graph generated
using this would throw an exception during ninja execution.
We should not do this check for `cc_api_library`. The prebuilt
library.so is missing in the build graph of the inner tree (expected),
but it will be present when the orchestrator creates the combined
multi-tree ninja graph.
Test: rm -rf out && multitree_build vendor/vendorimage
(I did not clean out/ in aosp/2260763, and therefore did not catch this
earlier)
Test: go test ./cc
Change-Id: I68d245acae3bfb777bfc8a72fb7cd4909cb0a289
Context
- Android-<target>.mk currently does not contain information about partition for its dependent unembedded jni libraries, but only lists the name of the unembedded jni libraries.
- If an android_app module depends on an unembedded jni library that is located in a different partition, make cannot find the library.
Implementation
- Create a string field partition in jniLib struct.
- Add variable "LOCAL_SOONG_JNI_LIBS_PARTITION_<target>", an array of mappings of the name of the jni library to its partition.
Bug: 154162945
Test: m
Change-Id: I6b8e1272ff59dc70e3dd6ce8c6c8e4686dad76df
* changes:
Move function PathForVndkRefAbiDump to Prevent unnecessary exports in paths.go
Change the type of parameter prevVersion to int in sourceAbiDump
This CL moves function PathForVndkRefAbiDump from android/paths.go to
cc/library.go to prevent unnecessary exports.
Test: make libz
Bug: 239915696
Change-Id: I1270e8d07edb09d93621c049acab9196757d356b
The AIDL source generation rule sets include flags based on the relative
path of .aidl sources. For .aidl sources provided by Bazel targets, e.g.
in a filegroup, the same directory could be added to the include path
twice. Instead we need to ensure that if a Bazel source provides the
include path, that we don't add it again from a Soong source.
Bug: 229251008
Test: USE_BAZEL_ANALYSIS=1 m api-stubs-docs-non-updatable
Change-Id: I4997039003242b43e0e52ccf41729acb4ad11324
Since vndk libraries don't need to maintain Cross-Version abi
compatibility. The newly proposed Cross-Version ABi checking mechanism
is applied to only ndk/ and platform/. The reference dumps of LLNDK
libraries should get from platform/ to be benefited from this checking
mechanism.
Test: mkdir -p prebuilts/abi-dumps/platform/UpsideDownCake/64/x86_64/source-based/ && cp prebuilts/abi-dumps/vndk/32/64/x86_64/source-based/libvndksupport.so.lsdump "$_"
make libvndksupport -j
Bug: 238383845
Change-Id: I663511d60623cd23619166d8d63fd26220a930e2
Add toolchains to support cross compiling to aarch64-linux-musl and
arm-linux-musleabihf.
Bug: 236052820
Test: build arm and arm64 musl sysroots
Change-Id: I47a9322929baff2492c6e8db989ece01fcbeb133
This change contains a prototype implementation for multitree. Several
interfaces and modules are added.
1. Imported/Exported
Modules implementing Exportable interface can export artifacts to other
components. "imported_filegroup" modules can import generated artifacts from other exported modules.
2. Multitree metadata
It contains information about imported/exported modules in each
component, and can be generated via "m update-meta".
3. cc library stub
It's based on prototype stub libraries. It uses imported/exported
mechanism to expose a C API, with a map.txt file and header files.
Bug: 230448564
Test: m
Change-Id: Id7ff7618e2c630c5617a564d8b23b60a1cc9c8e8
The ABI reference dump directory names contained TARGET_ARCH_VARIANT
or TARGET_2ND_ARCH_VARIANT. The builds with non-generic arch variant
could not find the reference dumps for their arches. This commit renames
the directories to <TARGET_ARCH> and <TARGET_2ND_ARCH>_<TARGET_ARCH> to
ensure that all arch variants are checked.
Bug: 213127415
Test: lunch aosp_cf_x86_64_phone-userdebug && make
Change-Id: I7617f3ffc0e1bc423eea8c39e0f2b745276b4244
While Bazel genrules will allow genrules to accept a directory as input,
the results can be unexpected to a user as changes to the contents of
the directory may not trigger a rebuild as expected. Restricting this
in Soong ensures that converted targets will behave as expected.
Test: CI
Change-Id: I8616f58d1df267005e6c0ca3f4730d06de52c0d9
Containers generally package the transitive installable
dependencies of their direct dependencies, track them as license
deps.
Bug: 207445310
Test: m checkbuild
Change-Id: Ic8640152cee0e0cfec5e85a1649a8adfd29d517a
* changes:
Fix InstallBypassMake symlink dependencies
Add tests for ctx.InstallFile
Fix OutDir vs SoongOutDir in tests
Fix go vet error
Add missing os.MkdirAll to WriteFileToOutputDir
Use pathtools.WriteFileIfChanged in translateAndroidMk
Call ctx.InstallFile on the primary install file last so that the
primary install file can depend on the extra install files, and so that
the primary install file can be inferred from the last installed file.
Add missing ctx.InstallFile calls for the dexpreopt and hostdex outputs.
Fix the install subdirectory for modules installing to the testcases
directory.
Bug: 204136549
Test: m checkbuild
Change-Id: I7edd7647be27439d3ca0ecc589ca5e89d4ba8474
Move cc module installation rules into Soong by overriding
InstallBypassMake. Update the locations that find host tools
to look in the Make installation directory instead of the Soong
installation directory, which will no longer be used.
The methods that find host tools are also used on go binaries,
so update the config methods that tell Blueprint where to install
go binaries to the Make installation directory too.
Bug: 204136549
Test: m checkbuild
Change-Id: Id172592c195e506102982a4af0084f6d9c68a896
Add PathForGoBinary that uses pathForInstall to return the install
path of a GoBinaryTool. This will replace various places that used
PathForOutput to reconstruct a path to a Go tool, and will support
moving Go tools to the Make install directory outside of the
PathForOutput directory in a future patch.
Bug: 204136549
Test: m checkbuild
Change-Id: I83a3be9f5c621975540f5ed601a0b9e2611c98b9
The next patches will make more InstallPaths use the Make output
directory, which is not inside PathForOutput. Fix the assumption
that the InstallPath is relative to PathForOutput by storing
enough information in the InstallPath to find the on-device path
without PathForOutput.
Bug: 204136549
Test: soong tests
Change-Id: Icb3eeef3f1c72f773f333267f8a7dfc503feacb5
These are: TopFile and GeneratingPrimaryBuilder.
Also re-shuffle the list of flags to make a bit more sense and finish
the rename of BuildDir and NinjaBuildDir to SoongOutDir and OutDir,
respectively.
Test: Presubmits.
Change-Id: I103ff5f09f1c0d16f695a7da5dea13b55028e33e
These are just out/ and out/soong/ and the old names were quite
confusing.
Test: Presubmits.
Merged-In: I999ca62162062f27e449f83ffb00fa580d4065b8
Merged-In: I2fcd80a9e376a6842a06aa104a9f6f5c5e16f8c7
Change-Id: Ib481d2aac40df8da1b74174dc8ec576c1cb48be0
This CL changes the logic in rust/builder.go so that some rustdoc flags
are only applied to external crates. This will allow us to since
warnings and deal with soft-failures in external crates while allowing
us to be more strict with our internal Rust code.
Bug: 195136952
Test: m rustdoc
Change-Id: Icdde304bbbb323cae9657e8f842f58ae79e811ce
Split the x86 host toolchain into glibc and musl variants
Create new musl toolchains that are based on the existing glibc
toolchains, and add the necessary flags for musl compiles.
This relands Ifc02f9e5afa61ff758be98b0c962f3a4b53d0546 with changes
for I46672e3a096b6ea94ff4c10e1c31e8fd010a163c.
Bug: 190084016
Change-Id: Iaa9f7a50ff601155ecd73acc5701a2c226be66dc
Test: TestArchMutator
Create new musl toolchains that are based on the existing glibc
toolchains, and add the necessary flags for musl compiles.
Bug: 190084016
Test: TestArchMutator
Change-Id: Ifc02f9e5afa61ff758be98b0c962f3a4b53d0546
Previously, a module reference in a path property would be parsed into
two parts, the module name and the optional output tag, which defaults
to an empty string if not specified. The output tag would be stored in
a sourceOrOutputDependencyTag which would then be used, along with the
module name to add a dependency on the module.
Later, when the paths were processed the same module reference would be
parsed into the same two parts again and the module name used to find a
matching Module by comparing it against the name returned by either
Module.Name(), ctx.OtherModuleName() or ModuleBase.BaseModuleName().
Once the module had been found then if it supported OutputFilesProducer
then the tag would be passed to its OutputFiles(tag) method. Otherwise,
it would fall back to treating it as SourceFilesProducer.
The problem with that is the module name retrieved from the module in
some way (either directly or through a context name) could be different
to that originally supplied when adding the dependency. e.g.
1. If the original dependency was added onto a source module but there
existed a suitable and preferred prebuilt module then the dependency
onto the source module would have been replaced by the prebuilt
module which has a different name.
2. If the path property included a fully qualified name that included
a qualifying path then it would not match the name retrieved from
the module which would not include the qualifying path.
This change circumvents that whole issue by adding the module name that
was originally used to add the dependency into the DependencyTag. Now
the DependencyTag uniquely identifies the original module/outputTag
pair parsed from the module reference. The pathDepsMutator guarantees
that they are unique as it dedups them before adding the dependencies.
It is possible that calling ExtractSource(s)Deps() would add some
duplicate but if they did they would be identical, i.e. the same
sourceOrOutputDependencyTag would be used to add a dependency onto the
exact same module. In that case it would not matter which of the
dependencies was found as it would still return the same module.
Bug: 193228441
Test: m nothing
Change-Id: I661514a2984818e5c26577411cede53eb57bcd02
PathForModuleSrc calls validatePath in order to convert the supplied
path components into a single path. Unfortunately, that corrupts a
fully qualified module name. So, when given "//namespace:module" it
treats it as a path and replaces "//" with "/". That replacement is
done by a call to filepath.Join().
This change simply concatenates the path components together textually,
using the path separator, to avoid the corruption. That ensures that
a fully qualified module name is preserved and processed properly. If
the path components do not contain a module name then expandOneSrcPath
will call pathForModuleSrc which validates the path so it does not open
up a way to create an invalid path as the validation was unnecessary
anyway.
Bug: 193228441
Test: m nothing
Change-Id: I0bb66feac182b77ce96c8d5d5f17e28ea28d75ba
Previously, modules customized behavior around the handling of
sourceOrOutputDependencyTag by comparing them to android.SourceDepTag
and retrieving the module using something like this:
ctx.GetDirectDepWithTag(m, android.SourceDepTag)
The problem with that is it does not allow an output tag to be
specified and does not handle fully qualified names properly.
This adds the following:
* IsSourceDepTag and IsSourceDepTagWithOutputTag to check whether a
blueprint.DependencyTag is a sourceOrOutputDependencyTag. The latter
also checks that it has the correct output tag.
* GetModuleFromPathDep(ctx, moduleName, outputTag) as a replacement for
ctx.GetDirectDepWithTag(m, android.SourceDepTag).
Replaces usages of:
* t == SourceDepTag with IsSourceDepTagWithOutputTag(t, "")
* ctx.GetDirectDepWithTag(m, android.SourceDepTag) with
GetModuleFromPathDep(ctx, m, "")
It also deprecates the following:
* android.SourcDepTag - as a follow up change needs to modify the
sourceOrOutputDependencyTag will make this useless.
* ExpandSources, ExpandsSources - copies existing deprecated messages
from the implementation to the interface so that they can be seen
by users of that interface.
Bug: 193228441
Test: m nothing
Change-Id: I8c397232b8d7dc1f9702c04ad45ea7819d4631ae
The default partition for RRO is "product/" in Make, but it was
"system/" in Soong. This CL ports the logic from Make to Soong
To implement this, a new function PathForModuleInPartitionInstall is
created that enables callers to provide the relevant partition
Bug: 158407753
Test: from build/soong, ran go test ./java
Change-Id: I05b02eae7fe57189aaad5109c26cccc5823518ef
Appending a Path or Paths to a slice of Paths is simple but appending
an OptionalPath requires conditional logic which makes OptionalPaths
harder to use. This change makes it easy to append the embedded Path,
if any, to a slice of Paths.
Bug: 179354495
Test: m nothing
Change-Id: Ibf80a23043c846162e17c3a98b2590bca653b170
Allow blueprint_go_binary to be used as test data by supporting it in
PathForModuleSrc.
Also make python use GetDirectDepsWithTag instead of VisitDirectDeps
so it doesn't fail on the dependency to a Blueprint (non-Android)
module.
Fixes: 186528269
Test: manual
Change-Id: Ibc1d7caf7bf1fa5db805c3b885a95df4baee0ecf
