Commit graph

232 commits

Author SHA1 Message Date
Colin Cross
dc35e21124 Consolidate *MutatorContext and ModuleContext into BaseModuleContext
Following 99bdb2ab4fd5168a71a20aecf10611425be47ec4 in build/blueprint,
move more methods into BaseModuleContext.

This reapplies I9f8df94f1ae2b55d3cccf7b9468247f3e7cb2ebd after fixing
missing errors thrown for missing defaults modules when
AllowMissingDependencies == true.

Test: m checkbuild
Test: defaults_test.go
Change-Id: Ia17b2bcbf2bac6889c419b2e73953946f6aa40ad
2019-06-10 16:30:35 -07:00
Colin Cross
380c69aab4 Revert "Consolidate *MutatorContext and ModuleContext into BaseModuleContext"
This reverts commit 7e0a2cb590.

Reason for revert: broke TARGET_BUILD_APPS builds

Change-Id: I5316a62f77bb38f6195e3df5e31b073dbd1eb682
2019-06-10 17:49:58 +00:00
Colin Cross
7e0a2cb590 Consolidate *MutatorContext and ModuleContext into BaseModuleContext
Following 99bdb2ab4fd5168a71a20aecf10611425be47ec4 in build/blueprint,
move more methods into BaseModuleContext.

Test: m checkbuild
Change-Id: I9f8df94f1ae2b55d3cccf7b9468247f3e7cb2ebd
2019-06-07 13:29:12 -07:00
Colin Cross
0ea8ba82fc Consolidate baseContext, BaseContext, and BaseModuleContext
blueprint.BaseModuleContext is the set of methods available to all
module-specific calls (GenerateBuildActions or mutators).  The
android package split the same functionality across baseContext (nee
androidBaseContext), BaseModuleContext, and BaseContext.
Consolidate all of them into android.BaseModuleContext.

Test: m checkbuild
Change-Id: I2d7f5c56fd4424032cb93edff6dc730ff33e4f1e
2019-06-06 20:01:31 -07:00
Colin Cross
25de6c30d3 Remove repetition in android package names
Types in the android package don't need to be prefixed with 'android'.

Test: m checkbuild
Change-Id: Ieefcfc1a2c4161d9c412f40ef31867149ec21aec
2019-06-06 17:24:01 -07:00
Colin Cross
41955e8895 Support tagged module references
There are cases where a module needs to refer to an intermediate
output of another module instead of its final output.  For example,
a module may want to use the .jar containing .class files from
another module whose final output is a .jar containing classes.dex
files.  Support a new ":module{.tag}" format in any property that
is annotated with `android:"path"`, which will query the target
module for its ".tag" output(s).

Test: path_properties_test.go, paths_test.go
Test: no unexpected changes in build.ninja
Change-Id: Icd3c9b0d83ff125771767c04046fcffb9fc3f65a
2019-06-04 10:22:51 -07:00
Logan Chien
41eabe63e8 Support opt-in ABI checks
This commit allows a module to opt in for ABI checks even when it is
not an LLNDK/VNDK module.

Bug: 131421213
Test: Add `header_abi_checker { enabled: true, }` to some module
Change-Id: Ie09d262e651cbb44d7d0eba652f55dc1e1e52962
2019-04-27 01:55:50 +08:00
Dan Willemsen
633c502295 Support RuleBuilder.Sbox to wrap commands in sbox
This essentially allows you to declare that everything in a directory
will be created by the rule, and we'll ensure that your command actually
writes out all of the claimed outputs, and remove any other files that
previously existed in that directory.

Test: built-in tests
Change-Id: I990dce2b3a0d89ebd2736ac1a0cadfb5864c6e73
2019-04-15 14:52:05 -07:00
Colin Cross
ba71a3fb11 Fix missing genrule srcs and tools with ALLOW_MISSING_DEPENDENCIES=true
Set the location label for missing srcs and tools to avoid
nonsensical errors when parsing the command.

Test: genrule_test.go
Test: paths_test.go
Test: unbundled branch with missing framework-res module needed by robolectric genrule
Change-Id: I9c1f1cd82a80f048c0e903b8e93910b1ae34b0b1
2019-04-07 19:40:02 +00:00
Colin Cross
8a49795df1 Replace ctx.ExpandSources with android.PathsForModuleSrc
Move the logic from ctx.ExpandSources into android.PathsForModuleSrc
and ctx.ExpandSource into android.PathForModuleSrc, and deprecate
them.  When combined with the pathDepsMutator this will let all
properties that take source paths also take filegroups or genrule
outputs, as long as they are tagged with `android:"path"`.

Test: All soong tests
Change-Id: I01625e76b5da19240e9649bf26a014eeeafcab8f
2019-03-20 19:36:13 +00:00
Colin Cross
07e51619a2 Remove ModuleSrcPath
ModuleSrcPath was designed as a type that ensured that modules only
referenced sources inside the directory that contained the Android.bp
file.  In practice they don't work very well, because allowing
filegroups and genrules as inputs to any module that takes a source
path means that the path might end up being to a file in another
source directory or to a generated file in the output directory.

Remove ModuleSrcPath, replacing it with SourcePath in the places
that need to explicitly refer to a path in the source tree, or
Path where it may be a source path or a generated path.

Make PathForModuleSrc return a Path instead of a SourcePath in
preparation for consolidation with ctx.ExpandSources, which will
make it possibly return paths to generated files.

Test: All soong tests
Change-Id: I973a78470ed14307eea5f6d0cc93942775a65715
2019-03-20 19:36:03 +00:00
Colin Cross
2fafa3ec49 Remove ExpandSourcesSubDir and ModuleSrcPath.WithSubDir
Replace ExpandSourcesSubDir with ExpandSources plus
PathsWithModuleSrcSubDir, which loops over the paths and uses
Join to create paths relative to subdir on any results that
are ModuleSrcPaths.

Test: All soong tests
Change-Id: I11a7face88641e2c26ccdca0a3117d5c38ab588e
2019-03-20 19:35:52 +00:00
Colin Cross
2cdd5df7cc Fix Rel() after ReplaceExtension or InSameDir
Copying p.rel doesn't work, as rel needs to match the end of the
value in path.  Apply the same transformation to p.rel as p.path.

Test: paths_test.go
Change-Id: I42d676c6c4fc18d9852c1a73f25e5a791d7553d0
2019-02-27 20:23:25 +00:00
Colin Cross
40e3373d41 Add more paths helper methods
Add PathsForOutput to convert multiple strings into WritablePaths.

Add OutputPath.InSameDir to build a new OutputPath pointing to a
file in the same directory as an existing OutputPath.

Add WritablePathForTesting and WritablePathsForTesting that mirror
PathForTesting and PathsForTesting but return WritablePaths.

Add PathContextForTesting to return a minimal PathContext
implementation.

Test: paths_test.go
Change-Id: I9708eb164b273514a96dae0a260ef9a963fb9bcf
2019-02-16 17:08:07 -08:00
Colin Cross
8854a5a805 Add OutputPath.ReplaceExtension()
Add a helper method that returns a new OutputPath with the
extension replaced.

Test: paths_test.go
Change-Id: I7f93b3475031a19b53652022740488373a1b7d97
2019-02-12 17:07:35 -08:00
Mikhail Naganov
ab1f518fb0 Improve error messages about paths
Quote the path parameter in the message to distinguish it
from the text. This makes messages more understandable.

Before: source path include does not exist
After:  source path "include" does not exist

Test: build Android
Change-Id: I99dbbce3cf090682a230d05bf120549a2cc7af3c
2019-02-08 13:32:08 -08:00
Colin Cross
0ddae7fddd Allow disabling implicit resource_dirs and asset_dirs
Specifying [] for resource_dirs or asset_dirs will prevent using
the default "res" or "assets" directories.

Test: TestResourceDirs
Bug: 124035856
Change-Id: I96e38ac1319260db43950299a8b1774da68ea85e
2019-02-08 00:24:01 +00:00
Colin Cross
7b3dcc31eb Avoid filepath.Abs
filepath.Abs is surprisingly expensive, it calls os.Getwd every
time, which involves multiple syscalls, a lock, and and allocations.
Use IsAbs and prefix matching instead.

Test: paths_test.go
Change-Id: Ia6cf34d6bef24c694702af1e7a6ff08ffd2d822b
2019-01-24 21:35:16 +00:00
Colin Cross
43f08db29e Dexpreopt soong modules inside soong
Port the dexpreopt logic from Make to the dexpreopt package in Soong,
and use it to dexpreopt Soong modules.  The same package is also
compiled into the dexpreopt_gen binary to generate dexpreopt scripts
for Make modules.

This relands Ib67e2febf9ed921f06e8a86b9ec945c80dff35eb and
I462182638bd57b1367b5bfb0718e975c11ae66f7, along with multiple fixes
to depsfile generation in dexpreopt_gen that caused .odex files for
modules in defined make to be missing dependencies on boot.art, and
a fix to not dexpreopt and strip tests.

Bug: 119412419
Bug: 120273280
Test: no differences to dexpreopt outputs on aosp_sailfish system/,
      only expected changes to dexpreopt outputs on system_other
      (.vdex files for privileged Soong modules no longer incorrectly
      contain .dex contents).
Test: OUT_DIR=$PWD/out m
Test: NINJA_ARGS="-t deps out/target/product/sailfish/obj/APPS/Contacts_intermediates/dexpreopt.zip" m
Change-Id: I6bb2c971cee65d2338839753aa0d84939f335b1b
2018-12-15 19:07:54 -08:00
Colin Cross
4ff85ebeb7 Revert "Dexpreopt soong modules inside soong"
This reverts commit 29ff88741e.

Test: none
Bug: 119412419
2018-12-14 11:49:58 -08:00
Colin Cross
359e6436be Revert "Fix MaybeRel when OUT is absolute"
This reverts commit fee2bff77c.

Test: none
Bug: 119412419
2018-12-14 11:49:58 -08:00
Colin Cross
fee2bff77c Fix MaybeRel when OUT is absolute
MaybeRel was failing with an error when comparing an absolute out
path to a relative source path.  Make it return false instead, and
add tests.

Bug: 119412419
Test: paths_test.go
Change-Id: I462182638bd57b1367b5bfb0718e975c11ae66f7
2018-12-13 21:05:18 +00:00
Colin Cross
29ff88741e Dexpreopt soong modules inside soong
Port the dexpreopt logic from Make to the dexpreopt package in Soong,
and use it to dexpreopt Soong modules.  The same package is also
compiled into the dexpreopt_gen binary to generate dexpreopt scripts
for Make modules.

Bug: 119412419
Bug: 120273280
Test: no differences to dexpreopt outputs on aosp_sailfish system/,
      only expected changes to dexpreopt outputs on system_other
      (.vdex files for privileged Soong modules no longer incorrectly
      contain .dex contents).
Change-Id: Ib67e2febf9ed921f06e8a86b9ec945c80dff35eb
2018-12-13 15:53:54 +00:00
Dan Willemsen
2d31a44b8a Stop using DIST_DIR in Soong
We're only using it to distribute files in case of failure, which isn't
well supported currently, but can be handled for now by using the
DIST_DIR environment variable during the command execution.

This was at least one cause that we'd be re-running Soong during every
build server build, as the DIST_DIR values are unique.

Test: m dist
Change-Id: Ibd5e6b6c46695350de80b745bfb6a6aa685033a0
2018-10-20 21:33:41 -07:00
Colin Cross
3f4d116496 Update soong for ShouldFollowSymlinks argument to pathtools.Glob
pathtools.Glob now takes a ShouldFollowSymlinks argument.  Add
pathtools.FollowSymlinks to all the calls to pathtools.Glob to
maintain current behavior.

Test: m checkbuild
Change-Id: I0215efb212be4ba4513669612d88c2baade43747
2018-09-21 16:08:16 -07:00
Colin Cross
fe4bc36f87 Allow '$' in some paths
The icu resource directories contain filenames that have '$'
characters.

Allow paths returned by the Glob functions to contain '$', on the
assumption that real paths on disk are unlikely to contain strings
that are valid ninja variables.  Fix the Build rules to escape any
paths that are passed as Path arguments.  Fix the resource rules to
manually escape the paths that are passed as strings.

Test: m checkbuild
Change-Id: Ie631bc6d96259e592adb280491a365c0df7ed0e2
2018-09-13 18:27:50 +00:00
Colin Cross
e3924e180b Allow paths containing glob characters in glob results
Some paths contain glob characters.  For now allow paths with glob
characters as long as they are in glob results.  In the future we
may need to allow escaping glob characters.

Test: m checkbuild
Test: paths_test.go
Change-Id: I1cbeea658e8fc4975ca0b6a50a8c24ac2de026c5
2018-08-16 17:03:56 -07:00
Colin Cross
3063b78ea5 Make :module provide the output file for java modules
Make :module on a java_library provide the output file, which is
normally the implementation jar.  For java_library modules with
installable: true or compile_dex: true this will be the dexjar
instead.  For android_app modules this will be the apk.

Bug: 80144045
Test: no change to out/soong/build.ninja
Change-Id: I739674aee60a38bfccb859369e4414b46f293d82
2018-08-16 16:13:58 -07:00
Dario Freni
fd05a74e06 Add support for /product-services partition
This is an adaptation of Icc4f8c16bc389fe20db680849f311d02df1299c3, to
support modules that are installed on the /product-services partition.

Bug: 80741439
Test: m -j both with and without enabling the new partition
Change-Id: I72b335ad38baff5848cd3da7489343f8cf98ff16
2018-07-19 14:04:30 +01:00
Logan Chien
7eefdc4ed2 Support text ABI dump file format
This commit adds an option to read ABI reference dump in text file
format directly from `prebuilts/abi-dumps`.  If both the text file and
the gzip format exist, an error will be emitted.

Bug: 78650426
Test: create libexif.lsdump.gz and it works as usual
Test: create libexif.lsdump (decompressed) and it works
Test: touch both libexif.lsdump.gz and libexif.lsdump and it errors
Change-Id: I420a5953fb80855cb5c07e5a4d347fb6709f0340
2018-07-11 18:37:50 +08:00
Logan Chien
5237bed1c4 Remove vndkVsNdk()
This commmit removes `vndkVsNdk()`, which is essentially
`!inList(ctx.baseModuleName(), llndkLibraries)`.

Test: lunch aosp_arm64_ab-userdebug && make
Change-Id: I8e2352f302df30057997944678f176f4550d3f75
2018-07-11 17:30:39 +08:00
Logan Chien
83c415f955 Remove isSourceDump from PathForVndkRefAbiDump()
This commit removes `isSourceDump` from `PathForVndkRefAbiDump()`
because the binary dump paths are not being used at all.

Test: lunch aosp_arm64_ab-userdebug && make
Change-Id: I7a6bb9053a6b052590fb1152982949fee897df8d
2018-07-11 15:27:36 +08:00
Jiyong Park
2e6743176a Recovery partition has same layout as system
Recovery partition has same layout as system. In other words,
executables are installed to <recovery_root>/system/bin and libs are
installed to ../system/lib.

This is made possible because the recovery partition is now
self-contained, i.e., shell, adbd, etc. are hosted in the recovery
partition and thus we no longer has to mount the real system.img
to /system. So /system is now available to executables built for
recovery mode.

Bug: 63673171
Test: `adb reboot recovery; adb devices` shows the device ID
Test: Select 'mount /system' in the recovery mode, then `adb shell`.
$ lsof -p `pidof adbd` shows that libm.so, libc.so, etc. are loaded from
the /lib directory.

Change-Id: I6c4f903ad21fd3dd10a07c4588be85d3d678c099
2018-06-09 01:57:00 +09:00
Jiyong Park
f9332f1c86 Support recovery and recovery_available
`recovery: true` installs a module to the recovery partition.
`recovery_available: true` makes a module to be available to other
`recovery:true` or `recovery_available: true` modules.

These to are very similar to vendor, vendor_available properties, except
for the target partition.

Bug: 67916654
Bug: 64960723
Test: m -j, toybox_recovery is installed to the recovery/root/sbin
Change-Id: Iaebe0593de16c69fa70de251a61f4d018a251509
2018-05-15 16:27:12 +09:00
Colin Cross
a140bb05f1 Reverse RRO dir paths when passing to Make
LOCAL_RESOURCE_DIRS puts the highest priority overlay first, but
aapt2 expects the highest priority overlay last.  Soong stores the
list in aapt2 order (low to high priority), but that means when it
exports to Make as LOCAL_SOONG_RRO_DIRS, which goes to
build_rro_package.mk and then package_internal.mk, it gets reversed
again and comes out backwards.

Bug: 78032566
Test: m checkbuild
Change-Id: If72bf929fbf1d126f9051a2f21ec1eb4e3030e6e
2018-04-17 10:53:54 -07:00
Dan Willemsen
45133ac184 Make Config.ProductVariables private
All access to these should be going through the methods on Config /
DeviceConfig.

Bug: 76168832
Test: m blueprint_tools
Change-Id: I47512dd58fb1a1a3f25838a9b1adaed2c41af8d3
2018-04-11 01:45:29 +00:00
Dan Willemsen
bc0c509267 Add DistPath to reference the dist folder
Instead of open-coding the logic of whether there is one, or where to
find it.

Test: diff out/soong/build.ninja without dist
Test: diff out/soong/build.ninja with dist specified
Change-Id: Ia3f1ef335e2d6e2175343338d04867d778a50300
2018-03-12 15:48:30 -07:00
Jayant Chowdhary
34ce67d454 Consider binder bitness while looking for abi references.
Binder bitness does not always correlate directly with primary arch, as
assumed earlier. For example: it is possible to have devices with
primary arch 'arm' and use 64 bit binder.

Bug: 74362434

Test: utils/create_reference_dumps.py;
      m -j vndk_package for aosp_arm64_ab, aosp_arm_ab invokes
      header-abi-diff on both arm and arm64 arches.

Test: mm -j64 in system/libhwbinder on troublesome internal target.

Change-Id: Iea0a24b57cdb3033e25b6fe126c5d5d717f45b4e
2018-03-08 16:49:59 -08:00
Jayant Chowdhary
ac066c6b90 For abi diffing, factor in arch variant and primary arch.
binder_size_t has a different size for builds with different primary
arches.

Also maintain seperate reference dumps for different arch variants,
since different cflags may be legally specified for them (similar to
what GSI does)

Test: create reference dump for libjpeg at
      prebuilts/abi-dumps/vndk/current/arm64/arm64_armv8-a/source-based/libjpeg.so.lsdump.gz
      mm -j64; header-abi-diff gets invoked.

Change-Id: I55eae4d4811c9754fe8dbd1009c7929fea119eeb
2018-03-01 17:19:25 -08:00
Dan Willemsen
540a78c1ce Turn GlobFiles into a Glob for files, use it
GlobFiles had allowed results to be anywhere in the source tree,
restrict it to results within the current module directory.

Then use it for ExpandSources and other places where we only want files.
This fixes using '*' in cc_test's `data` property, which can only
support files.

The only thing this changes today is that java_resource_dirs and
java_resources no longer pass directories to soong_zip's -f argument.
core-libart previously added some icu directories, now it only passes
files.

Bug: 71906438
Test: only expected changes in out/soong/build.ninja
Test: add data: ["**/*"] to a cc_test, build successfully
Change-Id: Iff1bd8c005a48e431c740706d7e23f4f957d8b1d
2018-03-01 14:05:20 -08:00
Colin Cross
192e97a95c Move AllowMissingDependencies check from PathsForSource to PathForSource
This reapplies Id7925999a27ea75a05e9301bbf1eb9f9a6bc4652 with
additional fixes to not use PathForSource in PathForModuleSrc.

PathsForSource was handling the AllowMissingDependencies case, but
PathForSource was not.  Refactor PathForSource and
ExistentPathForSource, and add logic to PathForSource to fall back
to behavior similar to ExistentPathForSource when
AllowMissingDependencies is set.

PathForModuleSrc uses PathForSource, which causes too many
globs (>50k).  The AllowMissingDependencies check doesn't make
much sense for PathForModuleSrc, since we already know the
project containing the definition of the module exists, we can
expect its local source files to exist.  Use pathForSource and
do an manual existence check instead.

Test: paths_test.go
Test: m ALLOW_MISSING_DEPENDENCIES=true
Change-Id: If1690a708393964d3030cb908beaf7b6897c0084
2018-02-23 11:31:45 -08:00
Colin Cross
461b445d62 Pass nil instead of []string{} to ctx.Glob* functions
[]string{} is unnecessary, just use nil.

Test: m ALLOW_MISSING_DEPENDENCIES=true
Change-Id: Ia9aeb2ffc483429787da0e473a7f1bc87eb4cad1
2018-02-23 11:31:45 -08:00
Colin Cross
c48c14369c Revert "Move AllowMissingDependencies check from PathsForSource to PathForSource"
This reverts commit 94a321045a.

Reason for revert: Broke builds with ALLOW_MISSING_DEPENDENCIES=true

Change-Id: I9604887f796a79809ef8d6c741597796219dcaf3
2018-02-23 07:09:15 +00:00
Colin Cross
94a321045a Move AllowMissingDependencies check from PathsForSource to PathForSource
PathsForSource was handling the AllowMissingDependencies case, but
PathForSource was not.  Refactor PathForSource and
ExistentPathForSource, and add logic to PathForSource to fall back
to behavior similar to ExistentPathForSource when
AllowMissingDependencies is set.

Test: paths_test.go
Change-Id: Id7925999a27ea75a05e9301bbf1eb9f9a6bc4652
2018-02-22 14:43:36 -08:00
Colin Cross
1ccfcc36bd Propagate errors out of validatePath
The next patch will need to more complicated custom error handling,
so make validatePath return an error and let the caller handle it.

Test: paths_test.go
Change-Id: I4fe11c3f319303d779596709f4819e828b5bdb9b
2018-02-22 14:43:36 -08:00
Colin Cross
32f3898f0b Remove unused intermediates parameter from ExistentPathForSource
Test: m checkbuild
Change-Id: Id2c0a5039c2ec3b3795385c135ffec022ccd691e
2018-02-22 14:43:36 -08:00
Jaekyun Seok
5cfbfbb67a Use product instead of oem for Product specific module
Bug: 64195575
Test: succeeded building product.img with BOARD_PRODUCTIMAGE_PARTITION_SIZE,
BOARD_PRODUCTIMAGE_FILE_SYSTEM_TYPE and PRODUCT_PRODUCT_VERITY_PARTITION.
Change-Id: Icc4f8c16bc389fe20db680849f311d02df1299c3
2018-01-25 07:50:35 +09:00
Jiyong Park
2db7692a74 Add [soc|device|product]_specific
Added three properties (soc_specific, device_specific, and
product_specific) that shows what a module is specific to.

`soc_specific: true` means that the module is specific to an SoC
(System-On-a-Chip) and thus need to be installed to vendor partition.
This has the same meaning as the old `vendor: true` setting.

`device_specific: true` means that the module is specific to the entire
hardware configuration of a device includeing the SoC and off-chip
peripherals. These modules are installed to odm partition (or /vendor/odm
when odm partition does not exist).

`product_specific: true` means that the module is specific to the
software configuration of a product such as country, network operator,
etc. These modules are installed to oem partition (or /system/oem when
oem partition does not exist). These modules are assumed to be agnostic
to hardware, so this property can't be true when either soc_specific or
device_specific is set to true.

Bug: 68187740
Test: Build. path_tests amended.
Change-Id: I44ff055d87d53b0d2676758c506060de54cbffa0
2017-12-21 12:16:29 +09:00
Colin Cross
74e3fe4cd4 Fix PathForPhony
PathForPhony was prefixing out/soong to all of the phony targets.

Test: m checkbuild
Change-Id: If54d9c692b79751a09eef76c734dd2a1f7db325b
2017-12-11 16:30:14 -08:00
Colin Cross
0db5568612 Make (Source|Output)Path.Join("foo").Rel() return "foo"
This allows TestModule.Output("foo") to be used to find
a file called "foo" that was installed using ctx.InstallFile.

Test: soong tests
Change-Id: I04833c9ee8ac5baa6b6afd35715d1191c1622a78
2017-12-06 04:16:00 +00:00
Colin Cross
6510f91a1c Replace ModuleContext.AConfig() with Config()
AConfig() now duplicates Config().  Replace the uses of AConfig()
with Config().  Leave AConfig() for now until code in other
projects is cleaned up.

Test: m checkbuild
Change-Id: Ic88be643049d21dba45dbd1a65588ed94bf43bdc
2017-11-30 00:44:18 +00:00
Colin Cross
aabf67968c Make *Context.Config return a Config instead of a interface{}
In Soong, a Config() method will always return a Config.  Make
ModuleContext, SingletonContext, TopDownMutatorContext and
BottomUpMutatorContext's Config() methods explictly return
a Config to avoid having to type-assert everywhere.  Overriding
the Config method requires duplicating the list of methods in
blueprint.BaseModuleContext and blueprint.BottomUpMutatorContext,
following the same pattern used by the other *Contexts.

Config() obsoletes the AConfig() method used in some places, which
will be cleaned up in the next patch.

Test: m checkbuild
Change-Id: Ibe21efde933959811d52443496967ab8ce71215e
2017-11-30 00:44:11 +00:00
Colin Cross
3bc7ffa59b Replace aapt support with aapt2
Use aapt2 instead of aapt to compile Android app resources.
Also generate all files into srcjars instead of individual
sources.

Test: m checkbuild
Change-Id: I5a67991a0daf0017e8159b46fcff7d5564a91468
2017-11-29 05:05:07 +00:00
Colin Cross
0875c52de7 Wrap PackageContext and SingletonContext
Wrap blueprint.PackageContext so that the *Func methods can provide
an android.Config instead of an interface{}.  The modified signatures
means that every method in ModuleContext and SingletonContext
that takes a blueprint.PackageContext now needs to be wrapped to
take an android.PackageContext.

SingletonContext wasn't previously wrapped at all, but as long
as it is, wrap everything like ModuleContext does.  This requires
updating every Singleton to use the android-specific methods.

Test: builds, all Soong tests pass
Change-Id: I4f22085ebca7def6c5cde49e8210b59d994ba625
2017-11-29 05:04:30 +00:00
Colin Cross
5e6cfbead9 Add searchable android.DirectorySortedPaths
Add an android.DirectorySortedPaths that stores paths sorted such
that all paths in a directory including subdirectories are in a
contiguous subslice.  This will allow efficient O(log(N)) finding
of all paths in a directory using a binary search on the directory
prefix.

Test: TestDirectorySortedPaths in paths_test.go
Change-Id: I5a06a89351ae06e88c06526be54a6b79075361b7
2017-11-13 15:11:08 -08:00
Colin Cross
b671544973 Move first/last unique elements utility functions to android package
Move firstUniqueElements to android.FirstUniqueStrings,
lastUniqueElements to android.LastUniqueStrings, and lastUniquePaths
to android.LastUniquePaths.

Test: m checkbuild
Change-Id: Ieac840405126c7f8f98afb4a4ef35c01a18fe7fb
2017-10-24 12:12:32 -07:00
Colin Cross
702e0f8ab0 Use full relative paths to get module outputs in tests
Make Rel() on ModuleOutPath and ModuleGenPath return the path
relative to the module out and module gen directories respectively,
and make TestingModule.Output() match against the full relative
path to the module.

Test: java_test.go still passes
Change-Id: Id5b2ec3fdef41d4169b943e68d032fc64a2b6f92
2017-10-19 20:44:11 +00:00
Jeff Gaston
294356f045 Automatically reorder C/C++ link dependencies in Soong
This uses knowledge of transitive dependencies to reorder
linker command line arguments such that if module A depends
on module B, then module A is automatically listed before
module B in the linker command line.

This should mostly remove the need for Android.bp files to
list all of their static dependencies in link order

Bug: 66260943
Test: reorder the entries of static_libs in an Android.bp and see that linking still succeeds

Change-Id: I20f851ab9f2f30031254e4f30023b6140d15d6c3
2017-10-18 14:30:05 -07:00
Jeff Gaston
af3cc2d23c Some clarifications in preparation to automatically order linker dependencies
Test: Browse the code and determine whether it's easier to understand
Bug: 66260943
Change-Id: I88c24a8a31ef68f428919087d206433659265684
2017-10-18 18:06:02 +00:00
Colin Cross
93e8595044 Initial kotlin support
Allow java libraries to specify .kt sources, precompile them with
kotlin, and then pass them in the classpath to javac.

Bug: 65219535
Test: java_test.go
Change-Id: Ife22b6ef82ced9ec26a9e5392b2dadacbb16546f
2017-10-17 16:25:22 -07:00
Colin Cross
0f37af0c15 Add java file resources and flag to include sources
Add a properties to allow including files as resources, including
support for filegroups.  Also add a flag that causes module sources
to be included in the final jar.

Test: java_test.go TestResources
Change-Id: Ida8ee59b28df9fe66952170f46470d3a09fd5d65
2017-09-29 14:02:22 -07:00
Dan Willemsen
866b563d4c Add target.linux_glibc and target.bionic
target.linux_glibc will apply to host builds with glibc, which is
identical to the current target.linux. In a future change, target.linux
will change to affect all targets using the Linux kernel (android,
linux_bionic, and linux_glibc).

target.bionic will apply to all OS variants using Bionic.

Bug: 31559095
Test: Add target.linux_glibc, target.bionic sections to an Android.bp, build
Test: m host
Change-Id: I677a67c22fba148fec264132311e355283f9d88d
2017-09-26 22:41:37 -07:00
Dan Willemsen
fe92c968b9 Remove more duplicated flags and dependencies
All the stats below are for a hikey960-eng build on AOSP master.

Number of order-only inputs in out/soong/build.ninja:
Before: 2847162
After:   606508

Size of:                       Before  After
out/soong/build.ninja          572MB   233MB  -59%
out/soong/Android-hikey960.mk   15MB     9MB  -40%
out/build-hikey960.ninja       367MB   358MB   -2%

Ninja time in `m nothing`:
Before: 6.9s
After:  5.0s

Soong generation time:
Before: 11.6s
After:   6.6s

Test: m nothing
Test: wrote script that counted inputs, only duplicates were removed
Test: treehugger
Change-Id: I6b7c5ef5b1395014b7bf5fd0b8112d42bee127bf
2017-08-29 13:39:46 -07:00
Dan Willemsen
00269f23ee Fix install location for vendor tests
These should be install in /data/nativetest* with the rest of the tests,
but had been moved to /vendor/nativetest* accidentally. Add some tests
so that this doesn't happen again.

Bug: 63393698
Test: m -j blueprint_tools
Test: compare out/soong/Android-aosp_arm64.mk
Test: compare out/soong/build.ninja
Change-Id: Id3b08a7e3908955df18a043a02ea576dc88086c3
2017-07-06 18:09:46 -07:00
Vishwath Mohan
87f3b24418 Fix install path for sanitized native tests. (Soong)
This CL fixes a bug with how Soong builds the install path for native
tests. Tests are installed in /data/nativetest* by default, but the
logic was wrongly redirecting sanitized versions to
/data/asan/system/nativetest*. With this fix, they are correctly
redirected to /data/asan/data/nativetest* instead.

Bug: 37942061
Test: lunch marlin_asan-userdebug && \
        SANITIZE_TARGET="address" m -j70  \
        # nativetests are generated in /data/asan/data/nativetest*

Change-Id: I0c132af5d443151c44219d231770049fddf79bbe
2017-06-07 12:31:57 -07:00
Jayant Chowdhary
715cac3971 Optimizations to abi checking.
We now add export_static_lib_headers, export_generated_headers to the
filters while dumping the abi of a library using header-abi-dumper
(through -I<dir> additions to the invocation of header-abi-dumper and
header-abi-linker)

Also add support for zipped reference source based abi dumps.

Test: mm -j64 in hardware/interfaces/nfc/default/1.0 produces
android.hardware.nfc@1.0.so.lsdump with abi filtered out using generated
headers.

Test: Copied the linked abi dumps produced by mm -j64 in bionic/libc to
prebuilts/abi-dumps/ndk and gzipped them. Ran mm -j64 again in
bionic/libc and verified header-abi-diff getting invoked.

Bug: 32750600

Change-Id: I26210af908c87a6143e39fa25f50307acb68a387
2017-05-25 14:46:58 -07:00
Jeff Gaston
734e3809ad Clarify paths.go somewhat
Bug: 35562758
Test: make

Change-Id: I4cf05ffdd0962186f490467829e3576b4e269c47
2017-05-15 11:31:48 -07:00
Colin Cross
5b52959c99 Use ctx.ModuleBuild for darwin ar
Test: builds
Change-Id: If90975c8545158012bc6201acadd136363c21260
2017-05-09 14:32:45 -07:00
Jayant Chowdhary
3e231fd8bd Add header-abi-checker for Vndk abi checks.
header-abi-dumper: dumps abi exported by source files for Vndk.
header-abi-linker: links abi dumps produced by header-abi-dumper.
header-abi-diff: compares linked dumps.

Test: mm -j64 showcommands > make_log in bionic/libc.
      This produced linked dumps in out/soong/.intermediates.
      Copied these dumps to
      prebuilts/abi-dumps/ndk/current/arm64/source-based/.
      Changed the abi and re-ran mm -j64 showcommands > make_log
      confirmed that the build reported compatibility breakge without
      actually failing (advisory mode).

Change-Id: Iccad6908fe68a80f47230751671d156893b96ead
2017-04-14 19:48:10 -07:00
Dan Willemsen
aa118f95a6 Implement vendor as a synonym of proprietary
The vendor image will have more than just proprietary modules in it
under Treble, so let's stop marking open source code as proprietary just
to move it to vendor.

Bug: 36452052
Bug: 37134596
Test: compare build.ninja before/after, no changes.
Test: Set vendor: true, ensure it works.
Change-Id: I44b0ec7007d0e311bdcbd44b238b1ef2d05cc6ff
2017-04-07 14:59:18 -07:00
Vishwath Mohan
1dd8839286 Refactor sanitized library on-disk layout - Soong.
This CL moves the location of ASAN-ified libraries on disk in the
following manner:
/data/lib* --> /data/asan/system/lib*
/data/vendor/* --> /data/asan/vendor/*

There are a couple of advantages to this, including better isolation
from other components, and more transparent linker renaming and
SELinux policies.

Bug: 36574794
Bug: 36674745
Test: m -j40 && SANITIZE_TARGET="address" m -j40 and the device
boots. All sanitized libraries are correctly located in /data/asan/*.

Change-Id: I06bf459260ec451d4495a02562f640ad622f34c2
2017-03-31 01:03:56 +00:00
Colin Cross
faeb7aa135 Support data properties for test binaries
Allow tests to specify a data property that lists files or filegroup
modules that will be packaged alongside the test.  Also add a path
property to filegroup modules to allow shifting the path of the
packaged files, and add ExpandSourcesSubDir to expand the filegroup
sources while including a shifted relative path in the Paths objects.

Test: soong tests, manually adding data to a module
Change-Id: I52a48942660e12755d313ef13279313361b4fc35
2017-02-02 16:23:30 -08:00
Colin Cross
294941bee9 Update soong to use pathtools.FileSystem
Update soong to follow changes in
https://github.com/google/blueprint/pull/141

Test: soong tests
Change-Id: I49a9b83cac7590dc75b26b31136b8707c188bc4a
2017-02-02 16:19:56 -08:00
Colin Cross
7fc17dbfce Remove extra basePath from ModuleSrcPath
ModuleSrcPath contains an embedded basePath as well as a
SoucePath that contains another basePath.  Remove the embedded
basePath, and make the SourcePath embedded.

Test: no change to build.ninja
Change-Id: I3cdf3477eca41ed35fac08a892aab22cbcdb2224
2017-02-01 15:14:31 -08:00
Dan Willemsen
4353bc4422 Support system/vendor for devices without a vendor partition
Test: lunch aosp_arm-eng; m -j vulkan.default
Change-Id: I91eabf98f53c455ec4ade95215f11a6f89631747
2016-12-05 17:21:13 -08:00
Colin Cross
7f19f37443 Move globbing to Blueprint
Move Soong's globbing-with-dependencies support into Blueprint so it can
be used for subdirs= lines in Android.bp files.

Blueprint has a slight change in behavior around subname= lines, it now
always uses the subname and doesn't fall back to Blueprints.  To support
the Blueprints files in build/blueprint, use them directly with build=.

Test: build, add source file that matches glob, rebuild
Change-Id: Ifd0b0d3bc061aae0a16d6c7ca9a1cd8672656b4d
2016-11-04 04:54:16 +00:00
Dan Willemsen
21ec49068f Add subdir to GenPath
We were emulating this for proto files, standardize it and make the
other generators use it as well.

Test: Compare out/soong/build.ninja before/after change
Test: mmma -j system/tools/hidl
Change-Id: I1888c7b981749060a398387bbb9b481270bf6d75
2016-11-02 21:19:11 -07:00
Colin Cross
4f6fc9c1d8 Add Path.Base helper
Add Path.Base to return the basename of a Path object.

Test: builds
Change-Id: I2bf1519dc5039bedb1586e7ec18297a293e32e11
2016-10-27 15:18:17 -07:00
Colin Cross
a1ad8d1889 Simplify arch target handling
Soong's multi-architecture building has grown complex, with the
combination of HostOrDevice+HostType+Arch necessary to determine how to
build a variant of a module, and three separate mutators to split each
into its variations.

Combine HostOrDevice+HostType into Os, which will be Linux, Darwin,
Windows, or Android.  Store Os+Arch as a single Target.

Change-Id: I92f2e2dac53617d595a35cc285d2bd348baa0fbd
2016-06-02 19:09:32 -07:00
Colin Cross
b9db480385 Revert "Simplify arch target handling"
This reverts commit 6713fb26cbcadf525cd75e47d7d0cbc23d282b3e.

Change-Id: Ic473cea2563b0b37dc08b0bc5d3a0ac8c4b6afe6
2016-06-03 01:50:47 +00:00
Colin Cross
54c7112c43 Simplify arch target handling
Soong's multi-architecture building has grown complex, with the
combination of HostOrDevice+HostType+Arch necessary to determine how to
build a variant of a module, and three separate mutators to split each
into its variations.

Combine HostOrDevice+HostType into Os, which will be Linux, Darwin,
Windows, or Android.  Store Os+Arch as a single Target.

Change-Id: Iae677eff61a851b65a7192a47f2dc17c1abb4160
2016-06-02 17:37:02 -07:00
Colin Cross
635c3b0157 Rename common to android
Rename the "common" package to "android", because common is too
generic.  Also removes all android.Android naming stutter.

Ran:
gomvpkg -from 'android/soong/common' -to 'android/soong/android'
gorename -from '"android/soong/android".AndroidModuleContext' -to 'ModuleContext'
gorename -from '"android/soong/android".AndroidBaseContext' -to 'BaseContext'
gorename -from '"android/soong/android".AndroidModuleBase' -to 'ModuleBase'
gorename -from '"android/soong/android".AndroidBottomUpMutatorContext' -to 'BottomUpMutatorContext'
gorename -from '"android/soong/android".AndroidTopDownMutatorContext' -to 'TopDownMutatorContext'
gorename -from '"android/soong/android".AndroidModule' -to 'Module'

Change-Id: I3b23590b8ce7c8a1ea1139411d84a53163288da7
2016-05-18 15:37:25 -07:00
Renamed from common/paths.go (Browse further)