Currently, the command line argments is being processed in multiple
places. In the main soong_ui arguments, there are several if statements
that checks if a specific argument was specified in order to execute
the requested operation. This does not scale well when adding or
removing a command in the near future. In order to support the build
commands (m, mma, etc...) from the envsetup.sh in soong_ui, a refactor
was required in order to add a command rather quickly and to have the
flexibiity to unit test the command. The soong_ui arguments format is
as follows:
soong_ui <command> [<arg 1> <arg 2> ... <arg n>]
The <command> is a specific operation to be executed. The arguments
after the command are processed by the command itself.
Below is the list of changes made in this commit:
* Created a new command infrastructure that allows adding
or deprecating a command easily.
* Fixed a bug when running ./soong_ui.bash directly would cause
a panic due to index being out of range for args list.
Bug: b/130049705
Test: Below is the list of testing done on this commit:
* Ran lunch and verified that the output is the same as the the output
without the modifications. lunch indirectly runs soong_ui passing
in the --dumpvar-mode (to read makefile variables such as
TARGET_PRODUCT) and --dumpvars-mode (to build the build cache).
* Ran ./soong_ui.bash directly (with unsupported flags and no flags)
and the proper message appeared that soong native UI is not yet
available.
* Ran m, mm, mmm, mma, mmma commands.
* Ran the make installclean command:
make -j80 TARGET_PRODUCT=aosp_arm64 TARGET_BUILD_VARIANT=eng dist DIST_DIR=/tmp/helloworld installclean
* Ran ./out/soong_ui -j80 --make-mode PRODUCT-aosp_arm64-eng dist checkbuild tests
Change-Id: Iee4de7ec0fa4661206fda8ae1fe6fa4487d9bb22
Merged-In: Iee4de7ec0fa4661206fda8ae1fe6fa4487d9bb22
This changes the default behavior of pom2bp to match pom2mk, and not
require pre-extraction of all of the manifests.
Test: convert an internal pom2mk user to pom2bp
Change-Id: I08120cce1f923ef28519563ffd772032b2e500ed
Allow callers of pom2bp to add dependencies on runtime libraries as
well as static libraries. Renames -extra-deps to --extra-static-libs
to make it clearer what it does.
Bug: 130287656
Test: prebuilts/sdk/update_prebuilts.py --stx 5280039
prebuilts/misc/common/androidx-test/update-from-gmaven.py
Change-Id: I8a3ee8056aa4ed9935a4f47d563ec423672cf38b
This will prevent it from being affected by future css location changes.
Test: m soong_docs
Fixes: 130567020
Change-Id: I80fd7b8a12a32bcea7b270423b8d8f8dc0976177
This essentially allows you to declare that everything in a directory
will be created by the rule, and we'll ensure that your command actually
writes out all of the claimed outputs, and remove any other files that
previously existed in that directory.
Test: built-in tests
Change-Id: I990dce2b3a0d89ebd2736ac1a0cadfb5864c6e73
The dependency type in the pom file is not always accurate, just
add all jar and aar dependencies to both the -nodeps and final
module.
Test: cd prebuilts/sdk/current/androidx && pom2bp -regen Android.bp && mma
Change-Id: I1784b8ebf3485db19405cb353c9bb5fb3ae837d8
android/soong/common was renamed to android/soong/android long
ago, but the pctx package path was still "android/soong/common".
This required all users of rules defined in android/soong/android
to import "android/soong/android" and then
pctx.Import("android/soong/common").
Test: m checkbuild
Change-Id: I20d096522760538f7cfc2bec3d4bfeba99b275d4
* changes:
Add a whitelist for services.art changing between builds
Add diff_target_files tool
Import files from compare_target_files for use in diff_target_files
Some tools used by genrules set the timestamp of the generated file
to an old value. For example, when using tar to extract files it
will set the timestamp of the extracted file to the value stored in
the tar file. This can cause unnecessary rebuilds due to output files
that are older than their inputs. Touch output files in sbox before
moving them into the output location.
Test: m checkbuild && m checkbuild
Change-Id: Idb0e71d2bf0ac3618338371132cc71be74bf9d9d
Add a tool that lists files that have changed between two builds by
comparing the target-files.zip artifacts.
Bug: 121158314
Test: unit tests
Change-Id: I3ca5dd1fdf652dffef923ad7307fd71e49e2e95a
Allow rules built with RuleBuilder to use depfiles. Ninja only
supports a single depfile with single output. If there are
multiple outputs in a rule, move all but the first to implicit
outputs. If multiple depfiles are specified, use new support
in dep_fixer to combine additional depfiles into the first depfile.
Test: rule_builder_test.go
Change-Id: I7dd4ba7fdf9feaf89b3dd2b7abb0e79006e06018
Bring property type strings to the start of property comments. This is
how source.android.com shows variables and paratemeters, and it is also
easier to read, especially when a comment ends with a code example.
Test: m soong_docs
Change-Id: Ic3e9db3a3711495998919ee95bbef6c7cb33b215
To add code format (<pre>...</pre>) support to module type texts.
Fixes: 124332708
Test: m soong_docs
Change-Id: Ibe9d3845a8f952fdef5007893d4af5836809ce1f
Follow the change to blueprint to make *Escape take and return a string
and add *EscapeList that take and return slices of strings. Fix up
a few places that were unnecessarily converting a string to a slice
and back to a string.
Test: m nothing
Change-Id: I3fa87de175522205f36544ef76aa2f04aef1b936
The href base added by I8a8ac0f9ba6c0d57ed83db2155955a95e070a265
doesn't really play nice with anchor links. This fixes the issue.
Fixes: 126550408
Test: m soong_docs
Change-Id: If62411a6391fd8151fa40cb2ecf32c5006797d6c
This adds a script to copy the current page's base url to hrefs in the
doc.
Fixes: 123473548
Test: m soong_docs
Change-Id: I8a8ac0f9ba6c0d57ed83db2155955a95e070a265
Previously the build doc was a gigantic list of modules and properties,
which can be overwhelming to new users. This change breaks it up by
packages, so that it is easier to look up and feels more coherent.
Bug: 123521276
Test: m soong_docs
Change-Id: I1a1a331f4b4deb8782d698e9076098c5a27b9566
Uses more python rather than C++, and skips less of Py_Main.
Test: build/soong/python/tests/runtests.sh
Change-Id: I03997d88e2e16047c96bb4e00e530229c42b3325
It now has module list on the left and the main panel containing
a section for each module. Each section contains the navigable list
of the module's attributes, grouped by attribute's origin.
Test: Manual
Bug: 120512870
Change-Id: Ib62ef58c61daa88950f0934a70ba8ed16f433bf2
bootstrap.ModuleTypeDocs needs a mapping of module types to factories
to support factories that are wrapped in ModuleFactoryAdapter closures.
It also returns ModuleType objects grouped into Package objects.
Bug: 67909957
Test: m soong_docs
Change-Id: I70eac9f0f0e13075580da92d4219792ca0b18fbf
If ANDROID_QUIET_BUILD environment variable is set to '1', do not show 20
lines of the environment variables. In addition, for the failing step show
only the step name and output, omitting the command proper (the verbose log
as well as error.log still contains it).
If build succeeds, the output of the build itself consists of a single
line:
```
> m androidmk
[100% NN/NN] <last command>
```
When it fails, the output does not contain sometimes very long command
line:
```
> m androidmk
[ 97% NN/MM] test androidmk
FAILED: <step>
--- FAIL: TestEndToEnd (0.01s)
androidmk_test.go:1025: failed testcase 'prebuilt_etc_TARGET_OUT_ETC'
input:
include $(CLEAR_VARS)
LOCAL_MODULE := etc.test1
LOCAL_MODULE_CLASS := ETC
LOCAL_MODULE_PATH := $(TARGET_OUT_ETC)/foo/bar
include $(BUILD_PREBUILT)
expected:
prebuilt_etc {
name: "etc.test1",
filename: "foo/bar",
}
got:
prebuilt_etc {
name: "etc.test1",
filename: "foo/bar",
}
FAIL
17:50:53 ninja failed with: exit status 1
```
[The related change in build/make/envsetup.sh suppresses timing display
when the same variable is set.]
Change-Id: I4d3c72457de031ff58a324c2fe98f4c1d10f8239
Test: treehugger
Test: Dumped the text formated based metrics file to out dir,
and checked the file.
Bug: b/63815990
Change-Id: Iff476f72a0be74eb53b6b26ef468d11c0f24a404
The zip file format does support scripts/programs prefixed to the
archive, though many of the offsets are supposed to start from the
beginning of the file. Some tools (python and zipinfo) are able to
read zip files with arbitrary prefixes, but others (libziparchive and
zipdetails) fail.
So pass the file to prefix onto the zip file into merge_zips so that we
can set the offsets from the real start of the file.
Test: m sepolicy_tests (runs the embedded python interpreter)
Test: zipinfo out/host/linux-x86/bin/sepolicy_tests
Test: zipdetails out/host/linux-x86/bin/sepolicy_tests
Change-Id: If73d4c2465581f7de5aa47959284ecf2059df091
Gold uses two PT_LOAD programs: RX, RW
LLD is using three PT_LOAD programs: R, RX, RW
All non-static host bionic executables are linked with LLD, as it's
required in order to embed the linker. The dynamic linker is considered
a static executable for this definition.
When the linker had been built with Gold, we inserted the the RX&RW
loads in the middle of the executable's readonly load, splitting the R
load into two. When this happened, all of the linker's loads existed 1:1
in the final executable.
Now that the linker is built with LLD, we're inserting the loads into
the same place, but LLD merges the first part of the executable's R load
with the linker's R load, so there's not a 1:1 matching.
So just make sure that all of the linker's loads are contained within
the executable's loads (and with the appropriate flags).
Test: m blueprint_tools (runs new tests)
Test: build host bionic binaries
Change-Id: If7868d2ab8980d27ba5ab8994fefe0d6142f215a
Patterns containing multiple globs or a recurisve glob may match
files with the same name in multiple directories. Keep the relative
directories of matches after the path entry containing a glob.
Bug: 117295826
Test: zip2zip_test.go
Change-Id: I5d663e546953af374175837551d23f484d568377
Stop writing out ninja files, as they become very large, especially when
multiplied by the number of defined products.
Test: treehugger
Change-Id: Id9529d14040acb72a0188e58b5db2911f142071e
In order to simplify the wrapper function, and stop using a linker
script, generate a set of flags to pass to LLD. Then run
host_bionic_inject on the linked binary in order to verify the
embedding, and give the wrapper function the address of the original
entry point (_start).
Bug: 31559095
Test: build host bionic with prebuilts/build-tools/build-prebuilts.sh
Change-Id: I53e326050e0f9caa562c6cf6f76c4d0337bb6faf
Don't warn on duplicate files in merge_zips if they have identical
CRC32 and size values.
Test: m checkbuild
Test: merge_zips_test.go
Change-Id: I61336ca4d4d3b7402c24a7abd337bd350fe10930
Make needs to exclude all files that match a filespec except for
those specified in a list. Add a -X argument that overrides the
-x argument to allow files be included that were previously
excluded.
Bug: 69500920
Test: zip2zip_test.go
Change-Id: Icc8aebc788b53bcdde73a39c9c7cf3107a049251
Add -0 flag to convert files in a zip to stored instead of deflated.
Bug: 69500920
Test: zip2zip_test.go
Change-Id: I6c2b10f3b200a53a3339e3c97a78f65192b309ca
Make the cleanspec & Android.mk runs of kati use much of the same code
and arguments.
Also renames 'Kati' to 'KatiBuild' in many cases, in the the expectation
that we'll have a 'KatiPackage' step in the future.
Use --no_ninja_prelude and move local_pool & _kati_always_build_ into
the combined ninja file. This will reduce the need to re-read makefiles
when Goma is enabled, and it allows us to include more than one
Kati-generated ninja file in the build graph.
Bug: 116968624
Test: build_test on downstream branches
Change-Id: Ibdac689b81f62dc293647fad917d84946f2c3cfa
Add test that cover basic command line usage of soong_zip. -D
is not covered yet as the implementation will be replaced with
one that is also more easily testable in the next patch.
Bug: 116751500
Test: zip_test.go
Change-Id: I5a1bcee74ebc9cb3cf332c36f89bc12c0e807ad2
Move the actual per-product builds out of the main function, combining
the product configuration and build sections. This makes it a lot more
readable.
This ends up saving a lot of FDs -- a -only-config build on aosp with 77
products used to require ~500 FDs, it succeeds with a limit of 150 FDs
now. I'm leaving the code to bump our FD limit though, since 2x the
number of internal products is too close to the 1024 limit for comfort.
Bug: 70370883
Test: prlimit -n150:150 build/soong/build_test.bash -only-config
Change-Id: Ia559beadc19deb8a5b9d50af6e0276e846fd8608
This uses a very large amount of intermediates, but can speed up
subsequent multiproduct_kati runs by not re-reading all the makefiles
unless necessary.
Since we'll no longer have the full build output on every run,
explicitly save away a std_full.log of the log from the last time we ran
Kati.
Test: build/soong/build_test.bash -products aosp_arm -incremental
Change-Id: Iaae580c0e49a2024528617d39b2b477e29d99250
Move some of the more complicated conditional logic out to helper
functions. Use the {{- }} syntax to strip previous whitespace to
allow spacing out the remaning conditionals in the templates.
Test: (cd prebuilts/sdk/current/androidx && pom2bp -regen Android.bp)
Change-Id: I766bd0e1837aa04375f322fbe796d923cd99ecde
