Commit graph

4 commits

Author SHA1 Message Date
Dan Willemsen
fb1271a52b Add a Kati-based packaging step
The idea is that we'd move the installation and packaging tasks over to
it, using data from Soong & the Kati reading Android.mk files.

This would allow us to make more fundamental changes about how we
package things without having to adjust makefiles throughout the tree.
Possible use cases:

* Moving some information from Soong's Android.mk output to a file read
  by the packaging step may allow us to read the Android.mk files less
  often, speeding up builds.

* Refactoring our current two-stage ASAN builds to run the Kati build
  step twice, writing into different object directories, then have a
  single packaging step that reads both outputs. Soong already has the
  capability of writing out a single ninja file with all the asan
  combinations.

* Running two build steps, one building the system-related modules
  using a "generic" device configuration, and one building the vendor
  modules using a specific device configuration. This could enforce a
  GSI/mainline system vs vendor split in a single build invocation.

* If all installation is through this tool, it will be much easier to
  track what should no longer be installed on an incremental build,
  reducing the need for installclean.

* Changing PRODUCT_PACKAGES should be a much faster operation, which
  means we could keep track of local additions to the images. Then
  `mma` would be more persistent, instead of installing something once,
  then never updating it again.

Eventually we plan on switching from Kati to something Go-based, but
this is a more incremental approach while we clean up everything else.

Currently, this just moves the dist-for-goal handling over to the
packaging step, so that we don't need to read Android.mk files when
DIST_DIR changes, or we switch between dist vs not.

Bug: 116968624
Bug: 117463001
Test: m nothing
Change-Id: Idec5ac6f7c7475397ba0fb65bd3785128a7517df
2018-10-19 09:55:00 -07:00
Dan Willemsen
59339a29e1 Fix go vet issues
Test: go vet ./...
Change-Id: Ifb936ccc5e2b5a2c3fcbbbcb54f680e2973ea1b3
2018-07-22 21:18:45 -07:00
Dan Willemsen
ed86952403 Normalize LANG / LC_* environment
This ensures that the current locale supports UTF-8, and that we're
getting a consistent (but still supported by the system) locale for
every configuration except user-facing messages. This should eliminate
any reproducibility problems around sorting, formatting, etc for all
built products, while still showing localized error messages where
available.

Bug: 71573630
Test: LANG=es_ES LANGUAGE=es: m   (check env in soong.log)
Change-Id: If33311899eaed8c44573113ee35c5a71cee503a0
2018-01-19 21:46:21 +00:00
Dan Willemsen
1e70446251 Add a Go replacement for our top-level Make wrapper
Right now this mostly just copies what Make is doing in
build/core/ninja.mk and build/core/soong.mk. The only major feature it
adds is a rotating log file with some verbose logging.

There is one major functional difference -- you cannot override random
Make variables during the Make phase anymore. The environment variable
is set, and if Make uses ?= or the equivalent, it can still use those
variables. We already made this change for Kati, which also loads all of
the same code and actually does the build, so it has been half-removed
for a while.

The only "UI" this implements is what I'll call "Make Emulation" mode --
it's expected that current command lines will continue working, and
we'll explore alternate user interfaces later.

We're still using Make as a wrapper, but all it does is call into this
single Go program, it won't even load the product configuration. Once
this is default, we can start moving individual users over to using this
directly (still in Make emulation mode), skipping the Make wrapper.

Ideas for the future:
* Generating trace files showing time spent in Make/Kati/Soong/Ninja
  (also importing ninja traces into the same stream). I had this working
  in a previous version of this patch, but removed it to keep the size
  down and focus on the current features.
* More intelligent SIGALRM handling, once we fully remove the Make
  wrapper (which hides the SIGALRM)
* Reading the experimental binary output stream from Ninja, so that we
  can always save the verbose log even if we're not printing it out to
  the console

Test: USE_SOONG_UI=true m -j blueprint_tools
Change-Id: I884327b9a8ae24499eb6c56f6e1ad26df1cfa4e4
2017-02-06 14:05:07 -08:00