Commit graph

967 commits

Author SHA1 Message Date
Colin Cross
76f2f97c51 Make defaults TopDownMutator parallel
Append .Parallel() to the defaults RegisterTopDownMutator call to tell
Blueprint it can run it in parallel.  Saves ~500ms in soong_build.

Change-Id: I43ddd9d6995674ccc06fed6928514f15a15712c1
2016-08-11 17:15:07 -07:00
Colin Cross
91169fe8ea Fix coverage sanitizer builds
Disable coverage for static binaries where address sanitizer is
disabled, and disable coverage in the mutator when disabling address
sanitizer.

Bug: 29188876
Change-Id: Ia1a21878c3f34cd295a6dec49608c412eb09e7b1
2016-08-11 16:54:01 -07:00
Colin Cross
e0b666c229 Merge "Make BottomUpMutators parallel"
am: aa99f84e57

Change-Id: I49cd56f16258f5405515275309bcb0329a32e639
2016-08-11 21:45:17 +00:00
Treehugger Robot
aa99f84e57 Merge "Make BottomUpMutators parallel" 2016-08-11 21:35:24 +00:00
Colin Cross
e8a67a7c58 Make BottomUpMutators parallel
Append .Parallel() to all of the RegisterBottomUpMutator calls to tell
Blueprint it can run them in parallel.

Test: identical build.ninja, passes race detector
Change-Id: I969a0689522d4cba7c8ff51e2aa00fe2fd338a89
2016-08-11 13:20:16 -07:00
Dan Willemsen
da8a307625 Merge changes from topic 'fast_dist'
am: 8bdd631a75

Change-Id: I4a3e8b06e155d8c7f830b4db1790461943278d8c
2016-08-11 19:33:14 +00:00
Treehugger Robot
8bdd631a75 Merge changes from topic 'fast_dist'
* changes:
  Rename soong_jar to soong_zip
  soong_jar: Parallel compression
  Add zip2zip tool to copy zip entries from one file to another
  Add archive/zip from go1.7rc5 tag
2016-08-11 19:28:14 +00:00
Dan Albert
585738ad21 Add support for NDK r13.
am: d015c4ac1b

Change-Id: I1d1c599bbe918537fb4afd4ca568937ee7859923
2016-08-11 16:52:59 +00:00
Dan Willemsen
47ec28f3b1 Rename soong_jar to soong_zip
This is a general purpose tool that happens to contain some jar specific
features.

Change-Id: I05f4654d4517c245ad7a3c15492e0d2368bbf64f
2016-08-11 01:30:16 -07:00
Dan Willemsen
017d8937fa soong_jar: Parallel compression
This compresses multiple files in parallel, and will split up larger
files (5MB+) into smaller chunks (1MB) to compress in parallel.

There is a small size overhead to recombine the chunks, but it's only a
few bytes per chunk, so for a 1MB chunk, it's minimal.

Rough numbers, with everything in the page cache, this can compress
~4GB (1000 files) down to 1GB in 6.5 seconds, instead of 120 seconds with
the non-parallel soong_jar and 150 seconds with zip.

Go's DEFLATE algorithm is still a bit worse than zip's -- about 3.5%
larger file sizes, but for most of our "dist" targets that is fine.

Change-Id: Ie4886c7d0f954ace46e599156e35fea7e74d6dd7
2016-08-11 01:30:16 -07:00
Dan Albert
d015c4ac1b Add support for NDK r13.
Test: make checkbuild
Bug: None
Change-Id: I6e7c92d6731be4ae47c7dc5e9444af08d338334b
2016-08-10 23:14:13 -07:00
Dan Willemsen
3bf1a08505 Add zip2zip tool to copy zip entries from one file to another
This doesn't do any decompression / recompression, but just copies over
the already compressed contents. So it's similar to zip -U, but allows
rewriting of the paths.

The first expected usecase is to replace img_from_target_files during
the build, since it does the equivalent of this:

zip2zip -i <target-files.zip> -o <img.zip> OTA/android-info.txt:android-info.txt IMAGES/*:.

Except it decompresses and recompresses the images, which takes over a
minute instead of a few seconds.

Change-Id: I88d0df188635088783223873f78e193272dbdf1c
2016-08-10 16:57:03 -07:00
Dan Willemsen
25a4e07df8 Add archive/zip from go1.7rc5 tag
In preparation to patch in some custom functionality (parallel
compression and zero-decompress zip to zip copying)

Change-Id: I96a36efc09c715f6b55290af40ebfdde9ae72e33
2016-08-10 16:10:16 -07:00
Colin Cross
adbf48c3df Convert defaults to a top down mutator
am: 1317701114

Change-Id: I3736f583432d56a7dd1889ba948b9253158c9543
2016-08-09 23:23:01 +00:00
Colin Cross
476861290e Update build.ninja.in for blueprint fs.go changes
am: 6debdbad42

Change-Id: I2fd3171f980ff02ff2b7665f80fdbed3535f489e
2016-08-09 22:07:06 +00:00
Colin Cross
1317701114 Convert defaults to a top down mutator
BottomUpMutators are going to lose their ctx.Visit* functions in order
to allow parallelizing them, move defaults to a TopDownMutator using
WalkDeps to only visit defaults modules.

Test: no changes to out/soong/build.ninja
Change-Id: I54ba65a7e2ae9503f4d217f63aa9178a7c5341f0
2016-08-09 13:54:03 -07:00
Colin Cross
6debdbad42 Update build.ninja.in for blueprint fs.go changes
Change-Id: I05bca49a563b13731ccb89d6f073edd59fee7049
2016-08-09 13:40:08 -07:00
Colin Cross
3099506f27 Merge "Use nil pointers for arch property structs"
am: 142d4fd01d

Change-Id: I006c7325e0b16aeeb947788545184fc96e018cf5
2016-08-09 00:19:18 +00:00
Treehugger Robot
142d4fd01d Merge "Use nil pointers for arch property structs" 2016-08-09 00:08:23 +00:00
Dan Albert
0f2f0fa282 More fixes for decorator refactor.
am: cdd4c24ff7

Change-Id: I169c2e8271d5e446f1a8a7fd358672c60c6b93c0
2016-08-08 23:13:01 +00:00
Colin Cross
62496a0d2e Use nil pointers for arch property structs
Blueprint can now handle creating a zero-valued struct when it
encounters a nil pointer to a struct, reducing the amount of allocations
and copying for arch structures that are not used.

Change-Id: If36da5603dbe6469fe6406c821f21a122ea6ea49
2016-08-08 15:56:19 -07:00
Dan Albert
cdd4c24ff7 More fixes for decorator refactor.
Didn't catch these the first time because I hadn't done a clean build.

Test: rm -rf out && make ndk
Change-Id: Ia66ad614d2163272889ee6937a6449c44db16ac0
2016-08-08 14:45:24 -07:00
Dan Albert
80f3a040cd Fix ndk_library following decorator refactor.
am: 705c84b9ee

Change-Id: I228224014450f4cb2cdbf13bdcf3d94e8f95aa23
2016-08-08 20:49:21 +00:00
Dan Albert
705c84b9ee Fix ndk_library following decorator refactor.
https://android-review.googlesource.com/#/c/253603/ accidentally
dropped the ndk_library properties and was using the base cc
properties instead.

Change-Id: Ia2dcd767f98871dbdbbd9d0ff6d2a73b9b4889ac
2016-08-08 12:44:23 -07:00
Dan Albert
df66aa3ecf Don't generate intermediate NDK libraries.
am: 7fa7b2efd3

Change-Id: I049ab976019bf085943cc4cf0bceb68e0c6ef59b
2016-08-08 18:44:14 +00:00
Dan Albert
7fa7b2efd3 Don't generate intermediate NDK libraries.
It's better to clip to the existing versions here rather than build
the lower levels as if they existed.

cc.Module's begin() will now normalize the sdk_version to a valid
value for the given architecture so that doesn't have to be checked
later.

Test: make checkbuild with all other libc ndk_library changes
Change-Id: I2fa834f7a1ac3b03a0723291778d86207fcfda72
2016-08-05 16:48:36 -07:00
Dan Willemsen
f7f5b8c1b3 Merge changes I5144bc62,Id6d6561f
am: 8a274392ac

Change-Id: I62a802f7ad60f9bd3a50ecf7f462642b10516fbd
2016-08-05 20:54:00 +00:00
Dan Willemsen
8a274392ac Merge changes I5144bc62,Id6d6561f
* changes:
  soong_jar: Hardcode time
  soong_jar: Implement symlinks, fix directories
2016-08-05 20:44:25 +00:00
Colin Cross
91790d64c7 Refactor cc modules to use decorators instead of inheritance
am: b916a38233

Change-Id: I3b43134205a739aa6e0baf2aef56507d1e1fa221
2016-08-05 18:45:32 +00:00
Colin Cross
987c4de3cd Remove unused flags.Nocrt
am: 01344df46e

Change-Id: I1c8c1dbf408072a1c6fb644151a3e33182665e52
2016-08-05 18:45:31 +00:00
Dan Willemsen
77a6b8655a soong_jar: Hardcode time
We want reproducible builds, so timestamps are not useful.

Change-Id: I5144bc62ac37c3c9d7e0638f3b3648a60d887702
2016-08-05 11:22:40 -07:00
Dan Willemsen
a59a3bc5ec soong_jar: Implement symlinks, fix directories
Ensure that we only create one directory entry, not one with a / at the
end and another without. Clean incoming paths to prevent other similar
duplicates.

When adding directories specified directly as inputs, use the relative
path just like files, not the full path.

Instead of traversing symlinks, addg them to the zip file directly.

Present an error instead of crashing when an input does not exist.

Change-Id: Id6d6561fe12c28398f6096bf9bcae602bb7aa491
2016-08-05 11:22:40 -07:00
Colin Cross
b916a38233 Refactor cc modules to use decorators instead of inheritance
For example , instead of trying to have libraryLinker inherit from
baseLinker and libraryCompiler inherit from baseCompiler, create a
single decorator object that wraps both baseLinker and baseCompiler.

Test: Builds, no unexpected changes to build.ninja
Change-Id: I2468adaea8466c203a240259ba5694b8b1df7a52
2016-08-05 10:25:09 -07:00
Colin Cross
01344df46e Remove unused flags.Nocrt
Nocrt is handled through deps.CrtBegin/End, flags.Nocrt is unused

Change-Id: I8ab5313bb2eb3a00e336504d6dd3813350fbe14d
Test: soong builds
2016-08-05 10:22:29 -07:00
Dan Albert
cbfe300d8a Auto-fill ndkMigratedLibs.
am: 7e9d295807

Change-Id: I2d20a3e4b7b87b86902fccd06e263f32d39a1494
2016-08-05 00:32:59 +00:00
Dan Albert
ae9175cefd Merge "Check NDK API level link compatibility."
am: 5c395b39b4

Change-Id: I1ffa2f563e0693fbf95be28d777dfe7ea4f7d981
2016-08-04 23:21:54 +00:00
Dan Albert
7e9d295807 Auto-fill ndkMigratedLibs.
Change-Id: Ifb91f62eccb17dedb71c1a9a0f5c1471edbcbd44
2016-08-04 16:10:54 -07:00
Treehugger Robot
5c395b39b4 Merge "Check NDK API level link compatibility." 2016-08-04 23:07:16 +00:00
Colin Cross
0a8fe7cf0d Merge "Don't cache InstallInData"
am: a7d1d4b0aa

Change-Id: I37aaf58a0e3d4b4b0ceee66d561576aea5163154
2016-08-04 21:43:18 +00:00
Colin Cross
a7d1d4b0aa Merge "Don't cache InstallInData" 2016-08-04 21:02:56 +00:00
Dan Albert
a1b846602f Use the version script for ndk_library.
am: 2bc91ba549

Change-Id: I5b07296ad3f458f901ee99297eefc6d5b536896c
2016-08-04 18:23:34 +00:00
Dan Albert
9e10cd4746 Check NDK API level link compatibility.
Mixing NDK API levels is only supported when linking against something
equal or older.

Change-Id: I2c3022a3d75465b736f2e65160da3113cfd6e635
2016-08-03 14:31:25 -07:00
Colin Cross
8d8f8e21cc Don't cache InstallInData
Change-Id: If38670cc68989a7ec11ba5f8c71b1fac6c69f3a6
2016-08-03 11:57:50 -07:00
Dan Albert
2bc91ba549 Use the version script for ndk_library.
We were generating this, but not actually using it yet.

Test: `make ndk` with libc/libm migration patches, readelf generated
      libs.
Change-Id: I9aadf58307bc645b33cab5207409552c5046937a
2016-08-02 13:45:40 -07:00
Dan Albert
4216e1e6f0 Merge changes I0f73b41a,I0cd2f80c
am: f77e8d7ffc

Change-Id: I4ddecc0a430e413016e65898470c7661829ca895
2016-08-02 01:33:33 +00:00
Treehugger Robot
f77e8d7ffc Merge changes I0f73b41a,I0cd2f80c
* changes:
  Add a real parser for the version scripts.
  Add tag for "versioned=API".
2016-08-02 01:26:41 +00:00
Colin Cross
5cf03973aa Merge "Rename compiler, linker and installer methods to be unique"
am: cab3d65445

Change-Id: Ide8b387dca1ba29eae3744cb39a9d6d1952ee9ba
2016-08-02 00:37:00 +00:00
Treehugger Robot
cab3d65445 Merge "Rename compiler, linker and installer methods to be unique" 2016-08-02 00:28:36 +00:00
Colin Cross
5d90e94a62 Merge "Fix mac soong build"
am: 35ef8bdf32

Change-Id: I1e9e3b5103fc627d5043316e8acc32e7d3cde1e5
2016-08-02 00:12:59 +00:00
Treehugger Robot
35ef8bdf32 Merge "Fix mac soong build" 2016-08-02 00:02:06 +00:00