Commit graph

11923 commits

Author SHA1 Message Date
Colin Cross
e264f22bec Merge changes Ia77090da,I95a57f76
* changes:
  Fix data races in finder_test.go
  Optimize sanitizerRuntimeDepsMutator
2019-06-21 17:01:23 +00:00
Treehugger Robot
45cdb71290 Merge "Soong: Add BUILD_MODULES build action in soong_ui." 2019-06-21 03:11:34 +00:00
Treehugger Robot
c4be0815d8 Merge "Remove experimental pass manager for fuzzer builds." 2019-06-21 00:51:26 +00:00
Treehugger Robot
17db3267f6 Merge "Make apexMutator parallel" 2019-06-21 00:04:01 +00:00
Patrice Arruda
3928206ecc Soong: Add BUILD_MODULES build action in soong_ui.
There was one case that did not work:
  1) Create dir in root source tree
  2) Run "m libc"

The build would fail as it was using the mma logic. A separate
one named BUILD_MODULES was added to allow building specific
modules in any directories using the "m" command.

Bug: b/130049705
Test: unit test cases, ran "m libc" in a tmp directory inside
      of the source tree.

Change-Id: I8d23e685a673a311001ee8edd89bd73b662392dd
2019-06-20 16:35:12 -07:00
Colin Cross
2f5a484ee1 Fix data races in finder_test.go
Fix data races by incrementing clock before starting the finder.
Fixes:
==================
WARNING: DATA RACE
Read at 0x00c0003be1c0 by goroutine 99:
  android/soong/finder/fs.(*MockFs).WriteFile()
      build/soong/finder/fs/fs.go:159 +0x869
  android/soong/finder.(*Finder).dumpDb()
      build/soong/finder/finder.go:1163 +0x296
  android/soong/finder.(*Finder).goDumpDb.func1()
      build/soong/finder/finder.go:337 +0x3c

Previous write at 0x00c0003be1c0 by goroutine 43:
  android/soong/finder.TestDirectoryNotPermitted()
      build/soong/finder/fs/fs.go:155 +0x49a
  testing.tRunner()
      prebuilts/go/linux-x86/src/testing/testing.go:865 +0x163

Goroutine 99 (running) created at:
  android/soong/finder.(*Finder).goDumpDb()
      build/soong/finder/finder.go:336 +0x8d
  android/soong/finder.(*Finder).loadFromFilesystem()
      build/soong/finder/finder.go:380 +0x12b
  android/soong/finder.newImpl()
      build/soong/finder/finder.go:197 +0x38d
  android/soong/finder.newFinderAndErr()
      build/soong/finder/finder_test.go:58 +0x245
  android/soong/finder.newFinderWithNumThreads()
      build/soong/finder/finder_test.go:42 +0xb3
  android/soong/finder.TestDirectoryNotPermitted()
      build/soong/finder/finder_test.go:38 +0x386
  testing.tRunner()
      prebuilts/go/linux-x86/src/testing/testing.go:865 +0x163

Goroutine 43 (running) created at:
  testing.(*T).Run()
      prebuilts/go/linux-x86/src/testing/testing.go:916 +0x65a
  testing.runTests.func1()
      prebuilts/go/linux-x86/src/testing/testing.go:1157 +0xa8
  testing.tRunner()
      prebuilts/go/linux-x86/src/testing/testing.go:865 +0x163
  testing.runTests()
      prebuilts/go/linux-x86/src/testing/testing.go:1155 +0x523
  testing.(*M).Run()
      prebuilts/go/linux-x86/src/testing/testing.go:1072 +0x2eb
  main.main()
      out/soong/.bootstrap/soong-finder/test/test.go:157 +0x1dc
==================

Test: run finder_test.go with race detector enabled
Change-Id: Ia77090dabaf8b14a80ac9ce938fef75f1d382574
2019-06-20 15:24:05 -07:00
Colin Cross
0b90833ea8 Optimize sanitizerRuntimeDepsMutator
sanitizerRuntimeDepsMutator only modifies the currently visited
module, it can visit modules in parallel.

Also, stop recursing into modules that are not static dependencies,
and stop recursing if the module already has all modifications that
the mutator could make.

Test: m checkbuild
Change-Id: I95a57f763a91940f1854ba3c587a2f70e8baba97
2019-06-20 15:23:41 -07:00
Elliott Hughes
63e70530d3 Merge "Allow gzcat as an alias for gzip's -dc mode." 2019-06-20 21:03:11 +00:00
Dan Willemsen
d9cf7fc48e Merge "Use prebuilt m4 instead of system m4" 2019-06-20 20:50:12 +00:00
Colin Cross
643614de24 Make apexMutator parallel
apexMutator only modifies the currently visited module, it can
visit modules in parallel.

Test: builds
Change-Id: I7d0ad142f5161742bd25dc57bac64c0f4a733ecf
2019-06-20 10:45:39 -07:00
Dan Willemsen
fe310be720 Fix typo in java system modules makevars
This was preventing us from actually using system module dependencies
from Make.

Fixes: 135679136
Test: EXPERIMENTAL_JAVA_LANGUAGE_LEVEL_9=true m legacy-performance-test-hostdex
Change-Id: Ic51c30d09cf39e35f2fe5f534a23273f450d9138
2019-06-20 10:17:35 -07:00
Treehugger Robot
0b4ca8605c Merge "OverridableModuleBase shouldn't embed ModuleBase." 2019-06-19 21:30:46 +00:00
Elliott Hughes
e671f5a397 Allow gzcat as an alias for gzip's -dc mode.
This gets us closer to toybox tar (and toybox gzip/gzcat).

Test: treehugger
Change-Id: If934270439b258eae55cdd3e97ac0e83b1c79c70
2019-06-19 14:02:02 -07:00
Dan Willemsen
c69d715d72 Use prebuilt m4 instead of system m4
Bug: 117561006
Test: treehugger
Change-Id: I588d867d95cc52963ce764018a9c09b26973c8c3
2019-06-19 12:50:32 -07:00
Jaewoong Jung
8985d527ae OverridableModuleBase shouldn't embed ModuleBase.
Individual module structs normally embed ModuleBase themselves. This
caused duplicate embedding, and so made common properties like notice
inaccessible.

Bug: 135460391
Test: app_test.go, TreeHugger
Change-Id: I5683b194e3de909d480c083a931cc7e871de1d74
2019-06-19 11:22:25 -07:00
Treehugger Robot
fedc47141e Merge "Support an action table that shows longest running actions" 2019-06-19 06:41:28 +00:00
Colin Cross
7dc6951c3d Merge "Support moving sources in srcjars in soong_zip" 2019-06-19 00:23:00 +00:00
Treehugger Robot
f8a698a0c6 Merge changes from topics "rbe-deps-dexpreopt-images", "rbe-deps-system-modules"
* changes:
  Expand the dexpreopt image dependencies to entire image
  Depend on all the files from system modules
2019-06-18 22:48:22 +00:00
Treehugger Robot
4b1f4fe138 Merge "Use SIGWINCH to update terminal size" 2019-06-18 21:23:43 +00:00
Colin Cross
9cb51dbc0b Support moving sources in srcjars in soong_zip
Add a -srcjar argument to soong_zip that causes it to read the
package statement of each .java file and use that to place the
source file at a path that matches the package.

Test: jar_test.go, zip_test.go
Change-Id: I36017e42445ba3b0a82a10a8d81e8ac0cca096f2
2019-06-18 13:33:20 -07:00
Patrice Arruda
db4c2f115e Soong: Use ctx.Writer in flag usage for dumpVar and dumpVars in soong_ui.
ctx.Writer is recommended stream to use for standard io.

Fixes: b/135487952
Test: Lunch and executed ./out/soong_ui --dumpvar-mode and
--dumpvars-mode to check the usage is output to the console.

Change-Id: I38351ebdb05f49f92d6b4bfb0423092299d1c5ea
2019-06-18 20:32:01 +00:00
Dan Willemsen
0f41678d00 Expand the dexpreopt image dependencies to entire image
Instead of just depending on the main .art file (boot.art, etc), also
expose the dependencies to the .oat/.vdex files (boot.oat/boot.vdex),
and all of the module files that get implicitly loading (boot-ext.*,
boot-framework.*, etc)

This is necessary for RBE, where the rule only gets the files that it
depends upon.

Test: treehugger
Test: build a system image with RBE
Change-Id: I0c7051f18582f1891d3398b46763b1521e4326c8
2019-06-18 06:40:38 +00:00
Treehugger Robot
2f93a2e07b Merge "soong_ui: Add --build-mode flag in soong_ui" 2019-06-18 03:43:24 +00:00
Colin Cross
04815d0bed Merge "Remove SOONG_FRAMEWORK_SRCS" 2019-06-18 02:02:54 +00:00
Treehugger Robot
5c3bf56ac2 Merge "Add more OWNERS" 2019-06-18 01:38:25 +00:00
Dan Willemsen
ff60a73d89 Depend on all the files from system modules
Instead of just one of the files that we pass into javac.

Test: treehugger
Change-Id: I8478e88656487c9f667893d7c17839f0ea63c78f
2019-06-18 01:29:02 +00:00
Patrice Arruda
b7b2282fd1 soong_ui: Add --build-mode flag in soong_ui
The --build-mode flag is a new flag in soong_ui that accepts a build
action (m, mm, mmm, mma, mmma), the directory where the build action
is occuring and an optional list of build arguments and invokes the
build option. This is to deprecate the build actions behavior in
envsetup.sh and start using the one defined in soong_ui.

Bug: b/130049705
Test: Ran lunch and ran all the build action commands in soong_ui.
      Ran a bunch of invalid commands such as no Android.mk file in
      a specific directory, invalid target name, etc...

Change-Id: I88e9e59f47f8c5cdc872fb83a0f3f0a0919885b6
2019-06-17 17:21:32 -07:00
Treehugger Robot
eec8d3aee3 Merge "Soong: Fix the package name in metrics.proto file" 2019-06-17 23:42:33 +00:00
Elliott Hughes
3b584b7861 Switch to toybox tar.
Test: treehugger
Change-Id: I4c561186c14cc44782a9e5c83d4d9c002addb3f6
2019-06-17 16:39:35 -07:00
Elliott Hughes
2d10e3f54e Merge "Revert "Revert "Switch to toybox timeout.""" 2019-06-17 23:35:16 +00:00
Colin Cross
06adae4c8c Add more OWNERS
Add paulduffin for EMEA.

Test: none
Change-Id: If4e4ce2cd668cf602479535a41175484dca54e33
2019-06-17 22:28:35 +00:00
Colin Cross
5d7409ba0f Remove SOONG_FRAMEWORK_SRCS
SOONG_FRAMEWORK_SRCS is not used in Make any more.

Test: m checkbuild
Change-Id: I61f7cdeb3d2e2fd89c8b3b302bee923624dba418
2019-06-17 14:27:15 -07:00
Paul Duffin
33ee25466f Merge changes I42cb181f,I6413c9b1
* changes:
  Add sdk_version:"core_platform" to replace no_framework_libs:true
  Remove the no_standard_libs property
2019-06-17 21:24:33 +00:00
Mitch Phillips
74384758a6 Remove experimental pass manager for fuzzer builds.
Sanitizer coverage is currently broken with the experimental pass
manager. See b/133876586 for more information. The patch is currently
being worked on upstream (https://reviews.llvm.org/D62888), but is not
ready yet. Hence, we disable it here (similar to the LTO bug).

Bug: 133876586
Test: Build anything with SANITIZE_TARGET='fuzzer', verify that it has
'sancov' symbols.

Change-Id: I4302a8cf05300015aaff137f40b61d5685dc3fea
2019-06-17 10:33:52 -07:00
Elliott Hughes
0e868d0391 Revert "Revert "Switch to toybox timeout.""
This reverts commit 50e91c1a7a.

Reason for revert: SIGRTMIN+2 support added.

Change-Id: I3cb251a102e427335315c4931e23a69b95253ee7
2019-06-17 16:45:15 +00:00
Nick Desaulniers
2deb017c04 Merge "Switch to clang-r353983d" 2019-06-17 16:25:51 +00:00
Paul Duffin
50c217c744 Add sdk_version:"core_platform" to replace no_framework_libs:true
Where possible this duplicates any tests that use no_framework_libs:true
with ones that use sdk_version:"core_platform". If not possible (e.g. in the
default targets included in java/testing.go) it switches some to use
sdk_version:"core_platform" to ensure that there is no regression in the
behavior of no_framework_libs:true.

Follow up changes will switch all usages of no_framework_libs:true over
to use sdk_version:"core_platform" at which point no_framework_libs will be
removed.

Bug: 134566750
Test: m droid

Change-Id: I42cb181f628b723c8f32a158ae4752b4c83365ae
2019-06-17 13:17:35 +01:00
Paul Duffin
301ba52886 Remove unused jars/sdk_version properties from java_system_modules
Bug: 134566750
Test: m droid
Change-Id: I2c5cd17f3f64bbde437eff65267d8c771d61e343
2019-06-17 11:21:39 +01:00
Paul Duffin
a3d0986cfc Remove the no_standard_libs property
Corrects an error message that refers to no_standard_libs.

Removes any tests that use no_standard_libs:true where possible as
there are duplicate tests for sdk_version:"none". Otherwise, switches
them over to use sdk_version:"none".

The androidmk mapping from LOCAL_NO_STANDARD_LIBRARIES to
no_standard_libs has also been removed. There was little point in
updating the tool to map it through to sdk_version:"none" as there are
only a couple of places where it is used, in art's test running mk
targets and in some unbundled packages to work around some limitation
in .mk based build.

Bug: 134566750
Test: m droid
Change-Id: I6413c9b1fe3e63b93753a6a017d2981e32b7e013
2019-06-17 11:21:39 +01:00
Treehugger Robot
b1856799b1 Merge "sysprop_library: Rename system scope to public" 2019-06-16 06:08:14 +00:00
Evgenii Stepanov
1c69e83030 Disable new LLVM pass manager for HWASan.
Current Android toolchain does not support hwasan with new pm.

Bug: 135298400
Test: manual, inspect the binaries for calls to __hwasan_init
Change-Id: I126c59e6ecabc3a1699e1bf34c546568b6e8a21f
2019-06-14 18:39:59 -07:00
Nick Desaulniers
36a39c9852 Switch to clang-r353983d
Test: build+boot sargo, TreeHugger
Bug: 131714073
Change-Id: Ieec2500027a78b80147f86b2bdaeaedaa1c18360
2019-06-14 16:31:34 -07:00
Patrice Arruda
0cc5b21516 Soong: Fix the package name in metrics.proto file
The metrics.proto needs to be imported to Google3 in order to unmarshal
the build metrics data from bigstore. Cleaned up the enum names to use
the Camel naming convention and renamed the build_metrics to
soong_metrics to be more specific to soong and to allow quering
soong_metrics files from bigstore.

Bug: b/135280521
Test: lunch andf m -j. Checked if soong_metrics was generated correctly
      by using printproto command.

Change-Id: I998c8d05db592e94a653d6ca32250b80df3c9b21
2019-06-14 15:27:46 -07:00
Dan Shi
9314e1124b Merge "Support require_root in auto-gen test configs" 2019-06-14 22:21:39 +00:00
Colin Cross
ad13e1185f Merge "Support sharding robolectric tests" 2019-06-14 21:59:12 +00:00
Treehugger Robot
fae8569647 Merge "Give Blueprint modules access to all namespaces" 2019-06-14 21:25:28 +00:00
Pirama Arumuga Nainar
50b382a9e3 Merge "Remove redundant -fexperimental-new-pass-manager" 2019-06-14 21:02:45 +00:00
Colin Cross
9a642dcde2 Support sharding robolectric tests
Explicitly pass the list of tests as LOCAL_ROBOTEST_FILES, and
support sharding the list of tests into multiple test runners.

Also filter out BaseRobolectricTest.java, which covers the only
use of LOCAL_ROBOTEST_FILES in Android.mk files.

Bug: 133878985
Test: m RunSettingsLibRoboTests runs the same number of tests before and after
Test: m RunSettingsLibRoboTests runs the same number of tests when sharded
Change-Id: Id85ffe03c98e722303eaa6def17812ed2244c6a6
Merged-In: Id85ffe03c98e722303eaa6def17812ed2244c6a6
2019-06-14 13:10:53 -07:00
Colin Cross
cd84b4e36b Give Blueprint modules access to all namespaces
Don't enforce namespaces on Blueprint modules like
bootstrap_go_package, their dependencies are handled before
namespaces are initialized in namespaceMutator.

Fixes: 135246048
Test: TestDependingOnBlueprintModuleInRootNamespace
Change-Id: I7cf1c26bb8512eed59d6b4eb42a49f7080ffa281
2019-06-14 11:26:09 -07:00
Dan Shi
37ee3b8f49 Support require_root in auto-gen test configs
require_root is added to allow auto-generated test config to include
RootTargetPreparer so the test runs with root permission.

Bug: 134509111
Test: add "require_root: true" to init_benchmarks and libpower_test
  build the modules, confirm the extra target preparer is added in the
  test configs.

Change-Id: Ia07503e338935d6aa92560e7cf7b18d2a4c51243
2019-06-14 11:10:13 -07:00