Commit graph

3807 commits

Author SHA1 Message Date
Jeff Gaston
66dd6e5a59 Support in soong_zip to write byte buffers in addition to copying files
This will enable writing a modified manifest without having to create a temporary file first

Bug: 64536066
Test: soong_zip --jar -o /tmp/out.zip -C . -l files.list && \
      # make sure that the output is binary equal
      # with and without this patch

Change-Id: I559d653e0e72e641e1ee6745924cb835bb0a355b
2017-08-25 16:07:29 -07:00
Jeff Gaston
8edbb3acc9 Have soong_zip add extended attribute to META-INF in jar mode
Bug: 64536066
Test: soong_zip --jar -o /tmp/out.zip -C . -l files.list && \
      zipdetails /tmp/out.zip | less \
      # and check that the first entry contains the \
      # "CAFE" extra attribute as shown below:

0000001E Filename              'META-INF/'
00000027 Extra ID #0001        CAFE 'Java Executable'
00000029   Length              0000

Change-Id: I12c4078591f2ce2afc1af5b9db20393b26601bca
2017-08-25 13:03:51 -07:00
Jeff Gaston
8cd5310504 Merge changes Id5c6ea9c,I4830bd33
* changes:
  Add '--jar' flag to soong_zip to move META-INF to the top of the zip
  Extract jar-sorting to be accessible to soong_zip
2017-08-25 20:02:53 +00:00
Jeff Gaston
b6530cc18f Merge "Have soong_zip not write a data descriptor for non-compressed files" 2017-08-25 20:02:40 +00:00
Jayant Chowdhary
ec5b3aa567 Merge "Allow apis from exported shared library headers as well, while dumping abi." 2017-08-25 16:50:32 +00:00
Isaac Chen
cd3d542187 Merge "Revert "Revert "Build support for 32-bit armv8-a""" 2017-08-25 01:55:56 +00:00
Jayant Chowdhary
af6eb71ad2 Allow apis from exported shared library headers as well, while dumping abi.
header-abi-diff has switched to recursive diffing of types and can
diff more accurately if types included in re-exported shared library
headers are included as well.

Bug: 62060883

Test: make -j64, android.hardware.light@2.0.so.lsdump contains structs
from libcutils as well (libcutils' headers are re-exported by
android.hardware.light@2.0)

Change-Id: I814819e4f7258b4b380350fe8ad0ccf8dbd5ce5c
2017-08-24 16:48:25 -07:00
Jeff Gaston
f3ccfa9b99 Fix crash when running androidmk against frameworks/base/Android.mk
The was a variable on the right-hand side of the assignment to
LOCAL_MODULE_CLASS , which wants a non-nil scope

Bug: 64266643
Test: androidmk frameworks/base/Android.mk
Test: androidmk prebuilts/misc/common/ddmlib/Android.mk

Change-Id: I52d33f5e5cb1179f84d4df149ef804268d67f7fb
2017-08-24 22:59:50 +00:00
Jeff Gaston
d55f4d01d5 Merge "Consolidate localIncludeDirs and exportIncludeDirs" 2017-08-24 22:59:30 +00:00
Jeff Gaston
a297695aa8 Add '--jar' flag to soong_zip to move META-INF to the top of the zip
Bug: 64536066
Test: soong_zip --jar -o /tmp/out.zip -C . -l files.list && \
      zipdetails /tmp/out.zip | less \
      # and check that the META-INF entries are earlier in \
      # the list than other entries

Change-Id: Id5c6ea9ce8c3a6fbfb8366db753e6603a076dbf8
2017-08-24 14:44:47 -07:00
Jeff Gaston
c5eb66d16b Have soong_zip not write a data descriptor for non-compressed files
Bug: 64536066
Test:  m -j blueprint_tools && cd /tmp && mkdir zip && \
       cd zip && touch empty-file && \
       echo empty-file > files.list && \
       soong_zip -o zip.zip -C . -l files.list && \
       jar -xvf zip.zip && echo ok

Change-Id: Iac5797aab5282237fa1cc902e6b068a7937c012a
2017-08-24 14:43:35 -07:00
Jeff Gaston
01547b23d2 Extract jar-sorting to be accessible to soong_zip
Bug: 64536066
Test: m -j # which runs unit tests
Change-Id: I4830bd331c9dab0b1d300a18aefaf25a6af4cfdc
2017-08-24 14:43:35 -07:00
Jeff Gaston
6dd56e0f4a Merge "Revert "Revert "Refactor rate_limit.go for more clarify""" 2017-08-24 21:41:14 +00:00
Jeff Gaston
21c21b7a75 Consolidate localIncludeDirs and exportIncludeDirs
in preparation to have LOCAL_AIDL_INCLUDES work similarly

Bug: 62882283
Test: m -j
Change-Id: I5df3d5ac2eac43d839440c291d4cdc0e26172dfa
2017-08-24 13:38:50 -07:00
Jeff Gaston
f355735fb9 Merge "Fix the Finder's ability to ignore permission errors" 2017-08-24 19:32:47 +00:00
Zach Riggle
06bbd89c16 Enhance coverage options to include those needed by Honggfuzz for coverage-driven fuzzing
Test: make m
Bug: 64903541
Change-Id: Id5f10b6d7c122444f875134883f7c0a95d71e12d
2017-08-24 19:25:05 +00:00
Treehugger Robot
2e78e5e37f Merge "Add dependency support for yasm" 2017-08-24 18:02:18 +00:00
Treehugger Robot
87291b110a Merge "Add product_variables.uml" 2017-08-24 01:14:36 +00:00
Jeff Gaston
821271b3b8 Fix the Finder's ability to ignore permission errors
It turns out the os was returning a different permissions error than
os.ErrPermission. Fortunately we can just use os.IsPermission().

Bug: 62455338
Test: mkdir not-permitted && chmod 000 not-permitted && m nothing

Change-Id: Ieb1653b9687ffdaae83184dcdcd4166ddb6a4613
2017-08-23 17:53:41 -07:00
Dmitry Shmidt
dba5419dea Add product_variables.uml
Bug: 64985489
Test: Manual

Change-Id: Ief6f06770b4df54c3630fb8f1904da2336f294ff
Signed-off-by: Dmitry Shmidt <dimitrysh@google.com>
2017-08-23 15:57:20 -07:00
Isaac Chen
a2a582413e Revert "Revert "Build support for 32-bit armv8-a""
This reverts commit 80b27e4e20.

Fixed CTS build issue by adding -march=armv7-a in cts/tests/tests/os/jni/Android.mk since the library built here needs/uses instruction (swp) obsolete in ARMv8.

Bug: 64964151
Test: lunch aosp_arm64-userdebug; make -j cts
      lunch aosp_x86-userdebug; make -j cts
      lunch aosp_sailfish-userdebug; make -j cts
      lunch aosp_bullhead-userdebug; make -j cts

Change-Id: I6ccafce043e101c769ad370a81ffa0eb62719b5e
2017-08-23 10:57:17 +00:00
Dan Willemsen
1d3e545a97 Add dependency support for yasm
Bug: 37719672
Test: lunch aosp_x86-eng; m native
Test: m out/soong/.intermediates/external/libvpx/libvpx/android_x86_static_core/obj/external/libvpx/libvpx/vp8/common/x86/copy_sse2.o
Test: NINJA_ARGS="-t deps out/soong/.intermediates/external/libvpx/libvpx/android_x87_static_core/obj/external/libvpx/libvpx/vp8/common/x86/copy_sse2.o" m
Change-Id: I01763b5870f994f7f0c6ac9d10f112f73addd7c7
2017-08-22 21:02:08 -07:00
Colin Cross
ef88ae2369 Make libdl.so be loaded after libc.so
Make sure that libdl is always after libc on the command line.
Simplifies the logic to always support system_shared_libs for
sdk and vndk builds.

For backwards compatibility without updating lots of Android.bp
files, allow libdl to be listed in shared_libs as long as it
is also in system_shared_libs or libc is not in system_shared_libs.

Remove all the places that libdl is added as a dependency, since
it will always be present unless explicitly removed now.

Bug: 62815515
Test: m -j checkbuild
Change-Id: I0233178ffea87a2f0b82190746022476304a68e2
2017-08-23 12:54:54 +09:00
Treehugger Robot
0b062130dc Merge "Add some tests for singlethreaded usage of the Finder." 2017-08-23 02:59:39 +00:00
Dan Willemsen
59fdf96fcf Split CleanSpec processing out of main Kati run
Instead of pairing the reading of CleanSpec.mk files with the reading of
Android.mk files, split them into separate, individually cachable steps.
This way we only read Android.mk files once after a clean/sync. We'll
still read the CleanSpec.mk files multiple times, but that's
significantly faster than reading all the Android.mk files.

This adds about 50ms if kati doesn't need to reread the CleanSpec.mk
files. Reading all the CleanSpec.mk files takes about a second.

Bug: 35970961
Test: m clean; m nothing; m nothing
Test: Add CleanSpec.mk line, see it executed.
Change-Id: I83bad15c50709510959d5b8b673a907b8aa7de82
2017-08-23 01:53:33 +00:00
Treehugger Robot
56ad256e70 Merge "Stop setting ASAN_OPTIONS for kati" 2017-08-23 01:52:37 +00:00
Jeff Gaston
d311952e93 Add some tests for singlethreaded usage of the Finder.
Also change the existing tests to use exactly 2 threads.
Also update some tests to also same number of threads as will
be used by the finder.

Bug: 62455338
Test: m -j # which runs unit tests

Change-Id: I2b9d39680ecddd6d28c951df982ac51c077d2777
2017-08-22 16:55:21 -07:00
Jeff Gaston
996716a8e2 Fix hanging of finder_test
Previously there could be an infinite loop in the Finder's db export when
running tests on a system with few cores.

The Finder divides the cache-serialization work amongst cores. On systems with
enough cores, each path in TestFileSystemRoot would get its own core and the
infinite loop in the prefix calculation logic would be avoided. On systems with
fewer cores, the computation of the maximum common path could run forever.

This shouldn't have affected the general usage of the Finder in builds other
than in its unit tests, because the builds don't ask to include the
filesystem root as a path to scan. This should only have affected
finder_test.go which tested the filesystem root anyway.

Bug: 62455338
Test: m -j # which runs unit tests
Change-Id: I7a1467fb32e4538fb96833791180bfbfad5a3911
2017-08-22 14:14:40 -07:00
Tobias Thierer
b3f1c8f5f5 Merge "Revert "Build support for 32-bit armv8-a""
am: 9dffe1d807

Change-Id: I4e040ff1e4ec5c12fab4091eead28badf4585a79
2017-08-22 10:36:38 +00:00
Tobias Thierer
9dffe1d807 Merge "Revert "Build support for 32-bit armv8-a"" 2017-08-22 10:30:29 +00:00
Dan Willemsen
e27ed543bf Stop setting ASAN_OPTIONS for kati
Kati sets detect_leaks=0 by default now.

Test: unset ASAN_OPTIONS; SANITIZE_HOST=address m nothing
Change-Id: I0cbf04ae9ed40de520abca3fb35c65e7f543682d
2017-08-21 20:54:06 -07:00
Dan Albert
fd7141ee24 Merge "Update the minimum NDK target API."
am: fb5a671c7c

Change-Id: I9c144909a9266d2fb45f77179166891c0874a2bc
2017-08-21 21:23:08 +00:00
Treehugger Robot
fb5a671c7c Merge "Update the minimum NDK target API." 2017-08-21 21:14:25 +00:00
Tobias Thierer
80b27e4e20 Revert "Build support for 32-bit armv8-a"
This reverts commit 9c02066b13.

Reason for revert: Broke the build for some targets (including marlin and angler).

$ make cts
build/core/combo/TARGET_linux-arm.mk:43: kryo is armv8-a.
build/core/combo/TARGET_linux-arm.mk:45: TARGET_2ND_ARCH_VARIANT, armv7-a-neon, ignored! Use armv8-a instead.
[...]
cts/tests/tests/os/jni/android_os_cts_CpuInstructions.cpp:88:20: error: instruction requires: armv7 or earlier
    asm volatile ( "swp r0, r0, [%0]" : "+r"(ptr) : : "r0" );
                   ^
<inline asm>:1:2: note: instantiated into assembly here
        swp r0, r0, [r1]
        ^

Change-Id: I3fa7ca74d88aedf6574855e988d35e341f6e5067
2017-08-21 15:03:26 +00:00
Isaac Chen
2bd5707d7a Merge "Build support for 32-bit armv8-a"
am: 1c508b6c53

Change-Id: I2c5ffff14717e7e71db3619a73bf209530bc396c
2017-08-19 06:16:43 +00:00
Isaac Chen
1c508b6c53 Merge "Build support for 32-bit armv8-a" 2017-08-19 06:10:56 +00:00
Dan Willemsen
68c15ddb2d Obsolete bootstrap.bash and ./soong wrappers
am: 99a75cd2a5

Change-Id: I12a217021354e5ab2812824e9b0209bcce2c9bcc
2017-08-18 22:41:37 +00:00
Dan Willemsen
99a75cd2a5 Obsolete bootstrap.bash and ./soong wrappers
We can call directly into the blueprint bootstrap.bash using values that
soong_ui has already calculated.

Instead of calling into blueprint.bash, build minibp with microfactory,
and directly run ninja. This allows us to get individual tracing data
from each component.

Test: m -j blueprint_tools
Test: m clean; m -j blueprint_tools
Change-Id: I2239943c9a8a3ad6e1a40fa0dc914421f4b5202c
2017-08-18 10:13:22 -07:00
Isaac Chen
9c02066b13 Build support for 32-bit armv8-a
Add armv8-a as a valid/supported 32-bit arm architecture variant. If
some known armv8-a core, like cortex-a53, cortex-a73, etc. is specified
as TARGET(_2ND)_CPU_VARIANT, the associated TARGET(_2ND)_ARCH_VARIANT
will be treated as if armv8-a is specified.

Bug: 62895439
Test: "bionic-unit-tests-static --gtest_filter=*strlen*" on Nexus 4
      (armv7-krait), emulator (armv7), and sailfish (armv8-kryo).
      The test binary for the first 2 is built with armv7-a as its
      TARGET_ARCH_VARIANT; The test binary for the last is built with
      armv8-a as its TARGET_2ND_ARCH_VARIANT.
      TARGET(_2ND)_CPU_VARIANTs of both binaries are set to "generic".

Change-Id: I53bee2974346cf485b2da54cf7aad010b0425910
2017-08-18 19:46:38 +08:00
Jeff Gaston
175f34c5c3 Revert "Revert "Refactor rate_limit.go for more clarify""
This reverts commit 526416b1e4.

Figured out a fix for the deadlocks; resubmitting the patch.
The first version was deadlocking because the switch statement in
zipWriter.write would choose a specific zip entry to start writing,
but the individual chunks may not have all necessarily been compressed
yet. When each individual chunk was made to require to request its own
allocations, the compression of the chunks of the file being currently
written could be blocked waiting for memory to be freed by chunks from
other files that hadn't yet started being written.

This patch is much like the original except it preallocates the memory
for the entire file upfront (and happens to use the total file size
rather than the compressed size, but I didn't observe that to cause any
performance differences).

Bug: 64536066
Test: m -j dist showcommands # which runs soong_zip to package everything

Change-Id: Id1d7ff415e54d3a6be71188abbdbbbab5a719fcf
2017-08-17 21:46:46 -07:00
Dan Albert
990419604a Merge "Expose the minimum supported SDK version to make."
am: fd697f4256

Change-Id: I1a7e7657258e6adbad88f9e51684e8238ab4ed18
2017-08-18 02:44:37 +00:00
Treehugger Robot
fd697f4256 Merge "Expose the minimum supported SDK version to make." 2017-08-18 02:34:36 +00:00
Jeff Gaston
66515e18f5 Merge "Fix Finder out path in multiproduct_kati"
am: bfbb8d242d

Change-Id: Ic00c2d27e9ff4717dc340fa0445ac4978e36c701
2017-08-18 01:50:45 +00:00
Jeff Gaston
a308122994 Merge "Fix androidmk crash if no CLEAR_VARS is detected"
am: d5476b11f3

Change-Id: Idf436e97131b87adae4f9aef179fd508d8af9da5
2017-08-18 01:50:27 +00:00
Jeff Gaston
bfbb8d242d Merge "Fix Finder out path in multiproduct_kati" 2017-08-18 00:54:39 +00:00
Colin Cross
e692428554 resolve merge conflicts of ee41cb8 to stage-aosp-master
Test: I solemnly swear I tested this conflict resolution.
2017-08-17 17:36:57 -07:00
Jiyong Park
ab0fd5f060 List of VNDK-related libs are exported to make
LL-NDK, VNDK-core, VNDK-SP libraries are exported to make as
SOONG_LLNDK_LIBRARIES, SOONG_VNDK_CORE_LIBRARIES, and
SOONG_VNDK_SAMEPROCESS_LIBRARIES. This can be used to auto-generate
ld.config.txt from a template.

Bug: 64013660
Test: BOARD_VNDK_VERSION=current m -j successful
Test: check out/soong/make_vars*.mk and look for SOONG_*_LIBRARIES

Merged-In: I0f4c5d05d9cd28c3fc9fdcca6ce0e6eaeaacbe8d
Change-Id: I0f4c5d05d9cd28c3fc9fdcca6ce0e6eaeaacbe8d
(cherry picked from commit d5b18a55bb)
2017-08-17 17:34:25 -07:00
Treehugger Robot
d5476b11f3 Merge "Fix androidmk crash if no CLEAR_VARS is detected" 2017-08-18 00:19:29 +00:00
Treehugger Robot
ee41cb8116 Merge "Keep vndk library lists sorted" 2017-08-18 00:10:16 +00:00
Colin Cross
2b3cb9cd39 Merge "Revert "Refactor rate_limit.go for more clarify""
am: 1115ddc838

Change-Id: Ic17c986c5058b463b529542588f67eb5e3ac83d5
2017-08-18 00:01:50 +00:00