Commit graph

64546 commits

Author SHA1 Message Date
Spandan Das
c0f9a2178f Merge "Re-enable TestMixedBuildUsesStubs" 2023-04-27 23:11:14 +00:00
Spandan Das
cc4da76511 Differentiate between empty and nil input
Previously, CopyOf on an empty list was returning nil. With the updates
to SortedUniqueStrings and FirstUniqueStrings, we need to differentiate
between empty lists and nil.

Test: m nothing
Change-Id: I91063ebbe5013cbda5d8f70efde4683c66581599
2023-04-27 19:34:08 +00:00
Cole Faust
03cf3181d3 Merge "Expand preprocessed flag to work on android_app_imports" 2023-04-27 17:18:59 +00:00
Cole Faust
3e0836e865 Merge "Load starlark files from soong" 2023-04-27 17:15:02 +00:00
David Fu
eb52c2e769 Merge "Changed assignee to triage_assignee in fuzzconfig" 2023-04-27 17:14:10 +00:00
Anton Hansson
c8cea365f3 Merge "Remove --skip-kati" 2023-04-27 11:26:50 +00:00
Treehugger Robot
da2f5ec78f Merge "Add allowed-deps tests" 2023-04-27 06:09:35 +00:00
Mark Dacek
aa2e898e62 Merge "Fix allowlist in order to mixed_build the entire prod allowlist." 2023-04-27 04:47:19 +00:00
Treehugger Robot
3c31266816 Merge "Remove makefile_goal" 2023-04-27 04:29:08 +00:00
Treehugger Robot
444b7ff1cc Merge "Add license visibility to prebuilt vsdk" 2023-04-27 01:47:18 +00:00
Spandan Das
f57a966b66 Ignore test apexes from bp2build generated tags
The core problem I am trying to solve is making sure that stub
libraries in Bazel have a single apex available. In Soong, this will be
enforced using some graph walk which is not easy to port to Bazel.

However, we might need to revisit this when we build the enforcement
mechanism in Bazel. We likely need a `test_for` on the top level
apex_test Bazel rule so that the test apex is _allowed_ to link against impl
of the library it is trying to test.

(This CL retricts this to cc_library, I can expand this to other modules
if this is the right approach.)

Bug: 277651159
Change-Id: Iaeec22c5626df79a33785c766ed29102b1da403e
2023-04-27 01:43:00 +00:00
Jooyung Han
40a35ee5f9 Merge "Run apex_sepolicy_test for supported fs_type" 2023-04-27 01:03:29 +00:00
Jooyung Han
4495f84f68 Add allowed-deps tests
These tests capture
- track updatable apexes
- do not track apex-only modules
- track transitive deps
- track external deps (with mark)

Bug: 274041915
Test: m nothing
Change-Id: I629015f5aa4a1a7627d0ba6d92fd42bb99c96287
2023-04-27 09:56:04 +09:00
Cole Faust
c9508aac4c Load starlark files from soong
There are a number of instances where we are exporting information
from soong to bazel via soong_injection. This could be more bazel-centric
if the information was instead held in bzl files, and both bazel and
soong read it from there.

Add a starlark package that will run
//build/bazel/constants_exported_to_soong.bzl at initialization time,
and then results can be retreived with GetStarlarkValue.

Since changes to the starlark files mean that soong has to rerun,
add them as ninja deps.

Unfortunately, the starlark code has to be run at runtime rather than
pregenerating their results, because tests run from intellij wouldn't
go through any pregeneration steps. This means that starlark is run
multiple times during the build, once per test package and once per
primary builder invocation. (currently 3, could be reduced to 2 if we
made the symlink forest generation into its own standalone tool) The
starlark code we have so far in this cl is very fast, roughly half a
millisecond, so it's not a big deal for now, but something to keep an
eye on as we add more starlark constants.

Bug: 279095899
Test: go test
Change-Id: I1e7ca1df1d8d67333cbfc46e8396e229820e4476
2023-04-26 17:18:19 -07:00
MarkDacek
b9e11c5b64 Fix allowlist in order to mixed_build the entire prod allowlist.
Bug: 279808471
Test: m nothing --ensure-allowlist-integrity

Change-Id: Ie7c18e1457d29f300e790ea8a38fbcdc4fdcc50a
2023-04-26 21:56:58 +00:00
David Fu
44fc9a826b Changed assignee to triage_assignee in fuzzconfig
Change-Id: Ifdf4e8a3d855c414c906d249674411f4f4b88396
2023-04-26 20:29:44 +00:00
Treehugger Robot
c741160d81 Merge "Update clang version to clang-r487747b" 2023-04-26 20:04:42 +00:00
Anton Hansson
50151a7489 Remove --skip-kati
Not used anymore.

Test: No
Change-Id: Ide0d29b773583d6a8faff5b41a82d650cc2f9bcb
2023-04-26 18:09:02 +00:00
Mark Dacek
dc4617aa66 Merge "Refactor MixedBuildsEnabled and add --ensure-allowlist-integrity." 2023-04-26 15:23:11 +00:00
Julien Desprez
3895c286ff Merge "Remove jar target not used" 2023-04-26 15:19:35 +00:00
Jooyung Han
a2b57abb4a Remove makefile_goal
This was added to use 'make' target in soong. It was a hacky way and
caused many problems:
- can't track dependency
- doesn't work with sanitizer
- VSDK snapshot doesn't support
- ..

Now references are all gone. Let's remove it.

Bug: n/a
Test: m
Change-Id: I0336a490780f37a4e77d343c61315686e23db643
2023-04-26 16:46:03 +09:00
Yi Kong
c57e3a8397 Update clang version to clang-r487747b
Test: presubmit
Change-Id: I98de36a9f301dee14c21d42e20901eb5aa157164
2023-04-26 15:11:43 +09:00
Jooyung Han
b7cdbba71e Run apex_sepolicy_test for supported fs_type
For now ext4 is the only supported fs_type.

Bug: 279689445
Test: m apex.apexd_test_erosf
Test: m apex.apexd_test_f2fs
Change-Id: Idd1b2e61e25110da6d3805b082195e6ec031f573
2023-04-26 14:42:50 +09:00
Treehugger Robot
dfd4c8b8a4 Merge changes Ifd3117a4,Ic4f8b2b1
* changes:
  Revert "apex: skip symbol files when replaced with prebuilt"
  override_apex can be replaced with apex_set
2023-04-26 02:55:35 +00:00
Treehugger Robot
167f917f58 Merge "Include license information for vendor and host snapshots" 2023-04-26 01:34:04 +00:00
Cole Faust
2f1da168ab Expand preprocessed flag to work on android_app_imports
Setting the preprocessed flag will now also verify that the apk is
zip-aligned and does not have compressed JNI libs or dex files.

Bug: 185811447
Test: m nothing
Change-Id: I01b7c25f390345b14385f6f9e1640f48a5d9dc93
2023-04-25 15:04:58 -07:00
Christopher Parsons
bcbb1728ed Merge "Refactor and cleanup of bazel handler" 2023-04-25 21:27:57 +00:00
MarkDacek
f47e142ffd Refactor MixedBuildsEnabled and add --ensure-allowlist-integrity.
Currently, there is little verification around allowlisted modules
actually being mixed-built. This flag would allow us to verify
that a module allowlisted is mixed-built for at least one variant.

Bug: 278910100
Test: m nothing --bazel-mode-staging --ensure-allowlist-integrity
Test: m nothing --bazel-mode-staging --ensure-allowlist-integrity
--bazel-force-enabled-modules=com.google.android.neuralnetworks (This
fails, as expected)
Test: build/soong/test/mixed_mode_test.sh

Change-Id: Icd5976f4f44f1a8caca1e5247d986642f7995f97
2023-04-25 20:55:07 +00:00
Julien Desprez
45bd092132 Remove jar target not used
Test: presubmit
Bug: 279638202
Change-Id: I21d6f699d665a42d84a1dd899bcfc054c2454879
2023-04-25 13:41:29 -07:00
Spandan Das
8a8714c781 Do not modify input in-place
SortedUniqueStrings and FirstUniqueStrings dedupes repeating elements
and returns the deduped list. Currently, it also modifies the input list
in-place, which causes non-determinisitc failures like b/275313114

Operate on a copy of the input so that the input remains untouched.

SortedUniqueStrings is O(NlogN) and FirstUniqueStrings is ~O(N), so
creating a copy (O(N)) should not result in major performance regressions.
Numbers for this single unit test:
```
go test . -run TestStubsForLibraryInMultipleApexes -v -count 1000
Before: 174s
After: 172s
```

Test: go test ./android
Test: go test . -run TestStubsForLibraryInMultipleApexes -v -count 1000
Change-Id: Id859723b2c2ebdc0023876c4b6fabe75d870bad7
2023-04-25 18:37:24 +00:00
Spandan Das
00bafc6693 Re-enable TestMixedBuildUsesStubs
This was disabled because of some non-determinism in Soong, which has
been been fixed in aosp/2559590

Bug: 275313114
Test: go test ./cc -run TestMixedBuildUsesStubs -count 1000
Change-Id: I261dc785c832e66f8e213132c19111b8b90e40e9
2023-04-25 17:50:05 +00:00
Jihoon Kang
b420705230 Merge "Utilize from text core platform api surface jar in build" 2023-04-25 17:48:25 +00:00
Sam Delmerico
cf88420414 Merge "add neuralnetworks to prod allowlist" 2023-04-25 17:43:10 +00:00
Treehugger Robot
d7b50181d9 Merge "Use copy of apex_available in AvailableToSameApexes" 2023-04-25 17:27:00 +00:00
Chris Parsons
c9089dcc7b Refactor and cleanup of bazel handler
- Creation of a bazel command is independent of test/real implementation
- Use a custom struct instead of cmd.Exec
- Move mock bazel runner to the test

Bug: 270989498
Test: m nothing
Test: USE_PERSISTENT_BAZEL=0 m nothing
Test: Treehugger
Change-Id: Ieec35dad5e21aac644d5b8dc79a92397d42db861
2023-04-25 15:53:58 +00:00
Sam Delmerico
969e375d55 Merge "Use stubs for system_dynamic_deps" 2023-04-25 12:31:04 +00:00
Justin Yun
1db9748805 Include license information for vendor and host snapshots
In addition to the license text files, include license kinds
information to json file. Also, use the original paths of license
text files that are copied to NOTICE_FILES directory.
This will be used when generating snapshots from the snapshot build.

Bug: 271539873
Test: m vendor-snapshot
Change-Id: I0c0427bb66f2c1fca322d5554aa66220a3b62fb3
2023-04-25 18:03:32 +09:00
Jooyung Han
418364213a Revert "apex: skip symbol files when replaced with prebuilt"
This reverts commit 9551384f65.

5bedfee448 is a better solution for the
same problem.

Bug: 279247159
Test: m checkbuild (presubmit)
Change-Id: Ifd3117a4450b5275c09258d1ce896adb41a75158
2023-04-25 15:58:50 +09:00
Jooyung Han
912c4ab16f override_apex can be replaced with apex_set
Bug: 279247159
Test: m nothing
Change-Id: Ic4f8b2b1ee936afe2bf26de7d6cae19d93ecc831
2023-04-25 15:57:38 +09:00
Sam Delmerico
75dbca2eae Revert "Revert "mixed builds correctly reference stubs libs""
This reverts commit 1db4348734.

Changes from original:
  - extracted function to add the current version to stub
    versions for bp2build.
  - added libc++ to mixed builds denylist

Reason for revert: re-uploading with fix in topic

Change-Id: Ifa0ed456bf8cb4a7f861d6826263adfedb4fdd9c
2023-04-25 00:25:20 +00:00
Spandan Das
4e1bfa06e9 Use copy of apex_available in AvailableToSameApexes
Modifying the list in-place causes some non-determinism.

Test: go test ./cc -run TestStubsForLibraryInMultipleApexes -count 1000
Bug: 275313114
Change-Id: Ia2519e146d97667ad5900cf68ab9935fcbaf08a4
2023-04-24 22:45:14 +00:00
Treehugger Robot
28cbb75a09 Merge "disable mixed builds for sanitized cc modules" 2023-04-24 20:12:34 +00:00
Trevor Radcliffe
aea506707f Merge "convert hidden visibility flag to feature" 2023-04-24 19:31:36 +00:00
Jeffrey Vander Stoep
b611cf4240 Merge "Add option to use protobuf3" 2023-04-24 14:56:58 +00:00
Sam Delmerico
ef69d47d03 disable mixed builds for sanitized cc modules
The Bazel rules don't currently support any sanitizers other than ubsan,
so we should disable mixed builds for modules which are sanitized.

Test: go test
Bug: 278772861
Bug: 253433725
Change-Id: Ia01fb8cb59154bdfb21a111b04af0350e1876b0b
2023-04-24 10:36:12 -04:00
Jeff Vander Stoep
c1490ec25e Add option to use protobuf3
Users of the rust_protobuf module can use the 'use_protobuf3'
option to select version 3 of the protobuf crate instead of the
current default of version 2. This unblocks teams that would like
to use protobuf 3 immediately and provides a mechanism for us to
incrementally migrate users of protobuf 2 to protobuf 3.

Test: Adds tests that are executed during every build.
Bug: 270895633
Change-Id: Ib6a18b77cfa7dc3cc604cea05269004b3712bbb6
2023-04-24 11:33:50 +02:00
Treehugger Robot
a4c93e3bfc Merge "Add TestApexes to ApexInfo" 2023-04-24 07:03:34 +00:00
Treehugger Robot
e7773041d6 Merge "Remove module with a cc_test_library static dep" 2023-04-24 04:08:52 +00:00
Spandan Das
ac693b2f81 Use stubs for system_dynamic_deps
system libraries (libc, libm, libdl) belong to the runtime apex api
domain. platform and other apexes should link against stubs and not impl
of these libraries. Update bp2build so that both apex/non_apex picks
stubs. These axes need to be more granular in the future, but for now
this works because
- non_apex axis (platform) is a separate api domain than the runtime apex
- com.android.runtime which matches apex axis has not been allowlisted
  yet.

Test: TH
Change-Id: I43f429a90875f30d176db17b144dd51674ec8adf
2023-04-24 01:51:33 +00:00
Mark Dacek
a23feefdc4 Merge "Squelch output for a failure case in mixed_mode_test" 2023-04-22 06:42:08 +00:00