Commit graph

216 commits

Author SHA1 Message Date
Aleks Todorov
1eb06c4a67 genrule: Make cmd configurable
Bug: 325067608
Bug: 325444956
Test: m
Test: m [custom genrule with select]
Change-Id: I3a2307a8d0f65ee419b935a30742f96e5733a81f
2024-06-04 18:20:29 +01:00
Justin Yun
40182b6ff3 Remove duplicated CollectDependencyAconfigFiles()
android.ModuleBase already calls aconfigUpdateAndroidBuildActions()
that is the same with CollectDependencyAconfigFiles(). Remove the
CollectDependencyAconfigFiles() to avoid duplication with
aconfigUpdateAndroidBuildActions().

To make the aconfig information available in
GenerateAndroidBuildActions() of all modules, call
aconfigUpdateAndroidBuildActions() before calling
GenerateAndroidBuildActions() of each module.

Also, we don't need SetAconfigFileMkEntries(), which is a duplicate
of aconfigUpdateAndroidMkData()

Bug: 335363964
Test: diff `adb shell printflags` before and after the change.
Change-Id: I52808e442e9fed7db1eae7b7c5ed0b1c5ba74f5d
2024-05-10 10:00:14 +09:00
Cole Faust
a963b94cde Make the enabled property configurable
This allows using select statements with it.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I6f3efaaa3d82505e38a91ee4ba0e18e404360191
Merged-In: If355d24506e3f117d27b21442a6c02bca3402dc7
2024-05-02 15:41:24 -07:00
yangbill
6d032dd911 Add a new property trim_extension for gensrcs
In order to provide a more flexible ability for gensrcs's output, due to
current output_extension property will only allow to replace the string
after the last "." of input, in order to handle input with multiple dot
in file name, provide a way to trim the suffix of input string.

Bug: 335536003
Test: cd build/soong/genrule ; go test -run TestGenSrcs
Test: cd build/soong/genrule ; go test -run TestGenSrcsWithTrimExtAndOutpuExtension
Test: cd build/soong/genrule ; go test -run TestGenSrcsWithTrimExtButNoOutpuExtension
Test: cd build/soong/genrule ; go test -run TestGenSrcsWithOutpuExtension
Test: cd build/soong/android ; go test -run TestPathsForModuleSrc

Change-Id: I033bbe1d225f207f0f6bdc140df308884f214b51
2024-04-30 05:06:56 +00:00
Jiyong Park
74ff1f80bc Merge "Ensure that genrule's host tool deps are for host" into main 2024-03-22 23:33:48 +00:00
Jiyong Park
8fb0e9734c Ensure that genrule's host tool deps are for host
Previously, when gathering transitive dependencies of genrule's host
tool, TransitivePackagingSpecs() was used if available. However, when
using this, it didn't check whether the dependencies are for host or not
(i.e. device). This was fine because host module can only depend on host
modules.

However, this is preventing Soong from implementing the installation of
the required modules by itself. Then a host module may depend on a
device module or vice versa (because that's the symantics of the
property).

To prevent the issue, check the partition of the transitive dependencies
of the host tool and filter out ones whose partition is not "" (which
means host).

Bug: 321626681
Test: builds
Change-Id: I296a920b3b88e7601315a4e1768af9727d0061e6
2024-03-20 16:17:21 +09:00
Treehugger Robot
1f4ffda2ed Merge "Support multilib property for cc_genrule" into main 2024-03-20 02:38:30 +00:00
kellyhung
750334a0c0 Support multilib property for cc_genrule
Change genrule Out to `android:"arch_variant"` for supporting
multilib variants.

Bug: 323295272
Test: go test -run TestMultilibGenruleOut

Change-Id: I102d64b45a5a2a5193f813001fc32da0d1fe9d36
2024-03-20 09:25:41 +08:00
Cole Faust
e762b94fe7 Remove SandboxingDenyPathList
It's no longer used.

Bug: 307824623
Test: m nothing --no-skip-soong-tests
Change-Id: I181de9d852f4025c6c5829cc4941ffb705f7cb50
2024-03-15 19:50:03 +00:00
Cole Faust
5549257422 Remove depfile support from genrules
All usages of it have been removed.

Bug: 307824623
Test: Presubmits
Change-Id: I9502b328a5aaa6840653f46011ca6bd05f3cba99
2024-01-25 18:05:18 -08:00
LaMont Jones
afe7baf47d Propagate aconfig providers for more modules.
Bug: 308625757
Test: manual
Change-Id: Iaf6d45a4259f1c6c34476c34c431344283ae2830
2024-01-10 14:03:30 +00:00
Colin Cross
402130276c Convert SetProvider to generic providers API
Convert all of the callers of SetProvider to use the type-safe
android.SetProvider API.

Bug: 316410648
Test: builds
Change-Id: If58f4b5355264ddab2045bc3591a4eac19cd58fc
2023-12-14 16:12:20 -08:00
Cole Faust
27609f15ff Update wording on depfile property error
Bug: 307824623
Test: Presubmits
Change-Id: I25919150d2ee2483099e7cb710330dc259d57e66
2023-12-12 18:10:36 -08:00
Colin Cross
a72573a727 Merge "Remove ConvertWithBp2build implementations" into main 2023-12-11 23:12:53 +00:00
Colin Cross
8ff105860d Remove ConvertWithBp2build implementations
Remove the ConvertWithBp2build implementations from all the module
types, along with the related code.

Bug: 315353489
Test: m blueprint_tests
Change-Id: I212672286686a318893bc7348ddd5a5ec51e77a7
2023-12-08 13:51:05 -08:00
Cole Faust
ce74a59bb2 individually escape results of $(locations)
proptools.ShellEscape escapes mostly by surrounding the string in
quotes. This makes all the files from a $(locations) be passed as
one shell argument if escaping was needed. Escape them individually
so that they're still passed as separate arguments.

Bug: 307824623
Test: running art tests with aosp/2860435
Change-Id: I9940941f4a7bb4d29d9388e4d57f8cbd732b6d32
2023-12-08 10:53:59 -08:00
Aditya Choudhary
26df39fe44 Add source file provider for genrule/srcs, python libraries and rust libraries.
Change-Id: I2d7d4684a10c15aeecc27b8db800ab27a807d2e2
2023-12-05 19:56:26 +00:00
Colin Cross
a644c263dd Stop collecting path entry for module_bp_java_deps.json from each module type
The jdepsGeneratorSingleton can get the module path directly, it doesn't
need to be collected by each module type that implements IDEInfo.  Fixes
module types (like android_library) that didn't reach the code that
collected the path.

Bug: 309835196
Test: out/soong/module_bp_java_deps.json contains path for ExtServices.core
Change-Id: If8cb81b4f708e0367f156ade164bee253bf53492
2023-11-15 11:40:48 -08:00
Christopher Parsons
c331812a59 Merge "Have ConvertWBp2build use Bp2buildMutatorContext" into main 2023-09-20 17:56:24 +00:00
Chris Parsons
637458d326 Have ConvertWBp2build use Bp2buildMutatorContext
This no-op refactoring facilitates some upcoming functional changes for
"bp2build allowlist v2". The work requires that the bp2build conversion
mutator be changed from a TopDown mutator to a BottomUp mutator.
Refactoring all bp2build-related methods so that they use Bp2buildMutatorContext
makes it easier to make this functional change without touching tens of
files and multiple projects.

Bug: 285631638
Test: m bp2build
Change-Id: I3d1ef3064146e959c6f0dc315350fc9764bf2bd2
2023-09-20 14:49:35 +00:00
Liz Kammer
9e2a5a7d6d Don't panic for unhandled product vars
Instead, we return an error. This allows us to access some product
variable information earlier when it will not be used as an attribute
without panicing

Test: m nothing
Change-Id: Id094b2b9e1364a8d174d99b3824fa149fb235b3e
2023-09-19 15:58:22 -04:00
Romain Jobredeaux
9973ace1c7 Bp2build support for app R8-related fields
Bug: 293304784
Test: CI
Change-Id: I0aefa57f968cb41e50d7c1d29b2d8a1480655355
2023-08-31 01:35:44 -04:00
Sam Delmerico
f877563735 update docs for genrule tool_files prop
According to go/roboleaf-busy-beavers-sandboxing, this field should just take in any data files needed by the tool. The existing description makes it sound like this property should only contain a single file that will be used as the tool itself.

Change-Id: I3ef3b8ceb52f7a7e6de9e0a897d5cc05c9c2d336
2023-08-14 23:54:55 +00:00
Cole Faust
f0d4d4f9f2 Bp2build genrule commands as configurable properties
So that users can use soong config variables / product variables
to adjust a genrule's command.

Bug: 295910468
Test: m nothing
Change-Id: I9fedf8d5d52e515c3fdb913411ce1b3fecb7ba81
2023-08-14 11:58:20 -07:00
Spandan Das
a99348dca4 cc Bp2build support for genrules that generate .proto file
If `srcs` contains a gensrcs/genrule module, the current bp2build module
will put it in the catch-all `srcs` attribute. This is reserved for .cpp
sources, so if the genrule produces a .proto/.aidl/... file, this will
fail.

This handles genrules that produce .proto files. To implement this, this
creates an additional partition that detects if the other module is a
genrule/gensrc that produces .proto files. If true, it will append it to
the proto partition.

This CL does not handle
- genrule that produce .c/.aidl/.yacc/.... files. They will continue to
  be partitioned into the catch-all partition
- java modules

Test: unit tests
Test: TH
Bug: 293205700
Change-Id: Ib720fdf3a053ff5dd256e6faa632e3fa7776966d
2023-08-03 18:56:05 +00:00
Liz Kammer
0db0e34c68 Gen a header library when genrules export includes
Embedding multiple includes from a genrule may be difficult to read,
instead we generate a header library that contains the headers and all
include dirs that Soong generates.

Test: go test bp2build tests
Change-Id: I590c74c133f015f27cccf5a2fd916153ad9c125e
2023-07-24 13:57:17 -04:00
Liz Kammer
d07cb48c6f Merge "Make genrule export_include_dirs more consistent" into main 2023-07-21 13:41:23 +00:00
Liz Kammer
d38c87c035 Make genrule export_include_dirs more consistent
Previously, genrule export_include_dirs always added ModuleDir to
exported include dirs when export_include_dirs is set but not when
export_include_dirs is not set. Now when export_include_dirs is set, we
also export the directory without the additional ModuleDir subdir.

Test: genrule go tests
Test: set export_include_dirs and test
Change-Id: I46e860b2c20c1a96bddd14367d7fa737d901994d
2023-07-17 09:58:50 -04:00
Justin Yun
4da4ccc203 Add write_if_changed property to genrule
When "write_if_changed: true" is set, it will call restat for ninja.
With this option the output file will be copied only if it is changed.

Bug: 290130959
Test: ninja rule include "--write-if-changed"
Change-Id: I8bd77b43b22eb0115e0bdc73718b2d6997d92652
2023-07-17 20:35:13 +09:00
Liz Kammer
796921d211 Ensure data is passed to all generation actions
Test: go test genrule tests
Test: build/soong/tests/genrule_sandbox_test.py \
      framework-cppstream-protos framework-javastream-protos
Change-Id: I16fa939280c7ff0f230a0a8cf18af5e55f16eb03
2023-07-11 08:27:51 -04:00
Chris Parsons
b7950a93df Add -gen suffix to genrule targets with conflicts
Fixes: 224580460
Test: m nothing
Change-Id: I3af3eb0fb7f65216087482eb620484229ac7d674
2023-06-16 17:41:42 +00:00
Yu Liu
accbd555a7 Merge "Fix a race problem in sandboxing genrule logic." 2023-06-14 04:38:55 +00:00
Yu Liu
e7f7cbfad2 Fix a race problem in sandboxing genrule logic.
Bug: 286978650
Test: locally build
Change-Id: I4fab03f74f5634666d22f7ed6e18ee69f780d6ca
2023-06-13 19:04:24 +00:00
Liz Kammer
8bd92429e5 bp2build data property -> data attr in gensrcs
Test: go test bp2build tests
Change-Id: I2192cf9846a5783385397337d40cca85a1ef73e7
2023-06-09 15:37:45 -04:00
Liz Kammer
81fec18a5e Add property data to gensrcs for additional inputs
This enables sandboxing for inputs that are necessary but do not need to
have a source file generated.

Test: GENRULE_SANDBOXING=true m framework-javastream-protos \
      framework-cppstream-protos
Change-Id: Id5ca1dab5799c25fa96b564a7d2008c2e7b5382b
2023-06-09 15:29:32 -04:00
Yu Liu
45d6af544a Refactor the genrule allowlists code to support internal allowlists
Bug: 281067655
Test: build locally
Change-Id: Iedac64c3d59fc48f1b5a1b461c53d00e92b653b5
2023-05-31 21:40:47 +00:00
Yu Liu
6a7940cc99 Support sandboxing genrule
Bug: 281067655
Test: local build
Change-Id: I123b5e51eaf1476d1215e5208331b6540d45fd98
2023-05-22 15:02:36 -07:00
Spandan Das
39b6cc5336 Ignore test apexes from bp2build generated tags
Soong does not enforce apex_available on the contents of test apex. To
prevent special-casing test apexes in the apex validation aspect in
Bazel, drop the test apexes from the tags altogether.

( The core problem I am trying to solve is making sure that stub
libraries in Bazel have a single apex available. apex validation happens
to be a nice side benefit)

Bug: 277651159
Test: go test ./bp2build
Change-Id: Ibb3cfedb5c0f2cda0464bf3758c70b67cb5885d1
2023-04-27 23:24:49 +00:00
Sam Delmerico
eddd3c0ba4 add APEX transitive dependency validation
Bug: 218419109
Test: b build //packages/modules/adb/apex:com.android.adbd
Change-Id: I93b0c99d6521e419e52c63271646448f6d708c22
2022-12-13 14:32:19 -05:00
Yu Liu
d6201013eb Make cc_genrule.srcs configurable.
Also disallow arch variant of genrule.out.

This is to be consistent with bazel where we are migrating to.

Bug: 254114674
Test: Manual
Change-Id: I685a2e64102b7bb68128b39931f0bc85878bc6de
2022-10-19 12:35:26 -07:00
Vinh Tran
32a98a5d55 Convert genDir to RULEDIR in gensrcs's cmd
Test: CI
Change-Id: I5ecc19dc73559bb518871080737b28a649edc162
2022-09-23 14:18:20 -04:00
Vinh Tran
f19b658803 bp2build to convert $(genDir) to $(RULEDIR) in genrule's cmd
`statslog.cpp` with cmd `"$(location stats-log-api-gen) --cpp $(genDir)/statslog.cpp"` produces `out/.intermediates/frameworks/proto_logging/stats/stats_log_api_gen/statslog.cpp/gen/statslog.cpp`. Hence, $(genDir) is equivalent to `<package-dir>/<module-name>/gen`.

Currently, bp2buld converts `$(genDir)` to `$(GENDIR)`. In Bazel, `$(GENDIR)` is only the base of the generated code (e.g. `bazel-bin`).

```
genrule(
    name = "statslog.cpp",
    cmd = "$(location :stats-log-api-gen) --cpp $(GENDIR)/statslog.cpp",
    outs = ["statslog.cpp"],
    tools = [":stats-log-api-gen"],
)
```
produces `bazel-bin/statslog.cpp` but expects to have `bazel-bin/frameworks/proto_logging/stats/stats_log_api_gen/statslog.cpp`.

By converting to `$(RULEDIR)` which is `bazel-bin/frameworks/proto_logging/stats/stats_log_api_gen`.

There had not been any genrule module allowlisted with genDir
yet. So this should not cause any issue with modules converted before
this CL.

Bug: 247536535
Test: go tests
Test: presubmit builds and tests
Change-Id: I65c6aafadab6b180b7ef700427e041547ae7e98a
2022-09-22 08:57:14 -04:00
Jihoon Kang
c170af4e5b Resolve escaping error in genrules sources with $
'$' cannot be included in the genrule source file name as it is an
invalid character.
One workaround to include a file with '$' in the filename is to use
glob(*) and match pattern.
However, shell currently evaluates the '$' sign and leads to unexpected behavior.
This change fixes the issue by shell-escaping the filepath in generating
build actions.

Test: m
Bug: b/194980152
Change-Id: I6fd919c568b5b6526e4de5155104a08ecadab307
2022-08-24 19:06:49 +00:00
Colin Cross
d079e0b270 Reformat build/soong for go 1.19
Test: none
Change-Id: I132368f0fcbdb5ea088b5b84dbe4ccfdd9e94cad
2022-08-17 10:43:13 -07:00
Vinh Tran
140d588a3d Deprecate depfile in gensrcs
All existing gensrsc modulePartners who use the property can use BUILD_BROKEN_DEP_FILE to bypass the error

Test: CI
Bug: 179452413
Fix: 179452413
Change-Id: I7cd39484b43eba693d79188b9a374f192198f90f
2022-06-27 12:07:27 -04:00
Cole Faust
01243368d7 Allowlist apexer for bp2build
This also introduces a workaround for the fact that
apexer depends on aapt2, but aapt2 doesn't build
with bp2build yet. Aapt2 is removed from apexer's
requirements during bp2build.

Bug: 204244290
Test: ./build/bazel/ci/bp2build.sh
Change-Id: I837597ce035c7d5c06e1a3957166583a7a94b5c7
2022-06-09 13:28:25 -07:00
Vinh Tran
b69e1aec66 Modify ConvertWithBp2build mutator to convert gensrcs to Bazel rule
Test: b build --platforms=//build/bazel/platforms:linux_x86
//art/runtime:art_operator_src

Change-Id: I942d68e17968327cc77f379edce9b73416e6c4fd
2022-06-01 16:00:28 -04:00
Chris Parsons
f874e46153 Refactor mixed builds to only take one pass
This large refactoring has both immense performance implications and
improves mixed builds complexity / usability. Summary:

1. Queueing calls to Bazel is done in a new mutator instead of a full
   soong_build pass. Normal soong_build flow is interrupted (via a
   functional hook in blueprint) to invoke bazel and parse its response.
2. Implementing mixed build support for additional modules is as simple
   as implementing MixedBuildsBuildable. In this interface, define the
   request that must be queued to Bazel, and then subsequently define
   how to handle the returned bazel cquery metadata.
3. Mixed builds consists of only a single pass. This greatly
   improves mixed build performance.

Result:
  A 33% runtime improvement on soong analysis phase with mixed builds.

Caveats:
  C++ BazelHandler handling still remains a bit of a mess; I did what
  I could within this CL's scope, but this may require additional cleanup.

Test: Treehugger
Test: Verified that aosp_arm ninja file is bit-for-bit identical with or
without this change.

Change-Id: I412d9c94d429105f4ebfafc84100d546069e6621
2022-05-20 10:04:13 -04:00
MarkDacek
ff851b83b6 Log information for Mixed Builds modules.
Test: Output matches expected. https://paste.googleplex.com/5913495636803584?raw
Performance evaluated: https://docs.google.com/spreadsheets/d/1X7eOVBKEZUwUWl5i8CDfBo9yUeZrDPXWi2JYO4BEZt4/edit?resourcekey=0-co8crIFW9dpiedhCMkhAgw#gid=0

Change-Id: I88780c7cc52a189a72216c5e2e499c96574b3731
2022-05-03 18:11:32 +00:00
Liz Kammer
619be4626b Restrict genrules to disallow directories as input
While Bazel genrules will allow genrules to accept a directory as input,
the results can be unexpected to a user as changes to the contents of
the directory may not trigger a rebuild as expected. Restricting this
in Soong ensures that converted targets will behave as expected.

Test: CI
Change-Id: I8616f58d1df267005e6c0ca3f4730d06de52c0d9
2022-02-23 21:29:22 +00:00