This reverts commit 8a038ed871.
Reason for revert: investigate breaking test suits Avatar and BumbleBluetoothTests
Change-Id: I8e4498fb5854bf62df6d161ac0c6cce49d471afe
embedded_launcher causes the python interpreter to be bundled in the
built zip file. This is beneficial because:
- You can export the built executable and run it on systems
without python or an old version of python. The CI servers have python
3.7, so it's useful to update that.
- Because the python version is known, the python sources will be
precompiled.
- It will start running the code directly from the zip file, without
extracting the binary to a temporary directory like non-embedded
launcher binaries. This is particuarly useful because some developers
use the extracted files directly instead of using
`importlib.resources`, so having the non-extracting implementation
gives them a nudge in the right direction.
Make embedded_launcher the default so that more developers will do the
right thing when developing python programs.
Bug: 331488610
Test: Presubmits
Change-Id: Ideb113a1d4d3b29ac04d57a48d111a99f77b2dfc
Remove the ConvertWithBp2build implementations from all the module
types, along with the related code.
Bug: 315353489
Test: m blueprint_tests
Change-Id: I212672286686a318893bc7348ddd5a5ec51e77a7
There was a request for using b with python tests. bp2build python
tests exactly the same way as python binaries so that they can be
used with `b`.
Bug: None
Test: go test
Change-Id: Id68a6a73572745a4885b3e5bb1b8452e36baa982
Still fall back to the hardcoded list for what to put in LOCAL_SHARED_LIBRARIES,
because I couldn't figure out a good way to get those modules at the moment.
Bug: 259718110
Test: Presubmits
Change-Id: Ia992ffea8819d8c67280b08bc45505f77af09de0
This signifigantly improves the startup time of soong-built
python binaries. For example, running
`m apexer && time out/host/linux-x86/bin/apexer` gives
0.734s before this cl, and 0.094s after.
Fixes: 259718110
Test: Presubmits
Change-Id: Ib19e83e2c60c39a849525be117279c318de3afa7
The goal of this cl is to simplify the python rules,
mostly by removing the "decorator" pattern that they
currently use, and instead making separate module
types for libraries, binaries, and tests that inherit
from each other.
Bug: 259718110
Test: Verified ninja files are unchanged (they only change in the list of soong sources because I added/deleted files)
Change-Id: I1e836e2cc4782c7818f91db7df7895de3b8db7ca
The new behavior has been enabled by default, and these
flags aren't necessary anymore.
Fixes: 245583294
Test: m py_dont_import_folder_of_entrypoint_test && /ssd/aosp-master/out/host/linux-x86/testcases/py_dont_import_folder_of_entrypoint_test/x86_64/py_dont_import_folder_of_entrypoint_test
Change-Id: I5b6f98da51791bc5d28662ef799a10c1bb6a35a0
1) Bp2build convert python_binary_host main attribute as LabelAttribute. Currently "main" attribute in python_binary_host is handled as string but for some modules (e.g certify_bootimg) the "main" attribute points to a file in its subpackage like "subpackage/file.py" and should be converted to "//.../subpackage:file.py".
2) Filter out duplicated labels in the merged list of "required" attributes of python_binary_host and its defaults.
Test: b build //system/tools/mkbootimg:certify_bootimg
Test: b build //build/make/tools/releasetools:check_target_files_signatures
Bug: 253081249
Bug: 253101186
Change-Id: Ic2cb4cadec2c1348da70af9f0730da9914d3a8ca
This reverts commit 4d5cc709fa.
Reason for revert: Acloud issue was fixed in aosp/2243150, make sure the host unit tests are run on this cl via go/abtd before submitting
Change-Id: Ie148430d331a01a9877adaf3cb88f1c57af136a0
Bug: 245583294
Bug: 247578564
Bug: 251688241
This reverts commit 5fe655d1c7.
Reason for revert: DroidMonitor: Potential culprit for Bug b/251688241 - verifying through ABTD before revert submission. This is part of the standard investigation process, and does not mean your CL will be reverted.
Change-Id: I1ea4df27557fc891d8f0156ec5681f9b0c601c98
Now that usages have been updated, enable the new
behavior by default. If everything goes well with
this, we can then remove the flags.
Bug: 245583294
Bug: 247578564
Test: Presubmits
Change-Id: Idac8d2dc0f96ae750dba4a99b32525676d0d6f25
The python interpreter will by default add the directory of
the entrypoint script to the beginning of sys.path. This
can be disabled in python 3.11+ (which is not released yet)
using the PYTHON_SAFE_PATH environment variable or the -P flag.
As a workaround to have this behavior in older python versions,
we can make an __soong_entrypoint_redirector__.py file at the
root of the zip file that is the entrypoint, and then that
file will redirect to the real entrypoint.
This brings non-embedded-launcher python modules closer to
the embedded launcher version. The embedded launcher binaries
already act like this because they start at an __main__.py file
at the root of the zip file.
Bug: 245583294
Test: m py_dont_import_folder_of_entrypoint_test && out/host/linux-x86/nativetest64/py_dont_import_folder_of_entrypoint_test/py_dont_import_folder_of_entrypoint_test
Change-Id: I39aaf04fb19c3ba7f5c9d98220872d6d08abf736
stub_template_host.txt added all the top-level modules to the
PYTHONPATH, which isn't correct, and caused absl.logging to
override the built-in logging module.
Removing this also makes it more consistent with python binaries
built with embedded_launcher: true. embedded_launcher: true
binaries don't add the top-level modules.
Fixes: 245583294
Test: m py_dont_add_top_level_dirs_test && out/host/linux-x86/testcases/py_dont_add_top_level_dirs_test/x86_64/py_dont_add_top_level_dirs_test
Change-Id: Id3069565d2b2c4b2bda0ff5301e757a7b4201751
This also introduces a workaround for the fact that
apexer depends on aapt2, but aapt2 doesn't build
with bp2build yet. Aapt2 is removed from apexer's
requirements during bp2build.
Bug: 204244290
Test: ./build/bazel/ci/bp2build.sh
Change-Id: I837597ce035c7d5c06e1a3957166583a7a94b5c7
Each conversion required defining a separate mutator, which will each
operate on _all_ modules and requires each to repeat checks whether the
mutator should operator. Instead, we introduce a single mutator and
modules can define a ConvertWithBp2build to implement bp2build
conversion for that module.
Test: bp2build.sh
Bug: 183079158
Change-Id: I99d4b51f441c2903879092c5b56313d606d4338d
Previously, could not set an empty string as a value of an attribute;
however, this is necessary in some cases. To not unnecessarily create an
empty string, use string pointers for attributes rather than strings.
Test: go test bp2build tests
Change-Id: I03b3a3567452d455246d22d81f86c317d06b7c39
Introduce `commonAttributes` & `fillCommonBp2BuildModuleAttrs used in
CreateBazelTargetModule
Adapt `bp2BuildInfo` to use `commonAttrs` instead of `Name`.
And thus also all downstream users of `CreateBazelTargetModule`.
As initial user, the Soong `required` property will be
translated to Bazel's `data`.
Bug: 198146582, 196091467
Test: build_converstion_test.go:TestCommonBp2BuildModuleAttrs
Test: go test
Test: mixed_{libc,droid}.sh
Change-Id: Ib500e40f7e2cb48c459f1ebe3188962fc41ec124
Have binary and library bp2build convert `libs` to
`deps` for bazel
Bug: 196083751
Test: python_{binary,library}_conversion_test.go
Test: build/bazel/ci/mixed_{libc,droid}.sh
Change-Id: I2d5f6ef2e83dd608910edb7adb2eb9a56d25293c
A performance improvement for bp2build as Blueprint/Soong no longer have
the overhead of additional modules. The creation of these modules
results in:
* traversal of additional modules for each subsequent mutator
* synchronization over a go channel to collect newly created modules:
https://cs.android.com/android/platform/superproject/+/master:build/blueprint/context.go;l=2594,2600;drc=1602226f23181b8c3fbfcaf3358f0297e839d7d3
We avoid both of these by storing the information directly in the
underlying module.
Also as a fringe benefit, removes some necessary boilerplate for
conversion.
For benchmarks, reduces runtime ~1% for 1% converted, ~24% for 100%
converted. See more: go/benchmarks-for-https:-r.android.com-1792714
Test: ran benchmarks/tests in bp2build
Test: build/bazel/ci/bp2build.sh
Change-Id: Ie9273b8cbab5bc6edac1728067ce184382feb211
This CL adds the support for specifying lists of directories in
build/soong/android/bazel.go, which are then written into
out/soong/bp2build/MANIFEST. Using this configuration,
modules/directories can either default to bp2build_available: true or
false, while still retaining the ability to opt-in or out at the module level.
It also ensures that ConvertWithBp2Build returns true iff the module
type has a registered bp2build converter.
Test: go tests
Test: demo.sh full
Test: TreeHugger presubmits for bp2build and mixed builds.
Change-Id: I0e0f6f4b1b2ec045f2f1c338f7084defc5d23a55
This CL adds a simple python_binary_host to native py_binary converter
for standalone modules that don't depend on libs. It also adds support
for the conditional py2/py3 build target based on the version prop.
Test: milestone-2/demo.sh full
Test: bazel query 'kind("py_binary rule", //bionic/...)'
Test: bazel run //bionic/libc/tools:genfunctosyscallnrs
Test: bazel run //bionic/libc/tools:genseccomp
Test: go tests
Fixes: 182236395
Change-Id: Ibe5ec6cd0dc12a61b3a449a8c723d80b891fae42
A host target is considered as being cross-compiled when the target
can't run natively on the build machine. For example, linux_glibc/x86_64
is a non-cross target on a standard x86/Linux machine, but is a cross
host on Mac. Previously, whether cross or not was a static attribute of
an OsType. For example, Windows was always considered as cross host,
while linux_bionic was not. This becomes a problem when we support more
host targets like linux_bionic/arm64 which should be cross-host on
standard x86/Linux machines.
This change removes HostCross from the OsClass type and instead adds a
property HostCross to the Target type. When a target is being added, it
is initialized to true when the target can't run natively on the current
build machine.
Bug: 168086242
Test: m
Change-Id: Ic37c8db918873ddf324c86b12b5412952b0f2be2
When multiple os/arch variants are supported, java_test_host could not
find a matching arch due to java having arch:common, whereas native
binaries support a specific architecture. This change adds the property
`data_native_bins` in order to support binaries with the appropriate
os/arch variants.
Test: m FirmwareDtboVerification with data_native_bins
Test: forrest
Bug: 153848038
Change-Id: I45adebff0fde2811d5ef5620c697b97b768c951f
Attribute `auto_gen_config` is added to test modules.
Test config will be generated if:
the attribute is not set and AndroidTest.xml doesn't exists
or
the attribute is set to true, whether or not AndroidTest.xml exists.
Test config will NOT be auto-generated if:
the attribute is not set and AndroidTest.xml exists
or
the attribute is set to false, whether or not AndroidTest.xml exists.
Bug: 141684102
Test: build test module with auto_gen_config set to true
Change-Id: I64fb003a83d8c32a967835e5f8d12fe4476043be
Mainly so that we can build a `python` prebuilt that acts like the
normal python install, though you could also build different ones with
more packages pre-installed.
Bug: 117811537
Test: move built py2-cmd into prebuilts/build-tools/path/linux-x86/python and build
Change-Id: I21215f6fd3754d89f8c65e1dfeb3f2deea23239f
The root cause is we didn't check if the optionalpath is valid or not,
the registerbuildation function directly invoke the path var and
cause the invalid memory error. We just return if the launcher doesn't
exist.
The ctx.VisitDirectDepsWithTag() also handles allowmissingdependency so
we are ok if launcher doesn't exist.
Test: N/A
Bug: b/116698229, b/67510844
Change-Id: I40941079a64d7797ab879fc5edaa29e835b493a0
Each Python module will generate a zip file containing source & data
files. The Python binary will collect all its dependencies and use
merge_zips to merge each zip file to create a final .par file.
Test: m -j checkbuild && real examples:
Bug: b/70568913
Change-Id: I9ff232d461d33e1c06026e7dcb5b124bf02c3ce5
Add a method on ModuleContext and TopDownMutatorContext to visit
direct dependencies that have a given dependency tag.
Test: m checkbuild
Change-Id: Ib875563091dcae6b7282b3e3427d0eb07d8c8af5
This reverts commit 0a32e5936b.
Reason for revert: <libsqlite was missing for darwin_x86_64 before>
Change-Id: I2e13e849a503a705ffad425df292380f2f73954e
Add python_test to generate executables for both host and device side.
And change python_library_host to python_library.
installation for python device test will be under
target/product/generic_arm[64]/data/nativetest[64].
Bug: b/69114465
Test: m clean && m -j
Device Test:
adb root; adb push nan_devicetest /data;
adb shell; ./nan_devicetest
print(sys.path)
['/data/nan_devicetest/runfiles', '/data/nan_devicetest', '/data/nan_devicetest/internal', '/data/nan_devicetest/internal/stdlib']
Change-Id: If8317070a3aa1b6dab2e84b8df2d037f495d7247
Also adds checks that the dependencies are android.Modules and
are not disabled.
Test: m checkbuild
Change-Id: I05e945f38915d49cd3c0ab72a86576949bc7eff2
Ubuntu has both aliases, but Mac only declares python2.7, not python2.
Test: `m sepolicy_tests` on internal master
Change-Id: Ia2b5d2d95195f307befbc1893a465752f14b8a7b
launcher.
For Python2, we bundle embedded launcher as bootstrapper within every
.par file. This feature is only enabled for linux_x86_64 for now. We
provide a user flag: hermetic_enabled within bp file. By default, Pyhon2
still use classic bootstrapping way to construct .par file and relys on
host interpreter. Once embedded_launcher is enabled, launcher will be
used to bootstrap .par file and execute user program.
For Python3, the launcher will be ready soon, and for now it still relys
on classic bootstrapping.
Test: Real example is used to test.
Bug: b/63018041
Change-Id: I28deba413d8ad3af407595e46f77d663e79a3705
Change module factories from returning a blueprint.Module and a list
of property structs to returning an android.Module, which holds the
list of property structs.
Test: build.ninja identical except for Factory: comment lines
Change-Id: Ica1d823f009db812c518f271a386fbff39c9766f
The base module handles all the common functionalites, such as version
compatibilty check, version variations split, source file format check,
source/data file duplicate check.
The library/binary module focuses on how to generate binary build actions,
such as setting up stub script, zipping, filling in __init__.py in
runfiles dir tree.
Bug: b/31676493
Test: go test under python package
Change-Id: I06608369f350f7195873d459e1c8d1bdb811e77e
