When making this script deterministic we accidentally changed it so that
none of the entries in the output zip file were compressed. Re-add
the compression.
Adding the compression makes atest go from 67M to 34M, at the cost of
0.33s startup time instead of 0.28s.
Test: Presubmits
Change-Id: I4e58eb76235ddabf0195694cdd3644f26b7e84ec
This reverts commit 8a038ed871.
Reason for revert: investigate breaking test suits Avatar and BumbleBluetoothTests
Change-Id: I8e4498fb5854bf62df6d161ac0c6cce49d471afe
Now a stacktrace involving precompile_python.py won't be shown,
and it will show all errors instead of just the first one.
Bug: 331488610
Test: Presubmits
Change-Id: I47376ca6764c0f84d1aaf5db329664e20143640a
embedded_launcher causes the python interpreter to be bundled in the
built zip file. This is beneficial because:
- You can export the built executable and run it on systems
without python or an old version of python. The CI servers have python
3.7, so it's useful to update that.
- Because the python version is known, the python sources will be
precompiled.
- It will start running the code directly from the zip file, without
extracting the binary to a temporary directory like non-embedded
launcher binaries. This is particuarly useful because some developers
use the extracted files directly instead of using
`importlib.resources`, so having the non-extracting implementation
gives them a nudge in the right direction.
Make embedded_launcher the default so that more developers will do the
right thing when developing python programs.
Bug: 331488610
Test: Presubmits
Change-Id: Ideb113a1d4d3b29ac04d57a48d111a99f77b2dfc
This CL fixes timestamps for zip entries to a hardcoded value to ensure
deterministic zip archives across checkouts, not just for incremental builds.
The value chosen for the fixed date is the same as the one used by soong_zip.
Bug: b/322788229
Test: Ran a few builds and verified that mdate for zipped files are
fixed as well as the generated zip files have identical hashes.
Change-Id: I3cd6fcf559d4d83d8813c93319b46e267ae64a2b
- Support test_config and test_config_template
- The `config` file extension is widly used by TF. Use `configv2` for
ATS 2.0 tests to avoid configuration errors in TF.
- Add `mobly` tag to support atest
Test: m CtsBluetoothMultiDevicesTestCases (with `runner = "mobly"`)
Bug: 299850687
Change-Id: Idbed90ab30e35aeed8cdeeb0aa0aa7adb75a3cd9
Timestamps in pyc files and zip archives were causing cache misses in RBE.
Bug: b/322788229
Test: Ran a few local builds and verified improved cache hit rate for metalava actions.
Change-Id: I9375694d26a107f5c0c43bbd685bd8e86349d187
Convert all of the callers of SetProvider to use the type-safe
android.SetProvider API.
Bug: 316410648
Test: builds
Change-Id: If58f4b5355264ddab2045bc3591a4eac19cd58fc
Remove the ConvertWithBp2build implementations from all the module
types, along with the related code.
Bug: 315353489
Test: m blueprint_tests
Change-Id: I212672286686a318893bc7348ddd5a5ec51e77a7
To generate module-info.json in Soong for b/309006256 Soong needs to
know the test data paths. Moving test data installation into Soong will
also help later for test suite packaging.
Add ModuleContext.InstallTestData that installs the files listed in a
[]DataPath alongside the test. The files will also be passed to Make
to allow it to continue packaging them into the test suites for now.
Update the module types that are producing LOCAL_TEST_DATA entries
in their Android.mk files to go through InstallTestData instead.
Bug: 311428265
Test: atest --host toybox-gtests --test-timeout=120000
Change-Id: Ia8b964f86e584ea464667fd86a48d754d118bead
This Cl adds a new rule to Soong to generate test spec metadata. Also, this CL adds a provider in various test module to provide test spec related data to the Soong rule.
Will add providers and test code to other Module in the future changes.
Provider added for the following test modules in this change: android_robolectric_test, android_test, bootclasspath_fragment_test, java_test, java_test_host, python_test, python_test_host, sh_test,and sh_test_host.
Bug: 296873595
Change-Id: I5f89f72d5874bb7838ae357efdb8c6ca208e18a7
This no-op refactoring facilitates some upcoming functional changes for
"bp2build allowlist v2". The work requires that the bp2build conversion
mutator be changed from a TopDown mutator to a BottomUp mutator.
Refactoring all bp2build-related methods so that they use Bp2buildMutatorContext
makes it easier to make this functional change without touching tens of
files and multiple projects.
Bug: 285631638
Test: m bp2build
Change-Id: I3d1ef3064146e959c6f0dc315350fc9764bf2bd2
This reverts commit b1094d602b.
Reason for revert: prebuilts updated in aosp/2726579
Bug: 278602456
Change-Id: I750b0f7e4dd94329a4455499f869a1823e660d77
* changes:
Handle proto.include_dirs in bp2build for py
Add proto.local_include_dirs support in bp2build
Add manual tag to proto_library at the root package
Followup to aosp/2711114 which added proto.include_dirs support for cc
Test: go test ./bp2build
Test: Added an integration test in build/bazel/examples
Bug: 285140726
Change-Id: Ic78dacbe46d9ea67e9441948852a3f94dabbb35e
If a python_library uses a pkg_path foo/bar, then the proto srcs in
that libray need to import the dep .proto as foo/bar/proto.proto.
This behavior is restricted to python modules. To implement this is in
bp2build, this CL creates a new interface with a single method
`PkgPath`. Only python module structs implement this interface, and
this method is only available during bp2build
Test: Added a bp2build unit test
Test: TH
Change-Id: If8d207c0b321f75337a053795826b283a5eaaf46
The precompilation logic assumes that the source tree matches the python
prebuilts, which is not necessarily true. This very much breaks during
major version upgrades, so just disable precompilation temporarily while
we generate new prebuilts.
Bug: 278602456
Test: treehugger
Change-Id: Idb206bb1d971e8c0fa556c75419623c3bc457898
Due to upstream changes in python path calculations, I'm simplifying the
python launcher to work more like a standard python distribution. This
is effectively changing the stdlib path from `internal/stdlib` to
`internal/python3.10` (or `3.11`, etc). This allows us to specify the
zip file as PYTHONHOME, set PYTHONPLATLIBDIR to `internal` and use the
default detection after that.
That does mean during upgrades that the stdlib pkg path will change, so
move the source vs prebuilt calculation from the Android.bp into Soong
to choose which stdlib module to pick up (with the corresponding
`pkg_path`)
Bug: 278602456
Test: treehugger with python3.10
Test: a python3.11 source + 3.10 prebuilt build
Test: a python3.11 source+prebuilt build
Change-Id: I8b02e7b22a1f1d1e02819ae1a31a99cdc985542c
We encountered a duplicate sources bug in python proto generation
because we're only using the base name. This CL makes it use the
entire path to avoid this.
Fixes: 295333879
Test: Presubmits
Change-Id: I4ab821508cd5979bd595e327c1ccd3d68110fba6
* changes:
Handle .proto files in different package for filegroups
Handle .proto files that end up in a different package
Allow creation of BazelTargets in a different directory
Bazel poses a strict requirement that .proto files and proto_library
must be in the same package. This CL handles this automatically by
creating the proto_library in a separate dir/package if necessary
Implementation details
- Partition the `srcs` by package. `srcs` has been computed using
`transformSubpackagePath`, so the information about packages is
available at this point
- Create a proto_library in each package by using
`CommonAttributes.Dir`. Collect all these additional libraries
and put them in `info.Proto_libraries` so that they get added as deps
of (cc|python|...)_proto_library
- Add an import_prefix to the proto_library in subpackages relative to
the current directory. This relies on the assumption that every src is
beneath the current directory (Soong will complain if a path in
Android.bp contains ../)
filegroup module type uses a separate code-path to create proto_library.
This will be handled in the next CL in stack.
Test: bp2build unit tests
Test: TH
Test: Built the failing internal module mentioned in
b/292583584#comment1
Bug: 292583584
Change-Id: I437fc89092321b26c5f0511387cde9e84084d6f9
Ic37c8db918873ddf324c86b12b5412952b0f2be2 converted python_binary_host
and python_library_host from HostSupportedNoCross to HostSupported,
but left python_test_host. Do the same for python_test_host.
Fixes dependencies between java_test_host modules that create
host cross variants and python_test_host modules that were missing
host cross variants.
Bug: 282918027
Test: builds
Change-Id: Iae687aa3aa0f0b005f2dd27469f631145247a008
There are no remaining Python 2 binaries using an embedded
launcher in Android.
Bug: 245854393
Test: m par_test py2-cmd py3-cmd && build/soong/python/tests/runtest.sh
Change-Id: I241bbaa417060b51b4d2883011ccb43e22ace4c8
There was a request for using b with python tests. bp2build python
tests exactly the same way as python binaries so that they can be
used with `b`.
Bug: None
Test: go test
Change-Id: Id68a6a73572745a4885b3e5bb1b8452e36baa982
Except if BUILD_BROKEN_USES_SOONG_PYTHON2_MODULES is set, and except for
some core py2 modules that can't be removed until python2 is fully gone.
Bug: 203436762
Test: m nothing
Change-Id: I62ccb6f5687eab1e79c372ffc234a90ca5b566ac
Mobly is a Python-based test framework that specializes in supporting
test cases that require multiple devices. There are some Mobly based
Android tests, but these tests cannot be added to xTS since they are
not using TF. Currently TF provides a 2-device Mobly test runner.
This change is a short-term solution to help teams (eg. bluetooth) to
add their multi-device Mobly tests to xTS without writing TF configs.
For long-term, device infra team and xTS infra team will provide a
solution for more complex multi-device tests in xTS and deprecate this
TF template.
Test: manually check the generated mobly xml config
Bug: 272239365
Change-Id: I252cc774094eebbd9264243d30023ace4839c2cd
Still fall back to the hardcoded list for what to put in LOCAL_SHARED_LIBRARIES,
because I couldn't figure out a good way to get those modules at the moment.
Bug: 259718110
Test: Presubmits
Change-Id: Ia992ffea8819d8c67280b08bc45505f77af09de0
This signifigantly improves the startup time of soong-built
python binaries. For example, running
`m apexer && time out/host/linux-x86/bin/apexer` gives
0.734s before this cl, and 0.094s after.
Fixes: 259718110
Test: Presubmits
Change-Id: Ib19e83e2c60c39a849525be117279c318de3afa7
