platform_build/core
Dan Willemsen e88ef9299a Rewrite test data logic to make it faster
It turns out we have a few tests with thousands of test data files,
which this logic does not handle very well. Just switching from `x :=
$(x) ...` to `x += ...` accomplishes the majority of these gains, but I
rewrote the logic to be less function-call+eval based to hopefully make
it easier to understand.

For an aosp-master/aosp_coral-eng build, this brings the time spent in
this logic from 7.6s to 0.3s (total kati runtime of 54s to 46s). Kati's
peak memory use (maxrss) also shrunk from 6.1GB to 3.8GB.

Bug: 158488548
Test: build-aosp_coral.ninja is the same before/after
Change-Id: Ia2ec8b29b5c1dd12fa7ea10b4f80fe802c692725
Merged-In: Ia2ec8b29b5c1dd12fa7ea10b4f80fe802c692725
(cherry picked from commit c5ec8e95bc)
2020-06-10 17:18:05 -07:00
..
clang Remove support for building Windows modules in Make 2019-02-05 00:19:41 -08:00
combo Add back the llvm-ar P flag 2019-09-06 15:19:59 -07:00
tasks Add tradefed-test-framework to the suites exported jar 2020-06-05 23:08:17 +00:00
aapt2.mk Allow passing --legacy in LOCAL_AAPT_FLAGS 2019-06-22 13:03:09 -07:00
aapt_flags.mk
allowed_ndk_types.mk
android_manifest.mk Allow codename.fingerprint format for minSdkVersion 2020-01-16 10:04:55 -08:00
app_certificate_validate.mk Access PRODUCT_ variables directly 2019-03-28 15:54:25 +00:00
app_prebuilt_internal.mk Incorrect file names in apkcerts.txt for Make modules 2020-05-03 10:57:01 +09:00
autogen_test_config.mk Update auto-gen test config template with EXTRA_CONFIGS placeholder 2019-07-15 17:29:42 +00:00
aux_config.mk Add back the llvm-ar P flag 2019-09-06 15:19:59 -07:00
aux_executable.mk Remove *_OUT_INTERMEDIATE_LIBRARIES 2018-09-15 10:52:13 -07:00
aux_static_library.mk
aux_toolchain.mk Move libgcc to libcrt.builtins 2018-11-08 13:40:34 -08:00
base_rules.mk Rewrite test data logic to make it faster 2020-06-10 17:18:05 -07:00
binary.mk Use sdk variant of Soong modules when LOCAL_SDK_VERSION is set 2020-04-27 14:35:20 -07:00
board_config.mk Introduce build variable OVERRIDE_TARGET_FLATTEN_APEX. 2020-05-19 02:06:45 +00:00
build-system.html Remove TARGET_ROOT_OUT_SBIN / TARGET_ROOT_OUT_SBIN_UNSTRIPPED 2019-04-17 11:12:32 -07:00
build_id.mk RESTRICT AUTOMERGE: rvc-dev is BUILD_ID=RVC 2020-03-29 16:03:39 -07:00
build_rro_package.mk Do not remove RRO resources 2020-01-14 17:00:13 -08:00
cc_prebuilt_internal.mk Restore "Linktype check for native:product" 2020-01-22 00:16:25 +00:00
ccache.mk
check_elf_file.mk Enable prebuilt ELF check for all targets 2019-12-09 14:00:25 -08:00
checktree
cleanbuild.mk
cleanspec.mk
clear_vars.mk Implement android_app_set module. 2020-06-03 09:50:48 -07:00
config.mk Add build support for LZ4 ramdisks 2020-05-29 22:46:31 +00:00
config_sanitizers.mk Stop linking libdl.a into static bins 2019-10-24 18:15:05 -07:00
configure_module_stem.mk
construct_context.sh Revert "remove telephony-common from boot class" 2019-09-24 22:00:18 +00:00
copy_headers.mk Stop removing valid copy headers 2019-12-26 16:52:34 -08:00
cxx_stl_setup.mk Update make build system for dynamic unwinder. 2020-01-10 11:09:52 -08:00
definitions.mk Use sdk variant of Soong modules when LOCAL_SDK_VERSION is set 2020-04-27 14:35:20 -07:00
deprecation.mk Mark the current BUILD_* warnings as errors by default 2020-02-19 18:11:18 -08:00
device.mk
dex_preopt.mk Add art dexpreopt to boot.zip archive 2019-12-04 16:50:17 -08:00
dex_preopt_config.mk Remove UseApexImage and GenerateApexImage. 2020-02-09 22:53:25 +00:00
dex_preopt_libart.mk Remove TARGET_TRANSLATE_2ND_ARCH 2019-09-18 16:23:04 +00:00
dex_preopt_libart_boot.mk Support for install multiple boot images. 2019-02-22 15:33:23 +00:00
dex_preopt_odex_install.mk Remove UseApexImage and GenerateApexImage. 2020-02-09 22:53:25 +00:00
distdir.mk Fix dist-for-goals with no targets 2018-10-20 16:59:41 -07:00
dumpvar.mk Fix dump-vars output to handle quotes. 2019-05-30 20:59:50 -07:00
dynamic_binary.mk Give strip depfile a unique name, properly include it 2020-01-27 22:06:21 +00:00
empty_test_config.xml
envsetup.mk Get rid of obsolete Make variable TARGET_CORE_JARS. 2020-03-24 17:09:47 +00:00
executable.mk Mark BUILD_HOST_EXECUTABLE as deprecated 2020-01-29 13:27:18 -08:00
executable_internal.mk Package coverage files as a zip. 2020-02-03 19:00:36 +00:00
executable_prefer_symlink.mk
filter_symbols.sh
force_aapt2.mk Don't set LOCAL_SDK_RES_VERSION to current if LOCAL_SDK_VERSION is core_current 2019-04-18 11:14:58 -07:00
fuzz_test.mk Fix to avoid breaking aosp_x86 build 2019-09-20 12:43:02 -07:00
generate_enforce_rro.mk Don't sign auto-generated RROs with platform cert 2019-05-01 16:42:56 +01:00
goma.mk Remove Goma resource check code. 2019-01-11 17:07:58 +09:00
header_library.mk Remove support for building Windows modules in Make 2019-02-05 00:19:41 -08:00
header_library_internal.mk Revert "Add soong_cc_prebuilt.mk" 2018-09-12 23:58:01 +00:00
host_dalvik_java_library.mk Set pools manually for rules that can run in RBE or goma 2019-11-12 13:59:49 -08:00
host_dalvik_static_java_library.mk
host_executable.mk Remove support for building Windows modules in Make 2019-02-05 00:19:41 -08:00
host_executable_internal.mk Add dependencies on the toolchain (clang, etc) 2019-06-12 21:34:52 +00:00
host_fuzz_test.mk Fix fuzzer builds. 2019-05-20 16:58:26 -07:00
host_java_library.mk Set pools manually for rules that can run in RBE or goma 2019-11-12 13:59:49 -08:00
host_java_library_common.mk Remove host- prefix from protobuf libraries 2018-10-26 23:55:49 -07:00
host_native_test.mk
host_prebuilt.mk
host_shared_library.mk Mark the current BUILD_* warnings as errors by default 2020-02-19 18:11:18 -08:00
host_shared_library_internal.mk Remove *_OUT_INTERMEDIATE_LIBRARIES 2018-09-15 10:52:13 -07:00
host_shared_test_lib.mk
host_static_library.mk Mark the current BUILD_* warnings as errors by default 2020-02-19 18:11:18 -08:00
host_static_library_internal.mk
host_static_test_lib.mk Deprecate next set of make module types 2019-05-08 12:27:53 -07:00
host_test_config.mk
host_test_internal.mk Remove support for building Windows modules in Make 2019-02-05 00:19:41 -08:00
install_jni_libs.mk Rename product_services to system_ext 2019-07-09 08:57:19 +00:00
install_jni_libs_internal.mk Don't check link type of Soong app JNI libraries 2020-05-15 10:09:57 -07:00
instrumentation_test_config_template.xml Update auto-gen test config template with PARAMETERIZED_STRINGS 2020-05-18 09:58:18 +08:00
jacoco.mk Use find(1)'s -exec directly. 2019-01-14 12:56:07 -08:00
java.mk Initialize common_proguard_flags for each module 2019-12-13 16:23:11 -08:00
java_common.mk Remove support for EXPERIMENTAL_JAVA_LANGUAGE_LEVEL_9. 2019-10-25 16:55:54 +01:00
java_host_test_config_template.xml Update auto-gen test config template with EXTRA_CONFIGS placeholder 2019-07-15 17:29:42 +00:00
java_library.mk Remove support for stripping dex. 2019-10-21 09:36:45 +01:00
java_prebuilt_internal.mk Remove support for stripping dex. 2019-10-21 09:36:45 +01:00
java_renderscript.mk Add labels to rewrapper cc compile action. Prevent ccwrapper from being 2019-08-12 14:13:31 -04:00
java_test_config_template.xml Update auto-gen test config template with EXTRA_CONFIGS placeholder 2019-07-15 17:29:42 +00:00
jetifier.mk Follow minor Jetifier usage change 2018-08-29 17:33:29 +00:00
line_coverage.mk rename libvixl to libvixl.com.android.art.debug 2020-05-05 18:49:46 +09:00
link_type.mk
LINUX_KERNEL_COPYING
local_systemsdk.mk Exempt RRO from the restriction on API level 2020-04-28 10:25:41 +09:00
local_vndk.mk Restore "Linktype check for native:product" 2020-01-22 00:16:25 +00:00
main.mk Merge "Clean up static_whitelist_patterns" into rvc-dev 2020-04-28 10:32:58 +00:00
Makefile fix spurious / on toplevel kernel module installations 2020-06-05 18:56:41 -07:00
misc_prebuilt_internal.mk Add RENDERSCRIPT_BITCODE to misc_prebuilt_internal whitelist. 2019-11-11 15:21:06 -08:00
module_arch_supported.mk Remove support for building Windows modules in Make 2019-02-05 00:19:41 -08:00
multi_prebuilt.mk Remove *_OUT_INTERMEDIATE_LIBRARIES 2018-09-15 10:52:13 -07:00
multilib.mk
native_benchmark.mk Eliminate ENABLE_DEFAULT_TEST_LOCATION as it is always true 2019-03-25 13:26:03 -07:00
native_benchmark_test_config_template.xml Update auto-gen test config template with EXTRA_CONFIGS placeholder 2019-07-15 17:29:42 +00:00
native_host_test_config_template.xml Update auto-gen test config template with EXTRA_CONFIGS placeholder 2019-07-15 17:29:42 +00:00
native_test.mk
native_test_config_template.xml Update auto-gen test config template with EXTRA_CONFIGS placeholder 2019-07-15 17:29:42 +00:00
ninja_config.mk Merge "Rename VTS to VTS10" am: 916be61b91 am: ed51dee98e am: 474ebdd3e9 am: 371d7b5194 2020-03-31 23:20:06 +00:00
node_fns.mk Add support for single-value product variables 2019-05-06 16:17:20 +01:00
notice_files.mk Use sdk variant of Soong modules when LOCAL_SDK_VERSION is set 2020-04-27 14:35:20 -07:00
OWNERS Update OWNERS for dex preopt related files. 2018-12-18 19:39:20 +00:00
pack_dyn_relocs_setup.mk Explicitly add lld's relr relocation flags for Android Pie and above 2019-07-30 22:36:29 -07:00
package.mk Remove TARGET_TRANSLATE_2ND_ARCH 2019-09-18 16:23:04 +00:00
package_internal.mk Incorrect file names in apkcerts.txt for Make modules 2020-05-03 10:57:01 +09:00
pathmap.mk
pdk_config.mk Replace framework.jar with framework-minus-apex.jar 2019-10-28 09:55:33 +00:00
pdk_fusion_modules.mk Fix pre-extracted PDKs 2018-11-05 10:57:24 -08:00
phony_package.mk
prebuilt.mk Remove TARGET_TRANSLATE_2ND_ARCH 2019-09-18 16:23:04 +00:00
prebuilt_internal.mk Add RENDERSCRIPT_BITCODE to misc prebuilt whitelist. 2019-11-11 09:30:38 -08:00
process_wrapper.sh
process_wrapper_gdb.cmds
process_wrapper_gdb.sh
product-graph.mk Ensure current product use PRODUCT_* vars directly 2019-10-30 14:29:42 +00:00
product.mk Support forcefully generating non-AB packages. 2020-05-07 12:48:17 -07:00
product_config.mk Add PRODUCT_SYSTEM_SERVER_JARS_EXTRA interface 2020-04-27 08:21:22 +00:00
proguard.flags Understand the support library/androidx @Keep annotations 2019-03-26 23:44:01 +00:00
proguard.jacoco.flags
proguard_basic_keeps.flags Keep proguard fields for lite runtime 2019-09-05 14:42:14 -07:00
project_definitions.mk
python_binary_host_test_config_template.xml ATest: Add test config template for python. 2018-11-20 15:31:44 +08:00
rbe.mk Update configuration for RemoteExecution actions 2020-04-02 07:13:39 -07:00
root.mk
rust_device_test_config_template.xml Add rust_device_test_config_template.xml 2019-12-02 17:44:47 +00:00
rust_host_test_config_template.xml Add rust_host_test_config_template.xml 2019-11-14 09:57:13 -08:00
sdk_check.mk
sdk_font.mk Move font xml files to "standard" directory 2020-01-29 18:26:32 +00:00
shared_library.mk Mark the current BUILD_* warnings as errors by default 2020-02-19 18:11:18 -08:00
shared_library_internal.mk Package coverage files as a zip. 2020-02-03 19:00:36 +00:00
shared_test_lib.mk
soong_android_app_set.mk Implement android_app_set module. 2020-06-03 09:50:48 -07:00
soong_app_prebuilt.mk Install .idsig files to test suite dirs. 2020-05-06 12:46:30 -07:00
soong_cc_prebuilt.mk Touch same_vndk_variant.timestamp 2020-04-30 04:02:20 +00:00
soong_config.mk Make RRO packages for system_ext 2020-04-20 13:59:52 +00:00
soong_droiddoc_prebuilt.mk Allow files generated by droidstubs to be copied in the SDK 2019-11-18 10:38:13 +00:00
soong_java_prebuilt.mk AIDEGen: Collect the classes.jar of the prebuilt modules into module-info.json 2019-06-14 10:47:18 +08:00
soong_rust_prebuilt.mk Set NATIVE_TESTS executable like EXECUTABLES 2019-11-15 14:10:09 -08:00
static_java_library.mk Add dependencies for files in LOCAL_PROGUARD_FLAGS 2019-08-09 09:19:52 -07:00
static_library.mk Mark the current BUILD_* warnings as errors by default 2020-02-19 18:11:18 -08:00
static_library_internal.mk Package coverage files as a zip. 2020-02-03 19:00:36 +00:00
static_test_lib.mk
suite_host_config.mk Generalize vts config module for other Suites 2019-10-18 18:28:23 +00:00
support_libraries.mk
target_test_config.mk
target_test_internal.mk Eliminate ENABLE_DEFAULT_TEST_LOCATION as it is always true 2019-03-25 13:26:03 -07:00
test_config_common.mk
use_lld_setup.mk Remove support for building Windows modules in Make 2019-02-05 00:19:41 -08:00
verify_uses_libraries.sh
version_defaults.mk Update Security String to 2020-08-05 am: d4b7f4de4b 2020-06-10 21:01:31 +00:00
WINPTHREADS_COPYING