No description
Find a file
Tao Bao 31b0807310 releasetools: Reduce the memory use in test_common.py.
test_common constructs a few 2GiB strings in memory, which leads to huge
memory footprint (18GiB). This CL moves away from in-memory strings to
generators, which reduces the memory use down to 41MiB. It also reduces
the time cost from 294s to 139s as an extra benefit for free.

The CL addresses some trivial pylint warnings as well.

* Before

$ /usr/bin/time -v python -m unittest -v test_common
...
----------------------------------------------------------------------
Ran 11 tests in 294.986s

OK
	Command being timed: "python -m unittest -v test_common"
	User time (seconds): 110.51
	System time (seconds): 109.34
	Percent of CPU this job got: 74%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 4:55.06
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 18894172
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 1
	Minor (reclaiming a frame) page faults: 20774908
	Voluntary context switches: 48
	Involuntary context switches: 3241
	Swaps: 0
	File system inputs: 184
	File system outputs: 8406424
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

* After

$ /usr/bin/time -v python -m unittest -v test_common
...
----------------------------------------------------------------------
Ran 11 tests in 139.100s

OK
	Command being timed: "python -m unittest -v test_common"
	User time (seconds): 59.00
	System time (seconds): 4.73
	Percent of CPU this job got: 45%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 2:19.17
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 41252
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 106569
	Voluntary context switches: 44
	Involuntary context switches: 103
	Swaps: 0
	File system inputs: 8
	File system outputs: 8422808
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

Fixes: 68988396
Test: See above.
Change-Id: I00f16603a4ee59fb085b189c6f5b5ee9d2378690
2017-11-08 17:10:33 -08:00
core Merge changes from topic "soong_ui_java" 2017-11-08 20:31:31 +00:00
target Merge "Add PRODUCT_ADB_KEYS" 2017-11-08 03:12:17 +00:00
tests Only set TARGET_PLATFORM version in lunch when explicitly requested 2017-05-03 02:47:59 +00:00
tools releasetools: Reduce the memory use in test_common.py. 2017-11-08 17:10:33 -08:00
.gitignore Add build subprojects to .gitignore 2015-07-23 13:18:47 -07:00
Android.mk Fix the build when using findleaves.py 2017-08-08 09:45:12 -07:00
buildspec.mk.default Add TARGET_PLATFORM_VERSION to lunch 2017-03-23 09:44:08 -07:00
CleanSpec.mk Remove PRODUCT_AAPT2_CFLAGS and PRODUCT_AAPT_FLAGS 2017-10-27 16:25:55 -07:00
envsetup.sh resolve merge conflicts of 0c8b471be3 to stage-aosp-master 2017-10-23 11:41:58 -04:00
help.sh Consolidate build system usage documentation into source control 2017-08-09 12:58:30 -07:00
OWNERS Add OWNERS in build/make 2017-06-21 16:09:56 -07:00
README.txt Consolidate build system usage documentation into source control 2017-08-09 12:58:30 -07:00
tapasHelp.sh Support 'tapas help' for more-detailed tapas help 2017-09-11 15:03:50 -07:00

Android build system usage:

m [-j] [<targets>] [<variable>=<value>...]


Ways to specify what to build:
  The common way to specify what to build is to set that information in the
  environment via:

    # Set up the shell environment.
    source build/envsetup.sh # Run "hmm" after sourcing for more info
    # Select the device and variant to target. If no argument is given, it
    # will list choices and prompt.
    lunch [<product>-<variant>] # Selects the device and variant to target.
    # Invoke the configured build.
    m [<options>] [<targets>] [<variable>=<value>...]

      <product> is the device that the created image is intended to be run on.
        This is saved in the shell environment as $TARGET_PRODUCT by `lunch`.
      <variant> is one of "user", "userdebug", or "eng", and controls the
        amount of debugging to be added into the generated image.
        This gets saved in the shell environment as $TARGET_BUILD_VARIANT by
          `lunch`.

    Each of <options>, <targets>, and <variable>=<value> is optional.
      If no targets are specified, the build system will build the images
      for the configured product and variant.

  An alternative to setting $TARGET_PRODUCT and $TARGET_BUILD_VARIANT,
  which you may see in build servers, is to execute:

    make PRODUCT-<product>-<variant>


  A target may be a file path. For example, out/host/linux-x86/bin/adb .
    Note that when giving a relative file path as a target, that path is
    interpreted relative to the root of the source tree (rather than relative
    to the current working directory).

  A target may also be any other target defined within a Makefile. Run
    `m help` to view the names of some common targets.

  To view the modules and targets defined in a particular directory, look for:
    files named *.mk (most commonly Android.mk)
      these files are defined in Make syntax
    files named Android.bp
      these files are defined in Blueprint syntax

  For now, the full (extremely large) compiled list of targets can be found
    (after running the build once), split among these two files:

    ${OUT}/build-<product>*.ninja
    ${OUT}/soong/build.ninja

    If you find yourself interacting with these files, you are encouraged to
    provide a more convenient tool for browsing targets, and to mention the
    tool here.

Targets that adjust an existing build:
  showcommands              Display the individual commands run to implement
                            the build
  dist                      Copy into ${DIST_DIR} the portion of the build
                            that must be distributed

Flags
  -j <N>                    Run <N> processes at once
  -j                        Autodetect the number of processes to run at once,
                            and run that many

Variables
  Variables can either be set in the surrounding shell environment or can be
    passed as command-line arguments. For example:
      export I_AM_A_SHELL_VAR=1
      I_AM_ANOTHER_SHELL_VAR=2 make droid I_AM_A_MAKE_VAR=3
  Here are some common variables and their meanings:
    TARGET_PRODUCT          The <product> to build # as described above
    TARGET_BUILD_VARIANT    The <variant> to build # as described above
    DIST_DIR                The directory in which to place the distribution
                            artifacts.
    OUT_DIR                 The directory in which to place non-distribution
                            artifacts.

  There is not yet known a convenient method by which to discover the full
  list of supported variables. Please mention it here when there is.