275 lines
11 KiB
Markdown
275 lines
11 KiB
Markdown
# Build System Changes for Android.mk Writers
|
|
|
|
### `export` and `unexport` deprecation {#export_keyword}
|
|
|
|
The `export` and `unexport` keywords have been deprecated, and will throw
|
|
warnings or errors depending on where they are used.
|
|
|
|
Early in the make system, during product configuration and BoardConfig.mk
|
|
reading: these will throw a warnings, and will be an error in the future.
|
|
Device specific configuration should not be able to affect common core build
|
|
steps -- we're looking at triggering build steps to be invalidated if the set
|
|
of environment variables they can access changes. If device specific
|
|
configuration is allowed to change those, switching devices with the same
|
|
output directory could become significantly more expensive than it already can
|
|
be.
|
|
|
|
Later, during Android.mk files, and later tasks: these will throw errors, since
|
|
it is increasingly likely that they are being used incorrectly, attempting to
|
|
change the environment for a single build step, and instead setting it for
|
|
hundreds of thousands.
|
|
|
|
It is not recommended to just move the environment variable setting outside of
|
|
the build (in vendorsetup.sh, or some other configuration script or wrapper).
|
|
We expect to limit the environment variables that the build respects in the
|
|
future, others will be cleared. (There will be methods to get custom variables
|
|
into the build, just not to every build step)
|
|
|
|
Instead, write the export commands into the rule command lines themselves:
|
|
|
|
``` make
|
|
$(intermediates)/generated_output.img:
|
|
rm -rf $@
|
|
export MY_ENV_A="$(MY_A)"; make ...
|
|
```
|
|
|
|
If you want to set many environment variables, and/or use them many times,
|
|
write them out to a script and source the script:
|
|
|
|
``` make
|
|
envsh := $(intermediates)/env.sh
|
|
$(envsh):
|
|
rm -rf $@
|
|
echo 'export MY_ENV_A="$(MY_A)"' >$@
|
|
echo 'export MY_ENV_B="$(MY_B)"' >>$@
|
|
|
|
$(intermediates)/generated_output.img: PRIVATE_ENV := $(envsh)
|
|
$(intermediates)/generated_output.img: $(envsh) a/b/c/package.sh
|
|
rm -rf $@
|
|
source $(PRIVATE_ENV); make ...
|
|
source $(PRIVATE_ENV); a/b/c/package.sh ...
|
|
```
|
|
|
|
## Implicit make rules are obsolete {#implicit_rules}
|
|
|
|
Implicit rules look something like the following:
|
|
|
|
``` make
|
|
$(TARGET_OUT_SHARED_LIBRARIES)/%_vendor.so: $(TARGET_OUT_SHARED_LIBRARIES)/%.so
|
|
...
|
|
|
|
%.o : %.foo
|
|
...
|
|
```
|
|
|
|
These can have wide ranging effects across unrelated modules, so they're now obsolete. Instead, use static pattern rules, which are similar, but explicitly match the specified outputs:
|
|
|
|
``` make
|
|
libs := $(foreach lib,libfoo libbar,$(TARGET_OUT_SHARED_LIBRARIES)/$(lib)_vendor.so)
|
|
$(libs): %_vendor.so: %.so
|
|
...
|
|
|
|
files := $(wildcard $(LOCAL_PATH)/*.foo)
|
|
gen := $(patsubst $(LOCAL_PATH)/%.foo,$(intermediates)/%.o,$(files))
|
|
$(gen): %.o : %.foo
|
|
...
|
|
```
|
|
|
|
## Removing '/' from Valid Module Names {#name_slash}
|
|
|
|
The build system uses module names in path names in many places. Having an
|
|
extra '/' or '../' being inserted can cause problems -- and not just build
|
|
breaks, but stranger invalid behavior.
|
|
|
|
In every case we've seen, the fix is relatively simple: move the directory into
|
|
`LOCAL_MODULE_RELATIVE_PATH` (or `LOCAL_MODULE_PATH` if you're still using it).
|
|
If this causes multiple modules to be named the same, use unique module names
|
|
and `LOCAL_MODULE_STEM` to change the installed file name:
|
|
|
|
``` make
|
|
include $(CLEAR_VARS)
|
|
LOCAL_MODULE := ver1/code.bin
|
|
LOCAL_MODULE_PATH := $(TARGET_OUT_ETC)/firmware
|
|
...
|
|
include $(BUILD_PREBUILT)
|
|
|
|
include $(CLEAR_VARS)
|
|
LOCAL_MODULE := ver2/code.bin
|
|
LOCAL_MODULE_PATH := $(TARGET_OUT_ETC)/firmware
|
|
...
|
|
include $(BUILD_PREBUILT)
|
|
```
|
|
|
|
Can be rewritten as:
|
|
|
|
```
|
|
include $(CLEAR_VARS)
|
|
LOCAL_MODULE := ver1_code.bin
|
|
LOCAL_MODULE_STEM := code.bin
|
|
LOCAL_MODULE_PATH := $(TARGET_OUT_VENDOR)/firmware/ver1
|
|
...
|
|
include $(BUILD_PREBUILT)
|
|
|
|
include $(CLEAR_VARS)
|
|
LOCAL_MODULE := ver2_code.bin
|
|
LOCAL_MODULE_STEM := code.bin
|
|
LOCAL_MODULE_PATH := $(TARGET_OUT_VENDOR)/firmware/ver2
|
|
...
|
|
include $(BUILD_PREBUILT)
|
|
```
|
|
|
|
You just need to make sure that any other references (`PRODUCT_PACKAGES`,
|
|
`LOCAL_REQUIRED_MODULES`, etc) are converted to the new names.
|
|
|
|
## Valid Module Names {#name}
|
|
|
|
We've adopted lexical requirements very similar to [Bazel's
|
|
requirements](https://docs.bazel.build/versions/master/build-ref.html#name) for
|
|
target names. Valid characters are `a-z`, `A-Z`, `0-9`, and the special
|
|
characters `_.+-=,@~`. This currently applies to `LOCAL_PACKAGE_NAME`,
|
|
`LOCAL_MODULE`, and `LOCAL_MODULE_SUFFIX`, and `LOCAL_MODULE_STEM*`.
|
|
|
|
Many other characters already caused problems if you used them, so we don't
|
|
expect this to have a large effect.
|
|
|
|
## PATH Tools {#PATH_Tools}
|
|
|
|
The build has started restricting the external host tools usable inside the
|
|
build. This will help ensure that build results are reproducible across
|
|
different machines, and catch mistakes before they become larger issues.
|
|
|
|
To start with, this includes replacing the $PATH with our own directory of
|
|
tools, mirroring that of the host PATH. The only difference so far is the
|
|
removal of the host GCC tools. Anything that is not explicitly in the
|
|
configuration as allowed will continue functioning, but will generate a log
|
|
message. This is expected to become more restrictive over time.
|
|
|
|
The configuration is located in build/soong/ui/build/paths/config.go, and
|
|
contains all the common tools in use in many builds. Anything not in that list
|
|
will currently print a warning in the `$OUT_DIR/soong.log` file, including the
|
|
command and arguments used, and the process tree in order to help locate the
|
|
usage.
|
|
|
|
In order to fix any issues brought up by these checks, the best way to fix them
|
|
is to use tools checked into the tree -- either as prebuilts, or building them
|
|
as host tools during the build.
|
|
|
|
As a temporary measure, you can set `TEMPORARY_DISABLE_PATH_RESTRICTIONS=true`
|
|
in your environment to temporarily turn off the error checks and allow any tool
|
|
to be used (with logging). Beware that GCC didn't work well with the interposer
|
|
used for logging, so this may not help in all cases.
|
|
|
|
## Deprecating / obsoleting envsetup.sh variables in Makefiles
|
|
|
|
It is not required to source envsetup.sh before running a build. Many scripts,
|
|
including a majority of our automated build systems, do not do so. Make will
|
|
transparently make every environment variable available as a make variable.
|
|
This means that relying on environment variables only set up in envsetup.sh will
|
|
produce different output for local users and scripted users.
|
|
|
|
Many of these variables also include absolute path names, which we'd like to
|
|
keep out of the generated files, so that you don't need to do a full rebuild if
|
|
you move the source tree.
|
|
|
|
To fix this, we're marking the variables that are set in envsetup.sh as
|
|
deprecated in the makefiles. This will trigger a warning every time one is read
|
|
(or written) inside Kati. Once all the warnings have been removed for a
|
|
particular variable, we'll switch it to obsolete, and any references will become
|
|
errors.
|
|
|
|
### envsetup.sh variables with make equivalents
|
|
|
|
| instead of | use |
|
|
|--------------------------------------------------------------|----------------------|
|
|
| OUT {#OUT} | OUT_DIR |
|
|
| ANDROID_HOST_OUT {#ANDROID_HOST_OUT} | HOST_OUT |
|
|
| ANDROID_PRODUCT_OUT {#ANDROID_PRODUCT_OUT} | PRODUCT_OUT |
|
|
| ANDROID_HOST_OUT_TESTCASES {#ANDROID_HOST_OUT_TESTCASES} | HOST_OUT_TESTCASES |
|
|
| ANDROID_TARGET_OUT_TESTCASES {#ANDROID_TARGET_OUT_TESTCASES} | TARGET_OUT_TESTCASES |
|
|
|
|
All of the make variables may be relative paths from the current directory, or
|
|
absolute paths if the output directory was specified as an absolute path. If you
|
|
need an absolute variable, convert it to absolute during a rule, so that it's
|
|
not expanded into the generated ninja file:
|
|
|
|
``` make
|
|
$(PRODUCT_OUT)/gen.img: my/src/path/gen.sh
|
|
export PRODUCT_OUT=$$(cd $(PRODUCT_OUT); pwd); cd my/src/path; ./gen.sh -o $${PRODUCT_OUT}/gen.img
|
|
```
|
|
|
|
### ANDROID_BUILD_TOP {#ANDROID_BUILD_TOP}
|
|
|
|
In Android.mk files, you can always assume that the current directory is the
|
|
root of the source tree, so this can just be replaced with '.' (which is what
|
|
$TOP is hardcoded to), or removed entirely. If you need an absolute path, see
|
|
the instructions above.
|
|
|
|
### Stop using PATH directly {#PATH}
|
|
|
|
This isn't only set by envsetup.sh, but it is modified by it. Due to that it's
|
|
rather easy for this to change between different shells, and it's not ideal to
|
|
reread the makefiles every time this changes.
|
|
|
|
In most cases, you shouldn't need to touch PATH at all. When you need to have a
|
|
rule reference a particular binary that's part of the source tree or outputs,
|
|
it's preferrable to just use the path to the file itself (since you should
|
|
already be adding that as a dependency).
|
|
|
|
Depending on the rule, passing the file path itself may not be feasible due to
|
|
layers of unchangable scripts/binaries. In that case, be sure to add the
|
|
dependency, but modify the PATH within the rule itself:
|
|
|
|
``` make
|
|
$(TARGET): myscript my/path/binary
|
|
PATH=my/path:$$PATH myscript -o $@
|
|
```
|
|
|
|
### Stop using PYTHONPATH directly {#PYTHONPATH}
|
|
|
|
Like PATH, this isn't only set by envsetup.sh, but it is modified by it. Due to
|
|
that it's rather easy for this to change between different shells, and it's not
|
|
ideal to reread the makefiles every time.
|
|
|
|
The best solution here is to start switching to Soong's python building support,
|
|
which packages the python interpreter, libraries, and script all into one file
|
|
that no longer needs PYTHONPATH. See fontchain_lint for examples of this:
|
|
|
|
* [external/fonttools/Lib/fontTools/Android.bp] for python_library_host
|
|
* [frameworks/base/Android.bp] for python_binary_host
|
|
* [frameworks/base/data/fonts/Android.mk] to execute the python binary
|
|
|
|
If you still need to use PYTHONPATH, do so within the rule itself, just like
|
|
path:
|
|
|
|
``` make
|
|
$(TARGET): myscript.py $(sort $(shell find my/python/lib -name '*.py'))
|
|
PYTHONPATH=my/python/lib:$$PYTHONPATH myscript.py -o $@
|
|
```
|
|
### Stop using PRODUCT_COMPATIBILITY_MATRIX_LEVEL_OVERRIDE directly {#PRODUCT_COMPATIBILITY_MATRIX_LEVEL_OVERRIDE}
|
|
|
|
Specify Framework Compatibility Matrix Version in device manifest by adding a `target-level`
|
|
attribute to the root element `<manifest>`. If `PRODUCT_COMPATIBILITY_MATRIX_LEVEL_OVERRIDE`
|
|
is 26 or 27, you can add `"target-level"="1"` to your device manifest instead.
|
|
|
|
### Stop using USE_CLANG_PLATFORM_BUILD {#USE_CLANG_PLATFORM_BUILD}
|
|
|
|
Clang is the default and only supported Android compiler, so there is no reason
|
|
for this option to exist.
|
|
|
|
### Other envsetup.sh variables {#other_envsetup_variables}
|
|
|
|
* ANDROID_TOOLCHAIN
|
|
* ANDROID_TOOLCHAIN_2ND_ARCH
|
|
* ANDROID_DEV_SCRIPTS
|
|
* ANDROID_EMULATOR_PREBUILTS
|
|
* ANDROID_PRE_BUILD_PATHS
|
|
|
|
These are all exported from envsetup.sh, but don't have clear equivalents within
|
|
the makefile system. If you need one of them, you'll have to set up your own
|
|
version.
|
|
|
|
|
|
[build/soong/Changes.md]: https://android.googlesource.com/platform/build/soong/+/master/Changes.md
|
|
[external/fonttools/Lib/fontTools/Android.bp]: https://android.googlesource.com/platform/external/fonttools/+/master/Lib/fontTools/Android.bp
|
|
[frameworks/base/Android.bp]: https://android.googlesource.com/platform/frameworks/base/+/master/Android.bp
|
|
[frameworks/base/data/fonts/Android.mk]: https://android.googlesource.com/platform/frameworks/base/+/master/data/fonts/Android.mk
|