Commit graph

44 commits

Author SHA1 Message Date
Jooyung Han
df09d173ca Generate linker config with packaged items.
Previously, we gather all deps without checking if a dep is actually one
of packaged items (CopyDepsToZip contents)

For example, following items shouldn't be listed.

provideLibs: "platform-mainline-sdk_libartpalette-system@current.so"
provideLibs: "platform-mainline-sdk_liblog@current.so"
provideLibs: "runtime-module-sdk_libc@current.so"
provideLibs: "runtime-module-sdk_libdl@current.so"
provideLibs: "runtime-module-sdk_libdl_android@current.so"
provideLibs: "runtime-module-sdk_libm@current.so"

Now, we gather only those modules contributing CopyDepsToZip to generate
linker config for android_system_image.

Test: m nothing
Test: m microdroid
Test: conv_linker_config print -s
    .../microdroid/android_common/root/system/etc/linker.config.pb
    shouldn't list sdk members.
Change-Id: Ife5d2193ab59970367d10e196aaaa922c6582862
2021-05-11 11:56:54 +09:00
Jiyong Park
fa616137a2 android_system_image that generates linker.config.pb
android_system_image module type is a specialization of the
android_filesystem module type. Currently, it adds a build rule for
creating linker.config.pb from the information about all the other files
in the filesystem and includes linker.config.pb to the filesystem as
well.

To do so, the filesystem module now provides a function pointer which
subtype modules like android_system_image can implement to pass extra
files that they want to package in the filesystem.

In addition, the linkerconfig package is revised to make it possible to
build linker.config.pb file outside of the package.

Bug: 185391776
Test: m microdroid and inspect etc/linker.config.pb in it.

Change-Id: Id89c40b519213062860d7306029b8413d8d36a2d
2021-04-21 09:11:21 +09:00
Jooyung Han
9706cbc1e9 Add filesystem_test.go
It has only one test function as a starting point.

Bug: n/a
Test: m (soong tests)
Change-Id: I785b096805014a40dfd600f7baaf884f4016c23c
2021-04-16 13:20:02 +09:00
Jiyong Park
da2d6ee7a2 Truncate vbmeta to 64KB
libavb expects to be able to read the maximum vbmeta size, so we must
provide a partition which matches this or the read will fail.

Bug: 181923506
Test: m MicrodroidHostTestCases
libavb tries

Change-Id: Icc5b86ccd98e3ed9c4269e584f35dbfbdf1b0730
2021-04-14 16:42:24 +09:00
Jiyong Park
30a083215b bootimg.cmdline is arch specific
microdroid_boot.img needs different kernel cmdline for x86_64 and arm64.

Bug: 185082754
Test: atest MicrodroidHostTests
Change-Id: I69be83cbdada6c49a033da2787908490ee76de36
2021-04-13 13:17:59 +09:00
Inseob Kim
a46b51cf20 Support super image size automatic calculation
If the size is set to "auto", the size will be automatically calculated
by lpmake.

Bug: 181107248
Test: boot microdroid
Change-Id: I9dd599ca64e4d442bfb83fe45b1f03080a74f1e1
2021-04-01 13:19:30 +09:00
Inseob Kim
152a702955 Add default group support for logical_partition
Default group has no size limit and becomes automatically minimized.

Bug: 181107248
Test: boot microdroid
Change-Id: Id38d3ab173db5fa01db3d471af15747d30b1820c
2021-03-31 10:46:46 +09:00
Treehugger Robot
9b6108f8de Merge "Fix PackagingBase.CopyDepsToZip" 2021-03-23 04:21:38 +00:00
Jiyong Park
e7569bc544 Merge "Add vbmeta module type" 2021-03-18 00:02:41 +00:00
Treehugger Robot
ab534a3b5c Merge "bootimg signs image using verity_utils" 2021-03-17 23:37:04 +00:00
Jooyung Han
092ef811a0 Fix PackagingBase.CopyDepsToZip
CopyDepsToZip() zips direct dependencies with tags implementing
PackagingItem interface.

Previously, it relied on InstallNeededDependencyTag which has a
different meaning.
- InstallNeededDependencyTag tells whether a dependency is required to
  be installed together.
- PackagingItem tells whether a dependency (of PackagingBase) is
  required to be packaged.

With the separation of InstallNeededDependencyTag and PackagingItem,
PackagingBase module can distinguish cases which were not available
before.(I = InstallNeededDependencyTag, P = PackagingItem)

   a (PackagingBase module)
   |
   |`--(I)--> b
   |
   |`--(P)--> c --(I)--> d
   |
    `--(I/P)--> e

a's CopyDepsToZip(): [c, d, e]

Test: m nothing (packaging_test)
Change-Id: I71fce29b19b0f00dc394981bcf4240e9c1041c7a
2021-03-17 11:54:53 +09:00
Jiyong Park
972e06c41f Add vbmeta module type
The module type is to create vbmeta image out of other partitions.

Bug: 180676957
Test: m microdroid_vbmeta microdroid_vbmeta_system
Inspect the built image using `avbtool info_image --image <image>`

Change-Id: Iac92e9ab1640dcd488af69842e09850a91262bf1
2021-03-16 11:34:11 +09:00
Jiyong Park
ac4076de9d bootimg signs image using verity_utils
Previously, bootimg signed the image using avbtool. This didn't work
because avbtool always requires that the partition size is given via
'--partition_size' parameter. The partition size is hard to estimate
especially when the image is not for a real physical partition, but for
a logical partition in a composite image.

With this change, the signing of bootimg is done by verity_utils.py
which internally uses avbtool. The python script is capable of
calculating the minimum required partition size when the partition size
is not given.

In addition, this change adds 'partition_name' property to the
`android_filesystem` module type so that we can customize the partition
name field in the vbmeta descriptor.

Bug: 180676957
Test: m microdroid-boot-5.10
Change-Id: I2e4aa626cf06a2177b4a8d90ff9b9006d2927ae4
2021-03-16 00:34:57 +09:00
Jiyong Park
ca5e611e0a Merge "bootimg supports v4 header" 2021-03-15 14:39:07 +00:00
Jooyung Han
d0b3574072 Merge "Fix bootimg.cmdline to use ShellEscapeIncludingSpaces" 2021-03-12 09:10:22 +00:00
Jooyung Han
32cddd0ba5 Fix bootimg.cmdline to use ShellEscapeIncludingSpaces
bootimg.cmdline should be passed to mkbootimg's --cmdline flag as a
single argument. Because cmdline can have spaces we should escape it
with ShellEscapeIncludingSpaces rather than blindly putting double
qoutes with ShellEscape, which also puts single quotes when necessary.

Bug: 182092664
Test: m nothing && see build.ninja (microdroid's build commands)
Test: boot microdroid on VIM3L
Change-Id: I5d766bb0a816d829569c9ebfddf97c4f846f78f8
2021-03-12 14:38:46 +09:00
Jiyong Park
b0eb3192c5 bootimg and logical_partition are OutputFileProducer
The two module types now implement OutputFileProducer so that we can add
them to the data dependencies of tests.

Bug: 181860941
Test: atest MicrodroidHostTestCase
Change-Id: If263fefb1e5cdb5b57c17eb389c6ecc11d8356f4
2021-03-09 20:31:51 +09:00
Jiyong Park
81aea9a0f2 bootimg supports v4 header
It's the latest version of android boot image header which supports
bootconfig. Bootconfig parameters are now passed via `bootconfig`
property.

Bug: 181936566
Test: m microdroid_boot

Change-Id: Iff8697434f7502fe56fca5bce5573e53f2f6ac60
2021-03-08 09:12:17 +09:00
Jiyong Park
b89e5e71b7 Add logical_partition module type
logical_partition builds a partition image (which is usually called
'super.img') out of one or more filesystem images.

Bug: 180921702
Test: m microdroid_super
Change-Id: I659607647e3a5bc82c82b576a049e6c6f91cbddb
2021-02-24 12:53:39 +09:00
Treehugger Robot
05785f0023 Merge "bootimg supports building (non-vendor) boot.img" 2021-02-22 23:53:15 +00:00
Treehugger Robot
277303f042 Merge "Add ctx to AndroidMkExtraEntriesFunc" 2021-02-22 22:40:31 +00:00
Colin Cross
aa2555387d Add ctx to AndroidMkExtraEntriesFunc
Add a ctx parameter to AndroidMkExtraEntriesFunc to allow them to
access providers.

Test: m checkbuild
Change-Id: Id6becc1e425c3c3d8519248f8c0ce80777fac7cc
Merged-In: Id6becc1e425c3c3d8519248f8c0ce80777fac7cc
2021-02-19 23:05:40 +00:00
Jiyong Park
4bbd6cfcaf bootimg supports building (non-vendor) boot.img
Bug: 178562516
Test: m microdroid_boot-5.10
Change-Id: I120fc954eb00362bdd2c02e3f3ef147670c69d8f
2021-02-19 18:26:32 +09:00
Inseob Kim
abf3f3c0ef Merge changes from topics "deps_base_dir_filesystem", "filesystem_rootdir"
* changes:
  Add dirs and symlinks property to filesystem
  Add base_dir property to filesystem
2021-02-18 00:19:48 +00:00
Inseob Kim
14199b07f7 Add dirs and symlinks property to filesystem
Dirs and symlinks will be created under the root of the filesystem.
Basic essential directories like "dev", "proc", "sys" and symlinks like
"bin -> /system/bin", "init -> /system/bin/init" can be created with
these properties.

Bug: 179652970
Test: boot with aosp_cf_x86_64_only_phone, see adb works
Change-Id: Ie06dc5a93635ea8b1e18be517ed8615b6c82fee6
2021-02-17 14:05:12 +09:00
Inseob Kim
2ce1b5dc3a Add base_dir property to filesystem
Deps have been installed to "system/" because of hard-coded mount point
"system". Now they are installed to base_dir, and mount point is set to
root.

Bug: 179652970
Test: see contents of microdroid.img
Change-Id: Ie03b539a1688db7002bb178823b39017a83ce840
2021-02-17 13:07:18 +09:00
Jiyong Park
1f55dbd0d6 boot_image modules inside APEX have correct names
This change fixed the problem that boot_image modules when installed to
an APEX get incorrect names like signed.img. The filename now is
"<modulename>.img" and can be overridden via the new `stem` property.

Bug: 178978059
Test: m
Change-Id: I1b25db04a4a2d888371b174c42f91a0cea87b877
2021-02-17 09:45:03 +09:00
Treehugger Robot
b68036f44b Merge "Add LOCAL_LICENSE_KINDS to build/soong" 2021-02-10 07:18:29 +00:00
Bob Badour
02040de891 Add LOCAL_LICENSE_KINDS to build/soong
Added SPDX-license-identifier-Apache-2.0 to:
  Android.bp
  android/Android.bp
  android/soongconfig/Android.bp
  androidmk/Android.bp
  apex/Android.bp
  bazel/Android.bp
  bp2build/Android.bp
  bpf/Android.bp
  bpfix/Android.bp
  cc/Android.bp
  cc/config/Android.bp
  cc/libbuildversion/Android.bp
  cc/libbuildversion/tests/Android.bp
  cc/ndk_api_coverage_parser/Android.bp
  cc/ndkstubgen/Android.bp
  cc/symbolfile/Android.bp
  cmd/dep_fixer/Android.bp
  cmd/diff_target_files/Android.bp
  cmd/extract_apks/Android.bp
  cmd/extract_jar_packages/Android.bp
  cmd/extract_linker/Android.bp
  cmd/fileslist/Android.bp
  cmd/host_bionic_inject/Android.bp
  cmd/javac_wrapper/Android.bp
  cmd/merge_zips/Android.bp
  cmd/multiproduct_kati/Android.bp
  cmd/path_interposer/Android.bp
  cmd/pom2bp/Android.bp
  cmd/pom2mk/Android.bp
  cmd/sbox/Android.bp
  cmd/soong_build/Android.bp
  cmd/soong_env/Android.bp
  cmd/soong_ui/Android.bp
  cmd/zip2zip/Android.bp
  cmd/zipsync/Android.bp
  cuj/Android.bp
  dexpreopt/Android.bp
  dexpreopt/dexpreopt_gen/Android.bp
  env/Android.bp
  etc/Android.bp
  filesystem/Android.bp
  finder/Android.bp
  finder/cmd/Android.bp
  genrule/Android.bp
  jar/Android.bp
  java/Android.bp
  java/config/Android.bp
  kernel/Android.bp
  linkerconfig/Android.bp
  linkerconfig/proto/Android.bp
  makedeps/Android.bp
  partner/Android.bp
  phony/Android.bp
  python/Android.bp
  python/tests/Android.bp
  remoteexec/Android.bp
  rust/Android.bp
  rust/config/Android.bp
  scripts/Android.bp
  sdk/Android.bp
  sh/Android.bp
  shared/Android.bp
  symbol_inject/Android.bp
  symbol_inject/cmd/Android.bp
  sysprop/Android.bp
  tradefed/Android.bp
  ui/build/Android.bp
  ui/logger/Android.bp
  ui/metrics/Android.bp
  ui/metrics/proc/Android.bp
  ui/status/Android.bp
  ui/terminal/Android.bp
  ui/tracer/Android.bp
  xml/Android.bp
  zip/Android.bp
  zip/cmd/Android.bp

Added SPDX-license-identifier-Apache-2.0 SPDX-license-identifier-BSD to:
  finder/fs/Android.bp
  third_party/zip/Android.bp

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all

Exempt-From-Owner-Approval: janitorial work

Change-Id: Ia47ca14f16b8c9f84f9d533a07e5b00e2c04e8d4
2021-02-06 04:23:21 +00:00
Treehugger Robot
b3f9025b6e Merge "Support uncompressed cpio" 2021-02-05 15:57:22 +00:00
Treehugger Robot
364da36115 Merge changes I6a5357e9,I199f070e
* changes:
  Install ko to /lib/modules
  android_filesystem is OutputFileProducer
2021-02-05 03:39:51 +00:00
Jiyong Park
837cdb2212 Support uncompressed cpio
Bug: 178978059
Test: atest VirtualizationHostTestCases
Change-Id: I1f69af40f10d14ec2f3de6111d12b3a9c5d6c75c
2021-02-05 00:29:47 +09:00
Jiyong Park
940dfd4db2 android_filesystem is OutputFileProducer
... to be able to reference the module in data property of test modules.

Bug: N/A
Test: m VirtualizationHostTestCases
Change-Id: I199f070e811011cea6189ef24c0b2d8a683f79d0
2021-02-05 00:29:47 +09:00
Inseob Kim
cc8e536a15 Add file_contexts property to filesystem
Filesystems like ext4 can store file contexts itself. This supports
passing file_contexts file to build_image.

Bug: 178993690
Test: boot and see selinux denials are gone
Change-Id: I97d4a981e4b9c89434ea2f1303173ae91cce94e3
2021-02-04 13:28:20 +09:00
Jiyong Park
1f7b93e976 Add bootimg module type
The new module type is to build the boot partition images. Boot
partition image consists of kernel, ramdisk, dtb, etc. which are
processed by the mkbootimg tool. There are two flavors of boot
partition: boot and vendor_boot. The choice is configured via the
'vendor_boot' property. Currently, only `vendor_boot: true` case is
supported.

But: 178978059
Test: m cf_bootimg-5.10
Change-Id: Iee689d9d666e8e6e5af7db02a76dd39d1e4d0e3a
2021-02-02 00:41:57 +09:00
Jiyong Park
11a65979d8 android_filesystem supports compressed cpio format
The format is used to create ramdisk image. Building ramdisk image in
Soong is required to have vendor_boot.img in the virt APEX.
vendor_boot.img consists of ramdisk and dtb.

Bug: 178980227
Test: m
Change-Id: Ie4e90cef8407b6e4bdf7f03f93724cdc3cd45c20
2021-02-02 00:41:57 +09:00
Jiyong Park
3db465ddd7 Don't use "echo -e" when creating prop file
We don't need to escape backslashes when generating the prop file. In
addition, the "-e" option doesn't seem to be supported in some build
environments.

Bug: 178443594
Test: watch presubmit tests
Change-Id: I167b25255a68b62a75b433f31c5e7c9d57f2579d
2021-01-26 14:08:16 +09:00
Jiyong Park
71baa7690a Sign android_filesystem with avbtool
Use_avb and other avb_* properties allows us to sign an
android_filesystem module with avbtool.

Bug: 172415113
Test: m

Change-Id: Ifa1ed8ded1b10170aaca9b34e6a14f0179dbab5d
2021-01-20 08:39:54 +09:00
Jiyong Park
7267831086 Refactor filesystem.go to make it easy to build prop file
The creation of the prop file is refactored to a function.

Bug: 172415113
Test: m

Change-Id: I969bf4a2476f7a4aa9571945d3645d7af52ff09a
2021-01-20 08:35:52 +09:00
Jiyong Park
12a719c0fc android_filesystem modules can be included in APEX
android_filesystem modules can be included in APEX via the new
`filesystems` property. The filesystem images are placed at
./etc/fs/<modulename>.img.

Bug: 172413888
Test: m nothing

Change-Id: I215ca7a32ff1988a0de4e1f71397684e189839ea
2021-01-07 15:35:25 +09:00
Colin Cross
f1a035e6be Pass pctx and ctx to NewRuleBuilder
Enable the RuleBuilder and RuleBuilderCommand methods to access
the BuilderContext by passing it to NewRuleBuilder instead of
RuleBuilder.Build.

Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I63e6597e19167393876dc2259d6f521363b7dabc
2020-12-01 16:22:16 -08:00
Jiyong Park
65c49f5979 android_filesystem is installable
Also document about the android_filesystem module type.

Bug: 172414391
Test: m
Change-Id: Iae07af9dd22f34c8e0e4c833333bafdf82dcab2a
2020-12-01 23:47:39 +09:00
Jiyong Park
65b6224471 Clients of PackagingBase can customize dependency tag to use
Previously, the dep tag used by PackagingBase was fixed, which prevented
some of its clients (e.g. cvd-host-package) from opting in to
android.InstallAlwaysNeededDependencyTag. Now, PackagingBase.AddDeps
accepts the dependency tag to use.

Also, dependencies toward rust dylib, rlib, and proc_macro are
configured to return true on InstallDepNeeded. This is required to
install shared_lib dependencies of the rust modules when they are
depended on by a rust binary.

Exempt-From-Owner-Approval: a trivial change after +2 from the owner.
This has to land ASAP as many users are affected by acloud being
unavailable.

Bug: N/A
Test: m
Test: acloud create --local-instance --local-image
Change-Id: If22aee7c6f314fcb03b9d4fe6901a2557f1e899c
2020-12-01 03:43:05 +00:00
Jiyong Park
6f0f688c4b add android_filesystem
android_filesystem is a module type that can be used to create filesystem
images out of binaries built for Android. Its initial use will be for
creating an Android-like OS image to run on virtual machines, but the
use is not limited to it.

The module type currently lacks a lot of features like the support for
other filesystem types, and the ability to do something like signing the
image using avbtool, etc. Those will be added in follow-up CLs as we
have actual demands.

Bug: 172414391
Test: m
Change-Id: I4b779d4586e04d9a960688e73b711166708558ab
2020-11-19 08:58:06 +09:00