We want to remove target specific host tools and since
fs_config_generate is compiled with a target specific header file, we
instead remove fs_config_generate entirely and allow python to build
the fs_config_files/dirs files directly from config.fs files and
parsed C headers.
Test: associated unit tests and new end to end test
Test: aosp_sailfish, aosp_crosshatch build produces valid fs_config files
Test: aosp_cf_x86_phone build correctly produces empty fs_config files
Change-Id: Idbc63ff56c0979e1e4c17721371de9d9d02dc8ff
This was deprecated along with TARGET_ANDROID_FILESYSTEM_CONFIG_H so
warn if it still exists.
Test: successful error if it exists, no error if it does not
Change-Id: I7dde3c88aaf7ecec23ffe018cbe8b42ebadb2bf8
TARGET_FS_CONFIG_GEN has existed as the preferred mechanism for two
releases, so we finally deprecate TARGET_ANDROID_FILESYSTEM_CONFIG_H.
Test: build
Change-Id: I299a4d1d1d893ac16d6e6ce2ec4659bfcdc19095
This prints the uid map in a very simple format, with
AID_CONSTANT<space>uid
on each line. This is super easy for other tools to
parse, and generate their own mappings, without requiring
edits to fs_config_generator.py itself.
Test: make, treehugger
Change-Id: I10e24ac29d440a24d43580880343d122ae1cdf02
The code to manage citing empty <partition>/etc/fs_config_{files|dirs}
content by specifically setting the file(s) to access mode 00000 is
bitrotted and ill-suited for multiple partitions and was written for
a more paranoid time when DAC and MAC rules were not nearly as mature.
Striking out the complications as it affected deterministic results.
Test: compile and boot
Bug: 119310326
Change-Id: Iec02e0c1194eaa799ab95adff4ff0951a24652a4
Mechanical extension of mount directories to also support product/
and product_services/ as individually supplied images.
NB: In the distant past, a prototype placing the make rules into a
macro was rejected as too difficult to maintain within the macro
rules, so rules are rolled-out to make them easy to understand.
Test: compile
Bug: 119310326
Change-Id: Ief137cbe62db7175ee02275804ff1d644858dc3d
Pages in .data.rel.ro are always dirty. Move whatever we can to
.rodata so that we reduce memory pressure.
The size of rodata, text, and data.rel.ro sections of libc on cuttlefish
before/after this change:
rodata: 0xfd70 -> 0x101d0 (+1120 bytes)
text: 0xb5715 -> 0xb56e5 (-48 bytes)
data.rel.ro: 0x57e0 -> 0x5230 (-1456 bytes)
While this change alone doesn't reduce the number of dirty pages
from data.rel.ro on cuttlefish, it is a step in that direction.
Test: Build and boot cuttlefish
Change-Id: Iff8203940495109dd01c40f31b034cfb2882e7f4
Content from fs_config designated for other partitions is leaking
into the system fs_config_* files. Notably /vendor is showing up.
Rule expansion happens late, after all parsing. This is the same in
make and kati. build/make/tools/fs_config/Android.mk ends with
fs_config_generate_extra_partition_list :=, so by the time you get to
rule expansion the value is empty.
Added a PRIVATE_PARTITION_LIST variable to record the value of
fs_config_generate_extra_partition-list.
Test: inspect result of ${OUT}/product/*/system/etc/fs_config_files
Bug: 119677224
Bug: 119310326
Change-Id: I968b4936ab2c5d174f164c760820af13434654ca
This was already done for partitions other than the system partition.
Add new, separate "_system" targets, and add new phony modules which
collects _system and _nonsystem targets for now.
Update the base_system.mk file to only depend on the system module, and
put the other ones in base_vendor.mk for now.
Bug: 80410283
Test: make fs_config_dirs[_*] fs_config_files[_*] and verify output
Change-Id: I7843578dbb9f27edf4c1e5b4a795a530f79e0be1
Previously we only created these if the device set
TARGET_FS_CONFIG_GEN, however there are now other targets that want to
depend on these. Instead of having those targets conditionally depend
on them, we always create them, defaulting to blank contents (by
reading /dev/null for TARGET_FS_CONFIG_GEN).
Test: builds succeed
Change-Id: Ie95286f5a800d891022eb66cd6fefcc967000c2e
The man page for the password file (man 5 passwd) does not mention any
provisions for comment line other than comment field
Bug: 109748241
Test: Build and check generated file
Change-Id: I2912d0844cf3f077184129d1a0121b50b09ef1e3
There is an internal buffer in bionic for user/group names that is 32
characters long including the trailing null, so we must restrict the
length of these names during compile time.
Bug: 27999086
Test: Successfully compile a valid config.fs
Test: Fail to compile a config.fs with AID name > 31 characters
Change-Id: I7fe887c630dd4d1033b86a5d8332480eb3b0fa07
These entries are vendor provided and belong on the /vendor partition.
Bug: 27999086
Test: end to end user/group check via config.fs and getpwnam, etc.
Change-Id: I9a5d56da594bf0d04de2b9ce7fd7d9a8151d4682
This change adds a flag so that the mksquashfs wrapper scripts generate
an entry that is fs_config-friendly for the root inode. Namely, the root
entry is expected to have an empty filename.
Bug: 72745016
Test: m -j100 dist
Change-Id: Iebdf79c5af0b9d999b7e5f5fe240abfe52cbadda
Clear fs_config_generate_extra_partition_list after use.
Test: build fs_config_files and fs_config_dirs targets.
Change-Id: Ie18bcd2df2ff1becb178e9ba45d226e2e9a8faab
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Misspelled fs_config_generate_extra_partition_list when used to
calculate LOCAL_REQUIRED_MODULES dependencies. This resulted in
fs_config_dirs and fs_config_files failing to generate the vendor,
oem or odm variants (ie: fs_config_files_vendor)
Test: build generates correct fs_config override files.
Bug: 37721445
Bug: 36071012
Change-Id: I053dd5cdb357c7cea05a5ff63b0686bd0bc99df6
Add build targets to generate the vendor, oem and odm slices of the
fs_config_dirs and fs_config_files targets for each of the respective
filesystems, and automatically trigger the individual and enabled
partition build objects based as dependents of fs_config_dirs and
fs_config_files.
Sensitive to the following board definition pairs:
BOARD_VENDORIMAGE_FILE_SYSTEM_TYPE BOARD_USES_VENDORIMAGE
BOARD_OEMIMAGE_FILE_SYSTEM_TYPE BOARD_USES_OEMIMAGE
BOARD_ODMIMAGE_FILE_SYSTEM_TYPE BOARD_USES_ODMIMAGE
Which when defined and set means the images are generated for vendor,
oem or odm respectively.
Test: manually build w/ m 'fs_config_(dirs|files)_(vendor|oem|odm)'
and check the expected products, or fails if not pertinent.
Test: build for product that produces split to system and vendor and
subsequently boots with vendor products correctly configured.
Bug: 36071012
Change-Id: I44289a3dd50b7657367c826ad813c671a3de6afc
Add a -P option to fs_config_generate that allows us to filter on
a comma separated partition whitelist or minus prefixed -blacklist.
A partition is defined as accessible as <partition>/ or
system/<partition>/.
No change for legacy, where no -P flag is specified.
For system.img, but not vendor, oem or odm file references:
fs_config_generate -P -vendor,-oem,-odm
For vendor.img file references:
fs_config_generate -P vendor
For oem.img file references:
fs_config_generate -P oem
For odm.img file references:
fs_config_generate -P odm
Add and fortify tests for each of these cases.
SideEffects: none
Test: gTest host fs_config-unit-test
Bug: 36071012
Change-Id: I0537453bb289d41ef98ad76e69f78ae49358a1b3
Fix pylint warning:
W:292,47: Cell variable lineno defined in loop (cell-var-from-loop)
For the closure function that prints the error message, we actually
want the value of modified lineno within the closure.
Test: run unit tests
Change-Id: I7d08184563d347c826ca1af7558fb8e378f151e1
Signed-off-by: William Roberts <william.c.roberts@intel.com>
A few changes got introduced that yapf re-styles. To make it easy,
keep this file formated to:
yapf -i --style=google fs_config_generator.py
Test: run unit tests
Change-Id: I3160b92d7fd07a4d315dde59f1d20fe4c7587da1
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Addresses the following build error:
"value" not in valid range [], got: 2901 for: "AID_ABC" file:\
"device/somc/abc/config.fs"
that happens when parsing device/somc/abc/config.fs:
[AID_ABC]
value: 2901
The problem is that the AID.PREFIX is 'AID_' so the _OEM_RANGE
expression is searching for AID__OEM_RESERVED, which never
exists. Drop the extra '_'.
Change-Id: I8a1556731e14e49cef66e3e91121e4be23a308e0
Test: ./build/tools/fs_config/fs_config_generator.py fsconfig --aid-header=system/core/include/private/android_filesystem_config.h device/somc/abc/config.fs
This reverts commit fad4b4b715.
Incorporating the following fixes:
1.
fsconfig: fix fs_config_* build for discovered headers
When android_file system_config.h is picked up from the device
directory, neither TARGET_FS_CONFIG_GEN or TARGET_ANDROID_FILESYSTEM_CONFIG_H
are specified. Thus, the build is not generating the required fs_config_files
and fs_config_dirs.
Test: Ensure that make fs_config_files works, and produces the same output as before
Build the system image and mount it as a loop back and dump the file system
capabilities with getcap. Verify that output to the supplied
android_file system_config.h
From the loopback of the system.img mount, from CWD system/bin:
$ getcap *
cnss-daemon = cap_net_bind_service+ep
hostapd = cap_net_admin,cap_net_raw+ep
imsdatadaemon = cap_net_bind_service+ep
ims_rtp_daemon = cap_net_bind_service+ep
logd = cap_setgid,cap_audit_control,cap_syslog+ep
mm-qcamera-daemon = cap_sys_nice+ep
pm-service = cap_net_bind_service+ep
run-as = cap_setgid,cap_setuid+ep
surfaceflinger = cap_sys_nice+ep
webview_zygote32 = cap_setgid,cap_setuid,cap_setpcap+ep
webview_zygote64 = cap_setgid,cap_setuid,cap_setpcap+ep
Compared to the android_filesystem_config.h:
{ 00700, AID_CAMERA, AID_SHELL, (1ULL << CAP_SYS_NICE), "system/bin/mm-qcamera-daemon" },
{ 00755, AID_SYSTEM, AID_SYSTEM, (1ULL << CAP_NET_BIND_SERVICE), "system/bin/pm-service" },
{ 00755, AID_SYSTEM, AID_SYSTEM, (1ULL << CAP_NET_BIND_SERVICE), "system/bin/imsdatadaemon" },
{ 00755, AID_SYSTEM, AID_RADIO, (1ULL << CAP_NET_BIND_SERVICE), "system/bin/ims_rtp_daemon" },
{ 00755, AID_SYSTEM, AID_SYSTEM, (1ULL << CAP_NET_BIND_SERVICE), "system/bin/cnss-daemon"},
2.
fsconfig: fix error message for duplicate AID
Fixes:
raise ValueError('Duplicate aid value "%u" for %s' % value,
TypeError: %u format: a number is required, not str
and
raise ValueError('Duplicate aid value "%s" for %s' % value,
TypeError: not enough arguments for format string
3.
fsconfig: add test for duplicate ranges
Add a test for duplicate range detection.
4.
fsconfig: skip AID_APP, AID_USER and all ranges
Do not output AID_APP, AID_USER and ranges. A range
is defined as ending with AID_ and ending in _START or
_END.
5.
fsconfig: test for skip AID_APP, AID_USER and all ranges
Test against AIDs that caused the bionic tests to fail.
Change-Id: I95569a9ccc83bd3231f8a6f395532cc2de316bd2
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Add some initial unit tests.
The unit tests themselves are inlined into the tool and
can be executed by running the "test" commandlet.
Example:
$ python -m unittest test_fs_config_generator.Tests
.............
----------------------------------------------------------------------
Ran 13 tests in 0.004s
OK
Test: run the test commandlet and observe for failures.
Change-Id: I1bada385fa841fd50fa958997d440f1198e15198
Signed-off-by: William Roberts <william.c.roberts@intel.com>
When configuring fs_config_files or fs_config_dirs for file_system
capabilities, drop the requirement that OEMs must add the target
to PRODUCT_PACKAGES. This limits the configuration requirement
to only needing to set the new and preferred TARGET_FS_CONFIG_GEN
or the older TARGET_ANDROID_FILESYSTEM_CONFIG_H method.
Test: That only setting TARGET_FS_CONFIG_GEN results in passwd and
group in the build image.
Change-Id: I818854fa1b3e94edaff59a32bd7cf23cf9b504aa
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Introduce a generator that outputs group files per man(5) group.
Succinctly, the output is a colon delimited string containing the following
fields:
* group name
* encrypted password (optional)
* gid (int)
* userlist (str,...)
Multiple colon delimited lines may exist, but will not be separated
across lines.
Sample generator output:
foo::2900:
foo_bar::2901:
custom_oem1::2902:
Test: That make group produces the group file.
Change-Id: Idd3fe925a09a227c6e894e1b5d2b3873b01531c6
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Introduce a generator that outputs passwd files per man(5) passwd.
Succinctly, the output is a colon delimited string containing the following
fields:
* login name
* encrypted password (optional)
* uid (int)
* gid (int)
* User name or comment field
* home directory
* interpreter (optional)
Multiple colon delimited lines may exist, but will not be separated
across lines.
When run, produces:
foo::2900:2900::/:/system/bin/sh
foo_bar::2901:2901::/:/system/bin/sh
custom_oem1::2902:2902::/:/system/bin/sh
Note that this generator allows for 0 or more config.fs files. This allows for:
* Unconditional inclusion of /system/etc/passwd in the generated image
* A blank passwd file if no config.fs files are specified.
This ensures that when OEMs add config.fs files, there is no additional steps
for proper functionality (simpler for OEMs).
The one draw back is the additional inode consumption on system for a possible
blank file.
Test: That it produces a valid passwd file.
Change-Id: I19691c8260f02147ed861f8a319aeab3f5b1738e
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Generate the friendly name in one location. This prepares
the tool for generatting passwd and group files.
Also support mapping friendly names to identifiers.
Test: That output files stay the same as before.
Change-Id: I12198611126613eae81ca61614ed269c2439b72b
Signed-off-by: William Roberts <william.c.roberts@intel.com>
When generating files restrict the characters allowed in
AID_<name> to upercase, numbers, and underscores.
This detects errors ahead of time for generated C files as
well as handles seperation characters for passwd/group files.
This also lends itself to automatic detection of collisions
on friendly names, since freindly names are the lowercase
version of <name>.
Test: That invalid aid values result in a build failure.
Test: That the output files are consistent with ones before
this change (hash and diff checks)
Change-Id: Ie8ec44c1157ba9c22100e9169d9187f615e71280
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Generate an OEM AID_<name> header file seperate from fs_config
header file and provide details on how to export this interface
into native code.
Test: That ls, ps, chown and services function for built in
services as before.
Change-Id: Ie8ce6585e0721b52633ee50d62dcfe796e178f65
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Implement an android_id generator that takes the data
acquired from parsing private/android_filesystem_config.h
and generates the android_id friendly name to uid mapping
for consumption in Bionic.
Test: That ls, ps, mkdir, chown, chgrp and services for built
in names work.
Change-Id: I1e55a401be0fca0ad162f8dc1e072e6afde7b927
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Rather than hardcode the OEM ranges, parse and extract
AID values from android_filesystem_config.h.
An AID is defined to the tool as:
* #define AID_<name>
An OEM Range is defined to the the tool as:
* AID_OEM_RESERVED_START
* AID_OEM_RESERVED_END
or
* AID_OEM_RESERVED_N_START
* AID_OEM_RESERVED_N_END
Where N is a number.
While parsing, perform sanity checks such as:
1. AIDs defined in the header cannot be within OEM range
2. OEM Ranges must be valid:
* Cannot overlap one another.
* Range START must be less than range END
3. Like the C preproccessor, multiple matching AID_<name> throws
en error.
The parser introduced here, prepares the tool to output android_ids
consumable for bionic.
Note that some AID_* friendly names were not consistent, thus a small
fixup map had to be placed inside the tool.
Test: tested parsing and dumping the data from android_filesystem_config.h
file.
Change-Id: Ifa4d1c9565d061b60542296fe33c8eba31649e62
Signed-off-by: William Roberts <william.c.roberts@intel.com>
This internally structures fs_config_generator.py to be able
to plug in generators to produce different outputs. This
prepares this tool for group and pwd file outputs.
Test: Checked diff and hash of before and after files.
Change-Id: Ie558518ac227dd946d70ab48027698b72a9bc94a
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Test: Checked diff and hash of before and after output files, same.
Test: Tested build for basic boot and functionality.
Change-Id: If7806427e3a2a9ddb7a2c9aa14e1e4f9bf696acf
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Testing on earlier versions of Python indicated an error in the
README. Python's ConfigParser prior to version 3.2 allows for
sections within the same file to override later sections.
Correct the README.
Test: Checked diff and hash of output file from before and after.
Change-Id: Ifd3f78414b4803e4b74decb1fdf6816d4902c4a2
Signed-off-by: William Roberts <william.c.roberts@intel.com>
The first line returned by awk is empty. In consequence, when
fs_config computes its mode it will consider this line as a file (no
trailing slash) and affect the default android_files mode which is
0644. The mode for the root directory should be the default
android_dirs mode 0755.
Add a special case in fs_config to consider empty line as a directory.
Change-Id: I9f33f6fcf4be05c31914db898e65c92b0a611518
Signed-off-by: Gaelle Nassiet <gaellex.nassiet@intel.com>
Support both OEM and OEM2 ranges as defined by
private/android_filesystem_config.h.
Change-Id: I1aef62fa446301bb68b0309b198f3fe8b5eb92c7
Signed-off-by: William Roberts <william.c.roberts@intel.com>
