Commit graph

31 commits

Author SHA1 Message Date
Dan Albert
a9e914dd2f Make multibyte result constants public.
The magic numbers that C defines are obnoxious. We had partial
definitions for these internally. Add the missing one and move them to
a public header for anyone else that may want to use them.

Bug: None
Test: None
Change-Id: Ia6b8cff4310bcccb23078c52216528db668ac966
2023-08-04 19:48:35 +00:00
Elliott Hughes
2bd4316bd6 Expose tzalloc()/localtime_rz()/mktime_z()/tzfree().
* Rationale

The question often comes up of how to use multiple time zones in C code.
If you're single-threaded, you can just use setenv() to manipulate $TZ.
toybox does this, for example. But that's not thread-safe in two
distinct ways: firstly, getenv() is not thread-safe with respect to
modifications to the environment (and between the way putenv() is
specified and the existence of environ, it's not obvious how to fully
fix that), and secondly the _caller_ needs to ensure that no other
threads are using tzset() or any function that behaves "as if" tzset()
was called (which is neither easy to determine nor easy to ensure).

This isn't a bigger problem because most of the time the right answer
is to stop pretending that libc is at all suitable for any i18n, and
switch to icu4c instead. (The NDK icu4c headers do not include ucal_*,
so this is not a realistic option for most applications.)

But what if you're somewhere in between? Like the rust chrono library,
for example? What then?

Currently their "least worst" option is to reinvent the entire wheel and
read our tzdata files. Which isn't a great solution for anyone, for
obvious maintainability reasons.

So it's probably time we broke the catch-22 here and joined NetBSD in
offering a less broken API than standard C has for the last 40 years.
Sure, any would-be caller will have to have a separate "is this
Android?" and even "is this API level >= 35?" path, but that will fix
itself sometime in the 2030s when developers can just assume "yes, it
is", whereas if we keep putting off exposing anything, this problem
never gets solved.

(No-one's bothered to try to implement the std::chrono::time_zone
functionality in libc++ yet, but they'll face a similar problem if/when
they do.)

* Implementation

The good news is that tzcode already implements these functions, so
there's relatively little here.

I've chosen not to expose `struct state` because `struct __timezone_t`
makes for clearer error messages, given that compiler diagnostics will
show the underlying type name (`struct __timezone_t*`) rather than the
typedef name (`timezone_t`) that's used in calling code.

I've moved us over to FreeBSD's wcsftime() rather than keep the OpenBSD
one building --- I've long wanted to only have one implementation here,
and FreeBSD is already doing the "convert back and forth, calling the
non-wide function in the middle" dance that I'd hoped to get round to
doing myself someday. This should mean that our strftime() and
wcsftime() behaviors can't easily diverge in future, plus macOS/iOS are
mostly FreeBSD, so any bugs will likely be interoperable with the other
major mobile operating system, so there's something nice for everyone
there!

The FreeBSD wcsftime() implementation includes a wcsftime_l()
implementation, so that's one stub we can remove. The flip side of that
is that it uses mbsrtowcs_l() and wcsrtombs_l() which we didn't
previously have. So expose those as aliases of mbsrtowcs() and
wcsrtombs().

Bug: https://github.com/chronotope/chrono/issues/499
Test: treehugger
Change-Id: Iee1b9d763ead15eef3d2c33666b3403b68940c3c
2023-06-16 08:10:47 -07:00
Elliott Hughes
2c96639eb2 Optimize the mbs fast path slightly.
From a logcat profile:
```
     |--95.06%-- convertPrintable(char*, char const*, unsigned long)
     |    |--13.95%-- [hit in function]
     |    |
     |    |--35.96%-- mbrtoc32
     |    |    |--82.72%-- [hit in function]
     |    |    |
     |    |    |--11.07%-- mbsinit
     |    |    |
     |    |    |--5.96%-- @plt
```
I think we'd assumed that mbsinit() would be inlined, but since these
functions aren't all in wchar.cpp it wasn't being. This change moves the
implementation into a (more clearly named) inline function so we can
trivially reclaim that 11%+6%.

Benchmarks before:
```
-------------------------------------------------------------------
Benchmark                         Time             CPU   Iterations
-------------------------------------------------------------------
BM_stdlib_mbrtowc_1            8.03 ns         7.95 ns     87144997
BM_stdlib_mbrtowc_2            22.0 ns         21.8 ns     32002437
BM_stdlib_mbrtowc_3            30.0 ns         29.7 ns     23517699
BM_stdlib_mbrtowc_4            37.4 ns         37.1 ns     18895204
BM_stdlib_mbstowcs_ascii     792373 ns       782484 ns          890 bytes_per_second=609.389M/s
BM_stdlib_mbstowcs_wide    15836785 ns     15678316 ns           44 bytes_per_second=30.4138M/s
```

Benchmarks after:
```
-------------------------------------------------------------------
Benchmark                         Time             CPU   Iterations
-------------------------------------------------------------------
BM_stdlib_mbrtowc_1            5.76 ns         5.72 ns    121863813
BM_stdlib_mbrtowc_2            17.1 ns         16.9 ns     41487260
BM_stdlib_mbrtowc_3            24.2 ns         24.0 ns     29141629
BM_stdlib_mbrtowc_4            30.3 ns         30.1 ns     23229291
BM_stdlib_mbstowcs_ascii     783506 ns       775389 ns          903 bytes_per_second=614.965M/s
BM_stdlib_mbstowcs_wide    12787003 ns     12672642 ns           55 bytes_per_second=37.6273M/s
```

Bug: http://b/206523398
Test: treehugger
Change-Id: If8c6c39880096ddd2cbd323c68dca82e9849ace6
2021-11-16 11:03:19 -08:00
Yi Kong
32bc0fcf69 Modernize codebase by replacing NULL with nullptr
Fixes -Wzero-as-null-pointer-constant warning.

Test: m
Bug: 68236239
Change-Id: I5b4123bc6709641315120a191e36cc57541349b2
2018-08-02 18:09:44 -07:00
Elliott Hughes
697f42afdb Hide various mbstate implementation details.
...by inlining them.

Also fix a couple of harmless bugs in passing. I've added tests, but in
both cases I don't think it was actually possible to hit the bad behavior:
we'd hit another test and fail immediately after in an externally
indistinguishable way.

Bug: N/A
Test: readelf
Change-Id: I8466050b0bfe2b7b94c76b383cf10c1d9d28debd
2017-07-14 17:00:05 -07:00
Dan Albert
3f03579589 Move wchar _l functions out of wchar.cpp.
We don't need these in libandroid_support, but we do need the other
parts of wchar.cpp, and they're not really related.

Test: make checkbuild
Bug: None
Change-Id: I40f3089b034abfd4873e81c0b6216a7cfd977d8d
2017-07-13 10:29:20 -07:00
Elliott Hughes
53de874c3c The default locale "" should be a UTF-8 locale.
"ls -q" (or "adb shell -tt ls") was mangling non-ASCII because mbrtowc
was returning multibyte characters as their individual bytes. This was
because toybox asks for "" rather than "C.UTF-8", and for some reason
we were interpreting that as "C" rather than "C.UTF-8".

Test: bionic tests, ls
Change-Id: Ic60e3b90cd5fe689e5489fad0d5d91062b9594ed
2016-10-24 14:50:31 -07:00
Elliott Hughes
89e29ee485 Fix mbsnrtowcs where dst is null.
POSIX is its usual unintelligible self
(http://pubs.opengroup.org/onlinepubs/9699919799/functions/mbsrtowcs.html),
but the ISO C11 standard (7.29.6.4.1 paragraph 2) is pretty clear: *src
should change if and only if dst is non-null.

Bug: https://code.google.com/p/android/issues/detail?id=166381
Test: bionic tests
Change-Id: Ibc631cfa5b1bf4a6f56963feba9f0eea27b07984
2016-09-29 17:26:13 -07:00
Dan Albert
3c5037f1b3 Fix incorrect parameter types for locale funcs.
strtoll(3), strtoull(3), wcstoll(3), and wcstoull(3) all take an _int_
as a base, not a size_t. This is an ABI compatibility issue.

Bug: 17628622
Change-Id: I17f8eead34ce2112005899fc30162067573023ec
2014-09-23 15:32:24 -07:00
Dan Albert
b6cc8e00cd Fix mbsrtowcs(3) src param for finished string.
A mistake I made while cleaning this up the first time through.
mbstrtowcs(3) sets the src param to null if it finishes the string.

Change-Id: I6263646e25d9537043b7025fd1dd6ae195f365e2
2014-07-31 11:31:03 -07:00
Dan Albert
6b55ba54ef Fix mbsrtowcs(3)'s handling of len parameter.
The len parameter is a _maximum_ length. The previous code was treating
it as an exact length, causing the following typical call to fail:

    mbsrtowcs(out, &in, sizeof(out), state); // sizeof(out) > strlen(in)

Change-Id: I48e474fd54ea5f122bc168a4d74bfe08704f28cc
2014-07-21 11:45:48 -07:00
Dan Albert
dfb5ce42bc Revert "Revert "Add locale aware APIs.""
This reverts commit 063e20c269.

Change-Id: Ib8c9004efefe75a5346b3af50dfe37952d91eb21
2014-07-11 16:21:31 +00:00
Dan Albert
063e20c269 Revert "Add locale aware APIs."
Accidentally verified against a dirty tree. Needs the companion change to libc++ to land upstream before I can submit this.

This reverts commit e087eac404.

Change-Id: I317ecd0923114f415eaad7603002f77feffb5e3f
2014-07-09 22:50:43 +00:00
Dan Albert
e087eac404 Add locale aware APIs.
Since we only support the C locale, we can just forward all of these to
their non-locale equivalents for correct behavior.

Change-Id: Ib7be71b7f636309c0cc3be1096a4c1f693f04fbb
2014-07-09 15:41:53 -07:00
Dan Albert
7a7f9952c1 Adds functionality specified by uchar.h
mbrtoc32 and c32rtomb get their implementations from mbrtowc and wcrtomb. The
wc functions now simply call the c32 functions.

Bug: 14646575
Change-Id: I49d4b95fed0f9d790260c996c4d0f8bfd1686324
2014-06-04 08:39:24 -07:00
Calin Juravle
15a6310e4b Support mb sequences across calls to mb*to*wcs* functions
Bug: 13077905
Change-Id: I5abdc7cc3c27c109b7900c94b112f18a95c35763
2014-05-13 00:24:25 +01:00
Elliott Hughes
0d0ccfe2ce Fix wchar_t signedness problems found on x86-64.
The existing tests caught this.

Change-Id: I6269844ae4301fd2c596241a59e97eb67ef166fa
2014-05-01 19:03:18 -07:00
Elliott Hughes
568c86a489 Fix wchar.cpp signed/unsigned comparison build failure.
Change-Id: Id20b91f3d57c4430987b5cc88ac99c245801d73b
2014-05-01 16:49:55 -07:00
Elliott Hughes
5a0aa3dee2 Switch to a working UTF-8 mb/wc implementation.
Although glibc gets by with an 8-byte mbstate_t, OpenBSD uses 12 bytes (of
the 128 bytes it reserves!).

We can actually implement UTF-8 encoding/decoding with a 0-byte mbstate_t
which means we can make things work on LP32 too, as long as we accept the
limitation that the caller needs to present us with a complete sequence
before we'll process it.

Our behavior is fine when going from characters to bytes; we just
update the source wchar_t** to say how far through the input we got.

I'll come back and use the 4 bytes we do have to cope with byte sequences
split across multiple input buffers. The fact that we don't support
UTF-8 sequences longer than 4 bytes plus the fact that the first byte of
a UTF-8 sequence encodes the length means we shouldn't need the other
fields OpenBSD used (at the cost of some recomputation in cases where a
sequence is split across buffers).

This patch also makes the minimal changes necessary to setlocale(3) to
make us behave like glibc when an app requests UTF-8. (The difference
being that our "C" locale is the same as our "C.UTF-8" locale.)

Change-Id: Ied327a8c4643744b3611bf6bb005a9b389ba4c2f
2014-05-01 14:46:54 -07:00
Elliott Hughes
94336d8ecf Switch to OpenBSD stdio wide printf functions.
Change-Id: Icf4f8685d021ec6b7482ca1cc021ce8184098e4a
2014-04-29 17:39:29 -07:00
Elliott Hughes
c932225e10 Switch to OpenBSD stdio wide get/put functions.
Change-Id: I71f8769cdea874e55d397ca7682d9d4e659d3dcb
2014-04-29 17:08:03 -07:00
Elliott Hughes
01ae00f317 Switch to the OpenBSD implementations of the wide scanf functions.
This also gets us the C99 wcstoimax and wcstoumax, and a working fgetwc and
ungetwc, all of which are needed in the implementation.

This also brings several other files closer to upstream.

Change-Id: I23b025a8237a6dbb9aa50d2a96765ea729a85579
2014-04-29 16:28:56 -07:00
Elliott Hughes
770491fb4f Fix build (signed char issue).
Change-Id: I05d78f4c1599ed9a0c1285f9eb1e89bc2f55c24d
2014-04-29 16:05:58 -07:00
Elliott Hughes
3d7a0d9b08 Switch to the OpenBSD wcsto* functions.
This replaces a partial set of non-functional functions with a complete
set of functions, all of which actually work.

This requires us to implement mbsnrtowcs and wcsnrtombs which completes
the set of what we need for libc++.

The mbsnrtowcs is basically a copy & paste of wcsnrtombs, but I'm going
to go straight to looking at using the OpenBSD UTF-8 implementation rather
than keep polishing our home-grown turd.

(This patch also opportunistically switches us over to upstream btowc,
mbrlen, and wctob, since they're all trivially expressed in terms of
other functions.)

Change-Id: I0f81443840de0f1aa73b96f0b51988976793a323
2014-04-29 14:53:11 -07:00
Elliott Hughes
0a5e26da1e Add mbtowc and fix mbrtowc.
Change-Id: I48786cd82587e61188d40f6fd6e11ac05e857ae9
2014-04-28 17:51:13 -07:00
Elliott Hughes
d299bcfdad Replace our broken wcswcs with the working upstream one.
Change-Id: I2952684df5674d10f0564d92c2cd42597725c0e3
2014-04-28 16:46:24 -07:00
Elliott Hughes
1b836ee6f8 Fix a wchar.wcstombs_wcrtombs test failure.
Looks like I screwed up a last-minute refactor and didn't re-run the tests.

Change-Id: I90a710ae66a313a9812859650aa0b4e8c6bc57f9
2014-04-18 13:32:33 -07:00
Elliott Hughes
0549371bd7 Upgrade to current vfprintf.
This gets us various bug fixes and missing C99 functionality.

Bug: https://code.google.com/p/android/issues/detail?id=64886
Change-Id: Ie9f8ac569e9b5eec1e4a1faacfe2c21662eaf895
2014-04-17 17:30:03 -07:00
Elliott Hughes
40b0579127 Add iswblank for libcxx.
We have similar degenerate implementations for all the other isw* functions,
so it's weird to exclude just one.

Change-Id: I659b97930e68598826c4882bb59f4146870fb6a0
2014-04-15 12:04:05 -07:00
Elliott Hughes
77e944fd46 Implement wctomb(3) for ltrace.
This is an implementation in the style of the rest: char == byte.

We might want to come back and implement UTF-8, but this is enough for ltrace.

Bug: 13747066
Change-Id: Ib2b63609c9014fdef9a8491e067467c4fc5ae3cc
2014-04-07 14:29:28 -07:00
Elliott Hughes
29c7f0b4d1 Move setlocale(3) and the wchar stubs over to .cpp.
Also separate out the C++ files so we can use -Werror on them. I'd
rather wait for LOCAL_CPPFLAGS to be in AOSP, but this also lets us
see which files still need to be sorted into one bucket or the other.

Change-Id: I6acc1f7c043935c70a3b089f705d218b9aaaba0a
2012-10-22 17:05:27 -07:00