Commit graph

93 commits

Author SHA1 Message Date
Ruchir Rastogi
e712eed24c Support atom-level annotations within AStatsEvent
Refactors implementation details to allow for atom-level annotations.
This CL does not change the API surface. addBoolAnnotation and
addInt32Annotation work for both atom-level and proto field annotations.

Test: bit libstatssocket_test:*
Bug: 151158794
Change-Id: I2c340b201aeb3bcddd5cfde171b7c4df1d8d69a4
2020-03-12 15:21:26 -07:00
Jiyong Park
c2a354be68 Merge "don't include liblog to APEXes" into rvc-dev 2020-03-12 04:10:15 +00:00
Jiyong Park
01aa81cec5 don't include liblog to APEXes
liblog is a platform library that provides stable C API. There is no
need to include the library, especialy by statically linking to it, in
any APEX. It not only wastes the storage/ram, but also is incorrect
because the socket interface to logd which is implemented in liblog is
not guaranteed to be stable.

Fixing this issue by converting static_libs: ["liblog"] into
shared_libs: ["liblog"], in which case the dependency to the library
is satisfied via the stub variant of the library.

As a result, we could restrict the availablity of the library to
the platform and the runtime APEX.

Exempt-From-Owner-Approval: already approved when this was in internal
master (ag/10572699)

Bug: http://b/151051671
Bug: http://b/150827719
Test: m
Merged-In: I5aab863cb12b8767b6979255c247000a59355b0e
(cherry picked from commit 95b6f45b0e)
Change-Id: I5aab863cb12b8767b6979255c247000a59355b0e
2020-03-11 18:30:06 +09:00
Ruchir Rastogi
7a287d0751 Use dlsym to load libstatssocket API
Also fixes a bug in the tests where the platform version was being
checked by querying version.release instead of version.codename
(currently, version.release == 10, while version.codename == R).

Test: bit libstatspush_compat_test:* (on Q and R)
Test: libsstatssocket.so is not placed in resolv apex (on R)

1|bonito:/ $ ls apex/com.android.resolv/lib64
libclang_rt.ubsan_standalone-aarch64-android.so
libcrypto.so
libnetd_resolv.so
libssl.so

Test: NETWORK_DNS_EVENT_REPORTED atom is logged to statsd (on R)
    - adb shell cmd stats print-stats (atom 116 count > 0)

Bug: 148743333
Change-Id: Ib3eaa32835905bcf6e3b003054bf0f3c4e7ec0a4
Merged-In: Ib3eaa32835905bcf6e3b003054bf0f3c4e7ec0a4
(cherry picked from commit 9848eb4f1c)
2020-03-09 15:16:11 -07:00
Luke Huang
7452e46ae0 Use libbase to get the elapsed time instead of libutils
To minimize the external library dependencies

Bug: 149721367
Test: atest
Change-Id: I7b79223ad58c2af94d01a35ad39be6cdd1de7f47
Merged-In: I7b79223ad58c2af94d01a35ad39be6cdd1de7f47
(cherry picked from commit cab9854fa9)
2020-03-06 22:09:44 +00:00
Tej Singh
d78f4dab02 Make libstatssocket a shared_lib
Make libstatssocket unable to be linked as a static lib on the platform.

Test: bit libstatssocket_test:*
Test: bit statsd_test:*
Test: atest LibStatsPullTests
Bug: 149340100

Change-Id: I5889dd718536f5f2c693b4c2e7331c9cc2eb2ac9
2020-02-27 11:37:33 -08:00
Muhammad Qureshi
3a40ee5402 //apex_available:platform should be first in list
Test: m
Change-Id: I6ede6f008bc149f587ae2fc024d5869463b2c64d
2020-02-18 17:05:19 -08:00
Ruchir Rastogi
d430c85c52 Merge "Delete stats_event_list from libstatssocket" 2020-02-15 01:34:34 +00:00
TreeHugger Robot
ea37af2c62 Merge "AStatsEvent_overwriteTimestamp" 2020-02-14 23:11:48 +00:00
Ruchir Rastogi
e1cff24259 Delete stats_event_list from libstatssocket
This change is done in internal master because the flag has not been
switched in AOSP, so stats_event_list is still being used there.

Test: m -j
Bug: 145573568
Change-Id: Ica525d0fd731b8d3e9d273ce2220a3d3179d93d9
2020-02-14 13:34:10 -08:00
Howard Ro
f8e70a5bf8 Merge "Expose AStatsSocket_close() from the statsd apex" 2020-02-14 19:43:21 +00:00
Ruchir Rastogi
e41a6c621c AStatsEvent_overwriteTimestamp
Create a function to overwrite timestamp for LogEvent tests. This
function is not exposed outside of the APEX.

Test: bit libstatssocket_test:*
Change-Id: Id0eac485af60e952cc700f7d1231b1b971400982
2020-02-14 10:56:49 -08:00
Muhammad Qureshi
2ebc2e3d8f Expose AStatsSocket_close() from the statsd apex
As part of statsd becoming a Mainline module in R, we need to formalize
the statsd APIs called from outside the statsd apex.

Bug: 145923416
Test: m
Change-Id: I296a018cc93d95bda77d95f8161acbbe983a5d51
2020-02-13 16:16:48 -08:00
Jeffrey Huang
b0a85dc46d Add apex_available to libs
Bug: 145922701
Test: m -j
Change-Id: If5cbe5651e47878520bdc5b943b17a4f72a2a2f3
2020-02-12 18:30:26 -08:00
Ruchir Rastogi
223dadb830 Migrate libstatspull to libbinder_ndk
Since libstatspull will be placed within the APEX, we need to move to
libbinder_ndk.

Test: m -j libstatspull
Test: atest LibStatsPullTests
Bug: 145213327
Change-Id: I86dd6e759efa5eb20bffc76492d5fa72cf279872
2020-02-10 16:07:18 -08:00
Jeffrey Huang
8095e4eef1 Update libstatspull lib
statsd_aidl was updated, so this is the matching change.

Test: m -j
Bug: 148682447
Change-Id: Iec0459cec9f649eb23c6d04b205075fff9b6e911
2020-02-07 12:17:40 -08:00
Muhammad Qureshi
b173a95ae8 Fix native stats_event logging
pass the byte array pointer; not the reference to the pointer.

Also turn on new socket protocol.

Bug: 148909969
Bug: 143979391
Test: m
Test: fastboot flashall
Test: atest android.cts.statsd.atom.UidAtomTests#testLmkKillOccurred
Test: atest LibStatsPullTests
Test: atest GtsStatsdHostTestCases
Change-Id: If7abae5cadb036555540d1b9b5e7f2cabe99beb8
2020-02-05 17:10:46 -08:00
Howard Ro
f49c38beb5 Merge "Expose libstatspull as a stable C API" 2020-02-05 18:28:12 +00:00
Tej Singh
0fca736e6c Add libstatssocket_test to TEST_MAPPING
Test: treehugger
Change-Id: Iddff5df4c498bf7c93e57b1ae214f813920243e1
2020-02-04 16:08:21 -08:00
Tej Singh
a8fd39d83f Expose libstatspull as a stable C API
Libstatspull will ship as a part of the statsd apex in R.

Also update libstatssocket to have version 30 to align with api level,
as per native api council feedback.

Change both libraries to use NDK style naming conventions (AStatsEvent,
AStatsManager), and remove unneeded APIs in libstatssocket.

Remove KeyValuePairs from libstatssocket. KeyValuePairs will not be
supported in native code.

Bug: 147499386
Test: make libstatspull
Test: make libstatssocket
Test: atest libstatssocket_test
Change-Id: Ie79771461215a057529aaac91db95e4334c3960e
2020-02-04 14:07:35 -08:00
Muhammad Qureshi
a37a2bc415 export libstatssocket_headers
Add export_header_lib_headers: ["libstatssocket_headers"]
to libstatspush_compat

Test: m
Change-Id: I70d7a214a5233c6a730b24c9c21b8bd654a78b04
2020-01-30 16:44:53 -08:00
Tej Singh
518d77d208 Register pullers in separate thread
This cl modifies getStatsService to use the blocking getService, but
also makes the binder call in a separate thread to not block the client
thread. This is needed because it is possible for pullers to be
registered before statsd starts, and calling checkService before statsd
is up will fail. We also would never receive the binderDied to register
it, because we would never have a binder object to linkToDeath on.

Bug: 147682855
Test: atest LibStatsPullTests
Change-Id: I68c04bc24c7fe066eca88cab4f6a76885581c1ee
2020-01-15 21:15:26 +00:00
Tej Singh
dab324f6e0 Unregistration of native pullers
Changes the API to return an int instead of a bool.
Creates API to unregister pullers.

Test: atest LibStatsPullTests
Bug: 144373250
Change-Id: I6d395237082337a22f37190343a1fc59703d39bf
2020-01-14 14:37:58 -08:00
TreeHugger Robot
d37759086f Merge "Remove extra indirection in native puller API" 2020-01-10 08:29:37 +00:00
Tej Singh
7e1b190832 Remove extra indirection in native puller API
The indirection is unneeded since we already take a function pointer.

Test: builds
Change-Id: Ice426d7dcb090aaf87a377ede233a4eb18f5eb30
2020-01-09 11:53:53 -08:00
Ruchir Rastogi
21c1f9c314 stats_event.h/c tests
Add unit tests for the native API to log atoms to statsd.

Test: bit libstatssocket_test:*
Bug: 145231901
Change-Id: If427c17319787200260cbe3b71075ca556c9a82b
2020-01-08 14:30:01 -08:00
Automerger Merge Worker
a8cdcadbd2 Merge "Fix bug in stats_event_benchmark" am: 3bf8858105 am: ae9375f13e am: 759c6ff425
Change-Id: I9446a60a7aa33d913d92b13f60690f5b7f903790
2020-01-07 01:13:13 +00:00
Treehugger Robot
3bf8858105 Merge "Fix bug in stats_event_benchmark" 2020-01-07 00:43:49 +00:00
Ruchir Rastogi
d177461a72 Fix bug in stats_event_benchmark
Now, the number of elements in each simulated stats_event is correctly
sampled from a uniform distribution.

Rerunning the benchmarking tests indicates that truncating the buffer
increases the number of cycles require to log an event by only 1.6%.

Test: m libstatssocket_benchmark
Test: bit libstatssocket_benchmark:*
Change-Id: I8d5c1734861038beec8eeb7ce1f84a24bde25a75
2020-01-06 23:10:51 +00:00
Automerger Merge Worker
5310401fe0 Merge "Expose libstatssocket as a stable C API" am: 2ff1eac71a am: 1662e44207 am: dada41d2cc
Change-Id: I33cd016ccbe2ce29b775b7a630c46bc00917114f
2020-01-06 21:41:15 +00:00
Ruchir Rastogi
2ff1eac71a Merge "Expose libstatssocket as a stable C API" 2020-01-06 20:57:55 +00:00
Ruchir Rastogi
5dbcae5020 Expose libstatssocket as a stable C API
We create a .map.txt file that lists all the stable entry points into
libstatssocket. This should allow other APEXes to link to libstatssocket
without having to copy the library within the APEX.

Test: m -j libstatssocket
Bug: 146377784
Change-Id: I9f77a0c380b6884d9ca60807a8974380420cfe0a
2019-12-27 15:04:54 -08:00
Automerger Merge Worker
a318188b70 Merge "Fix error checking in append_string" am: e2e2b2d3d3 am: 33e2002540 am: b2140e1246
Change-Id: I07dabf70064881875b5c6fd7522e9334dac3d8d9
2019-12-20 02:31:11 +00:00
Treehugger Robot
e2e2b2d3d3 Merge "Fix error checking in append_string" 2019-12-20 00:38:25 +00:00
Ruchir Rastogi
3bbc240263 Fix error checking in append_string
Test: m -j
Change-Id: Ifa806d5037acb17f83cbf45712e9795d6f56152f
2019-12-19 14:57:10 -08:00
Ruchir Rastogi
20516da9bc Merge libstatspush_compat and libstatssocket_q
This cleanup makes it clear that there are only two acceptable ways of
logging to the socket: either through libstatspush_compat or
libstatssocket (which will soon be renamed to libstatspush).

Test: m -j
Change-Id: I467b08bc300c2ef394b22422c6acf55a978a380a
Merged-In: I6a49b4995b56df462e1be901ea5e54acfe2e8bac
2019-12-18 17:48:28 -08:00
Ruchir Rastogi
79dd3eaa87 Improve stats_event memory usage
We now truncate the buffer to the appropriate length when clients call
stats_event_build().

Benchmarking tests indicate that truncating the buffer to the
appropriate length increases the cost clients pay to write to the socket
by 2%. This is negligible enough that I decided to truncate the buffer
for both pushed and pulled atoms in order to simplify the API.

Test: m libstatssocket
Test: bit libstatssocket_benchmark:*
Bug: 144126231
Change-Id: I35dec748ff87c0821d0d06779a406997e6e64966
Merged-In: Ife976bb383ecff8de5064730692a95e2a3a82c9d
2019-12-18 17:44:26 -08:00
Ruchir Rastogi
926fa88c35 StatsEventCompat
StatsEventCompat is a wrapper library that toggles between the old
logging scheme and the new logging scheme. It was designed particularly
for the DNS Resolver module and should not be used by others.

We will merge libstatspush_compat and libstatssocket_q in a future CL.

Test: m libstatspush_compat
Test: bit libstatspush_compat_test:* (passes on Q and R)
Bug: 145534143

Change-Id: Ib355031f3573101ea90bd8694861fbfc33b0c788
Merged-In: Idf35ccb6669798166475f08b2fbab40534b5db19
2019-12-18 17:21:27 -08:00
Muhammad Qureshi
a33f81bff5 Remove libstats/Android.bp
subdirs is not needed

Test: m -j
Change-Id: Idc034571eadae9c3135324af45fb4cd404ba8dd2
Merged-In: Ia7675fdf2e9a3a19bc52396ccbcc551556958803
2019-12-18 15:26:02 -08:00
Muhammad Qureshi
6d5a2fc44d Make stats_event write function parameters const
Bug: 143968790
Test: bit statsd_test:*
Change-Id: I413accb10cf3de4c5c49eb06a472168b1171039b
Merged-In: I2660cad5400a0e819c60b9caa706bb66ca2eef4f
2019-12-18 15:20:59 -08:00
Muhammad Qureshi
962cc23b9f Create stats_buffer_writer
Move iovec* construction and noteLogLoss from stats_event_list to
stats_buffer_writer.h.
Expose write_buffer_to_statsd that takes in a byte array and writes
it to the statsd socket. Currently exposed for StatsLog.write JNI call.

This change also allows getting rid of stats_event_list once all clients
have migrated to stats_event

Bug: 145619049
Test: m -j libstatssocket
Change-Id: I73f914985421fc1a4f094982c826bf364604b788
Merged-In: I0048e392c2f5039eb70dacf4e91a43d3f32e8749
2019-12-18 15:15:43 -08:00
Muhammad Qureshi
a3dcefc079 Create libstatssocket_q
Break up libstatssocket into libstatssocket_q and libstatssocket.
libstatssocket_q is for Q Mainline modules.

Bug: 145569088
Test: m -j
Test: bit statsd_test:*
Test: adb shell cmd stats print-logs && adb logcat "*:S statsd:*"
Change-Id: I2978dd544f4cb17714f3d14d5977cf6b46277ef9
Merged-In: I9d113b37640345ebad6aa269ab710db0d2179671
2019-12-18 15:11:53 -08:00
Ruchir Rastogi
a38d5a6fd6 Simplified stats_event.c; exposed getter methods
Simplifications/changes:
(1) We remove the bufPos field and track the end of the buffer using
    only the size field.
(2) We rename put_<type>() functions to append_<type>() to signify that the
    value will be placed at the end of the buffer. We also move the
    increment of event->size to within the append_<type>() functions;
    this improves readability of the write_<type>() functions.
(3) We immediately write the timestamp and atom id to the buffer in
    order to simplify stats_event_build().
(4) We never check for null pointers; checking for null pointers
    delays errors and obfuscates the root problem.
(5) We change the the annotationId and numPairs parameters to be
    uint8_t's. This helps signify to clients that these values must fit
    in 1 byte.
(6) Clients no longer have to pass in the length of strings. Instead, we
    expect them to pass in null-terminated strings, and we will
    calculate the length outselves using strnlen.

Test: m -j libstatssocket
Change-Id: I6d192768876a23d7016173bcdaf59f8b7e92b182
Merged-In: I0173f8bc76ef25118379dce5d2481f5f7a9b7519
2019-12-18 13:17:36 -08:00
Ruchir Rastogi
3822e61d7b General cleanup; add support for key-value pairs
Encoding of key-value pairs follows strategy described in
go/statsd-socket-doc.

Fixed bug where timestamp and atom id were not contributing to the
number of fields in the buffer.

Note: Similar to ag/9603344, which was abandoned because of git rebasing
issues.

Test: m -j libstatssocket compiles
Bug: 143079361
Change-Id: I15abdf13ad8debf83ae1f6c372676db4fe390289
Merged-In: I8d23ddf2dbfee518ad970894ac09ba3ad5b5b4ad
2019-12-18 13:11:28 -08:00
Ruchir Rastogi
a3bc1af1b8 Move type ids to stats_event.h
This allows the parsing code within statsd to access the type ids.
Also, we move stats_event.h to the include directory for consistency.

Test: m -j libstatssocket
Change-Id: I7030416872677f3a523c2baf7bf6bcb4ca0ae6a4
Merged-In: I1db7fc9e9d0efd2225612353a8fc49bb13220ff0
2019-12-18 11:58:42 -08:00
Ruchir Rastogi
f4d23fc353 Set default timestamp to elapsedRealtimeNanos
Test: m -j libstatssocket compiles
Change-Id: I52b0cb8e1e916413db3dc0fe1b9a7e922833bdf7
Merged-In: I62f4c6bb14c3914124a6af25ff46fb0a980204e7
2019-12-18 11:53:38 -08:00
Ruchir Rastogi
bb0da94f8d First draft of native StatsEvent API
Test: m -j libstatssocket compiles
Change-Id: I874968d4f77a5990764100123794b923847e5eb5
Merged-In: I0da9740d29749d79d14be39e5949b98c58df704d
2019-12-18 11:33:22 -08:00
Ruchir Rastogi
ffa2b31708 Merge "Merge libstatspush_compat and libstatssocket_q" 2019-12-17 19:33:58 +00:00
Ruchir Rastogi
7070b7ac74 Merge libstatspush_compat and libstatssocket_q
This cleanup makes it clear that there are only two acceptable ways of
logging to the socket: either through libstatspush_compat or
libstatssocket (which will soon be renamed to libstatspush).

Test: m -j
Change-Id: I6a49b4995b56df462e1be901ea5e54acfe2e8bac
2019-12-16 15:24:36 -08:00
Stan Iliev
5dc56738bf Change register_stats_pull_atom_callback cookie parameter to non-const
Test: Ran GRAPHICS_STATS puller
Bug: 146052586
Bug: 142665516
Change-Id: I98b4cbb99e35eda50fa9dc3069b3f0ddf16f5ff9
2019-12-16 12:14:08 -05:00