Currently there are deferral-relevant constants in arena.c and
background_thread.c. This commit moves them out into corresponding PAC
and HPA headers. Those commonly needed by both are put into a new header
deferral.h where the contract of deferral are clearly stated.
background_thread_sleep() computed the pthread_cond_timedwait deadline
from gettimeofday() (CLOCK_REALTIME). If the wall clock jumps backward
(NTP correction, VM snapshot, leap second), the deadline ends up far in
the future and the background thread stops purging dirty pages -
potentially for hours or days. This was observed in production where
NTP corrections caused unbounded RSS growth due to the background
thread stalling.
Switch deadline computation to nstime_init_update() (which already
handles the monotonic vs realtime fallback), and set CLOCK_MONOTONIC
on the condvar via pthread_condattr_setclock at both init sites
(background_thread_boot1 and background_thread_postfork_child), with
fallback to CLOCK_REALTIME if condattr init/setclock fails.
The split managed one ordering constraint: arena_choose() had to be
defined before arena_choose_maybe_huge() but after the tsd/tcache
inlines it depends on. After the malloc_dispatch refactor moved the
heaviest tcache-pulling inlines out of arena_inlines_b.h, the
remaining arena-side inlines all belong together. The merged
arena_inlines.h explicitly includes jemalloc_internal_inlines_a.h
and tcache.h (previously transitively pulled).
Convert the production source files in src/ (69 .c/.cpp) and
test/jemalloc_test.h.in to list the headers they actually use, then
delete the umbrella. Three consolidated headers (peak_event.h,
prof_sys.h, sz.h) also gain explicit transitive includes.
Every translation unit now declares what it uses. A missing include
now fails at the failing file rather than silently working because
something upstream pulled in the world.
The address of the local variable created_threads is a different
location than the data it points to. Incorrectly treating these
values as being the same can cause out-of-bounds writes to the stack.
Closes: facebook/jemalloc#59
If jemalloc is linked into a shared library, the RTLD_NEXT dlsym call
may fail since RTLD_NEXT is only specified to search all objects after
the current one in the loading order, and the pthread library may be
earlier in the load order. Instead of failing immediately, attempt one
more time to find pthread_create via RTLD_GLOBAL.
Errors cascading from this were observed on FreeBSD 14.1.
During boot, some mutexes are not initialized yet, plus there's no point taking
many mutexes while everything is covered by the global init lock, so the locking
assumptions in some functions (e.g. background_thread_enabled_set()) can't be
enforced. Skip the lock owner check in this case.
Following from PR #2481, we replace all integer-to-pointer casts [which
hide pointer provenance information (and thus inhibit
optimizations)](https://clang.llvm.org/extra/clang-tidy/checks/performance/no-int-to-ptr.html)
with equivalent operations that preserve this information. I have
enabled the corresponding clang-tidy check in our static analysis CI so
that we do not get bitten by this again in the future.
Fix or suppress the remaining warnings generated by static analysis.
This is a necessary step before we can incorporate static analysis into
CI. Where possible, I've preferred to modify the code itself instead of
just disabling the warning with a magic comment, so that if we decide to
use different static analysis tools in the future we will be covered
against them raising similar warnings.
This change allows every allocator conforming to PAI communicate that it
deferred some work for the future. Without it if a background thread goes into
indefinite sleep, there is no way to notify it about upcoming deferred work.
Previously the calculation of sleep time between wakeups was implemented within
background_thread. This resulted in some parts of decay and hpa specific
logic mixing with background thread implementation. In this change, background
thread delegates this calculation to arena and it, in turn, delegates it to PAI.
The next step is to implement the actual calculation of time until deferred work
in HPA.
For now, this is just a stub containing the ecaches, with no surrounding code
changed. Eventually all the core allocator bits will be moved in, in the
subsequent stack of commits.
This will eventually completely wrap the eset, and handle concurrency,
allocation, and deallocation. For now, we only pull out the mutex from the
eset.
Added a new stats row to aggregate the maximum value of mutex counters for each
background threads. Given that the per bg thd mutex is not expected to be
contended, this counter is mainly for sanity check / debugging.
Background threads may run for a long time, especially when the # of dirty pages
is high. Avoid blocking stats calls because of this (which may cause latency
spikes).
For low arena count settings, the huge threshold feature may trigger an unwanted
bg thd creation. Given that the huge arena does eager purging by default,
bypass bg thd creation when initializing the huge arena.
Before this commit jemalloc produced many warnings when compiled with -Wextra
with both Clang and GCC. This commit fixes the issues raised by these warnings
or suppresses them if they were spurious at least for the Clang and GCC
versions covered by CI.
This commit:
* adds `JEMALLOC_DIAGNOSTIC` macros: `JEMALLOC_DIAGNOSTIC_{PUSH,POP}` are
used to modify the stack of enabled diagnostics. The
`JEMALLOC_DIAGNOSTIC_IGNORE_...` macros are used to ignore a concrete
diagnostic.
* adds `JEMALLOC_FALLTHROUGH` macro to explicitly state that falling
through `case` labels in a `switch` statement is intended
* Removes all UNUSED annotations on function parameters. The warning
-Wunused-parameter is now disabled globally in
`jemalloc_internal_macros.h` for all translation units that include
that header. It is never re-enabled since that header cannot be
included by users.
* locally suppresses some -Wextra diagnostics:
* `-Wmissing-field-initializer` is buggy in older Clang and GCC versions,
where it does not understanding that, in C, `= {0}` is a common C idiom
to initialize a struct to zero
* `-Wtype-bounds` is suppressed in a particular situation where a generic
macro, used in multiple different places, compares an unsigned integer for
smaller than zero, which is always true.
* `-Walloc-larger-than-size=` diagnostics warn when an allocation function is
called with a size that is too large (out-of-range). These are suppressed in
the parts of the tests where `jemalloc` explicitly does this to test that the
allocation functions fail properly.
* adds a new CI build bot that runs the log unit test on CI.
Closes#1196 .
