We use BString in the build platform, so we need it to work
everywhere. Add a fallback for non-Haiku targets (though glibc and
others do have memrchr and so could keep this, if we added some
more logic here.)
It turns out PAGE_ACCESS assertions could be tripped in other cases,
e.g. when resizing a cache while pages were being read in. So,
properly unset ACCESS (and just leave 'busy' set) in the places
where pages are read in, and re-enable the check.
Change-Id: I22ac5c8cb432c87613bb85e82703a0c1aeb55d86
Reviewed-on: https://review.haiku-os.org/c/haiku/+/10627
Reviewed-by: waddlesplash <[email protected]>
Since we unlock the cache at various points in this loop,
we need to periodically recheck the size here, as otherwise
we might try to insert pages past the end.
This amends 4f8eaff87f. Re-tested
using the case described in that commit.
Change-Id: Id65e309f986a9b7f3e701d2b99c4f318a4eefbd3
Reviewed-on: https://review.haiku-os.org/c/haiku/+/10626
Reviewed-by: waddlesplash <[email protected]>
This allows the VMCache to remove these pages pending the I/O
completion.
Combined with the following commits, confirmed to fix#18390.
Change-Id: I9630a00d70a9d6ac5b36e6de70475a6032390806
Reviewed-on: https://review.haiku-os.org/c/haiku/+/10591
Reviewed-by: waddlesplash <[email protected]>
Previously, busy-writing pages just stayed in the cache until
I/O finished. This wasn't right, as they should've been unmapped
(and removed) immediately, as a TODO in the PageWriteWrapper noted.
Now they are, and they're moved into a queue on the cache pending
the I/O completing.
This paves the way for fixing #18390.
Change-Id: I51778ba1cfc1d26f34dd44f1ee293381be59937f
Reviewed-on: https://review.haiku-os.org/c/haiku/+/8627
Reviewed-by: waddlesplash <[email protected]>
Tested-by: Commit checker robot <[email protected]>
Before, it just used a hardcoded Element::link member. Now,
it uses whatever one is specified based on the template
arguments.
No source compatibility breakage intended.
Change-Id: I62128921766df0b705488eb64402f902334f2ed0
Reviewed-on: https://review.haiku-os.org/c/haiku/+/10589
Reviewed-by: waddlesplash <[email protected]>
Tested-by: Commit checker robot <[email protected]>
Haiku-Format: Haiku-format Bot <[email protected]>
This fixes a long-standing TODO, and a major inconsistency
that for different caches, "commitment" reserves from different
pools, so any two aren't necessarily convertible.
The new system introduces a virtual method "TakeCommitmentFrom",
which allows the VM's cut_area routine to behave as it did
previously.
They mostly contained descriptions of functions' operations, not
what they're for, so they weren't so useful. I've added some of
the remarks they had (or ones with a similar purpose) inline
as documentation comments, and deleted the rest.
The documentation file hadn't been updated in a very long time,
and referred to locks and paradigms that have long since been
dropped (e.g. sMappingLock was removed in 2010, in
afed2658f4ef3e3df5ff8390d1d7a51b5b897a9e.)
Add a few "protected by" comments in VMArea to clarify things,
but otherwise this document looks entirely obsolete (I went
through all of it.)
EV_ADD implies ENABLE, so this isn't needed, but a lot of applications
try to specify it anyway. So this extends source compatibility.
If in the future we ever implement EV_DISABLE, we can just add a
real definition for it then, and since ADD implies ENABLE existing
applications shouldn't break.
If the source area is now mapped write-combining, so will the
target area.
May fix some problems or slowness that occurred after the
framebuffers refactor commits.
This allows the guarded heap to be used in a much more targeted way,
if it's suspected specific object_caches have problems.
Enabled by default for KDEBUG. The new safemode/kernel option is
"guarded_heap_for_object_caches". This accepts a comma-separated
list of quoted strings, optionally with * at beginning and/or end
(but not middle.) Examples:
mbufs* # matches caches starting with "mbufs"
*jumbo9* # matches caches containing "jumbo9"
"*jumbo9 chunks","mbufs*" # matches caches ending with "jumbo9 chunks"
# or starting with "mbufs"
This should help with diagnosing #19973, among others, without needing
to compile custom versions of drivers or anything like that.
Unfortunately it is not possible to use this mechanism to selectively
replace parts of the slab malloc. The slab malloc free() relies on
the MemoryManager to figure out what cache an object belongs to, which
doesn't work if objects of some size classes aren't known to the
MemoryManager at all.
Before this commit, swap space was committed entirely orthogonally
from memory commitments. This was not handled at all correctly
when cutting areas (the area cutting logic tries to migrate commitments
between caches, but in the case of swapping caches, it didn't,
and couldn't, take the separate fCommittedSwapSize into account.)
Even without that problem, totally separate swap and memory commitments
caused other problems. For instance, suppose a setup with 4GB of RAM
and 4GB of swap. Many caches are created, and all 4GB of swap gets
reserved, but the caches with this swap reserved have no or few pages
in them. Then, all 4GB of RAM gets used up. The page daemon runs, and
looks for things to swap out, but it can't find anything because all
our swappable caches with pages have no swap reservations, so all it
can do is flush out cached pages, which just degrades performance.
Or, consider what would happen if the 4GB of swap was released, but
the 4GB of RAM was still in use, and then the kernel tried to
allocate memory. It would get OOM errors, because none of the
caches have swap commitments and so nothing can be written out.
Under this new system, the specific swap space reservation only happens
just before the page is first written out, so both these cases should
be handled smoothly, and swap should be used in a much more sensible
fashion overall.
This change seems to be more in line with what other OSes do (Windows
Task Manager's "Committed" clearly is RAM+swap, for instance.)
The only potential downside to this change is that swappable caches
may temporarily go over their commitment when reading or writing
out pages to swap, before the page daemon has a chance to come by
and free the page or the swap space. But as that is a temporary state
and should always be resolved by the page daemon, I think it's
probably an acceptable trade-off.
Change-Id: I928f4b4cacbec26ed6315179476eaff1a7f626c8
Reviewed-on: https://review.haiku-os.org/c/haiku/+/10305
Tested-by: Commit checker robot <[email protected]>
Reviewed-by: waddlesplash <[email protected]>
This begins paving the way for removal of user access to kernel
address space.
After the previous commits, BDirectWindow and BWindowScreen
still function properly.
Part of #19990.
Change-Id: I8477e43cd517ba67383755a4cb60b00bc117f5d7
Reviewed-on: https://review.haiku-os.org/c/haiku/+/10559
Reviewed-by: waddlesplash <[email protected]>
Tested-by: Commit checker robot <[email protected]>
This will be useful in the case of cloned device caches based
around physical memory, e.g. for framebuffers.
Change-Id: If2b864208e86f436e91379b73c4b866b0bd5e1d7
Reviewed-on: https://review.haiku-os.org/c/haiku/+/10558
Reviewed-by: waddlesplash <[email protected]>
Tested-by: Commit checker robot <[email protected]>
It can't work if the framebuffer is mapped per-application.
Instead, use the accelerant hook to get the framebuffer.
The BScreen methods this disables were deprecated and marked
private already, so no Haiku applications should be affected.
(If really necessary they could be reimplemented with BWindowScreen.)
Remove first _GetCardInfo in BWindowScreen; we invoke it
later, closer to where the buffer is actually used.
Tested with ParticlesII (which uses BWindowScreen) on VESA
with the new user-only-mapping accelerant.
Change-Id: I405bcfa8cb1eb2d0c346fafbc6233bef15196134
Reviewed-on: https://review.haiku-os.org/c/haiku/+/10572
Tested-by: Commit checker robot <[email protected]>
Reviewed-by: waddlesplash <[email protected]>
This paves the way for accelerants to map the framebuffer in
user memory, not kernel memory. It uses the ServerMemoryAllocator
in order to avoid mapping the framebuffer multiple times in
client applications.
While at it, clean up some names.
Tested with DirectWindowStars and GLTeapot, both work.
Change-Id: I4e0c003fae99891044af0a5e6e1d6a4506da6430
Reviewed-on: https://review.haiku-os.org/c/haiku/+/10571
Tested-by: Commit checker robot <[email protected]>
Reviewed-by: waddlesplash <[email protected]>
Instead of the kernel heap being a compile-time parameter, make it
a runtime parameter (if DEBUG_HEAPS is specified, at least; otherwise
we're hardwired to the slab heap as before.)
This is similar to what's done in userland: there LD_PRELOAD and
MALLOC_DEBUG control the heaps, here "kernel_malloc" and then
"guarded_heap_options" kernel settings control the heap.
While at it, apply fixes to the guarded and debug heaps to ensure
a minimum alignment of sizeof(void*), if no specific alignment
is requested.
Tested with a lot of build configurations (DEBUG_HEAPS 0, 1;
DEBUG_HEAPS_DEFAULT slab, guarded; USE_DEBUG_HEAPS_FOR_OBJECT_CACHE
0, 1) and with various parameters specified in the bootloader.
* Insert/Remove can't fail, so drop status_t return values.
* "TransferAreas" is really "TakeAreasFrom", so rename it. This
avoids confusion with the "transfer_area" API call, which moves
an area to another address space (keeping the same cache.)
* Give arch_vm_supports_protection a team_id argument (primarily
to allow the kernel address space to be more restricted than
user ones.)
* Move invocation of arch_vm_supports_protection to a static method,
and also call fix_protection from there.
* Consolidate protection checks.
It isn't needed now that it's set before sending the ICI. Saves
a lot (hundreds, in some basic testing of compile jobs) of
unnecessary scheduler invocations.
These allow us to move a lot of the basic logic out of the
TranslationMaps. In the process, it extends the fix applied
to x86 in the previous commit to all the other maps.
We may be called from the page daemon, which may be running on
a different CPU than the one this map is active on. So don't
bother flushing unnecessarily if we can avoid it.
This avoids the broadcast mailbox (and the incrementing of
other CPU's counters) entirely in this case.
As this is used for TLB invalidation, processes that are only
running two threads at once will benefit immensely from this.
It also seems to be triggered by the kernel's "idle_scan_active_pages"
logic, which sometimes sends invalidations to CPUs besides its own.
On the VM setup, this path was hit around 200 times during boot,
around 600 times during a rebuild of HaikuDepot + mime_db, and
50,000+ times while running Iceweasel (Firefox). On bare metal,
it was hit around 300 times during boot, 300 times during a rebuild,
and 90,000+ times while running Iceweasel (over a different length
of time, though.)
Following upon a KDL reported by atomozero, on wait-for-free-SMP-messages
on an unblock occurring with interrupts disabled.
In changing cpu_ent, move disabled to the beginning to avoid
enlarging the structure unncessarily.
On a 4-core VM, this skips around 200 sends during boot, and
over 3000 during a rebuild of HaikuDepot + mime_db (cold). On
bare metal (i3, 2x2), it skips around 150 during boot, and a bit
below 3000 during a rebuild of HaikuDepot + mime_db (over a much
longer time than in the VM, as the hardware is slower.)
Performance in the VM doesn't look much different. But this might
help in VirtualBox, or other situations where ICI latency is far
above what it should be.
Since the methods now don't just "send", the names are changed
also (which is fitting since they're now API-incompatible.)
The changes in the TranslationMaps to adopt this fixes a correctness
issue: previously TLB invalidations would be done either with or
without interrupts enabled depending on how they were invoked.
Now, they'll always be done with interrupts disabled. (This is
the same fix that was recently done to x86.)
The idea with processing the ICIs on the current CPU too is
that presumably the other CPUs will take just as much, if not
more, time processing the ICI than the sending CPU. So, we might
as well do the processing while we wait, rather than before or after.
Testing shows this to indeed be the case. On a bare metal i3, kernel
map invalidations (with global invalidations always forced) across
a boot-to-desktop go from around ~60.8ms cumulative before,
to ~51.8ms cumulative after this change.
Change-Id: Icd0343d462de7cf11043e6438f47b02f6b88bb8b
Reviewed-on: https://review.haiku-os.org/c/haiku/+/10472
Reviewed-by: waddlesplash <[email protected]>
Tested-by: Commit checker robot <[email protected]>
The homegrown usb_asix was partially based on the upstream FreeBSD
driver anyway, and hasn't seen much maintenance in recent years.
We might as well replace it with the FreeBSD version. In the process,
also add the "axge" driver, supporting the Gigabit version of this
hardware.
Tested by smrobtzz with the AX88179A: "it seems to work perfectly".
Change-Id: I37fdfc2cef6100916d6034817f117286f7f443b2
Reviewed-on: https://review.haiku-os.org/c/haiku/+/9600
Reviewed-by: waddlesplash <[email protected]>
Also delete an OpenBSD compatibility section that isn't used.
This includes a fix to mbufq_full (made in 2019 in FreeBSD)
to allow for mbufqs of unlimited size, apparently needed by
some drivers, found by @smrobtzz while testing USB ethernet
drivers.
The SMP_MAX_CPUS * 4 fixed pool size apparently goes all the way
back to NewOS, which supported only 4 CPUs max. As ours is now
64, this means the fixed pool size was very large even on systems
with only a few cores. So, instead, allocate 4 messages per CPU
(though often more, due to rounding up to the page size; e.g.
on x86_64, 1 page fits 56 smp_msgs.)
Also, put them a dedicated area, to keep them a bit more segmented
from the kernel heap, in case of problems.
Since the kernel is built with -ffreestanding, and thus -fno-builtin,
the compiler ignores the function names and doesn't perform optimizations
(e.g. inlining for small values) that it otherwise could. So, here,
we add a string.h that supplements the default, and uses #define to
reinstate the builtins, and thus the optimizations.
Linux and FreeBSD at least apparently do the same. A quick compile
benchmark in a VM doesn't show much difference, maybe a slight
decrease in sys time.
I checked all invocations of GetBit; as far as I can tell,
only the two adjusted here in smp.cpp are done while the Set
may be concurrently modified, and even then I don't know
that they really need to be atomics, anyway.
This allows read spinlocks to be used in check_for_message
and return_free_message.
On a 4-core VM, there don't seem to often be concurrent readers
(a counter I setup got only about 400-500 hits for booting up
and starting a web browser, at most.) But on bare metal (i3,
2 cores x 2 hyperthreads) we get it a lot more: 30,000+ times
during boot alone.
Kernel map invalidation (with a hack to invalidate all every time)
across boot-to-desktop takes a cumulative ~68.0ms before this change,
and ~61.0ms after, on the bare metal system. (On the VM it's also lower
after this change.)
Presumably on systems with more CPUs it will get hit even more
often still. So this optimization seems worth it.
They call out to other library functions, which are better optimized
than the old naive implementations.
Add -fno-builtin to the musl strings Jamfile, otherwise GCC
generates infinite recursion in strcpy, apparently.
Augustin Cavalier