docs/develop: More reorganization.
* There is now a 'busses' folder, and the extant USB/SDHCI/Bluetooth/etc. docs now live in it, instead of various other places. * kernel/ports is now kernel/arch, like it is in src/system. SPARC documentation is now in there, too. * VM files (these are rather outdated) are now in kernel/vm. * SCSI ASC info removed, this is easily available online and it doesn't seem to be very relevant.
This commit is contained in:
Augustin Cavalier
@@ -0,0 +1,52 @@
|
||||
Locks/Reference Counting:
|
||||
|
||||
vm_address_space:
|
||||
sem R/W for area creation/deletion and any other address space changes
|
||||
fields: areas, area_hint (is currently written in vm_area_lookup() without a write lock!),
|
||||
state
|
||||
ref_count: ensures validity of object beyond team lifetime,
|
||||
retrieved via the global sAddressSpaceTable's pointer (which is guarded by
|
||||
sAddressSpaceHashSem)
|
||||
vm_address_space_walk_next() is unsafe! (and obsolete, only used by the former
|
||||
page scanner)
|
||||
Problems: resize_area() does not lock any address spaces yet, but needs to lock all clones
|
||||
|
||||
vm_area:
|
||||
ref_count: ensures validity
|
||||
retrieved via the global sAreaHash's pointer (which is guarded by
|
||||
sAreaHashLock)
|
||||
vs. vm_area_lookup() which iterates over the address space's area list, not
|
||||
the hash - therefore, it checks ref_count against NULL (ugly)
|
||||
variable fields:
|
||||
size, protection: essentially unguarded! (can be changed by resize_area()
|
||||
and set_area_protection())
|
||||
mappings: guarded by the global sMappingLock (currently a spinlock)
|
||||
address_space_next: vm_address_space::sem
|
||||
hash_next: sAreaHashLock
|
||||
cache: guarded by vm_area_get_locked_cache()/sAreaCacheLock
|
||||
cache_next|prev: cache_ref::lock
|
||||
|
||||
vm_cache_ref:
|
||||
ref_count: ensures validity
|
||||
vm_cache_remove_consumer(): does scary things with the ref_count
|
||||
fault_acquire_locked_source(): tries to get a ref through the vm_cache
|
||||
cache, areas: guarded by lock
|
||||
|
||||
vm_cache:
|
||||
all fields: guarded by ref::lock
|
||||
BUT: ref may change, therefore it's generally unsafe to go from cache to ref
|
||||
without holding the ref's lock (which happens, by design, in vm_cache::source
|
||||
and vm_cache::consumers)!
|
||||
|
||||
vm_page:
|
||||
hash_next: guarded by sPageCacheTableLock spinlock
|
||||
queue_prev|next: guarded by sPageLock
|
||||
cache_prev|next, cache, cache_offset: guarded by vm_cache_ref::lock
|
||||
mappings: guarded by the global sMappingLock (currently a spinlock)
|
||||
state: in vm_page only used with the sPageLock held, other uses have the
|
||||
cache locked the page is in
|
||||
wired_count, usage_count: not guarded? TBD
|
||||
busy_reading, busy_writing: dummy pages only
|
||||
|
||||
vm_translation_map:
|
||||
TBD.
|
||||
@@ -0,0 +1,2 @@
|
||||
- unmap the page of all areas from the cache when a newly added page shadows one
|
||||
in a deeper cache
|
||||
@@ -0,0 +1,100 @@
|
||||
Haiku swap file support
|
||||
|
||||
This article describes how to use swap file in Haiku and how the swap system
|
||||
works.
|
||||
|
||||
1. How to use a swap file?
|
||||
|
||||
Like BeOS, Haiku uses "/var/swap" as default swap file. It is created
|
||||
during the boot process and its size is twice the size of physical memory by
|
||||
default. You can change its size through the VirtualMemory preference
|
||||
application and your settings will take effect after restarting the system.
|
||||
|
||||
The default swap file "/var/swap" may not satisfy your need. Haiku allows
|
||||
adding/removing a swap file dynamically. (This is *NOT* implemented yet, since
|
||||
I do not know how to add bin commands "swapon" and "swapoff" in the system.
|
||||
It needs to be done in the future.)
|
||||
|
||||
2. How swap system works?
|
||||
|
||||
The virtual memory subsystem of Haiku is very similar to that of FreeBSD,
|
||||
therefore our swap system implementation is borrowed from FreeBSD.
|
||||
|
||||
A swap system has two main functions: (1) maintain a map between anonymous
|
||||
pages and swap space, so we can page in/out when needed. (2) manage the
|
||||
allocation/deallocation of swap space. Let's see how these are implemented in
|
||||
Haiku.
|
||||
|
||||
In order to maintain a map between pages and swap space, we need to record
|
||||
the pages' swap address somewhere. Here we use swap blocks. A "swap_block"
|
||||
structure contains swap address information for 32 (value of SWAP_BLOCK_PAGES)
|
||||
consecutive pages from a same cache. So whenever we look for a page in swap
|
||||
files, we should get the swap block for it. But how to get the swap block?
|
||||
Here we use hash table. All swap blocks in the system are arranged into a global
|
||||
hash table. The hash table uses a cache's address and page index in this cache
|
||||
as hash key.
|
||||
|
||||
Here is an example. Suppose a page has been paged out to swap space and now
|
||||
its cache wants to page it in. It works as follows: look up the swap hash table
|
||||
using address of the cache and page index as hash key, if successful, we get
|
||||
the swap block containing the this page's swap address. Then search the swap
|
||||
block to get the exact swap address of this page. After that, we can read the
|
||||
page from swap file using vfs functions.
|
||||
|
||||
I draw a picture and hope it could help you understand the above words. If
|
||||
the pic becomes a mess on your computer, please set the tab width of your text
|
||||
editor to 4.
|
||||
|
||||
___________________________________________________________
|
||||
sSwapHashTable |__________|___NULL___|___NULL___|___________|____NULL____|
|
||||
| |
|
||||
| |
|
||||
___V___ ___V___
|
||||
swap_block /----|__0__| /--------|__5__|
|
||||
| |__3__|--------\ | /---|__6__|
|
||||
| |_..._| | | | |_..._|
|
||||
| |__2__|----\ | | | |__20_|--------------->
|
||||
| | | | |
|
||||
| _____________V___V_________V____V_________________________
|
||||
swap_file `->|slot|slot|slot|slot|slot|slot|slot|slot|slot|slot|....|
|
||||
|_0__|_1__|_2__|_3__|_4__|_5__|_6__|_7__|_8__|_9__|____|__
|
||||
|
||||
|
||||
The swap system also manages allocation/deallocation of swap space. In our
|
||||
implementation, each swap file is divided into page-sized slots(called "swap
|
||||
pages") and a swap file can be seen as an array of many swap pages(see the
|
||||
above picture). Swap page is the unit for swap space allocation/deallocation
|
||||
and we use swap page index (slot index) as swap space address instead of offset.
|
||||
All the swap pages in the system are given a unified address and we leave one
|
||||
page gap between two swap files. (e.g. there are 3 swap files in the system,
|
||||
each has 100 swap pages, the address range(to be exact, page index) for each
|
||||
swap file is: 0-99, 101-200, 202-301) Why leave a page gap between swap files?
|
||||
Because in this way, we can easily tell if two adjacent pages are in a same
|
||||
swap file. (See the code in VMAnonymousCache::Read()).
|
||||
|
||||
The efficiency of the FreeBSD swap system lies in a special data structure:
|
||||
radix bitmap(i.e. bitmap using radix tree for hinting.) It can operate well no
|
||||
matter how much fragmentation there is and no matter how large a bitmap is
|
||||
used. I have ported the radix bitmap structure to Haiku, so our swap system
|
||||
will have a good performance. More information on radix bitmap, please look
|
||||
at the source code.
|
||||
|
||||
Swap space allocation takes place when we swap anonymous pages out.
|
||||
In order to make the allocation less probable to fail, anonymous cache will
|
||||
reserve swap space when it is initialized. If there is not enough swap space
|
||||
left, physical memory will be reserved. Swap space deallocation happens when
|
||||
available swap space is low. The page daemon will scan a number of pages and
|
||||
if the scanned page has swap space assigned, its swap space will be freed.
|
||||
|
||||
3. Acknowledgement
|
||||
|
||||
Special thanks to my mentor Ingo. He is a knowledged person and always
|
||||
gives me encouragement. Without his consistent and illuminating instructions,
|
||||
this project would not have reached its present status.
|
||||
|
||||
If you find bugs or have suggestions for swap system, you can contact me
|
||||
via [email protected]. Thanks in advance.
|
||||
|
||||
Zhao Shuai
|
||||
[email protected]
|
||||
2008-08-21
|
||||
@@ -0,0 +1,41 @@
|
||||
Variables
|
||||
static void *page_cache_table
|
||||
|
||||
static spinlock_t page_cache_table_lock
|
||||
|
||||
static int page_compare_func(void *_p, const void *_key)
|
||||
Compares a vm_page's cache_ref and offset to those of key. Returns 0 on match, -1 otherwise.
|
||||
|
||||
static unsigned int page_hash_func(void *_p, const void *_key, unsigned int range)
|
||||
If p is not null, use it otherwise use key; computes a hash value for offset and reference.
|
||||
|
||||
int vm_cache_init(kernel_args *ka)
|
||||
Calls hash_init, sets page_cache_table_lock to unlocked and returns 0.
|
||||
|
||||
vm_cache *vm_cache_create(vm_store *store)
|
||||
Allocates a vm_cache structure and populates with 0's and NULLs (except for store).
|
||||
|
||||
vm_cache_ref *vm_cache_ref_create(vm_cache *cache)
|
||||
Allocates a vm_cache_ref, setting it to point to the passed in cache. It initalizes the references lock and returns.
|
||||
|
||||
void vm_cache_acquire_ref(vm_cache_ref *cache_ref, bool acquire_store_ref)
|
||||
If the cache_ref is null, panic. Otherwise, if we are to aquire a reference and there is an aquire reference function for this cache's store, call that function. Finally, increment this cache references' count.
|
||||
|
||||
void vm_cache_release_ref(vm_cache_ref *cache_ref)
|
||||
If this cache_ref is the last reference to its cache, we call destroy on the store, remove all of its pages from the page_cache_table and set the state of them to free, increase max_commit by the size of the now freed storage, remove the reference to the original cache, destroy the reference's mutex and its space.
|
||||
Otherwise, we call the store's release ref and return.
|
||||
|
||||
vm_page *vm_cache_lookup_page(vm_cache_ref *cache_ref, off_t offset)
|
||||
Lock the page_cache_table_lock and lookup the offset and cache_ref in the page_cache_table, then unlock.
|
||||
|
||||
void vm_cache_insert_page(vm_cache_ref *cache_ref, vm_page *page, off_t offset)
|
||||
Add this page to the page_cache_table.
|
||||
|
||||
void vm_cache_remove_page(vm_cache_ref *cache_ref, vm_page *page)
|
||||
Find this page and remove it from the hash list. Clean up the linked lists.
|
||||
|
||||
int vm_cache_insert_region(vm_cache_ref *cache_ref, vm_region *region)
|
||||
Add this region to the cache_ref's region list.
|
||||
|
||||
int vm_cache_remove_region(vm_cache_ref *cache_ref, vm_region *region)
|
||||
Remove this region from the cache_ref's region list.
|
||||
@@ -0,0 +1,33 @@
|
||||
Variables
|
||||
|
||||
bool trimming_cycle;
|
||||
Do we need free space?
|
||||
|
||||
static addr free_memory_low_water;
|
||||
|
||||
static addr free_memory_high_water;
|
||||
|
||||
static void scan_pages(vm_address_space *aspace, addr free_target)
|
||||
Finds a region in this address space to scan.
|
||||
Locks the region's cache_ref.
|
||||
For each page,
|
||||
if the page is present, do nothing with this page
|
||||
Lookup the page structure. If this page doesn't exist, do nothing with this page.
|
||||
If the page is written to or is hard wired (unswapable), do nothing with it.
|
||||
If the page is not accessed and is active and we need space (free_target), unmap it. If this is the last reference to that mapped page, put it on the inactive list.
|
||||
If the page has been modified, but wasn't on the active list, put it there.
|
||||
Move to the next region in this address space, wrap around until we hit the first one.
|
||||
|
||||
static int page_daemon()
|
||||
Walk through every address space:
|
||||
Adjust the size of the processes' working set (i.e. memory allocation) to be larger or smaller, to tailor to faults.
|
||||
Set trimming cycle if free pages is below the high water mark.
|
||||
Clear trimming cycle if free pages is above high water mark.
|
||||
Set free memory target to be the processes' mapped size minus the working set
|
||||
Call scan_pages
|
||||
|
||||
|
||||
int vm_daemon_init()
|
||||
Sets high water to pages/4 and low water to pages/8.
|
||||
Creates the page daemon as a kernel thread.
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
Variables
|
||||
extern bool trimming_cycle;
|
||||
Determines if we are trimming - if we are nearly out of space
|
||||
|
||||
static page_queue page_free_queue;
|
||||
The queue that holds unused (but not cleared) pages
|
||||
|
||||
static page_queue page_clear_queue;
|
||||
The queue that holds cleared (0'ed) pages
|
||||
|
||||
static page_queue page_modified_queue;
|
||||
The queue that holds altered pages
|
||||
|
||||
static page_queue page_active_queue;
|
||||
The queue that holds in use pages that are not altered
|
||||
|
||||
static vm_page *all_pages;
|
||||
Every page in the system
|
||||
|
||||
static addr physical_page_offset;
|
||||
The first address of real ram
|
||||
|
||||
static unsigned int num_pages;
|
||||
Total number of pages in the system
|
||||
|
||||
static spinlock_t page_lock;
|
||||
A lock to protect the queues.
|
||||
|
||||
static sem_id modified_pages_available;
|
||||
A semaphore to indicate if there are pages that need to be written to disk
|
||||
|
||||
static void clear_page(addr pa);
|
||||
Sets all values in this page to 0.
|
||||
|
||||
static vm_page * dequeue_page(page_queue *q)
|
||||
Standard queue remove first page; has count; no locking
|
||||
|
||||
void dump_page_stats(int argc, char **argv);
|
||||
Dumps counts of each state (active, inactive, busy, not used, modified, free, cleared, wired)
|
||||
|
||||
void dump_free_page_table(int argc, char **argv)
|
||||
Not done.
|
||||
|
||||
static void enqueue_page(page_queue *q, vm_page *page)
|
||||
Standard queue add a page; has count; no locking; SPECIAL - for page_modified_queue, if there is only one page modified (new one), release semaphore
|
||||
|
||||
static bool is_page_in_phys_range(kernel_args *ka, addr paddr)
|
||||
Determines if a physical address is in the physical memory that exists.
|
||||
|
||||
static void move_page_to_queue(page_queue *from_q, page_queue *to_q, vm_page *page)
|
||||
Removes the page from the from_q and adds it to the to_q.
|
||||
|
||||
static int pageout_daemon()
|
||||
aquires the "modified pages available" semaphore. Gets first page from modified list. Sets it to busy, updates all processes mapping this page that it is no longer a modified page, writes it out to disk, then puts it on the active list (i.e. not modified). Doesn't write out "anonymous" blocks unless we are trimming.
|
||||
|
||||
static int page_scrubber(void *);
|
||||
Every 1/10 second, takes a page from the free queue, memsets it to 0's, then puts it on the clear queue.
|
||||
|
||||
static void remove_page_from_queue(page_queue *q, vm_page *page)
|
||||
Removes a given page from the given page queue. No locking.
|
||||
|
||||
addr vm_alloc_from_ka_struct(kernel_args *ka, unsigned int size, int lock)
|
||||
Gets virtual space in the ka using vm_alloc_vspace_from_ka_struct, then uses vm_alloc_ppage_from_kernel_struct to find a physical page to map it to.
|
||||
|
||||
static addr vm_alloc_ppage_from_kernel_struct(kernel_args *ka)
|
||||
Attempts to extend, by one page, one of the phys_alloc_range blocks in the kernel args.
|
||||
|
||||
static addr vm_alloc_vspace_from_ka_struct(kernel_args *ka, unsigned int size)
|
||||
Attempts to find an address range that can be extended to hold size bytes.
|
||||
|
||||
vm_page * vm_lookup_page(addr page_num)
|
||||
Does the math to get vm_page pointer from a page number.
|
||||
|
||||
int vm_mark_page_inuse(addr page)
|
||||
Calls vm_mark_page_range_inuse with len of 1.
|
||||
|
||||
int vm_mark_page_range_inuse(addr start_page, addr len)
|
||||
Sets state to PAGE_STATE_UNUSED for all pages in this range.
|
||||
|
||||
vm_page * vm_page_allocate_page(int page_state)
|
||||
Gets a single page from the clear or free queue (from page_state), fall back to the other.
|
||||
|
||||
vm_page * vm_page_allocate_page_run(int page_state, addr len)
|
||||
Attempts to find a run of pages "len" long that are either free or clear. If found, pulls them from their queues and calls vm_page_set_state_nolock on them
|
||||
|
||||
vm_page * vm_page_allocate_specific_page(addr page_num, int page_state)
|
||||
Finds this page, removes it from the clear or free queue, clears it if necessary and returns. Returns NULL for not found or page in use.
|
||||
|
||||
int vm_page_init(kernel_args *ka)
|
||||
Initializes the free, clear, modified and active queues. Sets up the vm structures.
|
||||
|
||||
int vm_page_init2(kernel_args *ka)
|
||||
Properly maps the vm structures allocated in vm_page_init.
|
||||
|
||||
int vm_page_init_postthread(kernel_args *ka)
|
||||
Creates the page scrubber and the pageout daemon.
|
||||
|
||||
addr vm_page_num_pages()
|
||||
Returns the total number of pages.
|
||||
|
||||
addr vm_page_num_free_pages()
|
||||
Returns count of pages in free and clear queues.
|
||||
|
||||
int vm_page_set_state(vm_page *page, int page_state)
|
||||
Locks, then calls vm_page_set_state_nolock, then unlocks.
|
||||
|
||||
static int vm_page_set_state_nolock(vm_page *page, int page_state);
|
||||
Moves a page from the state that it is in to the state specified, and moves it from the old queue to the new one.
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
static void anonymous_destroy(struct vm_store *store)
|
||||
Free's the memory associated with store
|
||||
|
||||
static off_t anonymous_commit(struct vm_store *store, off_t size)
|
||||
Returns 0
|
||||
|
||||
static int anonymous_has_page(struct vm_store *store, off_t offset)
|
||||
Returns 0
|
||||
|
||||
static ssize_t anonymous_read(struct vm_store *store, off_t offset, iovecs *vecs)
|
||||
Returns unimplemented
|
||||
|
||||
static ssize_t anonymous_write(struct vm_store *store, off_t offset, iovecs *vecs)
|
||||
Returns 0
|
||||
/*
|
||||
static int anonymous_fault(struct vm_store *backing_store, struct vm_address_space *aspace, off_t offset)
|
||||
*/
|
||||
|
||||
|
||||
vm_store *vm_store_create_anonymous_noswap()
|
||||
Allocates space for a vm_store. Populates its ops with the above, sets its cache and data to none.
|
||||
@@ -0,0 +1,20 @@
|
||||
static void device_destroy(struct vm_store *store)
|
||||
Frees the space associated with this store
|
||||
|
||||
static off_t device_commit(struct vm_store *store, off_t size)
|
||||
Sets this store's commited size to size.
|
||||
|
||||
static int device_has_page(struct vm_store *store, off_t offset)
|
||||
Returns 0
|
||||
|
||||
static ssize_t device_read(struct vm_store *store, off_t offset, iovecs *vecs)
|
||||
Returns unimplemented.
|
||||
|
||||
static ssize_t device_write(struct vm_store *store, off_t offset, iovecs *vecs)
|
||||
Returns 0
|
||||
|
||||
static int device_fault(struct vm_store *store, struct vm_address_space *aspace, off_t offset)
|
||||
Should be called when a page is not mapped in. Locks the translation map, Finds the region in the cache that contains this page and maps it in. Unlocks the cache.
|
||||
|
||||
vm_store *vm_store_create_device(addr base_addr)
|
||||
Allocates memory for the vm_store structure, plus the data_store_device structure. Sets cache to null, and data to the device_store_data. Sets the device_store_data's base address to the base address passed in.
|
||||
@@ -0,0 +1,20 @@
|
||||
static void null_destroy(struct vm_store *store)
|
||||
Frees this stores space.
|
||||
|
||||
static off_t null_commit(struct vm_store *store, off_t size)
|
||||
Sets committed size to size and returns it.
|
||||
|
||||
static int null_has_page(struct vm_store *store, off_t offset)
|
||||
returns 1.
|
||||
|
||||
static ssize_t null_read(struct vm_store *store, off_t offset, iovecs *vecs)
|
||||
Returns -1.
|
||||
|
||||
static ssize_t null_write(struct vm_store *store, off_t offset, iovecs *vecs)
|
||||
Returns -1.
|
||||
|
||||
static int null_fault(struct vm_store *store, struct vm_address_space *aspace, off_t offset)
|
||||
Returns a Fatal page fault error.
|
||||
|
||||
vm_store *vm_store_create_null(void)
|
||||
Creates an empty vm_store struct.
|
||||
@@ -0,0 +1,23 @@
|
||||
static void vnode_destroy(struct vm_store *store)
|
||||
Frees memory associated with this store.
|
||||
|
||||
static off_t vnode_commit(struct vm_store *store, off_t size)
|
||||
Sets committed size to size and returns it.
|
||||
|
||||
static int vnode_has_page(struct vm_store *store, off_t offset)
|
||||
Returns 1.
|
||||
|
||||
static ssize_t vnode_read(struct vm_store *store, off_t offset, iovecs *vecs)
|
||||
Calls vfs_readpage.
|
||||
|
||||
static ssize_t vnode_write(struct vm_store *store, off_t offset, iovecs *vecs)
|
||||
calls vfs_writepage
|
||||
|
||||
static void vnode_acquire_ref(struct vm_store *store)
|
||||
Calls vfs_vnode_aquire_ref
|
||||
|
||||
static void vnode_release_ref(struct vm_store *store)
|
||||
Calls vfs_vnode_release_ref
|
||||
|
||||
vm_store *vm_store_create_vnode(void *vnode)
|
||||
Creates space for a vm_store and a vnode_store_data. Sets data = vnode_store_data. Sets the vnode_store_data's vnode to vnode.
|
||||
Reference in New Issue
Block a user