OSDev.org

The Place to Start for Operating System Developers
It is currently Mon Jan 30, 2023 5:57 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 7 posts ] 
Author Message
 Post subject: Remapping pages vs. memcpy
PostPosted: Fri Nov 18, 2022 2:29 pm 
Offline

Joined: Fri Nov 18, 2022 1:48 pm
Posts: 3
Is it feasible to use the paging machinery to remap the memory of a process? The alternative is copying data. 4K granularity is fine.

A few years ago I came across a discussion somewhere about this, in the context of a Linux kernel. If I remember correctly, the consensus was that the overhead of kernel IPCs and possibly the side-effects (caching, TLB, etc) make copying more attractive.

Assuming the client code is tightly integrated with the OS, what do you think? Are there additional issues with moving code around this way (other than the obvious need to fix-up dangling pointers)?

Thank you for your time.


Top
 Profile  
 
 Post subject: Re: Remapping pages vs. memcpy
PostPosted: Sun Nov 27, 2022 7:43 pm 
Online
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 4504
stack wrote:
Is it feasible to use the paging machinery to remap the memory of a process?

There's some point where moving larger amounts of memory will be faster using remapping instead of memcpy. That point moves depending on how well you optimize each of those functions.

stack wrote:
Assuming the client code is tightly integrated with the OS, what do you think?

There are optimizations that may not be possible in the general case but significantly reduce overhead when they are possible. For example, you can get rid of most of the TLB flushing overhead when the client or the CPU is single-threaded. A client that's aware of these situations can dynamically choose whether to call memcpy or to call the OS to remap pages.

stack wrote:
Are there additional issues with moving code around this way (other than the obvious need to fix-up dangling pointers)?

Code is usually not designed to be moved at runtime. You can move it when you load it, but why not load it at your desired address in the first place?


Top
 Profile  
 
 Post subject: Re: Remapping pages vs. memcpy
PostPosted: Sun Nov 27, 2022 10:53 pm 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1420
Creating a second mapping of the same physical memory has different semantics from having an independent copy. You'd need to implement copy-on-write semantics when doing that, and that means you suddenly get into the realm of TLB shootdowns when implementing this (since you need to revoke the write permission on at least the source page(s)), and that massively tips the scales towards just copying the memory.

Not to mention that it is possible that the application can be optimized not to require a giant memcpy().

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: Remapping pages vs. memcpy
PostPosted: Mon Dec 05, 2022 2:17 pm 
Offline

Joined: Fri Nov 18, 2022 1:48 pm
Posts: 3
Octocontrabass wrote:
Code is usually not designed to be moved at runtime. You can move it when you load it, but why not load it at your desired address in the first place?

My project involves code and data which can be garbage collected. Relocation and fixup require some finesse, but are not too much of an issue. I am investigating the possibility of using remapping to reclaim memory, something which would seem to require cooperation from the OS.


Top
 Profile  
 
 Post subject: Re: Remapping pages vs. memcpy
PostPosted: Sat Dec 10, 2022 3:35 am 
Offline

Joined: Fri Nov 26, 2021 11:08 am
Posts: 7
The Azul JVM does/did this.


Top
 Profile  
 
 Post subject: Re: Remapping pages vs. memcpy
PostPosted: Sun Jan 15, 2023 9:56 am 
Offline

Joined: Sun Jan 15, 2023 4:02 am
Posts: 2
From https://en.wikipedia.org/wiki/Accent_kernel:
Quote:
In 1979 one of the Aleph engineers, Richard Rashid, left for CMU and started work on a new version of Aleph that avoided its problems. In particular, Accent targeted workstation machines featuring a MMU, using the MMU to "copy" large blocks of memory via mapping, making the memory appear to be in two different places. Only data that was changed by one program or another would have to be physically copied, using the copy-on-write algorithm.


So some kind of memory protection mechanism must be involved here.

I'm experimenting on persistent data structure to replace MMU in some degree by structural sharing, but it makes the data very hard to debug. Plus, it seems like with MMU I still can't do mmap for persistent data structures where its pointers/leaves stayed on so I still have to rethink about it in some way.


Top
 Profile  
 
 Post subject: Re: Remapping pages vs. memcpy
PostPosted: Mon Jan 30, 2023 9:43 am 
Offline
Member
Member
User avatar

Joined: Mon Jun 05, 2006 11:00 pm
Posts: 2261
Location: USA (and Australia)
I'm doing this for my RPC.

My RPC has 'messages' and 'mini-messages'. Messages are arbitary sized and aligned to physical pages, and sending them to another process 'gifts' them the page. Mini-messages are fixed size messages that can fit in 4 64-bit registers (32-bytes or less).

A string to the VFS saying to open this file on this path is a arbitary sized message. A 2d vector of which direction the mouse moved is a mini-message.

Arbitrary sized messages can be as large as they want, but the most common size will be 4KiB, so I want to focus on optimizing these single page messages. One optimization: in each user process when we 'free' pages, I keep a pool of recently freed pages without actually releasing them to the OS.

_________________
My OS is Perception.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group