OSDev.org

The Place to Start for Operating System Developers
It is currently Fri Mar 29, 2024 8:09 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 8 posts ] 
Author Message
 Post subject: How can I use INVLPG to flush TLB for a Linux process?
PostPosted: Mon Feb 18, 2019 2:01 am 
Offline

Joined: Fri Nov 17, 2017 7:02 am
Posts: 20
Hi,
I am learning X86 and Linux.
Now I am trying to figure out how to flush TLB for a Linux process.
So here comes the first question, what is the source parameter for INVLPG instruction?

My codes consists of 2 parts, a Linux kernel module and a user space process.

The user space process malloc a buffer, then pass the buffer address (in the process's virtual address) to kernel module through an IOCTL.
Kernel module will call INVLPG with that virtual address, as follows,
static inline void invlpg(unsigned long addr)
{
asm volatile("invlpg (%0)" ::"r" (addr) : "memory");
}

Then, the 2nd question comes, how can I know this TLB flushing worked?

My understanding of the effect of TLB flushing is buffer access should be slowed down, since TLB is flushed, and memory page table walking should happen (and cost more cycles).
So that I can compare the access time spent before and after the TLB flush, is it correct?

Thanks,
-tao


Top
 Profile  
 
 Post subject: Re: How can I use INVLPG to flush TLB for a Linux process?
PostPosted: Mon Feb 18, 2019 2:28 am 
Offline
Member
Member

Joined: Tue Mar 04, 2014 5:27 am
Posts: 1108
Why do you want to flush the TLB? And if you really need to, I bet the kernel already provides a function for that (perhaps, that can do that on all CPUs as well).


Top
 Profile  
 
 Post subject: Re: How can I use INVLPG to flush TLB for a Linux process?
PostPosted: Mon Feb 18, 2019 3:21 am 
Offline

Joined: Fri Nov 17, 2017 7:02 am
Posts: 20
alexfru wrote:
Why do you want to flush the TLB? And if you really need to, I bet the kernel already provides a function for that (perhaps, that can do that on all CPUs as well).

What I want to do is just to learn the INVLPG could work in my way.

You are right, Linux kernel does provide this kind of API, I will test that later (in SMP).

Thanks,


Top
 Profile  
 
 Post subject: Re: How can I use INVLPG to flush TLB for a Linux process?
PostPosted: Mon Feb 18, 2019 3:54 am 
Offline
Member
Member

Joined: Tue Mar 04, 2014 5:27 am
Posts: 1108
wangt13 wrote:
What I want to do is just to learn the INVLPG could work in my way.

That's a confusing statement (to me).

What is there to learn?
If you want to see INVLPG effects other than timing, you need to break some code by removing this instruction from it instead of adding it.

What's your way?


Top
 Profile  
 
 Post subject: Re: How can I use INVLPG to flush TLB for a Linux process?
PostPosted: Mon Feb 18, 2019 4:23 am 
Offline

Joined: Mon Nov 26, 2018 9:14 am
Posts: 8
wangt13 wrote:
What I want to do is just to learn the INVLPG could work in my way.


INVLPG is a privileged instruction. Will not work on ring 3 (userland).


Top
 Profile  
 
 Post subject: Re: How can I use INVLPG to flush TLB for a Linux process?
PostPosted: Mon Feb 18, 2019 9:33 pm 
Offline

Joined: Fri Nov 17, 2017 7:02 am
Posts: 20
alexfru wrote:
wangt13 wrote:
What I want to do is just to learn the INVLPG could work in my way.

That's a confusing statement (to me).

What is there to learn?
If you want to see INVLPG effects other than timing, you need to break some code by removing this instruction from it instead of adding it.

What's your way?


Let me provide more details I want to learn.

INVLPG needs a parameter, I think it should be virtual address.
Now, I want to learn first is what is the virtual address?
It means, in which context the virtual address is, kernel space, or user space?

For example, tmp = malloc(0x1000) in user space, and to flush the TLB for that PFN translation, what should do?
INVLPG is a privileged inst. so it needs to tell kernel to do that, so is it OK to pass the tmp directly to the kernel (to the INVLPG).
If so, I think it should be in the process context, that is the CR3 rooted tree.

And my test showed this worked.

If I am wrong, please correct me.

Thanks,


Top
 Profile  
 
 Post subject: Re: How can I use INVLPG to flush TLB for a Linux process?
PostPosted: Mon Feb 18, 2019 10:56 pm 
Offline
Member
Member

Joined: Tue Mar 04, 2014 5:27 am
Posts: 1108
wangt13 wrote:
INVLPG needs a parameter, I think it should be virtual address.


Right. Did you know you could download CPU documentation for free? E.g. here.

wangt13 wrote:
Now, I want to learn first is what is the virtual address?

It's explained in the documentation and other places, e.g. Wikipedia.

wangt13 wrote:
It means, in which context the virtual address is, kernel space, or user space?

The context is the current address space. It may have only kernel-accessible pages or those and user-accessible ones. INVLPG doesn't care which of the two the address points to.

wangt13 wrote:
to flush the TLB for that PFN translation, what should do?
INVLPG is a privileged inst. so it needs to tell kernel to do that, so is it OK to pass the tmp directly to the kernel (to the INVLPG).

That's pretty much the only way to do it (reloading CR3 would flush the entire TLB (except the so-called global pages), not just a single page; there's also INVPCID in newer CPUs, but to keep things simple let's not go there).

wangt13 wrote:
If so, I think it should be in the process context, that is the CR3 rooted tree.

When you switch the process context, you switch the address space (and change CR3). INVLPG affects the current address space.


Top
 Profile  
 
 Post subject: Re: How can I use INVLPG to flush TLB for a Linux process?
PostPosted: Wed Feb 20, 2019 7:16 pm 
Offline

Joined: Fri Nov 17, 2017 7:02 am
Posts: 20
[quote="alexfru"][quote="wangt13"]
Thank you for the detailed reply.

Thanks,


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 8 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: KN9296 and 188 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group