OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Nov 21, 2019 5:42 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 6 posts ] 
Author Message
 Post subject: vSphere ESXi large page - EPT behaviour
PostPosted: Wed Apr 29, 2015 7:25 am 
Offline
Member
Member

Joined: Sat Dec 22, 2012 12:01 pm
Posts: 58
Hi,
I know this may not be the right place but I have found no answer elsewhere...

I am deploying a virtual lab based on vSphere ESXi 5.5 running on HP server with dual Intel Xeon E5620 processor supporting EPT (Extended Page Table) technology.
From vmware kb documentation, in "normal" condition ESXi 5.5 running on processor supporting EPT (or AMD RVI) make use of host large page (2MB) to back VM's guest physical pages (guest OS is unaware of that and can use 4KB or even 2MB as page size width)

Now, AFAIK, just when ESXi host is under memory pressure large pages are broken into 4k pages to allow Transparent Page Sharing (TPS) to work.

Now my doubt is: when we say "large pages are broken into small page to allow TPS to work" actually means the (EPT) address translation scheme is changed from now on and ESXi VMkernel has to build up on the fly the page tables needed to support the last level of page walk required by the new (EPT) translation scheme ?

thanks :)


Top
 Profile  
 
 Post subject: Re: vSphere ESXi large page - EPT behaviour
PostPosted: Thu Apr 30, 2015 2:39 am 
Offline
Member
Member
User avatar

Joined: Wed Oct 18, 2006 3:45 am
Posts: 9287
Location: On the balcony, watching the Swedish Chef
Quote:
ESXi host is under memory pressure large pages are broken into 4k pages

If a system has been running for a while and getting low on memory, there will be less contiguous chunks of memory because intermittent frees leave relatively small holes. At some point, fragmentation prevents whole 2MB pieces to be no longer available and the system has to hand out the next smaller size at 4K.

In addition, when running several VMs, the chance that you find the same 4K page in more than one VM is significantly larger than finding an identical 2M page. For that reason, if you want to reclaim memory from VMs by sharing, you might want to check on a 4K basis and arrange the backing page tables on a 4K basis, even if you do have 2M pages available. This includes removing 2M page entries and replacing them with a page table and a series of 4K pages of which you can share a few between VMs.

None of this should actually have to alter guest states.

_________________
"Certainly avoid yourself. He is a newbie and might not realize it. You'll hate his code deeply a few years down the road." - Sortie
[ My OS ] [ VDisk/SFS ]


Top
 Profile  
 
 Post subject: Re: vSphere ESXi large page - EPT behaviour
PostPosted: Mon May 04, 2015 1:13 am 
Offline
Member
Member

Joined: Sat Dec 22, 2012 12:01 pm
Posts: 58
Thanks.

Have you an idea how to check if hypervisor is making use of large page to back guest OS pages ?


Top
 Profile  
 
 Post subject: Re: vSphere ESXi large page - EPT behaviour
PostPosted: Mon May 04, 2015 1:56 am 
Offline
Member
Member
User avatar

Joined: Wed Oct 18, 2006 3:45 am
Posts: 9287
Location: On the balcony, watching the Swedish Chef
A version of "Cache missing for fun and profit" would be the most VM-agnostic way to do it. There might be MSRs or VM specific methods of querying though that can prove to be more helpful, but because all of these are basically side-channel attacks on the host you might find them blocked or otherwise impaired.

The more important question is however, do you even need to know how the VM maps guest to host memory? A VM is meant to work transparently after all.

_________________
"Certainly avoid yourself. He is a newbie and might not realize it. You'll hate his code deeply a few years down the road." - Sortie
[ My OS ] [ VDisk/SFS ]


Top
 Profile  
 
 Post subject: Re: vSphere ESXi large page - EPT behaviour
PostPosted: Tue May 12, 2015 6:39 am 
Offline
Member
Member

Joined: Sat Dec 22, 2012 12:01 pm
Posts: 58
A related question: can a VMM/Hypervisor force guest OS running into a VM to NOT make use of Large Page ? Previously in this thread I assumed is not possible because guest OS can independently choose the page size to use...

Does it make sense ?


Top
 Profile  
 
 Post subject: Re: vSphere ESXi large page - EPT behaviour
PostPosted: Tue May 12, 2015 7:22 am 
Offline
Member
Member
User avatar

Joined: Wed Oct 18, 2006 3:45 am
Posts: 9287
Location: On the balcony, watching the Swedish Chef
The VM normally shouldn't care what page size the client uses.

That said, you can prevent large pages to some extent, by faking that PSE/PSE36/PAE doesn't exist:
AMD SVM Manual wrote:
Various instructions and events (such as exceptions) in the
guest can be intercepted by means of control bits in the VMCB.
The two primary classes of intercepts supported by SVM are
instruction and exception intercepts.
(...)
CPUID
(...)
MOV to/from CRn
This is completely ineffective for 64-bit guests as long mode guarantees large page support.

_________________
"Certainly avoid yourself. He is a newbie and might not realize it. You'll hate his code deeply a few years down the road." - Sortie
[ My OS ] [ VDisk/SFS ]


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 6 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group