OSDev.org

The Place to Start for Operating System Developers
It is currently Tue Mar 19, 2024 2:18 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 5 posts ] 
Author Message
 Post subject: QEMU/KVM internals
PostPosted: Tue Oct 12, 2021 11:27 am 
Offline
Member
Member

Joined: Sat Dec 22, 2012 12:01 pm
Posts: 73
Hi,

I'd like to ask some details about QEMU/KVM implementation. On my Linux server (dual socket CPU w/ 12 core/24 HT each) I run a VM via qemu-system-x86_64:
Code:
root@eve-ng03:~# numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 24 25 26 27 28 29 30 31 32 33 34 35
node 0 size: 64180 MB
node 0 free: 22819 MB
node 1 cpus: 12 13 14 15 16 17 18 19 20 21 22 23 36 37 38 39 40 41 42 43 44 45 46 47
node 1 size: 64491 MB
node 1 free: 51078 MB
node distances:
node   0   1
  0:  10  21
  1:  21  10
root@eve-ng03:~#
root@eve-ng03:~#
root@eve-ng03:~# ps -wwf -p 45802
UID        PID  PPID  C STIME TTY          TIME CMD
root     45802 45786 99 Jun18 ?        153-02:15:51 /opt/qemu/bin/qemu-system-x86_64 -nographic -device virtio-net-pci,addr=3.0,multifunction=on,netdev=net0,mac=50:00:00:07:00:00 -netdev tap,id=net0,ifname=vunl0_7_0,script=no -device virtio-net-pci,addr=3.1,multifunction=on,netdev=net1,mac=50:00:00:07:00:01 -netdev tap,id=net1,ifname=vunl0_7_1,script=no -device virtio-net-pci,addr=3.2,multifunction=on,netdev=net2,mac=50:00:00:07:00:02 -netdev tap,id=net2,ifname=vunl0_7_2,script=no -device virtio-net-pci,addr=3.3,multifunction=on,netdev=net3,mac=50:00:00:07:00:03 -netdev tap,id=net3,ifname=vunl0_7_3,script=no -device virtio-net-pci,addr=3.4,multifunction=on,netdev=net4,mac=50:00:00:07:00:04 -netdev tap,id=net4,ifname=vunl0_7_4,script=no -device virtio-net-pci,addr=3.5,multifunction=on,netdev=net5,mac=50:00:00:07:00:05 -netdev tap,id=net5,ifname=vunl0_7_5,script=no -device virtio-net-pci,addr=3.6,multifunction=on,netdev=net6,mac=50:00:00:07:00:06 -netdev tap,id=net6,ifname=vunl0_7_6,script=no -device virtio-net-pci,addr=3.7,multifunction=on,netdev=net7,mac=50:00:00:07:00:07 -netdev tap,id=net7,ifname=vunl0_7_7,script=no -device virtio-net-pci,addr=4.0,multifunction=on,netdev=net8,mac=50:00:00:07:00:08 -netdev tap,id=net8,ifname=vunl0_7_8,script=no -device virtio-net-pci,addr=4.1,multifunction=on,netdev=net9,mac=50:00:00:07:00:09 -netdev tap,id=net9,ifname=vunl0_7_9,script=no -device virtio-net-pci,addr=4.2,multifunction=on,netdev=net10,mac=50:00:00:07:00:0a -netdev tap,id=net10,ifname=vunl0_7_10,script=no -smp 4 -m 8192 -name R_02 -uuid a368f2e7-eebf-418b-9ffd-5c190134a670 -drive file=virtioa.qcow2,if=virtio,bus=0,unit=0,cache=none -enable-kvm -smbios type=1,manufacturer=cisco,product=Cisco IOS XRv 9000,uuid=97fc351b-431d-4cf2-9c01-43c283faf2a3 -machine type=pc,accel=kvm,usb=off -serial mon:stdio -nographic -nodefconfig -nodefaults -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -realtime mlock=off -no-shutdown -boot order=c -serial null -serial null -serial null -cpu host
root@eve-ng03:~#
root@eve-ng03:~#
root@eve-ng03:~#
root@eve-ng03:~# ps -T -p 45802
  PID  SPID TTY          TIME CMD
45802 45802 ?        14-05:23:18 qemu-system-x86
45802   812 ?        00:00:00 qemu-system-x86
45802   824 ?        7-07:15:51 qemu-system-x86
45802   825 ?        7-06:48:23 qemu-system-x86
45802   826 ?        8-04:20:31 qemu-system-x86
45802   827 ?        116-01:07:17 qemu-system-x86
45802   658 ?        00:00:00 qemu-system-x86
45802   678 ?        00:00:00 qemu-system-x86
45802   681 ?        00:00:00 qemu-system-x86
45802   733 ?        00:00:00 qemu-system-x86
45802   734 ?        00:00:00 qemu-system-x86
45802   761 ?        00:00:00 qemu-system-x86
root@eve-ng03:~#

As you can see there are 12 threads that run inside the QEMU process (PID 45802). Digging inside qemu monitor (Ctrl+A C)
Code:
QEMU 2.4.0 monitor - type 'help' for more information
(qemu) info cpus
* CPU #0: pc=0xffffffff8d0454e6 (halted) thread_id=824
  CPU #1: pc=0xffffffff8d0454e6 (halted) thread_id=825
  CPU #2: pc=0xffffffff8d0454e6 (halted) thread_id=826
  CPU #3: pc=0x00007f7b8a64f3ff thread_id=827
(qemu)
I can see the four vCPU threads (as expected since the VM started with 4 vCPUs) and the main thread -- it should be the thread having Thread-ID (SPID) the same as the Process-ID (45802).

My question is: which are actually the tasks in charge of all other (non main or vCPUs) threads inside the QEMU process ? Thank you.


Top
 Profile  
 
 Post subject: Re: QEMU/KVM internals
PostPosted: Thu Oct 14, 2021 11:21 am 
Offline
Member
Member

Joined: Sat Dec 22, 2012 12:01 pm
Posts: 73
Any feedback ? Thank you.


Top
 Profile  
 
 Post subject: Re: QEMU/KVM internals
PostPosted: Thu Oct 14, 2021 11:49 am 
Offline
Member
Member

Joined: Fri Oct 04, 2019 10:10 am
Posts: 48
A random, mostly uneducated guess is that QEMU spawns up a thread for every physical CPU no matter what, and allocates those as needed to the VM. The other 8 all having a time of 00:00:00 suggests they all immediately went idle.

If you spawned up another 4 CPU VM, you'd probably see another 4 wake up.


Top
 Profile  
 
 Post subject: Re: QEMU/KVM internals
PostPosted: Thu Oct 14, 2021 2:11 pm 
Offline
Member
Member

Joined: Sat Dec 22, 2012 12:01 pm
Posts: 73
reapersms wrote:
A random, mostly uneducated guess is that QEMU spawns up a thread for every physical CPU no matter what, and allocates those as needed to the VM.

I take it as QEMU spawns up 12 threads since there are 12 core per socket on my system. Then 8 of them immediately go idle.

Does it make sense ?


Top
 Profile  
 
 Post subject: Re: QEMU/KVM internals
PostPosted: Thu Oct 14, 2021 2:40 pm 
Offline
Member
Member

Joined: Fri Oct 04, 2019 10:10 am
Posts: 48
Yes.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 5 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: songziming and 6 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group