OSDev.org

The Place to Start for Operating System Developers
It is currently Fri Mar 29, 2024 3:05 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 19 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: VESA standart PMode programming
PostPosted: Fri Mar 02, 2018 5:15 pm 
Offline
Member
Member

Joined: Sat Sep 24, 2016 12:06 am
Posts: 90
Hi everyone, sorry for my bad English, im from Russia. So, i want to develop VBE-like VESA driver for my OS, because i want make OS almost compatible with all types of x86-64 PCs. Can i(maybe, trought IO ports) set video modes/swap video pages and other WITHOUT BIOS calls? Because if u have a Intel DX/2 PC and 8 megabytes of RAM u do not want to use (in 800x600x(2^24)) about 1.5 mbytes of RAM to flat video mem. But if can, i need more than 64kb pages, may be 128kb or 256kb can i set size? In 64kb page you can store about 21.(3) thouthands of pixels, on screen u have 480 thouthand pixels. anyone can advise literature about this topic?


Top
 Profile  
 
 Post subject: Re: VESA standart PMode programming
PostPosted: Fri Mar 02, 2018 7:44 pm 
Offline
Member
Member

Joined: Sat Sep 24, 2016 12:06 am
Posts: 90
So, i has found a archive with SVGA/VGA implementations for some graphics cards. There are: http://bos.asmhackers.net/docs/vga/vgadoc4b.zip
"For any masochists who want to try writing SVGA code directly at the hardware level rather than using VESA, this contains detailed information about most of the common chipsets."


Top
 Profile  
 
 Post subject: Re: VESA standart PMode programming
PostPosted: Sat Mar 03, 2018 12:05 pm 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

MrLolthe1st wrote:
Hi everyone, sorry for my bad English, im from Russia. So, i want to develop VBE-like VESA driver for my OS, because i want make OS almost compatible with all types of x86-64 PCs.


There are only 3 choices:
  • Use BIOS and VBE (only works on systems that have BIOS)
  • Use UEFI (only works on systems that have UEFI)
  • Write a different native video driver for each different video card (which only works if you've written a video driver for the video card)

None of these options work on almost all x86-64 PCs by themselves; but it's multiple choice. For example, you can have a "BIOS boot loader" that uses VBE to setup a video mode and a "UEFI boot loader" that uses UEFI to setup a video mode and native video drivers for some video cards.

MrLolthe1st wrote:
Can i(maybe, trought IO ports) set video modes/swap video pages and other WITHOUT BIOS calls?


Yes, that's the "different native video driver for each different video card" option. There's about 500 different video cards and (if you can obtain documentation, which is considerably painful in a lot of cases) each driver will probably take an experienced developer an average of 2 weeks to get a minimal driver (e.g. video mode switches with no support for GPU or hardware acceleration) working and properly tested; so this will take about 20 years of work. However during that 20 years another 500 video cards will be released (and you won't support any of them until you spend another 20 years writing video drivers) so you'll end up continually writing minimal video drivers forever and you'll probably never support most x86-64 PCs bcause you're always missing about 500 video drivers.

MrLolthe1st wrote:
Because if u have a Intel DX/2 PC and 8 megabytes of RAM u do not want to use (in 800x600x(2^24)) about 1.5 mbytes of RAM to flat video mem.


Regardless of whether you're using BIOS or UEFI or native drivers; you mostly end up with a list of video modes that the video card can support. Typically you start by filtering this list - removing video modes that the OS doesn't support, and removing video modes that the monitor doesn't support. Then you select a video mode from whatever is left. This selection can involve giving each video mode a rating and then selecting the video mode with the best rating; where the rating can depend on whatever you like, including things like if there's enough RAM left over.

Note that there's "system RAM" and "video RAM"; and the video mode itself will use 0 bytes of "system RAM" (and will only use "video RAM"). The system RAM is used by code that generates data to put into the frame buffer (in "video RAM"); and this varies significantly depending on how you do it.

MrLolthe1st wrote:
But if can, i need more than 64kb pages, may be 128kb or 256kb can i set size? In 64kb page you can store about 21.(3) thouthands of pixels, on screen u have 480 thouthand pixels. anyone can advise literature about this topic?


The "legacy display windows" (which are normally two 64 KiB areas, one at 0x000A0000 and another at 0x000A0000) are obsolete trash from about 30 years ago (and are only supported for backward compatibility). Everything slightly modern uses a linear frame buffer (LFB) instead, where you'll have a large area at a much higher address (e.g. 16 MiB starting at the physical address 0xE0000000) that you can use to directly access the entire frame buffer. For VBE (which is also quite old and is effected by backward compatibility) you have to ask for LFB when setting a video mode, and it will tell you the address to use to access the frame buffer.

MrLolthe1st wrote:
So, i has found a archive with SVGA/VGA implementations for some graphics cards. There are: http://bos.asmhackers.net/docs/vga/vgadoc4b.zip
"For any masochists who want to try writing SVGA code directly at the hardware level rather than using VESA, this contains detailed information about most of the common chipsets."


That contains information for SVGA video cards that were common in the early 1990s. These video cards are extremely rare now and everything has changed (e.g. the old "fixed function accelerators" have been replaced by GPUs) so the information that "vgadoc4b.zip" contains is relatively useless.


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: VESA standart PMode programming
PostPosted: Mon Mar 05, 2018 3:46 am 
Offline
Member
Member

Joined: Tue May 13, 2014 3:02 am
Posts: 280
Location: Private, UK
Brendan wrote:
Use BIOS and VBE (only works on systems that have BIOS)


Technically, that's not quite true. The only thing that's requred for VBE is a video BIOS. A system BIOS/CSM amd/or BIOS-style boot is not required. Several operating systems (including Linux and Windows NT) have used a "BIOS emulator" to initialise and run Video BIOS code on non-BIOS systems (including non-x86 systems). The process of initialising "option ROMs" on PCI devices is fairly well documented.

As for writing "native" driver; there are only 3 GPU families that are really worth caring about these days; Intel, NVidia and AMD/ATI. Having VBE and/or GOP support is useful as a fallback and/or first prototype.

Thankfully, if all you're (initiallly) interested in is unaccellerated 2D, the basic 2D "framebuffer" interfaces of these families are fairly consistent and half a dozen or so drivers/variations can support most recent hardware. Linux, for example, supports all these families in just 4 "fbdev" drivers (one for older i810 Intel, one for newer Intel, one for NVidia and one for ATI/AMD; although this does contain seperate code for older "Mach64" and newer "Radeon" ATI/AMD GPUs); some of them even support a limited amount of 2D acelleration (bitblits, mostly). Note that these "fbdev" drivers are entirely separate from the "Direct Rendering" drivers used for 3D acelleration.

Of course, the biggest issue with developing such "native" drivers for a hobby OS isn't the difficulty of writing them, it's getting access to the hardware to test them... That gets very expensive rather quickly.

_________________
Image


Top
 Profile  
 
 Post subject: Re: VESA standart PMode programming
PostPosted: Mon Mar 05, 2018 11:01 am 
Offline
Member
Member

Joined: Sat Sep 24, 2016 12:06 am
Posts: 90
Thanks at all. Im will use VBE from BIOS. So, how i can increase graphics perfomace? One full update of screen needs in CPU time without GPU. Are there some methods to dont use a lot of CPU time for drawing at screen? In 1024x768x3bytes resolution i5-4440 have only 7 frames per second(direct memory access, without any computations, only one color), but with using QEMU emulator. Im hear, that i need to use shaders, and if i using(im wanna to write "if i will use", how correct?) them i can only call methods, GPU will compute it all and draw something at screen. Or how you drawing on screen?


Top
 Profile  
 
 Post subject: Re: VESA standart PMode programming
PostPosted: Mon Mar 05, 2018 3:16 pm 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

mallard wrote:
Brendan wrote:
Use BIOS and VBE (only works on systems that have BIOS)


Technically, that's not quite true. The only thing that's requred for VBE is a video BIOS. A system BIOS/CSM amd/or BIOS-style boot is not required. Several operating systems (including Linux and Windows NT) have used a "BIOS emulator" to initialise and run Video BIOS code on non-BIOS systems (including non-x86 systems). The process of initialising "option ROMs" on PCI devices is fairly well documented.


Let's assume that the video card's ROM is designed to use VGA compatible IO ports (which unlike memory mapped IO can be accessed in real mode) and that there may be multiple video cards where only one can respond to VGA compatible IO ports and the others cannot. This is enough (without looking at all of the other problems, like "UEFI only" ROMs, etc) to make it obvious that relying on this in a video card driver is incredibly stupid.

Let's also assume that ATI/AMD cards have interpreted byte-code for things like early initialisation (and possibly even video mode setting - I haven't found adequate documentation and I'm not sure exactly what functionality AtomBIOS does/doesn't provide), and that this also happens to be in ROM. This is enough to make it obvious that there are other reasons (that have nothing to do with VESA/VBE) why a video driver might want to access the card's ROM.

mallard wrote:
As for writing "native" driver; there are only 3 GPU families that are really worth caring about these days; Intel, NVidia and AMD/ATI.


That depends a little on the target market for the OS. If the OS is intended for servers I'd add Aspeed to the list (it's built into a lot of Xeon motherboards and more likely than NVidia until you get to HPC/"GPU compute"); and if the OS is intended for small/embedded stuff I'd add S3 to the list.


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: VESA standart PMode programming
PostPosted: Mon Mar 05, 2018 9:48 pm 
Offline
Member
Member
User avatar

Joined: Fri Oct 27, 2006 9:42 am
Posts: 1925
Location: Athens, GA, USA
Brendan wrote:
Note that there's "system RAM" and "video RAM"; and the video mode itself will use 0 bytes of "system RAM" (and will only use "video RAM"). The system RAM is used by code that generates data to put into the frame buffer (in "video RAM"); and this varies significantly depending on how you do it.


Uhm, despite what most people think, discrete VRAM is the exception, not the rule - the overwhelming majority of desktop and laptop systems use CPU integrated graphics, and set aside part of the system RAM for the IGPU/APU to use.

Intel has had integrated graphics on the majority of their consumer and enthusiast CPUs for over ten years. I am not sure if any of these IGPUs can work with a separate video memory bus, but AFAIK none of them require one.

AMD sells separate product lines with and without integrated GPUs, calling the models with the 'Advanced Processing Units' (APUs). They generally don't produce APU versions of their enthusiast and high-performance lines. The latest APUs, the Ryzen 3 2200g and the Ryzen 5 2400g, just came out last month, and are still getting through their teething pains (mostly relating to motherboards which need to be updated before they will work with the integrated graphics).

As it happens, I was just trying to learn about these this past week, in preparation of a new desktop system sometime in the coming year (I hope; I probably shouldn't be spending the retroactive payment I will be getting - hopefully in the tomorrow or Wednesday - but I think I can legitimately argue to myself it on mental health grounds). I am figuring if I can be certain that the motherboard will be an updated one - which should be possible by, say, early May, which is when I mean to do this if I can - which would give me a rather modern system for, I dunno, $800 US or so (including a monitor) if I am careful.

(BTW, Brendan, you might want to take a look at the TechYesCity YT channel and website on this, as he discusses the hardware market - including low-price builds - in Australia, and usually gives both USD and AUD prices he's found.)

However, the OP's question is relating to a type of system that one is unlikely to find outside of a museum today - the 486DX2 was a model which Intel sold briefly in 1992 to buy breathing space to get all the bugs (well, except the FDIV bug :-P) out of the original Pentium. While I have seen 20 year old PC systems still in regular use today, I don't think I have seen a 25 year old one that anyone other than a vintage hardware enthusiast was using regularly.

Sit down, I have a bit of a history lesson for all of you.

Now, in the PS/2 Video Graphics Array and the ISA/EISA cards meant to copy it, the video memory was dedicated... sort of. VGA used the 64KiB set aside in the original 5150 PC memory map (from 1981, which keep in mind was designed to support four text-mode video pages using an MDA, or two graphics pages on a CGA, neither of which exceeded 16KiB - they had the foresight to expect that this would need to be extended later, which obviously EGA and VGA did, as did things like Hercules, Tandy, and MCGA graphics), whose highest resolution modes was 640x480, 4-bit (16 colors), and highest color mode was 320x200, 8-bit (256 colors).

By late 1990, when 80486DX mobos and CPUs started to come out in notable numbers, VGA was pretty much the baseline, and early SVGA (original 800x600, 4-bit) were pretty common, with SVGA2 (1024x768, 8-bit) cards being the high-end cards (less for gaming than for AutoCAD, as most PC games only supported up to VGA, and often supported multiple older video models). When running in real mode, these continued to use the existing 64 KiB mapped memory space by bank-switching the video memory slot; generally, this was done in the card's implementation of VBE.

However, by 1992, as SXGA (1280x1024, 8-bit on a 5:4 aspect ratio, or 1280x960, 8-bit when retaining the 4:3 aspect ratio) started to really get traction, more games were using 32-bit extenders to run bigger games, and would need to either stick to the defined VGA registers and hope the card makers were diligent, or bundle drivers for several cards.

A few years later still - 1995-97 - it became common for a game developer to issue separate versions of their existing games, optimized for one or more of the video cards which were current in vogue (which at the time usually mean either S3 or 3Dfx), but by then, no one who was running those cards would have put them in a motherboard running a DX2 (even if the could find a motherboard which socketed 486 but had a PCI slot instead of a VLBus one).

Of course, by the time of Windows 98, you had releases of both DirectX and OpenGL for Windows which were no longer quite so terrible as those in 1995-97, which made it relatively easy to write 3D graphics software for Windows PC without having to write your own modified drivers (though it would be a while still before games using either of them were particularly common). By then the 486DX2 was... well, actually, a lot offices still had them, really, usually running windows 3.1, but they were already getting phased out even by dead-broke NPOs like the one in worked for in 1999.

_________________
Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTF
Ordo OS Project
Lisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.


Last edited by Schol-R-LEA on Wed Mar 07, 2018 7:18 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: VESA standart PMode programming
PostPosted: Tue Mar 06, 2018 2:07 am 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

Schol-R-LEA wrote:
Brendan wrote:
Note that there's "system RAM" and "video RAM"; and the video mode itself will use 0 bytes of "system RAM" (and will only use "video RAM"). The system RAM is used by code that generates data to put into the frame buffer (in "video RAM"); and this varies significantly depending on how you do it.


Uhm, despite what most people think, discrete VRAM is the exception, not the rule - the overwhelming majority of desktop and laptop systems use CPU integrated graphics, and set aside part of the system RAM for the IGPU/APU to use.


It makes no difference if it's integrated video or discrete video. For integrated video; part of the RAM becomes "system RAM" and part becomes "video RAM". Any "video RAM" that isn't being used for video (e.g. for frame buffer) can't be taken back and used as system RAM. The memory map (e.g. provided by "int 0x15, eax=0xE820") won't change when you set a different video mode. Setting a lower resolution video mode won't give the OS more usable RAM.


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: VESA standart PMode programming
PostPosted: Tue Mar 06, 2018 4:43 am 
Offline
Member
Member

Joined: Thu May 17, 2007 1:27 pm
Posts: 999
Brendan wrote:
It makes no difference if it's integrated video or discrete video. For integrated video; part of the RAM becomes "system RAM" and part becomes "video RAM". Any "video RAM" that isn't being used for video (e.g. for frame buffer) can't be taken back and used as system RAM. The memory map (e.g. provided by "int 0x15, eax=0xE820") won't change when you set a different video mode. Setting a lower resolution video mode won't give the OS more usable RAM.


This is only true if you use the BIOS/UEFI instead of a driver. The firmware drivers typically reserve only a small amount of system RAM for video RAM¹ (which prevents them from using high resolutions and/or multi-buffering). For Intel, the initially reserved video RAM is in the low MB range while the GPU supports 2 GiB of video RAM for newer chips. A real driver can indeed change the usage of individual physical pages from system RAM to video RAM and vice-versa. For Intel cards, this is controlled by the GTT (global graphics translation table) or PPGTT (per-process graphics translation table)². You cannot, however, access video RAM via its address in system RAM (as video RAM is not cache-coherent for Intel graphics).

¹ And this initially reserved RAM indeed comes from "stolen memory" that is not reported in the memory map. See the Intel documentation for more information.
² Let that sink in, folks who want to disable paging for performance reasons: The GPU always uses paging and there is no way to disable it.

_________________
managarm: Microkernel-based OS capable of running a Wayland desktop (Discord: https://discord.gg/7WB6Ur3). My OS-dev projects: [mlibc: Portable C library for managarm, qword, Linux, Sigma, ...] [LAI: AML interpreter] [xbstrap: Build system for OS distributions].


Top
 Profile  
 
 Post subject: Re: VESA standart PMode programming
PostPosted: Tue Mar 06, 2018 6:31 pm 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

Korona wrote:
Brendan wrote:
It makes no difference if it's integrated video or discrete video. For integrated video; part of the RAM becomes "system RAM" and part becomes "video RAM". Any "video RAM" that isn't being used for video (e.g. for frame buffer) can't be taken back and used as system RAM. The memory map (e.g. provided by "int 0x15, eax=0xE820") won't change when you set a different video mode. Setting a lower resolution video mode won't give the OS more usable RAM.


This is only true if you use the BIOS/UEFI instead of a driver. The firmware drivers typically reserve only a small amount of system RAM for video RAM¹ (which prevents them from using high resolutions and/or multi-buffering). For Intel, the initially reserved video RAM is in the low MB range while the GPU supports 2 GiB of video RAM for newer chips. A real driver can indeed change the usage of individual physical pages from system RAM to video RAM and vice-versa. For Intel cards, this is controlled by the GTT (global graphics translation table) or PPGTT (per-process graphics translation table)². You cannot, however, access video RAM via its address in system RAM (as video RAM is not cache-coherent for Intel graphics).

¹ And this initially reserved RAM indeed comes from "stolen memory" that is not reported in the memory map. See the Intel documentation for more information.
² Let that sink in, folks who want to disable paging for performance reasons: The GPU always uses paging and there is no way to disable it.


As far as I understand it; no, that's not how it works at all.

First, assume that all video cards (even ancient CGA, etc) have "video output control circuitry", which mostly consists of a clock source and some comparators, and exists to send data from a frame buffer to the monitor.

Now imagine "hypothetical integrated video with no GPU at all" - just the "video output control circuitry" and nothing else. In this case firmware would steal some RAM to use as "video memory" (e.g. frame buffer) and would configure the memory controller (and the MTTRs in each CPU) to make this stolen memory completely inaccessible by the CPU and avoid the need for any cache coherency for the stolen memory; but then the video card would make part of that memory accessible via. memory mapped IO (e.g. a normal BAR in PCI configuration space) so that CPU can access it indirectly in an "abnormal, not cache coherent, part of the device and not part of system RAM" way (for LFB). Of course to support multiple displays you might have duplicated "video output control circuitry" each with its own part of stolen memory for its own frame buffer.

Now imagine we add a GPU to this, where the GPU is completely independent from that "video output control circuitry". Assume that the GPU's physical address space has all the stolen memory in one zone and (for convenience) has all of system RAM in another zone, such that the highest bit of a "GPU physical address" selects which zone to look at (note: I think Intel GPU's use base registers instead of "highest bit" but the difference isn't important). Next; assume that the GPU includes its own paging, which does nothing more than convert "GPU virtual addresses" into "GPU physical addresses. In other words; the GPU's paging tables do not effect what is/isn't stolen at all.

Of course the GPU's paging tables have to go somewhere, and it'd be nice (to avoid cache coherency for things like textures) to have some "internal" RAM for the GPU; so let's just steal a lot more memory - let's steal 1 GiB of memory (where a few hundred KiB is used for GPU page tables, several MiB is used for framebuffers, and the remainder is used for "internal" GPU RAM). Again; the GPU's paging still does nothing more than convert "GPU virtual addresses" into "GPU physical addresses; and the GPU's paging tables do not effect what is/isn't stolen at all.

In theory; an OS might (or might not - Intel does use "write once" registers for some things in the memory controller) be able to reconfigure the memory controllers and MTTRs and the video controller (and the GPU's paging tables) to change how much RAM is/isn't stolen; but this would be extremely invasive, isn't something I'd expect any OS actually does, isn't something that a video driver should ever do, and (if the OS has "motherboard drivers" capable messing with memory controllers during boot) probably isn't something that a full "motherboard driver" should do either.


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: VESA standart PMode programming
PostPosted: Wed Mar 07, 2018 2:34 am 
Offline
Member
Member

Joined: Thu May 17, 2007 1:27 pm
Posts: 999
Brendan wrote:
[...]Next; assume that the GPU includes its own paging, which does nothing more than convert "GPU virtual addresses" into "GPU physical addresses. In other words; the GPU's paging tables do not effect what is/isn't stolen at all.

Of course the GPU's paging tables have to go somewhere, and it'd be nice (to avoid cache coherency for things like textures) to have some "internal" RAM for the GPU; so let's just steal a lot more memory - let's steal 1 GiB of memory (where a few hundred KiB is used for GPU page tables, several MiB is used for framebuffers, and the remainder is used for "internal" GPU RAM). Again; the GPU's paging still does nothing more than convert "GPU virtual addresses" into "GPU physical addresses; and the GPU's paging tables do not effect what is/isn't stolen at all.

No, it does not work that way (at least not for Intel and for those versions of Intel GPUs that I've written a driver for).

The way it works is as follows: There are two kinds of GPU page tables: The GGTT (this is global) and the PPGTT (exists once per process that wants to use the GPU). Display pipes (i.e. the hardware component that reads the framebuffers and emits VGA/HDMI/whatever signals) can only access the GGTT while the command units can access both the GGTT and the PPGTT. As a diagram, it looks like this:
Code:
  +--------+ +--------+ +----------------+
  |        | |        | |                |
  | Pipe 1 | | Pipe 2 | | Command engine |
  |        | |        | |                |
  +--------+ +--------+ +----------------+
     |          |          |        |
     |          |          |        |
     |          |          |        |
     |          |          |        |
+----v----------v----------v--+ +---v---+
|                             | |       |
| GGTT                        | | PPGTT |
|                             | |       |
+-----------------------------+ +-------+


Now, both the GGTT and the PPGTT contain a "mapping type" that specifies if the physical 4096-byte page that is referenced by an entry is cache-coherent (System Memory in Intel's terminology) or non-cache-coherent (Main Memory in Intel's terminology) memory. The OS can freely convert entries between those mapping types with the following caveats:
  • Some parts of the graphics unit need non-coherent memory. For example, this is true for the display pipes. You cannot display a framebuffer from coherent memory.
  • Before you convert memory to non-coherent you have to make sure that it is not in the CPU cache; you need to issue cache flushes as necessary and afterwards map the page as write-combining.
  • There is a PCI BAR that allows access to the non-coherent pages from the GPU's point of view. The BAR allows some other tricks that make dealing with graphics memory more convenient. For example, access through the BAR can transparently de-tile texture memory.

Initially, a small region¹ of the GGTT is filled with non-cache-coherent pages from stolen memory (i.e. memory that is not reported in the memory map). The GGTT itself also resides in stolen memory. All other entries of the GGTT point to a zero page and there is no PPGTT at boot. Thus, if the OS wants to use lots of framebuffers, it has to add non-cache-coherent memory to the GGTT.

¹ Lets say 16 MiB, the real size depends on the BIOS configuration (i.e. it can be configured by a bit in the memory controller until that bit is locked down).

_________________
managarm: Microkernel-based OS capable of running a Wayland desktop (Discord: https://discord.gg/7WB6Ur3). My OS-dev projects: [mlibc: Portable C library for managarm, qword, Linux, Sigma, ...] [LAI: AML interpreter] [xbstrap: Build system for OS distributions].


Last edited by Korona on Wed Mar 07, 2018 7:09 am, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: VESA standart PMode programming
PostPosted: Wed Mar 07, 2018 6:57 am 
Offline
Member
Member
User avatar

Joined: Tue Mar 06, 2007 11:17 am
Posts: 1225
Korona wrote:
Thus, if the OS wants to use lots of framebuffers, it has to add non-cache-coherent memory to the
GGTT ... BAR... ???

This sentence is incomplete, maybe you were thinking where the OS should add the non-cache-coherent memory, maybe the kernel can do so interchangeably to a degree.

_________________
Live PC 1: Image Live PC 2: Image

YouTube:
http://youtube.com/@AltComp126/streams
http://youtube.com/@proyectos/streams

http://master.dl.sourceforge.net/projec ... 7z?viasf=1


Top
 Profile  
 
 Post subject: Re: VESA standart PMode programming
PostPosted: Wed Mar 07, 2018 7:09 am 
Offline
Member
Member

Joined: Thu May 17, 2007 1:27 pm
Posts: 999
Yeah, thank you. I fixed that sentence.

_________________
managarm: Microkernel-based OS capable of running a Wayland desktop (Discord: https://discord.gg/7WB6Ur3). My OS-dev projects: [mlibc: Portable C library for managarm, qword, Linux, Sigma, ...] [LAI: AML interpreter] [xbstrap: Build system for OS distributions].


Top
 Profile  
 
 Post subject: Re: VESA standart PMode programming
PostPosted: Wed Mar 07, 2018 8:37 am 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

Korona wrote:
Brendan wrote:
[...]Next; assume that the GPU includes its own paging, which does nothing more than convert "GPU virtual addresses" into "GPU physical addresses. In other words; the GPU's paging tables do not effect what is/isn't stolen at all.

Of course the GPU's paging tables have to go somewhere, and it'd be nice (to avoid cache coherency for things like textures) to have some "internal" RAM for the GPU; so let's just steal a lot more memory - let's steal 1 GiB of memory (where a few hundred KiB is used for GPU page tables, several MiB is used for framebuffers, and the remainder is used for "internal" GPU RAM). Again; the GPU's paging still does nothing more than convert "GPU virtual addresses" into "GPU physical addresses; and the GPU's paging tables do not effect what is/isn't stolen at all.

No, it does not work that way (at least not for Intel and for those versions of Intel GPUs that I've written a driver for).

The way it works is as follows: There are two kinds of GPU page tables: The GGTT (this is global) and the PPGTT (exists once per process that wants to use the GPU). Display pipes (i.e. the hardware component that reads the framebuffers and emits VGA/HDMI/whatever signals) can only access the GGTT while the command units can access both the GGTT and the PPGTT. As a diagram, it looks like this:
Code:
  +--------+ +--------+ +----------------+
  |        | |        | |                |
  | Pipe 1 | | Pipe 2 | | Command engine |
  |        | |        | |                |
  +--------+ +--------+ +----------------+
     |          |          |        |
     |          |          |        |
     |          |          |        |
     |          |          |        |
+----v----------v----------v--+ +---v---+
|                             | |       |
| GGTT                        | | PPGTT |
|                             | |       |
+-----------------------------+ +-------+


Now, both the GGTT and the PPGTT contain a "mapping type" that specifies if the physical 4096-byte page that is referenced by an entry is cache-coherent (System Memory in Intel's terminology) or non-cache-coherent (Main Memory in Intel's terminology) memory. The OS can freely convert entries between those mapping types with the following caveats:
  • Some parts of the graphics unit need non-coherent memory. For example, this is true for the display pipes. You cannot display a framebuffer from coherent memory.
  • Before you convert memory to non-coherent you have to make sure that it is not in the CPU cache; you need to issue cache flushes as necessary and afterwards map the page as write-combining.
  • There is a PCI BAR that allows access to the non-coherent pages from the GPU's point of view. The BAR allows some other tricks that make dealing with graphics memory more convenient. For example, access through the BAR can transparently de-tile texture memory.

Initially, a small region¹ of the GGTT is filled with non-cache-coherent pages from stolen memory (i.e. memory that is not reported in the memory map). The GGTT itself also resides in stolen memory. All other entries of the GGTT point to a zero page and there is no PPGTT at boot. Thus, if the OS wants to use lots of framebuffers, it has to add non-cache-coherent memory to the GGTT.


This is mostly just a more detailed version of what I already described - the stolen video RAM/main RAM is non-cache-coherent and there's no easy way give it back and use it as system memory, but (with or without tricks like de-tiling) can be made accessible to the CPU via. memory mapped IO. The main difference (that I probably should've foreseen but didn't) is the ability to do the reverse (convert system RAM to non-cache-coherent and let the GPU have it for a while); but that's not particularly relevant.

Note that this whole discussion started because I got the impression that the original poster thought setting a higher resolution video mode (e.g. with VBE or otherwise) costs the OS normal usable system RAM and wanted to make sure that the original poster knows that this is not the case (and that video RAM is not system RAM). Nothing that has been said since suggests otherwise (that setting a higher resolution video mode "auto-magically" reduces the amount of normal usable system RAM).

Korona wrote:
¹ Lets say 16 MiB, the real size depends on the BIOS configuration (i.e. it can be configured by a bit in the memory controller until that bit is locked down).


For my Windows machine (Haswell, HD4600) it's 1 GiB - far too large for "a few frame buffers and not much else". I'd assume that part of the reason for this is that a lot of games are optimised for discrete graphics (where there's a fixed amount of video RAM and higher latency/lower bandwidth for system memory accesses and no point using "shared between CPU and GPU" memory).


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: VESA standart PMode programming
PostPosted: Wed Mar 07, 2018 12:24 pm 
Offline
Member
Member

Joined: Thu May 17, 2007 1:27 pm
Posts: 999
Brendan wrote:
Note that this whole discussion started because I got the impression that the original poster thought setting a higher resolution video mode (e.g. with VBE or otherwise) costs the OS normal usable system RAM and wanted to make sure that the original poster knows that this is not the case (and that video RAM is not system RAM). Nothing that has been said since suggests otherwise (that setting a higher resolution video mode "auto-magically" reduces the amount of normal usable system RAM).

Yes, allocation of stolen memory happens before the OS is given control over the computer. Not setting high modes does not make more memory available unless you have a driver for that specific card.

Brendan wrote:
For my Windows machine (Haswell, HD4600) it's 1 GiB - far too large for "a few frame buffers and not much else". I'd assume that part of the reason for this is that a lot of games are optimised for discrete graphics (where there's a fixed amount of video RAM and higher latency/lower bandwidth for system memory accesses and no point using "shared between CPU and GPU" memory).

But you got that value from Windows or from the memory map and/or the BIOS? I do not think 1 GiB is supported if I read the docs correctly. The value should be in bits 3:7 of PCI register 0x50 on PCI bus 0, device 2, function 0. The value is encoded by some lookup table. I'm not copying the whole table here but the documents mention that 160 MiB (corresponding to value 5) is default and 512 MiB (corresponding to value 16) is max.

_________________
managarm: Microkernel-based OS capable of running a Wayland desktop (Discord: https://discord.gg/7WB6Ur3). My OS-dev projects: [mlibc: Portable C library for managarm, qword, Linux, Sigma, ...] [LAI: AML interpreter] [xbstrap: Build system for OS distributions].


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 19 posts ]  Go to page 1, 2  Next

All times are UTC - 6 hours


Who is online

Users browsing this forum: Google [Bot] and 171 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group