OSDev.org

The Place to Start for Operating System Developers
It is currently Sat Oct 19, 2019 7:53 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 7 posts ] 
Author Message
 Post subject: RAM Probing - the "least unsafe" way
PostPosted: Wed May 21, 2008 11:37 pm 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

bewing wrote:
I was also wondering about your old "as safe as can be humanly attained" memory manual probing code. You posted a link or two to it on these forums, too. Are you interested/willing to have that posted permanently in the wiki, too?


Sure - why not. :)

First, some notes:
    - NEVER use manual probing unless you absolutely MUST. This implies that the manual probing code is only used for dodgy old computers, and that the code for manual probing needs to be designed for dodgy old computers (assumptions that are "good" assumptions for modern computers, like "it's unlikely that an ISA video card is present", don't apply).
    - minimize the amount of manual probing you do. For example, if the BIOS supports "Int 0x12" (they all do) then use it to avoid probing RAM below 1 MB. If "Int 0x15, AH=0x88" says there's 0xFFFF KB at 0x00100000 and you think there's more (because a 16-bit value can't tell you there's more if there is) then probe from the end of known RAM (not from 0x00100000).
    - don't assume that writes to "non-RAM" won't be cached (flush the cache with WBINVD or CLFLUSH after testing to make sure you're testing the physical address and not the cache).
    - don't assume that writes to "non-RAM" won't be retained due to bus capacitance (use a dummy write at a different address to avoid this, so you read back the dummy value and not the test value if there's no RAM at the address).
    - don't write a set value to an address and read it back to test for RAM (for e.g. "mov [address],0x12345678; mov [dummy],0x0; wbinvd; cmp [address],0x12345678") because you might be unlucky and find a ROM that contains the same value you're using. Instead try to modify what's already there.
    - test the last bytes of each block, not the first bytes of each block, and make sure that the size of each block is less than 16 KB. This is because some older motherboards relocate the RAM underneath the ROM area to the top of memory (e.g. a computer with 2 MB of RAM might have 128 KB of ROM from 0x000E0000 0x000FFFFF and RAM from 0x00100000 to 0x0020FFFF.
    - don't make any assumptions about the "top of memory". Just because the last byte of RAM is at 0x0020FFFF doesn't mean that there's 2176 KB of RAM installed, and just because there's 2 MB of RAM installed doesn't mean that the last byte of RAM will be at 0x001FFFFF.
    - it's better to assume that memory holes are present (and risk skipping some RAM) than to assume that memory holes aren't present (and risk crashing). This means assuming that the area from 0x00F00000 to 0x00FFFFFF can't be used and not probing this area at all (it's possible that some sort of ISA device is in this area, and that any write to this area can cause problems).
That's everything I can think of...

A "decent" piece of code to test for RAM would be:

Code:
;Probe to see if there's RAM at a certain address
;
;Input
; edx   Maximum number of bytes to test
; esi   Starting address
;
;Output
; ecx   Number of bytes of RAM found
; esi   Address of RAM


probeRAM:
    pushes eax,ebx,edx,ebp
    mov eax,esi             ;eax = starting address
    and eax,0x00000FFF      ;eax = offset within block
    xor eax,0x00000FFF      ;eax = number of bytes at starting address to skip due to rounding
    add esi,eax             ;esi = starting address (rounded up)
    xor ecx,ecx             ;ecx = number of bytes of RAM found so far (none)
    push esi                ;Save starting address for later
    sub edx,eax             ;edx = number of bytes to left to test
    jc .done                ;  all done if nothing left after rounding
    or esi,0x00000FFC       ;esi = address of last dword in first block
    shr edx,12              ;edx = number of blocks to test (rounded down)
    test edx,edx            ;Is there anything left after rounding?
    je .done                ; no

.testAddress:
    mov eax,[esi]           ;eax = original value
    mov ebx,eax             ;ebx = original value
    not eax                 ;eax = reversed value
    mov [esi],eax           ;Modify value at address
    mov [dummy],ebx         ;Do dummy write (that's guaranteed to be a different value)
    wbinvd                  ;Flush the cache
    mov edx,[esi]           ;edx = new value
    mov [esi],ebx           ;Restore the original value (even if it's not RAM, in case it's a memory mapped device or something),
    cmp edx,eax             ;Was the value changed?
    jne .done               ; no, definately not RAM,
                            ; yes, assume we've found some RAM

    add ecx,0x00001000      ;ecx = new number of bytes of RAM found
    add esi,0x00001000      ;esi = new address to test
    sub edx,1               ;edx = new number of blocks remaining
    jae .testAddress        ;Test the next block if there's blocks remaining
                            ;Otherwise we're done

.done:
    pop esi                 ;esi = the starting address (rounded up)
    pops eax,ebx,edx,ebp
    ret


Notes:
    - I didn't test the code above (but it should be fine).
    - Depending on how it's used some of the initial code could be skipped (e.g. if you know the starting address is aways aligned on a 4 KB boundary).
    - the WBINVD instruction seriously effects performance because it invalidates all data in all caches (except the TLB). It would be better to use CLFLUSH so you only invalidate the cache line that needs to be invalidated, but CLFLUSH isn't supported on older CPUs (and older computers is what this code is for). For older computers It shouldn't be too slow because the speed difference between cache and RAM wasn't so much and there's usually only a small amount of RAM (e.g. 64 MB or less). Modern computers have a lot more RAM to test and rely on caches a lot more. For example, an 80486 with 32 MB of RAM might take 1 second, but a Pentium 4 with 2 GB of RAM might take 30 seconds or more.
    - Increasing the block size (e.g. testing every 16 KB instead of testing every 4 KB) will improve performance (and increase risk). AFAIK 16 KB blocks should be safe (but I don't know for sure), and larger blocks sizes aren't safe. Large block sizes (e.g. testing every 1 MB) will probably work on modern computers (but you shouldn't need to probe at all on modern computers), and anything larger than 1 MB is guaranteed to give wrong results.
    - WBINVD isn't supported on 80386 and older computers. This means that for 80386 and older you can't flush the cache, but this shouldn't matter (for 80386 and older memory ran at the same speed as the CPU so there was no need for a cache). You will need to flush cache on later CPUs though. Having one routine that uses WBINVD and another routine that doesn't use WBINVD is probably better than doing an "if (CPU_is_80486_or_newer) { WBINVD }" in the middle of the loop.


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject:
PostPosted: Thu May 22, 2008 12:30 am 
Offline
Member
Member
User avatar

Joined: Wed Feb 07, 2007 1:45 pm
Posts: 1401
Location: Eugene, OR, US
Thanks :)

OK, into the wiki it goes!

Detecting Memory (x86)

Hopefully that article is complete enough now that the issue won't be coming up in the forums anymore (at least not very often :wink: ).

Cheers!


Top
 Profile  
 
 Post subject:
PostPosted: Thu May 22, 2008 7:27 am 
Offline
Member
Member
User avatar

Joined: Tue Oct 17, 2006 9:29 pm
Posts: 2423
Location: Canada
bewing wrote:
Thanks :)

OK, into the wiki it goes!

Detecting Memory (x86)

Hopefully that article is complete enough now that the issue won't be coming up in the forums anymore (at least not very often :wink: ).

Cheers!
Just an idea, but in the "x86 Examples" section, perhaps "Getting an E820 Memory Map and Getting a GRUB Memory Map" should go before the manual probing methods?

_________________
Image
Twitter: @canadianbryan. Award by smcerm, I stole it. Original was larger.
UNIX&BSD's, your only aspirations, to be imitated! 8)
Windows, are an opening in an otherwise solid and opaque surface through which light and, sometimes, even air can pass through; nothing more.


Top
 Profile  
 
 Post subject:
PostPosted: Thu May 22, 2008 6:23 pm 
Offline
Member
Member
User avatar

Joined: Wed Feb 07, 2007 1:45 pm
Posts: 1401
Location: Eugene, OR, US
Yeah, I kinda agree with that. It was just that this was technically an edit of the previously existing article, and that was the way the previous article went -- with the C code for probing immediately following the discussion about probing. I'm hoping to get rid of that C code entirely anyway. But yeah, I think rearranging it would be good.


Top
 Profile  
 
 Post subject:
PostPosted: Fri May 23, 2008 5:50 pm 
Offline
Member
Member

Joined: Sun Apr 06, 2008 7:04 pm
Posts: 33
Location: Southern California
I'll never use the manual probing code, but something odd caught my eye that may need some explaining or fixing for the wiki:

Code:
    mov eax,esi             ;eax = starting address
    and eax,0x00000FFF      ;eax = offset within block
    xor eax,0x00000FFF      ;eax = number of bytes at starting address to skip due to rounding
    add esi,eax             ;esi = starting address (rounded up)
    xor ecx,ecx             ;ecx = number of bytes of RAM found so far (none)
    push esi                ;Save starting address for later
    sub edx,eax             ;edx = number of bytes to left to test
    jc .done                ;  all done if nothing left after rounding
    or esi,0x00000FFC       ;esi = address of last dword in first block


Tracing through this the first couple of instructions always set the last three nibbles to 0xFFF. The 'or' at the end doesn't change that, obviously. I think it needs to be an 'and esi, 0xFFFFFFFC' instead otherwise it will simply stay at 0xFFF instead of 0xFFC.

The number of bytes left to test is also decremented incorrect I think. Say esi is 0x1000 and edx = 0x2000 (ie, two blocks) on input then eax will be 0x0FFF just before the "sub edx,eax" instead of 0x1000 (the size of the block).

So I think there needs to be an "inc eax" just before the sub

Or am I just not looking right at this code?

_________________
- Rob


Last edited by robos on Sat May 24, 2008 11:51 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject:
PostPosted: Fri May 23, 2008 7:32 pm 
Offline
Member
Member
User avatar

Joined: Wed Feb 07, 2007 1:45 pm
Posts: 1401
Location: Eugene, OR, US
Hmmm. OK, doing it in my head, I think I see problems -- so, how about this instead:

Code:
probeRAM:
    pushes eax,ebx,edx,ebp
    mov ebp,esi             ;ebp = starting address
    add esi,0x00000FFF      ;round esi up to block boundary
    and esi, ~0x00000FFF    ;truncate to block boundary
    push esi                ;Save corrected starting address for later
    mov eax, esi            ;eax = corrected starting address
    sub eax, ebp            ;eax = bytes to skip from original starting address, due to rounding
    xor ecx,ecx             ;ecx = number of bytes of RAM found so far (none)
    sub edx,eax             ;edx = number of bytes left to test
    jc .done                ;  all done if nothing left after rounding
    or esi,0x00000FFC       ;esi = address of last dword in first block
    shr edx,12              ;edx = number of blocks to test (rounded down)
    je .done                ; Is there anything left after rounding?

.testAddress:
    mov eax,[esi]           ;eax = original value
    mov ebx,eax             ;ebx = original value
    not eax                 ;eax = reversed value
    mov [esi],eax           ;Modify value at address
    mov [dummy],ebx         ;Do dummy write (that's guaranteed to be a different value)
    wbinvd                  ;Flush the cache
    mov ebp,[esi]           ;ebp = new value
    mov [esi],ebx           ;Restore the original value (even if it's not RAM, in case it's a memory mapped device or something)
    cmp ebp,eax             ;Was the value changed?
    jne .done               ; no, definitely not RAM -- exit to avoid damage
                            ; yes, assume we've found some RAM

    add ecx,0x00001000      ;ecx = new number of bytes of RAM found
    add esi,0x00001000      ;esi = new address to test
    dec edx                 ;edx = new number of blocks remaining
    jg .testAddress        ;Test the next block if there's blocks remaining
                            ;Otherwise we're done

.done:
    pop esi                 ;esi = the corrected starting address (rounded up)
    pops eax,ebx,edx,ebp
    ret


Top
 Profile  
 
 Post subject: Re: RAM Probing - the "least unsafe" way
PostPosted: Sat May 24, 2008 1:52 pm 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

@Robos: Doh - you're right. The instruction should have been "and eax,~0x00000FFF" or "and eax,0xFFFFF000" (but definitely not "and eax,0x00000FFF" as written).

@Bewing: Nice - I didn't even notice the unnecessary "test edx,edx" after the shift :)

I also have no idea why I saved and restored EBP in the original version (as EBP was never touched)...


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group