OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 2:53 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 13 posts ] 
Author Message
 Post subject: Dell 1950 Halting when CR0 Bit 30 Clear Cache Disable Reset
PostPosted: Sat Oct 04, 2014 6:02 pm 
Offline
Member
Member

Joined: Sun Jun 16, 2013 4:09 am
Posts: 333
Hi all.
A couple of servers have arrived and one of them is not starting my OS.
If someone could find my fault it would be appreciated.
Here is my Init code for the CPU.
Code:
FIL void tCPUs::InitThisCPU()
   {
      // Disable Cache
      asm volatile ("mov %cr0, %rax");
      asm volatile ("btr $29, %rax");         // Clear No Write Thru (Bit 29)
      asm volatile ("bts $30, %rax");         // Set Cache Disable (Bit 30)
      asm volatile ("mov %rax, %cr0");

      // Flush Cache
      asm volatile ("wbinvd");

      // Disable Paging Global Extensions
      asm volatile ("mov %cr4, %rax");
      asm volatile ("btr $7, %rax");            // Clear Paging Global Extensions (Bit 7)
      asm volatile ("mov %rax, %cr4");

      // Disable MTRRs and Configure default memory type to UC
      asm volatile ("mov $0x000002FF, %ecx");
      asm volatile ("rdmsr");
      asm volatile ("and $0xFFFFF300, %eax");   // Clear MTRR Enable (Bit 11), Fixed Range MTRR Enable (Bit 10), and Default Memory Type (Bits 7:0) to UC (0x00)
      asm volatile ("wrmsr");

      // Enable MTRRs
      asm volatile ("mov $0x000002FF, %ecx");
      asm volatile ("rdmsr");
      asm volatile ("bts $11, %eax");            // Set MTRR Enable (Bit 11), Only enables Variable Range MTRR's
      asm volatile ("wrmsr");

      // Flush Cache
      asm volatile ("wbinvd");

      // Enable Cache
      asm volatile ("mov %cr0, %rax");
      asm volatile ("btr $29, %rax");         // Clear No Write Thru (Bit 29)
      asm volatile ("btr $30, %rax");         // Clear Cache Disable (Bit 30)
      asm volatile ("mov %rax, %cr0");
      Progress("InitaliseCPU - Enable Cache Done");

      // Enable Floating Point
      asm volatile ("mov %cr0, %rax");
      asm volatile ("bts $1, %rax");            // Set Monitor co-processor (Bit 1)
      asm volatile ("btr $2, %rax");            // Clear Emulation (Bit 2)
      asm volatile ("mov %rax, %cr0");
      Progress("InitaliseCPU - Floating Point Done");

      // Enable SSE
      asm volatile ("mov %cr4, %rax");
      asm volatile ("bts $9, %rax");            // Set Operating System Support for FXSAVE and FXSTOR instructions (Bit 9)
      asm volatile ("bts $10, %rax");            // Set Operating System Support for Unmasked SIMD Floating-Point Exceptions (Bit 10)
      asm volatile ("mov %rax, %cr4");
      Progress("InitaliseCPU - FSSE Done");

      // Enable Math Co-processor
      asm volatile ("finit");
      Progress("InitaliseCPU - finit Done");

      asm volatile ("": : :"%rcx","%rax","cc","memory");
   }

It locks when this line is included:
Code:
asm volatile ("btr $30, %rax");         // Clear Cache Disable (Bit 30)


Any ideas ??

Alistair


Top
 Profile  
 
 Post subject: Re: Dell 1950 Halting when CR0 Bit 30 Clear Cache Disable Re
PostPosted: Sat Oct 04, 2014 6:12 pm 
Offline
Member
Member

Joined: Tue Mar 04, 2014 5:27 am
Posts: 1108
No ideas. But your inline assembly code is almost certainly broken because you never tell the compiler what registers, variables, etc you use. The line "asm volatile ("": : :"%rcx","%rax","cc","memory");" does not apply to any previous asm statements, just to itself. Oops. You need to learn to write proper inline assembly code or just make a separate assembly source with that asm code.


Top
 Profile  
 
 Post subject: Re: Dell 1950 Halting when CR0 Bit 30 Clear Cache Disable Re
PostPosted: Sat Oct 04, 2014 6:19 pm 
Offline
Member
Member

Joined: Sun Jun 16, 2013 4:09 am
Posts: 333
Quote:
No ideas. But your inline assembly code is almost certainly broken because you never tell the compiler what registers, variables, etc you use. The line "asm volatile ("": : :"%rcx","%rax","cc","memory");" does not apply to any previous asm statements, just to itself. Oops. You need to learn to write proper inline assembly code or just make a separate assembly source with that asm code.
Good points thanks.


Top
 Profile  
 
 Post subject: Re: Dell 1950 Halting when CR0 Bit 30 Clear Cache Disable Re
PostPosted: Sat Oct 04, 2014 9:50 pm 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

I'm also not sure why anyone would want to disable the MTRRs so that everything is "uncached". This will completely destroy performance.


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: Dell 1950 Halting when CR0 Bit 30 Clear Cache Disable Re
PostPosted: Sat Oct 04, 2014 9:52 pm 
Offline
Member
Member

Joined: Sun Jun 16, 2013 4:09 am
Posts: 333
Quote:
I'm also not sure why anyone would want to disable the MTRRs so that everything is "uncached". This will completely destroy performance.

Cheers


Top
 Profile  
 
 Post subject: Re: Dell 1950 Halting when CR0 Bit 30 Clear Cache Disable Re
PostPosted: Sat Oct 04, 2014 11:29 pm 
Offline
Member
Member

Joined: Sun Jun 16, 2013 4:09 am
Posts: 333
Hi all, how does this look for inline syntax and setting up caching?

The code is working now on the dell 1950 server, it looks like [MY] inline code was the issue!!


Code:
      asm volatile (
         // Enable SSE
         // Set Operating System Support for FXSAVE and FXSTOR instructions (Bit 9)
         // Set Operating System Support for Unmasked SIMD Floating-Point Exceptions (Bit 10)
         "mov %%cr4, %%rax;         " \
         "bts $9, %%rax;            " \
         "bts $10, %%rax;         " \
         "mov %%rax, %%cr4;         " \

         // Enables MTRR's & Enables Fixed Range MTRR's, Bit 10, Bit 11, and WB—Writeback (0x6) Bits 0-7
         "mov $0x000002FF, %%ecx;   " \
         "rdmsr;                  " \
         "mov $0x00000C06, %%eax;   " \
         "wrmsr;                  " \

         // Flush Cache
         "wbinvd;               " \

         // Enable Cache
         // Clear No Write Thru (Bit 29), Not used legacy olny.
         // Clear Cache Disable (Bit 30)
         "mov %%cr0, %%rax;         " \
         "btr $29, %%rax;         " \
         "btr $30, %%rax;         " \
         "mov %%rax, %%cr0;         " \

         // Enable Floating Point
         // Set Monitor co-processor (Bit 1)
         // Clear Emulation (Bit 2)
         "mov %%cr0, %%rax;         " \
         "bts $1, %%rax;            " \
         "btr $2, %%rax;            " \
         "mov %%rax, %%cr0;         " \

         // Init Math Co-processor
         "finit;                  " \

         // Clobber used registers
         "": : : "%rcx","%rax","cc","memory");

      return;


Top
 Profile  
 
 Post subject: Re: Dell 1950 Halting when CR0 Bit 30 Clear Cache Disable Re
PostPosted: Sun Oct 05, 2014 12:27 am 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

tsdnz wrote:
Hi all, how does this look for inline syntax and setting up caching?


Code:
      asm volatile (
         // Enable SSE
         // Set Operating System Support for FXSAVE and FXSTOR instructions (Bit 9)
         // Set Operating System Support for Unmasked SIMD Floating-Point Exceptions (Bit 10)
         "mov %%cr4, %%rax;         " \
         "bts $9, %%rax;            " \
         "bts $10, %%rax;         " \
         "mov %%rax, %%cr4;         " \


OK, so far; but I wouldn't preserve the previous values in CR4 at all, and would reduce it down to the 2 instructions "mov $0x00000600,%%rax" and "mov %%rax, %%cr4".

Code:
         // Enables MTRR's & Enables Fixed Range MTRR's, Bit 10, Bit 11, and WB—Writeback (0x6) Bits 0-7
         "mov $0x000002FF, %%ecx;   " \
         "rdmsr;                  " \
         "mov $0x00000C06, %%eax;   " \
         "wrmsr;                  " \

         // Flush Cache
         "wbinvd;               " \

         // Enable Cache
         // Clear No Write Thru (Bit 29), Not used legacy olny.
         // Clear Cache Disable (Bit 30)
         "mov %%cr0, %%rax;         " \
         "btr $29, %%rax;         " \
         "btr $30, %%rax;         " \
         "mov %%rax, %%cr0;         " \


This is mostly wrong. To change the MTTRs (which should be 100% unnecessary to begin with); you must:
  • disable caches in CR0 and then flush them (wbinvd); on all CPUs (not just one)
  • wait until all CPUs have done the previous step before continuing
  • update the MTTRs on all CPUs, so that the MTRRs are identical on all CPUs
  • wait until all CPUs have done the previous step before continuing
  • enable caches in CR0

In addition; you can not assume that "default memory type = write-back" is correct. For most systems the firmware sets the default memory type to "uncached" and then uses the variable range MTTRs to change anything that shouldn't be uncached to "write-back" (or whatever else); and therefore for most systems "default memory type = write-back" is wrong (e.g. you will end up with things like memory mapped PCI devices using "write-back" and breaking because of it).

However; for some systems the firmware sets the default memory type to "write-back" and then uses the variable range MTTRs to change anything that shouldn't be write-back to "uncached" (or whatever else); and therefore for some systems "default memory type = write-back" is correct.

Essentially there are only 2 sane choices. Either you don't touch the MTTRs at all and assume the firmware did it's job properly; or you reprogram all variable range MTTRs and the default memory type at the same time. Reprogramming all variable range MTTRs correctly and efficiently (e.g. using the least number of variable range MTTRs so you've got the most free for things like video) is complicated.

Code:
         // Enable Floating Point
         // Set Monitor co-processor (Bit 1)
         // Clear Emulation (Bit 2)
         "mov %%cr0, %%rax;         " \
         "bts $1, %%rax;            " \
         "btr $2, %%rax;            " \
         "mov %%rax, %%cr0;         " \

         // Init Math Co-processor
         "finit;                  " \

         // Clobber used registers
         "": : : "%rcx","%rax","cc","memory");


To be perfectly correct; you should tell the compiler that all floating point registers (and all MMX and 3DNow registers) are clobbered. This brings up another problem - what prevents the compiler from using FPU, MMX or 3DNow before this code is executed? There's only 2 ways to guarantee that doesn't happen. The first is to write the CPU initialisation code as pure assembly (not inline assembly) and use it as the kernel's entry point (where the pure assembly initialises the CPU before any C code is executed, and then passes control to the C code). The other way is to tell the compiler not to use FPU, MMX or 3DNow (but I'm not sure if that's possible to do for individual functions so you may end up never using FPU, MMX or 3DNow for anything anywhere in your entire kernel).

Note: I'd be willing to bet that you've already got some "pure assembly startup code" that (e.g.) sets up the stack.


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: Dell 1950 Halting when CR0 Bit 30 Clear Cache Disable Re
PostPosted: Sun Oct 05, 2014 12:49 am 
Offline
Member
Member

Joined: Sun Jun 16, 2013 4:09 am
Posts: 333
Quote:
This is mostly wrong. To change the MTTRs (which should be 100% unnecessary to begin with); you must:
disable caches in CR0 and then flush them (wbinvd); on all CPUs (not just one)
wait until all CPUs have done the previous step before continuing
update the MTTRs on all CPUs, so that the MTRRs are identical on all CPUs
wait until all CPUs have done the previous step before continuing
enable caches in CR0
Quote:
Essentially there are only 2 sane choices. Either you don't touch the MTTRs at all and assume the firmware did it's job properly; or you reprogram all variable range MTTRs and the default memory type at the same time. Reprogramming all variable range MTTRs correctly and efficiently (e.g. using the least number of variable range MTTRs so you've got the most free for things like video) is complicated.

Thanks, they are gone

Quote:
In addition; you can not assume that "default memory type = write-back" is correct. For most systems the firmware sets the default memory type to "uncached" and then uses the variable range MTTRs to change anything that shouldn't be uncached to "write-back" (or whatever else); and therefore for most systems "default memory type = write-back" is wrong (e.g. you will end up with things like memory mapped PCI devices using "write-back" and breaking because of it).

However; for some systems the firmware sets the default memory type to "write-back" and then uses the variable range MTTRs to change anything that shouldn't be write-back to "uncached" (or whatever else); and therefore for some systems "default memory type = write-back" is correct.

I see, again nice, thanks.


Quote:
To be perfectly correct; you should tell the compiler that all floating point registers (and all MMX and 3DNow registers) are clobbered
I do like to be tidy so I will add the registers you specified.


Yes, this code is run in the first few lines of code, there is no chance of MMX or 3DNow.
Code:
   void APStart()
   {
      tCPUs::DisableInterrupts();
      tCPUs::InitThisCPU();
      ABIT.CPUs->ConfigureLocalAPIC();
Code:
   void BSPStart()
   {
      tCPUs::DisableInterrupts();
      InitaliseCPU();




Many thanks.


Top
 Profile  
 
 Post subject: Re: Dell 1950 Halting when CR0 Bit 30 Clear Cache Disable Re
PostPosted: Sun Oct 05, 2014 1:06 am 
Offline
Member
Member

Joined: Sun Jun 16, 2013 4:09 am
Posts: 333
The servers crash if I just put 0x600 into rax

Code:
FIL void tCPUs::InitThisCPU()
   {
      asm volatile (
         // Enable SSE
         // Set Operating System Support for FXSAVE and FXSTOR instructions (Bit 9)
         // Set Operating System Support for Unmasked SIMD Floating-Point Exceptions (Bit 10)
         "mov %%cr4, %%rax;         " \
         "bts $9, %%rax;            " \
         "bts $10, %%rax;         " \
         "mov %%rax, %%cr4;         " \

         // Flush Cache
         "wbinvd;               " \

         // Enable Floating Point
         // Set Monitor co-processor (Bit 1)
         // Clear Emulation (Bit 2)
         "mov %%cr0, %%rax;         " \
         "bts $1, %%rax;            " \
         "btr $2, %%rax;            " \
         "mov %%rax, %%cr0;         " \

         // Init Math Co-processor
         "finit;                  " \

         // Clobber used registers
         "": : : "%rcx","%rax","cc","memory");

      return;


Top
 Profile  
 
 Post subject: Re: Dell 1950 Halting when CR0 Bit 30 Clear Cache Disable Re
PostPosted: Sun Oct 05, 2014 1:23 am 
Offline
Member
Member

Joined: Sun Jun 16, 2013 4:09 am
Posts: 333
Code:
The servers crash if I just put 0x600 into rax

Silly me, lol!!

I am setting up paging etc before this point in assembly.

Code:
;------------------------------------------------------------------------------

align 16
EnableExtendedProperties:

   mov      eax, cr4
   or      eax, 0x0000000B0   ; PGE (Bit 7), PAE (Bit 5), and PSE (Bit 4)
   mov      cr4, eax

   ret


Top
 Profile  
 
 Post subject: Re: Dell 1950 Halting when CR0 Bit 30 Clear Cache Disable Re
PostPosted: Sun Oct 05, 2014 1:32 am 
Offline
Member
Member

Joined: Sun Jun 16, 2013 4:09 am
Posts: 333
Here is the final Init:

Code:
   FIL void tCPUs::InitThisCPU()
   {
      asm volatile (
         // Enable SSE
         // Set Operating System Support for FXSAVE and FXSTOR instructions (Bit 9)
         // Set Operating System Support for Unmasked SIMD Floating-Point Exceptions (Bit 10)
         "mov %%cr4, %%rax;         " \
         "bts $9, %%rax;            " \
         "bts $10, %%rax;         " \
         "mov %%rax, %%cr4;         " \

         // Flush Cache
         "wbinvd;               " \

         // Enable Floating Point
         // Set Monitor co-processor (Bit 1)
         // Clear Emulation (Bit 2)
         "mov %%cr0, %%rax;         " \
         "bts $1, %%rax;            " \
         "btr $2, %%rax;            " \
         "mov %%rax, %%cr0;         " \

         // Init Math Co-processor
         "finit;                  " \

         // Clobber Registers
         "": : : "%rax","%rbx","%rcx","%rdx","%rsp","%rsi","%rdi","%r8","%r9","%r10","%r11","%r12","%r13","%r14","%r15","cc","memory","%st","%st(1)","%st(2)","%st(3)","%st(4)","%st(5)","%st(6)","%st(7)","%xmm0","%xmm1","%xmm2","%xmm3","%xmm4","%xmm5","%xmm6","%xmm7");
   }


Top
 Profile  
 
 Post subject: Re: Dell 1950 Halting when CR0 Bit 30 Clear Cache Disable Re
PostPosted: Sun Oct 05, 2014 3:04 am 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
tsdnz wrote:
Here is the final Init:


Almost :)

The "wbinvd" is now unnecessary (and is expensive); and you aren't clobbering rbx, rcx, rdx, rsp (which I'm surprised GCC accepts), rsi, rdi, r8, r9, r10, r11, r12, r13, r14 or r15.


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: Dell 1950 Halting when CR0 Bit 30 Clear Cache Disable Re
PostPosted: Sun Oct 05, 2014 1:21 pm 
Offline
Member
Member

Joined: Sun Jun 16, 2013 4:09 am
Posts: 333
Quote:
Almost :)

The "wbinvd" is now unnecessary (and is expensive); and you aren't clobbering rbx, rcx, rdx, rsp (which I'm surprised GCC accepts), rsi, rdi, r8, r9, r10, r11, r12, r13, r14 or r15.
Awake to a new day, looking forward to my cold leaving me!

Yes, left that in.
Thought I would add all the other registers in for fun, a clean entry point, and tell the compiler to clobber everything.

And thanks heaps.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 13 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 62 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group