OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 7:17 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 42 posts ]  Go to page Previous  1, 2, 3
Author Message
 Post subject: Re: Non-Volatile RAM
PostPosted: Wed Aug 05, 2015 5:07 pm 
Offline
Member
Member
User avatar

Joined: Wed Jan 06, 2010 7:07 pm
Posts: 792
Kevin wrote:
"Equivalent to file system corruption"... That's actually a scary thought in the context of a monolithic kernel: Any rogue pointer that would today cause memory corruption (which would be "fixed" by a reboot) could end up causing file system corruption when your whole "disk" is mapped as RAM.
The kernel doesn't have to map everything into virtual memory. I don't think there's any greater risk of file system corruption than there already is from things like buffer caches.

_________________
[www.abubalay.com]


Top
 Profile  
 
 Post subject: Re: Non-Volatile RAM
PostPosted: Thu Aug 06, 2015 10:28 am 
Offline
Member
Member

Joined: Sat Mar 01, 2014 2:59 pm
Posts: 1146
Rusky wrote:
A lot of misunderstanding going on here.

The ability to force a clean reset has almost nothing to do with NVRAM- it just means you put all your non-persistent state (approximately everything we store in volatile RAM today) in one particular section of NVRAM that you can easily discard by deallocating it and jumping to the initialization code, which will then act as if that state had been lost (it's really still there, just treated as leftover garbage now).
So it works like this?

In the event of a power failure:
"Volatile" data and "non-volatile" data are both preserved, and the system can carry on where it left off.

In the event of a system crash:
"Volatile" data and "non-voltaile" data are both preserved (although may be in an inconsistent state) and can be recovered via a memory dump.

In the event of an intentional system restart:
"Volatile" data is wiped and "non-volatile" data is preserved so that it can be reused at the next startup without having to be rebuilt via an algorithm (e.g. routing tables, etc.).

Am I correct? If so, this sounds elegant. If not, then it's still elegant and something worth considering :) .

_________________
When you start writing an OS you do the minimum possible to get the x86 processor in a usable state, then you try to get as far away from it as possible.

Syntax checkup:
Wrong: OS's, IRQ's, zero'ing
Right: OSes, IRQs, zeroing


Top
 Profile  
 
 Post subject: Re: Non-Volatile RAM
PostPosted: Fri Aug 07, 2015 4:46 am 
Offline
Member
Member

Joined: Wed Jun 03, 2015 5:03 am
Posts: 397
onlyonemac wrote:
In the event of a power failure:
"Volatile" data and "non-volatile" data are both preserved, and the system can carry on where it left off.

In the event of a system crash:
"Volatile" data and "non-voltaile" data are both preserved (although may be in an inconsistent state) and can be recovered via a memory dump.

In the event of an intentional system restart:
"Volatile" data is wiped and "non-volatile" data is preserved so that it can be reused at the next startup without having to be rebuilt via an algorithm (e.g. routing tables, etc.).

The size of the data is important in case of a combination of NVRAM and RAM. But if there is only NVRAM, then your cases are correct. And of course, the logic behind the cases should be more carefully studied and implemented, than is the case for sleep or hibernate procedures.

_________________
My previous account (embryo) was accidentally deleted, so I have no chance but to use something new. But may be it was a good lesson about software reliability :)


Top
 Profile  
 
 Post subject: Re: Non-Volatile RAM
PostPosted: Fri Aug 07, 2015 1:30 pm 
Offline
Member
Member

Joined: Sat Mar 01, 2014 2:59 pm
Posts: 1146
embryo2 wrote:
The size of the data is important in case of a combination of NVRAM and RAM. But if there is only NVRAM, then your cases are correct. And of course, the logic behind the cases should be more carefully studied and implemented, than is the case for sleep or hibernate procedures.
Yeah obviously in the case of a mixture of RAM and NVRAM one has to bear in mind
a) the size of the data
b) the performance impact of using NVRAM
c) whether or not the data is "volatile"
i.e. all "volatile" data should be stored in RAM unless RAM is full in which case it may go in NVRAM. "Non-volatile" data should always be stored in NVRAM or on disk if NVRAM is full. If RAM is available, excess RAM may be used to cache "non-volatile" data from NVRAM or from disk in order to give a performance boost, provided that changes are comitted immediately to NVRAM/disk in order to maintain system consistency at all times.

_________________
When you start writing an OS you do the minimum possible to get the x86 processor in a usable state, then you try to get as far away from it as possible.

Syntax checkup:
Wrong: OS's, IRQ's, zero'ing
Right: OSes, IRQs, zeroing


Top
 Profile  
 
 Post subject: Re: Non-Volatile RAM
PostPosted: Fri Aug 07, 2015 3:08 pm 
Offline
Member
Member
User avatar

Joined: Wed Jan 06, 2010 7:07 pm
Posts: 792
onlyonemac wrote:
all "volatile" data should be stored in RAM unless RAM is full in which case it may go in NVRAM. "Non-volatile" data should always be stored in NVRAM or on disk if NVRAM is full. If RAM is available, excess RAM may be used to cache "non-volatile" data from NVRAM or from disk in order to give a performance boost, provided that changes are comitted immediately to NVRAM/disk in order to maintain system consistency at all times.
Also known as virtual memory swapping and buffer caches- which means you're just treating NVRAM as a faster disk.

_________________
[www.abubalay.com]


Top
 Profile  
 
 Post subject: Re: Non-Volatile RAM
PostPosted: Sat Aug 08, 2015 7:36 am 
Offline
Member
Member

Joined: Wed Jun 03, 2015 5:03 am
Posts: 397
Rusky wrote:
which means you're just treating NVRAM as a faster disk.

Just like you treating RAM as another layer of a cache. NVRAM can give us yet another layer for the caching strategy elaboration.

_________________
My previous account (embryo) was accidentally deleted, so I have no chance but to use something new. But may be it was a good lesson about software reliability :)


Top
 Profile  
 
 Post subject: Re: Non-Volatile RAM
PostPosted: Sat Aug 08, 2015 7:52 am 
Offline
Member
Member

Joined: Sat Mar 15, 2014 3:49 pm
Posts: 96
A lot is being made of the 'no need to reboot''ness of NVRAM.

Do we imagine the registers or stack or belt or whatever inside the CPUs is going to be NV too?

We've had decent generally-available NVRAM for a long time already - chances are you've got it in your computer already, and its called SSD. It happens that its been far easier to treat it as 'disk' rather than RAM so as to make existing programs faster without changing paradigms.

Faster NVRAM than SSD e.g. memristors or 3D XPoint will be very welcome as faster disk. It just makes fsync() a no-op and suddenly real workloads that require durability just go magically much faster.

The next step will be for the DBMS to know that the mmapped file is on NVRAM and to do 'on-disk' sorts and such so as to avoid extra copies between RAM and NVRAM.

I'm trying to think what paradigms could change deeper in the kernel, though, and so far .. not so much. NVRAM is disk that may not need page buffers and so on, but those are small tweaks internal to the FS driver and not something that makes you change anything else much...


Top
 Profile  
 
 Post subject: Re: Non-Volatile RAM
PostPosted: Thu Aug 13, 2015 9:09 am 
Offline
Member
Member
User avatar

Joined: Fri Aug 21, 2009 5:54 am
Posts: 178
Location: Moscow, Russia
I've actually made a computer like that as a hobby project - 12.5MHz custom CPU, with 4Kb of volatile fast RAM and 64Kb of non-volatile (but only half as fast) FRAM.
I think i mentioned it here - viewtopic.php?f=15&t=27702

I always wanted to have a computer that can be turned off, then turned on and resume working from the same place.
Didn't quite get there, but it was fun.

Resetting to a known state is not a problem - just make the bootloader check for a reset button being pressed at power up.

The OS architecture was more or less transactional. Every inactive task stays where it is, active task gets a power-fail exception.
The idea was that on power-off the volatile stuff gets copied over to the FRAM (there was a low voltage detect interrupt and a few milliseconds worth of capacitors).
In practice that does not work beyond saving the task state, and i had to assume that whatever is in the volatile memory is gone.

On re-power, a bootloader checks for magic numbers, then calls the kernel's resume entry point. The kernel re-initializes the peripherals (which are volatile), and continues the execution.

All in all, once you out of the bushes it's not much different from a regular computer on the user level.
And there really isn't any advantages (that i found) to the non-volatility.
At least not on this scale - the whole thing can be cold-booted just as fast, and most of the time is spent (re-)initializing the peripherals.

However, debugging in such a system can be FUN with a capital F.
Uninitialized variables, loose pointers and similar things can produce mind-twisting effects if you ever forget that a reset means nothing.
The closest word to it is "haunted".
The system just feels haunted.
You have to exorcise it every now and then by a full reboot.


Top
 Profile  
 
 Post subject: Re: Non-Volatile RAM
PostPosted: Mon Aug 17, 2015 12:35 am 
Offline
Member
Member

Joined: Wed Dec 24, 2014 8:06 am
Posts: 29
Location: Germany
Well, a computer with 512 GB of non-volatile RAM would be much like a computer with no RAM but a very fast hard disk.
The RAM would have a filesystem and serve as a HDD.
Data such as process memory would simply be stored inside a swapfile.

_________________
All a good OS needs to do is to run Linux inside QEMU :-)


Top
 Profile  
 
 Post subject: Re: Non-Volatile RAM
PostPosted: Tue Aug 25, 2015 1:04 am 
Offline
Member
Member

Joined: Sat Mar 01, 2014 2:59 pm
Posts: 1146
StartOS wrote:
Well, a computer with 512 GB of non-volatile RAM would be much like a computer with no RAM but a very fast hard disk.
The RAM would have a filesystem and serve as a HDD.
Data such as process memory would simply be stored inside a swapfile.
That approach actually works for my OS design. I am managing RAM like a filesystem, where the lines between RAM and disk storage (and, also, NVRAM if it was present) are blurred and they are all treated equally as merely different places that "objects" can reside. Of course, objects are specified to be either volatile or non-volatie, and in the latter case the OS is obliged to keep them on non-volatile media even if they are also cached in RAM (the details of which caching system I am still working out).

_________________
When you start writing an OS you do the minimum possible to get the x86 processor in a usable state, then you try to get as far away from it as possible.

Syntax checkup:
Wrong: OS's, IRQ's, zero'ing
Right: OSes, IRQs, zeroing


Top
 Profile  
 
 Post subject: Re: Non-Volatile RAM
PostPosted: Tue Aug 25, 2015 6:41 pm 
Offline
Member
Member
User avatar

Joined: Sun Sep 19, 2010 10:05 pm
Posts: 1074
Quote:
I am managing RAM like a filesystem, where the lines between RAM and disk storage (and, also, NVRAM if it was present) are blurred and they are all treated equally as merely different places that "objects" can reside.

That sounds awfully familiar. :)

Although, rather than trying to use hardware to make non-volatile hard drive space look like RAM, I'm planning on letting the application decide when to use RAM, when to use hard drive space, and when to let the OS decide. However, reading and writing to all three will probably work the same way.

That's the plan anyway.

_________________
Project: OZone
Source: GitHub
Current Task: LIB/OBJ file support
"The more they overthink the plumbing, the easier it is to stop up the drain." - Montgomery Scott


Top
 Profile  
 
 Post subject: Re: Non-Volatile RAM
PostPosted: Wed Aug 26, 2015 1:39 am 
Offline
Member
Member

Joined: Sat Mar 01, 2014 2:59 pm
Posts: 1146
SpyderTL wrote:
Although, rather than trying to use hardware to make non-volatile hard drive space look like RAM, I'm planning on letting the application decide when to use RAM, when to use hard drive space, and when to let the OS decide. However, reading and writing to all three will probably work the same way.
I am leaving it up to wherever the object is created or moved to. So a thread creates an object and specifies what device to store it on and where in that device's object tree (although the UNIX filesystem concept is good, I decided not to implement such a design in my OS). A storage device in turn has a set of flags, one of which states if the device is "persistant" (i.e. non-volatile storage). It is up to the thread, or the user, to decide if it is appropriate for the object to be stored on a "persistant" device, or if it is OK for it to be stored on a "non-persistant" device (which, at this point, would be in RAM). (It is also worth noting that RAM is going to be implemented as multiple devices, at this stage I'm thinking one for the kernel+loaded modules and another for "userspace" (i.e. a scratchspace for loading running applications and any temporary data that they create).)

_________________
When you start writing an OS you do the minimum possible to get the x86 processor in a usable state, then you try to get as far away from it as possible.

Syntax checkup:
Wrong: OS's, IRQ's, zero'ing
Right: OSes, IRQs, zeroing


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 42 posts ]  Go to page Previous  1, 2, 3

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 27 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group