Hi,
goback wrote:
Sure, we can just create a small segment with the first L bytes of a given file. And when the user tries to read something at a position higher than L, we get a fault, so that we can read the file and update the segment descriptor. But then if the user reads the position associated with the the last byte wouldn't we need to copy the entire file into memory? Is there a clever way to do this?
The first "clever" trick I can think of is automatically deciding to use either a normal segment (program wants data near the start of the file only) or an expand down segment (program wants data near the end of the file only).
Note that:
- The general protection fault handler won't tell you the offset (in the segment) that was accessed. The only way to find this information (without paging) would be decode and analyse the instruction at EIP; and without that information you can't know whether to load the first part of the file or the last part (and would have to load the entire file as soon as its accessed).
- Software typically accesses multiple parts of a file. For example, if software reads one byte "offset 1234"; then you could allocate 1235 bytes of memory, create a 1235 byte segment and load those 1235 bytes; but then the software may read another byte at "offset 2345" and you'd have to re-allocate the memory, adjust the segment and load the next 1111 bytes. This means that for the "simplest case" sequential access pattern you'd be constantly diddling (reallocating memory and adjusting the segment).
- RAM must be physically contiguous. Physical memory will become fragmented (as programs allocate and free memory); and when allocating larger amounts of memory you'll probably also have to relocate everything in memory to be able to allocate the physically contiguous RAM needed (which is also going to be painfully slow and complicated; especially for multi-CPU where other CPUs are using things in memory while you're trying to de-fragment). In some cases (e.g. fast SSD drives) de-fragmenting RAM may be more expensive than loading a large file.
As an alternative, you could split the file into many small segments. For example, you could use 1 MiB segments, so that a 2 GiB file actually uses 2048 segments where any of those segments can be loaded or not individually. This solves the problems mentioned above; but also creates a new "excessively annoying for programmers" problem (e.g. if software wants 4 bytes at offset 0x000FFFFE, it'd have to load the segment for "offset 0" and get 2 bytes, then load the next segment and get the next 2 bytes). Mostly (to deal with this) programmers are probably going to write functions/wrappers - e.g. a "get_byte()" function that calculates which segment to use, then loads the segment register, reads the byte, etc (however in that case it's not much harder for these wrappers to manage a cache and do the file IO themselves).
There's also a limit to the number of descriptors you can have in the GDT and/or LDT. This causes compromises - for a single 2 GiB file you can't save more RAM by splitting it into a lot more smaller segments (e.g. 4 KiB segments) because you'll run out of descriptors; and for multiple files you'll probably run out of descriptors anyway (e.g. with four 2 GiB files and 1 MiB segments you'd need 8192 descriptors).
Of course if you could use lots of descriptors (e.g. and could use a massive number of 4 KiB segments) then it'd become an "excessively annoying for programmers" version of paging.
Cheers,
Brendan