OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Apr 18, 2024 2:21 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 9 posts ] 
Author Message
 Post subject: Compatible partitioning: MBR and GPT
PostPosted: Tue Mar 13, 2018 2:25 pm 
Offline

Joined: Sat Jul 28, 2007 2:23 am
Posts: 22
Can MBR and GPT partition tables refer to the same partition?

What I have in mind is a drive laid out roughly as follows.

* LBA 0: MBR with its normal embedded partition table
* LBA 1 to N: The N sectors of the GPT
* A small, dummy partition
* A data partition covering the bulk of the drive (or up to 2TB of it)
* The backup GPT

In the MBR:
- The partition 1 entry would ideally refer to the space occupied by the GPT and the dummy partition
- The partition 2 entry would refer to to the data partition

In the GPT:
- The partition 1 entry would refer to the dummy partition
- The partition 2 entry would refer to the data partition

The partition 1 entries are not essetial but would be nice to have to guard against various partitioning tools interpreting some of the space as free. The key point is that both tables refer to the same data partition; the partition would be seen as valid whichever partition table were to be consulted. I guess the partition numbers don't even have to match. In other words, the same partition could be partition 1 in one table and partition 2 in the other. But it might be handy for them to use the same number.

Any ideas on whether a compatible hybrid like that is possible, and do you know of any tools which will partition a drive that way?

_________________
--
James Harris


Top
 Profile  
 
 Post subject: Re: Compatible partitioning: MBR and GPT
PostPosted: Tue Mar 13, 2018 3:03 pm 
Offline
Member
Member
User avatar

Joined: Sat Nov 22, 2014 6:33 pm
Posts: 934
Location: USA
Hi James,

I have partitioned one of my test images just similar to this. The MBR entries point to a partition(s) as if the GPT was not there, as well as the GPT points to the same partition(s) as if the MBR entries were not there. I think this was the original intent, giving two SysID values for the MBR entry(s).

However, we have had this discussion here in this forum before and got reactions from absolutely not to absolutely yes, and everywhere in-between. viewtopic.php?f=15&t=32448

If you create the two partitions in the MBR table, the first to "protect" the GPT area and the second to point to a single partition, then I think the GPT only needs the one partition entry. It shouldn't need to "protect" itself.

There is a drawback about this though, and you mentioned it, the partition cannot be more than a 32-bit sector number from the base of the media and cannot be larger than a 32-bit number of sectors. For now, not a problem, since 2TB (512-byte sectors) disks are not a cheap option. Yet... Just as an interest only item, drives that can use 4096-byte sectors, natively, can still use 32-bit base offset and sizes, since this increases the distance from the start as well as the size of the partition--eight times to 16TB's each. Large disks.... In fact, if the OS is 48-bit disk capable, the MBR can still be 32-bit and you can handle a 32TB disk.

Ben
- http://www.fysnet.net/osdesign_book_series.htm


Top
 Profile  
 
 Post subject: Re: Compatible partitioning: MBR and GPT
PostPosted: Tue Mar 13, 2018 3:44 pm 
Offline
Member
Member
User avatar

Joined: Fri Feb 17, 2017 4:01 pm
Posts: 642
Location: Ukraine, Bachmut
It's incompatible with GPT and thus wrong. If there is GPT partitioning on a disk, only protective MBR is allowed. It should contain only 1 partition and that partition should be non-bootable and cover all the space of the disk or the maximum MBR addressing could cover.

But of course, you are free to devise your own deviations how you wish. There are already such crippled down things like Hybrid MBR or GPT.

PS. It's funny that you called it "compatible" whereas the only real purpose such things could do is breaking compatibilty/compliance/work.

_________________
ANT - NT-like OS for x64 and arm64.
efify - UEFI for a couple of boards (mips and arm). suspended due to lost of all the target park boards (russians destroyed our town).


Top
 Profile  
 
 Post subject: Re: Compatible partitioning: MBR and GPT
PostPosted: Tue Mar 13, 2018 11:03 pm 
Offline
Member
Member
User avatar

Joined: Sat Nov 22, 2014 6:33 pm
Posts: 934
Location: USA
zaval wrote:
It's incompatible with GPT and thus wrong.

Here is my opinion, and please understand that I place great emphasis on that word, opinion.

The existence *and* the name of the PMBR (Protective MBR) is proof that GPT was meant to be backwards compatible. It was meant, if only for legacy BIOSes to see that a MBR was present with at least one valid Partition entry, though with an unknown System ID field. Then the intent was for the BIOS to "express concern" and choke.

However, how often will you actually run across this? I mean, how often will someone have a non-aware GPT BIOS and a bootable--as the primary boot device--GPT partitioned drive? Seriously. Not counting us few that might try it just to try it.

If GPT really meant that there cannot be Legacy partition entry(s), they would have not had a PMBR and started the GPT either at LBA zero, or if you care about reliability of the "edge" of the disk, at LBA 2048 or even LBA 4096, specifying that every sector before this sector must be zeros.

This even means that they wouldn't care that the first sector contained zeros (add [bx+si],al) and eventually would get to LBA 2048 and try to execute the first sector of the GPT.

If GPT really meant absolutely no other form of partitioning may be used, there would be no other form including a PMBR.

UEFI 2.5, section 5.2.1 states that a Legacy MBR may reside at LBA 0, though UEFI firmware will not execute it. This MBR pay contain up to four partition entries and each may point to partitions on the disk. It does not say that these partitions cannot be partitions already described in a GPT.

"A legacy MBR may be located at LBA 0 (i.e., the first logical block) of the disk if it is not using the GPT disk layout (i.e., if it is using the MBR disk layout)."

Define "it" as in "if it is not using the GPT layout". Does this mean that the MBR is not using the GPT layout or the disk is not using the GPT layout? Most likely the latter than the former.

Just my opinion, so please don't take it as anything else. My Father-in-law always said, "Opinions are like ****holes, everyone has one and they usually stink".

Ben

P.S. Backwards compatibility will always be present in any general consumer electronics product, namely PCs. How much easier and cheaper do you think it would be for Intel/AMD/etc. to make a processor without Legacy support? It is in everything. The HPET describes in detail how the first two timers are used for Legacy PIT and RTC replacements. The 32-bit processor contains 16-bit modes, multiple 16-bit modes at that. The 64-bit processor contains a simplified 16-bit mode if only for backwards compatibility for booting. Disks with 4k sectors contain backwards compatibility with 512-byte logical sectors. In fact, they even include a mechanism to "pad" the first physical sector so that the Legacy LBA 63 lines up on a 4k boundary. How long ago was the requirement that the first partition start at LBA 63? This was back in the CHS days, yet it still affects a 4k sector disk that is not even common yet. As long as consumer electronics (i.e.: money) is concerned, there will *always* be some form of backwards compatibility.


Top
 Profile  
 
 Post subject: Re: Compatible partitioning: MBR and GPT
PostPosted: Wed Mar 14, 2018 11:57 am 
Offline

Joined: Sat Jul 28, 2007 2:23 am
Posts: 22
BenLunt wrote:
Hi James,


Hi Ben, good to talk to you on another forum. :-)

BenLunt wrote:
I have partitioned one of my test images just similar to this. The MBR entries point to a partition(s) as if the GPT was not there, as well as the GPT points to the same partition(s) as if the MBR entries were not there. I think this was the original intent, giving two SysID values for the MBR entry(s).



Is there a standard utility which will set up partition this way - i.e. matching in MBR and GPT - or did you write your own?

BTW, by SysID I guess you mean the partition type byte value but I am puzzled by there being two such values for an MBR entry.

BenLunt wrote:

However, we have had this discussion here in this forum before and got reactions from absolutely not to absolutely yes, and everywhere in-between. viewtopic.php?f=15&t=32448

If you create the two partitions in the MBR table, the first to "protect" the GPT area and the second to point to a single partition, then I think the GPT only needs the one partition entry. It shouldn't need to "protect" itself.



Thanks, I'll read the thread. Agreed that there shouldn't be a need for such protection. I suggested the dummy partition purely to occupy space prior to where it would be convenient to align the real partition, and suggested a partition entry to cover the space just to make sure nothing else would use it. I have long been suspicious of programs like Grub which write to unallocated space.

BenLunt wrote:

There is a drawback about this though, and you mentioned it, the partition cannot be more than a 32-bit sector number from the base of the media and cannot be larger than a 32-bit number of sectors. For now, not a problem, since 2TB (512-byte sectors) disks are not a cheap option. Yet... Just as an interest only item, drives that can use 4096-byte sectors, natively, can still use 32-bit base offset and sizes, since this increases the distance from the start as well as the size of the partition--eight times to 16TB's each. Large disks.... In fact, if the OS is 48-bit disk capable, the MBR can still be 32-bit and you can handle a 32TB disk.



I was thinking it would work for USB memory sticks which can be transported between machines. They are small enough and the partitioning would work whether the stick was viewed as MBR or GPT partitioned.

Speaking of which, would there be any value in including a GPT partition on it, especially if it is not to be booted from?

_________________
--
James Harris


Top
 Profile  
 
 Post subject: Re: Compatible partitioning: MBR and GPT
PostPosted: Wed Mar 14, 2018 1:55 pm 
Offline
Member
Member
User avatar

Joined: Sat Nov 22, 2014 6:33 pm
Posts: 934
Location: USA
JamesHarris wrote:
BenLunt wrote:
I have partitioned one of my test images just similar to this. The MBR entries point to a partition(s) as if the GPT was not there, as well as the GPT points to the same partition(s) as if the MBR entries were not there. I think this was the original intent, giving two SysID values for the MBR entry(s).

Is there a standard utility which will set up partition this way - i.e. matching in MBR and GPT - or did you write your own?
Not that I know of. I hard code my own image, just for testing. I don't have a utility for it yet.

JamesHarris wrote:
BTW, by SysID I guess you mean the partition type byte value but I am puzzled by there being two such values for an MBR entry.
I think most people are. One identifies the MBR as a Protected MBR for GPT and the other identifies it as a Legacy Bootable MBR, though the specs say there should not be any GPT after it.

JamesHarris wrote:
I was thinking it would work for USB memory sticks which can be transported between machines. They are small enough and the partitioning would work whether the stick was viewed as MBR or GPT partitioned.
I agree, and have no problem with it.

JamesHarris wrote:
Speaking of which, would there be any value in including a GPT partition on it, especially if it is not to be booted from?
Only if the partitions where larger than 2TBs. I don't have a USB thumb drive that big. Do you? :-)

Ben


Top
 Profile  
 
 Post subject: Re: Compatible partitioning: MBR and GPT
PostPosted: Wed Mar 14, 2018 3:47 pm 
Offline
Member
Member
User avatar

Joined: Fri Feb 17, 2017 4:01 pm
Posts: 642
Location: Ukraine, Bachmut
BenLunt wrote:
Here is my opinion, and please understand that I place great emphasis on that word, opinion.

The existence *and* the name of the PMBR (Protective MBR) is proof that GPT was meant to be backwards compatible. It was meant, if only for legacy BIOSes to see that a MBR was present with at least one valid Partition entry, though with an unknown System ID field. Then the intent was for the BIOS to "express concern" and choke.

However, how often will you actually run across this? I mean, how often will someone have a non-aware GPT BIOS and a bootable--as the primary boot device--GPT partitioned drive? Seriously. Not counting us few that might try it just to try it.

They say, PMBR was intended for non-aware utilities to not screw up GPT partitions (thus the name - "protective"). Legacy BIOS could use it, since MBR code in PMBR isn't prohibited and that code obviously should be able to understand GPT partitions¹.
I'm sure the authors were thinking that putting a GPT partitioned disk as a bootable media into a legacy BIOS system is a bad idea not worth considering.
Quote:
If GPT really meant that there cannot be Legacy partition entry(s), they would have not had a PMBR and started the GPT either at LBA zero, or if you care about reliability of the "edge" of the disk, at LBA 2048 or even LBA 4096, specifying that every sector before this sector must be zeros.

This even means that they wouldn't care that the first sector contained zeros (add [bx+si],al) and eventually would get to LBA 2048 and try to execute the first sector of the GPT.

If GPT really meant absolutely no other form of partitioning may be used, there would be no other form including a PMBR.

They clearly state the number of partitions in PMBR is 1. It answers everything.
PMBR isn't even partitioning, it's a remnant of the old partitioning scheme to protect the new one. From badass utilities.
Quote:
UEFI 2.5, section 5.2.1 states that a Legacy MBR may reside at LBA 0, though UEFI firmware will not execute it. This MBR pay contain up to four partition entries and each may point to partitions on the disk. It does not say that these partitions cannot be partitions already described in a GPT.

Sorry, but "legacy MBR" and protective MBR isn't the same in UEFI. The first may be a valid MBR (as MBR scheme), having up to 4 primary partitions. It also may contain EFI system partition (OSType 0xef). And at the same time it means this disk doesn't have GPT at all. On the other hand, if it has GPT, the only MBR partition is:
UEFI_2_4 chapter 5.2.2 wrote:
0xEE (i.e., GPT Protective) is used by a protective MBR (see 5.2.2) to define a fake partition
covering the entire disk
.

So it does say it. I see, they avoid situations with >1 schemes used simultaneously describing the same partitions. And I see it as a right way. Every "hybrid" approach fails on this. and introduces a lot more problems than resolves something. if it resoves anything.
Quote:
"A legacy MBR may be located at LBA 0 (i.e., the first logical block) of the disk if it is not using the GPT disk layout (i.e., if it is using the MBR disk layout)."

Define "it" as in "if it is not using the GPT layout". Does this mean that the MBR is not using the GPT layout or the disk is not using the GPT layout? Most likely the latter than the former.

Just my opinion, so please don't take it as anything else. My Father-in-law always said, "Opinions are like ****holes, everyone has one and they usually stink".

Ben

It's interesting. MBR isn't using vs. disk isn't using. I guess nobody took it so deep. :lol:
In my opinion it means "disk is not using gpt".

And in my opinion, I understand it like this:
1) for compatibility with legacy BIOS, you use legacy MBR and put EFI system partition (if it is still needed) in a MBR array, as a bootable partition. This would be understood by old and new systems.
2) for new systems, you use GPT only, that uses PMBR for the only purpose of protecting partitions from old utilities. You generally don't use this scheme on platforms with legacy BIOS.

¹ But some especially talented inventors might put a code into PMBR that reads GPTPA and finds there its own continuation say in the special "loader partition", loads it and jumps there. It's 100% compatible with GPT and I believe, it's 100% possible to fit GPTPA reading code into 440 bytes of space. This way a proper GPT disk might be used with legacy BIOS. Of course if everything BIOS does considering a disk as a candidate to boot from is reading first 512 bytes from disk and jumping there. As it's said here - didn't know that and forgot it. :lol:

_________________
ANT - NT-like OS for x64 and arm64.
efify - UEFI for a couple of boards (mips and arm). suspended due to lost of all the target park boards (russians destroyed our town).


Top
 Profile  
 
 Post subject: Re: Compatible partitioning: MBR and GPT
PostPosted: Thu Mar 15, 2018 9:52 am 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 5137
JamesHarris wrote:
Can MBR and GPT partition tables refer to the same partition?

According to the relevant specifications, no. With GPT, you are only supposed to use a protective MBR.

Of course, you're free to ignore the specifications; just be aware that things will break in new and exciting ways.

JamesHarris wrote:
do you know of any tools which will partition a drive that way?

Looks like gdisk can do it.


Top
 Profile  
 
 Post subject: Re: Compatible partitioning: MBR and GPT
PostPosted: Sat Mar 24, 2018 3:23 am 
Offline
Member
Member

Joined: Thu Jul 05, 2012 5:12 am
Posts: 923
Location: Finland
I simplified my master boot record (GPT) and created a GPT-partitioned USB stick and it seems to boot. Both Linux and Windows recognize the file system. The protective MBR and GPT partitions are strictly according to the specification (except the unique GUID in this example). I'm aware of the terse error messages in the boot chain but I take undefined behavior very seriously. It should work in a defined way even if there were errors.

There might be BIOSes that won't boot the protective MBR because there are no bootable partitions. I could activate the protective partition but I'm not very comfortable with that idea (i.e. saying that the GPT header is bootable).

This is not a request for comments but I liked to share this. OSDev is more fun than ever! :)

Code:
#!/bin/bash
set -e

if [ -z "${DEV_DANCY_DRIVE}" ]; then
  echo "Define environment variable DEV_DANCY_DRIVE" 1>&2
  exit 1
fi

cd Dancy
make
dd if=/dev/zero of=${DEV_DANCY_DRIVE} || true
sync

gdisk ${DEV_DANCY_DRIVE} << EOF
n
1
34
4095
0700
x
a
0

c
00112233-AABB-AABB-AABB-001122334455
i
w
y
EOF

mkdosfs ${DEV_DANCY_DRIVE}1
bin/dy-gpt ${DEV_DANCY_DRIVE} -g 00112233-AABB-AABB-AABB-001122334455
bin/dy-vbr ${DEV_DANCY_DRIVE}1

mkdir mnt
mount ${DEV_DANCY_DRIVE}1 ./mnt
cp LOADER.* ./mnt/
umount ${DEV_DANCY_DRIVE}1

_________________
Undefined behavior since 2012


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 9 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Bing [Bot], Google [Bot], Majestic-12 [Bot] and 124 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group