OSDev.org

The Place to Start for Operating System Developers
It is currently Wed Mar 03, 2021 9:15 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 85 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6  Next
Author Message
 Post subject: Re: Announcing POSIX-UEFI
PostPosted: Sun Feb 21, 2021 6:34 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 2815
bzt wrote:
I noticed it in fopen line 215.

Do you have a smaller example to reproduce the bug? (Or at least the flags you pass to GCC when compiling this file?)

kzinti wrote:
I have also run into a very similar problem where GCC optimizes malloc() + memset() into calloc(). The problem was that I was providing my own implementation of calloc() that was basically doing malloc() + memset(), resulting in another infinite loop.

I'm pretty sure you're supposed to build standard library functions with -fno-builtin.

bzt wrote:
And another, if you have a struct at address 0, then accessing any of its fields (at non-zero address) would generate an UD2 instruction instead of a MOV, how lovely, isn't it?

That's not a bug, using null pointers is undefined behavior (and using 0 to represent the null pointer is implementation-defined behavior).


Top
 Profile  
 
 Post subject: Re: Announcing POSIX-UEFI
PostPosted: Tue Feb 23, 2021 10:42 am 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 1303
Octocontrabass wrote:
bzt wrote:
I noticed it in fopen line 215.

Do you have a smaller example to reproduce the bug? (Or at least the flags you pass to GCC when compiling this file?)
No, I have no smaller example unfortunately. You have all the flags in the repo. To reproduce the error, simply replace the ifdef in uefi line 1344 with
Code:
#define MEMFNC(a) a
This will avoid the name prefix workaround, and should trigger the issue in stdio.o's fopen. Just compile one of the examples, like 05_file with USE_GCC=1, and then check out "objdump -d uefi/stdio.o".

Octocontrabass wrote:
I'm pretty sure you're supposed to build standard library functions with -fno-builtin.
Yes, but it didn't worked.

Octocontrabass wrote:
bzt wrote:
And another, if you have a struct at address 0, then accessing any of its fields (at non-zero address) would generate an UD2 instruction instead of a MOV, how lovely, isn't it?

That's not a bug, using null pointers is undefined behavior
You didn't listen, it is a bug because you're not using NULL pointers. With a struct, only the first member's address would be 0, not the rest, yet ALL member access will be replaced by UD2, not just the first, that's the bug. So for example
Code:
struct {
  unsigned char ivt[1024];
  unsigned char bda[256];
  ...
} memorymap_t;
Only ivt's address equals to 0, yet accessing bda (address 1024) would generate UD2 as well. (Let's not get into the part that in freestanding mode accessing address 0 (dereferencing NULL if and only if NULL is defined and it's defined as 0) is not just perfectly valid, but even required on some architectures. It can only be UB in hosted mode, simply because libc defines NULL, not the compiler.)

Cheers,
bzt


Top
 Profile  
 
 Post subject: Re: Announcing POSIX-UEFI
PostPosted: Tue Feb 23, 2021 11:02 am 
Offline
Member
Member

Joined: Mon Feb 02, 2015 7:11 pm
Posts: 572
bzt wrote:
You didn't listen, it is a bug because you're not using NULL pointers. With a struct, only the first member's address would be 0, not the rest, yet ALL member access will be replaced by UD2, not just the first, that's the bug. So for example

You are the one not listening. Accessing any member of a structure through a NULL pointer is undefined behaviour.

bzt wrote:
Let's not get into the part that in freestanding mode accessing address 0 (...) is not just perfectly valid, but even required on some architectures.

Where did you get that from? Freestanding or not, it is undefined behaviour to access anything through a NULL pointer. You are confusing the C standard with implementation details.

Null pointer and address 0x0 are not the same. It sounds like you think they are.

From the C99 standard section 6.3.2.3:

Quote:
An integer constant expression with the value 0, or such an expression cast to type void *, is called a null pointer constant. If a null pointer constant is converted to a pointer type, the resulting pointer, called a null pointer, is guaranteed to compare unequal to a pointer to any object or function.

Regarding integer to pointer conversion:

Quote:
An integer may be converted to any pointer type. Except as previously specified [i.e. for the case of null pointer constant], the result is implementation-defined, might not be correctly aligned, might not point to an entity of the referenced type, and might be a trap representation†.

So GCC decided on x86 that accessing something through the NULL pointer results in UD2 opcodes. That is implementation-defined behaviour and they are allowed to do this.

_________________
https://github.com/kiznit/rainbow-os


Last edited by kzinti on Tue Feb 23, 2021 11:17 am, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Announcing POSIX-UEFI
PostPosted: Tue Feb 23, 2021 11:17 am 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 1303
kzinti wrote:
You are the one not listening. Accessing any member of a structure through a NULL pointer is undefined behaviour. It doesn't matter whether or not happens to be defined as "address 0".
Then again: you are in freestanding mode, there's no such thing as NULL pointer. As a consequence, you cannot dereference something that doesn't even exists. And memorymap_t.bda isn't address 0 either (but even if it were, since NULL isn't defined, you can't dereference NULL).

kzinti wrote:
Where did you get that from?
Architecture specifications. There are MANY architectures where address 0 is used for something. For example: ARM32 vector table, x86 real mode IVT, C64 CPU configuration etc. etc. etc.

kzinti wrote:
Null pointer and address 0x0 are not the same. It sounds like you think they are.
Quite the contrary. You seem to think like that. I'm saying that NULL isn't defined at all (neither 0 nor otherwise), and offsetof(bda) != 0 either, yet you got UD2 instruction, so that's a bug.

Cheers,
bzt


Top
 Profile  
 
 Post subject: Re: Announcing POSIX-UEFI
PostPosted: Tue Feb 23, 2021 11:27 am 
Offline
Member
Member

Joined: Mon Feb 02, 2015 7:11 pm
Posts: 572
Dear bzt,

bzt wrote:
(...) in freestanding mode, there's no such thing as NULL pointer.

This is non-sense.

bzt wrote:
Architecture specifications. There are MANY architectures where address 0 is used for something. For example: ARM32 vector table, x86 real mode IVT, C64 CPU configuration, Amiga Chip memory etc. etc. etc.

Again you are confusing a null pointer with address 0x0. They are not the same. The fact that some compilers allow you to access address 0x0 through a NULL pointer is implementation specific behaviour which falls under the category of "undefined behaviour". This is permitted by the standard but not required.

bzt wrote:
I'm saying that NULL isn't defined at all (neither 0 nor otherwise), and offsetof(bda) != 0 either, yet you got UD2 instruction, so that's a bug.


NULL is defined by the standard whether or not you use freestanding mode. offsetof(type, member) has nothing to do with any of this. Applying the resulting offset to a NULL pointer is undefined behaviour. Accessing any member of a structure through the NULL pointer is undefined behaviour.

There is no bug going on here, just undefined behaviour that doesn't happen to align with what you want it to be.

_________________
https://github.com/kiznit/rainbow-os


Top
 Profile  
 
 Post subject: Re: Announcing POSIX-UEFI
PostPosted: Tue Feb 23, 2021 12:10 pm 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 1303
kzinti wrote:
bzt wrote:
(...) in freestanding mode, there's no such thing as NULL pointer.
This is non-sense.
No, not at all. NULL is defined by libc not by the compiler, take a look at musl stdlib.h for example.

kzinti wrote:
bzt wrote:
Architecture specifications. There are MANY architectures where address 0 is used for something. For example: ARM32 vector table, x86 real mode IVT, C64 CPU configuration, Amiga Chip memory etc. etc. etc.
Again you are confusing a null pointer with address 0x0. They are not the same.
Why do you think that? I was speaking about computer memory addresses, which has nothing to do with any languages, let alone a C specific definition. Strictly about computer memory addresses, nothing else.

kzinti wrote:
NULL is defined by the standard whether or not you use freestanding mode.
Nope, see the musl stdlib.h link above. NULL is defined in a libc header. Therefore in freestanding mode, when you don't have libc neither its headers, you simply don't have the NULL define. Simple as that. But try it out if you don't believe me!

kzinti wrote:
offsetof(type, member) has nothing to do with any of this.
Yes it does. You just don't understand what I'm saying. Look:
Code:
var = memmap.ivt; -> mov eax, 0; mov eax, [eax]; mov [var], eax
var = memmap.bda; -> mov eax, 1024; mov eax, [eax]; mov [var], eax         <-- no NULL in the C code, and no 0 address in the Assembly
So it doesn't matter if NULL is defined or if it's 0 or not, because neither the C code, nor the (correctly) compiled Assembly contains any of those.

Cheers,
bzt


Top
 Profile  
 
 Post subject: Re: Announcing POSIX-UEFI
PostPosted: Tue Feb 23, 2021 12:20 pm 
Online
Member
Member
User avatar

Joined: Fri Nov 22, 2019 5:46 am
Posts: 576
Hey, guys. It's nice that you keep polite. But you are not convincing each other. So this discussion can go on forever and ever and not leading anywhere...

Greetings
Peter

_________________
The "fanboy"
https://wiki.osdev.org/User:PeterX


Top
 Profile  
 
 Post subject: Re: Announcing POSIX-UEFI
PostPosted: Tue Feb 23, 2021 12:51 pm 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 855
bzt wrote:
No, not at all. NULL is defined by libc not by the compiler
It is defined by the language standard, actually. But you are mixing up concepts, again. NULL is the name of a macro that is defined in stdlib.h in hosted implementations, that expands to a null pointer constant. A null pointer constant is a constant integer expression with value 0 or such an expression cast to a pointer type. Due to this ambiguity, the NULL macro is pretty much useless, by the way, because any place where the type can be inferred based on context, NULL and 0 are equivalent, and any place where the type cannot be inferred, you need an explicit conversion anyway.

Back to the topic: While the NULL macro is only defined to exist in hosted implementations, null pointers are not so constrained, and are any pointer that compare equal to zero. How your implementation chooses to compare pointers and the integer zero is implementation-defined. Nonetheless, the null pointer is guaranteed to be unequal to the address of any object. Therefore, having an object with a null address is not even undefined, it is breaking the standard.

bzt wrote:
Why do you think that? I was speaking about computer memory addresses, which has nothing to do with any languages, let alone a C specific definition. Strictly about computer memory addresses, nothing else.
And you are free to manipulate such addresses as you see fit, in a language and implementation that supports that. However, a C implementation that implements null pointers as pointers to address zero is no such thing. You will have to either change the implementation (pointer to address negative one used to be a popular choice, once upon a time), or use a different language, such as assembly.

C is a more abstract language than people like to think. You are not programming your CPU, you are programming some abstract machine that will at some point be mapped onto your CPU. This indirection means that concepts that have a definite meaning in your world and in the world of your CPU might still be utterly meaningless in the middle step, and be implemented in a manner you did not foresee. In this case, having an object at address zero is to the C machine as a four-sided triangle is to you. It is an utterly alien concept.
PeterX wrote:
Hey, guys. It's nice that you keep polite. But you are not convincing each other. So this discussion can go on forever and ever and not leading anywhere...
Agreed, but... hope dies last, alright?

_________________
Thou hast outraged, not insulted me, sir; but for that I ask thee not to beware of Starbuck; thou wouldst but laugh; but let Ahab beware of Ahab; beware of thyself, old man.


Top
 Profile  
 
 Post subject: Re: Announcing POSIX-UEFI
PostPosted: Tue Feb 23, 2021 1:20 pm 
Offline
Member
Member

Joined: Mon Feb 02, 2015 7:11 pm
Posts: 572
PeterX wrote:
Hey, guys. It's nice that you keep polite. But you are not convincing each other. So this discussion can go on forever and ever and not leading anywhere...

Someone on the Internet is wrong. We have to correct this.

_________________
https://github.com/kiznit/rainbow-os


Top
 Profile  
 
 Post subject: Re: Announcing POSIX-UEFI
PostPosted: Tue Feb 23, 2021 1:56 pm 
Online
Member
Member
User avatar

Joined: Fri Nov 22, 2019 5:46 am
Posts: 576
kzinti wrote:
PeterX wrote:
Hey, guys. It's nice that you keep polite. But you are not convincing each other. So this discussion can go on forever and ever and not leading anywhere...

Someone on the Internet is wrong. We have to correct this.

Haha, nice motto! XD

_________________
The "fanboy"
https://wiki.osdev.org/User:PeterX


Top
 Profile  
 
 Post subject: Re: Announcing POSIX-UEFI
PostPosted: Tue Feb 23, 2021 5:44 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 2815
bzt wrote:
No, I have no smaller example unfortunately. You have all the flags in the repo. To reproduce the error, simply replace the ifdef in uefi line 1344 with
Code:
#define MEMFNC(a) a
This will avoid the name prefix workaround, and should trigger the issue in stdio.o's fopen. Just compile one of the examples, like 05_file with USE_GCC=1, and then check out "objdump -d uefi/stdio.o".

I tried it and was not able to reproduce the bug. Am I supposed to pass any additional flags or use a particular version of GCC?

bzt wrote:
NULL is defined in a libc header. Therefore in freestanding mode, when you don't have libc neither its headers, you simply don't have the NULL define.

NULL is defined in stddef.h, which is a freestanding header.


Top
 Profile  
 
 Post subject: Re: Announcing POSIX-UEFI
PostPosted: Wed Feb 24, 2021 6:03 am 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 1303
kzinti wrote:
Someone on the Internet is wrong. We have to correct this.
Someone on a technical forum is talking against facts and empirical evidence. Any real scientist would want to correct that.

By your reasoning, there's no point in correcting tutorials either, right? Those are just someone on the internet being wrong, right? And since you were so consistent, I'd like to kindly ask you to show us one implementation that defines NULL other than 0. Just one implementation! In a new topic, DO NOT derail this topic any more!

Octocontrabass wrote:
NULL is defined in stddef.h, which is a freestanding header.
How is this different to what I am saying??? First, stddef.h is provided by libc too (see musl, don't make a mistake that gcc has some built-ins, that's implementation specific and not language standard, use -nostdinc). Second, as I've shown, musl defines it in stdlib.h too (and in many others like time.h too for some reason). Third, the point is, it is defined in a header included by the programmer, and not by the compiler. Without a header, there's no NULL, and without NULL there's no NULL dereference, period.

Rant: why do I have a feeling that my topic is run over by paid trolls trying to derail it just because they are jealous that my POSIX-UEFI is actually simpler and better than the bloated EDK2? I don't want to offend anyone, but this is exactly how it looks like.

Is it really too hard to focus on POSIX-UEFI? Please keep on topic!

Cheers,
bzt


Top
 Profile  
 
 Post subject: Re: Announcing POSIX-UEFI
PostPosted: Wed Feb 24, 2021 6:17 am 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 1303
Octocontrabass wrote:
I tried it and was not able to reproduce the bug.
Good for you! Which version are you using? I'll try to update to the latest gcc and retry.
Octocontrabass wrote:
Am I supposed to pass any additional flags or use a particular version of GCC?
No. You just have to set USE_GCC in 05_file/Makefile. On test, 05_file/file.efi would stuck in an infinite loop (at memcmp in fopen function). By renaming memcmp to __uefi_memcmp everything works with gcc as expected.

Cheers,
bzt


Top
 Profile  
 
 Post subject: Re: Announcing POSIX-UEFI
PostPosted: Wed Feb 24, 2021 6:46 am 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 1303
Hm, this is interesting. If I use the distro provided gcc, then with and without -fno-builtin the problem exists, file.efi freeze on fopen when memcmp called. However if I use the gcc that I've compiled from the source, then with and without -fno-builtin the problem doesn't exists and file.efi works. Both gcc being the same version 10.2.0.
Code:
gcc (GCC) 10.2.0
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
I've upgraded my distro. Gcc was upgraded to 10.2.0-6, and now that works too (gcc still reports the same 10.2.0 version, but the package has an additional "-6" which increased since the last upgrade). I guess it's okay to remove the workaround, it's not needed any more. At least, let's hope so.

Cheers,
bzt


Top
 Profile  
 
 Post subject: Re: Announcing POSIX-UEFI
PostPosted: Wed Feb 24, 2021 11:54 am 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 855
bzt wrote:
Someone on a technical forum is talking against facts and empirical evidence. Any real scientist would want to correct that.
Any real scientist would be to busy with their science to care about one Hungarian professor misinterpreting the C standard. Not when on the internet, people misinterpreting the C standard are a dime a dozen.
bzt wrote:
How is this different to what I am saying???
stddef.h is required to be there in a freestanding implementation. You said the NULL macro does not exist in a freestanding implementation.
bzt wrote:
Rant: why do I have a feeling that my topic is run over by paid trolls trying to derail it just because they are jealous that my POSIX-UEFI is actually simpler and better than the bloated EDK2? I don't want to offend anyone, but this is exactly how it looks like.
Because obviously nobody could ever disagree with you without being paid to do so, right? You are correct that the topic has strayed from your invention, though, maybe we ought to remedy that.

Oh, and saying something offensive and then claiming that offence was not your intent really only works at most once.

_________________
Thou hast outraged, not insulted me, sir; but for that I ask thee not to beware of Starbuck; thou wouldst but laugh; but let Ahab beware of Ahab; beware of thyself, old man.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 85 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6  Next

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group