How common are malloc(0) and realloc(p, 0)?

[nullplan]

Hi all,

there's an ongoing thread on the musl mailing list right now (plus a couple of others) about malloc(0) and realloc(p, 0), specifically about making them consistent with each other. And while reading that thread I was constantly wondering if I was taking crazy pills.

In all my life, I have seen a lot of C code, I have written a lot of C code, and I have debugged quite a lot of C code. And in that time, never, not once, have I had occasion to use malloc(0) for anything. But suddenly there's this dude really passionately arguing that malloc should be required to attempt to allocate a zero-sized object if called with 0 as argument. And other people I previously respected agree with him! Why in three devils' name should malloc() do such a thing? C doesn't have any zero-sized types, and so cannot have zero-sized objects. Consequently, I made my version of malloc return NULL/EINVAL when someone attempts to allocate 0 bytes.

And realloc(p, 0) I have never understood. What is the point? realloc() can only return NULL or a valid pointer, right? And NULL means an error occurred and the original pointer is still valid. So realloc(p, 0) cannot have the meaning of freeing the pointer, because then p is no longer a valid pointer to return, and it cannot return NULL because that would mean error and no side effects. My version of realloc returns NULL/EINVAL and has no side effects.

So what about you guys? Have you ever seen malloc(0) and realloc(p, 0) honestly used for anything in an application? Or is that thread just an academic ivory tower discussion of the highest order?

[Octocontrabass]

Why in three devils' name should malloc() do such a thing?

The primary use, as far as I can tell, is being compliant with the C standard without requiring different versions of realloc depending on the version of the C standard.

I mean, it could hypothetically also be useful if you need a bunch of unique handles that each may or may not have additional data associated with it and don't want to allocate a separate pointer for each handle and eat the costs of an extra level of indirection to access the optional per-handle data... but that's only hypothetical, I don't think I've seen a situation like that in reality.

realloc() can only return NULL or a valid pointer, right? And NULL means an error occurred and the original pointer is still valid.

C90 says that realloc(p, 0) will always free the original pointer no matter what it returns.

C99 and C11 say that realloc(p, 0) won't free the original pointer if it returns NULL.

C17 says it's implementation-defined whether realloc(p, 0) will free the original pointer if it returns NULL.

So, allowing zero-sized allocations means there's no disagreement on whether the original pointer is still valid when realloc returns NULL.

(C23 says realloc(p, 0) is undefined behavior...)

[nullplan]

I mean, it could hypothetically also be useful if you need a bunch of unique handles that each may or may not have additional data associated with it and don't want to allocate a separate pointer for each handle and eat the costs of an extra level of indirection to access the optional per-handle data... but that's only hypothetical, I don't think I've seen a situation like that in reality.

What you are describing, I would just implement as an array of pointers, and set the pointers NULL if no additional data is present.

C90 says that realloc(p, 0) will always free the original pointer no matter what it returns.

That explains why it was off my radar: I don't aim that low.

Jokes aside, the progression in your post tells me the feature is less useful every year, and best avoided in applications for that reason. And in my implementation, I will keep everything just the way it is now. Thank you, this has been very helpful.

[PavelChekov]

The only application I can think of where malloc(0) would be truly useful is if you have a loop that expands a dynamic array with realloc every cycle, and you want a pointer to realloc to start with, because the first element would be initialized as part of the loop. I'm not really sure how allocating zero bytes would actually work, though.

[Octocontrabass]

You don't need malloc(0) for that, realloc(NULL,size) is equivalent to malloc(size) so you can initialize the pointer to NULL.

[nullplan]

As a test case, I have scoured the busybox source code for uses of this, figuring it would be a codebase with a certain reputation and of decent complexity. Busybox wraps all calls to realloc() into its own xrealloc() wrapper that does the right thing, actually, namely to treat NULL returns as a fatal error only if the size was nonzero (although that still means it invokes undefined behavior in C23).

But anyway, in all of busybox, I found a single use of xrealloc(p, 0): In its RPM code, if an RPM file contains 0 tags, then it will reallocate the NULL tags array to a size of 0 tags. That was the only place I could find where it might call xrealloc with a second argument of 0, assuming an absence of overflows (and in the presence of overflows, most of the code just breaks and crashes, anyway).

[PavelChekov]

You don't need malloc(0) for that, realloc(NULL,size) is equivalent to malloc(size) so you can initialize the pointer to NULL.

Wow, thank you! I've been doing too much for years.

[nzmjx]

No, I never seen malloc(0) or realloc(p, 0) in my career of 23-something years.

[Demindiro]

I've just yesterday encountered an instance of not just realloc(p, 0) but realloc(NULL, 0). And the program expects a non-null return value!

It isn't an obscure program either: the program in question is grep. (grep --version = grep (GNU grep) 3.7)

Proof: compile this with `gcc trace.c -shared -fPIC`, then run `LD_PRELOAD=./a.out grep wakwak`:

Code: Select all

// https://stackoverflow.com/a/5228735
#define _GNU_SOURCE
#include <dlfcn.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>

static void printhex(size_t x) {
	char p[32], *q = p + 32;
	do {
		int c = x & 15;
		x >>= 4;
		*--q = c < 10 ? '0' + c : 'a' - 10 + c;
	} while (x != 0);
	*--q = 'x';
	*--q = '0';
	write(2, q, p + 32 - q);
}

static void putstr(const char *s) {
	write(2, s, strlen(s));
}

static void trace(void *p, size_t len, void *res) {
	putstr("realloc(");
	printhex((size_t)p);
	putstr(",");
	printhex(len);
	putstr(") = ");
	printhex((size_t)res);
	if (p == NULL && len == 0)
		putstr("  <--------");
	putstr("\n");
}

void *realloc(void *p, size_t len) {
    static void *(*og_realloc)(void *, size_t);

    if (og_realloc == NULL)
        og_realloc = dlsym(RTLD_NEXT, "realloc");

	void *res = (og_realloc)(p, len);
	trace(p, len, res);
	return res;
}

Output:

Code: Select all

realloc(0x0,0x640) = 0x5635bd392860
realloc(0x0,0x400) = 0x5635bd392eb0
realloc(0x5635bd392eb0,0x800) = 0x5635bd392eb0
realloc(0x0,0x78) = 0x5635bd391a90
realloc(0x5635bd392490,0x80) = 0x5635bd392490
realloc(0x0,0xe0) = 0x5635bd3936c0
realloc(0x0,0x10) = 0x5635bd393fc0
realloc(0x0,0x80) = 0x5635bd393fe0
realloc(0x0,0x10) = 0x5635bd393f00
realloc(0x5635bd394070,0x10) = 0x5635bd394070
realloc(0x5635bd394150,0x10) = 0x5635bd394150
realloc(0x5635bd394070,0x10) = 0x5635bd394070
realloc(0x5635bd394070,0x10) = 0x5635bd394070
realloc(0x5635bd394090,0x7) = 0x5635bd394090
realloc(0x5635bd3940b0,0x7) = 0x5635bd3940b0
realloc(0x5635bd3940d0,0x7) = 0x5635bd3940d0
realloc(0x0,0x0) = 0x5635bd394070  <--------
realloc(0x0,0xe0) = 0x5635bd3954f0
realloc(0x0,0x100) = 0x5635bd394bc0
realloc(0x0,0x48) = 0x5635bd394cd0
realloc(0x0,0x80) = 0x5635bd394d20
realloc(0x0,0x80) = 0x5635bd394f20
realloc(0x0,0x80) = 0x5635bd394fb0
realloc(0x0,0x80) = 0x5635bd395040
realloc(0x0,0x80) = 0x5635bd3950d0
realloc(0x0,0x80) = 0x5635bd395160
realloc(0x0,0x80) = 0x5635bd3951f0
realloc(0x0,0x80) = 0x5635bd395280
realloc(0x0,0x80) = 0x5635bd395310
realloc(0x0,0x78) = 0x5635bd3953f0
realloc(0x0,0x40) = 0x5635bd395470
realloc(0x0,0xc0) = 0x5635bd3955e0
realloc(0x0,0x20) = 0x5635bd3938c0
realloc(0x0,0x10) = 0x5635bd394070
realloc(0x0,0x48) = 0x5635bd395470
realloc(0x0,0x10) = 0x5635bd3940f0

Guess what happens if realloc(0, 0) decides to return NULL?

Code: Select all

realloc(0x0,0x640) = 0x5618cab9b860
realloc(0x0,0x400) = 0x5618cab9beb0
realloc(0x5618cab9beb0,0x800) = 0x5618cab9beb0
realloc(0x0,0x78) = 0x5618cab9aa90
realloc(0x5618cab9b490,0x80) = 0x5618cab9b490
realloc(0x0,0xe0) = 0x5618cab9c6c0
realloc(0x0,0x10) = 0x5618cab9cfc0
realloc(0x0,0x80) = 0x5618cab9cfe0
realloc(0x0,0x10) = 0x5618cab9cf00
realloc(0x5618cab9d070,0x10) = 0x5618cab9d070
realloc(0x5618cab9d150,0x10) = 0x5618cab9d150
realloc(0x5618cab9d070,0x10) = 0x5618cab9d070
realloc(0x5618cab9d070,0x10) = 0x5618cab9d070
realloc(0x5618cab9d090,0x7) = 0x5618cab9d090
realloc(0x5618cab9d0b0,0x7) = 0x5618cab9d0b0
realloc(0x5618cab9d0d0,0x7) = 0x5618cab9d0d0
realloc(0x0,0x0) = 0x0  <--------
grep: memory exhausted

[Kevin]

So what about you guys? Have you ever seen malloc(0) and realloc(p, 0) honestly used for anything in an application? Or is that thread just an academic ivory tower discussion of the highest order?

I'm surprised that everyone here says they have never seen it. Of course, it's not usually literally written malloc(0), but instead something like malloc(n * size) where n could possibly be 0. You could argue that that should be calloc(), but then calloc() also zeroes the area, which maybe people don't want. Or they just don't think of the function. And it could be more complex expressions than a simple multiplication, too.

Long ago, someone in QEMU had the idea to assert a non-zero size in a malloc wrapper, and the result was that quite a few callers had to be updated to explicitly check for zero first while with an allowed malloc(0) they just worked fine without special casing it.

So yes, I've seen enough malloc() or realloc() calls with expressions that can evaluate to 0.

[nullplan]

I've just yesterday encountered an instance of not just realloc(p, 0) but realloc(NULL, 0). And the program expects a non-null return value!

It isn't an obscure program either: the program in question is grep. (grep --version = grep (GNU grep) 3.7)

I wanted to look at that example more deeply. So I downloaded the grep source code, built it with debug symbols, and added a breakpoint for realloc with $rsi==0. I found that there is a function, dfassbuild(), which apparently creates a copy of a DFA object and resets a bunch of things in there before copying the character classes:

Code: Select all

  *sup = *d;
// ...
  sup->charclasses = xnmalloc (sup->calloc, sizeof *sup->charclasses);
  if (d->cindex)
    {
      memcpy (sup->charclasses, d->charclasses,
              d->cindex * sizeof *sup->charclasses);
    }

It appears they are applying the zero-sized object extensions inconsistently. But anyway, what they are doing here is allocating a zero-sized array if d->calloc is zero (which BTW is undefined behavior, because calloc is a reserved name from the C standard that could also be a macro, but what the hell) and then skip the memcpy if d->cindex is zero. It appears they have the invariant that d->cindex <= d->calloc.

So the condition on avoiding the zero-sized allocation would be different from the one they already use to avoid the zero-sized copy.

But not spending that effort is of course still bad. On imlementations like musl, allocating zero bytes is implemented as allocating one byte instead. So the allocation can fail if the system is out of memory or the process has hit its VM size limit. Whereas setting sup->charclasses to null if d->calloc is zero can never fail.

I'm surprised that everyone here says they have never seen it. Of course, it's not usually literally written malloc(0), but instead something like malloc(n * size) where n could possibly be 0. You could argue that that should be calloc(), but then calloc() also zeroes the area, which maybe people don't want. Or they just don't think of the function. And it could be more complex expressions than a simple multiplication, too.

I would argue that it is best to avoid calling a fallible function for an infallible operation, because it reduces the number of failure cases. Calling calloc() wouldn't actually help, since calling it with zero arguments also just results in the same allocation I am complaining about here.

As an implementer of malloc() or realloc(), I see the request for zero bytes and wonder what they were thinking. Would a human ever go all the way to bank, to go to an ATM and request to withdraw 0€? You'd think he was barking mad. You'd think he should have realized he didn't need any cash before going to the bank. That is what avoiding the zero-sized allocation is about for me. Not the technical issue of avoiding a useless call to the allocator, but the effort to get there. As in the above code, if d->calloc is zero, you can skip the allocation and the copy.

Anyway, my goal in writing a libc is compatibility (else I could just design something completely different), and with so many systems in the past having supported zero-sized allocations, I think I shall have to as well.

[Kevin]

I would argue that it is best to avoid calling a fallible function for an infallible operation, because it reduces the number of failure cases.

Yes and no. You're right, the memory allocation can fail and it isn't strictly necessary here, so we may now error out unnecessarily. But if a single-byte allocation fails, then the chances that the program would have survived much longer without this allocation is minimal.

So you have the choice between an avoidable error in an exceedingly rare case and code that is easier to read and maintain. For me, maintainability wins this comparison in most contexts.

Calling calloc() wouldn't actually help, since calling it with zero arguments also just results in the same allocation I am complaining about here.

True. I only mentioned calloc() because I expected some reaction that malloc(n * size) isn't something you should write in the first place.

As an implementer of malloc() or realloc(), I see the request for zero bytes and wonder what they were thinking. Would a human ever go all the way to bank, to go to an ATM and request to withdraw 0€? You'd think he was barking mad. [...]
Anyway, my goal in writing a libc is compatibility (else I could just design something completely different), and with so many systems in the past having supported zero-sized allocations, I think I shall have to as well.

When implementing a standard library function, your job isn't second-guessing the callers, but implementing the contract as defined in the spec. So you only really have to decide with which spec you want to be compliant (in practical terms, POSIX is probably most useful).

[nullplan]

When implementing a standard library function, your job isn't second-guessing the callers, but implementing the contract as defined in the spec.

The spec offers minimal guidance here, which is why I made this thread. C23 says:

The malloc function allocates space for an object whose size is specified by size and whose
representation is indeterminate.

That is it. That is the entire definition of malloc(). And the other C standards say substantially the same. And look at Octo's first reply for the evolution of realloc() over the years. Then POSIX comes and says malloc(0) is implementation-defined either null or a non-null pointer that can't be deref'ed, but can be reallocated.

Given that C has no zero-sized types, making malloc(0) return null is a valid choice. Strictly speaking, it is undefined behavior in ISO-C, and implementation-defined in POSIX. But then Alejandro Colomar came on various libc related mailing lists claiming that it wasn't, and it was treated as common knowledge and accepted by everyone, including Rich Felker (musl maintainer) and Florian Weimer (glibc maintainer). Which left me terribly confused.

[Kevin]

Standard C says the same thing, that zero-sized allocations return either NULL or a unique pointer that can be freed. Have a look at the text at the beginning of the section for all memory management functions. Quoting from C23, 7.24.3 (but it has been largely the same since at least C99):

If the size of the space requested is zero, the behavior is implementation-defined: either a null pointer is returned [only since C17: to indicate an error], or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object.

So the only thing that is new is that newer versions of the standard explicitly call it an error case when NULL is returned.

Even without the above paragraph, malloc(0) would not be undefined behaviour (UB) in C, but at best unspecified behaviour.

What's different between C and POSIX is mostly that POSIX says something about errno in malloc(), while C leaves it unspecified, which means that the implementation can do anything with errno it wishes. Your implementation of malloc(0) seems compliant with both C and newer POSIX versions, but older POSIX versions don't specify EINVAL as a possible error.

realloc(p, 0) is a mess, and with the recent move of C23 to make it UB, it has become completely useless for applications. But because it means you can do whatever you want, your implementation is compliant with C23, too. Of course, it was already pretty useless in C17:

If size is zero and memory for the new object is not allocated, it is implementation-defined whether the old object is deallocated. If the old object is not deallocated, its value shall be unchanged.

Older versions of C didn't explicitly say anything about zero sizes in realloc(), but I think that effectively implies that returning NULL means that the pointer is not freed. This is what your implementation does, so it's compliant (but probably surprising to some).

From the mailing list thread, it's apparently not compliant with C89, which requires that it's equivalent to free(), but I don't have that text here. This might be incompatible with C99 and C11 (only "might be" because returning NULL isn't explicitly an error there, so freeing and then successfully returning NULL might be ok), but is explicitly allowed again since C17.