new linux standard

[kerravon]

So based on another post, I want to be able to write Linux apps such that they run under both PDOS-generic and real Linux.

Linux is set in stone, but PDOS-generic isn't. And also I have no problem invalidating all existing Linux apps, including my own. I never agreed on a standard, and if some professor or the POSIX committee or Thomo/Lillee/Richie Rich didn't allow for this 50 years ago or whatever - that's not my fault.

So I'll start with x64, since it is the most complicated. And this is just tentative first thoughts.

PDOS-generic may be running in userspace on Windows x64 with no access to interrupts, so I can't intercept the "syscall" instruction (whatever that is). I instead need all Properly Written (TM) Linux apps to gracefully accept a linux_syscall() callback override which would be part of the PDOS-generic OS structure.

But the app needs to know whether it is running under PDOS-generic or real Linux. And it can't e.g. look at an environment variable by doing a syscall, because syscall may not be allowed - that's the whole point.

So PDOS-generic can call the Linux app (entry point funcptr) with:

funcptr( (void *)(ptrdiff_t)-1, (void *)(ptrdiff_t)-2, (void *)(ptrdiff_t)-3, &OS_BLOCK);

The startup assembler code for the Linux app will push the following registers onto the stack:

rcx, rdx, r8, r9, (these first 4 are MSABI) rdi, rsi

rdi, rsi, rdx, rcx, r8, r9 is Linux calling convention for parameters.

I may wish to use either of those calling conventions depending on what compiler I have available.

I then pass a pointer to the stack to the C startup code, so it can figure out the environment in C instead of assembler.

On real Linux x64, I don't think any of those registers are guaranteed to be anything, so that could be a problem.

But otherwise, I can look for the -1, -2, -3 sequence in either rcx/rdx/r8 or rdi/rsi/rdx so I know that the caller is PDOS-generic and I know where to find the OS pointer and I can do my callbacks instead of real syscalls. If that sequence is not detected, I know it is real Linux and carry on as usual doing syscall, and also I know the location of argc/argv if I am forced to get that from the stack instead of /proc/.../cmdline (I use this so that PDOS/386 has a chance to provide parameters to a Linux app without creating a stupid stack - PDOS/386 handles a real INT 80H).


For 80386, PDOS-generic is still going to do the same call, but now everything is on the stack instead of in registers.

I again have assembler startup code to push the current stack pointer onto the stack as the parameter to the C startup code.

If this is PDOS-generic, the stack will have a return address followed by -1, -2, -3, OS. The OS will have an int80() callback function to override the real INT 80H. ie all Properly Written (TM) Linux apps are expected to call the callback function if the OS (no matter who - maybe real Linux will be updated one day for whatever reason someone has to not want a real interrupt done) tells it (via this mechanism) to do a callback, not a real interrupt.

If this is Linux I will have argc immediately, and I will not be able to differentiate that from a return address.

Real Linux will then have argc, and the value of -2 will not be (reasonably) valid. Possibly I could have an additional -1 parameter to be sure, so perhaps a sequence of -1, -1, -2, OS. Another possibility is to have three -1 to make sure argv[0] is an invalid pointer. Either -1, -1, -1, OS or -1, -1, -1, -2, OS. That last one will cause x64 to spill from registers into the stack, but that can be coped with, and may even be a good thing. Perhaps even having 8 or more parameters would be good for x64 detection purposes.

Whatever the number and value of parameters are, I expect it to be consistent regardless of ARM32/64 x86/x64, so that the PDOS-generic OS can be portable for the PDOS-generic app point of view. If it is acting as a mini Linux clone or mini Atari clone, I can have extra/different callback functions made available, but the PDOS-generic apps remain the same (ie with a view to setting the PDOS-generic API in stone too - PDOS-generic apps need to know to ignore the first x parameters and just get the parameter with the OS structure which contains everything they need).

Just initial thoughts.

[iansjack]

So based on another post, I want to be able to write Linux apps such that they run under both PDOS-generic and real Linux.

I’m afraid that my reaction is - why?

Why add extra layers, inefficiencies, and complications when you can just recompile programs for the target OS?

“The good thing about standards is that there are so many to choose from.”

[kerravon]

So based on another post, I want to be able to write Linux apps such that they run under both PDOS-generic and real Linux.

I’m afraid that my reaction is - why?

Why add extra layers, inefficiencies, and complications when you can just recompile programs for the target OS?

So that I can have a binary that works on both Linux and Linux-clone, instead of needing to find the source code and build a PDOS-generic executable.

It's not unusual to want binary compatibility.

[iansjack]

You are writing the programs and you need to search for the source code?

It’s not unusual to want binary compatibility

I’d disagree with your premise. IMO users want a program that will run on Windows, Linux, Mac OS, FreeBSD, etc., but very few have a desire for a single binary that will run on all four. Personally I prefer binaries that have benn compiled for, and take advantage of the efficiencies that entails, a single operating system.

[kerravon]

You are writing the programs and you need to search for the source code?

It's not necessarily me doing it. If someone has compiled a set of Linux executables, then yes, they will need to recompile for another environment.

It’s not unusual to want binary compatibility

I’d disagree with your premise. IMO users want a program that will run on Windows, Linux, Mac OS, FreeBSD, etc., but very few have a desire for a single binary that will run on all four. Personally I prefer binaries that have benn compiled for, and take advantage of the efficiencies that entails, a single operating system.

The program nominally only runs on Linux.

Or a clone of Linux.

I am providing a clone of Linux, similar to how Wine provides a clone of Windows.

For my clone to work, I need a slight change to the Linux standard.

I don't know (or care) if Wine requires privilege. My (proposed) Linux clone doesn't need privilege because it avoids having to do an INT 80H etc. The same way as my Atari mini-clone avoids a 68000 trap. By first changing the Atari standard.

[Octocontrabass]

I am providing a clone of Linux, similar to how Wine provides a clone of Windows.

The entire point of Wine is that it runs existing Windows binaries. If you needed special "Wine-compatible" binaries to use Wine, nobody would use Wine.

What you're suggesting is the equivalent of Wine requiring special "Wine-compatible" binaries.

[kerravon]

I am providing a clone of Linux, similar to how Wine provides a clone of Windows.

The entire point of Wine is that it runs existing Windows binaries. If you needed special "Wine-compatible" binaries to use Wine, nobody would use Wine.

I would. I would be happy to modify the way I build Windows executables so that Wine can handle them. E.g. not using undocumented APIs which is the main reason Wine won't handle existing binaries (I believe). So long as it is a minor change, and Wine doesn't expect a complete rebuild of my Windows apps to include an effective second copy of the application that uses Linux calls. I don't want to do Linux calls, I want to do Windows calls.

I guess that's the key difference. I'm interested in clones. I'm interested in "common subsets" of clones. I'm willing to build my executables so that they run on multiple Windows clones - be it real Windows, Linspire, ReactOS, PDOS/386, Wine, MSDOS 4.0+HX.

I may decide that a particular clone is just too bad - e.g. maybe ReactOS is just too bad - so I abandon that clone. But so long as it is "reasonable", I'm happy to take care when building my binaries to cover as many clones as possible. Ideally this should have been sorted out in 1993 - or even earlier - Win32 could have been made available on the Amiga in 1985 - but I'm not worried about being "late to the party".

Again - this is my goal, not someone else's goal. My expectations of Wine. I'm happy to listen to the manufacturers of Wine if they need assistance. If others demand 100% MS compatibility - take it or leave it - so be it - that's someone else's goal, not mine.

What you're suggesting is the equivalent of Wine requiring special "Wine-compatible" binaries.

Yes indeed. And I don't have a problem with that - or at least, I'm happy to ask the question - "what do you need, Mr Wine?". If they ask for too much, forget it. It so happens that the Wine people asked for nothing more than to use documented APIs - I can do that. If they ask for a little bit more - tell me what it is exactly and then I'll decide.


EDIT: thanks for teasing out that "unstated assumption". I don't know what unstated assumptions I am running on until I see a disconnect during a discussion.

[iansjack]

For my clone to work, I need a slight change to the Linux standard.

If I understand correctly you are asking the Linux community to adopt a new standard so that their programs will run on your particular clone, albeit less efficiently than they run on Linux.

Given that Linux is a free, well-developed operating system that runs on just an about any computer - why would any sane person go to that trouble to accommodate your requirements. Most would say “just rewrite your clone so that it works”.

[Octocontrabass]

The entire point of Wine is that it runs existing Windows binaries. If you needed special "Wine-compatible" binaries to use Wine, nobody would use Wine.

I would.

If you needed special "Wine-compatible" binaries to use Wine, you would be the only person using Wine.

I would be happy to modify the way I build Windows executables so that Wine can handle them.

But the whole point of Wine (and any other binary compatibility layer) is running binaries that you can't modify because you don't have the source code. If you had the source code, you wouldn't need Wine because you could build a native Linux binary instead.

[kerravon]

For my clone to work, I need a slight change to the Linux standard.

If I understand correctly you are asking the Linux community to adopt a new standard so that their programs will run on your particular clone,

Not necessarily "the Linux community". A subset of that. Even if the subset is 1.

albeit less efficiently than they run on Linux.

That's not correct. The Linux application will run at full native speed.

Given that Linux is a free, well-developed operating system that runs on just an about any computer - why would any sane person go to that trouble to accommodate your requirements. Most would say “just rewrite your clone so that it works”.

1. It only requires a change to the C library to work. Ignoring the question of sanity, I have control of a C library. I will do the work. No-one can stop me.

2. All applications that are subsequently linked against PDPCLIB will then be using the new standard.

3. Whether other C library authors adopt the new Linux standard or not is beyond my control. It's also not something I particularly care about.

4. I will not be rewriting my clone (PDOS-generic with Linux support) so that it "works". If you want to fork it, and make it privileged or whatever else you may need to do to get it to work - it's all public domain - go for your life. But me - nope. I'll tell you to use an appropriate C library. Even if that is exactly 1 C library.

If you insist that I'm the only person in the entire world who has this requirement, and no-one else will ever come along in the future who adopts this mechanism - so be it - your crystal ball - go for it. I'm not trying to win Miss America. I'm trying to stand up an unprivileged Linux mini-clone.

Ditto for an MVS clone, which uses SVCs (same as INT), and requires privilege if you want to install or intercept an SVC. Not everyone has privilege on MVS, but everyone has unprivileged on MVS. Although that would be if I wanted to support CMS (another OS) or VSE (another OS) on MVS - both of which use SVCs. Or vice-versa - run a mini MVS clone on VSE. Noting that IBM are no longer the vendor for VSE (now called zNext I think). And noting that CMS (z/VM) is easier (cheaper, or even free as a demo) than MVS too.

Linux on x64 is somewhat of a "proving ground". (another unstated assumption just teased out)

[kerravon]

The entire point of Wine is that it runs existing Windows binaries. If you needed special "Wine-compatible" binaries to use Wine, nobody would use Wine.

I would.

If you needed special "Wine-compatible" binaries to use Wine, you would be the only person using Wine.

I doubt that I am unique worldwide - at least in this respect. And anyhow, it's clearly not true. Wine is already in that situation. The special thing they need is to be bug-free and use only documented features. Some apps can get away with bugs because (by luck) Windows 11 (not necessarily Windows 12) doesn't have an issue with the bug (accessing freed memory or whatever). And some apps can and do get away with using undocumented features, because they do exist in real Windows regardless. That's why we're already in the situation where you have to take care if you wish your app to run on Wine. Even more care is required if you want to run on PDOS/386 too.

I would be happy to modify the way I build Windows executables so that Wine can handle them.

But the whole point of Wine (and any other binary compatibility layer) is running binaries that you can't modify because you don't have the source code. If you had the source code, you wouldn't need Wine because you could build a native Linux binary instead.

No I wouldn't. I have basically standardized on Win32 executables. I consider them to be universal already. They run on MSDOS 4.0 + HX. They run on Wine. They run on OS/2 under my PDOS-generic layer. They run on Windows.

Why would I want to maintain a separate set of binaries that only work on Linux? Answer: I don't.

But someone else may have the opposite opinion. They only want to produce Linux binaries and expect them to work everywhere. I can stand up a mini Linux clone on OS/2. On Windows. On MSDOS 4.0 + HX. For a trivial overhead - like 100k worth of executables. I can't and won't intercept interrupts though. I'm not sure about privilege on OS/2 as to whether I could be stopped from doing that. Windows can stop a user from having privilege though. Which would prevent the installation of WSL, presumably. I don't think lack of administrator privilege will stop me from installing my own unprivileged executables though. At least on a machine used by developers. I think I have literally been in a workplace where I was a developer but didn't have admin access - I think it's not unusual.

[iansjack]

Whether other C library authors adopt the new Linux standard or not is beyond my control. It's also not something I particularly care about.

I think that your definition of "standard" must differ from mine.

I'm still puzzled as to why you don't just compile your programs to run on your OS. Why do you even care about Linux?

I will not be rewriting my clone (PDOS-generic with Linux support) so that it "works". If you want to fork it, and make it privileged or whatever else you may need to do to get it to work - it's all public domain - go for your life.

I'll pass on that invitation. If I want to run a Linux program on - say - an IBM iSeries or my PPC Mac mini I'll just use Linux.

[kerravon]

Whether other C library authors adopt the new Linux standard or not is beyond my control. It's also not something I particularly care about.

I think that your definition of "standard" must differ from mine.

I'm still puzzled as to why you don't just compile your programs to run on your OS. Why do you even care about Linux?

I might get a job on Linux and want to bring my executables with me.

I will not be rewriting my clone (PDOS-generic with Linux support) so that it "works". If you want to fork it, and make it privileged or whatever else you may need to do to get it to work - it's all public domain - go for your life.

I'll pass on that invitation. If I want to run a Linux program on - say - an IBM iSeries or my PPC Mac mini I'll just use Linux.

That's not actually the scenario. You would still be on an x64 PC in this case.

And you may not have real Linux available. You may be on a Windows machine. And you either have to turn down the job offer, or go there without your executables. My Linux executables will work on the Windows machine. I just need to fire up the PDOS-generic-with-Linux-clone environment and they all work. Even if I don't have privilege.

[iansjack]

And you may not have real Linux available. You may be on a Windows machine. And you either have to turn down the job offer, or go there without your executables. My Linux executables will work on the Windows machine. I just need to fire up the PDOS-generic-with-Linux-clone environment and they all work. Even if I don't have privilege.

That would be your last day in that employment!

Seriously, your Linux executables will not work on that Windows machine. In the real world, if an employer wants to restrict what you run on a Windows machine then it's not a question of whether you have privilege or not. Have you ever actually worked in a corporate Windows environment? An administrator can easily restrict what you can run to a company-approved set of executables not programs that any Tom, Richard (really!), or Harry brings into the environment. If they want to let you run Linux programs on your Windows machine they'll give you WSL.

But surely this is all hypothetical. With your views on non-PD software you are never going to "get a job on Linux" or work in a Windows shop.

[kerravon]

And you may not have real Linux available. You may be on a Windows machine. And you either have to turn down the job offer, or go there without your executables. My Linux executables will work on the Windows machine. I just need to fire up the PDOS-generic-with-Linux-clone environment and they all work. Even if I don't have privilege.

That would be your last day in that employment!

No, not necessarily. You don't speak for every employer in the world. First of all a small company may not even have anything significant written down. And for a larger company, a lot of the world is just about covering your @$$. Even if something is written down, it's sometimes enough to just ask your boss if you can install xyz to help you with your job. In one interview I went to I asked in advance if I could bring my own editor as source code, and they said on the development machines it wasn't a problem, but the production machine I couldn't do that - or maybe they weren't authorized to approve that. Different companies have different policies.

Seriously, your Linux executables will not work on that Windows machine.

Yes they will. That's exactly what PDOS-generic-with-Linux-clone-extension is for.

In the real world, if an employer wants to restrict what you run on a Windows machine then it's not a question of whether you have privilege or not.

It can be.

Have you ever actually worked in a corporate Windows environment?

Yes I have. Multiple.

An administrator can easily restrict what you can run to a company-approved set of executables not programs that any Tom, Richard (really!), or Harry brings into the environment.

Yes, they CAN, but normally they DON'T. Not if you're doing a development role, anyway - you need to be able to produce new executables.

If they want to let you run Linux programs on your Windows machine they'll give you WSL.

They don't want you to run Linux programs. They are paying you to develop Windows executables. But if you are a Linux fan - hey - bring all your Linux executables with you - they'll work with the PDOS-generic layer added. You can download executables from multiple places if you want. They need to be built with PDPCLIB or equivalent to be THIS portable, obviously.

But surely this is all hypothetical. With your views on non-PD software you are never going to "get a job on Linux" or work in a Windows shop.

Pardon? Where on earth else am I going to get a job? And what views are you talking about? I'll work on any environment you pay me to work on.

I'm not willing to work FOR FREE on non-PD software though. Not normally, anyway. That's hardly surprising is it?!