OSDev.org

Octocontrabass wrote: ↑Sun Jul 27, 2025 10:11 am

kerravon wrote: ↑Sat Jul 26, 2025 8:32 amThe DBCS people (Japanese, Chinese, Korean) can go and change their culture and use their own alphabet if they wish to use PDOS. I'm not going to pander to them.

You can say you don't want to support internationalization without being rude. (Wasn't MS-DOS available in those languages?)

I do want to support internationalization - but it's going to rely on an alphabet - those 3 groups already have an alphabet (you could possibly quibble about the Chinese) - they're just not using it. I don't know about MSDOS.

kerravon wrote: ↑Sat Jul 26, 2025 8:32 amI'm not interested in nonsense like UTF either.

I agree that UTF-16 is nonsense, but you're stuck with it since that's what Windows uses. UTF-8, on the other hand, is the only half-decent way to encode text.

No - if I am Vietnamese, I expect Vietnamese text to be processed at full speed. A modified VISCII will do that for me without disturbing the microemacs control characters. But I will lose the box-drawing characters - or else lose the English alphabet - unless I switch to 9-bit chars. So I'm inclined to switch to 9-bit chars. I was fairly recently made aware that the box-drawing characters may be more important than I originally thought.

Although I just realized that I may be able to handle the box-drawing characters by putting them on the control key code points for output into the b8000 buffer, and using a (modified) ANSI escape sequence to render them. As the microemacs control keys are input-only.

kerravon wrote: ↑Sun Jul 27, 2025 11:29 am Although I just realized that I may be able to handle the box-drawing characters by putting them on the control key code points for output into the b8000 buffer, and using a (modified) ANSI escape sequence to render them. As the microemacs control keys are input-only.

Oh no - I can't do that. First, I thought the box-drawing characters used 32 bytes, and I realized I needed 6 of those for Vietnamese characters. But I just double-checked and I need 48 (B0 to DF):

https://en.wikipedia.org/wiki/Code_page_437

And here's the VISCII link for good measure:

https://en.wikipedia.org/wiki/VISCII

9-bit chars looks like the way to go.

Then C90 isprint() etc will work for Vietnamese, and I can still have a text-based windowing system.

Although still - I could use graphics mode and an escape sequence for those box-drawing characters.

How about Vietnamese users can choose from either of 3 solutions:

1. Modified hardware to support 9-bit chars

2. Render all box-drawing (modified) ANSI escape sequences as "+" when writing to 0xb8000.

3. Enable VGA graphics mode, so (presumably) slower screen rendering on an XT, but all Vietnamese characters plus box-drawing characters in an application such as msged (distributed with PDOS) can be displayed.

The actual application can use isprint() etc at full speed as C90 locales were designed to do.

Note that C89 was delayed for something like a year in order to add locales. I still haven't gotten around to using them, but they've been at the back of my mind for 35 years.

kerravon wrote: ↑Sun Jul 27, 2025 12:57 pm1. Modified hardware to support 9-bit chars

Why would you need to modify anything? EGA/VGA already supports this.

kerravon wrote: ↑Sun Jul 27, 2025 12:57 pm3. Enable VGA graphics mode, so (presumably) slower screen rendering on an XT, but all Vietnamese characters plus box-drawing characters in an application such as msged (distributed with PDOS) can be displayed.

Speed is irrelevant. Anyone trying to use an old junk PC for word processing will get the cheapest one they can find, not an expensive antique like an XT. The cheapest junk PC will be at least two orders of magnitude faster than the XT. (And now that you mention it, the Japanese version of MS-DOS did use VGA graphics mode.)

kerravon wrote: ↑Sun Jul 27, 2025 12:57 pmI still haven't gotten around to using them, but they've been at the back of my mind for 35 years.

In the past 25 years, dedicated localization libraries have progressed far beyond anything offered by the C standard, and all text encodings have been replaced with UTF-8.

Octocontrabass wrote: ↑Sun Jul 27, 2025 5:02 pm

kerravon wrote: ↑Sun Jul 27, 2025 12:57 pm1. Modified hardware to support 9-bit chars

Why would you need to modify anything? EGA/VGA already supports this.

Pardon? I can write 9 bits to 0xb8000? For a 9-bit index into a font table?

kerravon wrote: ↑Sun Jul 27, 2025 12:57 pm3. Enable VGA graphics mode, so (presumably) slower screen rendering on an XT, but all Vietnamese characters plus box-drawing characters in an application such as msged (distributed with PDOS) can be displayed.

Speed is irrelevant. Anyone trying to use an old junk PC for word processing will get the cheapest one they can find, not an expensive antique like an XT. The cheapest junk PC will be at least two orders of magnitude faster than the XT. (And now that you mention it, the Japanese version of MS-DOS did use VGA graphics mode.)

I'm not trying to cater to a particular "old junk PC" market. I'm trying to honestly compete with the XT in the actual timeframe. And make the 68000 a viable alternative. And just saying that the only thing that is required is a cultural change and software change.

kerravon wrote: ↑Sun Jul 27, 2025 12:57 pmI still haven't gotten around to using them, but they've been at the back of my mind for 35 years.

In the past 25 years, dedicated localization libraries have progressed far beyond anything offered by the C standard, and all text encodings have been replaced with UTF-8.

I don't consider that to be "progress". I'm rechallenging starting from C90.

kerravon wrote: ↑Sun Jul 27, 2025 11:09 pmPardon? I can write 9 bits to 0xb8000? For a 9-bit index into a font table?

Yes. You can program EGA/VGA to expect 9-bit characters using either INT 0x10 AX=0x1103 or the equivalent register manipulation. Bit 3 of the attribute byte acts as the additional character bit, so you may also want to reprogram the attribute controller to ignore that bit (which limits you to only 8 foreground colors).

kerravon wrote: ↑Sun Jul 27, 2025 11:09 pmI'm trying to honestly compete with the XT in the actual timeframe.

The XT was released in 1983. EGA wasn't available until several months after the AT was released in 1984. The XT was discontinued in 1987, at the same time the PS/2 and VGA were released.

If you're aiming for a typical XT, you need to aim for CGA, not VGA.

kerravon wrote: ↑Sun Jul 27, 2025 11:09 pmAnd just saying that the only thing that is required is a cultural change and software change.

Asking for that kind of "cultural change" is insensitive at best and a declaration of war at worst. I suggest you stop asking.

kerravon wrote: ↑Sun Jul 27, 2025 11:09 pmI don't consider that to be "progress". I'm rechallenging starting from C90.

So what do you consider to be progress? Modern computers can handle text in every language at the same time. That sounds like progress to me.

Octocontrabass wrote: ↑Mon Jul 28, 2025 1:03 pm

kerravon wrote: ↑Sun Jul 27, 2025 11:09 pmPardon? I can write 9 bits to 0xb8000? For a 9-bit index into a font table?

Yes. You can program EGA/VGA to expect 9-bit characters using either INT 0x10 AX=0x1103 or the equivalent register manipulation. Bit 3 of the attribute byte acts as the additional character bit, so you may also want to reprogram the attribute controller to ignore that bit (which limits you to only 8 foreground colors).

Ok, thanks. I don't care if I only have monochrome.

kerravon wrote: ↑Sun Jul 27, 2025 11:09 pmI'm trying to honestly compete with the XT in the actual timeframe.

The XT was released in 1983. EGA wasn't available until several months after the AT was released in 1984. The XT was discontinued in 1987, at the same time the PS/2 and VGA were released.

If you're aiming for a typical XT, you need to aim for CGA, not VGA.

I'm not trying to target that either. If you have to add a VGA card to an XT to support Vietnamese - so be it. My concern is whether it is possible to do "fast (modified) ANSI" on an 8086.

kerravon wrote: ↑Sun Jul 27, 2025 11:09 pmAnd just saying that the only thing that is required is a cultural change and software change.

Asking for that kind of "cultural change" is insensitive at best and a declaration of war at worst. I suggest you stop asking.

I didn't so much "ask" for cultural change, as say that that's what I consider to be the correct technical solution, and that PDOS won't cater for DBCS for that reason.

If the Japanese refuse to use PDOS for that reason - I don't care. If they fork it - I don't care. If they get upset that I won't incorporate their DBCS changes into the main PDOS - well - that's what happened when someone tried to introduce C99 stuff into PDPCLIB too. A clash of cultures in C programming. BTW, in a work environment, a couple of people got upset about me converting a K&R C source base into C90 - despite me being the one that had to maintain it. A clash of cultures where I was the one "too far advanced". That was in the mid 90s.

kerravon wrote: ↑Sun Jul 27, 2025 11:09 pmI don't consider that to be "progress". I'm rechallenging starting from C90.

So what do you consider to be progress? Modern computers can handle text in every language at the same time. That sounds like progress to me.

Modern computers require large amounts of memory and processing power to do what they do these days. I don't want to rely on such things. You've heard of "collapse os", right? I'm not saying that I agree (also not saying I don't agree - I simply don't know) that one day we will be scurrying around landfill for Z80s. If we are, Z80s have too little capacity for PDOS to operate. I am conscious that I am "splurging" in aspects of memory - I don't want to break the natural form of the code to cater for limited memory. But I am not willing to splurge on the CPU speed. I expect things to run at 4.77 MHz. And I don't expect to get a performance impact just because I switch from English to Vietnamese - both SBCS.

So that's progress for me - having locales implemented for Vietnamese. Currently PDPCLIB has no extra locales at all. Also progress would be for the Japanese to be able to touch-type in Katakana without ever needing to switch to the English keyboard, because software is designed to not be case-sensitive, and in any "press the letter C" type prompt, there is a Katakana character that can map to C when the programmer uses toupper(). I'm still expecting the software to be written in English - programmers aren't typically going to go to the effort to provide translations. But muscle-memory will allow them to operate the software at full speed using only Katakana. The Katakana will all be accepted by isprint() too. Touch-typing Katakana is likely to be done using the "moon array" keyboard from 2chan or whatever.

This is my rough opinion - I don't necessarily have all the underlying assumptions teased out, nor necessarily know how to vocalize it properly.

kerravon wrote: ↑Mon Jul 28, 2025 4:31 pmI didn't so much "ask" for cultural change, as say that that's what I consider to be the correct technical solution, and that PDOS won't cater for DBCS for that reason.

A cultural change is not a technical solution. DBCS are all garbage anyway, use UTF-8 instead.

kerravon wrote: ↑Mon Jul 28, 2025 4:31 pmModern computers require large amounts of memory and processing power to do what they do these days.

Sure, but how much of that goes towards handling text?

kerravon wrote: ↑Mon Jul 28, 2025 4:31 pmYou've heard of "collapse os", right?

No, but I've just looked it up, and the concept is pretty ridiculous. Should society somehow collapse to the point that modern computers are unusable, there won't be enough infrastructure left for an 8-bit computer to be useful in any way.

kerravon wrote: ↑Mon Jul 28, 2025 4:31 pmI expect things to run at 4.77 MHz. And I don't expect to get a performance impact just because I switch from English to Vietnamese - both SBCS.

Have you measured the performance impact of UTF-8? Is it really that much slower than a single-byte character set?

kerravon wrote: ↑Mon Jul 28, 2025 4:31 pmAlso progress would be for the Japanese to be able to touch-type in Katakana without ever needing to switch to the English keyboard, because software is designed to not be case-sensitive, and in any "press the letter C" type prompt, there is a Katakana character that can map to C when the programmer uses toupper().

That breaks as soon as the programmer tries to use toupper() to make a string uppercase and replaces perfectly-readable katakana with gibberish.

kerravon wrote: ↑Mon Jul 28, 2025 4:31 pmTouch-typing Katakana is likely to be done using the "moon array" keyboard from 2chan or whatever.

Standard Japanese keyboards are already labeled for direct hiragana/katakana input.

Octocontrabass wrote: ↑Mon Jul 28, 2025 6:24 pm

kerravon wrote: ↑Mon Jul 28, 2025 4:31 pmI didn't so much "ask" for cultural change, as say that that's what I consider to be the correct technical solution, and that PDOS won't cater for DBCS for that reason.

A cultural change is not a technical solution.

It is to me.

EDIT: Perhaps "technical recommendation" is a better term.

DBCS are all garbage anyway, use UTF-8 instead.

UTF-8 is to support the 3 DBCS and multiple SBCS.

kerravon wrote: ↑Mon Jul 28, 2025 4:31 pmModern computers require large amounts of memory and processing power to do what they do these days.

Sure, but how much of that goes towards handling text?

I don't know. I'm simply focused on low capability machines. It's been a massive decades-long struggle just to support English/ASCII.

kerravon wrote: ↑Mon Jul 28, 2025 4:31 pmYou've heard of "collapse os", right?

No, but I've just looked it up, and the concept is pretty ridiculous. Should society somehow collapse to the point that modern computers are unusable, there won't be enough infrastructure left for an 8-bit computer to be useful in any way.

I don't agree that computers are useless even in a remote village with no infrastructure whatsoever. And indeed, I have experimented with "living on the Sun" for all my computing needs so that I can disappear into a jungle (I'd still need a clearing for the Sun though). In recent days I have been discussing how Gaza would look if some software engineers had made the effort to invest in portable solar, now that the grid is gone for an extended period.

kerravon wrote: ↑Mon Jul 28, 2025 4:31 pmI expect things to run at 4.77 MHz. And I don't expect to get a performance impact just because I switch from English to Vietnamese - both SBCS.

Have you measured the performance impact of UTF-8? Is it really that much slower than a single-byte character set?

No - I haven't measured it. I can just see that it's creating an overhead that I don't believe should exist. People used English perfectly fine for decades. Vietnamese should be perfectly fine too. You shouldn't need to change your code just because you're switching languages because you live in Vietnam. I expect isprint() to work.

kerravon wrote: ↑Mon Jul 28, 2025 4:31 pmAlso progress would be for the Japanese to be able to touch-type in Katakana without ever needing to switch to the English keyboard, because software is designed to not be case-sensitive, and in any "press the letter C" type prompt, there is a Katakana character that can map to C when the programmer uses toupper().

That breaks as soon as the programmer tries to use toupper() to make a string uppercase and replaces perfectly-readable katakana with gibberish.

Good point - why is a string being uppercased? It may in fact be appropriate to translate katakana into uppercase English if it is an MSDOS filename. And you will need to press a katakana key that is translated into a valid English character that can be used in a filename.

kerravon wrote: ↑Mon Jul 28, 2025 4:31 pmTouch-typing Katakana is likely to be done using the "moon array" keyboard from 2chan or whatever.

Standard Japanese keyboards are already labeled for direct hiragana/katakana input.

The normal method appears to be using a separate shift key. You can't properly touch-type English either if you want all uppercase and you want to use the shift key instead of putting caps lock on. Someone created the "moon array" for a reason. I assume it is because they are trying to touch-type katakana and they recognize the existing method is inappropriate for that, and more suitable for people who are typing very slowly and converting everything to kanji. I'm not interested in kanji. I'm interested in keeping things in katakana. And typing katakana at the same speed as an English person can type.

kerravon wrote: ↑Mon Jul 28, 2025 7:22 pmNo - I haven't measured it. I can just see that it's creating an overhead that I don't believe should exist.

Switching between code pages creates an overhead that I don't believe should exist.

kerravon wrote: ↑Mon Jul 28, 2025 7:22 pmPeople used English perfectly fine for decades.

What on earth are you talking about?

kerravon wrote: ↑Mon Jul 28, 2025 7:22 pmI expect isprint() to work.

I expect a function that was defined before the existence of Unicode might need to be modified to support Unicode correctly.

kerravon wrote: ↑Mon Jul 28, 2025 7:22 pmIt may in fact be appropriate to translate katakana into uppercase English if it is an MSDOS filename.

This would be very surprising behavior, since MS-DOS is perfectly capable of storing katakana file names without translating them to gibberish.

kerravon wrote: ↑Mon Jul 28, 2025 7:22 pmThe normal method appears to be using a separate shift key.

Normally it's a lock key. The exact behavior might depend on software, though.

kerravon wrote: ↑Mon Jul 28, 2025 7:22 pmSomeone created the "moon array" for a reason. I assume

So they didn't explain why they created it? How do you know it was intended as a serious alternative to existing keyboards? Search engines aren't giving me any useful results.

Everyone knows Niklaus Wirth is a respected computer scientist, right?
Found in an old `fortunes` file. Another version without the middle verse is attributed to Guy L. Steele, Jr, 1978.

It strikes me that if you are planning a post-apocalypse operating system it is a big mistake to concentrate on obsolete processors such as the Z80 or even 68000. How many of those are you going to find in landfill or anywhere else. You might as well target the PDP8. Base your os on ARM SOCs, which need little extra support hardware, use little power, are infinitely more capable, and are to be found absolutely everywhere.

This may or may not help depending on your interview nerves and the quality of the interviewer, but technically, a job interview goes both ways: You can ask the interviewer questions to find out what's allowed and whether you'd be comfortable with it.

iansjack wrote: ↑Tue Jul 29, 2025 1:46 am It strikes me that if you are planning a post-apocalypse operating system

I am doing multiple different things.

it is a big mistake to concentrate on obsolete processors such as the Z80 or even 68000.

It is collapse os that is using Z80, not me. I can't fit PDOS into 64k.

And the 68000 is not for post-collapse - well - maybe it could be - but it is a conclusion of a competition that was lost on the day - the Amiga didn't take the world by storm. I expected it to replace the IBM PC. What was missing? What needed to change?

How many of those are you going to find in landfill or anywhere else. You might as well target the PDP8. Base your os on ARM SOCs, which need little extra support hardware, use little power, are infinitely more capable, and are to be found absolutely everywhere.

There is a reason that the 8086 was chosen by IBM. They weren't insane. I'm expecting PDOS to kick in with the PC AT where I have 2 MiB of memory - and possibly switching to the Amiga.

You can posit that there is a god/time machine that will take me back to the 1980s, but this time I will either be able to take my software with me, or write it much quicker this time, so that I can go head to head with MSDOS. Be it PDOS/286 on the PC AT or Win68k on the Amiga. And this also assumes I have some way of influencing programming culture as required. Maybe I'll sleep with Wirth or something.

Octocontrabass wrote: ↑Mon Jul 28, 2025 8:08 pm

kerravon wrote: ↑Mon Jul 28, 2025 7:22 pmNo - I haven't measured it. I can just see that it's creating an overhead that I don't believe should exist.

Switching between code pages creates an overhead that I don't believe should exist.

Where are code pages being switched?

kerravon wrote: ↑Mon Jul 28, 2025 7:22 pmPeople used English perfectly fine for decades.

What on earth are you talking about?

English-speaking people have been programming on computers ever since they were created and didn't have a need for UTF-8 or anything else like that. If computers had been invented in Vietnam, I expect that the Vietnamese would have had a very simple system that didn't need UTF-8 either. That's what I want to see. With some (limited) willingness to compromise for English-speakers.

kerravon wrote: ↑Mon Jul 28, 2025 7:22 pmI expect isprint() to work.

I expect a function that was defined before the existence of Unicode might need to be modified to support Unicode correctly.

Which is what I don't want to do.

kerravon wrote: ↑Mon Jul 28, 2025 7:22 pmIt may in fact be appropriate to translate katakana into uppercase English if it is an MSDOS filename.

This would be very surprising behavior, since MS-DOS is perfectly capable of storing katakana file names without translating them to gibberish.

Ok, so why is a string being uppercased then? If my surname was McDonald, I don't think I'd appreciate that.

kerravon wrote: ↑Mon Jul 28, 2025 7:22 pmThe normal method appears to be using a separate shift key.

Normally it's a lock key. The exact behavior might depend on software, though.

Well - lock and unlock all the time depending on which katakana character you want?

kerravon wrote: ↑Mon Jul 28, 2025 7:22 pmSomeone created the "moon array" for a reason. I assume

So they didn't explain why they created it? How do you know it was intended as a serious alternative to existing keyboards? Search engines aren't giving me any useful results.

Sorry, I posted the link ...

https://ja.wikipedia.org/wiki/%E3%81%8B ... D%E5%88%97

(translate and search for "chan" - for 2channel)

here ...

https://forum.vcfed.org/index.php?threa ... 0.1253897/

and didn't realize I hadn't posted the link on this forum.