OSDev.org

The Place to Start for Operating System Developers
It is currently Mon Mar 18, 2024 8:32 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 24 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Sat Jan 09, 2021 8:21 pm 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 676
Alternative title: "klange finally writes that Python clone they've been talking about doing for years"

Image

I am pleased to announce that my winter project, Kuroko, has reached a stage where I'm comfortable enough to make an official announcement.

Kuroko is a light-weight, embeddable, dynamically-typed, bytecode-compiled, garbage-collected Python-like scripting language. It features native list and dictionary (associative array) collection types, first-class functions, classes, exception handling, optional function arguments, keyword arguments, native iteration ("for-each") loops, function/method decorators, loadable modules (including native code modules), and more. Kuroko is easily embeddable, with minimal C standard library requirements and no external dependencies. The interpreter can be easily extended with C modules to provide bindings for native libraries or integration into a host application. The full-featured interpreter core can be built down to a 166KB static binary (with musl), or a 140KB shared object (with glibc). Kuroko's syntax is highly compatible with Python, so it is easy to get started and port existing code. Kuroko also features a syntax-highlighted repl with tab completion, so you can get started right away!

Source repository: https://github.com/klange/kuroko
Code examples: https://github.com/klange/kuroko#code-examples

And of course it works in ToaruOS:

Image

_________________
toaruos on github | toaruos.org | gitlab | twitter | bim - a text editor


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Sun Jan 10, 2021 1:46 am 
Offline
Member
Member
User avatar

Joined: Tue Sep 15, 2020 8:07 am
Posts: 264
Location: London, UK
Looks good! I’ve never been able to get on well with dynamically typed languages though :)

_________________
CuriOS: A single address space GUI based operating system built upon a fairly pure Microkernel/Nanokernel. Download latest bootable x86 Disk Image: https://github.com/h5n1xp/CuriOS/blob/main/disk.img.zip
Discord:https://discord.gg/zn2vV2Su


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Sun Jan 10, 2021 10:37 am 
Offline
Member
Member
User avatar

Joined: Mon May 22, 2017 5:56 am
Posts: 811
Location: Hyperspace
Nice! I had a lot of fun with Python itself 20 years ago, more than any other language at the time. I even tried to make an all-Python userspace for Linux, but in those days I would get discouraged when I couldn't implement exactly what I thought I wanted. I didn't have the skills to implement mount and other system utilities in Python, and felt I as spoiling the purity by bringing them in as binaries. (Concern for imagined standards of purity has always been my biggest problem with programming.) Anyway, I liked Python very much when it was simpler. If I ever get to thinking I do want complex syntaxes (not Forth, Lisp, APL...), I'll be sure to give Kuroko a try.

Before I saw the reference to the anime, I websearched and found kuroko are stagehands in traditional Japanese theatre. I think that's an excellent metaphor for a scripting language. ;)

Looking at the examples..

That's an interesting way of creating a static local variable.

I think you've got a typo under Lambda functions. The example defines myLambda but references myLambdaa.

Woot! Implicit `self`! I got so tred of typing that, back in the day. XD

Typo in the `set` example: the last print is "pritn".

The C-style loop has `i = i + 1`. Nothing wrong with that, but `i++` is the more instantly recognizable form.

Oh oh oh! `if` can be part of an expression and return a value! I always liked that much better than `?:` :D

What's the purpose of `with`? Just temporary variables?

Evaluating default arguments at call time is good. I remember being very confused by the way Python did it. :)

_________________
Kaph — a modular OS intended to be easy and fun to administer and code for.
"May wisdom, fun, and the greater good shine forth in all your work." — Leo Brodie


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Sun Jan 10, 2021 7:27 pm 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 676
Hey eekee, thanks for the feedback on the README - cleaned up the typos :)

Quote:
I websearched and found kuroko are stagehands in traditional Japanese theatre. I think that's an excellent metaphor for a scripting language.
Kuroko were dressed in full-body black clothing and would be tasked with manipulating set pieces on stage as a kabuki play progressed - their black outfits were supposed to hide them. Historians believe our modern image of ninjas actually comes from appearances in plays where they were dressed like the stagehands to surprise the audience!

Quote:
The C-style loop has `i = i + 1`. Nothing wrong with that, but `i++` is the more instantly recognizable form.
I think I wrote that section a few days before I added the increment/decrement shortcuts :)

Quote:
Oh oh oh! `if` can be part of an expression and return a value! I always liked that much better than `?:` :D
Getting the semantics for Python's ternary expressions working required a lot of thought, since the condition comes after the first result expression and my compiler has no AST stage; glad I got there in the end!

Quote:
What's the purpose of `with`? Just temporary variables?
The intent of `with` blocks is to ensure that cleanup tasks happen regardless of early returns (and also exceptions, but still working on that one). In some languages, this is done by allowing objects to automatically call functions when they leave scope or are de-allocated, but with a dynamic type system the overhead of checking for these methods every time a variable goes out of scope is considerable, so instead the "with" block explicitly marks such objects and the scope for which their entry and exit should be tracked. The most common usage of this in Python is opening files using a "with" statement - they'll close themselves automatically on exit. Python gets more use out of `with`, though, as its approach to scoping means you can't just throw a random block construct at it to reduce the lifetime of a variable.

_________________
toaruos on github | toaruos.org | gitlab | twitter | bim - a text editor


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Mon Jan 11, 2021 6:31 am 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 1584
Looks pretty good!

If I understand correctly, it is interpreted and not JIT compiled. Do you have some performance measurements? (FYI for security reasons I prefer interpreters even though they are slower)
Is the bytecode yours? Or did you use an existing one? (Unlike Java, WASM etc. Python bytecode is not standardized, that's why I'm asking)

Could you please compare your solution with this implementation? This is the one I was playing with so far (also not bytecode-compatible):
https://github.com/micropython/micropython/
This is a great embeddable Python interpreter, but has some shortcomings and very difficult to maintain, so yours looks very promising! (uPython also available on microcontrollers, but I'm not interested in those, only in x86, the version in "ports/unix" to be precise) It doesn't matter if your language is not 100% Python, just Python-like, that's perfectly okay.

Well done,
bzt


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Mon Jan 11, 2021 7:22 am 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 676
Quote:
If I understand correctly, it is interpreted and not JIT compiled. Do you have some performance measurements? (FYI for security reasons I prefer interpreters even though they are slower)
Yeah, execution is entirely running the bytecode through the VM and calling appropriate C functions to churn out results. I have a number of code snippets I've ported from Python that I run regularly and the results are promising - CPython's highly-tuned object implementations beat me out a bit, but the difference isn't all that big. I'll see if I can get some proper numbers and do a complete comparison...
Quote:
Is the bytecode yours? Or did you use an existing one? (Unlike Java, WASM etc. Python bytecode is not standardized, that's why I'm asking)
The bytecode started from the Lox bytecode in Crafting Interpreters, though I rather quickly introduced features for larger code sizes; since then, it's organically grown towards something very similar to CPython's - often the disassembly of the output from my compiler looks nearly identical to the same code run through CPython, though it does have a better optimizer.

_________________
toaruos on github | toaruos.org | gitlab | twitter | bim - a text editor


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Mon Jan 11, 2021 7:48 am 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 1584
klange wrote:
Quote:
If I understand correctly, it is interpreted and not JIT compiled. Do you have some performance measurements? (FYI for security reasons I prefer interpreters even though they are slower)
Yeah, execution is entirely running the bytecode through the VM and calling appropriate C functions to churn out results. I have a number of code snippets I've ported from Python that I run regularly and the results are promising - CPython's highly-tuned object implementations beat me out a bit, but the difference isn't all that big. I'll see if I can get some proper numbers and do a complete comparison...
Quote:
Is the bytecode yours? Or did you use an existing one? (Unlike Java, WASM etc. Python bytecode is not standardized, that's why I'm asking)
The bytecode started from the Lox bytecode in Crafting Interpreters, though I rather quickly introduced features for larger code sizes; since then, it's organically grown towards something very similar to CPython's - often the disassembly of the output from my compiler looks nearly identical to the same code run through CPython, though it does have a better optimizer.
Thanks! Performance is not everything, sometimes having a dependency-free and easy to maintain, easy to port source code worth more.

Cheers,
bzt


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Mon Jan 11, 2021 11:51 am 
Offline
Member
Member

Joined: Thu May 17, 2007 1:27 pm
Posts: 999
Looks nice. Where do you see the biggest differences compared to Python? Where does Kuroko improve upon Python?

Personally, I like Python a lot and use it regularly (our "meta-meta" build system xbstrap is written in Python, as well as our IDL compiler and some other tools) but IMHO some parts of it could really be done better. For example, attribute access is a mess and goes through 3+ layers of double underscore functions.

_________________
managarm: Microkernel-based OS capable of running a Wayland desktop (Discord: https://discord.gg/7WB6Ur3). My OS-dev projects: [mlibc: Portable C library for managarm, qword, Linux, Sigma, ...] [LAI: AML interpreter] [xbstrap: Build system for OS distributions].


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Mon Jan 11, 2021 2:48 pm 
Offline
Member
Member

Joined: Tue Feb 18, 2020 3:29 pm
Posts: 1071
bzt wrote:
klange wrote:
Quote:
If I understand correctly, it is interpreted and not JIT compiled. Do you have some performance measurements? (FYI for security reasons I prefer interpreters even though they are slower)
Yeah, execution is entirely running the bytecode through the VM and calling appropriate C functions to churn out results. I have a number of code snippets I've ported from Python that I run regularly and the results are promising - CPython's highly-tuned object implementations beat me out a bit, but the difference isn't all that big. I'll see if I can get some proper numbers and do a complete comparison...
Quote:
Is the bytecode yours? Or did you use an existing one? (Unlike Java, WASM etc. Python bytecode is not standardized, that's why I'm asking)
The bytecode started from the Lox bytecode in Crafting Interpreters, though I rather quickly introduced features for larger code sizes; since then, it's organically grown towards something very similar to CPython's - often the disassembly of the output from my compiler looks nearly identical to the same code run through CPython, though it does have a better optimizer.
Thanks! Performance is not everything, sometimes having a dependency-free and easy to maintain, easy to port source code worth more.

Cheers,
bzt

No, not everything, just 80% of it :wink: . To be fair, I would have to say that speed and security are number one, and stability is number two, as crashing for me is less annoying then bad performance. Looks nice, though @klange!

_________________
"How did you do this?"
"It's very simple — you read the protocol and write the code." - Bill Joy
Projects: NexNix | libnex | nnpkg


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Sat Jan 16, 2021 1:41 am 
Offline
Member
Member

Joined: Mon Dec 07, 2020 8:09 am
Posts: 212
:lol: kuroko as in shirai?


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Sat Jan 16, 2021 3:15 am 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 676
xeyes wrote:
:lol: kuroko as in shirai?
Yes, or did the logo, association with ToaruOS, and link to the Fandom article for the character in the README not make it clear ;)

Anyway...

I have some exciting news: I built an in-browser REPL with a WASM-compiled version of Kuroko, so you can try it out right now:

https://klange.dev/kuroko/

Try some of the examples from the README - and let me know if any of them are wrong, it was written as development progressed to track feature coverage and some things may have changed since their original README section was written.

_________________
toaruos on github | toaruos.org | gitlab | twitter | bim - a text editor


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Mon Jan 25, 2021 6:31 pm 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 676
I've officially marked off a release candidate for Kuroko 1.0.

If you want to try it out from your browser, the WASM repl has been updated and includes a detailed interactive tutorial.

I also cut a new release of ToaruOS that includes the Kuroko 1.0 RC and a demo of using it to make GUI applications.

_________________
toaruos on github | toaruos.org | gitlab | twitter | bim - a text editor


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Mon Feb 01, 2021 8:14 am 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 676
Since the last post, Kuroko got its first official release... and a bug fix release shortly after that. :roll:

I've also set up a little page around the WASM REPL so it's a more inviting website and not just a big terminal in a browser tab: https://kuroko-lang.github.io/

_________________
toaruos on github | toaruos.org | gitlab | twitter | bim - a text editor


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Wed Feb 03, 2021 2:25 pm 
Offline
Member
Member

Joined: Sun Aug 23, 2020 4:35 pm
Posts: 148
It seems there is a bug with the online Kuroko terminal thing at https://kuroko-lang.github.io/.
This bug could not be reproduced on the full-screen terminal thing at https://klange.dev/kuroko/.

Steps to reproduce:
1. Type any statement, for example "let a = 1"
2. Press enter
3. Press any printable key except enter
4. Observe two extra newlines added prior to the character

Additionally, if you type another line of script with those two newlines prior, (for example, "let b = 2") and press enter, it will not execute that line but instead create a newline. You have to remove the two prior newlines OR press enter again before it will execute the script.

Correct me if I'm wrong, but I believe the intended result is that, if Shift+Enter is pressed, it will not execute the current line but instead add a newline; then when Enter is pressed normally, it will execute all that you have written.

(If you prefer, I can open an issue on the repo in GitHub)

_________________
My OS: TritiumOS
https://github.com/foliagecanine/tritium-os
void warranty(laptop_t laptop) { if (laptop.broken) return laptop; }
I don't get it: Why's the warranty void?


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Wed Feb 03, 2021 4:26 pm 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 676
foliagecanine wrote:
It seems there is a bug with the online Kuroko terminal thing at https://kuroko-lang.github.io/.
This bug could not be reproduced on the full-screen terminal thing at https://klange.dev/kuroko/.
Looks like this is a bug in the code editor I'm using (Ace) when disabling its line wrapping mode; I've turned wrapping back on and it seems to have fixed this.

_________________
toaruos on github | toaruos.org | gitlab | twitter | bim - a text editor


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 24 posts ]  Go to page 1, 2  Next

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group