OSDev.org

The Place to Start for Operating System Developers
It is currently Sat Jan 23, 2021 12:28 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 11 posts ] 
Author Message
 Post subject: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Sat Jan 09, 2021 8:21 pm 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 420
Alternative title: "klange finally writes that Python clone they've been talking about doing for years"

Image

I am pleased to announce that my winter project, Kuroko, has reached a stage where I'm comfortable enough to make an official announcement.

Kuroko is a light-weight, embeddable, dynamically-typed, bytecode-compiled, garbage-collected Python-like scripting language. It features native list and dictionary (associative array) collection types, first-class functions, classes, exception handling, optional function arguments, keyword arguments, native iteration ("for-each") loops, function/method decorators, loadable modules (including native code modules), and more. Kuroko is easily embeddable, with minimal C standard library requirements and no external dependencies. The interpreter can be easily extended with C modules to provide bindings for native libraries or integration into a host application. The full-featured interpreter core can be built down to a 166KB static binary (with musl), or a 140KB shared object (with glibc). Kuroko's syntax is highly compatible with Python, so it is easy to get started and port existing code. Kuroko also features a syntax-highlighted repl with tab completion, so you can get started right away!

Source repository: https://github.com/klange/kuroko
Code examples: https://github.com/klange/kuroko#code-examples

And of course it works in ToaruOS:

Image

_________________
toaruos on github | toaruos.org | gitlab | twitter | bim - a text editor


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Sun Jan 10, 2021 1:46 am 
Offline
Member
Member
User avatar

Joined: Tue Sep 15, 2020 8:07 am
Posts: 218
Location: London, UK
Looks good! I’ve never been able to get on well with dynamically typed languages though :)

_________________
Building a single address space Microkernel, as used in embedded applications, for the desktop... Download latest build bootable Disk Image: https://github.com/h5n1xp/CuriOS/blob/main/disk.img.zip


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Sun Jan 10, 2021 10:37 am 
Offline
Member
Member
User avatar

Joined: Mon May 22, 2017 5:56 am
Posts: 476
Nice! I had a lot of fun with Python itself 20 years ago, more than any other language at the time. I even tried to make an all-Python userspace for Linux, but in those days I would get discouraged when I couldn't implement exactly what I thought I wanted. I didn't have the skills to implement mount and other system utilities in Python, and felt I as spoiling the purity by bringing them in as binaries. (Concern for imagined standards of purity has always been my biggest problem with programming.) Anyway, I liked Python very much when it was simpler. If I ever get to thinking I do want complex syntaxes (not Forth, Lisp, APL...), I'll be sure to give Kuroko a try.

Before I saw the reference to the anime, I websearched and found kuroko are stagehands in traditional Japanese theatre. I think that's an excellent metaphor for a scripting language. ;)

Looking at the examples..

That's an interesting way of creating a static local variable.

I think you've got a typo under Lambda functions. The example defines myLambda but references myLambdaa.

Woot! Implicit `self`! I got so tred of typing that, back in the day. XD

Typo in the `set` example: the last print is "pritn".

The C-style loop has `i = i + 1`. Nothing wrong with that, but `i++` is the more instantly recognizable form.

Oh oh oh! `if` can be part of an expression and return a value! I always liked that much better than `?:` :D

What's the purpose of `with`? Just temporary variables?

Evaluating default arguments at call time is good. I remember being very confused by the way Python did it. :)

_________________
Kaph link pending. code pending. design pending. plans in a state of flux! everything pending! choice of language still up in the air! why is nothing coming together?!?


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Sun Jan 10, 2021 7:27 pm 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 420
Hey eekee, thanks for the feedback on the README - cleaned up the typos :)

Quote:
I websearched and found kuroko are stagehands in traditional Japanese theatre. I think that's an excellent metaphor for a scripting language.
Kuroko were dressed in full-body black clothing and would be tasked with manipulating set pieces on stage as a kabuki play progressed - their black outfits were supposed to hide them. Historians believe our modern image of ninjas actually comes from appearances in plays where they were dressed like the stagehands to surprise the audience!

Quote:
The C-style loop has `i = i + 1`. Nothing wrong with that, but `i++` is the more instantly recognizable form.
I think I wrote that section a few days before I added the increment/decrement shortcuts :)

Quote:
Oh oh oh! `if` can be part of an expression and return a value! I always liked that much better than `?:` :D
Getting the semantics for Python's ternary expressions working required a lot of thought, since the condition comes after the first result expression and my compiler has no AST stage; glad I got there in the end!

Quote:
What's the purpose of `with`? Just temporary variables?
The intent of `with` blocks is to ensure that cleanup tasks happen regardless of early returns (and also exceptions, but still working on that one). In some languages, this is done by allowing objects to automatically call functions when they leave scope or are de-allocated, but with a dynamic type system the overhead of checking for these methods every time a variable goes out of scope is considerable, so instead the "with" block explicitly marks such objects and the scope for which their entry and exit should be tracked. The most common usage of this in Python is opening files using a "with" statement - they'll close themselves automatically on exit. Python gets more use out of `with`, though, as its approach to scoping means you can't just throw a random block construct at it to reduce the lifetime of a variable.

_________________
toaruos on github | toaruos.org | gitlab | twitter | bim - a text editor


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Mon Jan 11, 2021 6:31 am 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 1174
Looks pretty good!

If I understand correctly, it is interpreted and not JIT compiled. Do you have some performance measurements? (FYI for security reasons I prefer interpreters even though they are slower)
Is the bytecode yours? Or did you use an existing one? (Unlike Java, WASM etc. Python bytecode is not standardized, that's why I'm asking)

Could you please compare your solution with this implementation? This is the one I was playing with so far (also not bytecode-compatible):
https://github.com/micropython/micropython/
This is a great embeddable Python interpreter, but has some shortcomings and very difficult to maintain, so yours looks very promising! (uPython also available on microcontrollers, but I'm not interested in those, only in x86, the version in "ports/unix" to be precise) It doesn't matter if your language is not 100% Python, just Python-like, that's perfectly okay.

Well done,
bzt


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Mon Jan 11, 2021 7:22 am 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 420
Quote:
If I understand correctly, it is interpreted and not JIT compiled. Do you have some performance measurements? (FYI for security reasons I prefer interpreters even though they are slower)
Yeah, execution is entirely running the bytecode through the VM and calling appropriate C functions to churn out results. I have a number of code snippets I've ported from Python that I run regularly and the results are promising - CPython's highly-tuned object implementations beat me out a bit, but the difference isn't all that big. I'll see if I can get some proper numbers and do a complete comparison...
Quote:
Is the bytecode yours? Or did you use an existing one? (Unlike Java, WASM etc. Python bytecode is not standardized, that's why I'm asking)
The bytecode started from the Lox bytecode in Crafting Interpreters, though I rather quickly introduced features for larger code sizes; since then, it's organically grown towards something very similar to CPython's - often the disassembly of the output from my compiler looks nearly identical to the same code run through CPython, though it does have a better optimizer.

_________________
toaruos on github | toaruos.org | gitlab | twitter | bim - a text editor


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Mon Jan 11, 2021 7:48 am 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 1174
klange wrote:
Quote:
If I understand correctly, it is interpreted and not JIT compiled. Do you have some performance measurements? (FYI for security reasons I prefer interpreters even though they are slower)
Yeah, execution is entirely running the bytecode through the VM and calling appropriate C functions to churn out results. I have a number of code snippets I've ported from Python that I run regularly and the results are promising - CPython's highly-tuned object implementations beat me out a bit, but the difference isn't all that big. I'll see if I can get some proper numbers and do a complete comparison...
Quote:
Is the bytecode yours? Or did you use an existing one? (Unlike Java, WASM etc. Python bytecode is not standardized, that's why I'm asking)
The bytecode started from the Lox bytecode in Crafting Interpreters, though I rather quickly introduced features for larger code sizes; since then, it's organically grown towards something very similar to CPython's - often the disassembly of the output from my compiler looks nearly identical to the same code run through CPython, though it does have a better optimizer.
Thanks! Performance is not everything, sometimes having a dependency-free and easy to maintain, easy to port source code worth more.

Cheers,
bzt


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Mon Jan 11, 2021 11:51 am 
Offline
Member
Member

Joined: Thu May 17, 2007 1:27 pm
Posts: 803
Looks nice. Where do you see the biggest differences compared to Python? Where does Kuroko improve upon Python?

Personally, I like Python a lot and use it regularly (our "meta-meta" build system xbstrap is written in Python, as well as our IDL compiler and some other tools) but IMHO some parts of it could really be done better. For example, attribute access is a mess and goes through 3+ layers of double underscore functions.

_________________
managarm: Microkernel-based OS capable of running a Wayland desktop (Discord: https://discord.gg/7WB6Ur3). My OS-dev projects: [mlibc: Portable C library for managarm, qword, Linux, Sigma, ...] [LAI: AML interpreter] [xbstrap: Build system for OS distributions].


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Mon Jan 11, 2021 2:48 pm 
Offline
Member
Member
User avatar

Joined: Tue Feb 18, 2020 3:29 pm
Posts: 546
bzt wrote:
klange wrote:
Quote:
If I understand correctly, it is interpreted and not JIT compiled. Do you have some performance measurements? (FYI for security reasons I prefer interpreters even though they are slower)
Yeah, execution is entirely running the bytecode through the VM and calling appropriate C functions to churn out results. I have a number of code snippets I've ported from Python that I run regularly and the results are promising - CPython's highly-tuned object implementations beat me out a bit, but the difference isn't all that big. I'll see if I can get some proper numbers and do a complete comparison...
Quote:
Is the bytecode yours? Or did you use an existing one? (Unlike Java, WASM etc. Python bytecode is not standardized, that's why I'm asking)
The bytecode started from the Lox bytecode in Crafting Interpreters, though I rather quickly introduced features for larger code sizes; since then, it's organically grown towards something very similar to CPython's - often the disassembly of the output from my compiler looks nearly identical to the same code run through CPython, though it does have a better optimizer.
Thanks! Performance is not everything, sometimes having a dependency-free and easy to maintain, easy to port source code worth more.

Cheers,
bzt

No, not everything, just 80% of it :wink: . To be fair, I would have to say that speed and security are number one, and stability is number two, as crashing for me is less annoying then bad performance. Looks nice, though @klange!

_________________
Currently working on the Nexware project, an attempt to make a less bloated version of GNU. All repos for it can be found at https://github.com/Nexware-Project.


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Sat Jan 16, 2021 1:41 am 
Offline
Member
Member

Joined: Mon Dec 07, 2020 8:09 am
Posts: 39
:lol: kuroko as in shirai?


Top
 Profile  
 
 Post subject: Re: Kuroko: A light-weight, embeddable bytecode VM language
PostPosted: Sat Jan 16, 2021 3:15 am 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 420
xeyes wrote:
:lol: kuroko as in shirai?
Yes, or did the logo, association with ToaruOS, and link to the Fandom article for the character in the README not make it clear ;)

Anyway...

I have some exciting news: I built an in-browser REPL with a WASM-compiled version of Kuroko, so you can try it out right now:

https://klange.dev/kuroko/

Try some of the examples from the README - and let me know if any of them are wrong, it was written as development progressed to track feature coverage and some things may have changed since their original README section was written.

_________________
toaruos on github | toaruos.org | gitlab | twitter | bim - a text editor


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 11 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group