Oberon-2 -> C compiler
======================
(release 0.17.2)
220 LEX source files (scanner for lexical analysis) src/*.l
4,192 YACC source files (parser for syntactic and semantic analysis
and part of code generation) src/*.y
809 C header files (misc. definitions and interfaces) src/*.h
9,197 C implementation code files (main driver,
command line handling, symbol table, other auxiliary data types
and runtime system/library implementation) src/*.c
707 build, install, and test scripts (sh)
------
15,125 < 16K LOC
======
(+6,299 lines of Oberon-2 test code)
Such a very compact code base! Oberon-2 may never have reached
the distribution that Pascal had (which was an issue of timing as well due to influential distributions like P-System Pascal and Borland Turbo-Pascal/Delphi), but it's a great exercise in minimalism.
Not to brag -- hum, ok, YES, to brag --, but Dusk's Oberon compiler[1][2] is less than 1500 lines of Forth code and it compiles directly to native code (no transpiling to C). Yes, Oberon is simple.
Thanks for sharing that. I've actually been learning some Forth recently and had been wondering how a parser or a compiler might actually look using Forth, and you actually already made one, which is pretty cool. I was wondering this because deterministic parsers are often already stack based (LL and LR parsers can both be implemented as imperative programs that manipulate a stack based on the input tokens, or just recursive descent).
I still can't really understand everything just from reading the code, but I had assumed it would be some kind of explicitly stack-based state machine precedence parser, like this article describes: https://h14s.p5r.org/2014/10/shiftreduce-expression-parsing-...
Kind of looks like you did that in expr.fs? Not completely sure, though.
I'm not of the academic type and this type of vocabulary quickly loses me. Lots of words to describe something that, at its core, is pretty simple. Mind twisting, but simple.
Infix notation parsing happens in ast.fs[1]. It parses tokens left to right and builds a tree (the structure is Lisp-like cons) describe the expression in such a way that expr.fs[2] then just has to recursively process the top tree node to have proper codegen.
In other words, operator precedence, prefix/postfix operators, etc. are all considered in ast.fs. Once in expr.fs is reached, all those things have been processed.
AST tests[3] give a good example of what ast.fs does.
I'm not sure I'd say that Forth being stack-based particularly help there. This logic is more "recursive-based" than anything. Lots of recursion all around.
Thanks for explaining and giving me an overview of your code. I guess I was overly focused on Forth being stack based, when it's also perfectly capable of processing trees recursively.
Yeah, I just think parsers are really cool, and have read enough about different parsing algorithms that I'm used to vocab like that.
Wow! I have been in a potentially unhealthy love relationship with Oberon (especially the -07 version, once it appeared) since the mid 1990s. It has no basis in reality, my obsession, that I am aware of. But, so, on the one hand, I "get it" about Oberon. And, also, Forth - very long time Forth fan here! But, but, why, for DUSK, are you doing this? I hope you have the coolest frickin' reasons ever! :-)
To port Oberon itself to Dusk. The effort is already well underway[1]. I'm of the opinion that the architecture of Oberon, w.r.t. to its graphical user interface, is sound. Dusk so far only has a console-based interface, Oberon is going to give it a GUI.
To borrow your words, yes it's going to be frickin' cool.
The Oberon system version corresponding to Oberon-07 (see http://www.projectoberon.net/) is not really a "GUI" and very bare-bone compared to previous Oberon system versions. Wirth mainly wanted to implement his own computer including CPU using an FPGA; to reduce his effort he left out many things from the past. You could have a look at https://github.com/rochus-keller/OberonSystem3 instead which indeed has a GUI with an Oberon based widget framework; the referenced implementation should be easy to port to your OS (see ObxPal.obx).
My taste for minimalism are a bit on the extreme side. Dusk OS isn't really about having a GUI, but graphical applications such as a PDF viewer are part of its design goals.
I was planning on not having a Desktop Environment at all and just have graphical applications use the whole screen, but I was intrigued by Viewers and Texts architecture. Their fundamental simplicity is rather elegant and I feel that mixing this with Forth's own radically simple elements, there's something quite powerful to be had. I'm not sure yet how it will materialize, but I still have that feeling.
I'm not really looking to write new graphical applications for Dusk, so I'm not planning on ever needing a Oberon-based gadget framework. The idea behind Dusk is to port existing applications onto a radically simpler platform (compared to your typical POSIX) with a minimal effort, not to write new software.
I see. In case you're interested, I also made a version of the project oberon system with a minimal, portable plattform abstraction layer, see https://github.com/rochus-keller/OberonSystem/tree/FFI. Most of the code should be compatible with your compiler.
Early 2000s, compiler authors basically had to guess how types resolved and implement what made most sense. One week did not pass without filing a new kind of internal compiler error to gcc (well, I was lazy so I kept filing to debian). The specs improved immensely in the past 20 years. So much so that you can actually implement the entire grammar from the spec and the order of type resolution is well known. You could cut corners by ignoring every "compiler hint" and optional optimization that is left for the implementation to get things going, then go back and finish those.
I'm guessing we call it Oberon-07? The one from "The Programming Language Oberon", written in 2007, revised in 2016. As the documentation states, it's not 100% on specs, for a better fit with Dusk interoperability, but it's very close. 100% specs compliance wouldn't affect implementation size by much.
Ok, I see, thanks; so it's the same version as implemented in OBNC posted above. It's the last version Wirth specified, and quite different from the versions used for the Oberon systems 2, 3 and 4 in the nineties. Personally I consider Oberon-2 the most useful version, but I still had to get rid of a lot of orthodoxies to really use it in my projects (see https://github.com/rochus-keller/Oberon).
However I do agree it is very niche, one is better of with Go, D, C#, Swift, as modern compiled managed languages with low level language features for systems programming.
It’s used at ETHZ still if you count institutional use.
Commercially it had some popularity in industrial automation and robotics many years ago and some companies still maintain Oberon codebases for this reason. I believe this is the main target market for the commercial Astrobe Oberon compiler for Cortex-M, which sells enough to stick around.
I can’t think of a good reason to start a new commercial product in it and I’m not aware of any new commercial uses, but there are still lots of academic and hobby projects cropping up.
Not to brag -- hum, ok, YES, to brag --, but Dusk's Oberon compiler[1][2] is less than 1500 lines of Forth code and it compiles directly to native code (no transpiling to C). Yes, Oberon is simple.
[1]: https://git.sr.ht/~vdupras/duskos/tree/master/item/fs/doc/co...
[2]: https://git.sr.ht/~vdupras/duskos/tree/master/item/fs/comp/o...
Thanks for sharing that. I've actually been learning some Forth recently and had been wondering how a parser or a compiler might actually look using Forth, and you actually already made one, which is pretty cool. I was wondering this because deterministic parsers are often already stack based (LL and LR parsers can both be implemented as imperative programs that manipulate a stack based on the input tokens, or just recursive descent).
I still can't really understand everything just from reading the code, but I had assumed it would be some kind of explicitly stack-based state machine precedence parser, like this article describes: https://h14s.p5r.org/2014/10/shiftreduce-expression-parsing-...
Kind of looks like you did that in expr.fs? Not completely sure, though.
I'm not of the academic type and this type of vocabulary quickly loses me. Lots of words to describe something that, at its core, is pretty simple. Mind twisting, but simple.
Infix notation parsing happens in ast.fs[1]. It parses tokens left to right and builds a tree (the structure is Lisp-like cons) describe the expression in such a way that expr.fs[2] then just has to recursively process the top tree node to have proper codegen.
In other words, operator precedence, prefix/postfix operators, etc. are all considered in ast.fs. Once in expr.fs is reached, all those things have been processed.
AST tests[3] give a good example of what ast.fs does.
I'm not sure I'd say that Forth being stack-based particularly help there. This logic is more "recursive-based" than anything. Lots of recursion all around.
[1]: https://git.sr.ht/~vdupras/duskos/tree/master/item/fs/comp/o...
[2]: https://git.sr.ht/~vdupras/duskos/tree/master/item/fs/comp/o...
[3]: https://git.sr.ht/~vdupras/duskos/tree/master/item/fs/tests/...
Thanks for explaining and giving me an overview of your code. I guess I was overly focused on Forth being stack based, when it's also perfectly capable of processing trees recursively.
Yeah, I just think parsers are really cool, and have read enough about different parsing algorithms that I'm used to vocab like that.
Wow! I have been in a potentially unhealthy love relationship with Oberon (especially the -07 version, once it appeared) since the mid 1990s. It has no basis in reality, my obsession, that I am aware of. But, so, on the one hand, I "get it" about Oberon. And, also, Forth - very long time Forth fan here! But, but, why, for DUSK, are you doing this? I hope you have the coolest frickin' reasons ever! :-)
To port Oberon itself to Dusk. The effort is already well underway[1]. I'm of the opinion that the architecture of Oberon, w.r.t. to its graphical user interface, is sound. Dusk so far only has a console-based interface, Oberon is going to give it a GUI.
To borrow your words, yes it's going to be frickin' cool.
[1]: https://git.sr.ht/~vdupras/duskos/tree/master/item/fs/doc/ob...
The Oberon system version corresponding to Oberon-07 (see http://www.projectoberon.net/) is not really a "GUI" and very bare-bone compared to previous Oberon system versions. Wirth mainly wanted to implement his own computer including CPU using an FPGA; to reduce his effort he left out many things from the past. You could have a look at https://github.com/rochus-keller/OberonSystem3 instead which indeed has a GUI with an Oberon based widget framework; the referenced implementation should be easy to port to your OS (see ObxPal.obx).
My taste for minimalism are a bit on the extreme side. Dusk OS isn't really about having a GUI, but graphical applications such as a PDF viewer are part of its design goals.
I was planning on not having a Desktop Environment at all and just have graphical applications use the whole screen, but I was intrigued by Viewers and Texts architecture. Their fundamental simplicity is rather elegant and I feel that mixing this with Forth's own radically simple elements, there's something quite powerful to be had. I'm not sure yet how it will materialize, but I still have that feeling.
I'm not really looking to write new graphical applications for Dusk, so I'm not planning on ever needing a Oberon-based gadget framework. The idea behind Dusk is to port existing applications onto a radically simpler platform (compared to your typical POSIX) with a minimal effort, not to write new software.
I see. In case you're interested, I also made a version of the project oberon system with a minimal, portable plattform abstraction layer, see https://github.com/rochus-keller/OberonSystem/tree/FFI. Most of the code should be compatible with your compiler.
Thanks for sharing.
Yes, compare that to the smallest possible, ISO-compliant burst laughing C++ compiler implementation in any language...
Complicated means "likely full of errors that are hard to identify"...
Early 2000s, compiler authors basically had to guess how types resolved and implement what made most sense. One week did not pass without filing a new kind of internal compiler error to gcc (well, I was lazy so I kept filing to debian). The specs improved immensely in the past 20 years. So much so that you can actually implement the entire grammar from the spec and the order of type resolution is well known. You could cut corners by ignoring every "compiler hint" and optional optimization that is left for the implementation to get things going, then go back and finish those.
Cool. Which version of the Oberon language does it support?
I'm guessing we call it Oberon-07? The one from "The Programming Language Oberon", written in 2007, revised in 2016. As the documentation states, it's not 100% on specs, for a better fit with Dusk interoperability, but it's very close. 100% specs compliance wouldn't affect implementation size by much.
Ok, I see, thanks; so it's the same version as implemented in OBNC posted above. It's the last version Wirth specified, and quite different from the versions used for the Oberon systems 2, 3 and 4 in the nineties. Personally I consider Oberon-2 the most useful version, but I still had to get rid of a lot of orthodoxies to really use it in my projects (see https://github.com/rochus-keller/Oberon).
Your Forth implementation is fascinating.
I imagine its minimal optimizations at that point?
Is Oberon used anywhere, if so, where? Is it picked for new projects?
Yes, Astrobe is still in business for about 20 years.
https://www.astrobe.com/default.htm
ETHZ still uses Active Oberon somehow,
https://gitlab.inf.ethz.ch/felixf/oberon
This is the owner of that repo,
https://inf.ethz.ch/de/personen/person-detail.MTMyNjc0.TGlzd...
However I do agree it is very niche, one is better of with Go, D, C#, Swift, as modern compiled managed languages with low level language features for systems programming.
It’s used at ETHZ still if you count institutional use.
Commercially it had some popularity in industrial automation and robotics many years ago and some companies still maintain Oberon codebases for this reason. I believe this is the main target market for the commercial Astrobe Oberon compiler for Cortex-M, which sells enough to stick around.
I can’t think of a good reason to start a new commercial product in it and I’m not aware of any new commercial uses, but there are still lots of academic and hobby projects cropping up.
I use it as an example for simplicity enabling expression.
What I mean is if you pick the right primitives to begin with you don’t have to turn a system into horrid spaghetti.
Complexity costs everyone - consumers, people building products and supporting them etc.
And sometimes standardization is the enemy of the simple.
So these things will never be popular as long as committees exist to make our lives miserable.