It will help you improve your skills and understand how to start this journey!
I discovered this project on Reddit a few months ago, basically it is a transpiler for Bash.
I always looked for a way to write a better Bash and in the past, I searched for solutions like this, but they were all abandoned and incomplete.
Sure, shellcheck is amazing, but it is helpful to learn a better way to write Bash, but that doesn’t change the fact that you have to deal with this particular language.
I was surprised by the Amber language, as it is just a single binary, it’s written in Rust and includes a library internally that adds new functions. So you can have internal methods like `change_owner` that just check if the file/folder exists and execute `chown` to set that, for example.
I gave it a try, and after creating a few issues on the repository, I started doing some pull requests and after a while, I became one of the maintainers.
So in this article, I want to share my experience from the last few months!
Why use a Bash transpiler?
That’s a good question!
Bash is a language that, based on the version and also on what you are you trying to do, can be very cryptic.
After all, it was released in 1989, and the latest release, Bash 5.2 was 2 years ago.
We have to admit it, when you are trying to do complex stuff, the code is not beautiful to read.
Also, the language itself is not so advanced/cool, you are using a lot of external commands to do everything, such as grep, sed, awk, curl etc.
So you need to know not just the language, but also how every single command works, including that on OSX some of them are different (sed is the most famous case).
At the same time, writing something in another scripting language will require dependencies to be installed, so you need a package manager, and you also that you need to update them with the risk that everything breaks.
In the Bash case, as you are using commands that you install with the distribution package manager, it’s difficult for them to break your code because you are not using their internal APIs like in a usual scripting language.
Also, if you are using other gnu commands compliant alternatives like ripgrep, fdfind and many others, you can have the faster and best experience using the same tool experience without changing a line (just changing the command).
I started my testing by writing a simple script that I execute as root on my workstations, lsp-installer.
It is quite simple, just install various lsp for different languages on my machine, as not all of them are on my distro (Debian Sid) so I need to get them some from GitHub and others from pip, npm and gem!
That folder includes also the Bash version generated, but to read it you need an alert.
The project it is in Alpha stage so it is missing basically a code optimization to simplify the Bash code generated.
Take this Amber function an example:
fun move_to_bin(download_url, binary) { if silent download(download_url, binary) { unsafe $mv "{binary}" /usr/local/bin$ make_executable("/usr/local/bin/{binary}") } else { echo "Download for {binary} at {download_url} failed" exit(1) } }
The `download` and `make_executable` functions are provided by Amber (I implemented it 😅) natively.
Let’s see their code:
pub fun download(url: Text, path: Text): Bool { if { is_command("curl") { unsafe $curl -L -o "{path}" "{url}"$ } is_command("wget") { unsafe $wget "{url}" -P "{path}"$ } is_command("aria2c") { unsafe $aria2c "{url}" -d "{path}"$ } else { return false } } return true } pub fun make_executable(path: Text): Bool { if file_exist(path) { unsafe $chmod +x "{path}"$ return true } echo "The file {path} doesn't exist!" return false }
As you can see, they simplify a lot the code, thinking of the same lines in Python, it would be a lot more of an example.
Just to download a binary, move it, and make it executable.
Also, we see in my example that right now `mv` it isn’t built-in like `echo` (there is a PR for that right now), so you can write pure Bash if you need it.
Moving on to other interesting things:
silent unsafe $cd /tmp$
Right now the `cd` built-in has a PR to implement it (I already said that is experimental as language right?) but we see other 2 interesting things.
The `silent` keyword automatically in the bash generated will add a redirection to `/dev/null` and `unsafe` keyword is a way to manage the command by the compiler.
Because you can do this:
$npm i -g "{lsp}"$ failed { echo "Error! Exit code: {status}" }
Automatically, if the command fails (without being unsafe), there is a fallback that, in my case, I used to inform the user.
Of course, there is also a way also to get the status of the previous command just by using the `status` keyword, but it is another thing (that can be used in a lot of ways).
So my Amber script is 96 lines, and the Bash generated is 245 lines (without any optimization for the compiler).
The Bash code generated is a work in progress to be shellcheck valid (I am working on this), also on compiling it, it will be automatically prettified (there is a PR for that right now).
Sure, right now the code generated is not so easy to read, but we are working on this.
We can say that a Python script for that, that downloads, executes commands, and extracts packages, will not require any dependence to install, but I don’t think that it will be fewer lines.
What I got is a script that I can port all over my machines with no problems, I just need `jq` and `curl` installed, which is very common for developers.
Yes, we are working on a Bash Dependence Checker so the script itself will check for any commands you are using and alert the user if they are missing.
The project was started by Ph0enixKM for his CS thesis but has shown that it is solid enough to be expanded.
Consider that after the promotion on Reddit, there were already 3 releases, and we are planning another one while we are talking.
Why I am contributing To Amber
Because I like it!
There is a lot to do in the documentation to implement new functions and improve the Bash code generated, for example.
At the same time, working on the internals for what I did right now, it wasn’t so much difficult.
I think that when it will be 1.0 ready, or as we are saying, the stable release, it could be a change for someone who deals a lot with Linux but doesn’t thrust to write a Python script, for example.
I am learning a lot of how Bash works and how to improve them without writing it at the end.
Also, I want to improve the documentation, and for a developer, it is important to know how to write a good one. In my opinion, with a programming language, it is easier to learn that.
There is a TreeSitter module in progress and also a LSP and we have plans to create plugins/extension/configuration to add the support to the most famous editors.
What I am learning on working on a compiler
I never worked with Rust, including C/++ and other language that compiles to a binary. I always worked to script languages, so it is interesting to learn it with something that I get fun and I can manage without understanding everything.
I am learning that tests are very important, but not just important because you need to do for any single things, otherwise everything can break. We are talking about a programming language so you need to test if the condition statement works!
At the same time, you need to document the internals, be sure that everything is the same. Sure, we are in the alpha stage, so we can make breaking changes like we are doing with the next release, but it is a lesson about how to structure a codebase.
That example is that all the tests for the functions (we call it standard library) and for the syntax were in just 2 single Rust files. So there was a string with Amber code that it was escaped and a check if the Amber script works.
This is not something that you want because if there is a test failing it isn’t very comfortable to read/modify, so now every test (the majority) is a dedicated Amber file. So it is possible to run it manually for example and not compile everything again for a change.
I am learning that defining a function name is very important otherwise you will get `str_pos` as example instead of a comfortable `str_contains` (PHP 8 implement that).
The same it is for defining a new syntax, like for a `sudo` mode for example or how the compiler should map the generated code with the original Amber one (like with sourcemap).
Creating a compiler is not easy because you are compiling to a low level language but we are generating a Bash code that is not difficult at all.
Yes, it is a transpiler but it also includes a compiler because it validates types and so on.
We need your help
We are discussing this between the GitHub organization and the Discord server, so you are free to join us.
We need developers that want to do basically everything and help us to shape a new programming language.
The project is in the Alpha stage, and it is a simple way to create what you need without using a real scripting language but an evolved version of Bash.
So we need the help of everyone, especially of people with Rust experience, to improve the internals.
I will focus on the standard library and the documentation, but the next release will have the documentation generator as an example, so the way is not short.