Create your own programming in 10 minutes

Ever wanted to create your own esolang? With the rajiniPP project you can create one in less than 10 minutes

Tokens

In my previous post, I introduced to you, rajini++, an esoteric programming language with syntax made of Rajinikanth dialogues. In case if you haven’t read it here, you can find info about rajini++ here:

rajini++: The Superstar Programming Language

One feature / design choice that wasn’t discussed in the previous post was that the project was designed with flexibility in mind. That allows one to create their own esoteric programming language with all the features supported by rajini++ fairly easily and quickly! All you needs to do is edit one file, edit a couple of filenames, and you are good to go!

How does an interpreter / compiler work?

First things first, we first need to know what the building blocks of an interpreter / compiler are. Broadly speaking, a compiler has 3 major components:

1. The Lexer: The lexer’s role is to take the program as input and divide it into Tokens.
2. The Parser: The parser’s role is to do a syntax check of the program. It takes the list of tokens as input and create an AST as output.
3. The Code Generator: The code generator’s role is to transform the AST created from the parser into machine language.

rajini++ uses the rajinipp interpreter and has just the lexer and parser, it does not convert rajini++ code into machine language, itrather transpiles and executes rajini++ code in python.

More information about compiler construction can be found in this medium article:

Writing your own programming language and compiler with Python

Enough talk. Time to build your own programming language.

Now, we will see how you can create “your own” programming language in less than 10 minutes by just making some changes to the rajini++ code. In this example I will be creating a language called arnoldLang based on dialogues by Arnold Schwarzenegger.

Step 0: Prerequisites

• In this tutorial, we will modify some parts of the code written for rajini++. You need a text editor to do this.
• rajinipp and this tutorial are based on python. Some knowledge of python is required.

Step 1: Clone/Download the rajiniPP github repository

First things first we need to get a copy of the rajiniPP code. This can be done by either cloning the repository or downloading the code.

Note: For this tutorial, you need python 3.8.x installed on your machine.

Clone the repository

If you have git installed on your computer then clone the repository using the following command:

git clone https://github.com/aadhithya/rajiniPP.git

Download the repository

If you don’t have git or don’t know how to use git, you can download the code at this page: rajiniPP repository

At the page click on code and then on Download ZIP to download the code.

Fig 1: Download rajiniPP code.

Now you can unzip the code to a folder using your favourite archiver.

Note:

• We use poetry to manage environments, requirements and builds. You need to install poetry and set up the project:
  pip install poetry
  in the project directory:
  poetry install
  get environment path: poetry env info . copy the path to the virtual environment.
  activate environment: poetry env use path/to/env
  Or you can use your own virtual environment.
• Install project dependencies: poetry install

Step 2: Editing the token.yml file

Now, this is the most important part of creating the new arnoldLang programming language. rajiniPP is set up in such a way that the key words for the language are stored and loaded from the token.yml file. Therefore, changing the keywords in the token.yml file will de facto lead to a “new” esoteric language with a different set of keywords. Keep in mind that the syntax of the program still remains the same as rajini++ and all we are doing here is changing the keywords. Changing the syntax means changing the production rules and that is beyond the scope of this tutorial.

The token.yml file is present in rajiniPP/token.yml

Fig 2: The token.yml file is located in rajiniPP/token.yml

This is how the token.yml file for rajini++ looks:

Here, we are just interested in the PGM_START , PGM_END , PRINT , rajini++ and flow control commands. Now, in order to create your own language, all you need to do is change the values of these tokens to the ones you want. Your creativity is the limit here!

For my arnoldLang, inspired by ArnoldC, I have come up with the following keywords for the tokens:

And that is the most important part done! All we need to do now is to write programs in arnoldLang and run it!

The math ops, logical ops and other fields make use of regular expressions and it is too easy to mess up. So, unless you are experienced with regex, I suggest you leave them untouched!

Step 3: Change references to rajinipp and rajini++

The next step is to change all the references to rajini++ in the project to arnoldLang. The following changes need to be made:

1. rajinipp/__init__.py : In __version_str__ rajini++ with your language’s name. In our example’s case, it is arnoldLang.
2. rajinipp/__main__.py : change all occurrences of rajini++ and rajinipp with your language’s name. Here we replace it with arnoldLang.
3. rajinipp : change the name of the rajinipp folder to the name of your language(<yourLang>) . Make sure that the name contains no spaces. In this case we rename the rajinipp folder to arnoldLang.

And the rajinipp folder should have been renamed to arnoldLang like so:

Congratulations we are 99% done with creating our new arnoldLang. We can already start writing arnoldLang programs and testing the interpreter. Here’s the hello world program:

and will output the following:

arnoldLang hello world program

Assuming that you have renamed the folder properly, you should be able to run arnoldLang programs from the project root directory using the following command:

python -m arnoldLang run path/to/program.arnold

Step 4: Install your language as a package

At this point we have created arnoldLang and it can be invoked using the python -m command. If you are satisfied with this, you can call it a success and stop here, but if you want to make this as an installable package just like rajini++ and also potentially publish the package to the pypi registry so that others can try your language out, you are just a few steps away, continue reading!

In order to package the project and make it installable, we need to make the following changes to the pyproject.toml file:

1. pyproject.toml : change the name field to your language’s name and set a discription of your choice.
2. pyproject.toml : set the line underneath the [tool.poetry.scripts] section to the following: <yourLanguage> = <yourLanguage>.__main__:app
For our example this will be: arnoldLang = arnoldLang.__main__:app . Make sure that <yourLang> matches the name of the folder you set in step 3 of this section. What is happening is we are asking poetry to set the app function in the __main__.py file as the entrypoint when one calls yourLang in the terminal/command prompt.
3. under the [tool.semantic_release] change replace rajinipp in rajinipp.__init__:__version__ with <yourLang> . For our example this becomes: arnoldLang.__init__:__version__ . This is required by the semantic release tool to find set the version number automatically.

In the end for the arnoldLang, the pyproject.toml file should look like this:

now to create the installable package, all you need to do is run the following command:

poetry build

If everything worked fine, you’ll see such an output:

The package(whl file) can be found in the dist folder. You can now easily install the package using the following command:

pip install path/to/<whl-file>

in the case of arnoldLang it is pip install dist/arnoldLang-0.3.1-py3-none-any.whl

Test installation with arnoldLand version .

If you see the version information printed, congratulations, you have successfully installed your own language! You can now run programs by simply typing arnoldLang run path/to/pgm.arnold . You can also start the interactive shell by running the arnoldLang shell command! Enjoy!

Here’s an example program from rajini++ and its equivalent in our newely created arnoldLang:

Left: If-else conditional in rajini++, Right: if-else contitional in arnoldLang

as expected, the output of running the arnoldLang or rajini++ programs is the same.

Step 5: Congratulations!

Congratulations, you have now created your own programming language based on the flexible foundation provided by the rajini++ project! This tutorial has hopefully made you more interested about how programming languages are written. As a next step, dive into the code of rajini++ and try to understand how the BNF productions relate to the syntax of the program and how you can change the production rules to modify the syntax of the language. If this seems too complicated for you, have fun creating programming languages for your favourite actors, movies or tv shows! Enjoy!

Useful Resources

rajini++: The Superstar Programming Language

GitHub - aadhithya/rajiniPP: Programming language based on Iconic dialogues of Rajinikanth. 🤘🏽

Writing your own programming language and compiler with Python