TFoC - A compiler in 256 LoC

(This article is part of the The Fabric of Computing series: in search of simplicity)

The previous TFoC post was about a truly minimal setup, capable of just very simple computation. This time, let’s go all out and look into a 2-pass compiler for a reasonably high-level language, and what it takes to create a complete environment.

The language I’ll use for this is calledBCPL. Here is an example program:

GET "LIBHDR"

LET START() BE
    $( LET A, B, C, SUM = 1, 2, 3, 0
       SUM := A + B + C
       WRITES("Sum of 1 + 2 + 3 is ")
       WRITEN(SUM)
       WRITES("*NHello, World*N")
    $)

The reason this looks somewhat similar to C is that BCPL was a predecessor (it’s from the 1960’s) and it had a major influence on the syntax and design of the C language.

BCPL was created by Martin Richards, who still maintains a good reference page. Themanual (PDF) is a superb introduction to BCPL and overview of the entire system.

What puts BCPL squarely on the map for a TFoC post, is its elegance and very small size. The whole compiler + code generator is only 2,500 lines of (BCPL) source code. There are two intermediate file formats: OCODE and INTCODE, both plain ASCII.

The main pass of the compiler code uses about 20 KB of memory, and needs another 20 KB of data space to re-compile itself. It’s slightly too large to fit on the lowest-end STM32 µCs, although a $10 STM32F103RC board from eBay would most likely suffice.

BCPL is structured, yet untyped and word-oriented. It also predates byte-addressable memory, hardware stack registers, fancy “{” and “}” curly braces, and lowercase ASCII - who cared about that in the world of TELEXes, ASR33s, and ENIGMAs, eh?

On modern hardware with an interpreter, BCPL can compile itself in about 1 second. This is quite impressive for a high-level programming language environment.

Sooo… here’s what we need to be able to use BCPL, and even rebuild / extend it:

1. an interpreter for the “INTCODE” machine to which BCPL is targetted
2. an assembler / linker to generate the machine code for the interpreter
3. machine code for the compiler passes
4. a runtime library for some common I/O tasks, and a printf-like function
5. header files to access this library
6. source code for the compiler itself
7. and some sample code to play with

As with the previous article, theinterpreter andassembler have been implemented in Python, so that takes care of items 1 and 2.

Item 3 is a real tricky chicken-and-egg bit: you need a “compiled compiler” to be able to run it, even if the compiler source code is available. Luckily, BCPL already exists - no need to solve this bootstrap problem here.

Items 4 through 7 are easy, since the rest of the system is written entirely in BCPL - we just need to keep the source code around …

The steps to compile a “hello.b” BCPL program and run it are as follows:

python interp.py pass1 <hello.b
python interp.py pass2 <OCODE
cat INTCODE runtime.i >hello.i
python assem.py hello <hello.i
python interp.py hello`

Indeed, with the above hello.b we get:

Sum of 1 + 2 + 3 is 6
Hello, World

Two utility scripts make compilation and running code even simpler:

$ ./b -o fact fact.b

BCPL 2
OPTIONS  L6000
TREE SIZE 384

PHASE 1 COMPLETE
PROGRAM LENGTH = 57
1055 words
$ ./r fact
F(1), = 1
F(2), = 2
F(3), = 6
F(4), = 24
F(5), = 120
F(6), = 720
F(7), = 5040
F(8), = -25216
F(9), = -30336
F(10), = 24320
$

So yes, it all works splendidly - even if it’s a 16-bit machine with limited integer range.

Now we can also rebuild the compiler itself:

$ ./b -o pass1 syn.b trn.b

BCPL 2
OPTIONS  L6000
TREE SIZE 3070
TREE SIZE 4385
TREE SIZE 3586
TREE SIZE 3249
TREE SIZE 4093
TREE SIZE 3313
TREE SIZE 3899
TREE SIZE 3375
TREE SIZE 3198

PHASE 1 COMPLETE
PROGRAM LENGTH = 5098

BCPL 2
OPTIONS  L6000
TREE SIZE 3806
TREE SIZE 4446
TREE SIZE 4263
TREE SIZE 3769
TREE SIZE 4057
TREE SIZE 3600
TREE SIZE 4192

PHASE 1 COMPLETE
PROGRAM LENGTH = 4257
10523 words
$ ./b -o pass2 cg.b

BCPL 2
OPTIONS  L6000
TREE SIZE 3462
TREE SIZE 2412
TREE SIZE 3682
TREE SIZE 2890
TREE SIZE 2882

PHASE 1 COMPLETE
PROGRAM LENGTH = 2563
3797 words

Note that this takes about a dozen seconds on modern hardware. But … through the miracle of the PyPy JIT compiler, it actually can be speeded up ≈ 16x, proving that BCPL indeed can self-compile in under 1 second!

This code is available onGitHub. It’s all the work of Martin Richards and his team (half a century ago!), plus some new Python code and a bit of streamlining by yours truly.

Once you install this setup, you can make changes and “recurse” to re-build a next incarnation of the system. So you now have a real programming language“classic” which you can morph into whatever you like, syntax-wise and semantically. Enjoy!

PS. The title of this post is a bit of a stretch, and a play on an earlier one about a PDP-8, as “256 LoC“ refers to the size ofinterp.py.