#Bytecode Minimacy Virtual Machine reads Minimacy source code and compiles it to bytecode before running it. Bytecode stays inside the memory, there is no way to import or to export it: this prevents the Minimacy VM from running incompatible bytecode. However, Minimacy makes it easy to read the bytecode produced by the compiler. ##Stack based The bytecode is “stack based”: everything happens in a stack. To sum 1 and 2: - we push 1 - we push 2 - then, to perform the addition: o we remove 2 values (here 1 and 2) o we compute the addition (here 3) o we push the result ##fun f(x) = x+1 ;; Let’s show this: create a hello.mcy file with the following content. fun f(x) = x+1 ;; fun main() = dump #f ;; Run it, the output is: >-> fun f > bytecode: >| args=1 locals=0 >| 0 rloc.b 0 >| 2 int.b 1 >| 4 add >| 5 ret The first line gives the number of arguments and the number of locals variables (here 1 and 0). The compiler gives a number to each argument and local variable. Here there is only one argument “x”, its number is 0. Then we have 4 lines of bytecode instructions. We see three columns: - the first one is the address, starting from 0 - the second one is the opcode (operation code) as a short mnemonic - the third one may contain some operand { rloc.b 0 read and push “x” (numbered as “0”) int.b 1 push “1” add perform the addition ret return from the function } ##Args and locals Let’s see another example: fun g(x, y) = let x+y -> s in s*s ;; >-> fun g > bytecode: >| args=2 locals=1 >| 0 rloc.b 0 >| 2 rloc.b 1 >| 4 add >| 5 sloc.b 2 >| 7 rloc.b 2 >| 9 rloc.b 2 >| 11 mul >| 12 ret Here we have two arguments x and y, and one local variable s. There are numbered in this order: - 0: x - 1: y - 2: s We see something new in this example: { sloc.b 2 remove the last value from the stack and store it into “s” (numbered as “2”) } ##Branch Another example: fun isLess(x, y) = if x-> fun isLess > bytecode: >| args=2 locals=0 >| 0 rloc.b 0 >| 2 rloc.b 1 >| 4 lt >| 5 else >22 >| 9 int -1 >| 18 goto >24 >| 22 int.b 1 >| 24 ret { rloc.b 0 read and push “x” (numbered as “0”) rloc.b 1 read and push “y” (numbered as “1”) lt remove 2 values x and y and pushes true if x is lower than y (“lt”), else false. else >22 remove 1 value and if this value is not true, jump to address 22 int -1 push “-1” goto >24 jump to address 24 int.b 1 push “1” ret return from the function } We observe two opcodes “int” and “int.b” which seems to do the same thing: push an integer. Actually, “int.b” can only push an integer from 0 to 255, which needs only one byte. If we look carefully at the address, we see that “int.b 1” starts at address 22, and the next instruction “ret” starts at 24: “int.b 1” uses two bytes, one for “int.b” one for “1”. Things are different for “int -1”: this instruction uses 18-9=9 bytes: 1 byte for “int” and 8 bytes for “-1” because integers are 64 bits. Remember that the suffix “.b”, already seen in “rloc.b” and “sloc.b” usually means that the argument is only one byte. ##Call fun f(x) = x+1;; fun g(x) = f(x)*2;; We already studied the bytecode for “f” above. The bytecode for “g” function is: >-> fun g > bytecode: >| args=1 locals=0 >| 0 rglob.b 0 >| 2 rloc.b 0 >| 4 exec.b 1 >| 6 int.b 2 >| 8 mul >| 9 ret > links: > 0: > hello.f: fun Int -> Int Here we have a block “links” at the end of the bytecode. This block contains a numbered list of global definitions. Here we have the function “f” (hello.f, as our sample file is hello.mcy) with number “0”. { rglob.b 0 push the “f” function (numbered as global “0”) rloc.b 0 push “x” exec.b 1 call a function with 1 argument ... } When “exec.b” is reached, we have “f” and “x” in the stack (“x” is on the top of the stack). “exec.b” operand “1” means that there is one argument before the function in the stack. Then: - “exec.b” calls “f” with 1 argument “x” - then removes “f” and “x” - and pushes the result of “f” ##Constants The next example shows two globals: fun f() = strLength("abc");; >-> fun f > bytecode: >| args=0 locals=0 >| 0 rglob.b 0 >| 2 const.b 1 >| 4 tfc.b 1 >| 6 ret > links: > 0: > bios.strLength: fun Str -> Int > 1: > Str: "abc" Here we have two globals: - 0: the bios function to return the length of a string - 1: a constant string { rglob.b 0 push the strLength function (numbered “0”) const.b 1 push the "abc" constant (numbered “1”) tfc.b 1 similar to “exec.b” but is detected as “terminal function call”. } “Terminal function call” means that this is the last calculation of this function: it is possible to preserve the callstack and to replace the current function call with the new one. #C level Once we master the bytecode, we may want to go deeper and see the low-level computation actually performed by the CPU. ##Opcodes For example, we want to know how “int.b” works. In the Minimacy VM source code, we search for "int.b" (including double quotes). We find this line in “vm_opcodes.c”: >{OPintb, 1, "int.b"}, This means that: - OPintb: OPintb is the opcode for the mnemonic "int.b" - 1: this opcode needs 1 byte of operand Now we can search for OPintb. We find it in “vm_interpreter.c”: >case OPintb: > i = pc[0] & 255; > pc++; > STACKPUSHINT_ERR(th, i, INTERPRETER_OM); > break; This is part of a big “switch” for all opcodes: { i = pc[0] & 255; read 1 byte from the Program Counter and store it in local variable “i” pc++; move the Program Counter 1 byte forward STACKPUSHINT_ERR(th, i, INTERPRETER_OM); push “i” in the stack of the current thread “th” and returns an error “INTERPRETER_OM” when this thread is out of memory break; quit the “switch” and loop for the next instruction } ##Native functions Most of Bios functions are native C functions, such as [[strLength]] as seen above. Now we want to see the code. In the Minimacy VM source code, we search for "strLength" (including double quotes). We find this line in “system_str.c”: > { NATIVE_FUN, "strLength", fun_strLength, "fun Str -> Int"}, This line declares a native function in the Bios package with: - its name “strLength” - its C function “fun_strLength” - its type : fun Str -> Int Then we just search for “fun_strLength” (without double quote). We find it in the same file: >int fun_strLength(Thread* th) >{ > LB* a=STACK_PNT(th,0); > STACK_SET_INT(th,0,a?STR_LEN(a):0); > return 0; >} { int fun_strLength(Thread* th) a native function takes one argument, the thread, and returns an integer which is 0 when everything works fine. LB* a=STACK_PNT(th,0); read the first element of the stack (index 0) expecting a pointer STACK_SET_INT(th,0,a?STR_LENGTH(a):0); replace the first element of the stack (index 0) with 0 when the pointer “a” is null, or the string length } STACK_PNT, STACK_SET_INT, STR_LENGTH are small macros we can find in “vm_thread.h”. To understand them, we need to learn how data is stored in memory. #At memory level ##Blocks and words There are two concepts in Minimacy memory: - LB: a “lambda block”, which is a portion of memory - LW: a “lambda word” is a 64 bits value (32 bits on 32 bits hardware) There are three kinds of lambda blocks: - TYPE_BIN: a block of binary data - TYPE_ARRAY: an array of lambda words - TYPE_NATIVE: a block of binary data with memory management There are three kinds of lambda words: - VAL_TYPE_INT: a 32 bits signed integer, even on 64 bits hardware, to ensure total compatibility - VAL_TYPE_FLOAT: a 32/64 bits floating number, depending on the hardware processor - VAL_TYPE_PNT: a 32/64 bits pointer to a lambda block, depending on the hardware processor, it can be NULL It is important to understand how memory management works. Minimacy relies on a Garbage Collector, this means that at any time the Minimacy VM can compute which blocks are still in use and which blocks can be safely deallocated. For this task, Minimacy uses a classical “mark” algorithm: - every block is unmarked - there is a unique “root” block, which handles everything - then we call the “blockMark” function on the “root” block. The “blockMark” function does this: - if the block is already marked, do nothing - mark the block - if the block is TYPE_ARRAY, call this “blockMark” function recursively to all its LW words which are pointers (VAL_TYPE_PNT) - if the block is TYPE_NATIVE, call its native C “mark” function, which may recursively call this “blockMark” function - if the block is TYPE_BIN, do nothing As a result, TYPE_ARRAY blocks need to store the type of their lambda words. Compared to TYPE_BIN blocks, TYPE_NATIVE blocks only have two differences: - a dedicated “mark” function which allows the native block to reference other blocks - a dedicated “forget” function which is called when the block is freed by the GC. Each Lambda block starts with a header containing: - the block type - the block size - some information for the GC - its Memory manager (a way to manage a maximum allocation limit per thread) - a debug information, which basically contains the Minimacy type of the block ##Minimacy types We expose how some basic types of Minimacy language are implemented. ###nil [[nil]] is implemented as a VAL_TYPE_PNT Lambda word with value “0”. ###Int Integers are signed 32 bits integers, even on 64 bits hardware (else we could encounter libraries that run on 64 bits and fail on 32 bits). As any value in Minimacy language, an integer may be [[nil]]. It is considered as “0” in arithmetic expressions. There is no unsigned integer in Minimacy language: - you cannot have type inference when you mix signed and unsigned integers - most of the time, signed integers can be used instead of unsigned integers Remember that computers don’t really use mathematical integers, as they are stored on a limited number of bits. Instead, they use modular arithmetic. In modular arithmetic, let’s say on 8 bits, we have 256 “numbers” which are in fact “congruence classes”. Then, -2 and 254 belong to the same “congruence class”, as their difference is a multiple of 256. So, most of the time, computers are not dealing with integers but with “congruence classes”. When does it make sense to distinguish between -2 and 254? - when displaying a number or converting it into a decimal representation - when comparing two numbers - when dividing two numbers These three operations do not apply to “congruence classes”, first the computer need to convert “congruence classes” into integers: - with signed integer, congruence classes are converted to integers from -128 to 127 - with unsigned integers, congruence classes are converted to integers from 0 to 255 Then differences show up: - decimal representation is either “-2” or “254” - -2 is less than 0, 254 is bigger than 0 - dividing -2 by 2 gives -1, dividing 254 by 2 gives 127 What could be painful on 8-bit is easier with 32-bits. The lack of unsigned integers has a visible effect only when you need to deal with integers between 2^31 and 2^32, which is not common but may happen. ###Float Floats are simply Lambda words floats: 64 bits double-precision floating-point format (or 32 bits simple-precision on 32 bits hardware) As any value in Minimacy language, a float may be [[nil]]. It is considered as “0.0” in arithmetic expressions. ###Str/Bytes Strings and Bytes are implemented the same way as TYPE_BINARY blocks. Only the compiler ensures that strings are immutable. However, as in C, they are stored in memory with a NULL terminal character. That’s why the native C function could use the “strlen” Ansi function on these strings. However this would be a bad idea as Minimacy strings may contain NULL characters inside. As any value in Minimacy language, a string may be [[nil]]. ###Bool Booleans are either [[true]] or [[false]]. They are constant constructors for the sum “Bool”. This means that at a low level they are two TYPE_ARRAY blocks somewhere in the memory. As any value in Minimacy language, a boolean may be [[nil]]. It is considered as “false” in conditions. #Threads and stack Minimacy uses the popular concept of threads. Each thread has its own stack in which its calculations happen. Each thread has one stack, and each stack belongs to one thread. In each thread (defined in vm_thread.h), two fields manage the stack: >struct Thread >{ >... > LINT sp; > LB* stack; >}; - sp: an integer value which is the stack pointer - stack: a pointer to a TYPE_ARRAY Lambda block The initial value of “sp” is -1 and means “empty stack”. Stacks are special TYPE_ARRAY Lambda blocks: only the first elements of the array are active, from 0 to “sp”. “sp” is the index of the last element of the stack. “sp” increases each time a Lambda word is pushed into the stack and decreases each time a Lambda word is pulled. When “sp” reaches the size of the Lambda block, a new larger block is allocated and used as the new stack. That’s why each “push” may fire an Out of Memory error concerning the thread. Several macros are designed to manage the stack when writing a native function for the Minimacy VM. Here is a short list: { STACK_INT(th,index) read an integer at “index” position in the stack of thread “th” STACK_PNT(th,index) read a block pointer at “index” position in the stack of thread “th” STACK_FLOAT(th,index) read a float at “index” position in the stack of thread “th” STACK_IS_NIL(th,index) is non-zero when [[nil]] is at “index” position in the stack of thread “th” STACK_SET_INT(th,i,val) write integer “val” at “index” position in the stack of thread “th” STACK_SET_PNT(th,i,val) write block pointer “val” at “index” position in the stack of thread “th” STACK_PULL_INT(th) read and pull an integer from the stack of thread “th” }