REBOL/SDK - Preprocessor

SDK Documentation

Combine multiple source or data files into a single REBOL output file.

Contents

Overview

The preprocessor includes the full power of REBOL, allowing you to evaluate any REBOL expression during processing. The program allows merging of code, data, images, sound and other types of files into a single output file. It also supports conditional expressions, such as the conditional including of files.

The output is a reduced REBOL source file with extra spacing, comments, and include file headers removed. Processing is recursive, so included files can themselves contain commands to process. This is very powerful.

Part of Encap

The preprocessor is now part of REBOL/Encap. Source files provided as input to Encap will be preprocessed.

The preprocessor can also be run independently of Encap using any recent version of REBOL. See the instructions below.

Preprocessor Source Code

The preprocessor source code is provided as part of the SDK distribution. It includes two files:

prerebol.r - command line and interactive interfaces
prebol.r - main functions of the preprocessor

Under the conditions of the SDK license, any bug fixes or modifications must be sent to REBOL Technologies (to let other developers to take advantage of the changes).

Before adding any additional #commands to the preprocessor, please be certain that the #DO command does not already easily provide the same capability. We believe that it is unlikely that such command additions are needed, but if you think so, please contact us.

Changes in 2.0

The 2.0 version is a complete rewrite of the REBOL preprocessor and is now smaller, faster, and adds a few new features.

Running the Preprocessor Separately

The preprocessor is normally run as part of REBOL/Encap processing. No separate steps are required. However, if for testing purposes you want to run the preprocessor separately, the commands below will do so by calling a separate preprocessor interface file:

From a shell command line or an icon shortcut link:

rebol -s prerebol.r input.r output.r

From the REBOL console or a REBOL script:

do/args %prerebol.r [%input.r %output.r]

Preprocessor Commands

All preprocessor commands begin with a # character. The commands can appear anywhere within REBOL blocks (but not within strings), and are not restricted to the beginning of a line. In fact, commands can be part of expressions (see examples below).

Include Commands

The include commands are:

#include %file.r | (file-expr)

#include-string %file.txt | (file-expr)

#include-binary %file.bin | (file-expr)

#include-files %path [%file1 %file2 %file3]

The (file-expr) can be any valid REBOL expression that returns a filename. For example you can write:

#include (rejoin [%file version ".r"])

An include file may itself contain #include commands. When this occurs, the include files are processed relative to the directory location of the parent include file. This is the behavior you would expect in a preprocessor.

For example, if you include the file with the line:

#include %graphics/defines.r

and defines.r has:

#include %draw.r
#include %render.r
#include-binary %image1.jpg

all of the include files specified in defines.r are opened in the %graphics directory.

Including Groups of Files

The #include-files command requires additional explanation. This command returns a block of paired values: the input you provided (a word or filename) and its contents. This lets you write expressions such as:

foreach [word data] #include-files %graphics [
    header.png
    backdrop.png
    button.png
    arrow.png
][
    set word load data
]

view layout [
    image backdrop.png
    btn button.png [...]
]

This example will create an output file that contains the necessary images, then at run-time loads them and sets them to a word which is identical to their file names. This technique is useful if you require a lot of images or sounds in your program.

Note that in the VIEW LAYOUT a percent does not appear before backdrop.png because it is actually a word (a variable) that holds the image to be viewed.

Include File Headers

As part of preprocessing all include files will have their REBOL headers removed. However, the main file provided to the preprocessor will keep it's REBOL header.

For example, if your main file is:

REBOL [Title: "Main Program"]
#include %draw.r
...

and draw.r has:

REBOL [Title: "Drawing Functions"]

draw-box: func [...]

The output file will become:

REBOL [Title: "Main Program"]

draw-box: func [...]

This feature lets you keep your REBOL include files properly documented with headers (for identification, version control, authorship, etc.), but produce an optimal output file when preprocessed. It also lets you join together multiple include files without intervening data (such as when you want to concatenate several smaller files into a large table in the output file. You don't want the REBOL header being part of the resulting table).

Evaluation Commands

The evaluation commands let you execute any REBOL expression as part of preprocessing. They can be used to process include files in more detail (like building tables, etc.), dynamically create version numbers, auto-configure the output file, print status information during preprocessing, and much more.

The evaluation commands also eliminate the need for other types of preprocessor commands and allows an unlimited variety of expressions and results to be processed.

The evaluation commands are:

#do [expr]

#if [expr] [then-block]

#either [expr] [then-block] [else-block]

The #do, #if, and #either commands let you execute any REBOL expression within the preprocessor.

In the case of #do the result of the expression will be placed in the output file. For example:

version: #do [1.2.3 + 0.0.1]

will create the output:

version: 1.2.4

If you do not want the result of #do in the output file, return NONE.

#do [vers: 1.2.3 + 0.0.1  none]

The expression returns NONE, so nothing will be insert into the output. In addition, if you use an expression that returns nothing (such as print) nothing will be added to the output:

#do [print ["Version:" version]]

#do [start: time/precise  none]
...
#do [write %elapsed-time difference time/precise start]

The #if and #either commands will put the results of their blocks into the output file. For example:

version: #if [new-version] [2.3.4] [1.2.3]

will result in:

version: 2.3.4

if new-version is TRUE, or

version: 1.2.3

if new-version is false.

If you use an evaluation command and you need to insert a NONE as the result, you must return the word NONE, not the NONE value. For example:

colors: #do [either block? data [data]['none]]

This line would insert a block of data or the word NONE.

Conditional Includes

You can conditionally include files by using both the evaluation and include commands together. For example:

#if [flag] [
    #include %file1.r
    #include %file2.r
]

If FLAG is TRUE, then the block with the two include commands will be included in the output, and they will be processed (recursively).

Be sure to note the difference between the above expression and this one:

if flag [
    #include %file1.r
    #include %file2.r
]

The first example only includes file1 and file2 if the flag is set. The second will always include file1 and file2 in the program but will only evaluate their contents if the flag is set.

Example Usage

Here is an example program:

REBOL [Title: "Example"]

Include a couple source files:

#include %library.r
#include %actions.r

Compute a version number, print it during preprocessing, save it to a file, and include the version as part of the output file:

version: #do [
    vers: load %version.r
    vers: vers + 0.0.1
    print ["Version:" vers]
    save %version.r vers
    vers
]

Set a flag to check later during preprocessing:

#do [big-version: false]

Conditionally include two files if the BIG-VERSION option has been set.

#if [big-version] [
    #include %special.r
    #include %features.r
]

Include a file, but only if it exists:

#if [exists? %extra.r] [#include %extra.r]

Include one file or another, based on a condition:

#either [now/time > 12:00] [
    #include %afternoon.r
][
    #include %beforenoon.r
]

Include a string as text within the program:

text: #include-string %text.txt

Include a large string as compressed text within the program, and decompress it when the program runs:

text: decompress #do [compress read %text.txt]

Include an image file as binary. Keep the image in its compressed format until the program is executed:

image: load #include-binary %image.jpg

Hint for Creating Module Namespaces

If you use the preprocessor to combine a set of source code files, you can use REBOL objects to create separate namespaces for module-local variables. This is easily done by creating module files that have a format such as:

REBOL [Title: "Graphics Module" Version: 1.0.0]

graphics-module: context [

    data: ["Example" 123]

    local-func: func [arg] [
        ...
    ]

    set 'global-func func [arg] [
        ...
    ]

    window: layout [
        ...
    ]
]

When the CONTEXT function is encountered it will be evaluated and its contents initialized (variables and functions set, expressions evaluated, etc.) As with any object, variables that are set within the block (with word: syntax) will only be local to that module.

If you need to delay the module's initialization (for example if it needs variables that are not yet defined or uses a file in a directory that has not been set), you can accomplish this with:

graphics-module: [
    data: ["Example" 123]

    local-func: func [arg] [ ... ]

    set 'global-func func [arg] [... ]

    window: layout [ ... ]
]

init-graphics: does [
    graphics-module: context graphics-module
]

Within your main program, call the init-graphics function:

REBOL [Title: "Main Program"]

#include %graphics.r
#include %otherstuff.r

init-graphics

Including Scripts Not to be Preprocessed

On some occassions, you may need to include a script that contains preprocessor commands that you do not want processed. That can be done with a line such as:

do #include-string %script.r

This will embed the script as a text string within your program, then load and evaluate the script upon execution.

This is in fact how we include the REBOL preprocessor within our Encap products to prevent it's #include values from being processed.

Also, if within your script you need to refer to an issue datatype that is the same as those used by the preprocessor, you can use code such as:

if value = to-issue "include" [...]

to keep the preprocessor from evaluating it.