13. Miscellaneous Tools

Programming is more fun with a good tools. This chapter describes snarfing tools, and the guile-tools program which can be used to invoke the rest of the tools (which are self-documenting). Some of these are used in Guile development, too. Imagine that!

13.1 Snarfing		Grepping the source in various ways.
13.2 Executable Modules		Modules callable via guile-tools.

13.1 Snarfing

Because it's easier to maintain documentation, code, and other metainfo in one source file than in many files, there have evolved many methods for grepping source to lift and separate these kinds of info, in the process generating docs or fragments of source or what have you. This is known generally as snarfing, which comes from the verb "to snarf", here meaning "to unceremoniously extract information from a somewhat unwilling source."

This section documents the installed program guile-snarf which does init snarfing, and also touches upon guile's doc snarfing process which is not yet finalized (i.e., doc snarfing programs are not installed at this time).

13.1.1 Init Snarfing with guile-snarf		Exposing C subrs and friends to Scheme.
13.1.2 Doc Snarfing		Generating GDFv2 or texi from source.

13.1.1 Init Snarfing with guile-snarf

When writing C code for use with Guile, you typically define a set of C functions, and then make some of them visible to the Scheme world by calling the scm_c_define_gsubr function; a C function published in this way is called a subr. If you have many subrs to publish, it can sometimes be annoying to keep the list of calls to scm_c_define_gsubr in sync with the list of function definitions. Frequently, a programmer will define a new subr in C, recompile the application, and then discover that the Scheme interpreter cannot see the subr, because of a missed call to scm_c_define_gsubr.

Guile provides the guile-snarf command to manage this problem. Using this tool, you can keep all the information needed to define the subr alongside the function definition itself; guile-snarf will extract this information from your source code, and automatically generate a file of calls to scm_c_define_gsubr which you can #include into an initialization function.

13.1.1.1 How guile-snarf works		Using guile-snarf, with example.
13.1.1.2 Macros guile-snarf recognizes		How to mark up code for guile-snarf.

13.1.1.1 How guile-snarf works

Usage: guile-snarf [-o OUTFILE] INFILE [CPP-OPTIONS ...]

What guile-snarf does:

Process INFILE using the C pre-processor and some other programs. Write output to a file named OUTFILE or to the standard output when no OUTFILE has been specified or when OUTFILE is -.

If there are errors during processing, delete OUTFILE and exit with non-zero status.

If env var CPP is set, use its value instead of the C pre-processor determined at Guile configure-time.

During snarfing, the pre-processor macro SCM_MAGIC_SNARFER is defined. You can use this to avoid including snarfer output files that don't yet exist by writing code like this:

#ifndef SCM_MAGIC_SNARFER #include "foo.x" #endif

See section 13.1.1.2 Macros guile-snarf recognizes, for a list of the special (some would say magic) cpp macros you can use, including the list of deprecated macros.

For example, here is how you might define a new subr called clear-image, implemented by the C function clear_image:

#include <libguile.h> SCM_DEFINE (clear_image, "clear-image", 1, 0, 0, (SCM image_smob), "Clear the image.") #define FUNC_NAME s_clear_image { /* C code to clear the image... */ } #undef FUNC_NAME void init_image_type () { #include "image-type.x" }

The SCM_DEFINE declaration says that the C function clear_image implements a Scheme subr called clear-image, which takes one required argument (type SCM named image_smob), no optional arguments, and no tail argument. See section 13.1.2 Doc Snarfing, for info on the docstring.

This works in concert with FUNC_NAME to also define a static array of characters named s_clear_image, initialized to the string "clear-image". The body of clear_image may use the array in error messages, instead of writing out the literal string; this may save string space on some systems.

Assuming the text above lives in a file named `image-type.c', you will need to execute the following command to prepare this file for compilation:

guile-snarf -o image-type.x image-type.c

This scans `image-type.c' for SCM_DEFINE declarations, and writes to `image-type.x' the output:

scm_c_define_gsubr (s_clear_image, 1, 0, 0, (SCM (*)() ) clear_image);

When compiled normally, SCM_DEFINE is a macro which expands to a declaration of the s_clear_image string.

Note that the output file name matches the #include from the input file. Also, you still need to provide all the same information you would if you were using scm_c_define_gsubr yourself, but you can place the information near the function definition itself, so it is less likely to become incorrect or out-of-date.

If you have many files that guile-snarf must process, you should consider using a fragment like the following in your Makefile:

snarfcppopts = $(DEFS) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) .SUFFIXES: .x .c.x: guile-snarf -o $ $< $(snarfcppopts)

This tells make to run guile-snarf to produce each needed `.x' file from the corresponding `.c' file.

Aside from the required argument INFILE, guile-snarf passes its command-line arguments directly to the C preprocessor, which it uses to extract the information it needs from the source code. this means you can pass normal compilation flags to guile-snarf to define preprocessor symbols, add header file directories, and so on.

13.1.1.2 Macros guile-snarf recognizes

Here are the macros you can use in your source code from which guile-snarf can construct initialization code:

/* procedures */ SCM_DEFINE (FNAME, PRIMNAME, REQ, OPT, VAR, ARGLIST, DOCSTRING) SCM_PROC (RANAME, STR, REQ, OPT, VAR, CFN) SCM_REGISTER_PROC (RANAME, STR, REQ, OPT, VAR, CFN) SCM_GPROC (RANAME, STR, REQ, OPT, VAR, CFN, GF) /* everything else */ SCM_SYMBOL (c_name, scheme_name) SCM_GLOBAL_SYMBOL (c_name, scheme_name) SCM_KEYWORD (c_name, scheme_name) SCM_GLOBAL_KEYWORD (c_name, scheme_name) SCM_VARIABLE (c_name, scheme_name) SCM_GLOBAL_VARIABLE (c_name, scheme_name) SCM_VARIABLE_INIT (c_name, scheme_name, init_val) SCM_GLOBAL_VARIABLE_INIT (c_name, scheme_name, init_val)

REQ and OPT are numbers indicating required and optional argument counts, respectively; VAR is a number that, if non-zero, means the function will accept any remaining arguments as a list; DOCSTRING is a string (use \n\ at eol for multi-line); FNAME is a C-language identifier, CFN and GF and c_name likewise; PRIMNAME is a string denoting the name available to Scheme code, STR and scheme_name likewise; RANAME is the name of the static string (must match that declared by the associated definition of cpp macro FUNC_NAME); ARGLIST is an argument list (in parentheses); and lastly, init_val is a expression suitable for initializing a new variable.

For procedures, you can use SCM_DEFINE for most purposes. Use SCM_PROC along with SCM_REGISTER_PROC when you don't want to be bothered with docstrings. Use SCM_GPROC for generic functions (@xref{GOOPS,,,goops}). All procedures are declared static with return type SCM.

For everything else, use the appropriate macro (SCM_SYMBOL for symbols, and so on). The "_GLOBAL_" variants omit static declaration.

All these macros should be used at top-level, outside function bodies. Also, it's a good idea to define FUNC_NAME immediately after using SCM_DEFINE (and similar), and then the function body, and then #undef FUNC_NAME.

See section 13.1.1.1 How guile-snarf works, and also libguile source, for examples. See section 18.2.5.5 Subrs, for details on argument passing and how to write C functions.

13.1.2 Doc Snarfing

In addition to init snarfing (see section 13.1.1 Init Snarfing with guile-snarf), the libguile sources are also subject to doc snarfing, by programs that are included in the distribution (but not installed at this time). The output is the file `guile-procedures.txt' which is installed, and subsequently used by module (ice-9 documentation).

Here is a list of what does what according to `libguile/Makefile.am':

• guile-snarf-docs runs cpp defining SCM_MAGIC_SNARF_DOCS
• guile_filter_doc_snarfage parses guile-snarf-docs output to produce .doc
• ../scripts/snarf-check-and-output-texi makes guile.texi
• ../scripts/snarf-check-and-output-texi makes guile-procedures.txt
• guile-func-name-check checks source snarf-syntax integrity (optional?)
• guile-doc-snarf calls guile-snarf-docs (to make .doc) and guile-snarf

Note that for guile-1.4, a completely different approach was used! All this is rather byzantine, so for now NO doc snarfing programs are installed.

[fixme: Document further once doc snarfing is tamed somewhat. --ttn]

13.2 Executable Modules

When Guile is installed, in addition to the (ice-9 FOO) modules, a set of executable modules (scripts BAR) is also installed. Each is a regular Scheme module that has some additional packaging so that it can be called as a program in its own right, from the shell. For this reason, we sometimes use the term script in this context to mean the same thing.

As a convenience, the guile-tools wrapper program is installed along w/ guile; it knows where a particular module is installed and calls it passing its args to the program. The result is that you need not augment your PATH. Usage is straightforward:

guile-tools --help guile-tools --version guile-tools [OPTION] PROGRAM [ARGS ...] If PROGRAM is "list" or omitted, display contents of scripts dir, otherwise PROGRAM is run w/ ARGS. Options (only one of which may be used at a time): --scriptsdir DIR -- Look in DIR for scripts --guileversion VERS -- Look in $pkgdatadir/VERS/scripts for scripts --source -- Display PROGRAM source (ignore ARGS) to stdout

The modules are self-documenting. For example, to see the documentation for lint, use one (or both) of the shell commands:

guile-tools display-commentary '(scripts lint)' guile-tools --source lint

The rest of this section describes the packaging that goes into creating an executable module. Feel free to skip to the next chapter.

13.2.1 Writing Executable Modules

See template file PROGRAM for a quick start.

Programs must follow the executable module convention, documented here:

• The file name must not end in ".scm".

• The file must be executable (chmod +x).

• The module name must be "(scripts PROGRAM)". A procedure named PROGRAM w/ signature "(PROGRAM . args)" must be exported. Basically, use some variant of the form:

  (define-module (scripts PROGRAM) :export (PROGRAM))

  Feel free to export other definitions useful in the module context.

• There must be the alias:

  (define main PROGRAM)

  However, `main' must NOT be exported.

• The beginning of the file must use the following invocation sequence:

  #!/bin/sh main='(module-ref (resolve-module '\''(scripts PROGRAM)) '\'main')' exec ${GUILE-guile} -l $0 -c "(apply $main (cdr (command-line)))" "$@" !#

Following these conventions allows the program file to be used as module (scripts PROGRAM) in addition to as a standalone executable. Please also include a helpful Commentary section w/ some usage info.