Per Bothner's blog

This article talks about Kawa's support for compound symbols and namespaces. The basic support has been there for a while, but some features are new, and the documentation is more complete. (This is an extract from the SVN version of the Kawa manual.)

Namespaces and compound symbols

Different applications may want to use the same symbol to mean different things. To avoid such name clashes we can use compound symbols, which have two string parts: a local name and a namespace URI. The namespace-uri can be any string, but it is recommended that it have the form of an absolute URI. It would be too verbose to write the full URI all the time, so one usually uses a namespace prefix (namespace alias) as a short local alias to refer to a namespace URI.

Compound symbols are usually written using the infix colon operator:

     prefix:local-name

where prefix is is namespace alias bound to some (lexically-known) namespace URI.

Compound symbols are used for namespace-aware XML processing.

Namespace objects

A namespace is a mapping from strings to symbols. The string is the local-name of resulting symbol. A namespace is similar to a Common Lisp package.

A namespace has a namespace-uri, which a string; it recommended that it have the form of an absolute URI. A namespace may optionally have a prefix, which is a string used when printing out symbols belonging to the namespace. (If you want for “equivalent symbols” (i.e. those that have the same local-name and same uri) to be the identical symbol object, then you should use namespaces whose prefix is the empty string.)

The reader macro #,namespace is equivalent to the namespace function, but it is invoked at read-time:

     #,(namespace "http://www.w3.org/1999/XSL/Transform" xsl)
     (eq? #,(namespace "foo") (namespace "foo")) ⇒ #t

The form (,#namespace "" "") returns the default empty namespace, which is used for simple symbols.

Compound symbols

A compound symbol is one that belongs to a namespace other than the default empty namespace, and (normally) has a non-empty namespace uri. (It is possible for a symbol to belong to a non-default namespace and have an empty namespace uri, but that is not recommended.)

Two symbols are equal? or eqv? if they're symbol=?.

Namespace aliases

A namespace is usually referenced using a shorter namespace alias, which is is a lexical definition that binds a namespace prefix to a namespace object (and thus a namespace uri). This allows using compound symbols as identifiers in Scheme programs.

For example you might have a set of a geometry definitions defined under the namespace-uri "http://foo.org/lib/geometry":

     (define-namespace geom "http://foo.org/lib/geometry")
     (define (geom:translate x y)
       (java.awt.geom.AffineTransform:getTranslateInstance x y))
     (define geom:zero (geom:translate 0 0))
     geom:zero
       ⇒ AffineTransform[[1.0, 0.0, 0.0], [0.0, 1.0, 0.0]]

You could have some other definitions for complex math:

     (define-namespace complex "http://foo.org/lib/math/complex")
     (define complex:zero +0+0i)

You can use a namespace-value directly in a compound name:

     (namespace "http://foo.org/lib/geometry"):zero
       ⇒ AffineTransform[[1.0, 0.0, 0.0], [0.0, 1.0, 0.0]]

The variation define-xml-namespace is used for creating XML nodes.

     $ kawa --output-format xml
     #|kawa:1|# (define-xml-namespace na "Namespace1")
     #|kawa:2|# (define-xml-namespace nb "Namespace1")
     #|kawa:3|# (define xa (na:em "Info"))
     #|kawa:4|# xa
     <na:em xmlns:na="Namespace1">Info</na:em>
     #|kawa:5|# (define xb (nb:em "Info"))
     #|kawa:6|# xa
     <nb:em xmlns:nb="Namespace1">Info</nb:em>

Note that the prefix is part of the qualified name (it is actually part of the namespace object), and it is used when printing the tag. Two qualified names (symbols) that have the same local-name and the same namespace-name are considered equal, even if they have different prefix. You can think of the prefix as annotation used when printing, but not otherwise part of the “meaning” of a compound symbol. They are the same object if they also have the same prefix. This is an important different from traditional Lisp/Scheme symbols, but it is how XML QNames work.

     #|kawa:7|# (instance? xb na:em)
     true
     #|kawa:8|# (eq? 'na:em 'nb:em)
     false
     #|kawa:9|# (equal? 'na:em 'nb:em)
     true
     #|kawa:10|# (eqv? 'na:em 'nb:em)
     true

(Note that #t is printed as true when using XML formatting.)

The predefined html prefix could be defined thus:

     (define-xml-namespace html "http://www.w3.org/1999/xhtml")

XML literals are a new feature in Kawa. They make it more convenient to write XML data objects, especially if you're familar with XML syntax. (This is an extract from the SVN version of the Kawa manual.)

You can write XML literals directly in Scheme code, following a #. Notice that the outermost element needs to be prefixed by #, but nested elements do (and must not).

     #<p>The result is <b>final</b>!</p>

Actually, these are not really literals since they can contain enclosed expressions:

     #<em>The result is &{result}.</em>

The value of result is substituted into the output, in a similar way to quasi-quotation. (If you try to quote one of these “XML literals”, what you get is unspecified and is subject to change.)

An xml-literal is usually an element constructor, but there some rarely used forms (processing-instructions, comments, and CDATA section) we'll cover later.

     xml-literal ::= #xml-constructor
     xml-constructor ::= xml-element-constructor
       | xml-PI-constructor
       | xml-comment-constructor
       | xml-CDATA-constructor

Element constructors

     xml-element-constructor ::=
         <QName xml-attribute*>xml-element-datum...</QName >
       | <xml-name-form xml-attribute*>xml-element-datum...</>
       | <xml-name-form xml-attribute*/>
     xml-name-form ::= QName
       | xml-enclosed-expression
     xml-enclosed-expression ::=
         {expression}
       | (expression...)

The first xml-element-constructor variant uses a literal QName, and looks like standard non-empty XML element, where the starting QName and the ending QName must match exactly:

     #<a href="next.html">Next</a>

As a convenience, you can leave out the ending tag(s):

     <para>This is a paragraph in <emphasis>DocBook</> syntax.</>

You can use an expression to compute the element tag at runtime - in that case you must leave out the ending tag:

     #<p>This is <(if be-bold 'strong 'em)>important</>!</p>

You can use arbitrary expression inside curly braces, as long as it evaluates to a symbol. You can leave out the curly braces if the expression is a simple parenthesised compound expression. The previous example is equivalent to:

     #<p>This is <{(if be-bold 'strong 'em)}>important</>!</p>

The third xml-element-constructor variant above is an XML “empty element”; it is equivalent to the second variant when there are no xml-element-datum items.

(Note that every well-formed XML element, as defined in the XML specifications, is a valid xml-element-constructor, but not vice versa.)

Elements contents (children)

The “contents” (children) of an element are a sequence of character (text) data, and nested nodes. The characters &, <, and > are special, and need to be escaped.

     xml-element-datum ::=
         any character except &, or <.
       | xml-constructor
       | xml-escaped
     xml-escaped ::=
         &xml-enclosed-expression
       | &xml-entity-name;
       | xml-character-reference
     xml-character-reference ::=
         &#decimal-digit+;
       | &#xhex-digit+;

Here is an example shows both hex and decimal character references:

     #<p>A&#66;C&#x44;E</p>  ⇒  <p>ABCDE</p>

Currently, the only supported values for xml-entity-name are the builtin XML names lt, gt, amp, quot, and apos, which stand for the characters <, >, &, ", and ', respectively. The following two expressions are equivalent:

     #<p>&lt; &gt; &amp; &quot; &apos;</p>
     #<p>&{"< > & \" '"}</p>

Attributes

     xml-attribute ::=
         xml-name-form=xml-attribute-value
     xml-attribute-value ::=
         "quot-attribute-datum*"
       | 'apos-attribute-datum*'
     quot-attribute-datum ::=
         any character except ", &, or <.
       | xml-escaped
     apos-attribute-datum ::=
         any character except ', &, or <.
       | xml-escaped

If the xml-name-form is either xmlns or a compound named with the prefix xmlns, then technically we have a namespace declaration, rather than an attribute.

QNames and namespaces

The names of elements and attributes are qualified names (QNames), which are represented using compound symbols (see Namespaces). The lexical syntax for a QName is either a simple identifier, or a (prefix,local-name) pair:

     QName ::= xml-local-part
        | xml-prefix:xml-local-part
     xml-local-part ::= identifier
     xml-prefix ::= identifier

An xml-prefix is an alias for a namespace-uri, and the mapping between them is defined by a namespace-declaration. You can either use a define-namespace form, or you can use a namespace declaration attribute:

     xml-namespace-declaration-attribute ::=
         xmlns:xml-prefix=xml-attribute-value
       | xmlns=xml-attribute-value

The former declares xml-prefix as a namespace alias for the namespace-uri specified by xml-attribute-value (which must be a compile-time constant). The second declares that xml-attribute-value is the default namespace for simple (unprefixed) element tags. (A default namespace declaration is ignored for attribute names.)

     (let ((qn (element-name #<gnu:b xmlns:gnu="http://gnu.org/"/>)))
       (list (symbol-local-name qn)
             (symbol-prefix qn)
             (symbol-namespace-uri qn)))
     ⇒ ("b" "gnu" "http://gnu.org/")
     

Other XML types

Processing instructions

An xml-PI-constructor can be used to create an XML processing instruction, which can be used to pass instructions or annotations to an XML processor (or tool). (Alternatively, you can use the processing-instruction type constructor.)

     xml-PI-constructor ::= <?xml-PI-target xml-PI-content?>
     xml-PI-target ::= NCname (i.e. a simple (non-compound) identifier)
     xml-PI-content ::= any characters, not containing ?>.

For example, the DocBook XSLT stylesheets can use the dbhtml instructions to specify that a specific chapter should be written to a named HTML file:

     #<chapter><?dbhtml filename="intro.html" ?>
     <title>Introduction</title>
     ...
     </chapter>

XML comments

You can cause XML comments to be emitted in the XML output document. Such comments can be useful for humans reading the XML document, but are usually ignored by programs. (Alternatively, you can use the comment type constructor.)

     xml-comment-constructor ::= <!--xml-comment-content-->
     xml-comment-content ::= any characters, not containing --.

CDATA sections

A CDATA section can be used to avoid excessive use of xml-entity-ref such as & in element content.

     xml-CDATA-constructor ::= <![CDATA[xml-CDATA-content]]>
     xml-CDATA-content ::= any characters, not containing ]]>.

The following are equivalent:

     #<p>Specal characters <![CDATA[< > & ' "]]> here.</p>
     #<p>Specal characters &lt; &gt; &amp; &quot; &apos; here.</p>

Kawa remembers that you used a CDATA section in the xml-element-constructor and will write it out using a CDATA constructor.

Kawa has seemingly been pretty quiet, but lately I'm been making a lot of little improvements.

While it's going to take a while before Kawa can claim to implement R6RS, I've made a bunch of changes that bring it closer:

• Support the R6RS import keyword, including support for renaming.

• Implemented the (rnrs sorting) library.

• Various little changes in character literal and string literal syntax.

• The comment prefix #; skips the following S-expression, as specified by SRFI-62.

• All the R6RS exact bitwise arithmetic functions are now implemented and documented in the manual. The new standard functions (for example bitwise-and) are now preferred over the old functions (for example logand).

Kawa also support some new SRFIs: SRFI-62 (S-expression comments); SRFI-64 (Scheme API for test suites); SRFI-95 (Sorting and Merging); SRFI-97 (Names for SRFI Libraries). The last is a naming convention for R6RS import statements to reference SRFI libraries.

Also a lot of other changes and performance improvements. See the Kawa news page for more information.

It is (way past) time for an actual release, which will be called Kawa 1.10. I think we've got enough features (I'm hopying to add JSR-223 support before I call it done), but there is lots of little cleanups, plus documentation to write.

(This is an update of AndroidHelloScheme.)

Google's phone operating system "Android" is based on a custom Java virtual machine on top of GNU/Linux. So it occurred to me: How difficult would it be to get a Kawa application running on Android? Not that difficult, it turns out.

Here is "Hello world" written in Kawa Scheme:

(require 'android-defs)
(activity hello
  (on-create-view
   (let ((tv (android.widget.TextView (this))))
     (tv:setText "Hello, Android from Kawa Scheme!")
     tv)))

The following instructions have been tested on GNU/Linux, specifically Fedora 10, but if you've managed to build Android applications under (say) Windows, you should be able to appropriately modify these instructions. The article Android Phone development from the Linux command-line helped me figure out what to do.

Getting and building Kawa and Android

First download the Android SDK. Unzip in a suitable location, which we'll refer to as ANDROID_HOME.

$ ANDROID_HOME=/path/to/android-sdk-linux_x86-1.0_r2
$ PATH=$ANDROID_HOME/tools:$PATH

To get things to work I had to make some modest changes to Kawa, so you will need to get the Kawa developer sources from SVN.

You need to configure and make Kawa appropriately:

$ KAWA_DIR=path_to_Kawa_sources
$ cd $KAWA_DIR
$ ./configure --with-android=$ANDROID_HOME/android.jar --disable-xquery
$ make

Creating the application

Next, we need to create a project or activity, in the target directory KawaHello, with the main activity being a class named hello in a package kawa.android:

$ activitycreator --out KawaHello kawa.android.hello

Replace the skeleton hello.java by the Scheme code at the top of this note:

$ cd KawaHello
$ HELLO_APP_DIR=`pwd`
$ cd $HELLO_APP_DIR/src/kawa/android/
$ rm hello.java
$ emacs hello.scm

We need to copy/link the Kawa jar file so the Android SDK can find it:

$ cd $HELLO_APP_DIR
$ ln -s $KAWA_DIR/kawa-1.9.3.jar libs/kawa.jar

We also need to modify the Ant build.xml so it knows how to compile Scheme code:

$ patch < build-xml-patch.txt

Finally, we can compile our application:

$ ant

Running the application on the Android emulator

Start up the Android emulator:

$ emulator&

Wait until Android has finished booting, clisk the menu and home buttons. Click the tab above the menu key to show the installed applications. Now install our new application:

adb install bin/hello-debug.apk

The new hello application should show up. Click it, and you should see something like:

Running the application on the G1 phone

If the emulator is running, kill it:

$ kill %emulator

Connect the phone to your computer with the USB cable. Verify that the phone is accessible to adb:

$ adb devices
List of devices attached 
HT849GZ17337	device

If you don't see a device listed, it may be permission problem. You can figure out which device corresponds to the phone by doing:

$ ls -l /dev/bus/usb/*
/dev/bus/usb/001:
total 0
...
crw-rw-rw- 1 root wheel 189, 5 2009-01-18 16:52 006
...

The timestamp corresponds to when you connected the phone. Make it readable:

$ sudo chmod a+w /dev/bus/usb/001/006

Obviously if you spend time developing for an Androd phone you'll want to automate this process; this link or this link may be helpful.

Anyway, once adb can talk to the phone, you install in the same way as before:

adb install bin/hello-debug.apk

Some debugging notes

You will find a copy of the SDK documentation in $ANDROID_HOME/docs/documentation.html.

If the emulator complains that your application has stopped unexpectedly, do:

$ adb logcat

This shows log messages, stack traces, output from the Log.i logging method, and other useful information. (You can alternatively start ddms (Dalvik Debug Monitor Service), click on the kawa.android line in the top-left sub-window to select it, then from the Device menu select Run logcat....)

To uninstall your application, do:

$ adb uninstall kawa.android

(This has been updated here.)

Google's phone operating system "Android" is based on a custom Java virtual machine on top of GNU/Linux. So it occurred to me: How difficult would it be to get a Kawa application running on Android? Not that difficult, it turns out.

Here is "Hello world" written in Kawa Scheme:

(module-extends android.app.Activity)
(module-name kawa.android.hello)
(define (onCreate (savedInstanceState :: android.os.Bundle)) :: void
  (invoke-special android.app.Activity (this) 'onCreate savedInstanceState)
  (let ((tv :: android.widget.TextView (make android.widget.TextView (this))))
    (tv:setText "Hello, Android from Kawa Scheme!")
    ((this):setContentView tv)))

It's got some annoying boiler-plate, though it's similar to the Java version; hopefully we can simplify later.

Here is how to get this program running on the Android emulator on GNU/Linux. (I haven't yet figured out how to get it working on the actual phone.) This article Android Phone development from the Linux command-line was helpful in figuring out what to do.

First you need to download the Android SDK. Unzip, in a suitable location, which we'll refer to as ANDROID_HOME:

ANDROID_HOME=/path/to/android-sdk-linux_x86-1.0_r2
PATH=$ANDROID_HOME/tools:$PATH

To get this to work I had to make some modest changes to Kawa, so you will need to get the Kawa developer sources from SVN.

You need to configure and make Kawa appropriately:

KAWA_DIR=path_to_Kawa_sources
cd $KAWA_DIR
./configure --with-android=$ANDROID_HOME/android.jar
make

Next, we need to create a project or activity, in the target directory KawaHello, with the main activity being a class named hello in a package kawa.android:

activitycreator  --out KawaHello kawa.android.hello

Replace the skeleton hello.java by the Scheme code we started out with:

cd KawaHello
HELLO_APP_DIR=`pwd`
cd $HELLO_APP_DIR/src/kawa/android/
rm hello.java
emacs hello.scm

We need to copy/link the Kawa jar file so the Android SDK can find it:

cd $HELLO_APP_DIR
ln -s $KAWA_DIR/kawa-1.9.3.jar libs/kawa.jar

We also need to modify the Ant build.xml so it knows how to compile Scheme code:

patch < build-xml.patch

Finally, we can compile our application:

ant

Next start up the Android emulator:

emulator&

Wait until Android has finished booting, clisk the menu and home buttons. Click the tab above the menu key to show the installed applications. Now install our new application:

adb install bin/hello-debug.apk

The new hello application should show up. Click it, and you should see something like:

Some debugging notes

You will find a copy of the SDK documentation in $ANDROID_HOME/docs/documentation.html.

If the emulator complains that your application has stopped unexpectedly, start ddms (Dalvik Debug Monitor Service), click on the kawa.android line in the top-left sub-window to select it, then from the Device menu select Run logcat.... This shows log messages, stack traces, output from the Log.i loggin method, and other useful information.

To uninstall your application, do:

adb uninstall kawa.android

I talked about Scheme, XQuery, and JavaFX: Compiling using Kawa or Javac on September 26, 2008 at the JVM Language Summit. Slides are available here.

I've long been interested in improved commend-line interfaces. The latest Kawa (in SVN) has a new implementation of a command interface window. If you start up Kawa with the -w flag:

kawa -w

you get a new Swing JFrame, whose interesting component is a JTextPane for typing in expressions, and displaying the results. The following example uses Kawa's default language Scheme, but works for other Kawa languages, including XQuery.

Color coding

Here the command prompt is in green, and the input command is in bold-face, because it seems useful to emphasize the input as a way of visually separating one command from the previous. The standard error output is in red. Standard output has the plain default style. This is because standard output may have embedded in it other objects and nested styled text.

All of these are implemented using Swing style objects. There is currently no mechanism to modify one these styles, except by modifying the source code, but at some point we will support that.

Input editing

While the color-coding is pretty, even more important is command-line editing. That is you can move the input cursor along the input line, and insert, delete, or replace text before hitting Enter. This is previously available using the GNU readline library, which has some nice features, including a searchable history mechanism. However, you cannot move the cursor using the mouse, which is surprising to many.

Currently, the entire text pane is editable using the default JTextPane keystrokes and mouse handlers. (It would be better that at least the prompt be non-editable.) Nothing gets sent to the receiving reader until you type Enter. At that point the entire line containing the cursor, except for the prompt, is sent to the reader. (More precisely: If the cursor is at or after the output position, the rest of the line after the output position is sent. Otherwise, usually it's a way of repeating or modifying a previous line. In that case the contents of that line, except any initial prompt segment, are first copied to the end of the text buffer, as if typed there, before being sent to the reader.)

If you paste a multi-line string (commonly using ctrl-V), then ReplPane magically interleaves the prompt string and output with the input text. For example if you paste the following text:

(display "foo")
(newline)
(display
  "bar")
(newline)

You get this result:

Note that the prompt for line 4 is different because it's a continuation line. (Alas, this feature is not very robust; it is easy to get Swing exceptions.)

Embedding Components

A powerful feature is that you "print" java.awt.Component objects. These are embedded in the JTextPane. The following Scheme example creates a list of 3 JButton objects, and that list is then "printed":

We can of course save a reference to the button in a variable:

That allows us to modify properies of the button. Here we change its text property, in line 4, and the button is updated as soon as we hit enter:

One anomaly when "printing" a Component is that a Component can only appear once. If we "print" it a second time, it is as a side-effect removed from the first place it was printed:

This isn't really the right behavior, but it's unavoidable when "printing" a Component. To avoid the problem, we need to "print" a some kind of "model" object rather than "view" objects.

Embedding Viewable objects

The experimental swing-gui library provides "viewable model" objects that can also be "printed" to a TextPane. For example, the gnu.kawa.models.Button class goes beyond Swing's ButtonModel in also having text and an action procedure. When a gnu.kawa.models.Button is "printed" the implementation automatically creates a JButton that is co-ordinated with the gnu.kawa.models.Button:

Again, we can change the text property:

Here we do some more complex layout, creating a Row displaying the same button twice, separated by a spacer. Then we create a column that displays the same row twice, separated by a text field. (For some reason the text field isn't displaying properly.)

Playing with composable Java2D pictures

The swing-gui library also provides convenience wrappers to the gnu.kawa.models.Paintable interface, which makes it easy to create, compose, and transform Java2D Shape objects.

Issues

• The ReplPane is not very solid. It's not hard to type something which causes a Swing exception, which is usually not recoverable.
• There should be a View menu to modify fornts and styles, and a standard Edit menu to Cut/Copy/Paste, at least. The File menu should have a way to save the typescript.
• The Utilities->Purge Buffer menu item doesn't work.
• The display should be integrated with the Kawa pretty-printer, so that changing the window width recalculates line-breaks.
• It would be nice to emit styled "inline" (text) objects, such as a "red string". Maybe we should be using Swing's HTMLDocument instead of DefaultStyledDocument. Another toolkit to consider is Flying Saucer.