The use and distribution terms for this software are covered by the Common Public License 1.0, which can be found in the file CPL.TXT at the root of this distribution. By using this software in any fashion, you are agreeing to be bound by the terms of this license. You must not remove this notice, or any other, from this software.
My objective was to provide comprehensive, safe, dynamic and Lisp-y access to Java and Java libraries as if they were Lisp libraries, for use in Lisp programs, i.e. with an emphasis on working in Lisp rather than in Java.
The approach I took was to embed a JVM instance in the Lisp process using JNI. I was able to do this using LispWorks' own FLI and no C (or Java! *) code, which is a tribute to the LW FLI. On top of the JNI layer (essentially a wrapper around the entire JNI API), I built this user-level API using Java Reflection. This first version was built with, and contains code specific to, LispWorks.
jfli ("jay fly") provides:
jfli was built using LWM and LWW (using Apple's and Sun's JVMs respectively), and it works fine on both. More recently (2007) it's been tested extensively on LWL. It should be a trivial port to other LispWorks, and a possible port to any Common Lisp with a robust FLI. It should also work with any JVM with a conformant JNI implementation.
jfli is hosted on SourceForge.
jfli comprises: a small system of three Lisp files (and an accompanying defsys file), a system building utility (build-java-classes.lisp), and an optional Java .jar file. The first Lisp file, jni.lisp, defines a low-level API to the Java Native Interface, and is not documented here. The second, jfli.lisp, depends upon jni.lisp, and provides the user API documented here. Finally process.lisp provides recyclable processes.
To get started:
(load "defsys.lisp") (compile-system 'jfli :load t) (use-package :jfli) *jni-lib-path*.If you wish to allow for callbacks from Java to Lisp, you must place jfli.jar in your classpath when creating the JVM.
This sample session presumes you have already compiled the jfli system.
CL-USER 4 > (load "/lisp/defsys")
; Loading lisp file C:\lisp\defsys.lisp
#P"C:/lisp/defsys.lisp"
CL-USER 5 > (load-system 'jfli)
; Loading fasl file c:\lisp\jni.ofasl
; Loading fasl file c:\lisp\jfli.ofasl
; Loading fasl file c:\lisp\process.ofasl
(JFLI)
;The user API is entirely in the jfli package
CL-USER 6 > (use-package :jfli)
T
;tell the library where Java is located
CL-USER 7 > (setf *jni-lib-path* "/j2sdk1.4.2_01/jre/bin/client/jvm.dll")
"/j2sdk1.4.2_01/jre/bin/client/jvm.dll"
;this starts the VM and sets the class path (to the default plus
;jfli.jar, in this case)
CL-USER 8 > (connect-jvm ())
0
#<Pointer: JNI:PVM = #x081022A0>
#<Pointer: JNI:PENV = #x0086A858>
;define wrappers for the members of Object
CL-USER 9 > (def-java-class "java.lang.Object")
NIL
;and of Properties, a Hashtable-like class
CL-USER 10 > (def-java-class "java.util.Properties")
#<STANDARD-CLASS |java.util|:PROPERTIES. 2066B964>
;the above will create these packages if they do not already exist
;use the packages for easy name access
CL-USER 11 > (use-package "java.lang")
T
CL-USER 12 > (use-package "java.util")
T
;create a Properties instance, note keyword-style member inits, string conversion etc
;also note typed return value
CL-USER 13 > (setf p (new properties. :getproperty "fred" "ethel"))
#<PROPERTIES. 20664A94>
;virtual functions work as normal
CL-USER 14 > (object.tostring p)
"{fred=ethel}"
;setter was generated for member function because it follows the JavaBeans property protocol
CL-USER 15 > (setf (properties.getproperty p "ricky") "lucy")
"lucy"
CL-USER 16 > (object.tostring p)
"{ricky=lucy, fred=ethel}"
CL-USER 17 > (properties.size p)
2
;totally dynamic access, create wrappers as you need
CL-USER 18 > (def-java-class "java.lang.Class")
#<STANDARD-CLASS CLASS. 20680EC4>
CL-USER 19 > (class.getname (object.getclass p))
"java.util.Properties"
CL-USER 20 > (def-java-class "java.util.Enumeration")
#<STANDARD-CLASS ENUMERATION. 20669274>
;no need to wait for the vendor to enhance the language - you use Lisp!
CL-USER 21 > (defmacro doenum ((e enum) &body body)
(let ((genum (gensym)))
`(let ((,genum ,enum))
(do ()
((not (enumeration.hasmoreelements ,genum)))
(let ((,e (enumeration.nextelement ,genum)))
,@body)))))
DOENUM
;can't do this in Java yet
CL-USER 22 > (doenum (prop (properties.elements p)) (print (object.tostring prop)))
"lucy"
"ethel"
NIL
;doc strings are created giving original Java signatures and indicating
overloads
CL-USER 23 > (documentation 'properties.getproperty 'function)
"java.lang.String getProperty(java.lang.String,java.lang.String)
java.lang.String getProperty(java.lang.String)
"
CL-USER 24 > (documentation 'properties.new 'function)
"java.util.Properties()
java.util.Properties(java.util.Properties)
"
*jni-lib-path*
Set this to point to your jvm dll prior to calling create-jvm.
(setf *jni-lib-path* "C:/Program Files/Java/jre1.6.0_05/bin/client/jvm.dll")
(create-jvm &rest option-strings) -> unspecified
Creates/starts the JVM. You must call this prior to calling any other jfli
function, and you cannot subsequently make a further call to
either create-jvm or connect-jvm (a Java limitation).
The option strings can be used to control the JVM, especially the classpath:
(create-jvm "-Djava.class.path=/Lisp/jfli.jar")
See the JNI documentation for other initialization options.
(connect-jvm class-paths &optional option-strings) -> unspecified
Alternative, simpler interface to creating/starting the JVM. class-paths is a list of pathnames (to which jfli.jar will be added, and then each path will each be merged with the jfli source location). The option strings can be used to control other aspects of the JVM.
You must call this prior to calling any other jfli function, and you cannot
subsequently make a further call to either create-jvm
or connect-jvm (a Java limitation).
(connect-jvm '("/Lisp/jfli.jar"))
See the JNI documentation for other initialization options.
(enable-java-proxies)
-> unspecified
Sets up the Java->Lisp callback support. Must be called (once) before any calls to new-proxy, and requires jfli.jar be in the classpath.
(def-java-class full-class-name) -> unspecified
Given the package-qualified, case-correct name of a Java class as a string, will generate wrapper functions for its public constructors, fields and methods.
The core API for generation interfaces to Java is the def-java-class macro. This macro will, at expansion time, use Java reflection to find all of the public constructors, fields and methods of the given class and generate functions to access them.
(def-java-class "java.lang.ClassName")
you get several symbols/functions:
|java.lang| (note case)classname. (note the dot is part of the name)(classname.new &rest args) -> typed-reference, which
returns a typed reference to the newly created object
make-new, ultimately
calling classname.new, specialized on (the value of) the class-symbol
(classname.fieldname [instance]) -> field value(setf classname.fieldname [instance])*classname.fieldname*
If the type of the field is primitive, the field value will be converted to a native Lisp value. If it is a Java String, it will be converted to a Lisp string. Otherwise, a generic reference to the Java object is returned. Similarly, when setting, Lisp values will be accepted for primitives, Lisp strings for Strings, or (generic or typed) references for reference types.
(classname.methodname &rest args) -> return-valuegetSomething or isSomething and
there is a corresponding setSomething), then a (setf
classname.methodname) will be defined that calls the latter.
The same argument and return value conversions are performed as are for fields. The function documentation string describes the method signature(s) from the Java perspective.
(get-jar-classnames jar-file-name &rest packages)
-> list-of-strings
Returns a list of class name strings. Packages should be strings of the form "java/lang " for recursive lookup and "java/util/" (note trailing slash) for non-recursive.
(dump-wrapper-defs-to-file filename classnames) ->
unspecified
Given a list of classnames (say from get-jar-classnames), writes
calls to def-java-class to a file:
(dump-wrapper-defs-to-file "/lisp/java-lang.lisp" (get-jar-classnames "/j2sdk1.4.2_01/jre/lib/rt.jar " "java/lang/")) (compile-file "/lisp/java-lang") (load "/lisp/java-lang") (use-package "java.lang") ;Wrappers for all of java.lang are now available
(cl-user:build-java-classes output-file &optional class-paths option-strings) -> unspecified
Macroexpands def-java-class forms to sufficient extent that the results
can be compiled and loaded into an image which has not yet been connected to the
JVM. The utility connects to the JVM, generates source code and writes this to
output-file which for example could be one of the members of your application's
system. You can subsequently build that system in a fresh lisp, save the image and
only connect that lisp to the JVM when the image is restarted.
This function is defined in the utility file build-java-classes.lisp which is not part of the JFLI defsystem. To use it to generate forms and write them to output-file:
def-java-class forms in the file whose name is that of output-file
with ".src" tagged onto the endcl-user:build-java-classes; class-paths and option-strings
will be passed to connect-jvmread-from-string; the rest of the file -
comments, in-package forms, etc - is output as-is.build-java-classes again (and you may), class-paths
and option-strings will be ignored.build-java-classes are undefined.CL-USER 1 > (load "/home/nick/p4/project/frob/master/code/jfli/build-java-classes.lisp") ; Loading text file /home/nick/p4/project/frob/master/code/jfli/build-java-classes.lisp ; Loading text file /home/nick/p4/project/frob/master/code/jfli/defsys.lisp ;; Creating system JFLI ; Loading fasl file /home/nick/p4/project/frob/master/code/jfli/jni.ufasl ; Loading fasl file /home/nick/p4/project/frob/master/code/jfli/jfli.ufasl ; Loading fasl file /home/nick/p4/project/frob/master/code/jfli/process.ufasl #P"/home/nick/p4/project/frob/master/code/jfli/build-java-classes.lisp" CL-USER 2 > (build-java-classes "../wombat/java-classes.lisp" (directory "../wombat/*.jar")) ;; Reading /home/nick/p4/project/frob/master/code/wombat/java-classes.lisp.src ;; 12 def-java-class forms processed "../wombat/java-classes.lisp" CL-USER 3 >
(make-new class-symbol
&rest args) -> typed-reference
Allows for definition of before/after methods on constructors. Calls classname.new.
The new macro expands into a call to this.
(new class-spec &rest args) -> typed-reference
class-spec -> class-name | (class-name this-name)
class-name -> "package.qualified.ClassName" | classname.
args -> [actual-arg]* [init-arg-spec]*
init-arg-spec -> init-arg | (init-arg)
init-arg -> :settable-field-or-method [params]* value (note keyword)
| .method-name [args]* (note leading dot)
Creates a new instance of class-name, by expanding into a call to the make-new generic function, then initializes it by setting fields or accessors and/or calling member functions. If this-name is supplied, it will be bound to the newly-allocated object and available to the init-args:
(new (button. this) shell *SWT.CENTER* ;the actual args
:gettext "Call Lisp" ;a javabean property
(.addlistener *swt.selection* ;a method call
(new-proxy (listener.
(handleevent (event)
(declare (ignore event))
(setf (button.gettext this) ;this is bound to new instance
(format nil "~A ~A"
(lisp-implementation-type)
(lisp-implementation-version)))
nil))))
.setsize 200 100 ;can omit parens
(.setlocation 40 40))
Expands into:
(LET* ((#:G598 (MAKE-NEW BUTTON. SHELL *SWT.CENTER*)) (THIS #:G598))
(SETF (BUTTON.GETTEXT #:G598) "Call Lisp")
(BUTTON.ADDLISTENER #:G598
*SWT.SELECTION*
(NEW-PROXY (LISTENER.
(HANDLEEVENT
(EVENT)
(DECLARE (IGNORE EVENT))
(SETF (BUTTON.GETTEXT THIS)
(FORMAT NIL
"~A ~A"
(LISP-IMPLEMENTATION-TYPE)
(LISP-IMPLEMENTATION-VERSION)))
NIL))))
(BUTTON.SETSIZE #:G598 200 100)
(BUTTON.SETLOCATION #:G598 40 40)
#:G598)
(make-new-array type &rest dimensions) ->
reference to new array
Generic function with methods defined for all Java class designators:
Creates a Java array of the requested type with the requested dimensions.
(jlength array) -> integer
Like length, for Java arrays
(jref array &rest subscripts) -> reference
Like aref, for Java arrays of non-primitive (reference) types, settable.
(jref-xxx array &rest subscripts) -> value
Where xxx = boolean|byte|char|double|float|int|long|short. Like jref, for Java arrays of primitive types, settable.
Proxies allow the creation of Java objects that implement one or more interfaces
in Lisp, and thus callbacks from Java to Lisp. You must call
enable-java-proxies before using this proxy API. A significant
limitation is that LispWorks appears to not support calls back into Lisp other
than from threads initiated by Lisp, so you must ensure that the proxy will not
be called from an arbitrary Java thread!
(new-proxy &rest interface-defs) -> reference
interface-def -> (interface-name method-defs+)
interface-name -> "package.qualified.ClassName" | classname. (must name a Java
interface type)
method-def -> (method-name arg-defs* body)
arg-def -> arg-name | (arg-name arg-type) arg-type -> "package.qualified.ClassName
" | classname. | :primitive
method-name -> symbol | string (matched case-insensitively)
Creates, registers and returns a Java object that implements the supplied interfaces
(unregister-proxy proxy) -> unspecified
Stops handling for the proxy (which must have been created by new-proxy)
and removes references from the Lisp side. Make sure it is no longer referenced
from Java first!
(jeq obj1 obj2) -> boolean
Are the 2 java objects the same object? Note that this is not the same as Object.equals()
(find-java-class class-sym-or-string) ->
reference to Java Class object
Given a Java class designator, returns the Java Class object. Use this in preference to Class.forName() when using jfli.
(make-typed-ref java-ref) -> typed-reference
Given a generic Java reference, determines the full type of the object and returns an instance of a typed reference wrapper. classname.new/make-new/new always return typed references, but since Java methods might return Object or some interface type, and we don't want to always incur the cost of type determination, field and method wrapper functions return generic references. Use this function to create a typed reference corresponding to the full actual type of the object when desired.
(box-xxx value) -> reference to Java primitive wrapper class
Where xxx = boolean|byte|char|double|float|int|long|short|string. Given a compatible Lisp value, creates an instance of the corresponding Java primitive wrapper class, e.g. Integer. This should rarely be needed, but can be used to force overloading resolution.
(unbox-xxx ref) -> Lisp value
Where xxx = boolean|byte|char|double|float|int|long|short|string. Given an instance of a Java primitive wrapper class, creates an instance of the corresponding compatible Lisp value. This should rarely be needed, but can be used to unbox values returned by Java Object-based APIs.
This is one area of jfli which needs more attention. The JNI call DetachCurrentThread has proved most problematic (SEGVs in foriegn code, random corruption in Java internals and heaven only knows what else) and so we need another solution to prevent used threads from "leaking". The solution proposed, to recycle lisp threads rather than letting them go, is implemented in the file process.lisp which can be used as a standalone facility indepenently of the jfli.
(mp:cached-process-run-function name keywords function &rest args) -> mp:process
As for mp:process-run-function except that when the process terminates
it is cached for later reuse (by a subsequent call to mp:cached-process-run-function)
rather than dropped. It is recommended that all threads which are going to communicate
with the JVM should be created using mp:cached-process-run-function. This applies in
particular if your application uses lots of short-lived threads to talk to Java.
java-exception
An instance of this subtype of error will be signalled if an exception
occurs in the JVM.
(java-exception-exception java-exception) -> reference
Reader returning the Java exception whose occurance signalled
java-exception.
(describe-exception java-exception) -> string
This function is called when the java-exception is signalled to
generate a text description of the exception which is then stored in the exception and
used by its print-object method (if *print-escape* is
off). In other words, this is how the exception princs. A default method
is defined on java-exception; you might want to define others.
I hope you find jfli useful. It is my sincere intent that it enhance the utility and interoperability of Common Lisp, a language with which I am still becoming familiar, and grow to appreciate more every day. I welcome comments and code contributions.
Rich Hickey, July 2004
Updated by Nick Levine, April 2008