I Need Closure(s)

'Programming Collective Intelligence' in Common Lisp, Chapter 5 - Optimizations

2008-01-30T08:19:00.000-08:00

Back from the holidays and plowing through the chapters again. The explanation for the various optimizations was good, but I had a hard time figuring out a more "Lisp-like" way of doing the code. Either there isn't, or I haven't quite reached Zen-like Lisp nirvana. I'm open to improvements, here's the code for the generic optimizations and the social network.

For these python translations I'm growing fond of the (incf cl) utilities. It includes macros to make list comprehensions in Lisp, so


sol=[random.randint(domain[i][0],domain[i][1])
     for i in range(len(domain))]

becomes


(defun range-random (low high)
 (+ low (random (1+ (- high low)))))
...
(let ((sol (assemble (range-random (car x) (cdr x)) (<- x domain))))

I know that one would be just as easy with mapcar, see incf-cl for better examples.

Here's a graph made from a simulated annealing optimization of the social network graph:

Programming Collective Intelligence - Viewing Data in Two Dimensions

2007-10-11T05:25:00.001-07:00

An example of taking data in multiple dimensions and "scaling" it to two dimensions, where the distances between data points is proportional to the multi-dimensional distance. Again, this one is basically an imperative rewrite. I hope to be able to exploit more of Lisp's power in the next chapter, which includes some neural networks. The big revelation from this, if anyone says Lisp is unreadable, I'd show them


fakedist[i][j] = sqrt(sum([pow(loc[i][x] - loc[j][x], 2)
                                             for x in range(len(loc[i]))]))

I replaced clusters.lisp, new code is in it.

Zooming in,

'Programming Collective Intelligence' in Common Lisp, Chapter 3 - Hierarchical Clusters

2007-09-27T18:57:00.000-07:00

I have Common Lisp code for Chapter 3 up to Hierarchical Clusters. It's longer and not that interesting, so I'll just put up a link to the code. Once again, map and reduce are taking the place of list comprehensions, but other than that it's not particularly Lispy, no macrology or anything. Maybe next time.

Last time's tangent was a shortcut to hash syntax. This time I'm not happy with deleting items out of the middle of adjustable arrays. The code puts items on the end of an array and deletes them out of the middle, from a particular position.
In Python it's
del clust[lowestpair[0]]
What I have is


(defun truep (x) (declare (ignore x)) t)
(setf clust (delete-if #'truep clust :start (car lowestpair) :end (1+ (car lowestpair))))

Instead of PCI's use of the Python Imaging Library or some Lisp equivalent, I used cl-pdf, which was easy to use to make the simple graphs, lines and text. This is the full hierarchical cluster, then a zoom view.

'Programming Collective Intelligence' in Common Lisp, Chapter 2

2007-09-12T11:35:00.000-07:00

Like many others, I've been reading Toby Segaran's Programming Collective Intelligence. Toby's examples are in Python. Inspired by loucal's posting of code examples in Ruby, I've decided to put up my own Common Lisp examples. These are from Chapter 2, going up to page 15, "Ranking the Critics".

In order to just have the recommendations in the file, I used assoc lists instead of hashes. One place where Python (and Ruby) has it over Lisp is in hash syntax, just critics[person] instead of (gethash person critics) or (cdr (assoc person critics :test #'equalp)). I made a 'critics' function to keep down the verbosity. Is there a good way to change the syntax for hash lookup?

The other big difference is my use of mapcar and reduce everywhere instead of Python's list comprehensions, with the occaisional intersection thrown in.

If the Python source code gets posted, I may try some benchmarks with later examples.


(defparameter *RECOMMENDATIONS* 
  '(
    ("Lisa Rose" . (("Lady in the Water" . 2.5) ("Snakes on a Plane" . 3.5) ("Just My Luck" . 3.0) 
                    ("Superman Returns" . 3.5) ("You, Me and Dupree" . 2.5) ("The Night Listener" . 3.0)))
    ("Gene Seymour" . (("Lady in the Water" . 3.0) ("Snakes on a Plane" . 3.5) ("Just My Luck" . 1.5) 
                       ("Superman Returns" . 5.0) ("The Night Listener" . 3.0) ("You, Me and Dupree" . 3.5)))
    ("Michael Phillips" . (("Lady in the Water" . 2.5) ("Snakes on a Plane" . 3.0) 
                           ("Superman Returns" . 3.5) ("The Night Listener" . 4.0)))
    ("Claudia Puig" . (("Snakes on a Plane" . 3.5) ("Just My Luck" . 3.0) ("The Night Listener" . 4.5) 
                       ("Superman Returns" . 4.0) ("You, Me and Dupree" . 2.5)))
    ("Mick LaSalle" . (("Lady in the Water" . 3.0) ("Snakes on a Plane" . 4.0) ("Just My Luck" . 2.0) 
                       ("Superman Returns" . 3.0) ("The Night Listener" . 3.0) ("You, Me and Dupree" . 2.0)))
    ("Jack Matthews" . (("Lady in the Water" . 3.0) ("Snakes on a Plane" . 4.0) ("The Night Listener" . 3.0) 
                        ("Superman Returns" . 5.0) ("You, Me and Dupree" . 3.5)))
    ("Toby" . (("Snakes on a Plane" . 4.5) ("You, Me and Dupree" . 1.0) 
               ("Superman Returns" . 4.0)))))


(defun critics (reviewer &optional movie)
  (labels ((get-movie (ms m)
             (cdr (assoc m ms :test #'equalp))))
    (let ((movies (cdr (assoc reviewer *RECOMMENDATIONS* :test #'equalp))))
      (if movie (get-movie movies movie) movies))))

(defun similar (person1 person2 distance)
  (let* ((movies1 (critics person1))
         (movies2 (critics person2))
         (common-movies (mapcar #'car (intersection movies1 movies2 
                                                    :test #'(lambda (x y) (equalp (car x) (car y)))))))
    (if (null common-movies)
        nil
        (funcall distance person1 person2 common-movies))))

(defun euclidean-distance (person1 person2 common-movies)
  (let* ((sum-of-squares (reduce #'+ (mapcar 
                                      #'(lambda (cm) 
                                          (expt (- (critics person1 cm) (critics person2 cm)) 2)) 
                                      common-movies)))
         (distance (/ 1 (1+ sum-of-squares))))
    distance))

(defun sim-distance (person1 person2)
  (similar person1 person2 #'euclidean-distance))


(defun pearson-distance (person1 person2 common-movies)
  (let* ((n (length common-movies))
         (scores1 (mapcar #'(lambda (x) (critics person1 x)) common-movies))
         (scores2 (mapcar #'(lambda (x) (critics person2 x)) common-movies))
         (sum1 (reduce #'+ scores1))
         (sum2 (reduce #'+ scores2))
         (sum1-sq (reduce #'+ (mapcar #'(lambda (x) (* x x)) scores1)))
         (sum2-sq (reduce #'+ (mapcar #'(lambda (x) (* x x)) scores2)))
         (psum (reduce #'+ (mapcar #'* scores1 scores2)))
         (num (- psum (/ (* sum1 sum2) n)))
         (den (sqrt (* (- sum1-sq (/ (expt sum1 2) n)) (- sum2-sq (/ (expt sum2 2) n))))))
    (if (zerop den) 0 (/ num den))))

(defun sim-pearson (person1 person2)
  (similar person1 person2 #'pearson-distance))
         
(defun top-matches (person &optional (n 5) (similarity #'sim-pearson))
  (let* ((scores (mapcar #'(lambda (x) (cons (funcall similarity person x) x)) 
                         (remove-if #'(lambda (x) (equalp x person)) (mapcar #'car *RECOMMENDATIONS*))))
         (sorted-scores (sort scores #'> :key #'car))
         (len (length sorted-scores)))
    (if (<= len n)
        sorted-scores
        (butlast sorted-scores (- len n)))))

Redefining it

2007-05-17T21:06:00.000-07:00

"The best way to solve a problem is often to redefine it." - Michael Rabin, via Paul Graham
"I don't like assembly language." - Richard Cook

Last August I got a Lego NXT robot, and of course I wanted to program it in Lisp. I got Frank Klassner's RCXLisp code and the NXT programming manual, and made a start, looking at opcodes and twiddling bits, and jump offsets, but it was, well, opcodes and bits, and I just wanted to make the robot do things, and it was a big hurdle, and so I procrastinated, and did other things. All that munging with assembly wasn't that interesting a problem to me, and when (if?) I got some output, and it didn't work, then I'd have to try to decipher the disassembled code, and act like the NXT VM, and "run" it...

The light bulb went off a couple of weeks ago. Other people have done most of the heavy lifting for me - there were compilers for other languages for the NXT. I didn't have to get from Lisp to assembly, I just had to get from Lisp to the intermediary language, which would catch syntax errors and I could look at my program's output without going crazy. After some looking around I decided to go with NXC, which has a C syntax. The advantage of a C syntax instead of, say, Basic or Lua is that I have a pretty good head start. Since Javascript has a syntax that's close to C, I started with parenscript and am modifying it to turn it from a Lisp->Javascript compiler into a Lisp->NXC compiler.

I've made a few changes, enough to let me compile a test program


(defconstant *MYNUM* 100)

(defun runout (offset multiplier)
(-num-out 0 offset (* *MYNUM* multiplier)))

(defthread (main :primary t) ()
(let ((x 0))
 (setf x 1)
 (runout *LCD_LINE1* x)
 (setf x 2)
 (runout *LCD_LINE2* x)
 (-wait 10000)))

into


#include "NXCDefs.h"

#define MYNUM 100


sub runout(int 
           offset,
           int 
           multiplier) {
  NumOut(0, offset, MYNUM * multiplier);
}

task main() {
  int x = 0;
  x = 1;
  runout(LCD_LINE1, x);
  x = 2;
  runout(LCD_LINE2, x);
  Wait(10000);
}

which produces

I may pretty it up more when I understand parenscript more, but it compiles and runs and it isn't meant to be read by humans.

The next "architectual/philosophical" decision is how closely to follow the original RCXLisp syntax. That would give me some access to more example programs, but the NXT has more capabilities than the RCX, and there's a Google Summer of Code project "NXTLISP" working on a (I'm assuming) a Lisp->NXT VM compiler, so I may wait to see the syntax for that.

Or not. NXC has a rich set of C-style macros for things like turning on multiple sets of motors that's different than how RCXLisp does it, plus I'm thinking about keeping the language more generic so it can target multiple robot C compilers, such as the First VEX.

I'll post more as I go - the next thing to ponder is the best way to handle typing of NXC variables; int, short, long, byte, bool and string. Other robot C compilers may have other types. I may try Common Lisp type declares, or I may put the types in a list with the variable name, with just the variable name meaning "int".

Latest 'Crossing Borders' is about Lisp

2007-02-06T09:52:00.000-08:00

At IBM's Java page, Bruce Tate writes articles about other languages and the features that they have that other languages don't. His latest is on Lisp. The usual intro stuff, but he could have spent a little more time on macros; his example is
(defmacro times_two (x) (* 2 x))
With not much more he could have shown an actual language extension. Hey Bruce, would a 'while' have killed you?

Practical Ocaml - And I had such high hopes

2006-11-10T10:53:00.000-08:00

If you go back to my first posting, you'll see that before Common Lisp the language I used for home programming was OCaml. I still prefer Lisp's uniform syntax and macros, but Ocaml has a soft spot in my heart.

That's why I was looking forward to Apress' "Practical OCaml", which was going to be OCaml's answer to "Pratical Common Lisp", even doing some of the same projects - good for comparison.

Well, reviews are coming in, and the short version is, except for the prose and the code, the book's just fine. For the sarcasm impaired, the amazon reviews are all one star, and even the book's technical reviewer is not thrilled with it.

Some reviewers are even concerned that this will discourage others from writing a good OCaml book or learning OCaml. Thank you, Peter, for doing such a good job on "Pracital Common Lisp".

Parenscript out on its own

2006-09-18T07:58:00.000-07:00

I see that Parenscript has a home of its own. I'm glad it's out on its own - in the past I've tried to grab just what I needed out of UCW, or use KlammerScript, but the docs never quite matched the code.

Stealing it back

2006-06-01T18:59:00.000-07:00

At Artima, Topher Cyll wrote an article called If It's Not Nailed Down, Steal It where he implements pattern matching, S-expressions and destructuring in Ruby, then uses this to implement a modified Logo interpreter (it uses "(" and ")" instead of "[" and "]"). The Logo interpreter outputs SVG files for rendering.

The interpreter interested me, and hey, Lisp already has the stuff he's stealing, so I thought I'd have a go at implementing the interpreter. I wanted to stick to "vanilla Common Lisp" as much as possible, in other words not pull in big pattern matching libraries, just stick to CLOS multiple dispatch.

Most of the "stealing back" is pretty straightforward, except for the "run" methods. It looks like the Ruby code can distinguish methods by variable arguments, where Lisp methods implementing a generic function must all have the same argument signature, even if the last argument is a &rest. Fortunately, the Ruby methods just distinguish by the command argument, and the rest is destructuring, so once we're in the dispatched run method we can destructure afterwards.

The next bump was chopping off the "command" argument from the remainder of the S-expression to call the next run, which left a single argument for the &rest, the argument being a list of the remaining arguments, so that had to be car'ed, and you had to remember to do it everywhere, etc. It's messy.

"apply" to the rescue. From the CLHS, "apply" uses a "spreadable argument list designator", which means it will turn (1 2 (3 4 5)) into (1 2 3 4 5) when it gets back to dispatching on the method. So

(apply #'run lg '(right 90 forward 10))

turns into

(run lg right 90 forward 10)

which will match on

(defmethod run ((lg logo) (command (eql 'right)) &rest args)

and "args" will have (90 forward 10) in it, ready for destructuring.

One feature of the Ruby version that I can't figure out how to duplicate is dispatching on nil, in other words when we're at the end of the program S-expression. It looks like the methods can catch it, but calling "run" or "apply #'run" won't throw it. What I came up with was inserting a special "stop" token at the end of the programs and stopping on it, or checking at the end of each method for an empty remainder before calling run again. I opted for the empty check. If there's a better solution I'd appreciate a comment to let me know.

This gives me methods such as


(defmethod run ((lg logo) (command (eql 'forward)) &rest args)
  (destructuring-bind (distance &rest rest-args) args
    (move lg distance)
    (when rest-args (apply #'run lg rest-args))))

(defmethod run ((lg logo) (command (eql 'pendown)) &rest args)
  (setf (pen lg) t)
  (when args (apply #'run lg args)))

(defmethod run ((lg logo) (command (eql 'repeat)) &rest args)
  (destructuring-bind (repeat-count code &rest rest-args) args
    (dotimes (i repeat-count) (apply #'run lg code))
    (when rest-args (apply #'run lg rest-args))))

This works, but looking at the code you can see a lot of boilerplate code. Like any Lisper, to me this code smelled like it needed a macro to make an internal DSL to help out with the external DSL. So with


(defmacro defrun ((logo-var command-symbol vars) &body body)
  (let ((args (gensym))
        (rest-args (gensym)))
    `(defmethod run ((,logo-var logo) (command (eql ',command-symbol)) &rest ,args)
       (destructuring-bind (,@vars &rest ,rest-args) ,args
         ,@body
       (when ,rest-args (apply #'run ,logo-var ,rest-args))))))

the above methods turn into


(defrun (lg forward (distance)) (move lg distance))
(defrun (lg pendown ()) (setf (pen lg) t))
(defrun (lg repeat (repeat-count code)) (dotimes (i repeat-count) (apply #'run lg code)))

which is closer to the Ruby DSL.

logo-steal-when.lisp has the pre-macro version, and logo-steal-macro.lisp has the defrun'ned version. Happy stealing.

Am I supposed to like Smalltalk?

2006-05-16T10:51:00.000-07:00

When people talk in general about "high level, extensible, dynamic" languages on their blogs, they usually mention Smalltalk and Lisp in the same breath. Smalltalk programmers talk about their language in the same glowing terms as Lisp programmers; this prompts me to read about Smalltalk, and this weekend I tried out Squeak.

I think I'll stay with Lisp. I'm sure someone will say I didn't try it long enough, but there were a couple of issues I don't think will go away with extended use.

First of all is the environment/IDE. With Lisp/Emacs/Slime, or for that matter Java/Netbeans, to write method after method, you can just type. What I saw in Squeak was a bad combination of mousing and typing. You have to click in the '--all--' window of methods, then mouse down to the editing pane, and replace text, then save, then do it again for the next method. I'm sure this is a YMMV thing, but these mini-interruptions sure wouldn't keep me in 'flow' as much as just typing.

Second is the 'OOP all the way' flavor of the Smalltalk language. Sometimes a function is just a function, and doesn't need to be in a class. I prefer the options that Lisp provides, I use imperative, OO, and functional, depending on the problem.

All this is without bringing up macros, which I don't use that much now, but will do so more in the future, which will only widen the gap.

Have you tried Smalltalk, and why do you prefer Lisp?

Herds of Lispers?

2006-04-18T20:20:00.000-07:00

Ever VNC into a computer that has a VNC window open to the computer that you're on? That's how I feel when I link to a blog that's linked to mine.

About twice a week I search for 'lisp' on technorati, and it's still a little surprise for me when I see someone else writing about one of my articles.

Bob Congdon's entry is mainly a discussion about my entry about Java vs. Lisp language expansion, but at the end he talks about the likely number of Lisp programmers on one project - he himself worked on a project with five. I can't imagine much more than that, both because you wouldn't need that many and we seem to be so few and far between I'm not sure how you'd get that many together.

What's the biggest number of Lisp programmers you've seen on a project?

Let's scream

2006-03-01T05:45:00.000-08:00

I'd read about Screamer so by the time I'd heard of Sudoku I thought that this was something that just "screamed" for Screamer. Plus, when you tell your friends that you're working on a Lisp/AJAX RSS reader as a kind of laboratory for exploring ideas about natural language processing, you get a blank look. Tell them you're writing a Sudoku solver and they understand that.

Screamer has two parts. The first part is backtracking/search. You can write your program somewhat normally, but where your searching ends you put a "fail" call. Where your code doesn't "fail" is where your answers get collected. Brian Kuhn posted an "interview question" where you have a sequence of numbers, positive and negative, and you return the subsequence that would give the highest total. He had 100 lines of Java; with Screamer's help I had


(screamer:define-screamer-package :maxsub 
  (:export :findmaxsub))

(in-package :maxsub)

(defun findmaxsub (numbers)
  (let ((total -99999))
    (car 
     (nreverse  
      (all-values 
 (let ((begin (an-integer-between 0 (1- (length numbers))))
       (end (an-integer-between 1 (length numbers))))
   (when (>= begin end) (fail))
   (let ((subtotal (apply #'+ (subseq numbers begin end))))
     (if (> total subtotal)
  (fail)
  (setf total subtotal)))
   (subseq numbers begin end)))))))

Every time the subtotal is higher than the previous total there's a "success", and the sublist is collected by the "all-values" call. When it's done, the highest of them all is at the end of that list.

The other part is the "constraints" part, where you list the constraints on your variables, wind up Screamer, and let it go. The main problem I found with Screamer is that it doesn't handle arrays well, it does lists better, and individual variables best of all. This turned Sudoku into an exercise in typing. I had to list out 81 variables - mostly are "something between 1 and 9", some are the starting numbers - and 27 constraints, 9 row, 9 column and 9 box.

You can see the code here. If you have a Gmail account, you should get your Google page to host files if you don't want your own domain.

I've also wanted to learn LTK, so I wrote a separate LTK front-end for Sudoku. It uses LTK as well as the LTK "mega-widgets" in ltk-mw.lisp. Inside it uses a Sudoku solver that takes a 2D array as input, 0's for blanks, and returns a 2D array. Code is here. If you want to try this out as is you'll need to get Screamer loaded.

Something funny in the license for Screamer. Part of the conditions of its use is you have to send an e-mail telling them what version of Common Lisp you're using it with, so they can list it. I'll be adding OpenMCL 1.0 under Mac OSX and SBCL 0.9.0 under Linux. If you try this out then let them know.

Franz bustin' out all over

2006-02-06T08:32:00.000-08:00

In my latest Dr. Dobb's Journal, Franz had a two-page ad talking about what's possible and fast with AllegroCache.

Peter Coffee's latest column at eWeek, 'Exotic' Programming Tools Go Mainstream, has this to say:

Allegro CL has had two major updates since the last time eWEEK Labs reviewed it. The 8.0 version reflects performance-enhancing improvements so startling that if it played baseball, we'd expect a congressional probe.

Can't wait to get my hands on the upcoming Express Edition. I knew Lisp was what I wanted to learn so I was willing to get over the Emacs/Slime curve, but I understand people who'd want to explore with an IDE to see if Lisp is for them, and it would be good to be able to recommend a hassle-free Windows IDE.

Ever closer to Greenspun's Tenth

2006-01-17T11:19:00.000-08:00

Anyone who's glanced at LML's should find Markup for Ruby familiar:


 html do
   head do
     title action_name
     stylesheet_link_tag 'scaffold'
   end

   body do
     p flash[:notice], :style => "color: green" 
     self << @content_for_layout
   end
 end

Fits like a glove

2006-01-13T07:00:00.000-08:00

I've had a dry spell of coding at home, between the holidays and reading "Ajax in Action", but reading some of the long postings at Planet Lisp should be enough to guilt any slacker back into action.

This has uncovered yet another advantage of Common Lisp, at least for me; how short a time it takes to get back up to speed with the language after being away from it. It takes me considerably less time to get back "in the groove" than with other languages, with less references to language manuals. I don't know how this is for others, but Lisp fits my brain.

I think this is because of the lack of syntax. Instead of having to remember operator precedence, the operators themselves, strings of characters that look like Yosemite Sam swearing, it's just almost-English words and parentheses. The language is built on just a few core concepts that can be mixed and matched, instead of a bunch of rigid rules and details that have to be remembered.

New Year's Resolution

2005-12-18T19:03:00.000-08:00

Regular backups.
Also, thank Apple for putting 'terminal' on the install disk.
Thank you. That is all.

Portable Allegroserve as a de facto standard?

2005-12-08T09:31:00.000-08:00

The "Ruby as an acceptable Lisp" and Reddit discussions (I don't really need to supply links, do I?) brought back to the surface a question I ask myself often.

It's true that Common Lisp doesn't have multithreading or network clients in the standard, and people want that, but what's wrong with Portable Allegroserve? The acl-compat component has those covered (if the underlying implementation has support) for Allegro, LispWorks, CMUCL, SBCL, OpenMCL, SCL, Corman, CLisp. It appears to be used a lot, and has decent documentation.

Why not just use that and move on to the next problem? Is there something horrible about Portable Allegroserve I should know about?

Or....

2005-11-24T18:22:00.000-08:00

I received an email from JavaLobby, which had in it this article at EclipseZone, which talks about the plusses and minuses of adding features to Java. From the article:

Language guys love to tweak. I know, because I'm a language guy myself. I used to work on compilers, code generators, libraries, and debuggers. We were always adding new stuff, new commands, special keywords, etc. Did our users thank us for it? Maybe the one or two that wanted the new things, but the vast majority would just groan when a new release came out. What will this break now? Will I have to upgrade? Will I have to use somebody else's code that requires this new thing before I'm ready? Has it been ported to all the machines I need to run on? Boring, I know, but very practical and important issues.

Or... you can work in a language where if you want the one or two new things, you can just add them, and let the other guy that wants one or two other things added to the language add what he wants.

Or... I could just let Peter Siebel say it better:

DOLIST is similar to Perl's foreach or Python's for. Java added a similar kind of loop construct with the "enhanced" for loop in Java 1.5, as part of JSR-201. Notice what a difference macros make. A Lisp programmer who notices a common pattern in their code can write a macro to give themselves a source-level abstraction of that pattern. A Java programmer who notices the same pattern has to convince Sun that this particular abstraction is worth adding to the language. Then Sun has to publish a JSR and convene an industry-wide "expert group" to hash everything out. That process--according to Sun--takes an average of 18 months. After that, the compiler writers all have to go upgrade their compilers to support the new feature. And even once the Java programmer's favorite compiler supports the new version of Java, they probably still can't use the new feature until they're allowed to break source compatibility with older versions of Java. So an annoyance that Common Lisp programmers can resolve for themselves within five minutes plagues Java programmers for years.

Lisp with a different syntax

2005-11-22T11:00:00.000-08:00

I've been reading Ajax in Action, and have been reminded of the quote "Javascript is Lisp with a different syntax". It's usually used to stress the functional and closure features of Javascript, but there's another part that's also very close - attaching arbritrary properties to an object, including functions.

Compare


>var x = new Object()
>x.one = 1
1
>x.two = 2
2
>x.doubler = function(n) { return n * 2; }
function (n) { return n * 2; }
>x.two
2
>x.doubler(3)
6

with


CL-USER> (setf (get 'x 'one) 1)
1
CL-USER> (setf (get 'x 'two) 2)
2
CL-USER> (setf (get 'x 'doubler) #'(lambda (x) (* x 2)))
#<CLOSURE :LAMBDA (X) (* X 2)>
CL-USER> (get 'x 'two)
2
CL-USER> (funcall (get 'x 'doubler) 3)
6

In old Lisp textbooks I've seen this referred to as "data driven programming", used before anyone was talking about object-oriented programming. Now, no one would do OO this way in Lisp, but the quote didn't say "Javascript is Common Lisp with a different syntax".

Looking for Lisp/NLP starting points

2005-11-17T07:44:00.000-08:00

I guess I'm having trouble with the correct incantations into Google. I'm trying to find information on natural language processing, preferably in Lisp but I'll look at anything, specifically towards taking in text and summarizing it.

I'm thinking of doing an RSS reader where long-winded feeds can be automatically summarized down to "abstract" form, and including a link back to the original article that the user can decide to read or just move on to the next article.

Any suggestions for starting points would be appreciated.

A Cocoa-Lisp-Google Maps Mashup

2005-11-05T20:11:00.000-08:00

Just your run-of-the-mill Google Maps page, except for a couple of things; the server part is in Portable AllegroServe (no big deal), and the addresses came out of my Mac's Address Book via OpenMCL (a bigger deal).
I got interested in this after hearing an O'Reilly "Distributing the Future" podcast where someone mentioned that not everyone will want all their information on the web and there will be applications where part of the data is on the web and part is kept in the user's local storage.
The basic understanding of the Mac's AddressBook library came from this Mac Dev Center article, and help from Gary Byers got OpenMCL loading and searching the Address Book in OpenMCL. The rest is basic Portable AllegroServe.
The geocoder I'm using is Ontok, which has a REST api that takes up to 10 addresses at one time and returns CSVs that start with latitute and longitude co-ordinates.
Wil Shipley mentions that instead of using a database for storing customer orders he keeps them in XML files and uses Spotlight for searching. Something similar could be done with Address Book, just keep addresses in the built-in library instead of a database.
Code is here if you'd like to see.

Joel is influential, we must assimilate him

2005-10-25T10:17:00.000-07:00

Bill pointed out Joel Spolsky saying
"I had to install Lisp in a Box and start working through Seibel's new book on Common Lisp until my brain started functioning again."
What I find interesting is that the number of links to Lisp In A Box on Delicious' lisp tag has really shot up the past couple of days. Expect a newbie mini-flood on c.l.l.

Common Lisp and/or Scheme, and Y I should bother

2005-10-10T05:00:00.000-07:00

First things first. If there's a Y pun, then you know I must be talking about the Y combinator. Understanding it is on the computer science part of the ol' "Things to do before I die" list. To that end, I got and read "The Little Schemer", and had the same reaction I usually get when reading about the Y combinator, or continuations (another on my "to understand" list). It starts out very simple (cons, car cdr), adds a baby step, another baby step, etc. then they're off on this bizarro explanation that makes no sense to me. Instead of steps A, B, C, D, E, F it's more like A, B, C, W, X, Y; there were some important steps in between that I somehow missed. Is this something I need to eventually conquer to achieve enlightenment, or are they happy, productive Lispers out there who don't understand this?

A related question is Common Lisp and Scheme, specifically PLT Scheme. I've looked at both and prefer Common Lisp to get work done, using emacs and SLIME. Every now and then I look at PLT Scheme, mainly out of IDE envy, and it looks nice. Bill Clementson says it's the best Open Source Lisp. Common Lisp vs. PLT Scheme reminds me of the "even though Python is a better language, Java + IDE combos make you more productive" blog entries I see.

Everyone says the best tool for the job, but for Lisp users I'm thinking that's usually either Common Lisp or Scheme, and other non-Lisp languages. Do Lisp users just stick with one dialect of Lisp, or are there people that use Common Lisp and Scheme for different tasks?

An analogy - Lisp as Graffiti

2005-09-15T18:51:00.000-07:00

The Palm kind, that is.

Whining about Lisp's syntax is a favorite sport on the blogs, especially among students that have a Lisp assignment in a class. I look at it the same way Jeff Hawkins looked at Grafitti when developing the Palm (from a Fast Company article):

"People are smarter than appliances," Hawkins told his colleagues. "They can learn.'' He recalled a lesson from his Berkeley days: "People like learning. People can learn to work with tools. Computers are tools. People like to learn how to use things that work."

It's good to make things easy for the user, but there comes a point where if the user can compromise a little in one area they can gain a lot in others. For the original Palm, learning Graffiti meant the PDA could fit in your pocket, be cheap and run on two AAA's since the software could be simplified enough to fit in the restrictions of the machine. For Lisp, making a small change, going to prefix notation and "all those parentheses" gains you code as data as code, macros that match the syntax of the language, and everything else you've been reading about.

If you're a whiny student who doesn't want to believe me, try believing Robert Floyd, from his Turing Prize lecture (from here via here):

“Contact with the programming written under alien conventions may help. Visiting MIT on sabbatical this year, I have seen numerous examples of the programming power which Lisp programmers obtain from having a single data structure, which is also used as a uniform syntactic structure for all the functions and operations which appear in programs, with the capability to manipulate programs as data. Although my own previous enthusiasm has been for syntactically rich languages like the Algol family, I see now clearly and concretely the force of Minsky’s 1970 Turing Lecture, in which he argued that Lisp’s uniformity of structure and power of self-reference gave the programmer capabilities whose content was well worth the sacrifice of visual form.”

Fuzzy Queries on top of CLSQL

2005-09-04T17:18:00.000-07:00

I originally planned to incorporate Fuzzy SQL more into CLSQL, but I don't think it's necessary for my purposes. With CLSQL's def-view-class extension of CLOS I can use generic methods, and it will look like the rest of the class. Later in the book I'm using, Fuzzy Modeling and Genetic Algorithms for Data Mining and Exploration there's a chapter on Genetic Tuning of Fuzzy Models, which I think will work better with external functions and methods as opposed to MOP or macro-based solutions inside CLSQL. Besides, I couldn't even get db-reader to work with my classes, so I'll just leave well enough alone.

This will be based on my previous big and tall entry. My data setup is


rrc=# select * from btprospects;
   name   | height | weight | age 
----------+--------+--------+-----
 Saunders |     74 |    215 |  52
 Cassey   |     73 |    188 |  40
 Miller   |     71 |    157 |  25
 Freeman  |     70 |    202 |  34
 OMalley  |     65 |    163 |  48
 Jackson  |     63 |    170 |  38

and I have a corresponding CLSQL view-class


(clsql:def-view-class btprospects ()
  ((name :db-kind :key
         :type (string 20)
         :initarg :name
         :accessor name)
   (height :type integer
           :initarg :height
           :accessor height)
   (weight :type integer
           :initarg :weight
           :accessor weight)
   (age :type integer
        :initarg :age
        :accessor age)))

The key is each class will have a qcix (query compatibility index) method that will be the overall membership function, doing some combination of individual membership functions of individual columns in the database. For our "big and tall" prospects we'll have a tall method for membership based on height, and a heavy method for membership based on weight. These functions will return a value from 0.0 to 1.0, with a linear scale from the 0 cutoff to the 1 cutoff. For overall membership we'll average the two other membership function.


(defmethod tall ((prospect btprospects))
  (let ((raw-height (slot-value prospect 'height)))
    (cond ((<= raw-height 54) 0.0)
          ((>= raw-height 72) 1.0)
          (t (float (/ (- raw-height 54) (- 72 54)))))))

(defmethod heavy ((prospect btprospects))
  (let ((raw-weight (slot-value prospect 'weight)))
    (cond ((<= raw-weight 182) 0.0)
          ((>= raw-weight 220) 1.0)
          (t (float (/ (- raw-weight 182) (- 220 182)))))))

(defmethod qcix ((prospect btprospects))
  (float (/ (+ (tall prospect) (heavy prospect)) 2)))

CL-USER> (clsql:map-query 'list #'tall [clsql:select 'btprospects])
(1.0 1.0 0.9444444 0.8888889 0.6111111 0.5)
CL-USER> (clsql:map-query 'list #'heavy [clsql:select 'btprospects])
(0.8684211 0.15789473 0.0 0.5263158 0.0 0.0)
CL-USER> (clsql:map-query 'list #'qcix [clsql:select 'btprospects])
(0.93421054 0.57894737 0.4722222 0.7076024 0.30555555 0.25)

So far, so good. Now we want to select only some records that have a qcix over a particular threshold. We can use CLSQL's addition to loop:


(defun fuzzy-select (class-name cutoff)
  (loop for (rec) being the records in (clsql:select class-name)
    when (> (qcix rec) cutoff) collect rec))

CL-USER> (fuzzy-select 'btprospects 0.5)
; Warning: database-query-result-set not implemented for database type POSTGRESQL-SOCKET.
; While executing: #

Oops! Perhaps another reason not to delve into the innards of CLSQL. Plan B:


(defun fuzzy-select (class-name cutoff)
  (let ((result '()))
    (clsql:do-query ((rec) [clsql:select class-name])
      (when (>= (qcix rec) cutoff)
        (push rec result)))
    result))

CL-USER> (mapcar #'name (fuzzy-select 'btprospects .5))
("Freeman" "Cassey" "Saunders")

So, starting with a CLSQL def-view-class we can externally add generic methods to support fuzzy queries.

This wasn't even funny the first time (search friendly languages)

2005-09-01T09:15:00.000-07:00

Occaisionally I want to see what other bloggers are doing with Lisp, so I use Technorati to search on Lisp. Every time, over the course of months now, I see at least once or twice a search page "Whose idea was it to put an "S" in the word "lisp"? ". And of course each time they act like they were the first to hear of it.

Never mind the AI Winter reasons, Lisp, Scheme and Arc need better names for searching purposes. This kind of thing never happens to Ocaml or Nemerle.

September British LinuxUser reviews LispWorks Personal

2005-08-28T18:17:00.000-07:00

I had a look at it in the bookstore, didn't buy it. Overall favorable, although they didn't like CAPI built on just Motif, they mentioned it uses Aqua under Mac. Nothing the Planet Lispers don't already know; I just mention it because I think it's interesting that a Common Lisp environment is mentioned in a more "mainstream" computer magazine.

A Spotlight Lisp plugin in a day, or I never metadata I never liked

2005-08-21T19:25:00.000-07:00

While planning my approach into CLSQL to add Fuzzy SQL, I found myself wanting to just search for keywords, Spotlight-style. Well, how hard can it be? Not that hard, as it turns out. I basically just want to throw the text of a file to Spotlight and have it index the contents. Even if you have no Objective C or Cocoa experience like me, copying and modifying code is pretty straightforward.

I started with this article on Creating Spotlight Plugins. In the info.plist, I substituted his type of "com.apple.stickies.aaa" with "com.lisp.lisp", and "lisp" as the file extension. In the schema.xml, I made a custom metata for lisp files called com_lisp_lisp_data.

The XCode project makes a file called 'GetMetaDataForFile.c', which gets renamed to 'GetMetaDataForFile.m' since we use Objective C and Cocoa. The meat of this file is a callback function called 'GetMetaDataForFile'. All I do in this function is read the entire file into a NSString and add it to the 'mutable dictionary'.

Here's the function:


Boolean GetMetadataForFile(
         void* thisInterface, 
         CFMutableDictionaryRef attributes, 
         CFStringRef contentTypeUTI,
         CFStringRef pathToFile)
{
 Boolean success=NO;
    NSAutoreleasePool *pool;
 
 // Don't assume that there is an autorelease pool 
 //around the calling of this function.
    pool = [[NSAutoreleasePool alloc] init];
 
    /* Pull any available metadata from the file */

 NSString *fileText = [NSString stringWithContentsOfFile:pathToFile];
 
 [(NSMutableDictionary *)attributes setObject:fileText
  forKey:@"com_lisp_lisp_data"];
 
 // return YES so that the attributes are imported and
 // add more resilient error handling at your own discretion
 success=YES;
 
 //memory management
 [pool release];
 
 return success;
}

Build this in XCode, then:


My-Computer:~/LispImporter/build rrc$ sudo cp -r LispImporter.mdimporter /Library/Spotlight

We start with a small lisp file:

My-Computer:~/LispImporter/build rrc$ cat ~/small.lisp
;; A small lisp file
(format t "This is a small lisp file")

Index this file:

My-Computer:~/LispImporter/build rrc$ mdimport -d1 ~/small.lisp
2005-08-21 20:41:05.624 mdimport[630] Import '/Users/rrc/small.lisp' type 'com.lisp.lisp' using 'file://localhost/Library/Spotlight/LispImporter.mdimporter/'

See what Spotlight thinks about this file. Note the 'com_lisp_lisp_data' attribute:

My-Computer:~/LispImporter/build rrc$ mdls ~/small.lisp
/Users/rrc/small.lisp -------------
com_lisp_lisp_data             = ";; A small lisp file (format t "This is a small lisp file") "
kMDItemAttributeChangeDate     = 2005-08-21 20:41:05 -0400
kMDItemContentCreationDate     = 2005-08-21 20:37:33 -0400
kMDItemContentModificationDate = 2005-08-21 20:37:33 -0400
kMDItemContentType             = "com.lisp.lisp"
kMDItemContentTypeTree         = ("com.lisp.lisp", "public.data", "public.item", "public.content")
kMDItemDisplayName             = "small.lisp"
kMDItemFSContentChangeDate     = 2005-08-21 20:37:33 -0400
kMDItemFSCreationDate          = 2005-08-21 20:37:33 -0400
kMDItemFSCreatorCode           = 0
kMDItemFSFinderFlags           = 0
kMDItemFSInvisible             = 0
kMDItemFSLabel                 = 0
kMDItemFSName                  = "small.lisp"
kMDItemFSNodeCount             = 0
kMDItemFSOwnerGroupID          = 501
kMDItemFSOwnerUserID           = 501
kMDItemFSSize                  = 60
kMDItemFSTypeCode              = 0
kMDItemID                      = 1604367
kMDItemKind                    = "Document"
kMDItemLastUsedDate            = 2005-08-21 20:37:33 -0400
kMDItemUsedDates               = (2005-08-21 20:37:33 -0400)

Great! Now tell Spotlight to do all the lisp files:

My-Computer:~/LispImporter/build rrc$ mdimport -r /Library/Spotlight/LispImporter.mdimporter
2005-08-21 20:43:47.750 mdimport[633] Asking server to reimport files with UTIs: ("dyn.ah62d4rv4gk8y2wnxna", "dyn.ah62d4rv4ge8024pxsa", "com.lisp.lisp")

Go away for a while...
Then give it a try.

Refresh, renew, refill the Kool-Aid

2005-08-14T19:22:00.000-07:00

My wife and I went on a vacation to the mountains of Highlands, NC. The place had no Internet access, so I thought I'd take along "Hackers and Painters" to reread and get re-inspired. I have drunk the Paul Graham Kool-Aid; I got a Mac to have an easy-to-use Unix to work on Lisps, I suggest his essays to friends, yada yada.

What Paul probably got real right - page 56, where he starts talking about web apps taking over most of software. This was before Google showing us the AJAX way.

What Paul may have gotten wrong - page 164, "parallelism will be something that is available if you ask for it explicitly, but ordinarily not used". Lately we've been hearing that we're hitting the limits on a particular core, mulitcore chips are going to be more common and we'll have to write our software for it.

The big open question I came away with was Paul talking about "Good design comes in chunks". He compares 15th century Florence (mentions 10 famous artists from there and then) vs 15th century Milan (mentions 0).

"Nothing is more powerful than a community of talented people working on related problems". Can virtual communities serve this purpose today? I can't move to this era's "15th century Florence", even if I could find it. But would I need to? Do I just need to keep up with Planet Lisp or comp.lang.lisp or common-lisp.net? Is is something broader, like Ruby On Rails/Trails/Lisp on Lines communities? It would be pretty damn cool if anyone could just surf to "a community of talented people working on related problems". All we need to do is find it, or define it ourselves.

P.S. Paul - If you're reading this, RELEASE ARC ALREADY. Thank you.

Holy Crap! or, I didn't know you could do that

2005-08-07T20:08:00.000-07:00

As a lower priority project I've been looking for a "universal UI" for any Lisp projects I may do. I've been restricting it to Web clients backed by Lisp on the server, no unusual installs, etc. I also want to keep it to something that would work with most browsers and OS's today (ruling out Safari's canvas tag, SVG, etc.) but that would allow some graphics, probably charts.

I was originally thinking of Laszlo, which with version 3.0 you can take your Laszlo client and compile it into Flash. The standalone Flash app can send requests back to the server, get data in the form of XML and use XPath strings to get to the data.

Then I found this article on JavaScript Graphics.

This is just browser, no Flash, Java or plugins. Basically, you can use pure JavaScript, DOM and CSS to make graphcs in browsers. All the tricks work with standards compliant browsers, and some work with IE. I've tried them with Safari and Firefox.

I was going to go Laszlo, but I won't mind going back to the drawing board for this.

Fuzzy Logic and Fuzzy SQL, or Richard's Big & Tall Shop

2005-07-29T20:44:00.000-07:00

These explanations are based on my readings so far in Earl Cox's Fuzzy Modeling and Genetic Algorithms for Data Mining and Exploration

Fuzzy Logic, or Are you tall?

A more complete explanation can be found here, but here's a short version. Fuzzy Logic is a superset of Boolean Logic. In Boolean Logic, you're either in a set or not, and this can be represented by a membership function that returns either 0 or 1. In Fuzzy Logic, an item's membership in a set is measured in a continuous number from 0 to 1, determined by a membership function.

As an example, let's talk about "tall". A boolean membership function would be something like "if you're six feet or over, you're tall, else you're not". 0 or 1. A fuzzy membership function could be "if you're 5 feet or under, your tallness is 0. If you're 6 feet or over, your tallness is 1. From 5 feet to 6 feet it's a continuous slope". So, if someone is 5'9" their membership in the "tallness" set is .75.

In this logic system, a fuzzy AND of two numbers is the minimum of the numbers, a fuzzy OR is the maximum of the numbers, and a fuzzy NOT is 1 - a number. If you think of booleans as either 0 or 1, these operators are the same. Fuzzy is a superset of boolean.

Fuzzy SQL, or are you Tall and Heavy?

Let's say I'm opening a new chain of Big & Tall stores, and let's say I bought a database or broke into some insurer and have a huge list of names, addresses, heights and weights. I want to determine who in the database it would be worth my while to send a direct mailing. In regular SQL a query would look something like

select name, address from prospects where height => 74 and weight => 210;

That would give me a subset to work with, but what about someone who's 72 inches and 240? He's probably a pretty good prospect, but my boolean filtering would miss him. I could play with the cutoffs, but there would always be someone just outside, and if I move them too far I increase my chances of adding people who wouldn't be interested, and exclude the tall but not too heavy or heavy but not too tall.

Fuzzy SQL is a way of managing this problem. A Fuzzy SQL statement that we would use would look like

select name, address from prospects where height is tall and weight is heavy;

Looks kind of meaningless. What's "tall and heavy"? Well, first you start by computing individual membership numbers for the "tall" set and the "heavy" set using membership functions I wrote about above. Then with these individual numbers you compute what Cox calls a "Query Compatibility Index" (QCIX), which are different ways of combining the individual numbers. One way is the classic fuzzy AND, the minimum of the numbers. Another way is to average the numbers, perhaps with a weighted average, if for example I thought heavy was more important than tall. The Fuzzy SQL query should also return extra columns, such as the QCIX and perhaps the individual membership numbers calculated by the membership functions.

Sticking with a regular average would be OK for my direct mailing purposes. Tall and heavy would give a high QCIX, but also would very tall but not that heavy, and vice versa. So, the query is run and the rows sorted according to the QCIX, and we give a cutoff value for the QCIX below which rows are not returned. Print them envelopes, throw in a coupon and we're good to go.

The source code in the book is in Visual Basic, but what's the fun in that? What we really need is language that can add extra keywords such as the "is" and make it look like the rest of the language, that can take symbols like "tall" and "heavy" and associate them with membership functions....

It's OK if you want to do this one

2005-07-25T20:01:00.000-07:00

I've been working with ActiveMQ at work. In addition to being the Java Messaging System in Apache's Geronimo, it's also going to have open-source C libraries, and support from the financial community (See Here). What inspired me to write it that is also has REST API. This means that a fairly simple Lisp library could be whipped up to communicate with J2EE app servers, C applications, etc. I may get around to this eventually, but if someone else wants to have a crack at this that's OK with me.

CLHS & CLTL in Compiled Help format

2005-07-21T20:32:00.000-07:00

clhs-win
Originally uploaded by ralphrichardcook.

I've found where someone has put the CLHS and CLTL in WIndows Compiled Help format. What's so nice about this, at least on Windows, it that's much easier to search than navigating the links in a browser. It works great in Windows, pictured here, but the mac viewer doesn't do indexes or searches. I'll look for an X viewer and see if I can compile it on the mac.
The CHM files can be found here, scroll to the bottom for URL's.

Smug Lisp Weenies unite...or at least talk to each other

2005-07-18T19:01:00.000-07:00

For a while I've been hearing about Ruby on Rails, and how Ruby's metaprogramming makes it possible to do it with so few lines of code. Well, we all know that Lisp invented metaprogramming, so I was thinking of working on a Lisp version of Rails. Then I see this. Someone who's worked on it for months, come up with a better version than I could, and is giving demos on it.

Time for Plan B.

I'm reading Earl Cox's "Fuzzy Modeling and Genetic Algorithms for Data Mining and Exploration". Upcoming chapters use something he calls "Fuzzy SQL" to model fuzzy sets, and genetic algorithms for tuning. I'm thinking of modifying or extending CLSQL to incorporate the Fuzzy SQL concepts, and look at John Koza's genetic programming for tuning in Lisp.

So, I'm asking, IS ANYONE ELSE DOING THIS? Am I going to start working on this, only to find a blog entry one day where someone's been working on this for months, come up with a better version than I could, and is giving demos on it?

If so, I'd appreciate it if you'd drop a comment here.

I think Pesty is a Lisp programmer

2005-07-18T18:52:00.000-07:00

Optimization conundrums

2005-07-18T18:48:00.000-07:00

I've been programming professionally for 20 years, and programming Java professionally for about five. Something I think I have in Java is a good feel for optimization and performance, at least on the micro level. What code is going to cause a lot of object creation, what choice of libraries is the best for a certain set of circumstances, etc.

Let's say I don't have that in Lisp yet. I worked on a benchmark for the Computer Language Shootout, and my best optimizations with declarations weren't doing so well. I asked for help on Usenet, Nicolas Neuss had a look, and, well, I'll just show you.

Here's my code (testrrc package) and Nicolas' (testnn package) combined for one of the routines.

This is compiled and run in CMUCL on my 1 Ghz iBook.

(proclaim '(optimize (speed 3) (safety 0) (debug 0) (compilation-speed 0) (space 0)))

(defpackage :testrrc)
(in-package :testrrc)

(defconstant IM 139968)
(defconstant IA 3877)
(defconstant IC 29573)

(defparameter THE_LAST 42)
(declaim (inline gen_random))

(declaim (inline gen_random))
(defun gen_random (max)
(declare (type (signed-byte 32) IM IA IC THE_LAST))
(declare (double-float max))
(setq THE_LAST (mod (+ (* THE_LAST IA) IC) IM))
(/ (* max THE_LAST) IM))

(defpackage :testnn)
(in-package :testnn)

(defconstant IM 139968)
(defconstant IA 3877)
(defconstant IC 29573)

(defparameter THE_LAST 42)
(declaim (inline gen_random))

(defun gen_random (max)
(declare (type (unsigned-byte 30) IM IA IC THE_LAST))
(declare (double-float max))
(setq THE_LAST (mod
(+ (the (unsigned-byte 31) (* THE_LAST IA)) IC) IM))
(/ (* max THE_LAST) IM))

(in-package :common-lisp-user)

(defun main ()
(time (dotimes (i 7500000) (testrrc::gen_random 1.0d0)))
(Time (dotimes (i 7500000) (testnn::gen_random 1.0d0))))

And here's the results.

; Evaluation took:
; 1.45 seconds of real time
; 1.142364 seconds of user run time
; 0.025509 seconds of system run time
; 1,529,475,015 CPU cycles
; 0 page faults and
; 5,127,264 bytes consed.
;

; Evaluation took:
; 0.28 seconds of real time
; 0.133931 seconds of user run time
; 6.7e-4 seconds of system run time
; 294,265,464 CPU cycles
; 0 page faults and
; 0 bytes consed.

Not only does that small change make it 5 times faster, but I have no idea why my version needs to cons 5 MEGS of bytes, and Nicolas' conses ZERO bytes. Anyone care to explain that one to me?

I wish I'd said that.

2005-05-19T20:19:00.000-07:00

From Marginilia, a quote that sums up my feelings pretty well.

There are still things to learn about Java, too, and I’m sure there are many smart people working on it, but here is the difference: studying Lisp is like peeking behind the curtain and mastering the secrets of the universe, whereas studying Java is like learning how fix your old truck — lots of particulars but not much theory.

So what is a closure?

2005-05-19T20:12:00.000-07:00

For the Java and other imperative language users, here is a good explanation of what closures are. Read the sections titled "We're no longer in Kansas" and "Does a Fence Keep things in or out?".

What brought me to Lisp

2005-05-17T22:07:00.000-07:00

Since the main emphasis of my writings will be about my explorations in and around Lisp, I thought I'd start with how I came to Lisp. This is an abreviated version of what's here.

A few years back I started looking into functional programming, and started with Haskell, a pure functional language. This is fine for learning the basic concepts of functional programming, but imperative is better for some things, mainly I/O. Haskell's monads made my head hurt. Sometimes you just want to print something out or access a database without getting a PhD.

This led to OCaml, which seemed to have most of what I wanted. Functional, but not obsessively so (you can printf when you want to). Good execution speed. Support for objects. I read a lot of literature about functional programming vs. object-oriented programming and came to the conclusion that both have their uses so I wanted a language that had both. I started writing some simple OCaml programs as I was learning the language and I liked the way they came out, no having to declare variable types, first-class functions, etc. Then I got farther along in my studies, learning more about how objects and classes are used in OCaml. I started looking at how they handled the FP vs. OOP tension, namely adding functions to a existing class/module vs. adding classes and subclasses, and they started talking about parameterized classes and abstract modules and the syntax started getting more and more complicated.

In Common Lisp, the FP/OOP tension is handled with multimethods - much simpler. Metaprogramming in Common Lisp is done with defmacro, backquote and comma and is done in place, where in OCaml you need a separate preprocessor and a syntax that's rather different than the syntax of the language itself.

I would say that OCaml and Lisp give you much the same power to do things, but my experience is that while doing the simple stuff may be simpler in OCaml than in Lisp, doing the complicated stuff is simpler in Lisp than in OCaml, and I plan on doing a lot more of the complicated stuff than the simple stuff.

I don't want to expend my brainpower wrestling with my programming language, I want to expend it solving problems, and with Lisp I don't wrestle the language. Although some say Common Lisp is a 'big' language, it's basically just parentheses and keywords.

If you think the prefix arithmetic is hard, well it can be, but how often do you put the Pythagorean theorem in your code, vs. just making a function call with parameters? Is foo(bar) that much easier than (foo bar)?

Oh, and about the parentheses? Get a parenthesis-matching editor, and get over it.