Archive for the ‘Code’ Category

In case you thought Lisp was dead …

Wednesday, March 5th, 2008

Two days ago I went to the largest usergroup meeting of any kind I have ever seen. It was the Boston Lisp Meeting (although it should be called the Cambridge Lisp Meeting, IMO). There were about 40 people in attendance, and when I left (at 22:00, four hours after my arrival), it still rivaled any usergroup in size.

With 40 people, the sitting-around-a-table-drinking-beer format is perhaps not the most effective. Sure, there is plenty of conversation to be a part of, but any single participant necessarily misses the majority of what\’s going on. So, I apologize at the beginning that this can not be a comprehensive report, but rather an experiential summary of what I observed that fateful evening.

I showed up at 18:00, with probably over twenty people bulging from a single long table. Almost everyone who came after me ended up filling a second table. I took note of a few big names right off the bat (from #lisp and elsewhere): Rahul Jain (who traveled from New York) had come with me, and certainly Faré (our great organizer) was already there. Jeremy Jones (a founder of Clozure) and James Knight (foom) were also there before me. Alistair Bridgewater (aka, nyef) trekked down from New Hampshire, but I think Hans Hübner won the longest-distance award, having come all the way from Berlin. Ok, this list is getting crazy … there were also other well-known lispers in attendance. Gary King was supposed to have been there, but I didn\’t see (or perhaps recognize) him. Both Zach Beane and dto had to miss because of transportation issues, which was quite disappointing.

I was afraid the group would be dominated by ITA employees, but I think they made up maybe a quarter of the entire group. None of the guys near me at the beginning were ITA employees, in fact, I think they were all Lisp hobbyists. As a result, there is always plenty of discussion about how to go about making Lisp your profession, which is a topic I do like to talk about. One of my favorite questions was \”If you get to write Lisp at work all day, do you work in other languages when you get home?\” The answer is pretty much \”no\” … at least, I don\’t play with other languages any more than I did before I was writing Lisp professionally.

Later on in the night I moved around the table to see what was happening at Faré\’s end. There seemed to be a lot of discussion about when meetings should be held, and how to organize them, etc., which is great to hear. We\’ll be moving to a more presentation-oriented format, and I\’m sure Faré will have all the details about future meetings sent around soon. There was an XO laptop that got passed around so everyone could enter in their contact information and how they would like to be involved with future mettings. So there ended up being a bit of discussion about the XOs, and who managed to get one (like me) and who didn\’t.

Toward the end of the night, I got around the table to talk with Rahul and another of my co-workers, and had the chance to finally meet Alistair. Unfortunately, I had to leave shortly after we began talking … and he will be farther away than NH for the next few months because of various contracts he\’s working on. Hopefully he\’ll come to meetings when he\’s back in the region.

I know … there\’s not much Lisp content in this post. Honestly, it\’s hard to remember exactly what was discussed. The meeting was great for getting to know more Lispers, and there was plenty of Lisp talk, but my focus at the time was more on having a good time (and some beers) with people that I have a lot in common with. There will be plenty of in-depth Lisp-hackery at future presentation-based meetings. For now I\’m just excited about there being so many people here who are interested in it.

Continuing about restarts

Tuesday, February 26th, 2008

Update: these changes (along with an additional one for unbound slots) are now available in the CCL repo: http://svn.clozure.com/publicsvn/openmcl/trunk/<os><arch>/ccl

When I read Geoff Wozniak’s post (I know, I\’m a week behind) I was disappointed to see Clozure CL wasn’t even taken into account. I decided to check it out for myself. Initially, I was a bit disappointed, but after only a couple minutes of hacking, I managed to improve the situation a bit.

Situation Lisp implementations
Allegro CL Lispworks SBCL CLISP Clozure CL
No function defined -
Failed function lookup - - -
No class found - -
Division by zero - - - -
No method found -
No slot found - - -
Replace function with generic function -
Redefine a generic function - - 1

1 The existing code handled a defmethod followed by a defmethod with an incompatible lambda-list, but not a defmethod followed by a defgeneric with an incompatible lambda-list.

Initially (the black) we could claim superiority only to SBCL [Ed: just kidding, guys], but after my changes (in red) we’re now on par with Allegro. And Open Source. The squeaky wheel and all that. Thanks to Geoff for pointing out the issue.

Note: these changes currently exist only on my own box. I\’ll get them into CCL 1.2, though.

Boston Lisp Meeting

Monday, February 25th, 2008

It\’s been about four months since I last posted, which is really unacceptable. I feel like I have to jump on the bandwagon with this announcement, though, and maybe it\’ll get me back on track.

Next Monday (3 March 2008), is the inaugural Boston Lisp Meeting. Judging by talk around the office and in the community at large, it sounds like it\’ll be well-attended. If you know any lispers, send them along. If you want to get to know some, come along yourself.

automated testing with CL & darcs

Sunday, October 21st, 2007

I spent a good chunk of today trying to get darcs to run my unit tests automatically. I haven\’t actually gotten it right yet, but at least I have it running the tests before each commit (even if the commit happens regardless).

I set up the auto-testing with darcs setpref test \"chmod +x test/run-tests; test/run-tests\". test/run-tests is the test script, but darcs doesn\’t allow you to add executable files to the repository, so you have to run chmod on the script before it can actually be executed.

Here\’s the test script itself:

sh: /home/pfeilgm/bin/enscript: No such file or directory

What it should be doing is running the tests, and returning non-zero on failure. It runs the tests, but the failure bit isn\’t exactly happening yet.

I figured I\’d throw this out there and see if anyone else has a decent way of getting automated testing happening. The more I try to bridge between Lisp and Unix, the more I just want to live in a Lisp REPL.

Integration is the Sincerest Form of Flattery

Saturday, April 7th, 2007

A while ago, Paul Graham blogged about not competing with Google. He mentioned how Google Calendar crushed other calendering apps because of ?Google Calendar\’s integration with Gmail. The Kikos can\’t very well write their own Gmail to compete.? And he actually concludes that ?the best solution for most startup founders would probably be to stay out of Google\’s way.?

I think he\’s right that integration was a major factor, but wrong in his evaluation of how to survive. Kiko doesn\’t need to write their own mail app to integrate with, they just need some mail app to integrate with. All of these ?Web 2.0? guys need to think a bit more about the Unix idea of ?small, sharp tools?, and not re-implement social networking with each new feature.

I\’ve been toying with a little Web app of my own and while I knew it needed some social networking, I also knew that I wasn\’t going to waste time writing it. I\’m going to piggy-back on Facebook. They have a nice API that other sites (like Bill Monk) are already capitalizing on. I\’m not going to write my own e-commerce engine, either, I\’m just going to use Amazon\’s APIs.

To a startup, time is precious. You can\’t throw away months re-implementing something that isn\’t pushing boundaries. You just need to find someone who already did, and hop on for the ride. Don\’t get caught in the pride of thinking you can do it better than them. If that\’s true, you can prove it once you have a couple dozen employees and pretty solid ground, not when you need to make every hour count.

So, please, create a tool, give it a good API, and let the mashups take over. Being connected to everything is your best chance of survival.

Releases 1.1.0

Thursday, November 16th, 2006

Yesterday\’s post was perhaps a bit off the cuff. There were a lot of people really scared about my script, and I don\’t blame them. A lot of good points were brought up. It\’s not that I didn\’t consider them, it\’s just that they\’re already taken into account with how I do revision control. At least Luis had faith in me :)

Below is an example of how I tend to organize a project (my pre-1.0 releases are a bit different, so I\’m just leaving them out for the moment). There are effectively two kinds of tags, fixed (in black) and mobile (in red).

\"\"

Notice that all the fixed tags are three-number versions. This means that if you request my-library_1.1.1.tar.gz, you\’re going to get that same code point all the time. However, you probably don\’t want to do that …

(more…)

pre-Released

Thursday, November 16th, 2006

Quick followup to my last post before I go to sleep. Here\’s a partial and buggy implementation for CVS and darcs. Yay. Maybe it\’ll get better tomorrow.

sh: /home/pfeilgm/bin/enscript: No such file or directory

Breathe in … and Release

Wednesday, November 15th, 2006

I wouldn\’t say I disagree with Jack Unrue\’s statement about releases being necessary, but I think there\’s a way to make both groups happy (and maybe I should just get around to implementing it). If the no-release crowd thinks that we don\’t need to indicate functionality changes in some way, there\’s still an impasse. If that\’s not the case, I may have a solution.

We all know that a URL doesn\’t have to point to a file, so what if http://example.com/releases/my-library_0.3.4.tar.gz created the archive on the fly? It would be built from the \”my-library\” repository, using the tip of the \”0.3.4? branch, using tar -cz (if the user had typed \”.tar.bz2\” on the end, it would have used tar -cj). You can, of course, cache the archive (just check to make sure there are no new commits since the last build).

What are the benefits of this? Well, the developers don\’t need to explicitly build release packages. Also, any bug fixes that get merged into the branch are included whenever the next person requests that archive. It gives the user the same interface that they\’re used to. It works fine with ASDF, etc. (as long as you remember to set the latest tag on your current branch).

Actually, this may work better as a commit hook than being pull-based. This should be pretty trivial to build for any specific source control system. Maybe it\’s my project for this evening.

Subsequent Thoughts on Concurrency

Tuesday, November 7th, 2006

Over the past few years my opinions on concurrency have changed. I mean, threads were always a pretty scary thing. I figured that no matter what I did, there must be a race condition or deadlock in there somewhere (and really, if you ever think otherwise, you\’re just fooling yourself). I didn\’t know of any alternatives. Java\’s approach seemed to have some merits — all you had to do was write synchronized, right? — and I took it a bit further with a language I was designing some years back. Here\’s an image from the spec:

(more…)

YACLHTTPAPI

Thursday, August 31st, 2006

So, I wrote a new HTTP client package for Common Lisp. I know, there are already a bunch, but they all seem to be pretty limited in their functionality (yes, even the one in Closure).

For one, they expect character content, which is fine when you\’re trying to load a Web page (as long as it\’s in the encoding the client expects) but not so much when you\’re dealing with binary data. There also isn\’t much support for \”Transfer-Encoding: chunked\”. Closure does that, I think, but no one else. I also have some other nice features:

So, I needed to handle both binary data and chunked encoding for what I\’m doing at work. I figured I might as well take some time to make a full implementation of RFC 2616. It\’s not quite perfect yet, so here\’s a list of the known shortcomings:

  • sometimes the content is returned as a byte vector, other times as a stream;
  • no parsing of date headers (I\’ll probably just steal this from Closure);
  • it might be nice to return a string in cases where we have a Content-Type (but this might just add confusion); and
  • handling other header fields.

None of these things prevents it from complying with RFC 2616, though (I think). So, it\’s just a matter of making it more user-friendly at this point. I\’d appreciate any feedback. There\’s just the darcs repository now, but I\’ll make this into a CLNet project if people are interested.

LispWorks Personal under SLIME on OS X

Sunday, May 21st, 2006

A while ago, Bill Clementson wrote about how to make Lispworks Personal Edition work under SLIME. It’s a lot harder than you might think (well, harder than I thought, anyway). And it seems like they change things over time to make it even more difficult. So, Bill’s 18-month-old instructions no longer work. I did a lot of cursing while trying to fix it. Here’s what I came up with. (Again, this is mostly Bill’s work, I just changed what was necessary to have it work with LispWorks 4.4.6.)

(more…)

When is Testing Silly?

Saturday, May 13th, 2006

Gary King compares test-driven development with literate programming, saying that writing the tests often ends up being \”silly\”, since he doesn\’t yet know where he\’s headed and the tests get thrown out. I agree that tests are sometimes unnecessary, but I think part of Gary\’s problem must be that he\’s developing tests the wrong way.

The appropriateness of tests depends on why you\’re writing the code. Do you have some use for it in mind, or is it pure speculation and experimentation? If it\’s the former, writing tests is paramount and should be easy (or, at least it gets easy once you get into the habit); the closer you get to experimentation, however, tests become harder and less appropriate. I think that if you have any inkling of how you\’d use the library, you should write some tests for it. It just gives you a bit of direction, and a way to see what you still need to accomplish. This can mean just grabbing part of your REPL session and throwing it in a file.

How is it possible that testing can be easy? Well, if you\’re writing a library that you have a planned use for, the trick is to use it before you write it. You don\’t sit down and try to come up with an API or some spec, you just pretend that your library already exists — perfectly coded — and use it. Even before TDD (now better explained as Behavior-Driven Development) this was an approach I used frequently. Write examples and other usages of your new library so you have some target to reach.

These are your tests. These are the things that need to work, and they are your effective spec. Of course, new use cases are going to come along and some of your assumptions are going to change, but they should almost entirely come from your (and others) usage of the library. The hard part always seems getting a test framework in place so you can slice up your usage and make tests out of them.

I think BDD is a significant step up from TDD because of its shift in mindset — it encourages you to think of what should happen as opposed to thinking that you need to test all functionality. Only test what you use, and only implement what you test.

It\’s hard when you start, but with a bit of practice it becomes natural.

Note: I know testing end-user applications and Web sites is difficult. All I can offer is to make that layer as thin as possible, and do more things as libraries. This is the natural way of Lisp … you stop thinking in terms of applications at all, because you can play with everything interactively. Any user interface is there to simplify the API for less sophisticated users.

Lisp Packages are Domains

Sunday, May 7th, 2006

In C2:CommonLisp, or any language that uses C2:GenericFunction s instead of class encapsulation, the package actually defines a domain. By domain, I mean a vocabulary in which each word has only one meaning.

Since C2:CommonLisp only allows for a single function signature within each package, this meaning for the word must be well-defined.

Often, the same concepts appear in multiple domains. Sometimes they keep the same names and it’s easy to :use a package in order to extend that domain with your own. However, sometimes the concepts carry over, but the vocabulary doesn’t. Currently that means that for each word in the vocabulary, you need to do this:

sh: /home/pfeilgm/bin/enscript: No such file or directory

Of course, defalias is a popular utility to make this renaming easier, but it\’s not part of the standard. I wonder if this could be made even simpler with an idea stolen from Eiffel: feature renaming:

sh: /home/pfeilgm/bin/enscript: No such file or directory

This makes the domain translation explicit.

Thinking of packages as domains may help limit the naming conflicts that arise in C2:GenericFunction-based OO systems, since people often try to use the packages much like Java packages, rather than like Perlpackages (a much closer, albeit single-dispatch, analogue).

Testing Lisp

Friday, March 31st, 2006

Gary King wrote an article not so recently about how to set up ASDF test-ops. I basically stole it, but have changed it a bit to work more intuitively (IMO). He arrives at the following test system definition:

1
2
3
4
5
6
7
8
9
10
11
12
13
(defsystem moptilities-test
  :components ((:module "test"
                        :components ((:file "tests"))))
  :in-order-to ((test-op (load-op moptilities-test)))
  :perform (test-op :after (op c)
                    (describe (funcall (intern "RUN-TESTS" "LIFT")
                                       :suite (intern "TEST-MOPTILITIES"
                                                      "TEST-MOPTILITIES"))))
  :depends-on (moptilities lift))

(defmethod operation-done-p ((o test-op)
                             (c (eql (find-system 'moptilities-test))))
  (values nil))

However, it seems to me that (asdf:oos 'asdf:test-op :moptilities-test) should be testing the tests, not Moptilities. I would much prefer to see (asdf:oos 'asdf:test-op :moptilities), and this is possible with a little rearrangement.

You still need a test system, but it should be pretty minimal:

1
2
3
4
(defsystem moptilities-test
  :components ((:module "test"
                        :components ((:file "tests"))))
  :depends-on (moptilities lift))

Then, the other pieces should be in the primary system:

1
2
3
4
5
6
7
8
(defsystem moptilities
  :components (...)
  :in-order-to ((test-op (load-op moptilities-test)))
  :perform (test-op :after (op c)
                    (describe (funcall (intern "RUN-TESTS" :lift)
                                       :suite (intern "TEST-MOPTILITIES"
                                                      :moptilities-test))))
  :depends-on (...))

Yeah, in addition to moving things to the primary system, I also changed a couple strings to keywords … just a personal preference. Also, OPERATION-DONE-P should operate on the MOPTILITIES system, rather than MOPTILITIES-TEST. Oh, and I changed the package name from TEST-MOPTILITIES to MOPTILITIES-TEST. I?like to keep packages as nouns and functions as verbs.

I’m always searching through Gary’s archives to find this, and then re-remembering my variations, so this puts it all in one place for me (and maybe others) in the future. Leave a comment if you have any suggestions.

What I\’m playing with at home

Thursday, March 16th, 2006

This is an email I sent out to a few friends, talking about what we\’re doing with our time, and I realized that it\’s exactly the sort of thing I should be posting.

There are a number of models of computation, the most famous being the Turing machine and the λ-calculus (which are equivalent, but λ-calculus is prettier). They\’re both pretty old, dating from the 40s and 50s respectively. Of course, they\’ve pretty well stood the test of time. Two of the more modern ones that have held my interest are the ρ-calculus and the π-calculus. Both of these encode the λ-calculus very easily (meaning anything you can do with λ-calculus you can do with them).

(more…)

Multiple Paradigms

Wednesday, March 8th, 2006

I\’ve got a few things floating around in my brain, making some odd connections. I recently watched Richard Dawkins\’ The Root of All Evil? where he discusses many of the problems with religion. And today I had a discussion with someone at work who didn\’t see why I would do a project in Lisp. We discussed a few different reasons, but there\’s one in particular I want to focus on, and I think it ties in with \”the religion problem\”.

(more…)

The Language Game

Friday, February 24th, 2006

Those of you who know me well, may know of my aborted attempt at language design. While working on it, I hit what I call the \”language design singularity\” — a point at which you realize you\’re headed down the wrong path, and make that jump to the right one. I imagine a lot of would-be language designers hit the singularity. People get interested in design because they recognize the problems with the current system, and seek to improve the situation. At this point, their path begins to move closer to the singularity, eventually there is a jump, like when a metal rod slides close enough to a magnet, and you find yourself in a world where your problems don\’t exist.

For me, it happened like this … (more…)

Python Indentation

Tuesday, February 21st, 2006

So, one of the touted benefits of Python is its indentation-based structure. Who needs curly braces, begin-end blocks, or (heaven forfend) parentheses? Now, I agree … proper indentation makes reading code easier. However I don\’t indent my code, my editor does. And when I change the structure of something one keystroke fixes all the indentation. Does this happen in Python editors? How does it know which parts should be re-indented, and which are correct? Does the programmer walk line-by-line typing tab, or is the editor really smart enough to handle it?

Haskell behaves the same way as Python, but

  1. I don\’t write much Haskell either and
  2. the code I do write doesn\’t really have much indentation, thanks to pattern matching.

I\’ll admit — I\’m pretty hooked on parentheses. Working with an AST directly feels so powerful, and reader macros allow me to add syntax where necessary. I will try to listen to the Python side, though. I was pretty into the idea of indentation-based structure for a while.

BTW, this comes to mind because of Guido\’s posting where he says, \”I find any solution unacceptable that embeds an indentation-based block in the middle of an expression,\” which sounds more like an indictment of indentation-syntax than of lambdas.

Common Libraries for Common Lisp

Monday, December 12th, 2005

A frequent complaint about Common Lisp is that it doesn\’t have the sort of standard library that is so popular these days. Java, Python, Ruby … everything has a standard library now. When Common Lisp was standardized, its functions comprised an enviable library. But, times change, everyone else catches up, and you gotta keep on your toes to stay in the game. So what does CL need to do to get back in it? Not too much. There are a lot of libraries out there already: cl-containers, CL-XML, etc. But, whereas Java just comes with a pile of a few thousand useful classes, CL doesn\’t. The available libraries need to be collected, sorted through to see where there are still gaps as compared with other languages\’ libraries, and made 1-step retrievable (if not included in the distribution).

I started writing this a while ago, and in the meantime, Dave Robert\’s posted an article that made a very similar point, in which he decided to dive in and fill one of those gaps (a full-fledged networking library). I have spent (very little) time wrapping a few implementations\’ threading libs in the BORDEAUX-MP API.

Now I\’m stepping into the other half of the game — collecting and categorizing all the existing CL libraries, and trying to pull some of those into a competitive standard library. I\’ll post a link to my standard library comparison shortly.

Single-Dispatch\’s Win

Monday, December 12th, 2005

There is a common feature in single-dispatch OO systems that I haven\’t seen in multiple dispatch: handling of \”missing methods\”. In Ruby, it\’s method_missing(); in Perl it\’s AUTOLOAD; Smalltalk, doesNotUnderstand:; and Obj-C, forwardInvocation:. For the C++ and Java programmer, who lack such functionality, I\’ll describe it a bit:

If you define (say, in Ruby) method_missing() on a class, and a user calls a method on that class which does not exist, Ruby will pass the call to your method_missing() implementation and you\’re given the opportunity to handle the call there. This can be useful in a number of cases; one of those is where the user may expect certain things to exist as methods, but the set may be variable or unbounded. For example, creating an XML structure, and allowing the user to just assuming the existance of a method matching each element.

In Lisp (and other multiple-dispatch systems that I\’m aware of) there is no such ability. I\’ve run into a number of places where I\’ve wanted such a mechanism. I don\’t have any solution in mind to this now, but it is something that I would like to think about. And maybe get input from others about.