automated testing with CL & darcs

I spent a good chunk of today trying to get darcs to run my unit tests automatically. I haven\’t actually gotten it right yet, but at least I have it running the tests before each commit (even if the commit happens regardless).

I set up the auto-testing with darcs setpref test \"chmod +x test/run-tests; test/run-tests\". test/run-tests is the test script, but darcs doesn\’t allow you to add executable files to the repository, so you have to run chmod on the script before it can actually be executed.

Here\’s the test script itself:

states: undefined state `--language=html' %!PS-Adobe-3.0 %%BoundingBox: 24 24 571 818 %%Title: Enscript Output %%For: Greg Pfeil %%Creator: GNU enscript 1.6.4 %%CreationDate: Mon Sep 6 12:12:55 2010 %%Orientation: Portrait %%Pages: (atend) %%DocumentMedia: A4 595 842 0 () () %%DocumentNeededResources: (atend) %%EndComments %%BeginProlog %%BeginResource: procset Enscript-Prolog 1.6 4 % % Procedures. % /_S { % save current state /_s save def } def /_R { % restore from saved state _s restore } def /S { % showpage protecting gstate gsave showpage grestore } bind def /MF { % fontname newfontname -> - make a new encoded font /newfontname exch def /fontname exch def /fontdict fontname findfont def /newfont fontdict maxlength dict def fontdict { exch dup /FID eq { % skip FID pair pop pop } { % copy to the new font dictionary exch newfont 3 1 roll put } ifelse } forall newfont /FontName newfontname put % insert only valid encoding vectors encoding_vector length 256 eq { newfont /Encoding encoding_vector put } if newfontname newfont definefont pop } def /MF_PS { % fontname newfontname -> - make a new font preserving its enc /newfontname exch def /fontname exch def /fontdict fontname findfont def /newfont fontdict maxlength dict def fontdict { exch dup /FID eq { % skip FID pair pop pop } { % copy to the new font dictionary exch newfont 3 1 roll put } ifelse } forall newfont /FontName newfontname put newfontname newfont definefont pop } def /SF { % fontname width height -> - set a new font /height exch def /width exch def findfont [width 0 0 height 0 0] makefont setfont } def /SUF { % fontname width height -> - set a new user font /height exch def /width exch def /F-gs-user-font MF /F-gs-user-font width height SF } def /SUF_PS { % fontname width height -> - set a new user font preserving its enc /height exch def /width exch def /F-gs-user-font MF_PS /F-gs-user-font width height SF } def /M {moveto} bind def /s {show} bind def /Box { % x y w h -> - define box path /d_h exch def /d_w exch def /d_y exch def /d_x exch def d_x d_y moveto d_w 0 rlineto 0 d_h rlineto d_w neg 0 rlineto closepath } def /bgs { % x y height blskip gray str -> - show string with bg color /str exch def /gray exch def /blskip exch def /height exch def /y exch def /x exch def gsave x y blskip sub str stringwidth pop height Box gray setgray fill grestore x y M str s } def /bgcs { % x y height blskip red green blue str -> - show string with bg color /str exch def /blue exch def /green exch def /red exch def /blskip exch def /height exch def /y exch def /x exch def gsave x y blskip sub str stringwidth pop height Box red green blue setrgbcolor fill grestore x y M str s } def % Highlight bars. /highlight_bars { % nlines lineheight output_y_margin gray -> - gsave setgray /ymarg exch def /lineheight exch def /nlines exch def % This 2 is just a magic number to sync highlight lines to text. 0 d_header_y ymarg sub 2 sub translate /cw d_output_w cols div def /nrows d_output_h ymarg 2 mul sub lineheight div cvi def % for each column 0 1 cols 1 sub { cw mul /xp exch def % for each rows 0 1 nrows 1 sub { /rn exch def rn lineheight mul neg /yp exch def rn nlines idiv 2 mod 0 eq { % Draw highlight bar. 4 is just a magic indentation. xp 4 add yp cw 8 sub lineheight neg Box fill } if } for } for grestore } def % Line highlight bar. /line_highlight { % x y width height gray -> - gsave /gray exch def Box gray setgray fill grestore } def % Column separator lines. /column_lines { gsave .1 setlinewidth 0 d_footer_h translate /cw d_output_w cols div def 1 1 cols 1 sub { cw mul 0 moveto 0 d_output_h rlineto stroke } for grestore } def % Column borders. /column_borders { gsave .1 setlinewidth 0 d_footer_h moveto 0 d_output_h rlineto d_output_w 0 rlineto 0 d_output_h neg rlineto closepath stroke grestore } def % Do the actual underlay drawing /draw_underlay { ul_style 0 eq { ul_str true charpath stroke } { ul_str show } ifelse } def % Underlay /underlay { % - -> - gsave 0 d_page_h translate d_page_h neg d_page_w atan rotate ul_gray setgray ul_font setfont /dw d_page_h dup mul d_page_w dup mul add sqrt def ul_str stringwidth pop dw exch sub 2 div ul_h_ptsize -2 div moveto draw_underlay grestore } def /user_underlay { % - -> - gsave ul_x ul_y translate ul_angle rotate ul_gray setgray ul_font setfont 0 0 ul_h_ptsize 2 div sub moveto draw_underlay grestore } def % Page prefeed /page_prefeed { % bool -> - statusdict /prefeed known { statusdict exch /prefeed exch put } { pop } ifelse } def % Wrapped line markers /wrapped_line_mark { % x y charwith charheight type -> - /type exch def /h exch def /w exch def /y exch def /x exch def type 2 eq { % Black boxes (like TeX does) gsave 0 setlinewidth x w 4 div add y M 0 h rlineto w 2 div 0 rlineto 0 h neg rlineto closepath fill grestore } { type 3 eq { % Small arrows gsave .2 setlinewidth x w 2 div add y h 2 div add M w 4 div 0 rlineto x w 4 div add y lineto stroke x w 4 div add w 8 div add y h 4 div add M x w 4 div add y lineto w 4 div h 8 div rlineto stroke grestore } { % do nothing } ifelse } ifelse } def % EPSF import. /BeginEPSF { /b4_Inc_state save def % Save state for cleanup /dict_count countdictstack def % Count objects on dict stack /op_count count 1 sub def % Count objects on operand stack userdict begin /showpage { } def 0 setgray 0 setlinecap 1 setlinewidth 0 setlinejoin 10 setmiterlimit [ ] 0 setdash newpath /languagelevel where { pop languagelevel 1 ne { false setstrokeadjust false setoverprint } if } if } bind def /EndEPSF { count op_count sub { pos } repeat % Clean up stacks countdictstack dict_count sub { end } repeat b4_Inc_state restore } bind def % Check PostScript language level. /languagelevel where { pop /gs_languagelevel languagelevel def } { /gs_languagelevel 1 def } ifelse %%EndResource %%BeginResource: procset Enscript-Encoding-88591 1.6 4 /encoding_vector [ /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /space /exclam /quotedbl /numbersign /dollar /percent /ampersand /quoteright /parenleft /parenright /asterisk /plus /comma /hyphen /period /slash /zero /one /two /three /four /five /six /seven /eight /nine /colon /semicolon /less /equal /greater /question /at /A /B /C /D /E /F /G /H /I /J /K /L /M /N /O /P /Q /R /S /T /U /V /W /X /Y /Z /bracketleft /backslash /bracketright /asciicircum /underscore /quoteleft /a /b /c /d /e /f /g /h /i /j /k /l /m /n /o /p /q /r /s /t /u /v /w /x /y /z /braceleft /bar /braceright /tilde /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /.notdef /space /exclamdown /cent /sterling /currency /yen /brokenbar /section /dieresis /copyright /ordfeminine /guillemotleft /logicalnot /hyphen /registered /macron /degree /plusminus /twosuperior /threesuperior /acute /mu /paragraph /bullet /cedilla /onesuperior /ordmasculine /guillemotright /onequarter /onehalf /threequarters /questiondown /Agrave /Aacute /Acircumflex /Atilde /Adieresis /Aring /AE /Ccedilla /Egrave /Eacute /Ecircumflex /Edieresis /Igrave /Iacute /Icircumflex /Idieresis /Eth /Ntilde /Ograve /Oacute /Ocircumflex /Otilde /Odieresis /multiply /Oslash /Ugrave /Uacute /Ucircumflex /Udieresis /Yacute /Thorn /germandbls /agrave /aacute /acircumflex /atilde /adieresis /aring /ae /ccedilla /egrave /eacute /ecircumflex /edieresis /igrave /iacute /icircumflex /idieresis /eth /ntilde /ograve /oacute /ocircumflex /otilde /odieresis /divide /oslash /ugrave /uacute /ucircumflex /udieresis /yacute /thorn /ydieresis ] def %%EndResource %%EndProlog %%BeginSetup %%IncludeResource: font Courier-Bold %%IncludeResource: font Courier /HFpt_w 10 def /HFpt_h 10 def /Courier-Bold /HF-gs-font MF /HF /HF-gs-font findfont [HFpt_w 0 0 HFpt_h 0 0] makefont def /Courier /F-gs-font MF /F-gs-font 10 10 SF /#copies 1 def % Pagedevice definitions: gs_languagelevel 1 gt { < < /PageSize [595 842] >> setpagedevice } if %%BeginResource: procset Enscript-Header-simple 1.6 4 /do_header { % print default simple header gsave d_header_x d_header_y HFpt_h 3 div add translate HF setfont user_header_p { 5 0 moveto user_header_left_str show d_header_w user_header_center_str stringwidth pop sub 2 div 0 moveto user_header_center_str show d_header_w user_header_right_str stringwidth pop sub 5 sub 0 moveto user_header_right_str show } { 5 0 moveto fname show 45 0 rmoveto fmodstr show 45 0 rmoveto pagenumstr show } ifelse grestore } def %%EndResource /d_page_w 547 def /d_page_h 794 def /d_header_x 0 def /d_header_y 779 def /d_header_w 547 def /d_header_h 15 def /d_footer_x 0 def /d_footer_y 0 def /d_footer_w 547 def /d_footer_h 0 def /d_output_w 547 def /d_output_h 779 def /cols 1 def %%EndSetup %%Trailer %%Pages: 0 %%DocumentNeededResources: font Courier-Bold Courier %%EOF

What it should be doing is running the tests, and returning non-zero on failure. It runs the tests, but the failure bit isn\’t exactly happening yet.

I figured I\’d throw this out there and see if anyone else has a decent way of getting automated testing happening. The more I try to bridge between Lisp and Unix, the more I just want to live in a Lisp REPL.

4 Responses to “automated testing with CL & darcs”

  1. D Herring Says:

    Have you considered switching to git?

    • stores unix permissions (i.e. can store executables)
    • easy commit hooks; each .git directory contains a hooks subdir; you simply edit and chmod the script; in your case, the pre-commit hook would do the job
    • much other goodness (speed, gitk, git-citool, git-gui, crypto hash checks, …)
  2. Greg Pfeil Says:

    I actually meant to mention that in the post. I do plan to use git on any new projects. I don’t know if it’s available for common-lisp.net projects yet, though.

  3. D Herring Says:

    Unfortunately, Git is not yet installed on c-l.net. There is a consensus to install it, but nobody with the permissions has had the time.

  4. Robert Goldman Says:

    One of the issues you are running against is the fact that the asdf:test-op doesn’t have a nice means of returning answers to you to tell you if the test was passed or not. You might suggest modifying ASDF to make this possible (you can find the ASDF maintainers on cclan-list).

Leave a Reply