Shake - A Simple Lua Test Engine

Shake
A Simple Lua Test Engine

## Examples

### Running Shake in the command line

Assuming you have a module like LuaFileSystem installed and you go to its `/tests` directory and run Shake from there, the output would be:

    ~/workspace/luafilesystem/tests$ shake
    ->  test.lua OK!
    _________________

    Tests: 27
    Failures: 0
    Errors: 0

On the other hand, if you have a test script like the one below that includes two assertions that are supposed to fail (lines are numbered):

     1	items = 10
     2	-- checks the correct case
     3	assert (items == 10, "this should not fail")
     4
     5	items = 20
     6	-- checks an overflow case
     7	assert (items == 10, "wrong number of items")
     8
     9	print("Verifying the total")
    10	items = 10
    11	total = 30
    12	assert (items == total, "wrong total")

Shake would register the failures but would run the whole test script, reporting at the end:

    :~/workspace$ shake
    ----------------    test.lua failed!   ----------------

    -- checks an overflow case
       #7 assert (items == 10, "wrong number of items")
       items -> 20

    Verifying the total
       #12 assert (items == total, "wrong total")
       items -> 10
       total -> 30
    _________________

    Tests: 3
    Failures: 2
    Errors: 0

Note how much more informative this is when compared to the default output of running the test script with Lua:

    :~/workspace$ lua5.1 test.lua
    lua5.1: test.lua:7: wrong number of items
    stack traceback:
            [C]: in function 'assert'
            test.lua:7: in main chunk
            [C]: ?

### Implementing a simple Shake runner

Here we show how to use the Shake API to implement runners. Note that if you just want to execute tests, the built in runners (command line and CGILua app) may be enough. On the other hand, if you need to show more detailed information about the tests results or if you want to use Shake as part of your application runtime, then you will need to use the Shake API.

The minimal Shake runner would be like:

    require "shake"
    local run = shake.runner()
    run:test("somefile.lua")
    print (run:summary())

This would be basically equivalent to the command line Shake runner, and if you want to report more details than the default summary does, you will need to drill down the results using something like:

    function ReportModules(run)
       local results = run.results
       for cs, suite in ipairs(results.suites) do
           -- displays information about the suite.title
           if suite.error then
    	-- displays information about the suite error
           elseif suite.failed > 0 then
    	-- displays information about the suite.failed results
           else
    	-- displays information about the suite.passed and suite.failed results
           end
       end
    end

For even more details, you may want to inspect each test result and decide what to show:

    function ReportModule(run)
      local results = run.results
      for _, suite in ipairs(results.suites) do
        if suite.error == -1 then
          -- displays information about the error
        else
          for _, context in ipairs(suite.contexts) do
            if next(context.tests) then
              if context.output[1] ~= "" or context.comments then
                -- displays information about the context
                for _, output in ipairs(context.output) do
                  if output and output ~= "" then
                    -- using context.output
                  end
                end
                if context.comments and context.comments ~= "" then
                  --  or context.comments
                end
              end
              for _, test in ipairs (context.tests) do
    	  local linenumber = test.linenumber or "???"
    	  local op = test.op
    	   local val2 = tostring(test.val2)
    	  -- when there is no comparision operator, assume that this is an assert(x) case
    	  if not op then
    	    val2 = "True value" -- just to diferentiate from the Lua "true"
    	  end
    	  if not op or op == "==" then
    	    op = ""
    	  end
    	  -- displays information about the test result using
    	  -- linenumber, test.exp1, op, val2, test.val1 and test.msg
    	end
           end
        end
    end

Valid XHTML 1.0!

$Id: examples.html,v 1.5 2007/12/21 22:55:20 carregal Exp $