Find references for testcase.

Default logic to find references is following:

use basename of testcase, e.g. filename of testcase without last extension, and append ".ref.*" glob to that basename. Then search under same directory, where testcase is located.

For example:

/path/to/dirsuite/testcase/test01.exec

findFiles("/path/to/dirsuite/testcase", Glob("test01.ref.*"))

Find test cases under basedir, by using test pattern to match test cases.

Get separator for exec line splitting.

This separator is used to split one exec line to actual arguments. Overload this if default ";" is not good for you.

Ignore these dirSuite tests.

Overload this if youd don't like to see all testcases which are ignored. You could here just cal registerIgnoredTest and register e.g. pattern.

Ignore these dirSuite tests.

Overload this if youd don't like to see all testcases which are ignored. You could here just cal registerIgnoredTest and register e.g. pattern.

Map arguments before test execution. If you have to provide e.g. some default argument for each test case, then overload this function, and change arguments as needed. (Add, remove, or transform args).

Map output filenames based on reference paths.

use basename of testcase, e.g. filename of testcase without last extension, and prefix "out.*" to that basename. Then append reference's filename suffix to that name. Reference file suffix is stripped (basename + ".ref.").

For example:

/path/to/dirsuite/testcase/test01.exec /path/to/dirsuite/testcase/test01.ref.output-name.txt

output : out.test01.output-name.txt

Parse one exec file.

If you have to change exec file format, look for tokenizer to parse each exec line and getExecArgumentSeparator to change exec's arguments separator.

Register dirsuite

If you don't like see each test case to be printed on test output, you could override this function and just call testCaseExecutor here.

Register ignored test case.

If you don't like see ignored test case to be printed on test output, you can overload this function and just don't do anything. Or even better, overload ignoreDirSuiteTestCases and register whole dirSuite once, so you still get information that there are ignored tests.

Find and Run test cases based on basedir and testPattern.

In addition of normal parameter, there is actual test method as parameter. This test method will be executed for each exec line, with arguments wich are defined by that exec-line.

Below is listed typical test case:

runDirSuiteTestCases(testdir, Glob("success/basic[0-9]*.exec")) { args: Array[String] =>
  assertResult(DemoApp.SUCCESS) {
    app.doTxt(args)
  }
}

Run (and find) dual assertion test cases. These are typically test cases where first steps are supposed to behave one way (succeeds), and then the last step is supposed to behave differently (fails).

This is useful for example testing a case when triggering of error you must first run multiple successful steps.

runDualAssertionDirSuiteTestCases(testdir, Glob("success/multiStepFail[0-9]*.exec"))
  { args: Array[String] =>
    // All steps at first must succeed
    assertResult(DemoApp.SUCCESS) {
      app.doFlaky(args)
    }
  } { args: Array[String] =>
      // Then the last step must fail
      assertThrows[RuntimeException] {
        app.doFlaky(args)
    }
  }

Select validator for this (one) test vector.

Default logic is that TestValidator.txtValidator is used for "*.txt" files, and TestValidator.xmlValidator is used for "*.xml" files.

Execute one test case.

Tokenizer for exec-line.

If default separator for arguments is not good for you, then overload getExecArgumentSeparator.

If you have to use some other format for exec lines (e.g.) JSON, then overload this function. If one raw: exec line is:

exec := "exec:" TEXT

then this function will be fed with TEXT.