Writing tests¶
Test as a function¶
Here is a very simple example of a suite:
# suites/my_suite.py:
import lemoncheesecake.api as lcc
from lemoncheesecake.matching import *
@lcc.test()
def my_test_1():
check_that("value", "foo", equal_to("foo"))
@lcc.test()
def my_test_2():
check_that("value", "bar", equal_to("bar"))
$ lcc show
* my_suite
- my_suite.my_test_1
- my_suite.my_test_2
A test is a function (or a method, see later) decorated with @lcc.test()
.
Test as a method, suite as a class¶
Tests can also be implemented as class methods:
# suites/my_suite.py:
import lemoncheesecake.api as lcc
from lemoncheesecake.matching import *
@lcc.suite()
class suite_a:
@lcc.test()
def my_test_1():
check_that("value", "foo", equal_to("foo"))
@lcc.suite()
class suite_b:
@lcc.test()
def my_test_1():
check_that("value", "bar", equal_to("bar"))
$ lcc show
* my_suite
* my_suite.suite_a
- my_suite.suite_a.my_test_1
* my_suite.suite_b
- my_suite.suite_b.my_test_1
In that case, the class must be decorated with @lcc.suite()
and will be considered as a sub-suite of the current
module-suite.
Please note that if the module only contains one suite-class whose name is equal to the module’s name, then this suite will not be considered as a sub-suite of the module-suite, but as the suite itself:
# suites/my_suite.py:
import lemoncheesecake.api as lcc
from lemoncheesecake.matching import *
@lcc.suite()
class my_suite:
@lcc.test()
def my_test_1():
check_that("value", "foo", equal_to("foo"))
@lcc.test()
def my_test_2():
check_that("value", "bar", equal_to("bar"))
$ lcc show
* my_suite
- my_suite.my_test_1
- my_suite.my_test_2
Organizing suites within directories¶
Suites hierarchy can also be created through putting modules into sub-directories, such as this:
# suites/parent_suite.py:
import lemoncheesecake.api as lcc
from lemoncheesecake.matching import *
@lcc.test()
def test_in_parent_suite():
check_that("value", "foo", equal_to("foo"))
# suites/parent_suite/child_suite.py:
import lemoncheesecake.api as lcc
from lemoncheesecake.matching import *
@lcc.test()
def test_in_child_suite():
check_that("value", "foo", equal_to("foo"))
.
└── suites
├── parent_suite
│ └── child_suite.py
└── parent_suite.py
$ lcc show
* parent_suite
- parent_suite.test_in_parent_suite
* parent_suite.child_suite
- parent_suite.child_suite.test_in_child_suite
A directory without a *.py
associated file, will be also considered as a suite:
# suites/parent_suite/child_suite.py:
import lemoncheesecake.api as lcc
from lemoncheesecake.matching import *
@lcc.test()
def test_in_child_suite():
check_that("value", "foo", equal_to("foo"))
.
└── suites
└── parent_suite
└── child_suite.py
$ lcc show
* parent_suite
* parent_suite.child_suite
- parent_suite.child_suite.test_in_child_suite
Changed in version 1.5.0.
Since version 1.5.0, several prior requirements have been made optional:
the description for tests and suites is now optional
the
SUITE
variable in module is now optionalsuites can be created with only a directory containing
*.py
files, a*.py
file companion is no longer a requirement, in other words: for aparent_suite
directory aparent_suite.py
file (at the same level) is no longer mandatory
Metadata¶
Metadata can be associated to both tests and suites, they can be used to filter tests and will be displayed in the report:
description (a description is generated from the test/suite name by default)
tags
properties (key/value pairs)
link (an URL and an optional description)
Example:
@lcc.test("A test with a meaningful description")
@lcc.tags("important")
@lcc.tags("a_second_tag", "a_third_tag")
@lcc.prop("type", "acceptance")
@lcc.link("http://bugtracker.example.com/issues/1234", "TICKET-1234")
@lcc.link("http://bugtracker.example.com/issues/1235")
def my_test():
check_that("value", "foo", equal_to("foo"))
The @lcc.suite()
decorator also takes a description like @lcc.test()
does.
The @lcc.tags()
, @lcc.prop()
and @lcc.link()
decorators also apply to suite-classes.
In suite-modules, metadata are be set through a SUITE
module-level variable:
# suites/my_suite.py
import lemoncheesecake.api as lcc
from lemoncheesecake.matching import *
SUITE = {
"name": "another_name",
"description": "A suite with a meaningful description",
"tags": ["important", "a_second_tag", "a_third_tag"],
"properties": {"type": "acceptance"},
"links": [
("http://bugtracker.example.com/issues/1234", "TICKET-1234"),
"http://bugtracker.example.com/issues/1235"
]
}
@lcc.test()
def my_test():
check_that("value", "foo", equal_to("foo"))
As it can be seen in the previous examples, test and suites name are determined from the decorated object’s name or
from the current module.
It can be overridden in @lcc.test()
and @lcc.suite()
decorators:
@lcc.test(name="test_something")
def my_test():
pass
Note
Test/suite descriptions are optional and automatically generated from the test/suite name. Since names are somewhat technical, it is recommended to set an explicit description to the test/suite in order to provide a meaningful insight of what the test/suite does to the test report reader.
Disabling a test or a suite¶
A test or an entire suite can be disabled using the @lcc.disabled()
decorator:
@lcc.test("Test something")
@lcc.disabled()
def test_something(self):
pass
Disabled tests are visible in the report but they are not taken into account while computing the percentage of successful tests.
New in version 1.1.0: it’s possible to pass a reason
(str) argument to the decorator, it will be visible in the generated report.
If you want to completely hide a test or a suite from the test tree and the report, use @lcc.hidden()
.
Conditional tests and suites¶
A test or an entire suite can included or excluded from the test tree using the @lcc.visible_if(condition)
decorator.
This decorator can be associated to both tests and suites, it takes a callable as argument. This callable will
be called with the object to which it is associated (a module, a class or a function).
If the callable return a non-true value, then the test/suite
won’t be included in the test tree, meaning it won’t be executed, it won’t be present in the test report nor in the
lcc show
command output.
Usage:
@lcc.suite("My Suite")
class mysuite:
some_feature_enabled = True
@lcc.test("Test something")
@lcc.visible_if(lambda test: mysuite.some_feature_enabled)
def test_something(self):
pass
Dependency between tests¶
Dependency between tests can be added using the @lcc.depends_on()
decorator:
@lcc.suite("My Suite")
class mysuite:
@lcc.test("Test 1")
def test_1():
pass
@lcc.test("Test 2")
@lcc.depends_on("mysuite.test_1")
def test_2():
pass
If mysuite.test_1
fails, then mysuite.test_2
will be skipped.
Dependencies can be expressed as test paths.
New in version 1.14.1: Callable syntax
They can also be expressed using a callable that takes a Test
instance:
@lcc.suite("My Suite")
class mysuite:
@lcc.test("Test 1")
@lcc.tags("mytag")
def test_1():
pass
@lcc.test("Test 2")
@lcc.tags("mytag")
def test_2():
pass
@lcc.test("Test 3")
@lcc.depends_on(lambda test: "mytag" in test.tags)
def test_3():
pass
In this example above, the test_3
depends on all tests with the mytag
tag, in other words: test_1
and test_2
.
The lcc.depends_on()
decorator:
can take multiple test paths or callable
a test can depend on any test of a test project
it is only applicable to tests (not suites)
the test path must point to a test (not a suite)
Setup and teardown methods¶
Test suites provide several methods that give the user the possibility to execute code at particular steps of the suite execution:
setup_suite()
is called before executing the tests of the suite; if something wrong happens (a call tolog_error
or a raised exception) then the whole suite execution is abortedsetup_test(test)
takes the test instance as argument and is called before each test, if something wrong happen then the test execution is abortedteardown_test(test, status)
is called after each test, it takes the test instance as argument and the status of the test “so far”, if something wrong happens the executed test will be mark as failedteardown_suite()
is called after executing the tests of the suite
Note
code within
setup_suite
andteardown_suite
methods is executed in a dedicated context and the data it generates (checks, logs) will be represented the same way as a test in the test reportcode within
setup_test
andteardown_test
methods is executed within the related test context and the data it generates will be associated to the given test
Suites discovery¶
A project path can be either:
a
project.py
-like filea directory containing a
project.py
a directory containing a
suites
sub-directory
Here is how suites are discovered by lemoncheesecake (by order of priority):
in case of
lcc run
, with the--project/-p
is usedusing the
$LCC_PROJECT
environment variable if definedby searching a project path among the directory hierarchy
Changed in version 1.5.0: prior to 1.5.0, a project file (project.py
) was mandatory; since 1.5.0, lemoncheesecake can discover tests simply
from a suites
directory.