API reference¶

API compatibility / stability¶

lemoncheesecake follows the well know Semantic Versioning for its public API. What is considered as “public” is everything which is documented on http://docs.lemoncheesecake.io/. Everything else is internal and is subject to change at anytime.

Tests and suites¶

lemoncheesecake.api.suite(description=None, name=None, rank=None)¶

Decorator, mark a class as a suite class.

Parameters:

description – suite’s description (by default, the suite’s description is built from the name)
name – suite’s name (by default, the suite’s name is taken from the class’s name)
rank – this value is used to order suites of the same hierarchy level

lemoncheesecake.api.test(description=None, name=None)¶

Decorator, mark a function/method as a test.

Parameters:

description – test’s description
name – test’s name (by default, the suite’s name is taken from the class’s name)

lemoncheesecake.api.tags(*tag_names)¶: Decorator, add tags to a test or a suite.

lemoncheesecake.api.prop(key, value)¶: Decorator, add a property (key/value) to a test or a suite.

lemoncheesecake.api.link(url, name=None)¶: Decorator, set a link (with an optional friendly name) to a test or a suite.

lemoncheesecake.api.disabled(reason=None)¶: Decorator, mark a test or a suite as disabled, meaning it won’t be executed but will be visible in report. An optional reason can be passed to the decorator (new in version 1.1.0).

lemoncheesecake.api.visible_if(condition)¶

Decorator, the test or suite will only appear if the given callable return a true value.

Parameters:: condition – a callable that will take the test object if applied to a test or the suite class instance if applied to a suite.

lemoncheesecake.api.hidden()¶: Decorator, the test or suite won’t be visible in the resulting test tree.

lemoncheesecake.api.depends_on(*deps)¶

Decorator, only applicable to a test. Add dependencies to a test.

Parameters:

deps (Union[str, Callable]) –

dependencies can be either:

a full test path:
```
@lcc.depends("mysuite.mytest")
```
a callable that express dependencies as a filter (a Test instance is passed to the callable):
```
@lcc.depends(lambda test: "mytag" in test.tags)
```

New in version 1.14.1: callable syntax

Return type:

Any

lemoncheesecake.api.parametrized(parameter_source, naming_scheme=<function _default_naming_scheme>)¶

Decorator, make the test parametrized.

Parameters:

parameter_source (Iterable) –
it can be either:
- an iterable of dicts, each dict representing the arguments passed to the test
- a CSV-like mode, where the first element of the list represents the argument names as an str or a sequence (example: "i,j" or ("i", "j")) and the remaining elements are sequences of the arguments to be passed to the test.
naming_scheme (Optional[Union[Callable[[str, str, dict, int], Tuple[str, str]], Sequence]]) –
optional, it can be either:
- a optional function that takes as parameters the base test name, description, parameters, index and must return the expanded test name and description as a two elements list
- a tuple/list of two (name + description) format strings, example: ("test_{i}_plus_{j}", "Test {i} plus {j}")

Return type:

Any

lemoncheesecake.api.inject_fixture(fixture_name=None)¶

Inject a fixture into a suite. If no fixture name is specified then the name of the variable holding the injected fixture will be used.

Parameters:: fixture_name (Optional[str]) –

lemoncheesecake.api.add_test_into_suite(test, suite)¶

Add test into suite

Parameters:

test (Test) – a Test instance
suite (Any) – a suite decorated class instance (in that case the function must be called when the class is instantiated) or a module marked as a suite (in that case the function must be called when the module is loaded)

Return type:

None

class lemoncheesecake.api.Test(name, description, callback)¶

Internal representation of a test.

property path: str¶: The complete path of the test node (example: if used on a test named “my_test” and a suite named “my_suite”, then the path is “my_suite.my_test”).

name¶: name

description¶: description

tags¶: tags, as a list

properties¶: properties, as a dict

links¶: links, as a list

Fixtures¶

lemoncheesecake.api.fixture(names=None, scope='test', per_thread=False)¶

Decorator, declare a function as a fixture.

Parameters:

names – an optional list of names that can be used to access the fixture value, if no names are provided the decorated function name will be used
scope – the fixture scope, available scopes are: test, suite, session, pre_run; default is test
per_thread –
whether or not the fixture must be executed on a per-thread basis Please note that when per_thread is set to True:
- the scope can only be session or suite
- the fixture can only be used in tests or by fixtures with the test scope

Logging¶

lemoncheesecake.api.set_step(description, detached=NotImplemented)¶

Set a new step.

Parameters:

description (str) – the step description
detached (bool) – argument deprecated since 1.4.5, does nothing.

Return type:

None

lemoncheesecake.api.detached_step(description)¶: Context manager deprecated since version 1.4.5, it only does a set_step.

lemoncheesecake.api.log_info(content)¶

Log a info level message.

Parameters:: content (str) –
Return type:: None

lemoncheesecake.api.log_warning(content)¶

Log a warning level message.

Parameters:: content (str) –
Return type:: None

lemoncheesecake.api.log_error(content)¶

Log an error level message.

Parameters:: content (str) –
Return type:: None

lemoncheesecake.api.log_url(url, description=None)¶

Log an URL.

Parameters:

url (str) –
description (str) –

lemoncheesecake.api.log_check(description, is_successful, details=None)¶

lemoncheesecake.api.prepare_attachment(filename, description=None)¶: Context manager. Prepare a attachment using a pseudo filename and an optional description. It returns the real filename on disk that will be used by the caller to write the attachment content.

lemoncheesecake.api.prepare_image_attachment(filename, description=None)¶: Context manager. Prepare an image attachment using a pseudo filename and an optional description. The function returns the real filename on disk that will be used by the caller to write the attachment content.

lemoncheesecake.api.save_attachment_content(content, filename, description=None)¶: Save a given content as attachment using pseudo filename and optional description.

lemoncheesecake.api.save_attachment_file(filename, description=None)¶: Save an attachment using an existing file (identified by filename) and an optional description. The given file will be copied.

lemoncheesecake.api.save_image_content(content, filename, description=None)¶: Save a given image content as attachment using pseudo filename and optional description.

lemoncheesecake.api.save_image_file(filename, description=None)¶: Save an image using an existing file (identified by filename) and an optional description. The given file will be copied.

Threading¶

class lemoncheesecake.api.Thread(*args, **kwargs)¶: Acts exactly as the standard threading.Thread class and must be used instead when running threads within a test.

class lemoncheesecake.api.ThreadedFactory¶

New in version 1.9.0.

This factory class returns a new object per thread when calling get_object().

This class works by subclassing and:

implementing the setup_object() is MANDATORY
implementing the teardown_object() is optional

NB: if the __init__ method is overridden then the base class __init__ method must be called.

get_object()¶

Get the object, it will be only created once per thread.

Return type:: Any

teardown_factory()¶

Teardown the factory.

This method must be called if teardown_object() has been implemented.

Return type:: None

setup_object()¶

Create the object. This method MUST be implemented.

Return type:: Any

teardown_object(obj)¶

Teardown an object. You can implement this method if your object needs a special teardown phase.

Parameters:: obj (Any) – an object created by setup_object()
Return type:: None

Matching¶

Operations¶

lemoncheesecake.matching.check_that(hint, actual, matcher, quiet=False)¶

Check that actual matches given matcher.

A check log is added to the report.

If quiet is set to True, the check details won’t appear in the check log.

Parameters:

hint (Optional[str]) –
actual (Any) –
matcher (Matcher) –
quiet (bool) –

Return type:

MatchResult

lemoncheesecake.matching.require_that(hint, actual, matcher, quiet=False)¶

Require that actual matches given matcher.

A check log is added to the report. An AbortTest exception is raised if the check does not succeed.

If quiet is set to True, the check details won’t appear in the check log.

Parameters:

hint (Optional[str]) –
actual (Any) –
matcher (Matcher) –
quiet (bool) –

Return type:

MatchResult

lemoncheesecake.matching.assert_that(hint, actual, matcher, quiet=False)¶

Assert that actual matches given matcher.

If assertion fail, a check log is added to the report and an AbortTest exception is raised.

If quiet is set to True, the check details won’t appear in the check log.

Parameters:

hint (Optional[str]) –
actual (Any) –
matcher (Matcher) –
quiet (bool) –

Return type:

MatchResult

lemoncheesecake.matching.check_that_in(actual, *expected, **kwargs)¶

Equivalent of check_that() over items of a dict.

Example of usage:

check_that_in(
    {"foo": 1, "bar": 2},
    "foo", equal_to(1),
    "bar", equal_to(2)
)

The key can also be a tuple when checking for a nested item:

check_that_in(
    {"foo": {"bar": 2}},
    ("foo", "bar"), equal_to(2)
)

The function can take a base_key keyword-arg to pass repeating nested-keys as a tuple.

If an extra quiet keyword-arg is set to True, the check details won’t appear in the check log.

The function returns a list of MatchResult.

New in version 1.11.0.

Nested data can also be checked this way:

check_that_in(
    {"foo": {"bar": 2}},
    {"foo": {"bar": equal_to(2)}}
)

It also means that instead of calling check_that_in with a list of key/matcher pairs, it can now be called with a data structure as the “expected” argument.

Parameters:: actual (Mapping) –
Return type:: Sequence[MatchResult]

lemoncheesecake.matching.require_that_in(actual, *expected, **kwargs)¶

Does the same thing as check_that_in() except it performs a require_that() on each key-matcher pair.

Parameters:: actual (Mapping) –
Return type:: Sequence[MatchResult]

lemoncheesecake.matching.assert_that_in(actual, *expected, **kwargs)¶

Does the same thing as check_that_in() except it performs a assert_that() on each key-matcher pair.

Parameters:: actual (Mapping) –
Return type:: Sequence[MatchResult]

Matchers¶

lemoncheesecake.matching.equal_to(expected)¶

Test if value is equal to expected

Parameters:: expected (Any) –
Return type:: Matcher

lemoncheesecake.matching.not_equal_to(expected)¶

Test if value is not equal to expected.

Parameters:: expected (Any) –
Return type:: Matcher

lemoncheesecake.matching.greater_than(expected)¶

Test if value is greater than expected.

Parameters:: expected (Any) –
Return type:: Matcher

lemoncheesecake.matching.greater_than_or_equal_to(expected)¶

Test if value is greater than or equal to expected.

Parameters:: expected (Any) –
Return type:: Matcher

lemoncheesecake.matching.less_than(expected)¶

Test if value is less than expected.

Parameters:: expected (Any) –
Return type:: Matcher

lemoncheesecake.matching.less_than_or_equal_to(expected)¶

Test if value is less than or equal to expected.

Parameters:: expected (Any) –
Return type:: Matcher

lemoncheesecake.matching.is_between(min, max)¶

Test if value is between min and max

Parameters:

min (Union[int, float]) –
max (Union[int, float]) –

Return type:

Matcher

lemoncheesecake.matching.is_none()¶

Test if value is None

Return type:: Matcher

lemoncheesecake.matching.is_not_none()¶

Test if value is not None

Return type:: Matcher

lemoncheesecake.matching.has_length(length)¶

Test if value has a length of

Parameters:: length (Union[int, Matcher]) –
Return type:: Matcher

lemoncheesecake.matching.is_true()¶

Test if value is true (boolean type)

Return type:: Matcher

lemoncheesecake.matching.is_false()¶

Test if value is false (boolean type)

Return type:: Matcher

lemoncheesecake.matching.is_json(expected)¶

Test if the two data structures (that can be represented as JSON) match.

If the two values do not match, the match result description will be a textual diff between the two JSON representations.

Parameters:: expected (Any) –
Return type:: Matcher

lemoncheesecake.matching.starts_with(expected)¶

Test if string begins with given prefix

Parameters:: expected (str) –
Return type:: Matcher

lemoncheesecake.matching.ends_with(expected)¶

Test if string ends with given suffix

Parameters:: expected (str) –
Return type:: Matcher

lemoncheesecake.matching.contains_string(expected)¶

Test if string contains sub string

Parameters:: expected (str) –
Return type:: Matcher

lemoncheesecake.matching.match_pattern(pattern, description=None, mention_regexp=False)¶

Test if string matches given pattern (using the search method of the re module)

Parameters:

pattern (Union[str, Pattern]) –
description (Optional[str]) –
mention_regexp (bool) –

Return type:

Matcher

lemoncheesecake.matching.is_text(expected, linesep='\n')¶

Test if the two multi-lines texts match.

If the two values do not match, the match result description will be a diff between the two texts.

Parameters:

expected (str) –
linesep (str) –

Return type:

Matcher

lemoncheesecake.matching.is_integer(expected=None)¶

Test if value is an integer.

Parameters:: expected (Optional[Union[int, Matcher]]) –
Return type:: Matcher

lemoncheesecake.matching.is_float(expected=None)¶

Test if value is a float.

Parameters:: expected (Optional[Union[float, Matcher]]) –
Return type:: Matcher

lemoncheesecake.matching.is_bool(expected=None)¶

Test if value is a boolean.

Parameters:: expected (Optional[Union[bool, Matcher]]) –
Return type:: Matcher

lemoncheesecake.matching.is_str(expected=None)¶

Test if value is a string.

Parameters:: expected (Optional[Union[str, Matcher]]) –
Return type:: Matcher

lemoncheesecake.matching.is_list(expected=None)¶

Test if value is a list.

Parameters:: expected (Optional[Union[list, tuple, Matcher]]) –
Return type:: Matcher

lemoncheesecake.matching.is_dict(expected=None)¶

Test if value is a dict (key/value collection).

Parameters:: expected (Optional[Union[dict, Matcher]]) –
Return type:: Matcher

lemoncheesecake.matching.has_item(expected)¶

Test if the sequence has item matching expected

Parameters:: expected (Any) –
Return type:: Matcher

lemoncheesecake.matching.has_items(values)¶

Test if the sequence contains at least the given values

Parameters:: values (Collection) –
Return type:: Matcher

lemoncheesecake.matching.has_only_items(expected)¶

Test if the sequence only contains the given values

Parameters:: expected (Collection) –
Return type:: Matcher

lemoncheesecake.matching.has_all_items(expected)¶

Test if all the items of the sequence match expected

Parameters:: expected (Any) –
Return type:: Matcher

lemoncheesecake.matching.is_in(expected)¶

Test if the sequence contains the expected item

Parameters:: expected (Collection) –
Return type:: Matcher

lemoncheesecake.matching.has_entry(key_matcher, value_matcher=None)¶

Test if dict has a <key> entry whose value matches (optional) value_matcher. Key entry can a standard dict key or a list of key where each element represent a level of depth of the dict (when dict are imbricated)

Parameters:: value_matcher (Optional[Matcher]) –
Return type:: Matcher

lemoncheesecake.matching.is_(matcher)¶

If the function argument is not an instance of Matcher, wrap it into a matcher using equal_to, otherwise return the matcher argument as-is

Parameters:: matcher (Any) –
Return type:: Matcher

lemoncheesecake.matching.not_(matcher)¶

Negates the matcher in argument

Parameters:: matcher (Any) –
Return type:: Matcher

lemoncheesecake.matching.is_not(matcher)¶

Negates the matcher in argument

Parameters:: matcher (Any) –
Return type:: Matcher

lemoncheesecake.matching.all_of(*matchers)¶

Test if all matchers match (logical AND between matchers).

Parameters:: matchers (Any) –
Return type:: Matcher

lemoncheesecake.matching.any_of(*matchers)¶

Test if at least one of the matcher match (logical OR between matchers)

Parameters:: matchers (Any) –
Return type:: Matcher

lemoncheesecake.matching.anything()¶

Matches anything (always succeed, whatever the actual value)

Return type:: Matcher

lemoncheesecake.matching.something()¶

Same thing as the ‘anything’ matcher but use ‘to be something’ in the matcher description

Return type:: Matcher

lemoncheesecake.matching.existing()¶

Same thing as the ‘anything’ matcher but use ‘to exist’ in the matcher description

Return type:: Matcher

lemoncheesecake.matching.present()¶

Same thing as the ‘anything’ matcher but use ‘to be present’ in the matcher description

Return type:: Matcher

Matcher¶

class lemoncheesecake.matching.matcher.Matcher¶

build_description(transformation)¶

Build a description for the matcher given the instance of MatcherDescriptionTransformer passed as argument.

Parameters:: transformation (MatcherDescriptionTransformer) –
Return type:: str

matches(actual)¶

Test if the passed argument matches.

Parameters:: actual (Any) – the actual value to match
Returns:: an instance of MatchResult
Return type:: MatchResult

override_description(description)¶

Override the matcher description.

Parameters:: description (str) – the new description
Returns:: a new matcher wrapping the actual matcher
Return type:: Matcher

hide_result_details()¶

Hide the matching operation result details.

Returns:: a new matcher wrapping the actual matcher
Return type:: Matcher

class lemoncheesecake.matching.matcher.MatchResult(is_successful, description=None)¶

Parameters:

is_successful (bool) –
description (str) –

is_successful: bool¶: whether or not the match did succeed

description: str¶: optional description

classmethod success(description=None)¶

Shortcut used to create a “successful” MatchResult.

Parameters:: description (Optional[str]) –
Return type:: MatchResult

classmethod failure(description=None)¶

Shortcut used to create a “failed” MatchResult.

Parameters:: description (Optional[str]) –
Return type:: MatchResult

__bool__()¶: Returns whether or not the match is successful.

class lemoncheesecake.matching.matcher.MatcherDescriptionTransformer(conjugate=False, negative=False)¶

This class is used as a callable and passed to Matcher.build_description() to transform the leading verb in description according to the transformer settings.

conjugate¶: indicate whether or not the verb in the description will be conjugated

negative¶: indicate whether or not the description will be turned into the negative form

__call__(description)¶

Transform the description according transformer settings.

Parameters:: description (str) –
Return type:: str

Project¶

This is the class that must be used / inherited in your `project.py` file¶

class lemoncheesecake.project.Project(project_dir=None)¶

dir: str¶: The project’s directory path (optional, defaults to the caller dir)

metadata_policy: MetadataPolicy¶: The project’s metadata policy

threaded: bool¶: Indicates whether or not the project supports parallel execution of tests

show_command_line_in_report: bool¶: Indicated whether or not the command line (“lcc run…”) will be displayed in the report

reporting_backends: Dict[str, ReportingBackend]¶: The reporting backends of the project as a dict (whose key is the reporting backend name)

default_reporting_backend_names¶: The list of default reporting backend (indicated by their name) that will be used by “lcc run”

add_cli_args(cli_parser)¶

Overridable. This method can be used to add extra CLI arguments to “lcc run”.

Parameters:: cli_parser (ArgumentParser) –
Return type:: None

create_report_dir()¶

Overridable. Create the report directory when no report directory is specified to “lcc run”.

Return type:: str

load_suites()¶

Overridable. Load the project’s suites.

Return type:: Sequence[Suite]

load_fixtures()¶

Overridable. Load the project’s fixtures.

Return type:: Sequence[Fixture]

pre_run(cli_args, report_dir)¶

Overridable. This hook is called before running the tests.

Parameters:

cli_args (Any) –
report_dir (str) –

Return type:

None

post_run(cli_args, report_dir)¶

Overridable. This hook is called after running the tests.

Parameters:

cli_args (Any) –
report_dir (str) –

Return type:

None

build_report_title()¶

Overridable. Build a custom report title as a string.

Return type:: Optional[str]

build_report_info()¶

Overridable. Build a list key/value pairs (expressed as a two items tuple) that will be available in the report.

Example:

[
    ("key1", "value1"),
    ("key2", "value2")
]

Return type:: List[Tuple[str, str]]

Loading suites¶

lemoncheesecake.suite.load_suites_from_directory(dir, recursive=True)¶

Load a list of suites from a directory.

If the recursive argument is set to True, sub suites will be searched in a directory named from the suite module: if the suite module is “foo.py” then the sub suites directory must be “foo”.

Raise SuiteLoadingError if one or more suite could not be loaded.

Parameters:

dir (str) –
recursive (bool) –

Return type:

List[Suite]

lemoncheesecake.suite.load_suites_from_files(patterns, excluding=())¶

Load a list of suites from a list of files.

Parameters:

patterns (Sequence[str]) – a mandatory list (a simple string can also be used instead of a single element list) of files to import; the wildcard ‘*’ character can be used
excluding (Sequence[str]) – an optional list (a simple string can also be used instead of a single element list) of elements to exclude from the expanded list of files to import

Return type:

List[Suite]

Example:

load_suites_from_files("test_*.py")

lemoncheesecake.suite.load_suite_from_file(filename)¶

Load a suite from a Python module indicated by a filename.

Raise SuiteLoadingError if the file cannot be loaded as a suite.

Parameters:: filename (str) –
Return type:: Suite

lemoncheesecake.suite.load_suite_from_module(mod)¶

Load a suite from a module instance.

Parameters:: mod (Any) –
Return type:: Suite

lemoncheesecake.suite.load_suites_from_classes(classes)¶

Load a list of suites from a list of classes.

Parameters:: classes (Sequence[Any]) –
Return type:: List[Suite]

lemoncheesecake.suite.load_suite_from_class(class_)¶

Load a suite from a class.

Parameters:: class_ (Any) –
Return type:: Suite

Loading fixtures¶

lemoncheesecake.fixture.load_fixtures_from_directory(dir)¶

Load fixtures from a given directory (not recursive).

Parameters:: dir (str) –
Return type:: List[Fixture]

lemoncheesecake.fixture.load_fixtures_from_files(patterns, excluding=[])¶

Load fixtures from files.

Parameters:

patterns (Any[str, Sequence[str]]) – a mandatory list (a simple string can also be used instead of a single element list) of files to import; the wildcard ‘*’ character can be used
excluding (Any[str, Sequence[str]]) – an optional list (a simple string can also be used instead of a single element list) of elements to exclude from the expanded list of files to import

Return type:

List[Fixture]

Example:

load_fixtures_from_files("test_*.py")

lemoncheesecake.fixture.load_fixtures_from_file(filename)¶

Load fixtures from a given file.

Parameters:: filename (str) –
Return type:: List[Fixture]

lemoncheesecake.fixture.load_fixtures_from_module(mod)¶

Load fixtures from a module instance.

New in version 1.5.1.

Parameters:: mod (Any) –
Return type:: List[Fixture]

lemoncheesecake.fixture.load_fixtures_from_func(func)¶

Load a fixture from a function that has been decorated with @lcc.fixture()

Parameters:: func (Callable) –
Return type:: List[Fixture]

Metadata Policy¶

class lemoncheesecake.metadatapolicy.MetadataPolicy¶

add_property_rule(prop_name, accepted_values=None, on_test=None, on_suite=None, required=False)¶

Declare a property rule.

Parameters:

prop_name – the property name
accepted_values – an optional list of accepted values
on_test – whether or not the property can be used on a test
on_suite – whether or not the property can be used on a suite
required – whether or not the property is required

If neither on_test or on_suite argument are set, then the property is only available for tests.

disallow_unknown_properties()¶: Disallow unknown properties for tests and suites.

add_tag_rule(tag_name, on_test=None, on_suite=None)¶

Declare a tag rule.

Parameters:

tag_name – the tag name
on_test – whether or not the tag can be used on a test
on_suite – whether or not the tag can be used on a suite

If neither on_test or on_suite argument are set, then the tag is only available for tests.

disallow_unknown_tags()¶: Disallow unknown tags for tests and suites.

Report¶

New in version 1.6.0.

The structure of a Report object is the following:

Report (Report)
    (0-1) test_session_setup (Result)
        (1-N) steps (Step)
            (1-N) step logs (StepLog)
    (1-N) suites (SuiteResult)
        (0-1) suite_setup (Result)
            (1-N) steps (Step)
                (1-N) step logs (StepLog)
        (N) tests (TestResult)
            (N) steps (Step)
                (1-N) step logs (StepLog)
        (N) sub-suites (SuiteResult)
            [suites can embed other sub-suite hierarchy]
        (0-1) suite_teardown (Result)
            (1-N) steps (Step)
                (1-N) step logs (StepLog)
    (0-1) test_session_teardown (Result)
        (1-N) steps (Step)
            (1-N) step logs (StepLog)

Step logs (whose base class is StepLog) are one of:
    - Log
    - Check
    - Attachment
    - Url

lemoncheesecake.reporting.load_report(path, backends=None)¶

Load report from a report directory or file.

Parameters:

path (str) –
backends (Optional[Sequence[ReportingBackend]]) –

Return type:

Report

class lemoncheesecake.reporting.Report¶

start_time: Optional[float]¶: The test run start time.

end_time: Optional[float]¶: The test run end time.

saving_time: Optional[float]¶: The report saving time.

title¶: The report title.

nb_threads¶: The number of threads used for the test run.

property test_session_setup: Optional[Result]¶: The session setup result if any.

property test_session_teardown: Optional[Result]¶: The session teardown result if any.

property duration: Optional[float]¶: The test run duration.

property nb_tests: int¶: The number of tests in the report.

property parallelized: bool¶: Whether or not the tests were parallelized.

add_info(name, value)¶

Add extra information (name, value) in the report.

Parameters:

name (str) –
value (str) –

Return type:

None

add_suite(suite)¶

Add suite result to the report.

Parameters:: suite (SuiteResult) –
Return type:: None

get_suites()¶

Get suite results.

Return type:: List[SuiteResult]

is_successful()¶

Return whether or not the test run is considered as successful.

Please note that every result is taken into account, including tests but also setups and teardowns.

Return type:: bool

all_suites()¶

An iterator over all suite results contained in the report (recursive).

Return type:: Iterator[SuiteResult]

all_tests()¶

An iterator over all test results contained in the report.

Return type:: Iterator[TestResult]

all_results()¶

An iterator over all results (tests, setups, teardowns) contained in the report.

Return type:: Iterator[Result]

all_steps()¶

An iterator over all steps contained in the report.

Return type:: Iterator[Step]

build_message(template)¶

Build a message from a template that contains variable placeholders.

Example: with template “Test results: {passed}/{enabled} passed ({passed_pct})” the method will return for instance “Test results: 1/2 passed (50%)”.

The following variables are available:

start_time: the test run start time
end_time: the test run end time
duration: the test run duration
total: the total number of tests (including disabled tests)
enabled: the total number of tests (excluding disabled tests)
passed: the number of passed tests
passed_pct: the number of passed tests in percentage of enabled tests
failed: the number of failed tests
failed_pct: the number of failed tests in percentage of enabled tests
skipped: the number of skipped tests
skipped_pct: the number of skipped tests in percentage of enabled tests
disabled: the number of disabled tests
disabled_pct: the number of disabled tests in percentage of all tests

Parameters:: template (str) –
Return type:: str

save()¶: Save the report.

class lemoncheesecake.reporting.SuiteResult(name, description)¶

Contains the results of tests and sub-suites within the suite.

Variables:

~.name (str) – test name
~.description (str) – test description
~.tags (list) – test tags
~.properties (dict) – test properties, as a dict
~.links (dict) – test links, as a list of tuples ((url, description) where description can be None)

Parameters:

name (str) –
description (str) –

start_time: Optional[float]¶: The suite start time.

end_time: Optional[float]¶: The suite end time.

property suite_setup: Result¶: The suite setup result (if any).

property suite_teardown: Result¶: The suite teardown result (if any).

property duration: Optional[float]¶: The suite duration, which is the addition of all results contained in the suite and its sub-suite (recursively).

get_tests()¶

Return the tests contained in the suite.

Return type:: List[TestResult]

get_suites()¶

Return the sub-suites contained in the suite.

Return type:: List[SuiteResult]

property path: str¶: The complete path of the test node (example: if used on a test named “my_test” and a suite named “my_suite”, then the path is “my_suite.my_test”).

add_test(test)¶: Add test to the suite.

add_suite(suite)¶: Add a sub-suite to the suite.

class lemoncheesecake.reporting.Result¶

Holds the result of setup/teardown phase and it’s also the base class of TestResult.

STATUSES = ('passed', 'failed', 'skipped', 'disabled')¶: possible status values for a Result instance

parent_suite: Optional[SuiteResult]¶: Parent suite.

type: Optional[str]¶: Result type (it is one of the following: “test_session_setup”, “test_session_teardown”, “suite_setup”, “suite_teardown”, “test”).

start_time: Optional[float]¶: Result start time.

end_time: Optional[float]¶: Result end time.

status: Optional[str]¶: Result status (one of Result.STATUSES or None if the result is not yet complete).

status_details: Optional[str]¶: Result status details, if any.

add_step(step)¶

Add step to the result.

Parameters:: step (Step) –
Return type:: None

get_steps()¶

Get steps.

Return type:: List[Step]

is_successful()¶

Return whether or not the result is successful (even if the result is not yet complete).

Return type:: bool

property duration: Optional[float]¶: Return the duration as a float or None if the result is not yet complete.

class lemoncheesecake.reporting.TestResult(name, description)¶

Holds the result of a test. Inherits Result.

Variables:

~.name (str) – test name
~.description (str) – test description
~.tags (list) – test tags
~.properties (dict) – test properties, as a dict
~.links (dict) – test links, as a list of tuples ((url, description) where description can be None)

Parameters:

name (str) –
description (str) –

property path: str¶: The complete path of the test node (example: if used on a test named “my_test” and a suite named “my_suite”, then the path is “my_suite.my_test”).

class lemoncheesecake.reporting.Step(description)¶

This class holds logs occurring within a step.

Parameters:: description (str) –

description: str¶: Step description.

start_time: Optional[float]¶: Step start time.

end_time: Optional[float]¶: Step end time.

add_log(log)¶

Add a log to the step.

Parameters:: log (StepLog) –
Return type:: None

get_logs()¶

Get step logs.

Return type:: List[StepLog]

is_successful()¶

Return whether or not (as a boolean) the step is successful.

Return type:: bool

property duration: Optional[float]¶: Return the duration as a float or None if the step is not yet complete.

class lemoncheesecake.reporting.StepLog(ts)¶

Base class for logs contained in a Step instance.

Parameters:: ts (float) –

time: float¶: Log time.

parent_step: Optional[Step]¶: Parent step.

class lemoncheesecake.reporting.Log(level, message, ts)¶

The log resulting of log_info/log_debug/log_warning/log_error functions. Inherits StepLog.

Parameters:

level (str) –
message (str) –
ts (float) –

level: str¶: Log level.

message: str¶: Log message.

class lemoncheesecake.reporting.Check(description, is_successful, details, ts)¶

The log resulting of a check_that/require_that/assert_that functions. Inherits StepLog.

Parameters:

description (str) –
is_successful (bool) –
details (Optional[str]) –
ts (float) –

description: str¶: Check description.

is_successful: bool¶: Whether the check is successful or not (boolean).

details: str¶: Optional check details.

class lemoncheesecake.reporting.Attachment(description, filename, as_image, ts)¶

The log resulting of save/prepare_*attachment* functions. Inherits StepLog.

Parameters:

description (str) –
filename (str) –
as_image (bool) –
ts (float) –

description: str¶: Attachment description.

filename: str¶: Attachment filename, this is a path relative to the report directory.

as_image: bool¶: Whether or not the attachment should be interpreted as an image (boolean).

class lemoncheesecake.reporting.Url(description, url, ts)¶

The log resulting of log_url function. Inherits StepLog.

Parameters:

description (str) –
url (str) –
ts (float) –

description: str¶: Optional description.

url: str¶: Actual url.

Exceptions¶

lemoncheesecake.exceptions.AbortTest(*args)¶: Raising this exception will stop the currently running test.

lemoncheesecake.exceptions.AbortSuite(*args)¶: Raising this exception will stop the currently running suite.

lemoncheesecake.exceptions.AbortAllTests(*args)¶: Raising this exception will stop the currently running test and all the tests waiting to be run.

lemoncheesecake.exceptions.UserError(*args)¶: This exception is intended to be raised in pre-run and post-run phases of the project to indicate that a required state has not been fulfilled.

API reference¶

API compatibility / stability¶

Tests and suites¶

Fixtures¶

Logging¶

Threading¶

Matching¶

Operations¶

Matchers¶

Matcher¶

Project¶

This is the class that must be used / inherited in your `project.py` file¶

Loading suites¶

Loading fixtures¶

Metadata Policy¶

Report¶

Exceptions¶

Related Topics

Navigation

Links

API reference¶

API compatibility / stability¶

Tests and suites¶

Fixtures¶

Logging¶

Threading¶

Matching¶

Operations¶

Matchers¶

Matcher¶

Project¶

This is the class that must be used / inherited in your project.py file¶

Loading suites¶

Loading fixtures¶

Metadata Policy¶

Report¶

Exceptions¶

This is the class that must be used / inherited in your `project.py` file¶