API reference¶
API compatibility / stability¶
lemoncheesecake follows the well know Semantic Versioning for its public API. What is considered as “public” is everything which is documented on http://docs.lemoncheesecake.io/. Everything else is internal and is subject to change at anytime.
Tests and suites¶
- lemoncheesecake.api.suite(description=None, name=None, rank=None)¶
Decorator, mark a class as a suite class.
- Parameters:
description – suite’s description (by default, the suite’s description is built from the name)
name – suite’s name (by default, the suite’s name is taken from the class’s name)
rank – this value is used to order suites of the same hierarchy level
- lemoncheesecake.api.test(description=None, name=None)¶
Decorator, mark a function/method as a test.
- Parameters:
description – test’s description
name – test’s name (by default, the suite’s name is taken from the class’s name)
- lemoncheesecake.api.tags(*tag_names)¶
Decorator, add tags to a test or a suite.
- lemoncheesecake.api.prop(key, value)¶
Decorator, add a property (key/value) to a test or a suite.
- lemoncheesecake.api.link(url, name=None)¶
Decorator, set a link (with an optional friendly name) to a test or a suite.
- lemoncheesecake.api.disabled(reason=None)¶
Decorator, mark a test or a suite as disabled, meaning it won’t be executed but will be visible in report. An optional reason can be passed to the decorator (new in version 1.1.0).
- lemoncheesecake.api.visible_if(condition)¶
Decorator, the test or suite will only appear if the given callable return a true value.
- Parameters:
condition – a callable that will take the test object if applied to a test or the suite class instance if applied to a suite.
Decorator, the test or suite won’t be visible in the resulting test tree.
- lemoncheesecake.api.depends_on(*deps)¶
Decorator, only applicable to a test. Add dependencies to a test.
- Parameters:
dependencies can be either:
a full test path:
@lcc.depends("mysuite.mytest")
a callable that express dependencies as a filter (a
Test
instance is passed to the callable):@lcc.depends(lambda test: "mytag" in test.tags)
New in version 1.14.1: callable syntax
- Return type:
- lemoncheesecake.api.parametrized(parameter_source, naming_scheme=<function _default_naming_scheme>)¶
Decorator, make the test parametrized.
- Parameters:
parameter_source (Iterable) –
it can be either:
an iterable of dicts, each dict representing the arguments passed to the test
a CSV-like mode, where the first element of the list represents the argument names as an str or a sequence (example:
"i,j"
or("i", "j")
) and the remaining elements are sequences of the arguments to be passed to the test.
naming_scheme (Optional[Union[Callable[[str, str, dict, int], Tuple[str, str]], Sequence]]) –
optional, it can be either:
a optional function that takes as parameters the base test name, description, parameters, index and must return the expanded test name and description as a two elements list
a tuple/list of two (name + description) format strings, example:
("test_{i}_plus_{j}", "Test {i} plus {j}")
- Return type:
- lemoncheesecake.api.inject_fixture(fixture_name=None)¶
Inject a fixture into a suite. If no fixture name is specified then the name of the variable holding the injected fixture will be used.
- lemoncheesecake.api.add_test_into_suite(test, suite)¶
Add test into suite
- class lemoncheesecake.api.Test(name, description, callback)¶
Internal representation of a test.
- property path: str¶
The complete path of the test node (example: if used on a test named “my_test” and a suite named “my_suite”, then the path is “my_suite.my_test”).
- name¶
name
- description¶
description
- tags¶
tags, as a list
- properties¶
properties, as a dict
- links¶
links, as a list
Fixtures¶
- lemoncheesecake.api.fixture(names=None, scope='test', per_thread=False)¶
Decorator, declare a function as a fixture.
- Parameters:
names – an optional list of names that can be used to access the fixture value, if no names are provided the decorated function name will be used
scope – the fixture scope, available scopes are:
test
,suite
,session
,pre_run
; default istest
per_thread –
whether or not the fixture must be executed on a per-thread basis Please note that when
per_thread
is set toTrue
:the scope can only be
session
orsuite
the fixture can only be used in tests or by fixtures with the
test
scope
Logging¶
- lemoncheesecake.api.set_step(description, detached=NotImplemented)¶
Set a new step.
- lemoncheesecake.api.detached_step(description)¶
Context manager deprecated since version 1.4.5, it only does a set_step.
- lemoncheesecake.api.log_info(content)¶
Log a info level message.
- Parameters:
content (str) –
- Return type:
None
- lemoncheesecake.api.log_warning(content)¶
Log a warning level message.
- Parameters:
content (str) –
- Return type:
None
- lemoncheesecake.api.log_error(content)¶
Log an error level message.
- Parameters:
content (str) –
- Return type:
None
- lemoncheesecake.api.log_url(url, description=None)¶
Log an URL.
- lemoncheesecake.api.log_check(description, is_successful, details=None)¶
- lemoncheesecake.api.prepare_attachment(filename, description=None)¶
Context manager. Prepare a attachment using a pseudo filename and an optional description. It returns the real filename on disk that will be used by the caller to write the attachment content.
- lemoncheesecake.api.prepare_image_attachment(filename, description=None)¶
Context manager. Prepare an image attachment using a pseudo filename and an optional description. The function returns the real filename on disk that will be used by the caller to write the attachment content.
- lemoncheesecake.api.save_attachment_content(content, filename, description=None)¶
Save a given content as attachment using pseudo filename and optional description.
- lemoncheesecake.api.save_attachment_file(filename, description=None)¶
Save an attachment using an existing file (identified by filename) and an optional description. The given file will be copied.
- lemoncheesecake.api.save_image_content(content, filename, description=None)¶
Save a given image content as attachment using pseudo filename and optional description.
- lemoncheesecake.api.save_image_file(filename, description=None)¶
Save an image using an existing file (identified by filename) and an optional description. The given file will be copied.
Threading¶
- class lemoncheesecake.api.Thread(*args, **kwargs)¶
Acts exactly as the standard threading.Thread class and must be used instead when running threads within a test.
- class lemoncheesecake.api.ThreadedFactory¶
New in version 1.9.0.
This factory class returns a new object per thread when calling
get_object()
.This class works by subclassing and:
implementing the
setup_object()
is MANDATORYimplementing the
teardown_object()
is optional
NB: if the
__init__
method is overridden then the base class__init__
method must be called.- teardown_factory()¶
Teardown the factory.
This method must be called if
teardown_object()
has been implemented.- Return type:
None
- teardown_object(obj)¶
Teardown an object. You can implement this method if your object needs a special teardown phase.
- Parameters:
obj (Any) – an object created by
setup_object()
- Return type:
None
Matching¶
Operations¶
- lemoncheesecake.matching.check_that(hint, actual, matcher, quiet=False)¶
Check that actual matches given matcher.
A check log is added to the report.
If
quiet
is set toTrue
, the check details won’t appear in the check log.
- lemoncheesecake.matching.require_that(hint, actual, matcher, quiet=False)¶
Require that actual matches given matcher.
A check log is added to the report. An AbortTest exception is raised if the check does not succeed.
If
quiet
is set toTrue
, the check details won’t appear in the check log.
- lemoncheesecake.matching.assert_that(hint, actual, matcher, quiet=False)¶
Assert that actual matches given matcher.
If assertion fail, a check log is added to the report and an AbortTest exception is raised.
If
quiet
is set toTrue
, the check details won’t appear in the check log.
- lemoncheesecake.matching.check_that_in(actual, *expected, **kwargs)¶
Equivalent of
check_that()
over items of a dict.Example of usage:
check_that_in( {"foo": 1, "bar": 2}, "foo", equal_to(1), "bar", equal_to(2) )
The key can also be a
tuple
when checking for a nested item:check_that_in( {"foo": {"bar": 2}}, ("foo", "bar"), equal_to(2) )
The function can take a
base_key
keyword-arg to pass repeating nested-keys as atuple
.If an extra
quiet
keyword-arg is set toTrue
, the check details won’t appear in the check log.The function returns a list of
MatchResult
.New in version 1.11.0.
Nested data can also be checked this way:
check_that_in( {"foo": {"bar": 2}}, {"foo": {"bar": equal_to(2)}} )
It also means that instead of calling
check_that_in
with a list of key/matcher pairs, it can now be called with a data structure as the “expected” argument.- Parameters:
actual (Mapping) –
- Return type:
- lemoncheesecake.matching.require_that_in(actual, *expected, **kwargs)¶
Does the same thing as
check_that_in()
except it performs arequire_that()
on each key-matcher pair.- Parameters:
actual (Mapping) –
- Return type:
- lemoncheesecake.matching.assert_that_in(actual, *expected, **kwargs)¶
Does the same thing as
check_that_in()
except it performs aassert_that()
on each key-matcher pair.- Parameters:
actual (Mapping) –
- Return type:
Matchers¶
- lemoncheesecake.matching.equal_to(expected)¶
Test if value is equal to expected
- lemoncheesecake.matching.not_equal_to(expected)¶
Test if value is not equal to expected.
- lemoncheesecake.matching.greater_than(expected)¶
Test if value is greater than expected.
- lemoncheesecake.matching.greater_than_or_equal_to(expected)¶
Test if value is greater than or equal to expected.
- lemoncheesecake.matching.less_than(expected)¶
Test if value is less than expected.
- lemoncheesecake.matching.less_than_or_equal_to(expected)¶
Test if value is less than or equal to expected.
- lemoncheesecake.matching.is_between(min, max)¶
Test if value is between min and max
- lemoncheesecake.matching.has_length(length)¶
Test if value has a length of
- lemoncheesecake.matching.is_json(expected)¶
Test if the two data structures (that can be represented as JSON) match.
If the two values do not match, the match result description will be a textual diff between the two JSON representations.
- lemoncheesecake.matching.starts_with(expected)¶
Test if string begins with given prefix
- lemoncheesecake.matching.ends_with(expected)¶
Test if string ends with given suffix
- lemoncheesecake.matching.contains_string(expected)¶
Test if string contains sub string
- lemoncheesecake.matching.match_pattern(pattern, description=None, mention_regexp=False)¶
Test if string matches given pattern (using the search method of the re module)
- lemoncheesecake.matching.is_text(expected, linesep='\n')¶
Test if the two multi-lines texts match.
If the two values do not match, the match result description will be a diff between the two texts.
- lemoncheesecake.matching.is_integer(expected=None)¶
Test if value is an integer.
- lemoncheesecake.matching.is_float(expected=None)¶
Test if value is a float.
- lemoncheesecake.matching.is_bool(expected=None)¶
Test if value is a boolean.
- lemoncheesecake.matching.is_str(expected=None)¶
Test if value is a string.
- lemoncheesecake.matching.is_list(expected=None)¶
Test if value is a list.
- lemoncheesecake.matching.is_dict(expected=None)¶
Test if value is a dict (key/value collection).
- lemoncheesecake.matching.has_item(expected)¶
Test if the sequence has item matching expected
- lemoncheesecake.matching.has_items(values)¶
Test if the sequence contains at least the given values
- Parameters:
values (Collection) –
- Return type:
- lemoncheesecake.matching.has_only_items(expected)¶
Test if the sequence only contains the given values
- Parameters:
expected (Collection) –
- Return type:
- lemoncheesecake.matching.has_all_items(expected)¶
Test if all the items of the sequence match expected
- lemoncheesecake.matching.is_in(expected)¶
Test if the sequence contains the expected item
- Parameters:
expected (Collection) –
- Return type:
- lemoncheesecake.matching.has_entry(key_matcher, value_matcher=None)¶
Test if dict has a <key> entry whose value matches (optional) value_matcher. Key entry can a standard dict key or a list of key where each element represent a level of depth of the dict (when dict are imbricated)
- lemoncheesecake.matching.is_(matcher)¶
If the function argument is not an instance of Matcher, wrap it into a matcher using equal_to, otherwise return the matcher argument as-is
- lemoncheesecake.matching.not_(matcher)¶
Negates the matcher in argument
- lemoncheesecake.matching.is_not(matcher)¶
Negates the matcher in argument
- lemoncheesecake.matching.all_of(*matchers)¶
Test if all matchers match (logical AND between matchers).
- lemoncheesecake.matching.any_of(*matchers)¶
Test if at least one of the matcher match (logical OR between matchers)
- lemoncheesecake.matching.anything()¶
Matches anything (always succeed, whatever the actual value)
- Return type:
- lemoncheesecake.matching.something()¶
Same thing as the ‘anything’ matcher but use ‘to be something’ in the matcher description
- Return type:
- lemoncheesecake.matching.existing()¶
Same thing as the ‘anything’ matcher but use ‘to exist’ in the matcher description
- Return type:
Matcher¶
- class lemoncheesecake.matching.matcher.Matcher¶
- build_description(transformation)¶
Build a description for the matcher given the instance of
MatcherDescriptionTransformer
passed as argument.- Parameters:
transformation (MatcherDescriptionTransformer) –
- Return type:
- matches(actual)¶
Test if the passed argument matches.
- Parameters:
actual (Any) – the actual value to match
- Returns:
an instance of
MatchResult
- Return type:
- override_description(description)¶
Override the matcher description.
- class lemoncheesecake.matching.matcher.MatchResult(is_successful, description=None)¶
-
- classmethod success(description=None)¶
Shortcut used to create a “successful” MatchResult.
- Parameters:
- Return type:
- classmethod failure(description=None)¶
Shortcut used to create a “failed” MatchResult.
- Parameters:
- Return type:
- __bool__()¶
Returns whether or not the match is successful.
- class lemoncheesecake.matching.matcher.MatcherDescriptionTransformer(conjugate=False, negative=False)¶
This class is used as a callable and passed to
Matcher.build_description()
to transform the leading verb in description according to the transformer settings.- conjugate¶
indicate whether or not the verb in the description will be conjugated
- negative¶
indicate whether or not the description will be turned into the negative form
Project¶
This is the class that must be used / inherited in your project.py
file¶
- class lemoncheesecake.project.Project(project_dir=None)¶
- metadata_policy: MetadataPolicy¶
The project’s metadata policy
- show_command_line_in_report: bool¶
Indicated whether or not the command line (“lcc run…”) will be displayed in the report
- reporting_backends: Dict[str, ReportingBackend]¶
The reporting backends of the project as a dict (whose key is the reporting backend name)
- default_reporting_backend_names¶
The list of default reporting backend (indicated by their name) that will be used by “lcc run”
- add_cli_args(cli_parser)¶
Overridable. This method can be used to add extra CLI arguments to “lcc run”.
- Parameters:
cli_parser (ArgumentParser) –
- Return type:
None
- create_report_dir()¶
Overridable. Create the report directory when no report directory is specified to “lcc run”.
- Return type:
- pre_run(cli_args, report_dir)¶
Overridable. This hook is called before running the tests.
- post_run(cli_args, report_dir)¶
Overridable. This hook is called after running the tests.
- build_report_title()¶
Overridable. Build a custom report title as a string.
Loading suites¶
- lemoncheesecake.suite.load_suites_from_directory(dir, recursive=True)¶
Load a list of suites from a directory.
If the recursive argument is set to True, sub suites will be searched in a directory named from the suite module: if the suite module is “foo.py” then the sub suites directory must be “foo”.
Raise SuiteLoadingError if one or more suite could not be loaded.
- lemoncheesecake.suite.load_suites_from_files(patterns, excluding=())¶
Load a list of suites from a list of files.
- Parameters:
patterns (Sequence[str]) – a mandatory list (a simple string can also be used instead of a single element list) of files to import; the wildcard ‘*’ character can be used
excluding (Sequence[str]) – an optional list (a simple string can also be used instead of a single element list) of elements to exclude from the expanded list of files to import
- Return type:
List[Suite]
Example:
load_suites_from_files("test_*.py")
- lemoncheesecake.suite.load_suite_from_file(filename)¶
Load a suite from a Python module indicated by a filename.
Raise SuiteLoadingError if the file cannot be loaded as a suite.
- Parameters:
filename (str) –
- Return type:
Suite
- lemoncheesecake.suite.load_suite_from_module(mod)¶
Load a suite from a module instance.
- Parameters:
mod (Any) –
- Return type:
Suite
- lemoncheesecake.suite.load_suites_from_classes(classes)¶
Load a list of suites from a list of classes.
Loading fixtures¶
- lemoncheesecake.fixture.load_fixtures_from_directory(dir)¶
Load fixtures from a given directory (not recursive).
- lemoncheesecake.fixture.load_fixtures_from_files(patterns, excluding=[])¶
Load fixtures from files.
- Parameters:
patterns (Any[str, Sequence[str]]) – a mandatory list (a simple string can also be used instead of a single element list) of files to import; the wildcard ‘*’ character can be used
excluding (Any[str, Sequence[str]]) – an optional list (a simple string can also be used instead of a single element list) of elements to exclude from the expanded list of files to import
- Return type:
List[Fixture]
Example:
load_fixtures_from_files("test_*.py")
- lemoncheesecake.fixture.load_fixtures_from_file(filename)¶
Load fixtures from a given file.
- lemoncheesecake.fixture.load_fixtures_from_module(mod)¶
Load fixtures from a module instance.
New in version 1.5.1.
Metadata Policy¶
- class lemoncheesecake.metadatapolicy.MetadataPolicy¶
- add_property_rule(prop_name, accepted_values=None, on_test=None, on_suite=None, required=False)¶
Declare a property rule.
- Parameters:
prop_name – the property name
accepted_values – an optional list of accepted values
on_test – whether or not the property can be used on a test
on_suite – whether or not the property can be used on a suite
required – whether or not the property is required
If neither on_test or on_suite argument are set, then the property is only available for tests.
- disallow_unknown_properties()¶
Disallow unknown properties for tests and suites.
- add_tag_rule(tag_name, on_test=None, on_suite=None)¶
Declare a tag rule.
- Parameters:
tag_name – the tag name
on_test – whether or not the tag can be used on a test
on_suite – whether or not the tag can be used on a suite
If neither on_test or on_suite argument are set, then the tag is only available for tests.
- disallow_unknown_tags()¶
Disallow unknown tags for tests and suites.
Report¶
New in version 1.6.0.
The structure of a Report object is the following:
Report (Report
) (0-1) test_session_setup (Result
) (1-N) steps (Step
) (1-N) step logs (StepLog
) (1-N) suites (SuiteResult
) (0-1) suite_setup (Result
) (1-N) steps (Step
) (1-N) step logs (StepLog
) (N) tests (TestResult
) (N) steps (Step
) (1-N) step logs (StepLog
) (N) sub-suites (SuiteResult
) [suites can embed other sub-suite hierarchy] (0-1) suite_teardown (Result
) (1-N) steps (Step
) (1-N) step logs (StepLog
) (0-1) test_session_teardown (Result
) (1-N) steps (Step
) (1-N) step logs (StepLog
) Step logs (whose base class isStepLog
) are one of: -Log
-Check
-Attachment
-Url
- lemoncheesecake.reporting.load_report(path, backends=None)¶
Load report from a report directory or file.
- class lemoncheesecake.reporting.Report¶
- title¶
The report title.
- nb_threads¶
The number of threads used for the test run.
- add_info(name, value)¶
Add extra information (name, value) in the report.
- add_suite(suite)¶
Add suite result to the report.
- Parameters:
suite (SuiteResult) –
- Return type:
None
- get_suites()¶
Get suite results.
- Return type:
- is_successful()¶
Return whether or not the test run is considered as successful.
Please note that every result is taken into account, including tests but also setups and teardowns.
- Return type:
- all_suites()¶
An iterator over all suite results contained in the report (recursive).
- Return type:
- all_tests()¶
An iterator over all test results contained in the report.
- Return type:
- all_results()¶
An iterator over all results (tests, setups, teardowns) contained in the report.
- build_message(template)¶
Build a message from a template that contains variable placeholders.
Example: with template “Test results: {passed}/{enabled} passed ({passed_pct})” the method will return for instance “Test results: 1/2 passed (50%)”.
The following variables are available:
start_time
: the test run start timeend_time
: the test run end timeduration
: the test run durationtotal
: the total number of tests (including disabled tests)enabled
: the total number of tests (excluding disabled tests)passed
: the number of passed testspassed_pct
: the number of passed tests in percentage of enabled testsfailed
: the number of failed testsfailed_pct
: the number of failed tests in percentage of enabled testsskipped
: the number of skipped testsskipped_pct
: the number of skipped tests in percentage of enabled testsdisabled
: the number of disabled testsdisabled_pct
: the number of disabled tests in percentage of all tests
- save()¶
Save the report.
- class lemoncheesecake.reporting.SuiteResult(name, description)¶
Contains the results of tests and sub-suites within the suite.
- Variables:
- Parameters:
- property duration: Optional[float]¶
The suite duration, which is the addition of all results contained in the suite and its sub-suite (recursively).
- get_tests()¶
Return the tests contained in the suite.
- Return type:
- get_suites()¶
Return the sub-suites contained in the suite.
- Return type:
- property path: str¶
The complete path of the test node (example: if used on a test named “my_test” and a suite named “my_suite”, then the path is “my_suite.my_test”).
- add_test(test)¶
Add test to the suite.
- add_suite(suite)¶
Add a sub-suite to the suite.
- class lemoncheesecake.reporting.Result¶
Holds the result of setup/teardown phase and it’s also the base class of
TestResult
.- STATUSES = ('passed', 'failed', 'skipped', 'disabled')¶
possible status values for a Result instance
- parent_suite: Optional[SuiteResult]¶
Parent suite.
- type: Optional[str]¶
Result type (it is one of the following: “test_session_setup”, “test_session_teardown”, “suite_setup”, “suite_teardown”, “test”).
- status: Optional[str]¶
Result status (one of Result.STATUSES or None if the result is not yet complete).
- is_successful()¶
Return whether or not the result is successful (even if the result is not yet complete).
- Return type:
- class lemoncheesecake.reporting.TestResult(name, description)¶
Holds the result of a test. Inherits
Result
.- Variables:
- Parameters:
- class lemoncheesecake.reporting.Step(description)¶
This class holds logs occurring within a step.
- Parameters:
description (str) –
- class lemoncheesecake.reporting.StepLog(ts)¶
Base class for logs contained in a Step instance.
- Parameters:
ts (float) –
- class lemoncheesecake.reporting.Log(level, message, ts)¶
The log resulting of log_info/log_debug/log_warning/log_error functions. Inherits
StepLog
.
- class lemoncheesecake.reporting.Check(description, is_successful, details, ts)¶
The log resulting of a check_that/require_that/assert_that functions. Inherits
StepLog
.
Exceptions¶
- lemoncheesecake.exceptions.AbortTest(*args)¶
Raising this exception will stop the currently running test.
- lemoncheesecake.exceptions.AbortSuite(*args)¶
Raising this exception will stop the currently running suite.
- lemoncheesecake.exceptions.AbortAllTests(*args)¶
Raising this exception will stop the currently running test and all the tests waiting to be run.
- lemoncheesecake.exceptions.UserError(*args)¶
This exception is intended to be raised in pre-run and post-run phases of the project to indicate that a required state has not been fulfilled.