Skip to content

Configure and Run your first test

Est Shizard suggests using YAML format to describe the test configuration.

Note

Please note .yaml and .yml are both valid extensions for configuration files. The tool has no restrictions on file naming at all. The file can have any extension, but it must contain valid YAML syntax. It is recommended to use the .yaml extension.

Introduction to YAML#

If your don't familiar with YAML syntax, please consider reading a quick start guide. In any case, below is the basic information you need to start using the YAML in the Est Shizard.

YAML is a way to describe information using a simple and understandable syntax. It consists of esho main data types: maps and sequences.

Maps are used to represent structured data as key-value pairs. Keys serve as identifiers, and values contain the associated information. For example:

project:
  name: awesome
  version: 1.2.2
  repository: hub/awesome

In Python, this will match the next object {'project': {'name': 'awesome', 'version': 1.2.2, 'repository': 'hub/awesome'}}.

In this example, we have a map with the key "project", and its value is another map with keys "name", "version", and "repository". Each key has its corresponding value.

Sequences, on the other hand, are used to represent ordered lists of data. They are denoted by a hyphen before each list item. For example:

fruits:
  - apple
  - banana
  - orange

In Python, this will match the next object {fruits: ['apple', 'banana', 'orange']}.

In this example, we have a sequence with the key "fruits". It contains three elements: "apple", "banana", and "orange". The elements are ordered and can be accessed in the order they are specified.

Both maps and sequences can be nested within each other to create more complex data structures. They can also contain other data types, such as strings, numbers, booleans, etc. Thus, with the help of maps and sequences in YAML, we can organize and store structured data in a convenient and understandable format.

Another important thing to note in YAML is the use of indentation. It is crucial for defining the hierarchy and structure of the data. Indentation is done using spaces or tabs, but it must be consistent throughout the file.

Note

YAML doesn’t allow literal tab characters for indentation. Use only spaces.

There is one more thing that is useful to know, you can use an extra hyphen to nest a sequence. So the following syntax is valid:

fruits:
- apple
- banana
- - nested_in_banana # nested sequence
- orange

In Python, this will match the next object {fruits: ['apple', 'banana', ['nested_in_banana'], 'orange']}.

Writing YAML tests configuration#

Configure a test#

A test case is a set of instructions determining whether a something behaves as expected. In the minimum required case, the test can be defined in Est Shizard configuration as

- test: test echo
  run: echo "Hello World!"

The test field defines a name of the test. The run field defines a string that the test will run.

Configure a suite#

A test suite is a collection of test cases that are grouped for test execution purposes. It collects individual test cases based on their specific purpose or characteristics. For example we want to run esho tests in the testing echo suite:

- suite: testing echo
  tests:
    - test: test_hello
      run: echo "Hello"
    - test: test_world
      run: echo "World!"

The suite field defines a name of the suite and the tests field defines a collection of tests.

Est Shizard provides provides various ways to customize your tests. For example you can define environment variables for a test, set a maximum number of times failed tests can be re-run, or limit the parallel execution of tests within in a suite.

For more information about writing YAML configuration, see Configuration keyword reference.

Running tests#

To see Est Shizard at work, let's create a simple test config(for example, in the tests.yaml file):

max_num_workers: 2
tests:
  - suite: test world with options
    env:
      OPTS: good old
    tests:
      - test: test world with flags
        run: echo "test $OPTS world"
      - test: test a new world
        run: echo "test $OPTS world"
        env:
          OPTS: new
        description: new beautiful world

Now, to start the test execution, simply call:

est-shizard run tests.yaml

For more information about run command, see Command-Line Reference.