3. The Command-Line Test Runner

    When invoked as shown above, the PHPUnit command-line test runner will look for a ArrayTest.php sourcefile in the current working directory, load it, and expect to find a ArrayTest test case class. It will then execute the tests of that class.

    For each test run, the PHPUnit command-line tool prints one character to indicate progress:

    .

    F

    Printed when an assertion fails while running the test method.

    E

    Printed when an error occurs while running the test method.

    R

    Printed when the test has been marked as risky (see Risky Tests).

    S

    Printed when the test has been skipped (see ).

    I

    Printed when the test is marked as being incomplete or not yet implemented (see Incomplete and Skipped Tests).

    PHPUnit distinguishes between failures and errors. A failure is a violated PHPUnit assertion such as a failing assertSame() call. An error is an unexpected exception or a PHP error. Sometimes this distinction proves useful since errors tend to be easier to fix than failures. If you have a big list of problems, it is best to tackle the errors first and see if you have any failures left when they are all fixed.

    Let’s take a look at the command-line test runner’s options in the following code:

    1. $ phpunit --help
    2. PHPUnit 9.6.0 by Sebastian Bergmann and contributors.
    4. Usage:
    5.   phpunit [options] UnitTest.php
    6.   phpunit [options] <directory>
    8. Code Coverage Options:
    9.   --coverage-clover <file>    Generate code coverage report in Clover XML format
    10.   --coverage-crap4j <file>    Generate code coverage report in Crap4J XML format
    11.   --coverage-html <dir>       Generate code coverage report in HTML format
    12.   --coverage-php <file>       Export PHP_CodeCoverage object to file
    13.   --coverage-text=<file>      Generate code coverage report in text format [default: standard output]
    14.   --coverage-xml <dir>        Generate code coverage report in PHPUnit XML format
    15.   --coverage-cache <dir>      Cache static analysis results
    16.   --warm-coverage-cache       Warm static analysis cache
    17.   --coverage-filter <dir>     Include <dir> in code coverage analysis
    18.   --path-coverage             Perform path coverage analysis
    19.   --disable-coverage-ignore   Disable annotations for ignoring code coverage
    20.   --no-coverage               Ignore code coverage configuration
    22. Logging Options:
    23.   --log-junit <file>          Log test execution in JUnit XML format to file
    24.   --log-teamcity <file>       Log test execution in TeamCity format to file
    25.   --testdox-html <file>       Write agile documentation in HTML format to file
    26.   --testdox-text <file>       Write agile documentation in Text format to file
    27.   --testdox-xml <file>        Write agile documentation in XML format to file
    28.   --reverse-list              Print defects in reverse order
    29.   --no-logging                Ignore logging configuration
    31. Test Selection Options:
    32.   --filter <pattern>          Filter which tests to run
    33.   --testsuite <name>          Filter which testsuite to run
    34.   --group <name>              Only runs tests from the specified group(s)
    35.   --exclude-group <name>      Exclude tests from the specified group(s)
    36.   --list-groups               List available test groups
    37.   --list-suites               List available test suites
    38.   --list-tests                List available tests
    39.   --test-suffix <suffixes>    Only search for test in files with specified suffix(es). Default: Test.php,.phpt
    41. Test Execution Options:
    42.   --dont-report-useless-tests Do not report tests that do not test anything
    43.   --strict-coverage           Be strict about @covers annotation usage
    44.   --strict-global-state       Be strict about changes to global state
    45.   --disallow-test-output      Be strict about output during tests
    46.   --disallow-resource-usage   Be strict about resource usage during small tests
    47.   --enforce-time-limit        Enforce time limit based on test size
    48.   --default-time-limit <sec>  Timeout in seconds for tests without @small, @medium or @large
    49.   --disallow-todo-tests       Disallow @todo-annotated tests
    51.   --process-isolation         Run each test in a separate PHP process
    52.   --globals-backup            Backup and restore $GLOBALS for each test
    53.   --static-backup             Backup and restore static attributes for each test
    55.   --colors <flag>             Use colors in output ("never", "auto" or "always")
    56.   --columns <n>               Number of columns to use for progress output
    57.   --columns max               Use maximum number of columns for progress output
    58.   --stderr                    Write to STDERR instead of STDOUT
    59.   --stop-on-defect            Stop execution upon first not-passed test
    60.   --stop-on-error             Stop execution upon first error
    62.   --stop-on-warning           Stop execution upon first warning
    63.   --stop-on-risky             Stop execution upon first risky test
    64.   --stop-on-skipped           Stop execution upon first skipped test
    65.   --stop-on-incomplete        Stop execution upon first incomplete test
    66.   --fail-on-incomplete        Treat incomplete tests as failures
    67.   --fail-on-risky             Treat risky tests as failures
    68.   --fail-on-skipped           Treat skipped tests as failures
    69.   --fail-on-warning           Treat tests with warnings as failures
    70.   -v|--verbose                Output more verbose information
    71.   --debug                     Display debugging information
    73.   --repeat <times>            Runs the test(s) repeatedly
    74.   --teamcity                  Report test execution progress in TeamCity format
    75.   --testdox                   Report test execution progress in TestDox format
    76.   --testdox-group             Only include tests from the specified group(s)
    77.   --testdox-exclude-group     Exclude tests from the specified group(s)
    78.   --no-interaction            Disable TestDox progress animation
    80.   --order-by <order>          Run tests in order: default|defects|duration|no-depends|random|reverse|size
    81.   --random-order-seed <N>     Use a specific random seed <N> for random order
    82.   --cache-result              Write test results to cache file
    83.   --do-not-cache-result       Do not write test results to cache file
    85. Configuration Options:
    86.   --prepend <file>            A PHP script that is included as early as possible
    87.   --bootstrap <file>          A PHP script that is included before the tests run
    88.   -c|--configuration <file>   Read configuration from XML file
    89.   --no-configuration          Ignore default configuration file (phpunit.xml)
    90.   --extensions <extensions>   A comma separated list of PHPUnit extensions to load
    91.   --no-extensions             Do not load PHPUnit extensions
    92.   --include-path <path(s)>    Prepend PHP's include_path with given path(s)
    93.   -d <key[=value]>            Sets a php.ini value
    94.   --cache-result-file <file>  Specify result cache path and filename
    95.   --generate-configuration    Generate configuration file with suggested settings
    96.   --migrate-configuration     Migrate configuration file to current format
    98. Miscellaneous Options:
    99.   -h|--help                   Prints this usage information
    100.   --version                   Prints the version and exits
    101.   --atleast-version <min>     Checks that version is greater than min and exits
    102.   --check-version             Check whether PHPUnit is the latest version

    phpunit UnitTest.php

    Runs the tests that are provided by the class UnitTest. This class is expected to be declared in the specified sourcefile.

    --coverage-clover

    Generates a logfile in XML format with the code coverage information for the tests run. See Code Coverage Analysis for more details.

    --coverage-crap4j

    Generates a code coverage report in Crap4j format. See for more details.

    --coverage-html

    Generates a code coverage report in HTML format. See Code Coverage Analysis for more details.

    --coverage-php

    Generates a serialized PHP_CodeCoverage object with the code coverage information.

    --coverage-text

    Generates a logfile or command-line output in human readable format with the code coverage information for the tests run.

    --log-junit

    Generates a logfile in JUnit XML format for the tests run.

    --testdox-html and --testdox-text

    Generates agile documentation in HTML or plain text format for the tests that are run (see ).

    --filter

    Only runs tests whose name matches the given regular expression pattern. If the pattern is not enclosed in delimiters, PHPUnit will enclose the pattern in / delimiters.

    The test names to match will be in one of the following formats:

    TestNamespace\TestCaseClass::testMethod

    The default test name format is the equivalent of using the __METHOD__ magic constant inside the test method.

    TestNamespace\TestCaseClass::testMethod with data set #0

    When a test has a data provider, each iteration of the data gets the current index appended to the end of the default test name.

    TestNamespace\TestCaseClass::testMethod with data set "my named data"

    The test name for a PHPT test is the filesystem path.

    See Example 3.2 for examples of valid filter patterns.

    Example 3.2 Filter pattern examples

    1. --filter 'TestNamespace\\TestCaseClass::testMethod'
    2. --filter 'TestNamespace\\TestCaseClass'
    3. --filter TestNamespace
    4. --filter TestCaseClass
    5. --filter testMethod
    6. --filter '/::testMethod .*"my named data"/'
    7. --filter '/::testMethod .*#5$/'
    8. --filter '/::testMethod .*#(5|6|7)$/'

    See for some additional shortcuts that are available for matching data providers.

    Example 3.3 Filter shortcuts

    --testsuite

    Only runs the test suite whose name matches the given pattern.

    --group

    Only runs tests from the specified group(s). A test can be tagged as belonging to a group using the @group annotation.

    The and @ticket annotations are aliases for @group allowing to filter tests based on their authors or their ticket identifiers, respectively.

    --exclude-group

    Exclude tests from the specified group(s). A test can be tagged as belonging to a group using the @group annotation.

    --list-groups

    List available test groups.

    --test-suffix

    Only search for test files with specified suffix(es).

    --dont-report-useless-tests

    Do not report tests that do not test anything. See Risky Tests for details.

    --strict-coverage

    Be strict about unintentionally covered code. See for details.

    --strict-global-state

    Be strict about global state manipulation. See Risky Tests for details.

    --disallow-test-output

    Be strict about output during tests. See for details.

    --disallow-todo-tests

    Does not execute tests which have the @todo annotation in its docblock.

    --enforce-time-limit

    Enforce time limit based on test size. See Risky Tests for details.

    --process-isolation

    Run each test in a separate PHP process.

    --no-globals-backup

    Do not backup and restore $GLOBALS. See for more details.

    --static-backup

    Backup and restore static attributes of user-defined classes. See Global State for more details.

    --colors

    Use colors in output. On Windows, use or ConEmu.

    There are three possible values for this option:

    • never: never displays colors in the output. This is the default value when --colors option is not used.

    • auto: displays colors in the output unless the current terminal doesn’t supports colors, or if the output is piped to a command or redirected to a file.

    • always: always displays colors in the output even when the current terminal doesn’t supports colors, or when the output is piped to a command or redirected to a file.

    When --colors is used without any value, auto is the chosen value.

    --columns

    --stderr

    Optionally print to STDERR instead of STDOUT.

    --stop-on-error

    Stop execution upon first error.

    --stop-on-failure

    Stop execution upon first error or failure.

    --stop-on-risky

    Stop execution upon first risky test.

    --stop-on-skipped

    Stop execution upon first skipped test.

    --stop-on-incomplete

    Stop execution upon first incomplete test.

    --verbose

    Output more verbose information, for instance the names of tests that were incomplete or have been skipped.

    --debug

    Output debug information such as the name of a test when its execution starts.

    --loader

    Specifies the PHPUnit\Runner\TestSuiteLoader implementation to use.

    The standard test suite loader will look for the sourcefile in the current working directory and in each directory that is specified in PHP’s include_path configuration directive. A class name such as Project_Package_Class is mapped to the source filename Project/Package/Class.php.

    --repeat

    Repeatedly runs the test(s) the specified number of times.

    --testdox

    Reports the test progress in TestDox format (see ).

    --printer

    Specifies the result printer to use. The printer class must extend PHPUnit\Util\Printer and implement the PHPUnit\Framework\TestListener interface.

    --bootstrap

    A “bootstrap” PHP file that is run before the tests.

    --configuration, -c

    Read configuration from XML file. See The XML Configuration File for more details.

    If phpunit.xml or phpunit.xml.dist (in that order) exist in the current working directory and --configuration is not used, the configuration will be automatically read from that file.

    If a directory is specified and if phpunit.xml or phpunit.xml.dist (in that order) exists in this directory, the configuration will be automatically read from that file.

    --no-configuration

    Ignore phpunit.xml and phpunit.xml.dist from the current working directory.

    --include-path

    Prepend PHP’s include_path with given path(s).

    -d

    Note

    Please note that options can be put after the argument(s).

    TestDox

    PHPUnit’s TestDox functionality looks at a test class and all the test method names and converts them from camel case (or snake_case) PHP names to sentences: testBalanceIsInitiallyZero() (or test_balance_is_initially_zero() becomes “Balance is initially zero”. If there are several test methods whose names only differ in a suffix of one or more digits, such as testBalanceCannotBecomeNegative() and testBalanceCannotBecomeNegative2(), the sentence “Balance cannot become negative” will appear only once, assuming that all of these tests succeed.

    Let us take a look at the agile documentation generated for a BankAccount class:

    1. $ phpunit --testdox BankAccountTest.php
    2. PHPUnit 9.6.0 by Sebastian Bergmann and contributors.
    4. BankAccount
    5.  ✔ Balance is initially zero

    Alternatively, the agile documentation can be generated in HTML or plain text format and written to a file using the and --testdox-text arguments.