328 lines
14 KiB
HTML
328 lines
14 KiB
HTML
<!DOCTYPE html>
|
||
<html lang="en">
|
||
<head>
|
||
<meta charset="UTF-8">
|
||
<title>Controlling How Tests Are Run</title>
|
||
</head>
|
||
<body>
|
||
<h2 id="controlling-how-tests-are-run"><a class="header" href="#controlling-how-tests-are-run">Controlling How Tests Are Run</a></h2>
|
||
<p>Just as <code>cargo run</code> compiles your code and then runs the resultant binary,
|
||
<code>cargo test</code> compiles your code in test mode and runs the resultant test
|
||
binary. The default behavior of the binary produced by <code>cargo test</code> is to run
|
||
all the tests in parallel and capture output generated during test runs,
|
||
preventing the output from being displayed and making it easier to read the
|
||
output related to the test results. You can, however, specify command line
|
||
options to change this default behavior.</p>
|
||
<p>Some command line options go to <code>cargo test</code>, and some go to the resultant test
|
||
binary. To separate these two types of arguments, you list the arguments that
|
||
go to <code>cargo test</code> followed by the separator <code>--</code> and then the ones that go to
|
||
the test binary. Running <code>cargo test --help</code> displays the options you can use
|
||
with <code>cargo test</code>, and running <code>cargo test -- --help</code> displays the options you
|
||
can use after the separator. These options are also documented in <a href="https://doc.rust-lang.org/rustc/tests/index.html">the “Tests”
|
||
section of <em>The <code>rustc</code> Book</em></a>.</p>
|
||
<h3 id="running-tests-in-parallel-or-consecutively"><a class="header" href="#running-tests-in-parallel-or-consecutively">Running Tests in Parallel or Consecutively</a></h3>
|
||
<p>When you run multiple tests, by default they run in parallel using threads,
|
||
meaning they finish running more quickly and you get feedback sooner. Because
|
||
the tests are running at the same time, you must make sure your tests don’t
|
||
depend on each other or on any shared state, including a shared environment,
|
||
such as the current working directory or environment variables.</p>
|
||
<p>For example, say each of your tests runs some code that creates a file on disk
|
||
named <em>test-output.txt</em> and writes some data to that file. Then, each test
|
||
reads the data in that file and asserts that the file contains a particular
|
||
value, which is different in each test. Because the tests run at the same time,
|
||
one test might overwrite the file in the time between when another test is
|
||
writing and reading the file. The second test will then fail, not because the
|
||
code is incorrect but because the tests have interfered with each other while
|
||
running in parallel. One solution is to make sure each test writes to a
|
||
different file; another solution is to run the tests one at a time.</p>
|
||
<p>If you don’t want to run the tests in parallel or if you want more fine-grained
|
||
control over the number of threads used, you can send the <code>--test-threads</code> flag
|
||
and the number of threads you want to use to the test binary. Take a look at
|
||
the following example:</p>
|
||
<pre><code class="language-console">$ cargo test -- --test-threads=1
|
||
</code></pre>
|
||
<p>We set the number of test threads to <code>1</code>, telling the program not to use any
|
||
parallelism. Running the tests using one thread will take longer than running
|
||
them in parallel, but the tests won’t interfere with each other if they share
|
||
state.</p>
|
||
<h3 id="showing-function-output"><a class="header" href="#showing-function-output">Showing Function Output</a></h3>
|
||
<p>By default, if a test passes, Rust’s test library captures anything printed to
|
||
standard output. For example, if we call <code>println!</code> in a test and the test
|
||
passes, we won’t see the <code>println!</code> output in the terminal; we’ll see only the
|
||
line that indicates the test passed. If a test fails, we’ll see whatever was
|
||
printed to standard output with the rest of the failure message.</p>
|
||
<p>As an example, Listing 11-10 has a silly function that prints the value of its
|
||
parameter and returns 10, as well as a test that passes and a test that fails.</p>
|
||
<figure class="listing" id="listing-11-10">
|
||
<span class="file-name">Filename: src/lib.rs</span>
|
||
<pre><code class="language-rust panics noplayground">fn prints_and_returns_10(a: i32) -> i32 {
|
||
println!("I got the value {a}");
|
||
10
|
||
}
|
||
|
||
#[cfg(test)]
|
||
mod tests {
|
||
use super::*;
|
||
|
||
#[test]
|
||
fn this_test_will_pass() {
|
||
let value = prints_and_returns_10(4);
|
||
assert_eq!(value, 10);
|
||
}
|
||
|
||
#[test]
|
||
fn this_test_will_fail() {
|
||
let value = prints_and_returns_10(8);
|
||
assert_eq!(value, 5);
|
||
}
|
||
}</code></pre>
|
||
<figcaption><a href="#listing-11-10">Listing 11-10</a>: Tests for a function that calls <code>println!</code></figcaption>
|
||
</figure>
|
||
<p>When we run these tests with <code>cargo test</code>, we’ll see the following output:</p>
|
||
<pre><code class="language-console">$ cargo test
|
||
Compiling silly-function v0.1.0 (file:///projects/silly-function)
|
||
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.58s
|
||
Running unittests src/lib.rs (target/debug/deps/silly_function-160869f38cff9166)
|
||
|
||
running 2 tests
|
||
test tests::this_test_will_fail ... FAILED
|
||
test tests::this_test_will_pass ... ok
|
||
|
||
failures:
|
||
|
||
---- tests::this_test_will_fail stdout ----
|
||
I got the value 8
|
||
|
||
thread 'tests::this_test_will_fail' panicked at src/lib.rs:19:9:
|
||
assertion `left == right` failed
|
||
left: 10
|
||
right: 5
|
||
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
|
||
|
||
|
||
failures:
|
||
tests::this_test_will_fail
|
||
|
||
test result: FAILED. 1 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
|
||
|
||
error: test failed, to rerun pass `--lib`
|
||
</code></pre>
|
||
<p>Note that nowhere in this output do we see <code>I got the value 4</code>, which is
|
||
printed when the test that passes runs. That output has been captured. The
|
||
output from the test that failed, <code>I got the value 8</code>, appears in the section
|
||
of the test summary output, which also shows the cause of the test failure.</p>
|
||
<p>If we want to see printed values for passing tests as well, we can tell Rust to
|
||
also show the output of successful tests with <code>--show-output</code>:</p>
|
||
<pre><code class="language-console">$ cargo test -- --show-output
|
||
</code></pre>
|
||
<p>When we run the tests in Listing 11-10 again with the <code>--show-output</code> flag, we
|
||
see the following output:</p>
|
||
<pre><code class="language-console">$ cargo test -- --show-output
|
||
Compiling silly-function v0.1.0 (file:///projects/silly-function)
|
||
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.60s
|
||
Running unittests src/lib.rs (target/debug/deps/silly_function-160869f38cff9166)
|
||
|
||
running 2 tests
|
||
test tests::this_test_will_fail ... FAILED
|
||
test tests::this_test_will_pass ... ok
|
||
|
||
successes:
|
||
|
||
---- tests::this_test_will_pass stdout ----
|
||
I got the value 4
|
||
|
||
|
||
successes:
|
||
tests::this_test_will_pass
|
||
|
||
failures:
|
||
|
||
---- tests::this_test_will_fail stdout ----
|
||
I got the value 8
|
||
|
||
thread 'tests::this_test_will_fail' panicked at src/lib.rs:19:9:
|
||
assertion `left == right` failed
|
||
left: 10
|
||
right: 5
|
||
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
|
||
|
||
|
||
failures:
|
||
tests::this_test_will_fail
|
||
|
||
test result: FAILED. 1 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
|
||
|
||
error: test failed, to rerun pass `--lib`
|
||
</code></pre>
|
||
<h3 id="running-a-subset-of-tests-by-name"><a class="header" href="#running-a-subset-of-tests-by-name">Running a Subset of Tests by Name</a></h3>
|
||
<p>Running a full test suite can sometimes take a long time. If you’re working on
|
||
code in a particular area, you might want to run only the tests pertaining to
|
||
that code. You can choose which tests to run by passing <code>cargo test</code> the name
|
||
or names of the test(s) you want to run as an argument.</p>
|
||
<p>To demonstrate how to run a subset of tests, we’ll first create three tests for
|
||
our <code>add_two</code> function, as shown in Listing 11-11, and choose which ones to run.</p>
|
||
<figure class="listing" id="listing-11-11">
|
||
<span class="file-name">Filename: src/lib.rs</span>
|
||
<pre><code class="language-rust noplayground">pub fn add_two(a: u64) -> u64 {
|
||
a + 2
|
||
}
|
||
|
||
#[cfg(test)]
|
||
mod tests {
|
||
use super::*;
|
||
|
||
#[test]
|
||
fn add_two_and_two() {
|
||
let result = add_two(2);
|
||
assert_eq!(result, 4);
|
||
}
|
||
|
||
#[test]
|
||
fn add_three_and_two() {
|
||
let result = add_two(3);
|
||
assert_eq!(result, 5);
|
||
}
|
||
|
||
#[test]
|
||
fn one_hundred() {
|
||
let result = add_two(100);
|
||
assert_eq!(result, 102);
|
||
}
|
||
}</code></pre>
|
||
<figcaption><a href="#listing-11-11">Listing 11-11</a>: Three tests with three different names</figcaption>
|
||
</figure>
|
||
<p>If we run the tests without passing any arguments, as we saw earlier, all the
|
||
tests will run in parallel:</p>
|
||
<pre><code class="language-console">$ cargo test
|
||
Compiling adder v0.1.0 (file:///projects/adder)
|
||
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.62s
|
||
Running unittests src/lib.rs (target/debug/deps/adder-92948b65e88960b4)
|
||
|
||
running 3 tests
|
||
test tests::add_three_and_two ... ok
|
||
test tests::add_two_and_two ... ok
|
||
test tests::one_hundred ... ok
|
||
|
||
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
|
||
|
||
Doc-tests adder
|
||
|
||
running 0 tests
|
||
|
||
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
|
||
|
||
</code></pre>
|
||
<h4 id="running-single-tests"><a class="header" href="#running-single-tests">Running Single Tests</a></h4>
|
||
<p>We can pass the name of any test function to <code>cargo test</code> to run only that test:</p>
|
||
<pre><code class="language-console">$ cargo test one_hundred
|
||
Compiling adder v0.1.0 (file:///projects/adder)
|
||
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.69s
|
||
Running unittests src/lib.rs (target/debug/deps/adder-92948b65e88960b4)
|
||
|
||
running 1 test
|
||
test tests::one_hundred ... ok
|
||
|
||
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 2 filtered out; finished in 0.00s
|
||
|
||
</code></pre>
|
||
<p>Only the test with the name <code>one_hundred</code> ran; the other two tests didn’t match
|
||
that name. The test output lets us know we had more tests that didn’t run by
|
||
displaying <code>2 filtered out</code> at the end.</p>
|
||
<p>We can’t specify the names of multiple tests in this way; only the first value
|
||
given to <code>cargo test</code> will be used. But there is a way to run multiple tests.</p>
|
||
<h4 id="filtering-to-run-multiple-tests"><a class="header" href="#filtering-to-run-multiple-tests">Filtering to Run Multiple Tests</a></h4>
|
||
<p>We can specify part of a test name, and any test whose name matches that value
|
||
will be run. For example, because two of our tests’ names contain <code>add</code>, we can
|
||
run those two by running <code>cargo test add</code>:</p>
|
||
<pre><code class="language-console">$ cargo test add
|
||
Compiling adder v0.1.0 (file:///projects/adder)
|
||
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.61s
|
||
Running unittests src/lib.rs (target/debug/deps/adder-92948b65e88960b4)
|
||
|
||
running 2 tests
|
||
test tests::add_three_and_two ... ok
|
||
test tests::add_two_and_two ... ok
|
||
|
||
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s
|
||
|
||
</code></pre>
|
||
<p>This command ran all tests with <code>add</code> in the name and filtered out the test
|
||
named <code>one_hundred</code>. Also note that the module in which a test appears becomes
|
||
part of the test’s name, so we can run all the tests in a module by filtering
|
||
on the module’s name.</p>
|
||
<!-- Old headings. Do not remove or links may break. -->
|
||
<p><a id="ignoring-some-tests-unless-specifically-requested"></a></p>
|
||
<h3 id="ignoring-tests-unless-specifically-requested"><a class="header" href="#ignoring-tests-unless-specifically-requested">Ignoring Tests Unless Specifically Requested</a></h3>
|
||
<p>Sometimes a few specific tests can be very time-consuming to execute, so you
|
||
might want to exclude them during most runs of <code>cargo test</code>. Rather than
|
||
listing as arguments all tests you do want to run, you can instead annotate the
|
||
time-consuming tests using the <code>ignore</code> attribute to exclude them, as shown
|
||
here:</p>
|
||
<p><span class="filename">Filename: src/lib.rs</span></p>
|
||
<pre><code class="language-rust noplayground"><span class="boring">pub fn add(left: u64, right: u64) -> u64 {
|
||
</span><span class="boring"> left + right
|
||
</span><span class="boring">}
|
||
</span><span class="boring">
|
||
</span>#[cfg(test)]
|
||
mod tests {
|
||
use super::*;
|
||
|
||
#[test]
|
||
fn it_works() {
|
||
let result = add(2, 2);
|
||
assert_eq!(result, 4);
|
||
}
|
||
|
||
#[test]
|
||
#[ignore]
|
||
fn expensive_test() {
|
||
// code that takes an hour to run
|
||
}
|
||
}</code></pre>
|
||
<p>After <code>#[test]</code>, we add the <code>#[ignore]</code> line to the test we want to exclude.
|
||
Now when we run our tests, <code>it_works</code> runs, but <code>expensive_test</code> doesn’t:</p>
|
||
<pre><code class="language-console">$ cargo test
|
||
Compiling adder v0.1.0 (file:///projects/adder)
|
||
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.60s
|
||
Running unittests src/lib.rs (target/debug/deps/adder-92948b65e88960b4)
|
||
|
||
running 2 tests
|
||
test tests::expensive_test ... ignored
|
||
test tests::it_works ... ok
|
||
|
||
test result: ok. 1 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.00s
|
||
|
||
Doc-tests adder
|
||
|
||
running 0 tests
|
||
|
||
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
|
||
|
||
</code></pre>
|
||
<p>The <code>expensive_test</code> function is listed as <code>ignored</code>. If we want to run only
|
||
the ignored tests, we can use <code>cargo test -- --ignored</code>:</p>
|
||
<pre><code class="language-console">$ cargo test -- --ignored
|
||
Compiling adder v0.1.0 (file:///projects/adder)
|
||
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.61s
|
||
Running unittests src/lib.rs (target/debug/deps/adder-92948b65e88960b4)
|
||
|
||
running 1 test
|
||
test tests::expensive_test ... ok
|
||
|
||
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s
|
||
|
||
Doc-tests adder
|
||
|
||
running 0 tests
|
||
|
||
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
|
||
|
||
</code></pre>
|
||
<p>By controlling which tests run, you can make sure your <code>cargo test</code> results
|
||
will be returned quickly. When you’re at a point where it makes sense to check
|
||
the results of the <code>ignored</code> tests and you have time to wait for the results,
|
||
you can run <code>cargo test -- --ignored</code> instead. If you want to run all tests
|
||
whether they’re ignored or not, you can run <code>cargo test -- --include-ignored</code>.</p>
|
||
</body>
|
||
</html>
|