Command line parsing with clap

This post is part of a series on implementing SortaSecret.com.

clap is a library which provides the ability to parse command line options. For SortaSecret.com, we have relatively simple parsing needs: two subcommands and some options. Some of the options are, well, optional, while otherwise are required. And one of the subcommands must be selected. We'll demonstrate how to parse that with clap.

Keep in mind that in addition to the clap interface I'll be using below, there is also a structopt library that more directly parses arguments into structures. This article will not cover structopt at all; a future article may do that instead. Also, in the future it looks like structopt functionality will be merged into clap.

Also, final note: the API docs for clap are really good and include plenty of worked examples. Please check those out as well!

Reader prerequisites

This blog post will assume basic knowledge of the Rust programming language, and that you have the command line tooling (rustup, cargo, etc) installed. If you'd like more information, see our Rust getting started guide.

Simple example

Let's write a simple command line argument parser to make sure everything's working. Start off with with a cargo new clap1 --bin to start a new project, and then add clap = "2.33" underneath [dependencies] in Cargo.toml. Inside the generated clap1 directory, run cargo run, which should build clap and its dependencies, and then print Hello, world!. Or more fully, on my machine:

First run of clap1

Of course, this isn't using clap yet. Let's fix that. We're going to parse the command line option --name to find out who to say hello to. To do this with clap, we're going to follow the basic steps of:

Here's our code doing all of this:

extern crate clap;

use clap::{Arg, App};

fn main() {
    // basic app information
    let app = App::new("hello-clap")
        .version("1.0")
        .about("Says hello")
        .author("Michael Snoyman");

    // Define the name command line option
    let name_option = Arg::with_name("name")
        .long("name") // allow --name
        .takes_value(true)
        .help("Who to say hello to")
        .required(true);

    // now add in the argument we want to parse
    let app = app.arg(name_option);

    // extract the matches
    let matches = app.get_matches();

    // Extract the actual name
    let name = matches.value_of("name")
        .expect("This can't be None, we said it was required");

    println!("Hello, {}!", name);
}

You can run this with cargo run, which should result in something like:

$ cargo run
   Compiling clap1 v0.1.0 (/Users/michael/Desktop/clap1)
    Finished dev [unoptimized + debuginfo] target(s) in 1.06s
     Running `target/debug/clap1`
error: The following required arguments were not provided:
    --name <name>

USAGE:
    clap1 --name <name>

For more information try --help

To pass in command line option to our executable, we need to add an extra -- after cargo run, to tell cargo that the remainder of the command line options should be passed verbatim to the clap1 executable and not parsed by cargo itself. For example:

$ cargo run -- --help
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/clap1 --help`
hello-clap 1.0
Michael Snoyman
Says hello

USAGE:
    clap1 --name <name>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
        --name <name>    Who to say hello to

And, of course:

$ cargo run -- --name Rust
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/clap1 --name Rust`
Hello, Rust!

Exercises

Extracting to a struct

It's all well and good to get the matches. But one of the best draws of Rust in my opinion is strongly typed applications. To make that a reality, my preference is to have an intermediate step between the matching and the actual application code where we extract the command line options into a struct. This isolates our parsing logic to one area, and lets the compiler help us everywhere else.

By the way, this is exactly the kind of thing that structopt does for us. But today, we'll do it directly with clap.

The first step is to define the struct. In our example, we had exactly real argument, a name, which was a required String. So our struct will look like:

struct HelloArgs {
    name: String,
}

And then, we can essentially copy-paste our original code into an impl for this struct, something like:

impl HelloArgs {
    fn new() -> Self {
        // basic app information
        let app = App::new("hello-clap")
        ...
        // Extract the actual name
        let name = matches.value_of("name")
            .expect("This can't be None, we said it was required");

        HelloArgs { name: name.to_string() }
    }
}

And we can use it with

fn main() {
    let hello = HelloArgs::new();

    println!("Hello, {}!", hello.name);
}

Since the code is almost entirely unchanged, I won't include it inline here, but you can see it as a Github Gist.

Exercises

Better testing

The above refactoring doesn't really give us much, especially on such a small program. However, we can already leverage this to start doing some testing. To make that work, we first want to modify our new method to take the command line arguments as an argument, instead of reading it from the global executable. We'll also need to modify the return value to return a Result to deal with failed parses. Up until now, we've been relying on clap itself to print an error message and exit the process automatically.

First, let's look at what our signature is going to be:

fn new_from<I, T>(args: I) -> Result<Self, clap::Error>
where
    I: Iterator<Item = T>,
    T: Into<OsString> + Clone,

And using clap::Error's exit() method, we can recover our original new function working off of the actual command line arguments and exiting the program on a bad parse fairly easily:

fn new() -> Self {
    Self::new_from(std::env::args_os().into_iter()).unwrap_or_else(|e| e.exit())
}

And within our new_from method, we just need to replace the call to get_matches with:

// extract the matches
let matches = app.get_matches_from_safe(args)?;

And wrap the final value with Ok:

Ok(HelloArgs {
    name: name.to_string(),
})

Awesome, now we're ready to write some tests. First, one note: when we provide the list of arguments, the first argument is always the executable name. Our first test will make sure that we get an argument parse error when no arguments are provided, which looks like this:

#[cfg(test)]
mod test {
    use super::*;

    #[test]
    fn test_no_args() {
        HelloArgs::new_from(["exename"].iter()).unwrap_err();
    }
}

We can also test that using the --name option without a value doesn't parse:

#[test]
fn test_incomplete_name() {
    HelloArgs::new_from(["exename", "--name"].iter()).unwrap_err();
}

And finally test that things work when a name is provided:

#[test]
fn test_complete_name() {
    assert_eq!(
        HelloArgs::new_from(["exename", "--name", "Hello"].iter()).unwrap(),
        HelloArgs { name: "Hello".to_string() }
    );
}

Property checking

There's still one piece of this whole puzzle that bothers me. The caller of new or new_from knows it has received a nicely typed value. However, within new_from, we are using expect for cases that should be impossible. For example, our name_option sets required to true, and therefore we know the matches.value_of("name") call will not return a None. However, what guarantees do we have that we remembered to set required to true?

One approach to improve the situation is to use property testing. In the case of the argument parsing, I can state a simple property: for all possible strings I can send as input, the parse will either return a valid HelloArgs or generate a clap::Error. Under no circumstances, however, should it panic. And using the quickcheck and quickcheck_macros crates, we can test exactly this! First we add the following to the top of our file:

#[cfg(test)]
extern crate quickcheck;
#[cfg(test)]
#[macro_use(quickcheck)]
extern crate quickcheck_macros;

And then write the nice little property:

#[quickcheck]
fn prop_never_panics(args: Vec<String>) {
    let _ignored = HelloArgs::new_from(args.iter());
}

And sure enough, if you set required to false, this property will fail.

I'll include the full code for this at the end of the article.

Just scratching the surface

There are lots of other things you may want to do with clap, and we're not going to cover all of them here. It's a great library, and produces wonderful CLIs. You may be a bit surprised that this article claims to be part of the series on implementing SortaSecret.com. This is where SortaSecret comes in. The source code includes the cli module, which has an example of a subcommand and therefore uses an enum to handle the differents variants.

Full code

extern crate clap;

use clap::{App, Arg};
use std::ffi::OsString;

#[cfg(test)]
extern crate quickcheck;
#[cfg(test)]
#[macro_use(quickcheck)]
extern crate quickcheck_macros;

#[derive(Debug, PartialEq)]
struct HelloArgs {
    name: String,
}

impl HelloArgs {
    fn new() -> Self {
        Self::new_from(std::env::args_os().into_iter()).unwrap_or_else(|e| e.exit())
    }

    fn new_from<I, T>(args: I) -> Result<Self, clap::Error>
    where
        I: Iterator<Item = T>,
        T: Into<OsString> + Clone,
    {
        // basic app information
        let app = App::new("hello-clap")
            .version("1.0")
            .about("Says hello")
            .author("Michael Snoyman");

        // Define the name command line option
        let name_option = Arg::with_name("name")
            .long("name") // allow --name
            .short("n") // allow -n
            .takes_value(true)
            .help("Who to say hello to")
            .required(true);

        // now add in the argument we want to parse
        let app = app.arg(name_option);
        // extract the matches
        let matches = app.get_matches_from_safe(args)?;

        // Extract the actual name
        let name = matches
            .value_of("name")
            .expect("This can't be None, we said it was required");

        Ok(HelloArgs {
            name: name.to_string(),
        })
    }
}

fn main() {
    let hello = HelloArgs::new();

    println!("Hello, {}!", hello.name);
}

#[cfg(test)]
mod test {
    use super::*;

    #[test]
    fn test_no_args() {
        HelloArgs::new_from(["exename"].iter()).unwrap_err();
    }

    #[test]
    fn test_incomplete_name() {
        HelloArgs::new_from(["exename", "--name"].iter()).unwrap_err();
    }

    #[test]
    fn test_complete_name() {
        assert_eq!(
            HelloArgs::new_from(["exename", "--name", "Hello"].iter()).unwrap(),
            HelloArgs { name: "Hello".to_string() }
        );
    }

    #[test]
    fn test_short_name() {
        assert_eq!(
            HelloArgs::new_from(["exename", "-n", "Hello"].iter()).unwrap(),
            HelloArgs { name: "Hello".to_string() }
        );
    }

    /* This property will fail, can you guess why?
    #[quickcheck]
    fn prop_any_name(name: String) {
        assert_eq!(
            HelloArgs::new_from(["exename", "-n", &name].iter()).unwrap(),
            HelloArgs { name }
        );
    }
    */

    #[quickcheck]
    fn prop_never_panics(args: Vec<String>) {
        let _ignored = HelloArgs::new_from(args.iter());
    }
}

What's next?

I'd recommend checking out the rest of our SortaSecret content, and have a look at our Rust homepage.