Thinking in terms of features vs thinking in terms of use cases

HackerGarten ?

David and Kyle and me meet regularly with other people for an HackerGarten session in Lausanne (thanks Greg for organizing!), where we contribute some code to open source projects. It is a relaxing way to give back by collaborating with somebody you esteem instead than alone.

Enough about Conan to follow along

You don’t need to be into C or C++ libraries to follow along this article, which is about how to approach development and applicable to any language.

Conan is a powerful, user-level, cross-platform package and dependency manager written in Python, similar conceptually to Maven for Java or pip for Python. Its main focus is to provide packages for C and C++ libraries, but it can also behave like Homebrew, installing executables as opposed to libraries.

Conan is made of two parts, the client and the server. The Conan client, called conan, is used for two different goals: consuming packages (installing them) or authoring packages. In this article we focus on the most common scenario: consuming packages.

The packages are downloaded from one or more servers, called “remotes”. When you install conan on a developer machine, it comes with two predefined public remotes.

The command

conan install <PACKAGE_SPEC>

is used to install a package. From the conan install documentation:

If any requirement is not found in the local cache, it will retrieve the recipe from a remote, looking for it sequentially in the configured remotes.

The use case

1. As a devops engineer, in my provisioning script, I want to ensure that Conan doesn’t have any default remote configured, so that it is impossible to pull by error a package from a remote that doesn’t belong to my organization.

2. Since my provisioning script changes global state (it installs things!), it should be idempotent, so that I can run it multiple times without negative impact while developing it.

Context

Currently, when installing Conan from scratch, there will be the following two default remotes configured:

conan-center:  https://conan.bintray.com         [Verify SSL: True]
conan-transit: https://conan-transit.bintray.com [Verify SSL: True]

The command conan remote remove allows to remove a remote, but it will fail (will return a a non-zero exit code) if the remote doesn’t exist.

This means that when driving conan remote remove from a script, it requires additional logic to cover the case when the script has actually been already called and so the remote has already been removed.

Solution 1

Yesterday we jumped immediately into implementation mindset (adding --force to conan remote remove) because we thought that it was the best solution to our problem.

On the other hand, if we had thought more about the essence of the use case, I think that we would have found Solution 2 sooner.

Following the spirit of existing Unix utilities such as rm (delete a file), introduce a --force flag to conan remote remove. From the man page of rm:

-f  Attempt to remove the files without prompting for confirmation, regardless
    of the file permissions. If the file does not exist, do not display a
    diagnostic message or modify the exit status to reflect an error.

so that we could call from the provisioning script simply:

conan remote remove --force conan-center
conan remote remove --force conan-transit

without any additional logic to keep the script from failing.

This works, and also has a very important characteristics: it doesn’t change the existing behavior: without --force the command will still fail if the remote doesn’t exist.

The code changes to add this --force flag are minimal and acceptable, although the logic is slightly more complex (see the github PR).

So work done and we are happy because we contributed to open source, right ?

Well, let’s start again with the use case:

1. As a devops engineer, in my provisioning script, I want to ensure that Conan doesn’t have any default remote configured, so that it is impossible to pull by error a package from a remote that doesn’t belong to my organization.

The current shape of the provisioning script:

conan remote remove --force conan-center
conan remote remove --force conan-transit

suffers from two limitations:

1. It requires the script to explicitly list the default remotes to remove, which is an implementation detail.
2. It will fail (silently!) to give us a clean slate if tomorrow Conan upstream, for any reason, either adds additional default remotes or renames the existing ones.

Said in other words:

The fixation with --force, too low-level, gave us a brittle solution.

A better solution is to understand that what we really want (and the use case was telling us since the beginning) is to be able to clear the default remotes list, no matter how many and no matter their identity!

Solution 2

Add a clear subcommand to conan remote that will clear the remotes list. This will allow to write the provisioning script simply as:

conan remote clear

The script will be simpler, self-describing without the need of comments and will withstand changes to the default remotes.

Conclusions

Both we and Conan upstream agreed that a conan remote clean would be a better approach and we will code this on the next HackerGarten!

Upstream is suggesting to change clear to clean for uniformity with other conan commands, but the main concept and the abstraction level stays the same.

Update

We submitted the implementation for Solution 2, conan remote clean in October 2018 and Conan 1.9.0 shipped with it.