Configuration versioning and archiving in VyOS

Last time I promised "node copying/renaming, node comments, and other little known features of the VyOS CLI", but the post actually only mentioned copying/renaming and comments, but not other features. It's time to fix that: today we'll discuss configuration versioning and archiving.

One of the great things about the config model with editing and commits being distinct stages is that it's feasible to execute some actions when the config is changed. In fact, you can execute arbitrary actions via pre/post-commit hooks, but there are built-in actions as well, namely configuration versioning and archiving to a remote location. This model, first introduced by JunOS, makes configuration is a lot more manageable than older Cisco style models.

This approach renders tools like Rancid or Oxydized redundant since the system can make a snapshot of the running config when the change is made rather than periodically. Moreover, right on the router you can see who made this or that commit and view diffs between revisions.

An additional advantage of versioning is that even if you forget to save the config (or purposely powercycle a system with an unsaved config because you forgot to use commit-confirm), you can always view recover the lost changes from the history.

Let's see how to use it.

Versioning

Configuration

By default, VyOS only keep a local archive of previous config versions. The number of revisions it keeps is controlled by the "system config-management commit-revisions $number".

In older VyOS versions the default value is 20, while the 1.2.0 nightly builds have it set to 100. Since configuration files are usually not too big, especially compared to the size of modern flash memory and hard drives, and the versioning system uses gzip compression, I usually set it to 1000 on all my routers: set system config-management commit-revisions 1000

You can disable versioning by deleting the "system config-management commit-revisions" option.

Usage

Viewing the commit history

To view the commit history, use this command: run show system commit

Here is an example:

dmbaturin@reki# run show system commit 
0 2018-02-10 16:28:52 by dmbaturin via cli
increase commit revision number
1 2018-02-10 16:23:39 by dmbaturin via cli
2 2018-02-01 05:41:33 by dmbaturin via cli
3 2018-02-01 05:41:27 by dmbaturin via cli
4 2018-01-26 02:46:31 by dmbaturin via cli
5 2017-12-25 04:37:54 by root via boot-config-loader
6 2017-12-15 18:46:13 by dmbaturin via cli

The output is quite straightforward, though some things need clarification. The first field is the revision number, the second field is the timestamp, and the third field is the user who initiated the commit.

The "via" field indicates the "application" that the commit was initiated from. Commits from the interactive CLI have it set to "cli", and commits made by the config loader at boot time have it set to "boot-config-loader" (why are they archived? At boot time the config can be modified by migration scripts).

In the example above, you can see "increase commit revision number" under a revision. This is a commit comment, which you can set with this command: commit comment "some comment".

Viewing old revisions and diffs

To view a complete config file from an older revision, use this command: run show system commit file $revisionNumber
The revision 0 points to the latest commit.

To see what a certain commit changed, use: run show system commit diff $revisionNumber
It shows a typical diff:

dmbaturin@reki# run show system commit diff 0
[edit interfaces ethernet eth1]
-address 10.90.90.1/24
[edit system config-management]
>commit-revisions 2000
[edit system]
+options {
+    ctrl-alt-del-action ignore
+    reboot-on-panic true
+}

This command may seem rather limited because it only shows a diff between a single revision N and revision N-1. For more advances comparison operations, you can use the "compare" command.

With just "compare" command with no arguments, you can view the diff between the proposed (uncommited) and running configuration. Quite handy when you have a large config and don't want to spend time searching for the modified bits in the output of the "show" command:

dmbaturin@reki# compare 
[edit system options]
>ctrl-alt-del-action reboot

You can also use "compare saved" command to quickly view a diff between the running and saved configuration.

Running "compare $revision" (e.g. "compare 5") allows comparing the running config with that revision, showing a cumulative diff (as opposed to a diff between $revision and $revision-1 as "run show system commit diff $revision" does).

Finally, you can use "compare $firstRevision $secondRevision" (e.g. "compare 4 20") to view a diff between any two revisions.

Archiving

Apart from the local archive, you can automatically send config revisions to a remote locations. The supported protocols are TFTP, FTP, and SFTP/SCP.

The configuration option is: system config-management commit-archive "<tftp|ftp|scp>://$user:$password@$host/dir", for example, "ftp://admin:qwerty@192.0.2.1/vyos".

Confirmed commit and rollback

One more possibility that is opened by configuration versioning is failsafe commits for risky changes and rollbacks.

Suppose you are making a risky change and you aren't sure whether you will lose your SSH connections to the router or not. The traditional approach is to schedule a reboot, make the change, and cancel the reboot if you are still connected.

VyOS offers a better approach. When you are about to make a risky change, use the "commit-confirm" command, optionally with an argument that specifies timeout in minutes (e.g. "commit-confirm 5"). The default timeout is 10 minutes.

It will warn you about what is about to happen and ask if you want to proceed. Type "y" or "yes" and press enter.

dmbaturin@reki# commit-confirm 
commit confirm will be automatically reboot in 10 minutes unless confirmed
Proceed? [confirm]y

If the change works fine and you are still connected, enter the "confirm" command. If you don't issue the "confirm" commands in 10 minutes (or the time interval you specified) the system will automatically reboot and revert to the previous revision (not: not to the saved config, but to the revision just before your last commit even if it wasn't saved).

You can as well rollback to any previous revision by hand with "rollback $revision" command, but it will also reboot your system first, unlike in JunOS, which limits its utility in non-emergency cases. The reasons for this are quite complicated and lie in unfortuate design decisions of the old config backend that gave rise to the VyConf project. If you want to see non-destructive rollback sooner, consider joining the development of the new backend. ;)

7 responses
I wrote some of code, so I'm happy to see it's still alive and being improved. Great job Daniil
Stig is being humble as usual, in the commit archive context "some" should read "most". ;) Things are finally getting back on track, so I hope we can keep that system as innovative as Stig, An-Cheng, Bob, Stephen, and everyone else made it back then!
Yuriy Andamasov upvoted this post.
4 visitors upvoted this post.