tag:blog.vyos.net,2013:/posts blog.vyos.net 2018-01-21T17:34:36Z VyOS tag:blog.vyos.net,2013:Post/1234444 2018-01-19T15:00:55Z 2018-01-21T17:34:36Z Setting up GRE/IPsec behind NAT

In the previous posts of this series we've discussed setting up "plain" IPsec tunnels from behind NAT.

The transparency of the plain IPsec, however, is more often a curse than a blessing. Truly transparent IPsec is only possible between publicly routed networks, and the tunnel mode creates a strange mix of the two approaches: you do not have a network interface associated with the tunnel, but the setup is not free of routing issues either, and it's often hard to test whether the tunnel actually works or not from the router itself.

GRE/IPsec (or IPIP/IPsec, or anything else) offers a convenient solution: for all intents and purposes it's a normal network interface and makes it look like the networks are connected with a wire. You can easily ping the other side, use the interface for firewall and QoS rulesets, and setup dynamic routing protocols in a straightforward way. However, NAT creates a unique challenge for this setup.

The canonical and the simplest GRE/IPsec setup looks like this:

interfaces {
  tunnel tun0 {
    encapsulation gre
vpn {
  ipsec {
    site-to-site {
      peer {
        tunnel 1 {
          protocol gre

It creates a policy that encrypts any GRE packets sent to Of course it's not going to work with NAT because the remote side is not directly routable.

Let's see how we can get around it. Suppose you are setting up a tunnel between routers called East and West. The way to get around it is pretty simple even if not exactly intuitive and boils down to this:

  1. Setup an additional address on a loopback or dummy interface on each router, e.g. on the East and on the West.
  2. Setup GRE tunnels that are using and .2 as local-ip and remote-ip respectively.
  3. Setup an IPsec tunnels that uses and .2 as local-prefix and remote-prefix respectively.

This way when traffic is sent through the GRE tunnel on the East, the GRE packets will use as a source address, which will match the IPsec policy. Since is specified as the remote-prefix of the tunnel, the IPsec process will setup a kernel route to it, and the GRE packets will reach the other side.

Let's look at the config:

interfaces {
  dummy dum0 {
  tunnel tun0 {
    encapsulation gre
vpn {
  ipsec {
    site-to-site {
      peer @west {
        connection-type respond
        tunnel 1 {
          local {
          remote {

This approach also has a property that may make it useful even in publicly routed networks if you are going to use the GRE tunnel for sensitive but unencrypted traffic (I've seen that in legacy applications): unlike the canonical setup, GRE tunnel stops working when the IPsec SA goes down because the remote end becomes unreachable. The canonical setup will continue to work even without IPsec and may expose the GRE traffic to eavesdropping and MitM attacks.

This concludes the series of posts about IPsec and NAT. Next Friday I'll find something else to write about. ;)

Daniil Baturin
tag:blog.vyos.net,2013:Post/1230762 2018-01-12T09:05:06Z 2018-01-21T17:34:36Z How to setup an IPsec connection between two NATed peers: using id's and RSA keys

In the previous post from this series, we've discussed setting up an IPsec tunnel from a NATed router to a non-NATed one. The key point is that in the presence of NAT, the non-NATed side cannot identify the NATed peer by its public address, so a manually configured id is required.

What if both peers are NATed though? Suppose you are setting up a tunnel between two EC2 instances. They are both NATed, and this creates its own unique challenges: neither of them know their public addresses or can identify their peers by their public address. So, we need to solve two problems.

In this post, we'll setup a tunnel between two routers, let's call them "east" and "west". The "east" router will be the initiator, and "west" will be the responder.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1230731 2018-01-12T06:21:06Z 2018-01-21T17:34:36Z VyOS builds now use the deb.debian.net load balanced mirror

If there are any good things about that packages server migration and restructuring is that it promoted a revamp of the associated part of the build scripts.

Since the start the default Debian mirror was set to nl.debian.org for a completely arbitrary reason. This of course was suboptimal for most users who are far from the Netherlands, and while the mirror is easy enough to change in ./configure options, a better out of the box experience wouldn't harm.

Danny ter Haar (fromport) suggested that we change it to deb.debian.org which is load balanced, which I think is a good idea. There's a small chance that it will redirect you to a dead mirror, but if you run into any issues, you can always set it by hand.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1230330 2018-01-11T15:16:56Z 2018-01-21T17:34:36Z VyOS builds and HTTPS: build works again, HTTP still needs testing

We have restored VyOS builds. Nightly build should work as expected today, and you can build it by hand as well if you want. This is not exactly the end of the story for us since we need to finish some reconfiguration of Jenkins to accomodate the new setup, but nothing dramatic should happen to the ISO builds any soon, or so we hope.

HTTPS, however, is another story. It still doesn't work for me, and I'm not sure if it's APT itself to blame or anything in our build setup. Since this is not a pressing issue, I'm not going to put much effort into it right now, but if you have a build setup, please check if it works for you. If it doesn't work for anyone, then we can write it off as an APT issue.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1230142 2018-01-11T06:58:04Z 2018-01-21T17:34:35Z Follow-up: VyOS builds and HTTPS

We've made HTTP on the dev.packages.vyos.net host optional, and restored the real directory index (provided by the Apache HTTP's mod_autoindex) instead of using the DirectoryLister that was proven a bit problematic with APT.

Since we had to change the default repository URL anyway, I also took a chance to finally make it configurable rather than hardcoded (T519). Now you can specify a custom URL with the --vyos-mirror="$URL" option. It defaults to the plain HTTP URL right now for the reason stated below.

I have also found a way to make live-build include apt-transport-https packages at the bootstrap stage and enable it to use HTTPS servers for building images. However, for some reason it doesn't work for me, apt says it cannot fetch the package index, while fetching that file with curl works just fine from the same host. I'm not sure what the issue may be. If you verify that it works for you or doesn't, or you know how to make it work, please comment upon T422.

Actual builds "still" don't work but for a completely unrelated reasons: mdns-repeater package needed for the recently merged mDNS repeater feature is not yet in the repository. We will fix it shortly, now that the builds otherwise work.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1229513 2018-01-10T03:09:11Z 2018-01-21T17:34:35Z VyOS builds and HTTPS

For a while some people kept asking why we do not enable HTTPS on the servers with ISOs and repositories. Now we have enabled it, but it turned out it's not all that simple: since APT doesn't support HTTPS by default and the apt-transport-https package is not installed by the bootstrap stage of the live-build process, the dev.packages.vyos.net server is now unusable and image builds are broken.

We are looking for solutions. If you know how to make live-build use apt-transport-https for the boostrap stage, please tell us. If we don't find anything today, we'll try to make HTTPS optional, and if that turns out to be impractical in VestaCP (which we are using for the web frontend server), we'll just disable it there.

Sorry for the inconvenience!

Daniil Baturin
tag:blog.vyos.net,2013:Post/1229472 2018-01-10T01:09:13Z 2018-01-21T17:34:35Z No new features in Perl and shell and no old style templates since May 2018

Now that the Python library for accessing the running config and the generator of old style templates from new style XML command definitions are known to be functional, it's time to set a cutoff date for the old style code. We decided to arbitrarily set it to May 2018.

Since May the 1st, no new code written in the old style will be accepted. We will accept (and make ourselves) fixes to the old code when the bug severity is high enough to affect VyOS operation, but all new features get to be new style.

If you have some new feature already almost done, consider completing it until May. If you are planning a new feature, consider learning about the new style first.

If you are late to the party, please read these blog posts:

For those who are not, I'll reiterate the main points:

  • Contributors who know Perl and are still willing to write it are getting progressively harder to find and many people say they would contribute if it wasn't in Perl.
  • The reasons people abandoned Perl are more than compelling: lack of proper type checking, exception handling, and many other features of post-60's language designs do not help reliability and ease of maintenance at all.
  • Old style handwritten templates are very hard to maintain since both the data and often the logic is spread across dozens, if not hundreds of files in deeply nested directories. Separation of logic from data and a way to keep all command definitions of a feature in a single, observable file are required to make it maintainable.
  • With old style templates, there's no way to verify them without installing them on VyOS and trying them by hand. XML is the only common language that has ready to use tools for verification of its syntax and semantics: right now it's already integrated in our build system and a build with malformed command definitions will fail.

I'll write a tutorial about writing features in the new style shortly, stay tuned. Meanwhile you can look at the implementation of cron in 1.2.0:

  • https://github.com/vyos/vyos-1x/blob/current/src/conf-mode/vyos-update-crontab.py (conf mode script)
  • https://github.com/vyos/vyos-1x/blob/current/interface-definitions/cron.xml (command definitions)

Daniil Baturin
tag:blog.vyos.net,2013:Post/1228213 2018-01-08T03:32:28Z 2018-01-21T17:34:35Z Migration and restructuring of the (dev).packages.vyos.net hosts

The original host where the packages.vyos.net and dev.packages.vyos.net web servers used to live has kept having serious I/O performance issues ever since a RAID failure event, and the situation hasn't improved much. After we've started having very noticeable latency even with bash completion (not speaking of very slow download speed for our users and mirror maintainers), we decided that it's time to move it elsewhere.

We do not blame the hoster, to the contrary, we are grateful for hosting it for us at no cost for so long despite its bandwidth consumption, and for offering it to us at the point in the VyOS project life when it has exactly zero budget for anything. But, it seems now it's time to move on.

The dev.packages.vyos.net host is used for the package repositories of the current branch and for nightly builds, as well as experimental images we make available for testing. It is already migrated to the new host and operational, even though some things are not working as well as they should, including temporary (we hope) loss of IPv6 access to it, and somewhat untidy directory links generated by the (otherwise very nice) DirectoryLister script.

We are sorry for the migration taking longer than it should have and for not communicating it to the community. The dev.packages.vyos.net stayed inaccessible for a whole day because of miscommunication: we have migrated the data, but I assumed that Yuriy (syncer) will setup the web server, while he assumed I will. Our maintenance procedures definitely could use some improvement.

Note that apart from physical data migration, we have also done some directory restructuring. We hope it's now more logical and that it will not have any impact on anyone, but if it does, we can setup redirects to make old direct links work again.

The packages.vyos.net server will stay on the old machine for a bit as we also need to migrate the rsync setup that the mirrors are using. We are still not sure whether to use downloads.vyos.io as a central server for mirrors too.

As of building VyOS images, we'll make adjustments to the build scripts to make them use the new paths by default and images will be buildable again.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1227260 2018-01-05T23:14:16Z 2018-01-21T17:34:35Z Why IPsec behind 1:1 NAT is so problematic and what you can do about it

Not so long ago the only scenario when the issues with IPsec and NAT could arise was a remote access setup, while routers invariably had real public addresses and router to router IPsec operators were incredibly unlikely to run into those issues. A router behind NAT was seen as a pathological case.

I believe it's still a pathological case, but as platforms such as Amazon EC2 that use one to one NAT to simplify IP address management rose in popularity and people started setting up their routers there, it became the new normal. This new reality became a constant source of confusion for beginner network admins and people who simply are not VPN specialists. Attempts to setup a VPN as if there's no NAT, with peer external IP addresses and pre-shared keys as it's commonly done, just fail to work.

Sometimes I'm asked why is that so, if the NAT in question is 1:1, and no other protocol just refuses to work behind it without reconfiguration. The reasons are inside the IPsec protocols themselves, and 1:1 NAT is not much better in this regard than any other kind (even though it's somewhat better than the situation when there are multiple clients behind NAT).

IPsec pre-dates NAT by over a decade, and was explicitly designed with end to end connectivity in mind. It was also designed as a way to add built-in security to the IP protocol stack rather than a new protocol within the stack, so it made heavy use of those underlying assumptions. That's why workarounds had to be added when the assumption of end to end connectivity became incorrect.

Let's look into the details and then see how IPsec can be configured to work around those issues. Incidentally, the solutions are equally applicable to machines with dynamic addresses as well as those behind a 1:1 NAT, since the root cause of the issues is not knowing beforehand what your outgoing address will be next time.

AH and ESP

We all (hopefully) remember that there are three criteria of information security: availability, confidentiality, and integrity. Availability (when required) in the IP is provided by reliable transmission protocols such as TCP and SCTP. IPsec was supposed to provide the other two.

AH (Authentication Header) was designed to ensure packet integrity. For that, it calculates a hash function from the entire packet with all its headers, including source and destination addresses. Since NAT changes the addresses, it invalidates the checksum and NATed packets would be considered corrupted. So, AH is unconditionally incompatible with NAT.

ESP only protects the payload of the packet (whether it's some protocol data, or an entire tunnelled packet), so it, luckily, can work behind NAT. The UDP encapsulation of ESP packets was introduced to allow the traffic to pass through incorrect NAT implementations that may discard non-TCP or UDP protocols, and for correct implementations this should be irrelevant (I've seen a number of SOHO routers that couldn't handle typical L2TP/IPsec connections despite that UDP encapsulation — but that's another story).

Before we can start sending ESP packets, we need to establish a "connection" (security association) though. In practice, most IPsec connections are negotiated via the IKE protocol rather than statically configured, and that's where IKE issues come into play.

IKE and pre-shared keys

To begin with, original IKE was specified to use UDP/500 for both source and destination port. Since NAT may change the source port, the specification had to be adjusted to allow other source ports. Even more issues arise when there are multiple clients behind NAT and traffic has to be multiplexed, but we'll limit the discussion to 1:1 NAT where canonical IKE might work. Let's assume the IKE exchange has gone that far.

The vast majority of IPsec setups in the wild use pre-shared keys. I've configured connections to vendors and partners on behalf of my employers and customers countless times, and I believe I've seen an x.509-based setup once or twice. In a setup with pre-shared keys, routers need to know which keys to use for which peers. How do they know? Suppose you've configured a tunnel with peer and key "qwerty". The other side has address NATed to Will it be the source address of the peer that will be used for the key lookup? Not quite.

In the IKE/ISAKMP exchange, there are identifiers. It's the pairs of identifiers and pre-shared keys that must be unique to reliably tell one peer from another and use correct keys for every peer. There are several types of identifiers specified by the ISAKMP protocol: IP (v4 or v6) address, subnet, FQDN, user and FQDN pair (user@example.com), and raw byte string.

By default, most implementations (including StrongSWAN in VyOS) will use the IP address of the outgoing interface for the identifier, and it will be embedded in the IKE packet. If the host is behind NAT, that address is a private address, When the packet passes through the NAT, the payload will obviously remain unchanged, and when it arrives, the responder will try to find the pair of in its configuration rather than that is has configured, and will send NO_PROPOSAL_CHOSEN to the NATed peer.

The solutions

So, how do you get around those issues? Whether IKE is configured to use NAT detection or not, you'll get to use peer identification methods that do not rely on known and fixed IP addresses.

The simplest approach is to use a manually configured peer identifier. This approach will work when one side is NATed and the other is not. The only thing you need to remember is that VyOS (right now at least) requires that FQDN identifiers are prepended with the "@" character. They also do not need to match any real domain name of your machine. You also need to set the local-address on the NATed side to "any".

Here's an example:

Non-NATed side

vpn {
  ipsec {
    site-to-site {
       peer @foo {
         authentication {
             mode pre-shared-secret
             pre-shared-secret qwerty
         default-esp-group Foo
         ike-group Foo
         tunnel 1 {
             local {
             remote {

NATed side

  ipsec {
    site-to-site {
         peer {
             authentication {
                 id @foo
                 mode pre-shared-secret
                 pre-shared-secret qwerty
             default-esp-group Foo
             ike-group Foo
             local-address any
             tunnel 1 {
                 local {
                 remote {
Daniil Baturin
tag:blog.vyos.net,2013:Post/1226718 2018-01-04T22:39:27Z 2018-01-21T17:34:35Z The meltdown and spectre vulnerabilities

Everyone is talking about the meltdown and the spectre vulnerabilities now. If you are late to the party, read https://meltdownattack.com/ 

Of course we are aware of it and took time to assess the risks for VyOS. Since both vulnerabilities can only be exploited locally, the risk for a typical VyOS installation is very low: if an untrusted person managed to login to your router, you are already deep in trouble and unathorized access to the OS memory is arguably the least of your concerns since even operator level users can make traffic dumps.

The fix will not be included in 1.1.9. Since the fix is associated with an up to 30% performance penalty, in 1.2.x, we will make it optional is feasible.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1224554 2018-01-01T12:28:16Z 2018-01-16T12:08:18Z VyOS mission statement

Past year the VyOS project has turned four. Perhaps it's time to update the mission statement I've made back in 2013. This one doesn't exactly contradict it, but needs to include the most important lesson we've learnt: if there is essentially no influx of commited maintainers rather than occasional contributors, the emphasis should be on finding ways to build and retain the maintainers team and enable them to put more time and effort into the project, and thus the commercial services should be a part of the mission statement.

If anyone has fears of VyOS losing anything in the freedom part, please do not fear. There will be a post about release model change that has to do with reconciling the conflicting goals of people who want latest features even at cost of stability and people who want stability, and creating an incentive for commercial users to give back to the project, but none of those come at cost of compromising the software freedom.

So, here's the updated mission statement:

VyOS is a successor of the original Vyatta

VyOS started as a Vyatta Core fork when it became clear that Vyatta Core was abandoned and no one else was going to continue it as a free open source software product. The development team and a substantial part of the user community still consist of long time Vyatta Core users and the primary reason why VyOS came into existence is to not let Vyatta Core effort to die.

Now that Brocade vRouter is no longer sold and supported, VyOS can as well be an update path for its former users, though we cannot provide a zero-effort update path for them of course.

VyOS is a free and open source platform

We believe that software freedom is very important and every user should be free to inspect the code, modify it, contribute to it, build customized images.

We will always keep the entire codebase of VyOS under free licenses. We will always keep the entire toolchain for building VyOS images open source and publicly accessible. We will try to make it as easy to use as possible. We will collaborate with vendors to support latest hardware, virtualization and cloud platforms.

VyOS is an affordable and accessible platform for a diverse user base

We know that price is a serious contributing factor that makes people turn to software-based routers. We also know that many VyOS users are networking enthusiasts and small businesses from around the world who cannot afford or not willing to use expensive hardware, whether commodity or specialized.

We will never make VyOS require any specific hardware to be fully functional. We will try to make VyOS usable on wide range of hardware, from small devices to carrier grade servers.

We are open for collaboration with hardware/software manufacturers who want to adopt VyOS

Maintainers’ work needs to be compensated

While we hope that VyOS is a community-driven project, an important thing is that community contributors typically only do the work they need themselves, but the maintainers get to maintain even features they do not use themselves and do work that is necessary but offers no reward for anyone in the short run such as code refactoring and test writing etc. This work needs to be compensated.

Besides, a large number of VyOS users are for-profit companies and use VyOS to reduce their costs. The maintainers are committed to keep VyOS free as in freedom, but will commercialize the project as well to get compensated for their work and build a team of people required to keep the development sustainable.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1209205 2017-11-29T04:15:56Z 2017-12-25T05:53:45Z Possible re-appearance of the AWS public key loading bug and what to do if you see it

There were some reports of re-appearence of the infamous bug with SSH public key loading in Amazon EC2. None of those were confirmed so far and one of them was traced to a user error, and we and our community members alike created dozens instances in different regions but could not reproduce the bug, so chances are it's a false alarm.

It's too early to become concerned, but we cannot rule out anything just yet because its previous appearence also started as intermittent and then became permanent and propagated to all regions.

To prevent a wide scale disaster, we are watching the situation closely. Past time when we could reproduce the bug ourselves it has already propagated too all regions and made the old AMI unusable for all users, so we definitely wouldn't want to repeat that. I suppose we should deploy some kind of automated test for it as well.

Meanwhile, if you have any issues logging in to a newly deployed AMI:

  1. Check carefully if the permissions of the private key file are set to 600 (rw-------). OpenSSH will say permission denied if the permissions are not restrictive enough, after telling you about permissions (http://lpaste.net/360350), but it has nothing to do with the key itself, it's just that it ignores the key with wrong permissions
  2. If you still cannot login to a newly deployed instance, please do not terminate it!. Snapshot it and contact us so that we can investigate it together.

Since marketplace AMI updates can take up to two weeks, if the bug does re-appear, we will provide a community AMI with all platform checks disabled to make the key load by default as an interim solution.

We are not ready to say if 1.1.9 will include the code that checks the platform by verifying the digital signature of the environment data, but 1.2.0 definitely will.


Right now the marketplace page of VyOS (https://aws.amazon.com/marketplace/pp/B074KJK4WC) has the sole negative review about the original SSH key issue that was written (or at least published) after the issue was resolved and the update was announced on the blog, which gives our AMI overall rating of one star. It feels rather unfair that this is all we get for working hard to fix the issue that wasn't even our fault as soon as possible and communicate it to our users.

If you are using VyOS on AWS, please consider leaving a customer review there to give it a more balanced rating.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1207813 2017-11-25T00:27:13Z 2017-12-25T05:53:45Z Jenkins maintenance

We'll be migrating the CI server (https://ci.vyos.net) to a different host this weekend. It means over this weekend there will be no automatic package or image builds for a time.

We hope this will not take too long, but in the worst case we will have it working by Monday.

Nothing else will be affected by this maintenance so the overall impact on the end users should be very low.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1206722 2017-11-22T04:07:00Z 2017-12-25T05:53:45Z VyOS cloud support platform strategy

Recently we have merged the old VyOS product on Amazon (which was free as in price) with the new one listed by Sentrium (reminder: the company setup by VyOS maintainers to provide commercial services for VyOS) that is available at ~$60/year as a means to support the project. It’s probably a good time to talk about our cloud platform support strategy.

It’s quite obvious that cloud platforms are popular, and that VyOS is often used as an alternative to their own VPN and routing facilities or proprietary products, because of both functionality and price. So far we’ve been providing our official EC2 AMI for free. This approach is not exactly sustainable since we ourselves do not benefit from it in any way, but the support level required of an AMI listed on the marketplace creates work that has to be done, and so far (past the initial contribution of the person who wrote the first AMI build scripts) it has been done solely by the maintainers.

And, let’s face it, VyOS needs funding sources. There are many activities that get virtually no community participation, including documentation writing, code refactoring, developing automated tests and so on. We are grateful to everyone who contributes, but that’s not enough to keep VyOS as a service provider and enterprise quality router. Migration to Jessie was the point when making an enterprise quality router OS by few people in spare time stopped scaling, and now, at least until we refactor the whole system to be easy to maintain and extend, either it's going to take a very long time to get done or we need a way to free the time of the maintainers and hire some community members (or someone else) to get it done at reasonable pace.

Some companies mentioned a possibility of corporate sponsorships but those offers failed to materialize, donations likewise have been enough only to pay for domains and some hosting. Switching to a RedHat-like model with paid access to LTS releases is something that we are considering, but not until a stable 1.2.0 release is made. As for now, an alternative revenue stream is needed to break the cycle, and making the official cloud images paid looks like the most viable option. This wasn't an easy decision, but it had to be made.

We've been keeping both the old (free as in price) 1.1.0 listing and new paid listing updated to 1.1.8 active for a while and so far the new one has been positively received and is already bringing us a small profit. We hope most of the old AMI users will choose to support the project as well, but if not, you have time until February 2018 to migrate your setup to self-built or community AMIs.

Most importantly

VyOS will remain free as in freedom forever — the goal of the project was and is to make a free and open source router that people can study and modify. It will always be possible to build a cloud image yourself and use it for free. While we hope most cloud users who run unmodified images will choose to support the project, trying to prevent people from building their own image would be against the free software ideals. We are also not going to hide the build scripts we use for building the official images, so self-built images will be identical to the official ones — without the convenience of deploying them in one click from the marketplace.

The AMI build scripts for 1.1.x can be found at https://github.com/vyos/build-ami

The build process is fairly straightforward and boils down to "./build-ami $isoURL". Right now it only takes signed releases and ourselves we test new images by upgrading already deployed instances to them, but we may add an option to allow building AMIs from unsigned images in the future.

Free access for contributors

The point of the whole deal is not just to make money for the maintainers, it's to make VyOS a better system and people who support VyOS by contributing to it are equally valuable as people who support it financially by using the paid images.
Everyone who had merged pull requests before this time or made substantial contribution to the documentation, automatically qualifies. New contributors who contribute to high priority areas (particularly, fix bugs in 1.2.0 and/or document the yet undocumented features) will also get free access. Active evangelists may also quality.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1205244 2017-11-14T16:41:50Z 2018-01-01T21:31:22Z 1.1.8 followup

LLDP bug

James Brown reported on phabricator that LLDP is not working in 1.1.8. Quite a mess up on our side: the reason it's not working is that it's built against the old OpenSSL 0.9.8 which is no longer in VyOS, but since the debian/control was missing dependency on libssl, it wasn't detected as dependent on OpenSSL and thus wasn't rebuilt.

If you want LLDP back right now, you can install the helium3 package from our repository (http://dev.packages.vyos.net/legacy/repos/vyos/pool/main/l/lldpd/).

OSPF route-map

One feature is missing from the changelog because it was committed "stealthily", without a task number, and thus I missed it. It's the command for setting up route-map for installing routes imported from OSPF.

set protocols ospf route-map MyMap

This route-map can prevent routes from getting installed into the kernel routing table, but make no mistake, it is not incoming route filtering (which would be very bad for OSPF). The routes will stay in OSPFd and in the RIB (they will be displayed as inactive).

The future of 1.1.x

So far it looks like we are going to make an 1.1.9 release to fix the LLDP bug. Perhaps we should also cherry-pick something safe from 1.2.0 too, if you have any specific bugfix or a tiny feature is mind, let us know.

VyOS 1.1.8 on AWS

The official 1.1.8 AMI passed the automated tests and it's on its way to the marketplace, the manual review by the marketplace team will take perhaps a week or so.

If you want to make you own, you can already use the AMI build scripts and point it to the (signed) release image URL.

And while we are at it: IPv6 in VyOS 1.2.0

I've re-enabled the old patch for IPv6 VRRP in the current branch and it will be in today's nightly build. In 1.1.x, we had to disable it because in the older keepalived version IPv4 and IPv6 VRRP were mutually exclusive, however, in the current branch, it seems to work. If you feel adventurous, please test the nightly build (on lab VMs!) and tell us it it works for you.

Also, on the forum, it was reported that the current branch image doesn't build. I've resolved that problem today so if you want to build an image, it should work.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1204936 2017-11-13T16:46:15Z 2018-01-18T06:21:14Z 1.1.8 release is available for download

1.1.8, the major minor release, is available for download from https://downloads.vyos.io/?dir=release/1.1.8 (mirrors are syncing up).

It breaks the semantic versioning convention, while the version number implies a bugfix-only release, it actually includes a number of new features. This is because 1.2.0 number is already assigned to the Jessie-based release that is still in beta, but not including those features that have been in the codebase for a while and a few of them have already been in production for some users would feel quite wrong, especially considering the long delay between the releases. Overall it's pretty close in scope to the original 1.2.0 release plan before Debian Squeeze was EOLd and we had to switch the effort to getting rid of the legacy that was keeping us from moving to a newer base distro.

You can find the full changelog here.

The release is available for both 64-bit and 32-bit machines. The i586-virt flavour, however, was discontinued since a) according to web server logs and user comments, there is no demand for it, unlike a release for 32-bit physical machines b) hypervisors capable of running on 32-bit hardware went extinct years ago. The current 32-bit image is built with paravirtual drivers for KVM/Xen, VMware, and Hyper-V, but without PAE, so you shouldn't have any problem running it on small x86 boards and testing it on virtual machines.

We've also made a 64-bit OVA that works with VMware and VirtualBox.


Multiple vulnerabilities in OpenSSL, dnsmasq, and hostapd were patched, including the recently found remote code execution in dnsmasq.


Some notable bugs that were fixed include:

  • Protocol negation in NAT not working correctly (it had exactly opposite effect and made the rule match the negated protocol instead)
  • Inability to reconfigure L2TPv3 interface tunnel and session ID after interface creation
  • GRUB not getting installed on RAID1 members
  • Lack of USB autosuspend causing excessive CPU load in KVM guests
  • VTI interfaces not coming back after tunnel reset
  • Cluster failing to start on boot if network links take too long to get up

New features

User/password authentication for OpenVPN client mode

A number of VPN providers (and some corporate VPNs) require that you use user/password authentication and do not support x.509-only authentication. Now this is supported by VyOS:

set interfaces openvpn vtun0 authentication username jrandomhacker
set interfaces openvpn vtun0 authentication password qwerty
set interfaces openvpn vtun0 tls ca-cert-file /config/auth/ca.crt
set interfaces openvpn vtun0 mode client
set interfaces openvpn vtun0 remote-host

Bridged OpenVPN servers no longer require subnet settings

Before this release, OpenVPN would always require subnet settings, so if one wanted to setup an L2 OpenVPN bridged to another interface, they'd have to specify a mock subnet. Not anymore, now if the device-type is set to "tap" and bridge-group is configured, subnet settings are not required.

New OpenVPN options exposed in the CLI

A few OpenVPN options that formerly would have to be configured through openvpn-option are now available in the CLI:

set interfaces openvpn vtun0 use-lzo-compression
set interfaces openvpn vtun0 keepalive interval 10
set interfaces openvpn vtun0 keepalive failure-count 5

Point to point VXLAN tunnels are now supported

In earlier releases, it was only possible to create multicast, point to multipoint VXLAN interfaces. Now the option to create point to point interfaces is also available:
set interfaces vxlan vxlan0 address
set interfaces vxlan vxlan0 remote
set interfaces vxlan vxlan0 vni 10

AS-override option for BGP

The as-override option that is often used as an alternative to allow-as-in is now available in the CLI:

set protocols bgp 64512 neighbor as-override

as-path-exclude option for route-maps

The option for removing selected ASNs from AS paths is available now:
set policy route-map Foo rule 10 action permit
set policy route-map Foo rule 10 set as-path-exclude 64600

Buffer size option for NetFlow/sFlow

The default buffer size was often insufficient for high-traffic installations, which caused pmacct to crash. Now it is possible to specify the buffer size option:
set system flow-accounting buffer-size 512 # megabytes
There are a few more options for NetFlow: source address (can be either IPv4 or IPv6) and maximum number of concurrenct flows (on high traffic machines setting it too low can cause netflow data loss):
set system flow-accounting netflow source-ip
set system flow-accounting netflow max-flows 2097152

VLAN QoS mapping options

It is now possible to specify VLAN QoS values:
set interfaces ethernet eth0 vif 42 egress-qos 1:6
set interfaces ethernet eth0 vif 42 ingress-qos 1:6

Ability to set custom sysctl options

There are lots of sysctl options in the Linux kernel and it would be impractical to expose them all in the CLI, since most of them only need to be modified under special circumstances. Now you can set a custom option is you need to:
set system sysctl custom $key value $value

Custom client ID for DHCPv6

It  is now possible to specify custom client ID for DHCPv6 client:
set interfaces ethernet eth0 dhcpv6-options duid foobar

Ethernet offload options

Under "set interfaces ethernet ethX offload-options" you can find a number of options that control NIC offload.

Syslog level "all"

Now you can specify options for the *.* syslog pattern, for example:

set system syslog global facility all level notice

Unresolved or partially resolved issues

Latest ixgbe driver updates are not included in this release.

The issue with VyOS losing parts of its BGP config when update-source is set to an address belonging to a dynamic interface such as VTI and the interface takes long to get up and acquire its address was resolved in its literal wording, but it's not guaranteed that the BGP session will get up on its own in this case. It's recommended to set the update-source to an address of an interface available right away on boot, for example, a loopback or dummy interface.

The issue with changing the routing table number in PBR rules is not yet resolved. The recommended workaround is to delete the rule and re-create it with the new table number, e.g. by copying its commands from 'run show configuration commands | match "policy route "'.


I would like to say thanks to everyone who contributed and made this release possible, namely: Kim Hagen, Alex Harpin, Yuya Kusakabe, Yuriy Andamasov, Ray Soucy, Nikolay Krasnoyarski, Jason Hendry, Kevin Blackham, kouak, upa, Logan Attwood, Panagiotis Moustafellos, Thomas Courbon, and Ildar Ibragimov (hope I didn't forget anyone).

A note on the downloads server

The original packages.vyos.net server is still having IO performance problems and won't handle a traffic spike associated with release well. We've setup the https://downloads.vyos.io server on our new host specially for release images and will later migrate the rest of the old server including package repositories and the rsync setup.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1202621 2017-11-01T23:49:11Z 2017-12-25T05:53:45Z A VyOS 1.2.0-alpha image with FRR instead of Quagga is available for testing (and we've found a GPL violation in VyOS)


Now that 1.1.8 release candidate is out and is (hopefully) being tested by community members, we can get back to building the future of VyOS.

It's been obvious that Quagga needs a replacement, and, since we've been using a Quagga fork inherited from Vyatta Core that includes features that never made it to the mainline quagga, even more so. The mainline Quagga still doesn't have usable commands for configuring multiple routing tables, nor they seem to actively accept patches that would be OS-specific.

The options were discussed many times and so far it seems FreeRangeRouting is the best option. It's a fork of Quagga that is being actively developed, actively accepts contributions, and already includes a number of features that Quagga lacks, such as support for network namespaces, PIM-SM, working IS-IS and more. There's also work being done on non-disruptive config reload.

While FRR is more or less a drop-in replacement for Quagga, it's not identical, and many CLI adjustments will be needed to make VyOS work with it. It needs a lot of testing. For this I've built a custom image that has vyatta-quagga replaced with FRR.

You can download the image here: http://dev.packages.vyos.net/tmp/vyos-1.2.0-alpha-frr-test.iso Please test it and report any issues in the routing protocols configuration you find. It's obviously experimental and you shouldn't use it in real routers, the best way to test is to load your production configs into test virtual machines.

GPL violation

Now to the GPL violation we've found. That violation in fact has been there for over five years and no one noticed it! Then again, it's relatively indirect and subtle.

Quagga is licensed under GPL (and so is FRR). In Vyatta/VyOS, Quagga has been built with SNMP support, so it links with net-snmp. In turn, net-snmp is built with SSL support and links with OpenSSL. This is where the problem is, OpenSSL is licensed under a four-clause BSD license that is not compatible with GPL.

Sadly, there is no easy way out, so it will take some time to fix this violation. The options are:

  • Build Quagga/FRR without SNMP support, which means routing protocol data will not be available through SNMP
  • Build net-snmp without SSL support, which means SNMPv3 will stop working
  • Patch net-snmp to support another cryptographic library that is GPL-compatible

The third option is hardest to implement, but it's the most appealing of all since all functionality will be preserved. We'd like to hear your suggestions regarding the libraries that would be license compatible and that can co-exist with OpenSSL.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1202026 2017-10-30T19:07:25Z 2017-12-25T05:53:45Z 1.1.8-rc2 release (fixes the problem with VLAN interfaces)

A new image is available for testing: http://dev.packages.vyos.net/iso/testing/vyos-1.1.8-rc2-amd64.iso 

If you are using VLAN interfaces, avoid the rc1 one and install the rc2 right away. Thanks to Samir we have identified a bug in VLAN interface configuration scripts (https://phabricator.vyos.net/T435) that prevented them from loading correctly. It was caused by an overlooked syntax error in the new options for ingress/egress VLAN QoS options.

Oddly, the current branch had a correct version of that patch, so 1.2.0 builds are not affected.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1201191 2017-10-26T19:02:38Z 2017-12-25T05:53:45Z 1.1.8-rc1 release is available for testing

The long overdue 1.1.8 release candidate is available for download from http://dev.packages.vyos.net/iso/testing/vyos-1.1.8-rc1-amd64.iso

While a number of people have already been running 1.2.0 nightly builds in production, we do acknowledge there are people who are not in position to install updates that are not completely stable, and recently discovered vulnerabilities in dnsmasq that potentially allow remote code execution are impossible to ignore (unlike many older vulnerabilities that are only locally or aren't practical to exploint).

It's stable for all practical purposes, but since it includes pretty big updates and a few new features, I suppose it's better to go through the release candidate phase. If in say a week no one finds any issues.

The release is only available for 64-bit machines at the moment. We can provide it for 32-bit, but we are wondering if anyone still wants it, when even small boards have 64-bit CPUs.

You can read the full changelog here: https://wiki.vyos.net/wiki/1.1.8_changelog_proposal 

Among package updates, there are openssl 1.0.2l and dnsmasq 2.72. Since squeeze is long EOL, the OpenSSL update required re-compiling everything that depends on OpenSSL ourselves, which took longer than we hoped.

Among VyOS fixes and features, there are user/password authentication for OpenVPN, as-override option for BGP neighbors, as-path-exclude option for route-map rules, tweakable pipe (buffer) size for netflow/sflow (too small hardcoded value could cause pmacct crash on high traffic routers), peer-to-peer VXLAN interfaces, and multiple fixes for bugs of varying severity, such as overly high CPU load on KVM guests or protocol negation in NAT rules not working.

A lot of features from 1.2.0 are not backportable due to big code changes and dependencies on way newer software versions than 1.1.x could provide, so features for cherry-picking had to be carefully chosen and even that needed quite a bit of merge conflict resolution. Quite a few of those were meant for the ill-fated "lithium" release that was supposed to be named 1.2.0 and be the last squeeze-based release, but then squeeze EOL'd, then serious life circumstances forced Alex Harpin to put all his VyOS work on hold thus leaving the maintainers team even more understaffed, and then the company we started to fund VyOS development through commercial support and services had a hard time when it almost reached the point of bankruptcy and dissolution (and, since it's self-funded, its founders almost reached the point of personal bankruptcy along with it), so by the time we could get things back on track a feature release based on squeeze wouldn't be feasible, especially considering how much we had to change to make the old codebase run on jessie. In a sense, it's a lithium that could have been, at least partially, rather than a straight maintenance release with nothing but bugfixes.
But, many of those features spent so much time in the limbo without making it into a release called stable that we felt compelled to include at least some of them.

I would like to say thanks to everyone who contributed and made this release possible, namely: Kim Hagen, Alex Harpin, Yuya Kusakabe, Yuriy Andamasov, Ray Soucy, Nikolay Krasnoyarski, Jason Hendry, Kevin Blackham, kouak, upa, Logan Attwood, Panagiotis Moustafellos, Thomas Courbon, and Ildar Ibragimov (hope I didn't forget anyone).

Daniil Baturin
tag:blog.vyos.net,2013:Post/1200619 2017-10-24T18:33:37Z 2017-12-25T05:53:45Z A book on VyOS in German is available

Our community member Markus Stubbig wrote a VyOS book in German that is now available for purchase from Amazon or bob.de.

Markus says there are no definite plans for an English version yet because he's not confident about his translation skills and will need help. If you have those skills and want to offer your services, we can connect you with Markus.

We don't have copies of the book yet (and none of the VyOS maintainers are fluent in German either, though I can read it a little), we cannot provide any kind review of the book yet, but we have no reasons to doubt Markus' expertise. If you get the book and write a review, we may publish it in the blog.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1200021 2017-10-22T05:48:29Z 2017-12-25T05:53:45Z Update on the AWS SSH key fetching issue

We have fixed the issue with key fetching and submitted the updated AMI for review. It passed the automated scan, but manual review and deployment to the marketplace will take some time.

The new AMI also includes updates for dnsmasq security vulnerabilities that will be included in 1.1.8. If you want to install those updates on 1.1.7 by hand, you can use these packages: http://dev.packages.vyos.net/tmp/dnsmasq/

Daniil Baturin
tag:blog.vyos.net,2013:Post/1199492 2017-10-19T15:41:39Z 2018-01-12T20:48:58Z Permission denied issues with AWS instances

Quick facts: the issue is caused by an unexpected change in the EC2 system, there is no solution or workaround yet but we are working on it.

In the last week a number of people reported an issue with newly created EC2 instances of VyOS where they could not login to their newly created instance. At first we thought it may be an intermittent fault in the AWS since the AMI has not changes and we could not reproduce the problem ourselves, but the number of reports grew quickly, and our own test instances started showing the problem as well.

Since EC2 instances don't provide any console access, it took us a bit of time to debug. By juggling EBS volumes we finally managed to boot an affected instance with an disk image modified to include our own SSH keys.

The root cause is in our script that checks if the machine is running in EC2. We wanted to produce the AMI from an unmodified image, which required inclusion of the script that checks if the environment is EC2. Executing a script that obtains an SSH key from a remote (even if link-local) address is a security risk since in a less controlled environment an attacker could setup a server that could inject their keys to all VyOS systems.

The key observation was that in EC2, both system-uuid and system-serial-number fields in the DMI data always start with "EC2". We thought this is a good enough condition, and for the few years we've been providing AMIs, it indeed was.

However, Amazon changed it without warning and now the system-uuid may not start with EC2 (serial numbers still do), and VyOS instances stopped executing their key fetching script.

We are working on the 1.1.8 release now, but it will go through an RC phase, while the solution to the AWS issue is needed right now. We'll contact Amazon support to see what are the options, stay tuned.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1194057 2017-09-26T16:17:44Z 2017-10-21T11:03:14Z SDN NFV WORLD CONGRESS 2017


Layer123 organize event called SDN NFV WORLD CONGRESS 2017 in The Hague, NL.

It take place between 9 and 13 of october.

I was lucky enough to win conference pass on drawing organized by Netronome guys. 

Thank you for such opportunity Netronome!

If someone would like to meet there to talk about VyOS/OSS and drink some beers - count on me :)

Yuriy Andamasov
tag:blog.vyos.net,2013:Post/1189711 2017-09-08T11:51:02Z 2017-10-21T11:04:18Z VyOS development digest #10
At last, there are some news. In the order of immediate importance...

1.1.8 release plan

There have been some uncertainty over this issue and it wasn't clear if we'll be able to make an 1.1.8 release or not with squeeze's death, but recently Kim and I got squeeze builds to work again, and this enables us to finally make one.

What's certain is that bugfixes from 1.2.0 are going to make it there. What's not yet certain is which features we should cherry-pic. OpenVPN user/password auth, for example, is definitely safe and well tested enough to bring it to 1.1.8.

1.2.0 development status

1.1.8, of course, is nothing more than a maintenance release. But, we are way closer to a full feature release now that, especially with the work done by two awesome contributors, namely Christian Poessinger and Jules Taplin. Among recent contrubutions are multiple fixes to IPsec operational and configuration mode (in particular, "show vpn ipsec sa" works properly now), correct deletion of VTI interfaces, and there's also work being done on integrating mDNS repeater.

1.2.0, Python, and code rewrites

This was already discussed in http://blog.vyos.net/vyos-2-dot-0-development-digest-number-7-python-coding-guidelines-for-config-scripts-in-1-dot-2-0-and-config-parser-for-vyconf and http://blog.vyos.net/vyos-2-dot-0-development-digest-number-5-doing-1-dot-2-x-and-2-dot-0-development-in-parallel

By now, the Python library is "beta" rather than "alpha" and it has already been used to rewrite the cron ("set system task-scheduler") scripts by Tania Dzyubenko and me.

The library is now a proper Python package and it's installed as vyos.config module. You can use it for VyOS scripting, as well as code rewrites.

It has also been moved out of the vyatta-cfg package. The package where the new rewritten code goes is https://github.com/vyos/vyos-1x

You can find the rewritten cron script here: https://github.com/vyos/vyos-1x/blob/current/src/conf-mode/vyos-update-crontab.py
As you can see, it's architecturally pretty different from the older scripts. You can find the guideline it's written according to here in the wiki: https://wiki.vyos.net/wiki/Python_coding_guidelines

The architecture boils down to this: all VyOS config reads are confined to one function that converts it into an abstract representation, the rest of the logic is split into separate "verify", "generate", and "apply" stages that, accordingly, verify config correctness, generate configuration files, and apply them to the live system.

I'll re-iterate the reasons for these changes:
  • Testability: if only one place in the code really needs VyOS to work, the rest can be test on developers' workstations and build hosts, by hand as well as with automated unit and integration tests
  • Easier syntax changes: same, redesigned syntax can translate to the same abstract representation or a modified version of it, and there will not be need to weed out hundreds instance of the old syntax all over the script
  • Transactional commits: if the config correctness checking stage is clearly separated, once all scripts are rewritten in this manner, it will be possible to implement commit dry-run and abort commits if an error is detected before any change to the live system is made, thus greatly increasing the system's robustness

Scripts written in this manner will be reusable in VyOS 2.0 once it's ready with little change, thus ensuring more gradual rather than radical rewrite.

2.0 style command definitions in VyOS 1.2.0

If you look into the vyos-1x package, you will notice that there are no command "templates". That's right.

As you remember, the future VyOS 2.0 and its config backend will not be using the old style command "templates" (bunches of directories with node.def files). There is no way to get rid of them in VyOS 1.x, but we still can abstract them away, thus enabling a more gradual rewrite in this area too.

There are multiple problems with those old style templates. They are notoriously hard to navigate even for experienced developers and are a repellent for newcomers. They are equally hard to syntax check and the only real way to find out if they have any chance to work is to install a package on a test VyOS instance and try them by hand.
And last but not least, they allow embedded shell scripts that further spread the logic all over and make debugging even harder than it already is.

New style templates are in XML. Before anyone says "why not JSON", tell me if JSON got a widely accepted schema language and its implementation (I'm aware of some attempts, but...). XML had been machine-verifiable for almost two decades already.
XML interface definitions for VyOS come with a RelaxNG schema (https://github.com/vyos/vyos-1x/blob/current/schema/interface_definition.rng), which is a slightly modified schema from VyConf (https://github.com/vyos/vyconf/blob/master/data/schemata/interface_definition.rnc) so the definitions will be reusable with very minimal changes.

The great thing about schemas is not only that people can know the complete grammar for certain, but also that it can be automatically verified. The scripts/build-command-templates script that converts the XML definitions to old style templates also verifies them against the schema, so a bad definition will cause the package build to fail. I do agree that the format is verbose, but there is no other format now that would allow this. Besides, a specialized XML editor can alleviate the issue with verbosity.

Right now that script is complete enough to produce the templates for cron, but there's still work to be done. For example, it doesn't support the "allowed:" statement used for command completion. Any testing and patches are greatly appreciated!

Daniil Baturin
tag:blog.vyos.net,2013:Post/1161907 2017-06-08T19:19:05Z 2017-09-06T11:31:14Z issues with wiki/forum resolved now


issues with wiki/forum resolved now.

Problem was caused by raid rebuild performance degradation, but now back to normal

Yuriy Andamasov
tag:blog.vyos.net,2013:Post/1161474 2017-06-07T11:48:32Z 2017-06-07T11:48:32Z wiki/forum

Hi, as you already noticed we experiencing issues with host where wiki and forum are hosted

I will let you know once services restored

Yuriy Andamasov
tag:blog.vyos.net,2013:Post/1140890 2017-03-23T02:44:02Z 2017-05-06T20:56:46Z Donations and other ways to support VyOS

Hello, community!

We got many requests about how you can donate, we decided open this possibility to those who asked

After all, this is direct support to the project that all you offer, and we constantly need a support of all types.

As was mentioned before, you can contribute in many ways:

But if you would like to contribute via donation you are welcome to do so!

Raised money will be used for project needs like:

  • Documentation development
  • Tutorials and training courses creation
  • Artwork creation
  • Travels of project maintainers to relevant events 
  • Event organization
  • Videos
  • Features development 
  • Popularization of VyOS
  • Servers
  • Lab
  • Software
  • Hardware

Of course, that is not a complete list of needs that project have but most visible.

Thank you!

Yuriy Andamasov
tag:blog.vyos.net,2013:Post/1128826 2017-02-05T18:10:58Z 2017-07-31T18:41:05Z VyOS 1.2.0 repository re-structuring

In preparation for the new 1.2.0 (jessie-based) beta release, we are re-populating the package repositories. The old repositories are now archived, you still can find then in the /legacy/repos directory on dev.packages.vyos.net

The purpose of this is two-fold. First, the old repo got quite messy, and Debian people (rightfully!) keep reminding us about it, but it would be difficult to do a gradual cleanup. Second, since the CI server has moved, and so did the build hosts, we need to test how well the new procedures are working. And, additionally, it should tell us if we are prepared to restore VyOS from its source should anything happen to the packages.vyos.net server or its contents.

For perhaps a couple of days, there will be no new nightly builds, and you will not be able to build ISOs yourself, unless you change the repo path in ./configure options by hand. Stay tuned.

Daniil Baturin
tag:blog.vyos.net,2013:Post/1127419 2017-01-31T00:08:48Z 2017-01-31T00:08:48Z Phabricator migration is over

Hello there!

We finished migration of phabricator, 

and also used this opportunity to add SSL to forum and wiki, so now all resources using a secure connection.


Yuriy Andamasov
tag:blog.vyos.net,2013:Post/1126761 2017-01-28T01:28:11Z 2017-01-28T01:28:11Z Phabricator migration

I know you are tired of this already, but... yes, you guessed it right, we are migrating the phabricator again!

This time we are moving it to the host that currently houses the wiki, a VM at OpenITC (thanks, Sean!). This should be the last migration for a long while. We plan to consolidate all web resources on that host: while this is not so good for redundancy, it's easier to manage. Since the blog is not hosted there, and we also have a twitter, we still have out of band channels to notify people about outages and resolution timeframe estimates, should that server ever failed.

Today we'll shutdown the phabricator to migrate all the data on the new host and re-deploy it there (it would be nice to put it in read only mode instead, but it currently doesn't have it, it's planned for future versions). We also need to reconfigure the web server for the new setup, so there may be short periods of downtime for the wiki too.

We'll notify you when migration is complete.

Daniil Baturin