IPng Networks

Sunsetting SixXS

Posted:

  • Author: Pim van Pelt, Jeroen Massar
  • Contact: <staff@sixxs.net>
  • Date: March 2017
  • Status: Draft - Review SixXS - Review Admins - Final - Published

Summary

SixXS will be sunset in H1 2017. All services will be turned down on 2017-06-06, after which the SixXS project will be retired. Users will no longer be able to use their IPv6 tunnels or subnets after this date, and are required to obtain IPv6 connectivity elsewhere, primarily with their Internet service provider.

Introduction

SixXS (Six Access) is a free, non-profit, non-cost service for Local Internet Registries (LIR’s) and endusers. The main goal is to create a common portal to help company engineers find their way with IPv6 networks deploying IPv6 to their customers in a rapid and controllable fashion. To reach our goals, SixXS provides the following services:

  • IPv6 Tunnel Broker: a versatile and high performance IPv6 tunneling router
  • Ghost Route Hunter: an IPv6 route monitoring tool and various other services to help out where needed
  • IPv6Gate HTTP Proxy: IPv6-IPv4 and IPv4-IPv6 Website Gateway

SixXS has offered the RIPE, ARIN, APNIC, LacNIC and AfriNIC communities pre-production deployment expertise based on the experience gathered while running the IPng IPv6 tunnel broker since 1999 and, combined with its successor, SixXS, gaining more than 18 years of valuable IPv6 experience.

Userbase

As of March 2017, there are 38’393 7-day active users spanning 140 countries. These users configured a total of 44’673 tunnels spanning 118 countries, and 12’632 subnet delegations (28.28%). Our peak 7DA usage was over 50’000 users. Full statistics, including distributions by country, can be found on the SixXS website [link].

Growth

User engagement over time (as shown in the graphs below), shows strong growth from 2001-2011, followed by a stagnation and then decrease of new users and subnets leading through 2016. We believe this is due to saturation, all users with the ability and desire to receive service, obtained an account, a tunnel and a subnet.

Users/Year | Tunnels/Year

Images: Left - New users per year; Right - New tunnels per year.

Another way to visualize this data is to measure the cumulative requested subnets (which are /48 in size). The requests for subnets naturally follows the growth of users. In recent years (2014 onwards), requests for additional subnets were clearly tapering off - this is in line with our goal of SixXS. Therefore, in December of 2015, new user signups were suspended. Note: this explains the flatline of requests in the years 2016 and 2017.

Cumulative Requests/Year Image: Cumulative requests over time.

Traffic

As the project evolved, traffic initially grew significantly, with a daily average of just around 900Mbit/sec. We note the trend of traffic is on a downwards trajectory since H2 2015. We believe this is in part due to ISPs starting to offer IPv6, which yields organic attrition of users migrating away. This trend is in line with the goals of the project.

Users engage with SixXS passively - once they set up their tunnel and configure their router or computer, the system is largely zero-touch. Traffic is consistently diurnal with a 7:1 ratio between peak and trough, which indicates that the traffic flowing through the system is roughly equivalent to what access providers see. This has changed over time - in the early days of IPv6, the major use case was NNTP and IRC, now general Internet usage with the larger content providers all support IPv6.

Comparing our traffic pattern to a well known Internet exchange point [amsix], we see similar changes. Today, many of the larger content providers have offerings on IPv6 which shifts the usage of our service also more towards HTTP (and, to a diurnal pattern).

The following two graphs illustrate the usage: The first graph is average traffic in bits/sec between 2012 and 2017. Second graph is average traffic in bits/sec between 2017-02-24 and 2017-03-03.

image | image

Images: Left - traffic since 2012. Right - weekly traffic pattern Feb 2017.

Footprint

Looking at the total footprint of SixXS - In March 2017, 46 PoPs spread over 29 countries were offered by 40 unique Internet service providers. Over the lifetime of SixXS, a total of 65 different PoPs have been active.

image

Image: Map of SixXS PoPs

The full list of countries hosting PoPs: Australia, Belgium, Brazil, Czech Republic, Denmark, Estonia, Faroe Islands, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Luxembourg, Netherlands, New Caledonia, New Zealand, Norway, Poland, Portugal, Russia, Slovenia, Sweden, Switzerland, United Kingdom, United States and Vietnam.

Our deployment comprises the following subnet allocations: 85x /40 which is equivalent to a total of 21’760 /48’s or 1x /34 + 1x /36 + 1x /38 + 1x /40 – to cover our usage fully would require an IPv6 /33. We believe the SixXS footprint is one of the largest IPv6 deployments in the world, and are incredibly proud of the accomplishments and contributions we have made to the community.

Rationale

The mission of SixXS is to help prepare companies and individual users for the world of IPv6. Started in 1999, our goals were to build a distributed tunnelbroker system, which allowed professional and hobbyist users to learn how to operate IPv6 networks so that they would roll out IPv6 natively across the Internet. Back in 1999, we wrote: Our ultimate target is to conclude the tunnel brokering service when all the end users can get Native IPv6 directly from their own Internet provider.

For a decade, the industry was divided into content providers and access providers engaged a chicken and egg game:

  1. Content providers claimed that investing in IPv6 rollout would be useless because there were not sufficient numbers of large ISPs which offered it.
  2. Access providers claimed that investing in IPv6 would be useless because there were not sufficient numbers of large content providers which offered it.
  3. Both content providers and access providers claimed their customers didn’t demand it and there was no business justification in doing so.

One raison d’être of SixXS is to help break this cycle by offering IPv6 connectivity to users, so that they in turn can demand that content providers offer service over IPv6, and to companies, so their engineers can learn the intricacies of rolling out IPv6 for their business safely. After 18 years, here is where we stand:

  1. Content providers, largely, have switched to IPv6. Examples: Wikipedia, Google, Youtube, Facebook, Akamai, Netflix, Microsoft, Yahoo.

  2. Access providers are starting to move on IPv6 deployments: 18% of the Internet has IPv6 connectivity, roughly doubling year over year.

  3. The access providers generally still claim that there is not sufficient customer demand to invest in IPv6.

Today, SixXS plays an insignificant role in converting the opinion on (1), (2) and (3) and is more recently (2016 and beyond) being quoted by several large access providers as a compelling alternative for their few customers who asked for IPv6, along the worrying lines of “SixXS and Hurricane Electric offer tunnels, so we are not planning to provide native IPv6 at this time”.

A call to action in 2016, asking our users to call their ISP and ask about rollout plans, yielded some reasonable engagement from SixXS users (for which we are incredibly grateful), but arguably disappointing results from the ISPs, particularly the very large ones. Extensive data can be found on the SixXS wiki page [link].

Conclusion

Building up to our conclusion, we make some critical observations:

  1. SixXS penetration has hit a point of diminishing returns (see the ‘Growth’ subsection of this document).

  2. Content providers have shown great progress in enabling users to reach their websites via IPv6, in our opinion formally breaking the chicken and egg problem.

  3. Access providers have shown reasonable interest in providing IPv6 to users, but some have started to quote SixXS as a reason they do not have to show an interest.

  4. Consumers should not have to be involved in the discussion as they largely need not know, or care, how the Internet works, as long as they can reach the Internet resources they want, when they want them.

Our conclusion is that SixXS is no longer able to contribute to the solution, and is hampering its own goals of facilitating the migration of consumers to native IPv6. We have therefore decided to shut down our services on 2017-06-06.

Accomplishments

When we started in 1999, we set ourselves some pretty ambitious goals. They are described on our website as value propositions to ISPs, to endusers and we explain in detail what targets we want our project to achieve in order to help the technical community.

To the latter point (why do this?), we have by far exceeded our targets of creating 10 regional PoPs (we created 65, each using their own IPv6 address space); we developed and rolled out a provisioning system that manipulated tunnels and subnets w/o human intervention; gathered a wide array of statistics (traffic, latency, uptime, debugging); set up Multicast, DNS and DNSSEC delegations; and allowed for a feature rich web- and console interface to manage the system.

We felt that having zero-touch PoP servers would likely yield less probability for human error to cause outages, and we were right: in 18 years of operation we cannot remember any wide scale outages caused by human error.

However, we did not stop at writing solid automation. The following are notable contributions that SixXS has made to IPv6 deployment and the Internet in general:

  1. PoPs on five major continents, missing Africa and Antarctica
  2. sixxsd - software based router with high performance tunneling support
  3. Heartbeat protocol (IANA port 3740) and IETF draft
  4. AYIYA protocol (IANA port 5072) and IETF draft
  5. Community outreach (example RIPE, IETF, ISOC, AMS-IX IPv6 Awareness Day)
  6. Research done with GRH (Ghost Router Hunter) to:
    • Help eradicate global IPv6 routing issues (Ghost Routes)
    • IPv6 Bogon Route detection
    • Distributed Looking Glass and Traceroute
  7. AICCU (Automatic IPv6 Connectivity Client Utility)
    • Automatic setup of IPv6 connectivity providing only username + password.
    • Awards of Excellence in the Implementation Award Category in the IPv6 Application Contest 2004
    • Incorporated into commercial Draytek, ZyXel, Motorola CPE products
  8. Heartbeat & TIC support out the box in AVM Fritz!Box
  9. Very few outages over 18 years of operation
  10. Pre-production access to Google and Wikipedia IPv6 servers through SixXS DNS recursors

Over time, social and technical media picked up on SixXS activities and regularly acknowledged the significance of the project. For example at Heise, Linux Journal, and in other places.

Acknowledgements

As SixXS founders, we were not operating in a vacuum. We have had countless interactions with institutions and corporations, many mentors, and like-minded engineers across the world. We wanted to take this opportunity to call out these formative folks:

  • TU/e and MCGV Stack (the genesis of IPng in 1997)
  • Intouch (the continuation of IPng and first sponsor of SixXS in 1999)
  • Concepts ICT (long time sponsor of SixXS, including our NOC hardware)
  • HEAnet (long time sponsor of SixXS since 2002)
  • BIT (long time IPv6 advocate, and long time sponsor of SixXS since 2004)
  • AMS IX (for organizing the AMS-IX IPv6 Awareness Day in Oct 2002)
  • ISOC and Steve Deering (for inventing IPv6, and meeting us in Amsterdam)
  • IPv6 Flag Day effort (for addressing the chicken-and-egg problem)
  • IPv6 Ops (for an engineering focused mailing list)
  • Heise (for their kind attention over the years)

And lastly, we extend our gratitude to the men and women who professionally operate the network, those who arranged the physical or virtual hardware, and those who are in a position to commit to running all 65 SixXS PoPs!

FAQ

Will you reconsider your decision?

A lot of thought has gone into this decision. While we do understand that the service SixXS provides is very valuable to its users, we have seen the growth of IPv6 content providers as well as IPv6 access providers is exponentially growing. We are of the belief that IPv6 Tunnel Brokers are no longer facilitating access providers moving to IPv6, and as such do not wish for the project to be continued. We will not reconsider our decision.

Will you hand over the project to other folks?

We are fairly protective of our brand and position in the community. Due to the nature of SixXS, which rests on an open source client (aiccu) with a closed source server (sixxsd), we are not willing to hand over the project. However, that aside, the main justification for our decision as outlined in this document, is that we are of the belief that IPv6 Tunnel Brokers are no longer facilitating access providers moving to IPv6, and as such do not wish for the project to be continued. Handing it over to other folks will not allow us to satisfy our concerns.

Do you need help?

Running SixXS servers and infrastructure is very efficient and does not demand much time. The PoP servers are stateless and can be brought up based on a Debian or Ubuntu base install in a matter of minutes. Handling user questions is stressful at times due to the volume of requests, but overall it’s manageable because most of our user interaction is self-service on the website. For operating SixXS, we do not believe time is a major concern.

That said, help can be more productively offered in the area of IPv6 deployment in Internet content providers and access providers. If you are active in those areas, we would greatly appreciate it if you would champion with your leadership and engineering teams to roll out IPv6 for your users!

What happens to the users?

We will mark all the tunnels and subnets as deleted on 2017-06-06 at which point they will stop forwarding traffic. Users will not have access to IPv6 through SixXS anymore on that date. We will return all resources to the Internet service providers, shut down the PoPs and delete all personally identifiable data from our database.

What happens to the servers (PoPs)?

The servers will be shut down and returned to the ISPs who own them, as SixXS itself does not own the PoP servers. The SixXS website will continue to run on private servers, mostly serving as a tombstone documenting our efforts over the years.

What happens to the PII data you have?

In the lifetime of the project, we have taken privacy and individually identifiable information very seriously, and to our knowledge have never suffered an information breach or leek. After decommissioning the services we run, we will destroy all PII data, keeping only traffic statistics at the PoP level, and general statistics of our usage, like the graphs seen in this document.

What happens to WHOIS entries in RIPE and APNIC databases?

As we mark the tunnels and subnets as deleted, our automation will automatically purge these records (inet6num) from the RIPE and APNIC databases. We will also return the subnets SixXS operates on behalf of the PoP owners (these /40 supernets will be returned to the LIRs).

For user (person) records, see FAQ entry (Do I have to delete my RIR handle).

What is your timeline?

Our timeline for the sunset of our services started in early March 2017, traversing a dialog with PoP administrators in April, notifying users at the end of April, and offering 6 weeks of time for folks to find alternative solutions.

On 2017-06-06 we will shut down services, and close the sunsetting project on 2017-07-01.

What are my alternatives?

Users are encouraged to call their ISP for IPv6 connectivity – ultimately that is the best way forward. Provided sufficient numbers of paying customers request IPv6 service, ISPs may be compelled to invest in offering service.

There is sometimes healthy competition. ISPs are critically interested in retaining users. If a specific ISP offers IPv6 to users – consider switching providers to obtain native connectivity and making note of this with the old ISP.

In cases where this proves infeasible, there are myriad other IPv6 Tunnel Brokers available.

Will you open source SixXS code?

We do not currently have plans to open source any code that is not already publicly available. Although our provisioning servers and routing daemon (sixxsd) are very well thought out, they do have some intricate dependencies on how we built SixXS. As such, offering the code base will not be necessarily useful for others. Over time, given effort on Jeroen’s part, this may change. We cannot make any promises at this point.

Can I still use www.sixxs.net as connectivity checker (smokeping et al)?

We are aware (simply by looking at access logs), that many users have pointed their IPv6 network monitoring at our website. Among these users are some quite large companies as well. You can rest assured that we will keep www.sixxs.net running on highly available distributed servers ongoing. It is likely that the website will become static, making it somewhat more reliable than it already is.

What happens to the SixXS domains (sixxs.{com,net,org,…})?

While we are retiring our services, the website will remain up. All domains will remain delegated to the current nameservers, although only www and MX will remain. In particular, the PoP hosts and tunnel hostnames like cl-x.pop-yy.tld.sixxs.net will be removed.

I am hosting my nameserver, mailserver, etc on my tunnel, what happens?

It is quite common and entirely acceptable for folks to point NS or MX records towards their tunnel name (cl-x.pop-yy.tld.sixxs.net). Hosting servers with content behind the tunnels is also common. Considering the DNS entries for sixxs.net will cease to exist, and the tunnels will be decommissioned, users are recommended to move their services to other IPv6 providers (either colocated, natively routed to home or office connections, or behind another IPv6 Tunnel Broker if no other options exist).

Do I have to delete my RIR handle (RIPE/ARIN/APNIC/LacNIC/AfriNIC)?

If you are using the handle for other business, obviously you should keep them. If SixXS was the sole purpose of registering such a handle (we have required them for many years), we recommend removing your handle from the relevant RIR database. The call is ultimately yours, as SixXS does not own that data.

What happens with the ULA registry?

This registry was for educational purposes, and has no official status. Importantly, as ULA is random per definition, the chance of collisions is extremely low. It is fairly straight forward to get a prefix from one of the RIRs. While that does cost some money, it is an activity that is not the main charter of SixXS and will be expected to continue with the RIRs.

Timeline

2017-03-01: T-14wk Decision made by Jeroen and Pim to start the sunset project.

2017-03-06: T-13wk Communicate to PoP admins (e-mail, referencing intent and problem statement)

2017-03-13: T-12wk Communicate to PoP admins (e-mail, details (this doc))

2017-03-20: T-11wk Wrap up feedback from PoP admins, prepare publication to the users. Initial backup of SixXS PoPs completed.

2017-03-23: T-11wk Publish to SixXS website. One-time mail to all users, noting the sunset date and pointing to rationale and FAQ.

2017-03-29: T-10wk Communicate and field response from IPv6 communities, social media, et al.

2017-04-17: T-7wk Due date for converting the SixXS website to a static replica, for posterity.

2017-05-29: T-1wk Convert info@sixxs.net to an autoresponder pointing at rationale+FAQ.

2017-06-06: (Tuesday) Turn off TIC and SixXSd on PoPs, retire IPv6Gate, shut down whois server.

2017-06-12: T+1wk: Secondary backup of SixXS PoPs completed.

2017-06-19: T+2wk: Power off SixXS PoPs, IPv6Gate, return resources

2017-06-26: T+3wk: Destroy PII data (mysql database).

2017-07-01: Close out the sunset project.