IPng Networks

Fiber7 on LiteXchange

Posted:

Introduction

Init7 logo

In a pilot of the Fiber7 product on the LiteXchange platform, the author took service to vet the product stability and quality. The pilot ran from 2016-09-25 to 2016-10-12, in which the Fiber7 connection was used exclusively by the author in their home internet connection, both for IPv4/IPv6 service as well as IP Television (via Init7) and IP Telephony (via a third party provider).

Executive Summary

Fiber7 via ‘direct connect’ on the LiteXchange platform works as expected and very satisfactory, including native IPv6, which was made available for this pilot. Throughput, latency and jitter are superior due to the direct fiber connection, and significantly better than existing connections, exceeding expectations compared to competing FTTH offerings that use customer premise equipment. IPTV worked correctly with multiple STB devices.

Detailed findings

Architecture

The author currently has a subscription via EasyZone, an Init7 subsidiary, with gigabit ethernet symmetric connectivity. The EasyZone product delivers service via the LiteXchange platform, which is an L2 broker, offering end users a choice of multiple internet providers [site]. An ONT is supplied, an ISP managed device that takes the fiber connection and exposes service via one of four gigabit ethernet copper ports. The Fiber7 product delivers via LiteXchange a direct fiber connection without the ONT. Fiber7 offers a range of termination options, including plugging the fiber into a provided CPE (AVM Fritz!Box [vendor] or alternatively MikroTik RB2011UiAS [vendor]), a media converter (TP-Link [vendor]), or simply an SFP (Flexoptix [vendor]) to use in customer provided switch/router infrastructure.

Architecture (details)

For this pilot, we chose a barebones connection type, consisting of a bidirectional SFP (Flexoptix [vendor]) directly terminating the FTTH connection from OTO position 1 into our own managed switch (Unifi US-24-500W [vendor]). The L3 routers used are a pair of PC routers (PC Engines APU2 [vendor]), running Linux. They are configured in CARP failover on egress (to Fiber7) and ingress (to local network).

Configuring IPv4 address on egress interface is done via DHCP - initially, DHCPv6 was not active on LiteXchange, so a local tunnelbroker (SixXS, hosted at Init7) was used. Within one week, the engineers at Init7 informed me that DHCPv6 was ready, and it worked spotlessly after configuring it to request an NA and a /48 PD, and bumping accept_ra=2 on the egress interface (note: this allows forwarding while at the same time accepting router advertisements).

Additional details of the L3 connection:

  1. The routers operate an L2VPN to a third party provider (IP-Max, AS25091) which routes 194.1.163.32/27 via eBGP using GRE. The MSS on this tunnel is clamped to 1436 (from 1460) to allow for encapsulating IPv4 and GRE. AS13030 and AS25091 meet at CIXP in Geneva, with a round trip time of 4.2ms.
  2. The routers operate an IPv6 tunnel to a common tunnel provider (SixXS, AS13030), which routes 2001:1620:fb6::/48 via AICCU using SIT to the active router. The MTU is set to 1440 bytes to allow encapsulating IPv6 in IPv4. Note that the Fiber7 connection via LiteXchange provides native IPv6 as well, so this tunnel is used only via a secondary IPv4 uplink.
  3. The routers operate native IPv6 – with DHCPv6, a /128 address and a /48 delegated prefix are obtained. This prefix is stable due to the use of DUID client identification. The default gateway is obtained via RS/RA. For IPv6, reversed DNS delegation for fixed DUID/PD delegation is provided.

It is worth pointing out the very low technical entry barrier to both IPv4 and IPv6. The termination is principally plug and play. An end user can use standard issue DHCP for IPv4 and RA/RS for IPv6. DHCPv6 is not widely used - but similarly the /48 prefix acquisition is hasslefree.

Failover between the routers is managed by a script that swaps the CARP [source] master to the standby PC router (automatically in case of CARP heartbeat timeouts; or manually in case of maintenance), ensuring the L2VPN, DHCP client, and IPv6 tunnels are running on the active machine.

Policy based routing [source] is used to separate Fiber7/SixXS and L2VPN/IP-Max routing domains. Routing tables are maintained with a popular open source routing platform called BiRD [source], OSPF between the PC routers, and eBGP with the third party provider.

IP Television

In this pilot the author was sent an IPTV device (Amino Aminet A140 [vendor]), which operates with IPv4. The device acquires video streams using IPv4 multicast. Setting this up was straightforward, using an IGMP Proxy [github] also used in commercial CPEs. The IGMP Proxy was configured on the PC routers.

With two such Amino IPTV devices, tuning in to SRF1 and SRF2 (both HD channels), a stream of UDP from multicast servers within the Init7 network was started. At the time of writing, SRF1 is on multicast address 239.44.0.77 port 5000; SRF2 is on multicast address 239.44.0.78 port 5000; both coming from source 109.202.223.18 port 5000. Average bandwidth was 13.0Mbit/s with a peak of 17.1Mbit/s per HD stream, and 4.2Mbit/s with a peak of 5.3Mbit/s per SD stream.

Multiple Amino IPTV devices in multiple backend VLANs can be used at the same time:

$ ip mroute | grep 239.44.0
(109.202.223.18, 239.44.0.77)    Iif: eth0.9     Oifs: eth0
(109.202.223.18, 239.44.0.78)    Iif: eth0.9     Oifs: eth0.2

A list of channels available on the EasyZone IPTV provider (a subsidiary of Init7) can be found on their website [source].

Netflix: IPv6

Worth noting during the pilot is that Netflix, a popular online television streaming service [website], was served from within the Init7 network as well. Connections were observed from host netflix-cache-1.init7.net (AS13030) via IPv6, which is impressive.

UHD (4K) streaming is also available with Netflix - the device used to test this (Samsung JU7080 Series 7 [vendor]) has a native client but it does not support IPv6, as such the traffic was observed from host ipv4_1.cxl0.c117.ams001.ix.nflxvideo.net in AS2906 located in the Netherlands.

In both cases (local within Init7 and remote to AS2906), Netflix streaming was free of interruptions and great quality.

Test Results

Throughput

A throughput test was started on September 27, lasting 12 hours, from the active PC router to a machine in the Init7 network [caveat]:

$ traceroute to chzrh02.sixxs.net (213.144.148.74), 30 hops max, 60 byte packets
 1  77.109.172.1.easyzone.ch (77.109.172.1)  0.755 ms  0.813 ms  0.803 ms
 2  r1zrh2.core.init7.net (77.109.183.61)  0.379 ms  0.373 ms  0.377 ms
 3  r1zrh1.core.init7.net (77.109.128.241)  0.477 ms  0.429 ms  0.397 ms
 4  r1zlz1.core.init7.net (77.109.128.210)  8.810 ms  8.783 ms  8.738 ms
 5  chzrh02.sixxs.net (213.144.148.74)  0.545 ms  0.490 ms  0.469 ms

Using a popular network bandwidth tool (iperf [source]), IPv4 bandwidth was measured for 10 minutes each, both upstream (from the PC router to a machine in the init7 network: 891Mbit), and downstream (from the init7 machine to the PC router: 895Mbit). In IPv6, the results were similar (771Mbit upstream, and 831Mbit downstream).

A standard internet test was performed (Speedtest.net, using Init7) [link; results], yielding 925Mbit downstream and 893Mbit upstream. In addition to the direct link, the author’s L2VPN connection to a third party provider was tested (Speedtest.net, using Init7) [link; results], yielding 609Mbit downstream and 578Mbit upstream. The L2VPN throughput regression is explained by tunneling en/decapsulation.

Latency

Latency to Google was tested – Init7 AS13030 and Google AS15169 meet in Zurich, with very low latency. IPv6 was tested twice (once via SixXS tunnelbroker tunnel, and once natively when it was available). Tunneled IPv6 reports slightly elevated latency due tunneling to an on-net IPv6 tunnelbroker[caveat]. Native IPv6 reports equivalent latency to IPv4.

IPv4 google.com ping statistics:
10 packets transmitted, 10 received, 0% packet loss, time 9002ms
rtt min/avg/max/mdev = 0.566/0.579/0.594/0.025 ms

Native IPv6 google.com ping6 statistics:
10 packets transmitted, 10 received, 0% packet loss, time 9015ms
rtt min/avg/max/mdev = 0.705/0.771/0.828/0.043 ms

Tunneled IPv6 google.com ping6 statistics:
10 packets transmitted, 10 received, 0% packet loss, time 9011ms
rtt min/avg/max/mdev = 1.154/1.451/2.206/0.276 ms

Caveats

IPv6 was initially not natively available on this connection. IPv6 was tunneled via chzrh02.sixxs.net (on-net at AS13030). The IPv6 server endpoint runs on a virtualized platform, with slightly less than bare-bones throughput. Shortly thereafter, native IPv6 was configured on the Fiber7 product via the LiteXchange platform.

Each OTO delivered by the city of Wangen-Brüttisellen [site] holds four simplex single mode fibers. The first position of the OTO is typically used to connect the ONT and subsequently the enduser internet connection (in the author’s case an EasyZone connection). The other three positions on the OTO are reserved for future use. For some reason unknown to the author, the Fiber7 connection was installed on a second OTO, again with four simplex single mode fibers. The first position of the second OTO was used to provide the Fiber7 internet connection.

Appendix

Appendix 1 - Terminology

Term Description
ONT optical network terminal - The ONT converts fiber-optic light signals to copper based electric signals, usually Ethernet.
OTO optical telecommunication outlet - The OTO is a fiber optic outlet that allows easy termination of cables in an office and home environment. Installed OTOs are referred to by their OTO-ID.
CARP common address redundancy protocol - Its purpose is to allow multiple hosts on the same network segment to share an IP address. CARP is a secure, free alternative to the Virtual Router Redundancy Protocol (VRRP) and the Hot Standby Router Protocol (HSRP).
SIT simple internet transition - Its purpose is to interconnect isolated IPv6 networks, located in global IPv4 Internet via tunnels.
STB set top box - a device that enables a television set to become a user interface to the Internet and also enables a television set to receive and decode digital television (DTV) broadcasts.
GRE generic routing encapsulation - a tunneling protocol developed by Cisco Systems that can encapsulate a wide variety of network layer protocols inside virtual point-to-point links over an Internet Protocol network.
L2VPN layer2 virtual private network - a service that emulates a switched Ethernet (V)LAN across a pseudo-wire (typically an IP tunnel)
DHCP dynamic host configuration protocol - an IPv4 network protocol that enables a server to automatically assign an IP address to a computer from a defined range of numbers.
DHCP6 Dynamic host configuration protocol: prefix delegation - an IPv6 network protocol that enables a server to automatically assign network prefixes to a customer from a defined range of numbers.
NDP NS/NA neighbor discovery protocol: neighbor solicitation / advertisement - an ipv6 specific protocol to discover and judge reachability of other nodes on a shared link.
NDP RS/RA neighbor discovery protocol: router solicitation / advertisement - an ipv6 specific protocol to discover and install local address and gateway information.

Appendix 2 - Supporting data

Bandwidth with Speedtest

Directly on Fiber7: speedtest

Speedtest Fiber7

GRE via IP-Max: speedtest

Speedtest IP-Max

Bandwidth with Iperf upstream

(AS13030 IPv4) $ iperf -t 600 -P 4 -i 60 -l 1M -m -c chzrh02.sixxs.net
------------------------------------------------------------
Client connecting to chzrh02.sixxs.net, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 77.109.173.198 port 41199 connected with 213.144.148.74 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]    0.0-60.0 sec  6.23 GBytes   892 Mbits/sec
[  3]  60.0-120.0 sec  6.21 GBytes   889 Mbits/sec
[  3] 120.0-180.0 sec  6.22 GBytes   891 Mbits/sec
[  3] 180.0-240.0 sec  6.25 GBytes   894 Mbits/sec
[  3] 240.0-300.0 sec  6.25 GBytes   894 Mbits/sec
[  3] 300.0-360.0 sec  6.23 GBytes   892 Mbits/sec
[  3] 360.0-420.0 sec  6.22 GBytes   890 Mbits/sec
[  3] 420.0-480.0 sec  6.20 GBytes   888 Mbits/sec
[  3] 480.0-540.0 sec  6.21 GBytes   889 Mbits/sec
[  3] 540.0-600.0 sec  6.18 GBytes   885 Mbits/sec
[  3]   0.0-600.0 sec  62.2 GBytes   891 Mbits/sec
[  3] MSS size 1448 bytes (MTU 1500 bytes, ethernet)

(AS25091 IPv6) $ iperf -V -t 600 -P 4 -i 60 -l 1M -m -c charb02.paphosting.net
------------------------------------------------------------
Client connecting to charb02.paphosting.net, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 2a02:168:2000:4b:469:a025:5293:84ad port 45044 connected with 2a02:2528:503:1::83 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]    0.0-60.0 sec  5.22 GBytes   748 Mbits/sec
[  3]  60.0-120.0 sec  5.52 GBytes   791 Mbits/sec
[  3] 120.0-180.0 sec  5.67 GBytes   811 Mbits/sec
[  3] 180.0-240.0 sec  4.86 GBytes   696 Mbits/sec
[  3] 240.0-300.0 sec  4.85 GBytes   695 Mbits/sec
[  3] 300.0-360.0 sec  5.44 GBytes   779 Mbits/sec
[  3] 360.0-420.0 sec  5.97 GBytes   855 Mbits/sec
[  3] 420.0-480.0 sec  5.54 GBytes   792 Mbits/sec
[  3] 480.0-540.0 sec  5.17 GBytes   739 Mbits/sec
[  3] 540.0-600.0 sec  5.63 GBytes   806 Mbits/sec
[  3]   0.0-600.0 sec  53.9 GBytes   771 Mbits/sec
[  3] MSS size 1428 bytes (MTU 1500 bytes, ethernet)

Bandwidth with Iperf downstream

(AS13030 IPv4) $ iperf -t 600 -P 4 -i 60 -l 1M -m -c 77.109.173.198
------------------------------------------------------------
Client connecting to 77.109.173.198, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 213.144.148.74 port 56642 connected with 77.109.173.198 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]    0.0-60.0 sec  6.22 GBytes   891 Mbits/sec
[  3]  60.0-120.0 sec  6.25 GBytes   895 Mbits/sec
[  3] 120.0-180.0 sec  6.24 GBytes   894 Mbits/sec
[  3] 180.0-240.0 sec  6.23 GBytes   891 Mbits/sec
[  3] 240.0-300.0 sec  6.21 GBytes   889 Mbits/sec
[  3] 300.0-360.0 sec  6.23 GBytes   892 Mbits/sec
[  3] 360.0-420.0 sec  6.27 GBytes   898 Mbits/sec
[  3] 420.0-480.0 sec  6.25 GBytes   895 Mbits/sec
[  3] 480.0-540.0 sec  6.27 GBytes   897 Mbits/sec
[  3] 540.0-600.0 sec  6.26 GBytes   896 Mbits/sec
[  3]   0.0-600.0 sec  62.4 GBytes   894 Mbits/sec
[  3] MSS size 1448 bytes (MTU 1500 bytes, ethernet)

(AS25091 IPv6) $ iperf -V -t 600 -P 4 -i 60 -l 1M -m -c 2a02:168:2000:4b:20d:b9ff:fe41:94c
------------------------------------------------------------
Client connecting to 2a02:168:2000:4b:20d:b9ff:fe41:94c, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 2a02:2528:503:1::83 port 43499 connected with 2a02:168:2000:4b:20d:b9ff:fe41:94c port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]    0.0-60.0 sec  5.68 GBytes   813 Mbits/sec
[  3]  60.0-120.0 sec  5.50 GBytes   787 Mbits/sec
[  3] 120.0-180.0 sec  5.75 GBytes   823 Mbits/sec
[  3] 180.0-240.0 sec  6.06 GBytes   868 Mbits/sec
[  3] 240.0-300.0 sec  5.96 GBytes   853 Mbits/sec
[  3] 300.0-360.0 sec  5.95 GBytes   852 Mbits/sec
[  3] 360.0-420.0 sec  5.99 GBytes   858 Mbits/sec
[  3] 420.0-480.0 sec  5.56 GBytes   796 Mbits/sec
[  3] 480.0-540.0 sec  6.10 GBytes   874 Mbits/sec
[  3] 540.0-600.0 sec  6.21 GBytes   889 Mbits/sec
[  3]   0.0-600.0 sec  58.8 GBytes   841 Mbits/sec
[  3] MSS size 1428 bytes (MTU 1500 bytes, ethernet)

Appendix 3 - Configuration files

DHCPv6 Configuration

Two IPv6 access mechanisms were used. Firstly, IPv6 was acquired via SixXS [site] who are present at Init7. After it was made available (approximately one week into the pilot), standard issue WIDE DHCPv6 client was used with the following configuration file:

$ cat /etc/wide-dhcpv6/dhcpc.conf

interface eth0.9 { # interface VLAN9 - Fiber7
  send ia-na 1;
  send ia-pd 1;
  script "/etc/wide-dhcpv6/dhcp6c-script";
};

id-assoc pd 1 {
  prefix ::/48 infinity;

  prefix-interface lo {
    sla-id 0;
    ifid 1;
    sla-len 16;
  };

 # Test interface
  prefix-interface eth1 {
    sla-id 4096;
    ifid 1;
    sla-len 16;
  };
};

id-assoc na 1 {
  # id-assoc for eth0.9
};

IGMP Proxy Configuration

Taking IGMPProxy from github and the following configuration file, IPTV worked reliably throughout the pilot:

$ cat /etc/igmpproxy.conf
##------------------------------------------------------
## Enable Quickleave mode (Sends Leave instantly)
##------------------------------------------------------
quickleave

##------------------------------------------------------
## Configuration for Upstream Interface
##------------------------------------------------------
phyint eth0.9 upstream  ratelimit 0  threshold 1
  altnet 109.202.223.0/24
  altnet 192.168.2.0/23
  altnet 239.44.0.0/16

##------------------------------------------------------
## Configuration for Downstream Interface
##------------------------------------------------------
phyint eth0   downstream  ratelimit 0  threshold 1
phyint eth0.2 downstream  ratelimit 0  threshold 1

##------------------------------------------------------
## Configuration for Disabled Interface
##------------------------------------------------------
phyint eth0.3 disabled # Guest
phyint eth0.4 disabled # IPCam
phyint eth0.5 disabled # BIT
phyint eth0.6 disabled # IP-Max