Randy's Work Blog

The simple approach to running DynaMIPS on FreeBSD guest under VMware did not work at all. No packets from the LAN reached the DynaMIPS ether interface.

24 hours later, and with the help of Iain and Rob, it was sorted.

It required two hacks as follows:

The first hack was running em1, the interface dynamips was using, through a custom vswitch that had promiscuous mode enabled.

The problem here was that this let packets through so well that it looped and multiplied them. Pings from the same ether LAN had TTLs as low as 220, delays of many seconds, and massive duplication of packets.

The second hack was running it through a bridge/tap interface on FreeBSD. In /etc/rc.conf lingo, that’s

   cloned_interfaces="bridge0 tap0"
   ifconfig_bridge0="addm em1 addm tap0"
   ifconfig_em1=up
   ifconfig_tap0=up

and then the dynagen conf file used

   fa0/0 = NIO_tap:/dev/tap0

Either hack alone did not work. It took both and one or more rubber chickens. The result looked like the following:

 
    .--------------------------.
    |                          |
    | .----------------------. |
    | |                      | |
    | | .------------------. | |
    | | |                  | | |
    | | |     DynaMIPS     | | |
    | | |                  | | |
    | | |                  | | |
    | | |           fa 0/0 | | |
    | | `--------------|---' | |
    | |                |     | |
    | |              tap0    | |
    | |  FreeBSD       |     | |
    | |             bridge0  | |
    | |                |     | |
    | |  em0          em1    | |
    | `---|------------|-----' |
    |     |            |       |
    |  vswitch      vswitch    |
    |     |            |       |
    |     `------------'       |
    |            |             |
    |  ESXi      |             |
    `------------|-------------'
                 |
        /--~~-----------~~--/
                LAN

The DynaGen configuration then looked like

  [localhost]
    [[7200]]
      model = 7200
      image = images/c7200-adventk9-mz-rpki26-v152_1_s_xe35_tr-bl30.unzip
      ram = 2048
      npe = npe-400
      idlepc = 0x6204bd2c
      ghostios = True
    [[router r0]]
      console = 2001
      aux = 3001
      fa0/0 = NIO_tap:/dev/tap0

With the generous help of Sato san, I managed to get IPv6 back working on my Tokyo home Soekris gateway to IIJ.

This is the simple view, pretty much focused on IPv4

               .-------------------------.
               |                         |
               |                b --wlan0|
               |                r        | 192.168.0.0/24
    WAN IIJ    |                i --- vr1| LAN hosts,
    PPP/NAT ---|vr0[PPPoE]tun0--d        | DHCP Clients
210.138.216.50 |                g --- vr2| ...
               |                e        |
               |                0 --- vr3|
               |                         |
               `-------------------------'

Here is the /etc/rc.conf

# User ppp configuration.
ppp_enable=YES
ppp_mode=dedicated
ppp_profile=iij

# IPv4 internal LAN
wlans_ath0=wlan0
create_args_wlan0="wlanmode ap mode 11g channel 11 up"
cloned_interfaces=bridge0
ifconfig_bridge0="192.168.0.1 addm vr1 addm vr2 addm vr3 addm wlan0 up"
ifconfig_vr1=up
ifconfig_vr2=up
ifconfig_vr3=up

gateway_enable=YES
hostapd_enable=YES # Run hostap daemon.

# IPv6 internal LAN
ifconfig_bridge0_ipv6="inet6 fe80::0452:fdff:fe5d:b500/64"
ifconfig_bridge0_alias0="inet6 2001:240:6a8::1/64"

# IPv6 options
ipv6_activate_all_interfaces=YES
ipv6_gateway_enable=YES
route6d_enable=YES
route6d_flags="-A 2001:240:6a8::/48,gif0 -O 2001:240:6a8::/48,gif0"
rtsold_enable=YES
rtadvd_enable=YES
rtadvd_interfaces="vr0 bridge0"
gif_interfaces=gif0
gifconfig_gif0="210.138.216.50 210.138.77.245"
ipv6_static_routes=gif
ipv6_route_gif="default -interface gif0"

and the /etc/ppp/ppp.conf

default:
set log Phase Chat LCP IPCP CCP tun command
ident user-ppp VERSION (built 2008.04.01)
disable ipv6cp

iij:
set device PPPoE:vr0
set MRU 1454 # NTT suggests this value
set MTU 1454
accept CHAP
enable lqr
add default HISADDR
nat enable yes
set authname <user>@bnf1.iij.ad.jp
set authkey <password>

My presentation, with Kotikalapudi Sriram, given at Cisco NAG of the first results from modeling the signing and validation processor costs of BGPsec.

My take-away:

A trivial new Internet-Draft to deal with the issue that an AS number zero was never formally proscribed.

This document specifies that a BGP speaker MUST NOT originate or
propapate an announcement with an AS number of zero, and a BGP
listener MUST NOT accept an announcement which has an AS number of
zero in the AS-PATH attribute, and SHOULD log the fact that it has
done so.

In addition if a BGP listener recives zero as the peer AS in an OPEN
message, it MUST abort the connection and send a NOTIFICATION with
Error Code “OPEN Message Error” and subcode “Bad Peer AS” (see
[RFC4271] Section 6.2). Obviosuly enough, a router MUST NOT
initialte a connection claiming to be AS number zero.

Our paper should be out about now.

M. Roughan, W. Willinger, O. Maennel, D. Perouli, and R. Bush, 10 Lessons from 10 Years of Measuring and Modeling the Internet’s Autonomous Systems, IEEE Journal on Selected Areas in Communications, Vol. 29, No. 9, pp. 1-12, Oct. 2011.

From the introduction:

1) The notion of “inter-domain topology of the Internet” is ambiguous, at best, without more precise definitions of terms than typically provided.

2) The commonly-used practice of abstracting ASes to generic atomic nodes without any internal structure is an over-simplification that severely limits our ability to capture critical features associated with real-world ASes such as route diversity, policy diversity, or multi-connectivity.

3) The traditional approach of modeling the AS-level Internet as a simple connected digraph is an abstraction incapable of capturing important facets of the rich semantics of real-world inter-AS relationships, including different interconnections for different policies and/or different interconnection points. The implications of such abstractions need to be recognized before attributing network-specific meaning to findings derived from the resulting models.

4) The BGP routing data that projects like RouteViews or RIPE RIS have collected and made publicly available are of enormous practical value for network operators, but were never meant to be used for inferring or mapping the AS-level connectivity of the Internet. The main reason for this is that BGP was not designed with AS-level topology discovery/mapping in mind; instead, BGP’s purpose is to enable ASes to express and realize their routing policies without revealing AS-internal features and, to achieve this goal in a scalable manner, BGP has to hide information that would otherwise aid topology discovery.

5) The traceroute data that projects like Ark (CAIDA), DIMES, or iPlane have collected and made publicly available have been a boon to network researchers, but are inherently limited for faithfully inferring or mapping the AS-level connectivity of the Internet. The main reason for this is that traceroute was not designed with Internet topology discovery/mapping in mind; instead, it is a diagnostic tool for tracking the route or path (and measuring transit delays) of one’s packets to some host, and to achieve this diagnostic task, traceroute can ignore issues (e.g., interface aliasing) that would need to be solved first were topology discovery its stated objective.

6) Significant additional efforts are required before current models of the Internet’s inter-domain topology derived from the publicly available and widely-used measurement data can purposefully be used to study the performance of new routing protocols and/or perform meaningful simulation studies. At a minimum, such studies need to be accompanied by strong robustness results that demonstrate the insensitivity of reported claims to model variations that attempt to address or remediate some of the known shortcomings of the underlying models or data.

7) When examining the vulnerability of the Internet to various types of real-world threats or studying the Internet as a critical infrastructure, it is in general inappropriate to equate the Internet with a measured AS topology. In fact, meaningful investigations of most vulnerability-related aspects of the Internet typically require taking a more holistic approach to Internet connectivity, accounting for details of the physical infrastructure, of how physical connectivity maps to various types of more virtual connectivity, of protocol-specific features, and of traffic- related aspects that manifest themselves at the different connectivity structures.

8 ) While there is a valid role for “observational” studies of the Internet’s Autonomous System, the results of such studies are in general hard to interpret. A more promising method involves performing controlled experiments that allow one to discriminate alternative explanations for results and prevent the effects of one confounding factor from drowning out the effects of others.

9) Studies which start with a definite application, and proceed to collect the best data available for that application have shown a much higher rate of success than “fishing expeditions”; that is, studies that target datasets collected by third-parties and analyze them for the sake of analysis.

10) In an environment like the Internet where high-variability phenomena are the rule rather than the exception, and where the quality of the data cannot be taken for granted, it is paramount to apply data-analytic methods that have strong robustness properties to the known deficiencies in the observations and naturally account for the presence of extreme values in the data.

To: sidr wg list
Subject: [sidr] is a longer announce invalid or not found?
From: Randy Bush
Date: Fri, 30 Sep 2011 11:39:21 +0900

randy


[ http://archive.psg.com/110904.broadside.html ]

Do Not Complicate Routing Security with Voodoo Economics
a broadside

A recent NANOG presentation and SIGCOMM paper by Gill, Schapira, and Goldberg[1] drew a lot of ‘discussion’ from the floor. But that discussion missed significant problems with this work. I raise this because of fear that uncritical acceptance of this work will be used as the basis for others’ work, or worse, misguided public policy.
o The ISP economic and incentive model is overly naive to the point of being misleading,
o The security threat model is unrealistic and misguided, and
o The simulations are questionable.

Basic ISP economics are quite different from those described by the authors. Above the tail links to paying customers, the expenses of inter-provider traffic are often higher than the income, thanks to the telcos’ race to the bottom. In this counter-intuitive world, transit can often be cheaper than peering. I.e. history shows that in the rare cases where providers have been inclined to such games, they usually shed traffic not stole it, the opposite of what the paper presumes. The paper also completely ignores the rise of the content providers as described so well in SIGCOMM 2010 by Labovitz et alia[2]

It is not clear how to ‘fix’ the economic model, especially as[3] says you can not do so with rigor. Once one starts, e.g. the paper may lack Tier-N peering richness which is believed to be at the edges, we have bought into the game for which there is no clear end.

But this is irrelevant, what will motivate deployment of BGP security is not provider traffic-shifting. BGP security is, as its name indicates, about security, preventing data stealing (think banking transactions[4]), keeping miscreants from originating address space of others (think YouTube incident) or as attack/spam sources, etc.

The largest obstacle to deployment of BGP security is that the technology being deployed, RPKI-based origin validation and later BGPsec, are based on an X.509 certificate hierarchy, the RPKI. This radically changes the current inter-ISP web of trust model to one having ISPs’ routing at the mercy of the Regional Internet Registries (RIRs). Will the benefits of security – no more YouTube incidents, etc. – be perceived as worth having one’s routing at the whim of an non-operational administrative monopoly? Perhaps this is the real economic game here, and will cause a change in the relationship between the operators and the RIR cartel.

The paper’s simulations really should be shown not to rely on the popular but highly problematic[3] Gao-Rexford model of inter-provider relationships, that providers prefer customers over peers (in fact, a number of global Tier-1 providers have preferred peers for decades), and that relationships are valley free, which also has significant exceptions. Yet these invalid assumptions may underpin the simulation results.

—

Randy Bush
Dubrovnik, 2011.9.4

[1] P. Gill, M. Schapira, and S. Goldberg, Let the Market Drive Deployment: A Strategy for Transitioning to BGP Security, SIGCOMM 2011, August 2011.
http://conferences.sigcomm.org/sigcomm/2011/papers/sigcomm/p14.pdf

[2] [1] C. Labovitz, S. Iekel-Johnson, D. McPherson, J. Oberheide, and F. Jahanian, “Internet inter-domain traffic,” in SIGCOMM ’10: Proceedings of the ACM SIGCOMM 2010 conference on SIGCOMM, 2010.

[3] M. Roughan, W. Willinger, O. Maennel, D. Perouli, and R. Bush, 10 Lessons from 10 Years of Measuring and Modeling the Internet’s
Autonomous Systems, IEEE Journal on Selected Areas in Communications, Vol. 29, No. 9, pp. 1-12, Oct. 2011.
https://archive.psg.com/111000.TenLessons.pdf

[4] A. Pilosov, T. Kapela. Stealing The Internet An Internet-Scale Man In The Middle Attack, Defcon 16, August, 2008.
http://www.defcon.org/images/defcon-16/dc16-presentations/defcon-16-pilosov-kapela.pdf

After a lot of pissing and moaning, A+P was finally published as RFC 6346. It only took five years from when I first presented it.

All due to the hard work of Gabor Bajko, Mohamed Boucadair, Steven M. Bellovin, Luca Cittadini, Olaf Maennel, Reinaldo Penno, Teemu Savolainen, and especially Jan Zorz.

Kohno-san’s draft finally made it through the sausage machine!

RFC 6164 Using 127-Bit IPv6 Prefixes on Inter-Router Links. M. Kohno, B. Nitzan, R. Bush, Y. Matsuzaki, L. Colitti, T. Narten. April 2011.

Our PAM paper on RFD is out

Cristel Pelsser, Olaf Maennel, Pradosh Mohapatra, Randy Bush, and Keyur Patel, Route Flap Damping Made Usable, in PAM 2011, March 2011.