These are unedited transcripts and may contain errors.
The plenary session commenced on the 5th of October, 2009, at 2 p.m. as follows:
ROB BLOKZIJL: Good afternoon. It is about 2 o'clock, and I suggest we start. This is the opening of the 59th RIPE meeting, welcome to you all in Lisbon. I have a few administrative issues to communicate and then we will start with the programme.
First place, we are very grateful to NFSI Telecom, which is the local host of this meeting, and what I have heard from the RIPE NCC technical crew is that this has been one of those very rare meetings where everything went extremely smoothly the last couple of days, installing all the networks and facilities in the hotel, and the NFSI Telecom has really been extremely supportive so I would like to give a warm round of applause to NFSI Telecom. Thank you.
(Applause)
Today is a national holiday in Portugal, it is celebrating the Republic which means on a more practical level, almost all shops are closed so you took a wise decision to spend the afternoon here.
Tonight, we have after the last session, welcome drinks for you to meet your old friends in a relaxed atmosphere to make new friends and tonight is sponsored by Google and will take place, if the weather permits, outside downstairs on the terrace. If it is raining too much, it will be moved just inside, but it is downstairs near the terrace area.
Most of you will have taken your lunch already so you know that ?? where the lunch takes place, it's in the restaurant on the lobby level, one floor down. It's easy to find. Coffee breaks take place just outside the room on this area. All RIPE meeting rooms and technical rooms and service rooms are all on this level.
If you need technical support, there is RIPE NCC technical crew working walking around and you can recognise them by their blue badges. If you have any problems, if your laptop with connectivity, whatever, don't hesitate to approach them. And last but not least and very important, for many years already we have been ROSIE.ripe.net as our main means of communication meeting information, like updates of agendas, social events and everything else concerning this meeting. So, have a regular look at ROSIE.ripe.net and you won't be surprised by things that go a little bit different from your printed information.
Well, having said all that, welcome again to Lisbon, welcome to the RIPE 59 meeting. We have, as usually, a very packed agenda for the whole week, so without further ado, I would like to start the programme with the first speaker, Geoff Huston, who will talk about 4?Byte As Numbers.
It's one example of a simple technology we all agreed upon five years ago, six years ago, and now we just have to implement it. There are some details there which might interest you.
GEOFF HUSTON: Thank you. Good afternoon everyone and Hi, I am with APNIC and it's a pleasure to be here. This afternoon, I was going to talk to you about autonomous system numbers, and I seem to recall I have talked about this before, but none of you were listening at the time. So, this time, you have got to listen or I am going to do it again but I promise you absolutely no train wreck pictures at all, none whatsoever. So, anyway, where are we? As some of you might recall, there are precisely 65,536, in other words 2 to the 16th AS number of the two byte fashion in this world,, the exactly 1,042 of them, and of the ones that are left, I can see 32,000 in BGP, and 15,000 I can't see anywhere at all, they are not in the RIR system so you guys have them. I don't know what you are doing with them but you have them.
OK. That doesn't leave very many. In fact IANA only has, today, 8176 of these two byte numbers left and the RIRs currently have between them, 9,000 give away. So, we are running low. But if numbers are too much for you and you like pretty colours those are pretty colours. There is red, blue and green and, you know, they mean something, which is really quite cool. The bit you are worried about is the bit at the right, the yellow bit, the bit we need and that is the bit that is running out. So how fast are we getting through them? This is again in colours for you. This is the daily rate of AS number allocations for the last three years. And although there is a bit of seasonal variation, you go away and have summer holidays and come back and need a few AS numbers, on the whole, on it's actually been really quite steady. We have been burning up 12 ASs a day, and more lately close to burning up 13. That is interesting. Where? Well in APNIC in, fact in the Asia Pacific region we don't use AS numbers, we get through about 1.3, LACNIC haven't heard of them either and neither have AfriNIC. But RIPE, RIPE get through around 5.8 AS numbers a day, and ARIN get through, so to you northern hemisphere people, what is your problem? Why do you need AS numbers on this side of the planet whereas the other side, the superior side, doesn't? I have no idea why this northern hemisphere consumption of AS numbers so all of this is your fault. OK. Every last bit of this is your fault and, you know, that is the problem. So how long have we got? How long have we got? Same set of stuff as we used to do before, this is all about mathematical modelling and model the entire system pretty accurately, and yes, the AS numbers will run out and they will run out at around about 2011 and you can model this really quite accurately and this is so who runs out first? You guys, because you are using them all. So around about early 2011, just before v6 runs out, it's going to be a fun year, isn't it? Just before v4 runs out, sorry. V6 already has its problems and let's not go there. Just before v4 collapses, your BGP is going to collapse and then at the end of that year RIPE is going to be the first RIR to run completely out. So, so far IANA is going to do its last allocation to the RIRs in March 2011 from, memory I think it's going to be ARIN that gets the last block, lucky ARIN and RIPE is going to run out of its pool in October. About the 11th or so. Is that anyone's national holiday? Maybe the 12th that is when it's going to happen. So we knew this, we knew this years ago, this is a slide from March 2003 so no one can claim that that was a surprise. That was the model from back then, and six years ago we kind of guessed that things were going to run out sometime around now. So oddly enough you guys are nothing if not predictable. Imminently predictable isn't that good? So we had a problem and interestingly enough, not only this was not a surprise, we planned. So way, way back in, fact even back in 2000, we knew this was going to be a problem so we worked out the sort of plan of trying to make sure that everyone would manage to get to the new improved bigger AS number pool, all 32 bits before we ran out. So the next kind of question to have here is, how well, are we going with this plan? How well is everyone doing along the script of saying, guys, we have really got to change things? So, you know, we know the IETF is not the fasters of beastees, realise the IETF doesn't work on a scale of week so even back then we thought it might take us as much two years to get a standard through the IETF about how to get larger AS numbers, so back in 2004 we thought yeah right, you know, about two years or so we would get this through. At the same time, proposals were put into side RIRs, we thought that would take about six months and we thought that would give about a year for vendors to make the relatively minor change and it's not a big change to BGP so within about 2008 everyone would be running 4 byte AS numbers. So how do we go? Because it's not doing that well, is it? The IETF, the initial specification was actually done years ago, February 2001, and it was a very subtle change. Easily because unlike v4/v6 this is a hot by HOP protocol and with a little bit of mucking around you can make it backward compatible. I can run a new modified and you guys can run the same old stuff and things will work. You can send me packets and I can send you packets routing will always work and the way it was done was quite subtle: BGP does its open exchange, Hi I am Geoff, who are you? That is very good I am pleased to meet you. I handle 4 byte numbers, do you? No, never mind, let's just fake it. So I will look as if I have one of these 2 byte AS number and I will just repackage everything for you. So the way it worked was with a very subtle mechanism of both tunneling and translation simultaneously, BGP was going to be backward compatible. That specification started in February 2001 and after about 12 iterations through the IETF was ready for publication in late 2005. , you know, if you go and look at those 16 odd drafts and say how much did the IETF change the original idea? I think they added about three full stops and an adjective, not much change, it took a while. I am not sure why. The thing published late 2006 and was RFC ready in May 2007 so it was a little bit late but basically on time so the standard was done. This is good.
The RIRs were also looked at this and we gave presentations on this very subject back in 2004, early 2005, and the whole idea was, in light of this policy, we would all give ourselves some very, very clear milestones, and we were all meant to stick to them. ISPs, vendors, everyone and they all looked pretty reasonable. In January 2007 we had open up the 32 bit number registry so those folk who were earlier doctors and wanted to play with the technology, a new BGP could do so and there was two years of trying to get things ready and say to the vendors clear signal by January 2009 you should have done your job, two whole years to get this stuff that is updates to BGP. So by January 2009, the start of this year, the RIRs had a policy that if you came and said I want an AS number, you got a big number, not a little one. And you only get a little number if you say I really need a little number. And you go oh, yeah, OK. So, this year, we are meant to be giving out big numbers by default and little only if you really, really ask for them, because in just three months' time, goes the policy, we are meant to say it's all over. It's all finished. This transition is over. So that was the goal, that was the objective of this. And the policy went through, right? So, how well did the vendors go and how well have you gone in doing your bit of the work?
What about the vendors? Slowly and surely, the vendors have been putting out BGP, and this comes from a very good site as.4.clupon.net which is a WIKI site that is updated, but it gives a complete list of which versions of various forms of router vendor soft fair supports BGP 4 byte version. I should mention that the only thing that really got fantastic discussion in the RIR communities, in the standards communities, the only I think that engendered any conversation whatsoever was should we use a colon or full stop in between the lower and upper parts of the AS number or none at all? And interestingly, the mixed signals we gave, if you look at the right?hand column of that, all the vendors didn't quite figure out what was intended and we all do something subtly different. If you are using the box you use very big numbers, if you are using a box from Cisco you have to separate the lower and high bytes with the full stop etc., etc. So we stumbled upon notation created a standard and known ever adopted it. There is a win there somewhere but I fail to see it. Vendors are indeed doing BGP one way and another and it's currently gathering pace. So how well, are you and I doing in actually deploying this stuff?
Here are the numbers: Like I said, this year you /TKPWOEPBL got a big number by default and a little one if you really, really asked for it. So, so far, up until /#30*9 of September, 3169 folk really, really asked for little numbers, and only 480 got big numbers, of which 256 went to Brazil, which doesn't leave much more anyone else. So around 1,200 numbers went to this part of the northern /HEPLS sphere and the other 1,200 went to the other part, you guys aren't listening. There is no take up whatsoever of 4 byte AS numbers at this point. Here is the graph of the two, the big red line is the 16 bit AS numbers, which throughout the whole of this year has been going gang busters and the bottom green line is the 4 byte numbers and the only thing you see is is a block allocation to Brazil that lifted it up from the bottom of the screen. So it's not doing too well, is it? In in terms of using AS numbers, out of those 480, not very well at all. 107 that I see advertised and 492 unadvertised out of the entire block of 6014 byte AS numbers that have been handed out so most of them aren't visible whatsoever. So as far as I can see that is the plot of who is advertising stuff, and the green line ?? the blue line down the bottom is advertised 4 byte AS numbers, they are not being used. So this is looking like a bit of a stuff?up and indeed I so say time is running out and things are lagging badly. I lied. I lied. Sorry.
So, time is running out and a bit like this entire v6 problem, we really need to actually put some pressure on because at some point or another like v4 there is nothing left after we run out. Once we run out of the little numbers the little numbers are no more, you have to use big ones so how can we get people to use them? So just the last few minutes here I would like to run through some of the common excuses I have heard why people tell the RIRs "I can't use a big number, I have to use a little number, you have really got to give me one" because quite frankly there are a remarkable of common myths about 4 byte AS numbers and I would like to slap some of them down. The first thing we get is, I can't use it, no one can use it, we have all got to up braid BGP, this is the whole thing about v6 isn't it, we have all got to jump together. As soon as someone starts using a 4 byte AS numbers I won't be able to send my packets to them. Do you really have to upgrade your BGP just because I have upgraded mine? Are we really in this boat together? Of course not. This is a backward compatible transition. So quite frankly, BGP does work when I upgrade you don't have to. And indeed, even within your own AS, you can piecemeal upgrade router by router, iBGP /SPAOERBG by iBGP, it will work. Because the way BGP actually works on this transition, is that it is intended to be completely backward compatible. What happens is in a in your world, all BGP, all you see in the EBGP part of it is AS 23456 appearing everywhere, because that is the token holder. So I am AS 131072 as I recall. But in your version of BGP you will see my routes coming from AS 2, 3, 4, 5, 6. Do you care? Not in the slightest. So as far as I can see, there is no problem around if I upgrade. Do you? Not necessarily. So anyone who is running up a new network doesn't need a little number to talk to the rest of the Internet. Big numbers work just fine.
So, what about if you are an ISP and you go hang on a second, if all my customers and my peers and my up streams start to use 4 byte AS numbers will my network break? Should I insist, as some transit ISPs do today, that all their customers have to use little AS numbers? Is that correct technically? Do they need to wait for me to up grid my BGP before they can present me with a 4 byte AS number? No, of course not. Each upgrade is independent, completely independent. You need to do absolutely nothing whatsoever. Is that a question?
AUDIENCE SPEAKER: It is. /A*RPB Hughes 6 connect. One comment on this, it's fairly difficult when you depend on flow data and having lots of flow data from, 2, 3, 4 and not being I believe to identify where that is coming from is rather difficult. And also, there have been significant issues with code trains mostly with Cisco where double byte ASs are supported on some versions of codes and don't support the features that you need for the rest of your /PWRAEUGS.
GEOFF HUSTON: Some of this is covered in my next slide but thank you for the comment. Basically as far as it goes you need to do nothing but and there is a big but, a huge number of operational support systems, including things like NetFlow and traffic analysis, a lot of them use the customers or the other parties' AS number as the primary look up key. Your customer AS 3, your 5, your customers AS /#10RBGS as long as each has a unique AS number, if you are running out BGP everyone is /#2?RBGS 3, 4, 5, 6, and things start to fall apart. So, your operatings support systems, your O S S does need a careful look through, basically you have going to have to support multiple fears, customers up streams, all of whom are going to present to you on the wire, as AS 2, 3, 4, 5 and 6 and in your filter list, as AS 2, 3, 4, 5, 6 and in all your accounting, that is going to be a little bit of an issue. So your support systems better be able to work with this one way or another. So you have got 12 months, get busy.
Can I use communities for fewer byte AS numbers because we have had a lot of use of communities and there should be more because if you can either submerge BGP by sending out more specifics of everything under the planet or you can be a bit more intelligent and actually use communities to signal routing policies. Can you do routing policy by four?byte ASNs? Well not quite. It's one of those "Oops", because the field inside the community field in the old BGP can be only allowed for 16 bits of AS number and 16 bits of community number, which is really hard to stuff a 32?bit AS number and even a 1?bit community stream, let alone something like 16?bit. So, no, this doesn't quite work and the standards folk are working on it and the relevant draft is oddly enough, didn't go through the RDR working group. It went through the layer 3 VPN working group. There's a signal there about RDR, but I don't know what it means and that draft will be published as an RFC any day now. It's in the RFC editor queue. The good part of this is that unlike 4?byte ASNs, vendors who actually impliment 4?byte ASNs have chosen to impliment this spec before it was a standard, so you'll find that most versions that support 4?byte ASNs, also support extended communities, that actually allow you to use 4?byte ASNs with communities. But that bottom bullet there, you really should, when you look at doing an upgrade, ask your vendor when and how they're going to be supporting extended communities with 4?byte ASNs is a real signal. Make sure you do that. This last one, which I thought was really cool, you know, if I upgrade BGP, will it crash and you sort of think: "No, I shouldn't, should it?" ....Oops! I don't know if you were watching over the northern hemisphere summer, but there have been some pretty good bug reports coming out and this one, I kind of liked. That if you get into AS path stuffing and you get to the magic number of 1,000 elements, the Cisco BGP implimentation will crash, bang wallop. The bottom line is: The max AS limits setting really is your friend. There is a security advisary on it. It's there in 1?point font, for those of you who want to get in with a microscope, but go and consult Cisco and you'll find more about that. There's a deeper issue here about BGP, that I think we probably should flag. There is a funny idea about BGP, that dates back to its original invention, back in about 1988, 87 and I remember asking Jaako Richte, who had a lot to do with it at the time, you know ? what's the issue? Jaako was of the view that, at the time, I think, that the BGP session wasn't a bit thing and you talk to operators today and resetting BGP is one of those cardinal sins, where you might as well open the window and jump as soon as you do it. Nobody resets BGP. So BGP has this really strange behaviour written into the standard, that if you get just one update, that has a malformed attribute in it, you don't just ignore it. You bring down the session.I'm sorry, that was a really bad update. I think I will go and kill myself. Which I suppose is an answer, but you don't know that it's bad so I come up, we open the peering and you sipped me the bad update again, I am sorry that is a really bad update I am going to kill myself again and again and /TKHEPB /HR* again.
This is a bizarre behaviour but oddly it's the standard and one of the parts of the 4 byte standard says if you use AS con federations, that is a mall formed update. So some poor sod somewhere in the northern hem see if sets up and a number of routers all over the world went bang wallop crash, not a good idea? Why? Because therm only implementing the standard.
The standard is wrong, but realisticically these days re?setting BGP is is a cardinal sin and you should be fired for it. When you get a bad update, ignore T and interestingly, many of the BGP implementations now do that quite pragmatic response, if you see a lousy update, just don't do anything. And hand tell in a soft fashion. And the IETF is now refining the BGP specification to encode that rather than just saying oh my God that was all of I am going to die, actually do something useful and just move on.
The other thing I have seen, have you ever seen these on some of the rowing mailing lists? 2, 3 /#4?RBGS ?? the Internet is about to burn and 9 honest answer is calm down. There is nothing wrong and if you go and search for AS /#2?RBGS 3, 4, 5, 6 in a four bite BGP im/PHREL, you will find it.
Every single day folk are playing with BGP and that was my snapshot a few weeks ago and you will find it your /SEFPLT I have named the guilty folk: T K telecom operating in Poland. Where are you?
Naughty. A few others as well. It's not fatal, it really doesn't matter. Even when AS 23456 appears in a path when it shouldn't routing loops cannot form. When you think about it hybrid loops take a little bit of time to be deeffected but BGP is going to work just fine and the Internet is going to be just hunky. Has this stuff come of age? Are we getting folk lying in the AS number place? And of course the answer survery inventive folk, even today there is /PWO*E /TKPWOPBS. At the time only 70 advertised AS numbers and four of them were /PWO*E /TKPWOPBS. When too many? Telefonica, we know who they are. Fast web and D I A N E /TEFPLT, 2076901376, you crash your forehead on the key /PWAOFRPLTD well done guys, very invent /TEUFPLT here are some re source, some stuff that I have picked up in trying to go through. Read the RFC, it really is one of the most textbook ways of doing a backward compatible /TRAPB /SEUFPLGTS if only v6 have been like this we would have been far more down the track. It is good work. There is a refinement of that because you can't let good ideas go unpunished. There is also some documentation about exploring AS numbers, and there is a really good WIKI, the last re source down there, AS dot /KHRAOUP on net, has more information for ISPs about who is doing what when and how. Thank you, if there are questions?
AUDIENCE SPEAKER: Hi, this is from LACNIC. I want to clarify that the comment about Brazilian getting ? AS numbers because of our registration system where what we do is just hand them a bunch of ASs for to allocate them. So even though they show up in our stat file as allocated it doesn't /R?LT mean they have been actually allocated /KWREPT but I know it's the data that is public and there you are. But that is going to be changing in beginning of next year when we implement our EPP interface with /TPH*EURS.
GEOFF HUSTON: Good to hear, thank you.
AUDIENCE SPEAKER: We are getting lot of request for ISPs experiment, they have their customer come to us, ask for ASNs, we give them 32 bit ASN. Then after two months they realise that they go to the ISPs, through the whole process and tell us my customer has a 32 bit ASN, I need one 32 order to test with them if this works. So I guess that is one of the other things that maybe you can add in this data, you don't need 32 bit ASN to have a BGP session with your customer because we get a lot of this, I want to test 32 please hand me one and we cannot do that.
GEOFF HUSTON: Oh, if we all had time. There is only about 13 months left before the brick hits the wall on this one. Test all you want because you have only got 13 more months before we run out. So realisticically, the answer is do the stuff now.
To all of the since out there, the real message is we are basically completely run out of /#250EU78.
There is no more time to muck around running dual system, the 2 bit AS space /TKAEUPBG really quAUDIENCE SPEAKER: NetNod. Did you ever try /PWREUPBLTing that long number as 16 ? 16.
GEOFF HUSTON: It's just as bad except you bang your head half against the key /PWAOFRPLTD they are just random numbers.
RUDIGER VOLK: Little bit for /RO*BGy, well OK, Geoff, I agree we are dam late and seeing the first reports out of the field about bugs, having appeared about November/December last year, is kind of showing that well OKof showing that well OK the whole effort was delayed by one or two years, I think actually the testing and figuring out what bugs are there around, well OK, we take the view that the vendors always deliver perfect products so I don't need to test whether the inter operation between AS 16 and 32 within iBGP works, OK I can take the jump of faith but I may actually run into problems so I think if somebody is asking for a 32 bit number, just for testing, which would mean, well OK, the guy probably has on his backbone an old number and he needs an example for simulating a customer with new ?? who is on the new fashion, I think give it to them and have people and have people actually test, yes we are actually too late and the products have been delivered too late for doing reasonable testing.
GEOFF HUSTON: It's a tough situation, isn't it be? Yes.
AUDIENCE SPEAKER: I just say that my comment was at our current policy for experimental allocation, doesn't allow it. That is the problem w AUDIENCE SPEAKER: Suzanne in a from the RIPE NCC have a question on Jabber from Jan Hugo /PR*EUPBS.
He says I always understood when you run a recent version of any BGP router soft fair understands 4 byte you can just as well run a two byte AS.
GEOFF HUSTON: Absolutely, you can always run a little number in a new version of BGP and that is just fine, so you don't need to put big numbers into new BGP, you can put any number you like.
Yeah.
AUDIENCE SPEAKER: So, I kind of don't like the big number/little number. I think kind of like the /TKWHRAD these are just numbers now and they are bigger than they used to be but they are just numbers. The problem that I have with 4 byte ASs is that I have a really difficult time getting Cisco to give me the source code tG S so I can patch it so it will take 4 byte AS. Take it as a hint. If you have got an A G S plus for me, that is great, the point here being that there is a significant installed base and little or no time to do a hardware refresh.
GEOFF HUSTON: The problem here or the issue here, Bill, is the installed base does not need to change. In other words, you can keep on running your existing routers and code and AS, that is fine. What you are going to see is the rest of the world is going to look like AS 23456 as grand scheme of taking over the planet but you will still work, so your A G S plus ?? A G S will still operate BGP in /SORPL form or fashion and I don't know whether that is a comfort to you or something far more depressing. Thank you.
ROB BLOKZIJL: Thank you, Geoff
(Applause)
ROB BLOKZIJL: Right. The next /SPAOERBG is Martin Pels who is giving a presentation without slides?
MARTIN PELS: It should be on there. So as some of you may know, we recently upgraded our infrastructure to MPLS/VPLS and I would like to share some details about that and some of the operational experience we have had with it.
This is the AMS?IX platform as it used to be, this is a picture from June 2009. Here we have 6 locations, we currently building a /S*EPBT one in interaction apples five which is near Schipol airport in the Amsterdam region, and the old platform had two core switches, each running for a completely separate network, so we had red network and blue network which were completely separate.
We had customers, one gigabit E customers and 100 megabit customers connected to Founmegabit customers connected to Foundry and RX 8, we have had 10 big E customers connected to /TPO*T cross connects and from there they are connected to one of two 10 gig E switches, MLX 3 /# or 16s either in the blue or red network.
We ran /PWROE cad sport security on all customer interface toss prevent loops and we raP to initiate fail overs between red and blue platform.
This is our current traffic. We are pushing about over 700 gigabits of daily peak traffic and the average traffic we have is about 470, 480 gigabits per second and as you can see from the yearly graph, summer ended so everybody starts getting new ports again and traffic is growing again.
This is our port growth. As you can see, all of our growth comes from 10 gig e?ports currently. We do sell quite new e?ports but there is a lot of people upgradeing from gigabit to 10 big bit so in the end it stays flat. As you can see from the projections, we expect the same to happen when 40 big bit and 100 come along, then those will be responsible for the growth of the platform and ten gigabit will probably stay the same.
So the problem we have had with AMS?IX version 3 was, well the biggest one of all was the capacity of our core switches. We use MLX 32 which has 12810 gig. Ports but those were too small for us.
We used all of the ports, summer this year could do no more IS L upgrades and could not connect any additional extra switches so we need today change something there.
Also, the fail overs we had between red and blue networks, they introduced a short link flap on our customer ports which was nine when we started with it but more and more we are seeing that ths BGP flaps for people because connected to not ignored link flap or there is somecircuit between our platform and router which amplifies link flat to 16 second ?? another issue we have had people are buying more and more 10 gig. P bunkling them so getting fewer customers on the single switch and there is less local switching on that device and more traffic going through the core. So we had to scale up.
And this is what we did. We created a single active platform with four core dev2 still, and again, we have 10 gig. Customers connected to the /TPO*T switches and 1 gig connected directly to an Internet switch.
We use a single platform brocade MLX series. We up scaled to MLX 32 which is biggest switch /PWROBG /KAEUD has on the market and we introduced an peering platform on top of MPLS/VPLS. The network is still physical star but on logical level we have a full mesh between all the extra switches. We still operate a VLAN service for different purposes. We have VLAN for G RX which is GP RX roaming traffic, a number of private interconnects, and each of these VLANs gets their own VPLS instance on the network. And finally we replace port security by layer 2 ACLs, which means that blocking off malicious mega address sincerely handled in hardware instead of CPU. This is a close?up. We have four core devices and we do load balancing over them. So from each axis switch, PE as we call them, we have one LSP to every over extra switch over each wore /SKEUFP, if you have two PEs, you have four LSPs towards another PE.
And because LSP level switch paths are uni directional we have also four LSPs the other way so that is a lot of logical links.
We implemented redundancy in the platform by putting in twice as much core switches as we needed for full capacity platform so we are able to loose two cores and then still all is fine. And we implemented axis switch redundancy by only connecting customers to half of an axis switch and the other half is used as a backup for second axis switch, and we can switch customers around using flat on I can switches.
So, how does resilience work in the platform? When a core switch fails or backboning failSP is going over the core switch, change their path to their backup core, this will give service interruption of about 20 milliseconds and it will not show any link flap to any customer so it will be practically un/A*EPBable. This is what it looks like. On the right two cores that go down and all the LSPs switch to their secondary above and run over the other two core switches. And then we have the axis switch redundancy that we built in. When an axis switch fails, all the LSPs towards it will go down and we have piece of software that recognises that and will trigger all of fat on I can switches to swap over the customer to different axis switch and like on our previous platform this will only happen for set of switches that has problem and not /TPO*R the entire platform.
So this is what that looks like.
The axis switch on the right top goes down, this means that all the LSPs towards this axis switch from the other axis switches will go down. This generates a whole bunch of S N P traps and are received by a piece of software we developed and this software instructs the fought on I can switches to fail over all customers on the top fat on I can switch to be put on the axis switch on the left top.
A little bit about migration we did. What we did, first, is move all one gigabit axis sfirst, is move all one gigabit axis switches behind the fought on I can cross connect whthe fought on I can cross connect which means they act as 10 big E customer and are swapped around with the other ports on the fat on I can switch.
We need today do that because we wanted to run for a while with both on MPLS platform and standard two layer and customer ports cannot be both at the same time, ethernet axis port and VPLS axis port at the same time. So we did that. After that we moved one half of the platform to VPLS and we left that running for about five weeks and found no real big issues so we did the other half as well and then we merged both halves into a single active platform.
This is where we are now. We still need to connect the 1 gig E axis switches directly to the cores as PE devices and this is something we will be doing in the next couple of months. I will be giving more details about this next nurse in the EIX session, so if you are interested in that, come and have a look.
So, some of the operational issues we found:
First of all, we, between the core devices and the axis switches we run S ?? we have some layer 3 in our exchange. And we are running with BFD with by directional forwarding detection to signal, to do signalling. And what we saw was that sometimes these sessions would go down for no apparent reason and we looked further into that and we hav running every hour, Rancid, probably a lot of you will know it, logs in on the switches and pulse some data like a show running config, show chassis and all that kind of stuff, the /HRA*PBG /KA*RT CPU got so busy didn't respond to BFD packets any more, and the other guy is not responding any more so let's start on the OSPF session. Now, this was all handled fine and the LSPs fell over as planned so no customer ever noticed this happened. But this is into the good thing, of course. So what we did was basically increased the timers so that the LAN card CPU would have more time to answFD packets. We previously used the lowest value we could set, which was about 200 mill /SEFPBLGTDZ we now have it set to 500, so if there is a within 500 milliseconds ?? if there is a problem which becauses BFD not to function within 500 milliseconds the session will be torn down and there will be a failover.
Another issue we found was that when we, basically when we saw this issue with the BFD problem, we put the LSPs back on their primary BoF because they have failed over and then we saw traffic imbalance, we saw that some things were doing more traffic than others while they should be doing about the same. And what actually happened was that due to a bug in the codes, another which generates two ? events causing an for that LSP to be used twice, so what you get when you have two core switches, is in a the MPLS tunnel of the first core switch would be used twice and the MPLS of the second core switch would be used once so you get a traffic imbalance because it will do a load balancing over the amount of tunnels it has. /*PB fortunately, we got to work around to re store this situation to normal and remove the ghost ? and there is a fixed schedule for the next batch release which I believe is out already. So this is not really a problem.
Another issue we saw was with multicast replication. Multicast replication network works by doing replication on the ingress PE so if you have a customer connected to an axis switch sending 1?meg bit of multicast traffic and needs to go to customers on five other axis switches ?? needs to go to customers on five other axis switches then we ?? then you get five megabits of /TWAFBG towards core. Now, this can be fixed with multicast snooping ?? no, this cannot actually be fixed. This is always the case. But when ?? normally, this only uses the first link of an ago great of the first LSP so you get all this traffic will go over one link which is 1 gig 10 E link in our case so that would mean with a little multicast traffic you would be able to fill that one link.
So this can actually be fixed by enabling PIM snooping and after that it's possible to do a multicast replication over all the links on the platform, over all the links in all the LSPs.
So that has some works at the moment so we are not enabling it yet (bugs) we don't have any multicast traffic at the moment so it's not a problem for us but this is something that is high on the list for us to be fixed, and we should have major software release coming out any day now to address this.
Let's talk about this issue for a bit. This is interesting. We saw TTM traffic graphs on a website were showing that certainly there was a really high delay on our platform sometimes.
Normally, the delay is roughly, well below one millisecond but we started smillisecond but we started seeing these spikes of several tens of milliseconds in the end, we found out that this was something that was very typical to the TTM traffic, and that because TTM traffic is only one packet every three seconds, this caused some ?? this caused ?? yes, so TTM traffic is only one packet every 30 seconds and because we have Foundry VPLS CA M entries are programmed on each backbone port and edged out every 60 seconds, so with a bit ago great you get packets will be learned and programmed and in a couple of minutes later there will be another packet over the same backbone links and it needs to be learned again.
So this is not, fortunately not a problem for real world traffic because we are ?? this is only the case if you have really low volume traffic and we are talking about Mcto Mctraffic so that would only occur if a customer would send only one packet to another customer every 30 seconds, which is ?? basically does not happen. Nevertheless, we are looking into changing this, changing the K a.m. time out or disabling it completely. (CA M)
I am going to go over this. The good stuff. We also found some, well, we found that the network is pretty stable now. We have had some issues on our backbone, but these were handled by MPLS so they were not seen by any customer and in previous situations, the entire platform would fail over if there was a problem in backbone and now, this is not the case.
Axis switch failures are now handled for a single pair of axis switches so that no longer affects the entire platform as well. We can do Phase Three locations of traffic streams so if we have one PE which sends traffic to a couple of other PEs, we can say, well, we have this new link in production, let's move the stream from A to B ton this link and let's look at that and let's move the link from A to C, the traffic from A to C over that link, so we can do much more gradual things with that, and that is really handy.
We also see that loop traffic, network loops on customer ports are now handled completely in hardware and do not affect the line card CPU any more because we now use layer 2.
It's also a lot easier for us now to debug customer problems. Previously, we had an active platform and an inactive platform so we can just swap customers around to another switch because then they wouldn't be able to talk to anyone any more, and now every switch in the network is active so we can just put a customer on a different axis switch using the photonic cross connect and if their problem disappears we know that it was a problem object the original switch and debug it off line without any further customer problems.
And another good thing about new platform is that it's completely one single vendor, oneimplementation. We needed to do a lot of configs, new, we have 100s of MPLS paths and hundLSPs and probably go into the thousands soon so we described the whole network in XML and created ought /PHALT I can configure races from that and made sure we didn't run into any problems when we took this platform into production, because there were no mistakes due to manual misconfiguation.
Another good thing about the new platform is the scaleability. We can scale with this platform for ?? we can scale to in a couple of ways: We can put in bigger core devices. These do not even need to be MPLS capable; they just need to be simple plain ethernet, as long as they don't touch the MPLS traffic. They don't even need to understand VLANs for. We can do load sharing over more than 4 core switches which is currently the limit for the pro/KAEUD software. We have feature request planning. We can do load balancing using different core switches for different PE to PE flows. So if you have PE A and PE B and C, we can do traffic from PE A via four core switches to PE B, and traffic from A to C over four completely different core switches, which, of course, will require some traffic engineering and will pose some challenges but at least we are able to scale our network further which was something we couldn't do previously because there were simply no more room in the core. And another thing we can doin more core devices.
/SOP conclusions: We found some issues. Nothing with any customer impact, nobody really ?? nobody noticed that we changed the VPLS so thgood thing, I guess. We solved our scaling problem with traffic load balancing over multiple core switches.
We increased stability by no longer having complete platform failovers and by handling backbone failures inside the MPLS /KHRO*UD and only doing fail overs of single PEs and not a complete platform.
We have up scaled a number of axis switches so that we have some more port density and less traffic to the core and we have ?? because of the single hardware problem we can now do easy con generation and we are probably going to do a lot more with that.
So that is it. Any questions? Nobody.
ROB BLOKZIJL: Nobody has any questions. OK
(Applause)
ROB BLOKZIJL: The next speaker is Jan Schollhammer.
JAN SCHOLLHAMMER: Hello, and thanks at RIPE for being able to speak here. I think this is not so common that a vendor is able to speak here and thank you very much for the opportunity.
Yes, I would like to say just really some few words about AVM. The company itself has been founded in 1986 and at the moment has a market share of 65 percent of all ? shipped in Germany and if we look into Europe it has been 18 percent of all CPE DSL mod ems shipped in Europe. When we enable IPv6 in these routers, in these modems, then all the 65 or 18 percent get IPv6 ready so this is quitof can be switched on to IPv6 if we want that.
But I would like to talk a little bit aboVM network development department. It started 18 years ago with the development of a complete own IP routing stack for IS D N and went on and the essence of this slide probably is that we do most of what we can ourselves. We try to do as most as we can on ourselves, meaning, yes, we try to develop all those things on our own. And this is, of course, something that has advantages when it comes to new protocol because we can re use some of our structures and do not have to re invent everything from scratch but, yes, it was not so hard for us to migratemigrate to IPv6 so what we did is we started in 2008, at the beginning of 2008 with a development of our routing engine with the IPv6 development, and from March 2009 we did release, I must say it was a better release, it wasn't really a re /HRAOET, better release, for the C bit, /TPAEURBGS where we showed to the public what we did so far and this was basically a firm wear that has been re lesioned and this firm wear was completely, can't discuss that but completely IPv6 ready for the /SRA*PB site and LAN site.
So if you have a closer look at this, what does this mean on the LAN side we have quite much hosts in the home network, not only PCs and notebooks but also other host that is perhaps are already using IPv6, and on the other side there is the possibility to connect to the IPv6 network over native connection and also with tunnel connections.
In our firm, we have those two possibilities because we found out that there is not so much offerings to really an end user that is IPv6 enabled already, so what we did is we implemented tunnel protocols, two tunnel tunnel protocols, 6 to 4 and also to 6 protocol so that every end user has the ability to already experiment with IPv6 without having an ISP that already offers that to him.
But of course, the other things, the most important and also the more challenging part, of course, than implementing tunnel protocol. Tunnel protocols were, well, it was more like a game for developers they did overnight to play around for themselves at home.
What does this mean for our development team?
About 50 RFCs had to be implemented in the routing engine, mostly for the /WA*PB side, use some functionality /TK?RBGS /TK?RBGS for the /WA*PB side we did implement all those things ourselves about and about 50 RFCs had to be implemented and it was quite much, I think.
We did for the CE bandwidth a press announcement together with German free net that has been sold unfortunately now to united Internet so this doesn't exist /STPHEU more but we showed (CE B IT) a complete native IPv6 connection over real DSL line of German telecom, TD S L so?called, and it simply worked. It used PP P for the connection and IPv6 CP and del greated prefixes to us. Just really some kind of breakthrough that we experienced. We have a working device on real DSL line that is shipped to end customers to really home users.
So far we did four releases for that firmware so we tried to put all the feedback inside of it. Of course, we did not manage to get all the feedback inside it and there is still one lack which I will come to later in functionality, but we released four firm wares where we partly extended functionality so we now support much more LAN side protocols than we did at the beginning but also now PP O A than also those RFC 1483 and successors protocols meaning basically send directly over ethernet. There is international and German firmware available for this so everybody can try out the firmware, publically available and downloadable under these URLs here.
This is what it looks like. There is quite many options, like you ppp, meaning that your device gets an IP address itself, which is quite a nice thing for communicating.
There is also the possibility to not get an IP address for itself but some users off the delegated prefix and two tunnel options can be seen here.
This is what it looks like if it is connected.
Here we can see a tunnel connection 6 to 4 tunnel connection, but I hope you believe me that the others work, too.
What did that mean for us? We did some experiences with these ?? this firmware so, yes, we had to /PHEU great not only the routing stack itself but also quite many services that are running on such a Gateway in our days, it's not router, it's a little bit more; of course, it has web interface that had to be /PHEU greated and some ftp serveerMB but there is still a lot of stuff to do and there is even not for protocols and whole network is IPv6 enabled versions of it so for example when it comes to U PP protocols like U P V A for media sharing or U PP I G D which is protocol to control the router and especially the wire wall of the router and also to monitor the router, then you can see that there is not so much development there; there is a funny remark on some Microsoft documents I found about U P N P, if you are router vendor and implementing U P N PI G D with IPv6 please contact us, which we didn't because it's not really standardisation which they do, I think.
There is one more learning that we did, well, yeah, what to do when there is no globally prefix assigned to us? There is quite a lot of situations where we do not have any global prefix at all, for example the DSL line is down or there is one user credentials inside the router and the connection doesn't work because of these work credentials and so on and so on. Quite a lot of situations where there is no real globally valid IPv6 prefix. So what to do /STPHEPB how should local peers communicate then? There is link /HROEBG /A*PLal addresses but they are not really supposed for sending pay loads, really data, they are more supposed to for sending status messages and all those things but not really for data and especially it's not only ?? yes ?? it's not only academic question but really a question that most of the operating systems we know just don't Tuesday and don't communicate if they only have link local addresses. (Use it) so we decided to have unique local addresses in this case, so we assigned to the host and the LAN unique local address prefix. When there is no global valid prefix available. There is some solvable problems with WindWindows XP likes so much a that it sends packets with U LA source address even when there is other prefixes already assigned and they are more globally valid so it sends things to the Internet with U LA prefixes, which isn't really something that makes sense.
We weren't completely sure if eWe weren't completely sure if everybody wants it so we made this configureable. You have three possibilities and there is also, by the way, an English user interface of course in the international version. Just screen shows the German one, sorry. There is three options: Do not assign any unique local addresses; assign unique local addresses if there is no IPv6 connection at the moment; or assign them all the times.
We are next learning that some ISPs do not want and there is one and he is directly after me with his presentation, Marco, and they did not want to do numbered PPP but unnumbered PPP which was a problem at the beginning because we need an IP address for our own communication purposes, for example we have IP stack inside there. We do DNS requests of course, we have remote management protocols running there that communicate with peers in the Internet so we need a valid IPv6 address for that. What to do if nobody assigned us a valid IP address? We take one from the prefix that we have been assigned, from the network that we have been assigned.
Another learning that we did was we offer network seg mentation feature which people really like because we are having the kids with their virus computers and and my computer in another segment seems like a good idea so people really like this and use it for IPv4 so we found some ISPs that only offer us 64 networks and 64 is too small for this kind of purpose, because we are not allowed to assign networks that are smaller than 64 into an ethernet like network and if we want to do seg mentation we need bigger network, for most of the ISPs we see they are assigning now 56 netwoand so this isn't a problem.
And then one thing that I think Marco will also talk about soon is the IPv6 firewall. We have a very secure firewall at the moment inside ?? this is what we call our routers, well, it's quite secure because you can't configure it, there is no way to open it. There is only one way to open it, there is some a.m.lication level functionality already in site, for FTP or something like that, there is ways to open it from inner site but in general you can't configure it at the moment and of course, one good thing, not suchtask, simply extend your address fields in your current firewall configuration, use interface and everything is good. No, it's not that easy.
Because if you think that there is quidynamic stuff involved and then static ACLs wouldn't make any sense for firewall configuration because prefix changes at least, that is what we actually seen forthe ISPs that we are workiwith, one half are dynamic prefixes and this is the general case that we have to deal with, and also the host address may change because and there is privacy extensions telling exactly that, that one host should change its host address, at least when it's doing outer configuration. So ithat easy. In the former IPv4 times, we had the fact that probably one station at the LAN gets the same IP address, the same private IP address all the time, and it's easy to form a rule for that, but it's not so easy to do that for IPv6; there is real globally valid addresses in your LAN and these even change. So we are, at the moment, thinking about user?friendly implementation of this thing and this goes to the idea that we are using the host names that we know for almost all cases and simply use those host names for the firewall rules.
On the other hand, of course, one interesting question is what do I need a firewall rule for or why do I open up my firewall to addresses that change and are not reachable all the time from the same address? So, yes, the question is what is that good for? OK I can think about peer to peer other other things where this might make sense but for really servers that I want to run in my home network, dynamically addresss are not that well ?? this is where we have ended up with firewalls so far but we are open up to any inspiration that we get here from this meeting because there is simply no best practices for the CPEs at the moment we do not see so mimplementation whecould be inspired of or nobody has done that so far, as far as we can see it at least. So, it needs a little bit more of thinking. About what I can promise also to this audience and also to Marco is we will have a very simplified firewall where everything is getting configure today play around with but we do not think this is the best solution or this is some end customer friendly solution because they are not able to configure things like that with huge addresses, and so oranges which even change all the time.
To summarise things up: There is some additional specifications needed for the LAN services which we offer but this doesn't ?? it's not really a problem because everything works and of course we have a dual stack so IPv4 is still existing in the LAN and all those old Internet kitchen radios that have bought last year, still work, of course, but they are some services that need some re thoughts when it comes to IPv6. But it works and we can put this into production. We do not do that at the moment, we don't have so much ?? one we know is doing IPv6 not only inside the /HRA*PS but also for end customers, for really residential customers, and we could do that but, well, yeah, it's not so much ?? well, we feel, somehow, a little bit uncomfortable to put a feature inside a router and into production without having tested it at all with some real lines so we have test labs and have talked to many ISPs so far but yeah, it's little bit kind of a strange situation so we probably will continue with the better firmware that is available under those URLs I have showed and let's see what happens.
Yes, some questions that we really are asking us and perhaps we also are able to discuss that here.
What will it be at the end? Will thynamic prefixes? I think yes, but maybe there is another opinion there. What is the typical prefix length, I have seen quite often 56 and I think there is also some recommendations but let's see what real life will bring us. And one question at the end that I think is very interesting but probably won't be the case: There is such a wonderful feature in IPv6 that we have multicast streams tglobally weld, or we could have it, at least, and will there be some kind of this at the end? Let's see.
So, yes, it's really like, we do not have any reference implementation and really like to have feedback on the implementation that we did now, and there is the possibility to change things now and to establish best practices and, yeah, we would like to have your feedback. Thank you very much.
(Applause)
ROB BLOKZIJL: I think we have time for one or two /KES.
AUDIENCE SPEAKER: Alex. I understand som boxes have /S*EUT voice for does this sip /KHROEUPB also support IPv6.
JAN SCHOLLHAMMER: Not so far.
ROB BLOKZIJL: No more questions. Thank you.
(Applause)
ROB BLOKZIJL: Last speaker of this session is Marco Hogewoning and he will eat intffee break but rest assured you will have time to havebreak but rest assured you will have time to have a coffee. We will just shift the coffee break little bit.
MARCO HOGEWONING: As Rob said, I will be eating a bit of your time from the coffee break, but I will be fast.
IPv6 at XS4SALL, the big delay, you may have seen the last presentation I did at RIPE a.m. /STER /TKARPBLGS it's a bit of the agenda. Who am I, why am I here, a bit on our running pilot, an overview of the CPE devices we have encountered so far and tested and a bit on the delay.
Who am I: Marco Hogewoning, engineer, this is the regular slide of our K one of the oldest Dutch ISPs, currently owned by KPN. 200 employees and serve around 300,000 customers on most Lyon DSL, got quite a bit of hostinging and do some co?location. I was there at RIPE 58, you may have seen that presentation for those who haven't, it's still available in the archives. What we did there was we a/TPHOUBSed working set?up. We are using /TPREUTS /PWORBGSs for a couple of years now, issues their v6 release, we got it working within a couple of weeks, so we got up there at RIPE 58 and told us ?? told people where we would stand and what we were doing. At the same time, we issued press release, well that is nothing specyes, we did actually hit the major ?? one of the major newspapers in the Netherlands, the snippet is right there. Roughly translates to the Internet is getting an unlimited address book so some of it got lost in trance /HRAEUFPLGTS it was one of the first times IPv6 got mentioned in a major newspaper in the Netherlands and it's not only this but they couple of other newspapers also printed this.
Like I said, we started small scale pilotly friendly users, a couple of people we know who wanted to play around with it, in fact those guys from AVM themselves in the Dutch office /R?P connected up and I announce more to come at RIPE 59 so here I am at RIPE 59 with more to come. AP bit on the status then. We have approximately 40 people entered in our pilot now. We have seen around 30 or 35 of those connections live. We have got one or two configuration issues at the moment, especially with Cisco, they seem to be a bit difficult to configure if people are using the web interface. No wonder there. We have got two customers who are actually the current firewall is doing sensible thing by dropping all incoming connections but you can't turn behaviour off so we have two people stopped using it because it broke their mill set up and switched back to using tunnel and 3 or 4 people we actually lost. They simply no time to connect it yet or being lazy.
Those customers we entered claw couple of journalists and NCC staff members, like I said some people from vendors who connected up to us, we also got various other operators who are using our infrastructure to gain some experience in testing.
And technically, our infrastructure is more or less ready to roll out, some bits and pieces left over to do but we are pretty much ready to go.
Like I said, a bit on the CPE, /TPR*EUTS box. It's actually working, main issue like Jan said is the firewall and we have got some issues with the back board, got about 70,000 modems in our network which are slightly older model and would be nice we get v6 working on those as well. Obviously Jan with prefer me to buy 70,000 new modems but there is some discussion there. We will sort it out /TOPBLT after a beer or so.
The other one, more or less working. It's fully configureable, you don't have you can run IC L to stop packets from get not guilty or getting out.
Whatever you like. Reported issues on the wireless, I haven't got one with wireless interface myself but apparently the bridge interface you need for that to work isn't v6 capable so it's either v6 running your wireless, and not both at the same time.
The require some knowledge and expertise, we have been to hell with one of our support people who bought one and apparently my pristine simple configuration does work but if he pushes in his configuration it, all of a sudden, stops working for IPv6. There is probably some AC L in the way, used some interface to generate it, must be one or two bits wrong and main thing with Cisco, it's very expensive and not really suitable for a large scale residential deployment.
Juniper net screen, some of you may have been /TPHARPBD cope en/HAG enthree weeks ago, I gave an update there. We couldn't get it to work. IPv6 CP simply seems to be missing or broken. We can't see the negotiation coming up. Obviously, we couldn't test the D H ?? we couldn't get connection to work.
What happened since is is a day later, Juniper called me ah you are using called me ah you are using the wrong hardware so they issued new hardware and for test and I received it last week, so test is planned for this month, it's actually signature on my desk as soon as I get back into Amsterdam I can test this.
/THOPL son, about the same thing happened N cop enHague enit was they claimed to do IPv6 and it's not there three days later, I get an invitation for a meeting. The beta is ready for testing /TPHOUFPLT it's very basic installation is no problem. Only doing PPP O E at the mowing. It's forwarding packets, it's got some basic fire walling on it, there is a absolutely no services whatsoever, no configuration yet on the web inter /TPAEURBGS it's all text interface yet. But apparently it's there. We are trying to plan first test next week, they are coming over. They wouldn't release the software yet but coming over with a box with the software pre loaded and /T*EGSed on our network and we will see what happens there.
Bray tech it, the last of our list, nothing much happened, they asked happened, they asked us for specifications, in May already. We gave them, this is what we wanted implemented it. They ran off with it and we never heard from them again. It may be a good thing, maybe next week there is a /HREUG box on my desk.
You may not know but on the other hand it might as well be that they simply stopped, looked at the specs and never did anything with it.
So what is up? The infrastructure is clear, we have got everything on track. There obstacles in the way. This pick tire just ran into this situation Saturday here up in the old town.
Well, operational issues: Yes, of course, we have got some. Resources. It's a ?? got full support of our management and they recognise that IPv6 is important but of course, other projects are even more important so by the time you have got stuff around and all of a sudden your resources are gone.
Other main point is dual stack on Juniper E320 routers, we used to /TPERPL Nate connections.
They ?? there are maximum number of interfaces, dowel stack v4 and 6 are two interfaces so you lower the totality number of customers being able to connect to that box by 50 percent. Sounds easy, put in more hardware, but obviously you need money to do that. We are planning for ?? it's budgeted for next year and we donfor next year and we don't think that all our customers will deploy IPv6 at the sacustomers will deploy IPv6 at the same time, there is no ?? ain't going to be a /PHRAG day so with slow growth on IPv6 we can probably cope with it but it's something to think about and plan for.
Our major thing as I said in my presentation at RIPE 58 our provisioning pool is called V I, we are working on it, I have seen some design signs and people are code /TPHAOG test beds to get IPv6 provisioning working in a basic state.
Support, the other main headache, how to support customers. They obviously need training, well, how to create training material when stuff is still beta. There is no graphical user interface we can make screen shots of, it's still very hard for those guys itself to install a modem at home and test with it. It's pretty hard. On other hand, current feedback from current test group there ain't much to support. Apparently it's all working, I haven't heard much complaints from them.
So it seems all reasonable.
Sales, the other big issue, as usual demand is still very low. I have seen three requests for business to business on IPv6 this year. We have got a couple of 100 from residential, which is kind of strange. I get the feeling there is some form of coolness factor to actually have IPv6 at home.
We have got people can I get native IPv6, breakthrough the fence or something. The other main part for residentials are if in fact IT professionals who want to use their home connection to gain some experience and play around with it.
It's very strange. The same guys who make the decision buys wise say no to IPv6 and ask you if you can enabling it on their home connection.
There is something there. They are aware of the problem but not business?wise.
Bit on the legal side. Prepare to be annoyed. There is no disclaimer there. This is Dutch law, your local situation might abbit different. We have got 3 topics: First of all is lawful interception. The interpret laws in the Netherlands basically state that you should be able to do both v4 and v6 intercepting. At the moment, the specification is there but the implementation isn't. It's obviously very small market so you only have to talk to one or two vendors who have to implement this and, yes, they are not really jumping around like ah we are going to do this, technically they promise a lot and don't come forward and yeah, if we want to rush things we need to pay them and so, there is a bit of struggle there. But it's very important, we just ?? non?compliance, if I start deploying IPv6 and not able to fulfil intercept warrant, can you run up to 10 percent of your yearly revenue so you are basically bankrupt for not being compliant.
We are working on it, still no progress and this is basically the main thing that is holding our pilot back. At the moment we have got this one sorted, we can easily scale up to 500 or thousand customers but at the moment, this is currently stands, the lawyers simply forbid me to opening up the pilot and get things going.
Other minor thing is sealed presentation from those guys was at RIPE 58. It's database which basically holds all phone numbers, e?mail addresses, both v4 and v6, hooked up to names, addresses, we actually need to push out a date every 24 hours, apply the list of all our customers with all reference data. It's accessible by virtually every police employee without a warrant, they have to fill in a little number saying like, it's this research or I am working on this case, but there is no actual judge signing off on it. The database currently is running 250,000 requests a month and obviously that needs to fit with v6. We covered that one, the specification, it's an XML stuff is capable of running IPv6. We are capable of uploading it. We haven't testing it it yet and kind of wondering if they ever did. Maybe if we stuff in the IPv6 address the first one, the whole database goes belly?up but we will find out
Last point here: This is really specific for the Netherlands but, yes, it's a success. They have 3 million requests each year, for police terms that is a really successful installation. So they are show casing this to the rest of the EU and other countries are actually looking into deploying the same kind of structure so be advised that you may or may not encounter the same system within a couple of years.
Last one: Data retention, laws passed in the Netherlands. The conclusion we had a couple of weeks ago from Copenhagen was at the moment the collecting specification does specify IPv6 but in the retrieval specification they forgot to put it in. General consensus is somebody will eventually notice that error and will correct it.
Impact: Yet unknown. The good thing on this it's all fairly new and still has to be built but will probably be built dual stack from day one and so it's not really going to be a public or a blocking issue for v6 deployment, I think.
Well, was it useful so far? Yeah we stood up there at RIPE 58 claiming too far small pilot and opening it up which obviously we didn't do. The results so far, we have got some print coverage about actually the problem, the Internet is running out of address space, help. We have been contacted by vendors, like I said. You stand up here, you say something and all of a sudden you got that phone call like ah there is your test hardware or we can do this, and in fact like Jan showed up here to explain what they are doing, so it's working. We have been invited to various meetings and fora. I have been helping other operators, also talking to large enterprises who are researching what is that v6 thing and should we do something about it? But it's staking up.
And my reason for that is you can't do it all by yourself. Talk to each other, share experiences. We found it very useful to do that, actually tell what you are running into, show what you did already and actually help each other and one of the main things is join each other towards vendors. You all familiar with the fact you are always the first one to ask. You are never the second. Always if you phone up, can you do v6, oh we never get that request ? you are the first one to ask. Yes sure. Make the problem known to the public. We see sales demand is very low and as long as you don't get the word out in the newspaper, get that coverage on TV, the general public isn't aware of what is coming and it's very hard to get demand up and tell your boss to actually spend money on this. So, shout about it, tell everybody the world is going to end.
Now, you may have notice that had there are some bits of ?? Geoff skipped it, we know you all love trains. Now, the pictures I have shown so far, most of them are made in Amsterdam and they might look familiar. If you have been around in May, you see basically the city is one big ditch building area and they are actually doing a metro line, a new one, north ? south. Original idea 1968. In 1997 referendum over it, the result was no. They ignored the fact, decide today build it anyway in 2002. Original the build was started in 2003. At that time, the estimated delivery of that project was 2011. Now, at the moment, planning is that it will be ready somewhere near 2020, at the same time because there is run?up from one billion euros to over four billion euros so there is slight problems and people are discussesing should we abandon this. And the reason I put this up here is, I have seen the same thing happening on IPv6; maybe we should actually the whole v6 thing, let's go nuts and forget about the whole thing. You ain't going to know what you get. But in the end it will look like this and will all be fancy and stick with the big for another ten years and we will find out.
That is all I have to say. (Applause)
And hopefully more in Prague. Thank you. I think it's either questions or coffee.
AUDIENCE SPEAKER: I have a question from ?? I have a question from Neill Backer from AMS?IX on Jabber, what transport does axis for all support IPv6 over towards its home customers, PPPoE or only 48 bridged.
MARCO HOGENWONING: At the moment PPP only and at the moment we have thinking about the solution. It's very hard, at the same time, we are basically, our bridge connections are more or lessened of line all new connections being made are point to point so we are slowly phasing out IP or bridge connections so at the moment we are not focusing on that. It should be possible but it's hard and we focus on the main thing and about 75 or 80 percent of our customer base at the moment is PPP so that is the more important part.
Neil O'Reilly: Maybe we should learn from this Amsterdam approach to referenda in Ireland. But the real question is: Is there a T?shirt based on this last slide?
(Applause)
SPEAKER: I know there is one. She is not here I guess. But we are working on it. I will take your point, I will bring a T?shirt next time.
ROB BLOKZIJL: OK. No more questions. Thank you Marco.
(Applause)
ROB BLOKZIJL: So this is the end of the first part of the afternoon session. As promised, your coffee break has shifted a little bit. I would strongly invite you to have a coffee or whatever and be back at a quarter past 4.