On January 31, Gandi and CloudFlare presented a panel discussion as part of our joint Meetup series The Root Zone. which focuses on DNS-related topics. If you haven’t had the opportunity to attend before, or had the chance to catch up on the series so far, they are all available online.
This time marked a departure from the previous format of interviewing various DNS luminaries by bringing together a panel of experts to speak on a specific problem in DNS that notably reared its head last October. We’re talking, of course, about DDoS, or a Distributed Denial of Service attack.
CloudFlare hosted this panel in their space at the end of January
Cloudflare and Gandi invited DNS inventor Paul Mockapetris, Chris Baker, principal data analyst at Dyn, Brian Hartvigsen, who works as SRE Manager for OpenDNS, another DNS provider, Andrew Lewman from threat intelligence firm Farsight Security (and former CEO of the Tor Project), as well as Gandi’s own Chief Innovation Officer Pascal Bouchareine. The panel was moderated by Cloudflare Engineering Manager Gideon Redelinghuys.
You can watch the whole panel discussion on YouTube below:
Or jump to specific questions from the list below:
1. Describing October’s attack
After introductions and a brief history of DDoS, we wanted to start off describing the attack in October on Dyn’s services.
Gideon Redelinghuys from CloudFlare moderates (photo credit: Dani Grant, Cloudflare)Chris described how the October attack and others like it tend to start as spikes of anomalous activity. One reason we wanted to include OpenDNS was because of the solution they’ve devised to address the authoritative exhaustion problem. Brian from OpenDNS explained it in greater detail. That opened up the question of what, if any, is the ideal TTL, which was posed to Chris.
Of course, authoritative exhaustion and amplification attacks are nothing new. DDoS has been around for decades, which almost begs the question: why did this happen now? Andrew emphasized the fact that the October DDoS was the culmination of a lot of trial-and-error testing.
Paul made a good point about the asymmetry between increased bandwidth and centralized DNS hierarchy: don’t we have to think about the fundamental asymmetry in this type of attack?
As we started talking about root causes and solutions, we didn’t think this would last long without mention of BCP38.
For those who aren’t familiar with it, BCP38 recommends what’s generally called ingress filtering. It’s a proposed means to combat IP address spoofing that involves blocking IP packets entering the internet with forged source IP addresses, that is, not assigned to the device that’s sending them.
Brian brought up a good point, though, which is that when we talk about “solutions” the target is artificial.
Brian Hartvigsen from OpenDNS discusses DDoS attacks (photo credit: Dani Grant, Cloudflare)Paul highlighted the critical theoretical question underlying the discussion of solutions: Will we find a solution that preserves the network as we know it and want it?
And Chris elaborated on one direction Paul’s comments brought up, talking about the conflict between a free and open internet and the proverbial “golden list of IPs” came up, and the need for real, direct relationships between providers on different levels.
In the end though, as Brian put it, solving these problems comes down to something unglamorously non-technological.
4. In the middle of the attack
We thought it would be interesting for a little bit of an inside perspective on the experience of being in the middle of the storm as it were.
Chris described the fact that the experience as a data analyst in the middle of a big attack like October’s DDoS can be at odds with the NOC perspective, especially at Dyn last October.
At OpenDNS, however, the experience was very different, according to Brian.
5. Minor miracles and things to be thankful for
We asked what some of the blessings the panelists were thankful for during these attacks in the past. Brian’s answer was pretty unequivocal: Anycast and having a good playbook, and for his part, Chris followed up with a discussion about 20-bit case randomization to introduce entropy that helps with identifying spoofing.
6. The underlying problem hasn’t gone away
Paul turned the praise of Anycast into a pointed question about why the Mirai attack went away. Andrew pointed out that while this attack is mitigated, the underlying problem still hasn’t gone away. So what’s to stop it, then, from coming back with a vengeance? Chris reminded us of something crucial to remember about DDoS: DDoS is the bluntest instrument you can use.
Pascal Bouchareine from Gandi talks about the need for good communication (photo credit: Dani Grant, Cloudflare)While we were primarily focused on considering DDoS as a technical problem, we did take a moment to think about how DDoS is employed for specific purposes in the real world. Brian continued that thought, which led to a question we had for Paul. And while we had Andrew with us, we decided to poke him about Tor’s role.
8. Q&A
We finished off the meet up with a Q&A session from the audience:
Q. Why isn’t the fact that DNS doesn’t serve the last-known answer when no other answer is available considered a bug?
Brian and then Chris fielded this one.
Q. Is there some way for a customer to know what’s good vs. what’s bad?
“What’s good is what works for your network.”
Q. How do we protect ourselves, really?
Is centralization or decentralization the answer?
And with that, we wrapped things up. And while if you’re watching these, we assume you couldn’t make it, we hope you caught the gist of it here. We hope to be able to host a few more panel discussions on relevant DNS topics in the coming months. Follow the meetup page for The Root Zone. for information about future events, but we will also be sure to keep you updated here as well.
