ietf
[Top] [All Lists]

Re: Update Re: IETF Website Degradation

2015-08-03 18:20:14
Job -

Thank you so much!  Yes, we are aware of those numbers,  but thank you so
much for making sure we have them!  And thanks also for making them aware
of our situation here!

Best regards,
Glen
Glen Barney
IT Director
AMS (IETF Secretariat)

On Mon, Aug 3, 2015 at 3:50 PM, Job Snijders <job(_at_)instituut(_dot_)net> 
wrote:

I pinged two cloudflare employees via facebook to make them aware of the
fact you are trying to reach them.

Did you try the following to reach cloudflare:

    US callers: 1 (888) 99-FLARE
    UK callers: +44 (0)20 3514 6970
    International callers: +1 (650) 319-8930

Kind regards,

Job

On Mon, Aug 03, 2015 at 03:24:08PM -0700, Glen wrote:
All -

We have determined that the degradation was caused by a DDoS attack
against
the www.ietf.org website.  The attack was a slowly-escalating attack,
which
began several hours ago, and increased in load over the afternoon.  The
attack was directed at the Cloudflare servers, so we were not immediately
impacted.

However, as time passed, the results of the attack started to spill over
to
the actual IETF webservers, with the result that our webservers started
to
slow.  We were alerted to this by our own monitoring systems, which is
when
we did an initial check, and I then sent the initial report out.

At this point, we have been unable to reach a human at Cloudflare,
although
we are continuing to try.  We have therefore put our Cloudflare account
into "DDoS Mitigation Mode".

In this mode, users will see a brief interstitial page when browsing the
IETF website.  This page allows Cloudflare to perform testing on each
browser to determine whether the request is part of an attack or not.
You
may see this page as you approach the IETF website.  It is nothing to be
alarmed about, and is an expected side-effect of this protection mode.

It is unknown, at this point, why Cloudflare did not automatically
detect,
and block, the attack.

It is unknown, at this point, why the attack caused Cloudflare to start
spilling requests over to us.

It is unknown, at this point, why we are unable to reach a human there.
:-)

However, at this time, website service is restored, and, apart from the
interstitial page on the IETF website, everything is running as expected.
We will continue to reach out to Cloudflare to address these remaining
issues, and will get that check page deactivated as quickly as possible.

Thank you for your patience during that happily brief degradation.

Glen
Glen Barney
IT Director
AMS (IETF Secretariat)