FYI
The gateway machine at UCSD has been under a sustained DDOS attack now for over 24 hours, so if anyone is seeing heavy packet loss through the gateway, that’s why. The 1Gb/s interface is max’d out. You can view the interface stats here:
73, Chris - G1FEF — ARDC IT Director
Web: https://www.ardc.net
It would be interesting to know if BGPed 44 subnets are also seeing this?
On Sat, Sep 25, 2021, 01:56 Chris Smith via 44Net 44net@mailman.ampr.org wrote:
FYI
The gateway machine at UCSD has been under a sustained DDOS attack now for over 24 hours, so if anyone is seeing heavy packet loss through the gateway, that’s why. The 1Gb/s interface is max’d out. You can view the interface stats here:
73, Chris - G1FEF — ARDC IT Director
Web: https://www.ardc.net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
There was a ddos with SYN/ACK against ON3YH’s BGP subnet, but that is the only one I heard of uptil now. My own subnet does not show any signs (yet) nor does the general /16 for Belgium. Fingers crossed that it stays that way..
Ruben - ON3RVH
On 25 Sep 2021, at 17:38, K7VE - John via 44Net 44net@mailman.ampr.org wrote:
It would be interesting to know if BGPed 44 subnets are also seeing this?
On Sat, Sep 25, 2021, 01:56 Chris Smith via 44Net 44net@mailman.ampr.org wrote:
FYI
The gateway machine at UCSD has been under a sustained DDOS attack now for over 24 hours, so if anyone is seeing heavy packet loss through the gateway, that’s why. The 1Gb/s interface is max’d out. You can view the interface stats here:
73, Chris - G1FEF — ARDC IT Director
Web: https://www.ardc.net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
Maybe sign up for the UTRS feed from https://team-cymru.com/community-services/utrs/ we use it here for our BGP. Could help. ________________________________ From: 44Net 44net-bounces+colin.bodor=imperium.ca@mailman.ampr.org on behalf of Ruben ON3RVH via 44Net 44net@mailman.ampr.org Sent: Saturday, September 25, 2021 9:48:16 AM To: 44Net general discussion 44net@mailman.ampr.org Cc: Ruben ON3RVH on3rvh@on3rvh.be Subject: Re: [44net] Sustained DDOS attack
There was a ddos with SYN/ACK against ON3YH’s BGP subnet, but that is the only one I heard of uptil now. My own subnet does not show any signs (yet) nor does the general /16 for Belgium. Fingers crossed that it stays that way..
Ruben - ON3RVH
On 25 Sep 2021, at 17:38, K7VE - John via 44Net 44net@mailman.ampr.org wrote:
It would be interesting to know if BGPed 44 subnets are also seeing this?
On Sat, Sep 25, 2021, 01:56 Chris Smith via 44Net 44net@mailman.ampr.org wrote:
FYI
The gateway machine at UCSD has been under a sustained DDOS attack now for over 24 hours, so if anyone is seeing heavy packet loss through the gateway, that’s why. The 1Gb/s interface is max’d out. You can view the interface stats here:
73, Chris - G1FEF — ARDC IT Director
Web: https://www.ardc.net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
_________________________________________ 44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
Nothing here at AS 55016 ________________________________ From: 44Net 44net-bounces+colin.bodor=imperium.ca@mailman.ampr.org on behalf of K7VE - John via 44Net 44net@mailman.ampr.org Sent: Saturday, September 25, 2021 9:35:25 AM To: 44Net general discussion 44net@mailman.ampr.org Cc: K7VE - John k7ve@k7ve.org Subject: Re: [44net] Sustained DDOS attack
It would be interesting to know if BGPed 44 subnets are also seeing this?
On Sat, Sep 25, 2021, 01:56 Chris Smith via 44Net 44net@mailman.ampr.org wrote:
FYI
The gateway machine at UCSD has been under a sustained DDOS attack now for over 24 hours, so if anyone is seeing heavy packet loss through the gateway, that’s why. The 1Gb/s interface is max’d out. You can view the interface stats here:
73, Chris - G1FEF — ARDC IT Director
Web: https://www.ardc.net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
_________________________________________ 44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
Nothing directed to my /24
Matt
On 26/9/21 1:35 am, K7VE - John via 44Net wrote:
It would be interesting to know if BGPed 44 subnets are also seeing this?
On Sat, Sep 25, 2021, 01:56 Chris Smith via 44Net 44net@mailman.ampr.org wrote:
FYI
The gateway machine at UCSD has been under a sustained DDOS attack now for over 24 hours, so if anyone is seeing heavy packet loss through the gateway, that’s why. The 1Gb/s interface is max’d out. You can view the interface stats here:
73, Chris - G1FEF — ARDC IT Director
Web: https://www.ardc.net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
On 25 Sep 2021, at 10:40, Chris Smith via 44Net 44net@mailman.ampr.org wrote:
FYI
The gateway machine at UCSD has been under a sustained DDOS attack now for over 24 hours, so if anyone is seeing heavy packet loss through the gateway, that’s why. The 1Gb/s interface is max’d out. You can view the interface stats here:
Seems to be a DNS DDoS.
Can’t you filter upstream? Let me know if you need assistance.
Borja - EA2EKH
And how would that help? It won't make the bandwidth available, the packets would still saturate the public interface.
On 27/09/2021 09:15, Borja Marcos via 44Net wrote:
On 25 Sep 2021, at 10:40, Chris Smith via 44Net 44net@mailman.ampr.org wrote:
FYI
The gateway machine at UCSD has been under a sustained DDOS attack now for over 24 hours, so if anyone is seeing heavy packet loss through the gateway, that’s why. The 1Gb/s interface is max’d out. You can view the interface stats here:
Seems to be a DNS DDoS.
Can’t you filter upstream? Let me know if you need assistance.
Borja - EA2EKH
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
Marius,
If you filter it upstream (meaning filter it at the transit border level) it would not saturate the public interface anymore as it would not reach the transit pipe anymore.
73
Ruben ON3RVH
-----Original Message----- From: 44Net 44net-bounces+on3rvh=on3rvh.be@mailman.ampr.org On Behalf Of Marius Petrescu via 44Net Sent: Monday, September 27, 2021 08:18 To: 44Net general discussion 44net@mailman.ampr.org Cc: Marius Petrescu marius@yo2loj.ro Subject: Re: [44net] Sustained DDOS attack
And how would that help? It won't make the bandwidth available, the packets would still saturate the public interface.
On 27/09/2021 09:15, Borja Marcos via 44Net wrote:
On 25 Sep 2021, at 10:40, Chris Smith via 44Net 44net@mailman.ampr.org wrote:
FYI
The gateway machine at UCSD has been under a sustained DDOS attack now for over 24 hours, so if anyone is seeing heavy packet loss through the gateway, that’s why. The 1Gb/s interface is max’d out. You can view the interface stats here:
Seems to be a DNS DDoS.
Can’t you filter upstream? Let me know if you need assistance.
Borja - EA2EKH
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
_________________________________________ 44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
So, it was a TCP port 80 attack directed against one IP address.
I passed this on to my contacts at CAIDA who passed it onto the UCSD NOC who have got the IP blocked.
The traffic has now returned to normal.
Chris - G1FEF
On 27 Sep 2021, at 07:25, Ruben ON3RVH via 44Net 44net@mailman.ampr.org wrote:
Marius,
If you filter it upstream (meaning filter it at the transit border level) it would not saturate the public interface anymore as it would not reach the transit pipe anymore.
73
Ruben ON3RVH
-----Original Message----- From: 44Net 44net-bounces+on3rvh=on3rvh.be@mailman.ampr.org On Behalf Of Marius Petrescu via 44Net Sent: Monday, September 27, 2021 08:18 To: 44Net general discussion 44net@mailman.ampr.org Cc: Marius Petrescu marius@yo2loj.ro Subject: Re: [44net] Sustained DDOS attack
And how would that help? It won't make the bandwidth available, the packets would still saturate the public interface.
On 27/09/2021 09:15, Borja Marcos via 44Net wrote:
On 25 Sep 2021, at 10:40, Chris Smith via 44Net 44net@mailman.ampr.org wrote:
FYI
The gateway machine at UCSD has been under a sustained DDOS attack now for over 24 hours, so if anyone is seeing heavy packet loss through the gateway, that’s why. The 1Gb/s interface is max’d out. You can view the interface stats here:
Seems to be a DNS DDoS.
Can’t you filter upstream? Let me know if you need assistance.
Borja - EA2EKH
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
Tnx. Chris for the update.
I'm still wondering what the goal of such an attack is...
On 27/09/2021 10:30, Chris Smith via 44Net wrote:
So, it was a TCP port 80 attack directed against one IP address.
I passed this on to my contacts at CAIDA who passed it onto the UCSD NOC who have got the IP blocked.
The traffic has now returned to normal.
Chris - G1FEF
Yes it would be worthwile to research (with the IP address as information) what could be the reason behind this. Assuming it was not 44.0.0.1 but some amateur's IP, it could be some retaliation against that person and they may be able to identify a likely source and legal action may be possible.
Aside from that, I think there is too little headroom on this connection and it needs to be upgraded to 10Gbit or some teamed 1Gbit links when that is more practical. The background noise already takes up 650Mbit/s of the 1Gbit/s available...
Rob
On 9/27/21 9:33 AM, Marius Petrescu via 44Net wrote:
Tnx. Chris for the update.
I'm still wondering what the goal of such an attack is...
On 27/09/2021 10:30, Chris Smith via 44Net wrote:
So, it was a TCP port 80 attack directed against one IP address.
I passed this on to my contacts at CAIDA who passed it onto the UCSD NOC who have got the IP blocked.
The traffic has now returned to normal.
Chris - G1FEF
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
I agree with Rob! Normally you should start upgrading the connection if the average reaches 50% of it's capacity
-- Tim (PH4T)
On Mon, 27 Sept 2021 at 10:03, Rob PE1CHL via 44Net 44net@mailman.ampr.org wrote:
Yes it would be worthwile to research (with the IP address as information) what could be the reason behind this. Assuming it was not 44.0.0.1 but some amateur's IP, it could be some retaliation against that person and they may be able to identify a likely source and legal action may be possible.
Aside from that, I think there is too little headroom on this connection and it needs to be upgraded to 10Gbit or some teamed 1Gbit links when that is more practical. The background noise already takes up 650Mbit/s of the 1Gbit/s available...
Rob
On 9/27/21 9:33 AM, Marius Petrescu via 44Net wrote:
Tnx. Chris for the update.
I'm still wondering what the goal of such an attack is...
On 27/09/2021 10:30, Chris Smith via 44Net wrote:
So, it was a TCP port 80 attack directed against one IP address.
I passed this on to my contacts at CAIDA who passed it onto the UCSD
NOC who have got the IP blocked.
The traffic has now returned to normal.
Chris - G1FEF
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
There has been a plan in place to upgrade the link to 10Gb/s for nearly a year, there were two attempts at upgrading which failed due to the new link not working, but UCSD have now identified the issue and the upgrade has been expedited and, all being well, it will be upgraded within the next week or two.
Of course that just means the next DDOS will be hitting the gateway much harder, but at least we will have some decent headroom under normal circumstances.
73, Chris - G1FEF
On 27 Sep 2021, at 09:20, Tim de Boer via 44Net 44net@mailman.ampr.org wrote:
I agree with Rob! Normally you should start upgrading the connection if the average reaches 50% of it's capacity
-- Tim (PH4T)
On Mon, 27 Sept 2021 at 10:03, Rob PE1CHL via 44Net 44net@mailman.ampr.org wrote:
Yes it would be worthwile to research (with the IP address as information) what could be the reason behind this. Assuming it was not 44.0.0.1 but some amateur's IP, it could be some retaliation against that person and they may be able to identify a likely source and legal action may be possible.
Aside from that, I think there is too little headroom on this connection and it needs to be upgraded to 10Gbit or some teamed 1Gbit links when that is more practical. The background noise already takes up 650Mbit/s of the 1Gbit/s available...
Rob
On 9/27/21 9:33 AM, Marius Petrescu via 44Net wrote:
Tnx. Chris for the update.
I'm still wondering what the goal of such an attack is...
On 27/09/2021 10:30, Chris Smith via 44Net wrote:
So, it was a TCP port 80 attack directed against one IP address.
I passed this on to my contacts at CAIDA who passed it onto the UCSD
NOC who have got the IP blocked.
The traffic has now returned to normal.
Chris - G1FEF
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
Hi,
Note that this network is not the only one under attack. Voip.ms and bandwidth.com are both suffering. Voip.ms was ransomed for about 4 million. This should be reported to the fbi.
Chris, n3cgm
On Mon, Sep 27, 2021, 5:12 PM Chris Smith via 44Net 44net@mailman.ampr.org wrote:
There has been a plan in place to upgrade the link to 10Gb/s for nearly a year, there were two attempts at upgrading which failed due to the new link not working, but UCSD have now identified the issue and the upgrade has been expedited and, all being well, it will be upgraded within the next week or two.
Of course that just means the next DDOS will be hitting the gateway much harder, but at least we will have some decent headroom under normal circumstances.
73, Chris - G1FEF
On 27 Sep 2021, at 09:20, Tim de Boer via 44Net 44net@mailman.ampr.org
wrote:
I agree with Rob! Normally you should start upgrading the connection if
the
average reaches 50% of it's capacity
-- Tim (PH4T)
On Mon, 27 Sept 2021 at 10:03, Rob PE1CHL via 44Net <
44net@mailman.ampr.org>
wrote:
Yes it would be worthwile to research (with the IP address as
information)
what could be the reason behind this. Assuming it was not 44.0.0.1 but some amateur's IP, it could be some retaliation against that person and they may be able to
identify a
likely source and legal action may be possible.
Aside from that, I think there is too little headroom on this connection and it needs to be upgraded to 10Gbit or some teamed 1Gbit links when that is more practical. The background noise already takes up 650Mbit/s of the 1Gbit/s
available...
Rob
On 9/27/21 9:33 AM, Marius Petrescu via 44Net wrote:
Tnx. Chris for the update.
I'm still wondering what the goal of such an attack is...
On 27/09/2021 10:30, Chris Smith via 44Net wrote:
So, it was a TCP port 80 attack directed against one IP address.
I passed this on to my contacts at CAIDA who passed it onto the UCSD
NOC who have got the IP blocked.
The traffic has now returned to normal.
Chris - G1FEF
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
Hello Chris,
This 1Gbps link is to the IPIP gateway system in San Diego right? If so, even if you get a faster pipe, can this gateway server (I want to say this is a FreeBSD machine right?) actually forward at those rates?
--David KI6ZHD
On 09/27/2021 02:10 PM, Chris Smith via 44Net wrote:
There has been a plan in place to upgrade the link to 10Gb/s for nearly a year, there were two attempts at upgrading which failed due to the new link not working, but UCSD have now identified the issue and the upgrade has been expedited and, all being well, it will be upgraded within the next week or two.
Of course that just means the next DDOS will be hitting the gateway much harder, but at least we will have some decent headroom under normal circumstances.
73, Chris - G1FEF
On 27 Sep 2021, at 09:20, Tim de Boer via 44Net 44net@mailman.ampr.org wrote:
I agree with Rob! Normally you should start upgrading the connection if the average reaches 50% of it's capacity
-- Tim (PH4T)
On Mon, 27 Sept 2021 at 10:03, Rob PE1CHL via 44Net 44net@mailman.ampr.org wrote:
Yes it would be worthwile to research (with the IP address as information) what could be the reason behind this. Assuming it was not 44.0.0.1 but some amateur's IP, it could be some retaliation against that person and they may be able to identify a likely source and legal action may be possible.
Aside from that, I think there is too little headroom on this connection and it needs to be upgraded to 10Gbit or some teamed 1Gbit links when that is more practical. The background noise already takes up 650Mbit/s of the 1Gbit/s available...
Rob
On 9/27/21 9:33 AM, Marius Petrescu via 44Net wrote:
Tnx. Chris for the update.
I'm still wondering what the goal of such an attack is...
On 27/09/2021 10:30, Chris Smith via 44Net wrote:
So, it was a TCP port 80 attack directed against one IP address.
I passed this on to my contacts at CAIDA who passed it onto the UCSD
NOC who have got the IP blocked.
The traffic has now returned to normal.
Chris - G1FEF
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
I would think that FreeBSD of all things would be fine forwarding at much more than that.
At work we lease Dedicated servers to a wide variety of VPN providers, and they are all having them delivered with dual 10Gb interfaces - and pushing 6-7Gbps per server. Now most of the providers run customised *nix platforms, but one in particular (who I won’t name) runs Windows 2019 and uses RRAS and it handles it ok.. so provided the CPU is relatively recent and has the appropriate extensions to support hardware encryption (assuming the 44net tunnels are even encrypted), and the NIC has checksum offload, etc, FreeBSD shouldn’t have any problems whatsoever :)
—DG
VK2TDG/DGJ
On Tue, 28 Sep 2021 at 7:21 am, David Ranch via 44Net < 44net@mailman.ampr.org> wrote:
Hello Chris,
This 1Gbps link is to the IPIP gateway system in San Diego right? If so, even if you get a faster pipe, can this gateway server (I want to say this is a FreeBSD machine right?) actually forward at those rates?
--David KI6ZHD
On 09/27/2021 02:10 PM, Chris Smith via 44Net wrote:
There has been a plan in place to upgrade the link to 10Gb/s for nearly
a year, there were two attempts at upgrading which failed due to the new link not working, but UCSD have now identified the issue and the upgrade has been expedited and, all being well, it will be upgraded within the next week or two.
Of course that just means the next DDOS will be hitting the gateway much
harder, but at least we will have some decent headroom under normal circumstances.
73, Chris - G1FEF
On 27 Sep 2021, at 09:20, Tim de Boer via 44Net 44net@mailman.ampr.org
wrote:
I agree with Rob! Normally you should start upgrading the connection if
the
average reaches 50% of it's capacity
-- Tim (PH4T)
On Mon, 27 Sept 2021 at 10:03, Rob PE1CHL via 44Net <
44net@mailman.ampr.org>
wrote:
Yes it would be worthwile to research (with the IP address as
information)
what could be the reason behind this. Assuming it was not 44.0.0.1 but some amateur's IP, it could be some retaliation against that person and they may be able to
identify a
likely source and legal action may be possible.
Aside from that, I think there is too little headroom on this
connection
and it needs to be upgraded to 10Gbit or some teamed 1Gbit links when that is more practical. The background noise already takes up 650Mbit/s of the 1Gbit/s
available...
Rob
On 9/27/21 9:33 AM, Marius Petrescu via 44Net wrote:
Tnx. Chris for the update.
I'm still wondering what the goal of such an attack is...
On 27/09/2021 10:30, Chris Smith via 44Net wrote:
So, it was a TCP port 80 attack directed against one IP address.
I passed this on to my contacts at CAIDA who passed it onto the UCSD
NOC who have got the IP blocked.
The traffic has now returned to normal.
Chris - G1FEF
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
Do you have or can you get RTBH setup with UCSD? So from the 44net gateway box you can announce problematic /32s or larger with a black hole community e.g :666, which they then null route on their borders? Also that they then propagate to their upstreams.
This is a very common simple setup. If today the 44net space is static routed from UCSD to the gateway box then you could still do private BGP with them which only the blackhole prefixes are announced, leaving the static in place for the covering routes.
On Mon, 27 Sep 2021 at 22:12, Chris Smith via 44Net 44net@mailman.ampr.org wrote:
There has been a plan in place to upgrade the link to 10Gb/s for nearly a year, there were two attempts at upgrading which failed due to the new link not working, but UCSD have now identified the issue and the upgrade has been expedited and, all being well, it will be upgraded within the next week or two.
Of course that just means the next DDOS will be hitting the gateway much harder, but at least we will have some decent headroom under normal circumstances.
73, Chris - G1FEF
On 27 Sep 2021, at 09:20, Tim de Boer via 44Net 44net@mailman.ampr.org
wrote:
I agree with Rob! Normally you should start upgrading the connection if
the
average reaches 50% of it's capacity
-- Tim (PH4T)
On Mon, 27 Sept 2021 at 10:03, Rob PE1CHL via 44Net <
44net@mailman.ampr.org>
wrote:
Yes it would be worthwile to research (with the IP address as
information)
what could be the reason behind this. Assuming it was not 44.0.0.1 but some amateur's IP, it could be some retaliation against that person and they may be able to
identify a
likely source and legal action may be possible.
Aside from that, I think there is too little headroom on this connection and it needs to be upgraded to 10Gbit or some teamed 1Gbit links when that is more practical. The background noise already takes up 650Mbit/s of the 1Gbit/s
available...
Rob
On 9/27/21 9:33 AM, Marius Petrescu via 44Net wrote:
Tnx. Chris for the update.
I'm still wondering what the goal of such an attack is...
On 27/09/2021 10:30, Chris Smith via 44Net wrote:
So, it was a TCP port 80 attack directed against one IP address.
I passed this on to my contacts at CAIDA who passed it onto the UCSD
NOC who have got the IP blocked.
The traffic has now returned to normal.
Chris - G1FEF
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
Have to admit, I was a little confused to see anything under an attack for over 24 hours.. All of our upstreams support blackhole communities, and realtime mitigation. We’re constantly having them soak upwards of 80Gbits of DDoS at any time. Any single IP that ends up with > 20gbps of DDoS pointed at it, ends up with a /32 blackhole announced to all peers, and that traffic just goes away.
Why is UCSD’s upstream not washing the DDoS? (And why is UCSD not identifying the target and blackholing it upstream?)
Cheers,
DG
On Mon, 27 Sep 2021 at 4:17 pm, Borja Marcos via 44Net < 44net@mailman.ampr.org> wrote:
On 25 Sep 2021, at 10:40, Chris Smith via 44Net 44net@mailman.ampr.org
wrote:
FYI
The gateway machine at UCSD has been under a sustained DDOS attack now
for over 24 hours, so if anyone is seeing heavy packet loss through the gateway, that’s why. The 1Gb/s interface is max’d out. You can view the interface stats here:
Seems to be a DNS DDoS.
Can’t you filter upstream? Let me know if you need assistance.
Borja - EA2EKH
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
On 27 Sep 2021, at 08:20, Damien Gardner vk2tdg@gmail.com wrote:
Have to admit, I was a little confused to see anything under an attack for over 24 hours.. All of our upstreams support blackhole communities, and realtime mitigation. We’re constantly having them soak upwards of 80Gbits of DDoS at any time. Any single IP that ends up with > 20gbps of DDoS pointed at it, ends up with a /32 blackhole announced to all peers, and that traffic just goes away.
Why is UCSD’s upstream not washing the DDoS? (And why is UCSD not identifying the target and blackholing it upstream?)
We have a customer with a misconfigured router being used as a DDoS amplification. Sorry about it, I play whack-a-mole with them but sometimes I miss one or two.
On the bright side it gives me visibility over Hamnet's particular issue (and our customer is putting just around 600 Kbps) I can tell you that AMPRNet is suffering an ongoing DNS based DDoS attack since Sep 23rd , 10:00 UTC.
The destination addresses are two:
44.37.62.8 44.72.200.252
The traffic profile is a classic DNS DDoS. Large UDP packets with source port 53 and fragments (which on Netflow flows appear as source and destination port 0).
Mitigating this is particularly easy. Use an “internal” DNS resolver with a different IP address and make sure your upstream filters large UDP packets with popular DDoS source ports such as 53 (DNS), 123 (NTP), etc and fragments. No need to drop them all but you can cap them at, say, 100 Mbps so that they won’t flood your network port.
Avoid fragmentation wherever possible of course.
And setting up some Netflow monitoring is mandatory, so that you can quickly characterize hostile traffic.
As long as you have proper monitoring, you can ask your upstream to accept FkowSpec rules from you. FlowSpec is a specification to transport IP filtering configuerations over BGP so that you can install limited filters on your upstream router without their intervention.
Other sype of DDoS attacks, such as a TCP Syn flood can be a bit more complicated to mitigate and the best strategy is to use operating systems with enough resources and proper mechanisms (such as TCP syncookies) to withstand it.
Blackholing is a last resource measure.
Cheers,
Borja - EA2EKH
-
Borja Marcos -
Chris McCune -
Chris Smith -
Colin Bodor -
Damien Gardner -
David Ranch -
K7VE - John -
Marius Petrescu -
Matt - VK2FLY -
Nat Morris -
Rob PE1CHL -
Ruben ON3RVH -
Tim de Boer