Monday, January 14, 2013

PSAD - DoS'd my Linode and my G-mail account


Yes, I DoS'd myself.

How?

I was tuning my firewall so that PSAD wouldn't alert on traffic from my static IP.  This was caused by me using a half-baked firewall policy (ie, the firewall was allowing too much and not blocking what it should've).  So, while making the policy more secure, I ended up blocking some of DNS, which the firewall blocked via the clean-up rule (deny by default policy on all chains).  

I didn't double-check my work, and 24 hours later, I was checking the server via the admin console and saw that the disk I/O was extremely high when looking at the system graphs.  CPU utilization was also wayyy up.  

I initially couldn't figure out what was going on, until I used Webmin to access the server and found that it was taking forever for the server to resolve the domain address.  As well, there were like 30,000 e-mails in the Postfix e-mail queue.  I was basically spamming the hell out of Gmail and DShield.  I'd begun to wipe out all the Gmail notifications, but I soon realized that I wasn't making any headway, so I killed PSAD, cleared ALL the mail in the queue, then restarted PSAD.  It was still generating e-mails, though, so I turned off e-mail notifications, as well as syslog notifications.  I also killed my DShield log feed.  THEN I fixed DNS by just rolling back to a known good policy...then I told PSAD to not log on port 53/UDP (no real need to log that traffic anyways, unless it hits the catch-all rule, but that wouldn't happen now since I fixed DNS within the policy).

It took quite awhile for Gmail to finish processing the e-mails (the ones that I couldn't kill via Postfix).

I just now re-enabled syslogging and the DShield log feed, but may have to reach out to the SANS team to see if they can remove all DNS traffic that was logged by my static IPs.

I think I've everything fixed now.

No comments: