Researching and found an old flamefest spark

Reference:

http://mythtv.beirdo.ca/ircLog/channel/1/2008-07-14

Summary: At LQ.org, there was a discussion on the security forums on how vulnerable Linux was to attacks/malware. Someone didn't like what was being discussed because of typical Linux zealotry. What happened on LQ's forums spilled over into ##slackware on IRC. Dagmar, the instigator of a LOT of bad things that used to happen in ##slackware got perm banned by me. Later, documented in the link above, he is his typical self, not even attempting to objectively explain what the whole thing was about, pretty much slandering me about how flawed my thoughts are on the whole thing and is worrying that I'll propagate bad information.

Let me explain some things about myself. I'm an IT security engineer. I don't just mess with routers and I'm not some glorified network engineer. I'm a senior consultant. I not only consult, I'm able to find "needle-in-the-haystack"-type info using packet-level analysis. Most of what I do requires that I be a jack-of-all-trades in network engineering, but my specialty is security. I'm proficient in utilizing many industry-leading security tools, both freeware and commercial software. I work at a very large ISP/telecom within a large managed security services team. I am THE lead of a government security operations center. We manage well over 100 customers' security posture via firewalls, NIDS, HIDS, and IPS appliances, using ArcSight, an aggregation and correlation tool that is fast becoming the standard in security event monitoring.

Every day, we see machines being compromised...this is nothing new. The compromises span every mainstream OS. This includes Linux. Whether it is kernel level or application level is not the argument. The argument is that Linux is not as rock-solid as everyone makes it out to be. Sure, it has more safeguards than Windows-based systems, but it is still susceptible to application-level exploits. Whether this is a coder issue or PEBKAC/user/admin issue is besides the point.

People need to stop thinking that just because they are running Linux, they are safe. That is NOT the case. This is not paranoia speaking. It is from seeing such things happen on a daily basis during security event monitoring. Due to applications such as PHP-Nuke, it is becoming more difficult to secure back end applications. It is much harder to stop SQL injection than it is to stop SSH brute-forcing, for instance. This isn't the only issue, though. The issue is the perception that because Linux code is open and free, the code base is free of vulnerabilities. That is NOT the case. Also, many people think that a majority of the cracker focus is on Win32 because MS has a majority of the market share. That also is NOT the case. That is a big assumption. milw0rm and other such sites document many *nix-based vulnerabilities, along with Bugtraq at Securityfocus track all vulnerabilities. Sometimes, people justify Linux because its security model is better focused than Win32 systems. It is, but that does not mean that Linux is rock-solid. It has its own faults, whether it is the user, the admin, or the software developer (or even kernel developer).

Dagmar has a habit of blocking out people's opinions and sometimes beating people down with his own. Dagmar thinks he knows security more than anyone else when he's just a developer. I see attacks every day on all types of machines. Some of the attacks are successful. I doubt that Dagmar sees those. Dagmar need not worry about me "propagating" untruth, because what I say IS the truth. All you have to do to see the truth is to research and not be blind to other opinions.

Dagmar also stalked. After the IRC discussion, he began to frequent the LQ security forums and respond to every thread I posted to. He was hardly ever in those forums before then. I noticed this immediately (and also checked). I didn't mind this, but when it spilled back over into IRC, I tired of it and wanted it ended...it really had no place in ##slackware and I was fed up with his attitude about the whole thing. I don't suffer drama very well.

Now, Dagmar has been banned several times before for the lack of tact in the way he 'helped' people in ##slackware. He was walking a thin line to begin with. Those with operator status in ##slackware acknowledge that he is knowledgeable, but that is not grounds for him to be dismissed as an abusive ##slackware visitor. Sure enough, he did the same thing with a channel operator (me) and I banned him. I also discussed it with the other operators. The consensus was that he stay banned since his history of being banned was substantial.

That was why he got banned...not because his views went against my own, but because he started regressing back to his former self and became abusive. He did the same in the LQ.org forums, but I was able to filter his posts from my normal views. As an operator at Freenode.net, I can't and shouldn't filter any visitor from my views in ##slackware, so my only option was to ban him, and like I said before, he'd his own infamous nature that was going against him.

As a security consultant, I'm certainly not going to keep my thoughts quiet about what I think is a disservice to my favorite operating system. I certainly know more than someone who is not a security consultant about IT security...its what I get paid to do and its what I've been doing for years. It's the same as a person who has built his own car, vs. someone who works as a senior Mercedes mechanic.

As much as I can, I tell people that there is NO secure OS. It is only as secure as the admin makes it, and even if the admin puts 100% resources into hardening the box, it will never be 100% secure. The LQ security forums is itself proof that Linux systems get compromised more than most people think. 2-3 times a week, someone reports they've been compromised. There's even 4 threads on Linux-based vulnerabilities:

Kernel Vulns
Mozilla Firefox Vulns
The Problem with PHP Application Security
Failed SSH Login Attempts

I can post a ton of other links but why do this when there is Google?

No further issues with gOS so far

It has been a very good experience, so far.

In fact, I've also been leveraging Thunderbird...this is a first time for me (since Netscape Mail back in the 90s). It is very robust!

I've also aliased a few commands that I tend to use alot, mainly ssh commands that I use on remote hosts.

I've also found some decent background images that I've scaled (using Gimp) to 1024x600.

I've also been conducting my typical security audits (BASE and iptables and web server log perusing).

I've not used my Mac in like 3 weeks! I don't know if that's a good or bad thing (probably bad for the battery).

gOS v3.1 installed on Dell Mini 9

I took the plunge and installed gOS v3.1 Gadgets onto the Dell Mini 9.

The install went flawlessly.

The issues I have discovered so far:

1. Wireless would not work. I followed the instructions located at http://gosforums.org/viewtopic.php?f=21&t=48&p=203&hilit=broadcom#p223. I applied this fix (when I was using a cat5 connection) and it worked, so the wireless non-functionality is no longer an issue.

2. The Mini won't suspend when I close the lid. I can manually suspend, though. I'll hunt for a fix and apply it later.

I also just noticed that a swap partition was created and configured for use (automatically, when installing gOS). I already have a gig of physical RAM and I don't want to burn out my SSD card, so I'll disable it for now and consider a workaround if/when I need it.

Overall, this is a pretty solid distro and it is pretty cool to be able to use Google tools (this will save drive space and conserve the limited resources this machine has). The Gadgets can also be used offline, so I won't need to be connected to use them...now, that's cool.

Some screenshots:






EDIT:

Fixed the sound issue by following Step 4 of "Installing Ubuntu 8.10 on the Dell Inspiron Mini 9".

I turned off the swap partition by editing out the swap entry in /etc/fstab. I'll test to see how this impacts my install of gOS before removing the partition.

Power adapters and other news

Wouldn't it be nice if the laptop empire joined forces and standardized laptop power adapter cords? I've several laptop and each requires a dedicated cord. I forgot my cord for my Mini today...fortunately, it has a decent charge, but I now have to watch consumption of power. Good thing my Macbook is in my truck. May have to go get it!

Other news:

I'm still trying to script FW log parsing. I've pretty much nailed it for my BSD machine, but will need to edit what I have, as the script parses and adds IPs to a block list. Sometimes, I just want to parse and find the top 10 offenders. I also still need to do this on my Linux machines. I've a script that parses (for Linux) but it is very rudimentary. And, I still want to port the script to Perl or Python.

tcpdump, Dell Mini, and BASE

So, I'm wondering why tcpdump is missing from the default install of my Dell's Ubuntu...doesn't make sense. I was having issues with getting my wifi card associated with my WAP and wanted to see the packets leaving the wireless interface, so I tried to bring up tcpdump, but it wasn't available. I actually had to hook a cat5 cable to the Mini to get this package, just to troubelshoot. I noticed the same thing with Suse about a year ago.

Apparently, tcpdump was created on the permissive free software license, per Wikipedia. I don't know if this is actually GPL or a derivative of GPL. The manpage doesn't mention what license tcpdump falls under and I'm sometimes wary of Wikipedia, as I like to find the facts on my own to validate (or invalidate) internet claims.

I'll research this and post my findings here.

On another note, I found a very cool bag for my Dell Mini, at Dell's website. I'll try to post pics and a link soon (from my Macbook, as the Mini's keyboard slows me down a bit).

Lastly, I somehow broke access to my MySQL database, so now my snort sensors won't report to it. It's been down for about 2 weeks and I don't have the time to fix it. I'm going on vacation for my birthday and hope to have some personal (ie, QUIET) time to myself to fix this. I'll be visiting my parents for my birthday this weekend and will see about shelling in to fix it remotely.

Worked on...

Reconfigured mnwclient so that I can provide FW logs to MyNetworkWatchman (which is similar to Dshield).

I'd much rather get Dshield working on the Linode but for some reason, I been having difficulties using their supplied clients. I'll continue to work on it, as I had it working prior to the last Linode upgrade.

With that in mind, at some time I'm going to have to upgrade the Linode from v12.0 to v12.2.

Night...

What I've installed on the Mini so far...

So far, I've installed the following on my Dell Mini:

tcpdump (IMO, this is a mandatory package...should've been installed by default)
xchat
gdesklets (I'm going to remove this because desktop space is at a premium)

That's it, so far. Surprisingly, I find myself not needing much more than what apps are already installed. Then again, I've had the Mini for less than a week. :)

EDIT: also installed 'locate' but the install didn't include the updatedb.conf file...working on that now.

My Mini is here!

My Dell Mini has arrived. Went to pick it up at the local Fedex facility last night.

It came in a LITTLE box. Basically, it was just the netbook and the adapter that were in the box, with a few CDs and documents.

I took it out and plugged it up and the battery was almost fully charged already, but I charged it anyways.

The keypad is small and my big hands don't help much but the unit itself is pretty slick.

The tech specs are below:

32gb SSD drive
1.3mb vidcam
1gb RAM
Ubuntu Linux OS
Black in color

I opted for no bluetooth but now I'm thinking maybe I should've went that route (although it would've jacked up the price.

I now need a case. For now, I'm carrying it in my work laptop's bag.

My immediate problem is that it won't connect to any wireless networks, either at home or at work. I did a quick search and this appears to be a known issue. I'll have to delve more into this. When I attempt to have it automatically apply the wireless setting and check those settings, the settings are blank (no IP or DNS). Manually applying them doesn't help.

So, I'll be occupied the next few days... :)

Dell Mini 9 and system upgrades

I just ordered a Dell Mini 9.

It should be here next week. I purchased the 16GB SSD upgrade and webcam. It is also a Windows XP model. Later, I'll more than likely either install Linux on it (or OS X), although this will require a USB optical drive.

The plan is to also upgrade the SSD to 32GB (a $119 purchase at My Digital Discount).

More than likely, I'm going to sell my Macbook (and maybe use the money to get an iMac).

As for my dead motherboard, it looks like I'm looking at this. I've decided to go AMD Phenom. Why? Because, while I want to upgrade to quad core while I can, I don't want to go Core i7 because I'd have to spend butt-loads on the CPU and motherboard in addition to DDR3 RAM...too much upgrading. At this point, I don't even want to go Intel. While Intel has better bang for the buck regarding CPU power and robustness, I'm extremely loyal to AMD. There's nothing wrong with AMD's products if you don't benchmark (and I'm not one to hang his every decision on raw stats alone). I should have the parts within a month...then I can game again. This will leave me with an unused Pentium D 830 (dual core), though. That means I'll more than likely buy a cheaper motherboard just to utilize that CPU in one of my older systems...it is a nice CPU with good power.

EDIT: I upgraded my Mini order because they were offering a $50 discount on Presidents Day. It will have Ubuntu and it will also have a 32GB SSD. More than likely I'm going to send it back, though, since Asus has a better buy, sans the SSD, but also has BT.

About my mainboard and CPU upgrade. I went AMD. I bought a Phenom 940 with a Foxconn mainboard. It is up and running now. It is FAST and I've yet to see all 4 cores maxed out! I also bought another mainboard for the Pentium CPU. That's a project for another day, though.

Foxconn A79A-S AM2+/AM2 AMD 790FX ATX AMD Motherboard

AMD Phenom II X4 940 Deneb 3.0GHz Socket AM2+ 125W Quad-Core Black Edition Processor Model HDZ940XCGIBOX

System dead

Ref: http://slackfiles.blogspot.com/2007_02_18_archive.html

OK. Apparently when the Ultra PS died, it may have taken the motherboard with it. The machine has been blue-screening ever since the PS replacement, just not as frequently as before.

I also noticed that the HSF on the Northbridge isn't working. When searching Google for issues with the Northbridge, I found that this is happening to others and that it actually signifies a motherboard failure. The Northbridge gets HOT and I'm assuming that over time, the heat has killed the motherboard.

This is the time to upgrade, I guess. I'll more than likely buy a new motherboard/CPU combo and it'll more than likely be a quad-core CPU that I'll be getting (Phenom):

(Intel) http://www.newegg.com/Product/ComboDealDetails.aspx?ItemList=Combo.152153

or

(AMD) http://www.newegg.com/Product/ComboDealDetails.aspx?ItemList=Combo.150323

EDIT - I ended up purchasing a

Foxconn A79A-S AM2+/AM2 AMD 790FX ATX AMD Motherboard

and a

Retail AMD Phenom II X4 940 Deneb 3.0GHz 4 x 512KB L2 Cache CPU

and basically rebuilt this system...it is SMOKING. I was gaming and installing game patches for another game AND XP updates, and only two cores were being used while two were idle!! I put 3GB of RAM into the system (I almost put 4GB but then realized that XP 32-bit would only see 3GB of it...time for a 64-bit Windows OS install, I think). I really should install some high-grade RAM, though, as I think I only have 667mhz installed.

I also bought a

BIOSTAR TForce TP43D2A7 LGA 775 Intel P43 ATX Intel Motherboard

for to replace the motherboard in the Pentium-D system...I'll use this motherboard and CPU to rebuild my old Sempron 3000+ system (I'm starting to stack up 'older' motherboards and CPUs).

Recent Happenings

I've added 2 more GB to my gaming system (the Ultra that I bought from TigerDirect 2 yrs ago). The system now has a total of 4GB. It runs Windows XP, so its only seeing 3.25GB of the installed 4GB, as I'm running XP 32-bit. I need to either go to Vista or install XP 64-bit. I may just wait until the new version of Windows desktop is released.

I've also begun to (again) monitor my internal LAN traffic at home. I broke down my security systems this past Fall because I bought a house and had to move. I wanted to reinstall my external IDS but didn't want to run an ethernet tap yet, so I used the same machine to just log internal traffic for now. Later, I'll have this machine start a separate Snort process that will monitor external traffic via the tap.

I need to upgrade my Slackware boxes to v12.2. The current boxes are running v12.0, including my Linode (http://wigglit.ath.cx). I'll try to do this soon, although this will require upgrading from v12.0 to v12.1, then v12.1 to v12.2.

I'll also begin to attempt to convert all (which isn't many) my BASH scripts that manipulate text (FW log parsing scripts and such) to Perl. This is mainly as an exercise to force myself to learn Perl. I could also do Python. This is an educational 2009 goal for me. :)

Distributed SSH Brute Force Attempts, part 3

I wanted to be able to harvest the log data that the brute force attempts are generating, so I've decided to not move the SSH listening port. I'm also logging each event within my firewall logs. The particular machine I'm seeing the attacks on is a FreeBSD box (I should've mentioned that earlier) and I'm using PF as the firewall of choice.

I'd like to show you my listing of blocked IPs. I've been actively gathering them since approximately 11/17/2008. I was unhappy that I hadn't noticed the distributed attempts until November and wanted more trending data, so I reached into my SSH logs and parsed the files present with a very dirty script that added each unique IP to a PF table that is designed to block such activity. The script is below:

[root@delly ~]# cat IPscript
#/bin/bash

cd /var/log
bunzip2 pflog.*

cat /var/log/auth.log | grep sshd | grep -i 'invalid user' > /tmp/auth_IP_list_1
bzcat /var/log/auth.log.[01234567].bz2 | grep sshd| grep -i 'invalid user' >> /tmp/auth_IP_list_1
sed '/Failed keyboard-interactive/d' /tmp/auth_IP_list_1 > /tmp/auth_IP_list_2
awk '{print $10}' /tmp/auth_IP_list_2 > /tmp/auth_IP_list_3

tcpdump -nettttr /var/log/pflog > /tmp/fw_IP_list_1
tcpdump -nettttr /var/log/pflog.0 >> /tmp/fw_IP_list_1
awk '{print $9}' /tmp/fw_IP_list_1 > /tmp/fw_IP_list_2
nawk -F. '{print $1, $2, $3, $4}' /tmp/fw_IP_list_2 > /tmp/fw_IP_list_3
sed 's/ /./g' /tmp/fw_IP_list_3 > /tmp/fw_IP_list_4

cat /tmp/fw_IP_list_4 >> /tmp/auth_IP_list_4
cat /tmp/auth_IP_list_4 | sort -rn | uniq > /tmp/auth_IP_list_5
cat /tmp/auth_IP_list_5 | grep -v '64.62.231.220' > /tmp/auth_IP_list_6
cat /tmp/auth_IP_list_6 | grep -v '66.160.141.30' > /tmp/auth_IP_list_7
cat /tmp/auth_IP_list_7 | grep -v '10.150.1' > /tmp/auth_IP_list_8
cat /tmp/auth_IP_list_8
cat /tmp/auth_IP_list_8 | wc -l

pfctl -t bruteforce -T add -f /tmp/auth_IP_list_8

rm -rf /tmp/fw_IP_list_* /tmp/auth_IP_list_*

I apologize for the lack of comments in the script...as I said, it was an extremely dirty hack that required me to learn a tad of sed and awk. The script outputs the following file: http://wigglit.ath.cx/txt/bruteforce_IPtable

My logs don't actually go back that far. The FW logs go back to 14 Dec. The SSH service logs go back to 26 Nov. I probably have captured IPs reaching back to the beginning of November, though.

My FW policy prevents anything from entering the network unless specifically allowed (default deny policy). The reason I wanted to track the IPs was because my Denyhosts configuration wasn't catching most of these and it was working fine before this new trend occurred. I'm a security consultant and a researcher at heart, so I thought that tracking this would be cool. My home router has ports 22, 443, and 3306 exposed to the wild for the FreeBSD box. The FW only allows certain IPs in on those ports, though, so nothing will get in. The activity that doesn't get immediately blocked is blocked by the bruteforce_IPtable script that I run daily. The script is flawless and I will begin to have it run hourly via a cronjob. I'll also have it send an updated list to my website daily.

The IPs within my block table number 565. The script parsed 115 from the logs tonight, but only added 1 IP. The norm is usually 2-3 daily. The rest of the IPs are from logs over the last month and a half.

Oh yeah, I've another script does a daily copy of the IPs that are added to the table, so I can at least quickly determine (using 'diff') what was added on a certain day. I can create a script that will show me what was added daily for a given timespan (day/week/month).

Distributed SSH Brute Force Attempts, part 2

OK, so I'm still curious about the distributed SSH brute force attempts I've been seeing.

I wanted to check all my logs and not just the most recent logfile. Looking at my /var/log directory:

-su-2.05b# ls | grep auth
auth.log
auth.log.0.bz2
auth.log.1.bz2
auth.log.2.bz2
auth.log.3.bz2
auth.log.4.bz2
auth.log.5.bz2
auth.log.6.bz2
auth.log.7.bz2

I tailed the auth.log file and grabbed a random IP:

Nov 24 21:53:59 delly sshd[75490]: Invalid user bryan from 170.56.255.20
Nov 24 21:54:00 delly sshd[75490]: error: PAM: authentication error for illegal user bryan from 170.56.255.20
Nov 24 21:54:00 delly sshd[75490]: Failed keyboard-interactive/pam for invalid user bryan from 170.56.255.20 port 43229 ssh2

I then wanted to check all the auth.log.* files, but was curious as to how I could check compressed files. I found that there's a command called bzgrep that allows one to grep compressed files, so I used the following command and came up with quite a few hits for the referenced IP over seven (7) log files:

-su-2.05b# bzgrep '170.56.255.20' auth.log.*

The results show 2-3 instances of log entries per login attempt, so I wanted to isolate each instance without having to use arcane sed and sort commands, so I used the following:

-su-2.05b# bzgrep '170.56.255.20' auth.log.* | grep 'Invalid user' | wc -l
19

So, this particular IP generated 19 log entries between 22 and 24 Nov.

That's not particularly good. I'd love to create a script that would break down all of these IPs' unique login attempts (and possibly block them). Sounds like a project, no? :)

I've a bit of time, so I did the following (non-scripted). I cat'd the auth.log file and collected a screens-worth of data:

-su-2.05b# cat auth.log | less
Nov 24 15:00:00 delly newsyslog[74001]: logfile turned over due to size>100K
Nov 24 15:00:49 delly sshd[74014]: Invalid user brand from 218.80.215.198
Nov 24 15:00:50 delly sshd[74014]: error: PAM: authentication error for illegal user brand from 218.80.215.198
Nov 24 15:00:50 delly sshd[74014]: Failed keyboard-interactive/pam for invalid user brand from 218.80.215.198 port 19051 ssh2
Nov 24 15:02:25 delly sshd[74017]: Invalid user brandee from 83.19.224.11
Nov 24 15:02:26 delly sshd[74017]: error: PAM: authentication error for illegal user brandee from dum11.internetdsl.tpnet.pl
Nov 24 15:02:26 delly sshd[74017]: Failed keyboard-interactive/pam for invalid user brandee from 83.19.224.11 port 50163 ssh2
Nov 24 15:03:48 delly sshd[74020]: Invalid user brandee from 194.224.118.61
Nov 24 15:03:48 delly sshd[74020]: error: PAM: authentication error for illegal user brandee from 194.224.118.61
Nov 24 15:03:48 delly sshd[74020]: Failed keyboard-interactive/pam for invalid user brandee from 194.224.118.61 port 6345 ssh2
Nov 24 15:05:21 delly sshd[74026]: Invalid user brandee from 90.176.233.222
Nov 24 15:05:22 delly sshd[74026]: error: PAM: authentication error for illegal user brandee from 222.233.broadband9.iol.cz
Nov 24 15:05:22 delly sshd[74026]: Failed keyboard-interactive/pam for invalid user brandee from 90.176.233.222 port 46108 ssh2
Nov 24 15:06:42 delly sshd[74029]: Invalid user branden from 125.77.106.246
Nov 24 15:06:42 delly sshd[74029]: error: PAM: authentication error for illegal user branden from 125.77.106.246
Nov 24 15:06:42 delly sshd[74029]: Failed keyboard-interactive/pam for invalid user branden from 125.77.106.246 port 46495 ssh2
Nov 24 15:10:59 delly sshd[74035]: Invalid user brandi from 122.224.128.222
Nov 24 15:10:59 delly sshd[74035]: error: PAM: authentication error for illegal user brandi from 122.224.128.222
Nov 24 15:10:59 delly sshd[74035]: Failed keyboard-interactive/pam for invalid user brandi from 122.224.128.222 port 42253 ssh2
Nov 24 15:12:27 delly sshd[74051]: Invalid user brandi from 59.125.200.51
Nov 24 15:12:28 delly sshd[74051]: error: PAM: authentication error for illegal user brandi from 3w.upcc.com.tw
Nov 24 15:12:28 delly sshd[74051]: Failed keyboard-interactive/pam for invalid user brandi from 59.125.200.51 port 14046 ssh2
Nov 24 15:15:23 delly sshd[74057]: Invalid user brandice from 62.112.222.88
Nov 24 15:15:24 delly sshd[74057]: error: PAM: authentication error for illegal user brandice from 3e70de58.adsl.enternet.hu
Nov 24 15:15:24 delly sshd[74057]: Failed keyboard-interactive/pam for invalid user brandice from 62.112.222.88 port 42127 ssh2
Nov 24 15:16:49 delly sshd[74060]: Invalid user brandice from 218.80.215.198
Nov 24 15:16:50 delly sshd[74060]: error: PAM: authentication error for illegal user brandice from 218.80.215.198
Nov 24 15:16:50 delly sshd[74060]: Failed keyboard-interactive/pam for invalid user brandice from 218.80.215.198 port 57929 ssh2
Nov 24 15:18:11 delly sshd[74063]: Invalid user brandice from 65.203.231.41
Nov 24 15:18:11 delly sshd[74063]: error: PAM: authentication error for illegal user brandice from 65.203.231.41
Nov 24 15:18:11 delly sshd[74063]: Failed keyboard-interactive/pam for invalid user brandice from 65.203.231.41 port 38395 ssh2
Nov 24 15:19:43 delly sshd[74066]: Invalid user brandie from 123.14.10.64
Nov 24 15:19:44 delly sshd[74066]: error: PAM: authentication error for illegal user brandie from 123.14.10.64
Nov 24 15:19:44 delly sshd[74066]: Failed keyboard-interactive/pam for invalid user brandie from 123.14.10.64 port 4925 ssh2
Nov 24 15:21:07 delly sshd[74072]: Invalid user brandie from 200.170.141.134
Nov 24 15:21:07 delly sshd[74072]: error: PAM: authentication error for illegal user brandie from 200-170-141-134.static.ctbctelecom.com.br
Nov 24 15:21:07 delly sshd[74072]: Failed keyboard-interactive/pam for invalid user brandie from 200.170.141.134 port 39979 ssh2
Nov 24 15:22:44 delly sshd[74088]: Invalid user brandie from 80.51.31.84
Nov 24 15:22:44 delly sshd[74088]: error: PAM: authentication error for illegal user brandie from 80.51.31.84
Nov 24 15:22:44 delly sshd[74088]: Failed keyboard-interactive/pam for invalid user brandie from 80.51.31.84 port 39453 ssh2
Nov 24 15:24:02 delly sshd[74091]: Invalid user brandon from 200.157.176.13
Nov 24 15:24:03 delly sshd[74091]: error: PAM: authentication error for illegal user brandon from 200.157.176.13
Nov 24 15:24:03 delly sshd[74091]: Failed keyboard-interactive/pam for invalid user brandon from 200.157.176.13 port 54638 ssh2

I then checked every IP for unique log entries within all of my ssh logs:

-su-2.05b# bzgrep '218.80.215.198' auth.log.* | grep 'Invalid user' | wc -l
6

-su-2.05b# bzgrep '218.80.215.198' auth.log.* | grep 'Invalid user'
auth.log.0.bz2:Nov 24 10:55:25 delly sshd[73084]: Invalid user bjorn from 218.80.215.198
auth.log.1.bz2:Nov 23 23:22:28 delly sshd[70023]: Invalid user bahari from 218.80.215.198
auth.log.3.bz2:Nov 23 11:18:54 delly sshd[66908]: Invalid user archibald from 218.80.215.198
auth.log.4.bz2:Nov 23 04:11:52 delly sshd[65051]: Invalid user amy from 218.80.215.198
auth.log.5.bz2:Nov 22 23:20:53 delly sshd[63465]: Invalid user alize from 218.80.215.198
auth.log.7.bz2:Nov 22 01:07:10 delly sshd[57652]: Invalid user claire from 218.80.215.198

-su-2.05b# bzgrep '83.19.224.11' auth.log.* | grep 'Invalid user' | wc -l
8
-su-2.05b# bzgrep '194.224.118.61' auth.log.* | grep 'Invalid user' | wc -l
11
-su-2.05b# bzgrep '90.176.233.222' auth.log.* | grep 'Invalid user' | wc -l
0
-su-2.05b# bzgrep '125.77.106.246' auth.log.* | grep 'Invalid user' | wc -l
9
-su-2.05b# bzgrep '122.224.128.222' auth.log.* | grep 'Invalid user' | wc -l
8
-su-2.05b# bzgrep '59.125.200.51' auth.log.* | grep 'Invalid user' | wc -l
5
-su-2.05b# bzgrep '62.112.222.88' auth.log.* | grep 'Invalid user' | wc -l
10
-su-2.05b# bzgrep '218.80.215.198' auth.log.* | grep 'Invalid user' | wc -l
6
-su-2.05b# bzgrep '65.203.231.41' auth.log.* | grep 'Invalid user' | wc -l
15

-su-2.05b# bzgrep '65.203.231.41' auth.log.* | grep 'Invalid user'
auth.log.0.bz2:Nov 24 14:56:28 delly sshd[73982]: Invalid user bran from 65.203.231.41
auth.log.1.bz2:Nov 23 22:32:01 delly sshd[69793]: Invalid user azra from 65.203.231.41
auth.log.1.bz2:Nov 24 02:04:43 delly sshd[70680]: Invalid user bartholemew from 65.203.231.41
auth.log.1.bz2:Nov 24 04:07:45 delly sshd[71474]: Invalid user beck from 65.203.231.41
auth.log.2.bz2:Nov 23 19:02:03 delly sshd[68866]: Invalid user aurora from 65.203.231.41
auth.log.2.bz2:Nov 23 20:16:18 delly sshd[69213]: Invalid user avi from 65.203.231.41
auth.log.3.bz2:Nov 23 10:08:33 delly sshd[66592]: Invalid user april from 65.203.231.41
auth.log.3.bz2:Nov 23 10:24:43 delly sshd[66657]: Invalid user aquila from 65.203.231.41
auth.log.3.bz2:Nov 23 11:22:12 delly sshd[66933]: Invalid user archie from 65.203.231.41
auth.log.4.bz2:Nov 23 04:22:52 delly sshd[65094]: Invalid user anahid from 65.203.231.41
auth.log.4.bz2:Nov 23 05:32:46 delly sshd[65407]: Invalid user andra from 65.203.231.41
auth.log.5.bz2:Nov 22 23:00:09 delly sshd[63393]: Invalid user alisha from 65.203.231.41
auth.log.6.bz2:Nov 22 12:41:17 delly sshd[60534]: Invalid user abraham from 65.203.231.41
auth.log.6.bz2:Nov 22 16:14:07 delly sshd[61564]: Invalid user africa from 65.203.231.41
auth.log.7.bz2:Nov 22 11:48:33 delly sshd[60289]: Invalid user aaralyn from 65.203.231.41

-su-2.05b# bzgrep '123.14.10.64' auth.log.* | grep 'Invalid user' | wc -l
19
-su-2.05b# bzgrep '200.170.141.134' auth.log.* | grep 'Invalid user' | wc -l
6
-su-2.05b# bzgrep '80.51.31.84' auth.log.* | grep 'Invalid user' | wc -l
3
-su-2.05b# bzgrep '200.157.176.13' auth.log.* | grep 'Invalid user' | wc -l
4

So, someone appears to have a pool of compromised machines and is using each one in a scaled SSH brute force attack, based on the referenced user accounts being bruteforced. I'm seeing more of this than standard, blatant SSH BF attempts. I'll be checking Denyhosts' website to see if they've a resolution on how to track and ban such activity.

Distributed SSH Brute Force Attempts?

I'd read not long ago on the ISC Diary that someone has noticed that a there's a newly discovered way to avoid automated tools such as Denyhosts and Fail2ban. It appears that the attacks are now distributed across an IP pool of compromised machines. Maybe botnet masters are leveraging their botnets to attempt to bruteforce login attempts without risking the attacking hosts.

I think I'm seeing this in my home firewall logs:

Nov 23 15:23:01 delly sshd[67946]: error: PAM: authentication error for illegal user artois from 1-1-4-27a.vhe.sth.bostream.se
Nov 23 15:23:01 delly sshd[67946]: Failed keyboard-interactive/pam for invalid user artois from 82.182.188.187 port 35763 ssh2
Nov 23 15:24:18 delly sshd[67949]: Invalid user arty from 58.26.48.162
Nov 23 15:24:18 delly sshd[67949]: error: PAM: authentication error for illegal user arty from 58.26.48.162
Nov 23 15:24:18 delly sshd[67949]: Failed keyboard-interactive/pam for invalid user arty from 58.26.48.162 port 5785 ssh2
Nov 23 15:25:17 delly sshd[67955]: Invalid user arty from 200.170.141.134
Nov 23 15:25:17 delly sshd[67955]: error: PAM: authentication error for illegal user arty from 200-170-141-134.static.ctbctelecom.com.br
Nov 23 15:25:17 delly sshd[67955]: Failed keyboard-interactive/pam for invalid user arty from 200.170.141.134 port 57360 ssh2
Nov 23 15:26:31 delly sshd[67958]: Invalid user arty from 219.76.222.27
Nov 23 15:26:31 delly sshd[67958]: error: PAM: authentication error for illegal user arty from n219076222027.netvigator.com
Nov 23 15:26:31 delly sshd[67958]: Failed keyboard-interactive/pam for invalid user arty from 219.76.222.27 port 47176 ssh2
Nov 23 15:28:48 delly sshd[67963]: Invalid user arva from 58.196.4.2
Nov 23 15:28:49 delly sshd[67963]: error: PAM: authentication error for illegal user arva from 58.196.4.2
Nov 23 15:28:49 delly sshd[67963]: Failed keyboard-interactive/pam for invalid user arva from 58.196.4.2 port 50637 ssh2
Nov 23 15:33:27 delly sshd[67982]: Invalid user arvid from 125.77.106.246
Nov 23 15:33:27 delly sshd[67982]: error: PAM: authentication error for illegal user arvid from 125.77.106.246
Nov 23 15:33:27 delly sshd[67982]: Failed keyboard-interactive/pam for invalid user arvid from 125.77.106.246 port 51673 ssh2
Nov 23 15:34:40 delly sshd[67985]: Invalid user arvin from 85.39.252.226
Nov 23 15:34:40 delly sshd[67985]: error: PAM: authentication error for illegal user arvin from host226-252-static.39-85-b.business.telecomitalia.it
Nov 23 15:34:40 delly sshd[67985]: Failed keyboard-interactive/pam for invalid user arvin from 85.39.252.226 port 43706 ssh2
Nov 23 15:35:54 delly sshd[67991]: Invalid user arvin from 217.126.90.161
Nov 23 15:35:55 delly sshd[67991]: error: PAM: authentication error for illegal user arvin from 161.red-217-126-90.staticip.rima-tde.net
Nov 23 15:35:55 delly sshd[67991]: Failed keyboard-interactive/pam for invalid user arvin from 217.126.90.161 port 36755 ssh2
Nov 23 15:37:11 delly sshd[67994]: Invalid user arvin from 200.232.181.40
Nov 23 15:37:11 delly sshd[67994]: error: PAM: authentication error for illegal user arvin from 200-232-181-40.dsl.telesp.net.br
Nov 23 15:37:11 delly sshd[67994]: Failed keyboard-interactive/pam for invalid user arvin from 200.232.181.40 port 56318 ssh2
Nov 23 15:39:16 delly sshd[67997]: Invalid user arwan from 200.248.82.130
Nov 23 15:39:17 delly sshd[67997]: error: PAM: authentication error for illegal user arwan from 200.248.82.130
Nov 23 15:39:17 delly sshd[67997]: Failed keyboard-interactive/pam for invalid user arwan from 200.248.82.130 port 53388 ssh2
Nov 23 15:40:24 delly sshd[68003]: Invalid user arwan from 217.126.90.161
Nov 23 15:40:25 delly sshd[68003]: error: PAM: authentication error for illegal user arwan from 161.red-217-126-90.staticip.rima-tde.net
Nov 23 15:40:25 delly sshd[68003]: Failed keyboard-interactive/pam for invalid user arwan from 217.126.90.161 port 43871 ssh2
Nov 23 15:41:33 delly sshd[68006]: Invalid user arwen from 200.209.6.130
Nov 23 15:41:34 delly sshd[68006]: error: PAM: authentication error for illegal user arwen from 200.209.6.130
Nov 23 15:41:34 delly sshd[68006]: Failed keyboard-interactive/pam for invalid user arwen from 200.209.6.130 port 14808 ssh2
Nov 23 15:42:48 delly sshd[68016]: Invalid user arwen from 123.14.10.64
Nov 23 15:42:49 delly sshd[68016]: error: PAM: authentication error for illegal user arwen from 123.14.10.64
Nov 23 15:42:49 delly sshd[68016]: Failed keyboard-interactive/pam for invalid user arwen from 123.14.10.64 port 7600 ssh2
Nov 23 15:43:50 delly sshd[68020]: reverse mapping checking getaddrinfo for techregister.worcesteracademy.org [68.112.227.30] failed - POSSIBLE BREAK-IN ATTEMPT!
Nov 23 15:43:50 delly sshd[68020]: Invalid user arwen from 68.112.227.30
Nov 23 15:43:50 delly sshd[68020]: error: PAM: authentication error for illegal user arwen from 68.112.227.30
Nov 23 15:43:50 delly sshd[68020]: Failed keyboard-interactive/pam for invalid user arwen from 68.112.227.30 port 38273 ssh2
Nov 23 15:45:03 delly sshd[68039]: Invalid user arya from 196.28.50.162
Nov 23 15:45:03 delly sshd[68039]: error: PAM: authentication error for illegal user arya from www.cfse.gov.pr
Nov 23 15:45:03 delly sshd[68039]: Failed keyboard-interactive/pam for invalid user arya from 196.28.50.162 port 55647 ssh2
Nov 23 15:46:10 delly sshd[68042]: Invalid user arya from 81.12.221.74
Nov 23 15:46:10 delly sshd[68042]: error: PAM: authentication error for illegal user arya from em.asiban.ro
Nov 23 15:46:10 delly sshd[68042]: Failed keyboard-interactive/pam for invalid user arya from 81.12.221.74 port 16653 ssh2
Nov 23 15:47:23 delly sshd[68050]: Invalid user arya from 190.34.148.178
Nov 23 15:47:23 delly sshd[68050]: error: PAM: authentication error for illegal user arya from 190.34.148.178
Nov 23 15:47:23 delly sshd[68050]: Failed keyboard-interactive/pam for invalid user arya from 190.34.148.178 port 58738 ssh2
Nov 23 15:48:29 delly sshd[68053]: reverse mapping checking getaddrinfo for britannic-iss-medidean-working.e1-4-0-0-57.0.ar2.lon3.gblx.net [64.213.54.106] failed - POSSIBLE BREAK-IN ATTEMPT!
Nov 23 15:48:29 delly sshd[68053]: Invalid user asa from 64.213.54.106
Nov 23 15:48:29 delly sshd[68053]: error: PAM: authentication error for illegal user asa from 64.213.54.106
Nov 23 15:48:29 delly sshd[68053]: Failed keyboard-interactive/pam for invalid user asa from 64.213.54.106 port 42991 ssh2
Nov 23 15:49:46 delly sshd[68056]: Invalid user asa from 91.135.200.86
Nov 23 15:49:47 delly sshd[68056]: error: PAM: authentication error for illegal user asa from 91.135.200.86
Nov 23 15:49:47 delly sshd[68056]: Failed keyboard-interactive/pam for invalid user asa from 91.135.200.86 port 10262 ssh2
Nov 23 15:50:52 delly sshd[68062]: Invalid user asa from 200.20.187.222
Nov 23 15:50:53 delly sshd[68062]: error: PAM: authentication error for illegal user asa from 200.20.187.222
Nov 23 15:50:53 delly sshd[68062]: Failed keyboard-interactive/pam for invalid user asa from 200.20.187.222 port 52959 ssh2

Don't focus on the attacking IPs, but look at the referenced users. There are now tools that look like they're scaling attacks on a listing of common logins (or maybe even dictionary attacks) so that there's less risk of detection. There are current tools that look for attacks in a thresholded manner (example: 4 attacks in 5 sec warrants a block of that attacking IP). This new method of attack will not trigger the thresholding blocks.

More than ever, SSH key-based authentication should be used. This will prevent a successful login when under attack via brute forcing methods.

I can already see attack detection tools being adjusted to focus on tracking user accounts being bruteforced and banning all IPs that try to access user accounts based on time (example: 4 attacks on account asa in 5 sec will warrant a ban of all subsequent IPs for the next day or so...and not block if the IP is listed within a whitelist).

Your thoughts?

Asus ships software cracker on recovery DVD

Asus ships software cracker on recovery DVD:

Asus is accidentally shipping software crackers and confidential documents on the recovery DVDs that come with its laptops.

The startling discovery was made by a PC Pro reader whose antivirus software was triggered by a key cracker for the WinRAR compression software, which was located on the recovery DVD for his Asus laptop.

Script that parses FW logs?

What I'm trying to do now is create a script that will parse FW logs daily and break down how many entries each IP generated.

I want to do this first as a quick command, then leverage BASH scripting to automate this.

What I used in http://slackfiles.blogspot.com/2007/12/modsecurity-again.html apparently isn't working when trying to parse the firewall logs. I don't get it, but then again, I'm tired. I'll try again tomorrow.

The new logs look like the below (showing in /var/log/syslog instead of /var/log/messages of my old setup):

Jul 8 22:55:19 starchild kernel: BLOCKED: IN=eth0 OUT= MAC=fe:fd:40:3e:e7:dc:00:0c:db:fc:8b:59:08:00 SRC=64.47.32.59 DST=64.62.231.220 LEN=48 TOS=0x00 PREC=0x00 TTL=249 ID=39747 DF PROTO=TCP SPT=3405 DPT=445 WINDOW=16384 RES=0x00 SYN URGP=0

I may need to leverage Awk (I was close to doing this before I upgraded the server from v9.0 to v12.0).

Stay tuned!

Linode has been upgraded to Slackware v12.0

I decided to take the plunge and upgrade my Linode server from v9.0 to v12.0.

Things are semi-normal, yet I still have some things to do, such as reinstall denyhosts, a new iptables script, mnwclient, and dshieldclient.

So far, I've reinstalled the eggdrop bot (slackboy), using the latest version of the software. I've also reinstalled Apache, using v2.0.63. Snort is running on the system (installed using my v12.0 slackpack), and also Webmin (which eases my pain somewhat, especially when administrating from work...they frown upon using SSH outbound).

This verion of Slackware is barebones and I've had to manually install the following packages so far:

-rw-r--r-- 1 root root 122953 2006-11-01 02:50 checkinstall-1.6.1-i386-1.tgz
-rw-r--r-- 1 root root 551952 2006-05-27 16:24 cyrus-sasl-2.1.22-i486-1.tgz
-rw-r--r-- 1 root root 744871 2007-05-09 14:46 fetchmail-6.3.8-i486-2.tgz
-rw-r--r-- 1 root root 2268306 2006-03-26 19:04 groff-1.19.2-i486-1.tgz
-rw-r--r-- 1 root root 2088735 2007-07-01 19:12 httpd-2.2.4-i486-6.tgz
-rw-r--r-- 1 root root 1898552 2008-02-14 17:37 httpd-2.2.8-i486-1_slack12.0.tgz
-rw-r--r-- 1 root root 313616 2007-06-02 18:52 lsof-4.78-i486-1.tgz
-rw-r--r-- 1 root root 222760 2006-02-10 20:03 man-1.6c-i486-2.tgz
-rw-r--r-- 1 root root 1949092 2007-06-13 13:35 man-pages-2.55-noarch-1.tgz
-rw-r--r-- 1 root root 135893 2004-02-18 05:59 metamail-2.7-i486-2.tgz
-rw-r--r-- 1 root root 16756453 2007-03-24 00:52 mysql-5.0.37-i486-1.tgz
-rw-r--r-- 1 root root 16942549 2007-12-14 17:01 mysql-5.0.51-i486-1_slack12.0.tgz
-rw-r--r-- 1 root root 1483847 2007-05-07 01:15 ntp-4.2.4p0-i486-1.tgz
-rw-r--r-- 1 root root 3440059 2007-06-13 13:40 openssl-0.9.8e-i486-3.tgz
-rw-r--r-- 1 root root 836941 2007-06-13 13:40 openssl-solibs-0.9.8e-i486-3.tgz
-rw-r--r-- 1 root root 143713 2006-09-19 00:11 procmail-3.22-i486-2.tgz
-rw-r--r-- 1 root root 2450931 2003-10-29 01:08 rpm-4.2.1-i486-3.tgz
-rw-r--r-- 1 root root 4603 2007-01-04 22:28 rpm2tgz-1.0-i486-1.tgz
-rw-r--r-- 1 root root 212764 2007-02-10 14:52 rsync-2.6.9-i486-1.tgz
-rw-r--r-- 1 root root 1391679 2007-06-10 01:16 sendmail-8.14.1-i486-1.tgz
-rw-r--r-- 1 root root 276485 2007-06-10 01:16 sendmail-cf-8.14.1-noarch-1.tgz
-rw-r--r-- 1 root root 30270 2006-04-19 00:56 slocate-3.1-i486-1.tgz
-rw-r--r-- 1 root root 763394 2008-06-15 15:30 snort-2.6.1.5-i386-1.tgz
-rw-r--r-- 1 root root 137062 2006-02-06 14:00 sudo-1.6.8p12-i486-1.tgz
-rw-r--r-- 1 root root 1793130 2007-06-02 19:32 tcl-8.4.15-i486-1.tgz
-rw-r--r-- 1 root root 66773 2007-04-30 00:35 telnet-0.17-i486-1.tgz
-rw-r--r-- 1 root root 8614263 2007-06-20 16:34 vim-7.1.012-i486-1.tgz
-rw-r--r-- 1 root root 1882278 2007-06-20 16:50 vim-gvim-7.1.012-i486-1.tgz
-rw-r--r-- 1 root root 13573880 2008-05-26 00:19 webmin-1.420.tar.gz

Stability-wise, the server appears to be running somewhat more efficiently than before (for instance, the CPU and memory utilizations, along with IO, appear to be less 'spiky' when looking at chart readouts. When I shut down the server to upgrade (which is the way its done with Linode.com), I'd had 440+ days of uptime, so I can't argue about stability...it doesn't get much better than that!

Sooner or later, I'm going to try backing up my latest install, then trying to upgrade to v12.1 (just because I can).

wigglit.ath.cx being bombarded with scans of port 1028/UDP

The culprit?

24.64.0.0/13, or 24.64.0.0 - 24.71.255.255, which resolves to SHAWCABLE.NET

There are at least 311 hosts within that range that have tried to connect to UDP port 1028 in the last few days. This isn't really a broad scan but the pepperings of hosts every day for the last few days, each one being unique hosts that have never been logged makes it hard to establish a pattern so that I can block remote hosts that continue to scan for this port, so I've opted to initiate a broad block and keep the block in place for maybe 30 days.

I'll monitor this activity and maybe alert the ISC diary if the scans continue.

If you're caught up in this ban, let me know and I'll see about allowing traffic to specific hosts.

Metaspolit hijacked?

Monday morning, Metasploit.com was temporarily hijacked using an attack on the local area network of Metasploit's hosting provider.

More info here

Slamd64

I've installed Slamd64 on my new AMD system. Initially, I had SATA issues that have mysteriously disappeared (no idea why, other than maybe swapping out that SATA cables helped). I installed using Disk 1 only (I didn't download anything else), which has the core system components. I then wanted to boot X, which required me downloading the X and KDE software. In fact, I went ahead and copied a whole mirror site and will continue to rsync the site against my local copy, using the updated local mirror as a upgrade repository.

I had issues getting my mouse recognized. I've a Logitech MX1000 wireless mouse, which is connected to a KVM. Slamd64 detects it as a PS/2 mouse. Slamd64 has PS/2 mice blacklisted, so I had to unblock the psmouse module from /etc/modules.d/blacklist (by uncommenting the module entry). Then I loaded the module and rebooted. The reboot detected the mouse. It took me a while to find this tidbit of info (although, it was recorded on the Slamd64 forums). I'll be adding this to my local knowledgbase (in fact, I need to add a whole Slamd64 category first).

I've also installed phpsysinfo on this machine, so I can see the hardware and how it is detected by this tool. From what I've seen so far, the second CPU core takes the brunt of the load, with the first CPU core assisting when the second is maxed out...I don't know if this is normal or a software issue (maybe phpsysinfo needs to be optimized for dual core usage?), but I only tested this by refreshing the phpsysinfo browser session (it appears to put a quick load on the system...dunno if that's normal or not).

Anyways, if things go well with this distro, I'll be using this machine as my main Linux machine, eventually.