BSD machine fixed!

So, I swapped a known working motherboard into the BSD machine.  It now works.  I also decided to use a quad core AMD AM2+ CPU that I had sitting around.  That's all I changed.

I'd originally thought the problem was related to the hard disk.  So, I decided the night before to disconnect the drives (it has two SATA drives), to determine if it were the real issue.  It still experienced the same symptoms after boot-up attempts, which told me it wasn't a hard disk issue.  I also swapped out the RAM with a known working chip with the same results when trying to boot-up.

So, either the the old CPU (a dual-core AMD...I forget the model) died or something on the motherboard died (or maybe there was a short somewhere?).  I left it running a live instance of Linux Mint, just to see if it stays stable over the next 24 hours.

Next, I need to reinstall FreeBSD (wondering if I should try some others as well, such as OpenBSD or Mint).  I wiped the drive, thinking that there was some corruption issue...shouldn't have done that.

Slackware Reunited!

Well, I'm back to using Slackware.  I don't know if that's actually proper to say, since I still use Slackware as an IDS for my LAN, but that box is pretty much just monitoring the network...nothing else.  I had an issue with my new FreeBSD box (it won't boot properly) and I needed another box, so I powered up an old machine that had Slackware v12 on it.  Yes, I'll upgrade to the latest as soon as I can, since everything seems to be out-of-date, such as my browsers and such.

I was able to get onto irc.freenode.net (was previously logging in via Xchat-aqua using my Macbook), but had a problem with D-bus:

ron@slackbox:~$ xchat
process 7948: D-Bus library appears to be incorrectly set up; failed to read machine uuid: Failed to open "/usr/local/var/lib/dbus/machine-id": No such file or directory
See the manual page for dbus-uuidgen to correct this issue.
  D-Bus not built with -rdynamic so unable to print a backtrace
Aborted

No, I don't use irssi or BitchX (I used to, awhile back).  Found that I could kickstart D-bus with the following command:

dbus-uuidgen --ensure

Dunno why this was an issue, since I didn't have the issue before I powered off the machine...maybe something broke during the power-down cycle of that last shutdown?  Dunno.

I'm glad to be delving in Slackware again...I love tinkering with different environments, but I'm pretty spread thin with Windows 7 (necessary evil to do my hardcore gaming), FreeBSD, Ubuntu (my colo server), and now Slackware...been meaning to reactivate an old box with OpenBSD on it also.  We'll see how I can cope with with all this.  :)

FreeBSD Pains

My 'new' BSD machine (FreeBSD 8.2) crapped out again.  Actually, I think either my wife or one of my kids accidentally shut it down.  Now it tries to boot up and immediately shuts down during the boot-up process.  I can't look at logs because it keeps shutting down.  I tried single mode and it does the same thing.  I've been trying to see the last line of the boot-up process before it shuts down...that's like trying to capture (with your mind) one frame of a film...very hard to do.  Well, it looks like it can't mount the root partition (just from what I've seen the milli-second before it shuts down).  I need to run fsck on it but I can't do that if it's not booting up properly into single mode.

I cheated and tried to boot up live install of Linux Mint, Ubuntu (both of the latter are on USB sticks), and FreeBSD (on DVD)...they must be trying to mount the drive that the FreeBSD install is on, because they all shut down too.  So, I'm going to try a rescue version of FreeBSD (for memory sticks).

Once I fix this, I'm sure there's a rc.conf setting that I'll need to set to force an fsck during boot-up if needed.  This has happened ever since I installed FreeBSD and I'm a bit irritated...this should be enabled by default so that someone doesn't get 'locked' out of their system.  :/

Missing me some Slackware...

I haven't played with Slackware in quite awhile.  I still run a server through Linode.com but I no longer have Slackware installed as an OS (I'm using Ubuntu for ease of use...yes, it is easier to maintain compared to Slackware and I've not run into any 'gotchas' yet).  I run one machine that has Slackware installed (it's sorely in need of an update, though) and it is being used as a NIDS system.  I've another machine with Slack on it that hasn't been turned on in months (it's OS version is even older than the other system).  I'll probably turn on this system and begin to use it again, but it is in very sore need of cleaning (it has 4-5 hard disks with data ALL over the place).

I'm trying to resist the urge to run Slackware in a VM on my Alienware system.  It will require me to probably get more RAM (I'm trying to resist that idea for now).  I do not want to attempt a native install, as I don't feel like experimenting to get Slack to work on that system.  The integrated and dedicated GPUs will probably be an immediate issue, as well as the fact that my system is running two 750GB drives in RAID0.  And, that is also my gaming system.  There's no real need for me to install Slackware natively on my system.  But, I will definitely install Cygwin, since I can leverage it's tools (such as GnuPG) without having to open a shell and have an internet connection.  Cygwin is the less complicated of the aforementioned options.

But I am missing using Slackware, which is why I've been trying to be more active at ##slackware on irc.freenode.net.  The thing is, I also have a fetish for Open- and FreeBSD, so I've been focusing on both of those the past few years.

Snortreport install

I remember running snortreport awhile back and liked it. I want to try to use it again, but I was having issues installing it in FreeBSD.

It appears that the FreeBSD port of snortreport requires php4. I'm currently using php5 and want to run snortreport with minimal fuss. I do not want to try to run both php5 (for Apache and phpBB3) and php4, as it will break the server. There are several tutorials on how to run both but as I said, I don't want any fuss.

So, I delved a bit into the ports and makefiles. I looked at the makefile for snortreport and decided to remove the php check that stops me from installing the port. It then choked on jpgraph (a dependency)...it appears that jpgraph is actually the port that requires php4. I was going to edit the makefile for jpgraph to allow the install (by commenting out the line that checks for php4), but saw that there is another version of jpgraph called jpgraph2. I looked at that port's makefile and it didn't check for php4 (it did check for php5). I went ahead and installed jpgraph2 instead, then installed snortreport without any warning/error messages.

So, for those of you that want snortreport on FreeBSD and want to leverage the ports system, you can get around the php4 dependency issue by just installing jpgraph2.

Of course, I still have to fully get snortreport up and running before I claim 100% success, right? ;)

Trying to upgrade/revamp my lab

I'm trying to retire some of my older equipment in my lab.  The biggest move will be in migrating my old FreeBSD server to a new one.  Both are currently up and running.

The old:

FreeBSD 6.2-RELEASE #0: Fri Jan 12 11:05:30 UTC 2007
CPU: Pentium II/Pentium II Xeon/Celeron (447.69-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x652 Stepping = 2
Features=0x183fbff
real memory = 268427264 (255 MB)
avail memory = 252989440 (241 MB)
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1

The new:

FreeBSD 8.2-RELEASE-p3 #0: Tue Sep 27 18:45:57 UTC 2011
CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 4400+ (2210.20-MHz K8-class CPU)
Origin = "AuthenticAMD" Id = 0x60fb2 Family = f Model = 6b Stepping = 2
Features=0x178bfbff
Features2=0x2001
AMD Features=0xea500800
AMD Features2=0x11f
TSC: P-state invariant
real memory = 1073741824 (1024 MB)
avail memory = 1002987520 (956 MB)
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1

I've a LOT of data on the old system that I need to somehow offload and retain onto the new one.  I also have to stand up updates services (mysql, ssh, httpd) and apps (phpbb3, BASE).  I already have the new phpbb3 running (it is NICE), but still have to install BASE (although Snort is installed).

I'll keep you all updated on this.

Playing with the logs again

So, I've some logging going on. I typically look at my auth logs and my FW logs that reside within /var/log. I also archive my bruteforce blocking FW table (PF), as the table dumps when I reboot or when the system loses power.

I consolidated these logs into one massive file (333,603 IPs). Yes, there are probably many repeat IPs, but that's OK. Several (26 of them, consisting of two unique IPs) are when I accidentally blocked myself.

I took the resulting file and did this:

cat top10_1.txt | sort | uniq -c | sort -rn

which resulted in this file.

The IPs with a count of '238' are obviously part of a distributed brute forcing botnet...its intriguing the way it is depicted within this hack's output. Also, the actual number of unique IPs recorded is 2377.

Now, maybe I should script something to provide me something like this on a daily basis...meaning, I'd like to see only that day's activity (right now, I'm crunching logs from at least a year back).

Also, this is from my FreeBSD machine, which runs PF, has port 22 open to the world (locked down service, though), has port 3306 open, and is my security box.

Killing my usage of Snorby

I've stopped attempting to get Snorby running. Why? After digging into this for over two weeks, logging my attempts on this blog, I again asked for someone to guide me in the right direction at Snorby's Google group:

Any news on this issue?

I'm at a dead standstill in implementing...can't even get a login
prompt.

I realize your main focus is to get to v1.0 status, but its hard for
me to contribute to the project if I can't get it running even when
following the instructions specifically.

Thanks,
unixfool

The response?

Hello,

Version 1.0.1 is the current release. I very doubt you followed the
instructions properly as there are 20-30 people in the irc channel
that have had no issues. I am not even sure what your issue is. Did you rake snorby:setup RAILS_ENV=production

I have no problem helping when there are real errors but its quite
annoying when its just because someone did not read the docs.

Please post your logs and let me figure out a workaround.

- Dustin

My parting response:

I followed EXACTLY what was on your pages. If there's an issue with the way it was set up, it could be the fact that your instructions on your website need to be updated.

Look, I stated in my blog that I was going to test Snorby. You posted to my blog that you would like to know if there were any issues. I stated I had an issue and even gave you a LOT of debugging information, which is a far cry from what I've been seeing here in your Google group and now you're getting a bit snobbish?

I don't particularly like your tone, so from here on out, no Snorby for me. Cool project, but I shouldn't have to be a freaking Rails expert to use any security tool...really. The fact that I can set up Snort (and its deps) blindfolded and install most other frontends (and their deps) without issue or handholding tells me that I'm competent enough. I really don't need the attitude...and you did this on a freakin' group listing. An e-mail would've been more tactful, but in the end, your attitude would've rubbed me raw all the same.

And, you know what? You keep harping on visiting freenode. I've no problem with freenode, especially since I oper and have ownership of ##slackware, but if you would much rather leverage IRC for support, what do you have this group for? Really? If you respond to everyone here in such a manner when they ask questions about your tool, you're not going to get nearly the user base that you want. No one wants to be spoken down to in such a manner.

Anyways, I'm out. I've said my piece and will remove myself from this group. Please do NOT respond or send me e-mail. You've made yourself clear that you don't like helping people use your tool. I'm done.

The whole thread is here

Actually, I'm pretty pissed off. I don't like using someone's tool and trying to contribute but having issues even implementing the freakin' piece of software, especially when I get major attitude when asking questions. WTF is the use in supplying debugging traces when the developer doesn't even look at it and assess if there's something wrong with his code implementation or if the user is using it wrong. I have some project management skills and I can tell you now that if I developed a process at my work environment and my team had issues with my process, I'd want to know the who/what/when/where/why so that I can assess my process and see if I made an error or if it needs to be clarified. I NEVER tell my team something akin to, "you didn't read the process," especially if there is a high probability that they actually did. No one is infallible, not even this particular guy. I'd have been humbled if I'd found that there indeed were instructions that I'd missed...that's not the case, though, unless he's maintaining documentation in another place. I wouldn't know and I shouldn't have to visit a damned IRC channel to ferret out discrepancies or hunt for additional support in a new tool...WTF is the Google group for if I can't ask questions there? Can you imagine if everyone on the AOLS mailing group said, "visit the IRC channel for your answer"?

Belittling people alienates people. Not even US Army drill sergeants do this (don't believe everything you see on TV).

No Snorby coverage will happen here again. No Snorby usage will occur. We're closing this chapter right now!

EDIT: After this post and after a few days of cooling off a bit, I decided to determine if the issue was actually with me, the way I set up Ruby/Rails, or any configuration of Snorby. I was still 100% sure I followed the directions properly, so I didn't change any configs of Snorby or my Ruby/Rails setup. I only refrshed the Snorby environment by pulling the latest update. Guess what? Snorby worked. This leads me to believe that something in the Snorby code changed...something the developer changed after he pissed me off with his insistence that I hadn't read the instructions and that I was just another person using his tool who didn't know basic sysadmin skills. Kinda funny that the tool works now when I didn't change anything or reapply the instructions...I just refreshed the code. Something smells bad and it isn't me...

Ruby, Rails, Gems Redux Part III

I'm starting to get a bit annoyed. I still can't get this working properly. Getting the same error as I got in my last post. I haven't changed anything but I've double- and triple-checked.

Right now, I'm currently posting to the Snorby Goggle group to try to get some assistance, which I usually don't have to do...I hate being dependent upon others, but that's just me.

Anyways, so far, I've been able to rule out MySQL as the culprit, as I'm seeing connections from Ruby to the MySQL server. I'm also able to connect to the server as 'root' and as 'snort'. The web server continues to issue status 500 and the Ruby logs indicate that there's something wrong with the user_session/new.html.erb file (keeps saying 'no credentials provided').

One suggestion I got is to do a 'git pull' to update Snorby from the Snorby directory. That command pulled quite a few changes, but after the pull, I'm still receiving the same error:

root@slackbox:~/RAILS/RAILS/Snorby# git pull
remote: Counting objects: 604, done.
remote: Compressing objects: 100% (522/522), done.
Indexing 542 objects...
remote: Total 542 (delta 393), reused 43 (delta 12)
100% (542/542) done
Resolving 393 deltas...
100% (393/393) done
37 objects were added to complete this thin pack.
* refs/remotes/origin/cache_test: storing branch 'cache_test' of git://github.com/mephux/Snorby
commit: a30cf8e
* refs/remotes/origin/master: fast forward to branch 'master' of git://github.com/mephux/Snorby
old..new: e17ace1..7edf9e9
Updating e17ace1..7edf9e9

Fast forward
app/controllers/application_controller.rb | 2 +-
app/controllers/comments_controller.rb | 57 ++++++++++
app/controllers/events_controller.rb | 4 +-
app/controllers/pages_controller.rb | 25 ++++-
app/controllers/searches_controller.rb | 4 +-
app/controllers/user_sessions_controller.rb | 2 +-
app/helpers/application_helper.rb | 41 +++-----
app/helpers/comments_helper.rb | 2 +
app/models/comment.rb | 5 +
app/models/event.rb | 17 +++
app/models/importance.rb | 3 +-
app/models/report.rb | 2 +-
app/models/search.rb | 4 +-
app/models/user.rb | 17 +++-
app/views/comments/_comment.html.erb | 15 +++
app/views/comments/_form.html.erb | 9 ++
app/views/comments/create.js.rjs | 11 ++
app/views/comments/destroy.js.rjs | 2 +
app/views/comments/edit.html.erb | 3 +
app/views/comments/new.html.erb | 5 +
app/views/events/_comments_for_event.html.erb | 21 ++++
app/views/events/_event.html.erb | 21 +++-
app/views/events/_ip_data.html.erb | 15 ++-
app/views/events/_summary.html.erb | 8 +-
app/views/events/remove_event.js.rjs | 2 +-
app/views/events/send_event.html.erb | 4 +-
app/views/events/show.html.erb | 4 +
app/views/pages/category.html.erb | 13 +++
app/views/pages/category.js.rjs | 1 +
app/views/pages/dashboard.html.erb | 20 ++--
app/views/pages/severity.html.erb | 8 ++
app/views/pages/severity.js.rjs | 1 +
app/views/reports/send_report.html.erb | 2 +-
app/views/searches/send_search.html.erb | 2 +-
app/views/searches/show.html.erb | 4 +-
app/views/settings/index.html.erb | 2 +-
config/email.yml.example | 3 +-
config/routes.rb | 8 +-
db/migrate/20090719222259_create_comments.rb | 16 +++
db/schema.rb | 12 ++-
public/flash/clippy.swf | Bin 5380 -> 0 bytes
public/images/.DS_Store | Bin 12292 -> 12292 bytes
public/images/comment/comment_top.png | Bin 0 -> 4759 bytes
public/images/cross.png | Bin 655 -> 689 bytes
public/images/other/{destroy.png => destroy2.png} | Bin 715 -> 715 bytes
public/images/other/edit.png | Bin 0 -> 497 bytes
public/images/other/is_not_important.png | Bin 648 -> 633 bytes
public/images/other/no_comment.png | Bin 0 -> 604 bytes
public/images/other/slash.png | Bin 714 -> 689 bytes
public/images/other/slash2.png | Bin 0 -> 714 bytes
public/images/other/whois.png | Bin 0 -> 595 bytes
public/stylesheets/snorby.css | 118 ++++++++++++++++++++-
test/fixtures/comments.yml | 11 ++
test/functional/comments_controller_test.rb | 54 ++++++++++
test/unit/comment_test.rb | 7 ++
55 files changed, 504 insertions(+), 83 deletions(-)
create mode 100644 app/controllers/comments_controller.rb
create mode 100644 app/helpers/comments_helper.rb
create mode 100644 app/models/comment.rb
create mode 100644 app/views/comments/_comment.html.erb
create mode 100644 app/views/comments/_form.html.erb
create mode 100644 app/views/comments/create.js.rjs
create mode 100644 app/views/comments/destroy.js.rjs
create mode 100644 app/views/comments/edit.html.erb
create mode 100644 app/views/comments/new.html.erb
create mode 100644 app/views/events/_comments_for_event.html.erb
create mode 100644 app/views/pages/category.html.erb
create mode 100644 app/views/pages/category.js.rjs
create mode 100644 app/views/pages/severity.html.erb
create mode 100644 app/views/pages/severity.js.rjs
create mode 100644 db/migrate/20090719222259_create_comments.rb
delete mode 100644 public/flash/clippy.swf
create mode 100644 public/images/comment/comment_top.png
rename public/images/other/{destroy.png => destroy2.png} (100%)
create mode 100755 public/images/other/edit.png
create mode 100644 public/images/other/no_comment.png
create mode 100644 public/images/other/slash2.png
create mode 100644 public/images/other/whois.png
create mode 100644 test/fixtures/comments.yml
create mode 100644 test/functional/comments_controller_test.rb
create mode 100644 test/unit/comment_test.rb

root@slackbox:~/RAILS/RAILS/Snorby# script/server -e production -b 10.150.1.106 -p 3000
=> Booting WEBrick
=> Rails 2.3.2 application starting on http://10.150.1.106:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server
[2009-07-23 22:18:39] INFO WEBrick 1.3.1
[2009-07-23 22:18:39] INFO ruby 1.8.6 (2007-03-13) [i486-linux]
[2009-07-23 22:18:39] WARN TCPServer Error: Address already in use - bind(2)
[2009-07-23 22:18:39] INFO WEBrick::HTTPServer#start: pid=5752 port=3000


Processing UserSessionsController#new (for ::ffff:10.150.1.106 at 2009-07-23 22:18:40) [GET]
Parameters: {"action"=>"new", "controller"=>"user_sessions"}
Rendering template within layouts/application
Rendering user_sessions/new

ActionView::TemplateError (undefined method `login' for #) on line #8 of app/views/user_sessions/new.html.erb:
5: <% form_for @user_session, :url => user_session_path do |f| %>
6: <%= f.error_messages %>
7: <%= f.label :login %>

8: <%= f.text_field :login %>

9:

10: <%= f.label :password %>

11: <%= f.password_field :password %>


app/views/user_sessions/new.html.erb:8
app/views/user_sessions/new.html.erb:5
/usr/lib/ruby/1.8/webrick/httpserver.rb:104:in `service'
/usr/lib/ruby/1.8/webrick/httpserver.rb:65:in `run'
/usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:162:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:95:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `each'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:23:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:82:in `start'

Rendering /root/RAILS/RAILS/Snorby/public/500.html (500 Internal Server Error)


Processing ApplicationController#index (for ::ffff:10.150.1.106 at 2009-07-23 22:20:40) [GET]

ActionController::RoutingError (No route matches "/test/" with {:method=>:get}):
/usr/lib/ruby/1.8/webrick/httpserver.rb:104:in `service'
/usr/lib/ruby/1.8/webrick/httpserver.rb:65:in `run'
/usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:162:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:95:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `each'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:23:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:82:in `start'

Rendering /root/RAILS/RAILS/Snorby/public/404.html (404 Not Found)


Processing UserSessionsController#new (for ::ffff:10.150.1.106 at 2009-07-23 22:20:55) [GET]
Parameters: {"action"=>"new", "controller"=>"user_sessions"}
Rendering template within layouts/application
Rendering user_sessions/new

ActionView::TemplateError (undefined method `login' for #) on line #8 of app/views/user_sessions/new.html.erb:
5: <% form_for @user_session, :url => user_session_path do |f| %>
6: <%= f.error_messages %>
7: <%= f.label :login %>

8: <%= f.text_field :login %>

9:

10: <%= f.label :password %>

11: <%= f.password_field :password %>


app/views/user_sessions/new.html.erb:8
app/views/user_sessions/new.html.erb:5
/usr/lib/ruby/1.8/webrick/httpserver.rb:104:in `service'
/usr/lib/ruby/1.8/webrick/httpserver.rb:65:in `run'
/usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:162:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:95:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `each'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:23:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:82:in `start'

Rendering /root/RAILS/RAILS/Snorby/public/500.html (500 Internal Server Error)

Grrr. Something isn't quite right here. I've followed the documents properly in installing Snorby, but something was obviously missed. I'm totally reliant upon the developers at this point. While this is normal for some people, it isn't for me. At work, our dev team are the VERY last people I go to, because they tend to either try to make you look stupid or will say some shit like 'it is what it is'. I'm trying to keep in mind that my bad experience with developers is limited to work and not the open-source community. That being said, I've invested quite a bit of time and effort on the Snorby project. While I've learned a few things, I do have an end goal and I'm a goal-oriented person.

I'll stop updating on Snorby until I actually have it working.

Ruby, Rails, Gems Redux Part II

Did a little research on the gem for MySQL and decided to try this:

root@slackbox:~/RAILS/RAILS/Snorby# locate mysql_config
/usr/man/man1/mysql_config.1.gz
/usr/bin/mysql_config
root@slackbox:~/RAILS/RAILS/Snorby# gem install mysql -- --with-mysql-config=/usr/bin/mysql_config
Building native extensions. This could take a while...
Successfully installed mysql-2.7
1 gem installed
Installing ri documentation for mysql-2.7...
Installing RDoc documentation for mysql-2.7...
root@slackbox:~/RAILS/RAILS/Snorby#

Now about my Snort architecture, I'm thinking all I'm gonna have to do is copy my Snort database over to Slackbox and then have my two Snort machines (one internal and one sensor at a datacenter) report to Slackbox....OR, have the Snort sensors report to BOTH the FreeBSD server AND Slackbox! I think the latter will work and it sounds like the better solution.

I'll be updating this post with my successes and failures most of the night, I suspect, or at least until I get good and pissed off. LOL!

=====

Update:

There's nothing like backing up an 83MB database file on old hardware:

Starting: 6:31PM up 23 days, 19:27, 4 users, load averages: 2.89, 2.94, 3.13

Ending: 6:33PM up 23 days, 19:29, 4 users, load averages: 5.88, 3.98, 3.51

While I'm sure that's incomparable to an enterprise database, at one point, I thought the old dell system would lock up.

I also was trying to do this via phpMyAdmin on both machines, but I didn't know the dbase size was that large (4 yrs of sniffing data). phpMyAdmin on the BSD box would say it was finished exporting but I'd check the filesize and it was different each time (did it like 4 times before I decided to go commandline. phpMyAdmin kept giving me a filesize of between 20M and 40M. It must've been choking out. I optimized the dbase, also, so it was more than likely larger than 83MB.

=====

Update:

Had to upgrade MySQL, as my 83MB file wouldn't import into Slackbox's MySQL server. 30 seconds into the import, the import would lock up or die. Apparently, it's a known issue with MySQL's lower versions.

Anyways, after the import and creation of new MySQL users, I had to edit Snorby's config/database.yml file, specifically the development part. The reason:

root@slackbox:~/RAILS/RAILS/Snorby# script/server -p 11001
=> Booting WEBrick
=> Rails 2.3.2 application starting on http://0.0.0.0:11001
=> Call with -d to detach
=> Ctrl-C to shutdown server
[2009-07-17 21:48:14] INFO WEBrick 1.3.1
[2009-07-17 21:48:14] INFO ruby 1.8.6 (2007-03-13) [i486-linux]
[2009-07-17 21:48:14] WARN TCPServer Error: Address already in use - bind(2)
[2009-07-17 21:48:14] INFO WEBrick::HTTPServer#start: pid=3848 port=11001
/!\ FAILSAFE /!\ Fri Jul 17 21:48:17 -0400 2009
Status: 500 Internal Server Error
Can't connect to MySQL server on 'no_not_use' (111)

It's still not clear to me why I had to edit it, but I did because the production portion was populated with the proper credentials but I was still receiving the above error..."Can't connect to MySQL server on 'no_not_use'". When I did it, I stopped getting that error.

*** I found why I was getting the MySQL error. The config/database.yml development entry has 'mysql' for the database entry. It should be 'no_not_use'. I've edited this to what is was originally supposed to be and changed everything back to 'no_not_use'. I no longer get the error when using the production settings. ***

Also, notice that I ran in what I want to call 'debug mode' because I wanted to see what was hanging up the connection.

So, now, after some editing and fiddling, I get the following in 'debug mode':

root@slackbox:~/RAILS/RAILS/Snorby# script/server -e production -b 10.150.1.106 -p 11001
=> Booting WEBrick
=> Rails 2.3.2 application starting on http://10.150.1.106:11001
=> Call with -d to detach
=> Ctrl-C to shutdown server
[2009-07-17 21:55:37] INFO WEBrick 1.3.1
[2009-07-17 21:55:38] INFO ruby 1.8.6 (2007-03-13) [i486-linux]
[2009-07-17 21:55:38] WARN TCPServer Error: Address already in use - bind(2)
[2009-07-17 21:55:38] INFO WEBrick::HTTPServer#start: pid=3915 port=11001


Processing UserSessionsController#new (for ::ffff:10.150.1.106 at 2009-07-17 21:55:40) [GET]
Parameters: {"action"=>"new", "controller"=>"user_sessions"}
Rendering template within layouts/application
Rendering user_sessions/new

ActionView::TemplateError (undefined method `login' for #) on line #8 of app/views/user_sessions/new.html.erb:
5: <% form_for @user_session, :url => user_session_path do |f| %>
6: <%= f.error_messages %>
7: <%= f.label :login %>

8: <%= f.text_field :login %>

9:

10: <%= f.label :password %>

11: <%= f.password_field :password %>


app/views/user_sessions/new.html.erb:8
app/views/user_sessions/new.html.erb:5
/usr/lib/ruby/1.8/webrick/httpserver.rb:104:in `service'
/usr/lib/ruby/1.8/webrick/httpserver.rb:65:in `run'
/usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:162:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:95:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `each'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:23:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:82:in `start'

Rendering /root/RAILS/RAILS/Snorby/public/500.html (500 Internal Server Error)

The thing is, I see nothing in the web browser, but:

We're sorry, but something went wrong.

We've been notified about this issue and we'll take a look at it shortly.

Something else that is nagging me that I was trying to fiddle with is:

[2009-07-17 21:55:38] WARN TCPServer Error: Address already in use - bind(2)

There is only one Ruby service running and nothing is utilizing that port when I run Ruby. I'm ignoring it for now.

I would love to see what the WEBrick logs show, if there are any.

For now, its time to do some serious Googling and maybe hit up my three Ruby/Rails books.

Ruby, Rails, Gems Redux

I decided to use Slackware this time. I've had better luck.

My install already has Ruby 1.8.6 (the latest stable is 1.8.7, I believe).

Ran into an issue when following these instructions. Was supposed to do 'rake gems:install' but got a 'prawn' error

root@slackbox:~/RAILS/RAILS/Snorby# rake gems:install
(in /root/RAILS/RAILS/Snorby)
rake aborted!
no such file to load -- prawn

Fixed it by using 'gem install prawn'. After running that command, I was able to run the 'rake gems:install' without error.

Now I'm having a similar issue when running 'rake snorby:setup':

root@slackbox:~/RAILS/RAILS/Snorby# rake snorby:setup
(in /root/RAILS/RAILS/Snorby)
Setting Up Snorby Database.
!!! The bundled mysql.rb driver has been removed from Rails 2.2. Please install the mysql gem and try again: gem install mysql.
rake aborted!
no such file to load -- mysql

Running 'gem install mysql' give me a BUNCH of errors:

root@slackbox:~/RAILS/RAILS/Snorby# gem install mysql
Building native extensions. This could take a while...
ERROR: Error installing mysql:
ERROR: Failed to build gem native extension.

/usr/bin/ruby extconf.rb
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lm... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lz... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lsocket... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lnsl... no
checking for mysql_query() in -lmysqlclient... no
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers. Check the mkmf.log file for more
details. You may need configuration options.

Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/usr/bin/ruby
--with-mysql-config
--without-mysql-config
--with-mysql-dir
--without-mysql-dir
--with-mysql-include
--without-mysql-include=${mysql-dir}/include
--with-mysql-lib
--without-mysql-lib=${mysql-dir}/lib
--with-mysqlclientlib
--without-mysqlclientlib
--with-mlib
--without-mlib
--with-mysqlclientlib
--without-mysqlclientlib
--with-zlib
--without-zlib
--with-mysqlclientlib
--without-mysqlclientlib
--with-socketlib
--without-socketlib
--with-mysqlclientlib
--without-mysqlclientlib
--with-nsllib
--without-nsllib
--with-mysqlclientlib
--without-mysqlclientlib


Gem files will remain installed in /usr/lib/ruby/gems/1.8/gems/mysql-2.7 for inspection.
Results logged to /usr/lib/ruby/gems/1.8/gems/mysql-2.7/gem_make.out

Grrr...!!!

But, I'm a lot closer this time than last time. I'll sort it out either tomorrow night or this weekend.

Oh, and one more thing. Maybe this is more complicated than it has to be, because I've already got BASE running on a server who's internal IP is 10.150.1.103 (FreeBSD on a Dell server/workstation). The MySQL server is also on that box. Maybe I don't actually need the MySQL gem? Can I leverage the actual database on the FreeBSD box and maybe skip some steps? We'll find out, otherwise, I'm going to have to maybe copy the database over to the Slackware machine so I can test. Yeah, I really wanted Snorby on the FreeBSD box but for some reason I'm more comfortable with Slackware.

Rails, Ruby, Gems...PITA

I spent the evening trying to get Snorby installed.

I've sporadically messed with Ruby on Rails before, actually getting it installed and playing a bit with it before moving on to other things.

Now, I've been hindered by an out-of-date Rails install. I tried to install Snorby and every step I have to take two steps backward. I ended up reinstalling to the latest version that FreeBSD (yeah, doing this on a BSD install, as it seems easier to install this way), but that version wasn't high enough.

Now, I'm installing the latest by source. I'm as far as I've ever been tonight, which is good because I'm running out of time. I'm installing the Gems at the moment and the install is agonizingly slow (doing this on a dual proc 450MHz machine). It appears most of this are documentation installs. :/

Maybe I can get this done and still be able to get a bit of sleep before I've to get up for work.

So far, see below .txt file...looks good so far:

http://wigglit.ath.cx/ruby.txt

AAARGH!!

[root@delly /usr/local/www/data/Snorby]# rake snorby:setup
(in /usr/local/www/data/Snorby)
Missing these required gems:
javan-whenever

You're running:
ruby 1.8.5 at /usr/local/bin/ruby18
rubygems 1.3.1 at /root/.gem/ruby/1.8, /usr/local/lib/ruby/gems/1.8

Run `rake gems:install` to install the missing gems.

Almost there but I'm out of time...will continue tomorrow.

I Created some scripts for Snort

I've created (well, modified) a Snort initialization, restart, and shutdown script for Slackware and OpenBSD. They are linked below.

The OpenBSD script works solidly.

The Slackware script works sporadically and I've no idea how to debug it (although I haven't tried 'strace' yet). It appears to work manually every time, but when run as a cron job, it's sometimes, seemingly randomly, doesn't restart. The cron job runs every hours but because it sometimes doesn't start, I now have holes in my website's IDS coverage.

Note that I didn't HAVE to create start/stop scripts for Snort, as I could've started Snort by utilizing the rc.local file, but I'd have still had to manually kill the Snort process whenever I wanted to stop Snort. Having an init script do this is much cleaner.

The fact that I've gotten it working on the OpenBSD machine hints that I've a minor issue with the Slackware script that I have yet to account for, but its frustrating me, so I'll throw it online to see if someone can help with debugging. Yeah, I'd searched for help via Google but didn't see much of Snort init scripts for Slackware (although I may find something if I look at any scripts for other distributions).

I also got Snortalog to process my Snort raw logs into a statistical report, although I had to import 6.2MB of flat files to my FreeBSD box (which Snortalog is installed on), then have Snortalog crunch that data into a HUGE (3.9MB) HTML file! Needless to say, that HTML file takes almost 5 minutes to load into a browser. I've got to filter the logs and only have it crunch certain dates to make the file less bulky.

Snortalog definitely highlights that I could do some tuning, as it shows a very high amount of MS-SQL worm attempts (MS Blaster) hitting my server, amongst other things. This is a good tool that I'd previously used (and had forgotten) at a prior place of employment. It would be nice if I could figure out how to get it to crunch my IPF FW logs.

Another oldie but goddie is SnortSnarf. It is a perl script, as is Snortalog, that parses Snort files (the alert file and the payload files) into readable HTML pages, which is a bit better at searching via command-line. It is not as handy as ACID/BASE is, though, but has lower overhead. Sadly, SnortSnarf's home page is gone, but I've linked Snort.org's archive.

EDIT --

I've found my 'error'. What happened was that I had line 34 commented out and line 35 uncommented. Line 35 is specifically for usage with OpenBSD. Line 34 is specifically for Slackware. I rectified this by uncommenting line 34 and commenting line 35. I'll also put commentary explaining this. Consider this issue solved!

Edited 8/30/2007:

Revised Script that works! *yes, click here*

Shell Scripting | Creation of a Subnet and Securing Wireless Access Points

I've been trying to automate some things on my Linux and BSD boxes, so I've been scripting a bit lately.

For one, I like the mailed stats that FreeBSD and OpenBSD provides the administrator, so I've attempted to do the same for Slackware. I've a version that also runs on BSD, although you have to hack it to get it to work under a BSD machine. I'm currently looking over it to see if I can lessen the hacking of the script when using it on a non-Linux or non-Slackware machine, but for now it does work. It does not mail the admin yet, but it does keep a listing of stats every hour on the hour (via cron).

I've used several web-based resources to create this script:

http://www.linuxcommand.org/
http://www.freeos.com/guides/lsst/

I've also bought a few scripting books:

Unix Shells by Example, 3rd Edition, by Ellie Quigley
Linux Shell Scripting with Bash, by Ken O. Burtch
Learning the bash Shell, 2nd Edition, by Cameron Newham and Bill Rosenblatt

There's a ton of shell scripting books out there, along with a ton of free online tutorials, but the ones I've mentioned gave me the most insight.

I shall post a link to my script when I've finished working it to my liking.

I've also done a few things to my network during the last week.

I bought a Netgear VPN Firewall (FVS114). I want to play with hardware a bit and this unit was cheap. Sometime in the near future, I want to inplement a VPN tunnel from my residence to my linode. Sure, I can implement it via open-source software but I have to start delving with hardware if I want to sell myself as a professional security consultant. Anyways, I was previously utilizing a Linksys WRT54X4 Firewall/Router/WAP as my border router/gateway, which was fine where it was, but in order to utilize the Netgear to its fullest (which I plan to do), it needed to be placed at the border, so I put the Netgear in the Linksys' place. I then put the Linksys inside my LAN, as I needed it's WAP capabilities.

It took me a week to get things the way I wanted them. I wanted the Linksys on its own network, and that required creating a subnet. I opted to let it use its default network segment, 192.168.1.0/24. It pulled an IP from the Netgear (the Netgear is set up to serve IP addresses via DHCP). I was able to run a CAT5 from the Linksys to my Shuttle box and gain access to the administrative browser. Everything was going well, until I found that I couldn't ping other machines on the Linksys network segment. I spent a week trying to figure out why until I got a coworker to come over and take a look at things. He almost immediately got things working. I found that the Sygate firewall that I had installed on the Shuttle was impeding things. I turned it off and the laptop that was associated with the WAP was able to ping the Shuttle. One more problem was apparent: I wasn't able to get out to the internet on any laptop, although I could on the Shuttle. The reason? The Linksys FW was enabled. Once that was turned off, I was able to open up a browser and point it to MSN.com. Those were pretty simple solutions that any competent engineer should have been able to fix. My issue? Well, most engineers run standard tools in their work environment. I'd forgotten about the Sygate firewall, which is only installed on two of my many machines...so, its not so standard within my network environment, so it was easy to forget. That, and I was so wrapped up with getting this setup to work that I didn't check the obvious items.

All that is left now is to add a static route on the Netgear that will allow communcations from the Linksys netrange.

This is a decent setup, as you always want to segregate your WAP from your network. For home users, the basic setup is fine, but I'm not a regular home user. I want to at least TRY to do things the right way.

Now, since we're talking about WAPs, I'll let you know that I'm using WEP, the protocol that's branded 'unsecure'. Why am I using it? Because not all of my wireless devices can use the WPA protocol. Some people say that WEP is so insecure that its better off not using it...that's total B.S. You always want to use security in-depth anyways, which means you need to implement your security in layers to cover all potential weaknesses. Here's what I normally do:

1. Create a good, long password for the administrative GUI.

2. Either limit the DHCP pool to a very small amount (depending on how many wireless devices you have...I've at least 5, so I have a DHCP pool of 7), or turn off DHCP and assign your devices static IPs. This way, if someone gains access to your network, he's smart enough to get his own IP and not have it given to him. The lesson is to not make it easy to bust into your network.

3. Change your WEP key from time to time, maybe once a month. My Linksys will ask for any phrase and create 4 keys based off of that one phrase. Rotate those every once in awhile. Why do all this? The WEP key is supposedly easy to crack. A coworker of mine did attempt to crack a key. He couldn't. Many people think it is easy, but apparently its not as easy as many people say, but in case it IS easy, rotate your keys from time to time.

4. You can use MAC address authentication. Sure, someone could spoof a MAC, but remember that we're layering security...he may spoof a MAC but he won't be able to circumvent the rest of the things you've implemented.

5. Don't broadcast your SSID. Those who know how to hack will find it anyways, but what you want to worry about is the script kiddies out there and the ocassional bandwidth leech. Don't worry about the serious crackers out there, as if they wanted to pop your box, they could easily do it, with or without your basic security layering.

6. On some WAPs, you can dial down the power a bit so the wireless signal doesn't broadcast out to 2-3 blocks. Of course, when you do this, your wireless bandwidth gets throttled. Weigh the cost of this in your own mind and decide what would be best for your network. I don't throttle my signal but I do have my WAP in the basement, which does cut down on the signal. If you've a WAP on a 2nd or 3rd floor home or building, its going to be hard to throttle the signal to the point that others won't see it, since the WAP is physically on higher ground.

I may be missing a few wifi pointers, but I'll let others fill in the blanks. The above are the ways I keep my WAP and network secure. I hope you guys and gals can benefit from those tips.