Alt-JBlog
Tech notes: stuff for geeks and non-geeks - (but mostly geeks.)

System Administration


Red Bull gives you wings…or a big headache

Posted by Alt J
On July 28th, 2008 at 14:07

Permalink | Trackback | Links In |

Comments (2) |
Posted in News, System Administration

Last week, I was at the Facebook developers conference. It was a pretty good conference and I learned a lot. Here are a few things I learned:

  • Some companies still operate with their blinders on: One of the sessions I was most excited about was “Made for Mobile.” I was hoping for some insight into developing apps for mobile phones and maybe some new “stuff” from Facebook. I was sorely disappointed. This session should have been named “Made for iPhone.” Instead of ranting in this post, I think I’ll do a dedicated post to the blinders concept.
  • Facebook is on the cutting edge when it comes to social networks: They announced Facebook Connect. If you’re a digg/citysearch/six apart user, you can see its effects already. It’s a new and easy way to put a social network twist on any site (using Facebook of course.)
  • Red Bull gave me a headache: The hardest stuff I regularly drink is Mountain Dew. Red Bull was a sponsor at this conference and as a result, the stuff was given out. I decided to give it a shot. The taste wasn’t very good. I’m a Guaraná Antarctica fan and so I’m a little picky when it comes to guarana. The taste of this took guarana and made it disgusting. To top it all off, within about 10 minutes of drinking it, I got the worst headache I’ve had in a long time. Needless to say, it didn’t give me wings and I don’t think I’ll be trying it again.
  • Facebook does an awesome job at scaling: This is the stuff I really love. In one session, the explained how they handle the high load demand due to their feeds. Their feeds are what displays all of your friends’ activity/actions on the main page when you’re logged in. If you think about it, that’s a lot of data. Just to generate your custom feed they have to go out and get all of the recent feed items from all your friends, filter and prioritize them, and then display it on a nice pretty page for you to see. And they do it all in around 60 milliseconds. Man, that’s fast!

Over 5,000,000 phpBB sites hacked

Posted by Alt J
On May 15th, 2008 at 10:05

Permalink | Trackback | Links In |

No Comments |
Posted in Software, System Administration

I see no problem with running phpBB, but there are so many people that don’t keep their software up to date. This is one reason why I hate to have 50 different types of software running on my servers. It’s just more to keep track of and hard to stay up to date on everything.
Here’s a link to the ComputerWorld article.

Oh that reminds me, I need to update my version of wordpress. BRB.

OpenSolaris on Amazon’s EC2, Yay!

Posted by Alt J
On May 12th, 2008 at 15:05

Permalink | Trackback | Links In |

No Comments |
Posted in Amazon EC2, System Administration

CloudsFor us UNIX fans, Sun Microsystems is working with Amazon to offer OpenSolaris on EC2.
I’m not sure how I missed it, but they made the announcement last week.

I just signed up to be included in the beta. I’ll post an update here when I get in and start tinkering. My main focus is going to be the performance of MySQL on EC2 comparing Linux with OpenSolaris.

I’m not sure what their long term plans are, but for now they aren’t incurring any extra charges (beyond the standard EC2 charges.) I know that RedHat charges extra to run their Enterprise version on EC2.

Multimedia, Entertainment & Distractions In (K)ubuntu

Posted by Alt J
On May 8th, 2008 at 09:05

Permalink | Trackback | Links In |

No Comments |
Posted in (K)ubuntu, System Administration

TrainTo help you get the most out of (K)ubuntu, be sure to enable Medibuntu as a source for packages. This will allow you to easily play those wmv files and DVDs on Linux as well as most other multimedia that use proprietary formats.

The steps for setting it up involve running three commands (Details here)
For me, it was a matter of running:
sudo wget http://www.medibuntu.org/sources.list.d/hardy.list -O /etc/apt/sources.list.d/medibuntu.list
sudo apt-get update && sudo apt-get install medibuntu-keyring && sudo apt-get update
sudo sed -e 's/ non-free//' -i /etc/apt/sources.list.d/medibuntu.list

Good luck!

My interrupt-driven life

Posted by Alt J
On April 30th, 2008 at 13:04

Permalink | Trackback | Links In |

No Comments |
Posted in Scripts & Programming, Security, System Administration

SlimeHere I am, peacefully working at my computer when I’m interrupted by a text message on my phone:

** PROBLEM alert - someserver.somewhere.com/SSH is CRITICAL **

That’s not a good thing. Hoping it’s a false alert, I try to ssh in. No luck. I try again. Still no luck. And a third time, Yes! I’m in. Let the troubleshooting begin.

I check to see what processes are running and sure enough, I find a culprit. There are a ton of sshd processes going. I take a look at auth.log and it’s full of “Failed password for root from 218.145.160.100 port 55739 ssh2″ messages (about 9,000 of them.) Here’s what’s going on: someone is trying to login to the server most likely by trying a bunch of passwords in a brute force attack. A brute force attack consists of trying every possible password until you find the right one. The attack doesn’t really concern me since I don’t allow password logins on most of the servers I manage. The excessive login attempts are a little annoying.

One command later and all traffic from that IP address drops into oblivion.

iptables -A INPUT -s 218.145.160.100 -j DROP

With that band-aid applied, it’s time to get something better in place for the long term. A while back there was some discussion about preventing or slowing down such attacks on the SLLUG email list and some people posted scripts they use to deal with it. Here is my current version of one of those scripts:

#!/bin/bash
case "$1" in
start)
# Put IP addresses for allowed hosts into this, separated by spaces.
SSH_ALLOWED="123.45.67.89 98.76.54.32"

iptables -A INPUT -p icmp -m state --state NEW,ESTABLISHED --icmp-type echo-request -m limit --limit 2/s -j ACCEPT
iptables -A INPUT -s 127.0.0.1 -j ACCEPT

# Allow TCP/UDP connections out. Keep state so conns out are allowed back in.
iptables -A INPUT -p tcp -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -p udp -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p udp -m state --state NEW,ESTABLISHED -j ACCEPT

# Allow ICMP out and anything that went out back in.
iptables -A INPUT -p icmp -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p icmp -m state --state NEW,ESTABLISHED -j ACCEPT

#put any custom rules for you rserver in this section
iptables -A INPUT -s 218.145.160.100 -j DROP
iptables -A INPUT -p tcp -m tcp --dport 111 -j REJECT
iptables -A INPUT -p tcp -m tcp --dport 11211 -j DROP
iptables -A INPUT -p icmp -j DROP
iptables -A INPUT -p udp -j DROP

#now for the ssh stuff
iptables -N SSH_Brute_Force
iptables -F SSH_Brute_Force
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -j SSH_Brute_Force
for IP in $SSH_ALLOWED; do
iptables -A SSH_Brute_Force -s $IP -j RETURN
done
iptables -A SSH_Brute_Force -m recent --name SSH --set --rsource
iptables -A SSH_Brute_Force -m recent ! --rcheck --seconds 60 --hitcount 5 --name SSH --rsource -j RETURN
iptables -A SSH_Brute_Force -j LOG --log-prefix "SSH Brute Force Attempt: "
iptables -A SSH_Brute_Force -j DROP

;;
stop)
iptables -F
iptables -X SSH_Brute_Force
;;

*)
echo "Usage: $0 {start|stop}" >&2
exit 1
;;
esac

This is an init script, so I put it in my /etc/init.d directory and set it up to run when the server boots up. What it does is only allows 5 SSH connection attempts per minute based on the source’s IP address. From there, it blocks and logs any connection attempts. Two words of warning when working with IP tables; be careful. It’s very easy to block yourself from accessing your own server. I’ve done this more times than I care to mention and had to take a drive to the datacenter or call their helpdesk to make things available again.