Bash Scripts

Remove all installed Ruby Gems.

Posted in Bash Scripts, Linux on June 18th, 2013 by admin – Be the first to comment

This one liner will remove all of the installed gems on your systems, including all versions.

for gem in `gem list | awk '{print $1}'` ; do gem uninstall $gem -aix 2> /dev/null ; done

Pidinfo – Statistics for individual process.

Posted in Bash Scripts, Linux on November 24th, 2011 by admin – Be the first to comment

There are so many methods to obtain information about a particular process on Linux, however many tools have one specific purpose whether it’s reporting memory, IO statistics on an individual basis.  Alternatively, there is also the /proc file system which contains a wealth of information, it too is spread out.  I put together ‘pidinfo’ to be a simple tool to grab information from various places and resources and then compiling them into organized results.  It’s generates information very quickly and provides insight into a particular process by providing statistics such as cpu usage, memory usage, IO statistics, state, open file descriptors, executor, startup command and so fourth.

I will continue to update it to add more functionality(Download):  https://github.com/vigeek/nixutils

OFBiz/Opentaps service wrapper/startup script.

Posted in Bash Scripts on February 23rd, 2011 by admin – Be the first to comment

I was recently tasked with putting together a service wrapper for opentaps, unfortunately the official service wrapper and third party solutions are quite basic and generally just execute startofbiz.sh without anyway to legitimately determine whether the service is correctly running and responding to web requests. Executing the startofbiz.sh shell script simply reports “starting ofbiz”, it’s up to you from that point to determine whether or not the start succeeded, I wanted something more robust than this. This company in particular generates significant income on their website and heavily utilizes opentaps, some of the users executing this script would be junior level administrators therefore it was import for this script to take a lot of the guess work on what opentaps/ofbiz is doing during the start/stop phase.

The Script
Note: There are two versions of this script (I’ll be posting the other soon), one wraps around the existing stop/start scripts for opentaps/ofbiz. The other completely replaces it. Both achieve the same result.

Details: We essentially use CURL to determine whether the service is running, CURL will make a header request, if the header request delivers the desired results, it’s online and responding. Else, it’s offline. Depending on your server performance you may need to adjust the connect-timeout, but in virtually every case the default (2) should suffice.

Features:

  • debugging (optional): Features a simple ‘debug’ mode that after start, opens the console.log in more.
  • cache clear (optional): Allows the script to drop the Linux memory cache after stopping opentaps/ofbiz.
  • runuser: Allows you to run the service as any user (including root, however not recommended).
  • chkconfig: chkconfig-able (CentOS/RHEL) easily add service to start-up services. (i.e: chkconfig otaps on)
  • emailing (on): On failure or error, the script will send an e-mail with information regarding the failure.
  • error handling: Has built in error handling for most functions, will add to this shortly with traps.
  • configurable: Script can easily be edited, most values are definable.
  • distribution independent: Designed to work on any Linux distribution (tested on: CentOS/RHEL/Debian).

Functions:

  1. stats: The ‘stats’ option reports some information regarding a running opentaps/ofbiz instance. (active connections, memory usage, cpu use, runtime)
  2. start: Starts opentaps/ofbiz, utilizes curl to determine when the application is fully operational.
  3. stop: Stops opentaps/ofbiz utilizes curl to determine when the application is offline.
  4. status: Reports whether opentaps/ofbiz is running and reports the pid.
  5. restart: Simply calls the stop and start functions, allows for proper restarting.

Installation

(Assumes you’re using RHEL/CentOS)

  1. Copy the code listed below.  Or download script: otaps.sh
  2. Change into /etc/init.d (cd /etc/init.d)
  3. Open up new file in nano/vi  (nano otaps.sh)
  4. Paste all code, then save file.
  5. Edit the configurable options at the top of the script.
  6. Make file executable (chmod ug+x test.sh)
  7. Ensure it’s working properly (service otaps) should report the script usage.
  8. To enable startup/shutdown for current runlevel:  chkconfig otaps on
  9. Test & Enjoy (please report bugs/suggestions if any)
#!/bin/sh
# chkconfig: - 95 20
# description: Open Taps / OFbiz service wrapper/control script.
# Tested on:  CentOS/RHEL/Fedora
# Author:  Russ Thompson @ viGeek.net
# Objective:  Method to bring up opentaps/ofbiz instance and ensure it's fully operational via curl.

# Functions
# start - starts opentaps/ofbiz
# stop - stops opentaps/ofbiz
# restart - restarts opentaps.
# status - gets opentaps/ofbiz run status.  Aslo reports connections.
# stats - Gets number of active connections, memory usage, CPU usage, runtime etc.

# Source function library
. /etc/rc.d/init.d/functions

# Define runuser (root not recommended).  Ensure open taps/ofbiz path chowned to this user.
RUSER="svc-user"

# Set pidfile (not used in this version)
PFILE="/var/run/otaps.pid"

# OpenTaps/OFBiz installation path.
OPATH="/usr/local/opentaps"

# E-mail parameters (we e-mail on errors, failed starts etc)
EADDY="rthompson@reddit.com"
ESUB="$(hostname -s) E-mail subject message"
EFILE="tmpfile.txt"

# Host IP Address & PORT (We use this to curl to ensure instance is fully started).  Must be seperated by colon.
HIPP="69.178.69.69:8080"

# Define JAVA path (not needed if variables are exported for user).
export PATH=/opt/jdk1.6.0/bin:$PATH
export JAVA_HOME="/opt/jdk1.6.0/"
JAVA=$JAVA_HOME/bin/java

# Counter (Must be 0)
COUNTER="0"

# DEBUG (simply opens console.log after start, uses more)
DEBUG="0"

# Drop Linux memory cache after stop? May help with application performance after multiple restarts.
# Not recommended in shared environments, use at your own risk, test first.
DCACHE="0"

mail_send () {
/bin/mail -s "$ESUB" "$EADDY" < $EFILE
rm -f $EFILE
}

start() {
 if [[ -z "$(curl --connect-timeout 2 -m 1 --silent -f -I http://$HIPP | grep Apache-Coyote)" ]] ; then
# Opentaps/OFBiz is not running.  Let's bring it up.
        cd $OPATH ; if [ ! $PWD == $OPATH ] ; then echo -e "Failed to change directories on line $LINENO" | tee -a $EFILE ; mail_send ; exit 1 ; fi
        echo -e "\033[1mOpentaps/OFBiz service is currently down, bringing online...\033[0m"
        runuser -m $RUSER ./startofbiz.sh >> /dev/null
          while [[ -z "$(curl --connect-timeout 2 -m 1 --silent -f -I http://$HIPP | grep Apache-Coyote)" ]] ; do
          sleep 2
          let COUNTER+=1
             if [ $COUNTER == 30 ] ;then
                 # It has taken one minute to bring up, suspect failure, send e-mail.
                 echo -e "$(hostname) OFBiz/Opentaps failed to start in 60 seconds" | tee -a $EFILE
                 mail_send
                 exit 1
             fi
    echo -e "caught $COUNTER" >> test1.txt
          echo -n "."
          done
          echo -e "\033[0;32mOK\033[0m"
      if [ $DEBUG -eq "1" ] ; then cat /usr/local/opentaps/runtime/logs/console.log | more ; fi
 else
 echo -e "\033[1mOpentaps/OFBiz is already running....\033[0m"
 fi
}

stop() {
 if [[ -n "$(curl --connect-timeout 2 -m 1 --silent -f -I http://$HIPP | grep Apache-Coyote)" ]] ; then
    # Opentaps/OFBiz is running.... Let's stop it...
        cd $OPATH ; if [ ! $PWD == $OPATH ] ; then echo -e "Failed to change directories on line $LINENO" | tee -a $EFILE ; mail_send ; exit 1 ; fi
        echo -e "\033[1mOpentaps/OFBiz service is currently online, taking offline...\033[0m"
        runuser -m $RUSER ./stopofbiz.sh >> /dev/null
          while [[ -n "$(curl --connect-timeout 2 -m 1 --silent -f -I http://$HIPP | grep Apache-Coyote)" ]] ; do
            echo -n "."
            sleep 2
            let COUNTER+=1
               if [ $COUNTER == 30 ] ;then
              # It has taken one minute to bring up, suspect failure, send e-mail.
                   echo -e "$(hostname) OFBiz/Opentaps failed to stop in 60 seconds" | tee -a $EFILE
                   mail_send
                   exit 1
                  fi
          done
          echo -e "Result:  \033[0;32mOK\033[0m"
      if [ $DCACHE -eq "1" ] ; then sync; echo 3 > /proc/sys/vm/drop_caches | more ; fi
 else
   echo -e "\033[1mOpentaps/OFBiz is already stopped, please start....\033[0m"
 fi
}

ostatus() {
  STAT=$(ps -ef |grep "ofbiz.jar" | grep -v "grep" | awk '{ print $2}')
  if [[ -n $STAT ]] ; then
    echo -e "\033[1mOFBiz/Opentaps is running with pid:\033[0m \033[0;32m $STAT \033[0m"
    echo -e "\033[1mCurrent connection total:\033[0m" $(netstat -an |grep $(echo '$HIPP' | cut -d ":" -f2) | wc -l)
  else
    echo -e "\033[4;31mOFBiz/Opentaps service is OFFLINE\033[0m"
  fi
}

ostats() {
  STAT=$(ps -ef |grep "ofbiz.jar" | grep -v "grep" | awk '{ print $2}')
  if [[ -n $STAT ]] ; then
    echo -e "\033[1mOFBiz/Putting together open taps report for pid:\033[0m \033[0;32m $STAT \033[0m"
    echo -e "\033[1mCurrent connection total:\033[0m $(netstat -an |grep $(echo '$HIPP' | cut -d ":" -f2) | wc -l)"
    echo -e "\033[1mTotal process $STAT memory usage:\033[0m $(ps aux | grep $STAT | grep -v "grep" | awk '{ s += $6 } END { print s/1024, "Mb"}')"
    echo -e "\033[1mCurrent CPU utilization (percent):\033[0m  $(ps -eo %mem,cmd|sort -k2 -r | grep ofbiz | cut -d " " -f1)"
    echo -e "\033[1mThe process started on:\033[0m $(ls -ld /proc/$STAT | awk '{print $6,$7,$8}')"
  else
    echo -e "\033[4;31mOFBiz/Opentaps service is OFFLINE\033[0m"
  fi
}


case $1 in

start)
  start
;;
stop)
  stop
;;
restart)
  stop
  start
;;
status)
  ostatus
;;
stats)
  ostats
;;
  *)
    echo $"Usage: $0 {start|stop|restart|status|stats}"
    exit 1

# Cleanup
esac
exit 0

rSync cron script with error handling.

Posted in Bash Scripts on February 22nd, 2011 by admin – Be the first to comment

This is a simple rSync script I wrote in bash, has error handling and e-mail functionality, used primarily for automated/cron jobs.

#!/bin/sh
# Objective:  Handles rSync
# Author Russ Thompson @ viGeek.net
# This script assumes you're using password-less login (via keys).  User/PASS can be added easily via opts.

# Log & Email parms...
LOGF="$(pwd)/rsync-results.log"
ETO"whatever@email.com"
ESUB="$(hostname -s) - rSync - FAILURE"
EFILE="tmpemail.txt"

# Make initial log entry...
echo "rSyncd script kicked off at $(date) by $(whoami)" >> $LOGF

# Remote host machine name (supports multiple entries)
REMOTE_SERVERS="69.69.69.69"

# Directory to copy from on the source machine.
SRCDIR="/path/to/source/"

# Directory to copy to on the destination machine.
DESTDIR="/path/to/destination"

# Path to SSH
SSH=/usr/bin/ssh

# Does copy, but still gives a verbose display of what it is doing (CHANGE THESE)
OPTS="-rptgvv --remove-sent-files --delete-after"

# This deletes rsync-results.log after xyz (to prevent buildup)
find $LOGF -mtime +30 -exec rm {} \;
# Can also be deleted via size paramaters

function trap_clean {
    # Error handling...
    echo -e "$(hostname) caught error on line $LINENO at $(date +%l:%M%p) via script $(basename $0)" | tee -a $EFILE $LOGF
    echo -e "Please see the tail end of $LOGF for additional error details...">> $EFILE
    /bin/mail -s "$ESUB" "$EADDY" < $EFILE
    # Cleanup our temp e-mail file
    rm -f $EFILE
}

# Defined trap conditions
trap trap_clean ERR SIGHUP SIGINT SIGTERM

# Determine that files exist
if ls $SRCDIR ; then
  for REMOTE in $REMOTE_SERVERS
  do
      VAR=`ping -s 1 -c 1 $REMOTE > /dev/null; echo $?`
      if [ $VAR -eq 0 ]; then
      echo "rsync $OPTS $REMOTE::$SRCDIR $DESTDIR"
      rsync -e "$SSH" $OPTS $SRCDIR $REMOTE::$DESTDIR 2>> $LOGF
  else
      echo "Cannot connect to $REMOTE." >> $LOGF
      fi
  done
else
  echo "rSync script did not run : no files present" >> $LOGF
fi

# Fill log with line for easy reading...
echo "--------------------------------------------------------------------------------------------------------------" >> $LOGF

mySQL Backup Script.

Posted in Bash Scripts on February 22nd, 2011 by admin – Be the first to comment

The following script will backup all of the specified databases, create a report and throw them in a tar.gz. Script features error handling, e-mail notifications and easily customizable. I wrote this several years ago, so it’s probably due for an update soon…

#!/bin/sh
## MySQL Makeover Backup Script v1.0
## Authored By Russ Thompson 2008 @ viGeek.net

# Establish destination directory
DESTDIR="/pwd/whatever"

# Define log file ensure writable
LOGFILE="/var/log/dbbackup.log"

# Establish output file name
FONAME="db_$(hostname)_$(date +%m-%d-%Y).tar.gz"

# Establish Databases To Backup
# For multiple databases seprate via space
DBACKUP="db1 db2"

# mySQL Database Credentials (can store in users my.cnf)
# Only user with 'read only' ensure this file script chmoded correctly
# Better security... use shcrypt to secure or keyless entry.
DUSER="username"
DPASS="password"

# Define e-mail attributes for error handling messages
EADDY="email@whatever.com"
ESUB="mySQL Backup Error on $(hostname)"

# Seperation
echo -e "--------------------------------------------------------------------------------------------------------" >> $LOGFILE

# Pregame check if we are the active  (for clustered DRBD systems)
# Simply checks to ensure we are running.
if [ -e /var/lib/mysql/mysql.sock ]
  then
     echo -e "$(hostname) is active beginning backup procedure..." >> $LOGFILE
  else
     echo -e "$(hostname) is not active, aborting..." >> $LOGFILE
     exit 1
fi

# Kickoff
echo -e "$(date +%D) $(basename $0) has started at: $(date +%l:%M%p)" >> $LOGFILE

# Define Error Handling.
function clean_up {
        #Perform actions on errors.
        EMESSAGE="tmpout.txt"
        echo -e "$(hostname) caught error on line $LINENO at $(date +%l:%M%p) via script $(basename $0)" >> $EMESSAGE
        /bin/mail -s "$ESUB" "$EADDY" < $EMESSAGE
        echo -e "Caught error on $LINENO at $(date +%l:%M%p)" >> $LOGFILE
        rm -f $EMESSAGE
        exit 1
}
# Define our trap conditions
trap clean_up ERR SIGHUP SIGINT SIGTERM

# Change into working directory
cd $DESTDIR

echo -e "Changed into directory $(pwd)" >> $LOGFILE

# Before we continue ensure we're in the right place

if echo $(pwd)| grep $DESTDIR >> /dev/null
        then
        echo -e "Good" >> /dev/null
else
        echo -e "ERROR: expecting directory $DESTDIR however in $(pwd)" >> $LOGFILE
        clean_up
fi

# Remove the previous backup
rm -f *.gz

# Begin creating SQL backup files.
for i in $DBACKUP
        do
        mysqldump --user $DUSER --password=$DPASS $i > $i.sql
        echo -e "Created mySQL dump for database $i" >> $LOGFILE
done

# Create simple report to include in backup
REPNAME="backup-report.txt"
echo -e "Database backups expecting the following files:" > $REPNAME
echo -e "$DBACKUP" >> $REPNAME
echo -e "\nGetting the following files:" >> $REPNAME
echo -e "$(ls)" >> $REPNAME
echo -e "The following files are getting backed up:" >> $LOGFILE
echo -e "$(ls)" >> $LOGFILE
# Create our GNU Zip
tar -czvf $FONAME *
echo -e "Created backup file with a size of $(du -h $FONAME)" >> $LOGFILE

# Cleanup the mess.
for i in $DBACKUP
        do
        rm -f $i.sql
done

# Finish
echo -e "$(basename $0) has finished at: $(date +%l:%M%p)" >> $LOGFILE

SSH Root Logon Notification Script.

Posted in Bash Scripts on February 22nd, 2011 by admin – Be the first to comment

This is a very simple script I wrote several years ago, a client wanted to be notified when a user logged in and switched to root. They also wanted some other basic information included, this was the quick solution I put together.

Login to your server and switch to root.

nano /root/.bashexec

Paste the following code in this file.

#!/bin/bash
tmpout=tmpout.txt
stringz="$(tail -n 1 /var/log/secure | grep root | grep opened | sed 's/.*by\(.*\)(.*/\1/' | awk '{print $1""$2}')"
echo -e "The local root account has been accessed by user $stringz" > $tmpout
echo -e "\nThe system last updated on: $(sed -n '/Updated:/h;${;g;p;}' < /var/log/yum.log | cut -dU -f1)" >> $tmpout
echo -e "\nThe last five users to access the system (including active):" >> $tmpout
echo -e "$(last -n 5 | sed '/^wtmp/d')" >> $tmpout
echo -e "\nUptime Report: $(uptime)" >> $tmpout
/bin/mail -s "$(hostname) root account accesssed by $stringz" youremail@whatever.com < $tmpout
rm -f $tmpout

Now execute it via .bashrc

nano /root/.bashrc

Paste the following.

sh /root/.bashexec