Bash script to alert when memory gets low

We have a web server that’s running out of memory about once every couple of weeks. The problem is that when it happens the swap file is totally full, the system is unresponsive and it usually needs a hard reboot. So it’s a bit difficult to debug. To avoid digging through log files I don’t understand I elected to put a script in /etc/cron.hourly which checks the total amount of free memory (including swap and physical). If there is less than 256mb free (this server has 512mb of ram and a 1gb swap so at this point the situation is serious), it dumps the process list to /tmp/processes.txt and sends me an email with it attached.

Note that mutt must be installed (‘apt-get install mutt’ on Debian/Ubuntu, or ‘yum install mutt’ on RedHat/CentOS).

#!/bin/bash

free=`free -mt | grep Total | awk '{print $4}'`

if [ $free -lt 256 ]; then
        ps -eo %mem,pid,user,args >/tmp/processes.txt
        echo 'Warning, free memory is '$free'mb' | mutt -a /tmp/processes.txt -s "Server alert" [email protected]
fi

Then of course make it executable and symlink to cron.hourly:

chmod +x /etc/scripts/memalert.sh
ln -s -t /etc/cron.hourly/memalert.sh /etc/scripts/memalert.sh

9 thoughts on “Bash script to alert when memory gets low

  1. poisonbit

    Hello,

    Just a few tips for the next version:

    free -mt | grep Total | awk ‘{print $4}’

    is the same that:

    free -mt | awk ‘/Total/{print $4}’

    Also when I parse commands output, I use to set LANG=C into the script, to avoid locales (“Total” is not said “Total” in all languages)

    And in this case, we can avoid the mutt depend to attach files, and insert the txt into the mail body:

    
    #!/bin/bash
    LANG=C
    free=$(free -mt | awk '/Total/{print $4}')
    [ $free -lt 256 ] &mb"
        procs="$(ps -eo %mem,pid,user,args)"
        printf '%sn%s' "$msg" "$procs" | mail -s "Server alert" [email protected]
    }
    

    Greetings :)

    Reply
  2. Hamish

    Hey Alex,

    I’ve used mutt too, as I don’t know how to use mailutils. However in my server I have a laptop HDD installed in the name of noise, whenever memory is running low and it uses swap, everything slows down a lot, so I have a warning system for when memory is being used.

    #!/bin/bash

    memfree=”`cat /proc/meminfo | grep MemFree | cut -d: -f2 | cut -dk -f1`”
    memfreemb=$(echo “scale=5; $memfree/1024” | bc -l| awk -F. ‘{if(($2/10^length($2)) >= .5) printf(“%d.0n”,$1+1);else printf(“%d.0n”,$1)}’)

    if [ $memfree -lt 204800 ]; then
    ps -eo %mem,pid,user,args >/tmp/processes.txt
    echo ‘Warning, free memory is ‘$memfreemb’mb’ | mutt -a /tmp/processes.txt -s “Server Alert” — [email protected]
    fi

    Reply
    1. Alex

      Wow this seems old now. Cringing at my lack of bash (and Linux) skills.

      Kernc, the -o option tells you this, the first column is “%mem”, i.e %age of memory used. Now I would probably do ps -eo %mem,pid,user,args | sort -rn, so that the processes are sorted in order of memory usage. And of course pipe it straight to mail as suggested by poisonbit rather than attach a text file with mutt.

      Reply

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.