Wednesday 23 March 2016

Critical Server Alert Monitoring Through Hipchat App

What is HipChat:

HipChat is a Web service for internal/private chat and instant messaging. As well as one-on-one and group/topic chat, it also features cloud-based file storage, video calling, searchable message history and inline image viewing.

How we implemented HipChat in our Environment:

1.     Signup in HipChat.com using common email id [italerts@gmail.com]

2.     Created one chat Room [gai-italerts.hipchat.com]. Security=private

3.     Downloaded HipChat API to one of our monitoring server [alps]

Install HipChat API:
            [alps]# mkdir /usr/local/hipchat
            [alps]# cd /usr/local/hipchat
[alps]# unzip master
[alps]# vim /etc/hipchat
HIPCHAT_TOKEN=Shxfn45jWM4x70KoVWfDVjAcEEgQdk8DonZe7GaL
HIPCHAT_ROOM_ID=2566040
:wq

Here: HIPCHAT_TOKEN = Created from HipChat.comàLoginàasàAdminàGroup AdminàAPIàEnter the Admin PasswordàUnder Create New Tokenà
            Type: Notification
            Label: italertstoken
         HIPCHAT_ROOM_ID = is Rooms->API ID
      
4.     Create script for monitoring in Server [alps]

root@alps /]# cat /root/scripts/hipchat_alert.sh
#!/bin/bash
for server in server1.domain.com server2.domain.com server3.domain.com server4.domain.com server5.domain.com server6.domain.com
do
packet=`ping -c 3 -f $server | grep "packet" | awk -F% '{print $1}' | awk '{print $NF}'`

if [ $packet -eq 100 ]

then

/usr/local/hipchat/hipchat-cli-master/hipchat_room_message -i "$server NOT REACHABLE" -f zabbix.gai.net -n @all

fi

done
[root@alps /]#

5.     Schedule the Script in Cron:

###HIPCHAT alert created by Mansoor##
*/3 * * * * /bin/bash /root/scripts/hipchat_alert.sh > /tmp/hipchat_alert.txt

6.     Send Invitation to Users to Join into Group: Group AdminàUsersàAdd

7.     Install Mobile App from PlayStore & Desktop App [HipChat] from HipChat.com website

8.     Enable the Stay connected, Room Messages, One-to-One messages, Room mentions in Mobile App

9.     Add this HipChat Mobile App into “Protected Apps”, so that it won’t go offline


Extra:

 To send alert to All Room members

#/usr/local/hipchat/hipchat-cli-master/hipchat_room_message -i "$server NOT REACHABLE" -f zabbix.gai.net -n @all
OR
#echo “Server is Not Reachable” | /usr/local/hipchat/hipchat-cli-master/hipchat_room_message -f zabbix.gai.net -n @all

To send alert to All Specific Room members
#echo “Server is Not Reachable” | /usr/local/hipchat/hipchat-cli-master/hipchat_room_message -f zabbix.gai.net -n @mdmansoor