Learn Linux: 2014

Tuesday 16 December 2014

Mod_security setup against DDoS Attack

Version: mod_security2.6.72.el5

Apache version: 2.2.3

We can protect our website from hacker whenever they send the huge request using

mod_security module in Apache webserver. This module protection method is blocking IP

address if we receiving huge hits within particular interval.

If need, we can allow huge no. of request for particular IP address

If need, we can allow search engine websites using UserAgent method

Log file path: /var/log/httpd/modsec_audit.log & /var/log/httpd/modsec_debug.log

ModSecurity 2.x allows rules to be placed in one of the following five phases of the Apache

request cycle. We are using phase1 in our configuration:

Phase1: Request headers (REQUEST_HEADERS)

Phase2: Request body (REQUEST_BODY)

Phase3: Response headers (RESPONSE_HEADERS)

Phase4: Response body (RESPONSE_BODY)

Phase5: Logging (LOGGING)

For example:

Using below configuration, we can blocks users who send more than 20 requests in a 10 second

period from same IP. They will be blocked for 30seconds unless this has be a frequent

occurrence. If they were blocked more than five times within five minutes they will be blocked

for five minutes.

Add below lines in mod_security.conf file(Path: /etc/httpd/conf.d):

SecRule REMOTE_ADDR "^10.1.5.63$" "phase:1,t:none,allow,nolog,ctl:ruleEngine=off"

SecRule REMOTE_ADDR "^127.0.0.1$" "phase:1,t:none,allow,nolog,ctl:ruleEngine=off"

SecRule REQUEST_HEADERS:UserAgent "Googlebot"

SecRule REQUEST_HEADERS:UserAgent "Yahoo! Slurp"

SecRule REQUEST_BASENAME "!(css|doc|flv|gif|ico|jpg|js|png|swf|gz|pdf)$" "phase:1,nolog,pass,initcol:ip=%{REMOTE_ADDR},setvar:ip.requests=+1"

SecRule ip:requests "@le 2" "phase:1,nolog,expirevar:ip.requests=10"

SecRule ip:requests "@ge 20" "phase:1,pass,nolog,setvar:ip.block=1,expirevar:ip.block=30,setvar:ip.blocks=+1,setvar:ip.requests=0,expirevar:i

p.blocks=300"

SecRule ip:blocks "@ge 5" "phase:1,deny,log,logdata:'req/sec: %{ip.requests}, blocks: %

{ip.blocks}',status:403"

SecRule ip:block "@eq 1" "phase:1,deny,log,logdata:'req/sec: %{ip.requests}, blocks: %{ip.blocks}',status:403"

ErrorDocument 403 "<html><body><h2>Too many requests.</h2></body></html>"Rule Details:

Rule 1&2: Allowing huge no. of request for particular IP & local network. Here, you can

change you IP address instead of 10.1.5.63.

Rule 3&4: We are allowing huge no. of request for Google & Yahoo search engine websites

Rule 5: Ignoring media files, count requests made in past 10 seconds.

Rule 6: We want the var to expire and leave it alone. If we combine this with the

increments in the rule above, the timer never expires unless

there are absolutely no requests for 10 seconds.

Rule 7: if there were more than 20 requests in 10 seconds for this IP set var block to 1 (expires

in 30 seconds) and increase var blocks by one (expires in 5 minutes)

Rule 8: If user was blocked more than 5 times (var blocks>5), log and return http 403.

Rule 9: if user is blocked (var block=1), log and return http 403

Rule 10: Error message

More details:

UserAgent:

Since we should allow huge no. request for Search engine website like: Googlebot, Yahoo and etc.,

So we tried to allow these websites using domain based(eg., googlebot.com) but not able to do using

this method. But we can meet this requirement using Useragent.

When we check our website(LS) log file, we found below details:

Yahoo:

68.180.228.87 [26/Nov/2014:23:30:18 +0000] "GET /blog/secrettohealthyeating/ HTTP/1.1" 200

8490 515039 "" "Mozilla/5.0 (compatible; Yahoo! Slurp;

http://help.yahoo.com/help/us/ysearch/slurp)"

Google:

208.78.85.241 [25/Nov/2014:19:21:33 +0000] "GET / HTTP/1.1" 200 29633 421334 ""

"Googlebot/2.X (+http://www.googlebot.com/bot.html)"

98.85.37.192 [26/Nov/2014:00:13:19 +0000] "GET / HTTP/1.1" 200 29637 158050 "" "Mozilla/5.0

(compatible; Googlebot/2.1 +http://www.googlebot.com/bot.html)"

Here whenever they send the request to our webiste, they are using “Yahoo! Slurp & Googlebot”

Useragent. So we can allow them using below rule:Rule: SecRule REQUEST_HEADERS:UserAgent "ApacheBench" phase:1,nolog,allow,ctl:ruleEngine=off

Note: we tested using rule “SecRule REQUEST_HEADERS:UserAgent "ApacheBench"

phase:1,nolog,allow,ctl:ruleEngine=off”. If we send no. of request using ab command, at that time our

requests are allowed based on the Useragent.

Allowing IP:

We tested it in local network using IP address 10.1.5.63. If we send the huge request from this IP, all of

the requests are allowed and got the response code 200.

Allowing Media file :

We tested it too in local network. Created the gz format file and placed it in particular domain(eg:

http://abc.com/test.gz). When we send the no. request to this URL at that time also we got 200 response

code.

Log details :

If anyone send the no. request from Particular IP, that ip will be blocked and details stored in

modsec_audit.log & site access log in below format:

In modsec_audit.log:

d147a861A

[15/Dec/2014:05:19:24 +0530] xpMAYwoBBUEAACkxCdcAAAAF 10.1.5.63 58473 10.1.5.65 80

d147a861B

GET / HTTP/1.1

Host: gailoadtest.com

User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

AcceptLanguage: enUS,en;q=0.5

AcceptEncoding: gzip, deflate

Connection: keepalive

IfModifiedSince: Wed, 26 Nov 2014 09:21:57 GMT

IfNoneMatch: "c004d66b508bf8f269740"

CacheControl: maxage=0

d147a861F

HTTP/1.1 403 Forbidden

AcceptRanges: bytes

ContentLength: 4958

Connection: close

ContentType: text/html; charset=UTF8

d147a861H

Message: Access denied with code 403 (phase 1). Operator EQ matched 1 at IP:block. [file

"/etc/httpd/conf.d/mod_security.conf"] [line "104"] [data "req/sec: 0, blocks: 1"]

Action: Intercepted (phase 1)

Stopwatch: 1418600964620387 516 ( )

Stopwatch2: 1418600964620387 516; combined=157, p1=85, p2=0, p3=0, p4=0, p5=37, sr=38, sw=35, l=0, gc=0

Producer: ModSecurity for Apache/2.6.7 (http://www.modsecurity.org/).

Server: Apache/2.2.3 (CentOS)

d147a861ZIn access log:

10.1.5.60 [02/Dec/2014:17:19:00 +0530] "GET /index.html HTTP/1.0" 403 69 "" "ApacheBench/2.3"

Ref URL:

http://www.nytechgroup.com/2010/04/07/apachedenialofservicedosattack/

https://github.com/SpiderLabs/ModSecurity/wiki/ReferenceManual

https://www.atomicorp.com/wiki/index.php/Modsecurity_audit_log → Audit log details

NOTE: Above testing process completed in 10.1.5.65(vasu machine)

Friday 28 November 2014

Free IPA Active Directory Integration

Free IPA / ADS Users & Password Sync Document

IPA Server (ipaserver.gai.local) : 10.1.5.159 DNS1: 10.1.5.159CentOS 6

ADS Server (ad1.gai.local) : 10.1.5.158 DNS: 127.0.0.1 Windows 2008 R2

Step 1# Install IPA Server

# ipa-server-install

Enter the hostname. This is determined automatically using reverse DNS.
Server host name [ipaserver.gai.local]:
Enter the domain name. This is determined automatically based on the hostname.
Please confirm the domain name [gai.local]:
The script then reprints the hostname, IP address, and domain name.
The IPA Master Server will be configured with
Hostname: ipaserver.gai.local
IP address: 10.1.5.159
Domain name: gai.local
Enter the new Kerberos realm name. This is usually based on the domain name.
Please provide a realm name [gai.local]:
Enter the password for the Directory Server superuser, cn=Directory Manager. There are password strength requirements for this password, including a minimum password length.
Directory Manager password:
Password (confirm):
Enter the password for the FreeIPA system user account, admin. This user is created on the machine.
IPA admin password:
Password (confirm):
After that, the script configures all of the associated services for FreeIPA, with task counts and progress bars.
Configuring ntpd
[1/4]: stopping ntpd
...
done configuring ntpd.
Configuring directory server for the CA: Estimated time 30 seconds
[1/3]: creating directory server user
...
done configuring pkids.
Configuring certificate server: Estimated time 6 minutes
[1/17]: creating certificate server user
....
done configuring pki-cad.
Configuring directory server: Estimated time 1 minute
[1/32]: creating directory server user
...
done configuring dirsrv.
Configuring Kerberos KDC: Estimated time 30 seconds
[1/14]: setting KDC account password
...
done configuring krb5kdc.
Configuring ipa_kpasswd
[1/2]: starting ipa_kpasswd
[2/2]: configuring ipa_kpasswd to start on boot
done configuring ipa_kpasswd.
Configuring the web interface: Estimated time 1 minute
[1/12]: disabling mod_ssl in httpd
...
done configuring httpd.
Setting the certificate subject base
restarting certificate server
Applying LDAP updates
Restarting the directory server
Restarting the KDC
Restarting the web server
Sample zone file for bind has been created in /tmp/sample.zone.ygzij5.db
==============================================================================
Setup complete
Restart the SSH service to retrieve the Kerberos principal and to refresh the name server switch (NSS) configuration file:
# service sshd restart
Authenticate to the Kerberos realm using the admin user's credentials to ensure that the user is properly configured and the Kerberos realm is accessible.
# kinit admin
Password for admin@gai.local:
Test the FreeIPA configuration by running a command like ipa user-find. For example:
# ipa user-find admin
--------------
1 user matched
--------------
User login: admin
Last name: Administrator
Home directory: /home/admin
Login shell: /bin/bash
Account disabled: False
Member of groups: admins
----------------------------
Number of entries returned 1
----------------------------

Step 2# Install ADS Server

Hostname: ad1.gai.local

IP : 10.1.5.158 (IP Should be static)

Domain Name: gai.local

Step 2.1# ADS Installation

Start-->Server Manager-->Roles-->Add Roles--> Select Active Directory Domain Services --> Next --> Next (Intro for ADS)-->Install

Step 2.2# Reboot

Reboot Server After Installation

Step 2.3# ADS configuration

Start--> dcpromo.exe --> Enter--> Next (Welcome ADS Wizard) --> Next(OS compatibily) -->Select Create a new domain in new forest -->Next -->Enter domain as “gai.local” -->Next-->Select Forest functional level as “Windows Server 2008” --> Next--> Ticket DNS Server --> Next-->A delegation prompt will come click YES-->Next (for c:\windows\ directory location)->enter password for ads administrator account --> Next --> Next (Summary)

Step 2.4# Reboot

Reboot Server After dcpromo Configuration

Step 2.5# Creating CA Certificate for ADS

Start-->Server Manager-->Roles-->Add Roles-->Active Directoy Certificate Services -->Next-->Next-->Select Certificate Authoriy-->Next-->Select Enterprise-->Next-->Select Root CA-->Next-->Create a new private key-->Next(algorithm selection)-->10 Years -->Next-->Next-->Install

Step 2.6# Reboot

Reboot Server After dcpromo Configuration

Step 2.7# Export ADS CA

Start-->Certificate Authority-->Select AD1.gai.local CA -->Right Click-->Properties-->View Certificate-->Details-->Copy to File-->Next(export Wizard)-->Select Base-64 encoded x.509(.CER)-->Next-->Browse (give any file location with file name example: c:\temp\ADCA) -->Next-->Save-->

Step 2.8# Install IPA (.crt) Certificate in ADS

Copy IPA:/etc/ipa/ca.crt to ADS:c\tmp\

Go to ADS:c:\tmp\ Right Click on ca.crt-->Install Certificate-->Next-->Select Place all certificates in the following store-->Browse-->Select Trusted Root Certification Authorities-->Ok-->Next-->Finish-->Yes-->Ok (Import was successfull)

Step 2.8# Tranfer AD CA to IPA

copy ADS:c:\temp\ADCA.cer to IPA:/etc/openldap/cacerts (mkdir /etc/openldap/cacerts if not exists)

copy IPA:/etc/ipa/ca.crt to to IPA:/etc/openldap/cacerts

#cacertdir_rehash /etc/openldap/cacerts

#vim /etc/openldap/ldap.conf

#Add the below line

TLS_CACERTDIR /etc/openldap/cacerts

TLS_REQCERT allow

Step 2.9 # DNS Resolve

From ADS# ping ipaserver.gai.local should ping successfully (10.1.5.159)

From ADS# ping ad1.gai.local should ping successfully (10.1.5.158)

From IPA# ping ipaserver.gai.local should ping successfully (10.1.5.159)

From IPA# ping ad1.gai.local should ping successfully (10.1.5.158)

Step 2.10 # Search ADS Ldap users from IPA (so that the we can confirm the connection from IPA to ADS)

ldapsearch -x -D "cn=administrator,cn=users,dc=gai,dc=local" -h ad1.gai.local -W -b dc=gai,dc=local

Enter LDAP Password: (Enter ADS administrator password)

#Connection check via TLS (-ZZ)

ldapsearch -x -D "cn=administrator,cn=users,dc=gai,dc=local" -h ad1.gai.local -W -b dc=gai,dc=local -ZZ

Enter LDAP Password: (Enter ADS administrator password)

Step 3# Creating Users in ADS before sync

ADS: Go to -->Active Directory Users & Computers --> Go to Users --> Rightclick --> New--> User -- > Create User here (Please note that First Name & Last Name should be entered because IPA Will replicate only First Name & Last Name present users only from ADS)

Step 4# Start Replication between IPA & AD

#ipa-replica-manage connect --winsync --binddn cn=administrator,cn=users,dc=gai,dc=local --bindpw Desk13@ga! --passsync secretpwd --cacert /etc/openldap/cacerts/ADCA.cer ad1.gai.local -v

Result:

Added CA certificate /etc/openldap/cacerts/ADCA.cer to certificate database for ipaserver.gai.local
ipa: INFO: AD Suffix is: DC=gai,DC=local
The user for the Windows PassSync service is uid=passsync,cn=sysaccounts,cn=etc,dc=gai,dc=local
ipa: INFO: Added new sync agreement, waiting for it to become ready . . .
ipa: INFO: Replication Update in progress: FALSE: status: 0 Replica acquired successfully: Incremental update started: start: 0: end: 0
ipa: INFO: Agreement is ready, starting replication . . .
Starting replication, please wait until this has completed.
Update succeeded
Connected 'ipaserver.gai.local' to 'ad1.gai.local'

Note: Note above –passsync password which is useful when doing password sync setup in ADS & user for the Windows PassSync service. Both highlighted in Red Color

Step 5# Check the above Created ADS User is replicated to IPA or not

#ipa user-show username_above_created

Step 6# Trust the IPA ca.crt certificate in ADS Password Sync services

Start-->cmd-->Open as administrator

Go to below location cd “C:\Program Files\Red Hat Directory Password Synchronization\”

C:\Program Files\Red Hat Directory Password Synchronization> certutil.exe -A -d . -i “c:\tmp\ca.crt” -n “IPA CA” -t CT,,

To list the certificate: certutil.exe -L -d .

Passsync log file: C:\Program Files\Red Hat Directory Password Synchronization\Passsync.txt

Step 6.1# Install RedHat-PassSync-1.1.5-x86_64.msi in ADS and start the Password Sync Service

Install the RedHat-PassSync-1.1.5-x86_64.msi

1. Host Name: Give IPA Server hostname

2. Port : 636

3. User Name: You should find this from ipa-replica-mange command result and paste it here directly

4. Password: Password is secretpwd which you give above ipa-replica-mange command –passsync 'secretpwd'

5. Cert Token : Leave blank

6. Search Base: Use as above cn=users,cn=accounts,dc=gai,dc=local

Step 6.2 # Reboot the ADS Server

Step 7 # Reset the Password in ADS and check it via login IPA client

Configure IPA client in one linux machine and login it via IPA username/password then change the password for the same user in ADS, then again login the ipa client with new password.

Monday 27 October 2014

Apache Hardening

Change Apache Server Signature with mod_security

vim /etc/httpd/conf/httpd.conf
ServerTokens ProductOnly
ServerSignature Off

#Restart apache

[root@GAIDL-5013 ~]# curl -LI localhost
HTTP/1.1 302 Found
Date: Mon, 27 Oct 2014 14:39:40 GMT
Server: Apache
Location: http://localhost/ovirt-engine/
Connection: close
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 503 Service Temporarily Unavailable
Date: Mon, 27 Oct 2014 14:39:40 GMT
Vary: Accept-Encoding
Connection: close
Content-Type: text/html; charset=iso-8859-1

###To hide the above Server: Apache follow the below settings##

#yum install mod_security
#vim /etc/httpd/conf.d/mod_security.conf

##Add the below two lines after   SecRuleEngine On directive###
    ServerTokens Full
    SecServerSignature Mansoor_Web_Server

#Restart apache

[root@GAIDL-5013 ~]# curl -LI localhost
HTTP/1.1 302 Found
Date: Mon, 27 Oct 2014 14:41:21 GMT
Server: Mansoor_Web_Server mod_perl/2.0.4 Perl/v5.10.1
Location: http://localhost/ovirt-engine/
Connection: close
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 503 Service Temporarily Unavailable
Date: Mon, 27 Oct 2014 14:41:21 GMT
Vary: Accept-Encoding
Connection: close
Content-Type: text/html; charset=iso-8859-1

Reference: http://www.queryadmin.com/206/change-apache-server-signature-mod_security/ & http://www.howtoforge.com/changing-apache-server-name-to-whatever-you-want-with-mod_security-on-debian-6

Thursday 18 September 2014

How to log bash (all users or specific user) history to syslog

There are two methods to get this thing done.

1. Trap method - drop the following snippet into either the per-user or system-wide bash profile (~/.bash_profile or /etc/profile, respectively)
unset HISTSIZE HISTFILESIZE export HISTTIMEFORMAT='%F %T' function log2syslog
{
declare COMMAND
COMMAND=$(fc -ln -0)
logger -p local1.notice -t bash -i -- "${USER}:${COMMAND}"
}
trap log2syslog DEBUG

2. Prompt Method - this method logs by hacking the prompt command to call history and write to syslog.
PROMPT_COMMAND='history -a >(tee -a ~/.bash_history | logger -t "$USER[$$] $SSH_CONNECTION")'

Tuesday 16 September 2014

Process which using high swap memory in linux

Here the threshold is 40% . If any process using swap memory more than 40% we will print the process id, TotalSwap, SwapUsed, SwapUsed in percentage.

Note: Below script only works where the VmSwap line prints in /proc/processid/status file (Works in RHEL/Centos 6 2.6.32-279.2.1.el6.centos.plus.x86_64)

#!/bin/bash
##Author == Md. Mansoor
for id in `ps aux | sort -k 4 -k 3 -nr | head -n5 | awk '{print $2}'`
do
swapkb=`cat /proc/$id/status | grep -i swap | awk '{print $2}'`
swapusedMB=`echo $((swapkb/1024))`
total=`free -m | awk 'FNR == 4 {print $2}'`
foutyper=`bc <<< $total*40/100`
swapusedper=`bc -l <<< $swapusedMB/$total*100 | awk -F. '{print $1}'`
if [ $swapusedMB -gt $foutyper ]; then
user=`ps aux | grep $id | grep -v grep | awk '{print $1}'`
cmd=`cat /proc/$id/cmdline`
echo "Above 40% swap used ProcessID = $id TotalSwap = $total SwapUsed = $swapusedMB SwapUsedin% = $swapusedper% MB User = $user Command = $cmd"
fi
done

##Another Script:
##This works for all linux version including old kernel
##Server running lower than procpu-3.2.7-26.rpm package should instead of above use below script. Reference: https://rhn.redhat.com/errata/RHBA-2013-1338.html

#! /bin/bash
# Author: Md. Mansoor
for i in /proc/[0-9]*; do
pid=$(echo $i | sed -e 's/\/proc\///g')
#echo $pid
swap_pid=$(cat /proc/$pid/smaps | awk 'BEGIN{total=0}/^Swap:/{total+=$2}END{print total}')
total=`free -m | awk 'FNR == 4 {print $2}'`
swapusedMB=`echo $((swap_pid/1024))`
#echo "swap used in MB $swapusedMB MB"
swapusedper=`bc -l <<< $swapusedMB/$total*100 | awk -F. '{print $1}'`
#echo "$swapusedper in percentage"
foutyper=`bc <<< $total*10/100`
user=`ps aux | grep $pid | grep -v grep | awk '{print $1}'`
if [ $swapusedMB -gt $foutyper ]; then
cmd=`cat /proc/$pid/cmdline`
echo "Above 40% swap used ProcessID = $pid TotalSwap = $total SwapUsed = $swapusedMB SwapUsedin% = $swapusedper% MB User = $user Command = $cmd"
fi
done

Reference: http://www.quora.com/How-can-I-determine-which-process-is-contributing-to-paging-on-Linux

Wednesday 10 September 2014

Memcached Installation in RHEL/CentOS 5/6

cd /usr/local/src
mkdir -p /usr/local/libevent
wget
https://github.com/downloads/libevent/libevent/libevent-2.0.21-stable.tar.gz
tar xzvf libevent-2.0.21-stable.tar.gz
cd libevent-2.0.21-stable
./configure --prefix=/usr/local/libevent
make
make install

cd /usr/local/src
mkdir -p /usr/local/memcached
wget http://memcached.org/latest
tar -zxvf memcached-1.x.x.tar.gz
cd memcached-1.x.x
./configure --prefix=/usr/local/memcached --with-libevent=/usr/local/libevent/
make
make install

cd /usr/local/memcached/bin
./memcached -d -u ukpdl -m 100 -p 10001

Close ticket like below:

Installed path of memcached : /usr/local/memcached
Memcache running Port          : 10001
Memcache running User         : ukpdl
Memcache memory allocated    : 100MB

Amazon glacier Installation in RHEL/CentOS 6

1. Download Python & boto from below path

cd /usr/local/src/
https://github.com/boto/boto/downloads/boto-2.6.0.tar.gz
http://www.python.org/getit/releases/2.7.5/Python-2.7.5.tgz

2. Install python
yum install *gcc* [ for python support ]
yum install openssl-devel [ to prevent this error : AttributeError: 'module' object has no attribute 'HTTPSConnection' ]
tar xzvf Python-x.x.x.x
cd Python.x.x.x.x
./configure --disable-ipv6
make
make install

3. Install boto
tar xzvf boto-x.x.x
cd boto-x.x.x
python setup.py install

4. Configure boto

##create below file and add the lines

vim ~/.boto
[Credentials]
aws_access_key_id=Enter here
aws_secret_access_key=Enter here

[glacier]
region=us-east-1
logfile=~/.glacier-cmd.log
loglevel=INFO
output=prin

[Boto]
https_validate_certificates = False

5. Export variables
#export LD_LIBRARY_PATH=/usr/local/lib/python2.7
#export PATH=/usr/local/lib/python2.7/site-packages/boto:$PATH

6. Check the command
#glacier valuts

7. glcier-cmd Install

Amazon Glacier CLI
==================

Command line interface for Amazon Glacier. Allows managing vaults, uploading
and downloading archives and bookkeeping of creat

#cd /usr/local/src
#yum install python-setuptools [ dependency for glacier-cmd ]
#wget http://python-distribute.org/distribute_setup.py
#python distribute_setup.py
#easy_install pip
#wget https://github.com/uskudnik/amazon-glacier-cmd-interface/amazon-glacier-cmd-interface-master.zip
#unzip amazon-glacier-cmd-interface-master.zip
#cd amazon-glacier-cmd-interface-master
#python setup.py install

8. Create file for glacier-cmd and add the below lines

vim /etc/glacier-cmd.conf
[aws]
access_key=Enter here
secret_key=Enter here

[glacier]
region=us-east-1
logfile=~/.glacier-cmd.log
loglevel=INFO
output=print

9. Commands for glacier please refer below
/usr/local/src/amazon-glacier-cmd-interface-master/README.md
Reference: https://github.com/uskudnik/amazon-glacier-cmd-interface/blob/master/doc/Usage.rst

Puppet Master and Agent Install RHEL/CentOS 6

Master: alps.gai.net

Agent:

gai-1234.gai.net
(192.168.10.22)

Puppet
dashboard : http://alps.gai.net:3000
Username: puppet
Password:
puppet123

Pre-requisites: Configure NTP in Master & Agents

Step 1# Master Install & Configure:

##Install
Puppet Repo

#rpm
-ivh
https://yum.puppetlabs.com/el/6/products/x86_64/puppetlabs-release-6-7.noarch.rpm

#yum
install puppet-server

##Configure
Master

#vim

/etc/puppet/puppet.conf

[master]

certname=alps.gai.net

dns_alt_names=gateway.gai.net

Note:

dns_alt_names:


A list of valid hostnames for the master, which will be embedded in
its certificate. Defaults to the puppet master’s certname and
puppet, which is usually fine. If you are using a non-default
setting, set it

before


starting the puppet master for the first time.

Certname:


The name to use when handling certificates. Defaults to the fully
qualified domain name.

Default

:
(the system’s fully qualified domain name)

#/etc/init.d/puppetmaster
start

Step
2# Agent Install & Configure

##Install
Puppet Repo

#rpm
-ivh
https://yum.puppetlabs.com/el/6/products/x86_64/puppetlabs-release-6-7.noarch.rpm

#yum
install puppet

##Configure
Agent

#vim

/etc/puppet/puppet.conf

[agent]

    server=alps.gai.net

    report=true

    pluginsync=true

    certname=gai-1234.gai.net

    runinterval=60m

Here:

runinterval
= for applying configuration with master (This will update
configuration every 1hr with master, change based on your
requirement)

#/etc/init.d/puppet
start

Step
3##Creating & Sigining SSL Certificate for Agent in Master##

Note:
The Master and Agent connections running in SSL hence we have to sign
the agent certificate request in Master. Do the below steps for it.

Agent#
puppet agent --test

Master#puppet
cert list (#Once enter this command, you will see the agent FQDN)

##Sign
the Agent certificate:##

Master#puppet
cert sign xxxxx (Here append xxxxx as above command result client
FQDN)

##Checking
Agent###

Agent#
puppet agent --test (This will provide as Applying configuration with
some version number)

Step
4# Testing Master Agent:

In
Master:

vim
/etc/puppet/manifests/site.pp

##Add
the below lines

class
test_class {

   file
{ "/tmp/testfile":

      ensure
=> present,

      mode
  => 644,

      owner
 => root,

      group
 => root

#
tell puppet on which client to run the class

node
'

gai-1234.gai.net

'
{

    include
test_class

}

In
Agent:

#puppet
agent --test

Now
check /tmp/testfile will be created in Agent node.

Step
5# Installing Puppet Dashboard in Master server

Master#yum
install mysql mysql-server puppet-dashboard

#cp
/usr/share/puppet-dashboard/config/database.yml.example
/usr/share/puppet-dashboard/config/database.yml

#vim
/usr/share/puppet-dashboard/config/database.yml

#Modify
the lines as below

production:

  database:
dashboard_production

  username:
dashboard

  password:

secret_password

(Note:
this password should be same as below mysql user password)

  encoding:
utf8

  adapter:
mysql

#cp
/usr/share/puppet-dashboard/config/settings.yml.example
/usr/share/puppet-dashboard/config/settings.yml

#vim
/usr/share/puppet-dashboard/config/settings.yml

##Modify
lines as below

ca_server:
'alps.gai.net'

ca_port:
8140

enable_inventory_service:
true

inventory_server:
'alps.gai.net'

inventory_port:
8140

use_file_bucket_diffs:
true

file_bucket_server:
'alps.gai.net'

file_bucket_port:
8140

##Creating
Database & User & Tables

CREATE
DATABASE dashboard CHARACTER SET utf8;

CREATE
USER 'dashboard'@'localhost' IDENTIFIED BY '

secret_password';

GRANT
ALL PRIVILEGES ON dashboard.* TO 'dashboard'@'localhost';

flush
privileges;

#vim
/etc/mysql/my.cnf

#Add
below line

max_allowed_packet
= 32M

##Once
Dashboard has its database, it can create its tables, but this has to
be done manually with the db:migrate rake task. For typical use with
the production environment:

##Run
the below commands in terminal

#rake
RAILS_ENV=production db:migrate

##Testing
That Dashboard is Working

#sudo
-u puppet-dashboard ./script/server -e production (use the below
service instead of this)

or

/etc/init.d/puppet-dashboard
start

/etc/init.d/puppet-dashboard-workers
start

browse
and connect as http://alps.gai.net:3000

##Configuring
Puppet

Using
Dashboard for Reports

For Dashboard to receive reports, there are two requirements:

All agent nodes have to be configured to submit reports to the master.
The master has to be configured to send reports to Dashboard.

Make
sure that all agents have reporting turned on:

#
puppet.conf (on each agent)

[agent]

    report
= true

#
puppet.conf (on puppet master)

[master]

    reports
= store, http

    reporturl
= http://dashboard.example.com:3000/reports/upload

#Restart
services (Master)

#/etc/indi.d/puppetmaster
restart

#/etc/indi.d/puppet-dashboard
restart

#/etc/indi.d/puppet-dashboard-workers
restart

#Restart
services (Agent)

#/etc/indi.d/puppet
restart

##Enabling
Inventory for puppet aent

Master#
vim /etc/puppet/auth.conf

#Go
to end of file#

##Disable
below lines

path
/

auth
any

#Add
below lines

path
/facts

auth
any

method
find, search

allow
*

#Restart
services

#/etc/init.d/puppetmaster
stop

#/etc/init.d/puppetmaster
start

#/etc/init.d/puppetmaster-dashboard
stop

#/etc/init.d/puppetmaster-dashboard
start

#/etc/init.d/puppetmaster-dashboard-workers
stop

#/etc/init.d/puppetmaster-dashboard-workers
start

Now
browse: http://puppetmasterip:3000
check the inventory tab

Step
6# ##Running Puppet Master using apache instead of services
(puppetmaster)

Refer:

http://docs.puppetlabs.com/guides/passenger.html

#yum
install httpd httpd-devel mod_ssl ruby-devel rubygems gcc

#Install
Rack/Passenger

#sudo
gem install rack passenger

#sudo
passenger-install-apache2-module

#sudo
mkdir -p /usr/share/puppet/rack/puppetmasterd

#sudo
mkdir /usr/share/puppet/rack/puppetmasterd/public
/usr/share/puppet/rack/puppetmasterd/tmp

#sudo
cp /usr/share/puppet/ext/rack/config.ru
/usr/share/puppet/rack/puppetmasterd/

#sudo
chown puppet:puppet /usr/share/puppet/rack/puppetmasterd/config.ru

#chown
puppet:puppet /usr/share/puppet/rack/puppetmasterd/config.ru

#chown
apache:apache /usr/share/puppet/rack/puppetmasterd/public -R

#chown
apache:apache /usr/share/puppet/rack/puppetmasterd/tmp -R

#vim
/etc/httpd/conf.d/puppetmaster.conf

#Add
below lines

#
This Apache 2 virtual host config shows how to use Puppet as a Rack

#
application via Passenger. See

#
http://docs.puppetlabs.com/guides/passenger.html for more
information.

#
You can also use the included config.ru file to run Puppet with other
Rack

#
servers instead of Passenger.

#
#Below modules lines you will get while running above gem install
rack passenger

LoadModule
passenger_module
/usr/lib/ruby/gems/1.8/gems/passenger-4.0.37/buildout/apache2/mod_passenger.so


  <IfModule
mod_passenger.c>


    PassengerRoot
/usr/lib/ruby/gems/1.8/gems/passenger-4.0.37


    PassengerDefaultRuby
/usr/bin/ruby


  </IfModule>

#
you probably want to tune these settings

PassengerHighPerformance
on

PassengerMaxPoolSize
12

PassengerPoolIdleTime
1500

#
PassengerMaxRequests 1000

PassengerStatThrottleRate
120

#RackAutoDetect
Off

#RailsAutoDetect
Off

Listen
8140

<VirtualHost
*:8140>


   SSLEngine
On


   #
Only allow high security cryptography. Alter if needed for
compatibility.


   SSLProtocol
            All -SSLv2


   SSLCipherSuite
         HIGH:!ADH:RC4+RSA:-MEDIUM:-LOW:-EXP


   SSLCertificateFile
     /var/lib/puppet/ssl/certs/alps.gai.net.pem


   SSLCertificateKeyFile
  /var/lib/puppet/ssl/private_keys/alps.gai.net.pem


   SSLCertificateChainFile
/var/lib/puppet/ssl/ca/ca_crt.pem


   SSLCACertificateFile
   /var/lib/puppet/ssl/ca/ca_crt.pem


   SSLCARevocationFile
    /var/lib/puppet/ssl/ca/ca_crl.pem


   SSLVerifyClient
        optional


   SSLVerifyDepth
         1


   SSLOptions
             +StdEnvVars +ExportCertData


   #
These request headers are used to pass the client certificate


   #
authentication information on to the puppet master process


   RequestHeader
set X-SSL-Subject %{SSL_CLIENT_S_DN}e


   RequestHeader
set X-Client-DN %{SSL_CLIENT_S_DN}e


   RequestHeader
set X-Client-Verify %{SSL_CLIENT_VERIFY}e


   DocumentRoot
/usr/share/puppet/rack/puppetmasterd/public


   <Directory
/usr/share/puppet/rack/puppetmasterd/>


     Options
None


     AllowOverride
None


     #
Apply the right behavior depending on Apache version.


     <IfVersion
< 2.4>


       Order
allow,deny


       Allow
from all


     </IfVersion>


     <IfVersion
>= 2.4>


       Require
all granted


     </IfVersion>


   </Directory>


   ErrorLog
/var/log/httpd/alps_puppet_master_ssl_error.log


   CustomLog
/var/log/httpd/alps_puppet_master_ssl_access.log combined

</VirtualHost>

#
Stop puppetmaster & restart http

#/etc/init.d/puppetmaster
stop

#/etc/init.d/httpd
restart

#Check
listen


#netst
-tupln | grep LISTEN | grep 8140

Step
7# ##Running Puppet Dashboard using apache instead of services
(puppet-dashboard)

Refer:
http://docs.puppetlabs.com/dashboard/passenger.html

#cp
/usr/share/puppet-dashboard/ext/passenger/dashboard-vhost.conf
/etc/init.d/conf.d/

#vim
/etc/httpd/conf.d/dashboard-vhost.conf

#Add
below lines or modify

LoadModule
passenger_module
/usr/lib/ruby/gems/1.8/gems/passenger-4.0.37/buildout/apache2/mod_passenger.so

   <IfModule
mod_passenger.c>

     PassengerRoot
/usr/lib/ruby/gems/1.8/gems/passenger-4.0.37

     PassengerDefaultRuby
/usr/bin/ruby

</IfModule>

PassengerHighPerformance
on

PassengerMaxPoolSize
12

PassengerPoolIdleTime
1500

PassengerStatThrottleRate
120

Listen
3000

<VirtualHost
*:3000>

        ServerName
puppet.gai.net

        DocumentRoot
/usr/share/puppet-dashboard/public/

        <Directory
/usr/share/puppet-dashboard/public/>

                Options
None

                Order
allow,deny

                allow
from all

        </Directory>

  ErrorLog
/var/log/httpd/puppet.gai.net_error.log

  LogLevel
warn

  CustomLog
/var/log/httpd/puppet.gai.net_access.log combined

  ServerSignature
On

<Location
"/">

    Order
allow,deny

    Allow
from 192.168.0.125 # your puppet master's IP

    Satisfy
any

    AuthName
"Puppet Dashboard"

    AuthType
Basic

    AuthUserFile
/etc/httpd/htpasswd

    Require
valid-user

</Location>

</VirtualHost>

##Stop
puppet-dashboard services & restart http

#/etc/init.d/puppet-dashboard
stop

#/etc/init.d/httpd
restart

##Classes Examples:

#To Copy configuration file:

vim /etc/puppet/manifests/site.pp

class centos_6_repo_class {

file { "/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6":

owner => root,

group => root,

mode => 644,

source => "puppet:///modules/centos_6_repo/RPM-GPG-KEY-CentOS-6"

}

file { "/etc/yum.repos.d/CentOS-Base.repo":

owner => root,

group => root,

mode => 644,

source => "puppet:///modules/centos_6_repo/CentOS-Base.repo"

}

# tell puppet on which client to run the class

node 'gai-1234.gai.net' {

include centos_6_repo_class

}

To Install a package:

#vim /etc/puppet/manifests/site.pp

class ftp-0-17-54_install_class {

package { "ftp":

ensure => "0.17-54.el6"

}

node 'genistra.gai.net' {

include ftp-0-17-54_install_class

}

Pupper Agent for Windows:

Download from https://downloads.puppetlabs.com/windows/puppet-3.4.3.msi

create file as C:\Program Files\Puppet Labs\Puppet Enterprise\puppet\conf\puppet.conf

add below lines

[agent]
    server = alps.gai.net
    certname = thinkpad4
        report=true
        pluginsync=true
        runinterval=2m

In server site.pp file add below lines

if $osfamily == 'windows' {
    File { source_permissions => ignore }
}

class win-check {
file { "C:/temp/CentOS-Base.repo":
    ensure => present,
    group => Administrators,
    source => "puppet:///modules/centos_6_repo/CentOS-Base.repo"
}

}

node 'thinkpad4' {
    include win-check
}

URLS Referred:

http://docs.puppetlabs.com/dashboard/manual/1.2/bootstrapping.html#testing-that-dashboard-is-working

http://docs.puppetlabs.com/dashboard/passenger.html

http://www.stuartmacfarlane.co.uk/installing-puppet-3-master-and-agent-on-centos-6-3

http://www.6tech.org/2013/01/how-to-install-puppet-open-source-on-centos-6-3/

http://www.ende-der-vernunft.org/2012/10/19/howto-install-puppet-dashboard-passenger-activemq-and-mcollective-entirely-from-repos-on-centos-6/

http://usrlocal.com/2012/11/puppet-dashboard/

http://www.puppetcookbook.com/posts/select-a-file-based-on-a-fact.html

http://rene.bz/yum-repo-and-package-dependencies-puppet/