jqPlot Example for Website Graphs

This is a simple example usage of the jqPlot jQuery plugin. The example on the author website omits the “, {}” options in the .jqplot function call.

     
<html>
 <head>
 http://./scripts/jquery-ui/js/jquery-1.4.2.min.js
 http://./scripts/jqplot/jquery.jqplot.min.js
 <link rel="stylesheet" href="./scripts/jqplot/jquery.jqplot.min.css" type="text/css" media="all" />
 <!--[if IE]>http://./scripts/jqplot/excanvas.js<![endif]-->
 

 $(document).ready(function() {
 $.jqplot.config.enablePlugins = true;
 alert('1');
 $.jqplot('chartdiv',  [[[1, 2],[3,5.12],[5,13.1],[7,33.6],[9,85.9],[11,219.9]]], {});


 });
 
 
 </head>
 <body>
 
</body> </html>  

Rsyslog and Log Analyzer

These are the steps I took to create a centralised location of system logs. In this scenario multiple servers (earth, venus, mars) send their system logs to a central server (sun 192.168.1.1). I’m not going to cover the configuration of Apache, MySql except were it applies to Log Analyzer. Most of the servers are running Red Hat / CentOS 5. In this setup I am using 192.168.0.0 as the subnet and topsecret as the password. Change as appropriate.

Central Server (sun):

On the central server (sun) which will be running Log Analyzer, these steps only need to be taken once. If you only want to add more servers sending their syslogs to sun skip this section:

yum install httpd php mysql php-mysql mysql-server wget rsyslog rsyslog-mysql

Create the rsyslog database structure in MySQL:

mysql -u root -p < /usr/share/doc/rsyslog-mysql-3.22.1/createDB.sql

Create the MySQL user:

mysql -u root -p mysql
mysql> GRANT ALL ON Syslog.* TO rsyslog@localhost IDENTIFIED BY ‘topsecret’;
mysql> flush privileges;
mysql> exit

Edit the rsyslog config file:

vi /etc/rsyslog.conf

Add the following at the top:

$AllowedSender UDP, 127.0.0.1, 192.168.0.0/16
$AllowedSender TCP, 127.0.0.1, 192.168.0.0/16

#UDP log
$ModLoad imudp
$UDPServerRun 514
#TCP log
$ModLoad imtcp
$InputTCPServerRun 514

$ModLoad ommysql
*.info :ommysql:127.0.0.1,Syslog,rsyslog,topsecret

Amend the rsyslog startup options:

vi /etc/sysconfig/rsyslog

Set the options as follows:

SYSLOGD_OPTIONS=”-r -t154 -m 0″

Now disable the standard syslog and enable rsyslog:

chkconfig syslog off
service syslog stop
chkconfig rsyslog on
service rsyslog start

Install Log Analyzer:

cd /tmp
wget http://download.adiscon.com/loganalyzer/loganalyzer-3.0.7.tar.gz
tar xzf loganalyzer-3.0.7.tar.gz
mv loganalyzer-3.0.7/src /var/www/html/loganalyzer
mv loganalyzer-3.0.7/contrib/* /var/www/html/loganalyzer
cd /var/www/html/loganalyzer
chmod u+x configure.sh secure.sh
./configure.sh

Now browse the website e.g. http://sun/loganalyzer
Follow the installer adding your MySQL credentials when requested.

Amend the firewall on the central (sun) server to allow other servers:

vi /etc/sysconfig/iptables

Add:

-A RH-Firewall-1-INPUT -p udp -m udp –dport 514 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp –dport 514 -j ACCEPT

Restart iptables:

service iptables restart

Remote Servers
Configure Other Servers (mars, venus, earth) to send their syslogs to the central server (sun):
Install rsyslog:

yum install rsyslog

Edit the config:

vi /etc/rsyslog.conf

Add:

*.info  @192.168.1.1:514

I add this on line number 2 below $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
Set rsyslog as the default syslogger:

/sbin/chkconfig syslog off
/sbin/chkconfig rsyslog on
service syslog stop
service rsyslog start

Using *.info could collect a lot of messages so customise as necessary, for example changing to *.crit will collect less messages of higher importance.

WordPress Multisite Domain Mapping

We have WordPress (v3.0.5) installed to deliver blogs of the format domain.com/blog1 and domain.com/blog2, however we would also like to be able to host blogs with their own already registered domain name, so ….

  1. Get the MU Domain Mapping plugin and unzip somewhere safe
  2. Copy the domain_mapping.php file to ./wp-content/mu-plugins
  3. Copy the sunrise.php file to ./wp-content
  4. Edit the ./wp-config.php file
  5. Uncomment the line: define( ‘SUNRISE’, ‘on’ );
  6. Comment out the line:  define(‘COOKIE_DOMAIN’, ‘blogs.domain.com’);
  7. Visit the WordPress backend and click SuperAdmin -> Domain Mapping
  8. Enter the server IP address and I just ticked the options for: Permanent redirect and User domain mapping page
  9. Enable a domain name for a site by clicking Super Admin -> Sites, Choose ‘Backend’ for the site you want to amend
  10. Click Tools > Domain Mapping
  11. Enter the domain name and click ‘Add’

Note: You must ensure that your new domain DNS record is either an A record pointing to your wordpress server IP address or a CNAME record pointing to your wordpress domain name.

WordPress is the cat’s whiskers!

Building a distribution group in Exchange 2007 from a .csv

A user needed a distribution list, so that she could send a message to 2000+ users. Doing this in Exchange meant that the message would not be duplicated 2000+ times, but the information on the users was in a .csv file.

Using the following command, I was able to add the users to an empty Distribution Group:

Type C:\jjlist.csv | ForEach-Object -Process {Add-DistributionGroupMember -Identity “nameofyourlist” -Member $_}

This works where all users you wish to add to the group exist in the AD, and where you have first made the Distribution Group in Exchange.

Spam Assassin rules for ~

Recently I wanted to block a uri using a body-rule in SpamAssassin. The uri in question contained the ‘~’ symbol, and as this can be used as a regular expression, I thought I’d check to see if it needed to be escaped.

Testing this by escaping the character normally, by using ‘\’ didn’t work, so I tried not escaping it at all. This allowed me to block a uri with ‘~’ that I made up, and showed that this symbol does not need to be escaped.

MySQL Proxy

Today I wanted to create a straight-forward MySQL Proxy service which would forward MySQL queries from a development machine to another live server which was running MySQL. The aim was to allow people on the development machine to use the same connection string as on the live server, specifying the MySQL host as localhost but actually be retrieving data from the live database server.

  • Added the EPEL repository and set it to a priority higher than RPMforge (if using rpmforge). Otherwise you may get problems with the missing dependencies for liblua.5.1.so
  • Install MySQL-Proxy from EPEL and socat for later:

    yum install mysql-proxy socat

  • Run mysql-proxy with the following (we could add this to /etc/rc.local later):

/usr/sbin/mysql-proxy –proxy-backend-addresses=live.domain.tld:3306 –proxy-address=127.0.0.1:3306 –proxy-skip-profiling  –pid-file=/var/run/mysqld/mysqld.pid &

Now if we test that from the local machine with the following it will not work as the mysql client automatically looks for a local Unix socket rather than TCP/IP port 3306:

mysql -u jonny -p
ERROR 2002 (HY000): Can’t connect to local MySQL server through socket ‘/var/lib/mysql/mysql.sock’ (2)

So we could amend /etc/my.cnf adding the following at the bottom:

[client]
protocol=tcp

Now with this setting we can connect on the command line but still not from PHP which appears to be hardwired to using the socket. So we could use socat as follows:

socat UNIX-LISTEN:/var/lib/mysql/mysql.sock,fork,reuseaddr,unlink-early,user=mysql,group=mysql,mode=777 TCP:127.0.0.1:3306 &

With this running socat with forward socket requests to the MySQL port.
It would be nice to wrap this all up into an init startup script. I’m not saying this a good startup script but it helps make things a little more manageable. First I added the following config options to the MySQL config file:

vi /etc/my.cnf

[mysql-proxy]
proxy-backend-addresses=live.domain.tld:3306
proxy-address=127.0.0.1:3306
pid-file=/var/run/mysqld/mysqld.pid
socket=/var/lib/mysql/mysql.sock

Next the init script which should disallow starting the proxy if MySQL is already running.

vi /etc/init.d/mysql-proxy

#!/bin/bash
#
# mysql-proxy   This shell script takes care of starting and stopping
#               the MySQL Proxy (mysql-proxy).
#
# chkconfig: - 64 36
# description:  MySQL Proxy server.
# processname: mysql-proxy
# requires: socat which may need to be installed

# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
. /etc/sysconfig/network

prog="MySQL-Proxy"

# extract value of a MySQL option from config files
# Usage: get_mysql_option SECTION VARNAME DEFAULT
# result is returned in $result
# We use my_print_defaults which prints all options from multiple files,
# with the more specific ones later; hence take the last match.
get_mysql_option(){
        result=`/usr/bin/my_print_defaults "$1" | sed -n "s/^--$2=//p" | tail -n 1`
        if [ -z "$result" ]; then
            # not found, use default
            result="$3"
        fi
}

get_mysql_option mysql-proxy proxy-backend-addresses "127.0.0.1:3306"
proxybackendaddresses="$result"
get_mysql_option mysql-proxy proxy-address "127.0.0.1:3306"
proxyaddress="$result"
get_mysql_option mysql-proxy socket "/var/lib/mysql/mysql.sock"
socketfile="$result"
get_mysql_option mysql-proxy pid-file "/var/run/mysqld/mysqld.pid"
mypidfile="$result"
mysqllock="/var/lock/subsys/mysqld"
mysqlproxylock="/var/lock/subsys/mysql-socket"

start(){
        if [ ! -f $socketfile -a ! -f $mypidfile -a ! -f $mysqllock -a ! -f $mysqlproxylock ]; then
                /usr/sbin/mysql-proxy --proxy-backend-addresses="$proxybackendaddresses" --proxy-skip-profiling \
                --proxy-address="$proxyaddress"  \
                >/dev/null 2>&1 &
                result1=$?
                [ $result1 -eq 0 ] && touch $mysqllock

                /usr/bin/socat UNIX-LISTEN:$socketfile,fork,reuseaddr,unlink-early,user=mysql,group=mysql,mode=777 \
                TCP:$proxyaddress >/dev/null 2>&1 &
                result2=$?
                [ $result2 -eq 0 ] && touch $mysqlproxylock
                if [ $result1 -eq 0 -a $result2 -eq 0 ]; then
                         ret=0
                         action $"Starting  $prog: " /bin/true

                else
                         ret=1
                        action $"Failed Stopping $prog: " /bin/false
                fi

        else
                ret=1
                echo "Failed: Lock files or sockets already exist. Check MySQL is not already running"

        fi
        return $ret
}

stop(){
        MYSQLPROXYPID=`ps aux | grep proxy | grep -v grep | awk {'print $2'} | head -1`
        SOCATPID=`ps aux | grep socat | grep -v grep | awk {'print $2'}`
        if [ -n "$MYSQLPROXYPID" ]; then
            /bin/kill "$MYSQLPROXYPID" >/dev/null 2>&1
            result1=$?
        fi

        if [ -n "SOCATPID" ]; then
            /bin/kill "$SOCATPID"  >/dev/null 2>&1
            result2=$?
        fi

            if [ $result1 -eq 0 -a $result2 -eq 0 ]; then
                    rm -f $mysqllock
                    rm -f $mysqlproxylock
                    rm -f "$socketfile"
                    action $"Stopping $prog: " /bin/true
                    ret=0
           
            else
                ret=1
                action $"Failed Stopping $prog: " /bin/false
            fi
        return $ret
}
 
restart(){
    stop
    start
}

# See how we were called.
case "$1" in
  start)
    start
    ;;
  stop)
    stop
    ;;
  restart)
    restart
    ;;
  *)
    echo $"Usage: $0 {start|stop|restart}"
    exit 1
esac

exit $?

Set the MySQL-Proxy to start on boot:

chkconfig mysql-proxy on

MySQL query output to text or CSV

I needed to output the results of a MySQL query today for further analysis and found the answer here. For future reference…

SELECT * FROM errors WHERE mailfrom LIKE ‘a.n.other%’ INTO OUTFILE ‘/tmp/a.n.other.txt’ FIELDS TERMINATED BY ‘,’ LINES TERMINATED BY ‘\n’;

It is also possible to enclose the fields with quotes using:

ENCLOSED BY ‘”‘

Clamd and Iptables

Running Clam on a separate machine to our Mail Transfer Agent we needed to configure iptables to allow access to clam from those machines. Clamd listens for connections on port 3310 by default but appears to hand off to other ‘passive’ ports for each stream to scan. So allowing access to port 3310 was not enough.
In the /etc/clamd.conf file I uncommented the following lines:

StreamMinPort 30000
StreamMaxPort 32000

To restrict the ports which clamd would use and restarted clam with

/etc/init.d/clamd restart

Checking netstat with:

netstat -lntap | grep clam

I could see that clam was indeed now using ports 30000-32000 rather than the default of 1024-2048
So adding the following line to /etc/sysconfig/iptables allowed the machines in my class B network to make use of the clam service:

-A RH-Firewall-1-INPUT -m state –state NEW -m tcp -p tcp –dport 3310 –source 111.222.0.0/16 -j ACCEPT
-A RH-Firewall-1-INPUT -m state –state NEW -m tcp -p tcp –dport 30000:32000 –source 111.222.0.0/16 -j ACCEPT

Building an RPM Package (DansGuardian 2.10)

After putting together an RPM for DansGuardian v2.10.1.1 I thought I would make a few notes. The version of DansGuardian in the repositories is currently version 2.8 and the only other repo or rpm I could find was a 32-bit 2.10.  Why DansGuardian 2.10?

  • Content Scanning Support with Clamd or Kapersky
  • Regular Expressions to enforce Google Safe Search (without patching)
  • NTLM support

So I set about generating the rpm for version 2.10.1.1 for 64-bit CentOS. My CentOS version is 5.5. I found this IBM developer works article useful for guidance on creating RPMs. I also did this build with DansGuardian v2.8 already installed via yum so I was able to make use of the existing init.d and logrotate.d scripts.

cd /usr/src/redhat/SOURCES
cp /etc/init.d/dansguardian /usr/src/redhat/SOURCES/dansguardian.init-centos
cp /etc/logrotate.d/dansguardian /usr/src/redhat/SOURCES/dansguardian.logrotate-centos
wget http://dansguardian.org/downloads/2/Stable/dansguardian-2.10.1.1.tar.gz
cd ../SPECS
wget http://ftp.qb.com.au/pub/yum/SPECS/dansguardian-centos-clamav.spec

Edit the above spec file to reflect version 2.10.1.1 removing the %patch0 line and setting –enable-clamav=no \
or use the SPEC file listed below.

yum install clamd clamav-devel pcre-devel

Build the RPM – keep an eye out for errors and install dependencies as needed:

cd ..
rpmbuild -v -bb –clean /usr/src/redhat/SPECS/dansguardian-centos-clamav.spec

After compilation you should have an rpm in /usr/src/redhat/RPMS/x86_64 which you can install after removing the existing 2.8 version of dansguardian:

yum remove dansguardian
rpm -Uvh /usr/src/redhat/RPMS/x86_64/dansguardian-clamav-2.10.1.1-0.1.el5.clamav.0.95.x86_64.rpm

In order to make use of the clamd content scanning in dansguardian it may be necessary to to change the owner and group in /etc/dansguardian/dansguardian.conf to clamav & clamav.

The RPM I created is available here: dansguardian-clamav-2.10.1.1-0.1.el5.clamav.0.95.x86_64.rpm – just in case anyone else can make use of it. I make no warranties.
 
SPEC FILE:

# $Id: dansguardian.spec  2007-04-06 dpv $
# Upstream: Daniel Barron <author$dansguardian,org>

#
# What to change when you build a new version:
# Version – match upstream version
# Release – Usually this would stay the same if the version is changed.
#           It is only things NOT in the .tar.gz file that will cause a release number change.
# Packager – if desired
# Vendor – if desired
# Patch0 – could easily become obsolete if the upstream version has changed
# %files – if files are added or removed, this section needs to be updated accordingly
#

%define real_name DansGuardian

Summary: Content filtering web proxy
Name: dansguardian-clamav
BuildRequires: gcc-c++ zlib-devel pcre-devel
Requires: coreutils squid
License: GPL
Group: System Environment/Daemons
URL: http://www.dansguardian.org/
Version: 2.10.1.1
Release: 0.1.el5.clamav.0.95
Packager: Jonny McCullagh <webmaster@qub.ac.uk>
Vendor: Queens University Belfast

# all of these files must live in /usr/src/redhat/SOURCES
Source: dansguardian-%{version}.tar.gz
Source1: dansguardian.init-centos
Source3: dansguardian.logrotate-centos
#Patch0: dansguardian_gcc43.patch
BuildRoot: %{_tmppath}/dansguardian-%{version}-%{release}-root

%description
DansGuardian is a web filtering engine that checks the content within
the page itself in addition to the more traditional URL filtering.

DansGuardian is a content filtering proxy. It filters using multiple methods,
including URL and domain filtering, content phrase filtering, PICS filtering,
MIME filtering, file extension filtering, POST filtering.

%prep
%setup -q -n dansguardian-%{version}
#%patch0 -p0

%build
%{configure} \
        –enable-clamav=no \
        –enable-clamd=yes \
        –enable-email=yes \
        –enable-icap=yes \
        –enable-kavd=yes \
        –enable-ntlm=yes \
        –enable-pcre=yes \

%{__perl} -pi.orig -e ‘
                s|^(CHKCONFIG) =.*$|$1 = :|;
                s|^\tchown|#\tchown|;
                s|/usr/lib|%{_libdir}|g;
        ‘ Makefile

%{__make} %{?_smp_mflags}

%install
mkdir -p %{buildroot}/var/log/dansguardian/
#mkdir -p %{buildroot}/var/run
make install DESTDIR=%{buildroot}
%{__install} -D -m0755 %{SOURCE1} %{buildroot}%{_initrddir}/dansguardian
%{__install} -D -m0644 %{SOURCE3} %{buildroot}%{_sysconfdir}/logrotate.d/dansguardian
ln -s /etc/init.d/dansguardian %{buildroot}%{_sbindir}/rcdansguardian

%post
chown -R nobody /var/log/dansguardian
chkconfig –add dansguardian

%preun
if [ $1 -eq 0 ]; then
        /etc/init.d/dansguardian stop &>/dev/null || :

fi

%postun

%clean
%{__rm} -rf %{buildroot}

%files
%defattr(-, root, root, 0755)
%doc INSTALL README
%doc /usr/share/doc/dansguardian/*
%doc %{_mandir}/man?/*
%config %{_sysconfdir}/dansguardian/*
%config %{_sysconfdir}/logrotate.d/dansguardian
%dir /etc/dansguardian
%dir /usr/share/dansguardian
/usr/share/dansguardian/*
%{_sbindir}/dansguardian
%{_initrddir}/dansguardian
%{_sbindir}/rcdansguardian
%dir /var/log/dansguardian

%changelog
* Mon Nov 22 2010 Jonny McCullagh <webmaster@qub.ac.uk> – 2.10-1.1
– Update to DG stable release and built for x86_64

* Wed Mar 11 2009 Rick Saul <rpm@qb.com.au> – 2.10-0.3
– Update to DG stable release.

* Wed Sep 17 2008 Paul Gear <rpm@libertysys.com.au>  – 2.9.9.8
– Created CentOS version based on Don Vosburg’s SUSE spec file. See http://dansguardian.org/downloads/2/Beta/SUSE.txt