Archive | Linux

“error establishing a database connection” in a previously working WordPress site

My site just crashed!!! I received the oh-so-helpful error “establishing a database connection” when I went to my site. Other sites on the server were fine.  The site was working fine, and my config had not changed. The httpd error log showed nothing.

Then I found this in the MySQL logs (/var/log/mysqld.log)

[ERROR] /usr/libexec/mysqld: Table ‘./db_name/wp_options’ is marked as crashed and should be repaired

To fix I ran:

mysql -e “use db_name;REPAIR TABLE wp_options”

And we are back . . .

CentOS, NTPD, VMware and sleeping.

I have a MacPro at home, and I am running VMware Fusion on it. At night, I sleep the machine to save electricity. I have a CentOS guest running and the time is always out of sync. After the VM is restored from sleep, the NTP service is no longer running and my time really drifts.

I finally sat down and tried to figure out my time sync issues on CentOS and VMware. This document explains it all. At the bottom they say you should turn off VMware tools time sync and use NTPD.

To install NTPD (with the recommended changes from the above document)

  1. yum install ntp
  2. chkconfig ntpd on
  3. sed -i 1i”tinker panic 0″ /etc/ntp.conf
  4. sed -i “s/^server\t127.127.1.0/#server\t127.127.1.0/g” /etc/ntp.conf
  5. sed -i “s/^fudge\t127.127.1.0/#fudge\t127.127.1.0/g” /etc/ntp.conf
  6. service ntpd start

The “tinker panic 0” is the most important part. Now when my VMware Fusion wakes and the CentOS guest powers on, NTP gets everything setup correctly.

CentOS Kickstart with local CDROM media and a “http config file”

At our NYC office, I have a PXE/Kickstart system setup. All I need to do is boot to PXE, and I can install CentOS with very little intervention. I have to choose Server vs Desktop (each choice points to a different http hosted kickstart cfg file.) and I have to setup my partitons how I want them.

Recently I was tasked to setup a new office in LA. I had taken a CentOS iso with me, but I wanted to use the standard config file (hosted over http) at the central office. Basically I wanted to use the local bits with a remote config file. I learned a couple fo things going through this exercise.

  1. It is not easy to serach for KickStart config file examples becasue KickStart is the name of the process and the name of the config file.
  2. When booting from an ISO, if you want use local bits with a remote KickStart config file, the command is
    • linux ks=http://url.server.com/
    • The kickstart file must have the directive: cdrom
    • you can not have both “url” and “cdrom” in the same file. It will use the first one it finds (or last, I do not remember which)
  3. You can not combine both command line parameters and a kickstart file – the KickStart file overrides. For example I used:
    • linux ks=http://url.server.com/ks.cfg method=cdrom and I did not have “cdrom” in the config file. The installer prompted me for media type.

The only way I could use local bits with a KickStart file, was to specify “cdrom” in the config file. Which means I had to have yet another option/config file= dekstop,server, server-cdrom.

Comparing master and slave MySQL WordPress DBs

My VPS host – Rackspace contacted me the other day to tell me that my host server became unresponsive. After it came back online I went to my MySQL replicated slave to check the status. The /var/log/mysqld.log said:

Error reading packet from server: Client requested master to start replication from impossible position

This article siad

“9 times out of 10 it’s because the master crashed and when it came back online a new binlog file was made”.

That sounds about right, my virtual host was probably force rebooted, and my VM crashed. I recovered my DBs by re-mirroring my DBs over an AutoSSH connection between my two servers.

The article goes on to say:

Now if your super sensitive of lost events because a row or two could of been lost from this replication event, do some spot testing for tables written to often”

I wanted to verify my DBs were the same on both master and slave, so I wrote this BASH function:

function CompareDBs {
if [ $1 ]; then
DB=$1;
RDBDUMP=$(mysqldump -h <a href="tel:127.0.0.1">127.0.0.1</a> -P 7777 --order-by-primary --skip-extended-insert --skip-opt --skip-comments $DB)
LDBDUMP=$(mysqldump --order-by-primary --skip-extended-insert --skip-opt --skip-comments $DB)
diff -y --suppress-common-lines &lt;(echo &quot;$RDBDUMP&quot;) &lt;(echo &quot;$LDBDUMP&quot;)
fi
}

Call the function with the DB you want to compare. This script assumes that you have an SSH Tunnel between your two servers.

 

********* MAKE SURE YOUR TIME IS RIGHT ON BOTH SERVERS*********
This will save you some hair pulling!

WordPress 3.2 requirements and CentOS 5.6

Current requirements

  • PHP version 4.3 or greater
  • MySQL version 4.1.2 or greater

WordPress 3.2 requirements:

  • PHP version 5.2.4 or greater
  • MySQL version 5.0 or greater.

Default PHP on Cent0S 5.6 = PHP 5.1.6.

CRAP

Updated: See this post on how to install an updated PHP on CentOS 5.6

Script to compare RPMs on two different CentOS servers

I wanted to make sure that the same RPMs were installed on several servers. I wasn’t worried about versions of RPMs because everything should be kept up to date via yum. So I sat down and wrote the script below. It has been on my ToDo list for quite a while!

RRPM=$(ssh $REMOTESERVER "rpm -qa --queryformat '%{NAME}\n'" )
LRPM=$(rpm -qa --queryformat '%{NAME}\n')

echo "*** Missing from $REMOTESERVER" ***
grep -vf <(echo "$RRPM"| sort) <(echo "$LRPM"|sort)
echo
echo "*** Missing from Local system ***"
grep -vf <(echo "$LRPM"| sort) <(echo "$RRPM"|sort)
echo

This script connects to a remote machine and compares RPMs installed there to the RPMs that are installed locally.

AutoSSH on CentOS

I have been interested in MySQL replication over ssh and I wanted a way to make sure that the tunnel is always up. Everyone says to use AutoSSH. AutoSSH is not in EPEL, but is in rpmforge (Is that that same as DAG? Didn’t they merge?). I installed rpmforge:

rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm

I don’t like to do the RepoDance, so I disabled rpmforge:

sed -i "s/enabled = 1/enabled = 0/" /etc/yum.repos.d/rpmforge.repo

Next I installed AutoSSH

yum install --enablerepo=rpmforge autossh</p>

And finally my Bash function to create an AutoSSH tunnel:
 

function StartAutoSSH {
	. /etc/rc.d/init.d/functions
	AUTOSSH_PIDFILE=/var/run/autossh.pid # we are assuming only one autossh tunnel
	if [ ! -e $AUTOSSH_PIDFILE ]; then
	AUTOSSH_PIDFILE=$AUTOSSH_PIDFILE;export AUTOSSH_PIDFILE
	autossh -M29001 -f -N -L7777:127.0.0.1:3306 [email protected]
	else
	status -p $AUTOSSH_PIDFILE autossh
	fi
}

If you call this function, it will created the specified tunnel or if it is up and runnng, then it will spit back the PID.

Mirror all MySQL DBs to the local machine

Continuing on my quest to get MySQL replicating over ssh, I am using the following bash function to replicate all remote DBs locally:

 

function MirrorAllRemoteDBsToLocal {
	for REMOTEDB in $(mysql -h 127.0.0.1 -P 7777 --batch --skip-column-names -e "SHOW DATABASES")
	do
	LOCALDBEXISTS=$(mysql --batch --skip-column-names -e "SHOW DATABASES LIKE '"$REMOTEDB"';" | grep "$REMOTEDB" > /dev/null; echo "$?")
	if [ $LOCALDBEXISTS -ne 0 ];then
		echo "adding $REMOTEDB to local MySQL"
		mysql -e "create database $REMOTEDB;"
		echo "getting a dump"
		mysqldump -h 127.0.0.1 -P 7777 $REMOTEDB | mysql $REMOTEDB;
		echo " adding $REMOTEDB to my.conf"
		sed -i '/master-port/a\\treplicate-do-db='$REMOTEDB'' /etc/my.cnf
	fi
	done
}

Line 2 connects to the local AutoSSH tunnel and gets a list of all the remote DBs.

Then we loop through the DBs and if there is not a DB locally with that name, the script will create a database. Next the script gets (Line 9) a dump of the DB and copies it to the newly created DB.

And finally the script add the DB to the /etc/my.cnf (line 11).

All that should have to happen is to issue a slave stop and then slave start, and all DBs should be mirrored locally.

MySQL replication of WordPress dbs over ssh

I wanted to setup MySQL replication over ssh for a small WordPress database. I have a VM that lives in my house, I wanted to be able to bring it up, make it current, disconnect, and then hack away. Here is my proceedure.

On the master:

Setup the replication user (I had to use 127.0.0.1 becasue % did not let me connect from ssh):

grant replication slave on *.* TO repl@"127.0.0.1" identified by '[repl password]';

edit /etc/my.cnf

server-id=1
log-bin=mysql-bin
replicate-do-db=DB_TO_REPLICATE   #the name of the db you want to replicate.

From the slave:

edit /etc/my.cnf

server-id = 2
master-host = 127.0.0.1
master-user = repl
master-password = password
master-port = 7777
replicate-do-db = DB_TO_REPLICATE   #the name of the db you want to replicate.

Next, tunnel a ssh connection from slave to the source, and then make sure you can see the source databases:

ssh -f -N -L7777:127.0.0.1:3306 [email protected]
mysql -h 127.0.0.1 -P 7777 -e "show databases;"

Next create a local database:

mysql -e "create database DB_TO_REPLICATE;"

Next we want to find where the master’s log is and its position:

MASTERLOGPOS=$(mysql -h 127.0.0.1 -P 7777 --batch --skip-column-names -e "SHOW MASTER STATUS;" | cut -f2)
MASTERLOGFILE=$(mysql -h 127.0.0.1 -P 7777 --batch --skip-column-names -e "SHOW MASTER STATUS;" | cut -f1)

next we want to seed the local db

mysqldump -h 127.0.0.1 -P 7777 DB_TO_REPLICATE | mysql -u root DB_TO_REPLICATE;

and finally, tell the db to use the source as the Master:

mysql -e "use DB_TO_REPLICATE;CHANGE MASTER TO MASTER_LOG_FILE='"$MASTERLOGFILE"',MASTER_LOG_POS=$MASTERLOGPOS;"
mysql -e "start slave;"

To test I run:

mysql -e "show slave status\G;"

and I compare the following on the master and the client:

mysql -e "select id,post_parent,post_modified,post_title from jbmurphy_com.wp_posts"

My “Copy WordPress site to local” script

I wanted to write a script that would copy a WordPress site to the local machine from a remote server (securely over ssh). This would allow me to set up a test environment or backup. First I need to make sure the site exists on the source server. Then I want to get the current wp-config settings, like the database name:

echo "Checking $SRCSERVER for a site named $SRCSITENAME"
CHECKCMD="ls /PATH/TO/$SRCSITENAME/wp-config.php | grep wp-config.php > /dev/null; echo \$?"
CHECKFILE=$(ssh $SRCSERVER $CHECKCMD)
if [ $CHECKFILE -eq 0 ]; then
        echo "Source Site $SRCSITENAME on $SRCSERVER exists"
        SRCMYSQLSITEUSER=$(ssh $SRCSERVER grep "DB_USER" /PATH/TO/$SRCSITENAME/wp-config.php | cut -f 2 -d " " | cut -f 2 -d "'")
        SRCMYSQLSITEPASS=$(ssh $SRCSERVER grep "DB_PASSWORD" /PATH/TO/$SRCSITENAME/wp-config.php | cut -f 2 -d " " | cut -f 2 -d "'")
        SRCMYSQLSITEDB=$(ssh $SRCSERVER grep "DB_NAME" /PATH/TO/$SRCSITENAME/wp-config.php | cut -f 2 -d " " | cut -f 2 -d "'")
        echo "Source mySQL Site Database:"$SRCMYSQLSITEDB
        echo "Source mySQL Site Username:"$SRCMYSQLSITEUSER
        echo "Source mySQL Site Password:"$SRCMYSQLSITEPASS
else
        echo "Source Site $SRCSITENAME on $SRCSERVER does not exist"
        exit;
fi

Next I provision a new site with a new virtual host, and I copy over the files:

echo "Copying site from $SRCSERVER:/PATH/TO/$SRCSITENAME/ to /PATH/TO/$DESTSITENAME"
rsync --archive $SRCSERVER:/PATH/TO/$SRCSITENAME/ /PATH/TO/$DESTSITENAME

Next I provision a new blank MySQL database with the name, username, and password received in the first part for code, then I copy over the db data via ssh:

echo "Copying mysql data from $SRCSERVER to local database name $DESTSITENAME"
ssh $SRCSERVER "mysqldump -u root $SRCMYSQLSITEDB" | mysql -u root $DESTSITENAME

Finally I run a couple of SQL queries to change the URL that WordPress has inside the DB.


echo "What is going to be the WordPress URL"
read NEWURL

OLDSITENAME=$(mysql --batch --skip-column-names -e "use $DESTSITENAME;select guid from wp_posts LIMIT 1;" | cut -d"/" -f3)
echo "Changing the wp_posts guids from $OLDSITENAME to $NewURL

mysql --batch --skip-column-names -e "use $DESTSITENAME;UPDATE wp_posts SET guid = REPLACE (guid, '"http://$OLDSITENAME"','"http://$NEWURL"');"

echo "Changing the home and siterurl to $NewURL"

mysql --batch --skip-column-names -e "use $DESTSITENAME;UPDATE wp_options SET option_value = replace(option_value, '"http://$OLDSITENAME"', '"http://$NEWURL"') WHERE option_name = 'home' OR option_name = 'siteurl';"