WordPress Woot Watcher

WordPress Woot Watcher, will monitor and check woot, shirt.woot, kids.woot, wine.woot and sellout.woot for new products every 10 hours. When a woot-off is launched, the widget will update the woot product every 30 seconds. WordPress Woot Watcher is a sidebar widget that enables select the woot sites you want to monitor in the administration section. The default installation selects woot.com only.

For all the details, installation, and more, visit the WordPress Woot Watcher page.

array=( ${array[@]-} $(echo "$variable") )

The code above reads as follows:

array=( ….. ) : Set your array variable named, appropriately enough: array.

${array[@]-} : Read all variables from the array array. The hyphen at the end prevents an “unbound variable” error when -u or -o nounset is set and array is an empty array. This is useful for appending the first value to an empty array in the event that one of the aforementioned flags is set.

$(echo “$variable”) : Add the variable $variable to the array through the use of command substitution $(…) by reading the output of echo “$variable” as the input value for the array. The reason for command substitution is that if you try it without the $(…), bash will read echo and “$variable” as separate entries into the array instead of just the value of $variable

One of daily responsibilities as a database administrator is maintaining the health of our shared hosting environment. In doing so, I deal with plenty of WordPress blogs daily. The one thing I have noticed is that many WordPress plugins are very poorly designed and can cause problems in a shared hosting environment. WordPress at it’s core though, is very well designed, although I have noticed some areas of possible improvement. As I come across common issues, I will post the fixes for them here for all to use.

The following is one of the most common queries I see all the time that actually comes from the WordPress core:
SELECT comment_date_gmt
FROM wp_comments
WHERE comment_author_IP = '94.142.131.22' OR
comment_author_email = 'yigrzds@tkgmd.com'
ORDER BY comment_date DESC
LIMIT 1


The IP and email is obviously only sample data. On the example server, the MySQL Explain for the query showed the following:

id: 1
select_type: SIMPLE
table: wp_comments
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 76845
Extra: Using where; Using filesort

Now, a query that examines 76,000+ rows is not the end of the world, but with enough traffic, it can slow down the server immensely, mainly because the explain above indicates that no key is being used on the query and therefore a full table scan is being performed.

To help alleviate this, the following code can be executed from a MySQL command prompt, or a management type interface like phpMyAdmin:

mysql> ALTER TABLE wp_comments ADD INDEX(comment_author_IP), ADD INDEX(comment_author_email)

**Note: The above optimization assumes the default database prefix of “wp”. If your database prefix is different, replace “wp” in the above query with your databases prefix.**

Afterward, the explain on the exact same query returns the following:

id: 1
select_type: SIMPLE
table: wp_comments
type: index_merge
possible_keys: comment_author_IP,comment_author_email
key: comment_author_IP,comment_author_email
key_len: 302,302
ref: NULL
rows: 2139
Extra: Using union(comment_author_IP,comment_author_email); Using where; Using filesort

As you can see, the query now examines 2139 rows as opposed to 76,000+ by using an Index Merge, which means the query is pulling data using two different indexes on the same query. Now keep in mind, index merges are only available in MySQL 5.0+. If you are using MySQL 4.1 or below, an index merge will not work, and I would suggest adding an index on the comment_author_IP field as it will tend to have more unique values. Simply remove everything after the comma in the above command.

This optimization will improve the speed of the query on your database and will provide better performance on your WordPress blog. Please share your thoughts below on this hack and the results for you.

+-----------------------------------------------------+
| Extra                                               |
+-----------------------------------------------------+
| Impossible WHERE noticed after reading const tables |
+-----------------------------------------------------+

There are two things to know in order to decipher this. First off, what is a “const table”. The “type” column refers to the join type made on the tables. The “const” type is the bets type of join type you can make, after a “system” type, which is a special case of the “const” join type when the table has only 1 row in it. According to the MySQL manual:

The table has at most one matching row, which is read at the start of the query. Because there is only one row, values from the column in this row can be regarded as constants by the rest of the optimizer. const tables are very fast because they are read only once.

(source: MySQL :: MySQL 5.0 Reference Manual :: 7.2.1 Optimizing Queries with EXPLAIN)

A “const” join type occurs when you run a very specific WHERE clause in your query that matches at most, one row from the referenced table.

The second thing to know is the “Impossible WHERE noticed”. This does not mean that your WHERE clause is wrong, it simple means “MySQL has read all const (and system) tables and notice that the WHERE clause is always false” (source: MySQL :: MySQL 5.0 Reference Manual :: 7.2.1 Optimizing Queries with EXPLAIN).

So in summation, the statement “Impossible WHERE noticed after reading const tables” simply means, that your query returned an empty result set (0 rows). You won’t always get this error on an EXPLAIN command when 0 rows are returned, but when you do get it, that’s what it means. I hope this helps

In order to achieve this look in your browser, first go to google.com. If you have an iGoogle home page, make sure you click ‘Classic Home’ in the upper right if you want to see the home page differences. Second, copy and paste the entire code below into your browser’s address bar:
javascript:void(document.cookie="PREF=ID=20b6e4c2f44943bb
:U=4bf292d46faad806:TM=1249677602:LM=1257919388:S=odm0Ys-53ZueXfZG;path=/; domain=.google.com");

If you want to set it back to the default, you will need to delete the current cookies for google.com from your browser. What are your thoughts on this interface? Share them below.

This option is useful for dumping large tables. It forces mysqldump to retrieve rows for a table from the server a row at a time rather than retrieving the entire row set and buffering it in memory before writing it out.

(source: MySQL :: MySQL 3.23, 4.0, 4.1 Reference Manual :: 4.5.4 mysqldump — A Database Backup Program)

Your full command should look something like this:
$ mysqldump -u user -p password -q database > outfile.sql

Run a bash command from the MySQL command line:

When you are operating in the MySQL command line mode, use the command \! to execute a command as if you were in a normal bash shell. For instance, to run a top command from the MySQL command line, run the following code:
mysql>\! top

Kill a long running MySQL query:

If you have a query that has been running for a long time, you can kill a MySQL query same as you can kill a bash process. To kill a query, run a show processlist; to get the process Id. Then run a kill command with the Id:

mysql> show processlist;
+-------+-------+-----------+------+---------+------+-------+--------+
| Id    | User  | Host      | db   | Command | Time | State | Info   |
+-------+-------+-----------+------+---------+------+-------+--------+
| 19638 | admin | localhost | mysql| Sleep   | 23154| NULL  | NULL   |
+-------+-------+-----------+------+---------+------+-------+--------+
1 row in set (0.00 sec)

mysql>kill 19638;

Cancel the current command you are inputting:

While operating in a bash command line, you can hit Ctrl+C to cancel your input. If you execute this in a mysql command line interface, you will close the connection. Instead, before any query terminators ( ; or \G), end your query with \c:
mysql> select * from mysql.user where user=\c
mysql>

Change your AUTO_INCREMENT value

If you’re like me, you’ve done some development at one point or another on a live site and wanted to test your code changes. You execute the code and it inserts your data into your database flawlessly. Once you’re ready to implement your changes, you remove the rows you inserted. However, now you’ve increased your AUTO_INCREMENT counter a few times and need/want it back where it was. Find the last auto incremented value, and run a quick alter table to update it:
mysql>SELECT id FROM members ORDER BY id DESC LIMIT 1;
+----+
| id |
+----+
| 82 |
+----+
1 row in set (0.00 sec)
mysql>ALTER TABLE members AUTO_INCREMENT=83;

If you’re asking, why not run a SHOW CREATE TABLE members query to get the current value, the answer is because that will show the current AUTO_INCREMENT value, after the data you inserted. You need to find the last value in your table, and set the next AUTO_INCREMENT to be 1 higher than that.

Make your query output, more terminal friendly

As most everyone should know, the command delimiter in MySQL is the semi-colon: ;. However, if you are running a query with lots of column output, for example a SELECT * query, to make your output more readable, terminate your command with a \G:
mysql
mysql> SELECT * FROM mysql.user WHERE User='root' AND Host='localhost'\G
***************** 1. row *****************
Host: localhost
User: root
Password: *********************
Select_priv: Y
Insert_priv: Y
Update_priv: Y
Delete_priv: Y
...
1 row in set (0.00 sec)


Make output from `SHOW PROCESSLIST` more readable

When you have a very large processlist that causes queries to take up multiple rows and scrolls the screen, you can make it easier to read by paging the output. By setting the pager to less -S,this will page the output for better readability. With the -S switch, this will cause lines to display one line instead of wrapping. By setting the pager, you can navigate output with the arrow keys. Set output paging with the following command:
mysql>pager less -S
To set the pager back, close the mysql connection or run:
mysql>nopager

Retrieve the output of your mysql command in HTML or XML format

When running a MySQL command using the command line mysql command, pass the -H or -X switches to receive your output in HTML or XML output, then store the output in a file. You can suppress column headers with the –skip-column-names switch:
$ mysql -u user -p password mysql -e'select * from table' -H --skip-column-names > output.html'
$ mysql -u user -p password mysql -e'select * from table' -X --skip-column-names > output.xml'

Skip a one-off error in replication:

If your MySQL replication is not running and you have determined that the error is a one-off situation, you can tell MySQL to skip the error and continue replication with a simple command. This command can only be run when the Slave_SQL thread is not running. This command skips event groups in the binary logs. In an InnoDB, and other transactional tables, an event group corresponds to an entire transaction. For nontransactional tables like MyISAM, an event group corresponds to a single statement. While it is possible to skip more than one error at a time, it si not recommended. To skip the current error, execute the following command:
mysql>SET GLOBAL SQL_SLAVE_SKIP_COUNTER=1; START SLAVE;

Disable binary logging for the current session:

Let’s say that you’re developing on your production data, which many of us do. In order to prevent your changes from being logged, and therefore replicated, run the following command when you enter the MySQL command line:
mysql>SET SQL_BIN_LOG=0;
This is a session variable, meaning it will be re-enabled when you close the session, or you can set it back to 1.

Load an SQL data dump into MySQL

Say you have a raw SQL file full of data that you need to import into MySQL. Assuming your CREATE TABLE statement is part of that code, you can import it directly in one of two ways, either through bash or mysql command line:
$ mysql -u user -p password < /tmp/data_dump.sql
mysql>LOAD DATA INFILE '/tmp/data_dump.sql';

Tell the optimizer which index you want to use:

The MySQL Optimizer generally does a great job of executing queries and selecting the right indexes to use. But let’s face it, sometimes, it gets it wrong. In these instances, you can tell MySQL which index to use when executing your query. The only way to know this however, is to test, test, and test. You have to know your data (data, indexes, query times, etc). The only way to find out what index the optimizer is using, is to use the EXPLAIN command on your SELECT query, which is a whole other post.
mysql>SELECT * FROM table USE INDEX (index_name);


Choose proper columns (and column order) for indexes

I’m not going to get into this in depth as indexing tables is it’s own post. However, for a quick and dirty indexing solution, try the following. Say you have the following queries you run regularly on your site with varying WHERE clauses:
mysql>SELECT * FROM table WHERE a=x, b=y, AND c=z;
mysql>SELECT * FROM table WHERE a=x AND b=y;
mysql>SELECT * FROM table WHERE a=x;
At first thought, you may just think to use a covering index that will satisfy all 3 queries:
mysql>ALTER TABLE table ADD INDEX (a,b,c);
However this may not be the best solution. The covering index is correct, but the order may not be. The point of an index is to speed up query processing time by returning as few rows as possible. If you have 10,000 rows in your database, and the WHERE clause a=x satisfies 9,500 of those rows, you aren’t making MySQL’s job any easier. What you want to do is find out which of your conditions returns the fewest rows because then, you’re running additional conditions on a smaller data set and your query runs faster. For example, a=x may satisfy 9,500 rows and then of those 9,500 b=y may satisfy 5,000. After a=x, you’re running the next condition on 9,500 rows of data. However, if b=y satisfies 6,000 rows initially and a=x then satisfies 5,000 of those, you’re query will run faster. This is because your second condition is run on fewer rows, meaning faster processing times. This can all still be done through a covering index, but you need to find out which of your conditions is the most restrictive. To do that, run the following command:
mysql>SELECT SUM(a=x), SUM(b=y), SUM(c=z) FROM table;
What this will do is tell you the total number of rows that satisfy each condition. So if your output looks like this:

mysql>SELECT SUM(a=x), SUM(b=y), SUM(c=z) FROM table;
+----------+----------+----------+
| SUM(a=x) | SUM(b=y) | SUM(c=z) |
+----------+----------+----------+
|    10000 |     6000 |       17 |
+----------+----------+----------+
1 row in set (0.00 sec)

Then c=z condition is the most restrictive and so c should be the first column in your covering index. Run your query again, this time with a WHERE clause of c=z to find the second most restrictive condition, and so on and so forth.
mysql>SELECT SUM(a=x), SUM(b=y), SUM(c=z) FROM table WHERE c=z; and you may get

mysql>SELECT SUM(a=x), SUM(b=y), SUM(c=z) FROM table WHERE c=z;
+----------+----------+----------+
| SUM(a=x) | SUM(b=y) | SUM(c=z) |
+----------+----------+----------+
|        5 |        2 |       17 |
+----------+----------+----------+
1 row in set (0.00 sec)

From here, b=y is the next most restrictive condition. So your index should be built like this:
ALTER TABLE table ADD INDEX (c,b,a) and your queries will run faster.

The one drawback to this is that if c is the first column in your index, but it is not specified in the WHERE clause, as in the first two queries above, the index will be ignored. The only way to remedy this, is to create another index for those two queries, whereas the initial (a,b,c) index would satisfy all queries. The only way to determine which solution works best for you is to benchmark each option.

If you have any MySQL Tips or tricks of your own, please share them in the comments below to help everyone out.

Top 20+ MySQL Best Practices

What I’m wondering is, as the title suggests, where do you find your “reading material”? What is your blog, your business, your site, your whatever, about and what do you find to be the most effective means of finding new information, whether it’s for your site, or for your personal learning? Please share and hopefully we can all find some new means of gathering information.

By changing a few data types and adding a couple indexes, I’ve taken queries from this plugin from examining 140k plus rows, down to examining less than 1800 rows, a 78% decrease in rows examined. Instead, I recommend “Google Analytics for WordPress” or “Wordpress.com Stats” (the latter requires an API key from wordpress.com, the former is obviously used in conjunction with Google Analytics, which is a great stat tracking software).

I’ll agree that the features of the plugin are nice, including the real-time monitoring, the GUI, and the sheer amount of data collected. However, the design flaws mentioned previously can spell disaster for your website, especially if it is a popular site. The information provided by the StatPress plugin is useful. However, when you scratch just below the surface of this plugin, beyond the stats, the plugin itself is horrible.

I installed this plugin today to start tinkering with it to know it better so I could improve it. After only 9 hours of collecting data, I have 270 rows of data on a blog that receives only minor traffic. That’s approximately 720 rows of data per day and 21600 rows of data per month, and nearly 130,000 rows of data in 6 months. Anyone who knows anything about database design, querying a table with 130,000 rows of data and no index can spell disaster for your blog. And that’s only on a minor traffic blog. If your site gets more traffic than that, you’re in serious trouble.

I am currently working on fixing all the faults in the database design and query execution of this plugin and will post here once I have, as well as on WordPress.org. If you have further questions or comments about this plugin, please post them here and I will do my best to answer.

You may wonder why I didn’t link to the WordPress.org plugins page or the developer’s home pages for these plugins, but it’s because I don’t want to send traffic to them as, since i mentioned, they are not useful plugins. You can find them and install them on your own if you want, but as you can tell, I wouldn’t recommend it.