Spiders and Bots are always scanning through your site and indexing the information as best they can. But how do you test your crawlability, to make sure the information you’re providing is indexed properly?

Enter SpiderTest and SEOBook.com’s Spider Test Tool.

Both crawl your site, but each reports back a separate set of info to help with improving your crawlability. SpiderTest gives a bulleted list of small technical improvements that will help with addressing the issues Googlebot has crawling your site. SpiderTest also reports some keyword statistical information on the page you set to crawl. While SEOBook.com’s Spider Test Tool gives more of a blunt, and less helpful, review of the page. Spider Test Tool does manage to list out some basic important info, like links on the page, the cache state of the page on search engines, and some rawly unrefined keyword info.

I would say SpiderTest is the more accurate for trying to see what GoogleBot sees and Spider Test Tool is more like what a crappier bot would see. But they are both worth a try!

Header Mime Types are one of the oldest ways for the web to communicate what type of media a file is (I would assume file extensions are the oldest way). Mime types are strings of meta-data sent when HTTP is communicating what the content type of a file being transmitted is. But, with this meta-data categorization there has been differences between browsers and servers regarding mime type strings and their association. This type of overlaying, now-a-days obviously duplicating, information is pretty standard for the older web ways of doing things, especially if they are still used in various forms in todays web world.

A decent example of this is javascript mime types. As listed below:

• text/javascript
• application/js
• application/javascript
• application/x-javascript

Overtime there has been duplication for various uses, but now are all still in standard use. So with something as common place as compression of javascript in gzip format for faster transmission over http, all listed mime types must be listed for the compression system to accurately compress all javascript files.

A list of all mime-media types are available on the IANA website.

If you have a comment or question, please respond below.

RSYNC is the fastest way to clone and sync directories from any server.
Below is an rsync example using ssh tunneling on port 2222 to a server using cPanel, thus the exclusion of the .cpanel and public_ftp. /home/user is the folder to clone/sync to /Volumes/HardDrive/BackupLocation. And a log file is created to log any issues at /Library/Logs/rsync.log.

rsync -az6 --delete -e "ssh -p 2222" --exclude 'public_ftp' --exclude '.cpanel' user@site.com:/home/user /Volumes/HardDrive/BackupLocation > /Library/Logs/rsync.log

The ‘az6′ flags stand for a – archive, z – compress, 6 – IP 6 and e – using a remote shell. You can read about more flags and options on the rsync man page.

If you know of a faster way to sync/clone a directory, please leave a comment below.

A lot of times when creating a webpage you’d like to be able to open an app from a link on that page. Luckily there is a way for app developers to add support for custom URL schemas, which means you can launch an app from a link on a web-page. To do that, you need to know the URL schema for those apps so the link will open the app correctly.
Here are the most complete lists of iOS URL schemes:

• http://wiki.akosma.com/IPhone_URL_Schemes
• http://developer.apple.com/library/ios/#featuredarticles/…

When writing code to dynamically create background colours and text colours, there is always the potential those two colours will not look appropriate together, and will either render the text illegible or create oscillating text.

Enter Split Brain .org‘s Color Contrast Checker

It’s a very helpful script, to prevent some illegible dynamic text colors from occurring and helps create visible text in PHP.