I set up mod_disk_cache and mod_cache for Hardcoredreamer.com - specifically for browsing the albums.  We were  using mod_mem_cache on a server wide level. So far, the performance statistics are breathtakingly amazing. Recent pages are loaded right from the cache with long expiration times so PHP and MySQL are not even bothered.  I ran a test of 10,000 connections to the random page in 17 seconds. I may expand this to other sites, but it would need to incorporate mod_expires in the .htaccess files and some other tweaks.

Apache’s guide on caching. The Gallery2 forum page that gave the most tips!

Basically we have 4 levels of caching going on, and the goal is to balance out obsolete pages that disrupt users and the fastest loading time possible!

I wanted to help out the great site, The Free Sound Project, so I set up Squid proxy on our server. It is pretty amazing that we are now serving out twice as many bits of traffic, yet it has a negligible effect on our server load and other resources!

From their site:

The Freesound Project is a collaborative database of Creative Commons licensed sounds. Freesound focusses only on sound, not songs. This is what sets freesound apart from other splendid libraries like ccMixter. New to this site? Read the What is Freesound page to learn more!

What I’ve gained is much experience using Squid, which gives me ideas on how to use it to get much more performance out of webservers in general. What I find cool about this is that the use of the http://freesound.cyborgcow.net proxy is invisible to you, when you are browsing the site. When you preview or download a sound, it actually grabs it from one of several mirrors.

I am always looking for ways to contribute to Open Source or Creative Commons projects. The Creative Commons liscense comes in several flavors, but a good example is the share and share alike license that Nine Inch Nails released their Ghosts EPs. That means you can legally download those songs for free! On top of that, you can remix the songs and release them. This spurs innovation.

By assisting a large collection of free, Creative Commons licensed audio files and samples, I hope to assist artists all over the world in creating new and amazing projects!

OSX saves .DS_Store and other resource-fork metadata files such as .Appledouble onto hard drives and network shares. These normally are completely invisible to a Mac OSX user, but once you start to browse the drives with different operating systems, you get all sorts of helpful files that can be annoying or get in your way.

These show up either with period, or as a hex number beginning with a : with netatalk, which allows a faster and more native sharing of APF over TCP. With FTP or SMB servers, you can ignore or veto the creation of these files.  A way to avoid copying resource forks & metadata to network shares, you can try this hint to disable it as a finder preference.

It seems like nobody has ever discovered that the Darwin OSX man page for the terminal command “cp”, which is used to copy a file or directory recursively, has built a SIMPLE parameter to completely skip any of these!

First, I mounted my network share, which is just called rayray using Finder’s Connect To Server menu item. This put rayray in /Volumes/rayray Then I can run cp with the -X parameter!

cp -vRX /path/to/folder /Volumes/rayray

This will show each file as it is copied and happily it goes *fast* and skips all of those files!

By the way, I still have to test NFS’s speed, but with netatalk using Appletalk over TCP Apple File Transfers, I get 50-60MB/s over a gigabit crossover cable as opposed to 30-35MB/s with Samba. Kinda neat

I want to point out that this person’s blog was the most helpful resource to getting Avahi-Daemon to advertise exactly what I wanted, so if you’re setting up a Linux server on your network with services that you want to appear in Finder, Terminal, and other places automagically, check out HolyArmy.org

Success! Every thumbnail, resize, and movie still has been built! It took quite a bit of work and some switching around of certain plugins, but the gallery does not need to generate any images when a picture is requested.

In the meantime, I have munin monitoring the server and I am very pleased with how smoothly everything is running. I have given MySQL much more ram than it needs for the sake of performance, and have been watching the free memory to make sure there is enough for http processes. Of course, I was able to free up many processes by using mod_security smartly.

I’ve set up the new server with a nice new firewall, very powerful virus scanning, and found time to update to a new version of ffmpeg module for gallery. This one lets you convert to a flash file, but for some reason (even after a lot of wasted time) it says that it can’t do audio, even when it should. When that is fixed, videos can take up less space, load faster, and be embedded easily!

I was going to take this move slowly, but I have a job interview and pretty important things going on that make me want to get this done right now so that just in case there is still time to get information off the old server.

I have not left my computer in 8 hours, and I’m starting to hurt… I think I may deserve a break in another hour. I have technical questions to review for this job interview!

Dual Opteron Goodness!

The site will be slow while I regenerate all of the thumbnails and both midsizes, which are at 640×640 and 1024×1024 and at 95% quality. That’s 276,190 images that it is processing!

While I’m doing this so that everything can be even higher quality, I’m also using it to compare the processing power of the new server. =) Lets just say I am very pleased!

In moving the gallery, the first attempt took 6 hours and failed because my internet connection at home hiccuped. Overnight I used Rsync to transfer all of the images. It was so much easier, I will be using that more often.

I am going to be migrating all of us to a new Dual Xeon server within a week. Hardcoredreamer.com will be the first site to be moved, so I can make sure if anything is affected that it will be my site and nobody else’s. I’m hosting 153 domains right now, and take pride in everything running as smoothly as possible.

I am also going to get to post how-to guide for using your album in really fun ways- such as having random desktops on your computer, embedding into your Livejournal or Myspace, and how to get random pictures on your Facebook profile.

I am currently working on four websites getting them up and running, including my own, but things are going to get very fun! I have a lot to share that has been in the dreaming stage for far too long.

I am going to get together a Video gallery, which will be spearheaded by a documentary by M-Pire’s Matt Lipinski. [M-Pire Gallery]