A couple rsync problems

A couple days ago I decided to move my podcast management to my desktop from my macbook pro. Since I’m running an SSD, I don’t have much space on the mac to store 80+ GBs of podcasts. So after rsyncing my current list and then painstakingly resubscribing to all of my podcasts in iTunes and adding the existing episodes, I decided to try to back them up using rsync again to my FreeNAS box.

Unfortunately, since I first tried to duplicate the command on Windows (even though I knew the file permissions would not map from Windows to Unix), the permissions of my main share folder were changed to 0000. Naturally, these are useless permissions for a network share. They also prevented the rsync backup from working on my mac.

Once I discovered the changed permissions, all that was necessary to fix the problem with my mac was to restore them using chmod. Then I turned to Windows to fix the permission settings.

I removed the ‘archive’ option from the command and replaced it with just recursive directories and preserve modification times (-rt). This fixed the problem and allowed rsync to successfully copy over all of the new podcasts. To be sure I decided to check the permissions on the new files. I was surprised to see that they were set to 0000. So I turned to the Internet to find how to make rsync set permissions.

Luckily it didn’t take long and I found this post on superuser. The answer is to add a switch to rsync to tell it to change the permissions. According to the poster, adding “–chmod=ugo=rwX” to the rsync command will tell the receiver to make the new files using the filemask. This created the new files with the expected permissions of 0644 as all of the other podcast episodes have been.

Lastly, since I had already copied over eight or so podcast episodes with bad permissions I decided to fix them. I knew it would be quite a bit of work to type out the file names of all of them or to go to each folder to find them and change them so I decided to see if you could find files by permissions. Thankfully the find command is capable of doing this using the “-perm” switch. Combing this with xargs and chmod, I had a one-liner that would find the files with broken permissions and change them to the proper ones.

$ find Podcasts -perm ugo=-rwx -print0 | xargs -0 chmod 644
Posted in Uncategorized | Leave a comment

Using a raw disk as a VirtualBox drive in Windows 7

The other day I was listening to episode 386 of Security Now! and learned from a listener who had written in that VirtualBox was capable of using physical disks for virtual machines and that this would work with SpinRite. Using this feature it is possible to run a SpinRite scan on a secondary disk in a virtual machine while still using the host machine. So to make sure I don’t forget how this was done and to hopefully help someone else out there, here are the steps that I took to get this working. An interesting thing to note here, since you can create raw drives in OS X, you should be able to run in the virtual machine a SpinRite scan on a secondary disk on a Mac!

Since it was listed on the manual page and I don’t want anyone to lose data, read this:

Warning

Raw hard disk access is for expert users only. Incorrect use or use of an outdated configuration can lead to total loss of data on the physical disk. Most importantly, do not attempt to boot the partition with the currently running host operating system in a guest. This will lead to severe data corruption.

For more information check out this section on creating raw disks in the VirtualBox manual https://www.virtualbox.org/manual/ch09.html#rawdisk.

Another final note: while I have not verified that this occurs, it is possible that Windows will assign a different disk number to the drive after rebooting. A few options for handling this include:

  1. Create a raw virtual disk for each drive number naming them, so you can attach the correct one to the virtual machine before running it but not need to recreate the raw disk files.
  2. After running the SpinRite scan delete the raw virtual disk vmdk file to make sure you don’t use it accidentally.
  3. Follow the instructions on this post by Kevin Cave to create a raw virtual disk file that will always point to the correct drive.

Creating the Virtual Drive

1. Connect your hard drive.

This should be obvious. The first thing you should do is connect up the drive that you need to scan to the host machine.

2. Find the disk number.

After you have booted your machine back up you need to find out what number Windows has given to your disk. You can find this information in the disk management pane. First, open the start menu and right click on “Computer”. Select “Manage” from the menu. Next, select “Disk Management” from the pane on the left under the heading “Storage”. In the middle section you will see all of your hard drives and removable media drives listed. At the top of the bottom half should be at least two disks, the first labeled ”Disk 0″. Find the disk number for the drive you want to create the raw disk for (because you can’t use your host machine’s drive as a raw disk, obviously this drive should not have the (C:) partition on it).

Here’s a screenshot showing my desktop’s drives, the primary drive first as “Disk 0″ and the drive I want to run the scan on is the second, “Disk 1″.

screenshot of disk management

3. Run VirtualBox as an administrator.

Right click on the VirtualBox shortcut and select “Run as administrator” from the menu. If / when UAC opens a box requesting permission to run, select “yes”.

4. Open up a command prompt.

Open the start menu and in the search box type “cmd”. In the list above right click on “cmd.exe” and select “Run as administrator”.

5. Navigate to the VirtualBox folder.

Next, you need to navigate to the folder where you installed VirtualBox. For me, this is the default location and since I’m running 64-bit Windows 7, the installer installed the 64-bit version of VirtualBox. For me this means I needed to “cd” (change directory) into the regular Program Files folder.

cd C:\Program Files\Oracle\VirtualBox\

Note: For the more savvy Windows users, all you really need to do is add your VirtualBox install directory to the PATH environment variable.

5. Enter the following command

To create the raw disk for use with VirtualBox type the following command in the command prompt filling in the file name and the disk number in place of the # symbol:

VBoxManage internalcommands createrawvmdk -filename "FILENAME" -rawdisk "\\.\PhysicalDrive#"

Make sure that you include the double quotes around the filename and the disk name. This ensures that if there is a space in your filename that the command uses the entire path.

This is the command that I ran to create mine as an example using the disk number shown in the screenshot above and saving the disk image to my Downloads folder.

VBoxManage internalcommands createrawvmdk -filename "C:\Users\dan\Downloads\internalssd.vmdk" -rawdisk "\\.\PhysicalDrive1"

6. Create a new VirtualBox VM for running SpinRite.

In order to run SpinRite, you’ll need to create a VM and select the type “Other” then OS “DOS”. Follow the wizard and select how much RAM you’d like to allocate (I chose 128 MB which worked fine) and you don’t need to create a virtual hard disk since you are not installing an operating system.

7. Add the newly created virtual disk to your VirtualBox VM.

Navigate to the location of the file you created with the command and add it to your SpinRite VM. Lastly, you’ll have to mount the SpinRite.iso in the VM as well by adding a new CD/DVD drive and selecting the file on your machine.

8. Start up the VM and run SpinRite!

If everything went fine, SpinRite should discover the drive and show it in the list.

Note: You’ll need to run VirtualBox as an administrator anytime you want to run the VM so it can access the drive.

spinrite welcome screenspinrite drive selectionspinrite runningspinrite finished

Running on OS X and Linux

The only changes to the above steps that should be required for OS X and Linux (I have not tested these but if they work for you or a something different is required let me know) is to change the name of the raw disk in the command. On Linux use “/dev/sda” or whatever your drive is. On OS X use “/dev/disk1″ or whatever your drive is.

Troubleshooting

Hopefully those steps worked for you like they did for me. If you get an error like, “VERR_ACCESS_DENIED” you probably didn’t run the command prompt as an administrator. If you get an error like,

VBoxManage.exe: error: Failed to create the VirtualBox object!
VBoxManage.exe: error: Code CO_E_SERVER_EXEC_FAILURE (0x80080005) - Server execution failed (extended info not available)
VBoxManage.exe: error: Most likely, the VirtualBox COM server is not running or failed to start.

Then you probably didn’t run VirtualBox as an administrator.

Relevant Links not given above:

Posted in SpinRite, Virtual Machines, VirtualBox | Tagged , , , , , , | 6 Comments

Unicorn Rails Ubuntu init script

Today I’ve been updating the TrustAuth project website to reflect the changes I made the last couple of months. While I was deploying, I rediscovered the terrible state of the Ubuntu init script I had been using to run Unicorn. It wouldn’t properly stop Unicorn so I had to SSH into my server and restart Unicorn when I updated the website. If I didn’t it would still be serving the old Rails pages. So tonight I set out to fix this.

I started by trying to figure out why the script wasn’t creating the file for the PID. Looking through the script I had found in this setup guide I realized that it wasn’t writing the PID anywhere. I tried to use pidof to output the PID of the process but it wasn’t finding any unicorn_rails processes despite there being five listed with ps aux. This gave me the idea to look at one of the existing init scripts that I have been using already; specifically the Nginx script.

In the script I found that every command was using start-stop-daemon so I decided to look if it would work with Unicorn. I wasn’t sure because unicorn_rails is actually a Ruby script that launches Unicorn. Sure enough with a little bit of tinkering I was able to get a working init script that starts, stops, and restarts Unicorn. Not only that but start-stop-daemon outputs the PID so I have that too. Here’s the init script based off the one from the previously mentioned guide and with my modifications:

By using `which unicorn_rails` I don’t need to update the script when I update the version of Ruby I’m using. I hope this helps you as much as it has helped me. I tested this on Ubuntu Lucid Lynx with Ruby 1.9.3-p194.

Posted in Rails | Tagged , , , , | 1 Comment

Setting up a symbolic link for Apache in OS X Lion

Today I was fixing my OS X installation that had been wiped out before and ran across a problem that I forgot about. I store the code for the TrustAuth website in Dropbox for convenience and sym linked it to the appropriate folder to run the site locally. Apache was giving me the error:

Symbolic link not allowed or link target not accessible: /Users/dan/Sites

What I forgot was that I had to do before was change the permissions on the Dropbox folder.

In order for Apache to follow the symbolic links not only does the option “FollowSymLinks” need to be set in the config but every directory along the path from “/” needs to have the execute bit set. My Dropbox directory did not have this set so I had to set it with:

$ chmod a+x ~/Dropbox

Once I changed that Apache could read the files just fine.

Posted in Apache, Mac OS X Lion, PHP | Tagged , , , | Leave a comment

Site back up!

Well, my blog has been down for a while but I finally got it back up so that I can test out the TrustAuth WordPress plugin! Checkout the TrustAuth blog for the latest information about TrustAuth.

Posted in Uncategorized | Leave a comment