1. Ensure that you have Handbrake installed on your mac.
2. Insert the DVD; close the dvd player app that automatically starts.
3. Open Handbrake.
4. Choose the “Source” button, and choose the dvd.
5. In the right try, choose iPhone & iPod Touch
6. Now comes the hard part. You have to figure out what you actually want from the dvd. Different dvd’s partition their stuff differently, and it’s almost never on the chapter/scene partitions that you see in a regular dvd player. I typically look at the times. Something that’s 12 seconds, well, that’s probably what plays in the background for the dvd menu, or the intro when you first put it in. Something that’s 26 minutes, well, that’s be your payload – or at least one of them. It’s generally best to keep them in the same order that Handbrake shows them. If there are multiple payloads, you’ll have to rip each one by adding them to the ripping queue individually.
7. Choose the destination folder so you know where the ripped files will be stored.
8. Ensure that mp4 is selected – it will create m4v files (odd, I know, but it works). Also, the video codec should be H.264.
9. The filename ends up being what is displayed on the iPhone, and that display is not very much – perhaps 20 characters. As a result, don’t name your things like “French In Action Lessons – Lesson 1″ – because they’ll all look like “French In Action Less….” and you will have no idea which is which. Instead, name them “L01 – French In Action”.
10. Click “Add to queue” button. The first DVD section is now ready to be copied!
11. Repeat the previous steps for each of the segments you wish to rip from the DVD, ensuring that the file names are numbered somehow so you know which order to watch them in. In iTunes, each one will appear as a totally separate movie.
12. Click “Start” and Handbrake will begin the ripping.
13. Get yourself a cup of coffee.
14. Open iTunes, and bring up in Finder the folder that contains your ripped m4v files.
15. Drag your m4v files into iTunes, into the library area. They should automatically get added to the movie section.
16. Now sync with your iPhone like normal.

The solution: Attach the external hard drive to the mini (which will handle backing up the mini), and setup the MBP so that it backups up to the drive through the mini.

The steps:


1. Plug the external drive into the mac mini, ensure it’s powered up.
2. On the mini, setup the time machine to backup to the external drive like normal.
3. On the mini, open Preferences->Sharing
4. Check the correct box to enable File sharing
5. Ensure that the external hard drive appears in the “Shared Folders” window.
6. On the MBP, enable “unsupported” network volumes by entering this in the terminal window:

   defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1

7. Open finder in the MBP, ensure you’re connected to the mini. Click on the external hard drive. It should appear on your desktop.
8. Go into time machine, the external drive should now appear as an option. Select it and you’re done.

1. Boot up an instance of the AMI, any AMI, using elasticfox, the AWS console, or a CLI command.
2. Update the latest and greatest EC2 API tools (version 1.3-46266 2009-11-30 at a minimum)
   • For CentOS, copy the bin and lib files to /home/ec2 – you might need to blow away all existing files in these directories to work right.
   • For Ubuntu, mkdir /home/ec2 and copy -R the bin and lib dirs in there, then export EC2_HOME to /home/ec2
   • Get latest api tools from here.
3. Download your privateKey and 509Certificate pem files (this doc assumes you put them in /root)
4. Export the location of the pem files to your environment (not doing this causes certain commands to emit a “file not found: ””)
   
# export EC2_CERT=/root/my_509Certificate.pem
# export EC2_PRIVATE_KEY=/root/my_privateKey.pem

5. Ensure the EC2 binaries are in the beginning your path (/home/ec2/bin)
6. Download the AMI you want to convert from S3 using this formula:
   
# ec2-download-bundle -b $MANIFEST_PATH -a $ACCESS_ID -s $SECRET_KEY -k pk.pem -p $MANIFEST_PREFIX -d /mnt
Example:
# ec2-download-bundle -b ami.bucket -a AAAAAAAAA -s SSSSSS -k /root/my_privateKey.pem -p 20090926_centos -d /mnt

7. Unbundle the AMI using the following formula (will take a while)
   
# /usr/local/bin/ec2-unbundle -k pk.pem -m /mnt/$MANIFEST_PREFIX.manifest.xml -s /mnt -d /mnt
Example:
# ec2-unbundle -k /root/my_privateKey.pem -m /mnt/20090926_centos.manifest.xml -s /mnt -d /mnt
Creates one big file called /mnt/20090926_centos

8. Create a volume in the same availability zone as the instance you’re doing all this in, and attach it as /dev/sdh (elasticfox is the easiest way to do this)
9. Copy the ami file to the EBS volume (this may also take a while)
   
# dd if=/mnt/20090926_centos of=/dev/sdh

10. Now that the entire AMI has been copied onto the EBS, mount it
    
# mkdir /vol
# mount /dev/sdh /vol

11. Adjust the fstab in the EBS and remove the /mnt line – this is the ephemeral store that non-EBS instances get, but which EBS volumes don’t. Keep a backup, just because it’s good practice.
    
# cp /vol/etc/fstab /vol/etc/fstab.orig
# vi /vol/etc/fstab

12. Unmount the volume and detach it.
    
# umount /vol
Detach via elasticfox.

13. Create a new snapshot in elasticfox from this new EBS volume. This takes some time.
14. Register the snapshot as an image using this formula (use i386 or x86_64 for architecture):
    
# ec2reg -s $SNAP_ID -a $ARCH -d $DESCR -n $MANIFEST_PREFIX
Example:
# ec2reg -s snap-e92a2b80 -a i386 -d '20100310' -n 20090926_centos

15. It may take a few minutes, but an EBS backed volume should now appear in your AMI list. It is safe to delete the original EBS volume now; only the snapshot is used.

1. Insert the dvd/cd, wait until it appears on the desktop.
2. Type drutil status
3. In the output, find the name; it will be something like /dev/disk1
4. Unmount the disk (it will disappear from the desktop): diskutil unmountDisk disk1
5. If creating an ISO of a CD, you can’t use the name as-is: you need a slightly different device. If the name is /dev/disk1, then do ls /dev/disk1* to find the actual disk, something like /dev/disk1s1.
6. Perform the dd command for dvd: dd if=/dev/disk1 of=file.iso
7. Perofrm the dd command for cd: dd if=/dev/disk1s1 of=file.iso
8. If you want, mount the disk afterwards (so it appears back on the desktop): diskutil mountDisk disk1

I used disk1 in these examples, you would need to use whatever name that is specified in drutil.

With OS-X, Macintosh took the FreeBSD core and built their new operating system. For whatever reason, the ports collection didn’t come along with it. However, the open source community has come to the rescue with macports. And, once installed, its even easier to use than the FreeBSD ports collection.

First of all, go to the macports home page and download the latest copy. You’ll probably want to backup your system first.

The macports people were very kind in that all your ports will be installed in the /opt directory – not /usr, like in BSD. That’s so they don’t conflict with the Mac operating system files. That way, you can have the mac ports version of MySQL installed at the same time as the built-in mac version of MySQL – and they don’t have to know about each other (if they both try to listen on the same port then they will conflict, but the point is that the databases are segregated on disk).

So here are the basic steps I took to setup PHP 5.2 on my mac – it came with 5.3, but I needed 5.2.

1. Download XCode from apple (download here)
2. Download the Macports DMG file and install it
3. Update the system path to include: /opt/local/bin:/opt/local/sbin
4. Update the port tree: $ sudo port -v selfupdate
5. Search for php ports (partial okay): $ port search php
6. Install the port: $ sudo port install php52
7. Watch the output for instructions.

Note: It’s installed in /opt/local/var/macports
Note: Ports are in /opt/local/var/macports/source
Note: man 1 port to access man page
Note: Mac ports wiki is located here.

The instructions I saw that were useful consisted of the following items:

• To cause Apache to start at boot time: $ sudo launchctl load -w /Library/LaunchDaemons/org.macports.apache2.plist
• To customize PHP, copy:
  • /opt/local/etc/php5/php.ini-dist (if this is a development server)
  • /opt/local/etc/php5/php.ini-recommended (if this is a production server) to
  • /opt/local/etc/php5/php.ini and then make changes.
• To enable PHP in Apache, run:
  • $ cd /opt/local/apache2/modules
  • $ sudo /opt/local/apache2/bin/apxs -a -e -n "php5" libphp5.so

The original, 5.3 version of php was located in /usr/bin.

More information can be found at this excellent blog.

$ sudo vi /etc/paths

Enter your password, then add a new path anywhere in the file. Each line is its own path. For those who are not vi junkies, here’s how you add a path to the end:

1. Shift-G : goes to the bottom of the file

2. o : lower-case “o” will “open” a new line below the cursor and put you in edit mode

3. Type the path you want.

4. esc : Presssing escape returns you to command mode.

5. :wq : The colon tells vi you want to enter a command sequence; “w” means write (save), and “q” means quit (exit) the file.

6. You should now be back at the command line.

If you want more instruction on how to use vi, there are probably thousands of sites that explain things. This is a cheatsheet I found after spending about 15 seconds on google (I searched for “vi basics”).

Since I don’t use spotlight on my macs, it does nothing but take up valuable real-estate in the upper right corner. If I really wanted indexing, I’d probably use quicksilver, but even that isn’t really useful for me. I know where the stuff I want is.

The reason I want that real-estate in the upper right corner of my screen? I have recently purchased a 13″ macbook pro, and I found that with all the nifty toys I load, there are so many icons at the top of my screen that they start to conflict with the menu to the left. And no, I don’t want to upgrade to a larger screen: 13″ is the perfect size, given the clutter on my desk and the size of my lap.

Here are the commands, which should be entered in a terminal window (the dollar sign isn’t typed; that’s the prompt):

$ sudo mdutil -a -i off # this turns off indexing
$ cd /System/Library/CoreServices/Search.bundle/Contents/MacOS
$ sudo chmod 600 Search # this disables spotlight in the menu bar
$ killall SystemUIServer
reboot the machine

Note that the original permissions, in case you want to re-enable it, are 755.  Also, to reverse the indexer, just do the first command (mdutil) with”on” as the parameter instead.

The reason why 600 are the permissions is that it allows the system to perform any system updates to Spotlight without problems, yet disables it from running. The only possible downside is if there is some starter process that keeps trying to start Spotlight and logs its failures. I haven’t read anything about this, but you never know – I’ll have to keep an eye on my hard drive space.

First of all, I defined an application root directory – this would be something stored as an environment variable or some other mechanism so that every piece of code in this application was aware of it.  With certain exceptions, all codeknows about what happens from the application root directory down – anything outside of this master directory does not exist as far as the app is concerned.

The main reason for this is portability.  On my linux and FreeBSD installs, I use /usr1/myapp as the root directory, but on my windows dev box, I use C:\myApp.  When I develop using cygwin on windows, then my app root is C:\cygwin\home\corey\myApp.  The fact that all code works in relationship to this means that I can simply copy over all of my source, and everything is seamless.

Next, I decided to divide the responsibilities of my application with regards to boxes.  I imagined one or more boxes to house my web servers, one or more boxes for my databases, one or more for batch processing, and so on.  This is what makes it scalable; instead of starting small with everyhing on one machine and then going through headaches later trying to split them out, I started with them split out in the beginning – although with proper design, I can still run them all on one box if I want.  It’s simply a matter of configuration.

In its final form, the high level directory structure in my application root looked like this.  Notice that the directories start with “b_”; this is to indicate these are “box” directories.

• myApp/b_www – application specific code that is related to the http server.
• myApp/b_db – scripts, backup directories, and other items related to the database server
• myApp/b_batch – items related to back-office or batch processing; i.e., non-time critical operations like sending emails
• myApp/b_ftp – things related to the ftp or file operations.
• myApp/b_ms – Programs and support structure for the part of the system that can only run on a Microsoft platform
• myApp/b_common – scripts, programs, and configuration files that are common to all boxes

All code goes on all boxes. This enables the code to have access to practically any functionality while allowing me to find the code in its logical place. For example, the scripts I wrote for accessing Amazon S3 file storage – upload files, downloading files, getting listings, and so on – are located in b_ftp because they are related to file operations. However, many of my batch programs need to access the S3 functionality for backing up databases and such.

It requires long term discipline; when adding a new script or program to the system, I will first consider what the code is doing.  If it’s doing things that relate to both batch and ftp, for example, then perhaps it’s best to split it out – a library-type API for the lower level stuff and a high level script for the rest.  It’s very easy to put the wrong script in the wrong place.

Once the high level directories are defined, their contents can vary quite a bit.  For example, my b_www directory contains an htdocs and logs subdirectory, and in other respects looks very similar to any other web server’s area.  However, my b_ftp directory contains dozens of jailed user directories for the various user accounts that need ftp access.

By splitting the application into gross logical sections, maintance becomes much easier, and understanding the directory structure helps to understand the overall application architecture.

I was re-acquainted today with something that I had learned previously and forgotten:  Whenever you bundle a new linux AMI, always, always, always start it up and make sure you can get in.

Why, do you ask?  Because sometimes, just sometimes, something kills the sshd server shortly after it is started.  I see a signal 15 (SIG_TERM) being received by sshd.  I can’t remember exactly which log, either /var/log/messages or /var/log/secure, but it’s there.  I’ve googled for a solution to this, and the best anyone has come up with is “something is killing your process, go through your startup scripts with a fine-toothed comb and put in some debug, recompile the kernel, yada yada yada.”

Truth be told, the entire reason why I like linux on amazon is because I DON’T have to do any of that stuff.  I just configure the system, take a snapshot, and presto-whammo, I have the ability to create identical copies of my wonderfully configured OS at a click of a button.  So to be honest about the answer to the question posed in the title of this post, why doesn’t ssh work on my newly bundled instance?  Because something killed it.  What killed it?  I don’t know, and frankly, I don’t care.  It shouldn’t have happened.

I’ve also determined that once an AMI exhibits this behavior, it’s a goner: I every time I start one of them, sshd is always killed, every time.  So what to do about it?

Solution #1: Try again

If you’re like me, you make a lot of custom AMI’s until you finally get things exactly right.  That means that you’ll have previous images that may or may not do this whole “kill the sshd” thing.  The first thing to do is locate the most recent AMI where sshd wasn’t killed on startup, and startup an instance of that.   Then, apply your changes again (hopefully you’ve documented or remember them), and create a new AMI.

Solution #2: Wait a while, then try again

So I tried solution #1.  I found the last working AMI, started it up, made my changes, bundled a new AMI, started an instance of the new AMI – and… same problem.  I did it again to be sure, and sure enough, the second “new” AMI still didn’t work.  Were the Amazon gods cursing me?  Was my destiny to keep this last, golden AMI and modify it no further?

My gut told me to wait, so I did.  I kept my “vanilla” instance running and about four hours later, I tried making another snapshot.  And that AMI worked!

Solution #3: Start up the server manually

The instance I use of centos comes with webmin installed – and it’s worth its weight in gold.  I use it as a backup for everything else; because it’s designed to be an administrative interface, it can do most of what I normally do on the command line.  Like starting up sshd.  This is how I figured out that sshd was getting a SIG_TERM; after I started it using webmin, I could get in normally and nobody tried to kill it again.  Who knows, maybe this problem is created because of webmin trying to be everybody’s daddy, but at least it gives me the ability to fix things.

I wouldn’t consider this a long term solution, because I don’t want to have to do this every time I start one of my instances.

Solution #4: Start from scratch

I hope no one has to do this, especially if they’ve devoted a significant number of hours customizing an AMI.  But worst case scenario, you can always abandon all your AMI’s, find the original that you started from, and try applying all your changes in one shot.  It might be that an AMI that is only one bundle-build away from the original will be more reliable than one that is 30 away – but man, I hope not!  I’m already well past 30 away, and it really shouldn’t be that way anyway.

Solution #5: Figure it out

This is the option I skipped, but which more gifted and/or perseverant sysadmins might be willing to tackle.  I have no guidance here, other than telling you that google is your friend.

Solution #6: Abandon all hope…

Find a more profession more to your liking, like dive instructor or professional beach bum.  I hear that they don’t ever have to worry about these kinds of issues.

Conclusion

I’m sure that an operating system using zen to run on a cloud like Amazon’s is a complex beast, and there are doubtless things that might go wrong or vary where nobody ever expected them to vary.  It’s nice having the illusion that I have my own little dedicated server out there in internet-land, but issues like this slap me in the face occaisionally to remind me that my operating system is floating on a cloud of complexity.   The operating system next to my desk is sitting on hardware I can see and hear and touch, but my operating system on the cloud… yikes!

It’s like one of those little cocoons in which the humans were kept in the Matrix.  The human who is experiencing life in the cocoon has absolutely no idea what other cocoons are close to him, how the cocoon works, or what kind of support structure is built around the cocoon to ensure that the whole system stays up and running.  To the human, life isn’t about the cocoon.

And so it is with Amazon’s cloud.  I don’t know what other servers are close to my instance – for all I know, I’m on the same box as those people working on the successor to the genome project.  I don’t know how Amazon does its magic – other than in the abstract sense of understanding what virtual machines are all about – and I don’t know anything about what kind of resources they put into making sure everything works all the time.  But I sure appreciate that they do.