September 13, 2007

Carry the gibbon in your pocket: How to make a liveUSB of Gutsy Gibbon with persistence

The live CD is a fantastic tool to try out linux and many distros have them now. The newer feature is 'persistence' where changed settings are stored for future sessions. Combining persistence with a live USB allows you to really carry your desktop with you on just a USB stick. These are the steps that I needed to make a live USB with the Tribe 5 release of Ubuntu Gutsy.

This is not intended to be a standalone howto. I am not duplicating all steps and workarounds detailed in the Ubuntu wiki, so I suggest that you have that by the side if you are trying this. Also, I refer to the disk as /dev/sdb because that was what it was for me. I cannot emphasize enough that this may be different for you and blindly copying the commands can end up in formatting another drive

WHAT YOU NEED:
1. An USB drive with capacity of atleast 1 GB.
2. A live cd of Gutsy - The iso will also do.
3. A PC with linux or Windows.
I am doing this with a Maxtor 2 GB drive using a pc running Ubuntu Fiesty and a live cd of Ubuntu Gutsy.


STEP ONE: Partition the disk

You will need to create atleast 2 partitions.
One will be a primary FAT16 partition of 750 MB that is bootable.
Second will be a primary ext2/ext3 partition for the persistence feature.
I used 200 MB for the second partition and created a third partition in the remaining free space as FAT16.
So plug in the USB drive and identify the device name for the drive. It is /dev/sdb for me and it is important to find this correctly so you dont partition or format another drive. The output for 'dmesg |tail', 'df -h' or 'mount' should help identify the correct drive. Opening the gnome partition editor also makes it easy to recognize the device.
The partitioning itself can be done with either gparted, cfdisk or fdisk. The disk must be unmounted before doing the partitioning and all three methods are fairly easy, though someone used to GUIs may be more comfortable with gparted. These are a couple of screenshots with gparted.







Using fdisk and following the tutorial in the wiki also works perfectly (I tried it too). You will not yet be able to assign a label with any of these methods. As seen in the last screenshot, I ended up with the three partitions.
Now the following commands assign labels to these partitions - Ubuntu keeps automounting the partitions, so I had to unmount them before this step.


sudo mkfs.vfat -F 32 -n gutsy /dev/sdb1
sudo mkfs.ext2 -b 4096 -L casper-rw /dev/sdb2
sudo mkfs.vfat -F 32 -n data /dev/sdb3


Remember to change the device name if the name is not /dev/sdb* for you.
So I have the three partitions with the labels now. The second partition has to be labeled 'casper-rw' so that is is recognised by the system as the partition to use for storing changes. You can choose different labels for the other two, if you wish.
If the device is not mounted now, mount it manually or just unplug it and plug it back in to allow it to be automounted.


STEP TWO: Copy necessary files

I used the cd I had already burnt as the source for the files. If you have an iso only, you can mount the iso like this and use it as the source


mkdir /tmp/ubuntu
sudo mount /path/to/iso /tmp/ubuntu -t iso9660 -o loop


Now cd into the source directory (/media/cdrom or /tmp/ubuntu) from a terminal and then do the following to copy the files to the gutsy partition on the disk. I am assuming here that the partition is mounted as /media/gutsy.



#cd into source directory
cd /media/cdrom

#Copy these folders
cp -rf casper disctree dists install pics pool preseed .disk /media/gutsy

#Copy all files from isolinux directory
cp isolinux/* /media/gutsy

#Copy these files into the root destination directory
cp md5sum.txt README.diskdefines ubuntu.ico /media/edgy

#Copy the kernel image and initrd from casper directory and mt86plus from install dir
cp casper/vmlinuz casper/initrd.gz install/mt86plus /media/edgy

#rename isolinux.cfg as syslinux.cfg
mv isolinux.cfg syslinux.cfg


You can modify syslinux.cfg like this. Alternatively you can copy this and paste it in syslinux.cfg replacing everything else.


DEFAULT custom
GFXBOOT bootlogo
APPEND preseed/file=preseed/ltsp.seed boot=casper initrd=initrd.gz ramdisk_size=1048576 root=/dev/ram rw quiet splash --
LABEL custom
menu label ^Start Ubuntu in persistent mode
kernel vmlinuz
append preseed/file=preseed/ltsp.seed boot=casper persistent initrd=initrd.gz ramdisk_size=1048576 root=/dev/ram rw quiet splash --
LABEL live
menu label ^Start or install Ubuntu
kernel vmlinuz
append preseed/file=preseed/ltsp.seed boot=casper initrd=initrd.gz ramdisk_size=1048576 root=/dev/ram rw quiet splash --
LABEL xforcevesa
menu label Start Ubuntu in safe ^graphics mode
kernel vmlinuz
append preseed/file=preseed/ltsp.seed boot=casper xforcevesa initrd=initrd.gz ramdisk_size=1048576 root=/dev/ram rw quiet splash --
LABEL check
menu label ^Check CD for defects
kernel vmlinuz
append boot=casper integrity-check initrd=initrd.gz ramdisk_size=1048576 root=/dev/ram rw quiet splash --
LABEL memtest
menu label ^Memory test
kernel mt86plus
append -
LABEL hd
menu label ^Boot from first hard disk
localboot 0x80
append -
DISPLAY isolinux.txt
TIMEOUT 300
PROMPT 1
F1 f1.txt
F2 f2.txt
F3 f3.txt
F4 f4.txt
F5 f5.txt
F6 f6.txt
F7 f7.txt
F8 f8.txt
F9 f9.txt
F0 f10.txt


Note that you have most of the boot options as in the live cd, but the default is the custom option, which is the persistent mode here.


STEP THREE: Make USB drive bootable

We will install syslinux and mtools and use them to make the drive bootable


sudo aptitude install syslinux mtools
sudo umount /dev/sdb1
syslinux /dev/sdb1


Worked perfectly for me.


STEP FOUR: Checking it

Reboot and change bios settings so that you boot from the USB drive. I was able to boot in without problems. I created a username and password (default user is 'ubuntu' without a password). I logged into my new account and changed terminal settings, changed the metacity theme and installed gnumeric. Shutdown, reboot again - and voila - all the changes are there!

I had problem in a later session where I could not log into gnome with gnome complaining that it could not read the .ICEauthority file. Logging into a failsafe terminal and deleting the ~/.ICEauthority file allowed me to start gnome again.
The casper-rw partition is the place where all data and changes are stored without any compression. So if you store any data in your home folder, you can see it here - that means it is is easy to retrieve it without booting in with the drive and also has implications for privacy.

Things to explore:
Find what is the ideal size for the casper-rw partition?
Using an encrypted home or data folder?
Dual boot with another linux distro?
Is it possible to not automount the gutsy and casper-rw partitions by modifying udev rules so that all three partitions dont mount every time I insert the drive in my laptop?

Summary:
The process was much simpler than I had expected. Now I have a bootable Ubuntu drive with me all the time, while the 1 GB FAT partition is enough storage to use as a regular USB stick. Now when I am urging a friend to try out Linux, I know I have the gibbon ready in my pocket!

7 comments:

endoplasmic said...

Did you also consider this approach (which includes encryption):
http://feraga.com/
http://feraga.com/project/deb2flash

I'm curious of your reasoning to select the live option. With the automated flash installer at feraga, it's not clear to me why a (more restrictive?) live install is so popular (?). e.g. the value of a compressed filesystem seems dubious with 4GB keys at $25

I wonder if something like configsave used in the slax distribution would be a nice option too, when running from RAM - to write occasional changes to disk. I've heard Gnome is 'chatty' and writes to disk frequently when in the live+persistent mode(?) - any insights?

Thanks for the post!

endoplasmic said...

Also, I forgot:

-how do you access the fat 16 data partition and casper-rw partition on your key from a windows machine? I heard only the first partition on a key is visible in windows

-why fat16 and not fat32?

-why ext2 and not ext3?

Thanks!

endoplasmic said...

this likely is the wrong place, but comments were diabled on the original post - will you be releasing your python script described here:
http://reachbeyondgrasp.blogspot.com/2007/03/tagging-pdf-documents-with-python.html

Thanks! Very helpful information!

endoplasmic said...

regarding: "complaining that it could not read the .ICEauthority file. Logging into a failsafe terminal and deleting the ~/.ICEauthority file allowed me to start gnome again."

Are you sure this works fine for you now? I tried this, plus deleting tmp, etc.. My install was only 2 days old too - yet it had over 500 empty files in tmp, it certainly isn't conservative on disk writes - I can't see how anyone can advocate a persistent live USB debian install at the moment...I wasted two days with this little experiment by the time I installed apps, got my graphics card working, etc.

perhaps a different window manager? xmonad?

I've used Debian/linux on the server for years - but each time I try it on the desktop (every 1-2 years) I get burned- I'm sure I'll give it a go in another two years...

Raja said...

Hi endoplasmic,
This post was primarily a how-to on making a live-USB with persistence. I did not mull over the alternatives.
I think your concerns are valid about the possible frequent disk writes. I am not sure if it very gnome-specific though.
All the same, I dont get your last line. Why do you have problems working with Debian on the desktop? I dont see how it is related to the other issues. I have been happy with an exclusively linux desktop for the last 2 years.

endoplasmic said...

Hi Raja,

>>not sure if it very gnome-specific though.

me either - basing it on the X error log complaining of .ICE*, but no idea if this is causing the crashes (sessions are never stable longer than a few minutes now) - could be correlation w/o being the causative link though

hopefully the memory on the key is still good - have not checked this yet (brand new 4GB Corsair)

I'm presuming everybody has this problem though(?) - but it is rarely mentioned on all the howto's and wikis discussing persistent live usb installs

>>Why do you have problems working with Debian on the desktop?

I think it was frustration at the time of writing, since I was staring at a worthless debian key that I had spent 2 days on. "debian" should actually read "linux" since I've tried several distributions. Previously, I've tried desktop linux and become frustrated by lack of driver support, and several applications I use are windows only, so it was not possible to shift. In all cases I found myself spending much more time on OS debug issues though, rather than productivity in my field of study.

To be fair, this latest venture was a persistent usb install - and it is not even possible to do this with windows. When I have another free weekend, I'll probably give the Feraga approach a try - he claims to be successfully using his for more than a year now.

Raja said...

endoplasmic,
The code for medlib, the application to parse and organize pdfs is hosted at google code (http://code.google.com/p/medlib/), but is still under development. Though I a using it now, it is not close to being ready for release.