How to backup 8GB file or larger on fat32 from linux

I have external USB hard drive that use Fat32 (so I can use it on windows machine as well on GNU/Linux). I needed to reinstall my OS (Xubuntu) and repartition the disk, so I need to copy all my files to external drive. But I’ve got 15GB virtualbox file.

On GNU/Linux or MacOS X you can use standard tools to split the file and then merge it back.

Here is a code that will compress and create 4GB chunks.


tar czvf file.tar.gz directory/
split -b $(echo "4*(2^30)-1" | bc) --verbose file.tar.gz file.tar.gz-

it will create files like file.tar.gz-aa file.tar.gz-ab that can be copy to Fat32 partition because they are smaller then 4GB (2^30 is 1GB in bytes).

to merge those files together you can just cat them


cat file.tar.gz-aa file.tar.gz-ab > file.tar.gz

if you want to see progress bar you can use pv command. This is the smallest way I came up with to do this:


cat file.tar.gz-aa file.tar.gz-ab | pv -s $(ls -l file.tar.gz-aa file.tar.gz-ab | cut -d' ' -f5 | tr '\n' '+' | sed -e 's/+$//' -e 's/.*/echo -n $(( & ))/' | bash) > file.tar.gz

Here is a function that can be put into .bashrc file


function cat-pv () {
    cat $@ | pv -s $(ls -l $@ | cut -d' ' -f5 | tr '\n' '+' | sed -e 's/+$//' -e 's/.*/echo -n $(( & ))/' | bash)
}

Here I found shorter way to calculate size of list of files it use awk:


ls -lrt | awk '{ total += $5 }; END { print total }'

So the function can be strink to


function cat-pv () {
     cat $@ | pv -s $(ls -lrt $@ | awk '{ total += $5 }; END { print total }')
}

, , ,

  1. #1 by joseph on November 12, 2014 - 00:19

    split is a real nice utility.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: