Posts Tagged bash

How to backup 8GB file or larger on fat32 from linux

I have external USB hard drive that use Fat32 (so I can use it on windows machine as well on GNU/Linux). I needed to reinstall my OS (Xubuntu) and repartition the disk, so I need to copy all my files to external drive. But I’ve got 15GB virtualbox file.

On GNU/Linux or MacOS X you can use standard tools to split the file and then merge it back.

Here is a code that will compress and create 4GB chunks.


tar czvf file.tar.gz directory/
split -b $(echo "4*(2^30)-1" | bc) --verbose file.tar.gz file.tar.gz-

it will create files like file.tar.gz-aa file.tar.gz-ab that can be copy to Fat32 partition because they are smaller then 4GB (2^30 is 1GB in bytes).

to merge those files together you can just cat them


cat file.tar.gz-aa file.tar.gz-ab > file.tar.gz

if you want to see progress bar you can use pv command. This is the smallest way I came up with to do this:


cat file.tar.gz-aa file.tar.gz-ab | pv -s $(ls -l file.tar.gz-aa file.tar.gz-ab | cut -d' ' -f5 | tr '\n' '+' | sed -e 's/+$//' -e 's/.*/echo -n $(( & ))/' | bash) > file.tar.gz

Here is a function that can be put into .bashrc file


function cat-pv () {
    cat $@ | pv -s $(ls -l $@ | cut -d' ' -f5 | tr '\n' '+' | sed -e 's/+$//' -e 's/.*/echo -n $(( & ))/' | bash)
}

Here I found shorter way to calculate size of list of files it use awk:


ls -lrt | awk '{ total += $5 }; END { print total }'

So the function can be strink to


function cat-pv () {
     cat $@ | pv -s $(ls -lrt $@ | awk '{ total += $5 }; END { print total }')
}

, , ,

Leave a comment

Shell without ssh or root on your shared hosting

My site jcubic.pl is on shared hosting that don’t have ssh, and I wanted to have a shell access, so using my jQuery Terminal, I’ve stared a project leash. It’s a shell written in javascript with help from php and python cgi. It will give you a shell access, no need to install any new software, and you don’t need to be root (like with other shells like shell in a box, AjaxTerm or Buterfly).

, , , ,

Leave a comment

Concatenate files with progress bar on GNU/Linux

When I download files form rapidshare or other file hosting sites I occasionally download movies that are splited into parts, and to watch the movie they need to be concatenated.

I came up with this bash function which I put in my .bashrc file:

function concat() {
  files=${@:1:$(($#-1))}
  cat $files | pv -s $(du -cb $files | tail -n1 | cut -d $'\t' -f1) > ${@: -1}
}

If you have files like Movie.avi.001…Movie.avi.020, to concatenate these files just type:

concat Movie.avi.* Movie.avi

And you’ll see progress bar when output file is created.

, ,

Leave a comment

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: