jcubic

I'm graphic designer and programmer from Poland.

Homepage: http://jcubic.wordpress.com

How to backup 8GB file or larger on fat32 from linux

I have external USB hard drive that use Fat32 (so I can use it on windows machine as well on GNU/Linux). I needed to reinstall my OS (Xubuntu) and repartition the disk, so I need to copy all my files to external drive. But I’ve got 15GB virtualbox file.

On GNU/Linux or MacOS X you can use standard tools to split the file and then merge it back.

Here is a code that will compress and create 4GB chunks.


tar czvf file.tar.gz directory/
split -b $(echo "4*(2^30)-1" | bc) --verbose file.tar.gz file.tar.gz-

it will create files like file.tar.gz-aa file.tar.gz-ab that can be copy to Fat32 partition because they are smaller then 4GB (2^30 is 1GB in bytes).

to merge those files together you can just cat them


cat file.tar.gz-aa file.tar.gz-ab > file.tar.gz

if you want to see progress bar you can use pv command. This is the smallest way I came up with to do this:


cat file.tar.gz-aa file.tar.gz-ab | pv -s $(ls -l file.tar.gz-aa file.tar.gz-ab | cut -d' ' -f5 | tr '\n' '+' | sed -e 's/+$//' -e 's/.*/echo -n $(( & ))/' | bash) > file.tar.gz

Here is a function that can be put into .bashrc file


function cat-pv () {
    cat $@ | pv -s $(ls -l $@ | cut -d' ' -f5 | tr '\n' '+' | sed -e 's/+$//' -e 's/.*/echo -n $(( & ))/' | bash)
}

Here I found shorter way to calculate size of list of files it use awk:


ls -lrt | awk '{ total += $5 }; END { print total }'

So the function can be strink to


function cat-pv () {
     cat $@ | pv -s $(ls -lrt $@ | awk '{ total += $5 }; END { print total }')
}

, , ,

Leave a comment

Cross-Domain LocalStorage

As you may know, LocalStorage is domain based. You can’t read or write from localstorage that’s on different domain, even if that’s subdomain. But there is iframe trick that you can use to store data from domain to it’s subdomain.

Besically you create an iframe that’s hosted on your subdomain that set document.domain to the parent domain. Then you send PostMessage to that iframe and inside iframe you set that value of localStorage.

Here is the code

iFrame


document.domain = "domain.com";
window.onmessage = function(e) {
  if (e.origin !== "http://domain.com") {
    return;
  }
  var payload = JSON.parse(e.data);
  localStorage.setItem(payload.key, JSON.stringify(payload.data));
};

main file


<script>
window.onload = function() {
    var win = document.getElementsByTagName('iframe')[0].contentWindow;
    var obj = {
       name: "Jack"
    };
    win.postMessage(JSON.stringify({key: 'storage', data: obj}), "*");
};
</script>
<iframe style="display:none" src="http://sub.domain.com/iframe.html"></iframe>

If you want to save and load value from LocalStorage, you can PostMessage with method key that will indicate what action it need to take. You can also send message back from iframe to it’s parent with data read from localStorage.

iframe


document.domain = "domain.com";
window.onmessage = function(e) {
    if (e.origin !== "http://domain.com") {
        return;
    }
    var payload = JSON.parse(e.data);
    switch(payload.method) {
        case 'set':
            localStorage.setItem(payload.key, JSON.stringify(payload.data));
            break;
        case 'get':
            var parent = window.parent;
            var data = localStorage.getItem(payload.key);
            parent.postMessage(data, "*");
            break;
        case 'remove':
            localStorage.removeItem(payload.key);
            break;
    }
};

Your main page on domain.com


window.onload = function() {
    var win = document.getElementsByTagName('iframe')[0].contentWindow;
    var obj = {
       name: "Jack"
    };
    // save obj in subdomain localStorage
    win.postMessage(JSON.stringify({key: 'storage', method: "set", data: obj}), "*");
    // load previously saved data
    win.postMessage(JSON.stringify({key: 'storage', method: "get"}), "*");
    window.onmessage = function(e) {
        if (e.origin != "http://sub.domain.com") {
            return;
        }
        // this will log "Jack"
        console.log(JSON.parse(e.data).name);
    };
};

,

1 Comment

40 JavaScript libraries you may not know about

Here is a list of useful libraries I had in my bookmarks, libraries that I need to use in some of my future projects.

  • Drawing and animation
  • Language extensions
  • Utilites
    • URI.js – library for handling urls
    • rangy – Text range library
    • color-thief – pick color(s) from image
    • stacktrace.js – get the stacktrace
    • XRegExp – extended regular expressions
    • -prefix-free – CSS3 without prefixes
    • mathjs – An extensive math library for JavaScript and Node.js
    • numeraljs – library for formatting and manipulating numbers
    • String.js – string manipulation library
    • Ocrad.js – Optical Character Recognition in Javascript
    • filer.js – wrapper over HTML5 File API
  • DOM
  • Events
    • jwerty – keyboard events
    • jquery-simulate – simulate keyboard and mouse events
    • Mousetrap – keyboard shortcuts
    • jQuery hashchange – hashchange event
    • jquery-waypoints – Execute a function when you sroll to the element
    • Steady.js – A jank-free module to do logic on the onscroll event without performance regressions in a @media-query like conditions.
    • sysend.js – Send messages between open pages in the browser (this one is actually my own)

,

1 Comment

Shell without ssh or root on your shared hosting

My site jcubic.pl is on shared hosting that don’t have ssh, and I wanted to have a shell access, so using my jQuery Terminal, I’ve stared a project leash. It’s a shell written in javascript with help from php and python cgi. It will give you a shell access, no need to install any new software, and you don’t need to be root (like with other shells like shell in a box, AjaxTerm or Buterfly).

, , , ,

Leave a comment

Using exceptions to simulate tail recursion in JavaScript

JavaScript is very powerful language but it don’t have tail call elimination. But it turn out that you can simulate it using exception system. Here is very simple recursive function that calculate factorial:

function factorial(n) {
    function recur(n, result) {
        if (n == 0) {
            throw result;
        } else {
            recur(n-1, result*n);
        }
    }
    try {
        recur(n, 1);
    } catch(e) {
        return e;
    }
}

It turn out that in JavaScript (I read that in Douglas Crockford book JavaScript: The Good Parts) you can use any expression in throw and it will be send to variable in catch statement.

So what the above code does it simple exit from the recursive loop and pass result to catch statement. And this is exactly what tail recursion is, in language scheme this happen by default when inner function (you also need to create inner function in scheme) have recursive call as last expression. Here is scheme version of tail recursive factorial:

(define (factorial n)
  (let recur ((n n) (result 1))
     (if (= n 0)
        result
        (recur (- n 1) (* result n)))))

the code use named let but it can be rewriten with inner function and invocation. (this kind of trick is needed in script-fuGimp extension based on scheme).

, ,

6 Comments

Ready to use JSON-RPC Servlet in Java

I needed to write some Web based project in Java as an assignment, so I decide to use single page application and JSON-RPC service.

You can find the whole code in this gist.

Here is the code I use to create this task, step by step

First I found JSON-RPC library, I choose JSON-RPC 2.0 Base library.

I put both JSON Smart and JSON-RPC 2.0 Base jar files into lib directory (inside WEB-INF) of my java app.

In order to use JSON-RPC I needed to get raw POST data from request, I do this using this method (of the Servlet class):

    public String getInputData(HttpServletRequest request) throws IOException {
        ServletInputStream input = request.getInputStream();
        int buff;
        String output = "";
        while (true) {
            buff = input.read();
            if (buff == -1) {
                break;
            }
            output += (char)buff;
        }
        return output;
    }

Then I could parse that POST data (inside doPost Servlet method):

JSONRPC2Request reqIn = JSONRPC2Request.parse(this.getInputData(request));

Now I could extract data from the JSON-RPC request:

     Object id = reqIn.getID();
     Object[] params = reqIn.getPositionalParams().toArray();
     String method_name = reqIn.getMethod();

I decide to use other class as a Service and call method of that class based on JSON-RPC request, so I needed to use Reflection to do that

ClassLoader classLoader = ServletClass.class.getClassLoader();
Object service = classLoader.newInstance();
Class aClass = classLoader.loadClass("pl.jcubic.Service");
Method[] methods = aClass.getMethods();
                
Method method = null;
for (int i=0; i<methods.length; ++i) {
    if (methods[i].getName().equals(method_name)) {
        method = methods[i];
        break;
    }
}
if (method != null) {
    Object result = method.invoke(service, params);
    JSONRPC2Response respOut = new JSONRPC2Response(result, id);
    out.println(respOut);
} else {
    out.println(new JSONRPC2Response(JSONRPC2Error.METHOD_NOT_FOUND, id));
}

I use getMethods and search for method instead of using getMethod because I didn’t know what the classes I need to use for the params to get the right method.

I use class loader because I wanted to be able to recompile the Service (pl.jcubic.Service) and use it without the need to restart the app. But this loader don’t work as expected, the class file is loaded only once (when called the first time) the same as if I use import statement.

In order to have dynamic class load I need to write custom class loader, it look like this:

    class ServiceReloader extends ClassLoader {
        public ServiceReloader(ClassLoader parent) {
            super(parent);
        }

        public Class loadClass(String name) throws ClassNotFoundException {
            if (!"pl.jcubic.Service".equals(name)) {
                return super.loadClass(name);
            }
            try {
                String path = getClassPath(name);
                
                DataInputStream input = new DataInputStream(new BufferedInputStream(new FileInputStream(new File(path))));
                
                ByteArrayOutputStream buffer = new ByteArrayOutputStream();
                while (true) {
                    int data = input.read();
                    if (data == -1) {
                        break;
                    } 
                    buffer.write(data);
                }
                byte[] classData = buffer.toByteArray();
                return defineClass(name, classData, 0, classData.length);
            } catch (IOException e) {
                throw new ClassNotFoundException(e.getMessage());
            }
        }
    }

I write it as inner class of my Servlet. Then I use this class to load my Service class.

    ClassLoader parentClassLoader = ServiceReloader.class.getClassLoader();
    ServiceReloader classLoader = new ServiceReloader(parentClassLoader);
    Class aClass = classLoader.loadClass("pl.jcubic.Service");

Now I could recompile the Service class and use it from JavaScript without need to restart the app (don’t even refresh the browser if I write new method and wanted to test it)

One more thing that need to be written is catch exceptions and display proper JSON-RPC error responses.

You can find the whole code in this gist.

Now all you need is a class with methods that will become your JSON-RPC methods. Here is list of Java types mapend to JSON-RPC types. All paramters and return values of your methods need to be of those types.

, ,

Leave a comment

Add methods to Number object in JavaScript

In JavaScript everything is an object, even functions and numbers, and all objects even functions are first class citizens (which allow functional programming).

So this mean that you can call methods over numbers, and you can add new methods using Number.prototype. Here is example of range functions similar to that from Python, but bit different. (Python one return number from n to m – or from 0 to n – but this one return n numbers (n is this) starting from number passed as argument)

Number.prototype.range = function(n, result) {
    result = result || [];
    n = typeof n === 'undefined' ? 0 : n;
    if (this <= 0) {
        return result.reverse();
    } else {
        return this.range.call(this-1, n, result.concat([this-1+n]));
    }
};

And you can call this function/method using:

10..range(1)

It will return array [1, 2, 3, 4, 5, 6, 7, 8, 9, 10].

Note that there are two dots, it’s because when you use one it’s interpret as float number and thorw exception "SyntaxError: Unexpected token ILLEGAL" because it wait for a digit as next character.

Using this you can write factorial using

10..range(1).reduce(function(a, b) { return a*b; });

reduce function was added to Array.prototype in ECMAScript 5, You can check what browsers support them in this page.

You can wrap that code with a function and you can add it as Number prototype

Number.prototype.factorial = function() {
    return this.range(1).reduce(function(a, b) { return a*b; });
};

You can call it as:

10..factorial();

Here is example of times function (similar to that from Ruby)

Number.prototype.times = function(fn, self) {
    return this.range().forEach(fn, self);
};

You can use this function using:

10..times(function(i) { console.log(i); });

Leave a comment

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: