jcubic

I'm graphic designer and programmer from Poland.

Homepage: http://jcubic.wordpress.com

How to backup 8GB file or larger on fat32 from linux

I have external USB hard drive that use Fat32 (so I can use it on windows machine as well on GNU/Linux). I needed to reinstall my OS (Xubuntu) and repartition the disk, so I need to copy all my files to external drive. But I’ve got 15GB virtualbox file.

On GNU/Linux or MacOS X you can use standard tools to split the file and then merge it back.

Here is a code that will compress and create 4GB chunks.


tar czvf file.tar.gz directory/
split -b $(echo "4*(2^30)-1" | bc) --verbose file.tar.gz file.tar.gz-

it will create files like file.tar.gz-aa file.tar.gz-ab that can be copy to Fat32 partition because they are smaller then 4GB (2^30 is 1GB in bytes).

to merge those files together you can just cat them


cat file.tar.gz-aa file.tar.gz-ab > file.tar.gz

if you want to see progress bar you can use pv command. This is the smallest way I came up with to do this:


cat file.tar.gz-aa file.tar.gz-ab | pv -s $(ls -l file.tar.gz-aa file.tar.gz-ab | cut -d' ' -f5 | tr '\n' '+' | sed -e 's/+$//' -e 's/.*/echo -n $(( & ))/' | bash) > file.tar.gz

Here is a function that can be put into .bashrc file


function cat-pv () {
    cat $@ | pv -s $(ls -l $@ | cut -d' ' -f5 | tr '\n' '+' | sed -e 's/+$//' -e 's/.*/echo -n $(( & ))/' | bash)
}

Here I found shorter way to calculate size of list of files it use awk:


ls -lrt | awk '{ total += $5 }; END { print total }'

So the function can be strink to


function cat-pv () {
     cat $@ | pv -s $(ls -lrt $@ | awk '{ total += $5 }; END { print total }')
}

, , ,

Leave a comment

Cross-Domain LocalStorage

As you may know, LocalStorage is domain based. You can’t read or write from localstorage that’s on different domain, even if that’s subdomain. But there is iframe trick that you can use to store data from domain to it’s subdomain.

Besically you create an iframe that’s hosted on your subdomain that set document.domain to the parent domain. Then you send PostMessage to that iframe and inside iframe you set that value of localStorage.

Here is the code

iFrame


document.domain = "domain.com";
window.onmessage = function(e) {
  if (e.origin !== "http://domain.com") {
    return;
  }
  var payload = JSON.parse(e.data);
  localStorage.setItem(payload.key, JSON.stringify(payload.data));
};

main file


<script>
window.onload = function() {
    var win = document.getElementsByTagName('iframe')[0].contentWindow;
    var obj = {
       name: "Jack"
    };
    win.postMessage(JSON.stringify({key: 'storage', data: obj}), "*");
};
</script>
<iframe style="display:none" src="http://sub.domain.com/iframe.html"></iframe>

If you want to save and load value from LocalStorage, you can PostMessage with method key that will indicate what action it need to take. You can also send message back from iframe to it’s parent with data read from localStorage.

iframe


document.domain = "domain.com";
window.onmessage = function(e) {
    if (e.origin !== "http://domain.com") {
        return;
    }
    var payload = JSON.parse(e.data);
    switch(payload.method) {
        case 'set':
            localStorage.setItem(payload.key, JSON.stringify(payload.data));
            break;
        case 'get':
            var parent = window.parent;
            var data = localStorage.getItem(payload.key);
            parent.postMessage(data, "*");
            break;
        case 'remove':
            localStorage.removeItem(payload.key);
            break;
    }
};

Your main page on domain.com


window.onload = function() {
    var win = document.getElementsByTagName('iframe')[0].contentWindow;
    var obj = {
       name: "Jack"
    };
    // save obj in subdomain localStorage
    win.postMessage(JSON.stringify({key: 'storage', method: "set", data: obj}), "*");
    // load previously saved data
    win.postMessage(JSON.stringify({key: 'storage', method: "get"}), "*");
    window.onmessage = function(e) {
        if (e.origin != "http://sub.domain.com") {
            return;
        }
        // this will log "Jack"
        console.log(JSON.parse(e.data).name);
    };
};

,

1 Comment

40 JavaScript libraries you may not know about

Here is a list of useful libraries I had in my bookmarks, libraries that I need to use in some of my future projects.

  • Drawing and animation
  • Language extensions
  • Utilites
    • URI.js – library for handling urls
    • rangy – Text range library
    • color-thief – pick color(s) from image
    • stacktrace.js – get the stacktrace
    • XRegExp – extended regular expressions
    • -prefix-free – CSS3 without prefixes
    • mathjs – An extensive math library for JavaScript and Node.js
    • numeraljs – library for formatting and manipulating numbers
    • String.js – string manipulation library
    • Ocrad.js – Optical Character Recognition in Javascript
    • filer.js – wrapper over HTML5 File API
  • DOM
  • Events
    • jwerty – keyboard events
    • jquery-simulate – simulate keyboard and mouse events
    • Mousetrap – keyboard shortcuts
    • jQuery hashchange – hashchange event
    • jquery-waypoints – Execute a function when you sroll to the element
    • Steady.js – A jank-free module to do logic on the onscroll event without performance regressions in a @media-query like conditions.
    • sysend.js – Send messages between open pages in the browser (this one is actually my own)

,

1 Comment

Shell without ssh or root on your shared hosting

My site jcubic.pl is on shared hosting that don’t have ssh, and I wanted to have a shell access, so using my jQuery Terminal, I’ve stared a project leash. It’s a shell written in javascript with help from php and python cgi. It will give you a shell access, no need to install any new software, and you don’t need to be root (like with other shells like shell in a box, AjaxTerm or Buterfly).

, , , ,

Leave a comment

Using exceptions to simulate tail recursion in JavaScript

JavaScript is very powerful language but it don’t have tail call elimination. But it turn out that you can simulate it using exception system. Here is very simple recursive function that calculate factorial:

function factorial(n) {
    function recur(n, result) {
        if (n == 0) {
            throw result;
        } else {
            recur(n-1, result*n);
        }
    }
    try {
        recur(n, 1);
    } catch(e) {
        return e;
    }
}

It turn out that in JavaScript (I read that in Douglas Crockford book JavaScript: The Good Parts) you can use any expression in throw and it will be send to variable in catch statement.

So what the above code does it simple exit from the recursive loop and pass result to catch statement. And this is exactly what tail recursion is, in language scheme this happen by default when inner function (you also need to create inner function in scheme) have recursive call as last expression. Here is scheme version of tail recursive factorial:

(define (factorial n)
  (let recur ((n n) (result 1))
     (if (= n 0)
        result
        (recur (- n 1) (* result n)))))

the code use named let but it can be rewriten with inner function and invocation. (this kind of trick is needed in script-fuGimp extension based on scheme).

, ,

6 Comments

Ready to use JSON-RPC Servlet in Java

I needed to write some Web based project in Java as an assignment, so I decide to use single page application and JSON-RPC service.

You can find the whole code in this gist.

Here is the code I use to create this task, step by step

First I found JSON-RPC library, I choose JSON-RPC 2.0 Base library.

I put both JSON Smart and JSON-RPC 2.0 Base jar files into lib directory (inside WEB-INF) of my java app.

In order to use JSON-RPC I needed to get raw POST data from request, I do this using this method (of the Servlet class):

    public String getInputData(HttpServletRequest request) throws IOException {
        ServletInputStream input = request.getInputStream();
        int buff;
        String output = "";
        while (true) {
            buff = input.read();
            if (buff == -1) {
                break;
            }
            output += (char)buff;
        }
        return output;
    }

Then I could parse that POST data (inside doPost Servlet method):

JSONRPC2Request reqIn = JSONRPC2Request.parse(this.getInputData(request));

Now I could extract data from the JSON-RPC request:

     Object id = reqIn.getID();
     Object[] params = reqIn.getPositionalParams().toArray();
     String method_name = reqIn.getMethod();

I decide to use other class as a Service and call method of that class based on JSON-RPC request, so I needed to use Reflection to do that

ClassLoader classLoader = ServletClass.class.getClassLoader();
Object service = classLoader.newInstance();
Class aClass = classLoader.loadClass("pl.jcubic.Service");
Method[] methods = aClass.getMethods();
                
Method method = null;
for (int i=0; i<methods.length; ++i) {
    if (methods[i].getName().equals(method_name)) {
        method = methods[i];
        break;
    }
}
if (method != null) {
    Object result = method.invoke(service, params);
    JSONRPC2Response respOut = new JSONRPC2Response(result, id);
    out.println(respOut);
} else {
    out.println(new JSONRPC2Response(JSONRPC2Error.METHOD_NOT_FOUND, id));
}

I use getMethods and search for method instead of using getMethod because I didn’t know what the classes I need to use for the params to get the right method.

I use class loader because I wanted to be able to recompile the Service (pl.jcubic.Service) and use it without the need to restart the app. But this loader don’t work as expected, the class file is loaded only once (when called the first time) the same as if I use import statement.

In order to have dynamic class load I need to write custom class loader, it look like this:

    class ServiceReloader extends ClassLoader {
        public ServiceReloader(ClassLoader parent) {
            super(parent);
        }

        public Class loadClass(String name) throws ClassNotFoundException {
            if (!"pl.jcubic.Service".equals(name)) {
                return super.loadClass(name);
            }
            try {
                String path = getClassPath(name);
                
                DataInputStream input = new DataInputStream(new BufferedInputStream(new FileInputStream(new File(path))));
                
                ByteArrayOutputStream buffer = new ByteArrayOutputStream();
                while (true) {
                    int data = input.read();
                    if (data == -1) {
                        break;
                    } 
                    buffer.write(data);
                }
                byte[] classData = buffer.toByteArray();
                return defineClass(name, classData, 0, classData.length);
            } catch (IOException e) {
                throw new ClassNotFoundException(e.getMessage());
            }
        }
    }

I write it as inner class of my Servlet. Then I use this class to load my Service class.

    ClassLoader parentClassLoader = ServiceReloader.class.getClassLoader();
    ServiceReloader classLoader = new ServiceReloader(parentClassLoader);
    Class aClass = classLoader.loadClass("pl.jcubic.Service");

Now I could recompile the Service class and use it from JavaScript without need to restart the app (don’t even refresh the browser if I write new method and wanted to test it)

One more thing that need to be written is catch exceptions and display proper JSON-RPC error responses.

You can find the whole code in this gist.

Now all you need is a class with methods that will become your JSON-RPC methods. Here is list of Java types mapend to JSON-RPC types. All paramters and return values of your methods need to be of those types.

, ,

Leave a comment

Add methods to Number object in JavaScript

In JavaScript everything is an object, even functions and numbers, and all objects even functions are first class citizens (which allow functional programming).

So this mean that you can call methods over numbers, and you can add new methods using Number.prototype. Here is example of range functions similar to that from Python, but bit different. (Python one return number from n to m – or from 0 to n – but this one return n numbers (n is this) starting from number passed as argument)

Number.prototype.range = function(n, result) {
    result = result || [];
    n = typeof n === 'undefined' ? 0 : n;
    if (this <= 0) {
        return result.reverse();
    } else {
        return this.range.call(this-1, n, result.concat([this-1+n]));
    }
};

And you can call this function/method using:

10..range(1)

It will return array [1, 2, 3, 4, 5, 6, 7, 8, 9, 10].

Note that there are two dots, it’s because when you use one it’s interpret as float number and thorw exception "SyntaxError: Unexpected token ILLEGAL" because it wait for a digit as next character.

Using this you can write factorial using

10..range(1).reduce(function(a, b) { return a*b; });

reduce function was added to Array.prototype in ECMAScript 5, You can check what browsers support them in this page.

You can wrap that code with a function and you can add it as Number prototype

Number.prototype.factorial = function() {
    return this.range(1).reduce(function(a, b) { return a*b; });
};

You can call it as:

10..factorial();

Here is example of times function (similar to that from Ruby)

Number.prototype.times = function(fn, self) {
    return this.range().forEach(fn, self);
};

You can use this function using:

10..times(function(i) { console.log(i); });

Leave a comment

Working with SVG in jQuery

I recently notice that if you create element like a circle in SVG (inline SVG embedded into HTML), using inspector/firebug or jQuery, your circle is not visible on SVG until you refresh the SVG, I found a hack to force refresh of the SVG, I just get text of the SVG and insert it again into the DOM, and all elements that were not visible like circle mention before will be rendered.

Here is referesh function, as jQuery plugin


$.fn.xml = function() {
    return (new XMLSerializer()).serializeToString(this[0]);
};

$.fn.DOMRefresh = function() {
    return $($(this.xml()).replaceAll(this));
};

Those plugins should work with every XML embeded into HTML not only SVG. Don’t look very nice but it work, but I found a better way fix jQuery to work with SVG. I found that when I use function document.createElementNS instead of document.createElement elements added to inline SVG using for instance appendChild function everything work fine. So only thing that need to be done for jQuery to work with inline SVG is to replace this function if element is SVG. First I wrote method in jQuery object that test if element is SVG element, there are only three elements that have the same name in HTML and SVG is a, script, style and title tags so I didn’t put them in.

isSVGElement: function( o ) {
        if (o instanceof SVGElement) {
            return true;
        } else {
            if (typeof o === 'string') {
                return $.inArray(o, ['altGlyph', 'altGlyphDef',
                                     'altGlyphItem', 'animate',
                                     'animateColor', 'animateMotion',
                                     'animateTransform', 'circle',
                                     'clipPath', 'color-profile',
                                     'cursor', 'defs', 'desc', 'ellipse',
                                     'feBlend', 'feColorMatrix',
                                     'feComponentTransfer',
                                     'feComposite', 'feConvolveMatrix',
                                     'feDiffuseLighting',
                                     'feDisplacementMap',
                                     'feDistantLight', 'feFlood',
                                     'feFuncA', 'feFuncB', 'feFuncG',
                                     'feFuncR', 'feGaussianBlur',
                                     'feImage', 'feMerge', 'feMergeNode',
                                     'feMorphology', 'feOffset',
                                     'fePointLight',
                                     'feSpecularLighting', 'feSpotLight',
                                     'feTile', 'feTurbulence', 'filter',
                                     'font', 'font-face',
                                     'font-face-format',
                                     'font-face-name', 'font-face-src',
                                     'font-face-uri', 'foreignObject',
                                     'g', 'glyph', 'glyphRef', 'hkern',
                                     'image', 'line', 'linearGradient',
                                     'marker', 'mask', 'metadata',
                                     'missing-glyph', 'mpath', 'path',
                                     'pattern', 'polygon', 'polyline',
                                     'radialGradient', 'rect',
                                     'set', 'stop', 'svg',
                                     'switch', 'symbol', 'text',
                                     'textPath', 'tref',
                                     'tspan', 'use', 'view',
                                     'vkern']) !== -1;
            }
        }
    }

I get the list of elements from Mozilla MDN.

I added this method to main jQuery.extend({ that add methods to jQuery object. Next thing is to replace createElement with createElementNS, there are only 2 places with this in parseHTML (in the same jQuery.extend) and createSafeFragment function, only one is responsible for inserting elements – parseHTML. Below is the code for that function.


	parseHTML: function( data, context, keepScripts ) {
		if ( !data || typeof data !== "string" ) {
			return null;
		}
		if ( typeof context === "boolean" ) {
			keepScripts = context;
			context = false;
		}
		context = context || document;

		var parsed = rsingleTag.exec( data ),
			scripts = !keepScripts && [];

		// Single tag
		if ( parsed ) {
			if ( jQuery.isSVGElement( parsed[1] ) ) {
				return [ context.createElementNS(
					"http://www.w3.org/2000/svg",
					parsed[1]) ];
			} else {
				return [ context.createElement( parsed[1] ) ];
			}
		}

		parsed = jQuery.buildFragment( [ data ], context, scripts );
		if ( scripts ) {
			jQuery( scripts ).remove();
		}
		return jQuery.merge( [], parsed.childNodes );
	},

Unfortunetnly not all manupulation methods that create new elements will work but it’s better then refresh hack. The stuff that don’t work is when you create more then one element from a string or if you put some attributes. In this case jQuery use document fragments and innerHTML to create the DOM. If we will want to fix that too we will need to write a parser that will call createElementNS.

You can also include those function after you load jQuery so original code will not be changed.

The other way is to use document.createElementNS and element.setAttributeNS that need to be used if you want to add xlink:href attribute (it use http://www.w3.org/1999/xlink namespace).

,

Leave a comment

Image color picker in javascript that work without canvas in every browser

If you want to create color picker from image in javascript you probably will use canvas, but what if you need it work in IE as I needed. You can use some server side help to fetch pixels data from the server. I use php for that and GD library. The code is below.

cross-browser color picker

Server side code that return json pixel data


<?php

function rgb($color) {
    $result[] = ($color >> 16) & 0xFF;
    $result[] = ($color >> 8) & 0xFF;
    $result[] = $color & 0xFF;
    return $result;
}

function imagecreatefrom($filename) {
    $path = pathinfo($filename);
    switch($path['extension']) {
        case 'png':
            return imagecreatefrompng($filename);
        case 'jpg':
            return imagecreatefromjpeg($filename);
        case 'gif':
            return imagecreatefromgif($filename);
        default:
            return null;
    }
}

function getImageData($filename) {
    list($width, $height) = getimagesize($filename);
    $image = imagecreatefrom($filename);
    $image_data = array();
    for ($y = 0; $y < $height; ++$y) {
        $row = array();
        for ($x = 0; $x < $width; ++$x) {
            $row[] = rgb(imagecolorat($image, $x, $y));
        }
        $image_data[] = $row;
    }
    return $image_data;
}

if (isset($_GET['filename'])) {
    $filename = $_GET['filename'];
    if (file_exists($filename)) {
        $result = array(
            'error' => null,
            'result' => getImageData($filename)
        );
    } else {
        $result = array(
            'error' => "The file '$filename' don't exist",
            'result' => null
        );
    }
} else {
    $result = array(
        'error' => "You need to put filename",
        'result' => null
    );
}
header('Content-Type: application/json');
echo json_encode($result);

?>

Then you can fetch the data using ajax and add get invidial pixels on mousemove


    var img = $('img');
    $.getJSON('image_data.php', {filename: img.attr('src')}, function(data) {
        if (data.error) {
            alert(data.error);
        } else {
            $('.eyedropper').click(function() {
                picker = true;
                return false;
            });
            img.mousemove(function(e) {
                if (picker) {
                    var x = Math.round(e.pageX - offset.left);
                    var y = Math.round(e.pageY - offset.top);
                    if (x >= 0) {
                        $('.color').css('background-color',
                                        'rgb('+data.result[y][x].join(',')+')');
                    }
                }
            }).click(function(e) {
                picker = false;
            });
        }
    });

Now all you need to have is element with class eyedropper like a link in your html:


<a href="#" class="eyedropper">pick the color</a>

, ,

3 Comments

Mastering Emacs – Search inside minibufer

This is by far best feature in Emacs that you can use interactive search inside Minibuffer. If you want to find a function that you don’t know exact name, I do this a lot – who will remember function names in Emacs, you can use:

C-h f C-s

And search for functions. How cool is that? Almost all functions that have completion inside minibuffer work with search like

  • C-x b C-s – search for buffer to switch
  • C-h <KEY> C-s – search for help

When I switch to Emacs 24, I found that M-x C-s don’t work. So you can’t search for interactive function to execute. But you can get it back. As describe by this Emacs Bug Ticket by putting this to you .emacs file.

(defun read-extended-command ()
  "Read command name to invoke in `execute-extended-command'."
  (minibuffer-with-setup-hook
      (lambda ()
        (set (make-local-variable 'minibuffer-default-add-function)
             (lambda ()
               ;; Get a command name at point in the original buffer
               ;; to propose it after M-n.
               (let ((def (with-current-buffer
                              (window-buffer (minibuffer-selected-window))
                            (and (commandp (function-called-at-point))
                                 (format "%S" (function-called-at-point)))))
                     (all (sort (minibuffer-default-add-completions)
                                (lambda (a b) (string< a b)))))
                 (if def
                     (cons def (delete def all))
                   all)))))
    ;; Read a string, completing from and restricting to the set of
    ;; all defined commands.  Don't provide any initial input.
    ;; Save the command read on the extended-command history list.
    (completing-read
     (concat (cond
	      ((eq current-prefix-arg '-) "- ")
	      ((and (consp current-prefix-arg)
		    (eq (car current-prefix-arg) 4)) "C-u ")
	      ((and (consp current-prefix-arg)
		    (integerp (car current-prefix-arg)))
	       (format "%d " (car current-prefix-arg)))
	      ((integerp current-prefix-arg)
	       (format "%d " current-prefix-arg)))
	     ;; This isn't strictly correct if `execute-extended-command'
	     ;; is bound to anything else (e.g. [menu]).
	     ;; It could use (key-description (this-single-command-keys)),
	     ;; but actually a prompt other than "M-x" would be confusing,
	     ;; because "M-x" is a well-known prompt to read a command
	     ;; and it serves as a shorthand for "Extended command: ".
	     "M-x ")
     obarray 'commandp t nil 'extended-command-history)))

And tell me, why you will ever need Ido-mode

One note about that, when you use one interactive function inside another one you will need to call C-g twice to exit minibuffer

1 Comment

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: