Working with multiple versions of Python/Ruby in Debian/Ubuntu

Debian has a good mechanism for handling multiple versions of a runtime, like Python or Ruby, and it's a pity if you don't take advantage of it. This should also work in Ubuntu, although the package names may be a little different.

Ruby

Ruby1.9.1 brings exciting enhancements, not to mention speed enhancements thanks to Yarv, and 1.9.1 is considered stable, so it's a pity if you don't start playing with it. Rails 2.3 should be compatible enough, along with Rails 3 beta.

First install both Ruby versions ....

  sudo aptitude install ruby-full rubygems ruby1.9.1-full rubygems1.9.1

Then issue these commands ...

    sudo update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.8 500 \
        --slave /usr/share/man/man1/ruby.1.gz ruby.1.gz \
                /usr/share/man/man1/ruby1.8.1.gz \
        --slave /usr/bin/ri ri /usr/bin/ri1.8 \
        --slave /usr/bin/irb irb /usr/bin/irb1.8 \
        --slave /usr/bin/gem gem /usr/bin/gem1.8 \
        --slave /var/lib/gems/bin gem-bin /var/lib/gems/1.8/bin


    sudo update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.9.1 400 \
        --slave /usr/share/man/man1/ruby.1.gz ruby.1.gz \
                /usr/share/man/man1/ruby1.9.1.1.gz \
        --slave /usr/bin/ri ri /usr/bin/ri1.9.1 \
        --slave /usr/bin/irb irb /usr/bin/irb1.9.1 \
        --slave /usr/bin/gem gem /usr/bin/gem1.9.1 \
        --slave /var/lib/gems/bin gem-bin /var/lib/gems/1.9.1/bin

Then to switch to Ruby1.9 (or back to Ruby1.8) ...

  sudo update-alternatives --config ruby

A dialog is presented in which you select the version you want. It has the advantage that not only is the main interpreter "ruby" referencing the 1.9 version, but also the utilities "ri", "irb" and "gem" plus the ruby man-pages.

The rubygems bin path will be available in /var/lib/gems/bin which will point to the right gems directory. So add that in your PATH (in ~/.bash_rc or in ~/.profile). Myself I've installed Rails 3beta on Ruby 1.9 and Rails 2.3 on Ruby 1.8 ... and when switching between versions, the right "rails" command is available.

The only inconvenience is that any Rubygems you install, have to be installed twice for each version (and it's better this way, since there may be incompatibilities between packages). Ruby 1.9 gems get installed in /var/lib/gems/1.9.1/ and 1.8 gems get installed in /var/lib/gems/1.8/.

Python

First install Python2.6 and Python3.0 ...

    sudo aptitude install python3-all python3-dev python2.6 python2.6-dev


Then configure the alternatives ...


    sudo update-alternatives --install /usr/bin/python python /usr/bin/python2.5 500 \
        --slave /usr/share/man/man1/python.1.gz python.1.gz /usr/share/man/man1/python2.5.1.gz

    sudo update-alternatives --install /usr/bin/python python /usr/bin/python2.6 400 \
        --slave /usr/share/man/man1/python.1.gz python.1.gz /usr/share/man/man1/python2.6.1.gz

    sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 300  \
        --slave   /usr/share/man/man1/python.1.gz python.1.gz /usr/share/man/man1/python3.1.gz

And then ...


    sudo update-alternatives --config python

Out-of-date packages?

As I'm writing this, Python has an upgrade issue in Debian testing ... giving an error about the default selected Python package, so to install Python2.6 or 3.0, it's better to update/install the latest sources of these packages, and I'd prefer to keep them from the unstable (sid) repository.

To do that, first edit the file ... /etc/apt/sources.list by adding these lines ...

    deb http://ftp.debian.org/debian/ unstable main non-free contrib
    deb-src http://ftp.debian.org/debian/ unstable main non-free contrib


Then, create a new file (or edit the existing one if exists) in ... /etc/apt/preferences ...

    Package: *
    Pin: release a=testing
    Pin-Priority: 700

    Package: *
    Pin: release a=unstable
    Pin-Priority: 650


Then you can install packages from the unstable repository like so (after doing an apt-get update) ...

    sudo apt-get -t unstable install python python-dev python3-all python3-dev python2.6 python2.6-dev

And the great thing about it is that whenever updates are available in the unstable repository for these packages, they will get installed automatically on apt-get upgrate.

NOTE: in Ubuntu you have to use use their repositories, not Debian's. So if you're using Ubuntu Jaunty, the newer repository would be Karmic ... which would be useful when you want to use newer packages without having to upgrade the whole distribution. So be sure to make the appropriate changes.

Enjoy ~

Using the Best Tools in Programming: Not Really Doable

There's something that bothers me when it comes to starting a new project. You can't really use the best tool for a certain job, if that tool is not integrated with the rest of your platform. Let me explain.

At our startup we pride ourselves with our pragmatism. We are true polyglots :) capable of diving in any project, no matter the language it was written in. This also gives us the power to make educated choices about the technologies we're going to use for our own gigs.

Our programming language of choice is Perl, because of its flexibility and because usually there's no need to reinvent the wheel since you can find a CPAN module for almost anything.

But recently I began experimenting with data-mining techniques, flirting with various NLP libraries. You can find almost anything in CPAN's AI:: namespace. But I also knew about NLTK, a Python collection of libraries with excellent documentation, and I also found OpenNLP, MontyLingua, ConceptNet, link-grammar and various Ruby modules.

And all of a sudden I got cold feet. Java packages in OpenNLP may have the advantage of speed (just a guess and it doesn't matter for the purpose of this discussion). NLTK has pedigree and great documentation, not to mention that many books related to NLP, AI and data mining have Python samples (for example I own Programming Collective Intelligence and AIMA). Usually the solution is straightforward: you test all the options, and choose the best one.

But what if you want to combine them?

Well, then you're shit out of luck. Surely you can do that with inter-process communication, but for that you'll have to write glue-code and pay the price for extra latency, bandwidth and memory ... parsing millions of documents, moving results between processes, it's not really practical. Perl does have Inline::Java, but I would only use it in extreme situations.

That's why there's so much wheel reinvention around. Unless a module is written in C, for which any language has a FFI, almost nobody wants to use a Java module from Ruby, or a Python module from Perl. That's why there's Lucene, and then there's Lucene.NET, CLucene, Ferret, Zend_Search_Lucene, Plucene and Lucene4c.

What is really needed is a universal virtual machine with a flexible MOP, allowing seamless communication between languages. I'm happy there are a couple of efforts in this space, including Parrot, and the DLR. Also, the biggest obstacles of alternative implementations are the modules written in C. Fortunately, JRuby/Rubinius have a brand new implementation-independent FFI, and Ironclad will allow IronPython users to use CPython extensions (number one on their list being numpy).

These developments make me happy :)

Perl Gtk2::StatusIcon example

I had the need to add a status icon in the systray, enhancing the functionality of a web application. My needs are a little more complex than displaying a simple menu, but the problem was that you can't find many examples for scripting languages, like Perl, especially with StatusIcon.

The beautiful thing is that Gtk+ bindings try not to diverge too much from the main API, although some of them add useful extensions.

So I found this PyGtk example, converting it to Perl:
http://marcin.af.gliwice.pl/if-then-else-20070121143245.

#!/usr/bin/env perl
use strict;
use warnings;

use Gtk2 -init;

sub quit_cb {
   my ($widget, $status_icon) = @_;
 
   $status_icon->set_visible(0) if $status_icon;
   Gtk2->main_quit();
}

sub popup_menu_cb {
   my ($widget, $button, $time, $menu) = @_;

   if ($button == 3) {
   my ($x, $y, $push_in)
       = Gtk2::StatusIcon::position_menu($menu, $widget);

   $menu->show_all();
   $menu->popup( undef, undef,
             sub{return ($x,$y,0)} ,
             undef, 0, $time );
   }
}

sub activate_icon_cb {
   my $msgBox = Gtk2::MessageDialog->new(undef,
                     'GTK_DIALOG_MODAL',
                     'GTK_MESSAGE_INFO',
                     'GTK_BUTTONS_OK',
                     "Status Icon example!");
   $msgBox->run();
   $msgBox->destroy();
}

my $status_icon = Gtk2::StatusIcon->new_from_stock('gtk-home');

my $menu = Gtk2::Menu->new();
my $menuItem = Gtk2::ImageMenuItem->new_from_stock("gtk-about");
$menuItem->signal_connect('activate', \&activate_icon_cb);
$menu->append($menuItem);

$menuItem = Gtk2::ImageMenuItem->new_from_stock('gtk-quit');
$menuItem->signal_connect('activate', \&quit_cb, $status_icon);
$menu->append($menuItem);

$status_icon->set_tooltip("StatusIcon test");
$status_icon->signal_connect('activate', \&activate_icon_cb);
$status_icon->signal_connect('popup-menu', \&popup_menu_cb, $menu);
$status_icon->set_visible(1);

Gtk2->main();

~

Django - url tags for persistent GET parameters

I've been working on a project based on Django recently. The project was already established, and it's using Django version 0.96.

In the project's reporting interface I needed lots of filters to be passed around from the reports to the logs viewer. I also needed pagination on queries made by hand (i.e. not using django's models) ... just to give a little background. I'm not really proficient with Django, and there might be a better way to do this, but I wrote 2 tags that helped me dealing with persistence of GET parameters.

The code is posted here: http://www.djangosnippets.org/snippets/1175/

To install, you just drop the file in a "templatetags" directory, add the application to INSTALLED_APPS in settings, and in the template you have to include the templatetags file like so ...

{% load links %}

Usage is simple:

{% link_persist viewname arg1,arg2,kwarg1=value1,kwarg2=value2 %}

This tag has the same behavior as the normal url tag, with a couple of notable differences:

• GET parameters are included in the resulting url
• (key,value) arguments become GET arguments in the current url, overriding the value in the current request
• if you specify a value to a key as an empty string, or as a None, that key will be excluded from the resulting url ... useful for not persisting a set of parameters that only make sens for the current url

The other tag:

{% link_add kwarg1=value1,kwarg2=value2... %}

This tag adds to the url in the address bar the values specified in the list of named arguments. It has the same behavior as above, so for example, instead of adding to the current url, or replacing an existing value (useful for pagination) you can also remove parameters from the current URL.

~

Emacs - TextMate like snippets

I love Emacs, and while I'm no guru, I know my way around it.

At Adobe I had the option of using a MacBook Pro, so naturally, being a dynamic languages freak I've had the opportunity of experimenting with Textmate, as all the cool kinds use it in screencasts.

The coolest feature of Textmate is its intuitive and powerful snippets mechanism. The syntax is easy to understand and use and with it you can automate boring boilerplate code. If you don't know what I mean, take a look at the screencasts from rubyonrails.org. So yeah, I felt jelousy that my favorite text editor, Emacs, doesn't have it.

But then I found this wonderful Emacs plugin: Yasnippet. You can also watch a YouTube introductory screencast to whet your appetite. It uses the same Textmate syntax, with a notable difference: for text filters you can use elisp syntax.

Installing it is easy ... just download the latest version, extract the files to your "site-lisp" directory, which usually is "~/.emacs.d/site-lisp" on Linux. If you cannot find your "site-lisp" directory, just create one in "emacs.d/site-lisp" (or whatever) and add this line to your ".emacs" configuration file:

(setq load-path (cons "~/emacs.d/site-lisp" load-path))

Then you need to add to the same ".emacs" configuration file this line:

(require 'yasnippet)
(yas/initialize)
(yas/load-directory "~/emacs.d/site-lisp/snippets")

Be careful to change the specified directory ("~/emacs.d/site-lisp/snippets") to match your own setup.

Defining snippets is easy. It implies creating for every snippet a file containing specific syntax. Snippets are context-dependent ... you can define snippets that only work in html-mode, or only in java-mode (as in Textmate). Editing modes in Emacs have an inheritance model, as in ... java-mode inherits cc-mode, which in turn inherits text-mode. The "snippets" directory has a similar structure, so if you want to define a snippet for java-mode, you can define it in "cc-mode" and it will be inherited by all derived modes, like c++-mode and java-mode, or you can define it for java-mode.

To take an example, to define a snippet named "bean" for java-mode, you need to edit the file "snippets/text-mode/cc-mode/java-mode/bean". And when you'll type "bean" with a TAB following, it will expand. Such a snippet could have the following syntax:

private ${2:String} ${1:varname};

public void set${1:$(upcase-initials text)}(${2:String} new${1:$(upcase-initials text)}) {
${1:fieldName} = new${1:$(upcase-initials text)};
}

public ${2:String} get${1:$(upcase-initials text)}() {
return ${1:fieldName};
}

${N} is a hole in the snippet where you can type text. It can have a default, and you can also have filters for the typed text, by using elisp functions, as in the above example where the typed text for the variable name has the initial upper-cased (according to Java conventions).

Now for my next problem: I'm experimenting with Catalyst web-framework, which uses by default the Template Toolkit. I don't mind using html-mode for editing TT files, but it's a problem for snippets because I used and will use lots of other template managers with html-mode, so snippets names may clash, and I don't want to prefix the names I use for snippet-expansion.

But emacs is wonderful right? So why not define our own editing mode, inherited from html-mode?

To create an inherited mode from html-mode, I created a file named "site-lisp/template-toolkit-mode.el" with the following code:

;; specifies that this script provides an emacs module
(provide 'template-toolkit-mode)

;; defines a derived editing mode, inheriting from html-mode
(define-derived-mode template-toolkit-mode html-mode "Template Toolkit2" nil)

And in our ".emacs" configuration file we can initialize this mode automatically for files ending with ".tt" (the file extension):

(autoload 'template-toolkit-mode "template-toolkit-mode" nil t)
(add-to-list 'auto-mode-alist '("\\.tt$" . template-toolkit-mode))

That's it. Now we can define snippets only for template-toolkit file types. For example this is a snippet creating a foreach loop:

[% FOREACH ${1:item} IN ${2:items} %]
$0
[% END -%]

Btw, if your interested in my emacs configuration (nothing special about it, but may give you some hints), you can find it here: http://code.google.com/p/my-emacs-d/

Enjoy ~

Finding My Path

I was really impressed by a talk given by Jon Kabat-Zinn at Google:
Mindfulness Meditation.

Actually, I was so impressed that I bought two of his books:

• Wherever You Go, There You Are: Mindfulness Meditation in Everyday Life (amazon link)
  
• Full Catastrophe Living: Using the Wisdom of Your Body and Mind to Face Stress, Pain, and Illness (amazon link)

I'm in the process of reading them, and thus far it has been an enriching experience.

People in their twenties, after growing up :), sometimes face too much stress, associated with confusion, discontent and sleep deprivation. This comes from facing the real world, wanting to make a name for yourself, working harder and harder, mimicking other go-getters that have had success, while continually wanting to answer (unconsciously) really simple and fundamental questions:

1. Who am I?
2. What's my purpose?

This happened to me. And this confusion and endless stress can really cause pain, if uncontrolled.

Mindfulness meditation is a central theme in Buddhism, being all about awareness of one's thoughts, feelings and actions. It is said that some people live all their lives in a dream state, perpetually thinking about past mistakes, future actions and desires, continually having internal conflicts. Such people race through life without noticing the present moment, the "now", the only time when you actually live.

I noticed this pattern in my own being, and I really don't want to live a life of regrets, and sorrow.

In closing, I'll reproduce a beautiful poem recited by Jon Kabat-Zinn in the above linked presentation:

The time will come
when, with elation
you will greet yourself arriving
at your own door, in your own mirror
and each will smile at the other’s welcome,
and say, sit here. Eat.

You will love again the stranger who was your self.
Give wine. Give bread. Give back your heart
to itself, to the stranger who has loved you
all your life, whom you ignored
for another, who knows you by heart.

Take down the love letters from the bookshelf,
the photographs, the desperate notes,
peel your own image from the mirror.

Sit. Feast on your life.

– Derek Walcott

Be happy ~

Hacker Wannabe

It's the title of my other blog written in my native tongue.

Why wannabe?
Because I grew up with Eric Raymond's essays like How To Become A Hacker and with Paul Graham's Great Hackers, which while partly childish and narcissistic, I think it had irreversible effects on my personality.

Certain ideas stick around, and before you know it, you have faith without questioning assumptions. But if I were to name a single truth I learned is that "code talks, bullshit walks".

Somehow I could never grok that the end-result should matter more than the means to get there. That is why in my personal projects I tried really hard to stick to my beliefs. And personal projects are those projects where, you know, you have absolute control and none of that "bullshit" that goes with working on a big company. And yet I couldn't get the job done more efficiently at home.

Why? Because I would get pretty confused about the choices I had to make. Choices that I ended up judging based on politics and personal feelings.

Should I use Python? It makes me feel good. Yeah but it's not "turtles all they way down" like Scheme and its performance is not so good. But Scheme doesn't have libraries. And oh God, that Linq feature in C# is sooo cool, but C# is as verbose as Java. Hey, maybe I should use F#, since it is a ML dialect with all kinds of goodies and has access to a huge library, but it's proprietary, and why not use Scala? It's open-source, and good for DSLs, but somehow it feels dirty, and that JVM is consumating tons of resources which I can't stand. Ruby is a nice language and I can stand performance issues since it's so beautifull, but damn, its unicode support sucks.

Sounds childish, and it is.

Some of us can recognize this as the paralysis of choice. Don't fall into that. Don't use Lisp because of Paul Graham's classification of good/bad programmers. Don't use Linux because Eric Raymond said you cannot hack other systems. These are just tools. The end results is usually far greater than the sum of the parts you use.

And really, you can find a lot more beauty in algorithms.

Good tools make a difference, but when you're hit by choice paralysis, you should stop, take a deep breath, and choose what you're most familiar with, even if that's a painful decision. And in the end a far greater ingredient for success is passion. Nurture it, don't suffocate yourself with useless debates.

Sure, it's great to have mentors and to try the taste of multiple technologies and paradigms, and to reason about why some people have had success, but as they say "bad artists copy, great artists steal".

Making Programming Fun

Why is it so important to have fun while programming?

If you take the fun away, you lose your passion. And the thing to understand for outsiders is that programming is friggin' hard [1]. No passion, no results, it's that simple.

That is why it saddens me to see the reactions the new jQuery homepage received [2].

The new front-page logo displays a rockstar pounding his fingers on a keyboard beside him, and apparently people got offended because:

• some people don't like the idea that programmers can be superstars, although we do have our share of celebrities and for good reason
• some people don't like the non-professional look of this new design: worrying that they won't be able to convince their bosses anymore
  

With the first category I can relate, and it's usually people that don't feel good about their profession (boy, I do hate it sometimes). With the second category I do not agree: code talks, bullshit walks. Managers want results. Deliver those results, and the underlying technology will be accepted. If not, run for your life.

Amateurs (as in people that are programming for the fun of it) should be allowed to have a good time.

Profile Update

Not much "creating fun" lately. It appears I'm not in the mood.

I update my homepage with links to my photo collection (on Flickr), my LinkedIn account (newly created), my Google Reader shared items, and my Twitter account: http://lexoft.eu/

Right now I'm not in the mood for talking. I'll be 26 years old next month and I need to reevaluate my life, to understand what I want from it and find my path onwards.

When life gives you lemons, you should make lemonade!

First Post

Ok, so this is really nothing new.

I tried it before ... starting blogs using English as the default, but I can't really wrap my head around, since I'm still learning it. But I feel the need to express myself in ways that the whole world can hear ... not that it really matters, but it's nice thinking about the powers that be of online publishing, and getting people to hear you is a powerful freedom.

This blog will describe my personal journey of being a better programmer and a better person, a journey of self-discovery and fun.

Who am I?
I'm still trying to find out, but until then I posted a lame description on my homepage.

Since I can hardly restrain myself to write about useful things, I did some thinking, and these are the things that I want to talk about:

• programming languages
• interesting algorithms
• compiler theory
• usability and accessibility
  
• personal projects
• constructive criticism of software
  
• open source
• beauties discovered in my life
  

What this blog will NOT contain:

• reiteration of industry news without personal opinions
  
• open source/free software advocacy
• flames
• commercials
  
• unconfirmed rumors
• advices for matters in which I'm clueless
  

Also, to have high quality articles I am hereby stating that each article will benefit from a three day incubation period, in which I'll try to harass one of my friends in reading it.

Ok, here we go ...