Programming: the first thing to learn

Many of us had courses, attended lectures and speeches on programming. Many of us read (even many) programming books, and regularly read articles and blogs about programming.

Have you ever thought of what should have been the first thing to learn in programming? The usual "variables, operators, data structures, classes, templates, metaprogramming, ..." or something else?

I've come to the conclusion that the most important thing, the first thing to learn, the one that if you're not familiar with you shouldn't even keep studying programming, is the answer to this question: "How can I verify my code works?".

If I was a teacher, right before writing my name on the blackboard (yes, this is how I like it to imagine it), I'd write this: "How can I verify my code works?".

And I'll spend the entire lesson waiting for someone to come up with a reasonable solution. If nobody goes close to it, I'd assign it as an essay for the following lesson.

The concept I want to stress here is simple: The Code You Write Must Be Automatically Testable.

Emphasis on: Testable.

Emphasis on: Automatic.

1. At any given moment, you must be able to launch a script and verify that your code works as expected, while the result of the verification is automatically checked (refer to Unit Testing). Better if the script is fast.

2. Before changing your code, you must first write the tests to verify your changes will do what you expect (this is called TDD, Test Driven Development - there are many interesting and well written articles on this topic). Those tests will fail: your job will be making them pass.

3. You must not commit changes when some of the tests fail.

4. You must not build a package when some of the tests fail.

5. You must not be happy until all your tests pass.

(As a collateral note, you should beware of programming languages that don't provide mature and well maintained tools for unit testing.)

What happens if you don't follow this approach? Well, you will be able to write your code, have fun with it, pass your exams or even get paid for it, but sooner or later you'll end up with one or more of these questions:

"When did I break this?"

"How happened I didn't realize this wasn't working?"

"Now what else am I going to break if I add this new feature?"

"How can I test those 10K lines of code, without touching them?"

"How comes the testing guys never invite me for a beer?"

Inside the brilliant Head First - Software Development, you find the description of this simple process:

1. Write the tests

2. Make the code work

3. Refactor

4. Back to 1

On the Perl side, "Extreme Perl" gives a pragmatic view of testing.

Things start to get more complicated when the application you're writing has to interact with other "external" entities, which may be even remote. I'll write on this in the future.

Perl modules: from a new idea to a debian package in one minute

Time ago I wrote something about debianizing a perl module.

If you're not applying it to a tar downloaded from CPAN or in general a third party module, but on your own module, then you may find it useful to build the module's dir structure using module-starter (installed with libmodule-starter-perl).

The CPAN page for module-starter contains a simple description on how to use it and this post has interesting comments on even easier ways to achieve the same result (using dzil for example).

In summary this is what you need to do:

$ module-starter --module=My::AModule --author="Giacomo Vacca" --email="giacomo.vacca@email.email" --builder=Module::Install

Inside the created dir (My-AModule):

$ perl Makefile.PL

$ make

$ make test

(I'm not running make install on purpose)

Outside of My-AModule:

$ dh-make-perl My-AModule/

(builds the debian dir using the current configuration)

Inside My-AModule:

$ debuild

or

$ debuild -us -uc

if you don't want to sign the deb.

Then I just suggest to have dput configured and upload the package to your repo.

Building a debian package for a CPAN module, and a known issue

The good news is that building a debian package from the source of a CPAN module is easy, thanks to the dh-make-perl tool.

The bad news is that the configuration of dh-make-perl can lead to the generation of a debian package containing unneeded files, and as a collateral effect preventing package installation as the files belong to more than one package.

For example you can end up with something like:

dpkg: error processing /var/cache/apt/archives/scnlibpoe-filter-stomp-perl_0.01_all.deb (--unpack):

trying to overwrite `/usr/lib/perl/5.8/perllocal.pod', which is also in package scnlibpoe-component-client-stomp-perl

Somebody suggests a change in the configuration file used by dh-make-perl.

The same result can be achieved by applying this change to your debian/rules file:

--- debian/rules (revision 32211)

+++ debian/rules (working copy)

@@ -51,8 +51,11 @@

- rmdir --ignore-fail-on-non-empty --parents $(TMP)/usr/lib/perl5

+ rm -rfv $(TMP)/usr/lib

Creating (easily) a local debian repo

The goal is to make my life easier when the same package may go to different distributions.

This is not useful per se, but it becomes so when you use custom distributions names to distinguish separate product branches.

Let's assume you have an application app, which has different flavours, for example alpha and beta.

You may want to have two separate debian repos for app, one called alpha and one called beta.

Hosts within environment A may refer to the alpha repo, while hosts in another environment, let's say environment B, may want to refer to the beta repo.

This keeps the small or big differences in app isolated.

Now, assume app reaches a point when the only difference between A and B is the distribution name, so you can have the same version for the alpha and beta repo: how can you avoid building the package twice? Is there a way to make both repos happy?

These are the questions that led me to the need of having a local debian repo, to experiment a little on this topic (I know I should read all the Debian Policy docs and come up with one, single, incontrovertible answer. But I prefer the empirical way, sometimes).

So far I've just created a local repo using reprepro, and following this easy tutorial. I'll update this post with a simple procedure to build your own repo and some results on how to manage the distributions (although I must admit that in this particular moment I expect a simple solution to be:

1. Just stick both distributions inside debian/changelog

or

2. Add a Provides directive to debian/control

)

All you want to know about development on Debian

... and you never dared asking. Enjoy.

mysql and regular expressions

The first time I needed to have some kind of complex select statement, I started to pray about the ability to use regular expressions with mysql.

Easy as it may be: use REGEXP!

For example:

SELECT * FROM db.ip_addresses WHERE ip_address REGEXP '^192\.168\.0\.(133|135)$';

See the official documentation and details here.

Now save and quit... doh!

If you like me are annoyed by the "recording" feature on vim (or better, by the way you can accidentally start it), then you may find at least useful to learn how it can be actually used.

This post explains it.