Hacking our way through Astricon

This year I was speaking at Astricon for Truphone Labs (you can see my slides here if you're interested).

The week before Astricon I was invited try out respoke, a solution that allows you to build a WebRTC-based service.

They provide you with a client JavaScript library, and you need a server account (with an app ID and secret key) to connect your application to the respoke server and allow clients to communicate with each other.

This is an intuitive approach. As a service developer, you pay for the server usage, and you do so depending on how many concurrent clients you want.

I got a testing account, and started trying out the JS library. The process of building a new application was very straightforward, and the docs guided me towards building a simple app to make audio calls, and then video calls as well.

Soon I started thinking: respoke is from Digium, right, and Digium develops Asterisk. How is it possible that respoke and Asterisk cannot interconnect? What I’d like to do is place a call from web, and in some circumstances route it to a SIP client, or a PSTN line, or mobile phone.

It turned out that my expectation was quite justified: 36 hours before the beginning of the Astricon Hackathon, Digium announced chan_respoke, a new module for Asterisk (13) that allows Asterisk to connect as a respoke client and communicate with JS clients.

So that was the good news. The bad news was that we didn't have any time to prepare before the Hackathon, so we had about 8 hours to get up to speed and build something… sexy!

The other service that the Astricon Hackathon was encouraging to use was Clarify, which provides APIs to upload audio recording and is able to detect some specific “tag words” from the recordings.

Among the people discussing the formation of a team the most complete and compelling idea, and one that probably did require all 5 people working together, was GrannyCall. You can see some details here, with the list of team members.

GrannyCall was thought as a system for kids to call their granny (or daddy, mummy, etc.) from a simple web page, and get a score depending on how their vocabulary was appropriate and rich.

For example, we wanted to give some positive score for words like “love” or “cookie”, and perhaps a negative score for… well, you can guess some words that would score badly for a kid talking to his/her granny.

The project was quite ambitious, because the originating call would have been from a web page built with voxbone’s webrtc library, reach Asterisk over SIP, and then ring the granny on a web page built with the respoke client.

We used an Ubuntu VM from DigitalOcean to host Asterisk and the web servers (nginx) for the two web pages, and an external web server to interconnect to the Clarify APIs for uploading the recordings (with the desired tags).

Asterisk needed to be version 13, and chan_respoke was built and configured. The DigitalOcean box was on public IP so there wasn’t the need for any specific networking.

The part “kid to voxbone to asterisk” allowed for some preparation and went smoothly right after the time to build the web server and upload the client page.

While Asterisk was being built and configured, we built the granny web page with the respoke library. Again in this case it was quite easy and quick to have a call between two respoke clients, peer to peer, just to test the client application on the browsers.

The tricky part was originating the call from Asterisk to the granny web page, using chan_respoke.

For the sake of testing the connection and media establishment, we made some calls from the respoke client to Asterisk, hitting an announcement and an echo test. That worked almost immediately, and it was great!

Now it was the key moment: can we do the full flow (kid – Voxbone –asterisk – respoke – granny)? In terms of establishing the call, i.e. signalling, that worked too just after a few tweaks to chan_respoke’s configuration. But what about audio?

It turned out that there were some problems in the ICE negotiation between the respoke client and Asterisk: we had audio only in one direction. We were using Chrome at that moment, and moving to Firefox didn’t help, so we did think there could be possibly a bug in the libraries.

Considering the maturity of the libraries, this looked completely understandable, and the respoke guys spent a lot of time helping us investigating the problem and trying to find a proper solution before the submission deadline (this resulted in a patch on the server side being applied the next hours).

Honestly, I was happy that the Hackaton was scheduled for only a relatively short time as eight hours: would it had been any longer, we probably wouldn’t have dinner or had a proper sleep (and the 8 (or 9) hours jet lag was not particularly helpful!).

At submission time, we didn't have two-way audio. Also the debugging ate precious time to prepare the presentation to the judges, and this could be the reason why we weren't awarded any prize. Honestly, given the intensity of the effort and the complexity of the project, I was hoping for at least an honorable mention, but I hope we can gather again the same team in a different occasion and bring different results!

Jokes apart, it’s been an extremely useful experience. No documentation or remote communication can replace the live interaction and working on a proof of concept – in particular if you have a crazy deadline and the body full of caffeine and sugar (and a few hours' sleep in the last 36 hours).

Of course we took some shortcuts, like removing any firewall from the host, use a common linux user, authenticated via password and not SSH keys, edited files in place, etc. We did those things knowing they weren't best practices but aiming to complete a proof of concept as quickly as possible.

The takeaway from all this is very simple: if you’re developing a new technology, or a new solution oriented to developers, do whatever you can to involve the developers in a productive, challenging way. Hackathons represent a great solution, even if confined within a company, department or team. The excitement and the feedback (and debugging) you'll help to generate will have a tremendous value.

Removing all unused docker containers and images

docker containers are very easy to build, and the ability to re-use them is extremely handy.
They do require a lot of disk space though, and if like me you need to retrieve some of that space you can delete the containers that are not running with this command:

docker rm $(docker ps -a -q)

In fact 'docker ps -a -q' lists all the containers, but 'docker rm' won't destroy running containers, and the command above will do the job.

You still have the images though (visible with 'docker images'), and each one can be relatively big (> 400MB). I tend not to tag images, so in general I'm interested in the 'latest' ones, or the official ones (like ubuntu:14.04, where ubuntu is the repo and 14.04 is the tag).

Anyway even if selecting the "untagged" ones does the job for me, I can clean up the undesired images with:

docker rmi $(docker images -q --filter "dangling=true")

I've seen other commands just relying on the tag name (and assuming 'TAG' represents and undesired image), but the filter property above should be more reliable.

My presentation at Astricon 2014

Automatic Configuration Management For Kamailio And Asterisk from Giacomo Vacca

Accessing the full P-Asserted-Identity header from FreeSWITCH

I hope this can save the reader some time.
If you need to read the entire content of the P-Asserted-Identity header of an incoming INVITE, be aware that you should change the sofia profile by adding a param like:

param name="p-asserted-id-parse" value="verbatim"

FreeSWITCH will populate the variable accordingly and make it available with commands like (e.g. with lua):

PAID = session:getVariable("sip_P-Asserted-Identity")

If you don't add this parameter, you'll get the default behaviour, which is just filling the variable with the P-Asserted-Identity URI username part.

Possible value are: "default", "user-only", "user-domain", "verbatim", which I think are self-explanatory.

A recent reference here.

Comparing Configuration Management Tools The Hard Way

I've been using Puppet for more than 2 years now, and written almost 50 custom modules, for about 4K lines of code. The more I work with it, the easier is to find a 3rd party module to solve a problem for me, and I can just focus on very specific needs.
I rarely wrote any Ruby snippets, and basically relied on Puppet's DSL. A couple of times I submitted a pull request for a 3rd party module, and got my changes merged in.

I started using Puppet for a very specific reason: there was already expertise in the company, and most part of the infrastructure pre-requirements were deployed using it (even if in a Master-Slave mode, while I favour the Standalone approach).

All this said, I can't ignore what's going on in the Devops world; many other tools are being developed to solve automation of configuration management, and with the aim of being 1. Extremely easy to use 2. Reliable 3. Scalable.

In my hunt for comparisons I've stumbled upon this article from Ryan Lane.

In my opinion, this is the way comparisons should be made: the hard way, with a 360 degrees approach. Ryan explains how his team moved away from Puppet and chose between Ansible and Salt.

I strongly recommend you take the time to go through it, as it has precious insights that are typically shared only within a team or organization.

And if you have other examples of such thorough analysis, I'd be grateful if you could comment here or drop me a note.

Disclaimer: the opinions here are my own, and I'm not affiliated with either Puppet, Ansible or Salt.

Build a test bed with FreeSWITCH and PJSUA (with SILK)

Build and run FreeSWITCH

This is based on debian wheezy, and uses master FreeSWITCH. You can switch to stable when cloning.
In my experience this is the quickest way to get to a working, vanilla setup that you can use to automate tests with PJSUA (the PJSIP command-line client).

mkdir git
cd git
# To get master
git clone https://stash.freeswitch.org/scm/fs/freeswitch.git
cd git/freeswitch
./bootstrap.sh -j
# If changes are needed to the list of modules to be compiled
vi modules.conf
./configure --enable-core-pgsql-support
make
sudo make install
# Launch FS (in foreground)
sudo /usr/bin/freeswitch

You can find here the process for building it from source recommended by the FreeSWITCH team.

In /etc/freeswitch/vars.xml you may want to change the default password in:

cmd="set" data="default_password=XXXXXXXX"

If you're enabling SILK, add it to the list in modules.conf.xml:

load module="mod_silk"

and add it to global_codec_prefs and outbound_codec_prefs inside vars.xml, e.g.:

cmd="set" data="global_codec_prefs=SILK@24000h@20i,SILK@16000h@20i,SILK@8000h@20i,speex@16000h@20i,speex@8000h@20i,PCMU,PCMA"
cmd="set" data="outbound_codec_prefs=SILK@24000h@20i,SILK@16000h@20i,SILK@8000h@20i,speex@16000h@20i,speex@8000h@20i,PCMU,PCMA"

Build and run PJSUA

Note that you need to retrieve a copy of the SILK SDK, once available from skype's developer site (and put it in ~/silk-1.0.9.zip):

mkdir pjsip && cd pjsip
svn co http://svn.pjsip.org/repos/pjproject/trunk/@4806 pjproject
cp ~/silk-1.0.9.zip .
unzip silk-1.0.9.zip && cd SILK_SDK_SRC_FLP_v1.0.9 && make clean all
cd ../pjproject
./configure --with-silk=pjsip/SILK_SDK_SRC_FLP_v1.0.9
make dep && make

Now create a configuration file for PJSUA in pjproject dir, e.g. pjsua_test.cfg:

--id sip:1001@192.168.0.10
--registrar sip:192.168.0.10
--realm 192.168.0.10
--username 1001
--password THEPASSWORD
--local-port 5066
# SRTP - 0:disabled, 1:optional, 2:mandatory (def:0)
--use-srtp 0
# SRTP require secure SIP? 0:no, 1:tls, 1:sips (def:1)
--srtp-secure=0
# Disable the sound device. Calls will behave normally, except that no audio will be transmitted or played locally.
--null-audio
# Automatically stream the WAV file to incoming calls.
--auto-play
--play-file tests/pjsua/wavs/input.8.wav

where 192.168.0.10 is FreeSWITCH's listening IP address.

PJSUA can be local (i.e. run on the same host as FreeSWITCH), or remote (with obvious network reachability constraints).

Run PJSUA from pjproject with:

./pjsip-apps/bin/pjsua-x86_64-unknown-linux-gnu --config-file=pjsua_test.cfg

and let's the tests begin!

You may want to increase FreeSWITCH verbosity by enabling SIP traces and debug messages in fs_cli with:

sofia global siptrace on
console loglevel debug

It's also easy to use the same configuration and have PJSUA as one of the parties (caller or callee) and have a softphone on the other side, like Blink or X-Lite.
The advantage of PJSUA is that you can easily automate its execution and result parsing, for example with its python module.