Sunday, 21 January 2018

Cache busting when building Docker images

One of the handiest features of the docker build system is the caching system.

'docker build' tries to reuse the layers already built until something changes inside the Dockerfile. In this way, we can save several minutes when rebuilding an image if the changes happen further down the list in the Dockerfile.

Sometimes, though, we do want to invalidate the cache and ensure the next build won't use it.

To do this an option is to pass the '--no-cache' argument to 'docker build'.

When dealing with 'apt-get install' instructions though there are other tricks. I found this document on Dockerfile best practices very useful.

First of all an observation. If you have 'RUN apt-get update' as a single line of a Dockerfile, followed by the installation of a package, e.g.:

RUN apt-get update
RUN apt-get install -y nginx

then changing the list of packages and running again the build command won't trigger an 'apt-get update': that line hasn't changed so docker build reuses the cache. It might not be what you want.

To force cache invalidation for this specific case the recommendation is to use those commands in the form:

RUN apt-get update && apt-get install -y nginx

This will always install the latest version of the packages. It even has a name: "cache busting".

Another recommendation I like is to put each package on a single line, and have them in alphabetical order: this will ease visual inspection and prevent duplicates or other undesired conditions.

Of course, you can also specify exact versions for the packages as you would normally do with 'apt-get install'. That's "version pinning" and it invalidates the cache too.

You can find all this on the linked page on Dockerfile best practices; this is just my digested interpretation.

Just one more thing: a way to limit the size of a built image is to clean up the content of '/var/lib/apt/lists' in the same RUN command, e.g.:

RUN apt-get update && apt-get install -y \
    aufs-tools \
    automake \
    build-essential \
&& rm -rf /var/lib/apt/lists/* 

The command above will build an image layer that doesn't contain the apt cache.

If you had instead used this:

RUN apt-get update && apt-get install -y \
    aufs-tools \
    automake \
    build-essential
RUN rm -rf /var/lib/apt/lists/*

you would have had not only a larger layer, containing the apt cache, but also an additional layer generated by the second RUN command.


Saturday, 13 January 2018

SIP - ACK loose routing

If you've ever worked with SIP, you must have stumbled upon a trace with 200 OK to INVITE being retransmitted for about 30" and then the call just being set up fail.

The ACK was never received.

Then comes the interesting part: discovering why.

Here are some notes about what should happen, in particular when there are multiple proxies along the path, and with a little additional complexity of one of the proxies with two network interfaces. All this assuming loose routing everywhere. The main reference here of course is RFC 3261.

Isn't an image worth a thousand words? Then here's a sequence diagram:


All Record-Route headers are assumed to carry loose routing URIs (they have the ;lr attribute).

B, C and D, working as proxies that want to stay in the path, record route themselves. For this reason E, the UAS and "callee", receives an INVITE with a list of Record-Route headers with B, C and D.
In particular for B there will be two Record-Route headers, since B is using two separate interfaces, one facing A and the other facing C.

In typical cases the two interfaces represent the interaction with the public Internet on a side and a private infrastructure on the other. But it's not important for this discussion.

Omitting provisional responses for simplicity, let's assume E responds immediately with a 200 OK. This response will have the same list of R-R headers, in the same order, as received by E.
E will also add its URI in the Contact header of the 200 OK.

In this loose routing context the IP address in E's Contact's URI will be relevant only for D in the future.

D, C and B don't modify the list of Record-Route headers, and A receives it as sent by E.

Apart from the operations related to the media session set up, A will send the ACK to the 200 OK.
This ACK will have a Request URI with E's Contact URI (stripped of anything that can't se inside a Request URI), and a Route header list which is basically the received Record-Route header list inverted (see images).

A is saying: "Route this ACK to E, routing it via this list of hops".

When B receives that ACK, it must recognise that the topmost Route headers are B itself, remove them from the Route list, and pick b2 as the interface to deliver the ACK to C.

C and D will have an easier task to remove a single Route header, the one representing them, and deliver the ACK to the next route.

For D, the next route will be in fact E, and the ACK will be routed using only the Request URI, as the Route headers have all been eliminated. This is the only step where the IP address that E has set in the Contact of its 200 OK response needs to be visible by another entity, namely D.

ACK routing as explained in RFC 3665


To further reiterate this concept, let's look at a somewhat simpler example in RFC 3665.


The ACK part is:

You can see there's no requirement for Proxy 1 to be able to reach the UAC contact (client.biloxi.example.com might be completely unreachable from Proxy 1).
It's Proxy 2's responsibility to route the ACK in the last hop towards Bob.
Proxy 1 must leave the R-URI as is (see below for more details on proxy behaviour), strip itself from the list of Routes and route the ACK to the new topmost Route (Proxy 2).
Proxy 2 will strip itself from the Route list, being the topmost Route, and forward the ACK to Bob. There are no more Route headers.
Only at the last hop Bob's contact reachability is relevant, and it is for Proxy 2 only.

More about the behaviour of the proxies to corroborate the ACK routing


From RFC 3261, 16.4:

“  If the first value in the Route header field indicates this proxy,
  the proxy MUST remove that value from the request.”


From RFC 3261, 16.5:

“     A proxy can only change the Request-URI of a request during

      forwarding if it is responsible for that URI.”


APPENDIX - Why is the ACK to 200 OK to INVITE a separate transaction?


From RFC 3261, ch. 17:

     The reason for this separation is rooted in the importance of
      delivering all 200 (OK) responses to an INVITE to the UAC.  To
      deliver them all to the UAC, the UAS alone takes responsibility
      for retransmitting them (see Section 13.3.1.4), and the UAC alone
      takes responsibility for acknowledging them with ACK (see Section
      13.2.2.4).  Since this ACK is retransmitted only by the UAC, it is
      effectively considered its own transaction.






Tuesday, 9 January 2018

Copying a file from a Docker container to the host

Often things are more convoluted than you expect. Sometimes they are easier than you fear.

Here's an example: you're generating files inside a Docker container (with no volumes configured) and you realise you need some of those files available in the host.

A simple solution is to use 'docker cp', with a format like

docker cp CONTAINER_ID:FILE_PATH HOST_FILE_PATH

Official documentation.
A Stackoverflow question with more discussions.