Skip to main content

FOSDEM 2015 - part II

In "FOSDEM 2015 - part I" I made an overview of this Conference and a few comments on the talk "Python, WebRTC and you", where the a peer-to-peer WebRTC service was also described.

Fast forward to a Lightning talks on Day 1, afternoon: Emil Ivov about Jitsi VideoBridge.

In this case we’re considering video-conference scenarios, possibly with many participants, and the ability to add some presentation features, like highlighting the current speaker.

The architecture behind Jitsi VideoBridge aims to avoid a centralized mixer (MCU), but at the same time prevent the complexities of a Full Mesh approach.

Enter the SFU (Selective Forwarding Unit) concept: the server component is “simply” a router of media streams among conference participants. A IETF Draft describes the behaviour expected for an SFU. Each participant receives one stream per each other user: it’s then up to the receiving client to take care of stream presentation.

The SFU doesn't need to decode, mix, and then re-encode all the streams, and for this reason it can use just a small amount of resources, and scales well.

The critical point, as Emil is aware too, is dealing with mobile devices: low available bandwidth and relatively limited computational resources make it prohibitive for applications running on such devices to handle many different streams, some of each potentially carrying high-quality video.

What can help in this case is the concept of “simulcast”: compatible applications (like Chrome) are able to generate streams at different quality levels (resolution and framerate) at the same time. There are still some difficulties in using simulcast, as witnessed in this post. Different topologies involved in WebRTC networks are also well described in this other article.

Anyway the receiver of such streams can then instruct the server about which quality it is capable of receiving. Apps on mobile devices will get the low-quality streams; browsers on desktops will require the high-quality ones.

This is a clever solution, but I’d still like to experiment and see examples of mobile apps managing in an acceptable way more than 2-3 incoming streams, regardless of the optimized bandwidth usage. And if you have any reference, please feel free to post a comment!
Labels:

Popular posts from this blog

Troubleshooting TURN

  WebRTC applications use the ICE negotiation to discovery the best way to communicate with a remote party. I t dynamically finds a pair of candidates (IP address, port and transport, also known as “transport address”) suitable for exchanging media and data. The most important aspect of this is “dynamically”: a local and a remote transport address are found based on the network conditions at the time of establishing a session. For example, a WebRTC client that normally uses a server reflexive transport address to communicate with an SFU. when running inside the home office, may use a relay transport address over TCP when running inside an office network which limits remote UDP targets. The same configuration (defined as “iceServers” when creating an RTCPeerConnection will work in both cases, producing different outcomes.
Read more

VoIP calls encoded with SILK: from RTP to WAV

SILK is a codec defined by Skype, but can be found in many VoIP clients, like CSipSimple . It comes in different flavours (sample rates and frame sizes), from narrowband (8 KHz) to wideband (24 KHz). Since Wireshark doesn't allow you to decode an RTP stream carrying SILK frames, I was curious to find a programmatic way to do it. In fact, this has also allowed to me to earn a "tumbleweed" badge in stackoverflow . You may argue that a Wireshark plugin would be the right solution, but that's probably for another day. Initially I thought it was sufficient to read the specification for RTP payload when using SILK ; the truth is that I had to reverse engineer a solution by looking at SILK SDK's test vectors. There, I discovered that a file containing SILK audio doesn't have the file header indicated in the IETF draft ("!#SILK"), but a slightly different one ("!#SILK_V3"). More importantly, each encoded frame is not preced...
Read more

Extracting Opus from a pcap file into an audible wav

From time to time I need to verify that the audio inside a trace is as expected. Not much in terms of quality, but more often content and duration. A few years ago I wrote a small program to transform a pcap into a wav file - the codec in use was SILK. These days I'm dealing with Opus , and I have to say things are greatly simplified, in particular if you consider opus-tools , a set of utilities to handle opus files and traces. One of those tools, opusrtp , can do live captures and write the interpreted payload into a .opus file. Still, what I needed was to achieve the same result but from a pcap already existing, i.e. "offline". So I come up with a small - quite shamlessly copy&pasted - patch to opusrtc, which is now in this fork . Once you have a pcap with an RTP stream with opus (say in input.pcap ) you can retrieve the .opus equivalent (in rtpdump.opus ) with: ./opusrtp --extract input.pcap Then you can generate an audible wav file with: ./opusd...
Read more