Sunday 30 November 2014

Bridging WebRTC and SIP with verto


Verto is a newly designed signalling protocol for WebRTC clients interacting with FreeSWITCH. It has an intuitive, JSON-based RPC which allows clients to exchange SDP offers and answers with FreeSWITCH over a WebSocket (and Secure WebSockets are supported). It’s available right now with the 1.4 stable version (1.4.14 at the moment of writing).

The feature I like the most is “verto.attach”: when a client has an active bridge on FreeSWITCH and, for any reason (e.g. a tab refresh) it disconnects, upon reconnection FreeSWITCH automatically re-offers the session SDP and allows the client to immediately reattach to the existing session. I have not seen this implemented in other places and find it extremely useful. I’ve noticed recently that this does not fully work yet when the media is bypassed (e.g. on a verto-verto call), but Anthony Minnesale, on the FreeSWITCH dev mailing list said this feature is still a work in progress, so I’m keeping an eye on it.

Initially I was expecting an integrated solution for endpoint localization, i.e. what a SIP registrar can do to allow routing a call to the right application server. On second thoughts I don’t think this is a problem and there are ways to gather on which FreeSWITCH instance an endpoint is connected, and then route a call to it.

Once a verto endpoint hits the dialplan, it can call other verto endpoints or even SIP endpoints/gateways. I’ve also verified that verto clients can join conference rooms inside FreeSWITCH, and this is not only possible but can be done for conferences involving SIP endpoints as well, transparently.

This brings me to what I think it’s the strongest proposition of verto: interoperability with SIP.

In my opinion WebRTC is an enormous opportunity, and a technology that will revolutionize communications over Internet. WebRTC has been designed with peer-to-peer in mind, and this is the right way to go, however if you want to interoperate with VoIP (either directly or as a gateway to PSTN and GSM) you can’t ignore SIP.

I’m not worried about Web-to-Web calls: there are already many solutions out there, and each day there’s something new. Many new signalling protocols are being designed, since WebRTC standardization, on purpose, hasn't mandated any specific protocol for signalling. Verto is a viable solution when on the other side you have SIP.

I've been experimenting on this for some time now. In August I presented a solution for WebRTC/SIP interoperation, based on Kamailio andFreeSWITCH, at ClueCon. In that case signalling was accomplished with SIP on both sides (using the JsSIP library on the clients); unsurprisingly, after using verto, SIP on the web browser client side looks even more redundant, over-complex, but most of all with a steeper learning curve for web developers, and this is becoming every day a stronger selling point for new signalling protocols for WebRTC applications.

Web browsers running on laptops can easily manage multiple media streams incoming from a multi-party call. This is not true for applications running on mobile devices or gateways: they prefer a single media stream for each “conference call”, for resource optimization and typical lack of support respectively (1). Verto-SIP can represent a solution to bridge the web/multistream world with the VoIP/monostream one, for example by having the participants inside a conference room.

When video is involved though, things get as usual more complicated. WebRTC applications can benefit from managing one video stream per call participant, and a web page can present the many video streams in many ways.

But this can easily become too cumbersome for applications on mobile devices. We need to be able to send one single audio stream and video stream. And whilst the audio streams are “easy” to multiplex, how do you do that for video? Do you stream only the video from the active speaker (as FreeSWITCH does by default on conferences), or do you build a video stream with one video box per participant? The Jitsi VideoBridge is a clever solution leveraging a multi-stream approach, but again, how about applications running on mobile devices?

For what concerns signalling interoperation/federation there is an interesting analysis at the Matrix project blog. The experience gathered last Friday when hacking Matrix/SIP interoperability through verto/FreeSWITCH has also shown some key points about ICE negotiation: I recommend reading it.

My view is that there are two key points that will allow a solution to be successful in the field of Web-based communications involving “traditional” Internet telephony but also mobile applications:

  1. Interoperability with SIP.
  2.  The ability to provide one single media stream per application/gateway, should they require it.


What do you think?


(1)   Yes, I know that nothing prevents a SIP client to manage multiple streams, but practically speaking it’s not common.

27 comments:

  1. I am struggling with the documenation of Verto, calls are working fine, however some internal stuff is difficult to understand, where do you get the internal docs ?

    ReplyDelete
  2. Hi Grygoriy, to be honest I've mainly read the source code and more importantly the code for the demo the FreeSWITCH team has at https://webrtc.freeswitch.org.

    Feel free to send me questions you may have and I'll try to answer.

    Perhaps for the people interested it's worth creating a public doc about this (granted the module and protocol may continue to change).

    Regards.

    ReplyDelete
  3. Hi Grygoriy, to be honest I've mainly read the source code and more importantly the code for the demo the FreeSWITCH team has at https://webrtc.freeswitch.org.

    Feel free to send me questions you may have and I'll try to answer.

    Perhaps for the people interested it's worth creating a public doc about this (granted the module and protocol may continue to change).

    Regards.

    ReplyDelete
  4. Hi, to put it simple i have this issue:

    1: I call from line 1 my mate Sasha
    2: I put her on hold
    3: I call someone from the line 2 (so verto creates another dialog)
    4: I finish the conversation on line 2
    5: I bring back the line 1
    6: Sasha hears me fine, and i dont hear her
    7: I hit Reload button, page reloads(and verto do re-initialisation),
    and i can hear Sasha back again, our call was not cut.

    Looks like i a missing something obvious here, i was thinking "Bring
    back" from hold button should bring the audio stream back, but i sense
    that i must provide something to bring the correct dialog back in thie
    scenario.

    ReplyDelete
  5. Forgot to put our web interface example : http://s15.postimg.org/lxbhva2d3/S_lection_105.jpg

    ReplyDelete
  6. It's hard to say without seeing:
    - The verto JSON exchanged
    - FreeSWITCH logs

    Look into the logs for hints about problems re-establishing the session.

    The reason why the tab refresh works is that verto has a "attach" feature: when a client disconnects during a call FS keeps the session up and re-negotiates it as soon as the client comes back with the same id.

    ReplyDelete
  7. Hi, any chance we can talk, i did couple of traces here, with the problem.

    ReplyDelete
  8. Here is the log when agent 100000111 (m dev) put me back online 1212112 and i can hear him, and he does not hear me, only after re-attach

    http://dpaste.com/137VGM3

    ReplyDelete
    Replies
    1. HI Grygority, were you able to figure it out? I'm having the same issue.

      Delete
    2. Hi Grygoriy, were you able to figure it out? I'm having the same issue. Thanks.

      Delete
  9. Hi Grygoriy, I'll have a look next week, & will give you a way to have a chat. On mobile now.

    ReplyDelete
    Replies
    1. Hi Giacomo, were you able to figure out this issue? I'm having the same issue. Thanks

      Delete
  10. Hi Giacomo,

    I am having sip client and i want it to integrate it with freeswitch( acting both as media server and application server ) , i want to create a video conference, but i am confused how to implement, do i have to use verto or sofia, or both are same.
    Any help would be greatful.
    thanks

    ReplyDelete
  11. Hi, in brief verto doesn't use SIP, but a newly created protocol which uses JSON.
    If you already have a SIP client and wish to continue using it, then the sofia module will suit you fine.
    In general, you can have verto and SIP clients connected to the same conference: FreeSWITCH will do the "translation" for you.
    So perhaps you could spend some time experimenting with a verto client and see if it provides a valid option for your project, knowing anyway that you can use only SIP, or the two combined.

    ReplyDelete
  12. Hi Giacomo,

    is it possible to avoid stun or turnserver by using verto.js ?

    i am using sipjs and freeswitch on windows

    ReplyDelete
    Replies
    1. Hi akin,
      theoretically, if all the media can flow through FreeSWITCH, you don't need a STUN or TURN server. The clients won't need to discover all their public IP addresses because the other parties will always see FreeSWITCH as their media "interlocutor".
      This in fact is independent from the client technology/protocol, so it applies to both sipjs and verto.
      A proper answer though would be possible only understanding your architecture and call scenarios.

      Giacomo

      Delete
  13. There is any way to combine both sipml and verto js file to get running conferences details on web page...

    ReplyDelete
    Replies
    1. Hi Anil,
      I'm not sure I understand what you're trying to achieve. Do you want to provide a conferencing system where both verto and sipml clients are used at the same time? Or trying to chose one of them?

      Giacomo

      Delete
  14. Hi,
    I have configured FeeeSwitch with Outgoing Gateways / Trunks. with Sip client I am able to make Extension to Extension call and cane Route Calls to Gateway but when I use WebRT SIPML5 i can only make calls between extensions. With WebRTC calls are routed to Gateway(IDTExpress) but not connected I think I am missing something before Bridge to gateway.

    Please guide

    ReplyDelete
  15. Hi Imran, it would be interesting to see what's your dialplan.
    Verto endpoints are just specific endpoints, like SIP endpoints (managed by mod_sofia) are. In principle what you have to do in the dialplan managing calls from verto is originate calls specifying that the B-leg will be a SIP session towards the desired destination. FreeSWITCH will do the bridging for you, and it seems you already have such a configuration.
    For diagnosing the actual issue, try sharing your configuration/setup (sanitise any sensitive info first) and the logging you get when calls are failing.
    Anyway look at the signalling, in particular at the SDP messages exchanged, and try to figure out whether the IP addresses involved make sense, the ports dedicated to the RTP sessions are usable, and whether any of the entities involved is trying to send RTP packets to the wrong destinations.
    Ensure also that the outbound gateway is being offered the codecs that it expects (e.g. PCMA, rather than the unlikely to be supported/accepted Opus).

    I hope this helps anyway,
    Giacomo

    ReplyDelete
  16. Hi...
    I use fusion GUI for provisioning the freeswitch... I installed verto communicator on freeswitch but when I try to access it, I get 403 or 404.. I do have ssl certs and the path is provided in verto.conf.xml..I asked for help from freeswitch IRC but no luck in fixing the issue. could you or anyone help me how to make this working. your help is much appreciated.

    Thank you
    krish

    ReplyDelete
    Replies
    1. Hi Krish,
      when do you get the error? What do you see in fs_cli? In the logs? Those elements would be very useful to start investigating the issue.
      Increase console log level to debug.
      Once you have all this information, try sending an email to the freeswitch-users mailing list. They are very active and responsive, and I'm sure you'll end up with a solution. Feel free also to send that info here.

      Cheers,
      Giacomo

      Delete
  17. Please ignore my previous message as I fixed the issue.. its nginx index missing case under location path..I can access it now.. but now the issue is the verto not communicating with the server...

    ReplyDelete
  18. This comment has been removed by the author.

    ReplyDelete
  19. I am building webrtc app. I have problem with inbound call ring.

    When i logged in then inbound call ring rings.
    But from then no ring rings.
    I need to refresh application to listen to ring.

    ReplyDelete
  20. I am extending the functionality of verto and trying to implement chat apart from conference call. Is there any way I can use the chat module of conference call or do I need to implement a separate standard chat module to enable chat functionality between the registered users?

    ReplyDelete
  21. Hi Giacomo.

    How many users can support a verto communicator conference in mux mode?
    I know it's depend of hardware too, but I mean the html layout has a maximum?, example 20 users connected to conference room?

    Regards

    ReplyDelete

About ICE negotiation

Disclaimer: I wrote this article on March 2022 while working with Subspace, and the original link is here:  https://subspace.com/resources/i...