Tackling Three FFmpeg’s Technological Challenges to Further Shape TR-03 Standards for Broadcasters

Industry Context

“Today, the magic of film and television is half visual storytelling and half technical wizardry” (SMPTE, 2017). Major broadcasters (e.g., Radio Canada) largely depend on proprietary big ticket professional digital broadcasting equipment whose standards are written and shaped by Society for Motion Pictures and Television Engineers® (SMPTE®) which includes both the broadcasters and the manufacturers of the equipment. In short, SMPTE’s main job is to help advance “moving-imagery engineering” across the broadcasting industry. These heavy duty broadcasting machines are capable of transmitting raw data (i.e., uncompressed, unencrypted digital video signals) within television facilities. Recently, Savoir-faire Linux– a leading open source software service provider in Canada− has embarked on a journey (financially supported by Radio Canada) to test a technological possibility: can we adapt FFmpeg, using a general purpose server, in order to transmit raw data at volume/speed of 3.5 Gbps without relying on specialized hardware broadcasting equipment?

Savoir-faire Linux Takes Up the Technological Challenge

The journey started this winter which was by the way extremely cold and snowy in Montreal. A team of three from the product engineering department of Savoir-faire Linux decided to spend some time looking at FFmpeg’s internals in the hope of getting a working TR-03/SDI pipeline processing up to a several HD streams on a contemporary server while benefiting from FFmpeg’s easily available lower definition derivatives from the same video stream.

A few challenges arose from this endeavor. The first was the TR-03 format itself which wasn’t supported in the upstream version of FFmpeg. The second was the volume of data to be processed: we are talking about 3Gb/s of traffic here, which according to the FFmpeg developers might not be possible to process. Finally, the need to add transcoding to the pipeline which meant even greater CPU load.

Implementing TR-03 was not the biggest challenge, as the video format was pretty straightforward, and the team quickly got a working implementation. In this context, they used GStreamer, which was already very capable in terms of streaming as a sender for this volume of data.

Image result
Logo of FFmpeg

Once the data began to flow in, the performance bottlenecks became even more evident. To clearly locate these bottlenecks, they made the decision to write a few benchmark scenarios using a combination of unit tests and LTTng (i.e., an open source tracing framework for Linux). These benchmark scenarios allowed the team to detect where they were dropping a bunch of packets (i.e., losing data at different layers) at both the kernel socket and NIC buffers. Since the data processing process was closely monitored, it was relatively easy to tweak the buffer sizes, while keeping the delays within our acceptable range. However, another issue arose. The team noticed that FFmpeg’s data gathering/decoding thread was hogging a single CPU core causing occasional packet loss on the occasion when they were not fast enough to dequeue data from the socket buffer. To work around this, they decoupled the data gathering work from the decoding part. Following through these steps carefully, made it possible to have the packet drop-free pipeline, an achievement celebrated with a fine cake and a few drinks in front of a cool movie they could finally watch. As the head of the engineering team mentioned, “I could have never imagined that Big Bug Bunny was so much fun”.

Big Buck Bunny is a short computer-animated comedy film by the Blender Institute, released as an open-source film under Creative Commons License Attribution 3.0.

The Latest Update and the Road Ahead

Later, on April 5, 2017, the contributions developed and refined by the team were finally integrated into FFmpeg, meaning that, their proof of concept met with the stipulated standards of the community and became part of the platform.

For Savoir-faire Linux’ product engineering team, this was a very encouraging and promising adventure. Despite all odds, they empirically demonstrated the possibility to run SDI processing pipelines on top of a general purpose server equipment using open source software. This experiment was a success as it turned out there was great potential for broadcasters. To continue in this direction and further shape the standards of SMPTE 2110which are not frozen yet – being still in embryonic phase they face another main challenge. FFmpeg does not support the synchronization as described in SMTPE 2110; so, the team is now evaluating the possibility to provide and include the support needed.

Authors:

  • Damien Riegel,
  • Eloi Bail, and
  • Stepan Salenikovitch.

Savoir-faire Linux Opens a Red Hat Individual Exam Center in Montreal: A Timely Response to Shortage of Linux and OS Experts in Quebec

Montreal, March 28, 2017 – From April 1, 2017, Savoir-faire Linux, the leading free and open source expert in Quebec and Canada, will host Red Hat® individual exam sessions at its Montreal office.

From left to right : Pierre Lamarche (Director of Business Development and Strategic Partnerships), Raymond Cantin (Director of Training Dept.), and Eric Schaefer, Red Hat-Canada Channels Leader.

As a Red Hat® Advanced Business Partner, Savoir-faire Linux will now be equipped with a kiosk to run individual Red Hat certification exams on its premises in Montreal. The exam will take place on a personal test station, secured and accredited by Red Hat® within a Quebec-based company. The Red Hat® Individual Examination Booth will cover a wide range of certifications: Red Hat® Certified System Administrator (RHCSA), Red Hat® OpenStack System Administrator, Red Hat® Hybrid Cloud Management Expertise, Red Hat® Hybrid Cloud Storage Expertise, JBoss Developer, Jboss Administrator (RHCJA), and many more.

“We are very pleased that Savoir-faire Linux takes the lead in opening this unique individual exam center in Quebec. It has been logical to select Savoir-faire Linux because the company leads the free and open source initiatives in Quebec and has deep expertise in free open source software.”

Eric Schaefer, Canada Strategic Alliances and Business Development.

There is no need to emphasize that Red Hat® has already become the world leader in open source software technologies and it has done so within a relatively short time, about 2 decades or so. By collaborating closely with a broad network of IT leaders, open source supporters, developers and business partners, Red Hat® technologies have already changed the landscape of information systems and the companies success is echoed globally. Moreover, the digital revolution, marked by the rise of open technologies and the recognition of digital strategies at the heart of innovation and business growth, has already impacted the Canadian market and IT sector. According to Information and Communications Technology Council (ICTC)’s 2015 Labor Market Outlook, the cumulative hiring requirements for ICT talent in Canada will be approximately over 182,000 by 2019.

The Linux Foundation itself has published a report in 2016 in which it describes a growing shortage of Linux and open source specialists. 87% of professionals acknowledge the difficulty of recruiting adequately trained and certified experts; while 59% intend to increase the recruitment of these same specialists to meet the digital transformation challenges they face.

“In a rapidly changing technological sector, where competition is fierce to attract top talent, we consider these certifications to be essential to support the development of a pool of digital talents in Quebec and strengthen Quebec’s position in the digital economy of the North America. This kiosk installation was more than necessary to respond locally to the emerging technological trends”

Christophe Villemer, the Executive Vice-President of Savoir-faire Linux.

With this new service in Quebec, Savoir-faire Linux becomes the only Red Hat®’s partner in Quebec to offer both Red Hat® training and certifications in a collective and individual fashion. The company intends to consolidate its position as a major partner in the digital strategies of Quebec companies.

References:
Report “2016 Open Source Jobs Report” (May, 2016).
ICTC Press release (March 12, 2015).

Lemon LDAP::NG, Villeurbanne’s authentication system choice

The city of Villeurbanne (France) is interested in open source technologies and has chosen Lemon LDAP::NG to control its users’ access rights.

The city of Villeurbanne had many web applications where the authentication was already delegated to a central CAS server (Central Authentication Services), modified to Villeurbanne’s needs, to give access to current internal users in the Active Directory as well as to external users stored in a database.
By adding new applications to the city’s information system (IS), it became necessary for the IS managers required the need to control access rights in order to ensure that certain users will not access confidential information.

A solution proven to respond to the «unique authentication» issues of the city of Villeurbanne

Having searched for an appropriate authentication system and having consulted experts about different possibilities and the pros and cons of different systems, the town’IS makers’ decision was drawn to LemonLDAP::NG as a unique authentication system.
LemonLDAP::NG is a unique authentication software identity manager distributed under the GPL license and adapted to companies (centralized authorization management, interoperability with all the identity exchange protocols such as CAS, OpenID, Connect, SAML,…). With a very simple installation process, the configuration can be done in text or graphic mode. LemonLDAP::NG is generally compatible with a number of web applications.
«Within the scopeof this Intranet project, composed of open-source bricks, we were looking for a unique authentication tool which allows to assign access rights based on active directory account characteristics, explains Jean-Patrick Trauet, Tehchnical and Security Manager. After analyzing different solutions, we chose Lemon::LDAP which responded perfectly to our needs.
It also became an opportunity to create an authentication portal for our web applications. In the future, we envision to evolve our Lemon::LDPA towards strong authentication.»

«After analysis different solutions, we chose Lemon::LDAP which responded perfectly to our needs. It also became an opportunity to create an authentication portal for our web applications»
Jean-Patrick Trauet, Technical and Security Manager


A project lead by Savoir-faire Linux
The city of Villeurbanne asked Savoir-faire Linux to accompany them in the implementation of LemonLDAP::NG.To help them more effectively, we had to break the project down into several phases.

  • The implementation of an OpenLDAP directory to replace the referential external identity database
  • The creation of connectors to automatically synchronize the accounts and Active Directory groups in the OpenLDAP directory
  • The management of group rights in OpenLDAP associating internal and external users
  • The implementation of LemonLDAP::NG as a CAS server to replace the old system and personalize the authentication pattern
  • The migration of applications towards the new CAS server

The new authentication portal

 

The main benefits for the city of Villeurbanne

The city of Villeurbanne now has at its disposal a central identification (internal and external) directory and a WebSSO tool and access control. The possibility of using LemonLDAP::NG as a CAS server, but also as a supplier of headers, SAML, or OpenID Connect, allows to connect a large scope of applications.

Key data

  • Number of users impacted : approximately 1000 users
  • Number of project management days : 10 days
  • Number of concerned applications : between 5 and 10
  • Technologies used : OpenLDAP, LSC and LemonLDAP::NG , EZPublish, Alfresco, iTop, Piwik, Orchestra, développements internes en PHP

FOSDEM 2017 : our impressions

Packed amphitheatre for the keynote on the «Kubernetes on the road to GIFEE»

In the 2017’s FOSDEM event, held in Brussels, Soizik, Cécile, Clément, Adrien and Andreas represented Savoir-faire Linux. Here, they share briefly their experience and impressions from their participation in the event.

Soizik Froger, project manager

It was my first FOSDEM event, and I loved it. I have only one regret: I wish I could have attended every session, and talked with everyone!! It was a tremendous place for tech-savvy and free software lovers like me. Seeing those panelists with very high level of expertise has taught me a lesson about humility that comes with great knowledge.

I definitely give five stars to Frenchy Matthieu Totet’s talk on Graph Cycles. This is a passionate work on a real time graph on Twitter and how we can seize the fantastic potential of this technology!

I almost cried of joy at open Q&A session with members of the OpenJDK Governing Board: Mark Reinhold, Mario Torre, Andrew Haley, Georges Saab & Doug Lee (embedded). I secretly took a picture, and I’m sure I will keep it in front of my eyes until the next FOSDEM.

Open Q&A session with members of the OpenJDK Governing Board: Mark Reinhold, Mario Torre, Andrew Haley, Georges Saab & Doug Lee (embedded)

I give the best ‘orator palm’ to Christian Thalinger for his benchmarking of Compiler JIT Graal on Twitter services. It was such a clear and organized speech that a slow guy like me understood it all.

Finally, I discovered what technologies lie behind Ring. Our session was a great success. We were excited by the volume of the turn-out. We had a diverse and enthusiastic audience that posed questions about the architecture of Ring and its ambitious potential for scalability. High five to the Ring’s development team!

Cécile Delépine, Delegate Europe General Manager

That was my first FOSDEM too! It was a great opportunity to meet with the industry in a both cozy and professional environment.
My main focus was containers orchestration with Kubernetes, the fast growing Open Source community at the time being. FOSDEM is the place for technological intelligence, which I see as a BizDev not as an engineer.

Students and tech-related senior job seekers, please take note : FOSDEM is also a place for hiring talents (we invaded FOSDEM job corner with all our job posts for France and Canada). Our presentation on Ring , and Clement’s commitment to LemonLdap coupled with our presence on OW2 stand also significantly contributed to promoting our expertise at Savoir-faire Linux.

I wish I attended Mozilla conference too. What does motivate the open source community? This is the question posed by Rina Jensen, whose conclusions should be widely shared.

Some food for thoughts.

My five stars goes to Software Heritage, a project supported and initiated by Roberto Di Cosmo (INRIA, GTLL Systematic Member). The source code of our projects is a common good, thus it should be available to all and be centralized in one archive which does not depend un one solution and does not vary from one solution to another (Github, Bitbucket…)

Last but not least, I bought the book How Linux works and I almost finished the second chapter;-)

Clément Oudot, infrastructure and security expert

This year, I represented LemonLDAP::NG projet. I then met with Fusion Directory and Spoon communities., It was the first time we had an OW2 stand, and it showed that this consortium was not only Enterprise oriented but also community-oriented.

I was pleased by the large number of attendees at Perl stand and the packed Devroom.
I am still surprised to see Perl can attract so many people. What a wonderful turn-out for Perl!

During this weekend, I met with lots of people I knew from different communities : Framasoft, XMPP, OpenStack, VLC, PHP etc. That’s why I love going to FOSDEM!

The OW2 Stand with Clément Oudot (c), Cécile Delépine and Soizik Forger
Ring conference : What technologies lie behind Ring

Adrien Béraud, Ring system engineer
FOSDEM is an exciting and inspiring place. We met with lots of interesting people with interesting projects, There were too many people! [8000 people attended the event]

As for the Ring session, it was amazing to see how exciting the audience were. Last year, people discovered Ring, this year, they wanted to know a lot more about the technology. No wonder there were so many of them. Actually, there is a growing interest for distributed communication systems.

My five stars goes to Alok Anand, who presented Telepathy Connection Manager for Ring. He was able to develop it because he joined Google Summer of Code’s program which accommodated Ring from Savoir-faire Linux. Thanks to such a wonderful collaboration and such a nice program for young coders.

Andreas Traczyk, Ring developer

Like Soizic and Cecile, it was my first time at FOSDEM in Brussels. It surpassed all my expectations. There were so many passionate people, so many nerds in the same place;-) It was intense!

A packed room for Ring conference

I was really impressed by the turn-out of the audience during the Ring session, Although we did not haveso much time, we were well received, and the audience’s questions were challenging. Besides, I wish we had more space and more time!

My five stars for the best presentation goes to Daniel Pocock for his excellent talk on the real time communication solutions crowdfunding. It was so interesting, so inspiring.

Blockchain, a revolutionary tool for Ring

social-network-sphereBlockchain technology brings on a new revolution, as its advocates write on every wall, it is likely to deeply change our verticaly-shaped society.  Savoir-faire Linux uses its mechanism to develop Ring distributed ledger and users ID management.

Blockchain is the technology on which Bitcoin is based. This digital cryptocurrency displays both fear and covetness among the financial sector. However, beyond bitcoin, blockchain offers other exploitations, thanks to its decentralized, anonymous and secure operation. In addition, it reveals governance challenges, which can tumble organizations as we’ve seen them for decades. That’s why blockchain is one of the major issues of the 2016 Paris Open Source Summit, November 15th and 16th.

A decentralized society, without trusted third parties

There is no doubt about it : blockchain technology will have as much as an inpact as the invention of the web. According to its promoters, this technology is more than a tool, it will change the shape of society, towards a more decentralised one. Blockchain will overturn the trusted third party concept, if not remove it.
Blockchain works as a big public ledger, which registers every transaction between users. This large database is shared by all of them. Their identities are systematically verified by asymetric encryption. The transactions are put in a block, encrypted and certified by the network’s node – composed of other users aka miners- the block will be added to the chain and distributed through the network to all users.
The blockchain’s fundamentals are distintermediation, traceability and distributed consensus. As they are organized in decentralised nodes, users act as a validation authority. Hence, the transactions are traceable in the public and shared ledger. The distributed consensus lies in the following fact : every node receives the same chain of timestamped data through this huge ledger, and what’s more, every change has to be approved by a majority of miners.
Blockchain was born where cryptography and decentralised networks meet. The 2008 financial crisis and the lack of confidence towards banks were its breeding ground. Besides Bitcoin, created by Satashi Nakamoto, whose real name is not yet publicized, several trustless architectures have been set up. Among them, Ethereum is one of the most widely used. This non-profit foundation has a blockchain based system, which takes up the ID ledger and unforgeable history to execute any sort of code, called «smart contract»

Blockchain, a fundamental tool for Ring

Savoir-faire Linux’s team of experts has used Ethereum technology to include encryption standards in the creation of Ring’s database The smart contract is therefore the public ledger of names associated with a RingID, which is created by the application. As in every blockchain «each link depends on the previous elements to ensure the security of the database and to make it unforgeable» explains Adrien Béraud, Systems engineer, in charge of developing the distributed hash library OpenDHT.
Blockchain integration in Ring’s public key directory follows Savoir-faire Linux’s guidelines : to preserve the distributed network as Ring puts it, to ensure security with encryption keys and to let the user opt in or opt out of the public ledger. « This option offers the user the possibility to register or not the database and only use his RingID to communicate onto the network» details Adrien Béraud.
The smart contract coding is key. Given the fact that the decentralised registry is optional, users can preserve their pseudonymity through its RingID, which is then not associated with an alias or a username. Some smart contracts failed, because the code revealed some errors or vulnerabilities. Ring’s development team at Savoir-faire Linux tries toavoid any risk by following a basic principle: «we try to write a code with a simple design, with great care».
«Furthermore, we put the focus on a thorough QA, code review and we rely on external outputs», he concludes.
Contribute to Ring’s succes as well as Blockchain philosophy by coding with us!

References
The Revolution will (not) decentralised: Blockchains, Rachel O’Dwyer, Commons Transitions, 11 june 2016
Thinking through Law and Code – The future of State and Blockchain, Julian Feder, Backfeed magazine, 17 january 2016
Blockchain reaction, tech companies plan for critical mass, Ernst & Young Report, 2016
Privacy on the Blockchain, Vitalik Buterin, Ethereum Blog,15 january 2016
Comprendre la blockchain, Livre blanc, licence Creative Commons, U, January 2016
Les smarts contracts pour les non développeurs, Blogue Ecan, 23 juin 2016
Lexique de la blockchain, Blockchain France

Multidevice is now available on Ring

multidevice

Since its inception, Ring’s features and users have significantly increased. Savoir-faire Linux’s team of experts has recently released the Bêta 2 version offering without an exception new possibilities.

One account, many devices

This new release allows users to benefit from a new feature: multiple connection. As in centralized messaging systems, Ring now allows the connection of multiple devices – mobile or not – using a single user ID. Every device has its own ID connected to the same user (RingID).

Every connected device will ring once you receive a call, the others will be turned off once it is accepted or declined. This multi device feature will increase the fluidity between all of your electronic devices and will allow you to communicate anytime, with any device, as you would use centralized communication systems. However, it is important to remember that Ring has an element distinguishing it: a distributed and decentralized network.

Migration is key

The multi device feature offered in the Bêta 2 version, will make it incompatible with any previous version. Once upgraded to beta 2, users will be connected to another system, therefore beta 1 and 2 users will not be able to communicate to one another.

This compatibility break requires users to migrate to the newest version, by simply logging on to Ring; the system will run the necessary updates.

Security notice

You will need to provide a user name and password during the client update.

Your account password is important as it is used to encrypt the archive that contains the private keys linked to your Ring account. Please note that your password cannot be recovered.

When adding a new device, this archive is transferred over the network. It is important to choose a strong password to minimize any risks if your archive is intercepted.

Ring Bêta 2 is now available

ring-version-beta-2-en_wp

Ring, a GNU projet, is improving. Test the Gaston Miron version now! Distributed under license GPLv3, Ring is available on GNU/Linux, Windows, Mac/OSX, Android and soon on iOS and UWP (Universal Windows Platform).
Among the latest features in the beta 2 version, you will find:

  • A global account identification mechanism based on blockchain implementation on an Ethereum technology.
  •  Multi-device
  •  IPv6 support

Compatibility break

All users are invited to go to beta 2 due to a compatability break with the older version. Ring will handle the migration very easily.
Ring as a communication platform can be used to develop various projects: videoconferencing and distributed collaboration, IoT or robotics !

An active development process

We are continuously  improving Ring, in order to offer the best of communication to its clients. The software is in the active development phase and we invite the Free Software community to take part in this audacious project. All new ideas are welcomed to help its evolution.
https//ring.cx

Ring, officially a GNU package

heckert_gnulogo-ring-tagline-en-couleur

Since the month of October, Ring is officially a GNU package, following its integration into the GNU project.

Distributed under the GPLv3+ license, Ring is a free software which enables communication in multiple ways between its users. It is a phone, a conference station, a platform for exchanges and media sharing, and a link for the Internet of Things, all in a world where freedom and security go hand in hand. Running on a distributed network and encrypted from end to end, Ring tries to provide to Ring users’ a high level of privacy and confidentiality.

In the mid 2000s, Savoir-faire Linux has started a project called SFL Phone. The project then evolved to produce another: Ring. This shift is marked by various milestones such as the transition from SIP server to the OpenDHT distributed network in 2014, permitting decentralized communication, peer to peer mode. Becoming a GNU package is a new venture for the project and the free software movement.
Thanks to external contributions, which are the cornerstone of the Ring project, the Savoir-faire Linux team has over several months developed a beta 2, which fits perfectly with the philosophy of free software. Beyond the code, Ring has established links between the development team and the local partner universities such as the Université du Québec à Montréal (UQÀM), Polytechnique Montréal and the École de Technologie Supérieure (ÉTS ) which is home to the Maison du Logiciel Libre (ML2).

We want to thank all our contributors, all of whose assistance was greatly appreciated.

https://ring.cx/

Savoir-faire Linux, is Gold sponsor at the Liferay Symposium 2016

We are very happy to announce our presence to the community, our partners and clients at the Liferay Symposium 2016 which will be held in Chicago at the end of the month.

16nas-sponsor-gold-large

Like every year we are here to make you discover our Liferay expertise and at the same time our knowledges in Java, Web, DevOps et cloud services.

Every year, about ten Liferay partners are rewarded for their implication into the community.

We are hoping that this year our work will be appreciated.

Chart Portler, last updates

We are also very proud for all the hard work we put into this new version available here : https://web.liferay.com/marketplace/-/mp/application/48919330

We can’t wait to meet you in order to discuss about new business opportunities under Liferay 7 one of the biggest question in 2015.

As we have been doing, we will introduce you to our computerised deployment tools and the upgrade possibilities related to the user experience under Liferay 7.

Savoir-faire Linux, Red Hat and Liferay

Moreover, we would like to state once again that we are a Red Hat partner and a Silver Liferay partner and our developers and administrators are certified both of them.

Finally, this event is a wonderful opportunity for you to discover our last Free Software project for universal communication, Ring.

Hoping to see many of you at the event on 26 – 27 September, we invite you to visit our Ring website and download the application on your devices.

beta-version-features-en

https://ring.cx/en/download

Ring is now on Debian

On June 30, Ring was officially accepted in Debian testing, the development repositories for the next Debian release: Stretch.

That is great news for Ring users!

It is now possible to install Ring from your distribution’s repositories without having to add the custom ring.cx repositories.

Ring at Debconf16 and availability in Debian and Ubuntu repositories

CmYA4fXXgAAyhaf

The 17th edition of the annual Debian Developers conference took place in Cape Town, South Africa.

Members of the Ring development team were on site in order to present the project and work on Ring’s integration in official Debian repositories.

Ring in Debian and Ubuntu Repositories

If you are a Debian Testing user, you may now install Ring with the following command:

– “apt-get install ring”

The arrival of Ring in Debian repositories also means that it will be available in Ubuntu.

As of July 2, Ring was integrated in the Ubuntu 16.10.

Some Debian contributors have already started testing Ring on GNU Debian/kFreeBsd. We expect a package to be available in the near future.

A request and a patch was sent by a Debian user that would allow to build Ring on Debian Jessie. If all goes well, Ring might make it to jessie-backports soon.

Presentation of Ring at Debconf16

Alexandre Viau and Simon Désaulniers made a presentation about the technology that allows for making calls in a decentralized manner in Ring. The talk also incluses a quick preview of how Ethereum will be used as a decentralized username registrar.

A recording of the talk is available on the debian-meetings archives:
http://meetings-archive.debian.net/pub/debian-meetings/2016/debconf16/Decentralized_communications_with_Ring.webm

OpenDHT in Debian experimental

OpenDHT was developed for Ring but the project remains independent and could be used for other applications. The libopendht package made it to Debian experimental during DebConf.

Up and coming iterations of the package will include the dhtnote program, which allows to run an OpenDHT node.

This program will allow the community to host OpenDHT nodes without having to run dring, the Ring daemon, which is heavier in dependencies than dhtnode.