Press Review Inno #9

PyCon 2017 (Part 2!)

Talk: “Double Click: Continue Building Better CLIs”

Summarized by Kévin Barralon

This PyCon Canada talk was a presentation of Click, a Python package that allows you to create CLIs (Command-Line Interfaces) with only a few lines of code.

Click is simple and easy to use, in that the configuration of commands happens almost entirely using decorators around your Python functions. For example, here is a simple function that can be used to repeat the word you pass it as an argument:

# hello.py
import click<span style="display: inline-block; width: 0px; overflow: hidden; line-height: 0;" data-mce-type="bookmark" class="mce_SELRES_start"></span>
@click.command()
@click.option('--count', default=1, type=int, help='Number of greetings.')
@click.option('--name', prompt='Your name',
              help='The person to greet.')
def hello(count, name):
    for _ in range(count):
        click.echo(f'Hello {name}!') # Syntaxe de Python 3.6 !

if __name__ == '__main__':
    hello()

In just a few lines of code, you’ve created a command that can:

  • take an option --count with a default value of 1 and that is type-cast to an integer.
  • take an option --name which allows the user to state their name, thanks to the ‘prompt’ argument passed in the decorator

These values are passed directly as the count and name parameters of the decorated function.

To run this command in your terminal:

python3 test.py --count=3
Your name: Casimir 
Hello Casimir!
Hello Casimir!
Hello Casimir!

The other interesting (and practical!) point here is that if you pass the option --help when you call the command, you can see the docstring for your function, as well as information on each option that your comamnd supports thanks to the argument help passed in your decorators:

python3 hello.py --help

Usage: test.py [OPTIONS]

Simple program that greets NAME for a total of COUNT times.

Options:

Count INTEGER Number of greetings
Name TEXT The person to greet
Help Show this message and exit.

Rather than simply passing options to your command, you can also specify arguments. For example:

@click.command()
@click.argument('filename')
def touch(filename):
    click.echo(filename)

Then:

python3 touch.py foo.txt
foo.txt # Résultat

As you can see, the possibilities with Click are endless. Here are a few additional ones for fun:

  • Add the argument multiple=True in your decorator @click.option which will give you a tuple of all options passed to your command.
  • Add a callback as an argument to @click.option which will change the behaviour of your command (to define custom validation rules, for example).
  • Pass environment variables to your decorator (like so: envar="USERNAME").
    Define a limited or unlimited number of arguments that will return a tuple.
  • Add a File method as a type in your decorator (example: @click.argument('input', type=click.File('rb'))) which will automatically open a specified file and close it when necessary.

A list of all possibilities offered by this Python package is available in the documentation, which is indeed very well done.

Bonus!

The closing keynote on f-strings really seemed to motivate our team! Here’s an article by Maxime which adds to the one by Sébastien from last week:

What in the World Are F-Strings?

By Maxime Dupuis

The mechanism of Literal String Interpolation, known as f-strings, is a new arrival in the latest version of Python, 3.6. They are recognizable by the f (or F) at the beginning of a declaration of a string: f'My f-string!'. They are very similar in behaviour to template strings in JavaScript, which are defined using the ` character.

F-strings are useful because they can reference variables that will be automatically interpolated with the values of the local variables defined under the same name.

>>> age = 42
>>> first_name = 'Maxime'
>>> f"My name is {first_name} and I'm {age} years old"
"My name is Maxime and I'm 42 years old"
>>> # The use of an undefined variable will give us a NameError:
>>> f"{notdefined}"
Traceback (most recent call last):
  File "<stdin>;", line 1, in <module>
NameError: name 'notdefined' is not defined

They become very interesting when you consider their impact on performance. If we compare them to other ways to create strings in Python, we have the following:

python3.6 -m timeit -c 'age=42; first_name="Maxime"; f"My name is {first_name} and I'm {age} years old"'
10000000 loops, best of 3: 0.172 usec per loop
python3.6 -m timeit -c 'age=42; first_name="Maxime"; "My name is {first_name} and I'm {age} years old".format(first_name=first_name, age=age)'
1000000 loops, best of 3: 0.714 usec per loop
python3.6 -m timeit -c 'age=42; first_name="Maxime"; "My name is %s and I'm %s years old" % (prenom, age)'
1000000 loops, best of 3: 0.283 usec per loop
python3.6 -m timeit -c 'age=42; first_name="Maxime"; "My name is " + prenom + " and I'm " + str(age) + " years old"'
1000000 loops, best of 3: 0.449 usec per loop
python3.6 -m timeit -c 'from string import Template; age=42; first_name="Maxime"; Template("My name is $first_name and I'm $age years old").substitute(first_name=first_name, age=age)'
100000 loops, best of 3: 8.75 usec per loop

But it’s not only useful for its performance benefits. We can use f-strings to call functions, modify substrings, and many other things:

>>> total = 10.0
>>> taxes = 1.15
>>> f'Total: {total + taxes}'
'Total: 11.15'
>>> first_name = 'Maxime'
>>> f'My name is {first_name.upper()}'
'My name is MAXIME'

If f-strings interest you or you haven’t yet discovered all of the new things that Python 3.6 has to offer, check out the release notes.

Mobility

SwiftGen

By Thibault Wittemberg

When we develop an iOS app, we regularly need to access resources. The traditional way to use them in the code is to instantiate them with their identifiers (strings):

let avatar = UIImage(named: ‘darthVader’) // gets the ‘darthVader’ image from the assets folder
let name = NSLocalizedString (‘darthVader’, comment: ‘’) // gets the localized string with the key ‘darthVader’ from the Localizable.strings file

The same goes for fonts, storyboards and colors.
Swift is a strict language about type management and promotes the detection of errors at compile time rather than at run time.

That being said, what happens if one of the resources we want to instantiate is not found at run time? Since this instantiation is based on a string, it is subject to errors (typo errors). Therefore, we can use the SwiftGen tool which allows to secure this kind of manipulation:

  • It avoids typo errors
  • It allows auto completion
  • It ensures that the resources exist at compile time.

It operates as follows:

  • The “SwiftGen” executable is installed on the development workstation (either via Homebrew or via CocoaPods)
  • A custom build phase launches this executable for each compilation
  • SwiftGen scans the project’s resources and generates the Swift code representing them in a “type safe” manner

The resource identifiers will be transformed into enums that are much more secure and easy to use. Consider the following example:

let avatar = Asset.darthVader.image
let name = L10n.darthVader

It is even possible to customize the templates that will be used to generate the code if you want to apply a different coding style.

When combined with the “Reusable” framework (from the same author), we have tools that are almost mandatory (and we use them a lot at Savoir-faire Linux).

SwiftGen is made to be integrated with your continuous integration environment (the executable can be embedded in a CocoaPods dependency, it avoids installing it on the build machine). Read more on GitHub!

Contributions

New Release of num2words v0.5.6 with Support for New Languages!

By Ernesto Ortiz

It has been five months since the previous release of num2words v0.5.5, and these have been five really active months. A lot of new contributors have made some great improvements and have added support for new languages! Here are some of the latest changes, with “special thanks” to the contributors!

  • ‘Num2words’ has changed its coding conventions to follow PEP8. Code coverage has increased a lot, and we need your help to continue improving this, so check out the issues related to num2words
  • Now we can use other converters like to_currency, to_year, to_ordinal_num
  • Thanks to Tufan Kaynak for adding support for the Turkish localization
  • Thanks to Jesús Leganés-Combarro for adding support for Algerian French localization
  • Thanks to Armin Oliya for adding support for the Dutch localization
  • Thanks to Sergio Zholudov for adding support for the Ukrainian localization
  • Thanks to Blaž Bregar for adding support for the Slovenian localization
  • Thanks to Mārtiņš Šulcs for making a lot of improvements to the base code, to_currency functionalities and more!

Special thanks to everyone that has taken the time to report new issues, to make pull requests and to share their time and effort with this project!

You can check the new release at the following links:

Press Review Inno #8

Special Edition on PyConCa 2017

This week, we attended PyCon Canada which started on November 18 and lasted through 21. We are also proud and happy that we played a role in sponsoring this dynamic event. In this Special Edition of Press Review Inno, you find articles from our SFLers who actively participated in the event’s sessions.

PyCon 2017

Our Team Who Attended PyConCa 2017

 

Tutorial: Writing Tests that Write Themselves by David Kua

Summarized by Ernesto Rodriguez Ortiz

Stop worrying about corner cases and start loving the power property-based testing.

I just discovered the concept of property-based testing at PyCon Canada 2017, thanks to David Kua who prepared, showed and shared with us a great tutorial of property-based testing using the library Hypothesis. (Welcome to Hypothesis!)

In fact, the concept of property-based testing was popularized by the Haskell library Quickcheck. I am happy about my new discovery because I find myself too often thinking about corner cases in my tests, and now I don’t need to spend as much time considering this, as they will show up themselves with the help of Hypothesis.

So How Does This Work?

Hypothesis generates random data matching your specification and checks that your guarantee still holds up for each test case. Check out this really simple example:

# FILE NAME division.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
 
# You wrote these two lines way too fast...
def division(x, y):
    return x / y
 
# Hypothesis will now show you your mistakes...
from hypothesis import given, settings
import hypothesis.strategies as st
 
@given(st.integers(), st.integers())
@settings(max_examples=1000)
def test_commutativity_pbt(x, y):
    assert division(x, y) * y == x

Then run:

pytest --hypothesis-show-statistics division.py

You should see how Hypothesis has found the corner case:

ZeroDivisionError: division by zero

This is powerful stuff, so go play a little with the awesome tutorial prepared by David Kua and check the documentation of Hypothesis.

Closing Keynote – Mariatta Wijaya

Summarized by Sébastien Blin

During the closing keynote, Mariatta Wijaya, a Python Core Developer, presented us Python 3.6 through the lens of a PEP (Python Enhancement Proposal) that has been released in this version: PEP 498: Literal String Interpolation.

Mariatta explained the history of this PEP, from its beginnings as a Python mailing list post since a few years ago. The new feature introduced in this PEP, colloquially called “f-strings”, allows for an improved experience in creating strings with variables in them, all the while improving performance and making for a more codebase than when the % operator or str.format functions are used.

>>> import datetime
>>> name = 'Fred'
>>> age = 50
>>> anniversary = datetime.date(1991, 10, 12)
>>> f'My name is {name}, my age next year is {age+1}, my anniversary is {anniversary:%A, %B %d, %Y}.'
'My name is Fred, my age next year is 51, my anniversary is Saturday, October 12, 1991.'
>>> f’He said his name is {name!r}.'
"He said his name is 'Fred'."

She then gave a brief overview of the other PEPs released with Python 3.6 (including PEP 628 which adds math.tau, which did not exist prior, and the value of which is 6.28). The complete list of PEPs in Python 3.6 is available at Python 3.6 Release Schedule.

In any case, Mariatta invites us to upgrade to Python 3.6 as soon as possible, as well as to contribute to the evolution of the programming language through the creation of new PEPs at this GitHub repo.

Contributions

GitHub Releases a Team-Based Discussion Feature

By Romain Bertozzi

To contribute to open source software means to share with others, so it only follows that the ability to effectively communicate is critical. GitHub announced a new feature on the 20th of November centered around team-based discussions.

The idea of this feature is to create a space for conversations within a project that might not be best placed under issues or pull requests. These issues and PRs will now be freed up and will be less of a magnet for long, irrelevant comment threads. In addition to this, these new discussions, public or private, will have dedicated URLs and will be able to be easily shared and cited on other sites.

Here at Savoir-faire Linux, we welcome this news enthusiastically. We believe that it will bring more clarity around the discussions related to the creation, management and direction of a project. The centralization and the ease-of-use of these discussions are benefits that will allow for more transparency and longevity of conversations. It remains to be seen how this functionality will be adopted by GitHub users, who already have many different options available to them that they may be more used to.

Press Review Inno #3

Web Development

PyCon 2017

By Jacob Cook

PyCon Canada is soon upon us, and the national Python conference of record will be held in Montreal this year, from November 18th to the 21st.

As a software company well-anchored in the Python and Django communities in Montreal, Savoir-faire Linux will be sending a large contingent of employees to attend. The first two days of the conference include keynotes, talks and coding tutorials which cover some of the newest and most useful developments in the Python universe and in software development more generally. On Sunday the 19th at 11:50am (Abstract here), I will be addressing the conference and giving a talk on how to quickly create Django applications that utilize the dynamic front-end power of React. With a stellar speaking lineup announced and 70% of tickets already sold, this year’s PyCon Canada promises to be the biggest yet.

Free Software Projects and Communities

#Hacktoberfest or how a dormant Free Software project is now more active than ever

By Samuel Sirois

Istvan Szalaï and I are attempting to restart the development of a Free Software project that we started last summer but is presently inactive. This project is called RingMe.js. In order to motivate ourselves a bit more to make some contributions to the project, Istvan proposed that we create some issues in the project as opportunities for contributions to Hacktoberfest by DigitalOcean. What started out as simply being a use of “gamification” to motivate ourselves to put more effort into the project became a magnet to some small contributions by passing strangers that helped solve some pressing tasks, bugs and necessary improvements that were marked with the tag #Hacktoberfest. In fact, we were surprised to find that the contributors took time to propose quality code (Check out the code here: savoirfairelinux/ringme.js).

Why has this been such a success?

A few possibilities:

  • The tasks, bugs and new features are simple, precisely-worded and not overly broad in scope (thanks to Istvan);
  • JavaScript is considered a “sexy” programming language in 2017;
  • The codebase is sufficiently young and unique to be easy to understand for newcomers, and it’s also easy to propose coherent solutions;
  • The README, CONTRIBUTING and issue template files are present and well thought-out (thanks to Emmanuel);
  • The project maintainers (Istvan and I) try to respond to contribution requests and questions within a period of 48 hours;
  • #Hacktoberfest has become a popular Web development event within the open source community.

Thanks to:

Thanks to Istvan for having come up with the idea to mark our issues with #Hacktoberfest. Thanks to Jacob for having helped us effectively evaluate some pull requests due to his knowledge of the modern JavaScript ecosystem.  And thanks once again to all contributors to the project, including (at press time):

Mobility

Secret: Psss…  Have You Heard about Android View State Storing?

Who has never before had problems or encountered weird behaviors while trying to store/restore custom views on Android ? In this article Elye will explain how the system works and how one can avoid problems.

Read the Article: A must know secret about Android View State storing mechanism!

How does Ring communicate with connected devices # IoT

light-bulbWhen you leave home, are you sure the lights are turned off? With Ring, you can check it with a video camera and adjust them simply by sending a text message through live chat. Save energy!

Within an hour, a Savoir-faire Linux expert has built a device that allows him to control the lighting in his living room, while he is at the office. As Adrien Béraud has set his Ring account to automatically accept every call, he can see his living room remotely by simply initiating a video call to his home computer. He can then check if the lights are on, and turn them off as needed.

A simple device crafted with open hardware

Ring-arduinoAdrien first connected lamps in his living room to the Internet using a small connected system Particle Core (type Arduino) and a relay expansion board (Relay Shield). As his lightning is associated with a three-way switching circuit, it can easily control lights, even in the case of Internet failure or in case of problems with the relay. He then added a few lines of code in the documentation of the relay to connect it the Internet.

With some command line, your lights are connected

Once connected, he types a command line in the terminal of his Linux computer. For example, to turn off (LOW) the first lamp (r1):

curl https://api.particle.io/v1/devices/0123456789abcdef/relay -d access_token=123412341234 -d params=r1,LOW

Light control with Ring’s chat can then be done with this Python script:

#!/usr/bin/env python3
import re, json, http.client, urllib.parse
from controler import DRingCtrl

PARTICLE_API_TOKEN = '[your_api_token_here]'
PARTICLE_REQ_URI = '/v1/devices/[your_device_id_here]/relay'
PARTICLE_REQ_RE = re.compile('^(r[0-4],(?:HIGH|LOW))$')

class LightRingController(DRingCtrl):
    def __init__(self):
        super().__init__("LightController")

    def tryParticleReq(self, params):
        try:
            conn = http.client.HTTPSConnection('api.particle.io')
            p = urllib.parse.urlencode({'access_token': PARTICLE_API_TOKEN, 'params': params})
            conn.request("POST", PARTICLE_REQ_URI, p, {"Content-type": "application/x-www-form-urlencoded", "Accept": "text/plain"})
            ret = json.loads(conn.getresponse().read().decode())['return_value']
        finally:
            conn.close()
        return ret

    def onIncomingMessage(self, callid, ufrom, message):
        super().onIncomingMessage(callid, ufrom, message)
        msg = message['text/plain']
        res = re.search(PARTICLE_REQ_RE, msg)
        if res:
            op = res.group(0)
            print('Command:', op)
            ret = self.tryParticleReq(op)
            self.callmanager.sendTextMessage(callid, {'text/plain':'Got: '+str(ret)}, False)

if __name__ == "__main__":
    ctrl = LightRingController()
    ctrl.run()

IoT: endless possibilities

Many uses are possible, from simple to complex. For example, a Ring call can directly control lighting with such a script:

#...

class LightRingController(DRingCtrl):

    #...

    def onCallStateChanged(self, callid, state, statecode):
        super().onCallStateChanged(callid, state, statecode)
        if state == "HUNGUP":
            self.tryParticleReq('r1,LOW')
        elif state == "CURRENT":
            self.tryParticleReq('r1,HIGH')

So imagine what you could do if your Ring software was connected to your heating or your garage door… The possibilities are endless!

You have ideas to share? Feel free to comment!

Logic analyzer: visualizing latency between two digital signals in real time with sigrok and matplotlib

Logic Analyzer Salae In a previous post I described how to manipulate data extracted from a Saleae logic analyzer with Python, to view the latency between two digital signals and deduce the temporal jitter of one of them. In conclusion, I expressed the assumption that the Saleae SDK could allow us to retrieve information about the signals in real time, rather than via an intermediary CSV file. But at the end, the best solution I found came from a more generic free software.

The Saleae SDK solution

Saleae provides developers with two solutions:

  • An API (in beta at the moment) that only allows to control their visualization software from a socket;
  • A SDK that allows expanding their software and write your own protocol analyzer plugins.

We could have looked at the SDK and written a parser exporting the latency between two defined signals on the fly, but this solution is dependent on the Saleae software and its evolutions, ensuring very low durability. In addition, Saleae’s visualization software is not intended to be used with different types of logic analyzers. We were looking for a more generic and permanent solution.

Sigrok: an open-source software suite for logic analyzers

It was during a discussion with my colleagues that Emeric Vigier pointed to me a software called sigrok. Aiming to provide a free and generic logic analyzer, sigrok is a software suite for extracting data collected by various types of analyzers and displaying them or analyzing them using protocol decoder plugins.

This suite consists of several sub-projects:

  • libsigrok: a library written in C that standardizes access to different test and measurement devices.
  • libsigrokdecode: a C library that provides an API for protocol decoding. The decoders are written in Python 3 and later.
  • sigrok-cli: a command line interface to manipulate sigrok.
  • PulseView: a Qt GUI to manipulate sigrok.
  • Fx2lafw: sigrok also provides an open source implementation of the Cypress FX2 chip firmware, which is used — among others — by Saleae in all versions of its logic analyzers except the logic Pro 16. This firmware can program the embedded logic to function as a single logic analyzer hardware.

Sigrok usage example

An end-to-end example will be better than a thousand words. We will use PulseView to capture and visualize signals from a logic analyzer.

At startup, PulseView in its version 0.2.0 looks like this:

pulseview01

We’ll start a capture leaving the demonstration device generate random signals:

pulseview02

We will then add a pretty simple decoder that calculates the duty cycle of a digital signal:

pulseview03

Each decoder has its options. In our case we simply set the channel on which we want to apply the decoder:

pulseview04

The duty cycle calculation is applied to the channel and displayed directly into PulseView:

pulseview05

Sigrok: writing a decoder for determining the latency between two digital signals

The following is based on the libsigrok and libsigrokdecode in their version 0.3.0.
Each protocol decoder is a Python module that has its own subdirectory in the libsigrokdecode decoders directory.

A decoder consists of two files:

  • __ init __ py . This file is required to initialize the decoder and contains a simple description of the protocol
  • pd.py : This file contains metadata about the decoder, and its implementation.

We’ll look at the file pd.py that contains the code for our decoder. The libsigrokdecode API provides all the information on the captured signals.

To calculate the latency between two digital signals, our logic is to consider one signal as a reference clock, and the other as a result of it. The two signals are linked so each state transition of the clock signal will result in the other signal to change also transition with a more or less variable latency. We will discuss the fluctuation of this latency later.

The decoder options

The first thing to do is to allow our channel decoder to define which signal corresponds to the reference clock, and which channel corresponds to the resulting signal. The decoder class precisely allows to define some attributes specifying its options, including one that allows the user to select channels:

class Decoder(srd.Decoder):
   # ...
   channels = (
      {'id': 'clk', 'name': 'Clock', 'desc': 'Clock reference channel'},
      {'id': 'sig', 'name': 'Resulting signal', 'desc': 'Resulting signal controlled by the clock'},
   )
   ...

The decode() method

The second step will consist in implementing the contents of the decode(). This method is called by the libsigrokdecode every time a new block of data is available for being processed.

Each block actually depends on the sampling rate of our signal. For example, sampling at 100Hz value, we get 100 blocks of data per second. The data blocks contain the sequence number of the current sample, as well as the status of various signals at this time.

From these informations, it is very easy to implement a state machine that will note how the sample transition occurred between the clock signal and the resulting signal. The number of samples elapsed between the two events multiplied by the sampling rate, gives us the latency in seconds.

In its simplified version, our state machine looks like this:

def decode(self, ss, es, data):

    self.oldpin, (clk, sig) = pins, pins

    # State machine:
    # For each sample we can move 2 steps forward in the state machine.
    while True:
        # Clock state has the lead.
        if self.state == 'CLK':
            if self.clk_start == self.samplenum:
                # Clock transition already treated.
                # We have done everything we can with this sample.
                break
            else:
                if self.clk_edge(self.oldclk, clk) is True:
                    # Clock edge found.
                    # We note the sample and move to the next state.
                    self.clk_start = self.samplenum
                    self.state = 'SIG'

        if self.state == 'SIG':
            if self.sig_start == self.samplenum:
                # Signal transition already treated.
                # We have done everything we can with this sample.
                break
            else:
                if self.sig_edge(self.oldsig, sig) is True:
                    # Signal edge found.
                    # We note the sample, calculate the latency
                    # and move to the next state.
                    self.sig_start = self.samplenum
                    self.state = 'CLK'
                    # Calculate and report the latency.
                    self.putx((self.sig_start - self.clk_start) / self.samplerate)

    # Save current CLK/SIG values for the next round.
    self.oldclk, self.oldsig = clk, sig

The outputs of our decoder

The libsigrokdecode library offers different ways of feeding information back to our decoder. A register() method allows to record the output data our decoder generates. Each output has a defined type; for example, type OUTPUT_ANN is used to define an annotation type output that will be represented in PulseView by graphic widgets that we saw previously in the duty cycle decoder example.

In our example decoder, we mainly use two types of outputs:

  • An annotation type output (OUTPUT_ANN) to visualize latency informations in PulseView
  • A binary type output (OUTPUT_BINARY) to output raw latency values for later analysis.

The final version of our decoder also includes a metadata output (OUTPUT_META) reporting statistics on missed signals. But that’s not relevant here.

Both outputs are defined as follows in our decoder:

self.out_ann = self.register(srd.OUTPUT_ANN)
self.out_bin = self.register(srd.OUTPUT_BINARY)

To report information about these outputs we use the put () method, which has the prototype:

put(start_sample, end_sample, output_type, data)

where output_type is one of previously defined methods (out_ann, out_bin).

To report the latency between two signals using standard annotations, for example, let’s write:

put(clk_start, sig_start, out_ann, [0, [my_latency]])

Results obtained with our latency decoder

Here is an example of the results obtained with our decoder in PulseView:
pulseview06

And the results obtained with sigrok-cli in its version 0.5.0 — including the binary output for raw latency values:
sigrok-cli

Viewing real-time data to determine the temporal jitter

As we have seen, sigrok-cli allows us to get the raw latency values on the console in real time. We will now turn our attention to variations of this latency. Indeed, there is no guarantee that the latency between these two signals is constant over the time.

Considering that the clock signal is periodic and without fluctuation, in theory, the resulting signal should be good and the latency between these two signals should be constant. If this is not the case, it is because there is a fluctuation of latency that we have described in the previous section as the temporal jitter of a signal, and this is what we see now.

We can imagine retrieving those values on the fly and displaying them in a graph. To do this, we would once again write a Python script.

I had a look at the matplotlib library and its animation module which provides a FuncAnimation() feature, to define a function that we’ll call in order to update the chart whenever new data is available. This function takes parameters such as the figure we are working on, the animation function to call, and the data set to be processed.

anim = animation.FuncAnimation(fig, animate, frames=data_gen)

These data can be in the form of a Python generator that will pair well with the reading of a data stream (thank you Guillaume Roguez for presenting me this object type).

So every time a new latency value is written to the stream, our generator retrieves new data and the animation function is called.

The code looks like this:

# The data generator take its
# input from file or stdin
def data_gen():
    while True:
        line = fd.readline().strip()
        if line:
            yield line

Our animation function will, if necessary, update the abscissa to view all latencies, and add the new value supplied by the generator.

# The update graph function
def animate(data):
    global x_max, x_min, x, y

    try:
        # we must recalculate the abscissa range
        x_new = float(data)
        if x_max is None or x_new &gt; x_max:
            x_max = x_new
        if x_min is None or x_new &lt; x_min:
            x_min = x_new
        ax.set_xlim(x_min, x_max)

        # add the new plot coordinate
        x.append(x_new)
        y.append(0)
        line.set_data(x, y)

        return line,

    except KeyboardInterrupt:
        print(&quot;leaving...&quot;)

Finally, we just have to execute sigrok-cli with our latency decoder and retrieve these values in our visualization script.

sigrok-cli -d fx2lafw --config samplerate=24MHz --samples 2M -C 1,2 -P jitter:clk=1:sig=2 -B jitter | rt-draw.py

This gives us the following final result, which allows to visualize the resulting signal jitter:

rt_draw

Notes

  • The complete code is available on Github. It is intended to provide a simple example of visual rendering and can definitely be optimized.
  • The latency decoder was submitted to the sigrok project developers and it is available upstream
  • The Cypress FX2 chip firmware sometimes has trouble with high sampling rates and crashes.

Thanks to Jérôme Oufella and Uwe Hermann for their help with the english translation!

Modernize your deployment practices with Vagrant

Vagrant LogoA recurring problem in the world of web development is the complexity associated with setting up a suitable environment for each new project. If we’re lucky, we have up-to-date textual information to perform that setup. Even in that shiny world where we’re lucky, the time required to follow those textual instructions vary depending on the technical skills of the developer. Even when those instructions are up-to-date, it’s not rare at all to routinely spend many hours trying to set this new environment up.

Here at SFL, we recently started integrating Vagrant to our projects in order to modernize our deployment methods and results so far are very encouraging.

What is Vagrant?

Vagrant is a command line library allowing to quickly create and provision virtual environments through VirtualBox. It allows, with one vagrant up command, to create a VM from scratch and automatically configure it to host your project. For the configuration part, it’s integrated with Puppet, which is good for us because that’s already what we used here at SFL to manage our machines.

Benefits

Environment setup speed. What used to be a more or less relevant set of textual instructions is now executable Puppet code. This means that a new developer being brought to a project has one single command to type:

fab deploy

and within minutes, the site is ready to be used locally.

Setup reliability. Gone are the times of textual instructions that fall into obsolescence and make new developers want to tear all their hair off (well, the hair tearing part is not entirely gone, but at least it’s not over environment setup anymore). If the Puppet code becomes incorrect over time, it’s very easy to spot immediately and correct it.

Homogenous deployment environments. There used to be a time when we developers had to describe to our sysadmins the production environment that we needed for a new project, and then they would do their best to understand what we meant and build it. Now, with Puppet, developers and sysadmins speak the same language. The same code that is used to build a local environment can be reliably used by sysadmins to replicate that environment elsewhere.

How to use it?

How you use Vagrant depends on the type of project you use it with. For example, our Django projects use a mix of Vagrant and Fabric to perform setup and deployment. The Puppet side takes care of “OS specific” setup (installing packages, configuring Apache, creating databases) and Fabric takes care of “Python code specific” setup (pushing the code to the environment, pip dependencies, South migrations).

I’ve published, on Github, a sample Vagrant-enabled Django project that looks a lot like what we do internally so you can easily experience the potential of that solution. If you have Git, VirtualBox, Vagrant and Fabric installed on your machine, you can run this project (from nothing) with these commands:

git clone https://github.com/savoirfairelinux/django-vagrant-demo
cd django-vagrant-demo/deploy
fab deploy

Then, you can visit http://demo-django.local:8080 and see the newly-deployed site.

The deploy script will ask for your sudo password because Vagrant and the Fabric script need to modify your /etc/hosts file to make demo-django.local to 127.0.0.1.