Building a Burst FSK Modem in GNU Radio with Message Lambda Blocks and Eventstream

Lots of cheap electronics tend to use burst FSK modems for wireless communications. GNU Radio has long been able to work with these sorts of communications, but typically it has done so by running an FSK demodulator continuously and then correlating the output for a known preamble sequence such as a with the correlate access code block, and then adding some kind of monolithic special purpose block. This article proposes a slightly different approach to building such a burst FSK waveform to inter-operate with many wireless devices of this style and to be a bit more modular, flexible, simple and intuitive to work with.

Building the Transmitter

In most communications systems, the transmitter is computationally less demanding than the receiver, additionally many FSK burst modems are low baud rate and not particularly demanding from a computational complexity standpoint. I use these two facts as motivation to justify an exceedingly lazy modem design which boils a burst FSK transmitter block down to three simple message lambda blocks as shown below.

fsk_tx.grc

Since they aren’t visible in the screenshot above, the block expression used for the three message lambda blocks are:

Map [0,1] bits to symbols [-1.0,+1.0]
 * lambda x: numpy.array(x, dtype=numpy.float32)*2-1
Interp to Samps per Sym
 * lambda x: numpy.tile(x,[sps,1]).T.reshape([1,len(x)*sps])
FSK Modulate
 * lambda x: numpy.array(numpy.exp(1j*2*numpy.pi*((dev*x*numpy.arange(len(x)))/samp_rate)),dtype="complex64")

We generate random packets of bits, prepend an arbitrary known preamble, map bits to symbols, interpolate to N samples per symbol, and finally modulate our symbols up by a complex sinusoid at plus or minus some FSK deviation frequency. At this point, the modulated bursts can simply be dropped into a transmit sample stream using an eventstream source block whose output is then throttled and run through a simple channel model.

burst_tx_plots

Plotting our FSK burst transmit waveform’s output through the pulse shaping filter, we get a reasonable looking time and frequency profile of the transmitted output signal. This is kind of a fun example to show the power of lamda blocks, in this case we’ve been able to basically write an entire waveform from a couple python expressions all from within GRC with really minimal effort, for a nominal performance penalty, which in this case doesn’t really matter.   If performance was at some point deemed to be important here, blocks could easily be ported one by one to optimized C++ based equivalents.

Building the Receiver

We can take a similar approach to building a burst FSK receiver waveform with stream blocks, message blocks, and event detection. We simply run the received samples through a standard quadrature demodulator block matched roughly to the FSK deviation magnitude. Since this FLL is essentially a random walk when no signal is on the air, and nominally tracking carrier tightly when locked to a burst, we use the variance of the FLL’s output as a detection statistic for bursts, triggering an event when the variance goes below 15 in this case. (or the negative variance rises above -15 in this case).   This can be seen in the graph below.

fsk_rx.grc

In this case at every event we latch in a maximum burst length’s worth of FM demodulated float values containing one burst every time this threshold is reached using the trigger rising edge and eventstream sink blocks. These FM demodulated samples are run through a burst FSK time sync, which performs symbol timing recovery and outputs one soft float value per symbol. Please note this synchronization block is by no means an optimal implementation, and represents only a very quick stab at the problem – but seems to perform well enough in initial test. (I would love to clean this up with more optimal ML synchronization algorithms but haven’t had a chance yet).   The output bits can then be sliced to bits, packed into 8 bit chars, and passed in an output PDU downstream to feed a PDU Socket or TUNTAP interface. A screenshot of the receive waveform is shown below.

burst_rx_plots

Our diagnostic output on the terminal shows that we are indeed recovering the inserted preamble bits correctly (0x036cf is consistently received in our output) Although sometimes with alignment slipping a bit or two in either direction and adjacent noise and data bits flipping randomly – A problem that could be easily taken care of with a MAC header search and framer.

creating event @ time 368183, length = 5000
sps = 8.000000
* MESSAGE DEBUG PRINT PDU VERBOSE *
((tau . 5) (meta . 8) (es::event_length . 5000) (es::event_time . 368183) (es::event_type . edge_event))
pdu_length = 78
contents = 
0000: 40 36 cf 02 c5 d2 b7 48 0c bd 22 44 5c d1 84 c3 
0010: 0b 2e 49 2f 29 7b 96 dd 74 61 88 82 83 47 db 0b 
0020: 2a 40 20 55 81 7d b4 29 36 0e 1d 9e 65 86 4b af 
0030: 3e b9 b1 65 b0 f9 a1 4a be f2 54 1a b2 41 bf 02 
0040: 94 a9 c3 44 3d b2 bb b6 db 6d 2b e4 b4 67 
***********************************
creating event @ time 421379, length = 5000
sps = 8.000000
* MESSAGE DEBUG PRINT PDU VERBOSE *
((tau . 3) (meta . 8) (es::event_length . 5000) (es::event_time . 421379) (es::event_type . edge_event))
pdu_length = 78
contents = 
0000: 00 36 cf 0a 59 29 cb 7b 83 67 5f ac 29 c7 87 d7 
0010: 2e 63 d4 c4 3e ff 68 d2 01 56 cc ba 09 aa 33 16 
0020: 00 7a c7 54 57 12 e3 7b 5a 72 d0 4c 80 aa 44 f1 
0030: cf 77 16 28 65 31 a4 11 f3 12 07 ef e0 42 cc 47 
0040: 57 90 be aa 15 02 89 19 91 f6 c9 83 8a dc 
***********************************

This waveform is available on github for your enjoyment.  Please send pull requests for any improvements you make!

30 thoughts on “Building a Burst FSK Modem in GNU Radio with Message Lambda Blocks and Eventstream

  1. Any chance you could put the the Burst FSK Time Sync, Burst Binary Slicer, and Async PDU Packer source somewhere on github? (just the block code would be fine, doesn’t have to be an entire OOT module)

    If not, is the current “best” way to create something similar doing a copy/paste from work() from the equivalent stream blocks (clock_recovery_mm, binary_slicer, pack_k_bits) in to a clean PDU handler block? Just didn’t want to miss out on any useful tricks for turning stream blocks in to PDU blocks.

    1. I cloned your gr-fsk-burst and gr-psk-burst. I also and added all the dependencies and so I am missing a bunch of es blocks in the gnuradio flowgraph. Where can I find the following blocks?
      es_trigger_edge_f
      es_sink
      es_handler_pdu
      es_source

      I also am having some issues with the following Property in psk_burst_ldpc_tx.grc:
      LDPC Encoder Definition – ldpc_enc

      1. these blocks come from gr-eventstream

        what kind of issues are you having with FEC? not sure if something there may have been changed recently – since these aren’t widely used people haven’t been as sensitive to API breakage upstream …

        1. Tim,
          Here is some of the Ubuntu 14.04 running on Virtualbox terminal screen outputs when I run gnuradio_companion:

          <<>>

          Preferences file: /home/craig/.gnuradio/grc.conf
          Block paths:
          /usr/local/share/gnuradio/grc/blocks
          /home/craig/.grc_gnuradio

          Loading: “/home/craig/gr-fsk-burst/fsk_tx.grc”
          >>> Error: Block key “es_source” not found
          >>> Done

          Showing: “/home/craig/gr-fsk-burst/fsk_tx.grc”

          Loading: “/home/craig/gr-fsk-burst/fsk_rx.grc”
          >>> Error: Block key “es_sink” not found
          >>> Error: Block key “es_trigger_edge_f” not found
          >>> Error: Block key “es_handler_pdu” not found
          >>> Done

          Loading: “/home/craig/gr-psk-burst/psk_burst_tx.grc”
          >>> Error: Block key “es_source” not found
          >>> Done

          Loading: “/home/craig/gr-psk-burst/psk_burst_rx.grc”
          >>> Error: Block key “es_sink” not found
          >>> Error: Block key “es_handler_pdu” not found
          >>> Error: Block key “es_trigger_edge_f” not found
          >>> Done

          Loading: “/home/craig/gr-psk-burst/psk_burst_ldpc_tx.grc”
          >>> Error: Block key “es_source” not found
          >>> Done

          Loading: “/home/craig/gr-psk-burst/psk_burst_ldpc_rx.grc”
          >>> Error: Block key “es_sink” not found
          >>> Error: Block key “es_handler_pdu” not found
          >>> Error: Block key “es_trigger_edge_f” not found
          >>> Done

          1. Tim,
            Thanks for your feedback. When I built gr-eventstream, I had to first install log4cpp. After that it built correctly, and now no more errors. But when I built and ran it in gnuradio, I got the following errors:
            <<>>

            Preferences file: /home/craig/.gnuradio/grc.conf
            Block paths:
            /usr/local/share/gnuradio/grc/blocks
            /home/craig/.grc_gnuradio

            Showing: “”

            Loading: “/home/craig/gr-fsk-burst/fsk_tx.grc”
            >>> Done

            Showing: “/home/craig/gr-fsk-burst/fsk_tx.grc”

            Generating: ‘/home/craig/gr-fsk-burst/fsk_tx.py’

            Generating: ‘/home/craig/gr-fsk-burst/fsk_tx.py’

            Executing: /usr/bin/python2 -u /home/craig/gr-fsk-burst/fsk_tx.py

            Traceback (most recent call last):
            File “/home/craig/gr-fsk-burst/fsk_tx.py”, line 30, in
            import burst
            File “/usr/local/lib/python2.7/dist-packages/burst/__init__.py”, line 51, in
            from unpacker import *
            File “/usr/local/lib/python2.7/dist-packages/burst/unpacker.py”, line 6, in
            import bitarray, array
            ImportError: No module named bitarray

            >>> Done (return code 1)

          2. Tim,
            As I keep unwrapping the onion, the next error I am getting is
            Traceback (most recent call last):
            File “/home/craig/gr-fsk-burst/fsk_tx.py”, line 34, in
            import pyqt
            File “/usr/local/lib/python2.7/dist-packages/pyqt/__init__.py”, line 52, in
            from plotter_base import *;
            File “/usr/local/lib/python2.7/dist-packages/pyqt/plotter_base.py”, line 27, in
            import PyQt4.Qwt5 as Qwt
            ImportError: No module named Qwt5

            The platform I am running is Virtualbox running on a Windows 7 laptop. The virtual machine is running Ubuntu 14.04. From the research I have done so far, I am worried about installing Qwt5 because it appears that Qwt5 wants to run python 3. I am running python 2.7.6.

            Do you have a link I could go to that gives good instructions on how to install Qwt5 with above mentioned platform?

            Thanks,
            Craig

          3. Tim,
            That is good advice. There is another person in our group who has gotten pybombs to install correctly and it may be a few more weeks before I try installing it on my machine. Would it be better for me to wait until then or is there any way that I can temporarily get around this issue with a version of your flowgraph that uses Qwt4?
            Craig

          4. You should be able to install all of that from apt-get. Usually python 3 installs in parallel without effecting python 2 afaik. This fllowgraph shouldn’t be specific to one version, you can always remove the plotting widgets, also the main gr-qtgui plotters now support messages based plotting so they can be used instead of the gr-pyqt plotters

          5. Tim,
            So which way should I temporarily go until I get pybombs to install everything from the beginning? Should I try installing python 3 or change the widgets?
            Craig

          6. Tim,
            Good advice. Today I will change the message blocks until I get everything installed with pybombs.

            Craig

          7. Tim,
            I am so sorry for peppering you with all these questions and issues. I am hoping that once I get one of your flowgraphs working, that the rest will be easy.
            1. I removed your PDS Message Lambda Blocks which also resulted in removing the PyQT Real, Complex, and Complex Spectrum Plot. I connected the Burst Preamble Insert directly to the EventStream Source. Then I ran it and it worked without any errors. The FSK QT GUI Transmitter time and frequency sink ran, but there wasn’t really anything to see.
            2. The file sink should have produced a /tmp/fskburst.dat file but I couldn’t find one. What is the location of this tmp file? Should it be below the directory that I ran gnuradio-companion from? If so, it didn’t produce one.
            3. Am I able to create my own or modify your existing PDS Message Lambda Block, PyQT Real, Complex, and Complex Spectrum Plot Block using PyQT 4 from the available libraries so it would run the same?

            Craig

          8. – /tmp/fskburst.dat is a complete path, this is the file that should be created exactly
            – You should be able to use the Lambda block without any PyQT4 installed — there may be a python import or two that would need to be commented out for this to work.
            – Again I think the shortest path would just be to get all the dependencies installed properly, which pybombs will do for you.

          9. Tim,
            I agree completely. I am working on getting pybombs to install and work in my Windows 7/Virtualbox/Ubuntu 14.04 setup.
            Craig

          10. Tim,
            After taking your advice and installing PyBombs, your fsk flowgraphs are working without error! I am so impressed with what you created. Thank you so much for your patience!
            Craig

          11. Tim,
            I ran the psk_burst_rx and psk_burst_tx without errors. When I tried to run the psk_burst_ldpc_rx and tx.grc, I got errors. Am I supposed to be able to run these?
            Craig

  2. Thanks for demonstrating Python Lambdas and explaining burst FSK!
    I noticed a SQRT raised-cosine filter in the transmitter grc. Does the receiver’s quadrature demod include a SQRT raised cosine filter, or was this left out because of using a tmp file?

  3. Hi Tim!

    Thanks for publishing write-ups like this that explain the reasoning behind the things you do, rather than just publishing the examples themselves. I have found your talks from Cyberspectrum and GRCON15 helpful/informative as well.

    Do you have any idea why the GRC might complain about the connection between the bits_to_symbols and fsk_modulator? I have no missing connections or missing blocks, and it will generate code. But upon execution it reports:

    > Using Volk machine: avx2_64_mmx_orc
    > thread[thread-per-block[1]: ]: pmt_dict_keys: wrong_type : #
    > Could not find port: pdus in:
    > pdus
    > system

    > File “/root/git-repos/gr-fsk-burst/fsk_rx.py”, line 196, in __init__
    > self.msg_connect((self.blocks_pdu_remove_0, ‘pdus’), (self.burst_fsk_time_sync_0, ‘pdus’))

    > RuntimeError: invalid msg port in connect() or disconnect()

    I’m using the same lambda expressions you are, and I have the latest gr-burst and gr-fsk-burst. I’m running GRC 3.7.9rc1.

    Thanks for any ideas you might have!

  4. Hi Michael,
    Thanks for the feedback, I actually just went and ran a test and observed the same error you were seeing. However, upon updating to the latest gnuradio master branch, rebuilding and then updating/recompiling gr-burst, gr-eventstream, everything seems to be working happily again.
    I’m not sure whether this was an ABI incompatibility, or an actual message port bug in an earlier version, but everything does seem to be happy and working on gnuradio rev “161147d5e4d0e5951c6b6e3389353298a3eae05a”.
    I would suggest trying to do a clean build of this for now!
    Cheers,
    Tim

  5. Tim,
    Is it very easy to change the length of the fsk or psk burst? I would like to shorten it. Right now it is sending and receiving
    0000: f0 1b 67 81 92 ab 6a 18 57 77 5b 2f db 75 42 2a
    0010: 32 73 3b 49 5e a9 48 91 83 32 d9 a5 97 56 de 12
    0020: 70 8b 31 79 15 de 2d f5 5f b6 a8 33 fa f7 d9 0c
    0030: 92 56 01 89 42 4c 21 04 b8 60 21 6a 2f e3 75 cb
    0040: a7 d8 80 57 df 78 0d 4b 8a 8c 74 5b 3b 02
    I was going to start studying the code to see if I could just send the first eight bits (f0).
    Craig

  6. Tim,
    I would like to modify your FSK burst and PSK burst flowgraphs to send data either through two B210’s (cabled together) and/or an OSMOCOM RTL2032U (wireless) USB dongle. How difficult would it be to change from file source/sink to actual usrps? I tried a quick test using B210’s and it didn’t lock.

    Are your FSK and PSK algorithms Coherent? I am hoping that if the transmission is interrupted or interfered with, that the receiver will lock onto the signal once the transmission is resumed.

    Thanks,
    Craig

  7. Tim,
    I was able to get your FSK burst to run on a pair of B210’s through a cable, but I am struggling to get the PSK burst to run on USRPs instead of file sink and source. Is there any additional insight that you might be able to give me?
    Thanks,
    Craig

  8. I got this error while building ( In make step ) the module ( gr-burst ) but i do not know how to solve it . ?

    gr-burst/lib/length_detect_c_impl.cc:105:67: error: cannot convert ‘uint16_t* {aka short unsigned int*}’ to ‘unsigned int*’ in argument passing
    volk_32f_index_max_16u(&max_idx, &magavg[0], magavg.size());
    ^

Leave a Reply

Your email address will not be published. Required fields are marked *