Building a Burst FSK Modem in GNU Radio with Message Lambda Blocks and Eventstream

Lots of cheap electronics tend to use burst FSK modems for wireless communications. GNU Radio has long been able to work with these sorts of communications, but typically it has done so by running an FSK demodulator continuously and then correlating the output for a known preamble sequence such as a with the correlate access code block, and then adding some kind of monolithic special purpose block. This article proposes a slightly different approach to building such a burst FSK waveform to inter-operate with many wireless devices of this style and to be a bit more modular, flexible, simple and intuitive to work with.

Building the Transmitter

In most communications systems, the transmitter is computationally less demanding than the receiver, additionally many FSK burst modems are low baud rate and not particularly demanding from a computational complexity standpoint. I use these two facts as motivation to justify an exceedingly lazy modem design which boils a burst FSK transmitter block down to three simple message lambda blocks as shown below.

fsk_tx.grc

Since they aren’t visible in the screenshot above, the block expression used for the three message lambda blocks are:

Map [0,1] bits to symbols [-1.0,+1.0]
 * lambda x: numpy.array(x, dtype=numpy.float32)*2-1
Interp to Samps per Sym
 * lambda x: numpy.tile(x,[sps,1]).T.reshape([1,len(x)*sps])
FSK Modulate
 * lambda x: numpy.array(numpy.exp(1j*2*numpy.pi*((dev*x*numpy.arange(len(x)))/samp_rate)),dtype="complex64")

We generate random packets of bits, prepend an arbitrary known preamble, map bits to symbols, interpolate to N samples per symbol, and finally modulate our symbols up by a complex sinusoid at plus or minus some FSK deviation frequency. At this point, the modulated bursts can simply be dropped into a transmit sample stream using an eventstream source block whose output is then throttled and run through a simple channel model.

burst_tx_plots

Plotting our FSK burst transmit waveform’s output through the pulse shaping filter, we get a reasonable looking time and frequency profile of the transmitted output signal. This is kind of a fun example to show the power of lamda blocks, in this case we’ve been able to basically write an entire waveform from a couple python expressions all from within GRC with really minimal effort, for a nominal performance penalty, which in this case doesn’t really matter.   If performance was at some point deemed to be important here, blocks could easily be ported one by one to optimized C++ based equivalents.

Building the Receiver

We can take a similar approach to building a burst FSK receiver waveform with stream blocks, message blocks, and event detection. We simply run the received samples through a standard quadrature demodulator block matched roughly to the FSK deviation magnitude. Since this FLL is essentially a random walk when no signal is on the air, and nominally tracking carrier tightly when locked to a burst, we use the variance of the FLL’s output as a detection statistic for bursts, triggering an event when the variance goes below 15 in this case. (or the negative variance rises above -15 in this case).   This can be seen in the graph below.

fsk_rx.grc

In this case at every event we latch in a maximum burst length’s worth of FM demodulated float values containing one burst every time this threshold is reached using the trigger rising edge and eventstream sink blocks. These FM demodulated samples are run through a burst FSK time sync, which performs symbol timing recovery and outputs one soft float value per symbol. Please note this synchronization block is by no means an optimal implementation, and represents only a very quick stab at the problem – but seems to perform well enough in initial test. (I would love to clean this up with more optimal ML synchronization algorithms but haven’t had a chance yet).   The output bits can then be sliced to bits, packed into 8 bit chars, and passed in an output PDU downstream to feed a PDU Socket or TUNTAP interface. A screenshot of the receive waveform is shown below.

burst_rx_plots

Our diagnostic output on the terminal shows that we are indeed recovering the inserted preamble bits correctly (0x036cf is consistently received in our output) Although sometimes with alignment slipping a bit or two in either direction and adjacent noise and data bits flipping randomly – A problem that could be easily taken care of with a MAC header search and framer.

creating event @ time 368183, length = 5000
sps = 8.000000
* MESSAGE DEBUG PRINT PDU VERBOSE *
((tau . 5) (meta . 8) (es::event_length . 5000) (es::event_time . 368183) (es::event_type . edge_event))
pdu_length = 78
contents = 
0000: 40 36 cf 02 c5 d2 b7 48 0c bd 22 44 5c d1 84 c3 
0010: 0b 2e 49 2f 29 7b 96 dd 74 61 88 82 83 47 db 0b 
0020: 2a 40 20 55 81 7d b4 29 36 0e 1d 9e 65 86 4b af 
0030: 3e b9 b1 65 b0 f9 a1 4a be f2 54 1a b2 41 bf 02 
0040: 94 a9 c3 44 3d b2 bb b6 db 6d 2b e4 b4 67 
***********************************
creating event @ time 421379, length = 5000
sps = 8.000000
* MESSAGE DEBUG PRINT PDU VERBOSE *
((tau . 3) (meta . 8) (es::event_length . 5000) (es::event_time . 421379) (es::event_type . edge_event))
pdu_length = 78
contents = 
0000: 00 36 cf 0a 59 29 cb 7b 83 67 5f ac 29 c7 87 d7 
0010: 2e 63 d4 c4 3e ff 68 d2 01 56 cc ba 09 aa 33 16 
0020: 00 7a c7 54 57 12 e3 7b 5a 72 d0 4c 80 aa 44 f1 
0030: cf 77 16 28 65 31 a4 11 f3 12 07 ef e0 42 cc 47 
0040: 57 90 be aa 15 02 89 19 91 f6 c9 83 8a dc 
***********************************

This waveform is available on github for your enjoyment.  Please send pull requests for any improvements you make!

Burst PSK Modem with LDPC Coding in GNU Radio using Message Passing and Eventstream

Bursty wireless communications are widely used as a multi-user channel access sharing technique in the real world.   This has long been a frustration in GNU Radio as it has traditionally focused heavily on continuous stream processing blocks and applications.   More recently the message passing additions to GNU Radio and gr-eventstream’s burst schedulers have made building burst radio transmission and reception much simpler and more logical to implement than was possible before.   In this post we detail a demonstration of a burst QPSK modem with LDPC coding and a minimal packet header which is implemented in C++ and Python blocks, and can be put together and run easily in the GNU Radio Companion tool (GRC).

Transmit Waveform

The transmitter waveform is comprised of two processing domains which are joined by a an eventstream source block.   The first is a message passing framework which uses PDUs to pass around discrete “bursts” in various forms, and the second which is a traditional GNU Radio stream processing graph.   The eventstream source block provides the insertion of a discrete length “burst” of samples into the continuous, unbounded, stream of samples.

Transmit Waveform: Message Domain

In the message domain we start off with a “Message Strobe” block, this generates a PDU periodically ever 1000ms which triggers the creation of a random data PDU corresponding to each of these.   These random data PDUs are simply a vector of bits with random length and an empty dictionary which could hold information about the burst.

In reality an applicaiton might implement a higher layer MAC here, or simply use a TUNTAP or SOCKET block to allow real data PDUs to flow into the graph, but this is a convenient model for testing.

We pass these random bits into the framer block which adds a length field, a header checksum, and a payload checksum – which will allow us to verify correct reception and determine packet length upon receipt of the burst at the downstream receiver.

After this framer we pass the PDU through a “burst padder” block which simply adds zero bits to the end of the frame until the length reaches an integer multiple of the uncoded forward error correction (FEC) block size.   Since we must encode entire FEC blocks at a time this is a required step, when operating without FEC this could be easily removed.   At this point we go through a randomizer which XORs a random sequence onto the data to whiten the payload bits and pass it through the forward error correction encoder.

We then insert a known binary preamble sequence onto the front of each burst, and pass the burst bits through a QPSK bit to symbol mapper which converts the uint8_t bit vector into a complex float32 sample vector ready to be inserted into our transmission stream.

The image below shows the total transmit flow-graph as described above.

psk_burst_ldpc_tx.grc

Transmit Waveform: Bridging Domains

The last message block is the burst scheduler which decides when in time to schedule the burst.   It is simply a slotted aloha type of scheduler in this case which drops the burst in as soon as it can along some fixed slot boundary.   It sends bursts annotated with a sample-time to schedule them into the stream on to the eventstream source block, and receives asynchronous feedback from the same eventstream source block letting it know where in the stream it is “now”.   Out of this eventstream source block, we get a sample of complex zero samples with events copped into the correct offsets specified by their event time.

Transmit Waveform: Stream Domain

The stream domain then for this transmitter is quite simple, we go through a throttle block to limit the speed of the outgoing sample stream, go through an interpolating RRC filter to convert from 1 sample per symbol up to 2 samples per symbol for transmission, go through a standard channel model block, and finally stream the resulting continuous sample stream containing bursty transmissions out into a file sink and some QtGui plotters.

The graphical display below shows the plot of a single burst’s samples on the left side, and the mixed burst and off states with noise added in the standard stream plotters on the top and right side.   The bottom text area shows the contents of the PDU dictionary as it travels from the burst scheduler to the eventstream source, in this case the only key value pair is the event_time to schedule the burst at.

tx_waveform

Receive Waveform

The receive waveform similarly spans a message and a stream domain, bridged by using an eventstream sink block to extract discrete events from the continuous sample stream at appropriate times.

Receive Waveform: Stream Domain

The receive waveform begins with a sample stream, either from a radio or out of a stored sample file in this case.   To simulate real time operation, we throttle the data stream at the intended sample rate and then run through a matched filter for the presence of a preamble.   The output of this filter is then further filtered through a local comparison to a moving average, in the correlator filter block, and then the resulting detection metric is run into the eventstream trigger rising edge block.

This block detects whenever the detection metric rises over a certain threshold at the beginning of a burst and sends an asynchronous message to the eventstream sink block to extract a burst event beginning at that time in the sample stream.

Receive Waveform: Bridging Domains

 

The eventstream sink block, when triggered by the threshold detector, schedules a stream event which fires off the PDU handler.   This is kind of a dummy handler which simply converts the extracted samples into a standard PDU format message to be handled by any message passing blocks downstream.

Below you can see the flowgraph for the receive waveform as described.

psk_burst_ldpc_rx.grc

Receive Waveform: Message Domain

In the message domain, we start off with a chunk of samples extracted from the continuous stream containing the burst somewhere in it.   We have not done a fine synchronization and we do not yet know the length of the burst, so for now we pull out some upper bound on the length of all bursts worth of samples.    This is immediately plotted as power over time, and then run through a length detector block which attempts to trim some of the noise off the end of the burst based on its power envelope fall off / trailing edge.

At this point we have hopefully minimized the number of samples to run through our synchronization algorithm, which is of non trivial compute complexity.   We plot the constellation three places, before synchronization, after CFO correction, and after equalization and timing correction.   Since the synchronization block is operating on a whole burst atomically we are no longer restricted to using tracking loops which take time to converge at the beginning of a burst.   Instead we can produce an ML estimate for the CFO over the length of the burst, compute optimal timing and equalizer taps over the length of the burst, and then apply them atomically to the entire burst worth of samples.    The lower left hand plot below shows the correlation peak obtained during timing synchronization within this block,  with a clearly observable peak at the preamble.

Coming out of the synchronization block we have the burst’s symbols lined up in the correct rotation based on the preamble, at this point we can go through a soft demapper block, which translates all complex float symbols in the burst into a series of floating point soft bits in a PDU.   We can then use a “burst frame align” block which first strips the preamble bits off the front of the burst, and ensures that we trim all the remaining bits to a multiple of the coded FEC block size (in this case 271 bits).   We pass these soft bit vectors through an FEC decoder block to perform LDPC decoding and output hard bits, and then finally through a burst randomizer to remove the XORed whitening sequence.     The bottom middle plot shows the soft bits of a single frame before this decoding process, we can see that there is a bi-modal distribution representing clusters centered around the soft values for 0 and 1 bits for the useful portion of the frame, which is what we would hope to see.

At this point we can pass the decoded, derandomized hard bits through a deframer which provides a header CRC check, a payload length field to ensure we throw away bits past the end of the frame, and a payload CRC check which ensures none of the bits in the payload were corrupted.

Lastly we send the resulting PDU to a “meta text output” GUI widget, which allows us to look at the PDU’s dictionary values for each burst in a nice clean and easy way.    This is shown in the top right of the GUI below and contains a handful of dictionary entries which have been appended through event scheduling, synchronization, decoding, and deframing.   These include event time, center frequency offset, number of FEC iterations to decode, number of frames passed and failed, and header and payload pass rates, among other things.

rx_waveform

Future Use

The hope of this modem is largely that the message based components allow for very simple implementation of atomic burst operations, without needing to worry about book-keeping complex bursty state information within stream processing blocks.   A lot of effort has been put into trying to develop components which are NOT tightly coupled in any way, and have general applicability to many configurations with the same modem architecture.   I’m quite excited about having this available as a baseline because I think it will be a great starting point for building many projects in the future which seek to tailor various bits of the modem stack for specific situations without having to reinvent anything monolithic.   For the moment this does not represent an “adaptive” modem, but many of the components are already ready for that.   The demapper block can switch per burst based on a “modulation” dictionary entry, and a header deframer could easily schedule the extraction of an additional payload burst region back into the eventstream sink block if this were necessary, so I believe the architecture is already very well set up to handle these sorts of challenges which we are likely to throw at in the near future.

Waveform Information

This waveform both with and without LDPC can be put together using the GRC graphs in the gr-psk-burst repository which can be installed via pybombs.   This is available on github at https://github.com/osh/gr-psk-burst.   It leverages a number of tools from gr-vt as well as gr-pyqt to provide transmission, reception, and plotting.

I should also acknowledge the LDPC encoder and decoder was the work of Manu TS, originally in an out of tree module (https://github.com/manuts/gr-ldpc) developed as part of Google Summer of Code.   This work has now been integrated into GNU Radio’s gr-fec API and has been moved in-tree.   Additionally much of the underlying  work on burst syncrhonization and other burst modem blocks has been made available from Virginia Tech in the gr-vt repo, https://github.com/gr-vt, and represents the work of a handful of very talented people there.