Visualizing Radio Propagation in GNU Radio with gr-uhdgps, gr-eventstream and Google Earth

Radio propagation is a complex process in the real world, elaborate models are often used to predict expected propagation effects over known terrain, but in many cases there is no substitute for measuring ground truth.  This can now be accomplished quite easily, quickly, and cheaply using GNU Radio for a wide range of protocols.   Using gr-uhdgps (https://github.com/osh/gr-uhdgps), gr-eventstream (https://github.com/osh/gr-eventstream) and Google Earth, we demonstrate a tool to measure broadcast VHF FM Radio received signal strength over a metropolitan area. In this case we use WAMU 88.5 MHz, the local NPR radio station in Washington D.C., to demonstrate the tool and plot the results in Google Earth.

Building the GNU Radio Flowgraph

Looking at the WAMU FM Broadcast channel in a QtGui frequency and time sink plot is a good first sanity check.  Since FM broadcast is typically allocated a 200khz channel, we set up a USRP B210 to tune to 88.5 MHz with a complex sample rate of 200 ksamp/s shown below, the expected time and frequency spectrum behavior of the FM audio signal can be seen below.

      FM_WAMU_Plot   uhdgps_rssi_log.grc

In this flow graph we extract and event of 512 samples exactly every 1 second’s worth of samples from the incoming complex sample stream off the UHD source block.  Since we are throwing away the rest of the samples not extracted, this should be a low duty cycle, low CPU load process to run for the most part.   The trigger sample timer event allows us to trigger events at this sample-periodicity of a fixed length and pass them downstream to the “Handler PDU” block, this block simply converts the eventstream events into normal complex float PDUs which can be handled by any message passing block.

At this point we introduce a simple python message block called “CPDU Average Power” which computes the average sample power within the event’s samples, tags a “power” key and value to the PDU’s metadata structue and passes it downstream.    The functionality of this python block is contained within this simple python/numpy handler function which computes the power, adds the tag, and sends the output message — for now we are throwing away the PDU data field and sending PMT_NIL in its place since we don’t do anything with it downstream.

def handler(self, pdu):
    data = pmt.to_python(pmt.cdr(pdu))
    meta = pmt.car(pdu)
    data = data - numpy.mean(data) # remove DC
    mag_sq = numpy.mean(numpy.real(data*numpy.conj(data))) #compute average Mag Sq
    p = self.k + 10*numpy.log10(mag_sq)
    meta = pmt.dict_add(meta, pmt.intern("power"), pmt.from_float( p ) )
    self.message_port_pub( pmt.intern("cpdus"), pmt.cons( meta, pmt.PMT_NIL ) );

At this point we remove the “event_buffer” item from each PDU’s metadata structure using the “PDU Remove” block since it is also large and we don’t really want to keep it around downstream for any reason — trying to serialize this into our JSON storage would waste a lot of space and makes the message debug print outs massive.

Since the B210 is set up with an internal GPSDO board, we have both a GPS disciplined clock and oscillator as well as GPS NMEA fix data available from the UHD host driver.  By using the uhdgps gps probe block, we can poll the USRP source block for the NMEA fix data every time we need to store an updated signal strength record as shown in the flow graph above.  Passing our CPDUS through this GPS Probe block appends the NMEA data and a bunch of current USRP state information from the device which we can look at later.

Finally, we store a sequence of the JSON’ized PDUs to two files, one which has been filtered for records where “gps_locked” is “true” and one which simply stores all records regardless of GPS lock state.

Record Format

Each JSON’ized record corresponding to a single PDU from the flowgraph and a single eventstream extracted sample event, now looks like this in our output files.

We see that this structure now contains gps state information pulled from the GPS probe block, the stream tags which were added to the event/PDU by eventstream, and our power value computed by the python CPDU power block.

{
 "es::event_length": 512,
 "es::event_time": 20201024,
 "es::event_type": "sample_timer_event",
 "gain": 30.0,
 "gps_gpgga": "$GPGGA,003340.00,3852.6453,N,07707.3960,W,0,10,1.2,85.4,M,-34.7,M,,*6E",
 "gps_gprmc": "$GPRMC,003340.00,V,3852.6453,N,07707.3960,W,0.0,0.0,290315,,*3D",
 "gps_locked": "true",
 "gps_present": true,
 "gps_servo": "15-03-29 307 47943 -469.10 -3.21E-09 15 13 2 0x20",
 "gps_time": "1427589220",
 "power": -119.32362365722656,
 "ref_locked": "true",
 "rx_freq": [
 0,
 88499999.9999999
 ],
 "rx_rate": [
 0,
 199999.99937668347
 ],
 "rx_time": [
 0,
 [
 892243202,
 0.5858168473168757
 ]
 ]
}

Drive Testing

To test this graph I securely fasten a macbook into the passenger seat, wedge a B210 between the center console and passenger seat of the Subaru test vehicle, and plunk a GPS antenna and a poorly band matched magnetic duck antenna onto the trunk of the car and go for a drive.   After launching the application, it takes a while for the GPSDO to obtain GPS lock since it is not network assisted like many devices these days.  After a few minutes of waiting and impatiently moving the GPS antenna to the roof of the car, I obtained lock and pretty much kept it for the entirety of my drive.

seat-scaled  antenna-scaled

Inspecting Results

After driving a thorough route around town and past the broadcast tower, I dump JSON file through uhggps’s json_to_kml.py tool to convert the rss measurements into a Google Earth friendly KML file with vertical lines of height proportional to the RSS at each event location. This provides a nice intuitive visual way to look at and understand received signal strength effects over various terrain changes, obstacles, urban canyons and other such phenomena.   An overview of the route can be seen here.

plot_overview

A few notable spots around the route are shown below.

Driving through Rosslyn, VA we can see massive variation in received signal strength at each sample point likely due to massive amounts of urban multi-path fading resulting in large amounts of constructive and destructive interference.

rosslyn_variance

Driving from the George Washington Parkway North onto the US 1 memorial bridge on the far side of the river, we can see that going from a lower elevation shadowed by a small hill and a handful of trees up onto the elevated and unobstructed bridge resulted in a significant sustained climb in signal strength level.

memorial_bridge

Driving north on Connecticut Avenue, taking the tunnel underneath Dupont Circle, we briefly lose GPS lock, but upon reacquiring show a very low received signal strength down in the lowered, partially covered, and heavily shadowed roadway coming out of the tunnel.   To deal with this kind of GPS loss of lock we may need to implement some kind of host based interpolation of fixes occurring on either side of the positioning outage, but for now the samples are discarded.

dupont_tunnel

There are numerous other effects that can be observed in the KML plot from the drive which is linked below.

KML Issues

One issue that I’m still considering is how exactly to map RSSI to amplitude, there are two modes which KML supports and each has their own issue.   These plots are using “aboveGroundLevel” which maps “0” elevation to the ground elevation at a specific LAT/LON, this is good because we can just use 0 for our base height and not worry about putting a line underground somewhere, however its annoying in that the line elevation tracks with the ground contours distorting your perception of RSSI a bit.   Absolute doesnt have this problem, since 0 is (I think) locked at sea level, however in places farther from the coast this is annoying because low elevations wind up underground.  Ultimately it might be neccisary to do some kind of aboveAverageGroundLevel for a local area, but this is as far as I know not supported in Google Earth / KML.

References

The resulting KML file can be opened in Google Earth from https://github.com/osh/kml/blob/master/WAMU_RSS.locked_1427589112.17.json.kml.   Countless hours of fun can be had zooming around inspecting fades that occur passing various obstacles along the way and signal strength variation going up and down hills in various areas.

This flow graph is available at https://github.com/osh/gr-uhdgps/blob/master/examples/uhdgps_rssi_log.grc   To use it make sure you have gr-eventstream, gr-uhdgps, and relevent python modules installed.

Burst Transmission in GNU Radio Sample Streams with Eventstream

There are a handful of ways to transmit bursts of information using GNU Radio.   Perhaps most widely known is when using Ettus UHD based devices, burst transmit mode may be used to schedule bursts of samples within a properly tagged stream.   This results in accurately timed bursts and low latency transmission, but it also relies upon specific Ettus hardware and makes software only loopback simulation impossible.

This alternative method of burst transmission using the eventstream (http://github.com/osh/gr-eventstream) source block to interleave bursts into a sample stream is hardware independent, and allows precise timing of bursts in the sample stream in a hardware independent way which works well in loopback software simulation.   The performance cost is a slightly higher latency penalty than the UHD burst mode due to reliance on back-pressure within the GNU Radio transmit stream.  But it does have the nice side effect of resolving the ugly “flushing the end of a burst” nastiness which may be present when sending the end of a tagged stream burst through GR blocks and/or transfer to a UHD device.

Signal Source

Our burst signal source in this case is the random pdu generator block.   The message strobe block sends it a message every 500ms telling it to generate a new burst of between 50 and 2000 bits in length.   We ensure that each byte contains a single bit by anding a mask of 0x01 in this generator block.   Finally we go through a PDU PSK burst mapper block to map the random bits into a complex constellation points in our new complex float PDU.

These bursts are plotted using the gr-pyqt burst plotters for reference before continuing down the chain.   They are burst only and and no “off” time exists at this point in the graph.

Eventstream Source / Burst Scheduling

The eventstream source block, void of any events, simply produces zero valued samples at a rate limited by downstream back-pressure.   A variety of transmit handlers may be used to produce output into this sample stream at precise times and lengths.   One convenience handler which is provided is the PDU Handler, which is internal to the eventstream source block, and allows you to simply pass a PDU of samples into the block to be sequenced into the output stream without worrying about the more complex trigger/handler paradigm.

Once we pass a complex float PDU into the schedule_event port, the block checks for an “event_time” tag on the event to tell it what precies sample index time to schedule the burst at.   When this value is not provided, a time of 0 is assumed, which is the case in this example.   The source block defines and “Early Behavior” which defines what should happen if an event arrives with a time which is too early for the current stream nitems_written(0) value (number of samples which have been produced).   The options here are to DISCARD, throw away events which are too early, BALK, or throw an exeption and stop running when such event arrive (useful for debugging), or ASAP, to simply schedule them at the earliest possible time available.   In this case we use ASAP to schedule the events as soon as possible.

flowgraph_es_tx

Plotting

Coming out of the eventstream source block, we throttle the stream to limit production rate to our desired sample rate, add some gaussian noise to make plots more interesting, and then plot the power of the resulting sample stream with roughly time periodic bursts in it.   We can see that plotting a burst tirggered roughly every 500ms does result in a burst being scheduled into the stream roughly every 500ms.   If we wanted this to be exactly 500ms in samples, we could set the “event_time” tag on the event to the proper sample index desired, but this exercise will be left to a future example.

plots

Simple GNU Radio Eventstream Based Burst Extraction and Plotting

Both gr-eventstream and gr-pyqt have been around and available now for a while on my github page, but not enough good documentation material exists for either.   This article aims to address that and show a simple example of how they can both be used to plot synchronous windowed burst events occurring within a standard sample stream.   In order to run this example you will need to install both gr-eventstream (https://github.com/osh/gr-eventstream) and gr-pyqt (https://github.com/osh/gr-pyqt) for plotting.

Signal Source

The signal source for this example is a simple complex sinusoid with amplitude 1 added to complex Gaussian noise with mean amplitude 0.01.   This gives us a nice well understood signal to play with, but adds a bit of noise so plots aren’t identical each burst or sine period.

Detecting Bursts

Using eventstream, finite time-windowed events or bursts are scheduled by blocks known as “trigger” blocks.   There are many ways to do this such as a matched filter correlators, cyclostationary detectors, or otherwise; in this example we will use a generic “Rising edge trigger” block which simply picks up on a float stream rising over a constant threshold value.   Instead of implementing a new custom trigger block for you application, you may be able to use stream operators to produce a detector of interest and then simply pass the derived metric into the trigger block to perform detection.   In the case of this example, we use the real component of the complex sinusoid as our detection metric since it should be quite intuitive as to what is going on here to most readers.   This trigger block allows you to specify a threshold value, an event length (the number of samples contained in each event), a “lookback” length (how many samples before the actual rising edge should the event start), and a minimum trigger distance (a debouncing parameter to allow you to disallow subsequent events for N samples, incase of a noisy rising edge).   These can be seen in the flowgraph image below; upon meeting the threshold criteria, an event description is sent to the eventstream sink block.

Extracting Events

The Eventstream Sink block performs a seemingly simple, but tedious function.   It takes in “event” descriptions (event type, event time, event length) extracts the samples from a stream corresponding with the event time and length specified, and then sends the populated event to all “handlers” downstream which are associated with this type of event to operate.

Handling Bursts

Eventstream burst handlers are blocks which implement a special c++ GR Block interface which implements a handler( pmt_t msg, gr_vector_void_star buf ) method.  Since you probably don’t want to write a custom c++ handler function/class, the easiest thing to do in many applications is to use a “Handler to PDU” block.   This block takes the incoming event parameters and sample buffer from the event and converts it to a standard GNU Radio PDU formatted PMT asynchronous message to send downstream.  Standard GNU Radio asynchronous message blocks which operate on PDUs may then be used to operate on it.  That is the method we will use in this example.

Handling PDUS

Now that we have a standard PDU with complex floats contained in its data vector, we can send it downstream to two message blocks using fan-out of standard GNU Radio message ports.  The first connection is to a simple message debug block, this simply prints the message contents to the console for the user’s viewing pleasure.   The second is to a gr-pyqwt complex time plot, which plots the event as a single atomic plot every time a new message comes in.   These plotters were implemented to allow plotting of an entire event at once regardless of its sample length, as opposed to the existing gr-qtgui plots which are as of this writing, intended for stream plotting and by default plot 1024 samples every time, not variable as these are to the event buffer’s length.  This can be very useful for diagnosing the behavior of various burst waveforms.

Connecting Everything Up

Connecting everything up, we connect our raw sample stream input into the pass-through port on the trigger rising edge block, and we connect our detection metric (the complex to real float stream output) into the thresh_input port.

We then connect our trigger block’s passthrough_out port to the es.sink input block (We keep the sink downstream simply to ensure that the sink does not consume anything that the trigger has not yet, otherwise there would be no need to go through the trigger block), and the trigger’s “which stream” message port to the sink’s “schedule_event” port, this tells the trigger where to schedule the burst extraction when a trigger has been fired.

Finally, we connect the “edge_event” port on the trigger to the “handle_event” port on the “handler”.   This is perhaps the most confusing connection, it specifies that events of the type “rising edge” shall be handled by the “to pdu” handler, however it is a logical connection and does not represent the actual data path at runtime!  At runtime, the event goes to the eventstream sink, is populated with samples, and then passed to the handler to be converted to a PDU.   However, this direct connection is used to provide a more intuitive GRC connection line to show logically which events go to which handlers.   In reality when start() is called in the flowgraph, trigger blocks indicate their connections downstream to their corresponding es.sink blocks which which event types correspond to which handlers so that they then known at runtime which handler blocks to forward populated events to.

The final completed flowgraph can be seen below.

flowgraph

Plotting the results

When we run this flowgraph, we now expect to trigger every time the signal’s real component rises past a value of zero.   Since we have a bunch of added noise, we will actually also see a noisy “rise” past zero when we pass zero both on the way up and down, as you will see when you run the flowgraph.   You will see on the X axis below that each event is 1000 samples in length, which was specified in our trigger block’s “event length”.  In this case it is a constant size, but a trigger can specify a different event length for each burst if desired.  By playing with the lookback value to 500, you could center the zero crossing in the event if desired here.  Lastly since we have a debounce window of 300 samples. we only get one event per crossing even though the noise on top of the signal causes several local crossings in the area for each of these in reality.

Our event shown below, shows the real (green) and imaginary (red) portions of our complex sinusoid with the exact sample of the trigger occurrence being the first (leftmost) sample in the plot.

burst_plot

Example Source

The GRC source file for this example can be downloaded at: https://github.com/osh/gr-eventstream/blob/master/examples/demo_01_burst_rx.grc