Saturday, July 13, 2024

Re: Writing Drivers for Custom Hardware

Hi Walter,

interesting project!

The libpcap approach seems to be reasonable; backintheday, I used to capture at Ethernet
frame level using socket(PF_PACKET,…), but that's pretty non-portable and comes with its
own can of worms. pcap's the way to go there, I'd say, unless you want add a protocol
family to your operating system (don't know whether that is Linux or Mac OS), which I kind
of doubt.

However:
The PUB/SUB scheme is almost certainly not what you want here – that is for broadcasting
data to multiple subscribers (or dropping them, when the subscriber(s) aren't ready) from
potentially multiple transmitters. You might spot the problem here! If you attach a
waterhose on one end, and the other end doesn't fetch packets in intervals short enough
for the receive buffer to not overflow, these packets will just silently be dropped -
business as usual for a PUBlisher! Try the PUSH/PULL pattern: GNU Radio by principle will
need a block like the SUB block to fetch data as available, and call it back later at some
point. That will not work well in this use case.

So, to your core question, writing a GNU Radio block for your device is relatively easy,
probably; data rates aren't *that* high, so an extra memory copy here and there is
something I'd live with for a prototype.
Methodology would be this, roughly:
1. make an out-of-tree module. We cover this on https://tutorials.gnuradio.org ,
specifically in [1]. In short, `gr_modtool newmod yourmodname`.
2. Make a source block (`gr_modtool add -t source -l c++ hose_source`)
3. in the generated lib/something_impl.cc, add a `bool hose_source::start() {}`, and also
add tht `bool start() override;` method prototype to the _impl.h
4. in the _impl.h add private fields: a set of buffers, one for each channel, where you'll
put the data "deinterleavedly" from the NIC. Make each buffer some (say, 2²⁰) GNSS samples
long. Also add two unsigned integer counters: one read and one write index. And because
we're lazy and don't care *that* much about performance yet, two mutexes (one for securing
access to the read index, and one for the write index).
5. in the constructor, you allocate these buffers, set the read index to the length of the
buffers (-1) and the write index to 0
6. in the start() method, you spawn a thread that, in a loop
1. checks how much space there is between read and write index (get read mutex, fetch
read index value, release mutex, calculate difference)
2. uses pcap to fetch packets, (but only as much as the space calculated above allows
for!), deinterleaves data onto the buffers, finally
4. updates write index (get write mutex, update write index, release write mutex)
7. the block's work() method is called by GNU Radio regularly and
1. checks how much data is between write and read indices (get write mutex, read write
index, release mutex, calculate difference)
2. checks whether that's more or less than the space for output items available in this
current call to work(), takes the minimum of both
3. gets that amount of items from each buffer and writes them to the output buffer, as
passed as argument to the work() method (you could do type conversions here!)
4. updates the read index accordingly (get read mutex, update index, release mutex)
5. returns the number of written items

note that the index updating and distance calculation need to take the "wraparound" at the
end of the buffer into account.

Also note: very similarly, you could write a **SoapySDR** driver instead of a GNU Radio
block. You could then use the Generic SoapySDR Source block to get data from that driver,
and other, non-GNU Radio programs, could be using the driver just as well, without knowing
about the hardware. I don't think the basic principle would be much different: you need an
IO thread that keeps the NIC busy, and because readers might be slow, an internal buffer,
which you ideally use constructively (instead of just incurring a memory bandwidth
overhead), to deinterleave channels on ingress, and to convert data types on egress, if
you will.

Note that one *could* potentially, as mentioned above write a zero-copy-ish driver for GNU
Radio 3.10+ using our custom buffer framework and something like AF_XDP, dpdk, or
io_uring, but I think that would very much a) leave the scope of what anyone be able to
assist you with – to the best of my knowledge, you'd be the first to do that with a
network device – and b) we're "only" talking gigabit ethernet here, and you got a fast
machine. As you said in your email, in principle, things are plenty fast enough, so let's
not overcomplicate.

[1] https://wiki.gnuradio.org/index.php?title=Creating_C%2B%2B_OOT_with_gr-modtool

On 12.07.24 22:42, Walter Szeliga wrote:
> Hi all,
>     I have a GNSS Firehose
> (https://transitiva.com/product/gnss-firehose-receiver-tri-band-quad-constellation/
> <https://transitiva.com/product/gnss-firehose-receiver-tri-band-quad-constellation/>) and
> have been trying to get it working, in a streaming capacity, with Gnuradio. The Firehose
> sends packets over ethernet using the experimental ethertype 0x88b5. I've tried a few
> things to get data from the Firehose into Gnuradio, some have worked, others have not.
> Things that work:
>
> * Use tcpdump and filter on 0x88b5 and save to a file. Open and repackage each packet in
> the pcap dump as interleaved bytes of I&Q and save to a file. Read into Gnuradio.
>
> * Write a custom program using libpcap to filter on 0x88b5 on a selected interface,
> repackage the packets and write directly to a file. Read into Gnuradio.
>
> * Write a custom program using libpcap to filter on 0x88b5 on a selected interface,
> repackage the packets and PUB them using ZMQ. SUB to this PUB using a simple Python script
> and dump the message contents to a file. Read into Gnuradio. Both tcp and ipc PUB/SUB work
> equally well.
>
> Some things that do not work:
>
> * SUB to the ZMQ PUB/SUB using the Gnuradio ZMQ SUB Source block.
>
> * Write a custom program using libpcap to filter on 0x88b5 on a selected interface,
> repackage the packets and send them using UDP.
>
> The UDP approach doesn't work because too many packets get dropped and I have been unable
> to set sysctl values appropriately (on an M1 Mac) to avoid dropping too many packets.
>
> I'm not sure why the ZMQ approach does not work with Gnuradio. I've tried many simple
> flowgraphs to convert from vector to stream, ibyte to complex, you name it, and then
> dumping the data back out to a file using a File Sink (just to use existing software to
> check for data sanity). Data gets into Gnuradio, but it clearly loses something because
> constellation diagrams of the output data become blobs centered on the origin rather than
> pairs of point clouds offset from the origin as one would expect from the BPSK nature of
> the GNSS signals captured by the Firehose.
>
> I've come to the realization that it's probably best to write some sort of driver to get
> data straight from the Firehose into Gnuradio, but I have no idea where to start with
> this. Any ideas about how to fix my ZMQ approach or start writing a custom driver would be
> appreciated. There's a lot in here, so let me know if you would like code or flowgraph
> examples.
>
> Cheers,
> Walter

No comments:

Post a Comment