Hi Walter,
interesting project!
The libpcap approach seems to be reasonable; backintheday, I used to capture at Ethernet frame level using socket(PF_PACKET,…), but that's pretty non-portable and comes with its own can of worms. pcap's the way to go there, I'd say, unless you want add a protocol family to your operating system (don't know whether that is Linux or Mac OS), which I kind of doubt.
However:
The PUB/SUB scheme is almost certainly not what you want here – that is for broadcasting data to multiple subscribers (or dropping them, when the subscriber(s) aren't ready) from potentially multiple transmitters. You might spot the problem here! If you attach a waterhose on one end, and the other end doesn't fetch packets in intervals short enough for the receive buffer to not overflow, these packets will just silently be dropped - business as usual for a PUBlisher! Try the PUSH/PULL pattern: GNU Radio by principle will need a block like the SUB block to fetch data as available, and call it back later at some point. That will not work well in this use case.
So, to your core question, writing a GNU Radio block for your device is relatively easy, probably; data rates aren't *that* high, so an extra memory copy here and there is something I'd live with for a prototype.
Methodology would be this, roughly:
1. make an out-of-tree module. We cover this on https://tutorials.gnuradio.org , specifically in [1]. In short, `gr_modtool newmod yourmodname`.
2. Make a source block (`gr_modtool add -t source -l c++ hose_source`)
3. in the generated lib/something_impl.cc, add a `bool hose_source::start() {}`, and also add tht `bool start() override;` method prototype to the _impl.h
4. in the _impl.h add private fields: a set of buffers, one for each channel, where you'll put the data "deinterleavedly" from the NIC. Make each buffer some (say, 2²⁰) GNSS samples long. Also add two unsigned integer counters: one read and one write index. And because we're lazy and don't care *that* much about performance yet, two mutexes (one for securing access to the read index, and one for the write index).
5. in the constructor, you allocate these buffers, set the read index to the length of the buffers (-1) and the write index to 0
6. in the start() method, you spawn a thread that, in a loop
1. checks how much space there is between read and write index (get read mutex, fetch read index value, release mutex, calculate difference)
2. uses pcap to fetch packets, (but only as much as the space calculated above allows for!), deinterleaves data onto the buffers, finally
4. updates write index (get write mutex, update write index, release write mutex)
7. the block's work() method is called by GNU Radio regularly and
1. checks how much data is between write and read indices (get write mutex, read write index, release mutex, calculate difference)
2. checks whether that's more or less than the space for output items available in this current call to work(), takes the minimum of both
3. gets that amount of items from each buffer and writes them to the output buffer, as passed as argument to the work() method (you could do type conversions here!)
4. updates the read index accordingly (get read mutex, update index, release mutex)
5. returns the number of written items
note that the index updating and distance calculation need to take the "wraparound" at the end of the buffer into account.
Also note: very similarly, you could write a **SoapySDR** driver instead of a GNU Radio block. You could then use the Generic SoapySDR Source block to get data from that driver, and other, non-GNU Radio programs, could be using the driver just as well, without knowing about the hardware. I don't think the basic principle would be much different: you need an IO thread that keeps the NIC busy, and because readers might be slow, an internal buffer, which you ideally use constructively (instead of just incurring a memory bandwidth overhead), to deinterleave channels on ingress, and to convert data types on egress, if you will.
Note that one *could* potentially, as mentioned above write a zero-copy-ish driver for GNU Radio 3.10+ using our custom buffer framework and something like AF_XDP, dpdk, or io_uring, but I think that would very much a) leave the scope of what anyone be able to assist you with – to the best of my knowledge, you'd be the first to do that with a network device – and b) we're "only" talking gigabit ethernet here, and you got a fast machine. As you said in your email, in principle, things are plenty fast enough, so let's not overcomplicate.
[1] https://wiki.gnuradio.org/index.php?title=Creating_C%2B%2B_OOT_with_gr-modtool
On 12.07.24 22:42, Walter Szeliga wrote:
Hi all,
I have a GNSS Firehose (https://transitiva.com/product/gnss-firehose-receiver-tri-band-quad-constellation/ <https://transitiva.com/product/gnss-firehose-receiver-tri-band-quad-constellation/>) and have been trying to get it working, in a streaming capacity, with Gnuradio. The Firehose sends packets over ethernet using the experimental ethertype 0x88b5. I've tried a few things to get data from the Firehose into Gnuradio, some have worked, others have not. Things that work:
* Use tcpdump and filter on 0x88b5 and save to a file. Open and repackage each packet in the pcap dump as interleaved bytes of I&Q and save to a file. Read into Gnuradio.
* Write a custom program using libpcap to filter on 0x88b5 on a selected interface, repackage the packets and write directly to a file. Read into Gnuradio.
* Write a custom program using libpcap to filter on 0x88b5 on a selected interface, repackage the packets and PUB them using ZMQ. SUB to this PUB using a simple Python script and dump the message contents to a file. Read into Gnuradio. Both tcp and ipc PUB/SUB work equally well.
Some things that do not work:
* SUB to the ZMQ PUB/SUB using the Gnuradio ZMQ SUB Source block.
* Write a custom program using libpcap to filter on 0x88b5 on a selected interface, repackage the packets and send them using UDP.
The UDP approach doesn't work because too many packets get dropped and I have been unable to set sysctl values appropriately (on an M1 Mac) to avoid dropping too many packets.
I'm not sure why the ZMQ approach does not work with Gnuradio. I've tried many simple flowgraphs to convert from vector to stream, ibyte to complex, you name it, and then dumping the data back out to a file using a File Sink (just to use existing software to check for data sanity). Data gets into Gnuradio, but it clearly loses something because constellation diagrams of the output data become blobs centered on the origin rather than pairs of point clouds offset from the origin as one would expect from the BPSK nature of the GNSS signals captured by the Firehose.
I've come to the realization that it's probably best to write some sort of driver to get data straight from the Firehose into Gnuradio, but I have no idea where to start with this. Any ideas about how to fix my ZMQ approach or start writing a custom driver would be appreciated. There's a lot in here, so let me know if you would like code or flowgraph examples.
Cheers,
Walter
