Happy January! This month’s progress on the ADSBee project included a number of impactful firmware updates that added features requested by the community and fixed some important bugs. As of firmware version 0.7.2, here’s what I’ve been cooking!
- Settings Download and Restore from Web UI
- Hostname and mDNS Support
- Table-Based CRC
- 1-bit CRC Correction (and RP2040 multicore gotcha)
- Bug Fixes Galore!
- GitHub CI Builds
⚙️ Settings Download and Restore from Web UI
One of the top requests from our beta testers was the ability to download and restore settings from an ADSBee, allowing devices to be cloned and also allowing settings to be restored after a firmware update.
Most firmware updates don’t overwrite device settings, but some major updates (like transitioning from 0.6.X to 0.7.X) do make changes to the settings schema, meaning that feed settings and WiFi network information need to be re-configured after the firmware is upgraded. The new settings download and restore features turn this into a two-click operation!
There was some amount of debate about how best to implement settings download and restore functionality, but I eventually settled on using the existing AT command interface since it’s the most stable and backward-compatible control interface for the ADSBee. Sending all control commands (including USB console, Web Console, and remote script interaction) through a single command interface makes adding new features much easier, and reduces the risk for differing behaviors between interfaces.
Fortunately for me, utilizing the AT command interface in this manner made implementing settings download and restore features quite straightforward. Hitting the “download settings” button triggers a new AT command called AT+SETTINGS?DUMP
, which prints all of the current device settings to the console in AT command format. This output gets captured by the javascript on the Web UI webpage, which saves it as a .settings text file. Clicking the “Restore Settings” button prompts the user to upload a .settings text file, and then applies the AT commands within that file to the ADSBee via the web console.
NOTE: Currently, restoring settings from a .settings file requires a user to manually run AT+SETTINGS=SAVE
afterwards to make the new settings persist. This will be patched in a future update so that restored settings are persisted between reboots automatically.


🛜 Hostnames and mDNS Support
ADSBee receivers can now set their own network hostname with the AT+HOSTNAME
command! This is very useful if you have multiple ADSBees on your network that you’d like to differentiate (without remembering serial numbers), or if you just want a convenient way to find the Web UI of your ADSBee without bookmarking an IP address.
In order to allow users to visit ADSBee web UIs at <hostname>.local
, mDNS services have been added to the ADSBee firmware, so ADSBees can self-report their hostnames to other devices on the same network.
mDNS hostnames for other devices can also be used as feed endpoints! For instance, I have a readsb docker image on my laptop that I use to test the Mode S Beast output, and the hostname of the laptop is bird-book-pro.local
. Instead of fishing around for my IP address to use as a feed endpoint, I can set the feed URI using the hostname for a slightly less fussy experience.


🏓 Table-Based CRC
Previous implementations of CRC calculation for ADS-B and Mode S packets on ADSBee utilized an algorithmic technique that involved a number of repeated calculations. As of firmware 0.7.2, ADSBee calculates checksums using a pre-calculated table stored in flash memory, allowing checksums to be computed much faster and saving valuable processor cycles for more interesting things.
The CRC lookup table is essentially a 256-word array, with each word corresponding to a possible CRC Byte value as a processor is walking along a message. Instead of performing a rolling CRC on the message at every bit index, the processor can instead feed a full Byte of the message (along with some other info) into the CRC table and receive the pre-computed CRC value in return. Bits to Bytes, that’s roughly an 8x speedup!

If you’re interested in poking around or making your own table-based CRC calculator, the pre-calculated tables, interface code, as well as the Python script I wrote for generating CRC tables (and correction tables, more on that later), are available here in the GitHub repository.
👯 1-bit CRC Correction (and RP2040 multi-core gotcha)
Invalid ADS-B packets received by the ADSBee now have their “syndromes” (XOR of calculated and received checksums) compared to a pre-generated table of known syndromes corresponding to single bit errors at each location in the message. This allows single bit flips to be corrected, and a fraction of invalid packets to be recovered. In my testing, this seems to result in something like a 5-10% increase in decode rate, but results may vary.
Implementation of 1-bit CRC correction included porting the packet digesting code on the ADSBee onto the second processor core of the RP2040. This means that the ADSBee RP2040 application is now multi-core, with double the available compute resources! I’m excited to see how this windfall of processor cycles could be used in future application features.
While porting the RP2040 application to multiple cores, I found a nasty gotcha! If I put any significant amount of code onto Core 1 on the RP2040, Core 0 would execute until the start of the Core 1 code, and then crash. I could keep things working on both cores while using extremely small commands, like GPIO toggles, on Core 1, but anything more complex (like a printf) caused things to hardfault.
Eventually, I tracked down some very useful Raspberry Pi forum threads and a wonderful website writeup by Pete Warden that explain the issue quite well. For some reason, the Raspberry Pi foundation decided to dedicate an extremely small amount of stack memory to Core 0 (SCRATCH Y) by default, and located this stack just above the stack for Core 1 (SCRATCH X). When a single threaded application running on Core 0 exceeds its stack limit, it stomps on the stack of Core 1 before continuing to write into the much larger memory block dedicated to the Heap memory space.

The ADSBee application running on Core 0 uses a lot of stack memory, mostly due to my preference for pre-allocating buffers on the stack for most data transfer and processing tasks, as a bit of an embedded hack to avoid dealing with malloc
, free
, and the inevitable memory leaks they might bring. As a result, the Core 0 application absolutely obliterates the Core 1 stack space, which is normally not noticeable, as long as Core 1 doesn’t attempt to load or execute any code. As soon as I started up Core 1, it began writing into random parts of the Core 0 applications stack, causing aforementioned hardfaults.
Fortunately, the solution to this problem was relatively simple. Currently, there is very little memory used in the Core 1 stack, since it mostly reads from buffers that were already allocated on the Core 0 stack. Thus, by editing the linker scripts to allocate SCRATCH Y as Core 1 (small application), and SCRATCH X as Core 1 (big application), Core 1 would be free to overflow into stack memory without stomping on Core 0 along the way. With this change, multi-core applications seem to be running without issue, and firmware update 0.7.2 includes multi-core processing for checksum correction.
The linker files that I edited to resolve this issue are available on the GitHub repository, in case you or a friend find yourself in the same scenario!
🐛 Bug Fixes Galore!
A number of bug fixes were introduced with firmware updates this month. Here’s a few!
- ADSBees now properly set their unique receiver IDs when communicating with online aggregators like airplanes.live, adsb.lol, adsb.fi, etc. UUIDs set with the
AT+FEED
command are propagated into the Mode S Best feed during initial creation of the TCP socket stream, allowing a map of your local ADSBee’s feed to be queried from the aggregator using their API. Take a look at the instructions in this part of the README for some tips on accessing your local map! The instructions vary for each aggregator. If you have some details about how to access feed metrics or a local map for an additional aggregator, please drop me a note in Discord so that I can add the information to the quick start guide. - Mode S Beast feeds from the ADSBee now properly scale their RSSI values for detected aircraft.
- A nasty Compact Position Reporting decode issue was introduced in firmware 0.7.1, and then fixed shortly afterwards in 0.7.2 (oops)! This bug affected aircraft positions reported over local interfaces (CSBee, MAVLINK, GDL90) but didn’t affect raw feeds like Mode S Beast.
- The handshaking process used to update firmware over the network has been modified, and updates are now much more stable (but certainly not perfect–it may still take one or two tries to get a firmware to load properly onto an ADSBee over the network). There will be more work on this feature in the future!
- Build issues related to application linking order have been fixed! This was a cause for some intermittent build failures that varied by machine (there was a race condition that could cause some firmware images to be linked before they were built).
- A bug that caused one packet to be dropped during every SPI transaction between the RP2040 and ESP32 has been fixed (off by one, go figure). This wasn’t very noticeable in previous firmwares, but with checksum validation and correction on the RP2040, the ADSBee switched to only forwarding valid ADSB packets between the RP2040 and ESP32, making dropped packets much more apparent.
🚀 GitHub CI Builds
The ADSBee GitHub repository now has Continuous Integration (CI) builds enabled via GitHub Actions. These builds automatically build and run unit tests, then build and link firmware applications for both the RP2040 and ESP32, and produce the combined firmware artifacts for loading the application onto the ADSBee via USB (.uf2 file) or via the network bootloader (.ota file).
In addition to the automatic unit tests being a good goof check to ensure that pull requests don’t break things, this CI build functionality means that anyone can easily compile their own firmware for the ADSBee without a local build environment! While I wouldn’t recommend this approach for serious development (building via CI is sloooooow, and offers no debugger functionality), it does mean that it’s simpler than ever to try out a code change on your ADSBee receiver. Yay open source! 🌈✨
More to come!
I have many more features and improvements in the pipeline for this year, and I’m just as excited about the ADSBee project as ever. Stay tuned to Discord (and the mailing list) for more info!