Capturing Slack messages from the skies

If there is one thing any Slack user needs more of, it’s messages, and in this post we’re going to be looking at how to get more messages. Specifically, messages transmitted from the flight deck of nearby aircraft into your favourite productivity and GIF distribution tool, Slack.

I recently built a pipeline that takes Aircraft Communications, Addressing and Reporting System (ACARS) messages that are broadcast over VHF and captured by software-defined radio, filters out the interesting ones, and sends the message to Slack, as though it was a message from your co-worker rather than a person in the sky thousands of feet above you.

ACARS messages are used by automated systems to relay status/maintenance information, but can also carry text messages written by human pilots about things that have happened on the flight, such as delays, ride reports or broken cabin equipment. 90% of ACARS messages, for example, seem to be about broken toilets.

Anyone whose flown also knows that there is plenty of drama in the skies, and sometimes you can capture that drama as well, thus:

This setup may sound somewhat complex, but trust me — it’s not. The most complex part of it is making sure you have all the right bits and pieces to do it properly. That’s what we’re going to do.

Overview

So this is the pipeline. The good news, the majority of the work is done inside of an array of docker containers, leaving our main focus to be the last little bit. But, as we go through we’ll explain what every step does, because everyday is a school day after all. To note, my rig is a Ubuntu machine, anything should work though really.

Step One —Setting up your SDR Army

You will need a SDR receiver. At least one. Preferably two. Any more than that is evenbetter. These are $30-ish USB dongles that take in radio signals from an antenna and send said signals to your pipeline for decoding an onward processing.

In my rig, I have 4 SDR’s. Two of them are these. And the other two are these.

Is there any difference between the two types? No, not really. They all use the same chipset, so I’ve noticed no difference. The Nooelec one is less wide, so fits better side by side with a friend versus the RTL-SDR which has increased chonkiness.

Why do you need more than one?

Although this whole thing is about ACARS, sometimes ACARS messages are transmitted using a standard called VDLM2, which operates on different frequencies. We’ll be capturing both protocols (traditional ACARS and VDLM2), which will require at least two SDR’s, since they’ll be mapped to different decoders.

Why would I need more than two?

You don’t, however, ACARS and VDLM2 are broadcast on a range of frequencies depending on where you are located. The rig scans these frequencies, which essentially means it hops around looking for messages. The more hopping — the more likely you are to miss a message on one frequency. The more SDR’s, the less frequencies per SDR to hop around, which means you get better coverage.

Once you’ve gotten an SDR or twelve and connected a suitable VHF antenna (doesn’t need to be particularly fancy — since most planes are in the sky), you’ll need to make sure you can positively identify each of them individually. All of these units seem to ship with a flashed serial number of ‘00000000’, which can make identifying which one is which somewhat tricky, so the first step is making them all identifiable by flashing a new serial number to each.

You’ll need to use the RTL-SDR library to install a handy utility called rtl_eeprom, which lets you flash a new serial number onto the device for identification purposes. To use rtl_eeprom to write a serial number, insert one device at a time (so you know which one you are flashing), and execute the command as shown below, this will change the serial number from the default to make it identifiable:

mike@ubuntu ~ % rtl_eeprom -s 00000001
Found 1 device(s):
  0:  Generic RTL2832U OEM

Using device 0: Generic RTL2832U OEM
Found Rafael Micro R820T tuner

Current configuration:
__________________________________________
Vendor ID:  0x0bda
Product ID:  0x2838
Manufacturer:  Realtek
Product:  RTL2838UHIDIR
Serial number:  00000000
Serial number enabled: yes
IR endpoint enabled: yes
Remote wakeup enabled: no
__________________________________________

New configuration:
__________________________________________
Vendor ID:  0x0bda
Product ID:  0x2838
Manufacturer:  Realtek
Product:  RTL2838UHIDIR
Serial number:  00000001
Serial number enabled: yes
IR endpoint enabled: yes
Remote wakeup enabled: no
__________________________________________
Write new configuration to device [y/n]? y

Configuration successfully written.
Please replug the device for changes to take effect.

Repeat this step, setting a different serial number for each SDR you plan on using.

Step Two — Setup the docker containers to run the decoders and routers

The bulk of the brainwork, i.e. the decoding of messages captured via the SDR’s, and centralized collection of those messages happens in stack that can be spun up through Docker. This is great news, because there are a lot of moving parts here.

Everything that I’m about to spin up, and more, can be found via this wonderful Github profile: https://github.com/sdr-enthusiasts/. Everyone who contributed to this software should be awarded freedom of the universe.

Specifically, we’re going to be spinning up four different container images found on the SDR Enthusiasts page.

• acarsdec — ACARS Decoder, takes signal from your SDR and decodes it.
• dumpvdl2 — Similar to ACARS Decoder, but for VDLM2 messages.
• acars_router — scoops up the messages decoded by the two images above and sends them to the one below.
• acarshub — Provides a database, front end UI, and importantly, JSON output for ACARS messages collected by the rig.

So, make sure you’ve got your SDR’s setup and plugged in. Make sure Docker works on your machine, and then create a docker-compose.yml file thusly (and we’ll talk through all the things in here in a sec):

version: "3.8"

volumes:
  acarshub_run:

services:
  acarsdec-1:
#    image: ghcr.io/sdr-enthusiasts/docker-acarsdec:latest
#    something broken in the latest image, so I hardcoded to this release:
    image: ghcr.io/sdr-enthusiasts/docker-acarsdec@sha256:75f825b55059750b4fbcecd3bb40b5cf2a4d00b230af078b55cc751f466737ea
    tty: true
    container_name: acarsdec-1
    restart: always
    devices:
      - /dev/bus/usb:/dev/bus/usb
    environment:
      - FEED_ID=FEED-1
      - TZ=Etc/UTC
      - SERIAL=00000002
      - FREQUENCIES=131.550;131.725
      - SERVER=acars_router
      - SERVER_PORT=5550
    tmpfs:
      - /run:exec,size=64M
      - /var/log

  acarsdec-2:
#    image: ghcr.io/sdr-enthusiasts/docker-acarsdec:latest
#    something broken in the latest image, so I hardcoded to this release:
    image: ghcr.io/sdr-enthusiasts/docker-acarsdec@sha256:75f825b55059750b4fbcecd3bb40b5cf2a4d00b230af078b55cc751f466737ea
    tty: true
    container_name: acarsdec-2
    restart: always
    devices:
      - /dev/bus/usb:/dev/bus/usb
    environment:
      - FEED_ID=FEED-2
      - TZ=Etc/UTC
      - SERIAL=00000001
      - FREQUENCIES=130.025
      - SERVER=acars_router
      - SERVER_PORT=5550
    tmpfs:
      - /run:exec,size=64M
      - /var/log


  dumpvdl2-1:
#    image: ghcr.io/sdr-enthusiasts/docker-dumpvdl2:latest
#    something broken in the latest image, so I hardcoded to this release:
    image: ghcr.io/sdr-enthusiasts/docker-dumpvdl2@sha256:f7b3eca26c53fe73e43919564890888ac3b0b18c3a70100f98af4aa4e1793bfd
    tty: true
    container_name: dumpvdl2-1
    restart: always
    devices:
      - /dev/bus/usb:/dev/bus/usb
    environment:
      - FEED_ID=VDL2-1
      - TZ=Etc/UTC
      - SERIAL=00000003
      - FREQUENCIES=136.975
      - ZMQ_MODE=server
      - ZMQ_ENDPOINT=tcp://0.0.0.0:45555
    tmpfs:
      - /run:exec,size=64M
      - /var/log

  dumpvdl2-2:
#    image: ghcr.io/sdr-enthusiasts/docker-dumpvdl2:latest
#    something broken in the latest image, so I hardcoded to this release:
    image: ghcr.io/sdr-enthusiasts/docker-dumpvdl2@sha256:f7b3eca26c53fe73e43919564890888ac3b0b18c3a70100f98af4aa4e1793bfd
    tty: true
    container_name: dumpvdl2-2
    restart: always
    devices:
      - /dev/bus/usb:/dev/bus/usb
    environment:
      - FEED_ID=VDL2-2
      - TZ=Etc/UTC
      - SERIAL=00000004
      - FREQUENCIES=136.650
      - ZMQ_MODE=server
      - ZMQ_ENDPOINT=tcp://0.0.0.0:45555
    tmpfs:
      - /run:exec,size=64M
      - /var/log

  acars_router:
    image: ghcr.io/sdr-enthusiasts/acars_router:20220603.1258
    tty: true
    container_name: acars_router
    restart: always
    environment:
      - TZ=Etc/UTC
      - AR_SEND_UDP_ACARS=acarshub:5550
      - AR_SEND_UDP_VDLM2=acarshub:5555
      - AR_RECV_ZMQ_VDLM2=dumpvdl2-1:45555
    tmpfs:
      - /run:exec,size=64M
      - /var/log

  acarshub:
    image: ghcr.io/sdr-enthusiasts/docker-acarshub:latest
    tty: true
    container_name: acarshub
    hostname: acarshub
    restart: always
    ports:
      - 8080:80
      - 15550:15550
      - 15555:15555
    environment:
      - TZ=Etc/UTC
      - ENABLE_ADSB=false
      - ENABLE_ACARS=external
      - ENABLE_VDLM=external
    volumes:
      - acarshub_run:/run/acars
    tmpfs:
      - /run:exec,size=64M
      - /var/log

Run ‘docker compose up’ to get things going.

So, this is my setup, and it runs 6 containers.

• 2 x ACARS Decoder (one for each SDR doing ACARS capture)
• 2 x Dump VDLM2 (one for each SDR doing VDLM2 capture)
• 1 x ACARS Router
• 1 x ACARS Hub

Here are some important things to note about each.

ACARS Decoder:

I had been using the latest image for ACARS decoder, but something broke after a update so I rolled back to the version that is hardcoded int the YAML. It may be fixed now. I don’t know.

Environment variables:

• FEED_ID=FEED-1, TZ=Etc/UTC — these can be modified to be whatever you’d like, I use UTC for everything. The Feed ID is useful for ID’ing which SDR picked up your message
• SERIAL=00000001 — this is where our rtl_eeprom work from earlier comes in. This should be set to the serial number of the SDR you wish to attack to the particular container instance.
• FREQUENCIES=131.550;131.725 — this is where you add the frequencies you wish to scan for ACARS messages. The exact frequencies you use here depend upon your area. These are the ones that work best for me. To find out more about ACARS frequencies — look at this.

Dump VDLM2:

Pretty much the exact same story as ACARS Decoder, the frequencies can be found for VDLM2 here.

ACARS Router:

Not much to configure here, this just acts as the go between for the decoders and the hub.

ACARS Hub:

This container does a couple of very useful things for us. It provides a web UI you can use to search through captured messages, if you browse to port 8080 on your container host, based on my config above, you’ll see that UI.

It’s a great way to get familiar with ACARS/VDLM2 messages, and start to understand their content, so you can decide what you want to look for.

Secondly, I’m exposing two additional ports, 15550 and 15555. Both of these ports stream UDP traffic containing the raw JSON objects used to populate the web UI, which makes them good for parsing and turning into Slack messages.

ACARS Hub has a great deal of useful info about the number of messages received by each feed, and on what frequencies etc. So it’s your go-to source for tuning your setup. If you don’t need to do the Slack bit, you could stop here and still have a nice way to do pretty much everything with these messages.

Step Three — Turning the ACARS/VDLM2 messages into Slack messages

To push the ACARS messages I’m most interested in to Slack, I did a couple of different things.

1. I configured a Slack App with an incoming Webhook to post to a specific Slack channel. This is something that is extremely well documented and can be found on the build/your apps section of the Slack UI.

2. I wrote a daemon process to listen to the UDP stream of messages produced by ACARS Hub and capture them to a log file for parsing.

Sounds complicated, but it’s really not. I made use of the classic hacker tool ‘netcat’ and a bash script that just keeps on listenin’. Bash script one listens to port 15550 for ACARS messages, and the second one does the same for VDLM2 messages on port 15555.

#!/bin/bash

while true; do
nc -d 127.0.0.1 15550 >> /home/mike/alerter/acars/log.txt
done

#!/bin/bash

while true; do
nc -d 127.0.0.1 15555 >> /home/mike/alerter/vdlm2/log.txt
done

Both of these bash scripts are named ‘listen’, and are placed in the /home/mike/alerter/acars/ and /home/mike/alerter/vdlm2 directories respectively. To turn them into services that run all the time, I created a ‘.service’ definition file for each, in the ‘/etc/systemd/system/’ directory.

acars.service is as follows:

[Unit]
Description=ACARS Monitor
[Service]
User=mike
#Code to execute
#Can be the path to an executable or code itself
WorkingDirectory=/home/mike/alerter/acars
ExecStart=/home/mike/alerter/acars/listen
Type=simple
TimeoutStopSec=10
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target

vdlm2.service looks like this:

[Unit]
Description=VDLM2 Monitor
[Service]
User=mike
#Code to execute
#Can be the path to an executable or code itself
WorkingDirectory=/home/mike/alerter/vdlm2
ExecStart=/home/mike/alerter/vdlm2/listen
Type=simple
TimeoutStopSec=10
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target

To switch both services on after creating these definitions, execute the following commands:

sudo systemctl daemon-reload
sudo systemctl enable acars
sudo systemctl enable vdlm2
sudo systemctl start acars
sudo systemctl start vdlm2

With these running, your log files at /home/mike/alerter/vdlm2/log.txt (or wherever else, assuming your name is not Mike), should start filling up with messages. Running ‘tail -f log.txt’ will confirm this.

Last but not least, we need a parser that can look at the generated log files and pull out the messages we want to send to our Slack app. I knocked together a very quick one in bash, but obviously this can be done however you want.

The only special/possibly non-standard utility I used in my bash is ‘jq’ which is a command line JSON parser. Check you have this installed if using my script.

In both my ‘acars’ and ‘vdlm2’ directories, I created parse.sh, which looks like this for ACARS:

#!/bin/bash

cd /home/mike/alerter/acars

ts=$(date +"%s")
echo "Starting Run $ts" >> output-acars.log

cp log.txt log-$ts.txt
> log.txt

sed -i 's/{\"ack/\'$'\n{\"ack/g' log-$ts.txt
sed -i 's/\"/\\"/g' log-$ts.txt

while read line; do


        msg=$(echo $line |grep -E 'FAIL|HOTEL|TAXI|THREAT|SECURITY|MAYDAY|MTCRP|DSPTCH|C314|THX|PAX|SNAG|INOP|EMERGENCY|"label":"81|"label":"83|"label":"84|"label":"85|"label":"87|"label":"88')
        reg=$(echo "$line"|/usr/bin/jq .tail --raw-output)
        flight=$(echo "$line"|/usr/bin/jq .flight --raw-output)
        txt=$(echo "$line"|/usr/bin/jq .text --raw-output|sed 's/[a-z]//g')
        
        if [ -z "$msg" ]
                then
                echo "nothing found in $reg / $flight / $txt / $line" >> output-acars.log
                else
                echo "Sent $flight $reg  $msg" >> output-acars.log
                curl -X POST -H 'Content-type: application/json' --data "{\"text\":\"ACARS Message From: $reg/$flight \nMessage: $txt\"}" https://hooks.slack.com/services/xxxxxxxx/xxxxxxxx/xxxxxxxx #your slack hook URL
        fi

done < log-$ts.txt

rm log-$ts.txt

And like this for VDLM2:

#!/bin/bash

cd /home/mike/alerter/vdlm2

ts=$(date +"%s")
echo "Start of Run $ts" >> output.log

cp log.txt log-$ts.txt
> log.txt

sed -i 's/{\"vdl2/\'$'\n{\"vdl2/g' log-$ts.txt
sed -i 's/\"/\\"/g' log-$ts.txt

while read line; do

 msg=$(echo $line |grep -E 'FAIL|HOTEL|TAXI|THREAT|SECURITY|MAYDAY|MTCRP|DSPTCH|C314|THX|PAX|SNAG|INOP|EMERGENCY|"label":"81|"label":"83|"label":"84|"label":"85|"label":"87|"label":"88')
 reg=$(echo "$line"| /usr/bin/jq .vdl2.avlc.acars.reg --raw-output|sed 's/.//')
 flight=$(echo "$line"|/usr/bin/jq .vdl2.avlc.acars.flight --raw-output)
 txt=$(echo "$line"|/usr/bin/jq .vdl2.avlc.acars.msg_text --raw-output|sed 's/[a-z]//g')

if [ -z "$msg" ]
  then
  echo "nothing found in $reg / $flight / $txt / $line" >> output.log
  else
  echo "Found: $msg - $reg - $flight" >> output.log
  curl -X POST -H 'Content-type: application/json' --data "{\"text\":\"VDLM2 Message From: $reg/$flight \nMessage: $txt\"}" https://hooks.slack.com/services/xxxxxxxx/xxxxxxxx/xxxxxxxx #your slack hook URL
 fi

done < log-$ts.txt

rm log-$ts.txt

So, what do these scripts do?

Well, they dump the contents of log.txt, which will be full of captured messages from ACARS Hub into a new file called log-$ts.txt (where $ts is the timestamp at the start of the run).

They then delete the contents of log.txt, so new messages can be added without being processed multiple times.

The sed commands are used to make sure that each received message starts on a new line, and is wrapped with double quotes for parsing cleanliness.

The file is then processed line by line, with the various values (msg, reg, flight, txt) populated based on the JSON contents.

Of note, the $msg variable is used to determine if the message found is ‘interesting’ enough to be sent to Slack. If it’s empty, nothing is sent. I’ve picked a bunch of keywords and labels from interesting messages that I grep for, this can be customized of course.

The final ‘if’ statement checks to make sure the $msg variable has content (i.e. it matched our ‘interesting’ values list), and if so, sends it via curl to the Slack Webhook created when you set up the Slack app earlier.

I then added both of these scripts to run every minute via the crontab.

# m h  dom mon dow   command
*/1 * * * * bash /home/mike/alerter/vdlm2/parse.sh
*/1 * * * * bash /home/mike/alerter/acars/parse.sh

And thats it. The result of all this is you get a very curated set of messages in the Slack channel from ACARS and VDLM2.

Pretty cool — will continue to tune and play as time allows. If you make your own ACARS to Slack rig, be sure to share any interesting messages you get.

If you aren’t able to make your own rig, but still want to explore ACARS messages, check out https://app.airframes.io, which is a neat project that crowd-sources and aggregates ACARS/VDLM messages from distributed antennas.