Audio decoding

[Rabbe Fogelholm]

I am not very skilled at reading CW. It was fun to work out the content of the early 2020-07-05 transmission, but it took me quite some number of listenings…

I also looked around for audio decoding software, but I didn’t find any that was easy to operate and producing good results. So I ended up writing an audio decoding program myself. My program does not decode an incoming stream on-the-fly, instead it analyses a complete WAV recording. It locks in on the audio frequency, applies mild clipping, adapts to fading, and uses a sliding window to discriminate tone from silence. The message then emerges quite cleanly:

cq cq cq de saq saq saq = this is grimeton radio/saq in a transmission using the alexanderson 200 kw alternator on 17.2 khz. = in view of the present pandemic covid-19 we want to pay tribute to all parties concerned within healthcare knowing their efforts will pay effect eventually . = signed: world heritage grimeton radio station and the alexander[.-....-]grimeton veteranradios vaenner association + = for qsl info please read our website: www.alexander.n.se = + de saq saq saq <SK>

The audio frequency is reported as 690 Hz. With the SAQrx VFO set to 16500 Hz this suggests that the alternator would be running at 17190 Hz? Although there may be inaccuracies introduced along the way: The SAQrx program, the Audacity program for recording a WAV file, and the decoding program itself.

By counting inter-character spaces as 3 dot-time-units and inter-word spaces as 7, the average time per dot comes out as 79.93 ms, which gives a WPM of 1200/79.93 = 15.01. If the intention was to send at 15 WPM then the early transmission was an amazing accomplishment!

I can make the decoding program available if there is interest.

Topic

[Rabbe Fogelholm]

Until now the detection of dots and dashes in gerke-decoder has relied on finding the points in time where “silence” changes to “tone” and back. This method works fine with strong signals, but it is sensitive to noise: The timing gets disturbed, false dots may be detected, true dots may be missed, dashes may be misinterpreted as dots, etc. All of this causes the decoded text to be garbled.

There is a new version 3.2.3 of the decoder that doesn’t rely on tone/silence detection at all. Instead it computes integrals of the signal over short time intervals, thereby locating what are the most likely dots and dashes. This produces better results than previous versions, especially so for recordings with considerable noise.

Try it out if you like: Build from source at https://github.com/fowlay/gerke-decoder/ or use this executable jar: https://privat.bahnhof.se/wb748077/radio/3.2.3/gerke-decoder.jar

If you notice problems, let me know.

[rich.g.williams]

This is very interesting, I have seen gerke-decoder mentioned on other forums and realise now that you wrote it back in 2020. Mathematically it very interesting with the integration approach. Have you experimented with cross-plotting interval times (on a test basis) to give a graphic representation? Do you assign a statistical probability to dots, dashes, spaces?

I only started receiving SAQ last year and when I did my first decode was reading the screen and using pen and paper to work out each letter. This last decode on Alexanderson day was faster, perhaps because I knew what was coming and could guess what words were after two or three letters.

As I understand it (and have read), that to turn on and turn off transmission of the 17.2 kHz carrier takes time so SAQ so as you mentioned previously “the intention was to send at 15 WPM then the early transmission was an amazing accomplishment” !!

In the old days (I’m talking about the 1960’s) Short wave radio was used extensively, Morse speeds could get quite high even over 15 WPM. I think 20 WPM was not uncommon.

I was really interested in amateur radio and wanted to get a licence but passing the RSGB Morse test was a problem for me. As a young teenager there was getting to the exam, there was how to practice. I felt that I would never learn to transmit and receive Morse at the 12 WPM needed for the exam – so regretfully I left it back then and got interested in sound systems and microprocessors instead.

[rich.g.williams]

“The integrating decoder currently works like this: It looks for dashes by integrating the signal multiplied by a dash-like folding function over a time interval that is slightly longer than one dash, thereby sensing the edges as well as the bulk of the dash. This procedure is repeated at a great number of points along the entire telegram, of the order every 10 ms”

That’s very interesting the “dash-like folding function” sounds like Fourier or convolution/deconvolution mathematics.

For the sake of interesting discussion there are many algorithms in pattern recognition etc that look for patterns (or images) regardless of exact dimensions. It could be possible using your approach to look for letters rather than individual dots/dashes. It could even be possible to look for words then you are soon getting into the realms of predictive text and using AI chips or software to do the job.

“However, my impression is that the ear and brain is unbeatable when the signal to noise ratio is poor” That’s interesting, That got me thinking about why? In part there is a lot of redundant information in a Morse code stream, people know what’s coming next and what makes sense. Ideally to test your software (or any future versions) a stream of random morse would be useful and then compare efficiency of your algorithm versus human Morse reader.

So one practical and probably useful idea that occurred is as follows:- Consider receiving the SAQ carrier window at 17200 Hz and also one or two nearby windows say windows at 17000 Hz and a window at 17400 Hz with say a 100Hz bandwidth. Now noise is broadband so you can expect the noise pulses to appear in all these three windows. Whereas the SAQ carrier will not appear in the other two windows. This gives a way to weight the validity of the SAQ Morse stream and then either to subtract the noise pulse, or to define an extra condition (Carrier, No carrier or Noise), or the noise information could be an input to your “folding function over a time interval” approach.

Well done interesting work!

[Rabbe Fogelholm]

The integrating decoder currently works like this: It looks for dashes by integrating the signal multiplied by a dash-like folding function over a time interval that is slightly longer than one dash, thereby sensing the edges as well as the bulk of the dash. This procedure is repeated at a great number of points along the entire telegram, of the order every 10 ms. Since the length of an individual dash varies, this is then redone for a range of possible dash lengths.

The integrals with the highest scores point out the likely dashes. Once a dash has been located, many nearby candidates can be dropped as they would overlap.

The same procedure is then used to find the likely dots, using a shorter integration interval and a dot-shaped folding function.

Some dash and dot candidates will overlap, especially when a dash is a bit weak in the middle. A choice is then made, by comparing the strength of the dash candidate to the strengths of the two candidate dots. It turns out that the strength ratio limit is a quite sensitive parameter – either many “t” will come out as ”i”, or the other way around, if the limit value is not right. I tweaked the limit by testing against a set of SAQ recordings of varying quality.

I was happy to find that the integrating scheme seems to do better than the previous tone/silence edge-finding schemes. However, my impression is that the ear+brain is unbeatable when the signal to noise ratio is poor. And the brain does it with no floating-point arithmetic whatsoever!

[Rabbe Fogelholm]

Version 3.1.5 is available. The handling of unexpectedly long dashes is improved. Such “super-dashes” can appear if the dip between two tones becomes too short or shallow. Super-dashes are now interpreted as “dot dash”, “dash dot” or “dash dash”, depending on timing and position of the dip.

[Rabbe Fogelholm]

Version 3.1.1 is available. A new option -T has been added, controlling the output style (lower case, upper case, or capitalized, and line length). Furthermore .mp3 files can now be given as input; this requires the ‘mpg123’ (www.mpg123.de) conversion utility – use your host system package manager to install it.

[Rabbe Fogelholm]

There is now a new version of gerke-decoder: 3.1.0. This version fixes a coding fault – now decoding works well also in case there is a period of silence at the beginning of the recording.

A new option -M has been added, whereby histograms of tone lengths and space lengths can be drawn. Have a look here: https://privat.bahnhof.se/wb748077/radio/2023-02-13-grimeton/

As before, the program source and build instructions are here: https://github.com/fowlay/gerke-decoder

Building from source is straightforward on Linux. Alternatively you may use this pre-built executable jar: https://privat.bahnhof.se/wb748077/radio/3.1.0/gerke-decoder.jar – you simply run it with “java -jar gerke-decoder.jar”

Comments, questions and suggestions are welcome! You can reach me in this forum, or on github.

[Rabbe Fogelholm]

It is sad to hear that the Old Lady is not fit to sing out over the oceans this summer. Let’s hope for a steady and fast recovery.

Analyzing old recordings can be done anytime though! There is now a version 3.0.0 of the gerke-decoder program, where adaptivity against frequency drift and fading has been added.

The program expects input in .wav format and prints out the Morse code message as cleartext. Information about audio frequency, words-per-minute rating, timing of inter-character spaces, etc can be printed too. The signal can be displayed graphically for studying dashes and dots shapes, garbled characters and such.

Recordings in .mp3 format work well also, after converting to .wav format with some tool like Audacity.

By default the program listens for audio in the comfortable audible range, but a different frequency range can be specified – even up at 17200 Hz if a recording of the raw antenna signal has been made.

Find the program here: https://github.com/fowlay/gerke-decoder

Building from source is straightforward on Linux. If you don’t have a Linux box you may instead use this pre-built executable jar: https://privat.bahnhof.se/wb748077/radio/3.0.0/gerke-decoder.jar

Comments and ideas are always welcome.

[Rabbe Fogelholm]

The UN Day SAQ transmission is approaching – so what could be a better time for an updated version of “gerke-decoder” 🙂

The latest addition is a decoder routine based on least-squares fitting of straight line segments to the signal data, thereby locating the dashes and dots. My hope is that this could be a bit more noise insensitive than the other decoders, which all look for on/off transitions of the signal in one way or the other.

As before, try it out if you like: https://github.com/fowlay/gerke-decoder – if you have questions I will try to help, here or on github.

[Rabbe Fogelholm]

I have uploaded a new version (2.0.1) of gerke-decoder. More filtering and decoding options are available, please see the README.md file. If you recorded one of the recent SAQ transmissions it may be fun to try and decode it. Any feedback welcome, here or on github: https://github.com/fowlay/gerke-decoder

[Rabbe Fogelholm]

I have uploaded the decoder program to GitHub: https://github.com/fowlay/gerke-decoder – comments/questions/suggestions welcome.

By default the decoder listens for pleasant audio in the range 400-1200 Hz, but different ranges can be chosen. A minimalist setup would be to record the antenna signal from the microphone jack with e.g. Audacity, dump it to a .wav file and then use the decoder set for, say, 16000-18000 Hz. As yet untested though…

[Whitham Reeve]

Hello – I am very interested in the audio decoding program. I receive SAQ transmissions in Alaska and they are very weak, and my CW skills are pretty small. Please contact me at whitreeve@gmail.com. Thanks!

[Timo Salo]

Please some stations, try to learn how RST-code works!!! There was lots of impossible reports… Exc. report 333 was not real, tonequality was perfect!!!

Check:https://en.wikipedia.org/wiki/R-S-T_system

Best 73`s OH3DF Tim, licenced since 1975

[Author]

Replies