emCompress-ToGo — The on-target compression solution
emCompress-ToGo is a portable compression solution that compresses or decompresses data even on smallest microcontrollers.
Overview
emCompress-ToGo is a portable compression software, designed primarily, but not exclusively, for embedded systems. It can compress and decompress data with basically no RAM requirements.
The lossless compression of the emCompress-ToGo algorithm SMASH-2 is fast and efficient and can be done even on small embedded systems. There are very few, if any, other algorithms for compression on systems with constrained resources, especially those with limited RAM. Decompression is equally efficient.
Key features
- Compression and decompression on embedded systems
- SMASH-2 algorithm for use with minimal RAM requirements
- Optimized for the smallest of microcontrollers
SMASH-2
emCompress-ToGo uses SEGGER's SMASH-2 (Small Microcontroller Advanced Super-High compression scheme) algorithm to compress data. The proprietary SMASH-2 is an excellent all-around, tunable algorithm designed to provide fast and memory efficient compression and decompression.
SMASH compresses efficiently, decompresses quickly, requires only a few words of decoder state held in machine registers, and is very small. Compression is as efficient as other LZSS-based codecs. Decompression is far simpler than with these codecs, resulting in an ultra-small memory. The decompressor is also far simpler than in these formats and so also delivers an ultra-small memory footprint.
Compression ratios
SMASH-2 works well with virtually any kind of data. The table below displays the compression ratios for various use cases.
| Type of data | Window size | ||||
|---|---|---|---|---|---|
| 256 bytes | 512 bytes | 1024 bytes | 2 kB | 4 kB | |
| HTML file (324751 bytes) | 37.43% (121586 bytes) | 31.96% (103809 bytes) | 28.16% (91455 bytes) | 20.69% (67199 bytes) | 15.77% (51234 bytes) |
| Blog article (20895 bytes) | 49.22% (10285 bytes) | 44.14% (9224 bytes) | 39.99% (8356 bytes) | 36.77% (7685 bytes) | 34.20% (7147 bytes) |
| FPGA bitstream (114618 bytes) | 27.23% (31221 bytes) | 26.48% (30351 bytes) | 26.14% (29963 bytes) | 25.36% (29077 bytes) | 22.49% (25779 bytes) |
| MIDI file (56805 bytes) | 36.11% (20516 bytes) | 28.30% (16081 bytes) | 21.96% (12476 bytes) | 18.28% (10384 bytes) | 16.83% (9561 bytes) |
| Firmware (218824 bytes) | 48.22% (105530 bytes) | 44.04% (96381 bytes) | 41.80% (91472 bytes) | 40.53% (88703 bytes) | 39.85% (87202 bytes) |
| FirmwareHEX (615547 bytes) | 40.68% (250424 bytes) | 37.67% (231885 bytes) | 35.45% (218245 bytes) | 34.13% (210121 bytes) | 33.48% (206115 bytes) |
Performance
Compression performance and time increase as window size increases. Running on an Arm Cortex-M7 CPU clocked at 200 MHz and using RAM to RAM compression with a 256-byte window, the compression performance is about 400 kB/sec. An enhanced function that requires an additional work buffer in RAM will achieve about 1500 kB/sec compression speed.
Decompression performance is largely independent of the parameters, so that the window size does not have a major effect on the decompression performance. Similar to the performance numbers for compression, an Arm Cortex-M7 CPU clocked at 200 MHz has been used to measure the decompression performance. Using RAM to RAM decompression, a speed of 6.3 MB/sec is achieved with respect to the compressed input data stream, which corresponds to about 13 MB/sec for the decompressed output data stream. Using the stream interface, the decompressor will process an average of 4.3 MB/sec input data while generating 9 MB/sec uncompressed output data.
Resource usage
In most cases, no RAM is required. Since SMASH works with references, it uses the uncompressed part as a “window”. In the case of stream compression, blocks of 2 KB can be processed at a time. In some cases smaller blocks can be processed, down to as small as just 256 bytes.
ROM requirements depend on the option used for compression and decompression. emCompress-ToGo provides four options: from memory to memory (M2M), from streaming function to streaming function (F2F), from streaming function to memory (F2M), and from memory to streaming function (M2F).
Compression
| Function | ROM | RAM (client) | RAM (stack) |
|---|---|---|---|
| CTG_CompressM2M() | 1095 bytes | 0 bytes | 80 bytes |
| CTG_CompressM2F() | 1099 bytes | 0 bytes | 80 bytes |
| CTG_CompressF2M() | 1381 bytes | 514 to 16642 bytes | 160 bytes |
| CTG_CompressF2F() | 1403 bytes | 514 to 16642 bytes | 192 bytes |
| CTG_Compress() | 1275 bytes | 514 to 16642 bytes | 64 bytes |
| CTG_FastCompressM2M() | 1200 bytes | 768 to 33280 bytes | 100 bytes |
Decompression
| Function | ROM | RAM (client) | RAM (stack) |
|---|---|---|---|
| CTG_DecompressM2M() | 370 bytes | 0 bytes | 80 bytes |
| CTG_DecompressM2F() | 630 bytes | 256 to 16384 bytes | 160 bytes |
| CTG_DecompressF2M() | 394 bytes | 0 bytes | 80 bytes |
| CTG_DecompressF2F() | 668 bytes | 256 to 16384 bytes | 192 bytes |
| CTG_Decompress() | 536 bytes | 256 to 16384 bytes | 64 bytes |
Use cases
emCompress-ToGo enables compressing data on the smallest of embedded systems. While most common compression codecs require lots of RAM for compression, emCompress-ToGo can be used for compression and decompression with less than 1 kilobyte of RAM.
Saving resources on data logging
The amount of data collected and stored by data logging makes this a prime application for compression. emCompress-ToGo enables the storing of compressed data logs, saving storage requirements and memory resources.
Reduce IoT network traffic
Networks of connected IoT devices can suffer from congestion as more and more devices communicate with each other, with brokers, or with the Internet. With low power communication channels and mesh setups the available network bandwidth for each device is limited. To efficiently use the network, messages should be kept as small as possible.
With emCompress-ToGo messages to be sent on a network can be compressed and decompressed by the sender and receiver on the fly, resulting in less traffic on the network, reduced bandwidth requirements and in many cases lower power consumption for senders, receivers and relays.
Speed up long-range communication
In space and avionics applications, such as satellites, communication channels are not only restricted by bandwidth, but by the distance that data has to be transmitted. In long-distance communication, data packets can get lost more easily and require retransmission.
emCompress-ToGo can compress any communication data to reduce the number of packets to be transmitted, reduce retransmissions due to lost data and therefore speed up long-range communication significantly.
Learn more at:
https://casestudies.segger.com/using-emcompress-to-connect-machines-by-satellite-anywhere-anytime/
Media gallery
emCompress-ToGo — Intro | 5:36 min
emCompress — Example application | 2:43 min
View more...Evaluation packages
Compression and decompression with SMASH can be evaluated on a Windows PC with the tools of the software package available for download. It includes a compressor tool and a decompressor tool and the data files we used in the compression table previously mentioned.
Running ShowStats.bat
The software package includes a batch file, ShowStats.bat, which compresses the included data files. You can compare the compression results of the different files with the different window sizes being used.
You can also compress and decompress your own files to test the efficiency of the SMASH-2 compression algorithm and to find the best balance of compression ratio and window size (i.e. memory requirement) for your use cases.
Evaluation on a microcontroller
To evaluate the use of emCompress-ToGo in an embedded application, a trial package for Embedded Studio is available for download. It includes a project to test compression and decompression in Embedded Studio's simulator. It can also easily be configured to run on actual hardware.
Running the Simulator Project in Embedded Studio
Running the emCompress-ToGo demo project will generate some output on the terminal that provides the data of the compressed or decompressed file in different compression modes. emCompress-ToGo can run compression and decompression from memory to memory, from function to function, from memory to function and vice versa. All four resulting combinations are tested in the simulation project.
PC evaluation software package content
| File | Description |
|---|---|
| ShowStats.bat | Windows batch file that runs Smash.exe with window size parameters 8 to 12 for the input files mentioned below (Blog.txt, Firmware.bin, Firmware.hex, FPGA.bin, Index.htm, MIDI.mid) |
| Smash.exe | Windows executable of SMASH compressor |
| Unsmash.exe | Windows executable of SMASH decompressor |
| Blog.txt | Text content of "What's cooking at SEGGER" blog |
| Firmware.bin | Binary file of a firmware image for a microcontroller |
| Firmware.hex | Hex file of above mentioned firmware image |
| FPGA.bin | FPGA bitstream |
| Index.htm | HTML code taken from the SEGGER website |
| MIDI.mid | Standard .midi file |
To get started, follow these steps:
- Open Start.emProject with Embedded Studio.
- Select "Decode_Release" for testing the decompressor or "Encode_Release" for testing the compressor as the active build configuration.
- Build the project.
- Start a debug session and let it run.
- The debug terminal will show the compression or decompression results of the example application.
FAQ — Frequently asked questions
Q: Can the software be used for streaming data?
A: Yes. This is easily done. It actually comes with a sample showing how to do this.
Q: Can I use the software to compress data that I want to store in files?
A: Yes, no problem.
Q: I am storing a large file. Is there a way to "fast forward" in the file, to extract a portion of it without having to extract the whole file?
A: Yes, this is easily done by chunking the output and prefixing every chunk with the chunk size (compressed and uncompressed). The software comes with a sample showing how to do this.
Q: Can I test the software?
A: Sure, a Windows executable can be downloaded which compresses a file.
Q: Do I have to select parameters for compression?
A: No. There is basically only one parameter and that is the size of the window. This affects the amount of uncompressed data that needs to be in memory at any given time, as well as the compression speed.