Download xcd-1.2.tar.gz
Github site

A colorized hexdump tool

The program xcd is a simple variation on the standard hexdump utility xxd(1) that also colorizes its output. It assigns different colors to individual byte values, which assists in seeing patterns of repeated values. This is mostly useful in looking at data streams that contain a limited set of values.

Note that this program must be used in a terminal that is capable of displaying 256 colors. Assuming you're using an appropriate terminal program, this typically means setting your TERM environment variable to xterm-256color.

Here's an example comparing the output of xxd and xcd side by side for a data stream containing a clear pattern of byte values:

00000000: 0000 0100 0100 2020 1000 0000 0000 E802 ...... ........ 00000000: 0000 0100 0100 2020 1000 0000 0000 E802 è
00000010: 0000 1600 0000 2800 0000 2000 0000 4000 ......(... ...@. 00000010: 0000 1600 0000 2800 0000 2000 0000 4000 (@
00000020: 0000 0100 0400 0000 0000 8002 0000 0000 ................ 00000020: 0000 0100 0400 0000 0000 8002 0000 0000
00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000030: 0000 0000 0000 0000 0000 0000 0000 0000
00000040: 0000 0000 8000 0080 0000 0080 8000 8000 ................ 00000040: 0000 0000 8000 0080 0000 0080 8000 8000
00000050: 0000 8000 8000 8080 0000 8080 8000 C0C0 ................ 00000050: 0000 8000 8000 8080 0000 8080 8000 C0C0 ÀÀ
00000060: C000 0000 FF00 00FF 0000 00FF FF00 FF00 ................ 00000060: C000 0000 FF00 00FF 0000 00FF FF00 FF00 Àÿÿÿÿÿ
00000070: 0000 FF00 FF00 FFFF 0000 FFFF FF00 00B0 ................ 00000070: 0000 FF00 FF00 FFFF 0000 FFFF FF00 00B0 ÿÿÿÿÿÿÿ°
00000080: 0F0F 0F09 0909 0909 0909 0909 0909 0000 ................ 00000080: 0F0F 0F09 0909 0909 0909 0909 0909 0000
00000090: F0F0 F0F0 0000 0000 0000 0000 0000 0FF0 ................ 00000090: F0F0 F0F0 0000 0000 0000 0000 0000 0FF0 ððððð
000000A0: 0F0F 0F09 9999 9990 00FF FFFF FF00 0FFF ................ 000000A0: 0F0F 0F09 9999 9990 00FF FFFF FF00 0FFF ÿÿÿÿÿ
000000B0: 00F0 F099 9BB0 009F 000F F00F F00B 0FF0 ................ 000000B0: 00F0 F099 9BB0 009F 000F F00F F00B 0FF0 ðð°ððð
000000C0: F00F 0999 BB0B B099 F000 FFFF 00B0 0000 ................ 000000C0: F00F 0999 BB0B B099 F000 FFFF 00B0 0000 ð»°ðÿÿ°
000000D0: 00F0 999B B0B9 9B09 9F90 0FF0 0B99 00B0 ................ 000000D0: 00F0 999B B0B9 9B09 9F90 0FF0 0B99 00B0 ð°¹ð°
000000E0: 0F00 99BB 00B9 9B09 99F9 0000 B000 00B0 ................ 000000E0: 0F00 99BB 00B9 9B09 99F9 0000 B000 00B0 »¹ù°°
000000F0: F009 99B0 00B9 9B00 999F 900B B999 00B0 ................ 000000F0: F009 99B0 00B9 9B00 999F 900B B999 00B0 ð°¹¹°
00000100: 0F09 9BB0 0F0B B0F0 0999 F90B 0000 00B0 ................ 00000100: 0F09 9BB0 0F0B B0F0 0999 F90B 0000 00B0 °°ðù°
00000110: F099 9B00 F000 0B0F 0099 F90B 9999 00B0 ................ 00000110: F099 9B00 F000 0B0F 0099 F90B 9999 00B0 ððù°
00000120: 0999 BB00 00BB BBB0 F009 9F0B 0000 00B0 ................ 00000120: 0999 BB00 00BB BBB0 F009 9F0B 0000 00B0 »»»°ð°
00000130: 0999 B00F 0BB0 B0BB 0F09 9F00 B990 00B0 ................ 00000130: 0999 B00F 0BB0 B0BB 0F09 9F00 B990 00B0 °°°»¹°
00000140: 000B B000 0BBB BBBB 0F00 99F0 0B00 00B0 ................ 00000140: 000B B000 0BBB BBBB 0F00 99F0 0B00 00B0 °»»»ð°
00000150: 0000 00F0 BBB0 B0BB B0F0 99F9 00B0 00B0 ................ 00000150: 0000 00F0 BBB0 B0BB B0F0 99F9 00B0 00B0 ð»°°»°ðù°°
00000160: 0000 0000 B0BB BBB0 B0F0 099F 900B 0000 ................ 00000160: 0000 0000 B0BB BBB0 B0F0 099F 900B 0000 °»»°°ð
00000170: 0000 9F0B BBB0 00BB BB0F 0999 FF90 0BBB ................ 00000170: 0000 9F0B BBB0 00BB BB0F 0999 FF90 0BBB »°»»ÿ»
00000180: 0000 9F0B B00B 0B00 BB0F 0099 9900 0BBB ................ 00000180: 0000 9F0B B00B 0B00 BB0F 0099 9900 0BBB °»»
00000190: 0000 9F0B 00B0 00B0 0B00 F000 9000 0000 ................ 00000190: 0000 9F0B 00B0 00B0 0B00 F000 9000 0000 °°ð
000001A0: 0000 0F00 0B0F FF00 B0B0 0FFF 0000 0BBB ................ 000001A0: 0000 0F00 0B0F FF00 B0B0 0FFF 0000 0BBB ÿ°°ÿ»
000001B0: 0000 09F0 00FF 0FF0 B000 F999 0000 0000 ................ 000001B0: 0000 09F0 00FF 0FF0 B000 F999 0000 0000 ðÿð°ù
000001C0: 0000 09F0 00F0 00FF 0B0F 9990 0000 00B0 ................ 000001C0: 0000 09F0 00F0 00FF 0B0F 9990 0000 00B0 ððÿ°
000001D0: 0000 099F 0FFF FFFF F0F9 9900 0000 00B0 ................ 000001D0: 0000 099F 0FFF FFFF F0F9 9900 0000 00B0 ÿÿÿðù°
000001E0: 0000 009F 0FF0 00FF F0F9 9900 0000 0BBB ................ 000001E0: 0000 009F 0FF0 00FF F0F9 9900 0000 0BBB ðÿðù»
000001F0: 9000 009F 0FFF 0FFF 0F99 9000 0000 0B09 ................ 000001F0: 9000 009F 0FFF 0FFF 0F99 9000 0000 0B09 ÿÿ

The program also takes advantage of Unicode support to display more of the byte values in the far-right column.

Colors are selected by xcd to maximize contrast (assuming the standard xterm 256-color palette). The selection of which colors actually maximize contrast with each other was based on the works of several design researchers, most notably:

The former work cites several other researchers' color sets, including the latter. I used Kelly's color set as a starting point, modified to match the standard 256-color xterm palette. (I also removed several standard colors such as pure black and pure white to avoid collision with the terminal's normal colors.) I used a mixture of Green-Armytage's other recommendations to extend this color set to 27 colors, and then extended this set to 243 by assigning the remaining colors in the xterm palette so that colors with maximal overall color distance are used first.

The source code also provides a simple example of using the tinfo library to portably produce colored output without having to use the ncurses API. This is necessary for programs to remain well-behaved when piped through a pager, for example. (Though please note that the the –r option is required when using less(1), for the terminal control codes to work correctly.)

You can see an example of how this tool can be used in my essay Bootstrapping Understanding: An Introduction to Reverse Engineering.

The code in this distribution is made available under the MIT license. Share and Enjoy. Questions and comments should be directed to me at breadbox@muppetlabs.com.


Software
Brian Raiter