briarpig: thorn: demos: hex

demos are explained here; a menu at top column right indexes actual topic demos. Here we demo hex.

problem

As a debug aid Wil often prints raw memory dumps in hex. This is easily supported in the run representation of yv so Wil usually invokes hexdump printing from yv. This demo is basically an offshoot of the run demo devoted to hex dumping. (This demo is relatively short.)

Given a pointer to some memory and the number of bytes involved, you can make a new yv instance and hex dump the memory. Here's an example of hexdumping a 64 bit integer:

u64 num = 0x123456789abcdef0ULL; // « yv vnum(&num, sizeof(num)); // yout << vnum.quote() << yendl << ynow; vnum.vshow(yout, "vnum", /*max*/ 8192, /*base*/ 0); yout << ynow; // flush

As you can see from the output below, my laptop has a little endian processor and thus bytes of the integer appear

As seen in the run demo, yv::vshow() calls vhexmax() (cf «) and this demo is mainly about vhexmax() code listed below. But only one version will be shown. Wil has written many since the end of the 80's — this one is simpler than earlier versions (which were more concerned with minimal cost).

This hex dump code writes to the yo out stream api shown in the out demo. A typical work context variant usually uses fprintf() and writes to stderr; that one is trivially derived.

Let's start with some helper functions used by vhexmax(). But look for them at top column right. Code for leading offset prefix and trailing ascii comment is shown immediately at right.

hex

This version puts constants before the vhexmax() code, which is so simple you really ought to just step through under a debugger. But some explanation is offered anyway.

const unsigned khexcap = 48; // 16 per line « // const unsigned khexcap = 36; // 12 per line static const char yv_blanks[] = // 52 blank constant here: " "; //123456789_123456789_123456789_123456789_123456789_12

The khexcap constant limits hex capacity on each line; at three printed bytes per source byte, 48 is enough for 16 hexdumped source bytes per line. An alternative value of 36 is used only for our demos to make lines with fewer columns.

This yv_blanks constant is just a source of blanks for padding the format in vhexmax(). (The actual length is 72 and not 52 as shown here; this tweak shortens line length in this demo.) The comment is just an informal ruler so you can verify count of blanks by eye.

void yv::vhexmax(yo& o, u32 maxLen, u32 base) const { static const char* hex = "0123456789abcdef"; // digits « u32 line = yv::p32prefix(o, base); // length of current line u32 olen = line; // prefix offset length u8* p = v_p; u8* asc = v_p; // addr of last bytes not shown in ascii yet u32 limit = (maxLen < v_n)? maxLen : v_n; u8* end = p + limit; int c; for (/*prep preincr*/ --p; ++p < end ; ) { if (line >= (khexcap+olen)) { // 16 octets written so far? yv va(asc, p-asc); // last line to annotate in ascii va.vascii(o); line = yv::p32prefix(o, (p - v_p) + base); asc = p; // for next time: last octet not shown in ascii } c = *p; o.o3c(hex[(c>>4)&0xf], hex[c&0xf], ' '); line += 3; } if (asc < p) { // last trailing portion to write in ascii? if (line < (khexcap+olen)) { // need any blanks first? yv white(yv_blanks, (khexcap+olen)-line); o.ov(white); } yv last(asc, p-asc); // trailing fragment for ascii last.vascii(o); } if (limit < v_n) { // hit limit before yv was exhausted? o.ofn("# truncated after %d bytes; %d bytes more ignored", (int) limit, (int)(v_n-limit)); } }

In Wil's experience, structuring code to lazily notice a thing must be done sometimes results in simpler code. The handling of ascii annotation with vascii() falls in this category here. In the loop, once a line gets long enough, it's time to annotate and start a new line.

After the loop, if some ascii annotation is still pending because asc is less than the cursor, then one last annotation occurs. Everything else is pretty straight forward.

A submenu for demos appears below, letting you go to the page on a topic written as a demo (as the demos page defines it).

menu

thorn: todo, names, fd, iovec, assert, log, run, hex « Þ, crc, buf, in, out, quote, escape, compare, file, deck, cow, arc, blob, tree, slice, rand, time, stat, hash, heap, node, primes, page, book, pile, stack, atomic, lock, mutex, thread, map, meter, list, iter, ctype

(mu: toy, peg, imm, tag, box, symbol, token, number, bigint, class, method, reader, writer, eval, env, vm, gc, world, pcode, compiler, asm, lathe, lisp, smalltalk, design, weight, jar, card, harp, debug, profile)

Some demos are stubs: todo is a demo guide. See toy for mu updates on language pages; names introduces naming schemes.

helpers

A couple tiny helper methods are called from vhexmax(). (Listing them here helps balance columns on this page.)

u32 yv::p32prefix(yo& o, p32 pos) { // pos as hex prefix, return len « o.of("%05lx: ", (long) pos); return 7; // plan: 7 byte line prefix }

Each hex line is prefixed with an offset in hexadecimal and the format is decided here. The actual line prefix length is returned so vhexmax() can track whatever p32prefix() actually writes.

void yv::vascii(yo& o) const { // dump ascii "abc\x01" => "; abc.\n" « u8* end = v_p + v_n; o.occ(';', ' '); for (u8* a = v_p-1; ++a < end; ) { int c = *a; if (!isprint(c)) c = '.'; o.oc(c); // one octet ('.' if unprintable) } o.on(); // newline then indent to tab depth }

Hex is annotated by vascii() which assumes the encoding is Latin1 or whatever is understood by isprint() in ctype.h; the name vascii() is both shorthand and casual laziness.

base

The last argument to vhexmax() is the base offset used as the origin for printing hex offsets. The value of base defaults to zero, and you would only pass a nonzero value if you were showing a single byte sequence broken up into several pieces, and you still wanted to show the offset relative to the very start when hex dumping a current fragment.

This happens in the iovec demo when ydvp::pdump() shows all the iovecs comprising a larger sequence (cf «). Here's a reprise of yv::vshow() code from the run demo (cf «) since ydvp::pdump() calls it.

void yv::vshow(yo& o, const char* kind, u32 max, u32 base) const { « if (v_n) { yh32 h; h << *this; // yh32::hadd() in crc demo o.oftn("<%s p=%#lx n=%ld crc='%#lx:%lu'>", kind, (long) v_p, (long) v_n, (long) h.hcrc(), (long) h.hlen()); this->vhexmax(o, max, base); // see the hex demo o.ouend(kind); // out untab end (tag) } else o.of("<%s p=%#lx n=%ld/>", kind, (long) v_p, (long) v_n); }

dialog

"Why do you bother with this hex api anyway?" asked Stu.

"So I don't think about it," replied Wil. "I use it and I don't want to be bothered with detail. It's a time saver."

"What's the advantage of writing to yo?" wondered Stu.

"Mainly indentation," Wil downplayed. "When I print a hex dump inside some other object, this hex dump code indents correctly, so I can read the output more easily. It's trivial but quite helpful."

"Otherwise," Stu supposed, "you'd have everything flush with the left margin? Yeah, I hate that too."

Wil was nodding. "Yes, when I have to hex dump with fprintf() to stderr, there's no simple place to keep the current indent level, especially in threaded systems that might print concurrently."

"Are you saying the out streams are thread safe?" Stu doubted.

"No," Wil shook his head. "Not without locking. But a thread can have its own private out stream and then it's not an issue."

"How do you make yout to stdout threadsafe?" Stu asked.

"When I have to," Wil explained, "I use cooperative locking to make it threadsafe, when the stream is buffered. But no one has used my streams with threads in the workplace for a few years. Last time was around 2003. Worked fine then. I've no reason to expect problems."

"Most of my debug printing is single threaded," Stu noted.

"Yeah, that's normally the case," Wil agreed. "There's not much cause for debug printing to one stream from multiple threads. That's the kind of thing system logging is for — different problems."

demos « Þ