To unify treatment of output streams written with formatted content, Wil wants one api to write buffered i/o streams: yo is an abstract api he uses for this out-stream purpose. (A taste of yo's api appeared in the buf demo: ybo in column right.)
Because yo out-streams resemble yi in-streams, some parts of this page may loosely copy parts of the in demo, usually with all new wording to avoid redundancy (to make comparison useful).
Even though yo is the base class for all out-streams, it has a base class declaring all virtual methods used by yo. This origin base class adds zero suffix to say "origin"; it looks like this:
The same comments from the in demo about seek also apply here: the out-stream api has methods assuming random access is supported even though not all streams can seek. Streams that cannot seek will be unable to oseek(), oput(), or opwrite() and calls to these methods will fail — gently if possible, with log messages. So the base api over-promises features not all subclasses can support. Why is this a good idea?
Wil once tried (ten years ago) to define a set of orthogonal base classes for streams breaking features into consistent groups that could be gathered together as needed in base classes. The result was quite complex even though acceptably precise, and matters only became worse when Wil added support for slice expressions because possible slice types multiplied rapidly.
Even worse, an awkward form of type mismatch pressure in class apis arose. Wil compared this awkwardness to the use of const keywords in method and parameter types — you must use const carefully and transitively to stay correct, and const only involves one dimension. It's worse with more dimensions. In the end Wil decided too many orthogonal base classes were a mistake in the absense of pure interfaces with duck typing.
Trying to explain to himself why this happened, Wil felt a principle was involved: conservation of complexity. Some issues don't go away when you fold them differently: they just redistribute complexity in new ways — easily worse ways. So a problem that some streams can't seek doesn't get simpler than just saying those methods might not work on some types. Accepting that contract makes the api much simpler.
Wil defines the api so it suits a duck typing style of code: you can ask any stream to seek but it might be unable to do so, and you can arrange to avoid passing streams that cannot seek to places where seek is needed. (Among other things, this also resembles the way file and socket descriptors are used in the C runtime: an integer descriptor doesn't say whether a stream is able to seek or not, but you can still stay out of trouble.)
The yo0 base class provides a few simple defaults and trivial alternative methods calling standard virtual ones.
You needn't override oseek() and opwrite() unless your subclass needs random access. But nearly all subclasses need to override _oc() and owrite() since those are the main ways in which one writes single and N bytes at a time. Method owrite() resembles a write() system call, and _oc() is a protected part of oc() (explained below) which resembles fgetc().
You can see owritev() used in the iovec demo (cf «) to write all the iovecs in an array. Some subclasses can do this more efficiently; for example, yfo writing to a file descriptor can use a writev() system call. All these methods above are just shallow facades over owritev() or opwritev().
A main purpose of yo is to have the first triple (o_0, o_p, o_x) of pointers shown, describing a current buffer being written by an out-stream, so inlines like oc() can directly write octets without a virtual function call. As long as pointer o_p has not yet reached max o_x (one byte past the last in the buffer) then another byte can be written without need for more space via virtual flush, amortizing infrequent virtual methods across many octets; cheap cost to write octets is a main design motivation. As long as an out-stream is buffered, the space used can be buffered in this way.
An unbuffered stream can simply make the triple of pointers all equal to zero, indicating an empty buffer since o_p is then never less than o_x; a buffer is exhausted when o_p equals o_x.
As shown earlier in the buf demo where out-stream subclass ybo appears (cf «), each buffered out stream subclass uses triple (o_0, o_p, o_x) to create the following relation with respect to a buffer staging content being written to the stream:
As described in run and buf demos, the yb triple of members (v_p, v_n, b_x) mean the following:
- v_p: address of first byte in contiguous memory block
- v_n: number of consecutive bytes of logical content
- b_x: max physical capacity of buffer in bytes
Buffers are written from left to right, so bytes to the left of o_p are written and bytes to the right of o_p are yet to be written. Present length is o_p-o_0 and remaining space is o_x-o_p.
The constructor zeroes everything, and all the other methods above make error and log messages easier to generate. The o_e member is a place to store an error result, usually based on errno by convention. Without a place to capture errors in deeply nested i/o layers (say when wrapping a zlib.h object as an out-stream) it can be hard to preserve error conditions until better places to notice.
Method ogood() tests a yo instance for compliance to the main invariant of yo: that triple (o_0, o_p, o_x) is ordered as expected. A failure of this invariant can cause catastrophic memory corruption, so asserting when ogood() fails is a good idea.
The following bulk i/o api was first shown in the buf demo in the ybo implementation of otake() (cf «). The explanation below includes less context, so consult the reference above for amplification.
(Why are these pure virtual methods declared in yo's api instead of base yo0's api with all other out-stream virtual methods? Because otake() and ogive() know out-streams are buffered and — except for flushing — the idea of buffering appears in yo and not yo0. They aren't needed in yo0 to let a subclass of yo implement a generic buffered out-stream; the buffer gets added by the yo subclass.)
These two methods are pure virtual (requiring a subclass override) to force each out-stream class to consider this api a little; this is better than stubbing them out with do-nothing versions in yo which are easily ignored. This educates subclass writers a bit more.
This api lets you write directly into an out-stream's buffer space, optimizing reads from an in-stream (like a socket) and writes to a destination out-stream by not using an intermediary temp buffer. (But note: using temp buffers is simpler and thus often better in early versions of code, so you might delay use of otake().)
Normal usage: just call otake() first to get a description of the remaining buffer space returned in the yb buf parameter. The value returned by otake() is the number of bytes in the buffer — the same value as member dest.b_x in the buffer.
Next a caller writes up to dest.b_x bytes to address dest.v_p, putting the actual number of bytes written in dest.v_n to show actual length of buffer content. Finally a call to ogive() updates the buffer, differing from the original returned from otake() only in the value of v_n which must not exceed b_x; any other buffer change must fail, and an assert is even better.
Unless an out-stream runs completely out of space to hold more content, otake() should never return zero bytes of available buffer space. Instead if the current buffer is exhausted, otake() can flush the buffer and make it empty again, if a stream can hold more.
The o_take member variable is specifically intended to hold a copy of the buffer returned from otake() so ogive() can assert only v_n changed in the given buffer. By convention, o_take is also used to track whether otake() has been called and ogive() not yet called, since then an out-stream is basically locked (by convention) and cannot be used by any other out-stream method — doing so should signal an error in ogive(); only one otake() based transaction can be pending at a time. When an out-stream writes to fixed, pre-planned state (such as iovec vectors) otake() should move the (o_0, o_p, o_x) triple to next available space when a current buffer is empty.
Ideas in oc() below resemble those for byte i/o in yi in-streams using ic() (cf «). Method oc() is the reason yo exists: to write octets as cheaply as possible until a buffer is exhausted, after which protected _oc() refills a buffer (if possible) and tries again.
Like yi::ic(), this particular api is one of the oldest in þ (cf «) — dating to around 1993 — and has been used in many projects since. It was inspired by poor performance in Taligent i/o frameworks, which made Wil question whether virtual function calls were always necessary in the presence of abstraction. Both yi and yo are canonical examples of design exposing private state in a controlled way for speed, while still using virtual methods to keep options in subclasses.
But speed comes at the cost of fragility associated with changing the least amount of state when writing a single byte. Only a change in o_p's location shows another byte was added. This requires a lazy coding style in other out-stream methods, since they must check o_p's current position to maintain coherency.
Because oc() is inline and presumably generates more code at each call site than a non-inline method, you might sometimes use o1c() below when smaller seems more important than faster. Many yo inlines prefer using o1c() to oc().
The last three methods above write one more byte than each preceding, directly expressing a thing you might do, where otherwise you'd turn to a slower printf() style method.
Beyond virtual methods (performing the bulk of actual i/o) most of the yo out-stream interface is utility methods ultimately calling either oc() or owrite() to write content a byte at a time or one iovec at a time. The main purpose of yo's large api is reducing effort required to print objects using dump style debug printing methods shown in all the demos. Printing objects is a tedious business at best.
After years of writing zillions of object printing methods, Wil became tired of using several different method calls to get one specific boilerplate effect. So instead Wil added a method for each of the sorts of situations that ever arose when printing part of an object using his typical pseudo XML style of debug printing. So many util methods catenate several simpler yo methods, with names concatenating shorter names.
Several years' worth of earlier out-stream revs had longer method names until Wil finally got fed up with trying to recall exactly which long method name did what — especially when some classes took longer to print than it did to write all the other methods. Printing methods were aggravating, so Wil simply became cryptic to save time.
Except Wil can actually remember these names, so it's apparently less cryptic when the naming scheme is known. So let's go over the one-letter naming scheme used below. A handful of methods contain a word or syllable from an English word, but most methods are composed of letters standing for words. The leading o each time stands for out of course. Here's a legend for the rest:
- t - tab: indent one more on next newline
- u - untab: indent one less on next newline
- n - newline: end line then indent to tab depth
- f - format: format as if by printf()
- c - char: write a single octet (not character)
- s - string: write null terminated C string
- x - hex: write hex dump like yv::vhexmax()
- i - in: reads from an in-stream source
- z - slice: reads from a slice source
- p - pointer/position: one or more hex positions
- 0 - zero: describes unexpected nil values
These letters appear in the order a method applies the operation involved. For example, ofn() means format and then newline, while onf() means the reverse.
Above you see many ways to combine and order tab, untab, and newline-indent. Instead of using o_tab as the depth (at two spaces per depth), oncol() indents to a specific column (cf »).
Below are several variants of formatted printing methods using printf() style format syntax, with optional tab and newline-indent before and after the formatted print.
The following operator<<() inlines simply call virtual owrite(). (Such inlines for other classes are often outside the class, but these are member functions.) All these owrite() an iovec of some sort.
Methods ending in x below write source in a hex dump; methods ending in xp write positioned hex where offset display offset starts at a given value instead of zero.
The next 0 (zero) methods are highly specialized ways of reporting a field inside an object is nil. The arc value is the name of the field whose value is nil. (Terminology here models an object graph in terms of node objects and arc pointers — a nil pointer is a nil arc when it points from one object to another.) The who value becomes the tag name in pseudo XML, and purports to convey what meaning the referenced node might have with respect to the containing object.
The next char (octet) writing methods appeared earlier (cf «).
Method oline() just ends the line without indenting to tab depth. Wil rarely uses this, favoring indent after newline with on() (cf »).
An explicit otabs() indents the same way indentation works on on(): two space per tab, but with some upper limit so the method still terminates quickly with absurdly large tab values.
Each of these char (octet) writing methods include variations of following that octet with a line end, tab, untab, newline-indent, or combinations of these:
Next are (null-terminate C style) string writing methods with the usual variations of extra tab, untab, and newline-indent.
The word end in methods means a pseudo XML end tag is written with the given null-terminated C string as the tag name. Variations of optional newline-indent and one or more untab help minimize formatting code when laying out printed objects in XML style.
Inlines above are self explanatory, and many other methods above have comments showing how they might have been written inline as well — no explanation is offered for them either. For the most part they all involve some basic operation either preceded or followed by one or more calls to indent, exdent, or newline. Source below shows slightly interesting methods like on() which indents after; but first let's look at the boilerplate printing api, which debug prints the same way most classes do.
Class yoz slicing an out-stream is shown elsewhere (cf «) and you can find general remarks in the slice demo. Right here we just show the basic print, cite, and dump methods:
You can see samples of dump use in the buf demo (cf «).
A badly ordered triple is logged as below. (If it ever fires, Wil adds an assert — normally a reflex for such code in work contexts.)
The next section shows sources for the numerous util methods that are not inlines.
Many methods for yo are highly repetitious in nature, aiming to minimize code at call sites, but at the expense of duplicating similar code in yo. Here Wil violates a don't-repeat-yourself principle as a compromise, since Wil would rather repeat himself more in yo.cpp and less everywhere else objects print themselves.
Unfortunately, that means code below is many more lines longer than it would be if Wil minimized yo — which lacked priority.
Before relatively boring yo methods, let's look at writing tabs, since this might be least easy to imagine before you see how trivial it really is. (Wil doesn't really use tab chars — just two spaces per indent: the api calls each pair of spaces a tab — figurative term, not literal.)
The main use of otabs() is inside on() immediately below, which simply writes a newline (in this version) and the indents. Writing a line end only with oline() is basically on() wtihout otabs().
A Windows version would write a carriage return before the newline, but this is the Unix version of on(). Apparently much complexity in good formatting then involves only careful decisions to tab and untab using ot() and ou() in the right places, tweaking the value of o_tab. Out-streams need only know current indent for nice formats.
Code for oncol() is like on() aiming for an explicit column, but doesn't use otabs() because it can only indent an even number of blanks. A max column indent is imposed once again, assuming this code is used for human readable output — no one can follow a thousand space indent. For more indent above this limit, it's trivial to call owrite() yourself with as many blanks as you like.
Let's continue with many formatted printing methods. Note Wil wrote all these assuming vsnprintf() does not return the length, even though it usually does on most systems. So this code unnecessarily calls strlen() each time to get the length. In this situation Wil favors portability over tiny speed increase in code whose speed matters little to Wil. (If profiling an app shows an issue, Wil optimizes code further.)
A short dialog between Wil and Dex almost appeared right here, discussing the arbitrary limit of 2048 in formatted output above; but you can simulate Dex's obsession for one-size-fits-all perfection yourself. Wil simply notes a bigger buffer can be used if you like, but a caller of the api ought not to assume any specific limit anyway. So good practice would break larger messages into smaller ones, with "strings" of potentially unbounded size handled separately.
Error message printing methods below also use similar printf() style formatting. These are all wrappers atop ylog() (cf «) centralizing how a logged message is handled. Clearly something inefficient happens here since both these methods and ylog() call vsnprintf() — but speed on error paths is rarely a priority for Wil.
Next is o0() used when debug printing to briefly note a nil pointer field inside a containing object. The who is more of a what here: the nature of the object that might have been present, but whose pointer was nil. And arc is the name of the field that contained a nil pointer.
Using o0() minimizes Wil's code when printing expected objects found to be nil instead. The next two methods also fall in the category of very minor effect minimizing amount of typing.
Below appear various string writing methods for null-terminated C strings, which in Wil's code are most often static literal constants. Wil uses constructor yv::yv(const char*) (cf «) to convert such C strings to canonical run format with pointer and length fields v_p and v_n.
"Wait!" Stu stopped Wil when it looked like this section was done. "You're not done yet — where are the hex writing methods?"
"You're right" Wil grinned. "I owe you one more section on hex dumping methods — ones listed in the api above at onv2xp()."
"What happened to them?" Stu asked frowning.
"Are you frowning at me?" Wil marveled. "I could pretend I left that code out for brevity; but the real reason is code for those methods weren't in the code rev I now have in hand. Then, when I think about writing them for this demo, I don't think they'll add much besides length. So the motive not to write them is brevity."
"Why'd you leave them in the api?" Stu wondered.
"If present, that's what those methods would be named," Wil replied. "I tend to write apis before code. Then I code on demand when I need something, or I code at the same time as something very nearly the same. Anyway, code for those will show up here when it suits me to bother writing and testing them."
"What if I write ov2xp() for you?" Stu suggested.
"Using the hex demo's yv::vhexmax()?", Wil asked.
"Yeah," Stu nodded. "It's just a one liner, isn't it?"
"That was the easy one", Wil teased. "Why didn't you write the one with a yi in-stream as source?"
"I only do one liners for free," Stu smiled. "But I'm available to do the rest for a price. Are you hiring volunteers?"
"No, silly," Wil wagged a finger. "Do volunteers get hired?"
"Heh," Stu chuckled. "You had a bad volunteer experience once, right? Let me guess: no accountability?"
Wil rolled his eyes upward dramatically. "Ugh, don't get me started. I once had to work with wealthy volunteers. Egocentric, machiavellian, total bleeping asshole volunteers."
"Bleeping, huh?" Stu winked. "Sociopathic gameplayers?"
"Sorry, did I say that out loud? Folks who never plan to get paid can't be punished," Wil observed. "Especially when they no longer need money, or care about new allies. Avoid them like the plague — I do now."
"So they can do whatever they bleeping want?" Stu asked.
"Yes," confirmed Wil. "But it depends on the coder; not every rich guy turns into a scheming, morally corrupt angler."
"Okay," Stu held up a hand. "You just stepped over the line. You're still bent out of shape about it, aren't you?"
"Who, me?" Wil pretended. "I've forgotten every name. Where was I? Oh yeah, demos."
"Wait! What did you think of Hertzfeld?" Stu asked.
"He's a really nice guy," Wil asserted. "And brilliant, but his saintly personality is even better. Lots of fun to work with — lucky when I work with guys like him. That's it; no more names."
"Excuse me, but I've always wondered," Stu persisted. "I know it's none of my business. But why don't you complain about other people? When they deserve it? Do you just not mind?"
"Well," Wil sighed and paused. "I'm not a saint. It's not like I don't notice; witness my burst of whining a second ago. It's just talking about other people is petty and a waste of time. It makes me feel small to just think about it; my self esteem won't let me gossip."
"That's interesting," Stu considered. "So you'd rather just not defend yourself? What if someone really disses you?"
"It happens," Wil smiled. "And after I think of a really vicious response, I go on about my business. Mud slinging sucks."
"But it's really entertaining," Stu pressed. "You ought to give it a try. And folks might think you're a turn-the-other-cheek pacifist."
"Oh no," Wil shook his head. "I definitely think about getting out the nukes. I just know it would fan the flames. If I do anything, it's apt to be subtle, not direct. All I really want is to get the drivel of small minded people behind me so I can work — get the muck off my hands."
Stu wrinkled his nose and said, "You're way too high horse."
Method yozo::owrite() below appears here to juxtapose it with yozo::zo_pwrite() near the bottom of column right.
Compare this write method to others at bottom column right.
All this code is available only under the BriarPig mu-babel license described fully on the rights page. You do not have permission to reprint this page in any way. No feeds or repackaging is allowed. You can link this page if you want folks to read it.
thorn: todo, names, fd, iovec, assert, log, run, hex, crc, buf, in, out « Þ, quote, escape, compare, file, deck, cow, arc, blob, tree, slice, rand, time, stat, hash, heap, node, primes, page, book, pile, stack, atomic, lock, mutex, thread, map, meter, list, iter, ctype
(mu: toy, peg, imm, tag, box, symbol, token, number, bigint, class, method, reader, writer, eval, env, vm, gc, world, pcode, compiler, asm, lathe, lisp, smalltalk, design, weight, jar, card, harp, debug, profile)
Some demos are stubs: todo is a demo guide. See toy for mu updates on language pages; names introduces naming schemes.
The yo out-stream api uses many of þ's singleton classes, so most of them are shown below (even ones not used by yo). The character 1 (digit one) in each class name means singleton which, in the sense used here, means only one instance is expected to exist.
However, since no instance has a meaningful value beyond class type, any instance is the same as any other. For example, if you make more instances of y1now for some reason, it will work as well as the canonical instance ynow you're expected to use. For this reason new construction is not forbidden. In each case the integer member inside each struct is a dummy that's ignored. (An exception is ynil which converts to zero regardless of y1_nil's value.)
These singleton classes are used in þ only to overload methods — usually operators — so the right hand side of an operation can accept one of the singleton values. For example, many of the demos flush an out-stream like this using yo::operator<<(y1now const&):
Names chosen for the singletons are arbitrary, and largely what suited Wil at the moment. Obviously yendl imitates std::endl; but all other names were chosen freely at whim, but with an eye to brevity. Wil had the most trouble choosing names for indenting and exdenting a stream, finally settling on yaddi and ysubi even though the i suffix for indent here is inconsistent with both global þ and local yo conventions. (þ uses a trailing i to mean in-stream subclass, and yo uses t for tab and u for untab instead of i for indent.) In practice Wil rarely uses yaddi and ysubi to change tab depth.
Code above appears in source file y1.cpp at global scope inside the mu namespace. This is where the singletons are constructed currently, but it hardly matters where since the values of these objects is irrelevant. Anything that makes a linker happy for library structure is fine.
Singleton value ynil means nothing or empty and þ classes overload methods using class y1nil to specify what happens when ynil is used as value. Typically Wil uses ynil to express deletion of content: assigning ynil to a left hand side expression denoting a selection (eg a slice) means delete selection when this operation can be supported in an api.
(This behavior of ynil has little to do with yo; but since ynil is the singleton value with the most potentially confusing complex behavior in other classes, an additional note on normal meaning seemed useful here.)
A common expected use of ynil looks like this:
Overriding T::operator=(y1nil const&) is how this would be done for a selection of type T on the left hand side. Alternatively, instead of remove, assigning ynil to an object might zero it in some way — perhaps make it completely empty by calling a clear() method.
All the demos use yout as the global out-stream writing to stdout. This section shows how yout is declared and defined; then the next section belows shows the implementation of yfo for an out-stream writing to a file (descriptor).
The current yo.h header declares yout and yerr like this:
But the comments are only suggestive, and not intended as contract or specification. The actual definitions in yo.cpp are these:
Note how yfdw::we_stdout (cf «) is used instead of fileno(stdout), which should have exactly the same value of one (1). As long as Wil bothered to write the fd demo, he might as well use yfdw.h header constants.
The third argument, yfo::foe_fp_none, causes an overloaded yfo::yfo() constructor to be used that more-or-less means "do not seek before each write", because seeks on stdout and stderr don't make sense (and seeks can cause rather strange behavior, as Wil observed first hand).
The &yH::g_1H and 4096 arguments tell yout to heap allocate a 4K buffer used to stage content on its way to stdout. Singleton yH::g_1H from the heap demo is just a thin wrapper atop calls to malloc() and free().
The name of class yfo means thorn file out (stream); it supports the yo api for writing a file given a file descriptor. Because yo is mainly geared for buffered i/o, yfo prefers to always use a buffer, even if it's only a very small one inside yfo itself. (When yfo writes to traditionally unbuffered stderr, this might surprise the unwary; suggestion: remember to flush.)
Wil has written many versions of such file out-streams since the 90's, sometimes using file descriptors and sometimes using FILE from C's standard library. This version aims to be minimal with regard to effecting what Wil wants with minimal fuss, sometimes using very casual means of encoding intent. (Wil turns some features on and off using magic number values in slots.) Further abstraction isn't needed.
You might compare this yfo subclass writing a file to the ybo subclass writing a buffer in the buf demo (cf «).
Note yfo is not small in size. If Wil ever needed a very large population of file out streams using yfo, he might either reduce the size or (more likely) only instantiate yfo when actually writing, with some other representation holding state between use.
The file descriptor is m_fd and the m_heap pointer is only non-nil when a heap was passed to a constructor for buffer allocation. The buffer is m_buf and m_pos is the position in the file's stream corresponding to the first byte in the buffer. If a tiny default buffer is needed, m_body provides the space (Wil wanted "over one-line's worth" here) and m_weeLen expresses a size that is a small part of the buffer, for any buffer size.
The value in m_tag must always equal magic number e_fo_tag or else the object is assumed corrupt (or never a valid yfo in the first place). Two more magic numbers in m_fp and m_fg control how yfo behaves. The names fp and fg mean file pointer and file guard in exactly the same senses described in the fd demo.
When m_fg equals magic number e_fg_enable, it means yfo owns the file descriptor and should close the file eventually — the same way yfdg does in the fd demo (cf «). When a yfo constructor gets the file descriptor by way of yfdg, a transfer of ownership occurs
When m_fp equals magic number e_do_fp, it means yfo should seek when it writes the buffer using pwrite() instead of write() with m_pos as the destination position. This makes yfo's write position independent of anyone else using the same file. (It's a bad idea with file descriptors that don't support seek, so you can disable this at construction.)
The following inline getters access yfo members:
All the virtual yo methods are listed next. The implementation of the destructor is fohalt(), permitting early destruction.
The following protected methods are non-virtual with internal effects usable from multiple places. Instead of calling either fo_pwrite() or fo_write() directly, yfo methods call fo_put() instead, which knows whether pwrite() or write() is the default manner to write.
The yfo api closes with a variety of constructors. The last arg to each is an enum saying whether writes use seeking pwrite() — defaulting to true, so you must pass yfo::foe_fp_none to avoid pwrite().
By passing a heap instance, you tell yfo to allocate a heap buffer. (If you want to pass in an explicit buffer, just add more constructors; Wil has his hands full at the moment.) By passing a file descriptor in yfdg, you tell yfo it owns this file descriptor: it must closed when yfo halts; the yfdg no longer has the file descriptor inside after yfo steals it away.
Since we just presented yfo constructors, let's start there. Each defaults the buffer to m_body and then calls fo_init() to allocate a heap buffer if necessary, and then finish initializing yo triple (o_0, o_p, o_x).
Each constructor calls fo_init() below, which uses yfdp from the fd demo to seek the current file position (cf «). Since this is questionable when a stream cannot seek, yfdp is used to seek current position only when pwrite() usage is enabled by default with e_do_fp==m_fp.
Now let's look at destruction so you can see what resources are released there, comparing to what happens above. (Note fohalt() does not auto-flush; if you want flush before destruction, do so explicitly.)
If the magic number in m_tag is ever wrong, fogood() returns false and fobad() below logs the problem and asserts:
Now you have enough context for meaningful content writing.
Any time yfo must be flushed because the buffer contains pending content destined for the file descriptor, the actual flush is done by fo_flush() below, which uses inline fo_put() (cf «) to call fo_pwrite() or fo_write() depending on whether the stream seek to write.
The virtual oflush() just calls non-virtual fo_flush():
When yo::oc() runs out of buffer space (cf «), yfo::_oc() flushes the buffer with fo_flush() then does an inline oc():
Calls to olen() and oseek() also flush to consistently empty the buffer when a seek occurs. It's necessary for oseek() but gratuitous for olen() since current position doesn't change. Note getting the length of a file involve seeking eof which might not work in streams that cannot seek.
The next section shows writing methods that are not simply flushing buffered bytes.
Writing methods all use inline fo_put() (cf «) to call fo_pwrite() or fo_write() as appropriate, depending on e_do_fp==m_fp to explicitly seek position m_pos corresponding to the first byte in the buffer.
When the buffer is empty, this next block decides whether input is small enough to go in the buffer, or whether writing directly to the file descriptor seems better:
Here after flushing the buffer, we decide once again whether remaining content is large enough to go direct, or better staged in the buffer:
Next are fo_pwrite() and fo_write() methods called by inline fo_put(). The pwrite version was written first. As source comments note, a flush is only necessary when the target span intersects the file range covered by the buffer. But virtual opwrite() always flushes first for simplicity.
Note automatic handling of EAGAIN and EINTR which might otherwise get lost in layers of object-oriented apis. They're handled similarly in non-virtual fo_write() which is only used (as with yout writing to stdout) to avoid explicit seek operations when they make little sense.
The next section covers writing directly to the buffer inside yfo using the protocol for reserving available buffer space in yo subclasses.
Here we see bulk i/o copy using otake() and ogive(), to support the algorithm described earlier (cf «). Here each method uses fo_body() as an aggressive check for error conditions.
If otake() sees a full buffer, it flushes first to make space. A better check than is-full would also flush if remaining capacity size was under some small number of bytes like 8 or 16 (a trivial micro-optimization).
A subsequent ogive() call aims to finish the transaction started by otake(), checking for errors and adding used bytes to the buffer. Since otake() flushes on buffer-full, there's no need for ogive() to do more than watch for over-filling the buffer.
The yoz object below is the value returned by method yo::oz() (cf «) representing a slice of an out-stream, described as a beginning offset and total length. The yz slice type is described in the slice demo, along with examples of usage.
Code for yoz is not shown here now, but might appear later. The main idea to notice now is this: yoz is a triple (z_p, z_n, z_o) giving offset, length, and pointer to underlying collection. Even if yoz had no methods this state alone would be useful. For example, the yozo subclass of yo shown next is an out stream writing to a subset of another out-stream, where this subset is expressed as a yoz slice.
Please see yiz in-stream slices in the in demo for a good hint how code for yoz would look. In particular, yoz::zwrite() here resembles yiz::zread() in many ways (cf «) However, the yozo class shown below is more general and more useful than yoz, so there's no need to miss the code for yoz.
The yozo class in this section is a yo out-stream writing to a fragment of another yo out-stream. The destination segment is an offset range — a yo subset described using a slice (see the slice demo for more context) denoted by offset and length.
None of the code in this section for yozo has been tested. All the source code you see here is a first draft, written just today (18may2008), and hasn't been run before. (It resembles a vague memory of a similar class done a few years ago.) Wil would never use this code professionally without testing it first in a fairly good unit test. Perhaps a test demo will show a test for this and similar slice streams later.
Superficially yozo resembles yfo in the last section, except total length of the stream is constant, using a size fixed at time of first construction. The main purpose of yozo is to constrain all writes to affect only the slice described. No content outside the slice should ever be updated in the underlying yo using the yozo api. (This isn't quite as useful as the yizi class shown in the slice demo, which reads from a slice in an underlying in-stream, treating a slice as an independent file.)
The final destination is zo_o, which can be any yo subclass, like yfo shown earlier on this page (cf «) or ybo in the buf demo (cf «). The part of zo_o that can be written is described by slice zo_z: the N=zo_z.z_n bytes starting at offset zo_z.z_p inside zo_o; so obviously yozo always has length zo_z.z_n denoting eof in the slice.
Triple (o_0, o_p, o_x) points to space in iovec zo_v (see yv in the run demo): a buffer given as constructor arg, or an 8 byte zo_body[] array in yozo.
Valid offsets for yozo::oseek() vary from zero to zo_z.z_n, and zo_pos describes an offset in this range saying where first byte o_0 in the buffer will be written when the buffer is flushed. Since zo_z.z_p is the offset in underlying zo_o, the offset in zo_o corresponding to o_0 is actually zo_z.z_p + zo_pos.
Writes falling outside the target slice are ignored besides counting them in zo_lost — zero zo_lost means you never wrote outside the target slice. Total bytes written to zo_o are counted in zo_sum; this can be more than zo_z.z_n since nothing stops you from rewriting the same bytes again and again.
Source code for yozo below is described only briefly, since you can get the general idea for each method from descriptions of the same methods in yfo (higher in this column). Material here dwells on how limiting writes to a target slice affects otherwise generic code. Let's start with creation:
As you can see, a call to olen() on zo_o gives the entire length of zo_o — this is used to constrain the zo_z slice so it corresponds to an actual part of the underlying out-stream. Calls to yz::zrelative() and yz::zabsolute() convert negative offset and length to absolute values relative to total length (cf »).