This gathers random dialogs originally appearing on the qi page, which now fit no clear topic category.
prison model ¶
"Something doesn't fit," Eli puzzled. "If a client process is single threaded and can't block, and Lathe defers blocking operations to a client env, who can't do them either, then who makes blocking calls?"
"Nice analysis," Wil granted. "I couldn't put it better myself. But I think the answer is implied by your process of elimination. The client env is out. Who's left?"
"Another process?" Eli guessed. "You mean a client env delegates blocking calls to another process entirely? Isn't that a little extreme?"
"Well, it works," Wil shrugged. "If a client env refuses to block, can you think of another way to do it? Mind you, I don't care if a Lathe client blocks. That would make my life simpler. But I know at least one client who won't, so slow things are shipped off to other processes."
"I think I get it," Eli closed his eyes. "Conceptually, there's a thread pool somewhere doing lots of concurrent blocking operations. Maybe the pool is a a bunch of processes, or maybe just one. But this pool is not inside Lathe. Using inversion of control, Lathe says it happens outside and it doesn't care."
"Yes, perfect," Wil admired. "Once control is inverted, it can keep going: a client env can push things into other processes, and it still works."
"Is this client process psychotic?" Eli needled.
Wil glanced up to the left as shorthand indicating deliberate vagueness. "The client process is a prison," Wil explained. "And the warden is slightly kooky. But he has method in his madness. The warden has a system enriching himself and the inmates."
"Uh, what?" Eli tried to smile engagingly.
"The warden runs a business using inmates, who don't have freedom of folks outside prison," Wil continued. "So the inmates run the business by messaging folks in the outside world, who actually do stuff."
"Strange metaphor is losing me," Eli warned.
"Say you're sentenced to ten years," Wil kept going. "Sounds bad, but sentence is really 120 months served concurrently, and you get ten percent of the warden's gross when you get out. Your ten year sentence is done after one month, and you have a year's worth of wages in your pocket. So prison pays you ten to one."
"You saw Shawshank Redemption how many times?" Eli asked. "Better lay off prison movies."
"Maybe seven or eight times," Wil admitted. "I could use a guy who knows how to get things. I'm gonna start a charter fishing business down in Mexico. I hope the Pacific is as blue as it has been in my dreams."
"Okay," Eli tried to grin but largely failed. "While that's funny, it's basically a little too weird for me."
Wil pointed a finger at Eli. "It will get funnier the longer you work in the tech business," he claimed.
"I think I get what you're telling me," Eli held up both his hands enclosing a box. "Sometimes you don't get to choose where code runs. You might have no control at all — wish all you like, but you're shit out of luck."
"Your cynical view is coming along nicely," Wil praised. "Yes, Lathe can demand premium client env service only by risking a trip to the infirmary. The model I'm proposing is one that works under lockdown, which is in fact a normal real world working scenario."
"Needy languages need not apply?" Eli guessed. "How does this affect api bootstrapping?"
command lines ¶
"We want to invoke Lathe from command lines," Wil said. "Which would run it as a process, of course, unless a command line is routed as a client request to Lathe running as a server somewhere else."
Eli nodded tentatively. "How does that relate to running in a prison process without amenities?"
"A general Lathe api should work for both," Wil explained. "If we do command line use first, lazily expanding into doing whatever we want, then Lathe won't work in a constrained client env."
Eli drummed his fingers. "So we're going to design complex async Lathe api, which we invoke from a simple command line tool, making me ask why we bothered with async api and non-blocking semantics. But all along, you're thinking about running Lathe later in a prison."
"I've been too hard on you," Wil admitted. "You're tracking this perfectly. Keep it up and I'll stop being a supercilious snot. Yes, a bootstrapping context at first won't show one thing indicating why complex async api is better in later use, when constraints are grim."
"Okay then," Eli cleared his throat. "So as we explore Lathe invoked from a command line, the trick is to invert control so blocking stuff happens outside Lathe, where it looks weird now, but helps later."
"Terrific summary," Wil shook his head in admiration. "So command line stuff will look like a game whose aim is to get normal blocking stuff, like i/o etc, outside in a client env submitting stuff to Lathe in requests."
"Async requests—" Eli realized. "—each request gets nothing but a ticket, at most, which might or might not be redeemable later. Responses won't be tied to requests. Inputs and outputs run off the hook."
"Something like that," Wil agreed. "Internally, Lathe can multiplex requests as green threads or lightweight processes. What? Got a problem with that?"
processes ¶
"Can't put my finger on it," Eli concentrated. "But somehow you're violating the Unix way, which involves doing small things in each process, so you can combine multiple processes to get larger tasks done."
"I had in mind adopting the Unix way in Lathe," Wil countered. "Except by extending the idea of processes into Lathe. I think it'd be cool to connect lightweight processes in Lathe, piping output from one to another."
"There it is," Eli recognized. "The smell of wheel re-invention. Why not just use actual Unix processes? It seems dumb to re-invent a lot of standard Unix stuff — you'll never get done, among other things. You seem too pragmatic to fall for a mistake like that."
Wil shrugged. "Sometimes you can't control how many processes are involved in a project. One or very few processes can be forced on you. But if your tasks logically resemble thousands of processes, it would sure be nice if a familiar model could be imitated."
"It smells," Eli persisted. "Can't you carve out some small piece without cloning Unix processes?"
"Sure," Wil said. "I only wanted to clone stdin, stdout, and stderr in lightweight Lathe processes. Spawning a lightweight process would give you a green thread you can configure so i/o pipes the way you want."
Eli chewed his cheek. "Doesn't that start you on a slippery slope? Next thing you know, you're copying everything about Unix process forks."
"Just say no to slippery slopes," Wil said. "Here's the secret: breadth first cloning kills you dead. Depth first cloning lets you cherry pick exactly what you need right now. I want to cherry pick narrow slices."
"How do you benefit from similarity to Unix processes when you omit everything?" Eli wondered.
"I just want to say, 'This code acts like piping output of one process to input of another.' I don't want the whole ball of wax," Wil said. "Each time I want a feature, it need only involve a small number of things to work. In effect, you're accusing me of being unable to stop each time."
"Other folks don't seem to stop after they start," Eli noted. "It's like eating Lays potato chips."
"Most folks don't start, period," Wil corrected. "They see it's insane to clone Unix processes when you can use them as they are instead. But no one's multi-threaded apps manage to offer a simple process model."
"What do you have in mind, exactly?" Eli asked. "From a command line, how do multiple lightweight processes in Lathe look simple?"
"Maybe one heavyweight Lathe process can look like N bundled lightweight processes," Wil suggested. "On a Unix style command line, you could pipe a new stream into a new lightweight process in an existing Lathe process. Lathe would look like a little embedded OS or something."
"Isn't this a solution in search of a problem?" Eli accused. "What's your use case?"
"For diagnostic and debugging purposes," Wil suggested, "it might be useful if N independent things — running at one time inside Lathe — looked like separate processes you can examine one at a time as if others were absent. For example, suppose you can see logging for just one request, as if stderr existed only for that request."
"You want to be able to tease apart N things mashed together," Eli concluded. "But one at a time. How will this affect api design in bootstrapping for Lathe?"
"I don't know yet," Wil excused. "But I expect this to be a standing question: as I add a feature, can I make it look more like a process in its api? Expect it to come up all the time: Can I use a process approach here?"
"Wake me up if it happens, okay?" Eli joked. "You know, I see a theme here. You warned me api will favor async; now you warn Lathe api will emphasize process ideas — and processes run asynchronously."
"You noticed?" Wil echoed. "Okay, yeah: Async is required; process orientation is optional preference. So whenever you have an idea to simplify api with a process metaphor, I'm interested."
Here's a menu of pages under qi:
- process - lightweight process api
- dialog - miscellaneous dialogs
requirements ¶
"Uh, Wil," Eli broke gently. "I hate to correct you, but 2009 is not a leap year. There's no 29th of February this year. You should fix that."
Surprisingly, Wil looked very gratified. "You fell for my ruse. Consider for a moment: What if Zé someday learns to travel in time? He might get folks to reform the calendar. For all you know, 2009 might become a leap year. Today is the 29th of February by fiat, because I say so."
"Wil, that's deeply eccentric," Eli inched away.
"I'm joking," Wil sighed. "But I do have a point: don't argue with me. Whatever I say about Lathe goes. I'm not interested in arbitrary rules other folks like. When it's arbitrary, I get to pick when it comes to Lathe. So do we have a deal?"
"I have to agree, or I can't play?" Eli restated.
"Yeah," Wil nodded. "I don't really have a lot of time. And unlike sixteen years ago, I find I don't care much anymore about trivial programming language details. They're all the same in big ways. So I'll let you banter with me over bootstrapping Lathe, provided we don't sweat endless alternatives. I need a yes or no here."
"Fine!" Eli snapped. "I accept. But you don't have to be a prick about it. Don't have a cow, man."
"That's my cue to say 'Bite me!' — Yes I do have to be a prick about it," Wil insisted. "Otherwise, in your enthusiasm, you'll stonewall my progress through simple denial of service attack. So when I make this gesture, you shut up."
"I thought you were a nice guy," Eli said crossly.
"Sadly, it's human nature to never stop asking for more, no matter how much you're given," Wil claimed.
"Oh, so now it's about me," Eli rolled his eyes.
"Enough self pity," Wil commanded. "Let's start with requirements. You'll hate the first one."
"By the way, qi is a stupid name, " Eli sniped.
non-blocking ¶
"Lathe must be non-blocking," Wil announced.
"Non-blocking?" Eli screwed up his face. "Does that mean async? No blocking at all?"
"Yes, it means async," Wil confirmed. "No blocking at all — Lathe can't make blocking system calls."
"Because you say so," Eli covered his eyes.
"That's right," Wil smiled. "I want to use Lathe in embedded contexts where code must not block, or it will stall a process with great negative effect."
"You know you can't stop C extensions from calling arbitrary system calls," Eli warned. "You can't forbid blocking calls when Lathe incorporates new code someone links into the runtime. Am I right?"
"Yes, you're right," Wil granted. "But Lathe's core can forbid blocking calls. There's no reason to throw up hands in defeat at the start over folks screwing up later."
Eli pressed knuckles to his temples. "I'm sorry, this sounds incredible. You know allocating memory can block, right? Plan on allocating any memory?"
"Memory is always very complex," Wil countered. "I want it to be possible for a Lathe client to pre-allocate all memory, then dole it out as Lathe demands. But if a client doesn't do that, and then memory allocation blocks, that's not Lathe's fault — it's not forced."
"That so smacks of sophistry," Eli charged.
Wil shrugged. "The goal is to avoid forcing a client environment to block due to basic Lathe design," he said. "In some cases, a client env must help. If it doesn't, the client can't complain about blocking."
Eli looked like he tasted something sour. "So this whole non-blocking thing sounds like a careful dance between a client env and an embedded Lathe runtime. Is that a warning Lathe's async api might get complex?"
"Yes," Wil admitted. "Which is why you're not going to give me any grief about it. Any complexity due to async semantics is necessary, because I need async, and I don't need folks who prefer simpler blocking api."
"Whatever," Eli shrugged. "Suppose you tell me more about this async business."
"Sure," Wil made an easy gesture.
async ¶
"Can you define 'async' for me?" Eli asked.
"Yes, it's the opposite of sync," Wil explained. "Sync api invokes an operation — sends a request or calls a function — and the entire thing is done before the function returns, which occurs directly. So for example, if you call a synchronous function, it performs the desired task to completion and then returns afterward."
"So async means you don't finish before returning?" Eli asked. "Async functions only start a task?"
"That's right," Wil nodded. "An async function request starts a task, but the task doesn't finish by the time the function returns. Instead, a successful return from an async call means something like, 'yeah I started the desired effect,' and that's all."
"I guess non-blocking i/o is a typical case," Eli suggested. "If a disk read might take a while, say twenty milliseconds, then an async read is just a request to do the read. So when the async read finally completes, a reply happens afterward."
"Yes," Wil agreed. "In sync, a call is a request and the reply is its return. But in async, a single call is just a request, including its return. The async reply is a completely different function call, executing later."
"This will give me nightmares," Eli predicted.
"Maybe," Wil shrugged. "The point of the matter is: async is complex. Every operation you suppose is one function actually takes two to manage: a request and a reply. It can be hard to follow normal sync control flow; async control flow is harder."
"How does this work without threads?" Eli wondered. "Some kinda funky state machine?"
"An async call is something like a function A() taking two extra arguments: a reply function R and a context cx to pass back to the reply," Wil began. "So you call A(args, R, cx) and by the time it returns, you find it has asked you for G, H, and I in the meantime."
"So I fetch them and call g(G), h(H), and i(I)?" Eli guessed. "When does original A(args, R, cx) reply?"
"Eventually calculation completes and finally someone calls reply function R(cx, result)," Wil said.
"What was context cx for again?" Eli prompted.
"Well, the C call stack present when A(args, R, cx) was called is no longer there," Wil explained. "So cx is a clue indicating how to resume, since we can't keep continuity for the call in a C runtime stack."
"So, async is a form of living hell?" Eli inferred.
"That's the negative interpretation," Wil waffled.
"There's a positive point of view?" Eli wondered.
"Yes," Wil assured. "You can very flexibly interleave a lot of really complex stuff, all running concurrently, with some pieces waiting arbitrary periods of time to hear from distributed participants."
"You're saying if my app is total chaos, this async stuff helps?" Eli asked. "What causes chaos?"
"Tens of thousands of network connections," Wil said. "With gnarly stuff happening for each one."
"I didn't sign up for that," Eli objected.
"Well I did," Wil retorted. "So don't give me any crap about my app context not matching yours."
"You're a wee bit touchy," Eli noted. "You know, I forgot we were talking about Lathe. How does all this apply to api for Lathe runtime details?"
queuing ¶
"Ultimately it means requests queue inward and replies queue outward," Wil said. "Or it works roughly like that. Replies might be function callbacks instead of queued responses. But Lathe also queues requests to a client env — for stuff needed to satisfy client requests."
"What's an example?" Eli asked. "Maybe disk i/o."
"Okay, the env asks Lathe 'please do X for me' and Lathe finds a disk read D is needed to keep doing X," Wil explained. "So Lathe periodically returns to a calling env, which finds a request from Lathe to please do disk read D. When D is done, the env should send D into Lathe. Eventually Lathe replies to the X request."
"I thought it would be, like, please read and interpret Lathe code in this file," Eli said.
"Maybe interpreting that file opens lots of socket connections," Wil suggested. "So one initial request — intepret this byte sequence — turns into a lot other requests. Each socket event is another input to Lathe if sockets are handled in the Lathe runtime."
Eli rubbed his chin. "So you're saying every time Lathe needs to do something that might block — some blocking operation B — Lathe returns to the client env which finds a request to do B in the background?"
"Yes, you're doing quite well," Wil praised.
"I don't grasp what I'm saying yet," Eli admitted.
"Picture an old-fashioned switchboard operator working for the phone company early in the last century," Wil suggested. "A client env is like the operator, manually plugging together connections. And Lathe is the switchboard, which does something every time new data is plugged in someplace. Lathe goes as far as it can using data in hand, until a need to block occurs."
"Must Lathe ever yield voluntarily?" Eli wondered. "I mean, yield even when there's no blocking operation to be done by the client env operator?"
"That's a great question," Wil complimented. "The answer is a whole new Lathe requirement."
yielding ¶
"Lathe must return to the calling env in a fairly short time after any request," Wil said. "Even if Lathe could run indefinitely — an infinite loop is a pathological case — Lathe must return often to a calling client env, to avoid hogging cycles. Otherwise a calling env might miss handling new events in a timely fashion."
"So Lathe must return to the env in two cases," Eli restated. "One, when only blocking operations remain; and two, when some timeslice is exhausted. Or some approximation to a timeslice?"
"Very good," Wil smiled. "Lathe must return to the environment within a maximum timeslice at most, or earlier when only blocking operations remain."
"Why isn't the client env doing anything in the background while Lathe is running?" Eli wondered. "What about other threads in the same process?"
"There might not be other threads," Wil explained. "A process hosting Lathe might be single threaded. So any client env with realtime constraints might fail to honor contracts if Lathe ran too long without yielding."
"I want to say 'screw single threaded clients'," Eli admitted. "But I guess they matter?"
"Yes, single threaded clients matter because I'll often need to run Lathe in a single threaded process," Wil said. "If Lathe blocks, that process is toast."
Eli pulled at his collar like he was getting too warm, and then sheepishly joked, "This is a little scary."
"Now you understand what motivates the async api," Wil gestured easily. "It might look complex, but Lathe needs to bend over backward to avoid killing the host process by ruining any hope of hitting time contracts."
"I predict I won't like async api," Eli confided.
"Likely not," Wil agreed. "That's why making an api you like isn't a main goal. It would almost certainly conflict with my host process requirements."
back pressure ¶
"What if Lathe doesn't make progress fast enough?" Eli asked. "I mean, suppose a client env keeps queuing inbound requests, and Lathe replies slower than new requests arrive? Won't something get wedged?"
"Oh, yes," Wil agreed. "If nothing else, latency between request and later reply might get too big. So a client env must track use of resources in Lathe — total memory used or memory remaining, as well as queue lengths — and decide whether to throttle incoming requests."
"That's it?" Eli asked. " 'Just throttle.' Nothing else?"
"Servers that scale must always cope with this," Wil said. "Eventually there's some load you can't handle. The graceful thing to do is refuse service you can't provide. Don't say you'll do something, then do it miserably. Push back on the source of traffic."
"It sounds hard, though," Eli worried.
"I didn't say it was easy," Wil refined. "Just that throttling is the plan. It's the answer to questions sounding like 'What if I run out of resources?' The answer is: don't run out of resources. Say 'no' upfront faster."
"What if Lathe just runs too slow?" Eli considered. "What if desired request throughput isn't high enough?"
"Oh, well now you're just talking about optimization," Wil dismissed. "Anything that's too slow in Lathe should be moved out of Lathe, into a C or C++ extension."
"Why get Lathe involved at all, then?" Eli asked. "Just go straight to C and C++. What does Lathe add?"
"You forgot our recent conversation about interpreter contexts, didn't you?" Wil seemed wounded.
"Sorry," Eli apologized. "Slipped my mind."
