weight — considering pros and cons from Wil's point of view concerning toy programming language lathe: a lisp dialect plus smalltalk class system, using þ library under mu, using the mu-babel license. (Updates? See toy.)
One might pursue several goals and avoid many nuisances at once while working on any given project. This page explains pluses and minuses in Wil's perception of priorities and values, versus burdens and scourges. Little of this is relevant to programming language content. It only shows some of Wil's motives (which Wil guesses you don't grasp).
Getting this page fleshed out will likely take a lot of iterations and slow accumulation. Some material that would have appeared here already slipped Wil's mind soon after conception, and that's likely to be repeated. (Wil thinks his world view is "normal" and forgets it differs from popular convention, except during moments of passing, temporary insight about other folks.)
Wil originally planned to list pros and cons with number scores on this page, putting a positive or negative value next to an objective with suitable magnitude to show importance. But that's not how content appeared when written, so that idea is temporarily on hold.
Wil's first priority is finishing, as long as the end result approximately matches an original plan Wil liked. Anything increasing scope, complexity, difficulty, completeness, adaptabilty, or precision is likely to take longer, and could just as easily be put off until after a first pass is done.
Wil likes to think of classic time-budgeting problems, like writing an essay in English class given an hour maximum to do the whole thing. (Obviously programming projects are not the same thing, but might as well be compared to a sequence of such tasks; programming preserves the essential problem that time squandered is gone and can no longer be used.) The first planning constraint must be an aim to finish if partial work counts for little, and that's true in this case with lathe. Wil wants Lathe to do something, and be roughly described.
Doing more and better things can wait until later, after Wil gets a handle on how long it takes to do the worst acceptable thing. He suspects it's a worse-is-better kind of problem, because better things can run out of time. When choosing between too little and too late, too little sounds swell.
Wil isn't sure he can afford to do this. It might simply be too expensive in time if quality standards are too high. Since Wil isn't being paid, it must be a rush job, or else lathe might go up on cinder blocks in Wil's front yard: another white trash half-baked idea that wasn't triaged as loss a little sooner.
Wil's yard is already filled with software projects up on cinder blocks. Wil knows far too much about the Bayesian likelihood of code languishing on cinder blocks.
Did you like that cinder block metaphor? Wil will sell it to you for a dollar. It will help offset Wil's costs. A dollar here and a dollar there, and soon you're talking about real, useless pocket change. See, free humor with software; what a deal!
Wil has no interest in numerics, so code speed and type systems related to number hierarchies have zero priority. For example, Wil has not yet planned a single idea related to complex numbers, and expects never to need complex numbers himself when coding. This means the smallest amount of numeric code Wil needs will appear at any given time.
When Wil later aims to successfully support specific standard Scheme APIs, this might finally drag in more numerics than Wil wants to code. Maybe. If Wil can't dodge it.
Good approaches to threading are one thing Wil wants to pursue eventually. But in the near term, threads are a background problem shaping design ideas without direct support in toy language bits. Wil thinks about threading, but doesn't add thread features yet, unless you consider immutable data support as thread-friendly (and you should).
Threads are off the table as first-class language feature in early drafts because it increases scope and therefore violates a priority to finish an early pass. However, Wil still mentions thread constraints in places when discussing runtime mechanisms.
"Will you ever support native threads?" Dex demanded.
"Sure, why not?" Wil granted. "Obviously I'm going to need multiple threads in complex apps. But in one toy language OS process, I might only use one thread per lightweight process, using only green threads within."
"What's a lightweight process?" Dex puzzled.
"A disjoint subset of toy language address space in which code runs," Wil explained. "So shared mutable state is not accessed by more than one native thread, except where explicit language level IPC support has been implemented."
"You lost me," Dex admitted.
"That means we're done then," Wil announced. "No more questions about threads will be answered unless I feel like writing about something. Early language features will include green threads with cooperative scheduling, when I find I can't avoid doing them for some reason."
Wil prefers the smallest number of dependencies he can manage to get by on. For example, it's impossible to avoid using basic system calls and standard C library methods, so there's no point in thinking about such primitive dependencies.
However, any other dependency is conceivably avoidable, and any third party library can be avoided unless it's necessary for some feature, and can't be trivially mocked in new code. To avoid third party library dependencies, Wil avoids any feature needing one. But in modern software, it's hard to get by without platform basics for threads, networking, and Unicode support.
Wil plans to delay Unicode support as long as possible, hoping to make Unicode pluggable and optional, so language semantics do not depend directly on charsets. Early parsers will require ascii for keywords and primitive tokens like parentheses — in fact, nearly everything will depend on ascii encoding. So new parsers written in Lathe might support other charsets later.
Early UI will be minimalist Unix command line tools. Later GUI support might target the usual suspects in app frameworks and widget libraries. But Wil wants to migrate the GUI into a completely different process from the Lathe engine, so the GUI is essentially a client of a UI-less server. This won't happen until there's nothing left to do except pursue better UI architecture fitting Wil's idea of engine as local-or-remote server.
Every piece of code in a Lathe process not written in a safe managed runtime exposes a garbage collection system to risk of catastrophic failure as a result of stomping on single bytes from wild pointers in C. Copying garbage collection depends on bit-perfect interpretation of tags describing memory formats. A single corrupt tag — wrong by as little as a single bit — can cause all gc boxes downstream to be lost, essentially terminating that process abnormally once memory integrity is gone.
Therefore, integrating any third party library in a Lathe process is very dangerous, since a single bug in C which barely affects an ordinary C process might destroy a Lathe process when small scale memory corruption is greatly amplified by wrecking process garbage collection. Bugs in libraries will make Lathe look fragile, like a canary in a coal mine.
Wil's long term view of good UI includes browsing system state via HTTP using a browser — meaning the language runtime and development environment should host a web server fully integrated from an internal perspective, surfacing views of language, IDE, and developed libraries as Plan9 style virtual file systems like Linux's proc file system.
Of course, other non-browser clients would be better for some purposes; but it should be possible to get by using only a browser, though perhaps awkwardly. This would afford access to both development environment and administration interfaces at all times, even after a system is deployed as a server.
None of that has any direct effect on a toy programming language per se, but it influences the desired growth path of a development environment for a language as it evolves over time. Any more traditional IDE model at variance with the server based model described above won't fit where Wil is headed. This subtly influences context of language organization, from the start when module systems are drafted.
A server approach supplies context Wil uses for a top down view, into which bottom up designs must fit in the long run of evolution. Enabling this sort of model is one of Wil's motives for making a language. Substituting a different model of system context renders some of Wil's language plans relatively meaningless. Wil holds that server image in mind when asking what would make programming that server easiest — for Wil — especially when live update is desired, along with live concurrent server versioning with privileged debug modes.
Depending on other folks to make any part of that work holds Wil's personal platform dream hostage to game play.
Wil wants Lathe to be usable in large scale, long term software development. A toy version of Lathe should begin to support this context by aiming to support multiple simultaneous versions of Lathe libraries, all used in one process, but by different clients who mount different views of the module system. At first it's just a problem statement.
For concurrent software versions, Wil keeps asking, "What happens when there's more than one of these?" By default, Wil assumes there will be more than one of everything, including objects that seem "global" in scope. Then Wil asks how clients of these objects can keep them straight.
Keeping these options open should look like unnecessary complexity that could be eliminated from a toy language; but they won't be unnecessary in a later real language. The toy version isn't a target deliverable — it's just a milestone. So naming systems will aim to keep as much flexibility as possible, even when it's more confusing than a toy needs.
Wil has little to no interest in cooperation outside of work contexts in day jobs, where collaboration is simply essential and unavoidable to be effective.
In Wil's experience, most tech alliances produce no result beyond advancing reputations of alliance partners, which is apparently a main goal in creating tech alliances. Wil has no interest in fluffing up his reputation, so the value of gathering prestige through cooperation has no appeal.
In 1968 Melvin Conway introduced Conway's Law which says: "Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations." If Wil allied efforts with another party, the expected result would be a system with two distinct, loosely connected pieces with slightly different views of the world, remaining slightly at odds.
Wil hopes to get a simple and seamless system of parts fitting well together, all clearly doing specific jobs apparent to Wil's eye so a consistent global perspective works.
If working with someone could still yield a cohesive result, and could happen faster than Wil's doing it alone, then cooperation might have some appeal. But neither of those seems likely, if for no other reason than Wil doesn't think like you.
Wil politely ignores offers of help; public conversation is fine. Wil has a no-strings-attached model of interchange.
mu, toy, peg, imm, tag, box, symbol, token, number, bigint, class, method, reader, writer, eval, env, vm, gc, world, pcode, compiler, asm, lathe, lisp, smalltalk, design, weight « Þ, jar, card, harp, debug, profile
(thorn, todo, names, fd, iovec, assert, log, run, hex, crc, buf, in, out, quote, escape, compare, file, deck, cow, arc, blob, tree, slice, rand, time, stat, hash, heap, node, primes, page, book, pile, stack, atomic, lock, mutex, thread, map, meter, list, iter, ctype)
"Do you think this toy language makes you look professional?" Dex challenged. "Toys aren't professional."
"Who cares?" Wil replied. "No, actually I suppose the correct answer is: I don't give a damn. The project is largely for personal amusement, with minor real use potential."
"What makes you think it has any potential?" Dex asked.
Wil shrugged. "What purpose is served by justifying my own personal opinion?" Wil replied. "And what makes you think I care about your view of my judgment?"
"Well, obviously you just want attention," Dex asserted. "Why do you think you deserve any?"
"You're an idiot," Wil snapped. "Attention just leads to nuisance harassment, like you're giving me right now. I'd tell you to get lost if I thought it would do any good."
"Maybe I'll go then," Dex half rose from his seat.
"Don't let the door hit you in the ass on the way out," Wil waved. "Go on, shoo. Go snipe where someone cares."
Dex sat back down. "I wasn't done yet," he noted.
Wil rolled his eyes, muttering, "Whatever."
hack «
Dex spread his hands and acted like he was measuring something he wanted to say carefully. "I've been reading the toy language code," Dex sniffed, "and it's — I hate to say this — not much more than an amateur hack."
"No shit," Wil said. "What was your first clue, Sherlock?"
"Well, several things actually," Dex stumbled slightly, maybe expecting resistance. "You ignore trappings of real language work — you not only avoid standard tools, you seem to be mocking them. And native code — how can you put that off?"
"Have you ever heard the phrase 'premature optimization' before?" Wil asked. "And what part of the word 'toy' don't you understand? You can't be that dense."
Dex contrived to look down his nose. "Aren't you embarassed to publish amateur scribbling?" Dex needled.
profanity «
"Fuck off," Wil replied cooly. "I knew I should have put profanity in the license name. The kiss-my-ass license, or better yet the shut-the-fuck-up license. Are you any good with acronyms? When I say KMA or STFU, will you figure out what it means? Not that it would do any good, of course, but I might feel better."
"Ha," Dex crowed. "You broke first. And I have it on tape. Wil cussed me out. I can sell it on eBay. Say goodbye to your vaunted reputation now. You're a low class clown with no style."
"You're so tedious," Wil sighed. "If you have a story you wanted now, can you please leave? I have stuff to do."
"No I'm just getting started," Dex pulled out a pad of paper. "I have some notes. Oh, yes, here we go. Um, why don't you just write your application using an existing language?"
"What application is that?" Wil asked. "The app is the language. I make tools, you moron. There isn't any other app I want to make. I know all your idiot friends are writing 'applications' aiming for market share, but I'm not."
"You could write your language with an existing language?" Dex suggested. "You don't need to do your own."
"I'm writing it in C++," Wil said slowly. "Or did you miss C++ was an existing tool I didn't make myself?"
"I mean if you like dynamic languages, why not write it in one of those?" Dex wondered. "Isn't that what you want?"
"I didn't notice which one was written by me," Wil drawled. "I want one under my control, without interference."
selfish rules «
"That's pretty selfish," Dex charged. "What good is that to other folks? Wouldn't it be a better use of your time to focus on a useful project worthwhile to society? For the public good?"
"I don't care about what's useful to society," Wil said through clenched teeth. "I pay my good little zombie robot dues at the office. Fuck other people. On my time I do what I want. You party in the city at night, I write software for fun — fun to me."
"That's going in my blog," Dex smiled evilly, writing. "Wil says, 'screw other people, I'm the only one that matters.' Wil, you need to get with the program and make whatever the eloi demand. You're behind your quota. No one likes an uppity robot."
"Fuck you," Wil said without emotion.
"Come on," Dex coaxed, "say it like you mean it. You know, it loses it's charm when you keep saying it. Why not save it for special occasions? Going to say it all the time now?"
"No, we can do it by rule," Wil explained. "The rule is: every time you assume I'm at your service, or the service of humanity at large, the reply is the same: fuck you. So when I ask, 'What's the rule?' do you think you can remember what that means?"
"I don't know, I'm pretty stupid," Dex threw back. "After all, I am an idiot, you know. Will this be on the test?"
"Your time's almost up," Wil prompted. "Anything else?"
servers «
"Yes," Dex held up a hand. "You want to write servers. Why not use someone else's language there?"
"The answer has several parts," Wil warned. "Most importantly, I want a server to host the language itself, which I already told you I wanted to do. Instead of a batch compiler, I'm more interested in a process that stays up and lets you ask questions."
"That sounds ... strange," Dex puzzled. "What kind of questions? Integrated development environment questions?"
"Yes, and app questions when an IDE and app have no compile time versus runtime separation," Wil replied.
"Your toy language will do that?" Dex asked dubiously.
"Did I say that?" Wil snapped. "Obviously a toy language won't be able to do something later developed using it. Later is the key word. You're the one who asked about servers."
"Well there are dependencies ...," Dex muttered.
"Yes, and how I feel about dependencies is a motivation for the toy language," Wil agreed. "The phrase you're searching for without knowing it is 'software stack.' It implies a bunch of interrelated things that leverage each other somehow."
"Leverage is a bullshit corporate word," Dex objected.
"Yes, but when one person says it," Wil suggested, "maybe you should think of simple meanings like 'a machine can amplify the strength of one person' like a lever or a lathe, etc."
"Can you work inclined planes in too?" Dex joked.
"Shut up," Wil grouched.
"Any features you want to develop?" Dex asked.
"Yes," Wil granted. "Some of my toy language motivation is anticipating future fun of coding some things."
"Like what?" Dex prompted.
"None of your business," Wil said. "What's in it for me?"
"Uh," Dex struggled. "Saying what I'm thinking might make you ask, 'What's the rule?' Why do you describe some things? Why not future features you think might be fun?"
"The only things I describe are exactly what I'm ready to show right now," Wil explained. "That doesn't include future stuff. If I did, what would you call it?"
"Vaporware, of course," Dex nodded. "But I'd save that attack until after you were done entertaining me."
"Suffer," Wil suggested. "Entertain yourself."
"Any hints?" Dex whined. "Something new? Inventions?"
"No hints!" Wil barked using the same gesture and tone of voice E uses to say no capes in Pixar's The Incredibles. Then Wil laughed evilly. "Actually, I might plan to do nothing new whatsoever. Maybe I expect to enjoy old things new to me."
"Before we wrap things up," Dex added while taking notes, "what do you want fans to know about the project?"
"I don't want fans," Wil declaimed. "I want people to leave me the hell alone. That includes you by the way."
"Come on," Dex looked sidelong with doubting eyes. "We know that's just a line. Secretly you're eating up the idea of tiny smidgens of precious social approval."
"No I'm not," Wil grinned savagely. "If the result of telling folks to buzz off actually works, that would be a perfect result. I don't need readers, followers, or anyone for any purpose."
"You are so full of shit," Dex insisted. "How will you get users otherwise? You've got a full-on case of build-it-and-they-will-come — admit it. You crave users: they provide credibility."
users «
"What would I do with users and credibility?" Wil asked. "You keep projecting an agenda you suppose I have about the future. What if I just want to use my work without losing control of it or having people waste my time?"
"Just use it yourself?" Dex puzzled. "What good is that? No, what you want is social approval and respect. To get those you need to make others envy your winning technique of making yourself the center of attention through your work."
respect «
Wil was shaking his head. "Why should I care about respect from random people? Or respect from techies? Most of whom are idiots, I'm sad to say, much like yourself."
Dex's smile was slipping. "You keep talking like that and I'll start to believe you," he warned. "I did kind of like you before, but I can only take so much of this. Mutual respect is an important part of social interchange. I'm not getting respect here."
"That's because I don't respect you," Wil explained.
Dex's expression took on a stony quality.
"Which half explains why I don't care about your respect for me," Wil continued. "If you ran across a bunch of six year old kids making mud pies in the rain, would you care about their criticism of your poor mudpie making skills? Would you care if they partially understood what you were doing in your workshop?"
Dex was appalled. "That's so unacceptable. Where do you get off comparing other people to children? You're just, ah, you ... I don't have to take this. Fuck you." Jabbing an index finger.
"What's the rule?" Wil asked. Dex growled in anger.
status «
Dex collected his things stiffly. When Wil picked Dex's pen up off the floor, Dex almost slapped it out of his hand.
"Everyone does things for social reasons," Dex said aloud, more for himself than Wil. "Social status is the only thing that matters. Everything we do aims to improve status in eyes of others. Only losers tell themselves differently."
"Your self esteem requires you believe everyone shares your scoring system," Wil suggested. "But why score whores want to live in your world though is beyond me."
"I pity you," Dex sneered.
"You skipped part of Buzz Lightyear's line," Wil corrected. "First he says, 'You're a sad, sad little man.' Then he pities."
Dex froze up, mouth working slightly to avoid smiling.
"I want you to know," Wil said gently. "I don't think less of you each time you shout to infinity and beyond! in your zeal."
"Shut up," Dex finally smiled. "It's hard to hate you when you're funny. Why do you have to be such a jerk?"
"Somebody poisoned the water hole," Wil imitated Woody.
