design — aspects of toy programming language lathe differing from a lisp dialect with a smalltalk class system are discussed on this page. (Code uses þ library under a mu-babel license. See mu for broad context including a focus on usefulness to the author, or see toy for update notices.)
Characters Zé and Wil explore edges of topics resulting in Lathe features and plans. An earlier intro for Wil implies Wil (as a character) writes Lathe, sets priority, makes decisions, and gets anything he wants. So Wil lands things concretely after brusquely pruning options until results fit in time available, versus the role of Zé as noted column right.
"Why are you inventing a new programming language?" Stu asked. "Don't we have enough already?"
"I'm not inventing a new programming language," Wil replied. "I'm only coding two old ones: Lisp and Smalltalk. Newness for it's own sake is not a goal: it's to be avoided."
"What part is new then?" Stu asked. "And why do it?"
"Runtime and implementation is new," Wil said. "Libraries cherry picking what I like is new, as is mixing Smalltalk style method dispatch into a Lisp like retro Scheme. Why? Because I liked it the first time I did it, around 1991-92."
"I don't understand," Stu puzzled. "Why is that a new language? Isn't Lathe just a new code name? And aren't those new things just trivial features?"
"You understand perfectly," Wil countered. "It's just a new code name for a very old project. And yes, new bits are just features. New features don't make new languages."
"I suppose you're right," Stu pondered. "C++ evolved a lot, and no one said it was a new language, even when things like templates were added. Are names just marketing?"
"Names are sometimes marketing," Wil nodded. "At times they inform and at times they obscure. Lathe stands for lambda on thorn, informing about Lisp syntax and C++ runtime — the most salient aspects of form at top and bottom."
"Syntax alone says little," Stu objected. "So the name obscures how much it resembles anything else. Since you haven't committed to either Scheme or Smalltalk, it sounds like Lathe is a new language: a rogue elephant."
"Okay," Wil stipulated. "And folks in neither community seem interested in both joined in one environment, so it's a bit like a Reese's commercial where two people collide and say 'you got chocolate on my peanut butter'..."
"And 'you got peanut butter on my chocolate'," Stu chimed. "So why mix them? Why should anyone else care?"
"I don't know," Wil shrugged. "And I don't care. Mixing them is my preference, and there's no accounting for taste. I'm only explaining a design, not saying anyone will care."
"Hey, sorry I was late," Zé rushed in and took a seat. "Thanks for sitting in for me, Stu. I can take it from here. I'm familiar with the old Ygg codebase already. See you later."
"Oh," Stu got up. "I see I'm a third wheel. Bye."
"What did you use Ygg for again?" Zé asked. "It's been over ten years since you gave me a long run-down."
"Hey, that's relevant to the design of Lathe," Wil said, "since they're related and have similar purposes: to provide a pure-oo style dispatch when I want it, with plain vanilla Lisp syntax, for scripting code I write mostly in C++."
"Folks use other languages for that already," Zé noted. "Like Python. I hope you don't expect anyone to switch. No? I didn't think you were that dumb. When was that?"
"I think the first code was circa 1989," Wil said, "but it didn't get a Smalltalk class system until around 1992 I think. I used it as a scripting language to drive interactive tests for C++ code. I tested everything with it — Ygg was perfect."
"Except it had warts and you didn't know as much then," Zé gestured. "How did you wrap a C++ api for use in Ygg?"
"I did it by hand at first," Wil explained. "Then I developed a tool — a friend helped with stream editing — turning a C++ header into Ygg code that generated glue code."
"Sounds a little like SWIG, right?" Zé asked. "When did that come out? Around 1994? Yours was 1992?"
"Yes," Wil nodded. "And I'm sure swig was great for that task; but I kept using my own tool until around 2001 or so."
"I take it Ygg was your preferred way of testing IronDoc, for example?" Zé prompted. "How'd that go?"
"Yes it was great," Wil said. "In recent years, in day jobs, it's been extremely awkward to have no decent approved scripting environment to drive libraries I write."
"You should have maintained Ygg then," Zé chided. "It's your fault. When did you decide to do something about it? And why don't you just switch to, say, Python now?"
"When I started this site," Wil said, "I just wanted to follow through on some things I spent an awful lot of time on. At one point I decided to just upgrade Ygg to use thorn."
"Does using thorn have something to do with avoiding Python?" Zé persisted. "Wouldn't that work?"
"Probably," Wil nodded. "Except I don't want to write servers in Python. I like Lisp and Smalltalk more. It's about having fun. And I want to control the runtime as much as possible."
"Just because?" Zé asked. "Or for a reason?"
"My day jobs have been about controlling runtimes for a lot of years now," Wil said. "If I didn't do it in my own projects, I'd get a result inferior to what I know."
"Isn't that a bit pretentious?" Zé needled.
"Not my problem," Wil shrugged. "I don't care what it sounds like. I just want to scratch itches without a hassle. I learned it in my own side projects; now I want it to bear fruit."
"Is a toy language good enough?" Zé wondered.
Wil shrugged. "I'm not picky anymore, and Ygg was fun. So sure, a new toy in the form of Lathe sounds good."
"Let's talk a little about the design now," Zé suggested. "That's enough background context."
"What will the language syntax look like?" Zé asked. "Are you going to do some weird hybrid with Smalltalk?"
"A Scheme parser should accept the syntax," Wil said. "But I might add some new lexical extensions to include tokens Scheme would not like, but make sense in Smalltalk or C."
"C's number syntax, too?" Zé asked. "Why do that?"
"It's easier to re-use code if I don't have to change number syntax as often," Wil said. "And I'm not concerned about having Lathe source be read by Scheme. If a token can be understood as number, then why not? I'd rather include more."
"So basically it will look like Lisp parse trees," Zé summarized. "But atom tokens will be more inclusive than Scheme. What will Smalltalk method dispatch look like?"
"First let's talk about Scheme a little," Wil said, "I want code in Scheme to just work without change."
"That means you'll have the same special forms and primitive methods?" Zé asked. "What about macros?"
"Yes, but I'll only support a really early form of Scheme," Wil said. "So I don't think the Scheme community will consider it Scheme. I don't like syntax macros much. So I'll do a simpler verision of macros, like I did in Ygg."
"Better than nothing," Zé opined. "So it if looks like Scheme, generally Lathe does the same thing as Scheme. But how do you fit Smalltalk behavior in there? Like Ygg?"
"Yes, the way Ygg did," Wil nodded. "The following Scheme expression adds integer constants 3 and 4 because + is bound to a procedure which knows how to add the arguments."
"Okay," Zé granted. "Now show a Lathe expression."
"The last one is a Lathe expression, too," Wil said. "But Lathe also executes the following with the same result. It's illegal in Scheme because the first elem is an integer."
"Um," Zé tried to recall. "When evaluating a list, Scheme requires the first member to evaluate to a procedure or macro, if it's not a special form symbol ... right?"
"That's right," Wil said, "But in Lathe, if a first list member does not evaluate to a procedure or macro — when not a special form — it does a message send instead."
"In that case," Zé guessed, "does the second member of the list need to be a symbol? For the selector sent to the object? Like this expression."
"Like (vec at: 3 put: $c) in Smalltalk," Wil said. "Yes, the second list member must be the selector symbol. Your example does the same thing as this in Scheme."
"Cool," Zé smiled. "But I suppose the at:put: version would make Scheme folks barf. Is it slower? What advantage does it have? Why would I pick at:put:?"
"I don't care if they projectile vomit," Wil joked. "Scheme folks aren't my concern — I'm the user I care about. A message send is duck-typed, so the at:put: version works with any object understanding that message. It's polymorphic. Speed would depend on dispatch optimization."
"Are there any other syntax differences between Lathe and Scheme?" Zé asked. "Besides more legal tokens and turning an error case into a polymorphic dispatch?"
"Oh, more top level expressions, and special forms," Wil said. "How else could I send a message to a procedure? So I'll add a special form that means send for that."
"I guess you'd need special forms to define classes and such," Zé tapped his chin. "But would it still just be Scheme syntax, if you ignored the semantics of classes?"
"Sure," Wil nodded. "Should we talk about semantics?"
"No, first I want to ask about adding Smalltalk syntax," Zé insisted. "Can Lathe parse Smalltalk syntax?"
"I'm sure I'll write a Smalltalk parser using Lathe," Wil said. "But I'd call that syntax Scythe for smalltalk on thorn, to reduce confusion in folks who think syntax means language."
"Could both Lathe and Scythe run in the same process on the same runtime?" Zé asked. "Some methods written in Scythe and others written in Lathe?"
"I don't see why not," Wil shrugged. "But then folks reading code would need to know both. But it's a reason to make selector names the same in both syntaxes."
"Macros would be easier to write in Lathe syntax," Zé considered. "Would you add macros to Scythe?"
"Get real," Wil rolled his eyes. "I suggest folks write macros in Lathe. Macroizing Smalltalk isn't my problem."
"It's really like you just want both languages to interoperate in one runtime," Zé concluded. "How about Python, too?"
Wil narrowed his eyes. "What about Python?"
"You could parse Python syntax, then interpret or compile that, too," Zé conjectured. "Wouldn't that work?"
"Maybe," Wil equivocated. "But I won't have that kind of time on my hands for a year or two. Might as well call it forever since anything can happen in that time."
"I have a semantics question now," Zé warned.
"What about threads?" Zé asked. "Could more than one native thread run in Lathe at the same time?"
"I'll aim that way in the long run," Wil said. "In the short run, maybe just one native thread per VM in the same process. So two threads would work in two VMs."
"Can two VMs talk to each other?" Zé asked. "So one process could divvy up tasks in pieces handled by one VM in one thread per each piece of work?"
"Sounds good to me," Wil said amiably. "When I get two VMs to talk to one another, that ought to work. I don't know why I'd do that when one VM isn't very complete though."
"I'm just worried about mutable globals used by an interpreter or compiler," Zé explained.
"I'll try to avoid those," Wil assured. "But if worse comes to worst, I've done a lot of fine-grained locking in C++ before to share such things with low contention."
"Well, keep your eyes peeled," Zé jabbed a finger in emphasis. "Stay on top of it, so threads are supported."
"Any other issues besides threads?" Wil pleaded.
"Let's see," Zé pondered. "How much of Lathe will be implemented in C++? For example, you might write a lot of non-gc features exposed under a gc api."
"Yes, I could," Wil agreed. "Whatever is convenient. It's more likely in a feature sensitive to i/o speed in volume: I'd try to keep it out of gc space."
"And you can't really pass a pointer for an i/o buffer that might move to a C system call," Zé reminded.
"I expect to add non-gc buffers on the C++ side, yes," Wil granted. "But it's likely to be ad hoc for a while. It definitely figures into IPC for a Lathe VM too."
"How will the Smalltalk class system affect the Scheme view of things?" Zé asked. "That's all I think matters in semantics: Scheme or Smalltalk, or interference."
"I see them as orthogonal," Wil considered. "If you don't use Smalltalk features, they won't affect you."
"Unless you use them yourself implementing Scheme features," Zé guessed. "You can't rule that out, can you?"
"Okay, there's that possibility," Wil agreed. "And I'll also have to invent some way to manage namespace visibility, so Scheme won't see Smalltalk names except on purpose."
"Hey, I know: what about generic functions?" Zé asked. "You were working on your own Dylan compiler at Taligent. Adding any Dylan features to the mix?"
"I didn't like generic functions much," Wil squirmed. "No, it will just be vanilla old Scheme and vanilla Smalltalk. No exotic banana split features. Maybe token extensions."
"Maybe SRFIs: Scheme requests for implementation?" Zé suggested. "Can you follow those when possible?"
"Sure," Wil nodded. "And I'll look at other Scheme implementations for handy, small, useful features, too. When I have time. But I'll try to follow SRFI api specs."
"The more you use existing api specs, the less work to do writing docs, huh?" Zé realized.
"Damn straight!" Wil agreed enthusiastically. "Making stuff up from whole cloth is the last thing I need."
"Hey, what about static typing?" Zé suggested. "You said you wanted to use Lathe in large scale development later — don't you think that would help?"
Wil covered his face with his hands. "I suppose," he said finally. "I can add some optional static type declarations. Later. Why don't you just keep tacking new items on my todo list? It's not like I have anything else to do in my life."
"Would static types be hard?" Zé asked. "And would it mess up duck-typing?"
"Not too hard," Wil said. "In 1994 my day job was adding polymorphic types and 'declarations' to a visual programming language. I remember the basics. It needn't affect duck-typing at all — it would just generate static type errors or warnings when types didn't look right. Dispatch can remain the same."
"I wouldn't ask for static type add-ons," Zé justified, "but I'm worried about hacking very large codebases; it would be nice to get compilation warnings about api mismatches."
"You know, everything is doable," Wil claimed. "But there's only so much time."
"Is this where I remind you computationally intractable problems are not 'doable'," Zé teased. "Sorry, you know someone else is going to say that too. Noobs are so predictable."
Wil brushed it away.
"Tell me about digital trunk dialing," Zé demanded. "I must know everything."
"What?" Wil asked, deeply puzzled.
"Sorry, that was a quote from Time Bandits," Zé excused sheepishly. "It just kinda slipped out. Ha ha."
"You get loose around the edges after a while, don't you?" Wil chided. "Concentrate. Find your next can of worms."
"Right. Okay, are you going to do Smalltalk style images?" Zé asked. "You know: a snapshot of the whole world?"
"Good question," Wil said. "No, I won't have Smalltalk images. That's not a language feature, it's an environment feature. Why do folks lump everything together?"
"They just do," Zé shrugged. "It happens in Java too; the language, the libraries, and the environment are all mushed together in one big marketing ball of wax: Java."
"Well, when I talk about adding Smalltalk to Lathe," Wil clarified, "I only mean basic language stuff — not libraries and environment features."
"Well in theory you could parse and use existing Smalltalk libraries, couldn't you?" Zé asked. "As long as they didn't depend on things Lathe is missing?"
"Oh, heck," Wil gestured expansively. "Let's find out by trying it, okay? Is that too empirical for you?"
"I know you're not going to generate native code for a good while," Zé assured, "but what about compiler design? Anything interesting to say there?"
"I can do a Smalltalk blue-book style pcode compiler first," Wil said. "I'll just add a tail call opcode. I wrote one of those when I was at Taligent, around the same time I added a Smalltalk object system to Ygg, in 1992. Byte code compilers are easy."
"Don't you have to settle on a target virtual machine first?" Zé asked. "You have no idea how many times I've heard geeks discuss the ultimate VM to use. Java usually."
"They're all pretty similar in some ways," Wil shrugged. "I wrote an ad hoc virtual machine for assembling web pages in a reverse proxy server several years ago. The compiler and VM just accomodate each other."
"Didn't you say you wanted to support multiple instruction sets at once?" Zé asked, "That was a few years ago, in the Mithril design. Does that still apply?"
"Sure," Wil nodded. "I can have more than one compiler, and run code compiled by each all at the same time. As long as 'code' is typed for the engine to execute, it will work."
"What about cross-engine calls and continuations," Zé asked. "Can that cause a problem?"
"I don't see why," Wil said. "All you do is make sure function pointers are annotated by execution metainfo. I can even mix pcode, native code, and interpreted parse trees."
"You could debug just-in-time compilers by working with pcode variants," Zé suggested. "Before native code jit."
"Nice idea, I'll give it a shot," Wil nodded.
"You know, I have a theory about what you want to get out of your Lathe options," Zé proposed. "It's choice."
"Choice?" Wil asked. "To do what?"
"Whatever you want," Zé said, "As opposed to what you're told to do. You're removing constraints, to provide affordances letting you choose any technique for a task at hand."
"Interesting theory," Wil said politely. "But is it testable? Or is it just incoherent bullshit? Pardon my french."
"Let me give examples," Zé said. "Note each concerns your response to someone saying, 'You can't do that.' Your response is always, "Yes I can, if I choose to do that.'"
"I won't get it without examples," Wil shook his head. "What am I told I can't do?"
"Use garbage collection," Zé raised an eyebrow. "Write functional code. Use homoiconic syntax. Use a message passing actor model, or continuations. Have short names."
"If I use someone else's Lisp I can have those too," Wil furled his brow. "What are they saying I can't do?"
"Allocate memory manually, anywhere you want, any way you want," Zé replied. "Write really imperative code. Write C++ libraries. Control the runtime. Use simple macros."
Wil pondered, trying it on for size. "You're saying I want to have my cake and eat it too? All options at once?"
"Yeah, you don't want to be forced to use any limiting model," Zé said. "Lots of folks invent systems aiming to 'improve' the world by preventing some things."
"Like languages making it hard to modify state," Wil suggested. "Or hard to do systems programming? Or insist syntax looks one true way, or you can't play?"
"Now you're getting it," Zé encouraged. "You hate any time you're told 'our way or the highway' because it's good for you, or when folks demand denial in the name of a cause."
"You're just saying I want chaos," Wil demurred.
"No I'm not," Zé argued. "Lexical scoping can be used to bound rule changes, grouping rules in boundaries. Chaos is absence of boundaries — you want flexible ones."
"How do you test that?" Wil asked. "Sounds like a rationalization. What would audit the boundaries?"
"A compiler, or an IDE," Zé said. "Mixed models are not chaos if divisions are declared statically, and you can check."
"Wow, I didn't know I had such lofty plans," Wil said. "You can be my publicist. You make itch scratching sound like a crusade to unchain folks from fetters. I might run for office."
"No you're clearly selfish," Zé said. "But it's a perfectly egalitarian selfishness others can share. You don't require others play one way while you play yours."
"Look, skip any saintly overtones," Wil said. "I'd rather cast myself as a jerk: cuts down on prayers and co-opt moves. Nice to know there's method to my madness, though."
"Now, when someone says 'You can't do that' what do you say?" Zé prompted.
"I'll see you in hell!" Wil snarled. "Jim Carey said that over and over in this one great episode of Saturday Night Live. See, now you've got me making jokes."
"Uh, okay," Zé looked askance. "I'd say 'Yes I can' works with more audiences. It's direct and lacks that abusive quality you find so funny at the oddest moments."
"Aww!" Wil channeled a petulant 13 year old.
"I think you skipped over some potential Lisp and Smalltalk compatibility problems," Zé realized. "Like arity."
"They're both equally hilarious," Wil goofed.
"No, arity not hilarity, you twit," Zé mock slapped. "The number of arguments taken by a method."
"Oh, yeah," Wil had a dawning look. "It's all coming back to me. Lisp has lots of variable arg-count methods."
"And Smalltalk doesn't?" Zé asked, then watched Wil chew his cheek in concentration for a minute.
"I don't think so," Wil reported slowly. "There's unary, binary, and keyword methods. None are variable. Let's check google: I'll search for 'Smalltalk vararg methods'."
"What'd you find?" Zé leaned in to watch Wil read.
"D'oh!" Wil yelped. "As I feared: folks say Smalltalk doesn't do that. Workarounds discussed are awkward."
"Did you think of a way to fix that in Mithril designs?" Zé asked. "You did, didn't you? Why the long face?"
"It involved making up new stuff," Wil admitted. "In retrospect, it was more of a rat-hole than I realized. You know what a rat-hole is, don't you?"
"Of course!" Zé sang. "It's a near pointless wild goose chase over some exasperating technical detail. It's a primary risk for time waste in meetings ... or hobby coding."
"Yes," Wil nodded. "Well, I thought of a way to do variable Smalltalk arity, but it involved some weird new syntax to express, for example, splicing one vararg to another."
"Ew," Zé wrinkled his nose. "Weird new syntax puts me right off. But, uh, ... is it interesting?"
"Kinda," Wil smiled at this lapse in conviction. "But let's skip it for now, okay? Chalk it up as a minor issue."
"You're not going to tell me, are you?" Zé asked in alarm. "You scoundrel. I'll have to work it out myself."
"I remember another odd Smalltalk feature," Zé changed subjects. "You can tell one object to become another object, and that makes them swap identities."
"Oh," Wil perked up like he forgot to turn off the stove. "Oh yeah, that's right. Smalltalk had an object indirection table, so objects were able to swap table indexes."
"Do you think that's a good idea?" Zé prompted.
"It's neat," Wil said, "and makes clever class implementations possible. But the effect can be done some other way."
"What's the disadvantage?" Zé asked.
"Well, every box would effectively be four bytes bigger to have an indirection table entry," Wil said. "And even worse, you'd touch more memory each time you used a box."
"More cache line pressure?" Zé suggested.
"Yeah," Wil agreed. "But leaving that feature out would break even more standard Smalltalk technique."
"But you were going to break a lot of Smalltalk anyway," Zé reminded. "I don't know why you're bothering to say Lathe has a Smalltalk inside. Libraries count too. It matters."
"Yeah," Wil sighed. "I'll break so much taken for granted in Smalltalk, I can't worry about become: for objects. Folks will just have to add an indirection themselves."
"You sweated over a feature that was just indirection folks take for granted?" Zé marveled. "Get a grip."
"Smalltalk's a bigger language that Scheme," Wil warned. "So minimalism in Lathe will pare much more from Smalltalk. Maybe I should call the subset Gab, like I planned."
"If it was me, I'd link fictitious citations to a Gab subset dating back to the 80's," Zé smiled. "Would that be bad?"
"Whoa," Wil laughed. "It'd be funny, but a lot of folks don't have any sense of humor about history."
"Fiddlesticks," Zé said. "Past and future don't exist anyway. Time is just the rate of change in the now. Timelines are just models we use for simulations in our minds."
"Your perspective is, uh, very strange," Wil said quietly. "I wouldn't say that in front of normal people. What an odd view for someone who loves time travel stories."
"Why do you think I find them so amusing?" Zé smiled. "The whole idea of the past as a place you can go is comical."
"You must not be a big calendar fan," Wil said. "How do you think about schedules? Oh, I guess that rate of change thing works. Do me a favor: don't mention it again."
mu, toy, peg, imm, tag, box, symbol, token, number, bigint, class, method, reader, writer, eval, env, vm, gc, world, pcode, compiler, asm, lathe, lisp, smalltalk, design « Þ, weight, jar, card, harp, debug, profile
(thorn, todo, names, fd, iovec, assert, log, run, hex, crc, buf, in, out, quote, escape, compare, file, deck, cow, arc, blob, tree, slice, rand, time, stat, hash, heap, node, primes, page, book, pile, stack, atomic, lock, mutex, thread, map, meter, list, iter, ctype)
Zé is another character closely resembling the author in some ways, though less realistically than Wil because Zé is stuck with a function of being interesting — which won't appear much in toy language material because it would distract, since Zé has a lot of license. He's interested in too many things, brainstorms too much, considers too many options, and tends as far toward rich fantasy as Wil does to lean realism.
Note Zé is mainly destined for later fiction, but you can see an earlier appearance in a story (critiqued on 01mar08). His function in toy language material is to represent ideas not immediately relevant when they recall history, anticipate future work, explore options, or generally think outside a box bounding Wil's narrow focus. Zé guesses a lot, and his guesses always match Wil's even when Wil doesn't want to talk about that right now. Zé is about more to the extent Wil is about less. With respect to language suggestions, Zé is good cop to Wil's bad cop, and must moderate the scope of his ideas to avoid upsetting Wil's minimalist aims.
The two of them represent an almost clean yin/yang separation, except for deliberate contradictions — such as acting like one another — added to avoid reductionist effects. At times they might act like caricatures of imagination and executive function, which is what you're supposed to suspect (that they're right and left lobes, visual versus verbal, creative versus logical), even though this isn't quite right. They actually have the same abilities, but divide roles for convenience and efficiency, and to pull pranks on others.
Although he has implemented languages himself, Zé isn't directly familiar with Wil's current toy language code, despite having discussed earlier versions years ago. So usually Zé responds to first looks at this one.
Because toy language material is a mix of done and undone parts, plus even more planned and unplanned parts, Zé can help in dialogs to stratify content along such lines.
On LtU A couple years ago, in June of 2006, in a thread called Let's make a programming language!, one of the regulars (at that time) named Frank Atanassow wrote an interesting post, well received by the community and referenced multiple times since. All of Frank's post is quoted in light blue below, in boxed sections, so I can answer questions with respect to Lathe. I've been meaning to add a section like this to a web site ever since I first saw Atanassow's nice piece.
Even though Wil characterizes Lathe as old (just another Scheme and Smalltalk) it seems a good idea to answer nice pointed questions like these anyway, because incisive questions are good for taking measure of plans. Since Wil pitches absence of uniqueness in Lathe, absence of more design detail might make sense. (Note all links in quotes below go to the same entry on one LtU page.)
Here are Frank's questions:
Some words of advice on language design
Before you go off inventing new programming languages, please ask yourself these questions:
- 1. What problem does this language solve? How can I make it precise?
- 2. How can I show it solves this problem? How can I make it precise?
- 3. Is there another solution? Do other languages solve this problem? How? What are the advantages of my solution? of their solution? What are the disadvantages of my solution? of their solution?
- 4. How can I show that my solution cannot be expressed in some other language? That is, what is the unique property of my language which is lacking in others which enables a solution?
- 5. What parts of my language are essential to that unique property?
"Those questions lean toward wanting a justification for a new language, don't they?" Zé asked.
"I guess so," Wil said. "But I suppose similar questions apply to anything you do to existing languages. It's about grounding evaluation in specifics, not generalities."
Atanassow: If your answer to 1 is "it's cleaner", go home. If your answer is "it has this very small core which everything is definable from," nobody cares. (Well, I might care, but you will never convince me it's interesting without some mathematics. There is already a one combinator basis for untyped computation. SKI was known decades ago. For typed languages, it's more complex but also mostly pointless.)
"So, what problem does Lathe solve, Wil?" Zé asked.
"It lets me use C++, Lisp, and Smalltalk style conventions, all in one system, whenever I see fit," Wil replied. "I can use a tool fitting tactics and strategy I want to apply in each spot."
"Then you're just assuming C++, Lisp, and Smalltalk solve some problems well?" Zé asked. "Is there any particular part of each you find especially helpful?"
"C++ is good for high level assembler work," Wil said. "I want Scheme for full continuations and facile functional mapping. And I want Smalltalk for actor style duck-typed messaging."
"Okay, I see how leaving one out would deprive you of something," Zé granted. "Though I bet you could do Smalltalk features in the Scheme part."
"Another problem Lathe solves is providing me with a system I understand completely because I wrote all of it," Wil continued. "Which doesn't change except when I make changes, so the amount of time I'm in the flow is off the charts."
"How nice for you," Zé deadpanned. "I suppose you're claiming that happened with Ygg. I can't argue with that. How does that do other folks any good? Do you even care?"
"If one person knows all of Lathe," Wil hypothesized, "it couldn't be that hard for another single person to learn it all too. As a toy, cognitive footprint can't be large."
"I hope you're done with the list of problems solved now," Zé said. "Any last reason to prefer Scheme and Smalltalk?"
"Actually, yes," Wil said. "It was very convenient that almost no complex syntax was involved. I didn't spend any time on syntax, so more complex syntax would be a net loss."
Atanassow: If your answer to 2 is, "I will write programs in it after I have a prototype," then you have not thought very carefully about it; also, your feedback cycle is too long. If your answer involves more than one buzzword, you are kidding yourself.
"How do you show Lathe will solve those?" Zé asked.
"The earlier Ygg version I used in the 90's for hundreds of hours showed me I liked it," Wil said. "I'm only repeating parts I actually used a lot. Motive is practical, not theoretical."
Atanassow: If your answer to 3 is, "I don't know," then you don't know enough. There is always more than one solution. (Trust me; I never write, "the solution to this problem is..." anymore, and when I read it and I don't see a proof of uniqueness, the writer invariably turns out to be full of it.) If your answer involves only languages of one paradigm, likewise. Go study Scheme and Prolog and ML and Haskell and Charity and Lucid Synchrone and OBJ and Erlang and Smalltalk. Look at Epigram or Coq or HOL or LEGO or Nuprl. Aside from Java, these are the important ones. If you are familiar with all of these, then you are in a decent position. If you have only ever programmed in C/C++/Java and Lisp and scripting languages, you have been sitting in a corner your whole life. Perl, Python, Ruby, PHP, Tcl and Lisp are all the same language. (Scheme itself is only interesting for hygienic macros and continuations.)
"Might other languages work as well?" Zé asked. "What are the pros and cons from your perspective?"
"Other languages would work," Wil said. "Except I don't know another language with full continuations besides Scheme. Python would work as a substitute for Smalltalk; maybe Ruby too."
"So why Smalltalk?" Zé prompted.
"I know it already," Wil said, "I've written two different parsers and compilers for it. Relative cost savings over a new language is huge. And it's a rather simple language."
"Yeah, yeah," Zé sing songed. "And you know Scheme already too. Okay, so you have inertia on your side. Frank argues against new things; you argue to keep an old thing."
Atanassow: If you don't have an answer to 4, then your solution belongs in a library, not a language. In fact, the people on LtU are the perfect people to design good libraries, and libraries, on the average, are much more valuable than languages. Library design is also easier, and you won't waste as much time on syntax. (You will waste time on syntax. You will waste almost all your time on syntax.)
"Can you show other languages don't have the things you like in Scheme and Smalltalk?" Zé asked.
"Well, they have very few features other than the ones I want," Wil said. "Scheme is naked parse trees with continuations, and Smalltalk is nothing but duck-typed messages."
"So you're arguing other languages are either isomorphic and thus the same," Zé said. "Or they waste energy on things you don't care about. Sounds too cut and dried to me."
"Frank's point that what I want belongs in a library happens to be correct," Wil said. "I'm writing Lathe as a C++ library for an embedded language. At least at first."
"Oh, well you're confusing me by arguing both sides," Zé complained. "How do you get to be right both ways? And how come I haven't heard any praise for Lua yet?"
Atanassow: If your answer to 5 is "only this and this", rip those out of your language and add them to an existing language. ("Refactor mercilessly.") If your answer is, "almost everything contributes," I can almost guarantee you you are wrong. (If you aren't, then you are probably a researcher.)
"What parts are essential to the unique properties you want?" Zé asked. "Can you rip them out and add them to an existing language?"
"Yes, I can," Wil said. "I'm ripping out Scheme parse trees and continuations, plus Smalltalk message dispatch, and I'm adding them to C++. I'm doing what Frank suggests."
"Well la, di, dah," Zé announced. "You put yourself in opposition, didn't you? Or maybe Stu and I did that by making you defend Lathe. Hmm. Carry on."
Atanassow: The reason most new languages are pointless is that people rarely answer these questions honestly. That is why language design is hard, and why researchers hardly ever make new languages.
"I agree language design is hard; it's why I never make a new language," Wil said. "Do you think Lathe's a new one?"
"I have a conflict of interest," Zé excused. "But my real opinion is: kinda, sorta, I don't know. No. Maybe. Wait."
"You're a lot of help," Wil dismissed.
Atanassow: Oh, and, I know this will fall on deaf ears but: don't indulge in syntax design. Pick some other language's syntax style. Java, Lisp, Python, Haskell, it doesn't matter. Just get it out of the way and close the matter immediately. If your language's original contribution is syntactic, you are hopeless.
"Yeah baby," Wil crowed. "And that's why I like the nearly minimal syntax of Lisp."
"Why do folks love new syntax so much?" Zé puzzled.
"They hope notation does something," Wil suggested. "It only clarifies. It doesn't really change basic nature."
Atanassow: Think about language features in terms of asymptotic complexity — not of space or time, but of, well, complexity. (I would say semantics, but that's a dirty word.) Syntax changes can only reduce that complexity by a constant factor. A good language feature changes complexity from n-squared to n or to n log n. It increases modularity by localizing something which was global. The best features do so without compromising any other desirable properties (such as type safety).
"Complexity is one of my hot buttons," Wil claimed. "After school I told people design tradeoffs include complexity as well as space and time. It's at least three axes, not two."
"Do you agree with Frank?" Zé asked. "Can syntax change space or time complexity? He says not really."
"Clarity of syntax can help you avoid gross errors," Wil said. "But no, syntax doesn't magically reduce space or time complexity if you make no mistakes."
"Trying to follow a standard api might make you accept a worse big Oh complexity, though," Zé said. "It happens a lot."
"I know, I have to clean that stuff up," Wil said.
Atanassow: I would also add that there are more opportunities to innovate in typed languages than untyped, and concurrent languages than sequential. (Personally, I think the only interesting thing in sequential, untyped languages would be something involving delimited continuations or narrowing or compilation.)
"Do you care about language typing?" Zé asked.
"No," Wil said.
"No? Just no?" Zé angled for more. "Where's the meat?"
"Jesus wept, Zé," Wil pleaded.
Atanassow: Here endeth the lesson...
"That's a nice ending," Zé observed. "Sean Connery said that in The Untouchables when he played this crusty old experienced Chicago policeman during prohibition."
"Is that right?" Wil asked, rolling his eyes.
"Yeah," Zé smiled. "Do you think Frank works for the Chicago police department?"
"No, silly," Wil scolded. "I think he was in Amsterdam."
