ends versus means «
Perhaps you know the expression, "The ends [don't] justify the means." I mention this phrase only to clarify what I mean by ends and means: something like targets and techniques. An end is a strategic goal; a means is a tactical process. The end is where you're going, and means are how you get there.
I don't have any ends in mind for code I show on this site. I'm not going anywhere—there's no target goal. My only reason to show code is to explain means to do something, and sometimes to explore different means, but I don't have a goal to achieve. So it's hard to say whether means are appropriate for ends, because there are no ends. I know, you don't believe me.
You assume I have default ends, like goals many folks pursue: money, status, recognition, or some combination of those—if not immediately, then indirectly later. But this assumption is wrong. I don't see getting anything from code I present here, nor do I see it leading to software that does. My code is often useful for something later (most of it has been so far), but I don't know what that something is right now. It could be nothing.
Let's tackle an example. Let's say I show a stream class, and maybe a set of related classes using it. At this point you might ask, "What are you trying to do? What's the end goal? Isn't there an easier or less expensive way to reach that goal, instead of this?" In short, you tend to assume anyone who writes code does so in order to reach an end goal result. So you ask, what's the goal?
There's no goal. I can't tell you what it's for—it's for nothing right now. For me, that's enough reason to play with technique. Maybe I do this when you watch television; it's ritualized downtime. (Mine's better; TV is stupid.)
When I taught myself to juggle in my teens, I was asked the same thing: What is this for? Are you going to join the circus? No, I just find it interesting; and active play is a lot more fun than passive play. My idea of play has always looked a little like work to others. (And sometimes when I work, I enjoy it like it's play too, which seems wrong to a lot of people.)
Okay, I'm done. If you still don't get it, try to think of yourself as dumb (and unable to enjoy intellectual effort as a light game.)
emacs «
At work I use emacs for editing (and for several other things, like source level debugging under gdb). I've been using emacs about five years now. Before that I primarily used visual editors in work contexts. (The last one before emacs was Visual SlickEdit.) I learned emacs when I did because when I start a new job, I usually tell folks, "Show me exactly what you do." At Akamai more folks used emacs, so that's what I learned.
I'm not a big fan of emacs. But it does everything I need well enough to suit me. I used vi a lot in the 80's, but I hate it despite thousands of hours of practice. (I would say no to a job offer that required using vi exclusively; but it's hard to avoid using now and then on Linux.) At home I still use Xcode on my Mac laptop. In fact, that's what I'm using right now. (I swap between Xcode at home and emacs at work without blinking an eye, even though key bindings are not at all similar.)
I wrote this section only because I thought folks might read the next section on development environments, then say to themselves, "Hey, he should just use emacs—that does everything he wants." No, it doesn't. The idea is almost plausible, though. If I could build arbitrary C++ apps and servers in the same address space as emacs, then have complete control over the UI of emacs (including widgets I don't currently see being used), and reimplement internals of elisp, then it would almost come close. But I don't want to become an emacs developer to get there. Kitchen sink editor is not what I meant.
I want my development and production environments to be the same, with support for both C++ and a dynamic language in both contexts (development and production) with a fully programmable UI. Also, it would be nice if existing code was not a bletcherous snarl of crap so changing it was feasible. Feel free to publish pointers.
development environments «
I used to be interested in development environments as a pursuit: making better ones, especially ones usable by folks with modest coding skills. Apple's HyperCard, in the 80's, was my idea of the way to go, because newbies could make new scripts based on old ones, learning how to code in small, incremental steps. Using HyperCard, it was feasible to try things and see what happens: good experimental method, and good for learning.
If it was up to me, I'd give away free dev tools suitable for use by non-professionals. Among other things, this might cause more software to be created, but might also undercut high end salaries of professonal coders (maybe—but I don't really think so). I think absence of dev tools for non-professionals is partly due to artificial scarcity: both active and passive aggression, from some quarters, gets in the way.
There are lots of free dev tools now, yes, including both open source tools and high-end closed source tools. (For example, Apple gives away nice dev tools for MacOS X.) But I see only tools oriented toward folks who aim to develop code as a main focus. Exceptions, like maybe Squeak pitched as a toy for kids to learn coding, don't have the right stuff to take off without hand holding by professional folks who know more.
You may wonder where I'm going with this. It's about time I started making my point, but I wanted some context first. Part of the context I want to invoke is a question: Gee, why aren't there tools commonly used by non-professionals, so folks become producers of computing tech instead of just consumers alone. Isn't there obvious value in this? Note you might disagree and insist there are tools out there of the sort I claim are missing. (Fine: now stop bothering me; pretend you agree for the rest of this.)
I have several points, but no single point is very interesting, I think, to most other folks. They only matter to me. I was once motivated by pursuit of dev tools for the masses. That's what made me choose where I wanted to work, for years. First I hoped Apple would deliver, and then I put my bet on Netscape. Let's see, how do you think I made out? For example, can you use Firefox as a development environment? What if that had been a priority, for anyone? Personally, I think it's a dumb consumer app.
My first point: I don't know where to work. If I knew anyplace at all that aimed to make dev tools for the general public, that's where I would want to work. I don't know such a place; I don't think this sort of business is coming back any time soon. So I don't much care where I work. Instead, I aim only to do something interesting with folks I like and respect. For example, where I work now fits the bill, but I don't think it has any meaning; it just pays the bills. I'd rather have a purpose.
In theory I could do what I want on my own time. I once thought this was possible. But there's two problems: life is short, and software takes longer than you think. I know a lot about the latter. I have a lot of coding experience. Now I tend to have a realistic idea of how long things take. And in this case I know the answer is: too long. Note this is especially true in a contrary atmosphere unfriendly to the idea of such a thing, if only because it upsets a plan or worldview, and thus draws negative, um, influences. (If you don't know how shills work in tech contexts, I'm not going to explain.)
My second point: doing less than all of a dev environment is hard to do in useful chunks. The easy part is a language, if you want a programming language, especially if you've done one before. (And doing one again gets less interesting on each repetition.) A naked, standalone interpreter/compiler is not a dev environment. You also need an editor, machinery to hook up scripts and widgets, a debugger, and task-specific browsers. Compared to coding a language, which can be grown in pieces, these are all hard to approach incrementally, and still feel like you're being constructive. (For example, instead of starting your own editor from scratch, you can learn someone else's; but then you aren't actually making anything.)
Said another way, doing a small piece of a dev environment is a little pointless. Sure, it's a great learning exercise, but what if a learning exercise wasn't the goal? Working with other folks on a dev environment would make a lot more sense. As a job this would be perfect; but as a hobby diversion, this could be very time consuming. (Collaboration can be costly in communication, even if others want the same thing you do.)
I'd start a third point on desired dev environment features, but it would bloat obscenely and digress wildly as I fleshed out a wish list. What's the main problem with that? I know I'm not going to do it, so why give shape to still-born ideas? However, I might be able to characterize a consistent slant to the ideas. Okay, I'll try that:
My third point: both scripts and streams should be central to dev tool behavior. Selections should be potential sources or sinks of stream flows, and every UI behavior should be a script you can edit. A dev environment itself should be progammable using the same tools afforded by the environment. Everything a user does should invoke an interpreted script, which can be changed dynamically at any time. These can invoke statically compiled code for performance, but there's no reason to hardcode a UI so it can't be changed. For extra credit, make the backend engine a server, whose frontend can be in another process, or not; then support HTTP in the backend too, so you can browse the engine with a web browser, with pages built by scripts you can code in the environment. (I.e. web apps should be free.)
free browsers «
Until Microsoft gave away IE for free, Netscape charged for its browser. Income from the browser was significant. In practice, folks could use it at home for personal reasons free of charge, because licenses were not enforced for individuals. Only businesses had to pay up. Netscape's site license business was booming. A typical site license for users at a company, numbering in the thousands, was a seven figure number in dollars. I heard folks cite numbers in the low millions for individual site licenses landed.
Some businesses said, "We'll pay the $4,000,000 license if you add feature XYZ." For example, some companies wanted an address book scaling to 100,000 users. Adding that feature was worth millions in license fees. If you notice, this was a pretty good deal for Netscape. One engineer, working for a short period of time, costs a lot less.
In effect, Netscape had desired feature lists with dollar signs attached. I thought this was a great way to prioritize engineering—it's grounded in the real world, instead of fantasy-land priorities of architecture astronauts. Evidence is sweet. Anyway, deciding what to do was both rational and fairly easy. (How to do it was less clear.) Having software directly worth money was nice after years at Apple where only hardware counted.
Yes, Netscape also made money selling servers. But the browser made big money too, per engineer, and that's what matters. (Until Microsoft lowered price to zero.) Anyone who says different doesn't know what they're talking about.
Note I have nothing to say about how things went. Complaining about reality is rarely productive. But browsers made money, once upon a time.
missing features «
I agree repeated, boilerplate code can be a sign of missing features in a programming language. But if present as a feature, a one-size-fits-all version might not actually suit you, so be careful what you wish for.
One of my planned pages on code is about features I typically add to many classes, in almost boilerplate fashion, because C++ doesn't do them for me, automatically. For example, printing is a big cause of boilerplate in my code. (I don't print as much in commercial code for day jobs, so my work code has less boilerplate.)
In some languages, like many variants of Lisp, printing is a primitive builtin feature, so most expressions print something useful without any code written by a developer. Why do I think printing is so important? So you can get rapid feedback, which encourages scientific method in your style of development. Printing affords more evidence, so evidence oriented development happens more in languages with builtin printing.
In C++ I just bite the bullet and say things need to be printed. So boilerplate for printing doesn't count—very much. Would you rather do without it? In the absense of evidence to the contrary, many folks assume code does whatever they expect. Can you imagine that? Such optimists. If you close your eyes, problems won't be there.
I miss features showing you what you expect is true. So I add them. Those are consistent things I add as boilerplate patterns.
bicycles «
I have personal experience with how dangerous it is to ride a bicycle. Most of my riding was done in Iowa, in my teens. But I've ridden here in California too and, if anything, car drivers seem slightly more dangerous here... which is partly why I seldom ride bikes.
In Iowa my usual problem was very bad judgment on the part of car drivers. A classic move worked as follows. A car overtook you in a lane with enough room for the car to pass. Just after the car passed—tail-lights just beyond your front wheel—it slowed by letting up on the gas, or sometimes braking. What was about to happen? A sudden turn through space you'll occupy one half second from now, and without signalling first. You could avoid them only if you expect this. So any change in a car's speed must send off alarms in your mind, if you want to keep clear.
But that's not enough, if they brake hard enough. Once a car barely drew aside me, then braked hard into a turn for a store, so I rammed into them a fraction of a second later. I flew over the hood of their car in something like a forward roll, which apparently did me no harm while totalling my bicycle. My bike's frame was destroyed. By their reasoning I was at fault, since as soon as they managed to get in front of me, by any amount, they had the right of way. Basically they were just stupid, since it was impossible not to hit them. So I ate the cost of a new bike.
That might also have been the incident injuring my spine. I don't know. I had other experiences with flying through the air; that was just the most impressive one. But the timing coincides with start of sciatic nerve pain. I had a herniated disk, which I lived with for thirteen more years. All through the second half of my 20's I could often only barely walk, and was in major pain much of the time. I didn't have any health insurance. And my parents? Well, they were the same folks who didn't help me through school. (You want them? I'll trade for your parents.) I finally had it fixed when I was 33 years old. My disk was completely herniated: the doctor was impressed.
I'm still fond of telling folks, "You only get one spine." That's what I say when I see someone pursue risky sports. Now, bicycling shouldn't be risky, but is. The only way to be safe is by following advice offered by Neal Stephenson's Zodiac, in which the main character says the same thing I do: assume drivers want to kill you, if you only let them. In short, you can only ride a bike safely where a car cannot kill you if a driver deliberately tries to do so, because you get the same result when they do it accidentally, which sorta happens.
Fifteen years ago, when I worked at Apple, I took many long rides up and down a meadering road along a creek; my trail ended at Coyote Road, which had heavy traffic. One day, quite by accident, I saw a death notice in the newspaper: a cyclist had been killed at the end of my route, at Coyote Road. It was odd to read about someone killed on a bike in place I rode my own bike frequently.
A couple days later I hit that stretch of road again on my bike. A car was pulled over there, whose driver stood staring down at the road's surface. He looked up at me bleakly as I rode toward him; suddenly I was sure he knew the dead cyclist. He was either husband or father, here to grieve one more time. Apparently he read my expression, too. "A cyclist was killed here recently," he warned. Then seeing my recognition, he asked, "Did you know her?"
I told him, no, sorry, I had seen the newspaper report only.
questions «
I suppose I'll start writing fiction again soon, instead of just thinking about it. First I'm trying to get my new format jump-started in the new code section, which I'm almost ready to start populating with pages I'm holding offline in draft form. I'm afraid if I don't do something like work in my hobby time (writing about code), I'll get addicted to writing fiction alone. It was way too fun when I was writing and posting it. I fear falling into my navel and staying there; I need a viable alternative to fiction.
In fiction I like asking questions without necessarily answering them. For example, Finch has the most questions so far, which I hope readers ask themselves. What is Finch? Keeping fun alternatives alive is amusing. Readers are meant to ask: Is Finch an X? Is Finch Y? Then I try to keep as many X's and Y's possible as I can, in the sense they can't be ruled out. I'm tempted to list them here, but it undercuts your fun. However, any time a question occurs to you, but you think I haven't given enough reason to ask that, please go ahead and ask anyway, even things only hinted. Other characters ask these questions too. I expect Zé to pursue a supernatural angle more than others.
I have a big pile of stuff to try fitting into the next several scenes. Maybe this will make it better. I'll have to risk making it cluttered, to learn something from it.
optimization «
Professionally, as a coder, I'm a speed guy. Folks usually ask me how to do something faster, and I get assigned tasks to speed-up slow things. The more complex a problem is, the more effective I am compared to others. (I won't explain why this is true; maybe complexity bothers me less.) Generally what I do is remove latency, typically by doing less of whatever happens, so it gets done with less repetition or fewer prerequisites.
When a computation is very complex—say many folks have spent man-years getting code to where it is now—it gets harder to tell what was strictly necessary to get desired results. Often folks use a definition that wasn't minimal, and I think of something simpler. How do I do that? (Am I supposed to say I'm clever here?) Analysis and synthesis, both. You must understand a first version really well first to see parts that matter, so other parts can be shunted aside and minimized. Isolation of relevant facts is a skill.
I wrote this section as an introduction to latency, after writing the next section. But latency is a large topic. Systems slow down when you stack up individual parts contributing to total latency, either directly or indirectly. It might not be obvious. And worse, it can be statistical, with latencies added with some probability instead of always.
(I'm also a complexity guy. Around the time code gets too complex to change, I can still change it, because I can simplify first. Or, I can think of a simple way to add a next twist, if nothing can get simpler. I can alter something that frightens folks already, with high odds of success. After a while, folks think of me first here.)
Incidentally, I'm not looking for a job. In my experience, I get job feelers after I write material like I today's, because obviously I'm one of those folks in limited supply who get things done. When I want a job, I'll say so; the one I have now is fine: thanks for asking. I'm not even interested in leverage for a raise. I'm paid well enough.
compilation speed «
Yes, time to compile is mostly caused by reading header files. In fact, most of the time is opening each header file, not reading it. So for example, if you use a C-style classic "include sandwich" directive to prevent double inclusion, there's a wrong way to do it. A C++ header file named Foo.h might look like this:
Everyone who codes in C and C++ knows this part already: the part inside the ifdef is parsed only once, even when this header is included more than one time. This allows N different files to depend on Foo.h—each including Foo.h as needed—but only seeing definitions inside just once, the first time Foo.h is read.
This process is fastest only when each including file does this:
When every file—especially other header files—test the definition of Foo_H before including Foo.h like this, then Foo.h is only opened once per compiled source file, not N times, as when N different files do this instead:
The naked include shown directly above is typical coding style in some projects. It looks cleaner, so folk do this instead of always wrapping each include in an ifdef. When header files do this, files can be opened many times apiece, instead of just once. So latency goes up, since opening a file typically takes longer than reading and parsing a header.
I don't argue with folks about it, though. I do what everyone else does. But when folks complain about compilation speed, I tell them to use a faster include style instead, with more ifdefs. I once had a coworker convert all his headers and joyously report dramatic speed increases. He had been certain it wouldn't matter.
Some operating systems have incredibly clever file systems with respect to caching—Linux is one of these—so cost to re-open a file just opened is cheaper, but still not zero of course. So your mileage may vary. If you run micro benchmarks to test how much better it gets, pleased don't tell me: I don't care. I do mostly incremental builds.
In hobby projects I sometimes put everything in one header. This is considered very bad style in work environments, because it impedes code updates under source control, and makes it hard to find code you want when everything is placed together. But it sure makes compilation fast. I might distribute a copy of my thorn library this way later, as a lark. I did one early version of it this way; it was a few thousand lines long.
compiling html «
The piece on including source headers above reminded me of a project I did several years ago, because it made me realize just how fast I might be able to compile, say, Smalltalk source code. This was around 2001 and 2002.
I wrote a server to optimize web page access, in the form of a reverse proxy, along with a handful of other engineers in a startup. One guy did our sockets code, and handled the actual HTTP protocol. Another guy did our inter-system comm framework, so distributed nodes sent internal messages wherever they needed to go: for example, configuration of each optimization server node. I did the guts of the server code—what it actually did to content passing through—because getting this to occur as fast as possible was a goal. So most of what was stored in server memory was managed by my code.
When I enumerate all the things I did, it sounds like roles of several engineers. I wrote the utilities and server framework, parsed the html, compiled to a virtual machine I designed, drafted the VM's execution, and handled serialization, i/o and storage for compiled templates. (I also compiled server configurations to interpreted data structures, and compiled invalidation notices for fast matching at service time.) Each compiled template allowed a slightly different html page to be served, modified as necessary to make changes specific to variables in HTTP request headers, from cookies to URL query parameters. Instead of static caching like Akamai, this was dynamic caching, with dynamic substitution and editing in pages for individualization and the like.
(I figured out how to make this work with gzip-compressed pages, without having to run gzip to compress a page when it was served, so latency for gzip was zero after initial compilation. The format spec for gzip allows uncompressed "clear text" segments to be embedded, and zlib uses this escape hatch to emit original content unchanged whenever encoding makes the result bigger instead of smaller. But the zlib authors didn't think anyone would want to do this on purpose, so I had to add a new entry point in zlib's api to do this on demand. Then all I had to do was emit parts needing change later in the clear, so they could be patched when served, at cost of re-doing only the crc32 at the end. Checksum cost is much less than gzip compression, so latency is small.)
This is actually just a story about parsing html; all the rest is context and digressions on latency you might find intersting. Okay, so what about parsing?
My "compiler" parsed and compiled several megabytes of html a second on processors of the day, so I could keep up with lower end network connections without adding enough latency to be a bottleneck.
That it was megabytes a second made me wonder why compilers I used in the past were so slow. I was working on a hobby Smalltalk implementation at the time, and I realized a Smalltalk project of several megabytes in size would compile in under a second, even when all the code was compiled—provided this didn't involve opening a lot of files.
On a related note, the speed at which I could execute crc32 on megabytes of content told me how fast I could garbage collect heaps of that size, too. It was a lot faster than I had believed, making me like garbage collection more, if scope of gc could be bounded to a heap size under my maximum latency.
memory «
If you have a very good memory, and something bad happens to you, it's a problem. This isn't obvious at all until something quite bad happens. This might depend on how long a bad experience lasts. Worst is a situation lasting a long time, because it slowly captures a large part of your semantic network, as you touch more and of your entire network during a long-lasting situation. A nightmare that lasts a year can get in the path of almost anything you think about later. This happened to me starting about a year after my ex divorced me for the 20 year old Indian boy. That's when it got sticky.
I had about a year of high anxiety and adrenaline, feeling trapped all the time, wondering how I'd manage to live through it. That was about five years ago. A couple years after that, my girlfiend, who was in the psychology trade, decided I had a mild case of post traumatic stress disorder, because I treated it as a life-threatening situation for many months as it happened. I recalled the year like a car crash, and almost everything I thought about brought it to mind. It's taken me years to get over.
Getting over it means: it doesn't dominate my event horizon any more. For example, I no longer cry hysterically during all movies, at emotional parts where a character (any of them) experiences loss. I don't wake up with panic attacks. I don't feel fear all the time. And I have a vaguely positive outlook again—at least recently, like the last year or so. This is surprising in so far as my ex still controls my life, just as if we were married, because the legal system runs me like I'm bankrupt, on behalf of my kids.
I've gotten very good at not thinking about anything from the year or two starting from the very bad time. It's as if I have a giant plug-board carefully wired to route around that period, for all topics. Unfortunately, I had to learn how to do it without much practice in forgetting. I was used to fast recall about everything.
My memory is clear enough that I can see obfuscating effects of recall. As you remember a past event, your original recall gets harder to fetch, and you start to mix it with annotations you add later. I can tell later additions from earlier raw recall: they taste different. After I remember something enough times, I can't access original memory well at all, and it nearly all tastes like annotation. But as near as I can tell, I confuse them very little, and rarely conflate annotation with actual experience. To alter a memory deliberately, the best I can do is rehearse a replacement I want to recall instead.
This isn't true for many people. It's fairly easy to see folks edit memories with translation or re-interpretation after the fact. In high school I sometimes experimented with this. For example, I was aware many folks seemed to turn non-verbal events into verbal ones. I'll give you an example from one lunchtime experiment.
I wanted some salt, so I pointed at a salt dispenser and carefully said nothing, while raising my eyebrows. After my lunch partner duly handed me the salt, I asked her how I had asked for the salt. She said, "You said 'please pass me the salt.' "
"No, I didn't," I smiled at her. "I just pointed and raised my eyebrows."
"No," she said while concentrating. "You definitely said the words, 'Please pass the salt.' I'm sure of it. Why are you teasing me?"
I was famous for screwing with people in subtle ways. I told her I wanted to see if she'd remember her translation of what I did instead of what I actually did. I never got her to believe my version. She was dead certain.
dress code «
I wear sneakers and shorts with cargo pockets almost every day. (I put a lot in my pockets, so more is better.) Above, I wear tshirts or sports shirts. In cold weather I wear blue jeans or cargo pants. I wear no jewelry, not even a watch; my phone tells me time when I care, but it stays in my pocket.
So I always look like I'm on vacation, dressed as informally and comfortably as I can. Going to work makes no difference at all. Only one situation changes how I dress: job interviews. But the change is small—I avoid shorts, but otherwise dress the same.
If I had any choice, I'd always wear exactly the same thing, if weather never got cold and clothing never got dirty. I don't talk about, or even notice clothing beyond care to stay perfectly clean.
How I dress says little—except maybe: I don't care how I look. I think I'm seen as poor by others, since folks my age usually dress better, if only to stake a better socio-economic status. Which I don't care about, of course.
No one at work ever mentions how I dress, except a rare mockery of cargo pants as a fashion statement. I get an occasional funny look if I wear shorts when outdoor temperature is in the 50's, which I do.
So in the tech industry, in northern California, there's no dress code—at least for engineers. Sales folks usually dress better.
talking «
At home I talk to my sons, and at work I talk with coworkers often to discuss coding plans, problems, and progress. Otherwise I don't talk to anyone. I don't use my phone, and I don't use email away from work. My social footprint is as close to zero as you can get, and still have a job and family.
Does writing here count as talking? Maybe. I'm tempted to say no. But it really is talking, just not very much. I say far more to coworkers at lunch any given day. Why do I bother writing here? First, it's fun. Second, I do want other folks to know a tiny bit about me; this clearly influences how I write. But I'm unable to tell if publication is just a constraint making it more interesting in the fun department. If it had no effect at all, it'd be boring and I'd stop.
I started this piece only to report an exchange I had yesterday with a coworker at lunch. He told me he only used twitter to send tiny URLs. So I asked: Why? Most URLs are shorter than 140 characters, right?
"Yes, but tiny URLs are faster to enter into phones," he told me.
"Phones?" I asked, puzzled.
"It's part of the always-connected culture," he explained.
"Twitter is broadcast for phones!" I slapped my forehead.
He turned up his eyes and considered a second. He wanted to say this was close, but wrong. I forget his final remark. (See, I do forget things.)
I was pleased because one value of Twitter finally became comprehensible to me. I missed it since I rarely talk, and seldom keep touch with others. Of course it has other values, too. I'm inclined to agree with Aaron Swartz about it's role in maintaining an illusion of social closeness to relative strangers.
(When I say illusion you might think I mean a vacuum. Not so. Illusion is a main thing everyone wants: it's highly marketable. Illusion is in fact the product itself in many cases. Especially for Eloi, perception is reality.)
writing «
Writing is very easy for me now, but I hated writing in my teens, and hated writing papers in college. Writing papers was a rare cause of procrastination in college; I put it off as long as possible. I only started after I planned every move in my mind first, from beginning to end. It was a good way to do papers—but my papers were boring: good enough, but completely lifeless.
My current writing style is stream of consciousness. I have no conscious idea what I'll write next, sentence to sentence, paragraph to paragraph. Every decision is local and tactical. And yet it hangs together. Do I know where I'm going subconsciously? Or do I let readers impose more order than I add? Both, I think. The latter is something I spent a very long time thinking about.
A few years ago I wondered if reading was fun when inference is needed. I like making connections. Everyone does, as near as I can tell. But was this specifically the fun part? Maybe not, but I think it's a big part of fun in reading. As a writer you can cultivate this: put several interesting notions next to one another, then proceed without spelling out every consequence.
I find fiction especially amusing when a main idea is never actually said, explicitly, anywhere in the text—just implied by details dancing around it. Among other things, you can imply many things at once, in high density, by overlapping a lot of related things present as potentials.
Effective writing is based in part by running several leads at once while focusing on one at a time, thus priming what comes next by making it latent in prior context, so everthing does double duty. This is partly an efficiency argument: why let text do one thing when it can do two?
It's important to get a reader busy constructing your message for you, before you get there, so they're complicit in building ideas involved. Try to cultivate a "you know where I'm going" tone, because it works better in a game of constructing a mutual view held by you and readers. It signals a reader to fill in blanks, and reminds you to provide context and clues so a reader can. If an audience isn't nodding with a half smile, you're doing it wrong.
I write and think just as well any time of the day, but I'm more clever in the afternoon, either when coding or when writing fiction. The more awake I am, the better my imagination works; so yeah, lots of coffee is a good idea. If you sleep as much as you possibly can every day—seven or eight hours in my case—your subconscious works better when you write, and this matters, both in terms of having ideas and subliminally knowing where you're going.
To improve at writing, practice a lot and try to see what's weak in everything you do, so you get better. You slowly minimize what you dislike in your writing, but it never gets as good as you wish.
fragmented reality «
Yes, each sub-culture sees a different view of the world, informed by a constructed social narrative shared among group members. The pattern is fractal: a group of several friends is a small sub-culture embedded in a larger one, and so on upwards in larger groups of people. Shape is affected by several limiting factors.
Obviously there must be a communication channel, so changes in ability to talk have an effect on group beliefs. Faster propagation allows group consensus faster in the face of quick change in ideas about what is real. But many other things matter too: intelligence of members, homogeneity of views, breadth of imagination, tolerance of dissent, ability to ignore contrary evidence, etc. When group members are aware of this, say because they've been in different groups, an ability to self reflect on the process also has an effect. A group can be, but need not be, aware of group belief evolution.
And yes, it's always been this way. But it's still very easy to believe in a one-world-view version of reality, where anyone who thinks different is crazy. If you're a person who bubbles over with ideas and novel insight, you experience great pressure to stop showing this to others, because it makes you a scary outsider.
An ability to resist group consensus is admired only in rare situations, when a "correct" view can be verified by some objective measure. For example, suppose a school teacher withholds a "right" answer to see if the class can work it out. When you hold out against the other thirty, they're impressed when the teacher eventually backs you up. But any other time you're just an eccentric crank of unknown threat. So fitting in can be very important to success in normal life. Assuming you care, of course.
Obviously fiction I write has a bit of this flavor in it, making it taste a little like Philip K. Dick now and then. But it's not deliberate imitation. I never read Dick, except for a handful of short stories. I once tried Dick's Valis, but it was weird and I didn't like religious parts I felt were out of place. Lately I tried Philip Dick again to see why folks keep alluding to him. Okay, I see a similarity, but my approach is more rational.
Anyway, this multiple perspective stuff is just reality. It's consequences of it that are hard to figure out. When you suppose everyone else has a view like your own, it's easy to under-estimate just how different other views are, which you ignore.
But in my story, it's part of the plot so it can be explored, and becomes more explicit in some side eddies. For example, Wil writes a fable about Briar Pig, who slowly realizes he is just food to some other animals before he gets wise. (Maybe it's how Wil works through a reaction to others seeing him as a consumable resource; he never says.)
cautious writing «
A new code page is starting to gel. Maybe I'll post it soon; it's a fourth revision from scratch. I also didn't like a new third draft, but that one was worth keeping as a separate sub page. I'm almost done with generalities—then I can write about code again. Except this time, I'll write at a higher level.
By that I mean source code won't be the main purpose, even though I'll give source code. Instead, my main goal will be to explain why. Sample code just gives an example to illustrate ideas. Other code addressing the same concerns is just as good.
speed reading «
I don't think "speed reading" works. Not in a sense billed by speed reading training vendors. I seriously doubt reading very fast maintains the same retention as other speeds; but reading the same thing at different speeds ought to help a lot.
I tried speed reading in college many years ago. I gave it several hours focused study, but my comprehension didn't change much compared to first efforts. Maybe I was already good at skimming. Over years I had spent a few hundred hours in library stacks, skimming books to see if they had any useful information.
I have no idea whether I'm a fast reader now. I know my sons think I am, but I'm slow compared to scary people. (I can read aloud very fast, so my tongue trips, and several times faster when I read silently.) But I visualize a lot while reading, and this dramatically improves retention since my episodic memory is very good.
In high school, when we read novels, I normally scored 99% accurate on tests (meaning I missed one question in one hundred about book content), so I was almost perfect at the coarse granularity tested. I only had this level comprehension when I visualized a book like a movie while reading. To recall something boring takes multiple passes, comparing what I grasp already to any new finds.
These days I'm vaguely aware I read at more than one speed at the same time. I tend to partially read several lines at once, because I see and process words a couple lines below the one I scan. I skim a lot of online content when I suspect a writer says nothing whatever from my perspective, and only slow down when a part seems interesting—maybe about a third of what I see.
It's possible folks who get value from speed reading do so because they discard trivia under speed pressure, conserving bandwidth for what really matters. In other words, critical thinking matters more when you go fast and triage what needs attention. A smaller working set might fit into long term memory processing better. I have a high rate of long term memory retention, so triage under speed helps less.
As a writer you can help readers by saying only what matters.
prose bloat «
I'm having trouble writing the new code page. Each time I start anew, it grows with a cancerous life of its own, bloating until I see I need to kill it again. There's probably a zombie parody in there—why don't you write a parody for me? I know you're up to it.
Anyway, I have two longish, dead revisions that never made the cut and won't be posted. So, um, obviously my aim isn't minimalist enough. I have a couple ideas about a next attempt. For example, I could start with something I don't want to talk about: myself. An intro quickly summarizing myself as a coder might limit content nicely. I could say, "I'm not going to tell you about code, really, just say what kind of coder I am and then give examples of trivia." That might work.
Once I had this line of attack in mind, I started seeing online pieces giving me ideas. I could emphasize how I do niggling, fine-grained details which—stacked in just the right way—add up to big effects you want. Somebody has to do pointers, memory management, memory hierarchy, core algorithms, optimization and data structures. I'm that guy. And I fix hazards left behind by others less talented in minutiae, especially when they cause data loss under concurrency. Except I really don't want to say this, though it might keep length under control this time.
grit vs talent «
Is it better to have talent or to work hard? Both is better, of course. Talent is better understood—at least, it gets a lot of media play. I think most people get the idea of someone with such a high aptitude in something that results from first native efforts alone can surpass what a person with training and no talent can do. But if this sort of talent isn't harnessed by a modicum of discipline, nothing comes of it.
Grit is much less well understood. The scale of repetition necessary to really work through every angle is daunting to someone who isn't so inclined, and they grossly underestimate what's involved. You can actually be talented at grit itself, as a skill in pursuit of excellence through persistence. It helps a lot to tolerate repetition and losing. I have a high tolerance for repetition. (I'm pretty sure others watching me do something again have felt vicarious nausea on my behalf; the classic curled-lip, sea-sick look is a dead give-away.)
Tolerance for losing makes you a good sport. I'm not particularly talented at chess, but I liked playing it in high school. I played several dozens of chess games with a friend of mine named Brian, and I never won a game. He was quite good, and I started out very weak. It didn't bother me much when I lost, because I found it very interesting when he kicked my ass. I wanted to learn how he did it, so I studied and kept playing. He spotted me a queen in one early game and still won. But eventually he couldn't spot me anything and still control the game; by the end he had to sweat to win each game, but he still won. I'd have been less interested if I ever won, even once.
My ex and my older son are poor sports, and generally lose their tempers when they lose. This always surprises me because I expect others to be like me. What difference does losing make? It's a game, and you learned something, right? I suspect this stance has something to do with grit: failure is a source of information. People who are poor sports care less about knowledge per se, and just want status of winning. Gritty folk seek a path yielding skills to command a field of endeavor, based on what actually happens, as evidenced by results that dispel mysteries in early failure.
Grit is related to repetition. You only learn a small amount each time you retry something. Partly this seems because your brain is geared to see each occasion as nearly the same thing. More new stuff happens each time, but you miss some of it, and this failing is vaguely interesting itself: why do I make the same mistakes? If you enjoy this process rather than feeling frustrated, you have a natural bent to grit.
