Lately I write demos under thorn with a todo list using C++ under a BriarPig mu-babel license. The run and hex demos were done on 13apr2008; crc on 14apr2008; buf on 20apr2008; in on 27apr2008; ctype on 04may2008; out on 18may2008; slice on 25may2008; quote on 31may2008; escape on 31may2008; mutex on 14jun2008; rand on 16jun2008; stat on 17jun2008; primes on 19jun2008; list on 23jun2008; heap on 29jun2008; this menu links all demos:
mu: toy, peg, imm, tag, box, symbol, token, number, bigint, class, method, reader, writer, eval, env, vm, gc, world, pcode, compiler, asm, lathe, lisp, smalltalk, design, weight, jar, card, harp, debug, profile
thorn: todo, names, iovec, assert, log, run, hex, crc, buf, in, out, quote, escape, compare, file, deck, cow, arc, blob, tree, slice, rand, time, stat, heap, node, primes, page, book, pile, stack, atomic, lock, mutex, thread, map, list, iter, ctype
extremism ¶
Despite what you might have heard, extreme programming is not complete bullshit. There's a grain of sanity in the madness. I like the grain of sense which pursues more evidence of problems sooner. This is what I tell folks who ask me what I think of agile programming and its variants. I get asked this now and then in my day job where we pursue one flavor of agile process (when we don't backslide into schedule chicken).
Last year we had a phase in which we tried to use a scrum model from agile development. Folks kept asking whether I thought it worked. Here's what I emphasized: most folks will do anything to avoid evidence reflecting negatively on themselves or their plans. But avoiding such evidence of problems is counter-productive. I say I approve of any practice making folks face reality instead of telling fairy tales to put off reckoning.
A number of agile practices seem aimed at acknowledging reality faster, and this is good. However, I've almost never heard these practices described this way. Instead agile terminology has a touchy feely, mumbo jumbo quality when explaining what it means. So the explanation is mostly nonsense. The part that works is owning up to reality quickly.
Occasionally someone in charge does something bizarre and other engineers complain to me. I can almost always explain the problem in terms of willful refusal to acknowledge reality in a timely fashion. There are just so many games you can play to avoid accountability. Each one puzzles a few folks because any new tactic has some novelty. But they really all boil down to a similar base: far-fetched positive outcome will be achieved by new strategem with accounting for effectiveness conveniently far in the future where it won't get in the way.
Some of the old-timers (the gurus in their 60's) think my cynicism is funny, especially since my simple recipe for calling bullshit actually explains so many kinds of nonsense they've seen before. It's like they ask themselves, "Is that really all there is to it? Everyone desperately avoids negative feedback?"
Yes, that's pretty much it. But it does take a little inspired analysis to see how to apply it each time. You need to grasp the psychology of each person who constructs a new fantasy that they're immune to accounting, so perceiving negative evidence will be superfluous: "There won't be any contrary evidence, so why bother looking for it?" It's actually kinda funny when working in technology where scientific method is so important to evaluate success and quality.
barrel of monkeys ¶
I wrote the heap demo tonight after preparing the iter demo yesterday. I worked through iterator sample code, then wrote a new sample heap to show allocation effects, then wrote the heap demo. But I didn't have time for iterators.
I beefed up the heap demo by adding inlines for operator new() and operator delete(), along with sample code for a heap subclass revealing statistics about usage; this can be used to verify the inlines do something. (I also tested the template queue a little and fixed a couple bugs.)
Anyway, I was able to avoid any need to introduce the page/book heap beyond small references. So the bulk of material on that heap can appear in future page and book demos.
just for fun ¶
Today I'll actually blog. That is, I'll just say stuff I merely think is interesting, which might also be of use to someone else. As agenda, I plan to discuss how I treated my rsi (repetitive stress injury) several years ago, resulting in total cure after a year or two. I'll include my guess about a possible relationship between fitness and rsi.
But first I'll prattle about a few other things.
nature vs nurture ¶
Yes, some people are just plain born smarter than other people. I don't suppose it's easy for just anyone to tell though. You must be up to seeing it yourself, both from being smart enough (whatever that means) and from being good at reading others intellectually.
(I don't have Asperger's. I'm actually very good at reading fine detail in expressions, and my emotional IQ is excellent. For example, I'm keenly sensitive to subtle nuances in negative evaluations, no matter how carefully disguised. Some folks mistake my usually placid facial expression as an indication I'm unemotional, or that I don't see emotions. But this is just slightly backwards. You have trouble reading my emotions, and suppose I can't read yours? How interesting. You should think about that critically for a moment. For example, consider some counter examples, like how easy it is for you to read emotions in folks with weaker intellects.)
However, intelligence is a trainable skill. I figure it's easier to train if you have a head start, but anyone can do it. It helps a lot to have good memory, and to be able to run simulations merely by wanting to do so. Then you can work through more spaces and evaluate the results afterward since you'll remember. (If you remember for a really long time, you can review things from years ago with a fresh eye.) Reflection is useful training.
On an absolute scale, intelligence is somewhat flat, with everyone in the middle arranged on a gently increasing slope. What tends to make folks with IQ's around 130 or so such flaming assholes is their past experience with most folks being less smart than themselves, just based on statistical sampling. It's easy to assume "hey, I must be smarter than everybody!" when you're in that area, especially if you confuse dominant social game behavior with actual capacity. But playing king of the hill isn't fun unless you have close rivals.
Somewhere above that level, something interesting starts to happen, and the slope seems a little less uniform. Not only is the slope less flat, it also involves some forks. Very bright folks are all somewhat different from one another. There are many ways to be smart, and there's a lot of variety. That's why it's so odd many folks think intelligence is linear on just one scale.
I tend to like folks with an active internal life, whose minds are busy modeling the world independently of what's going on around them, partly because the world isn't enough stimulus, and thinking is a good alternative source of new things to ponder. People with a busy internal life simply shine, but it's hard to see unless you work similarly and recognize it. I'm only really good at reading people like this.
To model someone of normal mental abilities, I have to assume a rather strange combination of disabilities to get a good correspondence, including: flighty attention, rapid memory decay, self centered perspective, lack of empathy, absence of consideration, contempt for others, delusional wishful thinking, lossy data reception, inflexible models, closed categories, self-aggrandizing life story, uncritical associative reasoning, coarse pattern matching, fast sticking categorization, few options entertained, poor future sight, and inability to reach ideas unless found by constructing linear verbal descriptions.
In other words, normal people often scare me, like drunk drivers operating at all times under the influence, with a mean grudge against the world, wearing shades at night, driven by temper and lust, guided by random radio broadcasts, empowered by wild fantasies of safety, competence, and entitlement.
repetitive stress ¶
To fix repetitive stress problems, I recommend you follow at least three practices:
- exercise your arms more, with variety
- wear wrist braces only while sleeping
- massage when it hurts, where it hurts
This works for me. If you stop reading now, you already have my recipe to stay pain free.
However, if you keep reading I'll tell a couple stories, in which I explain how I came up with those. Note my theories might be flawed: just because I constructed an explanation doesn't mean it holds water.
First let's go back ten years. When I turned 40, I lost 45 pounds in eight months by cycling several hours a week and watching what I ate. My loss rate hit a steady one-pound-lost-every-five-days once I found a groove. I dropped from 210 pounds to 165. That's what I weighed at 18 in high school, so I stopped there.
I kept it off for five years, swinging from 165 to 170. But I didn't exercise my arms much. (I have big arms; they got as small as maybe 13 inches in those years.) Then I got a job in which I typed an incredible amount all the time, every day. I'd never had any rsi problem before.
Suddenly my right arm and hand hurt a lot. Then my right hand swelled up about 30% over it's normal size, and I could hardly move my fingers. I used my left hand exclusively and suddenly it was on the verge of hurting constantly too.
I would not have lost that job, I think, if I hadn't caught my wife having an affair with my sons' 19-year-old Taekwondo instructor a week after my hand swelled and became unusable, and she told me yes, we were getting divorced. After which the world became quite complex and I half lost my mind in the process. There's always some shock that's a little too much, like losing your partner of 23 years to materialism and whoopee.
(As I was trying to move out, my now ex-wife accused me of having psychosomatic rsi pain. Somehow she failed to notice this started a week before I caught her boffing the kid. In many ways she wasn't very bright.)
Moving right along, after a few months my rsi was improving enough that I started to work on the problem actively. I started juggling again and I saw a new doctor, and I massaged my forearms on advice from friends. I developed a theory about how the rsi crept up on me without my noticing. (I'll describe the creeping pain theory last.) The doctor's advice was crisp and clear.
He said: 1) take anti-inflammatory medication, and 2) wear wrist guards at night to avoid sleeping on your hands funny. The stress of bending your wrists backward or forward in your sleep was a problem for many people. All you have to do is wear guards in your sleep. I've been doing that over four years and it clearly helps. (I give myself nights off now and then for variety and comparison.) I suspect this practice matters most.
I juggled because I wanted to do so, not because I had a plan. I was trying to work my way up from five balls to seven. (I gave that up a couple years ago, but I'm much better at five now.) But it made my arms hurt worse when I juggled, but only while I was juggling. Afterward, my arms felt better. It wasn't hard to think of a treatment angle from this. I started going to the gym too.
That felt like gambling to me. I wasn't certain exercising would improve my arms, but it fit a theory I had about how I'd gotten the rsi, and it fit the way juggling made a large continual improvement.
A year later my rsi was gone. It hasn't been back in three years. I type a lot all the time (like I'm doing right now) with nary a problem.
I work out every two or three days now though. The last several years I worked out as much because it improved my mood as because it helped my arms. Once my arms were fine, why work out so much? It felt good. For a guy almost 50, I'm quite muscular now. Guess how much I weigh?
I'm up to 220 pounds. About ten of the new pounds is fat; another new forty pounds are muscle — ten pounds of new muscle each year. My arms are now 17 inches, same as my neck. My waist is 34 inches. I think it might cause a problem when I interview in the future, because I'm going to be typecast as a jock/moron or something.
Okay, so what was my theory? Keep in mind I know a lot about psychology, especially cognitive psych. Just think of me as having been a pysch minor in college if that helps you better categorize me. (I wasn't, but I could pretend to be without too much trouble.)
One of the things the human mind is good at is filtering out signals that remain the same. The brain is an engine geared for tracking things that change. For example, several eye illusions are based on a persistence of vision effect where standing images will inverse themselves. If you stare at one point in an image of the American flag for 60 seconds, then look away at white paper, you see the flag in inverse color as an after image. Why is the color inversed like that? Because your eye was trying to cancel the part of the signal that wasn't changing.
Anyway, the world is a busy place, and your brain is busy trying to filter out as much useless noise as possible. For example, things that don't change might easily be ignored, if the situation doesn't need your attention right this second. This gave me the idea that muscle pain in my arms might have slowly increased over time and I ignored it because it was so steady and regular, my brain didn't think this signal represented a clear and present danger needing attention.
What else fits this theory? Well, folks with arm stress from over-typing have sore muscles, which they only notice when those muscles are massaged, at which time it hurts like hell. Why only when you massage the muscles? Because that's different, and your brain says, hey that's interesting, lookit here.
If it hurts when you massage, then you have a problem, and you need the massage. So do it.
list demo ¶
The new list demo took two major writing sessions — I couldn't get it all done last night. It primarily describes one flavor of list only: double-ended circular lists of the deque flavor. The template version shown is a rough draft. (Well it might not look that rough, but any untested code is pretty rough.)
I wanted to write the heap demo next, but realized it had to wait because the base heap class wasn't very interesting as a bare bones abstraction of malloc() and free(). The interesting subclass was the one allocating pages and books, and I noticed it was really complex when I looked at it again. So I need to creep up on it first, writing demos for things used by the page/book heap.
In fact, the page/book heap will be so complex, it will have to be split between the future heap demo book demos, since otherwise heaps will have too much and books too little.
The new list demo just posted defers all iterators to a near future iter demo, which will likely be done in two phases, with iters for maps coming later.
Before I can write write about the page/book heap, I must also write a future node demo first (which will resemble the older refcounting page) because pages and books are refcounted nodes maintained in node lists using another variant of queue of the sort in the list demo.
The page/book heap is basic infrastructure I use under language runtimes I'll show later, and I knew none of the toy language code would make any sense unless you could follow the heap code all the way back to when locations in memory get read and written. That's the main reason for all the demos: around half are used by the page/book heap, and other demos use the page/book heap. For example, the deck demo will use pages as backing store for iovec scatter/gather memory-based files.
primes demo ¶
I wrote the primes demo start to finish. It's about using prime hashmap sizes to subdue potential bucket collisions when you know you're adding keys with problematic hashes (say when using addresses of page-aligned blocks as their own hashes).
It also has a dialog verging on discussion of heap fragmentation avoidance in designs appearing in future demos.
Dialogs are always a bit more fun to write than other things, for a very simple reason: I have almost no idea what I'm going to say in a dialog when I start. This is one of the reasons they often work well: I don't force them in a predetermined path. I let them go wherever seems natural. I simply try to simulate what the participants would think at each point.
If you ever wondered why one dialog or another wanders afield, this is why — and I like the effect. To me they're simply conversations in slow motion. (You read them several times faster than I can type them.) In some ways it's a lot like writing code, which is another way of thinking in slow motion.
second helpings ¶
I added a small dialog to the stat demo, and a long piece on pseudo random sequence partitioning in the rand demo. The latter is a bit interesting despite being somewhat trivial.
std dev demo ¶
Tonight I wrote all the stat demo, pulling content as necessary from some old blog posts. I plan to extend the rand demo with some random buffer iovec slice permutations, since this will be needed later by a future compare demo (already expected to run rather long).
random demo ¶
It took me just over an hour to write the new rand demo. (Formatting so much stuff takes longer than typing prose; I usually write around 800 to 1600 words an hour when composing technical prose — always done in one draft.)
Hmm. I just remembered I meant to add pseudo random permutations; maybe I'll tack that on later.
editing for brevity ¶
Occasionally I feel an urge to describe how I edit — because I do edit: for brevity. But I do it as I'm typing. As I think of my next sentence, I scan the last one and trim excess.
Such editing takes almost no thought after much practice. I hardly notice I'm doing it, which is why I haven't posted examples before. I recently caught myself on a very short example. I started with this clause:
"the status can be reversed by the time of return."
Which I immediately cut down to this:
"status can reverse by return time."
This shows how most of my words get cut: entire words are dropped when meaning doesn't change. (I did it again just now: I removed "the" before "meaning.")
I kill a lot of articles: a, an, and the. I try to excise commas. I collapse two words into one when possible. I lose weak adjectives (eg very) and whack as many "that" connectives I can. I axe many that's using gerund form for verbs (ending in "ing"). For example, you can change "that shine" into "shining" much of the time, improving clarity as well. Note "improving" is better than "which improves."
fathers day demos ¶
(Hanging out with my teenage sons trumped writing much today. I bought them new cards and new games.)
I'm in the middle of drafting new rand and stat demos, both of which are basically rewrites of boxmuller and normal blog entries from top column left in August2007. (Those in turn were rewrites of 2001 and 2005 blog posts.) Except these new versions of old material use the same style of exposition as other demos. I hope the new versions are more clear.
Some of my writing exploration the last year aimed to find a better way to describe code like that, and the new demos are the compromise I'll accept between lousy and ideal.
mutex finished ¶
I finished the mutex demo by writing a fresh new version of thread-safe request queue similar to others I've done before. This one uses STL lists as the queue storage mechanism because it makes the demo simpler and shorter. Tomorrow I'll write a new demo on another topic.
agent finch ¶
I must have spent at least a half hour today visualizing scenes from fiction explaining what happens to Ulf next. It stitches together several independent bits of business, though that would not be apparent given flow from one to the next. Rendering a nice written version would take a couple hours.
I'm not sure when I'll have time to write it up, though. In priority it loses to writing demos. However, it's getting interesting to me lately and I might make time for it soon. It's just that I must assume it's drivel to folks interested only in code.
Here's the really short version: Finch roughs up Ulf a little, which is surprising since Ulf is bigger. Zé calls it in before it happens by cell phone, interrupting Wil's long discussion with Eli while driving back from the archery range, where Wil was teaching Eli the basics of handling recurve bows.
Wil and Eli are comparing notes on Finch when Zé calls to say Ulf and Finch look poised to collide at Wil's place, where the group has recently setup temporary headquarters. Wil's having a problem with Finch's age which doesn't fit how much she knows.
The drive back and events at Wil's place are full of suggestive incident and conversation, with at least one surreal city street environment factor dismaying Eli. Wil thinks Finch's Glasshouse codename for his place sounds too cloak-and-dagger.
hammer time ¶
The mutex demo is now three quarters done — actually more like seven eighths. I'll either add a dialog or a thread-safe request queue template to finish it off.
expanded demo ¶
A mutex demo won't go up until tomorrow because parts are more fun to write than I expected — it'll be a lot longer than I guessed. Fun is good, so another day is fine. I had supposed I would not say much about semantically complex parts like how error detection works and why you'd do it. But I suspect I've told other developers the story so many times in day jobs that it's easy to rattle off one more time.
television ¶
As long as I'm posting something, I'll mention what a big waste of time television is, most of the time. I don't watch television very much, and haven't since I was 17. I didn't have a set for several years. My ex was always more into it than I was. I'm more a movie buff. I see movies on the tube more than anything else, when I'm not playing a DVD. (I missed all of Mork and Mindy except for re-runs because it aired during a total abstinence period.)
However, at any given time there's usually some tv series I try to watch — usually just one. For example, The X-Files was the only show I sought out for a few years. Lately I watch two things: House and High Stakes Poker. (I can bear to watch Mythbusters when my sons insist, but I wouldn't go out of my way.) I don't need to explain why House is entertaining if you've seen it. One of the gimmicks I love most is characters talking at the same time or interleaving independent conversations.
I don't care for comedy, unless it has a mean streak like House. I missed virtually all of Jerry Seinfeld, and couldn't sit through an episode when I tried. Short attention span stuff kills me. I'd rather watch geologists drone on science channels.
Liking poker was a surprise to me when I started watching a couple years ago. I like studying how good players process available information and simulate conditions they want other players to believe. It also helps develop better models of behavior in zero-sum game type people if you've a cooperative personality yourself. When I was young I was lousy at grasping aggressive tactics.
follow through ¶
Progress in writing demos lately is due only to follow-through commitment, which I possess in somewhat more abundance than a random joe. I'm no longer interested in demos at all. But my plans related to programming language exploration (my actual motivation) don't make any sense if my goals include full disclosure, unless I describe how I do basic data structures and memory management.
When I show how I code some toy languages, you won't be able to follow it unless my toolset looks easy.
Last week I started looking at ML again with an eye to implementing it, because that was a fun distraction. So I need to hurry demos along before they get too boring to finish.
miranda ¶
Eliot Miranda has a new blog (cf right sidebar) describing the new open source work he's doing on a new Smalltalk VM he's writing for the folks at Qwak. He says he's going to blog all about writing the VM, so that ought to be terribly interesting. Maybe I'll run across something I can use later myself. When I run a search on Eliot Miranda, I see he's been dabbling in Smalltalk implemention since the 80's. Fifteen years ago I'd have considered his job the ideal kind of job to have myself, in terms of being fun.
(I don't know what I'd consider ideal any more, and I don't expect ideal opportunities to occur anyway. I don't think any tech company in the industry currently has an interesting business model, so I don't imagine any place has fun work to do.)
next ¶
I'm writing the next mutex demo, but I won't put it up until tomorrow night (by Tuesday morning). I decided to watch Nicolas Cage in Next again on DVD (imdb: Next, wikipedia: Next) instead of finishing the mutex writeup tonight. In science fiction genre I like precognition almost as much as time travel and teleportation, and lately I've been getting a lot of all three. I read Philip K. Dick's original The Golden Man short story back in my teens, and I recognized Next derived from it when I first saw the trailer months before the movie arrived. I knew it would be good and I wasn't disappointed. Really good flick.
The new mutex demo will be mostly code and not very much explanation, at least in comparison to other demos. And there won't be very much sample code because it really needs to wait for the near future thread demo, which can throw in sample code for a thread pool handling queued requests, demonstrating mutex and condition variable use with the pthread library api.
Returning to movies as a topic, yesterday I watched Bender's Big Score (imdb, wikipedia) with my sons, and I was charmed to see it had a large time travel paradox running joke as a major sub-plot. I had worried it might only be tolerable, but the punchy time travel bits are fast and well done.
Last Sunday night I saw Darren Aronofsky's The Fountain (2006: imdb, wikipedia) which I suspected had a time travel aspect in early parts until I could see how historical elements were confabulated fiction within the fiction, done very nicely. I was only able to watch The Fountain because my younger son watched too. I can't handle sad love stories by myself any more; they twist me up far too much. I composed a piece of fiction while watching that I didn't post last week. I almost rendered it today because I still like the reaction Zé had to the movie, and the way he casually assumes time travel is feasible.
Time travel is one of the running threads I like to keep alive in fiction about the characters I use in stories. It's one of the less mundane plot elements, so the amusing thing to do is make it more mundane somehow, until it turns into an everyday issue like running out of cash at a startup.
cable ¶
Two weeks of awful Comcast service finally ended with a good conclusion today. Now my connection works quickly and reliably again. Replacing my Comcast-supplied modem was the solution, under a theory the old one was flaky. The new one has very different behavior if you judge by blinking green idiot lights; the new behavior is frenetic: several times the amount of activity as old. Too bad it took two service visits this week to fix it.
The first visit was Tuesday morning, during which a tech decided nothing looked wrong after some testing and investigation. So if the modem was shot (as per today's theory) the first guy saw nothing of the sort. Curiously my service was wonderful during Tuesday's morning time slot; then afterward it degraded to what it was before: after each modem power cycle, 100% packet loss followed soon after, with a half life of several minutes.
I shot a few evening hours this week trying to see a problem, finding almost nothing. I developed hare-brained theories without any evidence, which I won't inflict on you. I was very cross in my Thursday night call to Comcast; we scheduled another tech visit late this afternoon, hoping to see poor connections.
