The Counts

1 Comment »

countingMy attention was recently focused (thanks John) on an article by Flickr’s Kellan, describing how Flickr generates system wide unique IDs for user data like photos and comments. In short they have a dedicated ID generator that issues unique, bigint(20) numbers. They implemented this by keeping an almost empty table in MySQL and have the server do a non-standard REPLACE INTO and return the SELECT LAST_INSERT_ID(); of the table. The article claims that they needed a dedicated ID generator because they could not come up with a way to make independent servers generate a system wide unique ID and to prevent the introduction of a single-point-of-failure they came up with a way to make multiple independent servers generate a system wide unique ID. It is “a great example of the Flickr engineering dumbest possible thing that will work design principle.” (Kellan Elliott-McCrea) [Ticket Servers: Distributed Unique Primary Keys on the Cheap] But there is no such thing as dumbest-possible-thing. Give me a candidate dumbest-possible-thing and I’ll add a client-server layer and presto you have something even dumber. Like solving a multi-machine problem by building a multi-machine server.
However I am in favor of system design that goes for the “simplest possible thing that will work (but not simpler than that).” And others might rightfully argue that it’s impossible to prove a candidate simplest thing to actually be the simplest. Oh, I am such a hypocrite, and I did do nothing in my life that comes even remotely close to an installation like Flickr, they rock! But it’s amusing nonetheless.

HipHop: PHP to C++ Compiler


HipHopHipHop for PHP isn’t technically a compiler itself. Rather it is a source code transformer. HipHop programmatically transforms your PHP source code into highly optimized C++ and then uses g++ to compile it. HipHop executes the source code in a semantically equivalent manner and sacrifices some rarely used features — such as eval() — in exchange for improved performance. HipHop includes a code transformer, a reimplementation of PHP’s runtime system, and a rewrite of many common PHP Extensions to take advantage of these performance optimizations. (Haiping Zhao) [HipHop for PHP: Move Fast]
Given the totally adhock way PHP is structured combined with the fact that Zhao claims his compiler is not a compiler but a “source code transformer” makes me wonder what exactly is going on. These steps the transformer takes to do it’s transforming look suspiciously like compiler steps:
Whenever I heard people redefine the term “compiler” it usually turned out they were hand coding a recursive decent parser. I know, it’s easy to bash something you have no experience with, but if someone is talking about “reimplementation of PHP’s runtime system” any professional-coder should raise an eyebrow. Besides, would you want to clone all that PHP magic stuff into a new runtime system? Next to a PHP compiler, Zaho is also writing a PHP interpreter: We have also developed HPHPi, which is an experimental interpreter designed for development. When using HPHPi you don’t need to compile your PHP source code before running it. It’s helped us catch bugs in HipHop itself and provides engineers a way to use HipHop without changing how they write PHP. My guess is that facebook would have been better off with a formal language that is PHP like but much simpler and mostly orthogonal. On the other hand, being well designed is no prerequisite for being popular.

Control Is The Ultimate Illusion

No Comments »

peoplewareTom DeMarco has turned 180o, finally! As one of the initiators and most prolific proponents, if not principle responsibles of the software engineering myth, he finally sees his erring ways. After selling a fuckton of books on software-engineering like “Peopleware,” “Deadline,” “Controlling Software Projects,” and “Waltzing with Bears.” he now, finally, takes his distance. A bit late after 40 years, but I guess, the money was too good. He, at long last, acknowledges the folly of his most famous quote: “You can’t control what you can’t measure.” Never came so much stress to so many coders from so few words. Pity he does not apologize profoundly, but still it is good that he basically tells the world he has been wrong and misinterpreted. Finally he acknowledges, what all real-life-coders knew all along, the more you focus on control, the more likely you’re working on a project that’s striving to deliver something of relatively minor value. (Tom DeMarco) [Software Engineering: An Idea Whose Time Has Come and Gone?] It is ending the era of the software engineering myth. It has hit the community hard, just look at a few reactions: CH, iTwire, Stackoverflow, Software Architecture, Never In Doubt.

Kinda Like Assert


In 1983 my math teacher wrote on the blackboard: P{Q}R. Ever since, my code is loaded with assertions. Assertions should be used to document logically impossible situations and discover programming errors […] This is distinct from error handling: most error conditions are possible, although some may be extremely unlikely. (Wikipedia) [Assertion] We all have seen, but never produced, anti-code like:

  assert(buf = malloc(128));

However, I have produced code like:

  assert(!"Sorry, PNG routines not implemented yet.");

Fix-me-later-code like this might get shipped (with or without NDEBUG)… and some user might try a PNG file… after six hours of typing… without saving… We have all been there. You balance a todo-list of 20 items in your head, and it won’t survive deep thought on some unlikely potential problem. Put in an quick assert() and you can continue to work on your todo-list of, now, 19. The problem is not in using the assert(), but in forgetting to replace it with real code, eventually. So here is how I make them stick out:

#define NOT_HANDLED_YET(b) \
  (void)(!(b) || exit_error("Error '"#b"' not handled yet!"))
  (void)exit_error("Sorry, "s" not implemented yet!")

The examples above would read:

  buf = malloc(128);


And remember, no commits on trunk unless grep -c NOT_ *.[ch] returns 2.

Pointer To Pointer


Some hold that pointers in a language like C are dangerous and hard to master. Both maybe true, but so are scalpels. Since you are reading this, you know that pointer practice makes pointer perfect. The same goes for pointers to pointers. I remember vividly when I first saw some elegant code that used a pointer to a pointer (I was working on Amoeba). It was a coding changing event. Before I would write like:
Picture 2
After I would write:
Picture 3
I’ve even used that last bit of code in job interviews. If some self proclaimed hardcore C coder could not explain those lines of code I would know they were no experts. Now that this trick is out of the bag, I will have to use an other one, next time we meet.

Nice Language for Testing GUI Apps

1 Comment »

Sikuli is a research project developed by User Interface Design Group at the MIT […] Sikuli Script automates anything you see on the screen without internal API’s support. You can programmatically control a web page, a desktop application running on Windows/Linux/Mac OS X, or even an iphone application running in an emulator. [Project SIKULI homepage]
From the paper, it shows that Sikuli Script is motivated by the desire to address the limitations of current automation scripting languages like: AppleScript, Windows Scripting, DocWizards, Chickenfoot, CoScripter, and macro recorders like Jitbit9 and QuicKeys. On Slashdot it is promoted as quite a bit more. See [MIT Offers Picture-Centric Programming To the Masses With Sikuli]. Personally, I think, picture-centric programming for the masses will have to wait, but Sikuli is perfect for writing test programs for GUI applications. Thank you MIT for a scripting language that is reasonable indifferent to GUI element alignment changes.

Windowcitis Can Be Contagious

servet8In 2004 I had to make a set of napkins and tablecloths of a unique design. The theme was fallen leaves of the fern-leaved beech (Fagus sylvatica ‘Asplenifolia’). I decided to use PostScript (Ghostscript actually). It has instructions like infill to test collisions, it shrugs at using an aspect-ratio of 28×36 threads per cm and it is very intuitive. I could preview (in the after-shrink aspect-ratio) on screen and paper. I used a very naive algorithm. 1) Generate a unique leaf 2) pick a random position and direction for it and 3) shift it along until it did not collide with anything. It worked wonders. I could generate a napkin of 81×81 cm with 555 leaves in a few minutes. However doing the 2000 leaves of the tablecloth took much longer. About 32 hours. Since I had to send the CD to the weavery in two days, I started it on my wife’s computer and let it run. About 31 hours later my, than, 10 year old son came to me for some help with a computer game… You guessed it, he had rebooted the computer because it was acting “funny.” (It was running FreeBSD.) I could not blame him, rebooting is normal behavior for a Windows user. With the deadline four hours away, I had no choice but optimizing the code for three hours so I could generated the table cloth in minutes.
There are two lessons I learned. First, you can be hit by someone else’s windowcitis, and second sometimes optimizing can lower the risk of failure in unexpected ways. Speaking of lowering failure risk, nowadays all my family members run OS X.

Random Ordering

No Comments »

bugReplies to problems posted on are ordered by vote. So if you put up a question like “What are your best programmer/computer science/programming jokes?” you end up with an randomly ordered list of nerd-jokes. For example: An SQL query goes into a bar, walks up to two tables and asks, ”Can I join you?” is rewarded close to 900 points. Where When your hammer is C++, everything begins to look like a thumb. scores significantly lower. And If you put a million monkeys at a million keyboards, one of them will eventually write a Java program. The rest of them will write Perl programs. even lower. Pure random. To make it more complex some responses all most out-score the “answers” they comment on like: Jesus saves, but only Buddha makes incremental backups. Wanna read more? Go a head, lose your day.

Definitely Worth a Cat

No Comments »

microwave-cdA few months ago we bought a new digital camera, […] it came bundled with a CD of software. So [my wife] innocently ejected the DVD tray, and dropped the CD in. I happened to notice out of the corner of my eye […] screamed “noooooooooooo,” and frantically launched myself across the room in a desperate attempt to keep that CD from launching […] I nearly took out a cat in the process. […] Nobody hates software more than software developers. (Jeff Atwood) [Nobody Hates Software More Than Software Developers]

If Coders Made Ads

No Comments »