Welcome to the personal weblog of Wincent Colaiuta
This is my personal weblog where I write things that aren't always related to my business. If you're looking for information about my products please go to wincent.dev, and for official news see here.
March 25, 2008
Final server migration this week
I've been working on moving to the new server for nearly a month now, a process which has been quite slow because all the data has to be exported from a number of "legacy" applications and imported into a brand new custom Rails application written especially for this site. It's slow going because it's the first time I've ever deployed a Rails application and I've had a lot to learn... but now I know a fair bit about nginx, capistrano, mongrel, monit, and of course Rails itself. The server too is totally different, with a different operating system, so it's all been quite time-consuming.
Well, the crunch time is now coming and the real server move is about to happen this week (at the end of the March the old server will go offline forever, so the move has to happen this week, ready or not). So this will most likely be my last post ever to this weblog.
The forums are already in read-only mode, and I'm about to shut down the bug tracker too shortly so that I can do the final export of all the records and move them into the database on the new server. If you had an account on the old server you'll automatically have one on the new server too, although you will need to reset your password (I couldn't transfer across any passwords because none of them were stored in plain text, and rightly so of course).
At some point during this week email might temporarily go down, hopefully only for about an hour (delivery failures should be temporary), and the site itself will go down at roughly the same time too. When it comes back up we'll be running on the new server. If you make a purchase during that time you might have to wait an hour or two for your license information, but we'll soon be back to immediate automated delivery.
Keep your eye on my Twitter account if you'd like to be kept posted with updates; it will be my only avenue of communication during my brief stint in limbo:
More Miscellaneous articles
Posted by wincent at 1:18 AM
February 26, 2008
Server migration under way
The server migration from Rackspace to INetU is currently underway. My aim is do this very carefully, service by service, and keep downtime to a minimum; hopefully minutes rather than hours.
Just in case there is any interruption during the move, I've just opened a twitter account where I'll be posting updates along the way:
Not really a twitter fan myself — I've always felt that the best place to post that kind of info is on my own weblog — but in a case like this it makes sense to have an offsite feed.
More Miscellaneous articles
Posted by wincent at 11:53 AM
February 19, 2008
Microphone prices
I'm currently looking at getting a better microphone to do screencasts. The one I'm most interested in goes for between $84 and $100 in the US (57€ to 68€). Purchased locally here in Spain the mic goes for upwards of 130€.
Unfortunately, most of the retailers in the US don't want to ship outside of the US. I found one who would: they quoted me over $160 shipping, plus $50 wire transfer fee (they don't accept foreign credit cards) and the cost of the mic on top of that for a grand total of $294. Ridiculous. That's 200€. Needless to say I'll be holding off until I find a better option.
My past experiences ordering computer components from the US have always been excellent, but it seems that things are different in the audio realm.
Update
Found a reasonably priced European supplier.
More Miscellaneous articles
Posted by wincent at 9:51 PM
February 16, 2008
The joy of deleting code
Things have been quiet here of late because I've been feverishly trying to get things ready in order for the server move coming up at the end of this month. It's a big move because it's not just moving the box from one place to another; I'm changing data centers, hosting companies, operating systems, and application software all at once, and there is a lot of work to be done.
One of the things I'll be doing is replacing the teetering stack of disparate applications that I'm currently running (a mix of applications — Movable Type, UBB.threads, Mailman, Bugzilla, MediaWiki and a number of custom scripts — written in a mix of languages — Perl, PHP, Python and Ruby) with a single unified application, a custom job written in Ruby and built on Rails.
I'm taking the opportunity to greatly simplify things in the "Web 2.0" spirit (ugh, how I hate that term; but I've started to use it because it's lost some of its ridiculous hype feel and has now come to mean a set of widely understood characteristics). And it's actually quite a liberating experience. Each chunk of code that I delete makes me feel lighter, freer.
More Miscellaneous articles
Posted by wincent at 5:53 PM
February 2, 2008
Ragel wins! Fatality!
One of my projects is a fast wikitext-to-HTML translator. It's a Ruby extension written in C with speed being one of its top design goals (robustness, predictability and security being the others). I'm working on this because the Rails rewrite of this site will use wikitext markup for basically everything. Rails has a bad reputation for being slow and resource hungry, and that's why one of my primary objectives is speed; I don't want my wikitext parser being a bottleneck.
The translator itself consists of a hand-coded parser written in C paired with a scanner/tokenizer/lexer generated from a grammar specification.
Up until now the scanner used a ANTLR-generated lexer using the "C target" (although ANTLR itself is written in Java, it can target multiple languages, including C). The C target is the current speed king among ANTLR targets and is likely to remain so for the foreseeable future. It too is engineered with speed in mind and is quite frugal with resources (for example, when emitting tokens it maintains pointers into the input stream rather than making a copy of the substring for each token).
And the results were quite snappy. But for some time I've had the sneaking suspicion that a Ragel-generated scanner would be faster. I suspected this even though ANTLR uses all sorts of clever tricks like trying to predict which path to take and only backtracking if prediction fails.
You see, Ragel generates simple state machines: the building blocks of regular expressions. Whereas ANTLR is like a complex and unpredictable magic clock, Ragel state machines are totally transparent and predictable. I've spent many months trying to use ANTLR on a number of different projects and in every case except for this wikitext translator I've ended up putting it aside because I couldn't divine the incantations required to make it do its black magic. Ragel, on the other hand, is absurdly simple. You can build complex things out of it, but the base components are easily comprehensible. Compared to ANTLR, Ragel is incredibly easy to learn.
More Development articles
Posted by wincent at 6:07 PM
January 31, 2008
Farewell Rackspace
I wrote the other day that I was going to be moving to RHEL 5 in the near future. What I wasn't sure at the time was exactly how I'd be making the move. At the time it seemed that the most likely pathway was going to be installing a new drive for RHEL 5 and migrating the data over afterwards; the exact mechanics of this were never really defined (would the drive have RHEL 5 pre-installed on it prior to being slotted in? would they need to take the machine down and do the install "in place"?).
Ideally my host (Rackspace) would have slotted in a drive with a pre-installed OS on it, brought the server online so I could migrate my data over to the new drive and then rebooted from the new drive. I thought they use identical, standard hardware configurations to allow precisely this kind of thing.
So you'd be looking at two reboots, one of them accompanied by a few minutes of downtime to physically install the drive, and then a period in which some services didn't work properly because they'd need to be tweaked, set-up and configured only after booting into RHEL 5 for the first time. But overall impact to your website visitors would be fairly minimal. Once all that was done you'd be looking at another quick powercycle while they shut down the server to remove the old drive (you don't get anything for free from Rackspace).
I wonder how often Rackspace does this type of upgrade. As it was, they were suprisingly slow to offer the details of the procedure and offer options, and that's what makes me wonder.
After quite a bit of deep thought I decided that instead of staying with Rackspace I'd move on to another company. I've chosen INetU.
More Miscellaneous articles
Posted by wincent at 11:53 AM
January 28, 2008
Involuntary Reboot Log #25
Like I said the other day, looks like this iMac is dying. This time the machine simply turned itself off when I bumped the desk with my knee; it could barely even be called a bump.
The average time between failures is dropping. Not long ago it was about one month, but now it's approaching three weeks, and if you look at the average over the last month the failures have been almost weekly.
Apart from the risk of damage to my data, I am starting to get sick of these Disk First Aid runs (during which the machine is unusable), which go especially slowly because Spotlight decides to reindex all of my volumes every time I have one of these failures.
On a tangential note: I received an email in response to my last post in which I complained about the lack of RAM in the low-end iMacs. It was pointed out to me that real-world RAM really is cheap nowadays (even if RAM from Apple is not). You can max out an iMac to 4 GB for a mere $100 at my retailer of choice (OWC), and for someone like me living in Europe and buying the RAM from the US you're still looking at only 85€ including international shipping.
Involuntary reboot stats to date
- Operating system version: 10.5.1
- Kernel panics: 7
- Hard resets: 18
- Total failures: 25
- Start of recording keeping: 21 May 2006
- Total days to date: 617 days
- Average time between failures: 24.68 days
- Uptime at moment of failure(s): a day or so
More Involuntary Reboot Log articles
Posted by wincent at 9:39 PM
January 25, 2008
Rails and validates_uniqueness_of
I'm a Rails dabbler who has never actually deployed an application yet but has been working in spare moments on a new version of this site that will be powered by Rails.
I was puzzled today to see excess SELECT queries whenever I saved a record. The queries were for attributes that were entirely unrelated to what I was manipulating.
I'll spare you the long story and cut to the chase. It turns out that these extra queries are generated by validates_uniqueness_of. I guess I should have known. The entire model gets validated whenever you save, even attributes that you haven't touched. I guess this makes sense.
Now, I am not really sure about the point of validates_uniqueness_of because it is vulnerable to a classic race condition; basically Rails checks "is this unique?" and if it is goes ahead and saves the record. Of course, in a multi-user application it's possible that what was unique a millisecond again is no longer so.
Basically validates_uniqueness_of can't provide you with any guarantees, and if that's the case then why bother using it? I had it in my app mostly for experimentation purposes.
The only robust way to approach this is to do what we always do when we have race conditions to worry about: go ahead and save the record and only if it doesn't work worry about what you'll do. In order for this to work you need to impose a database-level constraint which will cause an exception to be thrown if someone tries to create a duplicate (and of course, the database won't permit the creation of the duplicate to succeed).
Something like this in a migration should do it:
add_index :things, :attribute, :unique => true
Try to insert a "thing" with a duplicate "attribute" and MySQL will complain about a "Duplicate entry" and Rails will duly raise an ActiveRecord::ActiveRecordError which you can rescue and deal with appropriately.
If you're doing that then is there any point in keeping those validates_uniqueness_of calls? Perhaps. I guess if the probability of races is low then you can keep your application logic simpler by relying on validates_uniqueness_of for informing the user nicely that "that name is already taken", and rely on database-level constraints to handle the really nasty race conditions (and throw an irrecoverable error). I suppose it depends on how much traffic and contention you're expecting. If traffic is high and performance is a real issue then you might just want to get rid of the validation; it doesn't provide any guarantees and those extra queries will only slow you down.
More Development articles
Posted by wincent at 8:51 PM
iMac finally dying?
I'm starting to get really worried about this machine.
Today we have the painters here painting all the rooms, which means that I've had to take the iMac out into the kitchen. As I've mentioned before, it has a wonky power socket, and this time it didn't want to boot up.
Several minutes of jiggling later I finally got the machine to turn on. Now I'm wondering which boot-up is going to be the last. One of these days the machine isn't going to power up.
Financially, I'm not in a position to buy a replacement yet. The cheapest model costs 1,169€ here but is handicapped with only a gig of RAM which would probably make it slower than this machine (which has 2 gig) despite having a faster processor (a 2 GHz Core 2 Duo as opposed to the 1.83 GHz Core Duo I currently have). RAM is very important for performance when compiling code in Xcode, mostly due to the amount of memory occupied by precompiled headers.
The top-end model would be a big improvement over my current machine — a 2.7 GHz Core 2 Extreme with 2 gig of RAM and a bigger screen (24" up from 17") and hard drive (500 GB up from 160 GB) — but it's out of my price range at 2,159€.
So really my only two options at this stage are to cross my fingers and hope that this machine holds up, or send it in for repair; and with the deadlines I have looming I just can't afford to be without this machine, even for a few days.
More Miscellaneous articles
Posted by wincent at 10:28 AM
January 21, 2008
Git 1.5.4-rc4
Git 1.5.4-rc4 is out, and the final 1.5.4 release is likely to come soon, probably before the end of the month. My favorite change in this build is the fix for the performance regression when commiting a single file specified on the command line (eg. git commit foo) as this is something that I do often.
The regression was only in the development version and has never made it into a released version of Git, but I've been tracking the master branch for some time now (see "Tracking the Git maint or master branches") so I had definitely noticed the problem.
More Development articles
Posted by wincent at 12:51 PM
January 19, 2008
Moving to RHEL 5
In October last year I wrote about changing UNIXes. Well, it's a new year now and it looks like it is going to happen.
I'll be migrating to RHEL 5 (from RHEL 3). I don't have a firm date for the migration yet, and I need to do quite a bit of research before then about the best way to get all my data across. This will basically be a clean install on a new hard drive, the server should only be down for a relatively short interval, and then we'll come back on line and over the following hours I'll be transferring user accounts, mail, databases, software and everything else from the old hard drive to the new. So it will be quite intense, and most services will be interrupted for at least some of the migration. I expect to be transferring and configuring for several hours, possibly even the entire day.
One way to minimize the interruption would be to install the drive with the new system on it and migrate some of the data before rebooting, but I'm not sure how much time that will actually buy me, nor whether my host, Rackspace, would be prepared to do that (it would require them to install RHEL 5 on the drive in another machine, then uninstall it from that machine and reinstall it into mine).
I'm very happy about this because it means I will be able to gradually retire bits of legacy software and replace all of this with a single, lean, unified Rails application which handles everything for me (bug tracking, customer support tracking, forums, the wiki, and so forth).
Deploying Rails is a bitch by all accounts, but it's easier on RHEL 5 than it is on RHEL 3, (specifically, there are fewer pieces of additional software that you have to install, and you deviate less from the supported, auto-updated configuration). And once you've deployed it you've got one application stack to worry about instead of half a dozen.
More Miscellaneous articles
Posted by wincent at 2:56 PM
January 17, 2008
Involuntary Reboot Log #24
Only a few days since the last kernel panic and I've just had another hard reset. This time the machine just froze completely with no explanation (mouse pointer frozen, clock frozen, no visible movement on screen or sound from the hard drive). This machine may well be on its last legs.
As always I did a Disk First Aid run after the crash. This time there was a bunch of damage that needed to be repaired. HFS+: is the F for Farce?
Involuntary reboot stats to date
- Operating system version: 10.5.1
- Kernel panics: 7
- Hard resets: 17
- Total failures: 24
- Start of recording keeping: 21 May 2006
- Total days to date: 606 days
- Average time between failures: 25.25 days
- Uptime at moment of failure(s): a day or so
More Involuntary Reboot Log articles
Posted by wincent at 7:05 PM
January 16, 2008
On the MacBook Air
I admit to not being really excited about this year's keynote. I've been so engrossed with work of late that I didn't even know what day the keynote would take place. And I knew that most of the things that would be talked about wouldn't excite me: Apple TV (don't have a TV), iPhone stuff (you can't buy iPhones in the country where I live yet), iTunes movie rentals (I don't like the rental model, I'd rather own stuff). Furthermore, the machines which most interest me, the Mac Pros, were already revamped before the show started.
But I must confess, even though I am not in the market to buy any new Apple product right now, I am really impressed with the new MacBook Air. If I were to buy a laptop, this would be the one for me. I am not one of those people who wants their laptop to be their only machine; for these people the Air is not the right product. I'm a person who wants a portable to be portable; so for me the decision to leave out the optical drive is a very clever optimization. Ever since Apple dropped a 12" model from their line-up I've felt it was a gap. I still have my 12" PowerBook G4 (the aluminum one). It was a great machine and has never skipped a beat in all these years (over four years now, eons in computer time).
So if I were to buy a laptop, the Air would be the one. The most portable portable ever!
More Apple articles
Posted by wincent at 10:26 AM
January 15, 2008
RSpec + Autotest + Objective-C
In this screencast I demo using RSpec and Autotest to do continuous integration/BDD (choose your favorite buzzword) of a compiled Objective-C tool:
I've got the flu right now, so apologies for the voice.
Update
I've just seen this post on David Chelimsky's weblog that confirms that this indeed new for 3.8.0 and I wasn't just imagining it...
More Screencasts articles
Posted by wincent at 12:26 PM
January 14, 2008
Open Source economics for the small guy
I like the open source model. I really do think it has the potential to produce better software. Note that I said potential; survey the market place and you'll find examples of open source projects which are best-of-class and others which are severely lacking.
I currently make a bit of open source software available but also work on closed source stuff. If I thought i could make everything open source and still survive economically then I'd do it in a heartbeat. I suspect I'm not alone in feeling this way; as Brent Simmons recently said:
But I will say that, for me personally, this is a dream come true. Every developer wants to be able to work on the software they love, make a living at it, and give it to the world for free. Usually you get to pick two out of three — if you're lucky. Me, I get all three.
This very question came up on Slashdot yesterday. I'm gradually edging towards releasing more source, but it's a difficult path to tread. I'm probably still quite a long way off being able to go fully open, if ever. Think about it: if a company like Apple can't go all the way, how is the small operator going to do so?
More Development articles
Posted by wincent at 1:50 PM
