Skip to content. | Skip to navigation

Personal tools

>>> ''.join(word[:3].lower() for word in 'David Isaac Glick'.split())

‘davisagli’

Navigation

You are here: Home / Blog / Be vigilant: Know your Buildout Threat Level

Be vigilant: Know your Buildout Threat Level

by David Glick posted May 09, 2010 09:09 PM
The Buildout Threat Level indicator reports the percentage of buildouts that have succeeded in the past 4 hours.

For those who missed my announcement last week on Twitter –

As a result of a mostly tongue-in-cheek conversation on IRC with Elizabeth Leddy, Alex Clark, and Matthew Wilkes, I decided to set up the global zc.buildout threat level indicator.

The indicator collects reports of success and failure from actual buildout runs, and displays the current threat level based on the percentage of buildouts that have succeeded in the past 4 hours.

I'm happy to report that the current threat level is LOW.

Buildout Threat Level: Low

Creating the indicator was mostly an excuse to try writing a Google App Engine app, but it's already proved its utility.  Last Thursday the threat level reached ELEVATED after a new distribute release started causing buildouts to fail under Python 2.4. Thanks to Tarek Ziadé for the quick response with a new distribute release to fix that.

You can help make the indicator more accurate!  Just add the buildout.threatlevel extension to your buildout:

[buildout]
extensions = buildout.threatlevel

This will display the current threat level as one of the initial steps when you run buildout (so that you have an opportunity to abort if it has reached SEVERE), and will ping my app engine app when the buildout is done to report success or failure. (No data is collected in this ping aside from a boolean indicating success, time, and IP address.)

Code for the GAE app and buildout extension is in the collective.

Tarek says:
May 09, 2010 11:20 PM
Great idea, great job ! :)

Now, if we were able to get an e-mail notificiation ala Nagios when the threat is high, that would be awesome.
Ian F. Hood says:
May 10, 2010 12:54 AM
Tongue in cheek or not, it was a worthy discussion and I'm certain this app will prove useful time and again. Thanks for putting in the time to do it.
Reinout van Rees says:
May 11, 2010 10:54 PM
Hi, I couldn't find a timeout setting or so in the code. So either offline mode or an appengine failure could stop a buildout dead in its tracks.

Did I miss something?

I wouldn't mind adding this extension. Looks useful. But I don't want to add an extra failure point so that I can be warned about other failure points :-)
David B says:
Jun 08, 2010 09:38 AM
Definitely useful, but a couple of suggestions:
1. I frequently Control-C my buildouts mid-operation because I'll forget something or simply don't need it to go on. The extension considers this to be a failure -- doing this a few times (even just normally) would probably affect things.
2. Some information about why buildouts are having trouble would definitely be useful. Knowing that the current buildout threatlevel is 'elevated' is good, but I guess I'm wondering should I be running around thinking the sky is falling or just carrying on?

My 2 cents worth of suggestions, anyway.

Navigation