Skip to content. | Skip to navigation

Personal tools

>>> ''.join(word[:3].lower() for word in 'David Isaac Glick'.split())

‘davisagli’

Navigation

You are here: Home

David Glick – Plone developer

by admin posted Apr 05, 2010 11:48 PM

on Zope, multiple cores, and the GIL

by David Glick posted Jan 17, 2010 04:12 PM

I recently installed HAProxy as a load-balancer for a site that had previously been running using a single Zope instance using 4 threads. I switched to 2 instances using 2 threads each, load-balanced by HAProxy. I wasn't anticipating that this change would have a noticeable effect on the site's performance, so was happily surprised when the client mentioned that users of the site were commenting on the improved speed.

But why did the site get faster?

Looking at a munin graph of server activity, I observed a noticeable drop in the number of rescheduling interrupts -- a change that coincided with my change in server configuration:

graph showing decreased contention when I switched to more Zope instances with fewer threads

I suspect that the "before" portion of this graph illustrates a problem that occurs when running multi-threaded Python programs on multi-core machines, wherein threads running in different cores fight for control of the Global Interpreter Lock (a problem Dave Beazley has called to the community's attention in a recent presentation) -- and that this explains the improvement in performance once I switched to multiple processes with fewer threads. By switching to multiple processes, we let concurrent processing get managed by the operating system, which is much better at it.

Moral of the story: If you're running Zope on a multi-core machine, having more than 2 threads per Zope instance is probably a bad move performance-wise, compared to the option of running more (load-balanced) instances with fewer threads.

(Using a single thread per instance might be even better, although of course you need to make sure you have enough instances to still handle your load, and you need to make sure single-threaded instances don't make calls to external services which then call back to that instance and block. I haven't experimented with using single-threaded instances yet myself.)

4 comments

Come improve Dexterity at the Tahoe Snow Sprint

by David Glick posted Jan 12, 2010 02:46 PM

This year the West Coast is hosting our own version of the infamous Snow Sprint. I'm really looking forward to spending a week coding, hanging out with Plonistas, and playing in the snow at the upcoming Tahoe Snow Sprint, organized by David Brenneman (dbfrombrc) and coming to California's Sierra Nevada this March 15-19.

The goal of the sprint is to improve the Dexterity content type framework (a modern alternative to Archetypes created by Martin Aspeli and others). As part of the Dexterity team, I want to offer the following list of potential projects to help get your creative juices flowing.

At the sprint, you could...


Implement one of Dexterity's missing features, such as:

Fix some of the other outstanding issues in the Dexterity issue tracker.

Create a ZopeSkel template for Dexterity-based projects.

Improve the through-the-web content type editor.

  • improve usability and/or sexiness
  • add UI for exporting types for work on the filesystem
  • add support for defining vocabularies
  • add support for selecting/configuring custom widgets

Create an editor that allows through-the-web editing of new behaviors (which can then be applied to existing types in a schema-extender-like fashion.)

Add a view editor to accompany the through-the-web schema editor. Deco is coming and will be great, but in the meantime it would be nice to at least have something that generates a basic view template based on your schema and then lets you tweak it and export it to the filesystem.

Build a better workflow editor to accompany the above.

Write a guide to migrating Archetypes-based content types to Dexterity. Or build a tool to do it automatically.

Create replacements for the ATContentTypes types using Dexterity types.

Determine how to handle existing content items sanely when editing schemas.

Devise a PloneFormGen successor that stores its schema in a fashion similar to Dexterity, and makes it easy to convert a form + results into a full-blown content type. Bonus points if the form editing is done using Deco. :)


There are so many interesting possibilities I'm having trouble deciding what to focus on myself. Space is limited, so if any of this strikes your fancy, head on over to Coactivate and sign right up to join us at the sprint!
2 comments

mr.igor

by David Glick posted Dec 14, 2009 02:28 PM

Today I released mr.igor, a utility for helping you write Python faster by filling in missing imports based on where you've imported the names from before.

Here's a one-minute screencast showing how it works.

1 comment
David Glick

David Glick

I am a problem solver trying to make websites easier to build.

Currently I do this in my spare time as a member of the Plone core team, and during the day as an independent web developer specializing in Plone and custom Python web applications.