Skip to content. | Skip to navigation

Personal tools

>>> ''.join(word[:3].lower() for word in 'David Isaac Glick'.split())

‘davisagli’

Navigation

You are here: Home

David Glick – Plone developer

by admin posted Apr 05, 2010 11:48 PM

Ten lesser-known improvements in Plone 4

by David Glick posted Sep 03, 2010 09:50 AM

The just-released Plone 4 includes rather a lot of big enhancements. Here are some of my favorite little ones.

Ten lesser-known improvements in Plone 4

Plone 4's "Sunburst" theme

As you've probably heard by now, Plone 4.0 final was released this week! (The cynical among you are probably muttering that we couldn't even beat Perl 6, but hey, it's here now :) ). Working on Plone 4 has been a huge part of my life for the past year, but I did far from all of the work. Thanks to every one of you who contributed; I couldn't be prouder of what we've created together!

See the link above for info on major features in the release, but there are a lot more little things included too. Without further ado, here are ten of my favorite Plone 4 improvements that nobody's talking about...

General / site admin features

1. New editor features. Check the control panel for TinyMCE (our new WYSIWYG editor) to find some useful buttons that aren't enabled by default:

  • Paste as Plain Text (preserves newlines)
  • Paste from Word (strips junk markup)
  • Insert/edit Media

(among others)

2. Better date/time handling. The display of event times in listings is now smarter. If the event begins and ends on different days, the time is not shown. If the start and end times are the same, it is not shown twice.

In addition, dates and times are now stored with a timezone, with proper consideration of daylight savings time.

3. Inline error tracebacks. If you're logged in as a Manager and encounter an error, the traceback is shown immediately, rather than requiring you to click through to the error_log in the ZMI.

4. Permission auditing. Thanks to a new feature in Zope 2.12, it's now easy to double-check what permissions a particular user will have in a particular context, via this button on the Security tab in the ZMI:

Permission auditing

For integrators

5. Better control over portlet columns. In Plone 3, it was possible to make a template prevent rendering of the portlet columns by filling METAL slots. In Plone 4, this can be done by setting the disable_plone.leftcolumn and disable_plone.rightcolumn flags on the request before calling main_template. For example:

<tal:block tal:define="foo python:request.set('disable_plone.leftcolumn', 1)"/>
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"
 xmlns:tal="http://xml.zope.org/namespaces/tal"
 xmlns:metal="http://xml.zope.org/namespaces/metal"
 xmlns:i18n="http://xml.zope.org/namespaces/i18n"
 lang="en"
 metal:use-macro="context/main_template/macros/master"
 i18n:domain="plone">

(snip)
</html>

(This might also be done in Python in the __init__ method of a browser view.) Handily, it's actually a 3-way flag: set it to True to force hiding the colum, False to force displaying it even if there are no portlets assigned, and None to use the default logic based on whether there are portlets assigned.

6. "All content" listing view and content-core macro. There is a new folder listing display option called "All content", which renders the full content of each item. (See it in action on the homepage of this website.)

This listing takes advantage of the new content-core slot which is defined in main_template following the title and description and between the various above/below viewlet managers, and filled by each content type's default view. This gives each content type control over how it renders in the "all content" listing. (The "all content" listing also takes advantage of some new slots/macros in the folder_listing template to reuse its logic for batching and fetching item properties; this may be a useful model for other custom folder listings.)

It also makes it simpler to write a view template for a content type, since one no longer needs to include the title, description, and common viewlet managers as boilerplate. The simplest view template is now (this is a copy of Plone 4's document_view):

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"
 xmlns:tal="http://xml.zope.org/namespaces/tal"
 xmlns:metal="http://xml.zope.org/namespaces/metal"
 xmlns:i18n="http://xml.zope.org/namespaces/i18n"
 lang="en" metal:use-macro="context/main_template/macros/master"
 i18n:domain="plone">
 <body>
  <metal:content-core fill-slot="content-core">
   <metal:content-core define-macro="content-core">
    <metal:field use-macro="python:context.widget('text', mode='view')">
     Body text
    </metal:field>
   </metal:content-core>
  </metal:content-core>
 </body>
</html>

For backwards compatibility, templates may still continue to fill the main slot instead of content-core, and provide their own title, description, and viewlet managers.

7. Flexible image scaling. There is now an Image Handling control panel which allows configuring the standard image scale sizes used by Plone.

In addition, since image scales are now generated on demand, it's quite easy for a template to request an image scale of a non-standard size, using this idiom:

<img tal:define="scale context/@@images"
     tal:replace="structure python: scale.scale('image', width=42, height=42).tag()" />

Including a standard image scale is simpler too:

<img tal:replace="structure context/@@images/image/mini" />

See plone.app.imaging for additional options.

For developers

8. Automatic registering of Zope 2 permissions. In Zope 2.10, registering a new permission took two steps (using the <permission/> ZCML directive to register a Zope 3 permission utility, and registering it as a Zope 2 permission, usually by calling Products.CMFCore.permissions.setDefaultRoles during product initialization).

In Zope 2.12, only the first step is required.  The equivalent of setDefaultRoles is automatically called after processing the permission directive, and sets the default roles to ('Manager',).

If you want different default roles, you must still call setDefaultRoles yourself, prior to loading of ZCML (e.g., at import time). This will improve in Plone 4.1 / Zope 2.13, where the permission directive gets an optional role subdirective to specify the desired default roles in ZCML.

9. Fixed ordering of CMF/Archetypes interaction during object creation. In Plone 3, when you created a new content item, CMF called the factory method of an Archetypes content type, then called its ObjectCreated event, then updated the item's portal_type attribute. This had the unfortunate effect that if you were trying to create a schema extender that used the portal_type to determine whether the extender should be applied (perhaps based on some configurable property), and your site used content types that shared the class of other content types, you were out of luck.

This is now fixed so that ObjectCreated is not fired until the portal_type is set. I think we also managed to avoid calling some event twice during object creation that was causing duplicate catalog indexing, but I could be misremembering that.

10. Folder ordering adapters. The order of items in a folder is now looked up via a named adapter of the folder to plone.folder.interfaces.IOrdering—and it is looked up dynamically when ordering catalog results by getObjPositionInParent, rather than consulting an index. This means that it should now be simpler to customize how a particular folder is ordered (e.g. to automatically sort alphabetically or by date), by writing a new adapter that provides IOrdering and setting a folder to use it. To set the name of the IOrdering adapter that is used for a particular folder, use its setOrdering method.

Out of the box, most folders use the DefaultOrdering adapter, which allows user-configurable order. This should be appropriate in many cases. Folders that were migrated from Plone 3's Large Plone Folders use the "unordered" ordering (but this shouldn't be needed for new folders that will store many items, as the default ordering is reasonably performant). plone.folder also has a "partial" ordering which allows specifying the order for some but not all subitems, but this is not directly used or supported in the Plone UI at this time.


 

On to 4.1 and beyond!

4 comments

plone.app.themeeditor 1.0a1 ... and a roundup of other recent releases

by David Glick posted Aug 04, 2010 10:55 PM

The Groundwire team and I have been busy bringing you updates to your favorite Plone add-ons.

Sometimes it's easy to get carried away fixing bugs and making new releases without remembering to announce them.  So here's a blog post to catch up on what the Groundwire web team and I have been up to in Plone add-on land the past few months.

Most noteworthily, David Bain has been working on plone.app.themeeditor, an improved UI for customizing Plone theme resources, as his Google Summer of Code (GSOC) project. This week we were finally ready to release the first alpha. Please try it out and give us feedback so that we can continue to make it better!  (Note the special installation instructions if you're on Plone 3.)  GSOC is almost over, but David is still at work trying to add support for automatically exporting customized items to a filesystem theme product.  If you missed the blog post where I introduced the theme editor, you can go watch the demo screencast.

And a bunch more...

pfg.donationform 1.0 - This product simplifies the process of setting up a PloneFormGen-based donation form that processes payments via PloneGetPaid. It provides a custom add form that creates a PloneFormGen form with fields for contact and billing info, and an adapter to fill a cart and initiate checkout. It includes a custom "donation field" and widget for use with PloneFormGen, which allows choosing from a list of predefined named donation levels or choosing an arbitrary amount to donate. It also supports an option to make a recurring donation if you're using a GetPaid payment processor that supports that (currently the Authorize.net and PayPal processors).

Products.PressRoom 3.8 - We've been fixing various issues with Plone 4 compatibility, and Matt Yoder just made a change so that the various listings (press releases, press clippings, etc.) are now based on Collections, for improved customizability.

Products.salesforcebaseconnector 1.3 - Query results can now be accessed from RestrictedPython in templates, and there is a new validateCredentials method to aid in remotely monitoring to confirm that the Salesforce connection is working.

Products.salesforcepfgadapter 1.6.2 - The new release includes a bugfix to support the case when you're using the adapter in update mode and the user is trying to clear a non-required field. (thanks to Matt Yoder)

collective.megaphone 1.4 - This new release of Groundwire's online advocacy campaign tool contains some fixes for Plone 4 compatibility, as well as a rewrite of the drag-and-drop UI for reordering fields to avoid a dependency on collective.jqueryui.  Look for a bigger Megaphone release including a new Petition feature soon.

collective.salesforce.authplugin 1.4 - Contains a couple bugfixes, plus some improvements to make it easier to use multiple auth plugins in the same Plone site (i.e. if you need to authenticate against multiple types of Salesforce objects).

collective.simplesocial 1.3 - Adds a couple features including a collective.googleanalytics plugin to track clickthroughs when prompting people to post to their Facebook feed, and an option to only show the latter after an edit. (thanks to Matt Yoder)

collective.z3cform.wizard 1.3.2 - Avoids some spurious inline editing errors.

plone.app.jquerytools 1.1.2 - This is mostly Steve McMahon's brainchild, but in the most recent release I added a stylesheet for Plone 3 based on the overlay styles from Plone 4's Sunburst theme, so that add-ons using jquerytools don't have to provide styles for the popup overlays.

Products.PloneFormGen 1.6.0b4 - We shifted to a different approach for getting keys for storing items in the saved data adapter, which should help avoid conflicts in heavy-write scenarios.

Finally, as I write this, Martin Aspeli is making a new beta release of the Dexterity content type system, incorporating numerous bugfixes.

2 comments

In-browser integration testing with Windmill

by David Glick posted Jun 09, 2010 11:16 PM

Windmill makes it easy to run automated tests of a Plone project in a real browser.

I really like writing integration tests for web projects using the Zope testbrowser, which is a convenience API around the mechanize library. But its Achilles heel has always of course been that it only operates on the HTML response, so can't test interactive functionality built with Javascript and AJAX. So I've wanted to try one of the options for running tests in a real browser for a while. Actually it's a testament to the utility of the Zope testbrowser (or my own laziness?) that I made it this long. But the Plone resource customizer uses a lot of AJAX, so it was time.

So first I tried Selenium. I'd heard about Selenium from a number of people, and it's cool. There is a Firefox plugin that lets you record actions and assertions and play them back. It also lets you export these tests to Python to be run through the selenium Python bindings in conjunction with Selenium RC. There's even a Selenium Grid for running tests in parallel on multiple machines.

Unfortunately, while I found collective.ploneseltest which looked like just what I needed—it provides a base Selenium test case for use with the Zope testrunner—I had trouble getting it to actually work. Using version 1.0.3 of Selenium RC and of the Python bindings, Selenium RC was sending an extra HEAD request before each GET request, which was getting interpreted incorrectly by the ZPublisher during traversal in Zope.  A query to Twitter yielded the information (from the creator of selenium himself) that new Python bindings for selenium 2.0 are available as of last week, but I found they don't work (at least not yet) with Python 2.4. And anyway, by that point Martin Aspeli had also replied to my query and suggested trying Windmill instead.

Windmill is another web testing framework that actually seems to have quite similar functionality to Selenium, at least from a cursory examination. It also has an in-browser controller for recording, and can export tests to Python code. There is a Zope testrunner integration for Windmill too, in the niteoweb.windmill package.

And Windmill was quite a bit easier to get working. I just added the following to my package's setup.py to define a new installation "extra":

extras_require = {
    'test': ['niteoweb.windmill',],
},

And then modified my buildout's test runner to include that extra:

[test]
recipe = zc.recipe.testrunner
eggs =
    ${instance:eggs}
    plone.app.skineditor [test]
defaults = ['--exit-with-status', '--auto-color', '--auto-progress']

I added a new test module called test_integration.py and made it use the base test case from niteoweb.windmill:

import unittest
from niteoweb.windmill import WindmillTestCase
from Products.PloneTestCase.setup import setupPloneSite
from Products.PloneTestCase.layer import onsetup
from Products.Five.zcml import load_config
from Testing import ZopeTestCase as ztc

@onsetup
def load_zcml():
    import plone.app.skineditor
    load_config('configure.zcml', plone.app.skineditor)
    ztc.installPackage('plone.app.skineditor')

load_zcml()
setupPloneSite(products=['plone.app.skineditor'])

class SkinEditorIntegrationTestCase(WindmillTestCase):

    def afterSetUp(self):
        """Setup for each test
        """
        ztc.utils.setupCoreSessions(self.app)
        self.setRoles(['Manager'])
        self.login_user()

    def test_customize_logo(self):
        import pdb; pdb.set_trace()

def test_suite():
    return unittest.defaultTestLoader.loadTestsFromName(__name__)

(Most of this is standard test setup boilerplate. I could probably be a bit more sophisticated about the test setup and use layers or something, but this is the tried and true test setup I've been using since Martin published it in his book. The load_zcml method and setupPloneSite method are deferred and run when the Plone test layer is set up by the test runner.)

So far I just added one test that enters pdb. At this point, I can run the test with bin/test -s plone.app.skineditor, and Windmill will start up the ZServer with a dummy Plone site, fire up a browser and a controller window, and then pause at the pdb. (Unlike Selenium, I don't have to install a browser plugin or have another process running first. Nice!) Then I can play around with the controller and start recording tests.  The afterSetUp method runs before each test, and calls niteoweb.windmill's "login_user" helper, which does an initial login to the site as a Manager user.  It also tells the test infrastructure to support sessions, which are needed for my app.

Actually recording a test is mostly a point-and-click affair—hit the "record" button and go at it—but with a few caveats that I'll note below. It took a little bit for me to get used to the ways of selecting parts of the page and making assertions, but at least Windmill provides some flexibility. You can select by id, via XPath expressions, via JQuery selectors, or several other methods.  And assertions range from asserting that certain text is on the page to asserting that an arbitrary Javascript expression evaluates to true.

Here's the full test case I ended up with (dumped from the controller using the "save" button):

def test_customize_logo(self):
    client = self.wm
    client.click(id=u'user-name')
    # load customizer
    client.click(link=u'Site Setup')
    client.waits.forPageLoad(timeout=u'20000')
    client.click(link=u'Theme Editor')
    # go into advanced mode and customize based on the logo in the
    # non-active "plone_images" layer (Windmill doesn't do file uploads)
    client.click(link=u'Advanced')
    client.click(id=u'plone-app-skineditor-name-field')
    client.type(text=u'logo', id=u'plone-app-skineditor-name-field')
    client.click(id=u'plone-app-skineditor-filter-button')
    client.waits.forElement(timeout=u'', id=u'plone-app-skineditor-browser')
    client.click(xpath=u"//a[@id='skineditor-logo.png']/dt")
    client.waits.forElement(xpath=u"//dd[@class='plone-app-skineditor-layers']")
    client.click(jquery=u"('a[href*=plone_images/logo.png/manage_main]')[0]")
    client.waits.forElement(jquery=u"('#pb_1 input[value=Customize]')")
    client.click(name=u'submit')
    client.waits.forElement(timeout=u'', id=u'pb_2')
    # now reload and make sure the logo has the height we expect from the
    # customized image
    client.refresh()
    client.asserts.assertJS(js=u"$('#portal-logo').height() == 57")
    # now remove the customization
    client.click(id=u'plone-app-skineditor-name-field')
    client.type(text=u'logo', id=u'plone-app-skineditor-name-field')
    client.click(id=u'plone-app-skineditor-filter-button')
    client.waits.forElement(timeout=u'', id=u'skineditor-logo.png')
    client.click(xpath=u"//a[@id='skineditor-logo.png']/dt")
    client.waits.forElement(xpath=u"//dd[@class='plone-app-skineditor-layers']")
    client.click(link=u'Remove')
    # and confirm we're back to the original height
    client.refresh()
    client.asserts.assertJS(js=u"$('#portal-logo').height() == 56")

And here's what it looks like to run that. (No clicking on my part involved!) The excitement starts about 0:17...


Finally, here are a few caveats I ran into while setting up my first test, to customize the logo using plone.app.skineditor, and some last observations.

  • Windmill can't do file uploads. This is a limitation of browser Javascript support / sandboxing, not of Windmill per se.  It would be nice if there were some command that would prime the Windmill HTTP proxy to add a particular file to the next HTTP request that comes through, so that uploads could at least be faked.
  • Windmill could be a bit smarter about handling AJAX requests when recording.  I needed to manually add a "waitForElement" step after each click that resulted in an AJAX load, to make sure that the load was complete before subsequent steps run.  It would be nice if there was a "wait for all ajax to complete" command so that I didn't have to identify a particular element to wait for.
  • I wish that there was a wrapper that would let me control Windmill using the same API as zope.testbrowser, to make it easier to convert existing tests to full browser tests.
  • So far I've only tried running tests in my default browser (Firefox), but it's possible to run in other browsers as well.  The niteoweb.windmill page on PyPI gives an example of a test layer for doing this.
  • It remains to be seen how cumbersome this sort of test will be to keep up-to-date as the product evolves.

Overall Windmill provides a decent experience for recording tests and a really nice one for running them. Perhaps it is a tool we can use to do real browser testing for Plone core?

If you've used Windmill and have any insights into how to use it effectively, I'd love to hear your thoughts in the comments.

3 comments
David Glick

David Glick

I am a problem solver trying to make websites easier to build.

Currently I do this in my spare time as a member of the Plone core team, and during the day as an independent web developer specializing in Plone and custom Python web applications.