Tuesday, March 06, 2007

SimpleWatcher

Like I mentioned last time, I have a newly released project available. I'm calling it SimpleWatcher.

It's exciting for me to release it. I've been using it in-house at my workplace for about 6 months now, so the functionality is fairly stable. I don't have any slick screenshots, since it's primarily a utility-based application, though I hope to change that. I've used wxPython for a while now, but I'd really like to try out Dabo, since the application needs a GUI config builder.

The project is basically the outcome of needing to build a workflow/queue/watch application for a lot of print-based work that I'm involved with. It handles branching and conditional logic, data manipulation, file moving, external script calls, etc, tying them all into simple xml configs. I've kept a lot of the production watch's I've built in as examples of how to build and use the watch configs.

I decided to open-source it as I didn't know of anything else out there that's doing this kind of thing, and I hoped others could get some benefit from the software. I can't imagine I'm the first one to run into this problem.

Regardless, feel free to browse over and take a look. Also, check out SimpleAuth... we have a new version out there, now supporting a get for all roles a user has per configured service, allowing for application-level caching.

Thursday, February 08, 2007

SimpleAuth

It's been a while since my last post. No point in rehashing everything that's happened since then, but, in a purely technical sense, it's been a pretty productive period. I've recently put out the first version of SimpleAuth, a new open source authentication aggregation application. It was a "scratch our own itch" effort by myself and a few of the developers I work with. If you're an enterprise developer who's ever had to struggle with integrating multiple applications, disparate authentication sources (Databases, NT Domain, LDAP, etc). Feel free to go take a look. I'll be releasing an early beta of another open source project shortly, based around workflowing action sets on an input queue. But I'll get into that more later.

Wednesday, July 05, 2006

about.com, phase 2

So, I got accepted into phase two (the Prep phase) of the about.com python Guide selection process. That's really exciting for me. My Prep period starts 7/14/06, running through the end of the month. In that time I'll create a sample Guide site, etc.

I find the whole thing interesting. I've been writing python code for about 2 years now, and have spent my last year-and-a-bit back and forth with Zope3. I love it and I hate it, I think it's one of the best pieces of software I've ever seen while simultaneously feeling like it's sometimes complicated for the sake of being complicated.

Still, after spending a few months with RoR, every time I come back to writing python code, it's rejuvenating. There's nothing wrong with Ruby, and I appreciate some of the language features... but I feel like I'm playing a little too fast and loose in Ruby, the same way I feel in Perl. In both Ruby and Perl, the question is rarely *can* I do it (both languages either can, or can be tricked, into doing most anything), but *what is the right way* to do it? But those aren't complaints so much as personal taste issues. Python just makes more sense to me.

A largely undirected ramble, but still. About.com. How cool is that?

Sunday, June 25, 2006

IRelationshipMapper

I've been thinking about relationships between different kinds of data. This is because I probably need more exciting hobbies. However, I still was, and so I'm going to jot down my thoughts about it.

To my mind, objects should not be cluttered with knowledge (or forced to be responsible for the tracking) of how they're related, and yet, all data, and most objects in a complex system, have some relationship of one-to-one, one-to-many, or many-to-many between them (as well as other more complex relationships). In the past, in order to mantain a relationship between a Zope3 content object and a relational db object, I've used constructs like "obj._db_key_ = some_primary_key".

But what I'm thinking about is a zope3 registry of objects. I've done this already, in a way, for the organization logic in a register/activate/login authentication scheme i developed for an application i was working on. But I think this has application far beyond something as simple as that.

We can create a utility that allows objects to register a relationship to other objects. something like "has_a(IObjectType, as=attribute)", "has_many(ISomeOtherObjType, as=attribute)", "belongs_to(IYetAnother, as=attribute)", etc.

Example, principals who own certain bits of data, but not in an annotated kind of way (assume other objects need access to the data as well). These principals need access to this data programmatically, but without having the content objects limited to certain containers (like home or group folders). In a registry of data, one object can look up it's associated data through the registry, and the relevant data can be given back to the seeking object. Setters and getters that utilize the registry could literally make it as simple as:


shop.customers.append(someNewCustomer)
shop.customers.remove(someOldCustomer)


where shop is an implementation of IShop, which has_many(ICustomers, as="customers"), with setter/getter code like:

relmap = component.getUtility(IRelationshipMapper, context=self)
def getCustomers(self):
return relmap.getRelationship(IShop, relationshipAttribute="customers", context=self)

def setCustomers(self, customerList):
customers = relmap.getRelationship(IShop, relationshipAttribute="customers", context=self)
for customer in customerList:
if customer not in customers:
customers.append(customer)

customers = property(getCustomers, setCustomers)



This is great since shop itself may not actually have a "contained" list of customers, since customers may belong to multiple shops. Customers may even be content objects that are associated with other principals in the system.

What's interesting is that in a registry like this, it could have multiple "item lookup" methods for investigation, so RDB constructed objects as well as ZODB objects could reference one another without having to be aware of what kind of object is, or where it persists. Between UniqId utilities and unique combinations of table/primary key for RDB objects, the Registry would not have to be a terribly complicated piece of software to track where everything is. The only significant complexity would be the relationship classes, that define what kind, and where. And even those wouldn't be terribly difficult. One-to-one would just be an index of object ids related to other object ids. Has_many (one-to-many) would just be an index of object ids with an associated list of other ids.

Catalog indexes for each ZODB object type. We may even delve into autocreating indexes for all Interfaces of registered objects. If object can merely implement (IRelationParticipant) or something similar, as a marker interface, then on notify(ObjectCreatedEvent(obj)), the utility will be a subscriber to that event, and check to see if the created obj implements IRelationParticipant. if so, it should check itself for the registered relationships (marked in the class through has_a, has_many, etc) on that object's uniq id, and set them. then, on every ObjectModifiedEvent, it will (as a subscriber), check to see if the attribute has been modified, and if so, that the new values implement the registered relationship interface (that obj.attr=val has val being IObjectType, if that was the relationship established at the outset). if so, it can be updated with the uniq id of the new set of values.

example:

class IShop(Interface):
"""shop interface"""
has_many(ICustomers, as="customers")

class Shop(object):
"""shop class. without the sawing and/or welding."""
implements(IShop, IRelationParticipant)
customers = HasManyRelationship()

How will the relationship help us? To know what kind of object this is, one to another. So i can say:

customers = relationshipMapper.getRelationship(IShop, relationshipAttribute="customers "context=myShop)


or something similar. or "myobj.children = ['some', 'list', 'of', 'objs']", and have the relmap utility automatically update these (setter/getter/property methods again, back to the registration utility).

Right now, all "contained" objects have __parent__ for persistent ZODB objects to describe the relationship, object to object, in the containment model. But objects have more relationships than simply "contained." It would even be possible to use the registration utility to mark "contained/contains" as a kind of relationship, so that ZODB persisted objects could "contain" RDB objects even (not sure how we'd need to implement IPhysicallyLocatable).

Consider a situation like this: Lets say I have two containers for two kinds of objects. Say I have a container called Shops and another called Customers. Lets say that both shops and customers need a way to be able to log in, and a way to access their data, so we have principal objects associated in an ownership capacity to both Shops and Customers, like ShopPrincipal and CustomerPrincipal, but distinct from these. We don't want Shops and Customers to merely be annotations of the principal, because we want this data able to be looked up and looked at by other logical entities in our application. However, there is a logical connection betweeen Shops and Customers. Shops->have_many->Customers, but Shops is not the __parent__ of it's customers, because when CustomerPrincipal logs in and wants to see her data we don't want to have to either track OR figure out which Shop the Customer data lives in. Especially if a Customer is potentially a Customer of more than one Shop, since Customer->has_many->Shops. the "has_and_belongs_to_many" relationship.

In an RDB, we would just create a join table, with foreign keys of each table's primary key. But in the ZODB, a non relational system, we don't do things that way. IRelationshipMapper could potentially create a software way of working with this kind of relationship model.

Saturday, June 03, 2006

zope3 configuration

so, i feel like my mental abs are sore. the tagline "xml situps" is an apt one.

zcml. the debate rages on in mailing lists, and probably will continue to. I've read arguments on both sides, and I don't really disagree with either side. but this is what i'm thinking about...

i read an email of Stephan Richter's recently where he said that he does all site configuration in python code. I'd considered that before, but didn't know if it was the "right" way to do it. if it is, then I'll probably start doing it that way too. Stephan is probably the right guy to take Zope3 queues from, if you're going to pick someone.

but it makes me wonder if the zmi should even exist. or if maybe it should be included in zope3 packages only if the developer wants it there. like "from zope.app.zmi.admin import controlpanel" or something. or "from zope.app.zmi.error import errorpanel".

which leads, sorta, to my next point. all of the zmi default display pages... they make things fast, certainly. but i think that they're almost immediately a crutch, too. i think they keep the developer from really having to dig into the zope3 page creation machinery, and i think that the page creation machinery (now formlib, which i like a lot) is something that any zope3 developer needs to be fluent in.

i'd like to see the day where "bin/runzope" just starts up a very small shell of functionality, with whatever the developer has included. maybe we could move the zmi to "zope3admin" or something. i don't know that i think the zmi has any real place "being there" in a live app. i don't see any reason that it couldn't be started on it's own port, and hook into the zodb on it's own. i know when i need to do any real debugging of zope3, i do it from zopectl debug anyway, not through the errors page.

so I don't know. there's a lot that the zmi does, certainly. but i think it prevents the "simplicity" that's necessary to get new developers on board for zope3. I know if I hadn't had to learn how to use zcml, zpt, the zope3 framework, AND the zmi all at once, i'd have been in better shape. zcml, zpt, and the framework are all non-domain-specific in their implementation. and it's enough to learn on it's own, even being basically just code and markup.

but then to learn the right order to register new utilities, deregister them, delete them, add them, chain the dependencies...

I am now, just now and I hadn't thought of it before, propsing a "Zope3 Site Builder" set of markup or template code. some way to set up a chain of dependencies, registrations, and necessary content objects. I haven't thought about it that much yet, but I can't imagine it wouldn't be doable. some new pythonic template that could set up and tear down a site in the right order.

Wednesday, May 24, 2006

I will not recant!

Or, maybe I will. Kinda.

I miss the hell out of Zope3.

I admit, I never said I didn't love developing in Zope3, because that would have been a lie. Nor did I ever say Zope3 wasn't a better dev environment than RoR. That would have been untrue. What I did indicate was that RoR is a more streamlined web-centric development environment. and that's probably still true.

But today I was assigned a new development task. Build a watch-style solution for a piece of document management software. I'm excited about the project for a number of reasons, some technological, some ideological, and some of pure dorkery. But I immediately turned to zope for interfaces and for inspiration on implementing an observer pattern. I immediately went to zope for things that you can't find other places.

And that's when it hit me, afresh, though I had known it all along. Zope3 provides functionality at such a powerful level (interface/adapter oriented programming, object observers and event notification, etc) that only when I really need/want these things elsewhere do I realize how much their lack hurts.

RoR is good, sure. It provides a lot of shortcuts and time savers. But the further I get into core design with it, the more I realize that, while it's quite good and I've learned a good bit from it, I wouldn't want to design something massive in it.

I'm not declaring any kind of back-turning on RoR. I still very much appreciate how data-driven the design is. There are a lot of things that I think Zope3 can learn from RoR. But I think, long term, Zope3 has more to offer.

I would like zcml to do less. I would like more view-oriented helpers (maybe integrating python-webhelpers into the framework, along with Mochikit?). I'd like to see an ORM (sqlalchemy, if I had a choice) tied in as a utility that allows certain objects to register themselves with the framework, and (using zalchemy?) ties the transactions together. I would like to see a query language (xpath or oql) implemented.

But y'know what? All of that can come. Maybe I can spend some of my time on it. But I don't know if I think they're powerful enough complaints against the framework to make me want to use anything else. I'll keep at it with RoR, one because I have to, and two because it makes my thought process more diverse/adept/adaptable/etc. But I want to be back on Zope3.

Friday, May 19, 2006

design patterns

quick post... reading the head first design patterns book by o'reilly press. I'm only on page 17 and I'm already impressed. I had, at the outset, almost decided against liking this book. The "visual kitsch" is not an approach I would normally care for, and find, on the surface, mildly irritating. but having read their rationale for designing it this way, I can't really argue with them. also, I think it's actually proving effective, which I wouldn't have expected.

The first chapter's Duck example is excellent. The separation of unique and potentially variable behavior into a class in it's own right is a brilliant idea. I've actually done that in the past, on my own, but never really considered the what and why of it. The explanation and description of the pitfalls is excellent. That they lead you through the other potential ways of solving the problem, and show why they don't work, help crystialize the reasoning behind the desing solution.

I'm impressed.

I'll probably update this post as I go.

about.com and python

It's recently been said in the python community, most notably by Guido himself, that Python needs an evangelist. I agree with that. I feel the same way about Zope3 (despite some of my recent seeming frustrations). What keeps python from having the same buzz that Java generated back in the day? What keeps Zope3 from having the same buzz that RoR gets? I think, really, it's just evangelism. I think Zope3 has too steep a learning curve (speaking as someone who is really pretty comfortable with Zope3 all things considered), but that can be remedied.

So, evangelists. I grew up in a frighteningly charismatic christian church, so i've seen my share of real, honest-to-god evangelists. Usually they have shocking hair, boistrous voices, and a disturbing glaze to their eyes that indicates they are not entirely "there." So I don't want to be one of those. Nor do I want to be "The Python Evangelist", if for no other reason than it would be amazingly presumptuous. I'm just not the most qualified person for the job.

But, I would *love* to be "one of the python evangelists" (notice the difference in case sensitivity). I'd like to be one of the Zope3 evangelists too, but there are some things I need to figure out about where I stand on the platform for that to be realistic. It does a lot of things that I love, and it's architecture is better than any i've seen (and i've looked). To that end, I was made aware that About.com had listed for a Python guide. Last night I put together a quick "intro to python, with hello world" piece, and sent it off as my writing sample. I think it would be an interesting opportunity.

Assuming the likely case that other, more qualified people will apply for and get the Guide position, I've decided to post my writing sample here, just for kicks. Below is the writeup.


What is Python? At its most concise, Python is a programming language. It is an interactive, interpreted, dynamically typed, object-oriented language, with similarities to Perl, Ruby, and Java. But unless you know those languages already, or have a pretty good technology background, none of that will really mean much to you. So a better question might be "Why Python?"

Why Python? While there are plenty of potent technical reasons to suggest Python, the reasons that matter most to anyone as they start out with a new language usually fall into the following categories: simplicity of syntax, ease of portability, and library support for common tasks. Python's syntax makes it simple and enjoyable to pick up and use the language. This is important not only to keep you coding and enjoying the code you write, but is a serious consideration for maintainability and longevity of a program. Relatedly, the fact that there are implementations of Python for Windows, Linux/Unix, and MacOS make most Python code extremely portable. Finally, and most importantly as your skill with any language grows, the extensive core and third party libraries for Python mean that you are rarely forced to solve a problem "the hard way" (unless you want to, which Python certainly has the capacity to do), with support for everything from web, email, and database applications, to GUI and game programming.

That all sounds pretty good, but what about getting our hands dirty? No one ever learned anything worth knowing until they ended up doing it themselves. So let’s look at a little bit of code. If you don't have Python installed, there are a number of places to get it. Both the Python website and ActiveState have installers and instructions for Python, though if you're on a *nix box, the odds are you already have Python installed by the distribution. If not, there are downloads and instructions available for those platforms as well.

Once you have Python installed, let’s get to the interactive interpreter. In *nix you need only to type "python", and it will bring you to this shell. In Windows, you'll need to go to Start->Programs->Python->Python(command line). This should bring up a window with the Python version and build information at the top, and a line with >>> prompt below it.

Let's establish that we're in a friendly environment. Type "help" at the prompt. It should present you with a nice message on how to use the help command. Try it the first way, with no arguments. Type "help()". The parentheses tell the interpreter (the bit running in the background) that you want to call the command "help" with no arguments. It should present you with a few paragraphs letting you know some basic information about Python, and the help utility. Feel free to come back later, but for now, just type "quit".

Now it’s time to really type some code. Because "Hello World" is the most common first program to write in a new language, we'll only flout the convention a little by changing the phrase a bit. Type in the following:

>>> greeting = "hello from python!"

What did we do here? It seems simple, and, remarkably, it is simple. We merely created a variable named "greeting" and assigned a string to it by using the equal sign. A string is just any piece of text-like data, and "=" is just the assignment operator. It says "whatever is on the left hand side is now set, or assigned, to whatever value is on the right hand side." As you might expect, the value on the left hand side has to be a variable name. A variable is just an identifier that can have things assigned to it. We could just as easily have made "greeting" the variable "hi" or "good_day" or "helloWorld". (Purists will have to forgive me here. Technically, "greeting" is not a variable, but an object of type "string", with a __repr__ value that happens to look like string data, and the assignment operator helped us create the new object. But object theory would be getting a little ahead of ourselves, so we're going to keep saying variable for now).

So now we have a variable, and that variable has a value. While there are a nigh-limitless number of things we can do with a piece of text, let’s just, for now, print it out to the screen. Printing something out to the screen is as simple as using the "print" command. Just like we used the "help" command earlier, we will tell the interpreter that we intend to invoke the print command, but this time with an argument. At the prompt, type the following:

>>> print(greeting)

hello from python!

Notice that "hello from python!" showed up immediately below it? The "print" command told the interactive interpreter that you wanted to have the value assigned to the variable "greeting" echoed out to the screen. By placing parentheses around the variable in question, the command "print" knew that you intended that as the first and only argument.

There are a few things to know about the interactive interpreter to make life easier. The first is that any value typed at the prompt, if it is able to be meaningfully represented as a piece of text, will echo back out to the screen. Try it like this:

>>> greeting

'hello from python!'

Notice, however, a subtle difference between this output, and the output of the print command above. The output of the "print" command does not have quotes around it. It is, in fact, the direct result of "print". The second output is not actually the result of any specific Python command, but is rather a string representation of the data referenced by the variable "greeting". When you typed it directly, the interpreter showed you that it was, in fact, a piece of string data, by enclosing it in quotes. Pretty nifty. The other things to know about the interpreter are fairly straightforward, and are, in effect, just other ways of looking at what we've already seen. Try typing a mathematical expression at the prompt. Something exotic, like:

>>> 1+1

2

The interpreter isn't really doing anything spectacular here, though it is cool. The interactive interpreter knows that 1 is a numeric type. It also knows that when the plus sign "+" is used between two numeric types, what is expected of it is to add those values together in a numeric fashion, and present a result. To see a counter example, try this:

>>> "1" + "1"

'11'

Without getting into the more complicated reasons about why this is happening, by putting quotes around both "1"s, we've told the interpreter that we want these values to be treated like text. The plus sign in this case says to concatenate the values. Also notice that when we perform operations at the interactive prompt, any value that we don't assign back into a variable shows once, and disappears. While we may know that 1+1=2, the interpreter has no record of the value "2" anywhere. if we wanted that value for later use, we would have needed to assign it into some other variable, like so:

>>> sum = 1 + 1

>>> print(sum)

2

That's probably about enough for now. With what you've got you can already do some interesting things. Try out "help()" again, and look over what it says. At the "help>" prompt, type "str" (string, like the greeting we used) and look over some of the things you can do with strings. Try it with "int" too. Try to use other math operators (+, -, *, /) and see what happens. Try them on strings, see what happens when you multiply a letter by a number. I think you'll be pleasantly surprised.