Sunday, June 25, 2006

IRelationshipMapper

I've been thinking about relationships between different kinds of data. This is because I probably need more exciting hobbies. However, I still was, and so I'm going to jot down my thoughts about it.

To my mind, objects should not be cluttered with knowledge (or forced to be responsible for the tracking) of how they're related, and yet, all data, and most objects in a complex system, have some relationship of one-to-one, one-to-many, or many-to-many between them (as well as other more complex relationships). In the past, in order to mantain a relationship between a Zope3 content object and a relational db object, I've used constructs like "obj._db_key_ = some_primary_key".

But what I'm thinking about is a zope3 registry of objects. I've done this already, in a way, for the organization logic in a register/activate/login authentication scheme i developed for an application i was working on. But I think this has application far beyond something as simple as that.

We can create a utility that allows objects to register a relationship to other objects. something like "has_a(IObjectType, as=attribute)", "has_many(ISomeOtherObjType, as=attribute)", "belongs_to(IYetAnother, as=attribute)", etc.

Example, principals who own certain bits of data, but not in an annotated kind of way (assume other objects need access to the data as well). These principals need access to this data programmatically, but without having the content objects limited to certain containers (like home or group folders). In a registry of data, one object can look up it's associated data through the registry, and the relevant data can be given back to the seeking object. Setters and getters that utilize the registry could literally make it as simple as:


shop.customers.append(someNewCustomer)
shop.customers.remove(someOldCustomer)


where shop is an implementation of IShop, which has_many(ICustomers, as="customers"), with setter/getter code like:

relmap = component.getUtility(IRelationshipMapper, context=self)
def getCustomers(self):
return relmap.getRelationship(IShop, relationshipAttribute="customers", context=self)

def setCustomers(self, customerList):
customers = relmap.getRelationship(IShop, relationshipAttribute="customers", context=self)
for customer in customerList:
if customer not in customers:
customers.append(customer)

customers = property(getCustomers, setCustomers)



This is great since shop itself may not actually have a "contained" list of customers, since customers may belong to multiple shops. Customers may even be content objects that are associated with other principals in the system.

What's interesting is that in a registry like this, it could have multiple "item lookup" methods for investigation, so RDB constructed objects as well as ZODB objects could reference one another without having to be aware of what kind of object is, or where it persists. Between UniqId utilities and unique combinations of table/primary key for RDB objects, the Registry would not have to be a terribly complicated piece of software to track where everything is. The only significant complexity would be the relationship classes, that define what kind, and where. And even those wouldn't be terribly difficult. One-to-one would just be an index of object ids related to other object ids. Has_many (one-to-many) would just be an index of object ids with an associated list of other ids.

Catalog indexes for each ZODB object type. We may even delve into autocreating indexes for all Interfaces of registered objects. If object can merely implement (IRelationParticipant) or something similar, as a marker interface, then on notify(ObjectCreatedEvent(obj)), the utility will be a subscriber to that event, and check to see if the created obj implements IRelationParticipant. if so, it should check itself for the registered relationships (marked in the class through has_a, has_many, etc) on that object's uniq id, and set them. then, on every ObjectModifiedEvent, it will (as a subscriber), check to see if the attribute has been modified, and if so, that the new values implement the registered relationship interface (that obj.attr=val has val being IObjectType, if that was the relationship established at the outset). if so, it can be updated with the uniq id of the new set of values.

example:

class IShop(Interface):
"""shop interface"""
has_many(ICustomers, as="customers")

class Shop(object):
"""shop class. without the sawing and/or welding."""
implements(IShop, IRelationParticipant)
customers = HasManyRelationship()

How will the relationship help us? To know what kind of object this is, one to another. So i can say:

customers = relationshipMapper.getRelationship(IShop, relationshipAttribute="customers "context=myShop)


or something similar. or "myobj.children = ['some', 'list', 'of', 'objs']", and have the relmap utility automatically update these (setter/getter/property methods again, back to the registration utility).

Right now, all "contained" objects have __parent__ for persistent ZODB objects to describe the relationship, object to object, in the containment model. But objects have more relationships than simply "contained." It would even be possible to use the registration utility to mark "contained/contains" as a kind of relationship, so that ZODB persisted objects could "contain" RDB objects even (not sure how we'd need to implement IPhysicallyLocatable).

Consider a situation like this: Lets say I have two containers for two kinds of objects. Say I have a container called Shops and another called Customers. Lets say that both shops and customers need a way to be able to log in, and a way to access their data, so we have principal objects associated in an ownership capacity to both Shops and Customers, like ShopPrincipal and CustomerPrincipal, but distinct from these. We don't want Shops and Customers to merely be annotations of the principal, because we want this data able to be looked up and looked at by other logical entities in our application. However, there is a logical connection betweeen Shops and Customers. Shops->have_many->Customers, but Shops is not the __parent__ of it's customers, because when CustomerPrincipal logs in and wants to see her data we don't want to have to either track OR figure out which Shop the Customer data lives in. Especially if a Customer is potentially a Customer of more than one Shop, since Customer->has_many->Shops. the "has_and_belongs_to_many" relationship.

In an RDB, we would just create a join table, with foreign keys of each table's primary key. But in the ZODB, a non relational system, we don't do things that way. IRelationshipMapper could potentially create a software way of working with this kind of relationship model.

Saturday, June 03, 2006

zope3 configuration

so, i feel like my mental abs are sore. the tagline "xml situps" is an apt one.

zcml. the debate rages on in mailing lists, and probably will continue to. I've read arguments on both sides, and I don't really disagree with either side. but this is what i'm thinking about...

i read an email of Stephan Richter's recently where he said that he does all site configuration in python code. I'd considered that before, but didn't know if it was the "right" way to do it. if it is, then I'll probably start doing it that way too. Stephan is probably the right guy to take Zope3 queues from, if you're going to pick someone.

but it makes me wonder if the zmi should even exist. or if maybe it should be included in zope3 packages only if the developer wants it there. like "from zope.app.zmi.admin import controlpanel" or something. or "from zope.app.zmi.error import errorpanel".

which leads, sorta, to my next point. all of the zmi default display pages... they make things fast, certainly. but i think that they're almost immediately a crutch, too. i think they keep the developer from really having to dig into the zope3 page creation machinery, and i think that the page creation machinery (now formlib, which i like a lot) is something that any zope3 developer needs to be fluent in.

i'd like to see the day where "bin/runzope" just starts up a very small shell of functionality, with whatever the developer has included. maybe we could move the zmi to "zope3admin" or something. i don't know that i think the zmi has any real place "being there" in a live app. i don't see any reason that it couldn't be started on it's own port, and hook into the zodb on it's own. i know when i need to do any real debugging of zope3, i do it from zopectl debug anyway, not through the errors page.

so I don't know. there's a lot that the zmi does, certainly. but i think it prevents the "simplicity" that's necessary to get new developers on board for zope3. I know if I hadn't had to learn how to use zcml, zpt, the zope3 framework, AND the zmi all at once, i'd have been in better shape. zcml, zpt, and the framework are all non-domain-specific in their implementation. and it's enough to learn on it's own, even being basically just code and markup.

but then to learn the right order to register new utilities, deregister them, delete them, add them, chain the dependencies...

I am now, just now and I hadn't thought of it before, propsing a "Zope3 Site Builder" set of markup or template code. some way to set up a chain of dependencies, registrations, and necessary content objects. I haven't thought about it that much yet, but I can't imagine it wouldn't be doable. some new pythonic template that could set up and tear down a site in the right order.