bookmark_borderHorde/Rdo ORM: PSR-4 and BC Breaks

Summary: Horde/Rdo ORM got upgraded for Namespaces. User code conversion is straight-forward. Backward Compatibility is limited.

If you ever wondered, RDO stands for Rampage Data Objects. This has been on my list for quite long, but it took some time to get it right. The horde/rdo library is horde’s Object Relational Mapping (ORM) solution. It allows you to store objects into sql databases or retrieve them without writing SQL. Originally, it has been written by Chuck Hagenbuch way back in the PHP 4 days and I have been a heavy user for years. If you know Laravel’s Eloquent, Doctrine, Hibernate, ActiveRecord, nHydrate or Dapper, this one is Chuck’s “as light as possible” implementation of the concept. Colleagues from B1 Systems have been users and contributors over the years. However, it has long been time to rethink Rdo in the light of newer capabilities of PHP 7.4 or even PHP 8. This time is now.

But you should not fear upgrading. First, the library still keeps the unnamespaced PSR-0 code, at least for the time being. Second, there is a straight-forward upgrade path for existing users.

Horde_Rdo_Base -> Horde\Rdo\Base
Horde_Rdo_Mapper -> Horde\Rdo\BaseMapper
Horde_Rdo_Factory -> Horde\Rdo\Factory
Horde_Rdo_List -> Horde\Rdo\DefaultList
Horde_Rdo_Iterator -> Horde\Rdo\DefaultIterator
Horde_Rdo_Query -> Horde\Rdo\DefaultQuery
Horde_Rdo_Exception -> Horde\Rdo\RdoException
Horde_Rdo:: -> Horde\Rdo\Constants::

It’s about as easy as it looks. Converting an application took me a few minutes. You might have noticed the names do not exactly match. Some names were not practical to simply turn into namespaced classes. In other cases, I turned class names into interface names. I found myself implementing the same enhancements over and over in multiple projects and I found myself wishing there was an easy way to do some others.

Less Boilerplate Mappers and Entities

Rdo is much more fun than some other ORMs as it comes with very little configuration. The library autodetects properties from the table columns. Datetime fields are automatically cast to Horde_Date objects. By default, Rdo tries for a convention over configuration approach for mapping table names, mappers, entities etc. Unfortunately, for most of my projects, that default does not fit too well to the class structure and file layout I choose. But still, implementing a new pair of mappers and entities takes two files and only two or three settings I ever need to think about.

Most often, subclassing the base mapper and the base entity is the right thing to do. But sometimes, you do not really care. If all you ever do to your entity is call ->toArray() and serialize it to json, you would be served very well by a generic entity instead. This is something on my list. I would even go one step further: If all you are changing to a mapper is subclassing and telling it the name of its database table and entity class, why subclass at all? Yes, I would want to turn the optional Factory class into something smarter. It will give you your mapper, be it an instance of the generic mapper with the right table name or something very customized.

Custom List Objects

Rdo queries always return a Horde\Rdo\List. This is a good default implementation and it makes common tasks easy. However, there are situations where you want your list of entities to be specific to an entity type or maybe a subclass of some base list class completely external to Rdo. Maybe you want to manipulate a list or merge results from two different queries.

Custom Entities
Sometimes the default entity implementation does not serve you well. There’s a range of things you would want.
– You want to inherit from a base class to attach behavior to your data. So you attach an interface and a trait to that foreign class to make it Rdo aware.
– You want to implement your own behaviour altogether
– You want Rdo to implement a proper repository with strongly encapsulated, less chatty domain objects. Rdo should provide a mechanism to produce those objects for you rather than having you cast or wrap Rdo\Base objects into your actual models. But it should not force you to think about such concepts before you really need them.

NoSql backends
Remember the name Rampage Data Objects? Rdo is mostly about mapping data to objects. It’s not about autogenerating the smartest SQL for the most obscure use cases. But once you have your prototype version ready, your first feedback comes in, you think about new features – and suddenly you want to support a new backend for some of your domain objects. Be it a NoSql database, a key/value store, a limited scope within a directory like LDAP or even a REST service. In a traditional horde application, you would wrap Rdo into a Driver called Driver/Rdo or Driver/Sql and implement a different backend. But what if you do not want to flip all your data to the new backend? Only the shopping list should go to the nosql backend but not the customers or the product inventory? You end up implementing individual drivers with individual backends. But you used Rdo’s relations feature … things get messy.

To achieve these capabilities, I want to make the Mapper less dominant. The formerly optional Factory gets promoted to take care of managing the right mappers, entities, backends, list types. This is what these interfaces are for. Mappers should mostly take care of mapping between an object class and a plain data format. Currently the mapper and query do too much, tightly coupled with the single mandatory list type. This will change.

Rampage Data Objects provides out of the box defaults for easy and common use cases. It gets you started really quick. We will add the capabilities needed when your application is maturing and your use cases get more demanding. This will be fun.

Backward Compatibility Breaks
The Horde\Rdo\Base* classes and their return types will be your best bet for backward compatibility. If you don’t try to use entities and mappers for side effects, you will be very safe. Factory’s constructor will change very soon. Factory should best be created from a Dependency Injector. Mappers should be created from Factory.

You should not rely on mappers exposing adapter or factory for creating other objects. Also, trying to manipulate sql session modes or transactions through Rdo’s adapter is not a good idea.

bookmark_borderHorde_Rdo Many to Many relations and Horde DB Migrator

Many to Many relations btween to object types or table rows are usually saved to a database using a third table.

For example, if every server can have multiple services and each service can run on multiple computers, we need a third table to store the relations:

server table:
server_id | server_name
        1 | hoellenhund.internal.company.com
        2 | forellenfisch.internal.company.com
service table:
service_id | service_name
         1 | tomcat
         2 | dovecot
relation table:
service_id | server_id
         1 | 1
         2 | 2
         2 | 1

Horde’s ORM Layer Horde_Rdo supports creating, checking and changing such relations but it’s not very prominently documented.

Let’s look at an example.

First, we need to create the database schema. Note that the relations table has no autoincrement key, only the two columns used for lookup


/usr/share/php5/PEAR/www/horde/hvview/migration # cat 1_hvview_base_tables.php
<?php
/**
* Create Hvview base tables.
*
* Copyright 2015-2015 B1 Systems GmbH (http://www.b1-systems.de/)
*
* See the enclosed file COPYING for license information (GPL). If you
* did not receive this file, see http://www.horde.org/licenses/gpl.
*
* @author Ralf Lang
* @package Hvview
*/
class HvviewBaseTables extends Horde_Db_Migration_Base
{
/**
* Upgrade
*/
public function up()
{

$t = $this->createTable('hvview_technical_landscapes', array('autoincrementKey' => 'landscape_id'));
$t->column('landscape_name', 'string', array('limit' => 255, 'null' => false));
$t->column('period_id', 'integer', array('limit' => 11, 'null' => false));
$t->end();

$t = $this->createTable('hvview_resource_pools', array('autoincrementKey' => 'resource_pool_id'));
$t->column('pool_name', 'string', array('limit' => 255, 'null' => false));
$t->column('landscape_id', 'integer', array('limit' => 11, 'null' => false));
$t->column('period_id', 'integer', array('limit' => 11, 'null' => false));
$t->end();

$t = $this->createTable('hvview_hardware_pools', array('autoincrementKey' => 'hardware_pool_id'));
$t->column('pool_name', 'string', array('limit' => 255, 'null' => false));
$t->column('landscape_id', 'integer', array('limit' => 11, 'null' => false)); /* possibly redundant, but may speed up things */
$t->column('period_id', 'integer', array('limit' => 11, 'null' => false));
$t->end();

/*Relations table*/
$t = $this->createTable('hvview_rp_hwps', array('autoincrementKey' => false));
$t->column('resource_pool_id', 'integer', array('limit' => 11, 'null' => false));
$t->column('hardware_pool_id', 'integer', array('limit' => 11, 'null' => false));
$t->end();

$t = $this->createTable('hvview_periods', array('autoincrementKey' => 'period_id'));
$t->column('period_ts', 'integer', array('limit' => 11, 'null' => false));
$t->end();

/* We collapse hypervisor and blade server objects into one for now - let`s see if this scales well */
$t = $this->createTable('hvview_servers', array('autoincrementKey' => 'server_id'));
$t->column('period_id', 'integer', array('limit' => 11, 'null' => false));
$t->column('hardware_pool_id', 'integer', array('limit' => 11, 'null' => false));
$t->column('hostname', 'string', array('limit' => 100, 'null' => false));
$t->column('state', 'string', array('limit' => 20, 'null' => true));
$t->column('os_release', 'string', array('limit' => 20, 'null' => true));
$t->column('comment', 'string', array('limit' => 255, 'null' => true));
$t->column('hv_free_vcpu', 'integer', array('limit' => 11, 'null' => true));
$t->column('hv_free_memory', 'integer', array('limit' => 11, 'null' => true));
$t->column('hv_free_disk', 'integer', array('limit' => 11, 'null' => true));
$t->column('hv_total_vcpu', 'integer', array('limit' => 11, 'null' => true));
$t->column('hv_total_memory', 'integer', array('limit' => 11, 'null' => true));
$t->column('hv_excluded', 'integer', array('limit' => 1, 'null' => true));
$t->column('hv_vm_count', 'integer', array('limit' => 3, 'null' => true));
$t->end();

// Indices not before we have an idea which of them we need most
// $this->addIndex('hvview_items', array('item_owner'));

}

/**
* Downgrade
*/
public function down()
{
$this->dropTable('hvview_technical_landscapes');
$this->dropTable('hvview_resource_pools');
$this->dropTable('hvview_hardware_pools');
$this->dropTable('hvview_periods');
$this->dropTable('hvview_servers');
$this->dropTable('hvview_rp_hwps');
}
}

The relations are defined in a Horde_Rdo_Mapper class which also knows how to spawn objects from the rows.

The Objects

/usr/share/php5/PEAR/www/horde/hvview/lib/Entity # cat ResourcePool.php 
<?php

class Hvview_Entity_ResourcePool extends Horde_Rdo_Base {
}

/usr/share/php5/PEAR/www/horde/hvview/lib/Entity # cat HardwarePool.php 

The Mappers:

/usr/share/php5/PEAR/www/horde/hvview/lib/Entity # cat ResourcePoolMapper.php 
<?php
class Hvview_Entity_ResourcePoolMapper extends Horde_Rdo_Mapper
{
    /**
     * Inflector doesn't support Horde-style tables yet
     */
    protected $_classname = 'Hvview_Entity_ResourcePool';
    protected $_table = 'hvview_resource_pools';
    protected $_lazyRelationships = array(
             'hwps' => array('type' => Horde_Rdo::MANY_TO_MANY,
                          'through' => 'hvview_rp_hwps',
                          'mapper' => 'Hvview_Entity_HardwarePoolMapper')
    
            );

}

/usr/share/php5/PEAR/www/horde/hvview/lib/Entity # cat HardwarePoolMapper.php 
<?php
class Hvview_Entity_HardwarePoolMapper extends Horde_Rdo_Mapper
{
    /**
     * Inflector doesn't support Horde-style tables yet
     */
    protected $_classname = 'Hvview_Entity_HardwarePool';
    protected $_table = 'hvview_hardware_pools';
    protected $_lazyRelationships = array(
             'rps' => array('type' => Horde_Rdo::MANY_TO_MANY,
                          'through' => 'hvview_rp_hwps',
                          'mapper' => 'Hvview_Entity_ResourcePoolMapper')
    
            );

}

The relation is defined in both direction and only loaded on-demand ("lazy") as opposed to upfront when the item is created from the database rows.
Now let's fetch two items and link them:

You can do this through the mapper or through one of the two partners

Adding a relation to an object using the object

// $rm is a ResourcePoolMapper instance
// $hm is a HardwarePoolMapper instance
$rp = $rm->findOne(); // In reality, you would not pick a random item but add some criteria
$hwp = $hm->findOne();
$rp->addRelation('hwps', $hwp);

Adding a relation to an object using the mapper

// $rm is a ResourcePoolMapper instance
// $hm is a HardwarePoolMapper instance
$rp = $rm->findOne(); // In reality, you would not pick a random item but add some criteria
$hwp = $hm->findOne();
$rm->addRelation('hwps', $rp, $hwp);

bookmark_borderHorde 5 Preview: Sesha Inventory App 1.0 and updated Rdo library

Dear folks, I am very pleased to announce:
The Sesha Inventory application is ready for Horde 5 and it is in good shape. Sesha is a simple inventory keeping application which originally developed by Bo Daley and Andrew Coleman on Horde 3. The product was never officially released but it went into production at several sites. Sesha release cycle can now start together with the Horde 5 Alpha release cycle.

Sesha inventory can be configured to hold any number of stock categories with any number and type of attributes.
Like the original version, Sesha for Horde 5 can provide its stock categories as ticket queues for the horde ticketing application whups.
There are a lot of plans and ideas for upcoming versions but for this time the focus was on finishing a releasable product.There are no surprises for existing users of Horde 3 based sesha. Most work happened invisibly under the hood:

  • The Horde_Template library was exchanged by new Horde_View code
  • A migration script for database was added
  • Users can keep their original Horde 3 Sesha tables and data.
  • The sql backend driver was completely reworked into a driver based on the Horde_Rdo ORM library The new Driver Api provides enhanced search capabilities but the current frontend doesn’t make use of it. I do not plan to add any features to the classic view but start working on an Ajax view once the Horde 5 Redesign is completed. This may ship with Sesha 1.1 later on.
  • Object oriented code has replaced complicated hashes in many places

The Horde Rdo library is the new work horse inside Sesha. Rdo means Rampage Data Objects and is a lightweight ORM layer by Horde founder Chuck Hagenbuch. It maps database tables to PHP Objects. This is similar to the ActiveRecord pattern. Each database row can be turned into one Rdo item. For Sesha and another – non-public – software project, some enhancements went into the Rdo library for Horde 5:

  • Rdo now provides a caching factory or root object which speeds up creation of mapper objects
  • Methods for add, removing or checking many-to-many relations have been added
  • A number of edge case bugs have been fixed

I think the Horde 5 release cycle will start with alpha1 releases sometime in May. I know we’re a little late but it’s worth the wait.
That said, I welcome any early testing or updates of the language files. Provided everything works as expected, Sesha will be shipped with Horde 5 for OpenSUSE 12.2