Posts Tagged ‘migration’

7. Dezember 2015

by

In: horde

Kommentare deaktiviert für Horde_Rdo Many to Many relations and Horde DB Migrator

Many to Many relations btween to object types or table rows are usually saved to a database using a third table.

For example, if every server can have multiple services and each service can run on multiple computers, we need a third table to store the relations:

server table:
server_id | server_name
        1 | hoellenhund.internal.company.com
        2 | forellenfisch.internal.company.com
service table:
service_id | service_name
         1 | tomcat
         2 | dovecot
relation table:
service_id | server_id
         1 | 1
         2 | 2
         2 | 1

Horde’s ORM Layer Horde_Rdo supports creating, checking and changing such relations but it’s not very prominently documented.

Let’s look at an example.

First, we need to create the database schema. Note that the relations table has no autoincrement key, only the two columns used for lookup


/usr/share/php5/PEAR/www/horde/hvview/migration # cat 1_hvview_base_tables.php
/**
* Create Hvview base tables.
*
* Copyright 2015-2015 B1 Systems GmbH (http://www.b1-systems.de/)
*
* See the enclosed file COPYING for license information (GPL). If you
* did not receive this file, see http://www.horde.org/licenses/gpl.
*
* @author Ralf Lang
* @package Hvview
*/
class HvviewBaseTables extends Horde_Db_Migration_Base
{
/**
* Upgrade
*/
public function up()
{

$t = $this->createTable('hvview_technical_landscapes', array('autoincrementKey' => 'landscape_id'));
$t->column('landscape_name', 'string', array('limit' => 255, 'null' => false));
$t->column('period_id', 'integer', array('limit' => 11, 'null' => false));
$t->end();

$t = $this->createTable('hvview_resource_pools', array('autoincrementKey' => 'resource_pool_id'));
$t->column('pool_name', 'string', array('limit' => 255, 'null' => false));
$t->column('landscape_id', 'integer', array('limit' => 11, 'null' => false));
$t->column('period_id', 'integer', array('limit' => 11, 'null' => false));
$t->end();

$t = $this->createTable('hvview_hardware_pools', array('autoincrementKey' => 'hardware_pool_id'));
$t->column('pool_name', 'string', array('limit' => 255, 'null' => false));
$t->column('landscape_id', 'integer', array('limit' => 11, 'null' => false)); /* possibly redundant, but may speed up things */
$t->column('period_id', 'integer', array('limit' => 11, 'null' => false));
$t->end();

/*Relations table*/
$t = $this->createTable('hvview_rp_hwps', array('autoincrementKey' => false));
$t->column('resource_pool_id', 'integer', array('limit' => 11, 'null' => false));
$t->column('hardware_pool_id', 'integer', array('limit' => 11, 'null' => false));
$t->end();

$t = $this->createTable('hvview_periods', array('autoincrementKey' => 'period_id'));
$t->column('period_ts', 'integer', array('limit' => 11, 'null' => false));
$t->end();

/* We collapse hypervisor and blade server objects into one for now - let`s see if this scales well */
$t = $this->createTable('hvview_servers', array('autoincrementKey' => 'server_id'));
$t->column('period_id', 'integer', array('limit' => 11, 'null' => false));
$t->column('hardware_pool_id', 'integer', array('limit' => 11, 'null' => false));
$t->column('hostname', 'string', array('limit' => 100, 'null' => false));
$t->column('state', 'string', array('limit' => 20, 'null' => true));
$t->column('os_release', 'string', array('limit' => 20, 'null' => true));
$t->column('comment', 'string', array('limit' => 255, 'null' => true));
$t->column('hv_free_vcpu', 'integer', array('limit' => 11, 'null' => true));
$t->column('hv_free_memory', 'integer', array('limit' => 11, 'null' => true));
$t->column('hv_free_disk', 'integer', array('limit' => 11, 'null' => true));
$t->column('hv_total_vcpu', 'integer', array('limit' => 11, 'null' => true));
$t->column('hv_total_memory', 'integer', array('limit' => 11, 'null' => true));
$t->column('hv_excluded', 'integer', array('limit' => 1, 'null' => true));
$t->column('hv_vm_count', 'integer', array('limit' => 3, 'null' => true));
$t->end();

// Indices not before we have an idea which of them we need most
// $this->addIndex('hvview_items', array('item_owner'));

}

/**
* Downgrade
*/
public function down()
{
$this->dropTable('hvview_technical_landscapes');
$this->dropTable('hvview_resource_pools');
$this->dropTable('hvview_hardware_pools');
$this->dropTable('hvview_periods');
$this->dropTable('hvview_servers');
$this->dropTable('hvview_rp_hwps');
}
}

The relations are defined in a Horde_Rdo_Mapper class which also knows how to spawn objects from the rows.

The Objects

/usr/share/php5/PEAR/www/horde/hvview/lib/Entity # cat ResourcePool.php 

The Mappers:

/usr/share/php5/PEAR/www/horde/hvview/lib/Entity # cat ResourcePoolMapper.php 
 array('type' => Horde_Rdo::MANY_TO_MANY,
                          'through' => 'hvview_rp_hwps',
                          'mapper' => 'Hvview_Entity_HardwarePoolMapper')
    
            );

}

/usr/share/php5/PEAR/www/horde/hvview/lib/Entity # cat HardwarePoolMapper.php 
 array('type' => Horde_Rdo::MANY_TO_MANY,
                          'through' => 'hvview_rp_hwps',
                          'mapper' => 'Hvview_Entity_ResourcePoolMapper')
    
            );

}

The relation is defined in both direction and only loaded on-demand ("lazy") as opposed to upfront when the item is created from the database rows.
Now let's fetch two items and link them:

You can do this through the mapper or through one of the two partners

Adding a relation to an object using the object

// $rm is a ResourcePoolMapper instance
// $hm is a HardwarePoolMapper instance
$rp = $rm->findOne(); // In reality, you would not pick a random item but add some criteria
$hwp = $hm->findOne();
$rp->addRelation('hwps', $hwp);

Adding a relation to an object using the mapper

// $rm is a ResourcePoolMapper instance
// $hm is a HardwarePoolMapper instance
$rp = $rm->findOne(); // In reality, you would not pick a random item but add some criteria
$hwp = $hm->findOne();
$rm->addRelation('hwps', $rp, $hwp);

1. Juni 2011

by

In: Allgemein, horde, Tech

Kommentare deaktiviert für Migrating Horde 3 to Horde 4 – Top 6 ways to mess up

There have been some migrations of Horde 3 to Horde 4 recently – not all went smooth from the start.

Some top issues of messing things up and how to avoid it:

  1. initial application dimpIn Horde 3 dimp was a separate application which provided an ajax interface to imp. It has since been merged into the imp application. If your Horde installation had dimp before migration to Horde 4, this might create runtime issues for your users when
    • when you locked the initial application to dimp
    • when your users decided that their initial application should be dimp

    To get around this you should

    • make sure you didn’t blindly copy your locked settings from horde 3 to horde 4
    • run a mysql update statement on the horde_prefs table to update column pref_value to „imp“ if it was „dimp“ before (Consider hiring a professional admin for the migration if you don’t know how that looks like)
  2. Making changes in backends.php or prefs.php
    In Horde 3 admins used to edit prefs.php or backends.php/servers.php to change values. Horde 4 ships backends.php and prefs.php as default values. Admins are supposed to copy these to backends.local.php and prefs.local.php and make their changes there. Changes to the original files will be overridden with the next rpm or pear update of the horde apps.
  3. Not unchecking utc time in kronolith
    The Horde 3 Calendaring app defaulted to store calendar events in local user time. The Horde 4 default is UTC timestamps. If you migrate from horde 3 you either have to uncheck that setting or run a migration script on the data.
    Warning: You might end up with an unrecoverable state if you add new data in UTC mode to a calendar backend which has not been converted to UTC timestamps
  4. Not converting turba and kronolith databases to utf8
    In Horde 3 installations, the calendar app kronolith and the addressbook turba often had their database tables encoded in latin1. The system wide default in Horde 4 is utf8. Not adapting this setting to the tables or the tables to this setting results in corrupted display of international characters and symbols.
    Warning: You might end up with an unrecoverable state if you add new data to addressbooks or calendars where backend encoding does not match the set horde encoding
  5. Relying on menu.php’s javascript onclick handler or target attribute
    In the ajax views of kronolith and imp there is currently no support for the target and onclick handler attributes. I do not know of any plans to re-add this support. If you want to link external sites into the iframe, consider creating a custom portal block. There was a recent blog post on creating this kind of blocks on The Upstairs Room
  6. Using the ldap prefs backend
    The ldap backend for preferences is currently not yet ported to horde 4. If you want to migrate, you first have to extract your prefs from ldap and then convert them to sql. If you need ldap prefs, consider hiring a consultant or sponsoring the development of this feature.