bookmark_borderA wicked problem from the past

In the last few evenings there was great despair. Trying to solve one problem would reveal yet another one. As a result, I hesitated to release some of the changes immediately. I don’t want to make people suffer, having to deal with new improved problems of the day whenever they run an update on their horde installations. I’m glad feedback on the mailing list has improved quite a lot, sometimes coming with patches and suggestions and even people telling me things start to just work. That’s great. But whenever you see the light, there’s a new challenge on the horizon. Or rather something very old lurking in the shadows.

Break it till you make it

So recently we felt confident enough to switch our local installation from a frozen state to the latest version of the wicked wiki and the whups bug tracker. We also updated the PHP container, the database version and the likes. This turned into a great opportunity to test the rollback plan. 🙄 Issues were cascading.

Bullets you might have dodged

Generally having both the composer autoloader and the horde autoloader work together creates some friction. This is complicated when the exact order in which they are initialized is not the same in all scenarios. As reported previously, Horde’s apps are not strictly conforming the PSR-0 autoloading standard while the libraries mostly do. Newer releases of the apps autoload using a class map instead of PSR-0 logic. But that class map was not restricted enough in the first iterations. Thus you might have the canonical locations of classes in the autoloader, but also the version in the /web/ dir or, in case of custom hook classes, the version from the /var/config/ directory. Torben Dannhauer suggested to restrict the rules and we will do that with the next iteration. The first attempt to fix this unfortunately broke overriding the mime.local.php config file. An update is in the git tree and will be released as a version later this week. But I’m glad we ran into this as it revealed another dark secret

We part, each to wander the abyss alone

Turned out the wicked wiki software carried a little library Text_Wiki in its belly. Hailing from PHP 4 days, it’s archaic in its structure and its way of implementing features. Parts of the test suite predate phpunit and the way it’s packaged is clearly not from our times. This library also exists as a separate package horde/text_wiki. Which also exists as pear/text_wiki both in the classic PEAR archive and the modern github repository. While the base package does not exist in packagist, the extension driver for mediawiki format does. Odd enough. It’s really a shame the software from the old PEAR ecosystem is slowly fading away because of its ossification. They all share ancestry and they were all largely written and maintained by the same set of people but they are all different in subtle ways.

Text_Wiki works great when it works. But it’s a real treasure trove of incompatibilities with modern PHP versions. Over the years, unicode support has evolved along with the strictness of PHP’s core. While I am very much willing to contribute back any changes to the official pear version, I have my doubts if they will be accepted or handled at all.

Rising again like the phoenix

I really, really did not want to deal with any of this. Creating a fourth version out of three splinters feels like madness. But what can you do?
The result is an upcoming version of horde/text_wiki and horde/wicked which is a radical break. The new text/wiki has no PSR-0 lib/ content. It’s all PSR-4, it’s namespaced, every single class name and file name has changed. A new test suite has been started but only a tiny fraction of unit tests have been ported so far. The different extension drivers for mediawiki and tiki dialects have been incorporated into the base package as I did not want to maintain a hand full of packages.

The whole thing couples as little to horde as possible. It has its own exceptions based on the PHP builtin exception tree, so no dependency on horde/exception. I even stripped the dependency on Pear_Exception as there is no value in keeping that nowadays. Anything which ties this revamped library into the horde ecosystem now lives in horde/wicked. Extra markup rules which render horde portal blocks into a wiki page. Adapters querying the database if a linked page already exists or must be created. Dynamic links created from the registry and application states. None of this glue belongs into the text_wiki library.

Many incompatibilities with modern PHP have been solved. Often this was a matter of mixing and matching bits from the three different splinter versions out in the wild. Some of it is just a chore to do. Of course, the pear XML files are gone and the composer file is fully compliant with packagist and most recent composer. At least this part has been contributed back already.

Dawn, finally

It will be another few evenings until the new horde/wicked and the new horde/text_wiki are ready for a tagged release, probably along with some of the changes to other libraries I explained above. There’s probably something that will break and need fixing. But that shouldn’t block progress for too long.

bookmark_borderPEAR down – Taking Horde to Composer

Since Horde 4, the Horde ecosystem heavily relied on the PEAR infrastructure. Sadly, this infrastructure is in bad health. It’s time to add alternatives.

Everybody has noticed the recent PEAR break-in.

A security breach has been found on the http://pear.php.net webserver, with a tainted go-pear.phar discovered. The PEAR website itself has been disabled until a known clean site can be rebuilt. A more detailed announcement will be on the PEAR Blog once it’s back online. If you have downloaded this go-pear.phar in the past six months, you should get a new copy of the same release version from GitHub (pear/pearweb_phars) and compare file hashes. If different, you may have the infected file.

While I am writing these lines, pear.php.net is down. Retrieval links for individual pear packages are down. Installation of pear packages is still possible from private mirrors or linux software distribution packages (openSUSE, Debian, Ubuntu). Separate pear servers like pear.horde.org are not directly affected. However, a lot of pear software relies on one or many libraries from pear.php.net – it’s a tough situation. A lot of software projects have moved on to composer, an alternative solution to dependency distribution. However, some composer projects have dependency on PEAR channels.

I am currently submitting some changes to Horde upstream to make Horde libs (both released and from git) more usable from composer projects.
Short-term goal is making use of some highlight libraries easier in other contexts. For example, Horde_ActiveSync and Horde_Mail, Horde_Smtp, Horde_Imap_Client are really shiny. I use Horde_Date so much I even introduced it in some non-horde software – even though most functionality is also somewhere in php native classes.

The ultimate goal however is to enable horde groupware installations out of composer. This requires more work to be done. There are several issues.

  • The db migration tool checks for some pear path settings during runtime https://github.com/horde/Core/pull/2 Most likely there are other code paths which need to be addressed.
  • Horde Libraries should not be web readable but horde apps should be in a web accessible structure. Traditionally, they are installed below the base application (“horde dir”) but they can also be installed to separate dirs.
  • Some libraries like Horde_Core contain files like javascript packages which need to be moved or linked to a location inside another package. Traditionally, this is handled either by the “git-tools” tool linking the code directory to a separate web directory or by pear placing various parts of the package to different root paths. Composer doesn’t have that out of the box.

Horde already has been generating composer manifest files for quite a while. Unfortunately, they were thin wrappers around the existing pear channel. The original generator even took all package information from the pear manifest file (package.xml) and converted it. Which means, it relied on a working pear installation. I wrote an alternative implementation which directly converts from .horde.yml to composer.json – Calling the packages by their composer-native names. As horde packages have not been released on packagist yet, the composer manifest also includes repository links to the relevant git repository. This should later be disabled for releases and only turned on in master/head scenarios. Releases should be pulled from packagist authority, which is much faster and less reliant on existing repository layouts. https://github.com/horde/components/pull/3

To address the open points, composer needs to be amended. I currently generate the manifests using package types “horde-library” and “horde-application” – I also added a package type “horde-theme” for which no precedent exists yet. Composer doesn’t understand these types unless one adds an installer plugin https://github.com/maintaina-com/installers. Once completed and accepted, this should be upstreamed into composer/installers. The plugin currently handles installing apps to appropriate places rather than /vendor/ – however, I think we should avoid having a super-special case “horde-base” and default to installing apps directly below the project dir. Horde base should also live on the same hierarchy. This needs some additional tools and autoconfiguration to make it convenient. Still much way to go.

That said, I don’t think pear support should be dropped anytime soon. It’s the most sensible way for distribution packaging php software. As long as we can bear the cost involved in keeping it up, we should try.

bookmark_borderInstalling Horde 4 pear packages to a custom pear location (SUSE)

When installing horde to a custom pear location, you need to run the pear of your custom location, not the system pear with the custom location’s config.

So the steps would be:

 
1  mkdir /srv/horde 
2  pear config-create /srv/horde/ /srv/horde/pear.conf 
3  pear -c /srv/horde/pear.conf install PEAR 

as the install docs say but then:

4 /srv/horde/pear/pear -c /srv/horde/pear.conf channel-discover pear.horde.org 
5 /srv/horde/pear/pear -c /srv/horde/pear.conf run-scripts horde/Horde_Role 
6 /srv/horde/pear/pear -c /srv/horde/pear.conf install --alldeps horde/groupware 

Otherwise running the Horde_Role script will fail saying

config-set (horde_dir, /srv/horde/, user) failed, channel pear.php.net

This was experienced on SLES11SP1, SLES11SP2 and openSUSE Factory.

I did not test this for any debian based products yet.

bookmark_borderTip of the day: Changing global php pear settings as root

Don’t forget: when you change pear settings as root, usually you want to set values in the system pear config, not in root’s personal config. The crucial third parameter is optional and defaults to ‘user’. We want ‘system’ instead

pear config-set test_dir /usr/share/php/tests systems

verify:
pear config-show

bookmark_borderHowto: Packaging 3rd party pear channel software with %php_pear_gen_filelist macro

The %php_pear_gen_filelist macro, maintained by Christian Wittmer, is really handy for packaging php pear software packages. It generates rpmlint-happy filelists and if you manage to get the dependencies right, packaging pear stuff for rpm is really a no-brainer. But the standard recipe for using this macro has one drawback: It’s ignorant of installed 3rd party roles and channels. 3rd party pear packages which depend on their channel being registered normally fail.

The workaround is easy: Copy the channel file to the build location.

Example:

#
# spec file for package php5-pear-Horde_Auth (Version 1.0.3)
#
# Copyright (c) 2011 Ralf Lang.
#
# All modifications and additions to the file contributed by third parties
# remain the property of their copyright owners, unless otherwise agreed
# upon. The license for this file, and modifications and additions to the
# file, is the same license as for the pristine package itself (unless the
# license for the pristine package is not an Open Source License, in which
# case the license is the MIT License). An “Open Source License” is a
# license that conforms to the Open Source Definition (Version 1.9)
# published by the Open Source Initiative
# Please submit bugfixes or comments via http://bugs.opensuse.org/

# norootforbuild

Name: php5-pear-Horde_Auth
%define pear_name Horde_Auth
%define pear_sname horde_auth
Summary: PEAR: Horde Authentication API
Version: 1.0.3
Release: 1
License: LGPL
Group: Development/Libraries/PHP
Source0: http://pear.horde.org/get/Horde_Auth-%.tgz
BuildRoot: %/%-%-root-%(% -n)
URL: http://pear.horde.org/package/Horde_Auth
BuildRequires: php5-pear >= 1.4.7
Requires: php5-pear-Horde_Exception < 2.0.0, php5-pear-Horde_Util < 2.0.0, php5-pear >= 1.7.0
Conflicts: php5-pear-Horde_Exception = 2.0.0, php5-pear-Horde_Util = 2.0.0
BuildRequires: php5-pear-channel-horde
Requires: php5-pear-channel-horde
BuildArch: noarch
BuildRequires: php-macros

# Fix for renaming (package convention)
Provides: php5-pear-% = %
Provides: php-pear-% = %
Provides: pear-% = %
Obsoletes: php5-pear-% < %
Obsoletes: php-pear-% < %
Obsoletes: pear-% < %

%description
The Horde_Auth package provides a common interface into the various
backends for the Horde authentication system.

%prep
%setup -c

%build
%install
% package*.xml %-%
cd %-%
PHP_PEAR_PHP_BIN=”$(which php) -d memory_limit=50m”
% %%/.channels/
% %/.channels/pear.horde.org.reg \
%%/.channels/

% -v \
-d doc_dir=/doc \
-d bin_dir=% \
-d data_dir=%/data \
-d test_dir=%/tests \
install –offline –nodeps -R “%” package.xml

% -D -m 0644 package.xml %%/%.xml

% -rf %/{doc,tmp}
% -rf %%/.{filemap,lock,registry,channels,depdb,depdblock}

cd ..

%php_pear_gen_filelist

%clean
rm -rf %

%post
if [ “$1” = “1” ]; then
% install –nodeps –soft –force –register-only %/%.xml
fi
if [ “$1” = “2” ]; then
% upgrade –offline –register-only %/%.xml
fi

%postun
if [ “$1” = “0” ]; then
% uninstall –nodeps –ignore-errors –register-only pear.horde.org/%
fi

%files -f %.files
%defattr(-,root,root)

Two parts are marked black: First you have to include the channel package with “BuildRequires:”. Second marked part copies the channel file from the installed location to the buildroot location.
Feel free to reuse or criticise this solution.

Jean claude van damme bodybuilding workout dianabol of dbol training met de beste bodybuilding-supplementen – fitness fectory.

bookmark_borderHorde 4 Alpha 1 released (pear)

Yesterday the horde project released alpha versions of Horde Framework 4 and the Groupware apps (Notes, Calendar, Email, Filter,Address Book, Tasks)

I did a test drive and they basically work. IMP has been improved a lot and now integrates the mobile and ajax interface versions which came as separate apps in Horde 3. DIMP (Ajax version) now plays more nicely together with classic non-Ajax horde applications.

I will begin distribution packaging for SUSE Linux around the official release on April 05, 2011.

See also: