PHP – ralf-lang.de

Horde’s new Two-Factor API

Posted on June 18, 2025 by Ralf

New Horde 6 feature: The horde/horde base app’s next release supports two factor logins.
Dmitry Petrov is working to release a new One-Time Password module which integrates with this new API.

Seemless integration for One Time Passwords.

Several years ago I did some downstream development for a customer. They wanted to use One Time Passwords (OTP) in their custom horde application as a way to offer Two-Factor Authentication (2FA). It worked well for the specific use case but it required patching the base Horde system or substantial reconfiguration, basically delegating authentication to this app. Unfortunately, this had several downsides.

Recently I was approached by Dmitry Petrov. He has built his own OTP solution for horde and offered to upstream his module. Time was ripe to finally provide an interface for Two Factor Authentication.

When horde detects the secondfactor/isEnabled API, it adds an additional field to the default login screen.

This also works in smartmobile view. The second factor is not required when connecting to JSON-RPC or CalDAV endpoints. It is only checked for UI logins. Support is currently restricted to the bare minimum. OTP authentication can be opt-in or mandatory – The horde base app does not know this. A future version may force the user into an OTP setup screen after login if no OTP is configured yet.

Sunsetting the Maintaina Horde Fork

Posted on June 4, 2025 by Ralf

A few years back I started a downstream fork of Horde to develop features I needed for foss and customer deployments without upstream dependencies. It went successful, was a great learning opportunity and a good exercise in critiquing our old tool chain and approaches. We had some well-known downstream users and contributors but I’d say it has run its course. It’s time to sunset Maintaina in a controlled way that’s fair towards our user base. As we are nearing a beta and prod release of horde 6 proper mostly built from Maintaina, we want to provide a smooth transition.

Horde 6 (upstream) is focusing on supporting PHP 8.4 without spamming warning&notices while Maintaina was originally targeted at PHP 7.4 through 8.1 – Still supporting anything before 8.2 is not a priority with upstream anymore. I will have to discuss with other maintainers of the fork.

Problems to solve:

Archive libraries which haven’t been touched for long
Coordinate upstreaming libraries with recent changes and archive them
Provide a feasible approach to consume only select maintaina packages and mostly upstream packagist
Clarify the future of changes downstream users want to keep but which compete with Horde upstream solutions
Invite maintainers of downstream code to maintain some upstream libraries to prevent stalling their own needs

I’ll keep you posted.

PHP: The case for standalone null parameters

Posted on June 2, 2025June 2, 2025 by Ralf

PHP 8.0 introduced null and false as members of union types but disallowed them as standalone parameter types. PHP 8.2 changed that and allowed null as standalone parameter types. What is this good for? Should they extend this to “never” one day? Why do I call standalone null parameters poor man’s generics?

What’s a null parameter? What’s a nullable parameter?

A null parameter is a parameter of type null which can only ever have one value null. Its core meaning is “not a value” as opposed to “empty string” or “false” or “zero”.

A union type which has other values but can also contain null is called nullable. For example, boolean is not a nullable type. It has the possible values true and false. If we want to allow a third state, we can create a new nullable type bool|null which has the possible values true, false and null.
In modern php, bool is a union type of true, which can only have the value true, and false, which can only have the value false. So bool|null is equivalent to true|false|null. Union types including null can also be written with a preceding question mark: bool|null is equivalent to ?bool

Isn’t this a bit pointless?

A null parameter by itself is not very interesting. After all, we know its only possible value. It is valuable as a type which can be extended. According to Liskov substitution principle parameters of subtypes should be contravariant to parent types. If the parent type accepts null as a parameter, the child type must accept null as a parameter but may accept anything else. The opposite is true for return types. The child class may have a more specific return type than the parent class but must not return anything the parent would not allow. This is called covariance. In PHP, the top type is called mixed and allows basically everything, even null values. The null type is at the other end of the scale. If the parent returns null, the child must not return anything else. There is one more restricted return type, never. A function of type never must not return at all. It is either an infinite loop or may only exit the whole program. But never is not an allowed parameter type. There is also a return type void which is in between the two. Void may return from functions, but it returns semantically nothing. Not even null.

What is it good for?

Defining an interface parameter as null is allows to define an almost generic interface which might be substantiated in derived classes. Let’s look at an example.

<?php

interface A {

        public function get(null $value): mixed;
}

class B implements A
{
        public function get(null $value): ?object;
        {
                return $value
        }
}

class C extends B
{
        public function get(A|null $value): XmlElement;
        {
            if (is_null($value)
            {
                $value = new SimpleXmlElement("Not Null");
            }

                return $value;
        }
}

class D implements A
{
    public function get(B $value)
    {

    }
}

If you don’t see the point, let me explain. interface A defines that any implementing class must have a method get(null $value) which has only one mandatory parameter. This parameter must accept null as a value. Any additional parameters must be optional. Any derived class may make the first parameter accept more types and even make it optional. The only drawback: Class D cannot be implemented because function get does not accept null as its first parameter $value.

Generics: I wish I had a little more nothing

This is as close to actual generics as one gets with PHP. Generics are code templates which do not have specific parameter and return types until an actual class is created from them. Some other languages do have them but PHP doesn’t. Larry Garfield and others have made the case for them over and over again.

There are some challenges in the engine which make them hard to implement. It would matter less if we had some tbd or changeme type which can only exist in interfaces and can be overridden by any other type. But we don’t. At least we have standalone null.

A wicked problem from the past

Posted on May 27, 2025May 27, 2025 by Ralf

In the last few evenings there was great despair. Trying to solve one problem would reveal yet another one. As a result, I hesitated to release some of the changes immediately. I don’t want to make people suffer, having to deal with new improved problems of the day whenever they run an update on their horde installations. I’m glad feedback on the mailing list has improved quite a lot, sometimes coming with patches and suggestions and even people telling me things start to just work. That’s great. But whenever you see the light, there’s a new challenge on the horizon. Or rather something very old lurking in the shadows.

Break it till you make it

So recently we felt confident enough to switch our local installation from a frozen state to the latest version of the wicked wiki and the whups bug tracker. We also updated the PHP container, the database version and the likes. This turned into a great opportunity to test the rollback plan. 🙄 Issues were cascading.

Bullets you might have dodged

Generally having both the composer autoloader and the horde autoloader work together creates some friction. This is complicated when the exact order in which they are initialized is not the same in all scenarios. As reported previously, Horde’s apps are not strictly conforming the PSR-0 autoloading standard while the libraries mostly do. Newer releases of the apps autoload using a class map instead of PSR-0 logic. But that class map was not restricted enough in the first iterations. Thus you might have the canonical locations of classes in the autoloader, but also the version in the /web/ dir or, in case of custom hook classes, the version from the /var/config/ directory. Torben Dannhauer suggested to restrict the rules and we will do that with the next iteration. The first attempt to fix this unfortunately broke overriding the mime.local.php config file. An update is in the git tree and will be released as a version later this week. But I’m glad we ran into this as it revealed another dark secret

We part, each to wander the abyss alone

Turned out the wicked wiki software carried a little library Text_Wiki in its belly. Hailing from PHP 4 days, it’s archaic in its structure and its way of implementing features. Parts of the test suite predate phpunit and the way it’s packaged is clearly not from our times. This library also exists as a separate package horde/text_wiki. Which also exists as pear/text_wiki both in the classic PEAR archive and the modern github repository. While the base package does not exist in packagist, the extension driver for mediawiki format does. Odd enough. It’s really a shame the software from the old PEAR ecosystem is slowly fading away because of its ossification. They all share ancestry and they were all largely written and maintained by the same set of people but they are all different in subtle ways.

Text_Wiki works great when it works. But it’s a real treasure trove of incompatibilities with modern PHP versions. Over the years, unicode support has evolved along with the strictness of PHP’s core. While I am very much willing to contribute back any changes to the official pear version, I have my doubts if they will be accepted or handled at all.

Rising again like the phoenix

I really, really did not want to deal with any of this. Creating a fourth version out of three splinters feels like madness. But what can you do?
The result is an upcoming version of horde/text_wiki and horde/wicked which is a radical break. The new text/wiki has no PSR-0 lib/ content. It’s all PSR-4, it’s namespaced, every single class name and file name has changed. A new test suite has been started but only a tiny fraction of unit tests have been ported so far. The different extension drivers for mediawiki and tiki dialects have been incorporated into the base package as I did not want to maintain a hand full of packages.

The whole thing couples as little to horde as possible. It has its own exceptions based on the PHP builtin exception tree, so no dependency on horde/exception. I even stripped the dependency on Pear_Exception as there is no value in keeping that nowadays. Anything which ties this revamped library into the horde ecosystem now lives in horde/wicked. Extra markup rules which render horde portal blocks into a wiki page. Adapters querying the database if a linked page already exists or must be created. Dynamic links created from the registry and application states. None of this glue belongs into the text_wiki library.

Many incompatibilities with modern PHP have been solved. Often this was a matter of mixing and matching bits from the three different splinter versions out in the wild. Some of it is just a chore to do. Of course, the pear XML files are gone and the composer file is fully compliant with packagist and most recent composer. At least this part has been contributed back already.

Dawn, finally

It will be another few evenings until the new horde/wicked and the new horde/text_wiki are ready for a tagged release, probably along with some of the changes to other libraries I explained above. There’s probably something that will break and need fixing. But that shouldn’t block progress for too long.

Horde 6: Return of the Git Tree

Posted on January 13, 2024 by Ralf

Over the last few weekends, Horde 6 code has been merged back from the Maintaina fork and from separate contributions to the former Horde development version, “master”. It was time to upgrade the development tool chain.

Back in the Horde 5 days, there was a utility called git-tools developed by Michael Rubinsky. It would checkout all horde library and application repositories of the github organization into a development directory – also called the git tree – and link into into another structure that resembled a regular Horde 5 installation – called the web tree. This required to hardcode some configurations what could otherwise be auto-detected but it allowed to organize the horde code base in a developer friendly flat structure and provide a ready to run test scenario that reflected the latest code changes even before they were committed to the official repositories. In the days of composer, half of what git-tools does is already covered by composer itself. The git-tools shell already had an integration for the horde-components tool which manages releases, change log, housekeeping tasks and some more.

It was obvious that I wanted a similar functionality for the composer-based setups. It took a while to get this right. The good news is I achieved that. Bear with me for another paragraph of incremental learning or skip to the next section with the results.

It’s getting better all the time

Originally I integrated a generator for “vcs” type repositories into the composer.json generation code. But these vcs repositories are relatively slow. This is OK for one or two apps or leave packages but it’s really really slow when you have a list of 80 or 120 dependencies each pulled from a separate vcs repository.

The next iteration involved generating a satis repository as a standin to releasing code into packagist. The satis repository would be updated by any commit or accepted pull request in any of the serviced repositories. Generating the satis repository is not particularly fast but I managed to scope refreshs to the individual component that needed updating. Reading from satis however is way faster than the vcs approach. Keeping an installation up to date with the latest development commits became much more viable. I also figured I could replace an already installed dependency with an actual git repository and composer would update the autoloader as needed. This works as long as you don’t change the autoloader rules, i.e. upgrade a package from PSR-0 to PSR-4.

Back to #1

Checking out individual libraries as root components and doing integration tests in the satis setup worked OK when the focus was on isolating individual pain points and solving bugs. For any development spanning changes on multiple libraries, it did not work out too well. I ended up implementing a new command.

horde-components github-clone-org

This will checkout a developer tree of git repositories containing apps, libraries or themes.

total 752
drwxr-xr-x 188 root root 4096 Jan 12 11:04 ./
drwxr-xr-x   4 root root 4096 Dec  4 17:44 ../
drwxr-xr-x   9 root root 4096 Jan 11 07:50 ActiveSync/
drwxr-xr-x   9 root root 4096 Jan 11 07:50 Alarm/
drwxr-xr-x   8 root root 4096 Jan 11 07:50 ApertureToAnselExportPlugin/
drwxr-xr-x   9 root root 4096 Jan 11 07:50 Argv/
drwxr-xr-x   9 root root 4096 Jan 11 07:50 Auth/
drwxr-xr-x   7 root root 4096 Jan 11 07:50 Autoloader/
drwxr-xr-x   7 root root 4096 Jan 11 07:50 Autoloader_Cache/
drwxr-xr-x   7 root root 4096 Jan 11 07:50 Backup/
drwxr-xr-x   7 root root 4096 Jan 11 07:51 Browser/
drwxr-xr-x   9 root root 4096 Jan 11 07:51 Cache/
drwxr-xr-x   9 root root 4096 Jan 11 07:51 Cli/
drwxr-xr-x   8 root root 4096 Jan 11 07:51 Cli_Application/
drwxr-xr-x   8 root root 4096 Jan 11 07:51 Cli_Modular/
drwxr-xr-x   8 root root 4096 Jan 11 07:52 Compress/
drwxr-xr-x   8 root root 4096 Jan 11 07:52 Compress_Fast/
drwxr-xr-x   8 root root 4096 Jan 11 07:52 Constraint/
drwxr-xr-x   8 root root 4096 Jan 11 07:52 Controller/
drwxr-xr-x  11 root root 4096 Jan 11 07:52 Core/
drwxr-xr-x   8 root root 4096 Jan 11 07:52 Crypt/
drwxr-xr-x   7 root root 4096 Jan 11 07:52 Crypt_Blowfish/
drwxr-xr-x   6 root root 4096 Jan 11 07:52 CssMinify/
drwxr-xr-x   8 root root 4096 Jan 11 07:52 Css_Parser/
...
drwxr-xr-x   7 root root 4096 Jan 11 08:17 Xml_Element/
drwxr-xr-x   7 root root 4096 Jan 11 08:17 Xml_Wbxml/
drwxr-xr-x   7 root root 4096 Jan 11 08:17 Yaml/
drwxr-xr-x  14 root root 4096 Jan 11 07:50 agora/
drwxr-xr-x  17 root root 4096 Jan 11 07:50 ansel/
drwxr-xr-x  21 root root 4096 Jan 11 07:51 base/
drwxr-xr-x  13 root root 4096 Jan 11 07:51 beatnik/
drwxr-xr-x   7 root root 4096 Jan 12 21:33 bundle/
drwxr-xr-x  12 root root 4096 Jan 11 07:51 chora/
drwxr-xr-x  15 root root 4096 Jan 13 16:54 components/
drwxr-xr-x  11 root root 4096 Jan 11 07:52 content/
drwxr-xr-x   2 root root 4096 Jan 11 08:02 dev.horde.org/
drwxr-xr-x   3 root root 4096 Jan 11 08:02 dns/
drwxr-xr-x  15 root root 4096 Jan 11 08:03 folks/
drwxr-xr-x   6 root root 4096 Jan 11 08:03 git-tools/
drwxr-xr-x  12 root root 4096 Jan 11 08:03 gollem/
drwxr-xr-x   9 root root 4096 Jan 11 08:03 groupware/
drwxr-xr-x  13 root root 4096 Jan 11 08:03 hermes/
drwxr-xr-x   6 root root 4096 Jan 11 08:08 horde-installer-plugin/
drwxr-xr-x   7 root root 4096 Jan 11 08:08 horde-support/
drwxr-xr-x   9 root root 4096 Jan 11 08:10 horde-web/
drwxr-xr-x  10 root root 4096 Jan 11 08:10 hylax/
drwxr-xr-x   7 root root 4096 Jan 11 08:12 iPhoto2Ansel/
drwxr-xr-x  14 root root 4096 Jan 11 08:11 imp/
drwxr-xr-x  14 root root 4096 Jan 11 08:12 ingo/
drwxr-xr-x   3 root root 4096 Jan 11 08:12 internal/
drwxr-xr-x  15 root root 4096 Jan 11 08:12 jonah/
drwxr-xr-x  12 root root 4096 Jan 11 08:12 klutz/
drwxr-xr-x  13 root root 4096 Jan 11 08:12 kolab_webmail/
drwxr-xr-x   9 root root 4096 Jan 11 08:12 koward/
drwxr-xr-x  18 root root 4096 Jan 11 08:13 kronolith/
drwxr-xr-x  13 root root 4096 Jan 11 08:13 luxor/
drwxr-xr-x  17 root root 4096 Jan 11 08:13 mnemo/
drwxr-xr-x  18 root root 4096 Jan 11 08:14 nag/
drwxr-xr-x   9 root root 4096 Jan 11 08:14 operator/
drwxr-xr-x  13 root root 4096 Jan 11 08:14 passwd/
drwxr-xr-x  11 root root 4096 Jan 11 08:14 pastie/
drwxr-xr-x  10 root root 4096 Jan 11 08:15 sam/
drwxr-xr-x  13 root root 4096 Jan 11 08:15 sesha/
drwxr-xr-x  11 root root 4096 Jan 11 08:15 shout/
drwxr-xr-x  13 root root 4096 Jan 11 08:15 skeleton/
drwxr-xr-x   8 root root 4096 Jan 11 08:16 timeobjects/
drwxr-xr-x  14 root root 4096 Jan 12 09:08 trean/
drwxr-xr-x  16 root root 4096 Jan 11 08:16 turba/
drwxr-xr-x  11 root root 4096 Jan 11 08:16 ulaform/
drwxr-xr-x  15 root root 4096 Jan 12 11:06 vilma/
drwxr-xr-x  10 root root 4096 Jan 11 08:17 webmail/
drwxr-xr-x  18 root root 4096 Jan 11 08:17 whups/
drwxr-xr-x  15 root root 4096 Jan 11 08:17 wicked/

This tree does not look like a composer based installation and would not easily work in a web browser. I did not want to reinvent composer with its autoloader and other benefits so I wrapped it into an installer. This installer creates a new copy of the horde/bundle base project and registers all the other libraries as a special type of composer repository “path”.

 /srv/git/horde/components/bin/horde-components -c ~/horde-testme.conf.php install
[  INFO  ] Installation directory is missing: /srv/www/testme
[   OK   ] Created installation directory: /srv/www/testme
Creating a "horde/bundle" project at "../../www/testme"
Installing horde/bundle (dev-FRAMEWORK_6_0)
  - Installing horde/bundle (dev-FRAMEWORK_6_0): Mirroring from /srv/git/horde/bundle
Created project in /srv/www/testme

When running the composer install command on this prepared setup, composer will not download the horde components from packagist or github but will use your local checkout. Only external dependencies are still downloaded from the web.

# composer install
No composer.lock file present. Updating dependencies to latest instead of installing from lock file. See https://getcomposer.org/install for more information.
Loading composer repositories with package information
Updating dependencies
Lock file operations: 92 installs, 0 updates, 0 removals
  - Locking horde/alarm (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/argv (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/auth (dev-FRAMEWORK_6_0 as 3.0.0alpha7)
  - Locking horde/autoloader (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/browser (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/cache (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/cli (dev-FRAMEWORK_6_0 as 3.0.0alpha6)
  - Locking horde/cli_modular (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/compress (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/compress_fast (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
  - Locking horde/constraint (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/controller (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/core (dev-FRAMEWORK_6_0 as 3.0.0alpha17)
  - Locking horde/crypt_blowfish (dev-FRAMEWORK_6_0 as 2.0.0alpha4)
  - Locking horde/css_parser (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
  - Locking horde/cssminify (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
  - Locking horde/data (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/date (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/dav (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
  - Locking horde/db (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/exception (dev-FRAMEWORK_6_0 as 3.0.0alpha4)
  - Locking horde/form (dev-FRAMEWORK_6_0 as 3.0.0alpha6)
  - Locking horde/group (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/hashtable (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
  - Locking horde/history (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/horde (dev-FRAMEWORK_6_0 as 6.0.0alpha7)
  - Locking horde/horde-installer-plugin (v2.5.5)
  - Locking horde/hordectl (v1.0.0alpha4)
  - Locking horde/http (dev-FRAMEWORK_6_0 as 3.0.0alpha8)
  - Locking horde/http_server (dev-FRAMEWORK_6_0 as 1.0.0alpha2)
  - Locking horde/icalendar (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/idna (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
  - Locking horde/image (dev-FRAMEWORK_6_0 as 3.0.0alpha6)
  - Locking horde/injector (dev-FRAMEWORK_6_0 as 3.0.0alpha11)
  - Locking horde/javascriptminify (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
  - Locking horde/listheaders (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
  - Locking horde/lock (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/log (dev-FRAMEWORK_6_0 as 3.0.0alpha9)
  - Locking horde/logintasks (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/mail (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/mime (dev-FRAMEWORK_6_0 as 3.0.0alpha6)
  - Locking horde/mime_viewer (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/nls (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/notification (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/pack (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
  - Locking horde/perms (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/prefs (dev-FRAMEWORK_6_0 as 3.0.0alpha7)
  - Locking horde/routes (dev-FRAMEWORK_6_0 as 3.0.0alpha6)
  - Locking horde/rpc (dev-FRAMEWORK_6_0 as 3.0.0alpha6)
  - Locking horde/secret (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/serialize (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/sessionhandler (dev-FRAMEWORK_6_0 as 3.0.0alpha3)
  - Locking horde/share (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/stream (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
  - Locking horde/stream_filter (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/stream_wrapper (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/support (dev-FRAMEWORK_6_0 as 3.0.0alpha6)
  - Locking horde/template (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/test (dev-FRAMEWORK_6_0 as 3.0.0alpha7)
  - Locking horde/text_diff (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/text_filter (dev-FRAMEWORK_6_0 as 3.0.0alpha4)
  - Locking horde/text_flowed (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/token (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/translation (dev-FRAMEWORK_6_0 as 3.0.0alpha3)
  - Locking horde/tree (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/url (dev-FRAMEWORK_6_0 as 3.0.0alpha6)
  - Locking horde/util (dev-FRAMEWORK_6_0 as 3.0.0alpha8)
  - Locking horde/vfs (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/view (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/xml_element (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking horde/yaml (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
  - Locking pear/archive_tar (1.4.14)
  - Locking pear/console_color2 (0.1.2)
  - Locking pear/console_getopt (v1.4.3)
  - Locking pear/console_table (v1.3.1)
  - Locking pear/pear (v1.10.14)
  - Locking pear/structures_graph (v1.1.1)
  - Locking pear/xml_util (v1.4.5)
  - Locking php-extended/polyfill-php80-stringable (1.2.9)
  - Locking psr/container (2.0.2)
  - Locking psr/http-client (1.0.3)
  - Locking psr/http-factory (1.0.2)
  - Locking psr/http-message (2.0)
  - Locking psr/http-server-handler (1.0.2)
  - Locking psr/http-server-middleware (1.0.2)
  - Locking psr/log (3.0.0)
  - Locking sabre/dav (4.6.0)
  - Locking sabre/event (5.1.4)
  - Locking sabre/http (5.1.10)
  - Locking sabre/uri (2.3.3)
  - Locking sabre/vobject (4.5.4)
  - Locking sabre/xml (2.2.6)
Writing lock file
Installing dependencies from lock file (including require-dev)
Package operations: 92 installs, 0 updates, 0 removals
  - Installing horde/horde-installer-plugin (v2.5.5): Extracting archive
  - Installing horde/util (dev-FRAMEWORK_6_0 as 3.0.0alpha8): Symlinking from /srv/git/horde/Util
  - Installing horde/translation (dev-FRAMEWORK_6_0 as 3.0.0alpha3): Symlinking from /srv/git/horde/Translation
  - Installing horde/exception (dev-FRAMEWORK_6_0 as 3.0.0alpha4): Symlinking from /srv/git/horde/Exception
  - Installing horde/compress_fast (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/Compress_Fast
  - Installing horde/cache (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Cache
  - Installing horde/stream_wrapper (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Stream_Wrapper
  - Installing horde/support (dev-FRAMEWORK_6_0 as 3.0.0alpha6): Symlinking from /srv/git/horde/Support
  - Installing psr/log (3.0.0): Extracting archive
  - Installing php-extended/polyfill-php80-stringable (1.2.9): Extracting archive
  - Installing horde/constraint (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Constraint
  - Installing horde/log (dev-FRAMEWORK_6_0 as 3.0.0alpha9): Symlinking from /srv/git/horde/Log
  - Installing psr/container (2.0.2): Extracting archive
  - Installing horde/injector (dev-FRAMEWORK_6_0 as 3.0.0alpha11): Symlinking from /srv/git/horde/Injector
  - Installing horde/controller (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Controller
  - Installing horde/crypt_blowfish (dev-FRAMEWORK_6_0 as 2.0.0alpha4): Symlinking from /srv/git/horde/Crypt_Blowfish
  - Installing horde/url (dev-FRAMEWORK_6_0 as 3.0.0alpha6): Symlinking from /srv/git/horde/Url
  - Installing horde/css_parser (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/Css_Parser
  - Installing horde/cssminify (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/CssMinify
  - Installing horde/text_flowed (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Text_Flowed
  - Installing horde/secret (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Secret
  - Installing horde/idna (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/Idna
  - Installing horde/text_filter (dev-FRAMEWORK_6_0 as 3.0.0alpha4): Symlinking from /srv/git/horde/Text_Filter
  - Installing horde/stream_filter (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Stream_Filter
  - Installing horde/stream (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/Stream
  - Installing horde/mime (dev-FRAMEWORK_6_0 as 3.0.0alpha6): Symlinking from /srv/git/horde/Mime
  - Installing horde/mail (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Mail
  - Installing horde/listheaders (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/ListHeaders
  - Installing horde/nls (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Nls
  - Installing horde/date (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Date
  - Installing horde/icalendar (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Icalendar
  - Installing horde/browser (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Browser
  - Installing horde/data (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Data
  - Installing horde/hashtable (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/HashTable
  - Installing horde/db (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Db
  - Installing horde/history (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/History
  - Installing horde/view (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/View
  - Installing horde/vfs (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Vfs
  - Installing horde/tree (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Tree
  - Installing horde/token (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Token
  - Installing horde/text_diff (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Text_Diff
  - Installing horde/serialize (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Serialize
  - Installing horde/xml_element (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Xml_Element
  - Installing horde/group (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Group
  - Installing horde/perms (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Perms
  - Installing sabre/uri (2.3.3): Extracting archive
  - Installing sabre/xml (2.2.6): Extracting archive
  - Installing sabre/vobject (4.5.4): Extracting archive
  - Installing sabre/event (5.1.4): Extracting archive
  - Installing sabre/http (5.1.10): Extracting archive
  - Installing sabre/dav (4.6.0): Extracting archive
  - Installing psr/http-message (2.0): Extracting archive
  - Installing psr/http-factory (1.0.2): Extracting archive
  - Installing psr/http-client (1.0.3): Extracting archive
  - Installing horde/http (dev-FRAMEWORK_6_0 as 3.0.0alpha8): Symlinking from /srv/git/horde/Http
  - Installing pear/pear (v1.10.14): Extracting archive
  - Installing pear/xml_util (v1.4.5): Extracting archive
  - Installing pear/structures_graph (v1.1.1): Extracting archive
  - Installing pear/console_getopt (v1.4.3): Extracting archive
  - Installing pear/archive_tar (1.4.14): Extracting archive
  - Installing horde/template (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Template
  - Installing horde/share (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Share
  - Installing horde/sessionhandler (dev-FRAMEWORK_6_0 as 3.0.0alpha3): Symlinking from /srv/git/horde/SessionHandler
  - Installing horde/prefs (dev-FRAMEWORK_6_0 as 3.0.0alpha7): Symlinking from /srv/git/horde/Prefs
  - Installing horde/pack (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/Pack
  - Installing horde/notification (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Notification
  - Installing horde/compress (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Compress
  - Installing horde/mime_viewer (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Mime_Viewer
  - Installing horde/logintasks (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/LoginTasks
  - Installing horde/lock (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Lock
  - Installing horde/javascriptminify (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/JavascriptMinify
  - Installing psr/http-server-handler (1.0.2): Extracting archive
  - Installing psr/http-server-middleware (1.0.2): Extracting archive
  - Installing horde/http_server (dev-FRAMEWORK_6_0 as 1.0.0alpha2): Symlinking from /srv/git/horde/Http_Server
  - Installing horde/cli (dev-FRAMEWORK_6_0 as 3.0.0alpha6): Symlinking from /srv/git/horde/Cli
  - Installing horde/autoloader (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Autoloader
  - Installing horde/auth (dev-FRAMEWORK_6_0 as 3.0.0alpha7): Symlinking from /srv/git/horde/Auth
  - Installing horde/alarm (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Alarm
  - Installing horde/core (dev-FRAMEWORK_6_0 as 3.0.0alpha17): Symlinking from /srv/git/horde/Core
  - Installing horde/dav (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/Dav
  - Installing horde/rpc (dev-FRAMEWORK_6_0 as 3.0.0alpha6): Symlinking from /srv/git/horde/Rpc
  - Installing horde/image (dev-FRAMEWORK_6_0 as 3.0.0alpha6): Symlinking from /srv/git/horde/Image
  - Installing horde/form (dev-FRAMEWORK_6_0 as 3.0.0alpha6): Symlinking from /srv/git/horde/Form
  - Installing horde/argv (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Argv
  - Installing horde/horde (dev-FRAMEWORK_6_0 as 6.0.0alpha7): Symlinking from /srv/git/horde/base
  - Installing horde/yaml (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Yaml
  - Installing horde/cli_modular (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Cli_Modular
  - Installing horde/hordectl (v1.0.0alpha4): Extracting archive
  - Installing horde/routes (dev-FRAMEWORK_6_0 as 3.0.0alpha6): Symlinking from /srv/git/horde/Routes
  - Installing horde/test (dev-FRAMEWORK_6_0 as 3.0.0alpha7): Symlinking from /srv/git/horde/Test
  - Installing pear/console_color2 (0.1.2): Extracting archive
  - Installing pear/console_table (v1.3.1): Extracting archive
81 package suggestions were added by new dependencies, use `composer suggest` to see details.
Generating autoload files
Applying /presets for absent files in /var/config
Looking for registry snippets from apps
Writing app configs to /var/config dir
Linking app configs to /web Dir
Linking javascript tree to /web/js
Linking themes tree to /web/themes
1 package you are using is looking for funding.
Use the `composer fund` command to find out more!

This composer environment works just like a regular installation. When you install turba, kronolith or passwd through composer, it will end up linking these apps and their library dependencies from the development tree.

Composer provides an option to copy files from the repositories rather than link the files. This would allow creating archives with artifacts for distribution packaging. The horde-components tool does not yet provide a switch to generate the necessary tweak to the repository files.

Maintaina’s Horde 6 goes upstream

Posted on October 27, 2023 by Ralf

It’s been a while. It’s been much too long actually. Let’s look forward though.
I recently started to onboard into FOSS development again. People have been asking for PHP 8.2 support with Horde but capacity to deliver that was very limited. But we’re getting there.

I used the “Maintaina” fork to deliver fixed and upgraded customized versions of Horde beyond what was allowed in Horde’s master branch. It’s time to move forward though. Maintaina used to target PHP 7.4 to PHP 8.1, composer2 based install from a custom satis repo and some quite invasive changes to CalDAV/CardDAV support. Most prominently, Maintaina introduced library level compatibility with several PSRs (container, http middleware, logging) and the all new horde/http_server component.

At the moment I am importing the FRAMEWORK_6_0 branches and the alpha releases based on this branch to horde’s upstream repository. So far I have imported enough libraries to make the Horde Base application barely install from packagist.

Get the current state of affairs

Your magic carpet is:

composer create-project horde/bundle .

Originally I wanted to mimic maintaina’s setup with a separate satis server as a QA stage before release to packagist. I re-considered and dropped the separate satis server. Every update of the development branches and every tagged release is consumed via packagist. I will have to fix some of my tools and workflows to reflect that. The tagged alpha releases still use maintaina’s satis server. The upcoming releases won’t.

Onboarding Procedure

I leave the FRAMEWORK_5_2 and master branches mostly untouched. FRAMEWORK_6_0 is the new default branch on github for anything I am handling. I only edit other branches if they block packagist’s import. Usually I also rebase FRAMEWORK_6_0 on any latest commits of master, but in some cases I only cherry-picked from master branch. In some cases, some entries of the changelog between 2022 late and now (october 2023) might be missing. Pull Requests, bug reports and patches welcome.

What’s next

Before I move forward with the actual applications, I want to make sure the necessary infrastructure is in place. I need to fix some aspects of the FRAMEWORK_6 version of composer. The workflow files in each repo need some review, too. Does PHPUnit and PHPStan still pass? Can we improve management of 130+ repo’s workflows?

Finally https://dev.horde.org should index from packagist, not from the satis server we originally planned to use.

What will become of maintaina repos?

The maintaina repos have had direct contributions from some trusted maintainers from the company I used to work for. They service some customers out of these repos and the related SATIS server so I won’t actively break it. However, with the move to Horde upstream, maintaina has served its purpose for me and I will not actively support the fork anymore. I suggest once everything is ported to upstream, maintaina should be archived. I will need to consult other stakeholders of this fork and the satis server.

Maintenance cost

Over the lifetime of the fork I have explored and applied various strategies for keeping the effort in check. Still, a fork of 100+ repos and the accompanying infrastructure for testing and deploying is a major burden which detracts from actually developing and maintaining code. I am glad I can save on this now and actually contribute to Horde directly in a way that doesn’t slow down activity too much. We now have the chance to speed up the cycle of feedback and releases. I hope this attracts some occassional and regular contributors.

Satis is now a Composer Plugin.

Posted on October 12, 2022October 12, 2022 by Ralf

Satis is the lightweight, static repository generator for the composer package manager. It enables admins to provide PHP software packages inside airgapped CIs, OS packaging environments and restricted data centers.

Back in August I added a plugin mode to satis to make it work as a regular composer plugin. While working on it, I also fixed some preexisting issues found by static analysis and made it compatible with the recent composer versions 2.3 and 2.4.

This week, the upstream maintainers merged my contribution. I feel a bit satis-fied 😉

Why make it a plugin?

When looking under the hood, it is easy to see that satis always has been some awkward kind of plugin or extension. It literally sits on top of an internal copy of composer. It hooks into its internals in not quite transparent ways, it uses its class and interface organs for its own vital functions. You might call it a parasite that attaches to composer’s body for its own needs. There are downsides to this approach. The first is that you need a private copy of composer. The second is that any refactoring of composer internals likely breaks satis compatibility. That happened some time ago when composer 2.3 and 2.4 were released and not for the first time. Composer has a maturing plugin API with nice, well-defined integration points. It provides some means to overload or amend core functionality but it also provides messaging between core and plugins. I only did the bare minimum work to make satis hook into the plugin API and not break the standalone mode. When installed as a dependency, package resolution will ensure that the API versions used by satis matches the API versions provided by the underlying composer.

I don’t quite understand… What is the benefit?

By itself, this change provides little benefit. It is a feature enabling feature.

Satis can be further refactored to make compatibility bread less often
Satis can send and receive events from composer or other composer plugins. This enables running satis as part off a custom command. Think passing unit and integration tests of a project and then conditionally updating the staging or canary package repository.
Satis’ schema could be amended to make a project’s root package also function as an instruction to build a repository of all dependencies with almost zero configuration. Add this to a workflow or add a collaborator plugin that handles the necessary push/release and you have a powerful tool for CI and developer laptop alike.

But as I went along, I also re-aligned satis with the latest breaking changes inside composer 2.3/2.4. This will benefit users who do not care about the whole plugin story.

What’s next?

With satis 3.0-dev merging this initial change, the next steps are obvious, but not urgent.
Making the new plugin mode play nice with the latest composer was already easier than fixing the standalone mode. Satis still has an internal, now updated dependency copy of composer which is only run in standalone mode.

Standalone mode should be refactored to be just a thin wrapper around composer calling into its satis plugin. Keeping intrusion into composer internals to the bare minimum to hide the builtin commands and re-brand it as satis, this would make breakage on upcoming updates much less likely. Eventually, we can maybe stop carrying around a source code copy of composer at all.

Finally, there is reaping the benefits. I want to leverage composer/satis functionality inside the horde/components tool. Rolling out new versions of horde stuff could be so much easier.

Resources

Horde on PHP 8.1 and Composer: Update

Posted on October 6, 2022 by Ralf

Regular readers of this blog and many other are aware that PHP 7.4 will stop receiving security updates when PHP 8.2 comes out in November. This has made many horde admins question if they can continue to run Horde. Some events in life have made progress slower than originally planned. So where are we?

Confirmed running under PHP 8.1 and composer 2.4

horde/base in Browser
essential Horde Base CLI tools like horde-db-migrate and hordectl
horde/base portal blocks and admin area
horde/components developer tool
horde/turba Addressbook App Reading and writing contacts in the UI
horde/mnemo Notes App UI and webdav
horde/nag Tasks Apps UI, webdav, caldav
horde/kronolith Calendar App UI, webdav, CalDAV
horde/passwd Password App – Changing passwords worked with the hordeauth driver
horde/gollem File Manager App – very limited testing so far
horde/imp Webmail – very limited testing.

I run on a setup with openssl3 and a recent mariadb against dovecot and postfix. You can also consume the openSUSE 15.4 based containers built nightly. There is still considerable log spam from deprecation notices: Mostly tentative return types and signatures, also some use of deprecated functionality like strftime. Each night a few of these disappear. They don’t stop you from running horde apps.

I also have an eye on PHP 8.2 compatibility – So far, there should not be too many surprises. I also check most unit tests against the development version of PHPUnit 10.

This code is quite solid on PHP 7.4 – production users run on it.
On PHP 8.1 I consider it ready for adoption tests. Breakage is expected, feedback is welcome. Be sure to have a backup of the database and of any mail accounts you connect to it.
There is a lot to be done over the next few weeks.

If it does not run for your combination of drivers, please contact me via the horde mailing list.

Known caveats:

imp config SHOULD have an explicit cache setting: Set it to false to disable caching or to ‘cache’ to use Horde’s default cache. The ‘sql’ option also seems to work but I do not recommend it.
The RPC interface has seen very little testing. The json-rpc protocol should work. I have no desire to look into xmlrpc though unless somebody voices his needs. Beware, the xmlrpc extension has moved out of mainstream into pecl.
I do not have the necessary setup to comment on ActiveSync currently
Kolab integration is very likely broken. I don’t think anybody really uses recent horde with ancient kolab versions.
Most likely the SyncMl and PHPGroupware drivers are useless. If anybody really uses that bridge, please give feedback
I usually test against sabre/dav 4.4 – if you use anything else and see bugs, let me know
I don’t currently test against postgresql. MariaDB, MySQL, PerconaDB should work.
As PHP’s LDAP extension has moved from resources to objects, the LDAP authentication and addressbook drivers likely need an update. I do not currently test against LDAP but this is something I want to change
I know my former colleagues run LDAP and Redis so likely they will give some feedback in that area – Cannot comment on the timeline. I will offer a redis option for the maintaina container setup soonish.

Modernizing horde/text_diff

Posted on September 15, 2022 by Ralf

If you ever read a github pull request or similar extension proposal, you will likely have seen side by side comparisons of the original and the changed file. You may also have seen some text format that highlights only differences and a little context but hides the unchanged rest of the file. Both of these formats are called Diff, named after the popular diff and patch utilities dating back to ancient Unix times. The git diff command does something very similar. The horde/text_diff library and its ancestor, the pear/text_diff library, are tools to generate and format such difference information for different usage scenarios.

Apart from Horde’s internal usage in its repository viewer, horde/chora, and its wiki software, horde/wicked, the tool is also used by external parties. WebSVN maintainer Michael O. approached me because he wanted to use a PHP 8.2 ready version of horde/text_diff to substitute an older component which did not do the job. Michael has been very helpful in getting me started, pointing me to some issues to solve and also providing his own solutions in some parts. The result is a conservative update of horde/text_diff that will run in the upcoming versions of PHP without causing any trouble. But this is only where I started.

Breaking bad habits

A closer look at the internal structure of the library showed that it deserved a major overhaul. The solution was to refrain from a verbatim upgrade to namespaces and the likes but to actually change some things. This meant breaking backwards compatibility. I go to great lengths to keep a conservative drop-in version of everything I touch in the lib/ folder. Sometimes it is just an interface or wrapper, sometimes the new and old code do not really share a lot.

I began with adding type hints to most methods. Targetting PHP 8.1+ for the src/ path allows to use union types and intersection types. A lot of knowledge hidden in phpdoc comments is moved into actual code and makes it more robust.

Exploring the code for base classes and interfaces, I noticed that some things I did not like.

Method signatures did not add up

Some method signatures did not add up. Depending on the type of Diff Engine, the diff() method would take different types of arguments. The interface was mixing the specifics of how the diff engine is set up with the command to create a list of operations objects. Loading the engine is now separated from running it. The running method is now always called in the same way.

Internal dependency creation

The Diff, Threeway Diff and Mapped Diff utilities all created their diff engine internally. To do this, they needed a very flexible constructor that allowed passing whatever is needed to set up the actual engine. That was bad enough but they also did it in different ways. The Differs’ constructor now only accepted a pre-constructed engine. For convenience, I created a factory which would take over the responsibilities originally assigned to the differs’ constructor: Building a differ from input and if no specific differ was chosen, selecting one by some priority logic. In the end it turned out that the Differ does not need the engine at all but rather needs the product of the Engine: A set of operations to transform document A to document B. Born is the OperationList class. I did not want to just pass an array. I added a small static method as a named constructor. It frees the actual constructor of too many responsibilities and allows to keep the interface clean and strict.

More explicit type juggling

Creation if diffs contains some interesting math. The algorithms use a lot of short variable names and operations that make sense if you know the underlying theory but otherwise look like garbage. I added some explicit conversions between string and integer and made some changes to ensure a number zero or an empty string is not mistaken for a “false” or “null” value which would have another meaning.
Overall it is now much easier for static analyzers to spot any issues.

Dual stacks have a price tag

Essentially maintaining two different sets of the library comes with some cost. One must ensure that unit tests targeted at the newer platform are not run when testing for compatibility with the older platform. The conservative lib/ is ready to run PHP 7.4 code but the modern version in src/ must be transpiled to be run in PHP 7.4 – I do that on release but still it is another aspect that needs minding. As I also use automated tools for some upgrade tasks, I need to ensure I do not upgrade the lib/ path. The price is worth it as I cannot convert the code base at once and I want to provide a good development experience to all who are caught in between maintaining an older release or creating new code. I am in that spot myself. Essentially it allows me to run two conflicting major versions of some critical libraries and pick the right one for different sub systems. The need will go away as code is gradually migrating towards the newer implementations. At some point a next major version will drop the conservative path. Anybody interested is free to maintain the older major version and keep using it.

Upcoming work

I consider the external interface of the newer horde/text_diff implementation fairly stable by now. Internally, however, there is a lot of room for improvement. Some functionality should move out of the base classes and into separate traits – which the base classes will use. Some getters should be added and used, preparing to move some public variables to internal state in a next major release. The new OperationList gets unloaded to plain arrays in several places – It needs to learn some tricks without degrading into a glorified array. None of this should stop early adopters from using the new code base. None of this is supposed to break any user code.

Out of scope for now

There are some items which I decided to postpone for now. One thing which bothers me is the amount of dependencies. While a dependency on horde/exception makes sense, it pulls in horde/translation for no good reason. Horde\Util is pulled in but really only used in two places: A horde/string call which could be reduced to a direct call to an internal function and one call to a helper for handling temporary files. That helper should maybe live in its own library, nicely decoupled from unrelated utilities. There was a reason why they were packed together but it is no longer relevant.

Also, some functionality is missing in the Xdiff-based engine. Most distributions do not even offer php-xdiff, including my own development platform. I will add that feature once I get it into CI and into the development setup. I do not want to delay other items to do that right now. Patches welcome 🙂

Tools to build better Tools faster

Posted on August 30, 2022 by Ralf

Behind every lofty architecture mantra there is mundane execution. This is best left to tools and I don’t mean anybody in particular but programs that help us make better programs. It basically goes like this: Build tool. Use tool. Build better tool. Build tool to build better tool. Build better tool to build better tool faster. And so on. Implementing this in practice can be quite boring but the alternative is to do boring things again and again and again and that’s enough already. So let’s see.

Maintaining 100+ libraries and programs involves doing a few things over and over again. Automating these seems natural but requires some thought. Developers want to spend their time in interesting and useful ways. Querying and manipulating git repositories is repetitive. Updating a changelog file with a select subset of messages also present in the git commits is repetitive. Rewriting project metadata and updating CI jobs for new PHPUnit and PHP versions or base operating systems is repetitive and requires no brains at all, why should I do this 100+ times?

Off the shelf tools

Using tools that already exist and are maintained by other parties is a no-brainer. Which tools can help?

PHPUnit helps us spot and eliminate regressions before any user is affected. The tool itself is maintained by Sebastian Bergmann but writing and upgrading the actual test code is a chore.
PHPStan or Psalm – I prefer PHPStan – are static analyzers which help developers spot places where signatures, types and assumptions don’t add up. To get the best out of it, either phpdoc annotations or parameter and return types must be added. No tests to write, which is good – but PHPStan is organized in progressively strict levels and each library needs to be checked against the level it is supposed to pass. Micromanaging that is boring as hell, tools are needed.
php-cs-fixer is developed by friendsofphp – it is a basic code manipulation tool which helps anywhere from adhering to PSR-12 or PER-1 to automatically upgrading from array() to [] notation. Configuring this beast is easy but ensuring the most current rules are used in every project is another burden.
rector is another tool that transpiles code either up or down to select standards. It will move implicit knowledge or phpdoc data into actual code or do the different thing. It will choose older ways to express something over new ones or vice versa. Configuring it to do only what is helpful is quite a challenge. Also ensuring the most recent config is used is just boring and cumbersome. Tool needed.

Homegrown tools

The horde project has some home grown tools that can help but need development themselves.

horde/git-tools by Michael Rubinsky used to be the way to assemble a bleeding edge developer copy from zillions of github repos. In a modern composer based installation this tool is less useful but it contains a lot of interesting capabilities that should be factored out
horde/components can generate composer and pear metadata from a self-defined yaml format. It can create tar archives from repositories, implements a basic workflow engine for release and quality check tasks and does some other things. Its internal architecture is rooted in history and while some of its functionality seems out of touch with 2022, many other parts deserve expanding or factoring out into modern self-contained libraries for reuse.
horde/hordectl is a command line tool to interact with a Horde installation. Inject users and passwords, configure permissions, groups or app-specific resources from yaml files and defaults. It needs some upgrading, it could do so much more to facilitate proof of concept, showcase or CI installations.
horde/horde-installer-plugin is a plugin for composer that helps bootstrap a horde installation and its web-readable part. Much of its code would best be moved out to separate libraries.

Building blocks

Existing and new libraries should inherit functionality moved out from existing tools or newly created

horde/vcs is a version control library. Its main origin are the horde/chora application and the installation/development tools. Recently I began to move or re-implement code from git-tools and horde/components into this library. I am less interested in the rcs, cvs and svn parts. The original library followed an approach abstracting the differences between git, cvs, svn & friends. This limits its usefulness. I see how it facilitates creating an application that consumes and shows code from these. Still, there should be a lower level of abstraction that provides the unique capabilities of git in a programmatic fashion. This is one thing I currently work on
horde/rampage used to be a dead end but I am reusing the library for deployment and introspection related code factored out from other tools.
horde/filesystem is a new library, focused on object-oriented filesystem traversal and manipulation. Still very immature but I hope to turn it into a standalone and reusable tool.
horde/registry is the stub of an upcoming redesign of the core bootstrapping process. No more globals, reliance on PSR-11 DI containers and PSR-4 autoloading – this registry will do less than its ancestors yet be much more powerful and easy to use. This is still much work.
horde/cli_modular is a tool to write extensible, pluggable commandline interfaces. It is used by horde/git-tools, horde/components, horde/hordectl and a few others. In the current upgrade cycle some redesign is necessary to make it viable for modern environments and free it from problems already solved by autoloaders or DI containers.

So much work to do but devoting some time to better tools is better than doing mindless conversions of existing code over and over.