bookmark_borderNet_DNS2 PHP 8.x compat issue

TLDR: When using horde/mail_autoconfig or other features using pear/net_dns2 under PHP 8.x, use the “master” branch or wait for a release of version 1.5.3 or higher.

While upgrading the Maintaina Horde codebase for PHP 8.1, I stumbled upon a problem:

[Sun May 15 12:15:15.026499 2022] [php:error] [pid 156] [client 172.23.0.1:43894] PHP Fatal error:  Uncaught ValueError: fread(): Argument #2 ($length) must be greater than 0 in /srv/www/horde/vendor/pear/net_dns2/Net/DNS2/Cache/File.php:147\nStack trace:\n#0 /srv/www/horde/vendor/pear/net_dns2/Net/DNS2/Cache/File.php(147): fread()\n#1 [internal function]: Net_DNS2_Cache_File->__destruct()\n#2 {main}\n  thrown in /srv/www/horde/vendor/pear/net_dns2/Net/DNS2/Cache/File.php on line 147, referer: http://localhost/horde/admin/config/config.php

It turns out Net_DNS2 as of latest version 1.5.2 has a problem in its filesystem cache when run under PHP 8.x – it tries to read the cache file content even if the file size is 0 or the file does not yet exist. In PHP 8.x this yields a ValueError because reading with a length 0 should not be attempted.

Mike Pultz, the maintainer of Net_DNS2 already has a fix in his master branch. I have kindly asked him to release a stable version with this fix. However I understand it may take a while or may not be very high on the daily priority list of todos. I am not sure what other changes the master branch contains which might impair its usefulness to your use case. I am hesitant to put work into a problem somebody else already fixed. I will only provide a downstream-packaged version in case no update happens within reasonable time.

bookmark_borderMaintaina Horde: Tumbleweed and PHP 8.1

PHP 8.1 is available off the shelf in openSUSE Tumbleweed. I will shortly prepare a PHP 8.1 / tumbleweed version of the maintaina Horde containers. These will initially be broken due to some outdated language constructs. As PHP 7.4 will EOL by the end of this year, I decided not to bother with PHP 8.0 and ensure compatibility with PHP 8.1 right away, while staying compatible with PHP 7.4 until end of year. This is not fun. PHP 8.x provides several features which allow for more concise code. I will not be able to use them.
This also means that for the time being I will produce code which you may find more verbose than necessary. While Constructor promotion is mostly about being less verbose, Readonly Properties and Enums kill some of the pro-method arguments in the eternal discussion if getter methods or public properties are more appropriate interfaces. Union Types and Intersection Types allow a flexibility of method interfaces which PHP 7.4 can only emulate. You can get far by type hints for static analysis combined with boilerplate guard code inside a method and dropping type hints all along or using insufficient surrogate interfaces. But it is really not shiny. Maintaining software which shows its age has its tradeoffs.

bookmark_borderSimplifying Routing / PSR-15 bootstrap in Horde

As you might remember from a previous post, Horde Core’s design is more complex than necessary or desirable for two main reasons:

  • Horde predates today’s standards like the Composer Autoloader and tries to solve problems on its own. Changing that will impair Horde’s ability to run without composer which we were hesitant to do, focusing on not breaking things previously possible.
  • Horde is highly flexible, extensible and configurable, which creates some chicken-egg problems to solve on each and every call to any endpoint inside a given app.

Today’s article concentrates on the latter problem. More precisely, we want to make routing more straight forward when a route is called.

A typical standalone or monolithic app usually is a composer root package. It knows its location relative to the autoloader, relative to the dependencies and relative to the fileroot of the composer installation. Moreover, the program usually knows about all its available routes. It will also have some builtin valid assumptions about how its different parts’ routes relate to the webroot.
None of this is true with a typical composer-based horde installation.

  • None of the apps is the root package
  • Apps are exposed to a web-readable subdir of the root package
  • While the relative filesystem location is known, each app can live in a separate subdomain, in the webroot or somewhere down the tree
  • Each app may reconfigure its template path, js path, themes path
  • The composer installer plugin makes a sane default. This default can be overridden
  • Each app can be served through multiple domains / vhosts with different registry and config settings in the same installation
  • Administrators can add local override routes
  • Parts of the code base rely on horde’s own runtime-configurable autoloader rather than composer.

This creates a lot of necessary complexity. The router needs to know the possible routes before it can map a request. To have the routes, the context described above must be established. Complexity cannot be removed without reducing flexibility. There is, however, a way out. The routing problem can be divided into three phases with different problems:

  • Development time – when routes are defined and changed frequently for a given app or service
  • Installation/Configuration time – when the administrator decides which apps’ routes will be available for your specific installation
  • Runtime – when a request comes in and the router must decide which route of which app needs to react – or none at all

Let’s ignore development time for now. It is just a complication of the other two cases. The design goal is to make runtime lean and simple. Runtime should initialize what the current route needs to work and as little as possible on top of that. Runtime needs to know all the routes and a minimal setup to make the router work. Complexity needs to be offloaded into installation time. Installation time needs to create a format of definite routes that runtime can process without a lot of setup and processing. As a side effect, we can gain speed for each individual call.

Modern Autoloaders are similar in concept: They have a setup stage where all known autoloader rules of the different packages are collected. In composer, the autoloader is re-collected in each installation or update process. The autoloader is exposed through a well-known location relative to the root package, vendor/autoload.php – it can be consumed by the application without further runtime setup. The autoloader can be optimized further by processing the autoloading rules into a fixed map of classes to filenames (Level 1), making these maps authoritative without an attempt to fail over on misses (Level 2a) and finally caching hits and misses into the in-memory APCu opcache. Each optimization process makes the lookup faster. This comes at the cost of flexibility. The mapping must be re-done whenever the installation changes. Otherwise things will fail. This is OK for production but it gets in the way of development. The same is true for the router.

The best optimization relies on the application code and configuration being static. The list of routes needs to be refreshed on change. Code updates are run through composer. The composer installer plugin can automatically refresh the router. Configuration updates can happen either through the horde web ui or through adding/editing files into the configuration area. Admins already know they need to run the composer horde-reconfigure command after they added new config files or removed files. Now they also need to run it when they changed file content. In development, routing information may change on the fly multiple times per hour. Offering a less optimized, more involved version of this route collection stage can help address the problem.

A new version of the RampageBootstrap codebase in horde/core is currently in development. It will offload more of horde’s early initialisation stages into a firmware stack and will reduce the early initialisation to the bare minimum. At the moment, I am still figuring out how we can do this in a backward compatible way.

bookmark_borderMaking horde/core more versatile

Hello, you may have seen my blog go quiet for a while. After two years of a global pandemic, I decided to take some more time with my family and also offload some knowledge and responsibilities around the Maintaina Horde codebase to my fellow team members at B1 Systems. I also took some time considering some very fundamental Horde design implications. One of these is horde/core.

You can use the Horde Framework and its libraries in two different styles. The first style I would call loosely coupled. Most libraries try to depend on as few as possible other horde libraries and concentrate on specific purposes. That makes them reusable outside of the wider context of the framework. For example, the backend for Michael Rubinsky’s blog uses the Horde Router, the Horde Controller library and the Horde Injector DIC as a kind of micro framework without much supporting code. A custom binder simulates setup of view paths normally performed by a horde environment.

The alternative is the “horde app” use case or tightly integrated use case. Your site will have a horde base app providing common capabilities and one or multiple modules or horde apps. They integrate with a framework-generated topbar, can use an inter-app API, react to a common styling choice etc. Developers benefit from being able to reuse solutions for preference storage and UI, configuration, permissions, user groups, predefined callbacks for webdav, rpc, language and presentation handling etc. This is all great if you just want to build some addon to your Horde Groupware installation. In other cases, Horde’s base does a lot of stuff you do not really need or want. I want to change that.

The horde/core library started out as a spinoff off the horde base app. It contains a lot of glue for putting together backends, drivers, config files, caches and it also provides base classes needed by the horde applications. The horde base app contains the endpoints for RPC, route-based UI, webdav, caldav. It also contains a micro bootstrapping file called horde/lib/core.php. This file installs an error handler, sets up some basic PHP sanity stuff, does some magic around Horde’s own autoloader, defines some constants and hooks into horde/config/horde.local.php, allowing the admin to inject some early-init custom magic. Our horde-installer-plugin for composer makes use of this hook to setup the composer autoloader (and some more constants) in a backward compatible way.

I think, outside of support for old-style code bases, none of this should happen. But our current implementation of the controller framework still depends on that magic and uses code in horde/core which also depends on it. In pre-composer environments, horde needs some tricks to find out basic facts about where it is, where everything else is, how to setup autoloading etc. In modern environments, we should neither pollute the global namespace with constants and global variables nor should we have an involved, tightly coupled setup process way before we even look at the app and route called for. We need to reorder how and when things are done.

Both developers and runtime operators benefit from this redesign. Developers can reduce boilerplate when relying on much, but not all of the Horde Framework and not exactly building a module for a horde installation. They are less entangled in conventions which do not make sense anymore. Operators will notice a lower memory, i/o and computation footprint, resulting in higher speed. This is because we throw out a lot of overhead unrelated to the current call. In the old setup we even created an IMAP connection when the current user was forbidden to use the mail component and the screen shown was for changing passwords or handling preferences. Yes, even when downloading an addressbook you would have triggered and IMAP connection handshake. We do not need that and by now, we have to tools to avoid it.

Stay tuned for more details in upcoming articles.