bookmark_borderSatis is now a Composer Plugin.

Satis is the lightweight, static repository generator for the composer package manager. It enables admins to provide PHP software packages inside airgapped CIs, OS packaging environments and restricted data centers.

Back in August I added a plugin mode to satis to make it work as a regular composer plugin. While working on it, I also fixed some preexisting issues found by static analysis and made it compatible with the recent composer versions 2.3 and 2.4.

This week, the upstream maintainers merged my contribution. I feel a bit satis-fied 😉

Why make it a plugin?

When looking under the hood, it is easy to see that satis always has been some awkward kind of plugin or extension. It literally sits on top of an internal copy of composer. It hooks into its internals in not quite transparent ways, it uses its class and interface organs for its own vital functions. You might call it a parasite that attaches to composer’s body for its own needs. There are downsides to this approach. The first is that you need a private copy of composer. The second is that any refactoring of composer internals likely breaks satis compatibility. That happened some time ago when composer 2.3 and 2.4 were released and not for the first time. Composer has a maturing plugin API with nice, well-defined integration points. It provides some means to overload or amend core functionality but it also provides messaging between core and plugins. I only did the bare minimum work to make satis hook into the plugin API and not break the standalone mode. When installed as a dependency, package resolution will ensure that the API versions used by satis matches the API versions provided by the underlying composer.

I don’t quite understand… What is the benefit?

By itself, this change provides little benefit. It is a feature enabling feature.

  • Satis can be further refactored to make compatibility bread less often
  • Satis can send and receive events from composer or other composer plugins. This enables running satis as part off a custom command. Think passing unit and integration tests of a project and then conditionally updating the staging or canary package repository.
  • Satis’ schema could be amended to make a project’s root package also function as an instruction to build a repository of all dependencies with almost zero configuration. Add this to a workflow or add a collaborator plugin that handles the necessary push/release and you have a powerful tool for CI and developer laptop alike.

But as I went along, I also re-aligned satis with the latest breaking changes inside composer 2.3/2.4. This will benefit users who do not care about the whole plugin story.

What’s next?

With satis 3.0-dev merging this initial change, the next steps are obvious, but not urgent.
Making the new plugin mode play nice with the latest composer was already easier than fixing the standalone mode. Satis still has an internal, now updated dependency copy of composer which is only run in standalone mode.

Standalone mode should be refactored to be just a thin wrapper around composer calling into its satis plugin. Keeping intrusion into composer internals to the bare minimum to hide the builtin commands and re-brand it as satis, this would make breakage on upcoming updates much less likely. Eventually, we can maybe stop carrying around a source code copy of composer at all.

Finally, there is reaping the benefits. I want to leverage composer/satis functionality inside the horde/components tool. Rolling out new versions of horde stuff could be so much easier.

Resources

bookmark_borderLet’s have a date. Revised horde/date ideas

It’s bad but not as bad as you think.

The way we write dates is very different among cultures and technologies. Even countries of the same language family might have totally different notions where to put the year, where to put the month, where to put the day when writing down a date in numbers. Apart from the order, we might use hyphen -, dot . or slash / to mark the sections. Leading zero yes or no. It gets much worse when writing out a date, like “Vlad was born on the 13th of June” or “Monday, October 10 I will have a barbecue”. Mix in different languages and different ways to abbreviate Monday as “Mo”, “Mon”, “M”.
The PHP-native ways to deal with date math and date formatting are a little quirky and best wrapped in some code that hides away the nasty details but exposes useful capabilities.

The horde/date library has been around for ages and it does its job. Due to changes in the PHP 8.1+ engine, it requires some internal re-engineering. As it is also due for a conversion to namespaces and PSR-4, I decided to think about the interface a little more. These are rough ideas and I am still figuring what will be fun and safe to work it and what is possibly over-thinking it.

It’s DateTime all right.

PHP provides internal data types to store and manipulate a date: DateTime and DateTimeImmutable. Horde_Date behaves similar to DateTimeImmutable but implements a lot of intricate math on its own. Still, it uses DateTime for some conversions. It makes sense to delegate most mutation math to the DateTimeImmutable class and make it also hold most state. No need to manually manage minutes, hours, days of weeks etc in most cases. Preferring Immutable safes us some clone magic.

Wrap it up or extend it?

There are two possible approaches to dealing with the builtin type: Either extend it or hide it in a wrapper. The third option would be to implement the interface but this is not possible.
Extending any of the two DateTime types would be handy as it is the lingua franca between different library ecosystems. We could simply inject our Horde Date type into any library that uses it. There are down sides to this approach though. While DateTimeInterface is reasonably small and mostly useful, DateTime and DateTimeImmutable expose a lot of functionality. Some of it is awkwardly named. Some of it would have to be wrapped in extending to avoid inconsistencies in our own object. Some of it possibly clashes with own naming schemes and blocks us from using preferred signatures. Some of it might not fit into our own notion of what belongs where. There is also a risk of exposing different functionality based on PHP versions. This is undesirable. Extending is not the way to go. Hiding away the DateTimeImmutable object and exposing it explicitly might seem a little verbose. It offers some interesting applications.

Clock Date – Now is the right time.

A clock date type always emits now. Asking it again at a later point will yield a later time without having to manipulate the object. This is useful for tracing duration of processes or for emitting status messages. We can also make this clock the second element of a time span. Its duration will automatically expand. We can add a Stop method to the clock which will return a regular, fixed date for further processing.

No Date – When it’s not right for you.

Sometimes we cannot rely on a date being present in the input. It might be optional. It might be required by the current data model but used to be optional or malformed. Traditional options would be expressing the non-date as null value, throwing an exception or silently assuming no date means now. These are appropriate, good solutions in many cases. Sometimes you may prefer to have the “no date” information behave a little like a date – until it reaches a point where it cannot. For example, a “not a date” can be formatted many ways. It can be serialized to the backend, provided the backend can deal with it. It can however not be calculated, mutated or cast to a DateTimeImmutable. I am not yet sure how to handle this. Maybe it should be confined to Formatters and Readers.

In The Zone.

PHP provides an own finite list of Time Zone identifiers. Wrapping PHP’s timezone objects allow dealing with well known but unsupported timezone names. We can map them to known names. We can safe the originally provided name for later usage. We might not carry a PHP timezone at all but signal the other parts of the library that some custom code must be applied.

In Good Format.

There are a plethora of ways to express a given date. There are three builtin date renderers in PHP, IntlDateFormatter (not always installed), strftrime (deprecated) and DateTimeInterface::format (English only). You might add your own. Each has its own dependencies, arguments, requirements.
It is much simpler if there is a Formatter type. Implementations can just configure it and load it with a date. Consumers have a simple interface to work with them regardless of how they are implemented.This also allows to keep the dependency footprint of the core date library low and makes adding more output formats very easy. The same is true for reading data. Reading values from various formats should not be the Date object’s concern. Another object should turn arbitrary string, integer or other data into dates – including legacy Horde_Date objects.

bookmark_borderHorde on PHP 8.1 and Composer: Update

Regular readers of this blog and many other are aware that PHP 7.4 will stop receiving security updates when PHP 8.2 comes out in November. This has made many horde admins question if they can continue to run Horde. Some events in life have made progress slower than originally planned. So where are we?

Confirmed running under PHP 8.1 and composer 2.4

  • horde/base in Browser
  • essential Horde Base CLI tools like horde-db-migrate and hordectl
  • horde/base portal blocks and admin area
  • horde/components developer tool
  • horde/turba Addressbook App Reading and writing contacts in the UI
  • horde/mnemo Notes App UI and webdav
  • horde/nag Tasks Apps UI, webdav, caldav
  • horde/kronolith Calendar App UI, webdav, CalDAV
  • horde/passwd Password App – Changing passwords worked with the hordeauth driver
  • horde/gollem File Manager App – very limited testing so far
  • horde/imp Webmail – very limited testing.

I run on a setup with openssl3 and a recent mariadb against dovecot and postfix. You can also consume the openSUSE 15.4 based containers built nightly. There is still considerable log spam from deprecation notices: Mostly tentative return types and signatures, also some use of deprecated functionality like strftime. Each night a few of these disappear. They don’t stop you from running horde apps.

I also have an eye on PHP 8.2 compatibility – So far, there should not be too many surprises. I also check most unit tests against the development version of PHPUnit 10.

This code is quite solid on PHP 7.4 – production users run on it.
On PHP 8.1 I consider it ready for adoption tests. Breakage is expected, feedback is welcome. Be sure to have a backup of the database and of any mail accounts you connect to it.
There is a lot to be done over the next few weeks.

If it does not run for your combination of drivers, please contact me via the horde mailing list.

Known caveats:

  • imp config SHOULD have an explicit cache setting: Set it to false to disable caching or to ‘cache’ to use Horde’s default cache. The ‘sql’ option also seems to work but I do not recommend it.
  • The RPC interface has seen very little testing. The json-rpc protocol should work. I have no desire to look into xmlrpc though unless somebody voices his needs. Beware, the xmlrpc extension has moved out of mainstream into pecl.
  • I do not have the necessary setup to comment on ActiveSync currently
  • Kolab integration is very likely broken. I don’t think anybody really uses recent horde with ancient kolab versions.
  • Most likely the SyncMl and PHPGroupware drivers are useless. If anybody really uses that bridge, please give feedback
  • I usually test against sabre/dav 4.4 – if you use anything else and see bugs, let me know
  • I don’t currently test against postgresql. MariaDB, MySQL, PerconaDB should work.
  • As PHP’s LDAP extension has moved from resources to objects, the LDAP authentication and addressbook drivers likely need an update. I do not currently test against LDAP but this is something I want to change
  • I know my former colleagues run LDAP and Redis so likely they will give some feedback in that area – Cannot comment on the timeline. I will offer a redis option for the maintaina container setup soonish.