bookmark_borderCode Generators: Bad, worse & ugly

Code generators have been invented and forgotten at least four times in software history. They have an appeal to developers like the sun to Daedalus’ son. Let’s not be Icarus, let’s keep them generators at a distance and watch them carefully.

Whenever a language, framework or paradigm forces developers to do the same thing over and over and over and over again they will try to get rid of that repetition. One approach is to automate writing code. It is not pretty but it saves time to concentrate on more interesting and useful things. Seasoned and reasonable software developers have resorted to that solution and many inexperienced have followed. Outside very narrow use cases I think generated code should better not exist at all.

Valid use case

Generated code and other kinds of boilerplate code are valid where avoiding them is not practical. This is often true for entry points. Depending on language, it might not be trivial for a the first piece of running code to find

  • its configuration
  • its collaborators, base classes, dependencies
  • useful data from previous runs

I have written a long, long piece on two-stage autoloaders and other two-stage bootstrapping topics and I keep rewriting it, breaking out separate topics. It is still unreleased for a reason. Any two-stage process that splits automated detection or definition of artifacts from the production run that uses them is essentially code generation. Avoiding it might be possible but impractical. Some level of repetition cannot be avoided at all and is best automated.

Another valid use case is generating code from code or transpiling. Nothing is wrong with that.

Unfortunate Use Cases

There are other use cases that should be avoided. Your framework follows convention over configuration so making magic work requires having some artifacts in the right places. Even if they have no natural place in your specific solution they are needed for technical reasons so you copy/paste or auto-generate the minimum sufficient implementation and make it fit. This is something to look for. Often there are ways around it. Another case is limits of the underlying language. You evolved from using magic properties and methods to implementing type safe, explicit equivalents but now you have to re-invent the type specific list or container type and you automate it. Bonus points if your ORM tool requires it. If your language does not support generics or another templating method, you are stuck between repetitive, explicit code and weakly typed magic. You end up using a code generator. Hopefully at one point somebody is annoyed enough and ventures to bring generics into your language. That would be the better solution but it is likely out of scope for your day to day work.

Stinking unnecessary use cases

Beyond that you are likely in the land of fluff where things just happen while they can and lines of code are generated just because it is customary. This is a foolish thing best avoided. Granted, automating code is better than hand-writing it. It does however not mean the code should exist at all. If you have no specific reason to repeat code, it is likely a design smell. This is not new, the Cunningham wiki had this thought a decade or more ago. Likely they were not even the first to recognize it. Refactoring, abstraction, configuring the specifics can help reduce the necessity for repetitive code.

My programming tools were full of wizards. Little dialog boxes waiting for me to click “Next” and “Next” and “Finish.” Click and drag and shazzam! — thousands of lines of working code. No need to get into the “hassle” of remembering the language. No need to even learn it. It is a powerful siren-song lure: You can make your program do all these wonderful and complicated things, and you don’t really need to understand.

Ellen Ullman: The dumbing-down of programming, Salon, May 1998 https://www.salon.com/1998/05/12/feature_321/

Let us take the input to a code generator and make it the input to abstracted, ready to run code instead. We will know when it is not practical, not performant or not possible. Then code generation is a blessed thing. Otherwise it is a sin.

bookmark_borderTurba Addressbook (II) – Architecture

Welcome back to our mini series on Turba.

Part I covered all the features and integrations provided by Turba.
Part II gives a dive into implementation, code structure etc.
Part III will consist of proposals for a changed architecture.

In the first chapter we looked at Turba’s features, APIs, Protocols. In the current installment, I want to present the concepts and structure of the code.

Turba is among the oldest horde applications. As such, it contains parts from various stages of Horde’s development.
Basically, it’s a layered architecture, but not fully fleshed-out or fully separated.

  • Presentation layer
  • Application Logic Layer
  • Storage/Backend layer

This is plugged together with some framework-provided integration points

  • with the sync services and inter-app API, RPC
  • with the portal/blocks service
  • with the Backup API
  • with the Content Tagger

Presentation Layer.

Turba provides both a desktop UI and a mobile UI. The following is mostly about the desktop UI. Blog de culturismo total fitness de lee hayward: debes probar estos entrenamientos musculares híbridos primobolan depot inicio – culturismo femenino y control de la natalidad, entrenamientos de culturismo femenino youtube – la casa de juegos del kama sutra negro.

Turba’s UI is organized into client pages rather than using a controller/route approach. This means, user visible URLs include files with a .php suffix.
The client pages build the UI, but are also API endpoints in a very traditional sense, catch interaction from forms or buttons as get variables and trigger actions in the backend
A representative example:

https://github.com/horde/turba/blob/master/search.php#L155
try {
$share = Turba::createShare(strval(new Horde_Support_Randomid()), $params);
$vid = $share->getName();
} catch (Horde_Share_Exception $e) {
$notification->push(sprintf(_("There was a problem creating the virtual address book: %s"), $e->getMessage()), 'horde.error');
Horde::url('search.php', true)->redirect();
}

Typically the top part of each client page is initializing the application, catching request and environment/session variables, setting up the business objects.
The middle part usually orchestrates actions depending on present or missing parameter scenarios.

The lower part will actually output the Horde Topbar, utility javascript, and the actual page content.

A rather extreme example is the data import/export part: https://github.com/horde/turba/blob/master/data.php
It contains mappings, attribute filtering, a longer cascade of if’s and switch statements and after about 360 lines, the actual UI logic starts.

This might sound messier than it really is. Actual functionality is mostly factored out into separate classes and the UI uses both some View classes and a form library.
Turba’s UI is heavy in forms and tables for the very reason that editing and displaying a highly configurable addressbook with hundreds of fields of data is
very crud-like by nature.

The Turba UI uses three types of helpers to compose the UI:

  • View classes
  • The Forms library
  • HTML templates with PHP snippets

View Classes

The View classes are very similar to the horde/view library and replicate some of its functionality, but are not using or inheriting from it.

Forms Library

Turba is a prominent user of the horde/forms library.
This utility allows to dynamically compose forms or multiple fields, check for internal/formal validity of entered values, missing mandatory values, etc.
It couples both a readonly and editable representation with a lot of processing logic.
Forms relies mostly on server-generated HTML with small parts of javascript injected for usability improvements.
The JavaScript snippets may utilize PrototypeJs and Scriptaculous, two formerly popular mainstream libraries.

Templates

The HTML templates provide most of the actual presentation apart from the forms – though they also provide some HTML forms to drive interaction.

Mobile View

Turba provides a read-only, touch friendly mobile phone presentation based on jQuery Mobile. This is completely separate from the rest of the turba UI.

Application Logic Layer and Backend Layer.

These two layers are closely tied together so it makes sense to discuss them as one.
From a problem domain perspective, it would make sense to expect these items:

  • Multiple addressbook sources or backends. These are individual configurations using drivers. Multiple sources on the same ldap driver can represent different directories or different views on one.
  • Addressbooks
  • Addressbook entries or contacts
  • Groups which are both entries and contain entries.

Turba’s logic layer consists of

  • Representations of actual addressbook entries via the Object and Object_Group classes.
  • A collection of reusable static functions in the Turba class.
  • Parts of the base backend driver.
  • Exporters/Importers for formats like LDIF and vCard.
  • specialised forms which deal both with presentation and state transformations.

Turba Objects and Groups are the common ground here. The objects glue together the actual data from the driver, files from VFS, permissions managed by the driver, the object’s change history and tags in the tagger app.
Groups are the only subtypes of Objects or Entries. Other subtypes suggested by the vcard standard like organisations or locations get no special treatment. Groups act as virtual addressbooks or views on addressbook data.
Turba’s groups only work with turba-accessible contacts. They cannot reference external contacts from other sources.

The addressbooks don’t really show up as entity objects. They are arrays of passive data managed by different parties.
This makes part of the logic a little hard to reason about and to setup unit tests.

A lot of Turba’s logic is data transformation. Backends have a native representation of data as SQL columns, LDAP objects, etc… as well as possibly native key names.
A person’s name may be a column “object_lastname” in one backend, but an attribute “givenName” in another one.

This is complicated by a highly configurable list of fields each backend can hold.
A driver transforms these native formats into a uniform format, transform date strings into date objects, handle blobs etc…
Both the driver dependent format and the “turba” format are hashes. It is up to the driver to actually generate a list object containing individual addressbook entry objects.

Another functionality delegated to the driver is deriving TimeObjects from addressbook data. TimeObjects are really not objects but hashes, representing anniversaries or birthdays.

Permission management in turba is using three distinct approaches:

  • The permission system allows restricting who can view or edit a certain addressbook
  • One addressbook source, usually the SQL db, can be configured as using the Shares system. In this case, the user can delegate access to addressbooks to other users or groups or make them world readable
  • The backing technology can restrict the user further. For example, the LDAP driver can be used to either bind using a service credential or using the user’s credential. Different users could have completely different data presented based on LDAP ACLs.

In the next article of this series, I am going to propose some modernisation approaches for Turba and discuss how they bring benefits in maintaining or extending the software vs being a tedious refactoring exercise.