Always close to the source
Install git and github cli through winget to avoid using a browser or manually entering the microsoft store tool. Install vscode and WSL if you need it.
winget install -e --id GitHub.cliwinget install --id Git.Git -e --source winget
## if visual studio code is your thing
winget install vscode
winget install Microsoft.WSLIssue: You can use FTDI based USB devices such as microcontroller boards and embedded products in Ubuntu Linux on physical devices and on full flegded VMs but under WSL2 the device does not work. You can successfully map the USB device in usbipd and you see it show up in lsusb, dmesg, journal. But no new serial device shows up under /dev/tty* and you cannot use the device.
Background: Ubuntu under Windows 11 WSL2 as of 2024/04 does not include the ftdi-sio driver in its default kernel builds. You cannot easily install that driver from the package manager either. Under WSL, similar to container environments and solaris zones, all linux guests use the same kernel.
Solution: Build and install a custom linux kernel with builtin FTDI drivers. At the time of writing, the default WSL kernel is 5.15 and the latest WSL branch is 6.1
Enter the Ubuntu WSL environment:
# If ubuntu is your only WSL or your default WSL, simply type "wsl"
# Double check with wsl -l if your ubuntu has another name like Ubuntu-22.04
# Mine is just called "Ubuntu"
wsl -d UbuntuNow download Microsoft’s modified kernel distribution and build it
git clone https://github.com/microsoft/WSL2-Linux-Kernel.git --depth=1 -b linux-msft-wsl-6.1.y
sudo apt update
sudo apt -y install build-essential flex bison libssl-dev libelf-dev bc
sudo apt -y install python3 pahole
cd WSL2-Linux-Kernel
## This line turns the FTDI driver from disabled or module to builtin
./scripts/config --file Microsoft/config-wsl --set-val CONFIG_USB_SERIAL_FTDI_SIO y
make -j$(nproc) KCONFIG_CONFIG=Microsoft/config-wsl
sudo make modules_install headers_install
## Use another target directory if you don't want the kernel in the root of C:\
cp arch/x86/boot/bzImage /mnt/c/bzImage-6.1
Leave the VM.
Back in Windows Terminal, run
wsl --shutdown
cd ~
notepad .wslconfig
Add or edit in the file:
[wsl2]
kernel=C:\\Users\\yourusernamehere\\bzImage-6.1
memory=4GB
Save the file.
Then run “wsl” or “wsl -d yourdistribution” to restart into the new kernel
To verify inside wsl, run
uname -a
## Expect output similar to
## Linux W-PF384K2W 6.1.21.2-microsoft-standard-WSL2+ #1 SMP Tue Apr 2 22:11:36 CEST 2024 x86_64 x86_64 x86_64 GNU/Linux
Note: In some cases, Windows will not actually terminate WSL and you need a full reboot of the host windows computer.
Over the last few weekends, Horde 6 code has been merged back from the Maintaina fork and from separate contributions to the former Horde development version, “master”. It was time to upgrade the development tool chain.
Back in the Horde 5 days, there was a utility called git-tools developed by Michael Rubinsky. It would checkout all horde library and application repositories of the github organization into a development directory – also called the git tree – and link into into another structure that resembled a regular Horde 5 installation – called the web tree. This required to hardcode some configurations what could otherwise be auto-detected but it allowed to organize the horde code base in a developer friendly flat structure and provide a ready to run test scenario that reflected the latest code changes even before they were committed to the official repositories. In the days of composer, half of what git-tools does is already covered by composer itself. The git-tools shell already had an integration for the horde-components tool which manages releases, change log, housekeeping tasks and some more.
It was obvious that I wanted a similar functionality for the composer-based setups. It took a while to get this right. The good news is I achieved that. Bear with me for another paragraph of incremental learning or skip to the next section with the results.
Originally I integrated a generator for “vcs” type repositories into the composer.json generation code. But these vcs repositories are relatively slow. This is OK for one or two apps or leave packages but it’s really really slow when you have a list of 80 or 120 dependencies each pulled from a separate vcs repository.
The next iteration involved generating a satis repository as a standin to releasing code into packagist. The satis repository would be updated by any commit or accepted pull request in any of the serviced repositories. Generating the satis repository is not particularly fast but I managed to scope refreshs to the individual component that needed updating. Reading from satis however is way faster than the vcs approach. Keeping an installation up to date with the latest development commits became much more viable. I also figured I could replace an already installed dependency with an actual git repository and composer would update the autoloader as needed. This works as long as you don’t change the autoloader rules, i.e. upgrade a package from PSR-0 to PSR-4.
Checking out individual libraries as root components and doing integration tests in the satis setup worked OK when the focus was on isolating individual pain points and solving bugs. For any development spanning changes on multiple libraries, it did not work out too well. I ended up implementing a new command.
horde-components github-clone-orgThis will checkout a developer tree of git repositories containing apps, libraries or themes.
total 752
drwxr-xr-x 188 root root 4096 Jan 12 11:04 ./
drwxr-xr-x 4 root root 4096 Dec 4 17:44 ../
drwxr-xr-x 9 root root 4096 Jan 11 07:50 ActiveSync/
drwxr-xr-x 9 root root 4096 Jan 11 07:50 Alarm/
drwxr-xr-x 8 root root 4096 Jan 11 07:50 ApertureToAnselExportPlugin/
drwxr-xr-x 9 root root 4096 Jan 11 07:50 Argv/
drwxr-xr-x 9 root root 4096 Jan 11 07:50 Auth/
drwxr-xr-x 7 root root 4096 Jan 11 07:50 Autoloader/
drwxr-xr-x 7 root root 4096 Jan 11 07:50 Autoloader_Cache/
drwxr-xr-x 7 root root 4096 Jan 11 07:50 Backup/
drwxr-xr-x 7 root root 4096 Jan 11 07:51 Browser/
drwxr-xr-x 9 root root 4096 Jan 11 07:51 Cache/
drwxr-xr-x 9 root root 4096 Jan 11 07:51 Cli/
drwxr-xr-x 8 root root 4096 Jan 11 07:51 Cli_Application/
drwxr-xr-x 8 root root 4096 Jan 11 07:51 Cli_Modular/
drwxr-xr-x 8 root root 4096 Jan 11 07:52 Compress/
drwxr-xr-x 8 root root 4096 Jan 11 07:52 Compress_Fast/
drwxr-xr-x 8 root root 4096 Jan 11 07:52 Constraint/
drwxr-xr-x 8 root root 4096 Jan 11 07:52 Controller/
drwxr-xr-x 11 root root 4096 Jan 11 07:52 Core/
drwxr-xr-x 8 root root 4096 Jan 11 07:52 Crypt/
drwxr-xr-x 7 root root 4096 Jan 11 07:52 Crypt_Blowfish/
drwxr-xr-x 6 root root 4096 Jan 11 07:52 CssMinify/
drwxr-xr-x 8 root root 4096 Jan 11 07:52 Css_Parser/
...
drwxr-xr-x 7 root root 4096 Jan 11 08:17 Xml_Element/
drwxr-xr-x 7 root root 4096 Jan 11 08:17 Xml_Wbxml/
drwxr-xr-x 7 root root 4096 Jan 11 08:17 Yaml/
drwxr-xr-x 14 root root 4096 Jan 11 07:50 agora/
drwxr-xr-x 17 root root 4096 Jan 11 07:50 ansel/
drwxr-xr-x 21 root root 4096 Jan 11 07:51 base/
drwxr-xr-x 13 root root 4096 Jan 11 07:51 beatnik/
drwxr-xr-x 7 root root 4096 Jan 12 21:33 bundle/
drwxr-xr-x 12 root root 4096 Jan 11 07:51 chora/
drwxr-xr-x 15 root root 4096 Jan 13 16:54 components/
drwxr-xr-x 11 root root 4096 Jan 11 07:52 content/
drwxr-xr-x 2 root root 4096 Jan 11 08:02 dev.horde.org/
drwxr-xr-x 3 root root 4096 Jan 11 08:02 dns/
drwxr-xr-x 15 root root 4096 Jan 11 08:03 folks/
drwxr-xr-x 6 root root 4096 Jan 11 08:03 git-tools/
drwxr-xr-x 12 root root 4096 Jan 11 08:03 gollem/
drwxr-xr-x 9 root root 4096 Jan 11 08:03 groupware/
drwxr-xr-x 13 root root 4096 Jan 11 08:03 hermes/
drwxr-xr-x 6 root root 4096 Jan 11 08:08 horde-installer-plugin/
drwxr-xr-x 7 root root 4096 Jan 11 08:08 horde-support/
drwxr-xr-x 9 root root 4096 Jan 11 08:10 horde-web/
drwxr-xr-x 10 root root 4096 Jan 11 08:10 hylax/
drwxr-xr-x 7 root root 4096 Jan 11 08:12 iPhoto2Ansel/
drwxr-xr-x 14 root root 4096 Jan 11 08:11 imp/
drwxr-xr-x 14 root root 4096 Jan 11 08:12 ingo/
drwxr-xr-x 3 root root 4096 Jan 11 08:12 internal/
drwxr-xr-x 15 root root 4096 Jan 11 08:12 jonah/
drwxr-xr-x 12 root root 4096 Jan 11 08:12 klutz/
drwxr-xr-x 13 root root 4096 Jan 11 08:12 kolab_webmail/
drwxr-xr-x 9 root root 4096 Jan 11 08:12 koward/
drwxr-xr-x 18 root root 4096 Jan 11 08:13 kronolith/
drwxr-xr-x 13 root root 4096 Jan 11 08:13 luxor/
drwxr-xr-x 17 root root 4096 Jan 11 08:13 mnemo/
drwxr-xr-x 18 root root 4096 Jan 11 08:14 nag/
drwxr-xr-x 9 root root 4096 Jan 11 08:14 operator/
drwxr-xr-x 13 root root 4096 Jan 11 08:14 passwd/
drwxr-xr-x 11 root root 4096 Jan 11 08:14 pastie/
drwxr-xr-x 10 root root 4096 Jan 11 08:15 sam/
drwxr-xr-x 13 root root 4096 Jan 11 08:15 sesha/
drwxr-xr-x 11 root root 4096 Jan 11 08:15 shout/
drwxr-xr-x 13 root root 4096 Jan 11 08:15 skeleton/
drwxr-xr-x 8 root root 4096 Jan 11 08:16 timeobjects/
drwxr-xr-x 14 root root 4096 Jan 12 09:08 trean/
drwxr-xr-x 16 root root 4096 Jan 11 08:16 turba/
drwxr-xr-x 11 root root 4096 Jan 11 08:16 ulaform/
drwxr-xr-x 15 root root 4096 Jan 12 11:06 vilma/
drwxr-xr-x 10 root root 4096 Jan 11 08:17 webmail/
drwxr-xr-x 18 root root 4096 Jan 11 08:17 whups/
drwxr-xr-x 15 root root 4096 Jan 11 08:17 wicked/This tree does not look like a composer based installation and would not easily work in a web browser. I did not want to reinvent composer with its autoloader and other benefits so I wrapped it into an installer. This installer creates a new copy of the horde/bundle base project and registers all the other libraries as a special type of composer repository “path”.
/srv/git/horde/components/bin/horde-components -c ~/horde-testme.conf.php install
[ INFO ] Installation directory is missing: /srv/www/testme
[ OK ] Created installation directory: /srv/www/testme
Creating a "horde/bundle" project at "../../www/testme"
Installing horde/bundle (dev-FRAMEWORK_6_0)
- Installing horde/bundle (dev-FRAMEWORK_6_0): Mirroring from /srv/git/horde/bundle
Created project in /srv/www/testmeWhen running the composer install command on this prepared setup, composer will not download the horde components from packagist or github but will use your local checkout. Only external dependencies are still downloaded from the web.
# composer install
No composer.lock file present. Updating dependencies to latest instead of installing from lock file. See https://getcomposer.org/install for more information.
Loading composer repositories with package information
Updating dependencies
Lock file operations: 92 installs, 0 updates, 0 removals
- Locking horde/alarm (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/argv (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/auth (dev-FRAMEWORK_6_0 as 3.0.0alpha7)
- Locking horde/autoloader (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/browser (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/cache (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/cli (dev-FRAMEWORK_6_0 as 3.0.0alpha6)
- Locking horde/cli_modular (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/compress (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/compress_fast (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
- Locking horde/constraint (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/controller (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/core (dev-FRAMEWORK_6_0 as 3.0.0alpha17)
- Locking horde/crypt_blowfish (dev-FRAMEWORK_6_0 as 2.0.0alpha4)
- Locking horde/css_parser (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
- Locking horde/cssminify (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
- Locking horde/data (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/date (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/dav (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
- Locking horde/db (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/exception (dev-FRAMEWORK_6_0 as 3.0.0alpha4)
- Locking horde/form (dev-FRAMEWORK_6_0 as 3.0.0alpha6)
- Locking horde/group (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/hashtable (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
- Locking horde/history (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/horde (dev-FRAMEWORK_6_0 as 6.0.0alpha7)
- Locking horde/horde-installer-plugin (v2.5.5)
- Locking horde/hordectl (v1.0.0alpha4)
- Locking horde/http (dev-FRAMEWORK_6_0 as 3.0.0alpha8)
- Locking horde/http_server (dev-FRAMEWORK_6_0 as 1.0.0alpha2)
- Locking horde/icalendar (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/idna (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
- Locking horde/image (dev-FRAMEWORK_6_0 as 3.0.0alpha6)
- Locking horde/injector (dev-FRAMEWORK_6_0 as 3.0.0alpha11)
- Locking horde/javascriptminify (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
- Locking horde/listheaders (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
- Locking horde/lock (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/log (dev-FRAMEWORK_6_0 as 3.0.0alpha9)
- Locking horde/logintasks (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/mail (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/mime (dev-FRAMEWORK_6_0 as 3.0.0alpha6)
- Locking horde/mime_viewer (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/nls (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/notification (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/pack (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
- Locking horde/perms (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/prefs (dev-FRAMEWORK_6_0 as 3.0.0alpha7)
- Locking horde/routes (dev-FRAMEWORK_6_0 as 3.0.0alpha6)
- Locking horde/rpc (dev-FRAMEWORK_6_0 as 3.0.0alpha6)
- Locking horde/secret (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/serialize (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/sessionhandler (dev-FRAMEWORK_6_0 as 3.0.0alpha3)
- Locking horde/share (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/stream (dev-FRAMEWORK_6_0 as 2.0.0alpha5)
- Locking horde/stream_filter (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/stream_wrapper (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/support (dev-FRAMEWORK_6_0 as 3.0.0alpha6)
- Locking horde/template (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/test (dev-FRAMEWORK_6_0 as 3.0.0alpha7)
- Locking horde/text_diff (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/text_filter (dev-FRAMEWORK_6_0 as 3.0.0alpha4)
- Locking horde/text_flowed (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/token (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/translation (dev-FRAMEWORK_6_0 as 3.0.0alpha3)
- Locking horde/tree (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/url (dev-FRAMEWORK_6_0 as 3.0.0alpha6)
- Locking horde/util (dev-FRAMEWORK_6_0 as 3.0.0alpha8)
- Locking horde/vfs (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/view (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/xml_element (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking horde/yaml (dev-FRAMEWORK_6_0 as 3.0.0alpha5)
- Locking pear/archive_tar (1.4.14)
- Locking pear/console_color2 (0.1.2)
- Locking pear/console_getopt (v1.4.3)
- Locking pear/console_table (v1.3.1)
- Locking pear/pear (v1.10.14)
- Locking pear/structures_graph (v1.1.1)
- Locking pear/xml_util (v1.4.5)
- Locking php-extended/polyfill-php80-stringable (1.2.9)
- Locking psr/container (2.0.2)
- Locking psr/http-client (1.0.3)
- Locking psr/http-factory (1.0.2)
- Locking psr/http-message (2.0)
- Locking psr/http-server-handler (1.0.2)
- Locking psr/http-server-middleware (1.0.2)
- Locking psr/log (3.0.0)
- Locking sabre/dav (4.6.0)
- Locking sabre/event (5.1.4)
- Locking sabre/http (5.1.10)
- Locking sabre/uri (2.3.3)
- Locking sabre/vobject (4.5.4)
- Locking sabre/xml (2.2.6)
Writing lock file
Installing dependencies from lock file (including require-dev)
Package operations: 92 installs, 0 updates, 0 removals
- Installing horde/horde-installer-plugin (v2.5.5): Extracting archive
- Installing horde/util (dev-FRAMEWORK_6_0 as 3.0.0alpha8): Symlinking from /srv/git/horde/Util
- Installing horde/translation (dev-FRAMEWORK_6_0 as 3.0.0alpha3): Symlinking from /srv/git/horde/Translation
- Installing horde/exception (dev-FRAMEWORK_6_0 as 3.0.0alpha4): Symlinking from /srv/git/horde/Exception
- Installing horde/compress_fast (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/Compress_Fast
- Installing horde/cache (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Cache
- Installing horde/stream_wrapper (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Stream_Wrapper
- Installing horde/support (dev-FRAMEWORK_6_0 as 3.0.0alpha6): Symlinking from /srv/git/horde/Support
- Installing psr/log (3.0.0): Extracting archive
- Installing php-extended/polyfill-php80-stringable (1.2.9): Extracting archive
- Installing horde/constraint (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Constraint
- Installing horde/log (dev-FRAMEWORK_6_0 as 3.0.0alpha9): Symlinking from /srv/git/horde/Log
- Installing psr/container (2.0.2): Extracting archive
- Installing horde/injector (dev-FRAMEWORK_6_0 as 3.0.0alpha11): Symlinking from /srv/git/horde/Injector
- Installing horde/controller (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Controller
- Installing horde/crypt_blowfish (dev-FRAMEWORK_6_0 as 2.0.0alpha4): Symlinking from /srv/git/horde/Crypt_Blowfish
- Installing horde/url (dev-FRAMEWORK_6_0 as 3.0.0alpha6): Symlinking from /srv/git/horde/Url
- Installing horde/css_parser (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/Css_Parser
- Installing horde/cssminify (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/CssMinify
- Installing horde/text_flowed (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Text_Flowed
- Installing horde/secret (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Secret
- Installing horde/idna (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/Idna
- Installing horde/text_filter (dev-FRAMEWORK_6_0 as 3.0.0alpha4): Symlinking from /srv/git/horde/Text_Filter
- Installing horde/stream_filter (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Stream_Filter
- Installing horde/stream (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/Stream
- Installing horde/mime (dev-FRAMEWORK_6_0 as 3.0.0alpha6): Symlinking from /srv/git/horde/Mime
- Installing horde/mail (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Mail
- Installing horde/listheaders (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/ListHeaders
- Installing horde/nls (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Nls
- Installing horde/date (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Date
- Installing horde/icalendar (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Icalendar
- Installing horde/browser (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Browser
- Installing horde/data (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Data
- Installing horde/hashtable (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/HashTable
- Installing horde/db (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Db
- Installing horde/history (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/History
- Installing horde/view (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/View
- Installing horde/vfs (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Vfs
- Installing horde/tree (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Tree
- Installing horde/token (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Token
- Installing horde/text_diff (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Text_Diff
- Installing horde/serialize (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Serialize
- Installing horde/xml_element (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Xml_Element
- Installing horde/group (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Group
- Installing horde/perms (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Perms
- Installing sabre/uri (2.3.3): Extracting archive
- Installing sabre/xml (2.2.6): Extracting archive
- Installing sabre/vobject (4.5.4): Extracting archive
- Installing sabre/event (5.1.4): Extracting archive
- Installing sabre/http (5.1.10): Extracting archive
- Installing sabre/dav (4.6.0): Extracting archive
- Installing psr/http-message (2.0): Extracting archive
- Installing psr/http-factory (1.0.2): Extracting archive
- Installing psr/http-client (1.0.3): Extracting archive
- Installing horde/http (dev-FRAMEWORK_6_0 as 3.0.0alpha8): Symlinking from /srv/git/horde/Http
- Installing pear/pear (v1.10.14): Extracting archive
- Installing pear/xml_util (v1.4.5): Extracting archive
- Installing pear/structures_graph (v1.1.1): Extracting archive
- Installing pear/console_getopt (v1.4.3): Extracting archive
- Installing pear/archive_tar (1.4.14): Extracting archive
- Installing horde/template (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Template
- Installing horde/share (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Share
- Installing horde/sessionhandler (dev-FRAMEWORK_6_0 as 3.0.0alpha3): Symlinking from /srv/git/horde/SessionHandler
- Installing horde/prefs (dev-FRAMEWORK_6_0 as 3.0.0alpha7): Symlinking from /srv/git/horde/Prefs
- Installing horde/pack (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/Pack
- Installing horde/notification (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Notification
- Installing horde/compress (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Compress
- Installing horde/mime_viewer (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Mime_Viewer
- Installing horde/logintasks (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/LoginTasks
- Installing horde/lock (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Lock
- Installing horde/javascriptminify (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/JavascriptMinify
- Installing psr/http-server-handler (1.0.2): Extracting archive
- Installing psr/http-server-middleware (1.0.2): Extracting archive
- Installing horde/http_server (dev-FRAMEWORK_6_0 as 1.0.0alpha2): Symlinking from /srv/git/horde/Http_Server
- Installing horde/cli (dev-FRAMEWORK_6_0 as 3.0.0alpha6): Symlinking from /srv/git/horde/Cli
- Installing horde/autoloader (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Autoloader
- Installing horde/auth (dev-FRAMEWORK_6_0 as 3.0.0alpha7): Symlinking from /srv/git/horde/Auth
- Installing horde/alarm (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Alarm
- Installing horde/core (dev-FRAMEWORK_6_0 as 3.0.0alpha17): Symlinking from /srv/git/horde/Core
- Installing horde/dav (dev-FRAMEWORK_6_0 as 2.0.0alpha5): Symlinking from /srv/git/horde/Dav
- Installing horde/rpc (dev-FRAMEWORK_6_0 as 3.0.0alpha6): Symlinking from /srv/git/horde/Rpc
- Installing horde/image (dev-FRAMEWORK_6_0 as 3.0.0alpha6): Symlinking from /srv/git/horde/Image
- Installing horde/form (dev-FRAMEWORK_6_0 as 3.0.0alpha6): Symlinking from /srv/git/horde/Form
- Installing horde/argv (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Argv
- Installing horde/horde (dev-FRAMEWORK_6_0 as 6.0.0alpha7): Symlinking from /srv/git/horde/base
- Installing horde/yaml (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Yaml
- Installing horde/cli_modular (dev-FRAMEWORK_6_0 as 3.0.0alpha5): Symlinking from /srv/git/horde/Cli_Modular
- Installing horde/hordectl (v1.0.0alpha4): Extracting archive
- Installing horde/routes (dev-FRAMEWORK_6_0 as 3.0.0alpha6): Symlinking from /srv/git/horde/Routes
- Installing horde/test (dev-FRAMEWORK_6_0 as 3.0.0alpha7): Symlinking from /srv/git/horde/Test
- Installing pear/console_color2 (0.1.2): Extracting archive
- Installing pear/console_table (v1.3.1): Extracting archive
81 package suggestions were added by new dependencies, use `composer suggest` to see details.
Generating autoload files
Applying /presets for absent files in /var/config
Looking for registry snippets from apps
Writing app configs to /var/config dir
Linking app configs to /web Dir
Linking javascript tree to /web/js
Linking themes tree to /web/themes
1 package you are using is looking for funding.
Use the `composer fund` command to find out more!This composer environment works just like a regular installation. When you install turba, kronolith or passwd through composer, it will end up linking these apps and their library dependencies from the development tree.
Composer provides an option to copy files from the repositories rather than link the files. This would allow creating archives with artifacts for distribution packaging. The horde-components tool does not yet provide a switch to generate the necessary tweak to the repository files.
Install a recent ubuntu/apache/mod-php setup into your local Windows WSL to get going.
(This is a dump – It will involve into an article someday)
Assuming you have installed WSL2 for Windows 10 or Windows 11
This is Ubuntu for WSL in the MS Store: Ubuntu – Microsoft Apps
Install via CLI or Store UI
Run Ubuntu in WSL: wsl -d Ubuntu
Now Upgrade, add the php repo, install some software
apt update ; apt dist-upgradesudo add-apt-repository ppa:ondrej/php
sudo add-apt-repository ppa:ondrej/nginx
apt install php8.2 php8.2-fpm php8.2-curl php8.2-dom composer gh php8.2-mbstring php8.2-sockets nginx
Add nginx config for passing php-fpm and for recognizing .php
File: /etc/nginx/sites-available/base.php82.horde
server {
listen 80;
root /var/www/base.php82.horde/web;
index index.php index.html index.htm;
server_name base.php82.horde;
location / {
try_files $uri $uri/ =404;
}
location ~ \.php$ {
include snippets/fastcgi-php.conf;
fastcgi_pass unix:/run/php/php8.2-fpm.sock;
}
}
Run these commands to activate PHP for our horde testing environment:
cp -ar /var/www/base.php82.horde/vendor/horde/horde/config/conf.php.dist /var/www/base.php82.horde/var/config/horde/conf.php
cd /var/www/base.php82.horde/
Open your Windows Notepad in “Administrator” mode and edit file C:\Windows\System32\Drivers\Etc\Hosts
Add new lines at the bottom and save
127.0.0.1 base.php82.horde
127.0.0.1 components.php82.horde
By now you can enter into your browser: http://base.php82.horde and get to a message saying
Please create a /var/www/base.php82.horde/var/config/horde/conf.php file and then run ‘composer horde-reconfigure’ to activate Horde
Start with a default horde config
cp -ar /var/www/base.php82.horde/vendor/horde/horde/config/conf.php.dist /var/www/base.php82.horde/var/config/horde/conf.php
cd /var/www/base.php82.horde/
composer horde-reconfigure
It’s been a while. It’s been much too long actually. Let’s look forward though.
I recently started to onboard into FOSS development again. People have been asking for PHP 8.2 support with Horde but capacity to deliver that was very limited. But we’re getting there.
I used the “Maintaina” fork to deliver fixed and upgraded customized versions of Horde beyond what was allowed in Horde’s master branch. It’s time to move forward though. Maintaina used to target PHP 7.4 to PHP 8.1, composer2 based install from a custom satis repo and some quite invasive changes to CalDAV/CardDAV support. Most prominently, Maintaina introduced library level compatibility with several PSRs (container, http middleware, logging) and the all new horde/http_server component.
At the moment I am importing the FRAMEWORK_6_0 branches and the alpha releases based on this branch to horde’s upstream repository. So far I have imported enough libraries to make the Horde Base application barely install from packagist.
Your magic carpet is:
composer create-project horde/bundle .
Originally I wanted to mimic maintaina’s setup with a separate satis server as a QA stage before release to packagist. I re-considered and dropped the separate satis server. Every update of the development branches and every tagged release is consumed via packagist. I will have to fix some of my tools and workflows to reflect that. The tagged alpha releases still use maintaina’s satis server. The upcoming releases won’t.
I leave the FRAMEWORK_5_2 and master branches mostly untouched. FRAMEWORK_6_0 is the new default branch on github for anything I am handling. I only edit other branches if they block packagist’s import. Usually I also rebase FRAMEWORK_6_0 on any latest commits of master, but in some cases I only cherry-picked from master branch. In some cases, some entries of the changelog between 2022 late and now (october 2023) might be missing. Pull Requests, bug reports and patches welcome.
Before I move forward with the actual applications, I want to make sure the necessary infrastructure is in place. I need to fix some aspects of the FRAMEWORK_6 version of composer. The workflow files in each repo need some review, too. Does PHPUnit and PHPStan still pass? Can we improve management of 130+ repo’s workflows?
Finally https://dev.horde.org should index from packagist, not from the satis server we originally planned to use.
The maintaina repos have had direct contributions from some trusted maintainers from the company I used to work for. They service some customers out of these repos and the related SATIS server so I won’t actively break it. However, with the move to Horde upstream, maintaina has served its purpose for me and I will not actively support the fork anymore. I suggest once everything is ported to upstream, maintaina should be archived. I will need to consult other stakeholders of this fork and the satis server.
Over the lifetime of the fork I have explored and applied various strategies for keeping the effort in check. Still, a fork of 100+ repos and the accompanying infrastructure for testing and deploying is a major burden which detracts from actually developing and maintaining code. I am glad I can save on this now and actually contribute to Horde directly in a way that doesn’t slow down activity too much. We now have the chance to speed up the cycle of feedback and releases. I hope this attracts some occassional and regular contributors.
ssh-agent does not persist over sessions in WSL2 with current openSUSE Linux and other common distributions. Linux native bashrc solutions don’t work here.
Let’s use keychain instead
zypper in keychain
Then let’s add some snippet to .bashrc and remove any eval ssh-agent lines
vim .bashrc
# Once per key to load by default
/usr/bin/keychain -q --nogui $HOME/.ssh/id_rsa
/usr/bin/keychain -q --nogui $HOME/.ssh/id_host_www.ralf-lang.de
# Only once
source $HOME/.keychain/$HOST-shThen open a new window. It will ask you to enter your passphrase once. Enter any additional windows, it won’t ask you
My 2023 started in a bad way. I did not attend to any non-work twitter, email or other communication and FOSS work to deal with that. Don’t worry, I will catch up soon. Sorry for any delays and inconveniences.
Satis is the lightweight, static repository generator for the composer package manager. It enables admins to provide PHP software packages inside airgapped CIs, OS packaging environments and restricted data centers.
Back in August I added a plugin mode to satis to make it work as a regular composer plugin. While working on it, I also fixed some preexisting issues found by static analysis and made it compatible with the recent composer versions 2.3 and 2.4.
This week, the upstream maintainers merged my contribution. I feel a bit satis-fied 😉
When looking under the hood, it is easy to see that satis always has been some awkward kind of plugin or extension. It literally sits on top of an internal copy of composer. It hooks into its internals in not quite transparent ways, it uses its class and interface organs for its own vital functions. You might call it a parasite that attaches to composer’s body for its own needs. There are downsides to this approach. The first is that you need a private copy of composer. The second is that any refactoring of composer internals likely breaks satis compatibility. That happened some time ago when composer 2.3 and 2.4 were released and not for the first time. Composer has a maturing plugin API with nice, well-defined integration points. It provides some means to overload or amend core functionality but it also provides messaging between core and plugins. I only did the bare minimum work to make satis hook into the plugin API and not break the standalone mode. When installed as a dependency, package resolution will ensure that the API versions used by satis matches the API versions provided by the underlying composer.
By itself, this change provides little benefit. It is a feature enabling feature.
But as I went along, I also re-aligned satis with the latest breaking changes inside composer 2.3/2.4. This will benefit users who do not care about the whole plugin story.
With satis 3.0-dev merging this initial change, the next steps are obvious, but not urgent.
Making the new plugin mode play nice with the latest composer was already easier than fixing the standalone mode. Satis still has an internal, now updated dependency copy of composer which is only run in standalone mode.
Standalone mode should be refactored to be just a thin wrapper around composer calling into its satis plugin. Keeping intrusion into composer internals to the bare minimum to hide the builtin commands and re-brand it as satis, this would make breakage on upcoming updates much less likely. Eventually, we can maybe stop carrying around a source code copy of composer at all.
Finally, there is reaping the benefits. I want to leverage composer/satis functionality inside the horde/components tool. Rolling out new versions of horde stuff could be so much easier.
It’s bad but not as bad as you think.
The way we write dates is very different among cultures and technologies. Even countries of the same language family might have totally different notions where to put the year, where to put the month, where to put the day when writing down a date in numbers. Apart from the order, we might use hyphen -, dot . or slash / to mark the sections. Leading zero yes or no. It gets much worse when writing out a date, like “Vlad was born on the 13th of June” or “Monday, October 10 I will have a barbecue”. Mix in different languages and different ways to abbreviate Monday as “Mo”, “Mon”, “M”.
The PHP-native ways to deal with date math and date formatting are a little quirky and best wrapped in some code that hides away the nasty details but exposes useful capabilities.
The horde/date library has been around for ages and it does its job. Due to changes in the PHP 8.1+ engine, it requires some internal re-engineering. As it is also due for a conversion to namespaces and PSR-4, I decided to think about the interface a little more. These are rough ideas and I am still figuring what will be fun and safe to work it and what is possibly over-thinking it.
PHP provides internal data types to store and manipulate a date: DateTime and DateTimeImmutable. Horde_Date behaves similar to DateTimeImmutable but implements a lot of intricate math on its own. Still, it uses DateTime for some conversions. It makes sense to delegate most mutation math to the DateTimeImmutable class and make it also hold most state. No need to manually manage minutes, hours, days of weeks etc in most cases. Preferring Immutable safes us some clone magic.
There are two possible approaches to dealing with the builtin type: Either extend it or hide it in a wrapper. The third option would be to implement the interface but this is not possible.
Extending any of the two DateTime types would be handy as it is the lingua franca between different library ecosystems. We could simply inject our Horde Date type into any library that uses it. There are down sides to this approach though. While DateTimeInterface is reasonably small and mostly useful, DateTime and DateTimeImmutable expose a lot of functionality. Some of it is awkwardly named. Some of it would have to be wrapped in extending to avoid inconsistencies in our own object. Some of it possibly clashes with own naming schemes and blocks us from using preferred signatures. Some of it might not fit into our own notion of what belongs where. There is also a risk of exposing different functionality based on PHP versions. This is undesirable. Extending is not the way to go. Hiding away the DateTimeImmutable object and exposing it explicitly might seem a little verbose. It offers some interesting applications.
A clock date type always emits now. Asking it again at a later point will yield a later time without having to manipulate the object. This is useful for tracing duration of processes or for emitting status messages. We can also make this clock the second element of a time span. Its duration will automatically expand. We can add a Stop method to the clock which will return a regular, fixed date for further processing.
Sometimes we cannot rely on a date being present in the input. It might be optional. It might be required by the current data model but used to be optional or malformed. Traditional options would be expressing the non-date as null value, throwing an exception or silently assuming no date means now. These are appropriate, good solutions in many cases. Sometimes you may prefer to have the “no date” information behave a little like a date – until it reaches a point where it cannot. For example, a “not a date” can be formatted many ways. It can be serialized to the backend, provided the backend can deal with it. It can however not be calculated, mutated or cast to a DateTimeImmutable. I am not yet sure how to handle this. Maybe it should be confined to Formatters and Readers.
PHP provides an own finite list of Time Zone identifiers. Wrapping PHP’s timezone objects allow dealing with well known but unsupported timezone names. We can map them to known names. We can safe the originally provided name for later usage. We might not carry a PHP timezone at all but signal the other parts of the library that some custom code must be applied.
There are a plethora of ways to express a given date. There are three builtin date renderers in PHP, IntlDateFormatter (not always installed), strftrime (deprecated) and DateTimeInterface::format (English only). You might add your own. Each has its own dependencies, arguments, requirements.
It is much simpler if there is a Formatter type. Implementations can just configure it and load it with a date. Consumers have a simple interface to work with them regardless of how they are implemented.This also allows to keep the dependency footprint of the core date library low and makes adding more output formats very easy. The same is true for reading data. Reading values from various formats should not be the Date object’s concern. Another object should turn arbitrary string, integer or other data into dates – including legacy Horde_Date objects.
Regular readers of this blog and many other are aware that PHP 7.4 will stop receiving security updates when PHP 8.2 comes out in November. This has made many horde admins question if they can continue to run Horde. Some events in life have made progress slower than originally planned. So where are we?
Confirmed running under PHP 8.1 and composer 2.4
I run on a setup with openssl3 and a recent mariadb against dovecot and postfix. You can also consume the openSUSE 15.4 based containers built nightly. There is still considerable log spam from deprecation notices: Mostly tentative return types and signatures, also some use of deprecated functionality like strftime. Each night a few of these disappear. They don’t stop you from running horde apps.
I also have an eye on PHP 8.2 compatibility – So far, there should not be too many surprises. I also check most unit tests against the development version of PHPUnit 10.
This code is quite solid on PHP 7.4 – production users run on it.
On PHP 8.1 I consider it ready for adoption tests. Breakage is expected, feedback is welcome. Be sure to have a backup of the database and of any mail accounts you connect to it.
There is a lot to be done over the next few weeks.
If it does not run for your combination of drivers, please contact me via the horde mailing list.
Known caveats: