Skip to content

Blog

Each week, I publish a summary of the open source work I completed, feature a musical, and discuss general events.


Week of 2024-02-25

Open source

I released ansible 9.3.0 this week. I used the new release automation developed by Anwesha Das, as well as the new release announcement automation I implemented in antsibull.

I cut a new release of Fedora’s Golang RPM Macros. It addresses some longstanding issues, including the ability to pass our compiler flags to upstream Makefiles and other build scripts and removing clashes between C LDFLAGS and Go LDFLAGS. The release also include a patch from Zbigniew JÄ™drzejewski-Szmek to enable build reproducibility for Go packages. Package build IDs were randomly generated previously and are now generated in a reproducible way.

I attended the bi-monthly Fedora Go Special Interest Group meeting. We discussed migrating our repositories from Pagure to Gitlab to allow us to take advantage of Gitlab CI and Packit. The CI situation with Pagure is not great, and the upstream project’s maintenance is in question. Fedora has its own namespace on gitlab.com, and it integrates with the Fedora Account System so we can give everyone in the Go SIG group access without having to manage permissions in multiple places. Gitlab’s interface is a bit… busy, but it works well enough. I guess its ‘open core’ model is better than Github’s completely closed source model.

I started a new go-vendor-tools project to enable creating vendored Go packages in Fedora while following the bundling and licensing guidelines. That project will be moved to the new fedora/sigs/go Gitlab namespace once Infra creates it for us. The tool definitely needs more work, but so far, it can:

  • Create Go vendor tarballs in a reproducible way
  • Crawl the vendor directory for license files using askalono-cli and generate a cumulative SPDX license identifier for the project
  • Implement a TOML configuration file to mark license files that askalono is unable to detect
  • Implement RPM macros to install all the detected license files into the package’s %{_licensedir} and verify that the package’s SPDX identifier matches the package’s RPM License: tag

The package uses a—dare I say—innovative Packit setup to run integration tests for the code and RPM macro side, in particular. If it ends up working out well, I may adopt a similar setup for the main Go macros and Ansible macros projects that I maintain. That probably warrants a separate bog post. In any case, I am a big proponent of automated testing, especially for RPM macros where one small mistake has the propensity to break all 23 thousand source packages in the Fedora package collection. I have explored various approaches for RPM macro testing, including pure pytest setups, a YAML file containing test cases, and the aforementioned Packit-based integration testing.

Musical of the week

The musical of the week is Little Shop of Horrors. The musical combines humor with deeper themes: morality; death; and the consequences of fame. I saw the show Presidents Day weekend off Broadway with Darren Criss (of “Glee” fame) playing the lead, Seymour Krelborn.


Until next time,

Stay sane and stay sure

Maxwell

Reply on the Fediverse

Week of 2023-12-10 (and the other weeks I missed)

Open source

I got back to improving our developer tooling for ansible-documentation. Amongst other tasks, I integrated the docs build process into the nox configuration.

I performed Fedora distgit and ansible-documentation code reviews as usual.

As discussed in My First Ansible Release, I have been working on improving the testing for the collections in the ansible package. My PR antsibull to enable this is now ready for review! The PR allows cloning all the collections in the ansible package, running sanity tests, and collecting the results. I also created https://github.com/ansible-community/package-test-results to store the test results and coordinate filing issues against broken collections. Feedback is welcome in the forum post!

I helped create a hot fix antsibull-docs release to address a new ansible-core change that broke the Ansible documentation.

I started working on plans for the next major release of antsibull-core, the library used by the other antsibull tools.

I started working on improving sourcehutx’s documentation.

I reported an issue against dnf5 for an issue that broke fedrq’s CI.

Speaking of fedrq, I have done some work on cleaning up the API and preparing to add support for recursively resolving package queries. The Repoquery.resolve_pkg_specs interface has gained some additional arguments to enable this. I spent some time going through dnf and libdnf5’s package Goal APIs and planning out a compatibility layer between them in the same way I’ve done for the Package and Query APIs.

I have also worked on tomcli, my CLI tool for querying and modifying TOML files and cut 0.4.0 and 0.5.0 releases (changelog). I improved the documentation a bit. I added new subcommands and more flexible output formatting. I reduced the dependencies by switching from typer to click for CLI parsing and from hatchling to flit_core for the Python build backend. The goal is to make tomcli easier to bootstrap so it can be adopted more easily by baseline Python packages in Fedora. I would like to release 1.0.0 and finish stabilizing the CLI API.

I submitted a PR to Fedora’s python-click package to make it easier to bootstrap.

Musical of the week

The musical of the week is In the Heights. Lin Manuel Miranda wrote In the Heights in college long before he wrote Hamilton. The musical centers around a group of Dominican American families living in the gentrified Washington Heights neighborhood of New York City. I have managed to memorize the entire opening number. I will be seeing a local production of the musical in February. There is a movie version of the musical on (HBO) Max1.

Maxwell

Reply on the Fediverse


  1. HBO must have payed a ton of money to purchase max.com… 

Week of 2023-11-19

Open source

I released Ansible 9.0.0 and subsequent Ansible 9.0.1. It turns out, there was an error in our new setuptools configuration, so we needed to release a hot fix.

I’ve continued my work on improving QA for the ansible community package. See Testing collections within the ansible package on the Ansible Forum for more details!

I updated ansible-core to 2.16.0 and ansible to 9.0.0 in Fedora. We have had packaged pre-releases up until now to enable support for Python 3.12.

I submitted PRs to the ansible-documentation repository to fix our broken 404 page template. Thanks to @felixfontein for helping diagnose the issue!

I also updated ansible in EPEL 8 to correspond to RHEL 9.3’s ansible-core 2.15.x update.

I worked locally on some changes to tomcli to make output formatting more flexible.

Per usual, I helped with ansible-documentation and Fedora distgit PR reviews.

Musical of the week

The Musical of the Week is Spring Awakening. Spring Awakening, based on the 1891 German play of the same name, explores adolescent sexuality through the lens of repressive late 19th century Germany. The musical deals with complex subjects, including religion, suicide, sexual abuse, abortion, and coming to terms with one’s sexual identity. The “taboo” nature of these topics makes them all the more important to discuss. Spring Awakening’s leads, Melchior Gabor and Wendela Bergmann, were Jonathan Groff and Lea Michelle’s breakout roles, respectively.

Maxwell

Reply on the Fediverse

Week of 2023-11-05

I released Ansible 8.6.0 and Ansible 8.6.1. See My First Ansible Release for more details!

I submitted a couple PRs to ansible-build-data to improve the release process documentation and to antsibull to improve parts of the release playbook and fix a regression I introduced in another change 😬.

FESCo approved my Pydantic V2 change. @music and I are now working on the package reviews and other steps to integrate the new major version.

I packaged bindep and ansible-builder for Fedora. They are currently waiting in the updates-testing repository.

I also packaged flit-core for the alternative python3.11 stack in EPEL 9.

RHEL 9.3 was released this week, so I updated the ansible package in EPEL 9 accordingly. In RHEL 9.2, ansible-core was built against python3.11, but it’s now back to the default python3 (python3.9) stack. I updated ansible-core to 2.16.0 in Fedora as well.

The musical of the week is Beetlejuice.

Maxwell

Reply on the Fediverse

My First Ansible Release

My first ansible package release and reflections on the process.

Intro

This week, the Ansible community package release process was opened up to the community (i.e., non-Hatters) for the first time! This is a pretty big milestone for our community. The Ansible Community Steering Committee made decisions and policies for the Ansible package, but we were never able to handle the releases ourselves — until now. I was the release manager for this week’s Ansible 8.6.0 release and @felixfontein, another community member, took care of Ansible 9.0.0b1. As the maintainer of the downstream Fedora package and co-maintainer of our build tooling, it was great to see the process in action!

Release process overview

Note

The ansible community package contains a set of independent Ansible collections that are curated by the community, and it pulls in ansible-core. Each major version of ansible depends on a specific major version of ansible-core and contains specific major versions of the collections it includes.

The release process involves putting the appropriate collection versions together, downloading the collection artifacts, and combining them into a single Python package with the proper metadata. Along with the package, a combined changelog and porting guide are generated for all of the collections in the package. The porting guide, changelog, and other static data about the specific collection versions included in each ansible release is kept in the ansible-build-data repository.

Hiccups

I also have a newfound appreciation for the work of the Ansible release manager and am already working on process improvement for future releases. The Ansible 8.6.0 release, in particular, was not without hiccups.

We had one upstream collection that had a semantic versioning violation and another collection that published a release with syntactically invalid Python files. The latter issue broke downstream packages even for users who do not utilize the broken plugins. This is unfortunately not the first time collection maintainers have published releases with Python syntax errors. In the end, I pinned the semver-violating collection to an earlier version and had to publish an 8.6.1 hottfix release to fix the other issue.

Testing and QA issues

We generally rely on upstream collections to follow our guidelines and run CI tests before publishing their releases. However, some collections, especially the commercial ones with convoluted internal <-> external repo sync pipelines, do not seem to consistently test releases before publishing them to Ansible Galaxy 😥.

The Steering Committee and Release Management WG are discussing running some basic smoke tests of our own on top of the ansible package. To this end, I submitted a PR to add a basic Python byte-compilation test to the ansible release playbook. This should at least catch the most egregious issues.

We are also discussing running at least a subset of ansible-test sanity’s tests on top of all collections, but that presents its own can of worms. Mainly, running all of the tests for every single collection is very time-consuming and resource intensive. Also, we would need to figure out how to retrieve the collection sources. The Galaxy artifacts are not guaranteed to include all of the files needed to run sanity tests while the content tagged in the upstream Git repository may not match the Galaxy artifact’s content.

Week of 2023-10-22 and Israel statement

Open Source

The Ansible Community Steering Committee is working on revamping our voting process to use the new community forum. We previously used Github Issues for discussions and created Github Discussions with +1/-1 comments for voting that we counted manually. The plan is to switch over to the forum and use the nice poll feature to count votes for us. I participated in discussions bikeshedding about specific process details and reviewed the policy change PR.

The Go macros in Fedora have a lua function/RPM macro to generate a Go package name based on an import path by replacing / with - along with other normalization. It turns out the way that the macro generates names for compat packages is contrary to the Packaging Guidelines and has been for the past five years. @eclipseo and I discussed the situation and he submitted PRs to go-rpm-macros and go2rpm to change the naming algorithm that I reviewed.

I worked on backporting these changes to the go-rpm-macros-epel EPEL 9 backport. We have had an entirely separate package that selectively overrides macros, as RHEL ignored our bugs and PRs to fix its broken patches. I decided to switch to a source git-based structure instead of keeping code in distgit. There is now a separate epel9 distgit branch in the upstream go-rpm-macros repository.

I have continued working on the dev tooling in the ansible-docuemntation repository and fixed an issue with our pipx installation instructions.

As usual, I reviewed various distgit PRs and Ansible documentation changes.

Musical of the Week

The musical of the week is Hadestown. Hadestown recounts the Greek tragedy of Orpheus, a wide-eyed songwriter and son of a muse, and Eurydice, a tortured young girl simply searching for something to eat and a warm place to sleep. Hadestown’s jazz-infused score and terrifying industrial re-imagination of the Underworld bring the chilling tale to life. I saw the musical in Chicago last summer and quite enjoyed it other than the overly pitchy Orpheus understudy.

Israel

I’m also featuring The Decolonization Narrative Is Dangerous and False from The Atlantic that counteracts some problematic rhetoric about Israel and covers the wider context of the conflict.

I condemn the Hamas terrorists who killed, mutilated, raped, and kidnapped innocent civilians, including grandparents, pregnant women, and children. The Kibbutzniks who Hamas attacked were peaceful, left-leaning Israelis, many of whom advocated for peace, opposed the hard right government and West Bank settlement expansion, and even spoke Arabic. I also mourn the loss of innocent Palestinians in Gaza, while recognizing Israel’s right to neutralize the Hamas terrorists in a manner that minimizes civilian deaths. Hamas seeks to sabotage the peace process and annihilate the Jewish people, all while deliberately endangering its own people.

Maxwell

Reply on the Fediverse

Week of 2023-10-15 (and more!)

My migraines and depression have been quite bad the past couple weeks, so I took a little break from these posts.

After much ado, I finally submitted my Fedora Pydantic v2 Change for review. After more ado and some poking from @decathrope, the Change was finally announced. Thanks as well to @music for his help with the Change.

I have done my usual work on the ansible-documentation repository to review PRs and improve dev tooling. I started a discussion on squashing vs. merging vs. rebasing when applying Github pull requests. I worked on an improvement to the way our issue/PR triager and pip-compile Github Actions authenticate with the Github API. I reviewed a couple PRs from the community. I updated our PR triager to warn contributors when their changes modify automatically generated files.

I also reviewed a change to enable codespell in the ansible-documentation repo. One of the issues is the naive way in which the tool parses its configuration. I plan to work on a PR to improve it next week.

I worked on packaging hatch-requirements-txt for Fedora. It’s a hatchling plugin that allows sourcing package requirements from a requirements.txt external to pyproject.toml. Of course, it had a new unpackaged test dependency that in turn required an entirely new build system called whey that in turned required even more dependencies. I decided to patch the new test dependency to use setuptools…

I preformed my usual ansible, ansible-core, and ansible-collection-* package updates and assisted with a couple distgit PR reviews.

I helped resolve a situation where duplicate packages were introduced into Fedora.

I’ve contributed some minor changes to the new maubot-fedora Matrix-native Zodbot replacement.

I helped with efforts to move Ansible Community Steering Committee votes to the new Ansible Community Forum.

The musical of the week is Waitress. Waitress, based on the 2007 movie by late director Adrienne Shelly, focusses on a masterful pie baker and waitress. Jenna grapples with life in a small Southern town and an abusive husband. The musical sensitively chronicles Jenna’s journey towards self reliance and realization, as well as her dream of ending her toxic marriage and establishing her own pie shop.

Maxwell

Reply on the Fediverse

Week of 2023-09-17

I wrote a draft Update_To_Pydantic_Version_2 Change to update Pydantic to version 2 in Fedora. I need to clean it up and submit it for community feedback.

As part of this work, I submitted a PR to Matrix synapse to add compatibility for the new version. For now, it uses the pydantic.v1 compat module until pydantic v1 support can be dropped completely.

I attended the FESCo meeting to discuss an issue relating to Fedora ELN. It seems we have reached some common ground.

I attended the Ansible Docs WG meeting and helped with the ansible-documentation repository branching process.

In case you haven’t heard, the Ansible community has a new forum at https://forum.ansible.com. I started a forum group for folks who maintain Ansible-releated packages in Linux distributions. So far, we have folks from Fedora, Arch, and the Ubuntu PPA!

I helped with some changes to the rust-packaging macros to fix an issue affecting written-in-Rust Python extension modules and automate vendoring for RHEL/ELN.

I participated in miscellaneous Ansible Community Steering Committee discussions and Fedora distgit PR reviews, as usual.

The musical of the week is Les Misérables, a classic and one of my favorites. I saw it twice, once in Chicago and again in London while I was in the vicinity for Flock to Fedora. It is a sung-through musical, which I particularly like. There is a reason it is one of the longest running musicals.

Maxwell

Reply on the Fediverse

Week of 2023-09-10

I have spent a good amount of time on packaging pydantic v2 for Fedora. The current state is documented on my wiki. I have a couple more packages to fix, and I need to actually write the change proposal.

I submitted an RPM PR to add a Provides generator for RPM lua modules. This was inspired by my work on the forge-srpm-macros lift and shift and wanting a way to express a dependency on the shared forge.lua code without needing to account for the old and new package name. Panu merged it on Friday.

I worked on enabling the rust extension module and updating python-libcst in Fedora.

I reviewed a couple other distgit PRs, including one from ELN.

I cut a small release of fedrq. This release adds supoort for pydantic v2, which fedrq uses to parse its config file. It also adds an experimental --extra-exact flag that I used as part of my pydantic v2 work. I need to go back to clean up and document that feature.

The musical of the week is Come From Away. It covers the experiences of the 7000 passengers stranded in Newfoundland, Canada when their planes were diverted after the 9/11 attacks. It is a touching story of community and human kindness. The show boasts great music, innovative staging, and well placed humor. I saw it in Chicago, and there’s also a recording of the full show on Apple TV+.

Dental surgery recovery has been no fun. The whole thing has also irritated my grinding problems and migraines.

Maxwell

Reply on the Fediverse

Week of 2023-09-03

My Revitalize_Forge_Macros Fedora Change was approved! This entails moving macros.forge and forge.lua out of redhat-rpm-config into a new upstream project. The Bodhi update was already merged into rawhide and eln.

I helped review a pyproject-rpm-macros PR that adds a flag to %pyproject_check_import to allow passing the test if no importable modules are found. This will be used for automated pyp2spec builds in Copr. It’s a bit of a fun one, involving new pytest unit tests, argument processing in lua, and escaping RPM macros.

I started a discussion amongst the Ansible Community Steering Committee about how the new Ansible Galaxy codebase should name roles.

I worked on packaging pydantic-core for Fedora. See my musings on Mastodon. Thanks to Fabio for reviewing the Rust dependencies.

This week’s musical of the week is Shucked. Alex Newell is great in it, but the show is a bit kitschy.

I got dental surgery this week. I’m currently mourning solid food.

I submitted a couple PRs to the ansible-documentation repo to improve our workflow automation (lockfiles, issue/PR labeling, etc.).

I also started this little blog.

Maxwell