bauerbill

2016-11-11 01:35 UTC
  • Xyne

Metadata

Description:

Extension of Powerpill with AUR and ABS support.

Latest Version:

2016.11

Architecture:

  • any

Dependencies:

Arch Repositories:

  • [xyne-any]
  • [xyne-i686]
  • [xyne-x86_64]

AUR Page:

bauerbill

Arch Forum Thread:

205834

Tags:

About

Bauerbill is an extension of Powerpill with AUR and ABS support. It includes a number of features to automate package configuration and basic trust management. It outputs build scripts in Bash that can be further customized as needed. See the list of features and the documentation below.

Usage Examples

Do a full system upgrade with AUR support:

sudo bauerbill -Syu --aur
./build/download.sh
./build/build.sh

This is equivalent to using the wrapper script, which will be used for the rest of the examples:

bb-wrapper -Syu --aur

Full system upgrade with all upgradable packages built from source via ABS:

bb-wrapper -Syu --build-targets

Install some AUR package foo with full dependency resolution:

bb-wrapper -S --aur foo

Generate the download and build scripts for some AUR package foo without pre-installing the sync dependencies. The dependencies will be managed by makepkg when the packages are built.

bauerbill -S --aur --no-predeps foo

Rebuild and re-install powerpill from the AUR instead of my repo:

bb-wrapper -S --aur AUR/powerpill

List all packages availabe in the repos and the AUR:

bauerbill -Sl --aur

Search for all repo and AUR packages with the term "foo":

bauerbill -Ss --aur foo

Get information about powerpill from both my repo and the AUR package:

bauerbill -Si --aur powerpill AUR/powerpill

Generate build scripts with dependency resolution for multiple AUR packages without pre-emptive sync dependency installation (i.e. this can be run without root privileges):

bauerbill -S --aur --nobqd aurpkg1 aurpkg2 aurpkg3 ...

Generate build scripts with dependency resolution for multiple repo and AUR packages without pre-emptive sync dependency installation (i.e. this can be run without root privileges):

bauerbill -S --build-targets --aur --nobqd repopkg1 repopkg2 repopkg3 aurpkg1 aurpkg2 ...

Project History

Gather around children!

Powerpill was the first community contribution that I posted years ago when I began using Arch. It started as a simple wrapper script around Pacman and Aria2 but thanks to multiple issues and noobish myopy, it gradually grew to a monstruous labyrinthine ecosystem of several Perl modules that re-implemented a lot of ALPM (Pacman's backend, written in C). Had I known better and had the skills, I would have tried to create proper Perl bindings for ALPM, but nope.

The modules worked and several package-oriented utilities arose from them (e.g. Reflector). Eventually a few years ago I ended up creating the original Bauerbill which was an extension of Powerpill with AUR and ABS support. It worked relatively well and experienced modest popularity during its first run. All was well and the land knew peace and updated systems, until one fateful day when Allan truly did break everything. :P

There was a major release of Pacman that converted the sync databases to archives along with some other changes iirc, which broke critical functionality in my Perl modules. This was entirely my own fault for having designed them so poorly (read: not at all) and although I could have fudged the code even further to work around the changes, it just wasn't worth it to me and so I let that ecosystem die.

I bid my farewell to Perl, whom I had been seeing less and less as I had begun spending ever more time with Python. pyalpm and numerous package APIs emerged and I started writing code around them. Several tools appeared or reappeared: by popular demand Reflector was quickly reborn, then a series of metalink scripts resulted in pm2ml, "powerpill lite" wrapper scripts, and finally a fully functional re-incarnation of Powerpill. New AUR APIs and functionality continued to appear as well and gradually python3-aur grew out of various interface wrappers. I was slowly collecting everything needed to recreate Bauerbill but I just didn't have the time to sit down and put it all together. I finally found some time during the last few days.

Bauerbill is finally reborn!

Bauerbill Reborn

With this new version of Bauerbill I have taken a different approach that will hopefully prove both more versatile and more maintainable. Instead of trying to be a Pacman emulator that transparently wraps the AUR and ABS, this is a Powerpill extension that generates scripts to build packages. A simple wrapper script (bb-wrapper) is provided to make this transparent to the user.

Using scripts to run all fo the build commands makes the process more transparent and configurable for the user while avoiding all of the concerns of privilege management (e.g. upgrading system as root, dropping root privileges to build packages, escalating privileges to install built packages).

The scripts invoke the utility bb-query_trust which can track the user's list of trusted AUR users and packages combinations (see below).

Scripts

When run with AUR or ABS support, bauerbill will generate 2 or 3 script files in the output directory:

download.sh
Download all of the target PKGBUILDs and related files, then download the package sources in parallel. Hooks can be run per package before retrieving the PKGBUILDs or before starting the source downloads.
build.sh
Build the target packages. Hooks can be run before each package is built.
* clean.sh
Clean up leftover dependencies installed for building.

Features

The generated scripts are configurable via the Bauerbill configuration file:

  • Select makepkg commands to download sources and build packages.
  • Run custom commands from the configuration file or scripts from a "hooks" directory at different points in the build process (preget, predownload, prebuild).
  • Select per-package arguments to pass to these scripts and commands.
  • Track a list of your own trusted users to avoid prompts for package inspections.
  • Remember package releases that you have already inspected and trusted for future rebuilds.
  • Search and query the AUR.
  • List all packages in the AUR.

Note that all of these hooks and commands can be configured per package using pattern matching for versatility. See the bauerbill.json man page for details.

bb-query_trust

bb-query_trust is a tool to check if a package should be trusted for building given its maintainers and modification time. If all maintainers are in a custom list of trusted users then the package will be trusted, otherwise if the specific combination of package, maintainers and optionally the mtime has previously been trusted then it will be trusted again. If neither condition is met then the user can choose to trust the package, trust the combination indefinitely, or trust the maintainers.

If the package is not trusted then the script will exit with an error. It is used to stop execution of the build and dlownload scripts.

bb-query_trust screenshot

bb-query_trust screenshot

Why Prompt The User?

PKGBUILDs are executable scripts and are run by makepkg but building packages. They come from an untrusted source and may harm your system due to incompetence or malice. You should always check a PKGBUILD and the related files befure running makepkg.

BAUERBILL.JSON(1)

Name

bauerbill.json - Bauerbill configuration file

Description

The Bauerbill configuration file is a plain JSON file. By default it is located at /etc/bauerbill/bauerbill.conf. The main object is a dictionary that holds multiple dictionaries. The latter are considered sections of the configuration file and contain options related to different parts of Bauerbill.

Sections

Note that all fields, including section names, are in lower case in the file. Upper case may appear in the man page during automatic conversion of the markdown file. For example, the first section is makepkg commands, not MAKEPKG COMMANDS.

bin

A mapping of expected executables to their commands, e.g. "pacman": ["/usr/bin/pacman"]. These can be used to wrap other commands or enforce certain options.

data ttl

The default time-to-live for cached data.

VCS patterns

This is just a list of glob patterns to match VCS packages, e.g. ["*-git", "*-svn"].

makepkg commands

This section contains 2 subsections: build and download. These determine the makepkg commands to use for building and download packages, respectively. These commands can be configured per package using glob patterns along with a default command. Both subsections follow the same format with the following keys:

custom
This key may be null or omitted. If included, it must be a list of 2-tuples. The first item is a glob pattern that will be tested against "/" (e.g. "/reflector" will match "community/reflector" and "AUR/" will match all AUR packages). The second item is a makepkg command with arguments as a list, e.g ['makepkg', '-irs', '--config', '/path/to/custom/makepkg.conf']. This command must install the package and accept --asdeps and --pkg as possible arguments. Simple example: "custom" : [["AUR/*", ["makepkg", "-irs", "--config", "/path/to/custom/makepkg.conf"]]]
VCS
A custom command for packages that match the patterns in VCS patterns. This key may be null or omitted.
default
The command to run if no match occurs.
common arguments
Arguments that will be appended to all commands in the section.

Each command must be a list where each item is a shell "word", e.g. ["makepkg", "-irs", "--config", "/path/to/some/config"]. Each item will be interpolated using Python string formatting so curly braces must be escaped (i.e. "{" should be "{{" and "}" should be "}}"). The possible mappings are the names in BuildablePkgMapping in Bauerbill.py (e.g. ["{Name}", "{LastModified}"]) and "PacmanConfig".

Additional makepkg arguments will be added to the command as necessary when generating the different scripts, for example --asdeps or --pkg.

hooks

Hook types
preget
Run before the PKGBUILDs are download.
predownload
Run before the sources are download.
prebuild
Run before the package is build.
Keys
directory
A path to a directory with hook scripts. This directory should contain subdirectories with package base names, which in turn contain subdirectories with the names of the hook types mentioned above. For example, setting this to /tmp/bbhooks will then look for scripts in /tmp/bbhooks/linux/prebuild before building the linux package.
arguments
This follows the same format as the makepkg commands sections above (default, VCS, custom), but with lists of arguments rather than commands. The first match will return the arguments to pass to the hook scripts. The arguments can be interpolated, e.g. ["{PackageBase}", "{LastModified}"]. This section may be empty.

commands ; This also follows the same format as the makepkg commands sections above but each value is a list of lists where the inner lists are commands to be run. Their arguments will also be interpolated. This section may be empty.

bauerbill Help Message

$ bauerbill --help

USAGE
  Bauerbill.py [Bauerbill.py options] [pacman args]

OPTIONS
  Bauerbill.Py should accept the same arguments as Powerpill, e.g.

      Bauerbill.py -Syu

  See "pacman --help" for further help.

  The following additional arguments are supported:

    --aur
        Enable AUR support.

    --aur-only
        Enable AUR support and limit some operations such as system upgrades to AUR packages.

    --bb-config <BB-CONFIG>
        The bauerbill JSON configuration file.

    --bb-quiet
        Suppress warning messages for AUR operations.

    --build
        Build target packages that can be built.

    --build-all
        Build targets and dependencies.

    --build-dir <BUILD-DIR>
        The directory in which to save generated scripts.

    --build-vcs
        Rebuild all rebuildable VCS packages (e.g. foo-git, bar-hg)

    --nobqd
        (no build queue deps) Do not install calculated sync deps for build queue directly. Use this option to allow makepkg to handle all sync deps. The advantage is that the build scripts can be generated without root. The disadvantage is that some deps may be installed and removed multiple times if they are required by multiple build targets.

CHANGELOG

2016-04-04

  • Refactored to modularize code and incorporate other backend changes. Bauerbill is now a class. Architecture-specific metadata should now be correctly handled in all cases through the backend.

2016-04-03

  • Add support for checkdeps.
  • Temporarily use AUR .SRCINFO until the RPC interface includes the architecture-specific arrays.

2016-02-15

  • Improved cleanup script to avoid removing previously installed packages and to skip packages that were not installed.
  • Added makepkgx which is a patched version of makepkg that accepts the old --pkg option for selective installation of packages from a package group. makepkgx also include an option to pass a custom configuration file to pacman (--pconfig)
  • Added PacmanConfig to the list of interpolable values in the configuration file commands.
  • Installation commands should now deal with selecting the package to install. The default commands in bauerbill.json have been updated.

2016-01-22

  • Improved permission management in bb-wrapper.

2016-01-20

  • The "custom" commands in bauerbill.json are now a list of 2-tuples which are tested in order.
  • The "custom" command patterns are now matched against the package base with the repo prefix for greater control, e.g. a custom command can be used for AUR packages with ["AUR/*", ["makepkg", "-irs", "--config", "/path/to/custom/makepkg.conf"]].
  • Bauerbill now passes the prefixed package base to bb-query_trust. Given that the only currently untrusted buildable packages are from the AUR, this just prefixes everything with 'AUR/' but in the future there may be other build sources. Edit your bb-query_trust combinations file and prefix 'AUR/' to preserve prevous combinations. See bb-query_trust --help to find the file's path.

2016-01-14

  • Added support for -Qu --aur.
  • Added --bb-quiet option to suppress AUR query warnings.

2016-01-12

  • Renamed option --build-targets to --build.
  • Added option --nobqd for generating build scripts without pre-emptive dependency installation. The scripts can now be created without ever needing root.
  • Added possibility to trust maintainer-pkgbase combinations with bb-query_trust.
  • Added colorized output to bb-query_trust.

2015-12-14

  • Added --build-vcs option to select VCS packages for rebuilding.

2015-12-05

  • Removed centralized data directory for which it was difficult to manage permissions for multiple users.
  • Use MemoizeDB to temporarily cache package information queries from archlinux.org.
  • Renamed --bb-scripts-dir to --build-dir.

TODO

TODO

  • Colorized output.
  • Rebuild all installed packages.
  • Rebuild packages that are older than their dependencies (see pkg-list_anachronistic_installations)
  • Maybe make the prompts configurable.
  • Rebuild VCS packages.
  • Per-package build locations.
Contact
echo xyne.archlinux.ca | sed 's/\./@/'
Feeds
Blog News
Validation
XHTML 1.0 Strict CSS level 3 Atom 1.0