Exherbo

Paludis is a multi-format package manager. One beneficial side effect of this is that the core code is sufficiently flexible to make handling things that aren’t really ‘packages’ in the conventional sense very easy; in the past, this has been used to ... [More] deliver unavailable, unwritten and unpackaged repositories. One of the things Exherbo inherited from Gentoo without modification was user and group management. In Gentoo, this is done by functions called enewuser and enewgroup from eutils.eclass; a package that needs a user or group ID must call these functions from pkg_setup. Although usable, this is moderately icky; Exherbo can do better than that. Really, user and group accounts are just resources. A package that needs a particular user ID can be thought of as depending upon that ID — the only disconnect is that currently dependencies are for packages, not resources. Can we find a way of representing resources as something like packages, in a way that makes sense? Fortunately, the obvious solution works. Having user/paludisbuild and group/paludisbuild as packages makes sense; adding the user or group is equivalent to installing the appropriate package, and if the user or group is present on the system, it shows up as installed. Then, instead of calling functions, the exheres can just do: DEPENDENCIES=" build run: user/paludisbuild group/paludisbuild " What about defaults? Different users need different shells, home directories, groups and so on. We could represent these a bit like options, but there’s a better way. If two or more ebuilds need the same user, they all have to do the useradd call. This means duplicating things like home directory information and preferred uid over lots of different ebuilds, which is bad. It would be better to place the users somewhere else. For Exherbo, we’ve gone with metadata/accounts/{users,groups}/*.conf. A user’s settings look something like this (the username is taken from a filename, so this would be metadata/accounts/users/paludisbuild.conf): shell = /bin/bash gecos = Used by Paludis for operations that require an unprivileged user home = /var/tmp/paludis primary_group = paludisbuild extra_groups = preferred_uid = And a group, metadata/accounts/groups/paludisbuild.conf: preferred_gid = We only specify ‘empty’ keys for demonstration purposes; ordinarily they would be omitted. We automatically make users depend upon the groups they use. The existing dependency abstractions are sufficient for this. There’s a bit of trickery in Paludis to allow supplemental repositories to override user defaults found in their masters; details are in the source for those who care. One more thing to note: with accounts specified this way, we can be sure that the package manager only manages relevant accounts. There’s no danger of having the package manager accidentally start messing with your user accounts. So what are the implications? We’re no longer tied to a particular method of adding users. If a user doesn’t want to use useradd and groupadd, they can write their own handler for the package manager to update users via LDAP or whatever. Paludis supports multiple handlers here. Users who would rather manage a particular account manually can add it themselves, and the package manager will treat it as being already installed and won’t try to mess with it. User and group defaults are in one place, not everywhere that uses them. It’s much more obvious when an account is going to be added. Accounts that are no longer required can be purged using the usual uninstall-unused mechanism. And what does it look like? $ paludis -pi test-pkg Building target list... Building dependency list... These packages will be installed: * group/alsogroupdemo [N 0] Reasons: *user/accountsdemo-0:0::accounts "alsogroupdemo" * group/groupdemo [N 0] Reasons: *user/accountsdemo-0:0::accounts "groupdemo" * group/thirdgroupdemo [N 0] Reasons: *user/accountsdemo-0:0::accounts "thirdgroupdemo" * user/accountsdemo [N 0] Reasons: *test-cat/test-pkg-2:2::ciaranm_exheres_test "A demo account" * test-cat/test-pkg::ciaranm_exheres_test :2 [N 2] <target> -foo build_options: recommended_tests split strip "Dummy test package" We can have a look at the accounts before they’re installed: $ paludis -q accountsdemo groupdemo * user/accountsdemo accounts: 0* {:0} Username: accountsdemo Description: A demo account Default Group: groupdemo Extra Groups: alsogroupdemo thirdgroupdemo Shell: /sbin/nologin Home Directory: /dev/null * group/groupdemo accounts: 0* {:0} Groupname: groupdemo Preferred GID: 123 Note the dependencies: $ paludis -qDM accountsdemo * user/accountsdemo accounts: 0* {:0} username: accountsdemo gecos: A demo account default_group: groupdemo extra_groups: alsogroupdemo thirdgroupdemo shell: /sbin/nologin home: /dev/null dependencies: group/alsogroupdemo, group/groupdemo, group/thirdgroupdemo location: /var/db/paludis/repositories/ciaranm_exheres_test/metadata/accounts/users/accountsdemo.conf defined_by: ciaranm_exheres_test The install is fairly boring: (4 of 5) Installing user/accountsdemo-0:0::accounts * Executing phase 'merge' as instructed >>> Installing user/accountsdemo-0:0::accounts using passwd handler useradd -r accountsdemo -c 'A demo account' -G 'alsogroupdemo,thirdgroupdemo' -s '/sbin/nologin' -d '/dev/null' >>> Finished installing user/accountsdemo-0:0::accounts And once they’re installed: $ paludis -q accountsdemo groupdemo * user/accountsdemo installed-accounts: 0* {:0} * group/groupdemo installed-accounts: 0* {:0} Exherbo will be migrating to this new mechanism shortly — package manager support is already there (it was only a few hours’ work), so it’s just a case of gradually hunting down and killing those enew* function calls. Posted in exherbo, paludis for users Tagged: accounts, ebuild, exherbo, exheres-0, gentoo, groups, paludis, users [Less]

So, been a while, but here's something I just threw together. It's still a little rough around the edges, but it seems to be more robust than tpb for my system, at least, and it doesn't require me to make /dev/nvram available. It uses ... [More] /sys/class/backlight/thinkpad_screen/actual_brightness (from the thinkpad_acpi module in the 2.6.22 kernel) to figure out the screen brightness, and /proc/acpi/ibm/volume to figure out the volume and mute states, also provided by the thinkpad_acpi module (I'll switch to using sysfs for this when it becomes available). It also requires the ruby-xosd package, which you can get from my overlay: git://git.pioto.org/pioto-overlay.git For lazy paludis users, just add this to /etc/paludis/repositories/pioto-overlay.conf: format = ebuild location = /var/paludis/repositories/pioto-overlay sync = git://git.pioto.org/pioto-overlay.git master_repository = gentoo names_cache = /var/cache/paludis/names write_cache = /var/cache/paludis/metadata Or, you can just do: playman -a pioto-overlay You can get the current version of this script from git at: http://git.pioto.org/gitweb/rbtpb.git Update: Fix links. [Less]

Paludis 0.34.0 has been released: instruo is now parallelised For NoConfigEnvironment, profiles are only loaded when needed. Posted in paludis releases Tagged: paludis

FOSDEM is only 3 weeks away now and as usual it's promising to be a great experience. It's by far the biggest Open Source conference in Europe. I've been to FOSDEM for the past 3 years and it's been a great experience every year. This year the staff ... [More] have (as usual) outdone themselves with a torrent of talks, stands, developer rooms and non-technical events such as the traditional beer event and a new offer of guided tours for spouses/partners.But this year is even more exciting for me as I've been invited to give a talk about Exherbo and some of the interesting things we're doing in our small part of the world. The talk is called 10 cool things about Exherbo and I hope to see all of you there. [Less]

Paludis 0.34.0_alpha2 has been released: Unpackaged repository now defines merge and strip phases, making phases useful for importare. Fixed build errors on certain systems. Stop pythons from eating all your rams when compiling. Updates to zsh completions. Posted in paludis releases Tagged: paludis

As various people have found out the hard way, and much to my annoyance because my laptop is memory starved, building Paludis can sometimes take lots and lots of RAM. Originally, we didn’t do anything about this. But unfortunately lots of users have ... [More] silly things like MAKEOPTS="-j9", which can result in the build process wanting somewhere in the region of eight gigs of RAM, which in turn leads to users whining about gcc internal errors or random processes being OOMed. So we stuck a nasty hack in the ebuild that would reduce MAKEOPTS based upon how much free memory you had — all very well, but it screws over distcc users and isn’t even necessary for many combinations of USE flags and CXXFLAGS. It’s worth working out exactly what makes the compiler memory usage so high. It’s fairly easy to figure out that it’s only an issue when building the Python bindings. We use Boost.Python for these, and unfortunately Boost will quite happily use horrible preprocessor hacks that result in huge generated source files and all sorts of nasty workarounds just to get code to work on ancient unsupported Microsoft compilers. It’s enough of a problem for enough people that there’s a tutorial section on reducing memory consumption for Boost.Python, but we already do those things. There’s something else interesting, though. With debugging turned on (which autotools does by default), we need something like 800MBytes to compile one particular Python binding file. With it turned off, we only need 300MBytes, which is much less likely to be a problem (and more to the point it won’t make my laptop start swapping). It turns out that building the Python bindings with -g isn’t even useful — gdb doesn’t give particularly useful backtraces on the Python interpreter, and there are better ways of tracking down problems there. So it looks like it makes sense to add -g0 (after checking that the compiler supports it) to CXXFLAGS for the Python bindings. Easy enough, right? Wrong. As with everything else involving autotools, we have to jump through all sorts of convoluted hoops to do this. CXXFLAGS is a user variable (so we aren’t supposed to change it), and it takes precedence over AM_CXXFLAGS (which we can change). There’s no ‘more important than the user variable’ variable, and we can’t sensibly override CXXCOMPILE, so this gets messy. We have to abuse configure.ac to remove the debugging options from the user’s CXXFLAGS and move them into something that ends up in AM_CXXFLAGS, which we can then override. Horrid. The next Paludis release will include this voodoo, which should improve things considerably and let us avoid the nasty MAKEOPTS mangling. But it’s still not ideal. Most Gentoo users have USE="python" set, either from profiles or explicitly. Most of these users do not want to build the Python bindings. Some of these users don’t think to look at the dependencies before moaning that Paludis requires Boost, so they don’t even realise it’s only because they’re using a USE flag they probably don’t want enabled. So what can we do about this? We can’t use IUSE defaults, since we don’t really want to use anything later than EAPI 0 for package manager ebuilds. We could turn off python selectively in package.use, but lots of users still have USE="python" set explicitly. So we use a different USE flag name. We’ve gone for python-bindings, along with a use description that makes it clear that thinking “well I have some things that use Python so I probably want this flag on” is wrong. For consistency, we’ve also renamed ruby to ruby-bindings. These are a lot more useful than the Python bindings (playman is written in Ruby), and a lot faster to build thanks to Ruby having a reasonably sane API, so we might end up having to mess with profiles to turn them on by default. I might end up reverting all of this if it turns out it does more harm than good. We’ll see. Tagged: boost, c , gentoo, paludis, python, ruby [Less]

Paludis 0.34.0_alpha1 has been released: Masks that have been overridden by the user are now shown. Support phases. Paludis internals are now threaded using pthreads, and provide the SGI guarantee.    Tagged: paludis   

Flag variables in Bourne shell programs Open in Browser Firefox extension, for all those webservers that get MIME types wrong. noremap <Leader>v `[v`] to select the thing just pasted in Vim. Inspired by gv, which I always expect to reselect pastes for some reason.    Tagged: bash, firefox, vim, vimrc   

Six months ago I posted an outline for encrypting your system with LUKS. Well, I figure it's about time for me to write up how to actually go about it. In this post, I'll outline the necessary kernel configuration. The basic kernel configuration ... [More] that I mentioned a long time ago still holds. Basically, you need to have the following options built into your kernel: CONFIG_DM_CRYPT, CONFIG_CRYPTO_CBC, CONFIG_CRYPTO_SHA256, and CONFIG_CRYPTO_AES. Most of those will be turned on when you enable: Device Drivers -> Multiple devices driver support (RAID and LVM) -> Device mapper support -> Crypt target support However, the SHA256 support will not be. It can be found at: Cryptographic API -> SHA224 and SHA256 digest algorithm On the note of kernel configuation: for this process, you will need to do a fair amount of work from within another linux environment (most likely a LiveCD). For Gentoo, the most recent CD image I've found that has all the necessary configuration is the 2006.1 version... As I recall, some of the intermediate minimal CDs, at least, didn't possess cryptsetup at all. And the most recent weekly build I tried, while it did have lvm and cryptsetup installed, didn't have CONFIG_CRYPTO_SHA256 either built in, or as a module. Probably the best bet is an Ubuntu 8.10 Alternative CD. You'll have to either boot in recovery mode, or go through a bit of the installation procedure, as it doesn't initially have cryptsetup available. But, once it's detected the CD and loaded the modules from it, you can just switch to another virtual console and do things from there. Next time, I'll outline creating the disk partitions. [Less]

Paludis 0.32.4 has been released by spb.    Tagged: paludis