Exherbo

This is part of a series on EAPI 3. In EAPI 2, use dependencies are only allowed to reference USE flags that are listed in IUSE for every package that matches the associated specification. Unfortunately, this isn’t enforced, and developers often ... [More] forget to pay special attention to it. Use dependency defaults provide a way of saying “if this flag isn’t listed in IUSE, pretend that it had this value instead”. The syntax is to append ( ) to the use flag name (the parentheses aren’t entirely pretty, but they’re necessary to avoid ambiguity) if the package manager should pretend that it is on if not present, and (-) if the package manager should pretend that it is off. So, one can do things like foo/bar[baz( )] or foo/bar[!baz( )?]. Because of EAPIs before 3 not being strict about what ends up in IUSE, there are still limitations. This can only be used on flags that would be listed in IUSE were they supported; it is not usable for USE_EXPAND values. Use dependency defaults still require a degree of care from the developer. In particular, it’s not possible to write ‘pre-emptive’ defaults to handle future possible use flag removals; one does not know whether a use flag removal will be because it is always on or because a feature will be removed. Use dependency defaults first appeared in exheres-0. Posted in eapi 3 Tagged: eapi 3 [Less]

This is part of a series on EAPI 3. Traditional dependencies and slots don’t really work very well together. If a package depends upon foo, and foo has multiple slots, the package manager doesn’t know what that means at runtime. Some packages just ... [More] need any foo, and don’t mind if a completely different version in a different slot is swapped in later. Others need the slot that was around when they were installed. The original theory was that all dependencies would be runtime switchable, and that any that weren’t would be controlled via use flags. But that’s extremely impractical, and largely pointless for heavily slotted things like db. Slot operator dependencies are a more pragmatic solution for the common case; thus, as of EAPI 3, any package that has both a DEPEND and an RDEPEND upon anything that can match more than one slot should specify either :* or := after the spec. A :* dependency means “I can use anything that matches the specification, regardless of what was there at build time”. This is often the case for packages using non-compiled dynamic language libraries. Any dependency that is just an RDEPEND and not a DEPEND is implicitly of this type. In the simple case, a := dependency means “I can be built against anything that matches the specification, but I need the same slot at runtime”. This is the case for most C-style linkage libraries that have a more or less stable API but not a stable ABI or shared object version (db is the canonical example), along with some sort-of-compiled dynamic languages that don’t maintain ABI stability. The definition of := is slightly trickier if there are multiple slots already installed at build time. Since slots are in no way orderable, it is taken to mean “the slot of the best version that was installed at build time”. We considered letting ebuilds specify which slot they used, but real world use has shown that this is unnecessary and impractical — in reality, packages either just go for the best version of something automatically, or require complex special case handling. Ebuild authors won’t see this, but anyone poking around in VDB may notice :=blah slot operator dependencies. This is how the package manager handles := dependencies — at install time, it rewrites the dependency specification to have the slot name straight after the equals. We decided against having a form meaning “I need all the slots that were there at compile time”. Although at first glance this might sound useful, a closer look reveals that it doesn’t solve the whole ABI problem (it just makes it even messier), and we’ve yet to run across a legitimate use case for it that isn’t merely a horrible hack to work around lack of multi-ABI support. It’s possible to combine either slot operator dependency form with normal dependencies just like EAPI 1 slot dependencies; >=cat/foo-1.3:= is fine. Slot operator dependencies do not make sense when combined with conventional slot dependencies. Widespread use of slot operator dependencies will allow a package manager to know whether older slots of slotted packages are really required or not, rather than having to assume that they are for safety reasons, or assume that they aren’t and risk causing horrible breakage. Slot operator dependencies were originally in exheres-0, and were used in kdebuild-1. Posted in eapi 3 Tagged: eapi 3 [Less]

This is part of a series on EAPI 3. EAPI 3 has a new phase function known as pkg_pretend. This can be used to carry out checks at pretend-install time, allowing many things that would otherwise abort in pkg_setup midway through the build process to ... [More] be caught whilst the user is still at the terminal. The default is to do nothing. Custom implementations might output warnings (which should be prefixed with the program name, to avoid confusion) or call die. Care must be taken when writing pkg_pretend, since there is no guarantee that conditions that hold at that stage will also hold later on. Unless developers are absolutely certain they know what they are doing, all checks that aren’t merely checking USE flags of the active package should be repeated in pkg_setup later on — PMS deliberately doesn’t do much to restrict what could have changed between when pkg_pretend was run and when the main build starts, so it’s easiest to assume that pkg_pretend might not have been run at all. For example, one might do something like this: pkg_pretend() { if use foo && ! use bar ; then ewarn "${PF}: Support for foo requires bar enabled, but you have USE=\"foo -bar\"." ewarn "${PF}: foo support will not be enabled." fi check_foo_not_strict } pkg_setup() { check_foo_not_strict } check_foo_not_strict() { if grep --quiet '^STRICT' /etc/foo.conf 2>/dev/null ; then die "/etc/foo.conf sets foo to STRICT mode, which this build cannot support." fi } Note the check for foo not being strict being repeated. It’s possible that foo won’t be installed when pkg_pretend is run, but that it will have been installed and incorrectly configured by the time we reach our build. Also note that there’s no environment saving for pkg_pretend. You can’t set variables in pkg_pretend and have them still around by pkg_setup. Nor are you allowed to modify the root filesystem or perform any action requiring privileges that would not be available in a normal phase. pkg_pretend was originally an exheres-0 feature. Posted in eapi 3, Uncategorized Tagged: eapi 3 [Less]

EAPI 3’s feature list has been provisionally approved by the Gentoo Council, subject to nothing going wrong when Portage implements it. It’s not yet usable in ebuilds. EAPI 3 is defined based upon EAPI 2. The changes are: pkg_pretend Slot operator ... [More] dependencies Use dependency defaults DEFINED_PHASES and PROPERTIES are now mandatory New default src_install Controllable compression dodoc -r doins symlink support dohard and dosed banned unpack xz support New default econf arguments pkg_info changes Profile-controlled IUSE injection AA and KV removed Replace version variables The S to WORKDIR fallback is sometimes gone RDEPEND=DEPEND is gone Most utility commands now die on failure, unless prefixed with nonfatal. Full and precise definitions will be provided in PMS (and can currently be found in a draft PMS branch); an overview of each feature will appear in subsequent posts. Posted in eapi 3 Tagged: eapi 3, gentoo [Less]

Paludis 0.36.1 has been released: Various Ruby 1.9 fixes. Hashes are now memoised, to speed up --pretend --install for certain cases. We now use pcrecpp rather than pcre to avoid the latter’s lack of thread sanity. Posted in paludis releases Tagged: paludis

Geldik, geldik ya,  kıymetlimiz dinar bandosu gelir de biz gelmez miyiz İtü’den alıp marsa götürdüler bizi, rakıyı sek içmenin önemini hatırlattılar. Kısacası çektiler bizi saykodelik bir aleme üflediler bizi güneşe…

Geldik, geldik ya, kıymetlimiz dinar bandosu gelir de biz gelmez miyiz ;)İtü’den alıp marsa götürdüler bizi, rakıyı sek içmenin önemini hatırlattılar.Kısacası çektiler bizi saykodelik bir aleme üflediler bizi güneşe…

Gentoo is slowly shuffling towards switching from CVS to Git. This is a good thing, because CVS stinks. Using Git will reduce the amount of time developers need to waste to get something committed, make it easier to apply patches from third parties ... [More] and make tree-wide changes merely a lot of work rather than practically impossible. What it will not do is make Gentoo in any way more ‘distributed’, ‘decentralised’ or ‘democratic’. Some of the Git work has already been done, in a reduced manner (no history and no mirroring), by Daniel Robbins’ Funtoo, which is purported to be more distributed than Gentoo. The problem is, there’s nothing there to back up the distributed claim. Distributed development, in the sense for which Git was designed (and ignoring the intervening BitKeeper stage), meant moving away from having a single central repository off of which everyone worked to having everyone work off their own, publishable repositories and providing easy ways of merging changes from one to another. ‘Good’ changes would tend to find their way from the authors up the food chain to the main repository whence official releases are made. Users requiring things that hadn’t made their way to the top would maintain their own repository, and merge in changes from elsewhere that they needed. Typical Git Workflow Model For a conventional codebase, this model works. But it’s not particularly nice, and it’s driven by necessity. You’ll note the big red dots in the diagrams. These represent places where people (assisted to some highly variable degree by Git) have to do merges. I chose big red dots rather than soft fluffy clouds because merges can be a lot of work (and because drawing clouds takes effort). If you’ve got a conventional codebase, you have to do merges to make use of things from multiple sources — the compiler takes a single codebase and produces a program from it. You can do the same thing with a distribution. Funtoo, for example has had the Sunrise repository merged in to the main repository. Such a change would likely not be possible with Gentoo’s current CVS architecture. It’s not entirely clear whether Funtoo intends to have users who want to use other overlays merge those overlays into their own tree. Doing so would be more Gitish. Apparent Funtoo Workflow Model But why bother? There’s no need to have a single codebase — there’s no compiler that has to take every input at once and turn it into a single monolithic product. Those big red dots are unnecessary. A lot of fashionable programs are moving away from the big monolithic binary model and towards a plugin-assisted architecture. If you want Firefox to do a few things it doesn’t, you don’t hunt around for people who have already written them and then try to merge their source trees together. You install plugins. Only for more severe changes do you have to dive into the source, and the severity of change requiring a trip to the source is gradually increasing. There’s a reason for this — whilst the merge model is a lot better than a single authoritative codebase and a bunch of patches, it’s a lot more work than providing limited composable extensibility at a higher level. What, then, would a plugin-based model look like for a Gentoo-like distribution? Presumably, one would have a centralised ‘main’ codebase. One could then add additional small extras to that main codebase to obtain new functionality (packages, in this case); these extras would rely upon parts of the main codebase and wouldn’t be able to operate on their own. Sound familiar? Yup, overlays are plugins. This whole “merging overlays into the main tree” thing is starting to look like a step in the wrong direction. What would be some steps in a better direction? One thing that comes instantly to mind is improving overlay handling. Portage’s overlay handling currently (at least in stable) looks like this: Portage Overlay Model Portage takes the main Gentoo repository, and then merges it with each overlay in turn, creating one ‘final’ overlay that ends up being used. I’ve used an orange dot here rather than a red one because it’s a different kind of merge. Rather than doing a source-level merge, the orange dot merge more or less (sort of) works like this: If there’s a package with the same name and version in the origin and the overlay we’re merging in, take the overlay version. If there’s an eclass with the same name and version in the origin and the overlay we’re merging in, sometimes take the overlay version. Do some horrid hackery to merge together any colliding profile things in an uncontrolled manner that doesn’t work for more than one merge. Pass everything else through. Now, to be fair, the orange dot merge usually works. Most overlays don’t try to override eclasses, don’t have eclasses that conflict with each other and don’t mess with profiles. For colliding versions, you end up being stuck with a single selected version, which isn’t always so good. Unfortunately, some overlays do try to override eclasses and profiles, and the result isn’t pretty. You’re ok so long as you only use a single overlay that does this, and so long as any eclass changes aren’t incompatible, but anything beyond that and weird stuff happens. A less dangerous model would be to make the package manager support multiple repositories. Presumably most overlays wouldn’t want to have to reimplement all the profile and eclass things in the Gentoo repository, so the model would look like this: Safer Overlay Model Here, repositories, rather than the user, have control over which implementation of eclasses and so on gets used. Paludis uses this model for Gentoo overlays unless told not to. Sidebar: one might want to go a step further, and allow repositories to use multiple masters. Some Exherbo supplemental repositories do this — the gnome supplemental repository, for example, makes use of both arbor (the ‘main’ repository) and x11: Exherbo Repository Model Note that we chose not to make a repository use its master’s masters. We could’ve gone either way on this one — it’s slightly easier if masters are inherited, but it can lead to unnecessary inter-repository dependencies. Unstable Portage, meanwhile, is starting to support controlled masters for eclass merging, but not version handling, which will eventually give: New Portage Overlay Model A multiple repository model is clearly safer than the Portage model, and does away with the manual merges required by the Funtoo model. This gives us: Model Multiple Repositories? Manual Merges? Unsafe Automatic Merges? Portage (Stable) No No Yes Portage (Unstable) No No Sometimes Funtoo No Yes No Safe Yes No No I consider the multiple repository model to be better for users even ignoring the merge or conflict issues. Here’s why: Users can make selective, rather than all or nothing, use of a repository. It becomes possible to mask the foo-1.2 provided by the dodgy overlay, and use the one in the main tree or a different overlay. Similarly, users can choose not to use anything from a particular overlay except things they explicitly request. It paves the way for handling repositories of different formats. There aren’t any downsides, either — so long as repositories have user-orderable importance, there’s no loss of functionality. Finally, I’d like to debunk the myth that the Git model is somehow ‘democratic’. There’s nothing in the least bit democratic about everyone having their own repository. At best, it could be said to be a way of allowing everyone to have their own dictatorship that anyone else can be free to visit — all very well, but when tin pot dictators fall back on old habits it does little to encourage collaboration. A democratic distribution would more likely make use of a special repository which lets people vote on unwritten packages and version bumps — clearly a recipe for disaster, since most people think “I haven’t noticed any bugs” means “stable it instantly”… The only thing switching Gentoo to Git will solve is the pain of having to use CVS. This alone is enough to make the move worthwhile, but it will do little to nothing to fix Gentoo’s monolithic design and inherently centralised model. Nor does Funtoo’s merge approach solve the problem — on the contrary, it replaces a model where the package manager automatically does unnecessary merging (and sometimes gets things wrong) with a model where people do unnecessary merging (which is a lot of work, and they will still sometimes get things wrong). The future is (or at least should be) in a multi-repository model with good support from the package manager that removes the costs of decentralisation. Posted in gentoo Tagged: funtoo, gentoo, paludis [Less]

sydbox-0.1_alpha is released. This is the initial release of sydbox. What is sydbox? Sydbox is a ptrace based sandbox implementation which is based in part upon catbox and strace. Being ptrace based, it doesn’t suffer from the well known security ... [More] issues that LD_PRELOAD based sandbox implementations suffer from. It has basic support to disallow network connections. OS/Arch Support At the moment only supported operating system is Linux and only supported architectures are x86 and x86_64 but adding support for new architectures should be trivial. If you’re interested in porting please send a mail to alip or leave a comment to this post. homepage: http://alip.github.com/sydbox/ tarball: sydbox-0.1_alpha.tar.bz2 sha1sum: e6d40372242951da90973490923c3fc14622c2a4 [Less]

sydbox-0.1_alpha is released.This is the initial release of sydbox. What is sydbox? Sydbox is a ptrace based sandbox implementation which is based in part upon catbox and strace.Being ptrace based, it doesn’t suffer from the well known security ... [More] issues thatLD_PRELOAD based sandbox implementations suffer from.It has basic support to disallow network connections. OS/Arch Support At the moment only supported operating system is Linux and only supported architectures are x86 and x86_64but adding support for new architectures should be trivial.If you’re interested in porting please send a mail to me or leave a comment to this post. homepage: http://alip.github.com/sydbox/tarball: sydbox-0.1_alpha.tar.bz2sha1sum: e6d40372242951da90973490923c3fc14622c2a4 [Less]