Jeremy's Weblog

Gentoo Prefix is still alive and going strong. In my opinion, Gentoo Prefix remains a strong point of Gentoo Linux and really establishes that Gentoo Linux is a metadistribution. In this post I want to focus on the numbers. The number of packages in the Gentoo Prefix tree, specifically. But first, a history lesson. It wasn’t until EAPI3 in Gentoo that “allowed” Gentoo Prefix variables into the main Gentoo Linux tree. That was in late 2011, but Gentoo Prefix existed much before then, all the way back to 2006 (at least). Before EAPI3, the prefix team made slight modifications to ebuilds and placed them in a repo and called it the tree of packages for Gentoo Prefix. This worked fine, but we had growing pains. The major issue was that we were getting too successful to manage the increased contributions from users. In other words, as the number of “forked” packages grew, the amount of maintenance time increased greatly – this is due to the fact that it is a chore to keep our forks synced. At least, a large chore for a small team. This is why we looked for help and adoption from the other pool of 200 Gentoo Developers, hence EAPI3 and beyond. Since supporting Gentoo Prefix is not a big use of overall developer time, this has gone over quite well in my opinion – yes, there are some pain points at times I do realize. Enough history, here are the numbers:

• Number of packages in Gentoo Linux: 15554 packages in 154 categories.
• Number of total* packages in Gentoo Prefix: 9483 packages in 154 categories.
• Number of KEYWORDED packages in Gentoo Prefix: About 3000 for the most popular arch
• Number of packages still NOT in the main Gentoo Linux tree: 369 packages

* The total packages in the tree also contains non-keyworded packages because that just makes life simple. Once packages started migrating to the main tree, I helped think of this “whitelist” concept. The short version of the whitelist is that if a package is listed in that text file, it gets included in the Gentoo Prefix tree as a direct copy of the version in the Gentoo Linux tree. The presense of the package in the old repo means that it is used instead. Eventually, this concept will go away and we will overlay the Gentoo Linux tree directly.

So why is it taking so long to migrate ALL packages to the Gentoo Linux tree? Well, that is where the rubber meets the road and we get into roadblocks. A roadblock for us could be a number of things, such as a disagreement with the Gentoo Linux maintainer, some patches existing that we don’t feel are a good fit for Gentoo Linux, or even us being lazy and not submitting stuff to upstream. We also don’t want to push invasive changes to Gentoo Linux for critical packages, like the toolchain for example.

It has long since been our agenda to not add anymore packages to the old repo and going forward only adding new stuff to Gentoo Linux directly. I hope we can make a dent in those remaining 369 in 2012!

Well… I finally figured out that the ucb package isn’t installed on Solaris 11 by default (resource). Unfortunately, the Oracle docs are confusing to follow. Here is a cheatsheet for installing the ucb package on your shiny Solaris 11 install.

1. Figure out the IPS installer, read man pages, get frustrated at lack of detail, run to Google.
2. Find the package you want on http://pkg.oracle.com/, in this case compatibility/ucb
3. Add the publisher link to your config, by the way, this link is not documented that I can find so I had to guess and check. A publisher is a package list of sorts, I guess.
   # pkg set-publisher -G '*' -M '*' -g http://pkg.oracle.com/solaris/release solaris
4. Install the package, # pkg install compatibility/ucb

# pkg install compatibility/ucb
Packages to install: 1
Create boot environment: No
Create backup boot environment: No

DOWNLOAD PKGS FILES XFER (MB)
Completed 1/1 80/80 0.4/0.4

PHASE ACTIONS
Install Phase 166/166

PHASE ITEMS
Package State Update Phase 1/1
Image State Update Phase 2/2

1. Behold, that you now have the compatibility libs for software that may need to use them

Whew…now, you might wonder what is so hard about that. Well, traversing Oracle docs is the hard part.

Here are the docs that I had open in my browser, they may or may not help and I fully expect the links to break in the future because Oracle is good at that.

• Copying and Creating Oracle Solaris 11 Package Repositories
• Oracle Solaris 11 Package Repository
• Oracle Solaris 11 Package Management with IPS
• Image Packaging System Man Pages

Colemak is my new keymap of choice. Luckily, Gentoo Linux supports it well. Unlike some of the crazy instructions people have posted out there, you only need to edit 2 files to convert your console and Xorg server. Note, I’m taking the time to write this because I couldn’t find easy instructions out there…


% cat /etc/conf.d/keymaps
# Use keymap to specify the default console keymap. There is a complete tree
# of keymaps in /usr/share/keymaps to choose from.
keymap="en-latin9"
<...>


% cat /etc/X11/xorg.conf.d/30-keyboard.conf
Section "InputClass"
        Identifier "keyboard-all"
        Option "XkbVariant" "colemak"
EndSection

(Preface: Target audience for this post is Gentoo Devs + GMail WebUI users, however, anyone that forwards bugmail to GMail and has procmail between them could also use this.)

I find it annoying that the GMail web interface chooses to thread messages based on subject name alone, this creates two threads for every new bug report sent to you from bugzilla. Sadly, we can’t control the threading that Google tells us is “the only way” (subject based threading or email header based threading, which bugzilla does correctly). If you want to follow the rabbit trail that I went on regarding this subject, I won’t stop you…

Or you can use procmail to rewrite the subject, that is, remove “New: ” from the first email:

# Remove "New: " from the subject so threading in gmail works
SUBJ_=`formail -xSubject: | expand | tr -d '\n' | sed -e 's/^[ ]*//g' -e 's/New: //'`
:0
* ^From: bugzilla-daemon@gentoo.org
{
    :0 fwh
    | formail -i"Subject: ${SUBJ_}"
}

Tangentially related that may be useful, is this rule that kills duplicate messages when you report a bug and are assigned the same bug (or in CC). The bugzilla software has no way of knowing what email aliases you may be in.

# Kill duplicate messages. If I am the reporter *and* the bug is assigned to a
# team I am in, delete the mail to me directly
:0
* ^To: username@gentoo.org
* ^From: bugzilla-daemon@gentoo.org
* ^X-Bugzilla-Reporter: username@gentoo.org
* ^X-Bugzilla-(Assigned-To|CC):.*(team1|team2)@gentoo.org
/dev/null

I like the GMail WebUI. I use it. Please don't suggest that I should use other clients, I already know that other clients can handle the threading fine.

Well, I got sick of setting -python -perl on my Gentoo hosts, I even consider them “questionable defaults” for a majority of Gentoo users..

So, let this be an advanced notice that you may see some rebuilds for useflag changes. There has been sufficient testing such that there should be few to nil problems, but we can’t test everything. Please file bug reports, if needed.

See also:

• Gentoo Announce Message
• Gentoo Developer Announce Message
• Bug 250179

I don’t know how many people know about per-package environment variables in portage since 2.1.9 or so. (ref: bug 44796) It is a worthwhile enhancement to know about, regardless. Like most people, I have my PORTAGE_TMPDIR on tmpfs to speed up compilation times and reduce I/O usage. My 2G tmpfs mounted on /var/tmp/portage is large enough for almost all packages, even multiple jobs at once, however, not all. Solution:

% cat /etc/portage/package.env
app-office/libreoffice notmpfs.conf
% cat /etc/portage/env/notmpfs.conf
PORTAGE_TMPDIR=”/var/tmp/notmpfs”

(More info available in the portage man page)

Now, when I find my next package that needs notmpfs, it is as easy as: echo "cat-egory/pkg notmpfs.conf" >> /etc/portage/package.env which is much easier than bashrc hacks or something else insane that I have seen. Of course you can extend that to most make.conf settings, hope that helps someone.

A few days ago, Linode.com announced native IPv6 roll out in their datacenters. Now, while I haven’t wrote about Linode in the past 6 months, I am still a happy customer. I am documenting the steps I took to migrate away from my HE.net tunnel.

1. Set the TTL low on any DNS addresses that you will be changing. Ideally, do this a fair amount ahead of time.
2. Send in a support ticket to get your /64 allocated. Sidenote: response time: 4 minutes
3. Reboot ‘node so the backend system deploys your IPv6 after it was allocated. Verify IPV6 status on your ‘node.
4. From a different IPv6 host, run nmap -6 on the existing address to verify listening services.
5. Update DNS, define static networking, be happy.

It has been awhile since I’ve posted about what I’ve been doing with Gentoo Linux. So, here is a general update for the team that I have been spending most of my time with.

• You may have seen the Bugzilla upgrade that Christian was working on. Gentoo moved from the bottom of the list provided from one of the upstream devs to the top of the list. (As of April 2011)
• I finally put an idea of mine into reality of graphing the number of “emerge –sync’s” against the rsync.gentoo.org rotation. Full graph and last 4 weeks
• A new reporting website was born: http://qa-reports.gentoo.org/ – The vision was: “Many Gentoo devs have useful scripts and many people complain that there is not a central place to see all the output.” This site is a solution, and open for all. repo: qa-scripts.git
• A new “Get Gentoo at a glance” website was born: http://get.gentoo.org/ that Matthew is still working on, so maybe expect some layout changes – The motivation for this was inspired from bug 350271, repo: get-gentoo.git
• Some behind the scenes work involving our mastermirror service. The current hardware running this important service is one of the oldest hosts we have.

Of course, there is always the untold hours to keep Gentoo Linux infrastructure running happily for all customers. As a final note, if you have a good idea, feel free to propose it via bugs or IRC. We will listen and definately avoid NIH syndrome if we can. Cheers.