Gentoo on Home of Jeremy Olexa

Gentoo: Graphing the Developer Web of Trust

Wed, 28 Nov 2012 00:00:00 +0000

“Nothing gets people’s interest peaked like colorful graphics. Therefore, graphing the web of trust in your local area as you build it can help motivate people to participate as well as giving everyone a clear sense of what’s being accomplished as things progress.”

I graphed the Gentoo Developer Web of Trust, as motivated by the (outdated) [Debian Web of Trust][2].

Graph (same as link above) – Redrawn weekly : http://qa-reports.gentoo.org/output/wot-graph.png
Stats per Node : http://qa-reports.gentoo.org/output/wot-stats.html
Source : http://git.overlays.gentoo.org/gitweb/?p=proj/qa-scripts.git;a=blob;f=gen-dev-wot.sh;hb=HEAD

Enjoy.

[2]: http://www.chaosreigns.com/code/sig2dot/debian.html

Gentoo Miniconf 2012

Wed, 24 Oct 2012 00:00:00 +0000

The Gentoo Miniconf is over now but it was a great success. There was 30+ developers that went and I met quite some users too. Thanks to Theo (tampakrap) and Michal (miska) for organizing the event (and others), thanks to openSUSE for sponsoring and letting the Gentoo Linux guys hangout there. Thanks to the other sponsors too, Google, Aeroaccess, et al.

More pics at the [Google+ event][3] page.

It was excellent to meet all of you.

[3]: https://plus.google.com/u/0/events/c0fn547i01b94gae3v9tkn2umqc

Gentoo: IPSec, L2TP VPN for iOS

Tue, 18 Sep 2012 00:00:00 +0000

There are thousands of guides out there on this subject, however I still struggled to set up an IPSEC VPN at first. This is a HOWTO for my own benefit – maybe someone else will use it too. I struggled because most of the guides involved setting up the VPN on a NAT’d host and connecting to the VPN inside the network. I didn’t do that on my linode, which has a static public IP.

My objectives were clear:

Create a connection point that was semi-secure while connecting to open wifi networks
Bypass some “You are not in the US” restrictions while on the road

Step 1: Install applications, net-misc/openswan, net-dialup/xl2tpd
Step 2: Configure openswan:

# cat /etc/ipsec.conf 
config setup
    nat_traversal=yes
    virtual_private=%v4:10.0.0.0/8,%v4:192.168.0.0/16,%v4:172.16.0.0/12,%v4:!10.152.2.0/24
    oe=off
    protostack=auto

conn L2TP-PSK-NAT
    rightsubnet=vhost:%priv
    also=L2TP-PSK-noNAT

conn L2TP-PSK-noNAT
    authby=secret
    pfs=no
    auto=add
    keyingtries=3
    rekey=no
    ikelifetime=8h
    keylife=1h
    type=transport
    left=1.1.1.1
    leftprotoport=17/1701
    right=%any
    rightprotoport=17/%any
    dpddelay=15
    dpdtimeout=30
    dpdaction=clear

# cat /etc/ipsec.secrets
1.1.1.1 %any: PSK "TestSecret"

Where 1.1.1.1 is your public eth0 address and 10.152.2.0 is the subnet that xl2tpd will assign IPs from (can be anything, I picked this at the advice of a guide because it is unlikely to be assigned from a router on a public network)

Step 3: Configure xl2tpd:

# cat /etc/xl2tpd/xl2tpd.conf
[global]
ipsec saref = no

[lns default]
ip range = 10.152.2.2-10.152.2.254
local ip = 10.152.2.1
require chap = yes
refuse pap = yes
require authentication = yes
ppp debug = yes
pppoptfile = /etc/ppp/options.xl2tpd
length bit = yes

The local IP must be inside the subnet but outside the IP range above.

# cat /etc/ppp/options.xl2tpd
refuse-mschap-v2
refuse-mschap
ms-dns 8.8.8.8
ms-dns 8.8.4.4
asyncmap 0
auth
lock
hide-password
local
#debug
name l2tpd
proxyarp
lcp-echo-interval 30
lcp-echo-failure 4

The ms-dns lines are configurable to any DNS server you have access to.

# cat /etc/ppp/chap-secrets
# Format:
# client server secret IP-addresses
#
# Two lines are needed since it is two-sided auth
test l2tpd testpass *
l2tpd test testpass *

Step 4: Configure kernel parameters (sysctl)

# cat /etc/sysctl.conf
# only values specific for ipsec/l2tp functioning are shown here. merge with
# existing file
# iPad VPN
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.icmp_ignore_bogus_error_responses = 1

Remember that sysctl.conf is evaluated at boot so run sysctl -p to get the settings enabled now as well.

Step 5: Configure firewall (iptables):
This is the critical step that I wasn’t grokking from the existing guides in the wild. Even when bringing the firewall down to test, you need the NAT/forwarding rules:

# iptables -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
# iptables -A FORWARD -s 10.152.2.0/24 -j ACCEPT
# iptables -A FORWARD -j REJECT
# iptables -t nat -A POSTROUTING -s 10.152.2.0/24 -o eth0 -j MASQUERADE

Step 6: Configure the device/client:
Settings -> General -> Network -> VPN -> Add VPN Configuration

L2TP
Description: Description
Server: 1.1.1.1 (or the hostname)
Account: test
RSA SecurID=OFF
Password: testpass
Secret: TestSecret
Send All Traffic=On

Step 7: Verify it works by going to some IP display webpage and it should show 1.1.1.1

Conclusion: The above examples should be enough to get the VPN working. There are some tweaking oppurtunities that I didn’t document or elaborate on. There is plenty of examples out there to look at or research, however. This was all setup without the firewall configuration and the client would connect but there would be no onward internet activity. It acted just like there was a invalid DNS server configured, at that point I looked into setting up a NAT, dnsmasq on the local interface, and other wierd things. In the end, just needed to forward the traffic properly.

With that knowledge of the firewall issue, the ultimate instructions would probably be this page: [https://www.openswan.org/projects/openswan/wiki/L2TPIPsec_configuration_using_openswan_and_xl2tpd][6]

[6]: https://www.openswan.org/projects/openswan/wiki/L2TPIPsec_configuration_using_openswan_and_xl2tpd

Linux: Easy/Free CLI Cloud Backup (box.com)

Fri, 10 Aug 2012 00:00:00 +0000

Preface: I use to use rsync.net for my offsite backup needs. They offer a nice solution, especially with the F/OSS contributor discount, but my needs were lesser than their offerings and I didn’t quite feel comfortable paying monthly for such needs. That is, get my critical (but small) backups off my host, MySQL, a few important files, etc. Most of my online world is now externally hosted (cloud) or easily rebuilt with key pieces of info.

My current free 5G offsite backup solution is with box.com. I don’t really use box.com as intended – ‘a collaborative file sharing platform’ but instead take advantage of the [WebDAV][3] access that they provide. So, a quick walk-through:

emerge davfs2 (I pushed fixes in version >=1.4.7 in Gentoo Linux, I recommend that version)
echo "https://www.box.com/dav /mnt/box.com davfs rw,user,noauto 0 0" >> /etc/fstab
Mount the mountpoint as your user (enter your box.com credentials), copy files to the mounted filesystem.

(There are some finer details, like modifying $HOME/.davfs2/secrets and turning off use_locks, but I’m sure you smart people can figure that out via google.)

Personally, I have a simple cron job that mounts, rsyncs, umounts that runs daily. Now, I have an accessible location to restore critical files as needed, just don’t look at the webui since it doesn’t understand compressed tarballs that well and is useless, heh.

[3]: http://en.wikipedia.org/wiki/Webdav

Gentoo Miniconf / Linux Days 2012

Wed, 18 Jul 2012 00:00:00 +0000

I’ll be there.

[][1]

[1]: http://www.linuxdays.cz/en/

Linode don't use barriers and ext-4

Tue, 14 Feb 2012 00:00:00 +0000

On my Linode running Gentoo Linux, I converted to ext4 some time ago and didn’t have any issues until now for some reason, mostly because I don’t reboot that often to notice. The symptoms are :

Every reboot, you will see:

EXT4-fs error (device xvda): ext4_journal_start_sb:296: Detected aborted journal<br /> EXT4-fs (xvda): Remounting filesystem read-only<br />
and a subsequent reboot fixes this by a forced run of fsck. Now that is an annoying one, every other reboot results in a crippled system and otherwise a fsck “fixes” it and you have no issues.

So, after some research I found that barriers are enabled by default and they don’t really make sense on a hosted vm guest. I qualify the last statement by google research, not an expert but it seems that the common knowledge is that disabling barriers is safe for battery backed up storage, and Linode disabled barriers completly.

The solution to the above problem is simply disabling barriers. Like so:

In /etc/fstab:
/dev/xvda / ext4 noatime,barrier=0 0 1

Source: Linode Forums

Gentoo: Colemak keymap support

Fri, 11 Nov 2011 00:00:00 +0000

Colemak is my new keymap of choice. Luckily, Gentoo Linux supports it well. Unlike some of the crazy instructions people have posted out [there][3], you only need to edit 2 files to convert your console and Xorg server. Note, I’m taking the time to write this because I couldn’t find easy instructions out there…

`
% cat /etc/conf.d/keymaps

Use keymap to specify the default console keymap. There is a complete tree

keymap=“en-latin9”
<…>
`

% cat /etc/X11/xorg.conf.d/30-keyboard.conf 
Section "InputClass"
        Identifier "keyboard-all"
        Option "XkbVariant" "colemak"
EndSection

[3]: http://forums.gentoo.org/viewtopic-t-639368-start-0.html

Tip: "Intelligent" bugzilla mail threading in GMail using procmail

Mon, 24 Oct 2011 00:00:00 +0000

(Preface: Target audience for this post is Gentoo Devs + GMail WebUI users, however, anyone that forwards bugmail to GMail and has procmail between them could also use this.)

I find it annoying that the GMail web interface chooses to thread messages based on subject name alone, this creates two threads for every new bug report sent to you from bugzilla. Sadly, we can’t control the threading that Google tells us is “the only way” (subject based threading or email header based threading, which bugzilla does correctly). If you want to follow the rabbit trail that I went on regarding this subject, I won’t stop you…

Or you can use procmail to rewrite the subject, that is, remove “New: “ from the first email:

# Remove "New: " from the subject so threading in gmail works
SUBJ_=`formail -xSubject: | expand | tr -d '\n' | sed -e 's/^[ ]*//g' -e 's/New: //'`
:0
* ^From: bugzilla-daemon@gentoo.org
{
    :0 fwh
    | formail -i"Subject: ${SUBJ_}"
}

Tangentially related that may be useful, is this rule that kills duplicate messages when you report a bug and are assigned the same bug (or in CC). The bugzilla software has no way of knowing what email aliases you may be in.

# Kill duplicate messages. If I am the reporter *and* the bug is assigned to a
# team I am in, delete the mail to me directly
:0
* ^To: username@gentoo.org
* ^From: bugzilla-daemon@gentoo.org
* ^X-Bugzilla-Reporter: username@gentoo.org
* ^X-Bugzilla-(Assigned-To|CC):.*(team1|team2)@gentoo.org
/dev/null</pre>
<p></p>


*I like the GMail WebUI. I use it. Please don't suggest that I should use other clients, I already know that other clients can handle the threading fine.*

Gentoo: Removing USE="python perl" from the default profile

Wed, 05 Oct 2011 00:00:00 +0000

Well, I got sick of setting -python -perl on my Gentoo hosts, I even consider them “questionable defaults” for a majority of Gentoo users..

So, let this be an advanced notice that you may see some rebuilds for useflag changes. There has been sufficient testing such that there should be few to nil problems, but we can’t test everything. Please file bug reports, if needed.

Gentoo: per-package PORTAGE_TMPDIR settings

Fri, 16 Sep 2011 00:00:00 +0000

I don’t know how many people know about per-package environment variables in portage since 2.1.9 or so. (ref: bug 44796) It is a worthwhile enhancement to know about, regardless. Like most people, I have my PORTAGE_TMPDIR on tmpfs to speed up compilation times and reduce I/O usage. My 2G tmpfs mounted on /var/tmp/portage is large enough for almost all packages, even multiple jobs at once, however, not all. Solution:

% cat /etc/portage/package.env
app-office/libreoffice notmpfs.conf
% cat /etc/portage/env/notmpfs.conf
PORTAGE_TMPDIR=“/var/tmp/notmpfs”

(More info available in the portage man page)

Now, when I find my next package that needs notmpfs, it is as easy as: echo "cat-egory/pkg notmpfs.conf" >> /etc/portage/package.env which is much easier than bashrc hacks or something else insane that I have seen. Of course you can extend that to most make.conf settings, hope that helps someone.

Gentoo: Infra team update

Sun, 10 Apr 2011 00:00:00 +0000

It has been awhile since I’ve posted about what I’ve been doing with Gentoo Linux. So, here is a general update for the team that I have been spending most of my time with.

You may have seen the Bugzilla upgrade that Christian was working on. Gentoo moved from the bottom of the list provided from one of the upstream devs to the top of the list. (As of April 2011)
I finally put an idea of mine into reality of graphing the number of “emerge –sync’s” against the rsync.gentoo.org rotation. Full graph and last 4 weeks
A new reporting website was born: http://qa-reports.gentoo.org/ – The vision was: “Many Gentoo devs have useful scripts and many people complain that there is not a central place to see all the output.” This site is a solution, and open for all. repo: qa-scripts.git
A new “Get Gentoo at a glance” website was born: http://get.gentoo.org/ that Matthew is still working on, so maybe expect some layout changes – The motivation for this was inspired from bug 350271, repo: get-gentoo.git
- Some behind the scenes work involving our mastermirror service. The current hardware running this important service is one of the oldest hosts we have.
Of course, there is always the untold hours to keep Gentoo Linux infrastructure running happily for all customers. As a final note, if you have a good idea, feel free to propose it via bugs or IRC. We will listen and definately avoid [NIH][8] syndrome if we can. Cheers.

[8]: http://en.wikipedia.org/wiki/NIH_syndrome

Tip: Single Purpose Password-less SSH Key

Fri, 11 Feb 2011 00:00:00 +0000

Scenario: You need to setup a service that requires ssh access to a remote host, possibly/probably by the root user. This service needs to run at regular intervals and it is critical that it works without a human entering a passphrase (even once).

Solution: The obvious solution that comes to mind is a ssh key. But a password-less key that allows root login? RED FLAG. However, there is a way to accomplish this without allowing a root login completely. That is to create, what I call, a single purpose key. I feel like this not a widely known trick, so I am archiving it so I don’t forget myself.

Details:
(Where local host is the host that needs ssh access and remote host is the host that you are “opening” up or allowing ssh access to)

On the local host, create a ssh key without a passphrase for the root user, this is widely documented via other sources
On the remote host, add the key to the /root/.ssh/authorized_keys file. However, start the line in that file with command="/root/bin/validate-ssh.sh"

On the remote host, the /root/bin/validate-ssh.sh script is a simple script that allows access to your service and exits for anything else. An example of allowing rsync access [only]:

% cat /root/bin/validate-ssh.sh 
#!/bin/bash
case "$SSH_ORIGINAL_COMMAND" in
    rsync\ --server*)
        # uncomment for debug
        # echo "$(date +%Y%m%d): $SSH_ORIGINAL_COMMAND" >> /var/log/ssh-cmd.log
        $SSH_ORIGINAL_COMMAND
        ;;
    # debug
    testconnect)
        echo "You successfully connected to $(hostname)"
        ;;
    *)
        echo "Sorry, command '$SSH_ORIGINAL_COMMAND' is not allowed"
        exit 1
        ;;
esac

Optional, if you only want to allow this access from a small set of hosts add from="192.168.1.11,10.80.80.1" to the same line in /root/.ssh/authorized_keys

So, now, you can use that password-less ssh key as root (assuming the remote host allows root logins via ssh) and you should see something. ssh root@remote testconnect will return that string. rsync root@remote:/file will work. Everything else will get the message that indicates it wasn’t allowed. This is expandable to just about everything provided that you know the “$SSH_ORIGINAL_COMMAND” – on another host I use it to allow password-less sshfs access, so SSH_ORIGINAL_COMMAND=/usr/lib/misc/sftp-server and so-forth.

Naturally, this will work for other users/uses as well. I’ve seen references that some admins are using this to allow access if and only if they enter a sekrit token, etc. I’ll also say that you should be smart with this, opening up root access is a hole – if anyone compromises the local host, I suppose they could get access to the remote host if they knew how.

Updating Intel Atom processor microcode

Tue, 25 Jan 2011 00:00:00 +0000

The problem:

% dmesg | grep -i micro
Atom PSE erratum detected, BIOS microcode update recommended
Atom PSE erratum detected, BIOS microcode update recommended
microcode: CPU0 sig=0x106c2, pf=0x4, revision=0x208
microcode: CPU1 sig=0x106c2, pf=0x4, revision=0x208
microcode: Microcode Update Driver: v2.00 , Peter Oruba

I found out that some recent kernel that I loaded at an unknown time, was able to point out that I had ‘old’ microcode for my processor on my Aspire1 ZG5. I searched around for a BIOS upgrade, but since this is an older generation netbook, I quickly gave up when my searching yielded nothing useful. There is a userspace tool that you can use to ‘upgrade’ the microcode provided by Intel on every boot. In Gentoo Linux, that is called sys-apps/microcode-ctl. Simply install that and enable the init script on boot.

The output after:

% dmesg | grep -i micro
Atom PSE erratum detected, BIOS microcode update recommended
Atom PSE erratum detected, BIOS microcode update recommended
microcode: CPU0 sig=0x106c2, pf=0x4, revision=0x208
microcode: CPU1 sig=0x106c2, pf=0x4, revision=0x208
microcode: Microcode Update Driver: v2.00 , Peter Oruba
microcode: CPU0 updated to revision 0x218, date = 2009-04-10
microcode: CPU1 updated to revision 0x218, date = 2009-04-10

References:

https://wiki.archlinux.org/index.php/Microcode
[Flameeyes’s Weblog : Microupdates for microcodes][2]

Edit: Added a reference

[2]: http://blog.flameeyes.eu/2011/01/17/microupdates-for-microcodes

Gentoo: A critical look at the QA process

Wed, 29 Dec 2010 00:00:00 +0000

This post will probably annoy some people, or bring bad “spotlight” to Gentoo Linux. I call it a case study, but it is just my opinions…

The QA team has said that there is some sort of “policy” on masking packages that break reverse dependencies. I’ll subscribe that that policy for the sake of not breaking users machines on purpose, however, let’s take a look at the current case study: poppler-0.16

package.mask (in context, name removed because it isn’t needed):
+# Masked because of ABI change, breaks
+# depending packages. Keep masked until depended packages
+# got fixed (adjusted dependency or fixing version bump).
+# tracer bug 349918
+=app-text/poppler-0.16.0

…and there was some discussion on IRC. The QA team (at least a few members) says that the “tree is broken” with poppler-0.16. At time of this writing, 7 packages were reported on the tracker bug and 2 were fixed already. So, it is my opinion that progress for Gentoo is hampered because of this masking. I’ll explain why but first the different theories to package testing that I have observed.

Theory 1:
Run full arch for stability. The problem with this theory is that some group of people/machines need to test ~arch packages first and report issues, otherwise they will hit the stable users and the concept of the arch/~arch tree will break down. The advantage being less compilation in general, and the goal being stability or less breakage. I consider this theory for people that are new to Gentoo.

Theory 2:
Run full ~arch to find interaction issues. This is a fine theory, but not for me. Even though I am a Gentoo dev, I don’t have copious amounts of free time to contribute how I want if I am running ~arch. It is my personal opinion that users should not run full ~arch if they do not want to be bothered contributing back to gentoo/upstream, via bug reports, patches, etc. The advantage here, is that full ~arch users/machines will be able to find issues with the package before it is deemed “stable” – in theory, finding issues fast and being fixed fast. The downside being more compilation, occasional breakage, and/or random mistakes (devs are human, after-all). The Gentoo Handbook says that ~arch needs more testing and isn’t enabled by default. I’d consider this theory for power users. “I only want the latest packages” is no excuse for using this theory if you are going to complain about the above mentioned downsides.

Theory 3:
Run mostly arch and a few ~arch packages. This is what most of my machines run. That is, arch (stable) system and ~arch for packages that I maintain (or help maintain). The advantage here is what I consider real integration testing. The theory is that everything in ~arch will become arch, at one point. These ~arch packages are coming in one-by-one, so it makes no sense to only test a full ~arch tree if the package will be entering the arch tree. This is what the arch team does when they mark packages stable. The downside being, more management, maybe some dependency issues if they aren’t stated properly, etc. I’d consider this theory for people that prefer stability but enjoy the latest packages for some apps.

I think it is valuable to have a user base that is doing all of those theories. …back to poppler.

As we have seen in the past, once a package is “masked for testing” – it may take years to lift that mask because no one actually tests it. And why should you? It is masked and therefore not even in the testing category (~arch)!! Bingo, the crux of this case study. Technically speaking, there is nothing wrong with poppler-0.16.0, it is perfectly fine to be in ~arch by its own criteria. On the other hand, it’s ABI change breaks packages that haven’t been fixed to work with the new ABI yet. By masking a package that changes ABI with a soname change, it could (and should, in my opinion) be considered slowing progress for Gentoo. It should not be considered that the “[whole] tree is broken” – these 7 packages may just be the tip of the iceberg in the breakage category, but we won’t know if it is masked and users are not contributing. Let’s remind ourselves that the arch tree is still functioning properly at this point and “stable” users won’t see any issues…now. For them, thankfully, the ~arch users (including devs) are contributing to Gentoo via bug reports & patches. It should also be noted that nothing is being broke “on purpose” here, bugs are being filed and bugs are being fixed – this is the goal of a software project, right?

So, the way I see it, is a chain reaction.

Package enters the tree that causes a slight nuisance. ⁷⁄₁₄₂₀₅ known packages affected or 0.04 % of the tree.
The QA team deems this problem package as too severe to allow in ~arch and masks it. (Personal opinion is the 0.04% of the tree is not a servere problem)
The maintainer of the problem package is trying to identify breakages by leaving it in ~arch (and relying on the power users)
Since the package is masked, the effects of this problem package will be prolonged because “no one” will be testing it in ~arch.
The 7 affected packages are fixed, the problem package is unmasked
A new set of affected packages are found
repeat above

And now for some hard questions:

How should Gentoo make it easier to add packages that break reverse dependencies if ~arch isn’t appropriate and nothing really happens when package.mask’d?
Is ~arch becoming a second stable tree and thus losing value in general?
Related, are overlays hurting Gentoo? The Xfce team subscribes to the theory that ~arch is the place to get useful testing feedback. The Xfce pre releases are in ~arch so the final release is ready for arch asap. This means that we have less work in the long run because we are maintaining less versions sooner and in better quality. The GNOME team subscribes to the theory that their overlay is for new releases and it takes longer to get packages in ~arch and thus longer to arch (my observation only, not official)
portage-2.2 has this cool “preserved-libs” feature that wouldn’t help compilation issues in this case but might help runtime issues by keeping the old lib(s) around. I like this feature but it is still masked for majority of the users, even though many devs are using it – is it time to release it to the world with all the [bugs][2] it has?
Related, does masking a feature block contributors? I want to say yes simply due to exposure.

Disclaimer: This was in NO WAY meant to be a personal attack against anyone. I can live with agreeing to disagree with people. But I can’t agree with not bringing something up because it is a controversial topic.

[2]: http://bugs.gentoo.org/240323

Gentoo: Xfce 4.8pre2 Released

Mon, 06 Dec 2010 00:00:00 +0000

Xfce-4.8pre2 was announced yesterday. Gentoo has it available already thanks to the work of Samuli (ssuominen). The pre1 version was available with 0day bumps too, that proved to have the normal beta release “issues”, there are many bugs fixed in pre2 (ChangeLog). I saw several Gentoo users participating on the Xfce bugtracker, thanks. If you are of the type to test and report bugs, feel free to upgrade to 4.8pre2 and help make 4.8 final a solid release by participating upstream ([bug tracker][3]).

The final 4.8 release is scheduled for January 16th, 2011. The Gentoo Xfce team will continue to bring you 0day bumps if possible. The 4.8 series will not hit the Gentoo stable tree until after the final release.

[3]: http://bugzilla.xfce.org/