Fedora is Self-Hosting

Fedora 12 (Beta available now), is self-hosting.

What does this mean? Simply put, it means that you can use a copy of Fedora 12 to rebuild, from source, all* of Fedora 12 again.

Why is this important? One of the key tenets of Free and Open Source software is that anyone can get a copy of the source code, make modifications to it, built it, an use the modified version. Simply publishing the source code, without also allowing people a way to rebuild and use that code, doesn’t accomplish this goal.

Source code tends to bitrot over time. Libraries that your code uses will change, get updated, add features and bugfixes. Compilers improve and update to later standards. Your code needs to keep up. So, for each Fedora release, we run an “Fails To Build From Source” pass, which rebuilds every package in the distribution, using the packages in the distribution. We started the Fedora 12 development cycle with about 400 packages which couldn’t build (still, less than 5% of the total packages) for various reasons. Over the last few months, members of the Fedora Packager community have been whittling away at these, fixing their packages, sending patches to their respective upstream projects, and therefore improving the quality of the open source ecosystem as a whole.

The result?  You see immediate improvements (smaller package sizes due to new compression methods being used, future-proof security through the use of stronger hashes to guarantee package integrity), and increased flexibility should you wish to remix Fedora for your own purposes.

Thank you packagers!

* Truth in advertising: All in this case means 8448 of the 8485 packages in the Fedora 12 tree. There are 37 problematic packages (0.4%), none critical to a vast majority of users, which still need some love.

Installing Fedora 12 and saving the environment

If you’re like me, chances are you have a system or three with DVD / CD burners in them.  Aside from their use for backups, I have tended to use my burners to create Linux install DVDs, done my install, and then given it to someone else, or (ashamedly) thrown it away.  What a waste.

I also prefer to do network-based installs, where I don’t have to download a whole 4GB DVD image, or even 700MB CD image, and burn it.  Instead, I download the 160MB “netinst” network install ISO, burn that to a CD, boot that CD, and point the installer at a Fedora mirror to grab all the packages.  This works great, but still, I’m left with a netinst CD when I’m done that I may no longer need.

Enter isohybrid, new in Fedora 12 (Beta).  I’ve got a few USB keys of various sizes, most larger than 160MB.  Instead of burning a CD (which I can still do, the process is unchanged), I can write the netinst ISO file directly to a USB key, and boot it.  Amazing!

Give it a try when you install Fedora 12 Beta, and save one more CD from becoming landfill.

$ wget http://download.fedoraproject.org/pub/fedora/linux/releases/test/12-Beta/Fedora/x86_64/iso/Fedora-12-Beta-x86_64-netinst.iso
$ sudo dd if=Fedora-12-Beta-x86_64-netinst.iso of=/dev/sdc bs=1M
$ eject /dev/sdc

Replace /dev/sdc with the actual device name of your USB key. You will want to unmount any file systems that are mounted on that key before writing to it.

Then boot that USB key, and you’re off to the races. When prompted for which local file system contains your install image, simply click “Back”, select the “URL” install method, and use a URL of your favorite mirror.

Special thanks to H. Peter Anvin for writing isohybrid and including it in syslinux.

TPMs are good for something

TPMs (Trusted Platform Modules) have long been avoided on Linux, given that their primary use cases have historically been around licensing and Digital Rights Management, concepts which are mostly foreign to Free and Open Source software.  However, as new use cases, such as “trusted boot” have emerged, developers have added TPM device drivers to the Linux kernel to enable these uses.  One often-overlooked feature of the TPM is that it has a hardware pseudo-random number generator.

A while back, Jeff Garzik and others were discussing this on the linux-kernel mailing list (summarized on LWN.net), where it was suggested that the TPM could be used to feed the rngd (random number gathering daemon) tool, just as it reads from other hardware random number generators.  The rngd program reads from hardware-based random number generators and feeds entropy into the kernel’s entropy pool.  Easy in concept, but lacking in TPM implementation.

As it happens, quite a few Dell systems include a TPM chip, including the PowerEdge 11G servers such as the R610 and R710.  So, I asked Dell’s crack team of Linux developers to see what they could do.  The result: a patch to rngd which adds the TPM as another source of random numbers for feeding the kernel’s entropy pool.

We’re working with Jeff to get this patch applied to the rng-tools upstream sources, and from there into the various distributions as their schedules permit.

So, should you find yourself running out of entropy on your servers, and not having a keyboard or mouse attached as ways to feed the entropy pool, you can run enable the TPM in BIOS SETUP, run rngd, and never lack for randomness again.

Google Voice: Why do I need a home phone?

For the past 3 months I’ve been using Google Voice, and I must say, I like it.  But I’m not exactly using it as intended.

I’ve had the same home phone number for 10 years.  A lot of people have that number.  Not a lot of people call it (what that says about my popularity I don’t really want to know), and we don’t make that many outgoing calls a month, but the thought of changing it everywhere is daunting.  More so for anyone with a number for even longer.  I’ve started doing so, but only opportunistically.

What to do?  I don’t want to give up my home number, and I can’t yet transfer my number to Google Voice.  And in theory, I get a discount on my phone/cable/internet by having all three, they’d charge even more for having just two.

My trick?  Time Warner offers unlimited free call forwarding.  So, my home number forwards to GV.  GV then forwards to my cell phone, email, Celeste’s cell phone, etc.  I dropped the voicemail from TW, as now GV takes care of that.  And I can drop the long distance with TW and use GV for that too.  Everything works great.

At some point, when I can transfer my home number to GV and have two numbers for the account (old home number and new GV number I’ve been giving out), and if TW’s rates change again so it’s cheaper to drop their phone service, I will.  Or they will get enough competition to realize that for a couple dozen calls a month, charging $$ for phone service won’t work and they just throw it in for free.  Here’s to hoping.

MirrorManager automatic local mirror selection

MirrorManager 1.3.2 (plus a hotfix) is now running on all Fedora Infrastructure application servers.  This brings one new interesting feature – automatic mirror detection.  How’s that you say?

As you know, Internet routing uses BGP (Border Gateway Protocol), and Autonomous System Numbers (ASNs) to exchange IP prefixes (aa.bb.cc.dd/nn) and routing tables.  By grabbing a copy of the global BGP table a few times a day, MM can know the ASN of an incoming client request, and Hosts in the MM database have grown two new fields: ASN and “ASN Clients?”.  MM then looks to see if there is a mirror with the same ASN as each client, and offers it up earlier in the list.

I’ve pre-populated the MM database, for public servers only, with ASNs, and set “ASN Clients?” = True, meaning such will offer to serve all clients on the same ASN.  If you have a private server and wish to do likewise (remember, this doesn’t work for home systems or those behind NATs), you can fill in those fields yourself.  The Fedora wiki page on mirroring gives an example on how to look up your ASN.  I recommend this for all schools, research organizations, companies, and ISPs.

The mirrorlist lookup code now goes in preferential order:

  • same netblock
  • same ASN
  • both on Internet2
  • same country
  • same continent
  • global

For ISPs and schools, this should mean that most of the possible Fedora traffic will stay within your network – no transit costs.  And as netblocks change, MM will keep up with them automatically.

To see this in action, try a query as such, and look for the ‘Using ASN ####’ in the result comment line.

$ wget -O – ‘http://mirrors.fedoraproject.org/mirrorlist?repo=fedora-11&arch=i386’

# Using preferred netblock Using ASN XXXX country = US country = MX country = CA

I hope you enjoy this new feature.

Fedora services IPv6-enabled

As Mike McGrath, Fedora Infrastructure team lead announced last week, several Fedora services are now IPv6-enabled.  Thanks to our good friends at ibiblio.org, who have native IPv6 connectivity, we were able to set up one web server and one DNS name server, with more services to come over time.  The web server in particular means that nearly all Fedora Infrastructure-hosted web pages and web applications are immediately reachable over IPv6.  This week, over 5000 unique IPv6 addresses have been served.

However, this has not come without a cost.  There have been a handful of individuals having difficulty reaching our web pages.  In one case, the user needed to lower the MTU (maximum transmission unit) for his ethernet adapter from the default 1500 to 1472, to accommodate both IPv6 and his PPPoE connection.  For others, particularly those using 6to4 routing (the default method in Fedora if you don’t already have native IPv6 connectivity), some packets are getting dropped elsewhere on the Internet (pings reach our server, responses don’t make it back).  These are the growing pains we’ll have to live through, and which will resolve themselves over time as more network operators deploy native IPv6 to their end users.

If you have troubles reaching Fedora web sites, take a look at the Known Problems section on our IPv6 wiki page for common workarounds, add your own workarounds as you find them, and if all else fails, join us in #fedora-admin on irc.freenode.net for assistance.  There’s not a lot we can do about the wider Internet and its routing, but we’ll help if we can.

If you’d like to help get additional services IPv6-enabled, check out our IPv6 page for tasks we’d like to do, and offer your own ideas.

CDs are Dead. Long live CDs.

I was running some stats on the Fedora 11 release, and an interesting thing caught my eye. Very few people are downloading the six (or in the case of PPC, seven) CDs to perform a “Fedora” install. Very Very few. In fact, at most, six people downloaded split media CDs using the Fedora mirror servers in the first few days. This in contrast to the over 234,000 direct downloads of DVDs and LiveCDs in the same amount of time. BitTorrent statistics are a little better for CDs: 908 completed downloads of the split media CDs, out of 41,235 total downloads (or ~2.2 %).

Which leads to the question, “Do we really need split media CDs for Fedora 12?”

A few more points lend credence to this idea.

Looking only at the BitTorrent stats for Fedora 9, 10, and now 11, we see an interesting trend. Figure 1 shows that the interest in split media CDs has been decreasing over the past year.
Figure 1

I have a suspicion. As the number of x86_64 users grows, it’s more likely that x86_64 systems will have DVD readers as opposed to older CD readers. Figure 2 shows the growth of x86_64 vs x86 over the past year, again extracted from BitTorrent statistics.
Figure 2

The entire Fedora 11 release as sent to the mirrors is ~143GB. Of that, CD and DVD ISOs represent ~34GB; the split media CD ISOs represent ~15.5GB of that. As most of the rest of that 143GB is all hardlinked, we’re really only transferring out all these ISO files. 10% of the disk space, and 45% of the time/bandwidth needed to get a release out to the mirrors, for about 2% of the user base, and declining.

CDs had their place, back when DVD readers weren’t commonplace, and before we had LiveCD/LiveUSB medias. Now, DVDs are fairly common, the LiveCDs work great for a lot of installs, and we have both a small (158MB) network-based bootable CD installer for new installs that would require a CD, and preupgrade for upgrading from an older distro version to the next. Let’s kill off split media CDs for Fedora 12.

Your thoughts?

Fedora 11 Metalinks!

I didn’t manage to get these onto http://get.fedoraproject.org/, but we have metalinks available for all of the Fedora 11 main content, as well as the Fedora Electronics Lab spin.  Metalinks can be used with metalink-aware download tools, like aria2 and the DownThemAll! FireFox plugin, to let the end user tool decide from which mirror to download the actual content.

Fedora 11 i686 Live CD images:

Live Desktop i686
Live KDE i686

Fedora 11 x86_64 Live CD images:

Live Desktop x86_64
Live KDE x86_64

Fedora 11 i386 CD and DVD images:

Network Install

Fedora 11 x86_64 CD and DVD images:

Network Install

Fedora 11 ppc CD and DVD images:

Network Install

Fedora 11 Fedora Electronics Lab spin Live CD images:

FEL Live i686
FEL Live x86_64

Fedora 11 Source Code CD and DVD images:


Fedora Elections: Voting now open

I’d like to take a moment to thank everyone involved in this Fedora election cycle.

Moderators: John Rose, Max Spevack, Chris Tyler, and Paul Frields
Questionnaire coordinator: Thorsten Leemhuis
Election application: Nigel Jones
Fedora 12 Naming Process: Josh Boyer

and of course the 5 individuals running for the Board seats and the 11 running for the FESCo seats.  I appreciate the efforts you put into attending the Town Hall sessions, answering the questionnaire, and for the commitment you’ve shown to Fedora already.

Fedora Voted

Fedora Voted

You have until 2359 UTC on 22nd June 2009 to vote.