Fedora is Self-Hosting

Fedora 12 (Beta available now), is self-hosting.

What does this mean? Simply put, it means that you can use a copy of Fedora 12 to rebuild, from source, all* of Fedora 12 again.

Why is this important? One of the key tenets of Free and Open Source software is that anyone can get a copy of the source code, make modifications to it, built it, an use the modified version. Simply publishing the source code, without also allowing people a way to rebuild and use that code, doesn’t accomplish this goal.

Source code tends to bitrot over time. Libraries that your code uses will change, get updated, add features and bugfixes. Compilers improve and update to later standards. Your code needs to keep up. So, for each Fedora release, we run an “Fails To Build From Source” pass, which rebuilds every package in the distribution, using the packages in the distribution. We started the Fedora 12 development cycle with about 400 packages which couldn’t build (still, less than 5% of the total packages) for various reasons. Over the last few months, members of the Fedora Packager community have been whittling away at these, fixing their packages, sending patches to their respective upstream projects, and therefore improving the quality of the open source ecosystem as a whole.

The result?  You see immediate improvements (smaller package sizes due to new compression methods being used, future-proof security through the use of stronger hashes to guarantee package integrity), and increased flexibility should you wish to remix Fedora for your own purposes.

Thank you packagers!

* Truth in advertising: All in this case means 8448 of the 8485 packages in the Fedora 12 tree. There are 37 problematic packages (0.4%), none critical to a vast majority of users, which still need some love.

8 comments on this post.
  1. Ka-Hing Cheung:

    What kind of errors are common, and what kind of systems built those binaries that wouldn’t build on F12? F11?

  2. mdomsch:

    They were built on Fedora 11, a few on 10, and at least one that couldn’t rebuild since Fedora 9. In the F11-F12 transition, we picked up a new version of GCC, which was stricter with C++ header files than before, and that tripped up quite a few applications.

  3. Jon:

    Excellent stuff. Nice to see some of the good ideas from Debian percolate through the community.

  4. Max:

    This is awesome, Matt. I think that the self-hosting feature, if you want to call it that, is a very important one. Congrats on achieving it for F12.

    How close did past releases of Fedora come?

  5. mdomsch:

    Jon: you bet. I had credited Debian for the FTBFS moniker and concept when I started doing the same in Fedora.

    Max: while the exact numbers are lost to history, here’s what BZ shows, we had about 200 FTBFS bugs opened during the F10 timeline, about 400 during the F12 timeline, and in total, about 1440. Of those, we’re down to about 40 left, and dropping.

  6. Ajamison:

    Ok so you talked about what self-hosted ment but how do you rebuild the entire distro from the distro without running an individual build on all 8k packages. Also can this method be use to back port to say a i386 arch from the default i686 if need be?

  7. mdomsch:

    Ajamison: yes, we rebuilt all 8k packages. I have a set of Dell PowerEdge blade servers I use to rebuild all the packages, it takes about 30 hours. In addition, the Fedora Release Engineering team scheduled a rebuild of all the packages using the koji build system and Fedora’s own build servers.

    In fact, for F12, one of the changes that was picked up during the rebuild was to bump the architecture from i386 to i686. http://fedoraproject.org/wiki/Features/F12X86Support so yes, you could do likewise to backport all the packages should you so desire. i386 could be done as a Secondary Architecture http://fedoraproject.org/wiki/Architectures#Secondary_Architectures if there were sufficient contributor manpower and interest to do so.

  8. MartinG:

    Cool. This reminds me of “Reflections on trusting trust”: