MirrorManager at FUDCon Lawrence

Two weeks ago I once again had the opportunity to attend the Fedora User and Developer Conference, this time in Lawrence, KS.  My primary purpose in going was to work with the Fedora Infrastructure team, and develop a plan for MirrorManager maintenance going forward, and learn about some of the faster-paced projects that Fedora is driving.

MirrorManager began as a labor of love immediately after the Fedora 6 launch, when our collection of mirrors was both significantly smaller and less well wrangled, leading to unacceptable download times for the release, and impacts to Fedora and Red Hat networks and our few functional mirrors that we swore never to suffer or inflict again.  Fedora 18 launch, 6 years later, was just as downloaded as before, but with nearly 300 public mirrors and hundreds of private mirrors, the release was nary a blip on the bandwidth charts, as “many  mirrors make for light traffic”.  To that end, MirrorManager continues to do its job well.

However, over the past 2 years, with changes in my job and outside responsibilities, I haven’t had as much time to devote to MirrorManager maintenance as I would have liked.  The MirrorManager 1.4 (future) branch has languished, with an occasional late-night prod, but no significant effort. This has prevented MirrorManager from being more widely adopted by other non-Fedora distributions.  The list of hotfixes sitting in Fedora Infrastructure’s tree was getting untenable.  And I hadn’t really taken advantage of numerous offers of help from potential new maintainers.

FUDCon gave me the opportunity to sit down with the Infrastructure team, including Kevin, Seth, Toshio, Pierre, Stephen, Ricky, Ian and now Ralph, to think through our goals for this year, specifically with MM.  Here’s what we came up with.

  1.  I need to get MM 1.4 “finished” and into production.  This falls squarely on my shoulders, so I spent time both at FUDCon, and since, moving in that direction.  The backlog of hotfixes needed to get into the 1.4 branch.  The schema upgrade from 1.3 to 1.4 needed testing on a production database (Postgres) not just my local database (mysql) – that revealed additional work to be done.  Thanks to Toshio for getting me going on the staging environment again.  Now it’s just down to bug fixes.
  2. I need not to be the single point of knowledge about how the system works.  To that end, I talked through the MM architecture, which components did what, and how they interacted.  Hopefully the whole FI team has a better understanding of how it all fits together.
  3. I need to be more accepting of offers of assistance.  Stephen, Toshio, and Pierre have all offered, and I’m saying “yes”.  Stephen and I sat down, figured out a capability he wanted to see (better logging for mirrorlist requests to more easily root cause failure reports), he wrote the patch, and I accepted it.  +1 to the AUTHORS list.
  4. Ralph has been hard at work on fedmsg, the Fedora Infrastructure Message Bus.  This is starting to be really cool, and I hope to see it used to replace a lot of the cronjob-based backend work, and cronjob-based rsyncs that all our mirrors do.  One step closer to a “push mirror” system.  Wouldn’t it be cool if Tier 2 mirrors listened on the message bus for their Tier 1 mirror to report “I have new content in this directory tree, now is a good time to come get it!” , and started their syncs, rather than the “we sync 2-6 times a day whenever we feel like it” that mirrors use today ?  I think so.

Now, to get off (or really, on) the couch and make it happen!

A few other cool things I saw at FUDCon I wanted to share (snagged mostly from my twitter stream):

  1. OpenLMI = Open Linux Management Infrastructure software to manage systems based on DMTF standards. http://del.ly/6019VxOl
  2. Mark Langsdorf from @calxeda is demonstrating the ECX1000 #armserver SoC based build hardware going in PHX at #fudcon pic.twitter.com/hgfo2mO7
  3. @ralphbean talking about fedmsg at #fudconhttp://del.ly/6015VxTD . I need to think about how @mirrormanager can leverage this.
  4. Hyperkitty is a new Mailman mailing list graphical front end, bringing email lists into the 21st century.

I look forward to next year’s FUDCon, wherever it happens to be.