call to software vendors… package it right

One of the tasks that I have been responsible for performing over the last several years is packaging software into RPM Package Manager (RPM) packages.  All of our internal RPMs are fairly simple, the tricky part is the 3rd party software.  There are several problems with the distribution of commercial off the shelf (COTS) software in the Linux ecosystem.

  • They are rarely RPMs
  • Sometimes they use InstallAnywhere installers (more on that later)
  • When they are RPMs they either think they know better than RPM (can be explained as a lack of understanding as well) or they try too hard to make a single RPM that works on all RPM-based distributions.

Before I go further I would like to say, if you are packaging your software as a real RPM (or any other native Linux packaging system), even if it is not perfect, THANK YOU. We appreciate it.  Please take my commentary as constructive criticism.  I am not angry with you, and will gladly help you with packaging issues if I can.  I am not the best either, but I have a fair bit of practice.  I am sure others would gladly assist as well.

Moving on… So today I got stuck attempting to automate the installation and configuration of some of unnamed vendor’s system management RPMs via puppet.  I made the mistake of looking into the scriptlets and was frustrated by some of their practices.  I started to write a package by package evaluation of the scriptlets, but one package in particular would have taken forever (it attempted to cover every possible RPM-based Linux distribution via the scriptlets).  I recalled a conversation I had a year or two ago with the individual who heads up this company’s Linux packaging group, and they had expressed interest in feedback.  At the time I had a few points to provide, but I did not have time for a more in depth analysis.  So I decided to finally write up a general set of bullet points to pass their way, if I am not trying to help then I am part of the problem, right?.  I figured it would not hurt to put it out here as well.  So I am re-wording a touch to make it less specific to just them, and more of a general call to all software vendors.  Also, I would imaging most of what is stated directly translates to Debian, conary, and other native packaging systems; but it is not intended to be a definitive guide.

Things to keep in mind when packaging software for native Linux distribution:

  • It helps to build (compile) your software from source using the packaging tools (RPM), instead of just packaging up the binaries.  You do not have to distribute the source (SRPM, tarball, etc), but the build process can potentially be cleaner and more manageable.  Yes, I know you paid all kinds of money for some fancy handle everything build system.  Are your customers (the system administrators) happy with the output?  I know I am not.
  • With the modern build processes and systems available there really is no good reason to build a mangled cluster of scripts that attempt to make one platform independent RPM instead of building distribution specific RPMs.  They can even still come from the same consolidate spec files, thus allowing you to reduce duplicate work.
  • Scriptlets
    • Should be short and concise
    • Almost the entire build and installation of the software should occur in the %build and %install sections, respectively.
    • Any setup or file layout should be handled in %build and %install sections, if its host specific it should be a documented post-install exercise for the admin.
    • User/group creation, symlinking files, chkconfig and service commands are all acceptable.
    • Should never touch a file owned by another package.  If it needs a setting or switch flipped, document it for the administrator. At worst, include a “check” or “setup” script that the administrator can run manually if they want you to do the work for them, cause the majority of us don’t.
  • Files should only be placed on the system via RPM’s built in methods.
    • Symlinks can be a caveat of this, but should not be abused. UPDATE 2014.03.19 – Retracting since I just built a package that managed symlinks right. No excuses. :)
    • File ownership and permissions should be set by the RPM, not in the scriptlets.
    • Do not provide library files that the OS already provides.  Get past the “i need this specific version, and no other will do” or “well we used that one but slightly modified” mentality, and when you can’t, then require the correct compatible library.  Most distributions do provide them already.  If you needed to modify it, are you properly following the licensing? Wouldn’t it be better to just submit a patch and stop having to maintain it and not worry about licensing issues?
  • You should not be adjusting security settings on the system for the administrator, you can provide them (i.e. SELinux policy files, default firewall rules, a file for /etc/sudoers.d/, etc), but implementing them for me is bad security at its worst.
  • If you provide an SELinux policy that does not change any existing policies on the system directly, you can implement that.  But if you change something existing, let us do the work so that we are aware.
  • Do not flip SELinux booleans related to other bits, let the admin or find the right way.
  • Get help… it is out there.  There is selinux mailing lists, and ya know what? Call Red Hat.  They helped get SELinux going, they know what to do.  You are not in charge of my system’s security, I am.  If you need a change to an existing policy, talk to that policy’s maintainer or implement it in your own policy.
  • Users and groups – for the most part this hasn’t looked bad except:
    • Deleting and re-creating a user is a annoying thing to do.  If the admin changed something about the user and everything still functions, don’t touch it.  If I need to fix it I can delete the user myself and then let you create it by re-installing, or just look at your newly clean scriptlets to discover the exact syntax.
    • Technically, you should not be deleting any user you create on the system… another exercise for the admin to avoid stranded files.
  • If your company decides not to supply good RPMs, consider a tarball(s) with an simple install script, we can do the rest.  I was avoiding naming names, and I hate to give this software credit, but IBM’s DB2 has got to be the easiest 3rd Party software I have ever packaged and deserves credit for the fact that there was just a bunch of tarballs and a few commands to install.  Setup was another matter. heh.

To summarize, write the RPMs according to a public packaging guideline, such as the Fedora Packaging Guidelines.  If Fedora would accept it into the EPEL repositories then you have succeeded.  I realize you want your packages to support other distributions, but odds are that if the spec file for the RPMs are cleaned up to meet Fedora’s guidelines, the other distributions should be easy to support, and us administrators would be ecstatic.  Plus the use of tools like mock, koji, OpenSuSE Build Service, etc can greatly ease build and distribution issues.

Since I already named one name, might as well point out a negative one.  Flexera’s InstallAnywhere… So InstallAnywhere is a universal installer with an interesting feature.  Their site claims: “Install native packages, such as RPM, on Linux, Solaris, and HP-UX“.  This is inaccurate, at least towards RPM.  What they produce is a Java-based installer that injects RPM metadata into your system’s RPM database.  This is not an RPM.  We can not distribute this software via Yum or Spacewalk or Red Hat Network Satellite.  They should be ashamed. :(

So some useful reading:

UPDATE: Some basic grammar fixes. 2011.10.19

cobbler webUI cleanup

Related threads:

I’ve been pushing get get cobbler implemented as part of our build infrastructure at work, and there have been very small little bits about the web interface that I’d like to see improved.  After a few conversations with James Cammarata and throwing together a few mockups I decided to site down start working on the change.

I’ve thrown up a github repo for the changes so people can follow along.   At some point I’m going to write up a bit detailing the thoughts behind some of the changes.

getting func-y

So it happened.  What you ask?  A very long anticipated release.  Gnome 3! Wait, no… well, yes, Gnome 3 was released, and is kinda kewl and you can try it.  I’m not totally sold on it yet though, maybe I’ll review it sometime or another. Oh wait, what was I talking about? Yes! The much anticipated release! It was not Gnome 3… it was Func and Certmaster!  They have officially reached 0.28.  Ya.. not as fancy of a version number as 3, but.. hey it gives us something to aspire to.  The releases will be making their way into EPEL-testing soon, but can currently be downloaded in tarball form or from koji.[1]

What are Func and Certmaster? Hrm.. well… so basically they are a set of programs that work together to let a administrator tell N number of servers what to do at the same time, and it will give back a report of what happened.  That is the 30,000 ft view.  Up close and personal, well… its a programmatic systems management tool that allows you to perform actions across large numbers of systems in an orderly fashion that allows great extensibility and control; which also exposes the basic functionality via a cli so that you can run quick one liners without to much effort.  Loose anyone there?  Don’t worry, I loose my self trying to follow the code in these programs some times, that’s all part of the learning process.

Why do I suddenly care?  Well, its not suddenly really.  I’ve been following func for a very long time.  I’m not sure if I heard about it before the 2008 Red Hat Summit or not, but it was sometime around there.  Its a very useful and wonderful tool, and due to my semi-regular use of it I had started sending bug requests reports and patches to the mailing list.  Recently, I was asked if I was willing to channel some of that energy into helping develop func and certmaster directly.  I was very happy to accept, if a bit trepidatious.  After several weeks of learning git and my way around some of the code, I was happy to help Seth Vidal (the primary maintainer) get func and certmaster ready for a new release.

We’ve still got lots of work on our plate, but its been a very useful experience for me, and i’m glad to contribute back to an open source project so directly for once.

[1] Certmaster on RHEL 5 does require a patched, non-supported pyOpenSSL package until RHEL 5.7 is released.  See the release announce thread for more information.