To be even more specific, the problem is that the packaging systems for python etc are written by programmers rather than systems administrators, and programmers typically have little or no interest in the system that their application (or library or whatever) is running on, their interest and their focus is on THEIR application.

The classic example is that sysadmins see important systems stuff running out of someone’s home directory and dependent on their idiosyncratic environment as a hanging offence, while programmers can’t see what the problem is or what the fuss is about.

This is, of course, an exaggerated generalisation – but there’s enough truth in it to indicate the source of the problem.

It probably explains why perl and CPAN modules integrate well with systems – perl is a sysadmin’s language, while python is a programmer’s language. The difference in focus results in a very different point of view and attitude towards systems.

Aslo, as a sysadmin and as a user, I *don’t want* a dozen different incompatible versions of libfoo installed. I want one, and I want it to work, and I want all programs that use it to work with that version. If appA requires libfoo 0.1 and appB requires libfoo 0.2 then I regard that as a bug in appA that needs to be fixed – it isn’t keeping up with developments in an important library that it depends upon. I most emphatically do not see it as a sufficient reason to have both libfoo 0.1 and libfoo 0.2 installed except in special circumstances (at least C libraries allow different sonames for different lib versions – perhaps that’s something that other languages should learn from).

If a developer happens to need a different version while they’re working on something then installing and using it from their home dir is entirely appropriate – but when their work is ready for use by others, it should integrate with the existing system (i.e. use the system version of libfoo, or create a new system package for the latest libfoo if required) rather than require everyone else to duplicate the developer’s idiosyncratic environment.

These are all lessons that we learnt during the transition from proprietary unixes in the 80s and early 90s to package-managed linux distributions in the mid 90s and later – faffing about with hunting for and manually downloading and resolving dependencies and conflicts is a major PITA, something that the system should handle. We seem to be in the process of forgetting that important lesson.

(btw, there seems to be a problem with your comment field. cuts off the RHS of the comments, no matter what font size i try. width specified in pixels, perhaps.)

There’s definitely a part of me that feels that this is an appropriate design tradeoff. I wonder whether one system can actually meet both sets of needs without being so complex as to be unattractive to one or both audiences.

I think this is a case of having a very fine hammer, and trying to apply it to unsuitable nails. The Debian system’s goal is to produce a coherent installation of OS and applications which is maintained in a simple and consistent way. It does this job well enough that the end result is consumable by casual users (e.g. Ubuntu) as well as experienced system administrators. In contrast, systems like nix and conary seem to serve a much narrower audience.

I would not argue that this approach is without its shortcomings, but I think it has emerged as a solution to many of the problems described here.

The mistake here is to assume it makes any sense whatsoever for a user, who happens to be a developer, to expect to be able to use system dev tools for userspace development work.

It doesn’t.

The default assumption should be that if a user wants to do development work they should install the dev tools they need as applications, not as system components. Traditionally Un*x systems have included dev tools such as perl and python interpreters and exposed them for the use of users, but this blurs a vital distinction, as the situation you describe shows. This is because developers traditionally have special status in Un*x culture as semi-system level users.

Going forward there needs to be a sharp, clear and rigorously enforced distinction between components that are part of the system and components that are user applications. I might even go so far as to split out the system shell interpreters from user shells and package those as applications. After all, suppose my system is running bash v3 and I want to use bash v4, or zsh, or fish?

All the successful user space platforms (Windows, MacOS X, iOS, Android, etc) make a sharp distinction between system component installation and update management and application installation and update management, and for very good reasons.

Applications should not ever (if at all possible) depend on the presence of optional system level components. Dependency management just shouldn’t be a user application level concern.

I honestly believe this is the major problem holding back Un*x systems from becoming mainstream platforms. Modern Un*x package managers are great, powerful systems with lots of very clever features, but the requirements for the installation and management of third party applications, especially commercial ones, are very different. I firmly believe Un*x will never have a truly strong, vibrant third party desktop app ecosystem until the rpms and debian packages of this world are firmly pushed back into the system maintenance role and a robust system for third party user application management that eliminates dependency management is agreed upon.

Most (binary, implemented in C or C++) libraries are packaged in distros exactly that way to support run-time dependencies on a specific ABI. Am I misunderstanding?

I’d say that this is a primary reason why local installs are attractive to end users even when they are using a mature system such as Ubuntu with thousands of packaged libraries – they get greater functionality with less failure rates than when using dpkg packages.