27 January 2009

Too little, too late: plug-ins for gcc

The GCC Steering Committee announced that they have created a new license exception that will---among other things---allow for the creation of a plug-in framework in GCC. In effect it will create a means by which compiler writers, researchers and tinkerers can write additions to the compiler without having to understand the vast majority of GCC.

I argue this is too little, too late.

GCC is open source software. This means that they give everyone the blueprints, so anyone can (potentially) go out and modify it to suite their needs. Such modifications might be fixing a bug, adding a new feature, or making it work slightly better in their situation. In that respect, GCC has done a wonderful job on providing a platform on which innovation can occur. But...

Software can be complicated. People have dedicated their careers to making software easier to write, understand, modify and maintain, or even prove that software does what it should. Despite all the effort, there's still no silver bullet. As software gets more mature---with more features and more compatibilities to maintain---it gets worse. And as software goes, compilers are typically some of the most complicated to write (not to toot my own horn as a compiler writer).

GCC suffers from the trifecta of software nightmares. GCC is as old as I am, and uses ancient techniques such as an RTL intermediate representation. GCC is feature-rich; its standard release supports at least seven source languages and at least twenty target architectures. GCC is a compiler. Take a look inside and you'll find a mind boggling nine hundred eighty four thousand lines of code (according to conservative estimates by David Wheeler in 2001). I'm not saying that GCC is bad software, but that it would be very difficult for even the most talented programmer to learn its internals in a short time.

Software engineers have seen this growth pattern before, and have come up with schemes to try to mitigate this risk. One such scheme is a plug-in framework. In a plug-in framework, the original developers present small, clean, easy to understand interface, and then anyone can write a plug-in that talks to that interface. This plug-in can do everything it needs to do, but it is significantly easier to create a plug-in than it is to modify the original code. Because of the constraints on the interface, the programmer can be blissfully ignorant of the operation of majority of the software.

Or, as an analogy: suppose you are a doctor, and your patient has a broken ankle. It's a really bad break, and you need to operate. So, you decide to remove all of the patient's skin so you can get to the break. Oh wait, that would be ridiculous... Instead, you only open up the ankle... much less risk of disturbing something else.

With compilers, there are some easy ways to support plug-ins. The easiest is to write the intermediate representation to disk in some form, let the plug-in do whatever to it, and then read it back and verify that it is well-formed. Implementing this sort of plug-in framework is very little additional work; in any sane software effort, this might be one of the earliest things designed, if for no other reason than convenience during the debugging phase.

But, for some reason GCC has not supported plug-ins until now. Twenty five years of software enthusiasts suggesting it, and the project hasn't done it. Why would GNU ignore feature requests, and even software engineering best practices?

From the GCC Exception FAQ:
...For a while now, the GCC developers have considered adding a plugin framework to the compiler. This would make it easier for others to contribute to the project, and accelerate the development of new compilation techniques for GCC. However, there have also been concerns that unscrupulous developers could write plugins that called out to proprietary software to transform the compiled code—effectively creating proprietary extensions to GCC and defeating the purpose of the GPL. The updated exception prevents such abuse, enabling the GCC team to look forward to plugin developments. [emphasis is mine].
Yes, that is correct. They were afraid people would write software plug-ins---completely separate from GCC---and that those plug-ins would not be free software. They think that any software that interacts with GCC should be governed by GCC's license. In effect, the GNU and the GCC project tried to strong-arm other developers into writing free software. I'm all for free software, but I think this was (1) a prick move, and (2) the wrong decision from a project management perspective.

Why is this a prick move?

In short, I don't like having my hand forced. Let's ask again: what is open source? According to the Open Source Initiative's (OSI) definition (which is about as close as we'll get to a standard, accepted definition):

6. No Discrimination Against Fields of Endeavor

The license must not restrict anyone from making use of the program in a specific field of endeavor. For example, it may not restrict the program from being used in a business, or from being used for genetic research.


8. License Must Not Be Specific to a Product

The rights attached to the program must not depend on the program's being part of a particular software distribution. If the program is extracted from that distribution and used or distributed within the terms of the program's license, all parties to whom the program is redistributed should have the same rights as those that are granted in conjunction with the original software distribution.

9. License Must Not Restrict Other Software

The license must not
place restrictions on other software that is distributed along with the licensed software. For example, the license must not insist that all other programs distributed on the same medium must be open-source software.


Although this definition has been carefully crafted to dodge the question, it seems to me that GNU could not place the no-proprietary-plug-in restriction in their license, and so instead they mutilated their code to discourage such unscrupulous behavior.

But, ultimately, my code is my code, and my code is separate from GCC. The GCC project cannot accept that, and so they try to prevent me from writing it.

Why was this a bad project management decision?

For all of the reasons that make plug-ins a good idea, and then some.

I presented the notion of plug-ins as a best practice---and they are---but realize that all best-practices are just rules-of-thumb. We could never get universal agreement that GCC is bad code because it doesn't support plug-ins. In my opinion, the lack of this feature is a flaw in GCC, but there are bigger problems.

At the end of the day, I don't care if open source software is good code. Authors can write code however they want to. But it doesn't mean I'm going to take part in their software project if they are working against me. If you write code to prevent me from participating, then it cannot be open source.

As academics researching compilers (that's fun to say), we have simply moved away from GCC as an experimentation platform. There are compilers with less restrictive licenses and which encourage third party development. This, ultimately, is GCC's loss and not ours.

Take for instance llvm, introduced by Chris Lattner's thesis in 2002. The llvm project not only has better documentation (including howtos on writing plugins), but also has a bit of a social contract. From llvm intermediate representation documentation:

The LLVM code representation is designed to be used in three different forms: as an in-memory compiler IR, as an on-disk bitcode representation (suitable for fast loading by a Just-In-Time compiler), and as a human readable assembly language representation. This allows LLVM to provide a powerful intermediate representation for efficient compiler transformations and analysis, while providing a natural means to debug and visualize the transformations. The three different forms of LLVM are all equivalent. [emphasis is mine].
Or, in other words "yeah, we're not going to pull that crap." Because of all of the llvm project's support, llvm is quickly becoming a popular platform for compiler research, and will assuredly overtake it (if it hasn't already). I'm not saying that we don't use GCC; rather that the next hundred cool optimizations are being developed for other compilers, and will only later be back-ported to GCC.

In summary, an important part of the management of an open-source software project should be to enable new programmers to enter the project with little initial investment. GCC has not only missed this opportunity, but actively avoided it. As a result, GCC has alienated some of the community, and is losing ground to newer open source projects.

22 January 2009

Easily miter tubes

[Update: see version 2]

Have you ever want to join two tubes at some specific Linkangle? For example, imagine you were building a bicycle frame. To ensure a good clean weld, you want to miter one of the tubes so that it perfectly fits against the surface of the other. This, unfortunately, can be a pain in the butt.

So, I created a quick and dirty ruby script to do the math for me. I enter the diameter of the two tubes, and the angle at which they should join, and it draws the curve as a postscript document. I can then print that template, tape it to the smaller tube, and cut along its length.

Say for instance I wanted to join two toilet paper rolls (1.75" diameter) at 30 degrees. I simply type:

$ ./pipemiter.rb 1.75 1.75 30 in >tp.ps

or, if you prefer something more guided and interactive,

$ ./pipemiter.rb >tp.ps
Preferred unit (one of mm, m, ft, in, cm): in
Outer Diameter of larger pipe: 1.75
Outer Diameter of smaller pipe: 1.75
Angle of joint (degrees): 30

and it gives me a postscript document, which looks like this curve on the right.

Perfect 30degree miterAs a proof of principle, I tried it out with two toilet paper rolls
(to your left). They join perfectly!

The source code for the script is available for free.

20 January 2009

Happy OBAMA day!

Or, more importantly, happy NO MORE BUSH DAY!

Know that your people will judge you on what you can build, not on what you can destroy.