Next article: Friday Q&A 2009-01-09
Previous article: Friday Q&A 2008-12-26
Tags: fridayqna privateapi
It's a new year, and that means a new Friday Q&A! This week I'm going to take Steven Degutis's suggestion and discuss the ups and downs of using private APIs.
Getting Started
I'm not going to discuss what private APIs are out there or how to figure them out, as that would cause me to badly miss my deadline and make this thing way too long. Instead I just want to address this question: should you use them at all, and if so, when?
There are two pretty obvious extremes to the answer, and a lot of people who believe each end. One extreme is that private APIs should never be used, period, full stop. They're bad, don't want to touch them, don't even acknowledge that they exist. The other extreme is that they're fine and dandy, use them like you'd use anything else.
As with most things, I believe the truth lies somewhere in the middle. But where, exactly, and how do you determine if something is worth using?
First let's review the disadvantages, which you're probably familiar with already. An API is essentially a contract between the creator of the API (generally Apple in the context of this blog) and the user of that API. When an API is public, the creator promises not to change that API in an incompatible fashion. With private APIs no such promise exists, and they can change at any time. This change can cause your application to malfunction, crash, or refuse to start.
And the advantages? Well that one's easy. Private APIs let you do stuff you couldn't otherwise do.
So like most of engineering, it's a tradeoff. You have benefits and disadvantages, and you have to decide which one is more significant.
Elements of the Tradeoff
Using a private API is, ultimately, a maintenance issue. (Except on the iPhone, where it's a legal issue, but that's outside the scope of this post.) If you use nothing but public APIs, your app is basically guaranteed to work forever. (Where "forever" really means "until Apple decides not to maintain backwards compatibility anymore". But note that ancient PowerPC-only Carbon apps still run on the latest Mac OS X, and that Classic didn't disappear until 10.4; Apple still keeps old stuff working for a good long time.) If you use a private API, your app is likely to break at some point.
But when? That's one of the big questions you need to answer. There are basically four levels to consider:
- Never. Sometimes a private API may be so fundamental and so widely used that it gets essentially fixed in stone despite not being public. A good example of this on Mac OS X is the
machAPIs, which are technically private but which underly everything at a very fundamental level. - Major releases. Most private APIs fall into this category, where you can be reasonably (although never 100%) confident that they will continue to work throughout the lifetime of the current major OS release. In other words, it will keep working on 10.5 but is likely to break on 10.6. Typically private APIs end up forming part of a support structure for the public APIs and can't be changed without a major reworking of those public APIs, and that only happens with a new major release.
- Minor releases. Occasionally something can't even be relied upon to keep working during the life of a major release. Early versions of LiveDictionary were like this. They relied on fiddly internal details of WebCore's implementation, like C++ method and ivar layout. These offsets were subject to change at pretty much any time, so LiveDictionary generally broke every single time Safari got updated. (Later on public APIs became available that I was able to use instead, which solved the problem once and for all. For more details of what was going on under the hood in those dark days before the public APIs were available, see Hacking C++ From C.)
- Any time. Generally this means that you aren't using the private API right (much more common than with public APIs since you don't have any documentation, you have no guarantee as to the API's requirements, constraints, preconditions, postconditions, etc.) and so is really a property of your usage, not the API itself, but it still happens.
Another big question you need to answer is how bad the break, when it comes, is likely to be. Again, there are different levels to consider:
- No effect. It's unlikely that you'll get here. If there's no effect from having it break, why are you even using the thing?
- Lose a feature. Often you can write your code in such a way that the breakage is likely to be detectable and so you can simply disable whatever feature uses it.
- Crash your app. This is pretty common.
- Crash other apps. For developers of stuff that loads into other programs this is very common, for self-contained processes not so much. LiveDictionary did this. When LiveDictionary broke, it didn't just crash, it crashed Safari too.
And lastly you need to figure out how long it will take you to fix the break. This depends greatly on your skill, your availability, what you're using, how critical it is, how it broke, and other such factors.
Coming to a Conclusion
Now you have enough information to run the cost/benefit analysis. The benefit side is pretty easy. The cost side can be determined by looking at how often you're likely to break, how bad it's likely to be, and how much time and effort it will take to fix. If it's a huge feature and will almost never break and will be trivial to fix when it does, then go for it. If it's a minor feature and will cause huge problems when it breaks every three months, pass. For LiveDictionary, the entire app was built around this feature, so it was worth it even though it required frequent difficult fixes.
Remember that the cost is not just to you, but to your users. If you're really unlucky the break will be so bad that it's not even obvious that it's your fault, and they'll figure it out only after much head-scratching. Once they do figure it out, they will hate you if your fix doesn't come really fast. This means that for a really crucial and breakable feature, you need to stay available and ready to create and release a fix.
Private APIs can be invaluable, but their use must be weighed carefully. Sometimes it pays off very well, and sometimes it's a terrible choice. By carefully examining your app's vulnerability to breakage and your ability to fix it, you can decide whether it's the right move for you.
Tune in Next Time...
That wraps it up for today. Come back next week for another edition of Friday Q&A. Please post your suggestions in the comments or e-mail them (and note that I will use your name unless you say otherwise). I'm running a bit low on topics so send in your ideas! I have a couple left, but you don't want to make me resort to coming up with topics on my own. With a bit of effort on your part, such unspeakable unpleasantness can be entirely avoided.
Comments:
;-)
As of 10.4.6 (the first Intel build), you could no longer rely on all customers having Classic available to them, so to a developer, you could say it was as good as dead at that point.
But that is splitting hairs. PPC builds kept classic through 10.4.11, and it wasn't until 10.5.0 that they lost it.
Historically, Apple obsoleted public APIs with that today would be considered wild abandon: QuickDraw GX, OpenDoc, AOCE, the list goes on. These days they only kill the big stuff, like entire processor architectures or promised 64-bit support for Carbon.
A small but significant number of developers were still building code for Classic when it got axed, although I don't think any of them were building completely new apps for it. Classic was considered an acceptable solution by their customers and why bother learning something new when it works? Of course the answer is because it was going to go away, but not everybody realizes that.
In the modern world, processor architecture shouldn't affect very many people at all, and I view the people who got shafted by the removal of Carbon 64-bit as similar to the ones who got shafted when Classic disappeared. Apple could have been clearer about what they were doing, but the writing was still on the wall from way back.
The whole concept of balancing possible code breakage with how hard/easy it would be to "fix it in the next release" leans rather heavily on the idea of subscription software or that the customers you sell to are going to be yours for life, deeply interested in always getting the latest version of your software, etc. etc.. None of this is actually true for the average software purchaser/consumer however.
The majority of people still buy software as a unitary "thing" as if it was a car or a toaster, and want it to work for a lot longer than the next point release. If as a customer, I was aware that the coder was engaging in these kinds of trade-offs I would purposely never buy from that source. Obviously most won't know, so you can continue with the methodologies that work for you as a developer, but this whole argument seems quite antithetical to the idea of making solid, "good" software that I as a consumer want to purchase.
This method will get you good software that any other developer would think was fantastic. It won't always get you good software in the sense of something that sells and is popular though. You would be purposely selling software that is likely to break with the next major release (or sooner) and relying on the consumer to be pro-active enough to upgrade it, and also not get mad at you for the initial "break."
Not everyone follows their favourite software developers blog, or is so involved in their tools that they want to be in a dialogue with the person who created them. Sometimes it seems that you folks are so wrapped up in "developer world" that you fail to see the forest for the trees.
With the widespread use of built-in updaters, it's entirely reasonable to expect users to stay up to date. This is completely unrelated to "subscription software", which is software which requires periodic payments.
And don't think that avoiding private APIs is a surefire way to avoid breaking when a new version of Mac OS X comes out. Plenty of apps which use nothing but public APIs break. Often it's because they have relied on some behavior which is not guaranteed, typically completely inadvertently. Sometimes it's because Apple has decided that a particular API contract really needs to be broken. Sometimes it's because Apple accidentally broke that contract.
Anyone who has used computers for a while knows that installing a new OS release has a good chance of breaking at least some of their software. If you want to own software for life, then you need to be very conservative with OS updates or be willing to actually keep that software up to date. This is true whether that software uses private APIs or not.
Every developer engages in these trade-offs. If you avoid developers when you're aware that they're doing it, all you do is end up using products from developers who hide it better. Engineering is about tradeoffs, period. You think it's bad with software? Think about the tradeoffs that go into engineering every single car you ride in, bridge you drive or walk over, building you enter, and electronic product you use. Unlike software, those products have the potential to take human life. The engineers for those products are making a complex trade-off between cost, complexity, functionality, aesthetics, and risk to human life. And while they try to hold risk down as much as possible, you can be sure that the number they're assigning to risk is not zero.
Engineering is always about trade-offs. It holds true everywhere. Yes, as developers, we need to take our customers into account when making those trade-offs. But the answer is not to avoid making them, because that's simply impossible.
Thanks for the thoughtful response and I agree (with what I infer is your standpoint), that perhaps I overstated my argument. Despite the undue vigour with which I argued it however, I think that it still stands.
From the point of view of the end consumer, using any API that is likely to change in any *reasonable* amount of time is (IMO) not good practice unless it's absolutely necessary to the design of the software. Of course, the developer is the one that has to make the determination as to what's a "reasonable" amount of time and what's necessary to the application, so in terms of practice, I'm probably not suggesting anything different from you.
As a technician but not a developer, (I haven't done any development work to speak of, but I do create and maintain huge complex databases so some of the same principles apply), I guess I'm not really in a position to argue anything definitively but I would still argue that *most* of the time, the use of private APIs is not always as necessary as it may seem at the time.
It's also true that by doing more work, one can avoid private API's altogether, so unless the amount of work involved is critical to the project, there is always the option of recreating the same effect "from scratch." A recent example is the creation of CoverFlow like interfaces from scratch on the iPhone instead of using the (private) API.
I think my main point was supposed to be more a critique of the concept of thinking you can "fix" things with a later release. In the real world, the customer might just as easily switch to your competitor's product than wait for you to fix a problem that shouldn't really be there in the first place. Anyway, I didn't mean to be hyper-critical anyway (if that's how I came across.)
First, while it's true that a lot of private API usage is fairly gratuitous, sometimes it's really essential. Not all private APIs can be easily replicated, and some can't be replicated at all. If your app depends on one which can't be replicated then you have little choice. The whole point of this analysis is to tell you which is which, and whether it's worthwhile.
Second, in my experience, there are extremely few users who won't give you a few days to fix your software but who instantly upgraed to any new version of Mac OS X that Apple releases. The vast majority of people are happy to keep up with updates. Of those who don't, the vast majority don't keep up with Apple's updates either.
Third, as I pointed out in my last message, avoiding private APIs won't always save you. There's always a chance that a major Mac OS X release will break your software even if you use 100% public APIs.
In any case I welcome disagreement and different opinions so please do continue!
David: From my point of view Carbon was clearly a legacy technology and the smart move was to move away from it as much as possible. But perhaps from the other side this was much less clear. Apple's move to cancel Carbon 64-bit after promising to support it was undoubtedly very crappy and wrong. However when talking about how long stuff stays supported, I'll note that this cancellation didn't kill any shipping code. It made a whole lot of code bases much less valuable and destroyed a bunch of work, but it didn't cause any apps to suddenly break, so from the point of view of how long your shipping code will last, the continuing life of Carbon 32-bit is what counts. From the point of view of which technologies to rely on, obviously there's some kind of important lesson in the Carbon 64-bit cancellation.
Comments RSS feed for this page
Add your thoughts, post a comment:
Spam and off-topic posts will be deleted without notice. Culprits may be publicly humiliated at my sole discretion.