Next article: Friday Q&A 2009-11-13: Dangerous Cocoa Calls
Previous article: Friday Q&A 2009-10-30: Generators in Objective-C
Tags: frameworks libraries linking
It's another Friday, and thus another Friday Q&A. I have recovered from the confusion of the Daylight Saving Time transition and am now ready to talk about Mac OS X linking, install names, @executable_path, and friends.
Static Libraries
These are so simple they barely need discussion. When you link against a static library, the contents of that library are copied into your application when you build. From that point on, the code acts just like the code you wrote yourself.
Dynamic Libraries
When you link against a dynamic library, things are less straightforward. The linker basically makes a note that your references to various symbols are to be found in this library, and that your binary depends on that library. Then at runtime, when your application is loaded, the dynamic linker also loads that library.
The big question for today is, how does the dynamic linker know where to find it?
Install name
The answer to that question varies greatly from one OS to another, but on the Mac, the answer is install names.
An install name is just a pathname embedded within a dynamic library which tells the linker where that library can be found at runtime. For example, libfoo.dylib might have an install name of /usr/lib/libfoo.dylib. This install name gets copied into the application at link time. When the dynamic linker goes looking for libfoo.dylib at runtime, it will fetch the install name out of the application and know to look for the library in /usr/lib.
Frameworks are just dynamic libraries with a funny wrapper, so they work the same way. Foo.framework might have an install name of /Library/Frameworks/Foo.framework/Versions/A/Foo, and that's where the dynamic linker will search for it.
@executable_path
Absolute paths are annoying. Sometimes you want to embed a framework into an application instead of having to install the framework into /Library or a similar location.
The Mac's solution to this is @executable_path. This is a magic token that, when placed at the beginning of a library's install name, gets expanded to the path of the executable that's loading it, minus the last component. For example, let's say that Bar.app links against Foo.framework. If Bar.app is installed in /Applications, @executable_path will expand to /Applications/Bar.app/Contents/MacOS. If you intend to embed the framework in Contents/Frameworks, then you can just set Foo.framework's install name to @executable_path/../Frameworks/Foo.framework/Versions/A/Foo. The dynamic linker will expand that to /Applications/Bar.app/Contents/MacOS/../Frameworks/Foo.framework/Versions/A/Foo and will find the framework there.
@loader_path
Finding the executable isn't always good enough. Imagine that you ship a plugin or a framework which embeds another framework. Say, Foo.framework embeds Baz.framework. Even though Foo.framework is the one requesting the load, the dynamic linker will still go off of Bar.app's location when figuring out what @executable_path refers to, and this won't work right.
Starting in 10.4, Apple provided @loader_path which does what you want here. It expands to the full path, minus the last component, of whatever is actually causing the target library to be loaded. If it's an application, then it's the same as @executable_path. If it's a framework or plugin, though, then it's relative to that framework or plugin, which is much more useful.
@rpath
While the above is sufficient for anything in theory, it can be troublesome in practice. The problem is that a single copy of a library can only be used in one way. If you want Foo.framework to work when embedded in an application or when installed to /Library/Frameworks, you have to provide two separate copies with two different install names. (Or manually tweak install names later on using install_name_tool.) This is doable, but annoying.
Starting in 10.5, Apple provides @rpath, which is a solution to this. When placed at the front of an install name, this asks the dynamic linker to search a list of locations for the library. That list is embedded in the application, and can therefore be controlled by the application's build process, not the framework's. A single copy of a framework can thus work for multiple purposes.
To make this work, Foo.framework's install name would be set to @rpath/Foo.framework/Versions/A/Foo. An application that intends to embed Foo.framework would then pass -rpath @executable_path/../Frameworks to the linker at build time, which tells the dynamic linker to search for @rpath frameworks there. An application that intends to install the framework would pass -rpath /Library/Frameworks, telling the dynamic linker to search there. An application that for some reason doesn't want to commit to one or the other at build time can just pass both sets of parameters, which will cause the dynamic linker to try both locations.
Conclusion
Now you hopefully know a little bit more about how dynamic linking works on Mac OS X and how the dynamic linker finds your libraries, including how to embed frameworks in applications, in plugins, and in other frameworks.
Come back next week for another exciting edition. Friday Q&A is driven by your submissions, so if you have an idea for a topic that you would like to see covered here, please send it in!
Comments:
Thanks for the explanation, Mike--this has had me blocked on a side project for a while!
Is there a way to examine a binary to see what list of rpath locations it was compiled with? On Leopard, I can use otool -L to see link entries of the form "@rpath/Foo.framework/Versions/A/Foo", but I haven't found a way to discover what directories @rpath will expand to at runtime.
Since Brent has mentioned the Xcode way to use -rpath, I'll mention that if you invoke the linker through gcc (on the command line, or in a Makefile, etc) you have to use the "-Wl,option" gcc flag. So, "-rpath path" becomes "-Wl,-rpath,path".
Also: if you're compiling a library and want the linker to look for a dependency in the same directory as the library, you have to do "-rpath @loader_path/.", not just "-rpath @loader_path". That one took me a little while to figure out.
otool -l to list the binary's load commands, the rpaths will show up as LC_RPATH commands.Wanted to thank you for your series of Friday Q&A. It's hard to find deep OS and development topics covered in such detail for the Mac platform, and your blog is a precious resource.
Thanks
Ben
#!/bin/bash
for file in $@; do
echo "$file":
otool -l $file | grep -A 3 LC_RPATH | grep path
done
I'd like to point out though a problem with your problem examples as that problem is a bit misleading
embedded in an application or when installed to /Library/Frameworks, you have to provide two separate copies with two different install names.
If you want Foo.framework to work when embedded in an application or when installed to /Library/Frameworks, you have to provide two separate copies with two different install names. (Or manually tweak install names later on using install_name_tool.) This is doable, but annoying.
These statements would be 100% correct with "~/Library/Frameworks", however not with "/Library/Frameworks" if you set a framework to any install path whatever it may be @executable, @loader, or just a random directory, if the framework does not exist where the install name says it should, by default Mac OS X will check
/Library/Frameworks
/Network/Library/Frameworks
/System/Library/Frameworks
So if a framework is compiled with @executable_path or @loader_path it is still consumable by an app that wants to put in in any of the above 3 fallback directories (and luckily /Library/Frameworks is most likely and most common alternative).
Certainly rpath is more flexible and should be used when targeting only 10.5 or later, but for the common specific case of designing a framework to be used in a bundle or in /Library, frameworks targeting 10.4 using @loader_path aren't really as annoying as mentioned above.
DYLD_FALLBACK_FRAMEWORK_PATH
This is a colon separated list of directories that contain
frameworks. It is used as the default location for frameworks
not found in their install path.
By default, it is set to /Library/Frameworks:/Net-
work/Library/Frameworks:/System/Library/Frameworks
This is really confusing wording because DYLD_FALLBACK_FRAMEWORK_PATH is not actually set on my system at all (yet the behavior works as you describe). I assume what they mean to say is that, if DYLD_FALLBACK_FRAMEWORK_PATH is not set, then the default is those paths.
/Network/Library/Frameworks/ isn't in that list, I've never tried that location personally though.
http://developer.apple.com/mac/library/documentation/MacOSX/Conceptual/BPFrameworks/Tasks/InstallingFrameworks.html#//apple_ref/doc/uid/20002261-BBCCFBJA
This is important if a framework is expected to be loaded by another framework. For example, when building the Sparkle framework into an app which accessed Sparkle via another framework, the app crashed upon launching outside of Xcode, because the Sparkle framework was not found, until I changed the Sparkle framework's Installation Directory from
@loader_path/../Frameworks, to @loader_path/../../../../Frameworks. In this case, the old fashioned @executable_path/../Frameworks worked too (because the path starts at Contents/MacOS/MyApp). I decided that was a better choice.Now JXZip is built as a framework itself so it has to include “libzip Mac.framework”. It’s “Installation Directory” (INSTALL_PATH) is already set to @rpath. I do the usual “Build Phases” dance when building JXZip.framework:
- Add the “libzip Mac” target to “Target Dependencies”
- Add the build product to “Link Binary With Libraries” and “Copy Frameworks”
As Jerry mentions above, the @loader_path is relative to the binary doing the loading. In this case it’s “JXZip”. For frameworks, the contained “Frameworks” is a subfolder of the folder where the binary is located. So we have to set “Runpath Search Paths” (LD_RUNPATH_SEARCH_PATHS) in the “Build Settings” to "@loader_path/Frameworks".
Set the environmental variable DYLD_PRINT_LIBRARIES and launch the application. It will print out exactly what it's loading.
https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/LoggingDynamicLoaderEvents.html
e.g. install_name_tool -add_rpath '@executable_path' myapp
Best of both worlds.
Comments RSS feed for this page
Add your thoughts, post a comment:
Spam and off-topic posts will be deleted without notice. Culprits may be publicly humiliated at my sole discretion.