As I have mentioned previously, I’m working to port OpenBSD’s
net80211 layer, which is an
IEEE 802.11-2007 WiFi stack, to Mac OS X.
Haiku, an open-source operating system, managed to do this last year for their own kernel. So why is it taking us so long to do a similar thing for OS X?
To start off, although it appears that the OS X kernel being partly based on BSD should make things easier, it actually makes it more complicated to attempt a port such as this. Since almost half of the OS X kernel consists of FreeBSD code, there are lots of symbols predefined, like
mbuf. Apple’s own “Kernel Programming Interface” (
KPIs) provide access to these data structures and functions through a slightly different API which is designed to prevent direct access. Unfortunately, most of the code from other BSD kernels heavily relies on direct access (most notable being the
mbuf handling routines). To top it off, several of these data structures have been slightly modified in the XNU kernel compared to their BSD counterparts, making their usage very risky.
Hence, simply including the source code for
net80211 and some compatibility glue code (like Haiku does) would result in massive naming collisions between predefined structures and functions in the kernel vs. those redefined in the compatibility layer.
Apple already includes a port of the
net80211 stack for their drivers for the Atheros wireless adapters. Without access to that source code, it is difficult to figure out (at least for me) what approach they have taken. My approach has been to convert the entire
net80211 source code to C++ and wrap them in a namespace. This will automatically cause all symbols to be prefixed with the namespace, hopefully avoiding collisions with their kernel counterparts.
This sounds easy enough, but for some things it just doesn’t work. The
mbuf handling routines in
net80211 make heavy use of direct access to the
mbuf structure. This is not possible in OS X (or not easily possible). OS X drivers use the
mbuf_t structure and the
mbuf KPI to manipulate mbufs. Now, mbufs are used to store all network data coming in and going out of the system. Thus, it is used extremely frequently. The two ways to deal with this were to either:
mbuf API and structures in our compatibility layer. Bad option in my opinion because of the naming collisions, and more importantly, from a performance point of view, and being badly integrated with OS X’s networking tools.
Wade through the hundreds of thousands of instances where
mbufs have been used in
net80211 source code, and convert them to their
mbuf_t KPI equivalents.
I chose the 2nd option as that was the only one which made sense, despite being extremely lengthy. I was actually recommended this by Apple kernel engineers on the
darwin-drivers mailing list.
So, the end result was to use Xcode’s “Find” feature to find every single
mbuf instance, and replace it with their
mbuf_t equivalent, after understanding its context. This was frustrating. But thankfully it’s done. Here’s an example snippet:
And here’s the list of files which needed to be manually reviewed line by line (compiler errors are not always useful) to make sure every instance was properly converted:
The other issue was that a lot of functionality from OpenBSD (e.g. kernel cryptography) is not present in OS X (it probably is present, but inaccessible from the
KPIs). So I included all of those. There were also several undefined symbols which I defined as equivalents. All in all, there is a compatibility layer for the compatibility layer:
For what it’s worth, there probably is a much better and more elegant way to do this. But sadly I am not knowledgeable enough to figure out how Apple/Atheros developers did it. How I wish I could at least get a glimpse of their source code! Ralink probably also uses the
net80211 stack (or something similar) in the OS X drivers for their USB wifi adapter range. A few days ago I tried to contact Ralink to get access to their source code (in return for making their drivers Airport compatible), but have not received any reply yet.
Anyway, with most of this mechanical porting done, what remains is to write code to glue the
net80211 information/function calls to IOKit’s “Airport” interface, which is referred to as
Kernel.framework from Leopard and older. Incidentally, Apple stopped supplying this particular set of header files from Snow Leopard onwards. Recreating the proper headers for Snow Leopard was a challenge which deserves a post of its own. Maybe later.
So that’s a survey of the process and the difficulties so far. This week I’ll be working to finish off the “compatibility layer” for
net80211 based drivers to access the hardware, and finally hooking it all up to parts of VoodooWireless so it can talk to Airport. Hopefully this approach will work. As is evident, I have not tested even a single line of all this code yet - simply because there are millions of nuts and bolts that all must fit in together before the whole thing will fly. This month I’m hoping it’ll all start to take shape. If not, well tough luck to us.
Postscript: I mainly wrote this post so that future developers who might want to pick up on this project after I leave, can get an idea of what work is being done and how it’s intended to work. Feel free to write to me at firstname.lastname@example.org if you have more technical questions (don’t write to ask “how much longer” or “it doesn’t work”, though).
Thanks for reading!