Raj Advani

I’m experimenting with using Google’s new LevelDB database for an iOS application. LevelDB is a fast, persistent key-value store, with a dirt-simple API: you put things, get things, and iterate over things.

The closest thing to it is probably Tokyo Cabinet, but due to licensing restrictions it’s difficult to use Tokyo Cabinet in an iOS app. Implementation-wise, the two are very different, but performance-wise they’re comparable, with a few differences. It appears that TC has slightly faster read performance but less consistent write performance, particularly when writing to random locations. LevelDB’s performance is much more consistent throughout.

I’m going to be doing more “real-world” testing of LevelDB soon, but I thought I’d share the Makefile I used to get build this for iOS, along with some precompiled static libraries. Note that this Makefile is not the cleanest thing in the world, but it’s meant to create a library usable by both the Simulator and the device, so I lipo together both x386 and armv7 architectures (I didn’t bother with armv6, but it’s easy to add). 

Build Instructions (if you just want the precompiled libraries, skip to the bottom!)

1) First apply this helpful OSX patch developed by davids, to the version 1 source code of LevelDB. 

2) Modify /port/port_osx.h, changing AtomicWord to Atomic32 on line 67.

3) Replace the makefile with this one. Modify the SDK variable to point to the version of the iOS SDK you’re using.

4) Run these commands:

make clean 
make library            // makes the simulator .a file
make clean 
make IOS=1 library      // makes the iOS .a file
make lipo               // lipos the two together


5) You’ll end up with libleveldb.a. Add this to your XCode project, and add the headers in the /include directory to your project as well. Then you’re done! 

Precompiled libraries (iOS 4.2 & Simulator)

Finally, if you don’t want to go through the above patch and build process, here are the the static libraries themselves, compiled against iOS 4.2. You can either use one of the architecture-specific libraries (386 or armv7), or you can use the lipo library that works with both simulator and iOS. Either way, here are the header files you’ll need.

The legislative language was just released for the 2011 budget compromise, and the electronic government fund — responsible for a number of government transparency initiatives — is getting cut from $32 billion to $8 billion. Data.gov is part of this fund, and it’s a promising platform with a lot of data. I hope they manage to keep the lights on. Small companies don’t have the luxury to buy multi-million dollar datasets; accessible government data, like the NYC Data Mine, enable them to compete in the space with large corporations like Google and Microsoft. Fortunately, even if Data.gov shutters, the USGS and related agencies still provide a large number of national data sets.

“When a consumer gets the phone and they wanna play a game that uses our technology, it’s got to be a consistent experience, and we can’t guarantee that [on Android]. That’s what held us off of Android”

Tim Sweeney on what’s stalled Epic from developing on Android. The easy solution to this, I think, is to allow developers to whitelist specific devices on the Android Market. Currently that’s not possible; so if someone downloads your app on a device that can’t run it, you’ll get a one-star review. And those will pile up. 

The Android Market does let you specify what features your app needs, but that’s not enough. I need to be able to target a specific CPU, a specific GPU, and sometimes even specific OpenGL extensions, all of which I could do myself if they let me whitelist devices.

We’ve been working a lot recently with the Android NDK. The NDK allows you to write native code for an Android device, allowing you to bypass the Dalvik VM and write directly to the platform. Now, we already have a functioning version of our app written in Java, so why did we put in the effort to port it over? Speed, of course. For those facing a similar decision, here are three things we’ve learned:

1) The Dalvik VM is slow for CPU heavy applications. Slow is a relative term. For your average “tables and text” application, it’s more than fast enough. But if you want to write applications with heavy CPU usage, then you’re much better off with the NDK. Specifically, we found that the Dalvik VM really suffered when running highly recursive algorithms: the method call overhead was unacceptable. Now, you could eliminate a lot of this method overhead via public fields and manually inlined code — but if you’re going to go that far, you may as well write the thing in C++. Note that this applies even to the newest crop of devices (Nexus S) with the latest operating system (Gingerbread).

2) Garbage collection kills frame-rates. Again, not for tables and text applications. But if you’re writing a game or anything with a tight, optimized render-loop, you must avoid generating any garbage. Unfortunately the Android SDK spills metric tons of it: Android internal classes aren’t written with optimization in mind, they’re written for ease of use. The IO classes freely allocate and discard byte arrays and can’t be re-used, and the NIO classes — which are as optimal as the SDK gets — are almost as bad. So even if you write your app efficiently, merely using basic SDK classes will produce significant garbage, eventually triggering the Garbage Collector and pausing your app, producing stutter. Using the NDK avoids this problem.

3) NDK libraries are extremely limited. This is gradually improving, especially with NDKr5, but for now you’re fairly constrained in what you can do. You get the standard C libraries and support for STL. But if you want to do something high tech, like, say, send an HTTP request, you have to learn your BSD sockets and write 100 lines of code. Or you could download the Android source and compile libcurl, which is perhaps even more of a pain. That said, there is one other alternative: JNI. For advanced functionality, JNI enables you to uplink to Java, do things in the SDK, then return the results back to the NDK. We do this a lot, and it works quite well — except, of course, it generates a lot of garbage.