Developing OpenCV computer vision apps for the Android platform

Eric Gregori, BDTI

January 31, 2013

Eric Gregori, BDTI

Continued from page 1

The OpenCV Manager
With the release of OpenCV version 2.4.2, NVIDIA introduced the OpenCV Manager, an application that can be downloaded and installed on an Android device from the Google Play Store or directly installed using the Tegra Android Development Pack (Figure 5). Once installed, the OpenCV Manager manages the OpenCV libraries on the device, automatically updating them to the latest versions and selecting the one that is optimized for the device.



The best practice for Android OpenCV applications is to use dynamically linked versions of the OpenCV libraries. Said another way, the OpenCV libraries should not be a part of the application (i.e., statically linked); instead, they should be dynamically linked (i.e., runtime linked) when the application is executed. The biggest advantage of dynamically linking the application to an OpenCV library involves updates. If an application is statically linked to the OpenCV library, the library and application must be updated together. Therefore, every time a new version of OpenCV is released, if the application is dependent on changes in the new version (bug fixes, for example), the application must also be upgraded and re-released. With dynamic linking, on the other hand, the application is only released once. Subsequent OpenCV updates do not require application upgrades.

An additional advantage of dynamic linking in conjunction with the OpenCV Manager involves the latter's automatic hardware detection feature. The OpenCV Manager automatically detects the platform it is installed on and selects the optimum OpenCV library for that hardware. Prior to NVIDIA's release of the OpenCV Manager, no mechanism existed for selecting the optimum library for a particular hardware platform. Instead, the application developer had to release multiple versions of his application for various hardware types. [15]

OpenCV for Android Tutorials
The OpenCV4Android project has developed a series of tutorials that walk the reader through the process of creating an OpenCV4Android host build machine and developing an OpenCV4Android application. The root node of the documentation tree can be found in the Android section of the OpenCV website.

The first tutorial, "Introduction into Android Development," covers two methods of creating an Android host build machine. The automatic method, using NVIDIA’s TADP, is described later. The manual method requires that you install the following software:

• Sun/Oracle JDK 6
• Android SDK
• Android SDK components
• Eclipse IDE
• ADT plugin for Eclipse
• Android NDK, and
• CDT plugin for Eclipse

The next tutorial, "OpenCV for Android SDK," covers the OpenCV4Android SDK package, which enables development of Android applications using the OpenCV library.

The SDK structure is illustrated here: [16]

OpenCV-2.4.3-android-sdk
_ apk
_ OpenCV_2.4.3_binary_pack_XXX.apk
_ OpenCV_2.4.3_Manager.apk

_ doc
_ samples
_ sdk
_ etc
_ java
_ native
_ 3rdparty
_ jni
_ libs
_ armeabi
_ armeabi-v7a
_ x86

_ license.txt
_ README.android

• The sdk folder contains the OpenCV API and libraries for Android.
• The sdk/java folder contains an Android library Eclipse project, providing an OpenCV Java API that can be imported into developer’s workspace.
• The sdk/native folder contains OpenCV C++ headers (for JNI code) and native Android libraries (*.so and *.a) for ARM-v5, ARM-v7a and x86 architectures.
• The sdk/etc folder contains the Haar and LBP cascades distributed with OpenCV.

The apk folder contains Android packages that should be installed on the target Android device to enable OpenCV library access via OpenCV Manager API. On production devices that have access to the Internet and the Google Play Market, these packages will be installed from the Market on the first application launch, via the OpenCV Manager API. Development kits without Internet and Market connections require these packages to be manually installed. Specifically, you must install the Manager.apk and corresponding binary_pack.apk, dependent on the device CPU (the Manager GUI provides this info). However, installation from the Internet is the preferable approach, since the OpenCV team may publish updated versions of various packages via the Google Play Market.

The samples folder contains sample applications projects and their prebuilt packages (the APK). Import them into Eclipse workspace and browse the code to learn ways of using OpenCV on Android.

The doc folder contains OpenCV documentation in PDF format. This documentation is also available online at the preceding link. The most recent (i.e. nightly build) documentation is at this location. Although it's generally more up-to-date, it can refer to not-yet-released functionality.

Beginning with version 2.4.3, the OpenCV4Android SDK uses the OpenCV Manager API for library initialization.

Finally, the "Android Development with OpenCV" tutorial walks the reader through how to create his or her first OpenCV4Android application. This tutorial covers both Java and native development, using the Eclipse-based tools. This tutorial also provides a framework for binding to the OpenCV Manager, to take advantage of the dynamic OpenCV libraries. The example code snippet that follows is reproduced from the OpenCV website. [17]

public class MyActivity extends Activity implements HelperCallbackInterface
{
private BaseLoaderCallback mOpenCVCallBack = new BaseLoaderCallback(this) {
@Override
public void onManagerConnected(int status) {
switch (status) {
case LoaderCallbackInterface.SUCCESS:
{
Log.i(TAG, "OpenCV loaded successfully");
// Create and set View
mView = new puzzle15View(mAppContext);
setContentView(mView);
} break;
default:
{
super.onManagerConnected(status);
} break;
}
}
};

/** Call on every application resume **/
@Override
protected void onResume()
{
Log.i(TAG, "called onResume");
super.onResume();

Log.i(TAG, "Trying to load OpenCV library");
if (!OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_2, this, mOpenCVCallBack))
{
Log.e(TAG, "Cannot connect to OpenCV Manager");
}
}

NVIDIA’s TADP (Tegra Android Development Pack)
NVIDIA has been a significant contributor to the OpenCV library since 2010. NVIDIA has continued its support of OpenCV and Android more generally with the TADP (see "Designing visionary mobile apps using Tegra AndroidDevelopment Pack”). The development pack was originally intended only for general Android development. However, with release 2.0, OpenCV was added as part of the TADP download direct from NVIDIA.

Per NVIDIA's documentation, the Tegra Android Development Pack 2.0  installs all software tools required to develop for Android on NVIDIA’s Tegra platform. This suite of developer tools is targeted at Tegra devices, but will configure a development environment that will work with almost any Android device. TADP 2.0 is available on Windows, Mac OS X, Ubuntu Linux 32-bit and Ubuntu Linux 64-bit (Figure 6).


Tegra Android Development Pack 2.0 features include:

• Android Development
• Android SDK r18
• Android APIs
• Google USB Driver
• Android NDK r8
• JDK 6u24
• Cygwin 1.7
• Eclipse 3.7.1
• CDT 8.0.0
• ADT 18.0.0
• Apache Ant 1.8.2
• Python 2.7

Tegra libraries and tools include

• Nsight Tegra 1.0, Visual Studio Edition (Windows only)
• NVIDIA Debug Manager for Eclipse 12.0.1
• PerfHUD ES 1.9.7
• Tegra Profiler 1.0
• Perf for Tegra
• OpenCV for Tegra 2.4.2
• Tegra samples, documentation and OS images
• Tegra SDK samples (all of which can also be imported into an Eclipse workspace, see Figure 7)
• Tegra SDK documentation
• Tegra Android OS images for Cardhu, Ventana and Enterprise development kits