McObject Benchmarks Embedded Databases on Android Smartphone
On three of the four TestIndex metrics, Perst performed database tasks significantly faster -- more than 16 times faster, in the case of database inserts
March 9, 2009 — In what is likely the first-ever database benchmark conducted on an off-the-shelf mobile phone, McObject® has successfully deployed its TestIndex application on T-Mobile’s Android-based G1smartphone, to measure performance of the SQLite embedded database alongside McObject’s open source Perst™ embedded database as the two perform typical data management functions.
The comparison showed that gaps in database performance can be measured as easily in this pocket-sized test bed as in more traditional benchmark environments with rack-mounted hardware. On three of the four TestIndex metrics, Perst performed database tasks significantly faster—more than 16 times faster, in the case of database inserts.
McObject has gathered benchmark results and analysis into a report, available for free download from McObject’s Web site. McObject also offers the TestIndex application, with complete source code, to run in the Android emulator as well as on the T-Mobile G1 and any future phones based on Google’s Android platform.
Mobile Devices’ Database Challenge
To support sophisticated features, applications running on mobile devices manage increasing volumes of complex data. Examples include location-based services (LBS) that rely on GIS data, MP3 players that index songs and playlists, and field sales and service applications that store enterprise data.
Incorporating an off-the-shelf embedded database for these tasks reduces coding and QA, and improves reliability and performance. Given mobile phones’ typically limited processing resources, though, the chosen database must be highly efficient.
Two quite different embedded database systems that have emerged for use on Google’s Android are SQLite, a relational embedded database that supports the SQL application programming interface (API), and Perst, an object-oriented embedded database that works directly with Java objects.
To compare Android databases in an “applets to applets” test, McObject developed the TestIndex benchmark application, which measures performance along the following parameters:
* Insert – loading 10,000 simple records into the database (with string and integer keys, using two B-tree indexes)
* Search – seeking these records via the indexes (there are 10,000 searches for random values in each index, so searching is performed 20,000 times)
* Scan – performing two traverses (sequential scans) through all records using both indexes, so that records are sorted by integer key and string key
* Delete – locate and remove all 10,000 records
SQLite vs. Perst: Benchmark Results
For the first TestIndex benchmark test in a “real” Android phone, McObject obtained T-Mobile’s G1 phone, equipped with all standard features.
The TestIndex application with both embedded databases was downloaded via USB from a PC to the G1 phone, where it received its own “Benchmark” icon. Clicking this launched TestIndex, which returned the following results (in milliseconds):
Operation SQLite Perst Performance Disparity
Insert 299,166 18,214 Perst is 16.43 times faster
Search 104,796 15,743 Perst is 6.65 times faster
Scan 6,457 9,747 SQLite is 1.51 times faster
Delete 203,253 37,811 Perst is 5.38 times faster
Examining the Results
According to McObject CEO Steve Graves, the performance disparity in insert and delete operations likely results, at least partially, from the lack of explicit transaction support in the SQLite Android API. Each update must be performed as a separate transaction, in autocommit mode, resulting in significant transaction processing overhead.
Perst’s edge in searching for records probably stems from latency in SQLite caused by using Android’s Java interface to access the native C language database, and SQLite’s overhead of parsing, optimizing and executing the interpreted SQL, Graves said. In contrast, Perst is all Java, and its interface works directly with database objects – no interpretation of an intermediate language is needed.
SQLite’s advantage in scan operations probably stems from the test’s simple tabular data layout, Graves said. As a relational database, SQLite organizes data in rows, and these rows are physically close to one another in storage, like rows of a spreadsheet. This proximity lends an edge in sequentially fetching the rows. In contrast, in Perst, everything is an object, including index pages, and objects are interleaved in the storage.
More details, including database definitions used in the test and an expanded performance analysis, are included in McObject’s freely downloadable benchmark report.
Perst is part of McObject’s family of powerful small footprint, high performance embedded database software products. The eXtremeDB™ in-memory embedded database from McObject is used widely in devices including MP3 players, WiMAX base stations, digital TVs, telecom/network communications equipment and military/aerospace technology. Perst is available for Java and .NET, including Java ME and .NET Compact Framework. CA’s Wily Technology credits Perst with delivering a shortened development cycle and a ten-fold performance improvement within its real-time Java application.
Founded by embedded database and real-time systems experts, McObject offers proven data management technology that makes applications and devices smarter, more reliable and more cost-effective to develop and maintain. McObject counts among its customers industry leaders such as Chrysler, Maximizer Software, Siemens, Phillips, EADS, JVC, Tyco Thermal Controls, F5 Networks, DIRECTV, CA, Motorola and Boeing. McObject, based in Issaquah, WA, is committed to providing innovative technology and first-rate services to customers and partners. Visit www.mcobject.com.