Android: Sqlite Version

Going back to that db corruption issue I wrote about a few weeks ago. Had a lot of questions on this one. Some of the posts I found talk about it being fixed in Version 3.5.4 (2007-12-14). But that might just be one cause of the file corruption.  3.* versions and Ice Cream Sandwich run on 3.7.4. 4.1/4.2 upgrade to 3.7.11. There are probably multiple causes of the corruption and a backup/restore strategy should be used if appropriate.

List is at:  http://stackoverflow.com/questions/2421189/version-of-sqlite-used-in-android

SQLite 3.7.4:

15-4.0.3-Ice Cream Sandwich
14-4.0-Ice Cream Sandwich
13-3.2-Honeycomb
12-3.1-Honeycomb
11-3.0-Honeycomb

Android: Database corruption issue

I recently ran into an issue where all the rows of all the tables in a user's database were deleted. For this app, there is some security code that would do this, but it would also delete temp files and backup data files. No exceptions were thrown and no app crashes were reported.Those files were still there which indicated that the service for wiping the db didn't run, but I wasn't able to duplicate it and there were no other instances of this happening.

The other day I was running some backend tests and after the test run, went to restore the previous version of the DB to rerun the test and got this error : android.database.sqlite.SQLiteDatabaseCorruptException



12-29 10:22:39.190: D/(14156): Here
12-29 10:22:39.200: I/SqliteDatabaseCpp(14156): sqlite returned: error code = 11, msg = database corruption at line 46978 of [8609a15dfa], db=/data/data/com.myapp.android/databases/db_file
12-29 10:22:39.200: I/SqliteDatabaseCpp(14156): sqlite returned: error code = 11, msg = database corruption at line 46978 of [8609a15dfa], db=/data/data/com.myapp.android/databases/db_file

/data/data/com.myapp.android/databases/db_file
12-29 10:22:39.200: E/DefaultDatabaseErrorHandler(14156): Corruption reported by sqlite on database: /data/data/com.myapp.android/databases/db_file
12-29 10:22:39.200: E/DefaultDatabaseErrorHandler(14156): deleting the database file: /data/data/com.myapp.android/databases/db_file

12-29 10:22:39.220: E/SQLiteDatabase(14156): android.database.sqlite.SQLiteDatabaseCorruptException: error code 11: database disk image is malformed
12-29 10:22:39.220: E/SQLiteDatabase(14156): at android.database.sqlite.SQLiteStatement.native_executeInsert(Native Method)
12-29 10:22:39.220: E/SQLiteDatabase(14156): at android.database.sqlite.SQLiteStatement.executeInsert(SQLiteStatement.java:112)
12-29 10:22:39.220: E/SQLiteDatabase(14156): at android.database.sqlite.SQLiteDatabase.insertWithOnConflict(SQLiteDatabase.java:1737)
12-29 10:22:39.220: E/SQLiteDatabase(14156): at android.database.sqlite.SQLiteDatabase.insert(SQLiteDatabase.java:1610)
12-29 10:22:39.220: E/SQLiteDatabase(14156): at com.myapp.android.database.DBHelper.createRecord(DBHelper.java:80)
12-29 10:22:39.220: E/SQLiteDatabase(14156): at com.myapp.android.contentproviders.SynchLogProvider.insert(SynchLogProvider.java:117)
12-29 10:22:39.220: E/SQLiteDatabase(14156): at android.content.ContentProvider$Transport.insert(ContentProvider.java:203)
12-29 10:22:39.220: E/SQLiteDatabase(14156): at android.content.ContentProviderNative.onTransact(ContentProviderNative.java:153)
12-29 10:22:39.220: E/SQLiteDatabase(14156): at android.os.Binder.execTransact(Binder.java:339)
12-29 10:22:39.220: E/SQLiteDatabase(14156): at dalvik.system.NativeStart.run(Native Method)
12-29 10:22:39.220: E/DatabaseUtils(14156): Writing exception to parcel

The database was completely wiped. Turns out that if a Sqlite database is corrupted, Android automatically deletes the database file and recreates it. This is a known Android bug/feature.

I didn't get this error until I tried to restore the db backup file and Android even thought that a desktop backup of the file was corrupted (although I was able to open it up in a third party tool on the desktop.) Likely the cause was the backup (simply a file copy) occurred while a service was updating the db.  We use the backup all the time and this was the first time that it had occurred. (The user issue was caused by something else corrupting the db file not the backup)

http://code.google.com/p/android/issues/detail?id=10127

http://stackoverflow.com/questions/2960015/android-database-disk-image-is-malformed

http://www.sqlite.org/lockingv3.html#how_to_corrupt

Solution appears to be to have a backup and restore routine so that data can be recovered if necessary.

Android Content Provider Limiting rows returned

To limit the number of rows returned by a content provider, try this code
Product obj = new Product();

        Cursor cursor = getContentResolver().query(
                ProviderHelper.determineURI(obj), null, null, null, " _ID DESC LIMIT 5000");

This will return the last 5000 rows in descending order.Limit needs to be last and don't forget the column name you want to descend.

Android Cursor code example

Processing cursors is a very common task that developers do when creating android apps. Here's some sample loops for processing cursors. Pay attention to log files to make sure that there are no null pointer exceptions when developing your cursor processing.

Cursor cur = dB.rawQuery("SELECT firstname, salary FROM demos " +
           "where salary>100,000 LIMIT 10", null);

if (cur != null ) {

    if  (cur.moveToFirst()) {

        do {
            String firstName = cur.getString(cur.getColumnIndex("firstname"));

            int salary = cur.getInt(cur.getColumnIndex("salary"));

            results.add("" + firstName + ",salary: " + salary);
        } while (cur.moveToNext());

    }
}
cur.close();

Organizing Resource Files

For very complex projects, I like to try to group the layout files together so that activity files, fragment 

files, adapter files and include files are all together. Developers working on an activity or fragment can immediately find the file that they need to change.  I do this by placing a key at the front of a file. Doesn't matter what the key is just as long as it makes sense. I typically make the activity files "views" and prepend a vi_ to their file names.Why? Eclipse will display the files alphabetically and since they files are not modified frequently, I want them at the bottom of the list and out of the way. Same goes for adapter template files.  

Here's what I use:

1, Container/Activity Files

Name container files the same as activity where possible. In this case I’ve named them vi_{ACTIVITYNAME}_container_portrait.xml and vi_{ACTIVITYNAME}_container.xml (for horizontal mode) I named them vi_* so that they end up towards the bottom of the directory since we won’t be touching them much and they’re more out of the way then.

2.       Fragments

In the example above, I named the fragment xmls, {activityname}.xml but we might want to call them fr_{ACTIVITYNAME}_top.xml, fr_{ACTIVITYNAME}_bottom.xml etc

3.       Include files for things like common headers I call inc_{FILEPURPOSE}.xml for example inc_topnav.xml

4.       Adapter list item files (I don’t see any in the screen shot) I would call li_{purpose}.xml  Purpose is generally related to the adapter name. For example, an adapter that shows a list of products would be called ProductsAdapter.java  with the list item being li_productsadapter.xml

Android Market Share now a dominating 68%

IDC reports that Android market share of the smart phone market has increased to 68%. Nearly 105 million Android phones shipped in last quarter, twice as many as a year ago. Apple also grew their shipments to 26 million.  Take a stab at Apple's market share.Where do you think it is? Way down at 17% of the global market.  Open platform is certainly the way to go.  Apple should have learned that way back in 1980s when it lost it's enormous advantage in personal computers to IBM compatible.

Android: Business models Some Analysis of Downloads

A potential client contacted me the other day to talk about an idea that they have. It's a great idea and wanted to bounce around some business model ideas.  The question that they had was the following:

So, we are trying to figure out for Super App Idea, the kind of download traffic we might get if we were free vs $1 vs $2?  Do you have a feel for that?  I am sure it might be a little different on iPhone as opposed to Android as well.  Do you have any sense of conversion rates  as well?  

I did a quick look for a few apps that had a free version and a paid version. I decided to use a pill tracking system since that was an app I downloaded to test the other day.  The first app was a free pill box app (https://play.google.com/store/apps/details?id=com.mobilepills.pillbox&hl=en) that received between 10,000 and 50,000 downloads.  The first comment was put in on September 2010 so it's been around about 2 years. We'll assume that the first comment was put in right around launch date. So conservatively it is generating about 5000 downloads a year.  Note these downloads don't include downloads from third party distributors like Amazon. At a high end, they've received 25,000 downloads per year

The second one I looked at was a pay to use app that has similar functionality.   This app (https://play.google.com/store/apps/details?id=com.sartuga.android.pillboxalert) received between 1000 and 5000 downloads since their first comment on November 8, 2009. They are selling their app at $1.99. So taking the conservative 1000 downloads, that means that they've received about 300 downloads a year or revenue of about $600 per year. At a high end, they've received 1600 downloads per year or revenue of about $3200 per year.

Based on this example, free apps will get about ten times the downloads as paid apps.

I'm going to create a few more blog entries in the near future taking a look at a couple of ways to potentially increase the downloads if you have a paid application.

Determining where your Android APK is installed

A little bit of code can tell you where your apk files are being installed. Typically it will be /data/app/package.name.apk



07-25 09:15:59.065: D/Main(3000): Source directory at: /data/app/com.appulearn.supersecretapp.android-1.apk

This bit of code will tell you....


  PackageManager  pmMgr = getPackageManager();
        List pkgList = pmMgr.getInstalledPackages(PackageManager.GET_ACTIVITIES);
        List appinfo_list = pmMgr.getInstalledApplications(0);
        for (int x=0; x < pkgList.size(); x++){    
          Log.d(TAG, "Source directory at: " +appinfo_list.get(x).sourceDir);
        }

Designing Android Databases

Most of my app use a sqlite database and I use data scripts to create, alter and bootstrap tables. ADB comes with the ability to look at the database on a USB connected device, but I often test the scripts in an opensource tool called SqliteBrowser.  Unfortunately it's a little bit unstable, and seems like development on it has stopped but it's great for testing sql statements that you want to run within an Android app/

http://sqlitebrowser.sourceforge.net/

Android: Disabling or Blocking the back button

One of the tablet apps I've been working on is a massive data collection app. It is set up to run in vertical mode and it's easy for the user to accidentally push the back button and exit a data collection form potentially losing their work.  Luckily it is also easy to disable the back button by preventing the click action or putting a dialog up when it is touched to confirm the exit of the screen.

Here you go. Nothing to it.

@Override
public void onBackPressed() {
}

Android logcat disappears with multiple devices attached

I was recently working on an Android project testing on multiple devices attached via USB and rotating the testing between them. After a while, all trace stopped being displayed and wouldn't come back. I disconnected devices, reset the debugging perspective, restarted Eclipse, restarted the laptop but nothing worked.  I typically have more than one Android device plugged in and in the past two years have never had any problems.  The solution was to go into DBMS perspective and select the specific device in the upper left hand panel. For whatever reason, it wasn't automatically switching based on what was running.

Data type, Android and Sqllite

When working with Android content providers and cursors, pay close attention to the datatypes you are using to pull the data from the database. If you set the data type of the sqlite field to NUMERIC and are using it to store LONG values, be sure to pull it from the database as a long and not a int. Pulling it as an Integer will lead to truncation of the long value. So for example,
Since CreateDate is a long, I need to pull it like this
data.setCreateDate(c.getLong(c.getColumnIndexOrThrow(DatabaseConstants.F_CREATEDATE)));  

and not like this

data.setCreateDate(c.getInt(c.getColumnIndexOrThrow(DatabaseConstants.F_CREATEDATE)));  

Here's the complete cursor processing.

  while (!c.isAfterLast()) { 

      MyData data = new MyData(); 

      data.setFirstName(c.getString(c.getColumnIndexOrThrow(DatabaseConstants.F_FIRSTNAME))); 

      data.setLastName(c.getString(c.getColumnIndexOrThrow(DatabaseConstants.F_LASTNAME))); 

      data.set_id(c.getInt(c.getColumnIndexOrThrow(DatabaseConstants.F__ID))); 

      data.setCreateDate(c.getLong(c.getColumnIndexOrThrow(DatabaseConstants.F_CREATEDATE))); 

      data.setIsdeleted(c.getInt(c.getColumnIndexOrThrow(DatabaseConstants.F_ISDELETED))); 

      data.setLastUpdate(c.getLong(c.getColumnIndexOrThrow(DatabaseConstants.F_LASTUPDATE))); 

      list.add(data); 

      c.moveToNext(); 

 }  

More Android code standards

When mapping edit text fields, select boxes and other elements that have GUI interaction with the user  within Android Activity Java code, the first thing you need to know is the resource id where the data will be coming from. Flipping back and forth between the xml resources and the Activities is a pain. Be sure to put a comment in your code next to the variable that indicates the xml resource id like below. This easy piece of documentation and code commenting is invaluable to the developers who will be maintaining your code


 private EditTextView mPersonId;   //   R.id.PersonIdETV
 private Spinner mPersonJob;     //   R.id.PersonJobSPN

Randomly generating a date before or after today

I was recently building a calendar type app for a client and needed to test out the placement of items on the calendar. Unfortunately their test data returned all items with the exact same date. So I wrote a little bit of code that would randomly replace the date in the data with AM PM (the calendar data didn't have a time just whether or not it was in the morning or afternoon and randomly assign a date within a week both before and after today.

 private String generateRandomAmOrPm() {  
         final Random myRandom = new Random();  
         boolean amOrPm=        myRandom.nextBoolean();  
         if (amOrPm) {  
              return "AM";  
         } else {  
              return "PM";  
         }  
      }  
      private int generateRandomDate() {  
            final Random myRandom = new Random();  
              boolean plusOrMinus=        myRandom.nextBoolean();  
              int dateToMove = myRandom.nextInt(6); // plus or minus 6 days  
              if (plusOrMinus) {  
                   return 0-dateToMove;  
              } else {  
                   return dateToMove;  
              }  
      }  

Android HTTP Clients

Making a backend http call to retrieve or send data is a very common Android task.  There are a couple of different ways to do it from Apache HTTP Client to HttpURLConnection. The blog article below summarizes the advantages and disadvantages of both and is a must read.

http://android-developers.blogspot.com/2011/09/androids-http-clients.html

Launching settings from a button

I was recently working on an Android tablet project that needs to detect connectivity to a network. If the user wasn't connected to the Internet, I would throw up a dialog box telling them to connect. I decided to take this one step further and take them to the setting screen that would allow them to connect to wifi.

So to do this, I needed to send an intent that would launch settings. Easy enough.The Android snippet below does that.


Intent intent = new Intent(Intent.ACTION_MAIN);
 intent.setClassName("com.android.settings", "com.android.settings.Settings");
startActivity(intent);


Now that's ok, but it takes the user to settings main screen. Which is nice but a better way to do that is to send them directly to the wireless settings screen.  This can be done using the code below.


Intent intent =new Intent(Settings.ACTION_WIRELESS_SETTINGS);
startActivity(intent);

Naming Conventions for Views

I recently worked a project as a lead developer working with several other developers on a very complex tablet app that was collecting medical history information. So lots of complex forms, widgets, and layouts.  One of the things that they worked on is setting up the xml layout files which is the first step in the Android development process. The screens looked great but we didn't have consistency on the naming of the individual layouts. For most simple apps, this isn't a big deal. But some of these forms had a dozen checkboxes, plus a dozen radiogroups plus a dozen spinners plus the text views, linear layouts and edit texts.

When I went to do the mapping to our json, I couldn't look at an R.id.field and know exactly what it it was for. For example, we had R.id.rash_field1 but that didn't tell me that it was an edit box that goes with SKIN-->Rash-->Color-->OtherValue

I would have name the resource R.id.skin_rash_color_othervalueET

This tells me a lot about what is going to be entered into the field and what the field purpose is as well as where in the data model I'm likely to be binding it to.

I try to use the following abbreviations for naming fields. This isn't an exhaustive list but will get you thinking on how you want to name the fields.

EditText: R.id.fieldnameET
TextView: R.id.fieldnameTV
Spinner: R.id.fieldnameSpn
Linearlayout: R.id.fieldnameLL
Checkbox: R.id.fieldnameCB
RadioGroup: R.id.fieldnameRG
RadioButton: R.id.fieldnameRB

Automated Testing on Android

For my most recent project, a very large tablet app, I've been experimenting with the Android Testing Framework.  My piece of the app is the backend integration consisting of a number of different API calls that send and receive JSON, handle login and more. The app will utilize content providers to enable other apps to access the database.  I'm specifically using the test framework to exercise these content providers as well as the services that integrate with the backend.

While there is some overhead with learning the test framework. it's very useful. There's a few gotchas that slow down setting up your tests. A good one is why doesn't my test run?  Test methods need to start with the word test. So testMyInsertCode() will run while doMyInsertCodeTest() won't.  Content provider testing has some specific requirements to set up the mock statements as well.  Be sure when testing queries that you don't forget to put the =? on the end of the where clauses....

Android Service not found error

I was recently helping a developer debug some services when he ran into an error saying that the service was not found. You get this error if your manifest doesn't have the service registered. Be sure to check for typos in the service tag.

06-19 13:26:32.543: W/ActivityManager(306): Unable to start service Intent { act=com.yourapp.special.android.services.BigService }: not found

< service

            android:name=" com.yourapp.special.android.services.BigService "

            android:exported="true" >

           

               

           

       

Content Provider and Count queries

I'm creating a number of custom content providers for a client and on one of the screens we want to show the count of some of the tables.
  Now in sql world that would be easy, simply run a query that is

  select count(_id) from cartable

  But in content provider world, it's a bit harder. After stumbling around a bit playing with provider.call methods,
  Now just like you can tell the content provider to give you an instance of a car by id like this:

  content://find.me.a.car/car/123


  You can define a new URI format and use that to have the content provider's query method do something other than return a cursor that contains a
  list or a single row.  You can have it return a cursor that is a count.

  Here's our test method. Since it's a test, we're inserting a record first then getting the count of the table.  The URI would look something like

  content://find.me.a.car/car/COUNT


 

  public void testCountCall(){ 

         Log.d(TAG, "--------------testInsert()-------------"); 

     ContentProvider provider = getProvider(); 

     Log.d(TAG, "Running insert of new Car record"); 

     Car obj = createObject(); 

         mUriRInsert=provider.insert(ProviderHelper.determineURI(obj), obj.createContentValues()); 

        Log.d(TAG, "Content inserted at: " + mUriRInsert); 

        Log.d(TAG, "------- testinsert will complete successfully if the returned uri is not null"); 

     Uri uriCount = Uri.withAppendedPath(ProviderConstants.CAR_CONTENT_URI, "COUNT"); 

     Cursor c = provider.query( uriCount, null, null, null, null); 

     c.moveToFirst(); 

     int count = c.getInt(0); 

      Log.d(TAG, "new Count:" + count); 

      assertEquals(1, count);      

   }  

 
    --- NEXT STEP: Define a pattern for the COUNT in your provider
 
    The first two are standard, the third pattern will match the count.
 
 
 

  static 

        { 

             sUriMatcher = new UriMatcher(UriMatcher.NO_MATCH); 

             sUriMatcher.addURI(ProviderConstants.CAR_AUTHORITY, ProviderConstants.CAR_PATH, 

                 ProviderConstants.CP_TYPE_LIST); 

             sUriMatcher.addURI(ProviderConstants.CAR_AUTHORITY, ProviderConstants.CAR_PATH+"/#", 

                 ProviderConstants.CP_TYPE_ITEM); 

             sUriMatcher.addURI(ProviderConstants.CAR_AUTHORITY, ProviderConstants.CAR_PATH+"/COUNT", 

                 ProviderConstants.CP_TYPE_COUNT); 

      }  

----------- Third Step: Modify the query() method of the provider so it knows what to do with a URI that matches the COUNT

 @Override 

      public Cursor query(Uri uri, String[] projection, String selection, 

          String[] selectionArgs, String sortOrder) 

      { 

           Log.d(TAG, "uri:" +uri); 

           Log.d(TAG, "where:" +selection); 

           SQLiteQueryBuilder builder = new SQLiteQueryBuilder(); 

           switch (sUriMatcher.match(uri)) 

           { 

           case ProviderConstants.CP_TYPE_LIST: 

                builder.setTables(TABLENAME); 

                builder.setProjectionMap(sDataProjectionMap); 

                break; 

           case ProviderConstants.CP_TYPE_ITEM: 

                builder.setTables(TABLENAME); 

                builder.setProjectionMap(sDataProjectionMap); 

                builder.appendWhere(DatabaseConstants.F__ID + " = " 

                    + uri.getPathSegments().get(1)); 

                break; 

           case ProviderConstants.CP_TYPE_COUNT: 

                builder.setTables(TABLENAME); 

                HashMap<String, String> countMap = new HashMap<String, String>(); 

                countMap.put("count", "count(*)"); 

                builder.setProjectionMap(countMap); 

                break; 

           default: 

                throw new IllegalArgumentException("Unknown URI: " + uri); 

           } 

           Cursor queryCursor = mDBHelper.runQueryBuilder(builder, projection, 

               selection, selectionArgs, null, null, null); 

           queryCursor.setNotificationUri(getContext().getContentResolver(), uri); 

           return queryCursor; 

      }  

Pretty straightforward and very handy.