App Lifetime Measurement on iOS & Android

Posted on

I was working on a mobile app analytics system recent months, how well the system works relies on the app usage data captured on the users' devices.

This post is about technically how I did it in both iOS and Android operating systems.

Making Thread

When a user opens an app, an open session event needs to be captured, when a user tap the home button, app transitions to the background, the session needs to be marked as closed, when a user view an important page, or click a fancy button, counters need to be increased. Those needs must be handled in background threads since I don't want my app UI freezing by some time-consuming "needs".

In iOS, Grand Central Dispatch (GCD) is amazing, create a serial dispatch queue, and a background thread is alive.

1
2
dispatch_queue_t dispatchQueue =
       dispatch_queue_create("com.company.app", DISPATCH_QUEUE_SERIAL);

Every time a user activity need to be tracked, a task is added to this queue as follows, and all tasks will be executed sequentially.

1
2
3
4
5
6
7
- (void)open
{
    dispatch_async(self.dispatchQueue, ^{
        // task to mark session as open
        [self openSession];
    });
}

Android is not cool, I chose to use Handler.

A Handler allows you to send and process Message and Runnable objects associated with a thread's MessageQueue.

In the first place, design a handler class SessionHandler inherits Handler.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
public class SessionHandler extends Handler {
    public static final int MESSAGE_OPEN = 0;
    public static final int MESSAGE_CLOSE = 1;

    public SessionHandler(final Context context, Looper looper) {
        super(looper);
    }

    public void handleMessage(final Message msg) {
        super.handleMessage(msg);
        switch (msg.what) {
            case MESSAGE_OPEN:
                this.open();
                break;
            case MESSAGE_CLOSE:
                this.close();
                break;
            default:
                Log.i(SessionHandler.class.getName(), "Can't handle this message");
        }
    }

Then, create a HandlerThread.

1
2
3
4
5
6
7
8
9
private static final HandlerThread sessionHandlerThread =
        makeHandlerThread(SessionHandler.class.getSimpleName());

static HandlerThread makeHandlerThread(final String name) {
    final HandlerThread thread = new HandlerThread(name,
            android.os.Process.THREAD_PRIORITY_BACKGROUND);
    thread.start();
    return thread;
}

Third, create a Handler instance using that handler thread.

1
2
Handler sessionHandler =
    new SessionHandler(context, sessionHandlerThread.getLooper());

Finally, every time we need to track something, just send a message to the Handler instance.

1
2
sessionHandler.handleMessage(
    sessionHandler.obtainMessage(SessionHandler.MESSAGE_CLOSE));

Binding System Event

If these tracking methods are provided as an SDK, app developers may not want to add those tracking methods one by one in their code base. So, the interfaces of this kind of SDK should be minimum. There are things can be done automatically.

As in iOS, we could bind method calls to system events, for example, when app transitions to background or foreground.

1
2
3
4
5
6
7
8
9
10
11
12
13
NSNotificationCenter *center = [NSNotificationCenter defaultCenter];
[center addObserverForName:UIApplicationDidEnterBackgroundNotification
                    object:nil
                     queue:[NSOperationQueue mainQueue]
                usingBlock:^(NSNotification *notification) {
                    [self close];
                }];
[center addObserverForName:UIApplicationWillEnterForegroundNotification
                    object:nil
                     queue:[NSOperationQueue mainQueue]
                usingBlock:^(NSNotification *notification) {
                    [self resume];
                }];

I added below code to main method in SDK, app developers only need to call this main method in their code base, believe me, they will love your SDK.

In Android, sadly, I couldn't find anything similar. It does provide system event bindings in Application, but only onCreate and onTerminate, useless for my needs. So I gave up, and looked for alternatives in Activity, maybe Application.ActivityLifecycleCallbacks could be helpful, but API level (14) is too high for me.

Sending Data

Process

The data captured are on the users' devices, they should be transferred to server for analysis.

Although the data is anonymous, it's about users' behavior, encryption is needed to keep it safe during transfer.

The target devices often connect to cellular network, it's much expensive than WiFi, compression is needed to keep the data size as small as possible.

For encryption, I implemented two ways, AES and RSA, RSA is more secure, but AES is easy to implement and use. Below are AES implementation in iOS and Android.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
+ (NSData *)encryptAES128Data:(NSData *)data withKey:(NSData *)key
{
    Byte *keyByte = (Byte *)[key bytes];
    if( [key length] != kCCKeySizeAES128){
        return nil;
    }

    Byte *valueByte = (Byte *)[data bytes];
    int valueLength = [data length];
    int bufferSize = valueLength + kCCBlockSizeAES128;
    void *buffer = malloc(bufferSize);
    size_t numBytesEncrypted = 0;
    CCCryptorStatus cryptStatus = CCCrypt(kCCEncrypt, kCCAlgorithmAES128,
                                          kCCOptionPKCS7Padding | kCCOptionECBMode,
                                          keyByte, kCCKeySizeAES128,
                                          NULL,
                                          valueByte, valueLength,
                                          buffer, bufferSize,
                                          &numBytesEncrypted);
    if (cryptStatus == kCCSuccess) {
        return [NSData dataWithBytesNoCopy:buffer length:numBytesEncrypted];
    }
    free(buffer);
    return nil;
}

The Java way is more elegant.

1
2
3
4
5
6
7
8
9
10
11
12
static final String AES128_ECB_KEY = "ampassword";
static byte[] encryptData(byte[] data) {
    try {
        SecretKeySpec secretKey =
                new SecretKeySpec(AES128_ECB_KEY.getBytes(), "AES");
        Cipher cipher = Cipher.getInstance("AES/ECB/PKCS7Padding");
        cipher.init(Cipher.ENCRYPT_MODE, secretKey);
        return cipher.doFinal(data);
    } catch (Exception e) {
        return null;
    }
}

For compression, gzip is enough.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
- (NSData *)gzipDeflateData:(NSData *)data
{
	if ([data length] == 0) return data;
	z_stream strm;
	strm.zalloc = Z_NULL;
	strm.zfree = Z_NULL;
	strm.opaque = Z_NULL;
	strm.total_out = 0;
	strm.next_in=(Bytef *)[data bytes];
	strm.avail_in = [data length];

	if (deflateInit2(&strm, Z_DEFAULT_COMPRESSION, Z_DEFLATED, (15+16), 8, Z_DEFAULT_STRATEGY) != Z_OK) return nil;
	NSMutableData *compressed = [NSMutableData dataWithLength:16384];  // 16K chunks for expansion
	do {
		if (strm.total_out >= [compressed length])
			[compressed increaseLengthBy: 16384];
		strm.next_out = [compressed mutableBytes] + strm.total_out;
		strm.avail_out = [compressed length] - strm.total_out;
		deflate(&strm, Z_FINISH);
	} while (strm.avail_out == 0);
	deflateEnd(&strm);
	[compressed setLength: strm.total_out];
	return [NSData dataWithData:compressed];
}

Still, I like the Java way.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
static byte[] gzipDeflatedData(String jsonStr) {
    GZIPOutputStream gzipOutputStream = null;
    try {
        byte[] data = jsonStr.getBytes("UTF-8");
        final ByteArrayOutputStream byteArrayOutputStream =
                new ByteArrayOutputStream(data.length);
        gzipOutputStream = new GZIPOutputStream(byteArrayOutputStream);
        gzipOutputStream.write(data);
        gzipOutputStream.finish();
        gzipOutputStream.flush();
        return byteArrayOutputStream.toByteArray();
    } catch (final UnsupportedEncodingException e) {
        return null;
    } catch (final IOException e) {
        return null;
    } finally {
        if (null != gzipOutputStream) {
            try {
                gzipOutputStream.close();
            } catch (final Exception e) {
            }
        }
    }
}

Transfer

In iOS, I wrapped sendSynchronousRequest of NSURLConnection to a dispatch queue.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
NSMutableURLRequest *request =
    [NSMutableURLRequest requestWithURL:serverUrl
                            cachePolicy:NSURLRequestReloadIgnoringCacheData timeoutInterval:60.0];
[request setHTTPMethod:@"POST"];
[request setValue:@"application/x-gzip" forHTTPHeaderField:@"Content-Type"];
[request setValue:@"gzip" forHTTPHeaderField:@"Content-Encoding"];
[request setValue:[NSString stringWithFormat:@"%d", [jsonData length]] forHTTPHeaderField:@"Content-Length"];
[request setHTTPBody:dataEncrypted];

dispatch_group_async(self.dispatchGroup, self.anotherDispatchQueue, ^{
    NSURLResponse *response = nil;
    NSError *responseError = nil;
    NSData *responseData = [NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&responseError];
    NSInteger responseStatusCode = [(NSHTTPURLResponse *)response statusCode];
    if (responseError) {
        // handle failure
    } else {
        if (responseStatusCode == 200) {
            // do things
        } else {
            // handler failure
        }
    }
});

Why I use dispatch group is because I don't want the uploading task block the tracking tasks, so the task is dispatched to another queue, in many circumstances, we have to wait all the tasks in tracking thread and uploading thread are finished, then do somethings.

For example, if I want to send all the data captured every time app transitions to background, five seconds is not enough sometimes, app will crash and it's a bad user experience. We can add a long-running task by using beginBackgroundTaskWithExpirationHandler, when all the tasks in this dispatch group are finished, notify the system our long-running task is finished.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
[center addObserverForName:UIApplicationDidEnterBackgroundNotification
                    object:nil
                     queue:[NSOperationQueue mainQueue]
                usingBlock:^(NSNotification *notification) {
        UIApplication *application = [UIApplication sharedApplication];
        __block UIBackgroundTaskIdentifier taskId = [application beginBackgroundTaskWithExpirationHandler:^{
                Dispatch_async(dispatch_get_main_queue(), ^{
                        // If task is overtime, end it
                        if (taskId != UIBackgroundTaskInvalid) {
                            // Failed to finish background task, cleanup
                            [application endBackgroundTask:taskId];
                            taskId = UIBackgroundTaskInvalid;
                        }
                    });
            }];

        [self close];
        [self upload];

        dispatch_group_notify(self.dispatchGroup, dispatch_get_main_queue(), ^{
                // Finished executing background task
                if (taskId != UIBackgroundTaskInvalid) {
                    [application endBackgroundTask:taskId];
                    taskId = UIBackgroundTaskInvalid;
                }
            });
    }];

In Android, as I did before, create another handler UploadHandler in SessionHandler for the uploading tasks, HttpClient can be used to send the data to server via HTTP.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
final HttpClient httpClient = new DefaultHttpClient();
final HttpPost httpPost = new HttpPost(url);
httpPost.addHeader("Content-Type", "application/x-gzip");
httpPost.addHeader("Content-Encoding", "gzip");
httpPost.setEntity(new ByteArrayEntity(dataEncrypted));
try {
    final HttpResponse response = httpClient.execute(httpPost);
    final int statusCode = response.getStatusLine().getStatusCode();
    if (statusCode == 200) {
        // do things
    } else {
        // handle failure
    }
} catch (Exception e) {
    Log.w(TAG, "Error occured during data sending, abort reason: " +
            e.getMessage());
    // handle failure
}

User Identification

Each iOS device has a Unique Device Identifier (UDID), it could be used to identify the app user, but Apple Has Started Rejecting Apps That Access UDIDs.

In iOS 6, Apple provides ADID, we could use it to identify people, but app user can switch it off if they don't want to be tracked.

I use OpenUDID and ADID to identify app user, what OpenUDID does is generating a UUID and caching it for later look up.

In Android, it has IMEI, ANDROID_ID, and mac address, none of them is reliable, so I hash those three with device model use SHA to identify the app user.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
// Android UUID known to be duplicated across many devices due to manufacturer bugs
public static final String INVALID_ANDROID_ID = "9774d56d682e549c";

public static String getUDID(Context context) {
    String imei = "";
    if (context.getPackageManager().checkPermission(Manifest.permission.READ_PHONE_STATE,
            context.getPackageName()) == PackageManager.PERMISSION_GRANTED) {
        TelephonyManager tm = (TelephonyManager)context.getSystemService(Context.TELEPHONY_SERVICE);
        imei = tm.getDeviceId();
    }
    String androidId = Settings.Secure.getString(context.getContentResolver(),
            android.provider.Settings.Secure.ANDROID_ID);
    if (androidId == null || androidId.toLowerCase().equals(INVALID_ANDROID_ID)) {
        androidId = "";
    }
    String macAddr = (getMacAddr(context) == null) ? "" : getMacAddr(context);
    return SHA(imei + androidId + macAddr + Build.MODEL);
}

public static String getMacAddr(Context context) {
    String macAddr = null;
    if (context.getPackageManager().checkPermission(Manifest.permission.ACCESS_WIFI_STATE,
            context.getPackageName()) == PackageManager.PERMISSION_GRANTED) {
        final WifiManager manager = (WifiManager)context.getSystemService(Context.WIFI_SERVICE);
        final WifiInfo info = manager.getConnectionInfo();
        if (null != info) {
            macAddr = info.getMacAddress();
        }
    }
    else {
        // no permission
    }
    return macAddr;
}

static String SHA(final String toConvert) {
    try {
        final MessageDigest md = MessageDigest.getInstance("SHA");
        final byte[] digest = md.digest(toConvert.getBytes("UTF-8"));
        final BigInteger hashedNumber = new BigInteger(1, digest);
        return hashedNumber.toString(16);
    } catch (final NoSuchAlgorithmException e) {
        throw new RuntimeException(e);
    } catch (final UnsupportedEncodingException e) {
        throw new RuntimeException(e);
    }
}