How I got Android App Installs + IAP numbers mailed to my Inbox

As has become the norm for me in 2018, I’m going off again to travel for a couple of months. During which time I plan to have a little detox from coding which means, aside from bugs which need to be fixed & support emails that need to be answered – I won’t be doing much else regarding work on my apps.

However, as an indie dev I still need to keep on top of things like installs, uninstalls, usage, IAP purchases & the like, mostly to identify any issues that might not necessarily trigger a Crashlytics email but could still substantially impact my user base. Unfortunately for me, that still requires more manual interaction then I want — I can live with the Play Developer console app but Firebase Analytics requires firing up a laptop, and sifting through the data across all my apps on the Firebase Dashboard — which definitely does not qualify as a digital detox of any stretch! Not to mention the WiFi in Asia can be pretty flakey at best & the Firebase console isn’t exactly light on resources..

Hence the reason I built a solution that would collate all the data I wanted to know about my apps, and send me a daily overview of it — straight to my inbox! Zero manual checks necessary, with a cost of $0 (or close to zero — we’ll see how the usage affects billing 🤷).

Now if you just want to get all the analytics you care about sent straight to your inbox & have zero interest in how it happens or doing it yourself, you can skip the rest of this post & just sign up here. However for those of you who might want to roll your own or are just curious about how it all came together, let me explain how I did it..

Putting the pieces in place

The first thing when you’re building something like this is to figure out all the moving parts & how they all slot together. In this case I knew I’d need:

• a database component to store the information
• Some server side API component to collate that data submitted across multiple installs throughout my user base
• A way to email that data to my inbox
• And an email template to display the data
• Oh and I’d need to build a client side SDK I could integrate across all my apps to make my life a little easier.

And it had to cost me $0 dollars. #poorindiedev

In the end I went with the combination of Firestore for storage, Firebase Functions to act as my API layer, MailGun as the email functionality for it’s easy to use SDK & great free pricing tier, and I nabbed a free email template from SendWithUs.

I also decided that to make this as simple as possible (there’s no easy way to get detailed analytics in an email & I wasn’t going to reinvent the wheel – if I want details I’ll have to suck it up & sign in to the dashboard), the SDK was going to be type & event based. Simply put you define a type ie “Feature Used” & you record an even for that type ie “Search performed”. The SDK will increment with every event & at the end of the day I’d know my search feature was used X amount of times today across Y installs. Simple.

The API layer

Now that I knew my data structure I could go ahead & build out my Firebase Function to handle the collation & uploading of the data from my as-yet-to-be-built client side SDK. (I’m assuming you know how Firebase works but if not, here you go.)

Client side I’d be using Android’s SharedPreferences to record the data for it’s simplicity, I just needed a way to get it into the Firestore database without requiring the Firestore SDK. Mostly because I wanted to keep the SDK as lightweight as possible but also because I already use the Firestore SDK in a number of apps & I didn’t want the hassle of updating this SDK anytime they’d be a clash in version numbers. Also all we’d be doing is making 1 post request a day so a full SDK seemed kind of excessive.

Side note: If you’ve never used Firestore before, outside of creating some rules to protect your data there’s no real setup, you just start writing straight to the database.

Luckily Firebase functions provides you with the ability to expose any functions you write as a HTTP endpoint, which is exactly what I did:

exports.uploadStats = functions.https.onRequest((req, res) => {

  return uploadStats(req, res) //This function is detailed next

});

To that endpoint we’d be sending a JSON payload in the format of:

{
  some_type: {
    some_event: number_of_occurences
  },
  some_other_type: {
    some_event: number_of_occurences
  },
  ...
}

And then compile that data with all the data collected from other users by first grabbing the package name of the app, which I’d be sending in the header:

var packageName = req.get(‘packageName’)

And then using that to grab any existing stats for the app already in Firestore:

var docRef = admin.firestore().doc("appStats/" + packageName)
return docRef.get().then(snap => {
  var appStats = snap.data();
  //If no stats exist yet create an empty map
  if(isNull(appStats))
    appStats = {}

  ...

});

Next we’d loop all the type/events from our JSON payload to see if they already exist in the data we just pulled from Firestore and if so, we simply increment the count with the new user data, but if there’s types that don’t exist we add them:

//This function loops over the types
Object.keys(inputStats).map(function(type) {
  var inputTypeMap = inputStats[type];
  if(isNotNull(inputTypeMap) && isNotEmptyMap(inputTypeMap)) {
     var typeMap = appStats[type]
     if(isNull(typeMap))
        typeMap = {}

        //Inner function loops of the events per type
        Object.keys(inputTypeMap).map(function(key) {
          var value = inputTypeMap[key];
          var count = typeMap[key]
          if(isNull(count))
             count = 0

          count = count + value
          typeMap[key] = count

       });

       appStats[type] = typeMap
  }
});

And finally we write the updated values back to Firestore.

return docRef.set(appStats)...

A total of 1 database read, 1 database write & 1 function run per install. Efficient if I do say so myself.

Building the SDK

Now in the name of keeping things lightweight I also (much to your horror I know) didn’t want to include Retrofit as again — did I want to include a whole other library just for 1 Post call? So I did things the old school way.

internal fun uploadStats(context: Context) : Boolean {
   //The user has the option to disable all analytics/crashlytics from the ui
if(!SdkUtils.getCollectionEnabled(context))
return false

   val serverURL: String = BASE_API_URL + “/uploadStats”
   val url = URL(serverURL)
val connection = url.openConnection() as HttpURLConnection
   connection.requestMethod = “POST”

   //30 seconds because Triggers can be slow to start/return
connection.connectTimeout = 300000
   connection.connectTimeout = 300000

   connection.doOutput = true

//This method simply loops through my shared preferences & puts them
//into a map of type of:
//Map>

   val map = SdkUtils.buildMap(context)
val json = JSONObject(map)
val postData: ByteArray =
                 json.toString().toByteArray(Charsets.UTF_8)
   connection.setRequestProperty(“Content-length”,
                                  postData.size.toString())

   val outputStream = DataOutputStream(connection.outputStream)
   outputStream.write(postData)
   outputStream.flush()
   Log.d(TAG, “Response code: “ + connection.responseCode)
if (connection.responseCode != HttpURLConnection.HTTP_OK) {
val reader: BufferedReader = 
           BufferedReader(InputStreamReader(connection.errorStream))
val output: String = reader.readLine()
       Log.d(TAG,”Api threw error $output”)
return false
}
return true
}

And I wrapped it in a kotlin coroutine for good measure. (I know, that is an extra library where I could just use an AsyncTask, but I’d just learnt it so it’s going in there – logic be damned!)

launch {

  uploadStats(context)
  StatManager.clear(context) //Clears daily stats regardless of success

}

Now we just need a way to trigger the upload and the SDK is complete. For that I turned to the GCMNetworkManager replacement Firebase-JobDispatcher. Again this is a bit more weight to the SDK but it greatly improves the efficiency on the device compared to the alternatives so to me it was worth it.

val dispatcher = FirebaseJobDispatcher(GooglePlayDriver(context));

val myJob = dispatcher.newJobBuilder()
             .setService(UploadService::class.java)
             .setTag(UploadService::class.java.simpleName)
             .setRecurring(true)
             .setLifetime(Lifetime.FOREVER)
             .setTrigger(Trigger.executionWindow(windowStart, windowStart + toleranceInterval))
             .setReplaceCurrent(true)
             .setRetryStrategy(RetryStrategy.DEFAULT_LINEAR)
             .setConstraints(Constraint.ON_ANY_NETWORK)
             .build();

dispatcher.mustSchedule(myJob);

The job will run in a 1 hour window every 23–24 hours from the time the app is installed, so once a day for efficiency. The timing means depending on the time zones stats will be uploaded at different times but as I just want a basic overview it doesn’t matter so much if some of yesterdays stats end up in today’s for some users. Plus the varied scheduling means 10’s of thousands of installs won’t all be trying to upload at the same time.

Oh and I used the on app updated receiver to set the job.

class AppUpdatedReceiver : BroadcastReceiver() {

@CallSuper
override fun onReceive(context: Context, intent: Intent) {
   Log.d(TAG, “app updated..”)
   SubmissionManager.scheduleUpload(context)

...

Which, if you don’t know, is declared in the manifest with the intent:

<intent-filter>

 <action android:name=”android.intent.action.MY_PACKAGE_REPLACED” />

intent-filter>

And the SDK is complete!

Note: I purposefully omitted how I’m logging the actual stats as it’s simply incrementing a SharedPreference, but for reference I’m storing each SharedPref as a Long value with the name type:key separated by a colon, which I split at the time of upload to extra the type & key.

Building the email

So right now we have an SDK to collect the stats & a way to upload them to our API layer which will do it’s thing & then add them to our Firestore database, meaning all that’s left is generating & sending the actual email.

First things first we need an email template, for which I turned to SendWithUs’s free open source email templates.

All I did was slightly modify the template I chose to work with HandlebarsJS so I could easily populate the email template directly with the JSON pulled from FireStore. Also, as the types & keys can be absolutely anything it needed a generic solution to display how I wanted. It looks something like this:

..header of the email..

{{#eachInMap this}} //Top level type

   {{key}} //Create type title with key

     {{#eachInMap value}} //Nest event:value for each type

       Event: {{key}}

       Count: {{value}}

     {{/eachInMap}}

{{/eachInMap}}

..footer of the email..

Where eachInMap is a custom handler I s̶t̶o̶l̶e̶ borrowed from this Stackoverflow post.

var handlebars = require(‘handlebars’)
handlebars.registerHelper( ‘eachInMap’, function ( map, block ) {

   var out = ‘’;
   Object.keys( map ).map(function( prop ) {
      out += block.fn( {key: prop, value: map[ prop ]} );
   });
   return out;

});

With that done I created an endpoint in my Firebase Functions that could be hit to trigger the email. It takes a query string which is the packageName of an app, which it then uses to grab the data from Firestore.

exports.sendEmail = functions.https.onRequest((req, res) => {

   var packageName = req.query.package_name
   var docRef = admin.firestore().doc(appStatsPath + packageName)
   return docRef.get()...

Checks if there’s any data to be emailed

if(isNull(appStats))
  //If empty I throw a 400 response

If there is, it grabs out email template from the filesystem (if you keep it in the same folder as your index.js file it’ll get uploaded when you deploy), and builds the email using handlebars.

var fs = require(‘fs’); //Filesystem

var source = fs.readFileSync(“./email_template.html”,”utf-8");

const template = handlebars.compile(source, { strict: true });

var html = template(appStats);

And then we send the email with mailgun.

var data = {
   from: ‘[email protected]’,
   subject: ‘Your daily email overview!’,
   html: html,
   ‘h:Reply-To’: ‘[email protected]’,
   to: ‘[email protected]’
}

var mailgun = require(‘mailgun-js’)({apiKey : mailGunApiKey, domain : mailGunDomain})

return mailgun.messages().send(data, function (error, body) {
   console.log(body)
   send(res, 200, { message: ‘Success’, body, });

});

And we’re done!

Well, actually not quite. We still need a way to trigger this email daily so we don’t have to keep manually hitting out sendEmail endpoint. For this I went super high tech… IFTTT 😁 That’s right, I wrote me a quick recipe (using IFTTT’s webhooks plugin to hit the url):

IF ‘it’s midnight’ THEN ‘hit this url’.

Simple. 👌

And that’s how it’s done!

With that done, this morning, just like every other morning these days – I woke up to this lovely email! 🙌🙌

No dashboards, no manual interaction, just an email, sent to my inbox, like clockwork! Nowww I can go off on my travels knowing that, should there be any major fluctuations in users daily use that might require some developer intervention, I’ll know about it without even having to look! (Plus I still get my Crashlytics emails so I’ll know about actual crashes also 😏)

Now if you’re reading this thinking “this is cool, I want to do the same thing” but you don’t want to/don’t have the time to put in the work to replicate what I’ve done, you might be interested to know I’m considering opening up the service that I built to all devs. One easy to integrate SDK & you’d be all set!

If that’s something that does interest you can let me know by registering here and if enough people are into it — it’ll be rolling out as soon as I’m back! 😄

And that’s that folks- thoughts, questions, suggestions.. any feedback whatsoever? I’m all ears.. 👂👂🤓