Mobile Pilot Scorecard




As described in my March 9 blog entry, Yes, You Can Deliver Mobile Learning, we recently completed our first mobile learning pilot with the gracious assistance of our vendor partner OnPoint Digital, using their CellCast platform.   The results are in:   as is typical of pilots we had some successes (hooray!) and we learned a few things (which always reminds us why we do pilots in the first place).

In terms of our success criteria, we fully achieved three out of four goals. Not too bad for a pilot.  We were even partially successful with our fourth goal.   So here is a look at our mlearning pilot scorecard:

1. Producing and converting content for mobile delivery: Success! - we were able to successfully deploy audio, video, .pdf documents, tests and surveys, and an interactive product placement map to our pilot audience.  Producing the learning content was the easy part.  After all, that is what my team is already good at.  For the pilot, we relied on OnPoint to convert the content to the format required mlearning.
2. Deploying content using a mobile platform - Success! - as previously stated, we used the CellCast app from OnPoint Digital.   It worked very well for us.  We had a few technical glitches that required some troubleshooting in the first few days of the pilot.  But doesn't that happen with all roll-outs that involve technology?   Once we got those glitches ironed out, it was pretty smooth sailing for the rest of the way.
3. Deploying content to a variety of mobile devices - Partial Success - We were able to successfully deploy to Android and iOS devices.  However, the majority of our pilot audience was equipped with older model Blackberries.  We knew going into the pilot that we were likely to hear grumblings from this group about the user experience. We were prepared to get that feedback, but there were two communication issues that made this a bigger challenge than I would have liked:  (1) Blackberry Enterprise Server (BES) software - this had to be unlocked so our users could download the app to their devices.  Nothing like this was required for either the Android or iOS devices.  (2) Lack of SD Cards - many of the users had never installed SD cards in their devices, so there was no place for their downloads to land.  This effectively put a good chunk of the pilot audience out of the game.   Many of the users without the SD cards, used personal devices to complete their training.  A handful of people went out and bought new SD cards and installed them so they could participate, but most just missed the opportunity.
4. Testing and recording participant completion results - Success! - one of the advantages of mobile devices that should not be overlooked is that they can be used for two-way communication.  Since this was our first pass at mlearning, we didn't go beyond receiving test scores and survey data back from our participants through their mobile devices, but it was encouraging to see the amount and type of data we could capture.   For example,  the image accompanying this blog entry is of a data processing unit.  It is a still capture from one of our videos that was viewed on average over six times by the pilot participants.  Most of the other videos averaged slightly less than three views each.  I'm interpreting that as telling us we need to beef up the DPU segment of our initial classroom training.  Great information, don't you think?

All in all, I would call our pilot a success.  We learned a few things about how to present our content, how to make downloading easier, and about what will work and what won't in terms of mobile learning.   And as far as our Blackberry device issue, well that has now been solved.  Reporting the results of our pilot to senior leaders gave them the last piece of information they needed to justify a device upgrade for our users.   How is that for a nice unintended outcome?