Development, Google, Mobile

Google Play Store in Numbers – The numbers that Google won’t share

Google’s Published Data is as poor as it can be

Every now and then Google releases the latest numbers regarding the Android platform (which you can find Here).

According to Google this Dashboard :

“provides information about the relative number of devices that share a certain characteristic, such as Android version or screen size. This information may help you prioritize efforts for supporting different devices by revealing which devices are active in the Android and Google Play ecosystem”.

This summarized data helps the developer and there’s no doubt about it but, given the ammount of data Google has, don’t you think those Dashboards should be richer ?

If Google won’t release any detailed data, lets gather it ourselves

I have recently tried to gather data on the web about the actual situation of the Google Play Store so that i could generate some richer data for our fellow developers. Needless to say that i failed. There’s no such data available online.

At this moment i’ve had all the questions in my head but no way to find the answers. That was when i decided to dig into my own crawler. For those who are not familiar with this word, crawlers are programs designed to scrape (or Mine) data out of a source, in an automatic way, without the need of any human intervention during the process.

The crawler should be able to visit pages (of apps) of the Google Play Store, parse data about each page (Number of Ratings, Price, Developer, Stars, Downloads, Version…) and store it into my own Database in a structured way so that i could query it the way i wanted to.

After a week of work i’ve had my own server and crawler setup for running without any problems (which you can find the code in My GitHub Repository).

All i needed now was to wait for it to do it’s work, so i waited, and waited, and waited.

My Result: Google Play Store in Numbers

After running my crawler (in multiple processes) for 7 days i started to get IP Banned/Blocked by Google’s black magic. Even using proxies and changing my crawler’s behavior dinamically I would still get blocked really fast. As slow as it was, I have managed to mine data out of roughtly 1.010.000 apps (which is something near 95% of the Play Store according to Wikipedia).

If you’re curious about the result, you can find the Excel file i created right here. Note that the database is public, so you can hit the GitHub page and connect to it in order to do your own queries if you like, as well as running the crawler in your home to help “Crowdsource” our own Database of Play Store Apps data.

In the end I have learned that we shouldn’t be depending on information given to us by the “Big Players”. There are tons of public data sources on the web, waiting to be mined and studied. We might be surprised with some of the results we find.

If you have any questions, considerations, comments or suggestions, leave it on the comments or find me at About.Me

Update: Thanks to the help of this blog readers, the crawler was able to populate the database with around 1.1 Million apps from the Playstore, which may translate into roughtly 95% of Reach. That’s how big “Crowd Crawling” can be

Update 2: By popular demand i have decided to try and implement a crawler on the same model of the Play Store one. You can find it on this Public Github Repository
 

Standard
Cloud, Uncategorized

Living in the Cloud

Are you ready for the cloud ?

What is the cloud exactly?

The first thing you should understand about the cloud is that it is not a physical thing. The cloud is a network of servers, and each server has a different function. Some servers use computing power to run applications or “deliver a service.”

Other servers in the network are responsible for storing data.

For example, when you take a picture on your smartphone, it is stored on your phone’s internal memory drive. However, when you upload the photos to Instagram, you are uploading it to the cloud.

Is “The Cloud” really safe?

Before talking about some advantages of using the cloud for your day-to-day tasks and services, it`s worth mentioning that there’s no such thing as a “fail proof” cloud service. Most of the cloud services we use, by consequence, our documents, photos, musics and so on, are not fully safe. Big players in the cloud realm such as Dropbox, Sony (Playstation Network) and even Evernote have been hacked, at least once, exposing at least the username and password of their users, leading to a potential theft of sensible data of their users.

Another cenario that you should have in mind is that the mother nature is unpredictable. There’s no way to make a Datacenter “tsunami-proof” or “meteor-proof”. Shit happens, and there’s no way the services can protect your data from these situations. The best they can do is to duplicate your data across multiple geographical zones, making a “backup” in different regions of the globe, making sure that for instance, if an earthquake hits their datacenter in China, they still have another one in the US mimicking the one that was destroyed.

Ok, so, why should i move to the cloud?

Allright, so we now know that the cloud is not completely safe, but, how about your house ? The chances of you living in a bunker are very low. So, the data sitting in a DVD, Flash Drive or External Hard drive carefully placed in your drawer by the bed, are also vulnerable to the same disasters that the datacenters that hold the cloud services are. Ok, i bet that now the idea of having professional people taking care of your data, seems a little better, right ?

Making use of cloud services correctly, means that your data will always be backed up, and available everywhere no matter what kind of data we are talking about.

There are cloud services that allow you to storage all your files, no matter which format. Be it the last meeting excel sheet, or the report presentation to your boss and even your last vacation pictures. The sky is the limit, or should we say, the cloud is the limit ?

Since the files you uploaded to the cloud service you are using, are not only on your computer, they are acessible everywhere, from every computer with internet access, making it easier to get to that brownie recipe you uploaded to Evernote or to that very secure password for your personal email that Lastpass have saved securely for you, so you don`t even have to remember it.

The cost of moving to the cloud?

Nowadays there’s a plethora of cloud services with free tiers (usually limiting the maximum storage you can use), meaning that the majority of the internet users can start living in the cloud, for free.

Here are some examples of cloud-storage services, that offer a decent free tier:

  • Dropbox – 2GB Free (+500 MB for each friend you invite)
  • Skydrive – 7GB Free
  • Box.net – 10GB Free
  • Google Drive – 15GB Free

So, are you ready for the cloud ?

Are you ready to migrate your workflow and personal files to the cloud ? Do you have any fears ?

Let me know in the comments what you thought about this post, what are your favorite cloud services and any sugestions / advices for people who are interest in trying the cloud.

See you in the next post

Standard
Mobile

Mobile Cross-Platforms Development Tools

Image

Write Once, Run Everywhere

As mobile OSes grow and shrink, sprout and vanish, the pressing question remains. How do you choose which mobile development platform to focus, and which ones you should omit / put on hold ?

Well, there’s still no answer for this question, but the bright side is that, if you actually don’t want to choose a single platform / target OS, you can develop for all of them. Using the same code (in theory).

This whole “Write Once, Run Everywhere” (AKA W.O.R.A) is neither new, or perfect but it’s definitely on it’s best moment with the HTML 5 hype.

Web Application vs Native App

When it comes to targeting mobile market, there are three and only three options, as we will see now each one of them has it’s own place and level of capabilities.

Web Applications (aka WebApps) are basically mobile version of websites, which are not platform-specific. Sites that got scaled / redesigned to fit the small screens of the handsets that we all have in our pockets. As this is only a remodeled site, it must be accessed through a browser, rendering an HTML/CSS/Script bundle. Because of today’s security patterns, each browser will have a “sandbox” that prevents any code from reading/writing directly from/to your device. Since your WebApp runs on top of a sandbox it will not be capable of accessing any feature of the phone/tablet itself, and this is the biggest limitation of the WebApps, since this will generally lead to a much “less immersive” experience for the user.

Native Applications, on the other hand, are platform specific, meaning that if you want it to work an specific platform/OS, you will have to code it in the target’s platform language (Objective-C for iOS, Java for Android, C# for Windows Phone), which is called Native Code. If you want to target multiple platforms, you will have to write the same application on each one of the languages in order to be able to distribute it to the target OSes. Well, the bright side is that, because you are so close to the OS itself, you have access to every feature the phone has to offer you, be it camera, geolocation/geofencing or receiving calls/sms, for instance.

Web Hybrids

According to the dictionary, Hybrids are : anything derived from heterogeneous sources, or composed of elements of different or incongruous kinds. What the dictionary does not say is that, generally, the result is not better than any of the sides, and it’s usually worse than both.

Web Hybrids, in the mobile development realm, are applications that are (usually) written using HTML5 / CSS / Javascript. Those hybrids are built using specific frameworks that are able to expose the native features of the phone (camera, gallery, geolocation), but not all of them. As you may have noticed the hybrids are between the WebApps and Native Apps in both implementation effort (easier than Native Apps but harder than WebApps) and features richness (more features than WebApps, but less than Native Apps). The reason why the hybrids are able to use the handset features while the webapps can’t is because they do not run into a browser. The development framework uses a Native Wrapper (just like a shell around your web code), allowing it to run side-by-side with other native applications, outside the browser’s sandbox. Those wrappers also implement the “bridge” between the native features of the mobile device and the script call to them, just like an “API”, allowing you to make use of hardware functionalities via an script call for instance.

Here are some of the Frameworks that works this way:

Non Web Frameworks

As we have covered before, the “Web Hybrids” cannot access all the hardware features the phone may have to offer. Say, for example, you need hardware acceleration or a specific GPU command for a game, the Web Hybrids will not be able to access them, but there is someone who can.

When it comes to non-web frameworks, you can still find some W.O.R.A solutions for mobile, and i will try to cover them really quick. The reason behind those tools is the same : Write your app using a single programming language, and compile/deploy it to multiple platforms using the same code. Of course you will still have to build different UI’s most of the cases, but the “core” code still reusable on multiple platforms once it’s compiled correctly.

Most of those frameworks are focused into mobile game development, anyways, lets see what we have available.

Ansca Mobile Framework

Corona SDK

One of the players of this market is the Corona SDK. The language used here is Lua programming language, and it’s main focus is building games, but you can still build anything from business, to education apps and even Ebooks.

Recently Ansca Mobile announced that a free version of Corona would be released, so you can use it without paying a dime for it, with it’s own limitations of course. The Pro version is priced at 599 USD per year and the Enterprise one ranges from 999 USD per year to 2.499 USD per year.

No matter which licence you acquire, you can always compile your code to iOS, Android, Nook (Barnes and Noble) and Kindle. Unfortunately, no Windows Phone here.

Unity 3D

There’s no way to mention cross-platform gaming development without mentioning Unity 3D Framework. Unity is, no doubt about it, the biggest player in this market, mainly because it is a Game Engine with a built-in IDE. Using the Unity 3D allows you to compile your code to basically any platform available out there, including XBOX, PS3, Wii-U, Web (using unity web player), Android, iOS, Windows Phone, Linux, Windows and OSX.

The engine is built in C/C++ and it supports code written in C#, Javascript, Boo (a phyton syntax inspired script language). The pricing for this Titan, depends on which platforms you want to target. Each platform needs it’s licence if you want to compile code to it, and the prices for each platform range from Free (with limitations) to up to like 80.000 USD for the PS3. Check it’s store for the prices.

There are free licence options for Android and iOS, with their own limitations but the Pro one is 1500 USD per platform (one time fee). It may also worths noting that there is an “Assets Store” on their site, meaning that you can buy sprites, logos, 3D scenarios and characters for your game from their store, created and maintained by it’s own developers community. Kudos to Unity.

MOAI SDK

Moai SDK is a relatively new player, competing directly with the Corona SDK, targeting iOS, Android and Chrome as possible outputs and also using LUA as the main programming language for writing the apps, but the main difference is that, this guy is Open Source.

The prices range from Free, to up to 499 USD per Month, depending on your scale. Seems like it’s only possible to build games using it, which automatically puts it behind Corona since both of them have a free licence option.

Xamarin Mono Touch

The last competitor from today’s list is Mono Touch, from Xamarin. In my opinion, this is hands down the best framework for building non-game apps. Mono is a very robust framework, using C# as it’s input language and allowing you to deploy either to iOS and Android (but since the code must be written in C#, you can basically copy and paste the code and use it to build a Windows Phone version of your app).

For deploying to Android, you will need a Windows PC. As for iOS, you will need a MAC. This is the only downside of the framework, but the reason for this limitation is that it uses XCODE integration as it’s IDE for iOS. That’s right, you can use your beloved XCODE to code using C#, and even the Interface Builder will work just as you expect.

For the Android version, you can either use it’s own IDE (Xamarin Studio) or, if you want to pay for it, Visual Studio. The same integration is found here, allowing you to code into the best C# IDE out there.

The prices start from Free (but trust me, you don’t want this. This is basically a joke since you can’t build apps with more than 120kb, which is ridiculously small), to 1899 USD. The cheapest licence allowing visual studio integration is priced at 999 USD. Every price is bound to one Platform, so multiply it by 2 if you want to target two platforms.

TL:DR

The post was long, i know, but i’ve been willing to write about it for two weeks, and there was no way i could make it smaller without ommiting content.

If you have managed to reach this section, it means that you know now that there are lots of options for you to build your app using cross-platform deploying technology, be it Web Based or Close to Native.

I hope this helps someone, someday.

Standard
Mobile, Opinion

Mobile Market Fight

Big Mobile Players

The whole world is living a mobile boom. The smartphones and tablets are everywhere with a higher market penetration every month. This whole hype around the mobile realm is catching the eyes of milions of developers worldwide, resulting in a boom of mobile apps aswell. Those great apps have managed to turn already great platforms into even better and richer ones when it comes to features and usability. As those platforms become better and better, more customers get hooked. With the increasingly number of potential buyers for mobile apps, more and more developers tend to hop into mobile development. Seems like a cycle, right ?

As the smartphones and tablets started to grow, the big players wiped the small ones, and nowadays, all we have left are the Fantastic Four of platforms fighting for their share in the mobile market. Apple, Google, Microsoft and RIM are the titans in the arena, each one with its own mobile operational system (iOS, Android, Windows Phone and Blackberry). This is the basics, but how about some real data about these giants ? No Problem.

Data, Data, Data…

Mobile Infographic

Gains and Losses

As we all know, Apple is losing market share mainly because of the growth of the Android platform. On the other hand, the Windows Phone is gaining market quickly. Innovation is the key everywhere in the tech realm, and since Apple have been lacking creativity/new features that “wow” their faithful users, some of them already started their exodus to the rivals Android and Windows Phone. Both players have shown some significant quality boost both in the devices (hardware) and OS versions (software), while Apple shrinked the iPad.

“Competition is good for consumers”

You may be asking, ok, what do I win ? We, as consumers, end up winning overall better mobile platforms, devices and apps.

The more intense the competition gets between the big players out there, the more they are going to put their efforts into evolving their own “OS’es”. As we saw in the introduction to this post, this is where the “cycle” comes in. The better a platform/OS is, the more consumers (like you and me) it gathers, resulting in more developers focusing their efforts into the platform, which will develop better/more apps for the OS, making the platform even better, and attracting even more consumers.

My Prediction

If I were to bet in a single platform to “rule them all”, i would definitely bet on Android. Apple is quickly losing their loyal apple-fans to Samsung, HTC (which are two of the bigger members of Open Handset Alliance – OHA) and Nokia (which is the only manufacturer of Windows Phones nowadays), while Android have been eating Apple’s slice (no pun intended) of the market.

I wouldn’t say Android will pass 70% of the market, but i would say that Windows Phone will top Apple’s iOS by 2016 (when it comes to Smartphones only, the Tablet market is completely different, the iPad is the King, followed by the Kindle Fire from Amazon), and they will fight for the 30-35% left untouched by Google.

I couldn’t think of any brilliant phrase or conclusion to this post (mainly because this is my opinion about the mobile world competition) so here is a picture of a cat. Everyone loves cats.

Cat Closing the Post

Standard
Google

The Hidden Gems From Google IO 2013

Google IO 2013

Google IO is an annual event, where Google shows its latest goodies and news. Last year, Google revealed a lot of hardware such as the first tablet of the Nexus family (The Nexus 7) and a media streaming device called Nexus Q (which has been a major fail in sales). Also, there was a great highlight when Google announced the Android 4.1 Jellybean OS followed by a quick demo of it’s features. As if it wasn’t enough, google came with the cherry on the cake, the awesome Project Glass

The Google I/O 2013, on the other hand, announced no new hardware. Many people considered this a huge leap backwards, since they were expecting either a new Nexus 7, Nexus Q or the Phone X (the Nexus branded phone from Motorola). For those who think that since there was no new awesome hardware, the IO event was a waste of time, here I am trying to show the reason they should care about the releases of this year’s event.

No Hardware. No Fun. Are you Sure?

Here is the list of everything announced at the Google IO 2013, and the reason you should or should not care about it.

    Game-On !

    Google Play Games Service : For those who play games in the android ecosystem, be it casual or hardcore, this is a huge step forward. This service promises to increase the engagement between friends when playing into the Android platform.

    Why should you care?
    Google Play Games announced a new feature which will sync the state of your game between your devices. You can start playing Angry Birds on your Tablet, and have your phone synced to where you stopped playing in less than a minute. Great, Right ?

    Google Cloud Messaging Improvements

    Better Google Cloud Messaging (GCM) integration with Google Play Services: Developers can now send tons of notifications and push data to multiple users of their app, with a higher efficiency.

    Why should you care ?
    To be honest, there is nothing awesome here. Developers now can, for example, send a message to your device to “signal” your device to execute a certain action, such as an check for upgrades. The reason for the hype, is that this now works for multiple devices at the same time

    Google Music is a Threat to Pandora, Rdio, Groveshark and Live365

    Now you can pay 9.99 USD a month and stream every one of the 20 millions song available on the Google music service.

    Why should you care?
    If you live outside the US, you should not care, since it’s only available in the US. But, if you live within the United States this means that, you can now access a Radio service based on your preferences (since Google knows you better than your mom does), ads free, with over 20 millions songs available, with the ability to allow you to cache songs offline on your Android device.

    Google+ Major Overhaul

    From now on, Google+ is even more awesome than it was. Every single day the engineers at Google are working their ass out to make Google+ a better platform, and they are on the right track.

    Why should you care?
    If you are not a Google+ user, than you probably won’t win much here. The new features of the Google+ includes AutoImageEnhancing (basically an “Auto Photoshop Me” button), Autotagging of content to help you find what you want faster, Automatic Highlights selection of photos (if you upload every one of your 1000 photos from your last trip, Google will pick the bests ones and make an album. This is incredibly precise), just to name a few features.

    API’s are the Foundation of Awesome Apps

    Using the latest Google API, now, allows developers to build even greater apps. The new API’s include a more battery efficient way to track your location (1% battery juice spent per hour of use), a Geofencing feature (developers can now trigger actions based on users locations within a pre configured area) and a Activity recognition API, which is able to identify whether you are walking, biking or on a car/bus.

    Why should you care?
    Well, API’s are the foundation of must of the apps with awesome features you see out there. Apps now, can easily perform tasks such as : When you enter the nearest market, it will remind you to buy something that you said you needed, or, once you start biking, an app can now turn the volume of your phone to the max so you can hear it while you ride your bike. Even tho there is no “Killer App” with those features, the APIs now make it easier for more apps to implement those features.

    In the end, it wasn’t bad

    Even without KeyLime Pie (Android 4.3 or Android 5.0), or a new Nexus device, google announced very solid features and product changes this year, which the impacts will be visible soon enough.

    This was my first blog post ever, so, i hope you like it and feel free to leave any comment, compliments or tips.

Standard