Firebase adds three new capabilities in ML Kit and Performance Monitoring for Web Apps Firebase, is getting its biggest update this y...
Firebase adds three new capabilities in ML Kit and Performance Monitoring for Web Apps
Firebase, is getting its biggest update this year at Google’s annual developer conference, Google I/O. Today, Google announced new ways they’re improving the accessibility of machine learning for developers; Google is also extending its performance monitoring tools to help web developers speed up their web apps.
Google announced ML Kit at last year’s I/O to take the mystery out of machine learning for developers. They started out with a couple of APIs for the most common use cases, and this year they’re expanding the SDK with the addition of 3 new APIs: an on-device API for translation, an API for object detection and tracking, and an API to easily create custom ML models. Native app developers can integrate the Performance Monitoring SDK into their app to collect performance data which they can then analyze in Firebase Performance Monitoring; soon, web developers will also be able to track the performance of their web apps in Firebase. I spoke with Francis Ma, Head of Product at Firebase, to learn more about these changes.
New ML Kit APIs
Google’s ML SDK currently supports 7 APIs: text recognition, face detection, barcode scanning, image labeling, landmark recognition, smart reply, and language identification. The last 2 were only recently added in April, but now they’ll be joined by the 3 aforementioned APIs. Here’s a high-level summary of the 3 new ML APIs for developers:
- On-device API for translation: Using the same model that powers the Google Translate app’s offline translation, this new API allows developers to provide fast, dynamic translations between 58 languages.
- Object detection and tracking API: This API lets an app locate and track the most prominent object, marked by a box around it, in a live camera feed. Developers can then identify the most prominent object by querying a cloud vision search API. As an example, IKEA is said to be experimenting with this API for visual furniture shopping.
- AutoML Vision Edge: For developers who want a custom ML model with minimal expertise needed, AutoML Vision Edge lets you build and train your own custom model to run locally on a user’s device. To train a model, one simply uploads their database (eg. a set of images) to the Firebase console and click “train model” to train a TensorFlow Lite model against the database. Google announced that a company called Fishbrain used this API to train a model to identify the breed of a fish, while another company called Lose It! trained a model to identify the categories of food in an image.
Machine learning is a quickly growing field in computer science, so it’s natural for developers to show interest in it. However, building and training ML models effectively without a data scientist on staff can be difficult, which is why Google is simplifying the process by automating the training of models with ML Kit. Developers can focus on building new apps with powerful functionality using the power of ML without having to devote significant time and effort into learning data science. With the addition of these 3 new APIs in ML Kit, we’ll hopefully see a lot of new useful apps in Google Play.
Firebase Performance Monitoring for Web Developers
Consumers demand good performance from the apps and websites they use, but Firebase has thus far only provided native app developers with the means to effectively monitor the performance of their products. At Google I/O 2019, Google announced that Firebase Performance Monitoring will be made available for web developers using Firebase Hosting. Web developers can keep users engaged on their platforms by improving the speed of their web apps; to help web developers spot the key weaknesses in their sites’ performance, Firebase will provide web-centric tools and telemetry measurements to show how real-world users experience a website. For instance, web developers will be able to monitor aspects like the time to first paint and input delay, how soon people first see and interact with content on a web page, and the mean latency. The overview dashboard will show these and other metrics to help web developers optimize the experience for their users, whether by country or globally.
