You have unlimited access as a PRO member
You are receiving a free preview of 3 lessons
Your free preview as expired - please upgrade to PRO
- Setting Up Datalab
- Python Data Analysis
- Deploy to Google Cloud ML Engine
- Predict from a Firebase Cloud Function
- The End
- Angular Elements Advanced Techniques
- TypeScript - the Basics
- Cloud Scheduler for Firebase Functions
- Testing Firestore Security Rules With the Emulator
- How to Use Git and Github
- Infinite Virtual Scroll With the Angular CDK
- Build a Group Chat With Firestore
- Async Await Pro Tips
- Sortable Drag and Drop Lists in Firestore
Machine Learning Engine TutorialEpisode 141 written by Jeff Delaney
Health Check: This lesson was last reviewed on and tested with these packages:
- python v2.7
- functions v2
Find an issue? Let's fix it
Source code for Machine Learning Engine Tutorial on Github
Building a predictive model is only half the battle when it comes to machine learning - you also need a way for clientside apps to consume it and add value to the user experience. Today we’re going to wire up a collection of Google Cloud services to train and deploy a Python-based ML model to the cloud. The following lesson is designed for product-minded developers who want to get an idea of the end-to-end ML process in the cloud. Here’s a quick summary of our milestones:
- Setup Datalab
- Build a custom machine learning model with Python and Scikit Learn
- Deploy it to Cloud ML Engine
- Expose it as a public API via Firebase Cloud Functions
Looking to train your data in a Firebase Cloud Function with TensorFlow.js? Checkout this Radi Cho’s demo.
Datalab is an environment built on top of Google Compute Engine that allows you to stream your cloud data directly to a Python Jupyter Notebook with virtually unlimited compute resources. This means you can spin up a GPU-powered cloud supercomputer from your laptop, then shut it down in a few hours when you’re done.
Follow the QuickStart Guide, which boils down to the following commands.
gcloud components update
After running the commands, you should be able to access your notebook environment via
The only thing missing from the datalab environment is the Firebase Admin SDK. You can install packages in the datalab by running pip install
!pip install firebase-admin
Next, go to the Firebase dashboard and download your service account from project settings, then save it to the datalab environment.
Now that we have datalab running, it’s time to start filling our notebook with some python code.
We need to load our data from Firestore into a Pandas dataframe suitable for ML. For this demo, I have generated synthetic data that is half random and half fixed. The goal is to build a model that can find the signal hidden within the noise/randomness.
import pandas as pd
With the dependencies imported, we can then generate some random data and save it to the database. Keep in mind, saving this data is completely optional - I’m only doing it to simulate a real-world machine learning problem.
devs = db.collection('developers')
It’s always best to start with an exploratory data analysis to gain a better understanding. Pandas provides a variety of methods to explore the data.
df = pd.read_csv('data.csv')
Now we’re ready for the fun part - training the model. To make our data suitable for training, we need to encode any string columns to numeric values and split it into training/validation sets.
# Encoding to Numeric
For our simple use case we will use a Random Forest algorithm, which usually does well out of the box.
from sklearn.ensemble import RandomForestRegressor
Now that our model is trained, we need to save it in a Storage Bucket to so it can be picked up by ML Engine in the next step.
from sklearn.externals import joblib
Our model is saved in Google Cloud Storage, allowing us to connect it to ML engine.
Go into the GCP console and enable the required Cloud Build and ML Engine APIs.
The next part is just a matter of clicking a few buttons. On the ML engine console we need to (1)create a model, then (2) create a version of it that points to the
joblib file in the storage bucket from the previous section.
It’s finally time to share our ML Engine model with the universe by exposing it as an API endpoint via Firebase Cloud Functions.
firebase init functions
import * as functions from 'firebase-functions';
Now you can make calls to this API to get predictions in matter of milliseconds directly from your clientside apps.
Datalab gives you an environment to analyze data and train predictive models, while ML Engine makes is much easier to maintain and release production versions to the cloud. When you combine these tools with Firebase you get a fullstack solution for building real-world AI products.