Articles

Develop Solutions Faster w/ GCP Marketplace: Home Depot QuoteCenter’s BigQuery Story(Cloud Next ’18)


[MUSIC PLAYING] ANIL DHAWAN: All right,
thank you for joining us. We’re super excited
to talk to you today about how you can
accelerate your solution development in Cloud
using our GCP marketplace, and we’re super thrilled to have
a great customer, Home Depot QuoteCenter, with us, along with
one of our partners, Matillion, who you may have already
heard of yesterday or throughout the conference
talking about their solution and the marketplace. And so I’ll be covering
the Cloud Marketplace part, and I’ll invite Ian
and Laura to talk you through what the customer
journey and the partner journey looks like for Marketplace. So how many of you are
already in Google Cloud today? Great. How many of you
are in the process of migrating to Google Cloud? OK, a few of you. So as you know,
the moving to cloud journey can be
quite overwhelming. You’ve got to migrate
your existing workloads. You’ve got to figure out how
you’re open source, your VMs, if you’re doing
containers on-prem, how all of this is
going to work in cloud, and how your commercial
solutions– those ISVs, those third-party
relationships you have– how those are going to work
in this new cloud world. At the same time,
you’re in Cloud, so you’re learning about all
the great new technologies that GCP has to offer, whether
it’s BigQuery, Kubernetes Engine, Cloud ML, et cetera. And what you’re trying
to do in the process is figure out, how do
I string all of these together deliver my
offerings in Cloud? So you’re trying to put
together GCP products, your own custom-built solutions,
third-party solutions, and new applications that you
might be building, and putting them all in on Google Cloud. So at the same time,
Cloud is powerful. GCP lets you pay only for what
you’re using, which is great. You get per-second, per-minute
billing for various services. It allows you to easily
prototype, spin stuff up, spin it down. If you’re running it in
a large organization, you can have different
departments dabbling with different technologies. So it really makes it easy
to kind of get quickly up and running in Cloud. At the same time, once
you’ve got a workload and you want to run
it in production, it’s also very easy to scale up. And that’s the beauty of Cloud Now, while the Cloud
problem is quite strong, the reality is there
are always gaps. And this is really where
the ISV ecosystem comes in to fill in those
gaps, whether it’s adding an additional capability
on top of maybe a Google Cloud Platform networking
function or actually solving a specific problem,
like we learned about today with Matillion and ETL. ISVs are there to fill in
the gaps in the ecosystem and make sure that you’re
successful in your Cloud journey. And what we’ve done with the
GCP marketplace is allow you, as users, to tap into
that ecosystem of ISVs that we work with
and we partner with, we vet, and we bless to help
you discover great solutions, deploy them easily,
and get a single bill. So you have one single
place to go with Cloud. You can build your
own infrastructure. You can add on ISV
goods, get a single bill, and be up and running
on your Cloud journey. So why use GCP marketplace? How many of you are already
marketplace customers today? How many of you were
using marketplace when we were called Launcher? OK, some of you. So last week, we rebranded
from GCP Launcher to GCP Marketplace. So we got a new shiny
logo and a bunch of new capabilities I’ll walk
you through really quickly. So what is a GCP Marketplace? Simply put, you can think
about it as your Google Cloud Platform app store. All the different things you
want to do with app stores, like finding goods,
trying them out, see what solutions
work for your problems. We standardize deployments. So what that means is when
you find a nice fee solution, you can be guaranteed
that you’re running their solution using
the best patterns and practices on Google Platform. So it’s really not a great
reason to kind of roll your own or do it yourself anymore. ISVs are working with
us, packaging the goods in a standard way, making it
really, really easy to deploy and get up and running. The next is Cloud-native
billing models. You may be used to
enterprise agreements where you’re locked in
for maybe multiple months or multiple years. As you know, with Cloud,
those dynamics change. We’re used to per-second,
per-minute billing, and the same is true
with our Marketplace. So many of our
marketplace solutions are actually built along those
models, where you’re just paying for what you
use and not more than what you actually need. And you’re getting this
all in a single bill. So from a procurement
perspective, you’re able to try things out. Ultimately, when
you try to run him in production and workloads,
you get one bill from Google across all of your
third-parties, and your infrastructure usage
on Google Cloud Platform. As I mentioned before,
all of our solutions in the Marketplace
are vetted by Google. And what that means is we
have deep relationships, both contractual and otherwise,
with each of our vendors, and that means when
there’s a vulnerability or there is an
update, those vendors are obligated to push
updates to the marketplace. And what that means for
you as a customer is that you can be assured that
if you’re deploying something from the marketplace,
you’re getting the latest and greatest, and
it’s going to be maintained. And these aren’t
vendors that are going to disappear on you next year. We actually have a whole
suite of technologies behind the scenes that
we use to security scan all of our solutions. We check for the latest CVEs. So if there is a vulnerability
in our core operating system image that maybe is not
from the vendor itself but the vendor is using
that operating system image, we’re able to detect that,
work with the vendors, get it patched and upgraded. So really a secure way
of running any software from my ISVs in the Cloud. And last but certainly
not least here is easy access to
support from vendors. The marketplaces are great
because it’s a self-service. Model you don’t need
to pick up the phone and call a sales person. But if you do ever
need help, we connect you to support quite easily
throughout the product. As I mentioned before,
we’re a growing ecosystem, the Marketplace. We’re now up at 800 solutions
from over 150 partners. If you want to applause,
that’s fine, yeah. [CHUCKLES] [APPLAUSE] It’s a huge number. We’ve been growing tremendously. This is 4x where
we were last year. So it’s a really,
really tremendous pace of growth for
Google, for you all, and for our whole ecosystem. You probably recognize
a lot of big names here. They have products
across the board from security, networking,
storage, data analytics, I mean you name it. And we’re constantly
adding more. We’re also proud to
announce as of last week support for Kubernetes
applications, the commercial
Kubernetes applications, specifically, in
our marketplace. We’re actually the only cloud
platform provider– public cloud platform provider today
offering commercial Kubernetes applications, and
we’ve been partnering with a great set of companies
you see up here already. These are all live today in the
Marketplace you can try out, and we have a whole
host more coming. A couple quick points–
how many of you are running Kubernetes
Engine today Google Cloud? Great. With the solutions we’ve
built on the marketplace, all of these can
be click-deployed onto a Kubernetes
Engine cluster. So that means you can come to
the user experience and just, in a matter of a couple
of minutes, you can get up and running with any
of these solutions. You can actually run
these solutions not just in Kubernetes engine. You can actually take
them and run them on-prem or in another
Kubernetes environment. Maybe you’re running
Kubernetes on GCE or using other infrastructure. You can still use
these solutions. And for the
commercial solutions, you still get one
single bill from Google. So think about
that for a moment. You start your journey in Cloud. Maybe you prototype your
offering or your larger application in Cloud using
third-party software. And you can actually
take that same stack and bring it into a
different environment, and for the commercial
Kubernetes applications, you still get one
unified bill from Google for your usage across clouds. It’s a pretty
powerful capability, something we’re really proud
of and actually really excited to offer you. All of our solutions
that are commercial get metered through Google,
so you get one bill from us. But more importantly, you’re
only paying for your usage. So in the virtual machine
world, you typically pay for maybe VM
hour or core hours. With our Kubernetes
commercial solutions, you can actually pay on
different dimensions. For example, a database
vendor might charge you based on the number
of gigabytes stored, not the compute instance
hours, so a lot of flexibility. A lot of these solutions are
building based on dimensions that matter more
to the application. So it gives you a
lot of flexibility for pricing, costing, and
evaluating these solutions. Like I said, you
can deploy anywhere. You still continue to get that
single great bill from Google. Our partners are super excited,
too, to be trailblazers. Here is a quick quote from
Aqua Security, a large partner of ours, just talking about
the opportunity of getting in front of customers. I think the thing I want you
to take away from this message is it’s not just Google pushing
these solutions with you. It’s actually vendors looking
at the Marketplace and saying, this is the channel by
which we want to transact, and this is the channel by
which we want our customers to consume our goods– so a really great
cohesive relationship. Like I said, we’re over
800 solutions this year and growing like crazy. So with that, why don’t I
quickly turn to the demo and walk you through some
of the interesting points of the actual Marketplace? So the first thing to note
is the Google Cloud Platform marketplace is a part of the
Google Cloud Platform console that you’re already using. It’s really easy to access. It’s actually the
first menu item. We’ve done this intentionally
because if the Marketplace is going to be your
place for discovering all kinds of products, we want
that to be front and center. I want to highlight
a few other things we’ve done around discovery. You can, of course, access the
Marketplace from the menu here. But often times, you might
be in a particular part of the Console
experiencing a problem. Maybe you’re configuring
a network setting in GCP, and you realize that you need
an additional capability that’s not currently offered
by the platform. One of the things
we’ve been doing in the platform is saying, how
can we make it easier for you, as a customer, to discover
solutions from ISVs wherever you may be in the Console? Let me quickly show you
what that looks like. So here I am at my Home
page in Cloud Console. And let’s say I were in
that networking section, so let’s say go
Network Services. What you’ll see
here is, of course, I can configure my load
balancing settings and whatnot. But you can also see a
callout here directly to the marketplace. And when you click
this, you actually get this overlay of
tailored solutions that are specific
to that section. So in this case, our
engine was smart enough to pick up that you were
in the networking area, and it’s going to show
you networking solutions that it thinks might
be useful for you. And we’ve done
these integrations throughout the
console, so you don’t have to go directly to the
Marketplace all the time. You can actually
discover the solutions from wherever you might be. I’m gonna talk to
you really briefly– or take us through what the
Marketplace actually looks like and some of the behaviors there. As I mentioned, we have
lots and lots of products, and they span of variety
of different types. So we have virtual machines,
container images, public data sets, APIs and services,
Kubernetes solutions, operating systems, et cetera. But what we also do is we also
have all of our Google Cloud Platform products there. And the reason why
that’s important is because what
we’re trying to do is create a way where
you, a customer, if you have a problem, you can come to
one place and find a solution. Sometimes, that solution
might come from Google, and many times a solution
might come from ISVs. But we don’t believe
in a world where you have to go to
one place to find Google products and another
place to find others. We want to have
one kind of house where you can find all of these. And not just that, you should
be able to find solutions across types, whether it’s a
data set, a virtual machine, or a container. You can be able to choose
what works best for you. And so we have a really
robust filtering. So if you were
interested only in, let’s say, Kubernetes
applications. You could click that. You can actually see
only those solutions. And of course, you can filter
down into subcategories and get into, for example, the
database solutions in there. The next thing I
want to show you is what we’ve done with
our deployable products, so virtual machines and
Kubernetes applications. I’m going to show
you how those work and how you can quickly
deploy those yourselves. So I’ll quickly take,
let’s say, WordPress, kind of the Hello World of
Cloud, as sometimes we call it. So here, you see the
basic details page. You get a good estimate of
how much it costs to run. And you can see that it’s
a really popular solution– there have been a lot of
deployments recently– learn about the solution. We give you a table
on the side, which is showing you exactly
how much the solution is going to cost you. So we have a standard
default configuration. All of our deployable products
are built with smart defaults that we work with
the vendors with. And so you know that if you’re
just click, click, deploying, you’re going to be in
a known good state, and you also get great
pricing understanding just right off of the screen here. And if you’ve been using Google
Cloud Platform for some time, you’re probably familiar with
our sustained use discounts. A lot of the
products you’re using are probably going to be
running for weeks or months, and so we actually
factor in that pricing right here as well. Many of our solutions
also offer trials, and so I’m going to show you
that really quickly here. And so we’re showing
Matillion later as well. But you can see here’s a
Matillion solution offering a 14-day free trial. And what that means
is you, as a customer, only pay for the infrastructure
costs, so the VMs that are running this product
but not the license fee for that time period. It’s really a great
way to try out new products really risk-free. And if you decide you like it,
you can just continue running. You don’t need to
shut things down. You don’t need to
migrate your environment. You just continue
what you were doing. And if you decide it’s
not for you or you want to try something else,
you can just shut it down, and you won’t be charged
for those license fees. So another great benefit
of Google Cloud Platform marketplace. Another is the ability
to try a free test drive. So we kind of have two
kinds of trial modes. One is the trials I mentioned
earlier and another test drive. Test drives are a capability
that allow you, as a customer, to quickly see
the application up and running in an
environment, oftentimes with a preloaded
tutorial or sample data. These typically run for a
few days or sometimes even shorter periods of time. They’re really meant
to just give you a quick familiarity
with the solution and see if that’s right for you. So you can really start
your journey, maybe, with a test drive, understand
what the solution is about, talk to your team,
see, OK, maybe this is a good solution for
us, then kick up a trial– for this case 14 days, put
your data in, try it out. And if it’s going to be good for
you, you just continue running; if not, you spin down. So really, really
flexible model there. And with our
commercial solutions, again, on the pricing table,
we show you detail costings. So you can see here
you have the usage fee. You get a solution trial
credit for those first 14 days. We show you, again, the
basic instance infrastructure cost as well. So that covers virtual machines. Our Kubernetes solutions
are very similar. We also offer services. I’m going to show you that next. So SendGrid is a great email
API I’m sure many of you may be already using today. In this case, I’ve already
subscribed to a plan. But one thing you’ll
take away here is I’ve actually
subscribed to Send Grid, and I can actually
manage my subscription right from within the
Google Cloud Platform Marketplace itself. So I don’t need to go to Send
Grid site and log in again. I can change my
plans right here. So it’s really,
really integrated. Again, single bill
from Google to manage all of these services. I want to quickly jump
next to Kubernetes Engine and show you what
we’ve been doing there. So keeping on with
this theme of, how do we get applications
in context of the problem that you might be
having, we’ve actually done an integration
with Kubernetes Engine, where you can actually see
Kubernetes applications from right within the console. So here I am. I click Applications. I can deploy from
Cloud Marketplace. Again, I get a filter list. I only see Kubernetes
applications. We have both commercial
ones and open source. And I can go through
any of these and deploy. You can see the
similar details pages. And I can get up and running. So hopefully, that
gives you a good sense of the different kinds
of inventory we have, the different products. Again, you’re getting a
single bill from Google. Really easy to spin up, try
things, or shut things down, as the case might be. Can we switch back to slides? So another interesting benefit
of the Google Cloud Platform Marketplace is actually
helping you run open source. And we’ve actually made a
pretty substantial investment over the years in
this program we called Google Click to Deploy. Many of you may be using
these packages already, and we’ve made some
recent improvements. So I want to quickly talk
through what that looks like. The spirit behind the
Click to Deploy program is really accelerating your
use of open source in Cloud. It’s really that simple. The way we do that is we take
standard open source products, and we package them in ways that
you would have done yourself. So these are vanilla
applications. We intentionally
do things in a way where you can continue
following blog posts, stack overflow
tutorials, et cetera. We don’t tamper the packages. What we do do is make sure
that they are optimized and run well on GCP. So we kind of have that
trust that if you’re deploying Django,
Ruby, or any of these, you’re going to be in a known
good state from the start. Vanilla configurations,
as I mentioned. They continue to be
community-supported. And so you can follow
in any of the tutorials, blogs that you might
have out there, or work with any of the
vendors that offer support on top of open
source applications. Many of these solutions are
actually packaged in a way that you can actually run them
in production really well. So it’s one thing to run
a single-node LAMP stack, but it’s another two run a
seven-node cluster MongoDB instance. And if you’ve ever had to
configure something like that, you know that that takes hours,
sometimes days of your time, and getting it right
can be quite tricky. With our program here, you can
actually run a clustered Mongo, for example, in under
five minutes, so really, really easy to get up and
started with open source. We offer VM images. We offer base container
images you can use as well. So if you’re hosting in your
own container environment, you can just use those. We have Kubernetes applications
that we’ve added as well. So really big investment
from the open source side from Google. For the general GCP
Marketplace, we’re also focused on making sure
that it’s a great way for you to accelerate your prototyping. So we talked about
some of these benefits of deploying a few clicks. We have these smart
pre-configured settings, so you’re always going
to start in a good state. Each of our solutions
comes with really robust getting started
guides and tutorials. One of the most frustrating
experiences we all relate to is trying something out and
then just being stuck and not knowing what to do next. So that’s kind of the thing
with all of our solutions, whether it’s from Google or
from a third-party provider, really robust tutorials
getting started guides, making sure you don’t get stuck. Again, really low risk. We talked about trials. You can try solutions out. We have trials ranging
from 14 days to 30 days, depending on the vendor,
so really robust. You can actually apply
free trial credits. So if you’re new to
GCP or evaluating GCP, you have that budget of $300. You can actually use that
budget towards any solution on our marketplace, so it could
even be a commercial solution. And you can, in addition to
that, use free trial solutions while you’re on a GCP free
trial if that makes sense. So really, really
low risk way to trial the platform and
check out new apps. So it’s great for open source. It’s great for prototyping. But it’s also really
great for production. So I want to talk to you briefly
about how we think about that. So the journey just doesn’t
stop when you’re prototyping. You want to have these multi-VM
cluster configurations that you can depend on in production. And that’s what we’ve done,
whether it’s our open source solutions through the
Click to Deploy program or whether it’s through our
vendors, clustered applications that you can actually
take to production. We also offer a lot of flexible
pricing models, including BYOL. So if you have existing
relationships with the vendor, you can tap into those. Support integration–
one of the key things of going into production
is making sure that the vendor is going
to have your support. And all of our
marketplace vendors are contracted to offer
that support that they list on their details page. With Kubernetes, you actually– I’ve written it as one line. But actually, the
power of Kubernetes is quite strong here. With Kubernetes solutions,
you can actually take the commercial application,
bring it down, open it up, customize it, and redeploy it. Again, you still get
metered, still get billed. You get one single
bill from Google, and with the power
of Kubernetes, you get some of the additional
benefits, like auto-healing. So if one particular
instance or node goes down, it’ll quickly come back up
just because Kubernetes has that capability built for you. Another big focus
for us is making sure our solutions are secure. So we talked about the
verification pipeline that constantly runs
to make sure the latest vulnerabilities are detected. And we work very closely
with our vendors to make sure that the latest versions are
put up in the Marketplace as soon as possible. And in addition to all
of those, another thing you want to do in
production is make sure that you have a way to
tweak these configurations. You’re not just getting
stock things from Google and if you need to
break glass, you have to start from
scratch again. So I’ve talked a
little bit of how we do that on the Kubernetes side. We also do something very
similar on virtual machines. So all of our virtual
machine solutions are packaged using
Deployment Manager. How many of you are familiar
with Deployment Manager Today or are using it? OK. So Deployment Manager is
our kind of Cloud resource configuration language. It’s a declarative configuration
language where you can write up exactly what you
need to run, give it to an engine called
Deployment Manager, and it will bring up
that infrastructure. And what you should
be taking away from this with the marketplaces
that all of our VM solutions are packaged with
Deployment Manager. So once you deploy,
you can bring it down. You can open it up. You can tweak things. In this case, I’m showing
you a MongoDB instance where you can actually tweak the
zone or the number of replicas or number of arbiters or even
the sizes of the machines, and, again, just from
the command line, quickly redeploy that in a
matter of maybe a minute or two and be up and running, perhaps,
in a new availability zone. So really robust, great
for the CICD workloads. Really easy to deploy
and tear down as well. Just to summarize,
we talked about a lot of different inventory
today in our marketplace, and the list keeps growing,
which is really exciting. So we have virtual machines. We deploy on Compute Engine. We have base container images
where you can use, actually, in any environment. These are open source
packages maintained by Google that you can layer on top
of in your applications. We have Kubernetes applications,
both commercial and open source. And we offer a wide variety
of APIs and services as well. We also have a huge investment
in public data sets. So for those of
you using BigQuery, it’s a great way to get started. We have dozens of public
data sets from weather data to geospatial data
there for you to use. And as I mentioned earlier,
all of the GCP products are listed in the Marketplace. You have one-stop shop to find
any solution to a problem. In terms of management, many of
us solutions are self-managed. If it’s a case of
virtual machines, in Kubernetes applications,
and for our services, those are more of
a SaaS style, so when vendors are taking care
of the management for you. And when it comes to billing,
we’re a lot of flexibility here. We have free solutions,
like our open source ones. We have trials. We have test drives. Most of our solutions have
the integrated billing, so you get one single
bill from Google. Many of the services are built
from the partner directly. And we also offer BYOL,
so really, really flexible depending on your needs. What I want to
share with you next is a story of how a customer,
like perhaps yourselves, actually got in touch with
the Google Cloud Marketplace and explored it, found
a great solution, and are running
it in production. So I want to
introduce Iain first, from the Home Depot
QuoteCenter and, in a bit, Laura from Matillion. I’ll hand it over to you. IAIN KNIGHT: Thanks, Anil. ANIL DHAWAN: Yup. [APPLAUSE] IAIN KNIGHT: I’m Iain Knight. I’m a business intelligence
architect for QuoteCenter. I’ve been in IT
for about 18 years. I’ve worked in various roles– development engineering,
architecture, and leadership. I’ve worked in local
government, the retail web, and financial services sector. QuoteCenter was founded
about 10 years ago and acquired by Home
Depot four years ago. Our goal is to be the center of
excellence driving contractor sales, which is a
$600 billion market. We want Home Depot
to be the place of choice for contractors,
just as it is for consumers. To support that, our goal was
to migrate our existing BI environment, which
provided reporting and analytic capabilities. The current environment
had scalability issues. The code had become a
little bit unwieldy. We had a company
strategy to move to GCP. We looked to a
serverless architecture first and settled on
BigQuery as the platform. We tried to run
our existing tools, but it really wasn’t a good fit. Now, our team and our
current environment were built around
the Microsoft stack, so this was a big culture
and technology shift for us. So we needed a new approach. So we got the team in
a room, and we came up with some criteria
that made sense for us. Cost was definitely a concern. We wanted a tool that would
help transition our skills. We wanted something that would
integrate with source control. We had some system requirements,
like SQL, Salesforce and Elasticsearch. We knew we wanted to
transform that data and apply our business rules. We wanted a tool
that was scalable. We knew we needed
to schedule jobs, and we knew we wanted
monitoring capabilities. We also wanted a tool that
was hosted in Google Cloud. Another important
thing for us was that we had a tool that worked
in mobile environments– our dev, test, QA,
and production. We have about 50 years combined
experience in our team, and we did what any
self-respecting IT professional would do. We Googled BigQuery ETL tools. That took us to
the partner page, and there were a lot of our
friends in the partner page. We talked to some of them. We did some demos. We got some high-pressure
sales pitches. It wasn’t optimal. Another theme we saw
in the partner page was a lot of the offerings
were actually hosted on AWS, which, from
a security and performance perspective, wasn’t
really optimal for us. There were also other solutions
outside of the marketplace. But generally, they required
you to install and manage your own infrastructure–
again, not optimal. So one of the partners
we found was Matillion. One of the things we liked
was for $0.99 an hour, we could just spin
the instance up. We didn’t need to
talk to a sales rep. We didn’t need to negotiate
a proof of concept. So that’s what we did. We launched the instance. We poked around. It looked like a tool
that might meet our needs. The interface was
pretty familiar. The transformations
made sense to us. It was hosted in
Google Cloud Platform. So we reached out to Matillion,
and they worked with us. They gave us an introduction. They helped us as
we moved forward. We actually went back
to the Marketplace and launched beefier instance to
run our production environment on. One of the things
we really liked was we could make
use of the APIs and import all our code
and configuration straight from source, so
definite benefit there. So where are we now? With the combination of BigQuery
and Matillion, our team– [INAUDIBLE] Phil, and Vitale– have been able to load source
and model data into BigQuery. We’ve been following engineering
practices probably more commonly seen in app dev teams. One of our engineers,
Vitale, was able to automate our deployments
between environments. He integrated source control. He integrated Alert
in with Slack. We’re currently in the process
of building data quality checks within Matillion. We’re about 80% migrated
from our on-prem. And Home Depot teams
outside of QuoteCenter now solely use our
BigQuery implementation. We feel like we’re
much better positioned for the analytic demands
of a growing company, and we put ourselves
in a better position as we move more
towards a streaming architecture and real-time
analytic capabilities. A few takeaways– BigQuery is fundamentally
different from other platforms we’ve used, so
embrace its strengths. One of the things
we’ve done is working with Home Depot’s
implementation of BigQuery, we’re easily able to integrate
the work they’ve done and use their projects and
data sets rather than recreate in the same work on our side. We’ve been able to use
Cloud Launcher, or Google Marketplace, to go back
and launch new instances, as Matillion has
come back with– come out with updates. And we’ve been able
to test them to make sure there were no breaking
changes in the way we used Matillion. One of the things
taken advantage of the parametrization
in Matillion, we’re able to point to a
completely blank environment and deploy all our
staging table views permissions all straight
from source control. One final thing–
as we start to do things that weren’t out
of the box for Matillion, we learned to use the Python
scripting and the Google API to fill in the gaps. I’d like to hand it over
to Laura from Matillion. [APPLAUSE] LAURA MALINS: That’s it. Than you, Iain. IAIN KNIGHT: Yeah. LAURA MALINS: Thank
you very much, Iain. So hopefully, you
can hear me there. Thank you very much, Iain. So hi, everyone. My name is Laura Malins. And I’m a solution
architect here at Matillion, and today, I’m just
going to give you some of the
highlights of what is available within the
Matillion product. So I’ll just switch
across to my demo machine. And here, we can see the
Matillion ETL page for BigQuery on the GCP platform,
so available within the marketplace,
as Anil showed. And here, we’ve got our 14-day
free trial of the product. This is really popular
with our customers, so they can see how
the product will work within their own environment. You can also try out the
test drive of the product, so you can try the product in a
sandbox environment completely outside of your GCP project. So you can see what
the product does. And then when you
decide to launch, you just click
through on the Launch to Compute Engine button,
and it is very, very quick to launch on Matillion. And the really nice thing we
find about launching Matillion within the marketplace is
that you’ve got the ability to configure those
resources as you need it, so you can choose how many
CPUs you want in your Matillion instance. You can also choose how much
RAM to allocate to that. And then once the
instance is launched, it’ll literally spin up into
minutes, and spin up as a VM within your own GCP project. And now if I jump
across, you access that VM via the web
browser, and this is Matillion an ETL for BigQuery. And there are two main jobs
available within Matillion. There is a
transformation job, which is used for transforming all
your data within BigQuery, and there’s an
orchestration job, which is used to bring all
that data down into BigQuery. So Iain used an
orchestration job to reach out to his on-premise
databases, so his MS SQL databases, to connect
out to that database and, first of all,
move everything off the legacy systems. But also, you can
use it to bring data from incremental updates from
data source systems as well. One of the real
benefits of Matillion is it connects to a whole host
of different source systems. So on your screen now, you can
see an example orchestration jobs that we’ve got. And I’ll just talk you
through this briefly. So we’ve got four things
happening in parallel here. First of all, we’re creating
some blank staging tables first on BigQuery. We’re then using our Cloud
storage load component to connect into a flat
file, which is stored in a Cloud Storage bucket. And it will copy the
contents of that flat file down into BigQuery. In parallel, we’re checking
for an if condition, so we’re checking if some data
already exists within BigQuery. And then we’re loading some
data up from Google Analytics. We’re also loading out some
data from Microsoft SQL Server from our on-premise system. And we’re connecting
into Salesforce, so our Salesforce
instance in the Cloud, and we’re pulling some
accounts and opportunities data down here. And we’re saying if that
count load fails, actually that’s a key thing my whole job,
I want my whole job to fail. And once this is
loaded, we can then transform our data as
required within BigQuery and run that transformation
job down there, and all of this is very, very easy to
configure with Matillion. So I can see a list of
components available down here. I can expand out my
load/unload folder. And here, this will
give you a flavor of all of the
different data sources that Matillion
will connect into. So I want to add a new source in
here, say HopSpot, for example. I just drag it and
drop it, and it’s a very user-friendly interface. It’s all dragging and
clicking and dropping. And it presents this
visual view of Matillion. So I’ll just scroll
down and show you some more of the
sources available here. So I think at the last count,
we were up to about 60-odd data sources, but we’re
constantly receiving feedback from customers. And we’re always adding to
these sources here as well. And then finally, we’ve got
this API query component. So for a native
source for your data that Matillion doesn’t
have a connector for, you can use this API query
component to connect in, and Matillion can pull
up data from your API down into BigQuery. And these sources are
very easy to configure. You can simply click
into one of them, and it’s just a
selection of options. So in Salesforce, I can
choose my authentication via username password
or via [INAUDIBLE].. Then I pick the authentication
I’ve already got set up. And now what Matillion does is
it will connect into Salesforce as you as a user. So it takes a few
seconds to do that, because it’s
connecting in through. And what it will
do now is it will present a list of
all of the objects I’ve got available
for me in Salesforce that I’ve authenticated
it to access. So it’ll show that
list, and it will show all of the custom
objects within Salesforce now. So I think it’s taking a
couple of seconds to load that, because the Wi-Fi is quite slow. And then once you’ve configured
from that data source, you’ve picked down
that dropdown, you select which fields
you want to bring through from that source. And then, you simply give
the name of the new table that Matillion is going to
create for you in BigQuery to write that data through into. So here now, we can see the
list of all those data sources, and we can see that
Matillion will bring any custom one back for us. And then once I’ve got all the
data available within BigQuery, I can then transform it. So here’s my transformation job. I’ve got different
components because this is all about working
within BigQuery and really harnessing
the power of BigQuery to manipulate and transform
our data as we require. So then you can
feed that data out into your business
intelligence tool, or you can pass it on to a
data scientist, as Ian did. OK perfect, so I’ll
just hand back over to Anil for the last bit. [APPLAUSE] ANIL DHAWAN: And just to recap– hopefully what you’ve
taken away today is you can really accelerate
your journey in Cloud and your solutions development
using the GCP marketplace. You get that single bill
from Google for your licenses as well as your
infrastructure, making it really really easy to
procure and try things out. And hopefully, you’ve
also understood that we have solutions
across the board, whether it’s data sets,
containers, VMs, APIs, et cetera, so really,
really broad inventory, 800-plus solutions. Here’s a link,
cloud.google.com/marketplace. Go give it a spin. Check it out for a trial. If you’re a partner and you
want to participate and be part of our growing ecosystem,
got a link up here where you can actually get in touch
with our sales team and get you onboarded. And last but not
least, don’t forget to drop in your feedback in
the Google Cloud Next app about the session. [MUSIC PLAYING]

Comment here