Let’s learn about Google Cloud Shell and how it can help you as a developer to get more productive working with Google Cloud Applications.

Cloud Shell (https://lnkd.in/dcS444yp) is an interactive shell environment for Google Cloud that makes it easy for you to learn and experiment with Google Cloud and manage your projects and resources from your web browser.

How much space does it give you and is it linked to a specific GCP Project ?
Cloud Shell provisions 5 GB of free persistent disk storage mounted as your $HOME directory on the virtual machine instance. This storage is on a per-user basis and is available across projects.

What kind of tools does it come pre-installed with?
For a start, we have multiple editors, Google Cloud Tools (gcloud, gsutil), Build and Package Tools and more. For the entire list, check out: https://lnkd.in/dteQhK5N

What kind of Programming Language SDK support can I get in Google Cloud Shell?
Check out the list of languages that are supported. https://lnkd.in/deX7wZHi

Does it come with a Visual Editor?
Yes. Google Cloud Shell comes with an editor, based on Eclipse Thiea. You can launch it from within Cloud Shell or even directly via https://lnkd.in/dBWfMufS. It is integrated with Cloud Code (https://lnkd.in/drN_mm95), that has integrated support for k8s, Cloud Run apps and exploring various Google APIs. Looking to learn about using the Cloud Shell Editor? Check out this playlist: https://lnkd.in/dVPpcuQ2

How much does it cost?
No additional cost for Google Cloud customers.

Can I upload and download files?
Yes. Check out https://lnkd.in/dZSw3sf8

Can I locally run web applications in Cloud Shell and preview/test them?
Absolutely. Check out https://lnkd.in/ds-MvM9u

I have never used Cloud Shell? Whats the quickest way to get started?
Cloud Shell 101: https://lnkd.in/d5c9cYhS. Once done with that, explore our other Quickstarts: https://lnkd.in/dBfRW3Q9

Any other cool features?

1. Open in Cloud Shell: This allows you to publish a link that opens the Google Cloud Console with a Git repository cloned into Cloud Shell. Check out : https://lnkd.in/dpBPb3n9

2. Interactive Tutorials : Write interactive tutorials that takes a user step by step while still remaining within Cloud Shell and GCP console?
Check this https://lnkd.in/dZrVfk6c

3. Activate Google Cloud Shell within Documentation :
Now you can run code samples within documentation inside of Cloud Shell with single click. Read more: https://lnkd.in/dywbUS_h

What is/are your favourite feature(s)?

Google Cloud Shell — Resources to get started was originally published in Romin Irani’s Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

• Part 1 : Sustained Use Discounts and Committed Use Discounts
• Part 2 : Custom Machine Types
• Part 3 : Preemptible VMs

In Part 1 , we covered On-demand pricing , where you are charged for the time that you keep the Virtual Machine instances running. We also covered the scenario, that in development and testing environments, one of the recommendations is not to keep the instances running all the time. You can save a lot of money by identifying the block of time that you typically use these VMs and then keeping them in a Stopped state during other days.

An example scenario could be that you would like all your Development machines to be started at 9:00 AM and stopped at 5:00 PM everyday. This will result in you getting billed for just 8 hours a day instead of 24 hours a day. This will result in significant savings just in on-demand pricing for that VM. Just for numbers, cost of running a n1-standard-1 instance for 24 hrs / day and all days a week v/s running it for 8 hours a day and all days a week is shown below. You can see the cost of approx. $29 v/s $13 / month.

Google Cloud recently announced the ability to schedule start and stopping your Compute Engine instances in GA and this article will show how we can do the same.

What is an Instance Schedule?

To get started, we need to first understand the concept of an Instance Schedule. The Instance Schedule is meta information about when you want to start an instance, stop an instance, in which compute engine region it can be applied to, the time zone for the start/stop time and whether you want apply it daily/weekly/monthly with various combinations. Think of Cron expressions that specific when it can be applied.

Having said that, a sample Instance Schedule might have the following definition:

Name + Description : <Sample Name + Description>
Compute Engine Region : asia-south1 (Mumbai) 
Start Time : 9:00 AM
End Time : 5:00 PM
Time Zone : IST (UTC + 5:30)
Initiation Date (Optional) : Leave it empty
End Date (Optional) : Leave it empty
Frequency : Daily / Weekly / Monthly (Depending on that you can select days of the week and days of the month)

So, if we go back to our plan of having an Instance Schedule, where we want our machines to be running only from 9:00 AM to 5:00 PM on all days of the week (daily) and apply it to various instances in Mumbai region, I would go with the following values:

Name + Description : MyDevMachine-Instance-Schedule
Compute Engine Region : asia-south1 
Start Time : 9:00 AM
End Time : 5:00 PM
Time Zone : IST (UTC + 5:30)
Frequency : Daily

That’s it. Before we get to the mechanics of setting this up, I suggest to read up on the limitations, a few of which I reiterate below:

• An instance schedule is specific to a region and you can only attach it to instances in that region.
• Each VM instance can be associate with only one instance schedule.
• The instance schedule needs a window of about 15 minutes to the time that you have specified to complete the start and stop of the instance. In my sample runs, it took around 10 minutes after the stop time for example, for the instance to actually stop. So you might want to factor that but do it with care since you cannot predict it precisely.

Instance Schedules in Action

To see the instance schedules work in action, we are going to go with our original requirement i.e. start the VM instances at 9:00 AM and stop them at 5:00 PM, on all days of the week.

Given that, I went ahead and created two Compute Engine VMs — nothing fancy here, just go with the n1-standard-1 or smaller VMs. In my case, I created them in the region = asia-south1.

If you are familiar with gcloud, you can use the following commands to create a couple of VM instances.

gcloud compute instances create instance-1 --machine-type=n1-standard-1 --zone=asia-south1-a

gcloud compute instances create instance-2 --machine-type=n1-standard-1 --zone=asia-south1-a

If you visit the Google Cloud Console and Compute Engine → VM Instances, you should see the two of them listed as shown below:

If you notice, you see a tab INSTANCE SCHEDULE next to INSTANCES in the above screenshot. Simply click on that.

At this point, we don’t have any instance schedules. Remember, all of the stuff that I am showing can be done via the gcloud utility or even the API, but I will use the Cloud Console here.

Let’s create our instance schedule now. Click on CREATE INSTANCE SCHEDULE and you will see the following screen. Go ahead and populate it as I have shown below:

Click on SUBMIT button and the schedule gets created.

The instance schedule is not yet associated (attached) to any Compute Engine VM Instances. So let’s apply it to both our VMs (instance-1 and instance-2).

Click on the instance schedule record i.e. (dev-start-stop-schedule under the name column). You should see the screen below. It contains the definition of our instance schedule (dev-start-stop-schedule) and you can see that there are no attached instances yet.

Click on the ADD INSTANCES TO SCHEDULE link shown above and you will see the list of VMs that can be attached.

Select the VMs (instance-1 and instance-2 in our case) and click on the ADD button. You see that the instances are now attached to the instance schedule and it is in effect (if you get an error about service account permission, see the next section).

Service Account Permission

Keep in mind that when we create an instance schedule and attach it to the Compute Engine instances, Compute Engine behind the scenes will need to have the permission to perform a start instance and stop instance operation on those compute resources (VM instances : instance-1 and instance-2).

For that to happen, you must ensure that the Google Managed Compute Engine System Service account (not your default Compute Engine service account) has the permissions to start and stop the instance.

The Google Managed Compute Engine System Service Account is of the form:

service-PROJECT_NUMBER@compute-system.iam.gserviceaccount.com

This account is not visible in the list of service accounts when you go to that section in the IAM or Service Accounts section. We will get to that in a while, but first up, we will create a Custom Role : Instance-Start-Stop-Role , which only has permissions to start and stop Compute Engine instances.

Go to IAM & Admin → IAM section in the Google Cloud Console and click on CREATE ROLE at the top as shown below:

This will bring up a form, where you can fill it up with the Custom Role name (in my case, it is Instance-Start-Stop-Role):

Click on ADD PERMISSIONS. This will bring up the following screen , where you can filter by role by typing in Compute Instance Admin and selecting Compute Instance Admin (v1) as shown below:

Then from the list of permissions, filter it first by compute.instances.start and then click on ADD.

Similarly, filter it again via compute.instances.stop and click on ADD.

Ensure that you have both these permissions and then create the role.

This will create a Custom Role. Now, all we need to do is assign this role to the Google Managed Compute Engine System Service Account. Let’s do that.

Visit IAM and Admin → IAM. Select the checkbox for Include Google-provided role grants as shown below.

This will show the system accounts too and you should look out for the service account of the form:

service-PROJECT_NUMBER@compute-system.iam.gserviceaccount.com

We can now edit this role by clicking on the Edit Pencil icon at the end. This will bring up the current list of roles assigned to this service account. Click on ADD ANOTHER ROLE and select our Custom Role that we just defined i.e. Instance-Start-Stop-Role. Click on Save. You will be brought back to the list of Accounts, just ensure that the new custom role is visible in addition to other roles for the service account.

Note : In case you got the permission error while attaching the instance schedule to the instance, go back now and try the same again. You should be able to move attach the same successfully.

That is pretty much what you have to do to schedule your instances to start and stop as per your pattern.

Monitoring the Start / Stop Schedule in Action

You can use Cloud Logging to view the start and stop schedule in Action. I created an instance schedule of just 30 minutes to test the same in action i.e. I defined the instance schedule to start at 12:00 PM and stop at 12:30 PM.

To view the same, I visited Cloud Logging → Logs Explore as shown below:

Next up, click on Query builder. Then do the following:

• For the Resource dropdown, select VM Instances from the Resource Type as shown below:

• For Log name, we need to go to the Cloud Audit Log and Activity as shown below:

Now run the log query again , and you see a lot more specific logs. I went back to the logs and noticed that the stop command was given as shown below:

You can see that the stop command was given a good 10 minutes after my instance schedule specified time i.e. 12:30 PM. So as I earlier mentioned and which the documentation states clearly, we should factor a 15 minute window after the start / stop time for the action to be performed.

Hope this was useful. Do note that we also have available in beta, the ability to suspend/resume a VM.

Hands-on Guide to Scheduling VM Instances to Start and Stop was originally published in Romin Irani’s Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

In the previous part, we took a look at Custom Machine Types in Google Compute Engine and how you could provision a VM as per your specific vCPU / RAM requirements and save on the cost.

In this article, we are going to look at how you can save a lot more of the list price of a VM via what are know as Preemptible Virtual Machines. We will also understand more importantly, where you can ideally use this type of a VM.

What is a Preemptible Virtual Machine?

I would like to present the definition via two points:

• This is an instance that you can get at a much lower price.
• This is an instance that can be stopped if Compute Engine requires the infrastructure for some other tasks.

Having defined the above, it means that these instances which could be stopped any time , at-least once across a 24 hour period, should not be used for any kind of workload. You cannot use this instance for a customer facing website or a critical workload that is not designed to survive an instance stopping, etc.

As a result, it is best suited for batch processing or fault tolerant jobs, where the Preemptible Virtual Machines (PVMs) are part of a cluster. Think of Big Data clusters, where the frameworks are fault-tolerant , can distribute their workloads across instances and can tolerate a few instances going up and down. It can be well suited to a Managed Instance Group too.

I won’t go too much into the finer aspects of how you will be notified if a PVM is going to be preempted, how to write a shutdown script to handle that for some work to be done before the VM terminates, etc. You should read up on all those details over here. Google Cloud has also published a set of best practices to reduce your chances of getting preempted, interesting … isn’t it?

GCP Calculator for PVM Pricing

Let’s continue with the example that we have been using so far for provisioning a n1-standard-1 instance type (1 vCPU and 3.75 GB RAM). Launch the GCP Calculator https://cloud.google.com/products/calculator and use the following:

# instances : 1
What for? : Server-1
OS : Free
Series : N1
Machine Type : n1-standard-1
Region : Mumbai

This comes to our standard pricing that we saw earlier as shown below:

Remember that I have chosen 24 hours per day and 7 days a week as running time for this instance.

Now, let’s see the cost if we go for a Preemptible version of the above. Go with the same settings, except that in the GCP Calculator, you will see a field called “Machine Class”. Click on that dropdown and select Preemptible instead of Regular.

With all the other settings remaining the same, the pricing comes to:

See the big difference in cost? Use this knowledge and the fact that if you have fault tolerant workloads that are run across cluster of VMs, you could save tons of $$$ using PVMs. Just make sure that you understand the tradeoffs that are mentioned in the documentation.

Conclusion

I would like to highlight that there are a lot more things to consider from a total cost perspective, when provisioning a Virtual Machine and not just list price. My goal was to help you understand the list prices via the Google Cost Calculator and for you to better understand the following terms:

• Sustained Use Discounts
• Committed Use Discounts
• Custom Machine Types
• Preemptible Virtual Machines

Hope this series was useful to you. Look forward to any feedback and some lessons that you have learnt while juggling with these options.

A working guide to understanding Google Compute Engine Pricing Options — Part 3 was originally published in Romin Irani’s Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

In the previous part, we took a look at understanding the basics of Sustained Use Discounts (SUD) and Committed Use Discounts (CUD) and how they are related to pricing a Virtual Machine in Google Compute Engine.

In this article, we are going to take a look at one of the biggest differentiators in Google Cloud Platform : Custom Virtual Machine Types.

Before we get to that, let us continue with the Virtual Machine instance type that we used in the previous article i.e. n1-standard-1 , which as per the documentation gives us 1 vCPU and 3.75 GB RAM.

Google Compute Engine gives us a list of standard machine types, which come with a fixed vCPU and RAM combination and you have a variety of them to choose from. Let’s look at the n1-standard series and the different fixed machine types that we can choose from:

This choice is good since your workloads might have varied requirements in terms of the vCPU and RAM that they need.

We need a higher configuration !!!

Now consider a scenario that you have been happily chugging along with our n1-standard-1 configuration and we are hitting limits in terms of performance. You are told to upgrade to more vCPU and RAM. The first thing that you might end up doing is to look at the list above and start matching the machine type that corresponds to the vCPU and Memory that you want. You have 2 options here:

• If you find a perfect match, you pick the machine type and we are done.
• If you don’t find the perfect match, you are likely to choose the next higher configuration.

Let’s concentrate on the 2nd option. You end up choosing the next higher configuration, which means that you have made a compromise and gone for a larger capacity in terms of vCPU and RAM. Nothing wrong with it but the first hit when you do that is going to be the cost ($$$) of running that instance.

We are talking about just one VM over here, but think about a medium sized deployment of 10s of VMs and you are likely to end up in this scenario a lot more than you expected, thereby over provisioning a lot more additional vCPU and RAM, and therefore paying a lot more.

Enter Custom Machine Types

Luckily , there is a 3rd option available in Google Cloud Platform and that is Custom Machine Types. What if you could settle for something in the middle in terms of the vCPU and RAM that you need. What this means is that you come as close as possible to what you need and you end up paying less too. You pay less, because logically the configuration you want is now somewhere between the original configuration and the next higher configuration.

There will certain ratios that you need to maintain in terms of vCPU and RAM for Custom Machine configurations and you should check out the documentation for the same.

Let’s work out the details for our example. We have the following scenario:

• We had our original VM i.e. n1-standard-1 (1 vCPU and 3.75 GB RAM)
• We want to upgrade to a configuration where we need 6 vCPUs and 24 GB RAM.

Question: Is the 6 vCPU and 24 GB RAM configuration available among the standard n1-standard-* machine types, as per the list below:

It’s not an exact match and you might be tempted to go to the n1-standard-8 configuration for a start. Nothing wrong in that, but let’s see what happens when we use the GCP Calculator.

Visit the GCP Calculator at https://cloud.google.com/products/calculator and use the following:

First, let’s look at our original pricing for n1-standard-1 for 24 hours a day and 7 days a week, which we did in the earlier article as shown below:

# instances : 1
What for? : Server-1
OS : Free
Series : N1
Machine Type : n1-standard-1
Region : Mumbai

We get the following pricing as shown below:

Next, let’s look at the pricing for a n1-standard-8 configuration as shown below:

# instances : 1
What for? : N1-Server2
OS : Free
Series : N1
Machine Type : n1-standard-8
Region : Mumbai

We get the following pricing:

This is all fine but we would like to now use the option of custom machine types and see what it will cost us for provisioning exactly what we need:

Once again in the GCP Cost Calculator, enter the following to get the pricing for our custom configuration as per the specifications below:

# instances : 1
What for? : N1-CustomServer
OS : Free
Series : N1
Machine Type : <<< How do we select this >>> ??? 
Region : Mumbai

So, how do we select a custom machine type in the GCP Cost Calculator. In the form, click on the dropdown list for Machine Type and you will see Custom Machine Type right at the top, as shown below:

On selecting that, you will see a slider for selecting the vCPU and RAM combination as shown below and we will select the values:

Once you add this to the estimate, you will see the pricing as shown below:

You can see how the pricing dropped from nearly $230/month to $178/month. That is a lot of savings and this is true cloud pricing in terms of provisioning exactly what you want and getting charged for that.

Additional considerations

Like with everything else, it can’t be that simple, isn’t it ? There are additional things that you need to be aware about:

• There are certain minimum RAM per VCPU expected in different machine types.
• The RAM that you can allocate needs to be a multiple of a certain amount i.e. 256MB.
• Each machine type will be constrained in terms of max vCPUs and RAM supported.
• There is a max vCPU to RAM ratio that can be supported across different machines. So if you need additional RAM , you will need to provision Extended Memory and some caveats are there to that too in terms of you not being able to make use of Committed Use Discounts on Extended Memory.

Whew ! Don’t worry , just familiarize yourself with the documentation.

This concludes the part on Custom Machine Types , a true differentiator available in Google Cloud. Next up, let us discuss Preemptible Virtual Machines.

A working guide to understanding Google Compute Engine Pricing Options — Part 2 was originally published in Romin Irani’s Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is series of articles that I plan to write to help understand the various pricing levers that you have in Google Cloud Platform’s core compute offering : Google Compute Engine. I plan to explain this vis easy to understand and practical examples with the help of the Google Cloud Cost Calculator, that is available publicly.

Let’s get going.

Google Compute Engine provides a core infrastructure component : Virtual Machines (VM). These VMs are classified into different families

• General purpose
• Memory Optimized
• Compute Optimized

Each family has various machine types. How do you select which machine type, what kind of workloads are they best suited for? You can use this guide as a start for the same:https://cloud.google.com/compute/docs/machine-types and I produce a screenshot from there that covers the various workloads that are best suited for which machine types.

Each of the machine types (e.g. N1 general purpose) will have several *fixed* or *pre-defined* configurations (instance types) of VPU and RAM. You can choose the appropriate instance types depending on your requirements.

A set of instance types for N1 (General purpose) machine types is shown below:

An important thing to consider when you choose a machine type is whether it is available in the GCP Region (actually zone) since VMs are zonal resources.

Check out the list of regions that GCP is available in and more importantly, the availability of machine types per region. Bookmark this since this can change. https://cloud.google.com/compute/docs/regions-zones

VM is a rented resource and hence you are charged for the time that you use the Virtual VM. Let’s qualify that. You are charged for the amount of time that you keep a VM in running state. If a VM is stopped, you are not charged for it. https://cloud.google.com/compute/vm-instance-pricing

Is there any minimum charge and then charged by second/minute/hour — whats the granularity by which you are charged? I reproduce this from the official documentation linked above:

• All vCPUs, GPUs, and GB of memory are charged a minimum of 1 minute. For example, if you run your virtual machine for 30 seconds, you will be billed for 1 minute of usage.
• After 1 minute, instances are charged in 1 second increments.

Each VM instance type has a pricing associated with it i.e. per hour pricing. As you consider higher vCPU and more RAM, the pricing inches upwards. Take a look at pricing for N1 instance types (https://cloud.google.com/compute/vm-instance-pricing#n1_predefined).

Remember* the prices are not the same in each region. When we get to using the Google Cloud Calculator in a while, ensure that you choose the correct region for the pricing to avoid any surprised.

In the screenshot below, you will find that there is a dropdown for the region that you can select and see the prices.

The pricing is classified into:

• On-demand price
• Preemptible price
• 1 year commitment price
• 3-year commitment price

This can be confusing but let’s understand it with an example that we shall work through with the help of the GCP Cost Calculator. We will also throw in some other variations.

Our Running Example

First up, we will finalize that we are interested in provisioning a Compute Engine VM instance type (n1-standard-1), which has the following configuration:

vCPU     : 1
RAM (GB) : 3.75 GB
Region   : Mumbai
OS       : Free (Linux)

Remember that I mentioned that GCP charges you for duration that you keep the VM running. So you run it 24 x 7 i.e. 7 days a week and for 24 hours a day i.e. you keep the VM running. GCP charges you by the rate per hour for that instance type.

But do consider that not everything needs to run 24 x 7. For e.g. consider organizations that have a development , test and production environments. Sure the production environments run 24 x 7 but my development / test environment might only run for 5 days a week (Mon-Fri) and 8 hours a day (9:00AM — 5:00 PM).

So I should be charged less right ? Correct. Let’s go step by step.

First up, we will use the GCP Cost Calculator available at (https://cloud.google.com/products/calculator) and see what it costs to run this VM.

Visit the URL and enter the values as:

# instances : 1
OS : Free
Series : N1
Machine Type : n1-standard-1
Region : Mumbai

Ensure that you select 24 hours per day and 7 days per week as shown below. Click on “Add to Estimate”

This will show the pricing to you on the right side:

Note 3 things here first:

• The pricing is for the full month : 730 hours per month i.e. your VM is in running state throughout the month.
• You can change the currency to one that you want. This is very useful to share with clients that would like to be billed or want to see what it costs for them in their currency of choice.
• You can email this estimate or even click on Save Estimate to get a URL to share and keep collaborating on the pricing.

Notice one of the entries in the billing details “Sustained Use Discount : 30%” . What is going on here? Google Cloud provides an automatic discount (upto a certain %) off the list price, when you use the VM for more time , in our case the full duration of the month.

The more you use, the more discount you get. Check this graph out.

If you use it for just 15 days a month, we still give you a % discount. This is Sustained Use Discount (SUD) and it is covered here: https://cloud.google.com/compute/docs/sustained-use-discounts

That’s great , isn’t it ? No need to do complex calculations — we pass on the discount as per your usage.

What about Developer and Test Environments?

Next up, remember the dev and test environments? What if you wanted to run this for the following:

• 5 days a week (Monday — Friday)
• 8 hours a day

Let’s click on the Edit button for the pricing as shown below. This will populate the entries in the form of the GCP Calculator, where you can then tweak any settings.

Now go ahead and change the values as shown below:

• 8 hours per day
• 5 days per week

And click on “Add to Estimate” again.

This will bring up the revised pricing as shown below:

That’s a nice discount , isnt it ? So always plan out your different environments (Production, Development , Testing) and how long the VMs need to be running.

Don’t pay for resources that are idle and you are not using. Best Practice : https://cloud.google.com/compute/docs/instances/schedule-instance-start-stop

What are Committed Use Discounts (CUD)?

Now, what let’s come to Committed Use Discounts (CUD). If you are willing to commit to running these instances all the time for a period of 1 yr or 3 yr, Google Cloud can give additional discounts. Read about CUD here (https://cloud.google.com/compute/docs/instances/signing-up-committed-use-discounts )

Let’s see how to do that in the calculator. Let’s edit the previous entry back to running it 24x7 , so that the total is shown as below. This time, we will also label this line item as : SUD Server.

Next up, add another VM resource with the following:

# instances : 1
What for : CUD-1yr
OS : Free
Series : N1
Machine Type : n1-standard-1
Region : Mumbai
Committed Use Discount : 1 yr

Click on “Add to Estimate”

Next up, add another VM resource with the following:

# instances : 1
What for : CUD-3yr
OS : Free
Series : N1
Machine Type : n1-standard-1
Region : Mumbai
Committed Use Discount : 3 yr

Click on “Add to Estimate”

This will bring up the following pricing and you can see how it is beneficial to go for CUD pricing, if you plan to commit to GCP.

Hope this was useful.

I plan to cover more some nuances next on Compute Engine pricing vis-a-vis Custom Machine Types and Preemptible VMs in another thread.

Till then, Happy Pricing !

A working guide to understanding Google Compute Engine Pricing Options — Part 1 was originally published in Romin Irani’s Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Dialogflow is a great tool for designing and building out Agents that can run in a variety of Messaging Platforms. In my interactions with customers, a frequent request that comes up is about support for multiple languages. Not just that but complex scenarios like “Can an Agent support more than 1 language in a single utterance?”, etc.

The objective of this blog post is to show you step by step how you can start designing an Agent in Dialogflow that can support two languages. It will cover just about enough to give you a start and you can build upon it.

Pre-requisites

I shall assume that you know how to use Dialogflow to build out an Agent and are familiar with the environment and key building blocks like Intents, Entities, etc.

I am going to use English and Hindi as two languages that the Agent understands, so I will expect you to make sense of the Hindi that you see in this blog post.

What are we going to build?

We are going to build an Agent for a Hotel chain that lets you book a room in one of the cities that it has resorts in. We are not going to make things complex here since the idea is to show you the multilingual feature.

Here is a sample interaction with my Agent in Hindi:

And here is the same interaction in English:

Let’s get started

Log in to the Dialogflow flow console with your account and click on Create New Agent, as shown below:

This will bring up a form as shown below. Provide a name to your Agent, go with all other defaults, including English as the default language and click on Create. Let’s assume that the name of the Agent for this tutorial is MyMultilingual Agent.

You will see screenshots with the Agent name as Romin-MultilingualBot but for your reading, you can assume the name is MyMultilingual Agent

On successful creation, you will have the standard two intents : Default Fallback Intent and Default Welcome Intent created for you.

Go to the Settings page for the Agent and then to the Languages. You will only see English — en listed, which is the default language that we had selected at the time of creation of the Agent. Click on Select Additional Language and then select Hindi from the list.

If you are upto it, pick another language that you are familiar with instead of Hindi and follow along accordingly.

You should see something similar to the screen shown below. Also notice , how on the left-pane, you now see two chips indicating the two languages that the Agent now supports i.e. en and hi. Click on Save button.

Important → Every time that you have to work with a specific language, make sure to select the language chip from the Top Left. This way you know which language you are in and for which, you are now configuring Intents, Entities and so on.

Support for English

For the default English language, we are not going to do anything special. If you are familiar with the default Intents (Welcome and Fallback) that are generated, we are good. We are not going to modify anything over there.

Ensure that you have select the English language en for the Agent. Then let’s do a couple of things for this Agent.

• Add a resort-location Entity

We don’t want users to select any city in which this Hotel chain has resorts in. We would like to restrict it only to cities in which they have resorts. To do this, we are going to define a Custom Entity named resort-location and restrict the values over there.

Go ahead to Entities and create a Custom Entity resort-location as shown below and click on Save.

• Add a new Intent Book Holiday Intent

Click on Intents and then add a New Intent named Book Holiday Intent. In the sample phrases give the following and make sure that the custom entity @resort-location is detected currently for the parameter.

Go to Actions and Parameters section and mark the resort-location parameter as required.

If the user does not provide us with the required parameter, we need to prompt them for the value. For that, I set the Prompt text for this variable (last column in the above table) as follows:

Finally, provide a simple Text Response that the Intent gives out as follows:

Click Save and let the training get completed.

There we go ! We have our Agent ready and you can test it out in the Test console on the right. Remember that you have already selected the English Language (en) for the Agent, hence the Test console will assume an interaction in English language.

Give it a spin as shown below:

We are looking good now for our English support. Let’s figure out Hindi language support now.

Support for Hindi

First thing that you need to do now (I am sure you know what that is…) is to switch to Hindi language. From just below the Agent name, switch to Hindi by choosing the language chip as shown below:

Now, visit the Intents section and you will see all the 3 intents that we have so far:

• Default Welcome Intent
• Default Fallback Intent
• Book Holiday Intent

Click on either the Welcome or the Fallback Intent and you will notice that Dialogflow has provided us with sample phrases in the language for both the intent. Wonderful, isn’t it? Sample screenshots for the two intents are given below:

and the Fallback Intent is given below:

This means that if you would like to test these two intents right away in the language, go ahead and test them out in the console.

Let’s get back to the two things that we need to modify for Hindi language:

• resort-location entity
• Book Holiday Intent

First, up visit the Entities section. You will see that the resort-location entity is present but it will not have any values in Hindi language. So go ahead and enter the Hindi specific values as given below and remember to click the Save button.

Next up, visit the Intents and click on Book Holiday Intent. The values will once again be empty, but let’s fill them up in the right sequence as given below.

First up, provide the sample phrases for booking a room as given below. You can use your own nuances too in Hindi (and please excuse my language skills).

See how it is able to map the city value to the resort-location entity.

Next up, in the Actions and Parameters section, we need to provide a prompt for our parameter in Hindi language. Click on prompt for the resort-location parameter and provide some text. An example is shown below:

Finally, let’s provide the text response too as shown below:

Remember to Save. That’s it. We can now go and test the same in the console. Ensure that the language selected is still Hindi. A sample interaction in the Test Console is shown below:

and another interaction:

Testing the Integration

You can even test it out in the Integrations → Dialogflow Messenger , which is in Beta:

It will show you a snippet of code that you can insert into your web site or you can try it out there itself. For those of you with sharp eyes , you will notice that the language is indicated in the iframe snippet.

Click on TRY IT NOW button and experience your Agent.

Hope you liked the tutorial. Let a thousand multilingual Agents bloom !

Writing in a Multilingual Agent in Dialogflow was originally published in Romin Irani’s Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is part of a Google Cloud Functions Tutorial Series. Check out the series for all the articles.

By now, you will be familiar with creating Cloud Functions (foreground and background) and associating the appropriate trigger i.e. HTTP Trigger, PubSub Trigger and others. We usually find that we have a need to trigger the Cloud Function at regular intervals.

Consider the following Cloud Functions that you might have written:

1. A function that checks a Stock Price every 10 minutes and sends you an email.
2. A function that runs once a day at 3:00 AM to begin the process of taking a backup.

You could even have complex scheduling like run a Cloud Function every Monday, Wednesday and Friday at 10:00 AM.

In all the above situations, you have the following requirements:

1. A Service that will trigger your Cloud Function.
2. A Service that can accept simple to complex scheduling intervals. For e.g every NN minutes, every NN hours, every week on Friday at 10:00 PM and more. In other words, support a CRON like expression that we know from Unix world.

Cloud Scheduler

The Cloud Scheduler now available in Beta helps us out in the above requirements.

As per the official documentation, Cloud Scheduler is an enterprise grade job scheduler that will help you automate your jobs across various Google Services in a standard fashion and comes with retry mechanisms that you can configure. It can also trigger your jobs in a variety of ways and currently supports invoking a HTTP endpoint, send a message to Cloud Pub/Sub and trigger an internal App Engine URL.

In our example here, we will be looking at triggering a sample Cloud Function every minute. Let’s go.

Before we begin, a few assumptions:

1. You have an existing Google Cloud Platform project with Cloud Functions API enabled.
2. You have a sample Cloud Function ready with a HTTP Trigger associated with it. Let us assume that this Cloud Function is called function-1. Note down the HTTPs Trigger URL of the Cloud Function.
3. You need to enable the Cloud Scheduler API for your Google Cloud Platform project. From GCP console, visit APIs and search for Cloud Scheduler API as shown below:

Click on Cloud Scheduler API and click on Enable:

Once enabled, you might see a message that says that you will need to have credentials in order to use the API. That is fine for now, since we plan to do everything via the GCP console and Cloud Functions deployed inside the same GCP Project. The API credentials would be required in case you were making any requests to this API from external Apps.

This completes our initial setup.

Using the Cloud Scheduler

We have completed the initial setup and have our Cloud Function function-1 ready. All we need to do now is to create a job in the Cloud Scheduler. All the details like the schedule, what to trigger and other parameters will be configured at the time of Job creation itself and are associated with the Job.

From the GCP Console main menu, visit the Cloud Scheduler page:

Click on the Cloud Scheduler option. Since we do not have any jobs yet, we will see the message as shown below:

Click on Create job. This will prompt you to select a region in which you would like to initialise the Cloud Scheduler. Select the appropriate region. You should see a message as shown below:

Once it is initialised, you will be led to a form to create a new job and you can provide the details as shown below:

The key things to note in the Job details form are:

• Name : Give a unique name to your job.
• Frequency : Provide the cron expression. Since we want to trigger the function every minute, I use : * * * * *
• Timezone : You can select a specific timezone in which to execute the function. The Cloud Scheduler will use that.
• Target : As mentioned earlier, the Cloud Scheduler currently supports App Engine, HTTP and Pub/Sub. We will select HTTP i.e. ask the Cloud Scheduler to invoke a HTTP endpoint that we will specify next. This HTTP endpoint will be that of our HTTP Trigger based Cloud Function.
• URL : Provide the HTTPs endpoint of the Cloud Function i.e. function-1.
• HTTP method : This applies to HTTP invocations. You can choose from one of the HTTP methods like POST, GET, etc. In our case, we will use the POST method.

Click on Create button. This will create the Job and you will see it listed in the list of jobs. You can either wait for the execution to happen (look at the Result column) since it is every 1 minute or if you want to invoke it directly click on Run now.

Once it has executed, you will see the status getting updated as shown below (only the specific columns are shown):

Click on View button to see the logs. Here is the sample log statemens from one run of the job:

Note that we could also have a Pub/Sub Trigger configured Cloud Function get triggered by the Scheduler by using the same steps given above to create a job. Except that in the target, you will select Pub/Sub and provide the Pub/Sub Topic name. At the scheduler trigger time, the Cloud Scheduler will publish a message to the topic with the message body that you specify.

This completes our note on triggering a Cloud Function using the just released Cloud Scheduler service.

Google Cloud Functions Tutorial : Using the Cloud Scheduler to trigger your functions was originally published in Romin Irani’s Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is a book review of The Fuzzy and the Techie : Why the Liberal Arts will Rule the Digital World by Scott Hartley.

This book caught my attention a few months back since I am firmly of the belief that to create great products, it is not just about having technical prowess but a lot of other things and I specifically wanted to understand the role of liberal arts. It is this intersection of streams that is of deep interest to me.

The book is firm in its theme and gives numerous examples of how folks with either just a liberal arts background, or a combination of humanities + tech have been making significant contributions to digital products. This is not just from an ideating perspective but even working with technical folks to bring their ideas to reality.

The book is a good read. It gave me numerous examples of companies that either I had heard about or had no clue about. The latter was more interesting since I often found myself looking them up on the internet for precisely what they are doing. It was interesting to understand the background of the founders and honestly I have been humbled since I was not really sure if people with a very diverse background or completely unrelated background could go so deep and build a cutting edge or futuristic digital product. There are examples throughout the book and enough food for thought in multiple places.

What I would have liked to see more in the book would have been a deeper discussion into how specific diverse backgrounds played a role. I would have loved to hear more from the folks themselves. I do understand that a book might not be an effective format for that kind of discussion but I did feel like I wanted to understand the founder a bit more.

The book takes up multiple themes ranging from creating better learning outcomes, making technology work for humans, designing responsible products, the job skills needed for the future and more. Each of these themes can be a book in itself but it made me better understand that to effectively solve these problems would require a very different mindset and most likely a different stream that they took up instead of just engineering or medicine. I appreciated the fact that the author concluded that in the end , it is a partnership between the Techies and Fuzzies that will yield the best results.

The book is also a gentle reminder to parents, including myself (especially within the context of where I live … India) that we have to let go of current mentalities that push our kids only into Engineering and Medical streams. This book has been educating in multiple ways for me.

I give this book 3.5/5 stars.

Book Review : The Fuzzy and the Techie was originally published in Romin Irani’s Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is part of a Google Cloud Functions Tutorial Series. Check out the series for all the articles.

At the time of writing this article, support for Environment variables in Google Cloud Functions is in Beta.

As you start work your way through Cloud Functions, one of things that you will notice as you write them is that you often end up accessing other systems or define some constants, etc. You might want to connect to an external system for which you need to have a hostname, service name, etc.

The questions to ask are:

1. Is it possible to define a Cloud Function and associate one or more Environment variables with it at the time of deployment?
2. Will the Cloud Functions runtime make it available to our function when it runs it?
3. Can we read these Environment variables directly in our code?
4. Can we use the standard gcloud functions command or the GCP Console to manage the Environment variables?

The answer to all of the above questions is yes ! Enter Environment Variables in Cloud Functions.

Using process.env

Let us take a look at the existing environment variables that are already present. This is not something you want to do too much about but it comes in handy to understand which of these are reserved environment variables and used in some way or the other by Google Cloud Functions runtime.

exports.helloEnvVariables = (req, res) => {
 res.status(200).send(process.env);
};

If you deploy the above Cloud Function, you should get an output that looks like this. It gives you an interesting amount of information.

{
 “X_GOOGLE_CODE_LOCATION”: “/user_code”,
 “WORKER_PORT”: “8091”,
 “X_GOOGLE_SUPERVISOR_INTERNAL_PORT”: “8081”,
 “X_GOOGLE_WORKER_PORT”: “8091”,
 “FUNCTION_IDENTITY”: “mygcpproject@appspot.gserviceaccount.com”,
 “X_GOOGLE_LOAD_ON_START”: “false”,
 “X_GOOGLE_FUNCTION_REGION”: “us-central1”,
 “GCLOUD_PROJECT”: “mygcpproject”,
 “FUNCTION_NAME”: “myEnvVarFunction”,
 “X_GOOGLE_FUNCTION_MEMORY_MB”: “128”,
 “SUPERVISOR_HOSTNAME”: “169.254.8.129”,
 “X_GOOGLE_GCLOUD_PROJECT”: “mygcpproject”,
 “PATH”: “/usr/local/bin:/usr/bin:/bin”,
 “FUNCTION_REGION”: “us-central1”,
 “PWD”: “/user_code”,
 “FUNCTION_TRIGGER_TYPE”: “HTTP_TRIGGER”,
 “FUNCTION_TIMEOUT_SEC”: “60”,
 “X_GOOGLE_FUNCTION_TRIGGER_TYPE”: “HTTP_TRIGGER”,
 “NODE_ENV”: “production”,
 “SHLVL”: “1”,
 “X_GOOGLE_FUNCTION_NAME”: “myEnvVarFunction”,
 “X_GOOGLE_ENTRY_POINT”: “helloVersion”,
 “X_GOOGLE_FUNCTION_IDENTITY”: “mygcpproject@appspot.gserviceaccount.com”,
 “DEBIAN_FRONTEND”: “noninteractive”,
 “CODE_LOCATION”: “/user_code”,
 “X_GOOGLE_GCP_PROJECT”: “mygcpproject”,
 “FUNCTION_MEMORY_MB”: “128”,
 “GCP_PROJECT”: “mygcpproject”,
 “X_GOOGLE_SUPERVISOR_HOSTNAME”: “169.254.8.129”,
 “PORT”: “8080”,
 “SUPERVISOR_INTERNAL_PORT”: “8081”,
 “X_GOOGLE_FUNCTION_TIMEOUT_SEC”: “60”,
 “ENTRY_POINT”: “helloVersion”,
 “OLDPWD”: “/var/tmp/worker/”,
 “_”: “/usr/bin/env”,
 “HOME”: “/tmp”
}

Specifying your own Environment Variables

To associate Environment Variables with your Cloud Function, you can do it in multiple ways:

1. Use the GCP Console to define them at the time of creating/updating a Google Cloud Function
2. Use gcloud functions command to set and update them individually, a list of them or via a configuration file.

In this tutorial, we will use the GCP Console, but if you would like to do the same via the gcloud functions command, you can read up the documentation here: Using environment variables.

Let us go ahead and create a Google Cloud Function via the GCP Console. I am assuming that you know that by now via this series and are able to bring up the Create Function form on your own.

If you scroll right down, you will find a More link. Click that and the Environment Variables section should show up as shown below:

Click on the Add variable button to add variables as you want. We will define four variables as shown below:

To use any specific environment variable value in your Google Cloud Function, you will have to use the process.env statement as shown below:

exports.helloEnvVariables = (req, res) => {
  let minVer = process.env.MIN_VER;
  let maxVer = process.env.MAX_VER;
  let maxResults = process.env.MAX_RESULTS;
  let hostName = process.env.HOSTNAME;
  res.status(200).send("Function version:" + maxVer + "." + minVer);
};

If you deploy and run this function, it will send back the following response text:

Function version:1.2

A few points around using Environment Variables

Using Environment Variables in Google Cloud Functions is straight forward but there are a few things to keep in mind:

1. The size of the variable names and values is limited to 32KiB. Additionally, there are reserved names for Environment variables and you saw that in the first example that we ran.
2. Once you have deployed a Google Cloud Function with Environment variables or you are associated Environment variables for the first time for an existing Google Cloud Function, you cannot just apply any changes to Environment variables without deploying the function. In other words, any change to Environment variables requires a full function deploy.
3. Extending point (2), keep in mind that when you deploy a new version of the function with updated Environment variables, the changes may not be immediately available on the next invocation of your function. I strongly recommend this talk by Jason Polites (Twitter: @jplts ). If you are working with Google Cloud Functions — do spend time with this

4. The scope of the Environment variables is limited to your specific Cloud function. You cannot share it across multiple functions.

5. Environment variables are not the place to put sensitive data. So do not keep credentials (usernames, passwords, keys) or anything that you do not want others to get a hold off. Take a look at this article by Seth Vargo (Twitter: @sethvargo) on managing secrets in serverless.

Secrets in Serverless | Seth Vargo

Here is an excellent article on using process.env and specifically I would recommend the section “When to use it” to understand common use cases that are a good fit for Environment variables.

process.env: What it is and why/when/how to use it effectively

6. In case you are providing Environment variables to your Cloud Functions via a configuration file at the time of gcloud functions deploy , do remember to add that file potentially to your .cloudignore and .gitignore files.

Reference

Official Documentation on Environment Variables → https://cloud.google.com/functions/docs/env-var#using_environment_variables

Google Cloud Functions Tutorial : Using Environment variables was originally published in Romin Irani’s Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is part of a Google Cloud Functions Tutorial Series. Check out the series for all the articles.

In the series so far, we have deployed our source code directly. What I mean by that is that we either had the source code present in a local folder and then we used gcloud to deploy the function or we used the Cloud Console Web UI to paste our source code directly. If you have taken a closer look, you would have noticed that one of the options available for deploying your Function is to use a Source Code Repository like Git to host your function code and deploy it directly from there.

The ability to host our Source code in a Git repository and use that as a basis to deploy our function code has multiple benefits. It fits with a typical development workflow that we use today. Additionally, you can maintain versions of your function code and also have automatic builds done for you via the source code checkin trigger. Let’s learn the mechanics of how to do this now.

First up, if you visit the Cloud Console Web UI for creating a Cloud Function, you can see the option for Source Code Repository as shown below (only relevant sections from the Create Function form ):

Google Cloud Source Repositories

Did you know that each Google Cloud Platform project comes with a Git repository ? This is available in the Cloud Console main menu via the Source Repositories option shown below:

What we shall be doing is the following:

1. As mentioned earlier, all the code in this series is available at Github at https://github.com/rominirani/googlecloudfunctions-training. So we already have a Git repository created for us over there. I am assuming that you have cloned this repository under our own Github account. This is important and do not miss this step.
2. We are going to initialize a repository in our GCP Project and connect it to the Github repository that we have created in Step 1. All code that you push / update to this Github repository will be kept in sync by the Google Cloud Source Repository.
3. At the time of deploying our function, we can refer to the Google Cloud Source Repository, specify the branch and the folder in which the specific function source is present and the deployment then happens automatically for us.

Let’s do it.

Google Cloud Source Repositories has been revamped significantly and a Beta version is available, at the time of writing this blog post. I shall use the Beta version and in case you see a message asking you to use the Beta version, please do so.

The Beta version available at https://source.cloud.google.com/ looks like this:

If you click on View All Repositories, you will see all your GCP Projects and the associated repositories for the same. We shall go ahead and create a new respository. To do that, click on the New repository button in the top right corner.

You have a couple of options as shown below and I am choosing the 2nd option since I would like to connect this repository to an already initialized and available repository that I have for this series at Github.

Click on Continue. This will bring up a screen below where you should select your existing GCP Project and choose the provider as Github. Give the appropriate consent and complete the formalities of connecting to Github.

Once you click on Connect to Github, you will need to authorize Google Cloud Platform to access your Github account and repositories that you have under it and select the particular repository on Github to connect to. Please do that. You should see a message similar to the following:

If all goes well, your Github repository will be available under Source Repositories as shown below:

We are able to see our README.md file and that is a good thing ! :-)

This completes our section of connecting an existing Github repository to Cloud Source Repositories. You can continue to use your standard Git workflow to checkin files into Github repository and the same will get mirrored into the Cloud Source Repositories. The synchronisation is done for you.

The important thing to note from the above screen is the repository name and the branch name. An example is shown below. This should also help you understand that a GCP Project can have one or more repositories (a default one and multiple others like these as shown below).

You can click on the Repository and Branch drop down arrows to view the details. What I particularly liked was that you could look at various branches, tags, commits and everything that has happened in Github directly from Cloud Source Repositories.

Deploying our function from Cloud Source Repository

We are going to deploy a HTTP Trigger based function that is present in the helloworld-http folder of our repository as shown below:

Few things to note here:

1. Repository Name : github_rominirani_googlecloudfunctions-training
2. Branch/Tag : Branch
3. Branch Name : master
4. Folder in which the index.js file is present : helloworld-http
5. Function to execute : helloGET

Once ,we have this information, we can visit the GCP Console for New Function and provide the information as given above and click on Create Function. Your function should deploy successfully and ready to be invoked!

This is part of a Google Cloud Functions Tutorial Series. Check out the series for all the articles.

Google Cloud Functions Tutorial : Deploying from a Source Repository was originally published in Romin Irani’s Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.