Google Cloud Platform(GCP) Creating a Project and a Compute Engine Instance with a Startup Script Attached and Auto-scaling Process

Image for post
Image for post

Plan of the post:

Creating a Project

There are three ways of creating a project in GCP the first one from GCP web UI , the second one from Terminal and the third one is via API. We will examine web UI and terminal.

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Google Documentation
$gcloud projects create compute-engine-terminal
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Enable Compute Engine Service API

Before, enabling the Compute Engine Service API, I will be deliberately fired Error for you to see differences.

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Creating Instance

As always there are three ways to create a compute engine instance from terminal, web UI and via API. I will continue with Web UI after this point of post but you can try doing the same things by terminal or API. For example, you can create one from the terminal even though it needs a little bit configuration.

Image for post
Image for post
Terminal Command if you want to use the terminal
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Starting Instance with Startup Script

We have a Compute Engine instance but it does nothing and this is not satisfying for me. So I want to do some fancy staff and to use as much as a feature I can. I am going to create a startup script that runs when the instance starts every time also create a bucket to put my log files.

Creating Bucket

We select Storage from the side menu,

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Troubleshooting

After I deployed my script, I got “AccessDeniedException: 403 Insufficient Permission” error when instance tried to copy the log to the storage(bucket). It is self-explanatory but means that my VM instance doesn’t have the required permission.

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Creating a Startup Script

You can find detailed usage of the startup script at this link. Also, I am going to add all the sources that I use to create this startup script.

#! /bin/bash#
# -x : the xtrace option can be applied to a script by invoking Bash with the -x switch. This tells Bash to show us what each statement looks like after evaluation, just before it is executed.
# -v : oposite of -x, shows each line before it is evaluated instead of after. Options can be combined and by using both -xv
# GCE startup script output shows up in "/var/log/syslog".
#
set -x
#
# noninteractive : This is the anti-frontend. It never interacts with you at all, and makes the default answers be used for all questions.
#
export DEBIAN_FRONTEND=noninteractive
#
# up to date system
# -y : Automatic yes to prompts; assume "yes" as answer to all prompts and run non-interactively
# -q : Quiet. Produces output suitable for logging, omitting progress indicators.
#
apt-get -yq update
apt-get -yq upgrade
#
# Install Google's Stackdriver logging agent
# https://cloud.google.com/logging/docs/agent/installation
#
curl -sSO https://dl.google.com/cloudagents/install-logging-agent.sh
bash install-logging-agent.sh
#
# Install "stress" tool
# Run the stress command to spawn 8 workers spinning on sqrt() with a timeout of 120 seconds
#
apt-get -yq install stress
stress -cpu 8 -timeout 120
#
# Report that we're done.
#
# Metadata should be set in the "logs-bucket" attribute using the "gs://mybucketname/" format.
log_bucket_metadata_name=logs-bucket
log_bucket_metadata_url="http://metadata.google.internal/computeMetadata/v1/instance/attributes/${log_bucket_metadata_name}"
worker_log_bucket=$(curl -H "Metadata-Flavor: Google" "${log_bucket_metadata_url}")
# We write a file named after this machine.
worker_log_file="machine-$(hostname)-finished.txt"
echo "Eureka :) Work completed at $(date)" >"${worker_log_file}"
# And we copy that file to the bucket specified in the metadata.
echo "Copying the log file to the bucket..."
gsutil cp "${worker_log_file}" "${worker_log_bucket}"
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Outcome of Startup Script

1- Making available system logs in Stackdriver Logs

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Last Touch: Providing the Startup Script via Cloud Storage

We have to make a couple of changes to get and run the startup script. First, we have to update Cloud API access scope of VM instance and give Read/Write permission to the Storage

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Auto Scaling Compute Engine Instance

Creating Instance Template and an Instance from Template

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Creating Managed Instance Groups (MIG) and Setting Up Auto-Scaling

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Resources:

Junior Full-Stack Developer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store