How Do You Use Google’s Cloud Pub/Sub with Python?

Problem scenario
You want to use a Data Analytics or a Big Data tool that publishes messages and subscribes to listening to messages being published. You know GCP has a Pub/Sub tool. You know it supports synchronous and asynchronous messaging. How do you use it with Python?

Solution

1. Log into GCP via the web UI.
2. Go here: https://console.cloud.google.com/cloudpubsub/
3. Click "Create Topic".
4. Enter a name and click "Create".
5. In a different browser tab, go here:
   https://console.cloud.google.com/projectselector/appengine/create?lang=flex_python&st=true&_ga=2.217969685.-287673474.1542719370
6. Click "Select." Choose an existing project. If there is none, click "Create" instead of "Select".
7. This assumes that you have set up the App Engine and in cloud shell cloned a repository. If you need assistance, see this posting How do you use GCP's App Engine?
8. From the cloud shell run this command:

cd python-docs-samples/appengine/flexible/pubsub

9.a. Draft these commands, but substitute contint_topic with your topic name, contint_subscription with your subscription name, PROJECT_ID with the project ID*, contint_token123 with a password like "token" of your choice:

gcloud pubsub topics create contint_topic
gcloud pubsub subscriptions create contint_subscription \
     --topic contint_topic \
     --push-endpoint \
     https://PROJECT_ID.appspot.com/pubsub/push?token=contint_token123 \
     --ack-deadline 10

9.b. FYI: The push endpoint will use the above to verify requests.

10.a. Modify the app.yaml in this directory. (If it is not there, find this file: appengine/flexible/pubsub/app.yaml)
10.b. Edit it such that it looks like this (with contint_topic and contint_token123 being references to the placeholder values in the previous step):

env_variables:
     PUBSUB_TOPIC: contint_topic
     # This token is used to verify that requests originate from your
     # application. It can be any sufficiently random string.
     PUBSUB_VERIFICATION_TOKEN: contint_token123

11. Run these commands:

pip install protobuf google-cloud-bigquery google-cloud-pubsub
export GOOGLE_CLOUD_PROJECT=[your-project-id]
export PUBSUB_VERIFICATION_TOKEN=[your-verification-token]
export PUBSUB_TOPIC=[your-topic]
python main.py

12. In the upper right-hand corner of the GCP console there should be an icon that when you hover over it, a message is displayed "Web preview". Click this icon, then choose "Preview on port 8080".

13. A new tab in the web browser should appear. You should see an opportunity to enter text in the website. Type in some text and click "Submit Query". Go back to the cloud shell. You should see some lines of activity for this "POST" and "GET" activity.

14. You are now done.

*If you want to find the ID of your project, from the GCP web UI, go to "My First Project" or something similar at the top middle part of the screen. Click on it. The pop up should show the project IDs according to their names.