Distributed applications rely on Virtual Private Clouds (VPCs) to increase security and reduce latencies. Traffic is routed through the VPC, rather than the public internet, eliminating the need for any network hops along the way.
In my continued development of The Largest River, I've chosen to deploy my application instances inside of a Google Cloud VPC.
To access these resources in the cloud, both in development and production environments, there are multiple considerations. How will I connect to my YugabyteDB clusters? And how will I connect from my local machine to remote resources in the VPC?
The following outlines my solutions to these problems.
YugabyteDB Deployment in a VPC
YugabyteDB Managed relies on VPC Peering, to keep network traffic within the cloud provider's network and to establish connectivity between nodes from different cloud regions.
This is an optional configuration in single-region clusters, but for multi-region deployments, it is mandatory. I’ve configured two multi-region clusters for this application, so VPC peering is required.
For this to work, I also need to set up VPC network peering in the Google Cloud.
That was easy!
Now our application nodes and database nodes have a peered network connection inside of Google Cloud. This also means that all of our database connections must come from machines inside of this network.
What does this mean for local development?
Setting up SSH Tunneling
This was uncharted territory for me. I needed to find a way to establish database connections to various multi-node clusters from my development machine, in order to iterate quickly.
Initially, I thought about writing application code directly on a VM using remote SSH in Visual Studio Code. This was problematic, in that I would still need to push and update the code on all other application instances. I needed a solution which didn’t require continuously deploying code to remote servers within the VPC for testing.
Here is where I discovered the power of SSH tunneling (also known as port forwarding). By tunneling connections from my local machine through a server in the VPC, I'm able to establish a database connection to each and every database node relying on this peered VPC connection.
Here's an example:
# development machine command line
ssh -N -i /path/to/gcp/ssh/key -L 5000:fake-database-node-url.gcp.ybdb.io:5433 vm_username@[VM_IP_ADDRESS]
This command forwards localhost:5000 to fake-database-node-url.gcp.ybdb.io:5433, through a VM inside of the VPC, to which I'm able to establish and SSH connection.
Despite being incredibly simple to set up, this discovery has been instrumental in my development journey. By utilizing port forwarding, my Node.js application development environment has remained unchanged.
When the NODE_ENV environment variable is set to development, the app knows to establish connections through a particular local port for each database node. When the environment variable is set to production, the application knows requests should be made directly to the database URL. It just works, and it does so securely.
Much the same, SSH tunneling can be used within a database client. Here's what this looks like inside of my client of choice, DBeaver.
Wrapping Up
For those working as IT administrators, devops, or network engineers, etc., this might seem like a rudimentary discovery!
SSH tunneling has a wide range of use cases and I'm sure I've just scratched the surface. However, nothing feels better than finding a simple, frictionless solution to a blocking issue you've never faced before.
Without SSH tunneling, I was essentially stuck, doomed only to deploy my code to an instance within the VPC in order to test its validity. I will not be apologizing for my excitement at this time. 😎
We’re approaching the finish line, stay tuned!