Constructing a Video Sport Recommender System with FastAPI, PostgreSQL, and Render: Half 2

Getting Began

In half 1 we walked by the method of establishing a board recreation suggestion system leveraging FastAPI and PostgreSQL. In Half 2 we proceed this undertaking and present find out how to deploy this undertaking to a cloud service, on this case Render, to make it accessible to customers.

To make this a actuality, we’ll be engaged on establishing our PostgreSQL database on Render, populating it with our knowledge, Dockerizing our FastAPI software, and eventually deploying it to a Render Internet Software.

Desk of Contents

1. Deploying a PostgreSQL database on Render
2. Deploying a FastAPI app as a Render Internet Software
   – Dockerizing our software
   – Pushing Docker Picture to DockerHub
   – Pulling from DockerHub to Render

Tooling Used

• Render
• Docker Desktop
• Docker Hub

Deploying on Render

Now we have now a PostgreSQL database and a FastAPI software that work domestically, and it’s time to deploy on a cloud service that may be accessed by a front-end software or finish consumer (by way of Swagger). For this undertaking, we’ll use Render; Render is a cloud platform that, for small tasks, provides a extra simple setup expertise than bigger cloud suppliers like AWS and Azure.

To get began, navigate to Render and create a brand new account, then you may create a brand new undertaking by deciding on the ‘New Venture’ button proven beneath. Be aware, as of the time of this writing, Render has a trial interval that ought to will let you observe alongside at zero price for the primary month. We’re calling this undertaking fastapi-test, we then navigate into that undertaking after it’s created.

Every undertaking incorporates all the things required for that undertaking to work in a self-contained surroundings. On this case, we’d like two parts: a database and an online server for our FastAPI software. Let’s begin with creating our Database.

That is quite simple, we choose ‘Create New Service’ as proven in Determine 3 after which choose ‘Postgres’. We’re then navigated to the sphere proven in Determine 4 to arrange the database. We title our database “fastapi-database” and choose the free tier to get began. Render solely lets you use the free tier database for a restricted time, however will probably be advantageous for this instance, and in case you wanted to take care of a database long run, the pricing may be very affordable.

After inputting our database info and deciding on ‘Create’ it is going to take a minute to arrange the database, and also you’ll then be introduced with the display proven in Determine 5. We’ll save the Inside Database URL + Exterior Database URL variables in our .env file, as we’ll want these to attach from our FastAPI software. We will then take a look at our connection to the database utilizing the Exterior Database URL variable(connecting from our native machine is outdoors the Render Setting) and create the tables from our native machine earlier than transferring on to establishing our FastAPI software.

We then run our take a look at database connection script, which makes an attempt to hook up with our database by utilizing the External_Database_Url variable because the connection string and create a take a look at desk. Be aware that our External_Database_Url is our full connection string for the database, so we will cross this as our single enter. A profitable run ought to lead to a printout as proven in Determine 6.

from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker, Session
from sqlalchemy.ext.declarative import declarative_base
import os
from dotenv import load_dotenv
from utils.db_handler import DatabaseHandler
import pandas as pd
import uuid
import sys
from sqlalchemy.exc import OperationalError
import psycopg2

# Load surroundings variables from .env file (override=True reloads modified values)
load_dotenv(override=True)
# loaidng exterior database URL
database_url = os.environ.get("External_Database_Url")
if not database_url:
    print("❌ External_Database_Url not present in surroundings variables")
    print("Please verify your .env file incorporates: External_Database_Url=your_render_postgres_url")
    sys.exit(1)
print(f"Database URL loaded: {database_url[:50]}...")
# Parse the database URL to extract parts for testing
from urllib.parse import urlparse
import socket
def parse_database_url(url):
    """Parse database URL to extract connection parts"""
    parsed = urlparse(url)
    return {
        'host': parsed.hostname,
        'port': parsed.port or 5432,
        'database': parsed.path.lstrip('/'),
        'username': parsed.username,
        'password': parsed.password
    }
db_params = parse_database_url(database_url)
def test_network_connectivity():
    """Take a look at community connectivity to Render PostgreSQL endpoint"""
    print("n=== Community Connectivity Exams ===")
    # 1. Take a look at DNS decision
    strive:
        ip_address = socket.gethostbyname(db_params['host'])
        print(f"✅ DNS Decision profitable")
    besides socket.gaierror as e:
        print(f"❌ DNS Decision failed: {e}")
        return False
    
    # 2. Take a look at port connectivity
    strive:
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        sock.settimeout(10)  # 10 second timeout
        end result = sock.connect_ex((db_params['host'], int(db_params['port'])))
        sock.shut()
        
        if end result == 0:
            print(f"✅ Port {db_params['port']} is accessible")
            return True
        else:
            print(f"❌ Port {db_params['port']} is NOT accessible")
            print("   This may point out a community connectivity challenge")
            return False
    besides Exception as e:
        print(f"❌ Port connectivity take a look at failed: {e}")
        return False
# Run connectivity exams
network_ok = test_network_connectivity()
if not network_ok:
    print("n🔍 TROUBLESHOOTING STEPS:")
    print("1. Verify your web connection")
    print("2. Confirm the Render PostgreSQL URL is right")
    print("3. Guarantee your Render PostgreSQL occasion is energetic")
    print("4. Verify if there are any Render service outages")
    sys.exit(1)
print("n=== Trying Database Connection ===")
# connect with the database utilizing psycopg2
strive:
    conn = psycopg2.join(
            host=db_params['host'],
            database=db_params['database'],
            consumer=db_params['username'],
            password=db_params['password'],
            port=db_params['port'],
            connect_timeout=30  # 30 second timeout
        )
    
    # If the connection is profitable, you may carry out database operations
    cursor = conn.cursor()
    
    # Instance: Execute a easy question
    cursor.execute("SELECT model();")
    db_version = cursor.fetchone()
    print(f"✅ PostgreSQL Database Model: {db_version[0]}")
    
    # Take a look at making a easy desk to confirm permissions
    cursor.execute("CREATE TABLE IF NOT EXISTS connection_test (id SERIAL PRIMARY KEY, test_time TIMESTAMP DEFAULT NOW());")
    conn.commit()
    print("✅ Database permissions verified - can create tables")
    
    cursor.shut()
    conn.shut()
    print("✅ psycopg2 connection profitable!")
    
besides psycopg2.OperationalError as e:
    print(f"❌ Database connection failed: {e}")
    if "timeout" in str(e).decrease():
        print("n🔍 TIMEOUT TROUBLESHOOTING:")
        print("- Verify your web connection")
        print("- Confirm the Render PostgreSQL URL is right")
        print("- Verify if Render service is experiencing points")
    elif "authentication" in str(e).decrease():
        print("n🔍 AUTHENTICATION TROUBLESHOOTING:")
        print("- Confirm the database URL incorporates right credentials")
        print("- Verify in case your Render PostgreSQL service is energetic")
        print("- Make sure the database URL hasn't expired or modified")
    sys.exit(1)
besides Exception as e:
    print(f"❌ Surprising error: {e}")
    sys.exit(1)
# If we get right here, connection was profitable, so exit the take a look at
print(f"n✅ All exams handed! Render PostgreSQL connection is working.")
print(f"✅ Related to database: {db_params['database']}")
print("✅ Prepared to be used in your software!")

Loading Database

Now that we’ve verified that we will connect with our database from our native machine, it’s time to arrange our database tables and populate them. To load our database, we’ll use our src/load_database.py file, which we beforehand walked by the person items of this script initially of this text, so we gained’t go into additional element on it right here. The one notable factors are that we’re once more utilizing our External_Database_Url as our connection string, after which on the finish, we’re utilizing the test_table perform that we’ve outlined as a part of our DatabaseHandler class. This perform makes an attempt to hook up with the desk title handed to it and returns the variety of rows in that desk.

Working this script ought to lead to an output as proven in Determine 11, the place every of the tables was created, after which on the finish we recheck that we will return knowledge from them and present that the output rows match the enter rows.

from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker, Session
from sqlalchemy.ext.declarative import declarative_base
import os
from dotenv import load_dotenv
from utils.db_handler import DatabaseHandler
import pandas as pd
import uuid
import sys
from sqlalchemy.exc import OperationalError
import psycopg2

# Load surroundings variables from .env file
load_dotenv(override=True)

# Assemble PostgreSQL connection URL for Render
URL_database = os.environ.get("External_Database_Url")

# Initialize DatabaseHandler with the constructed URL
engine = DatabaseHandler(URL_database)

# loading preliminary consumer knowledge
users_df = pd.read_csv("Information/steam_users.csv")
games_df = pd.read_csv("Information/steam_games.csv")
user_games_df = pd.read_csv("Information/steam_user_games.csv")
user_recommendations_df = pd.read_csv("Information/user_recommendations.csv")
game_tags_df = pd.read_csv("Information/steam_game_tags.csv")


# Defining queries to create tables
user_table_creation_query = """CREATE TABLE IF NOT EXISTS customers (
    id UUID PRIMARY KEY,
    username VARCHAR(255) UNIQUE NOT NULL,
    password VARCHAR(255) NOT NULL,
    e-mail VARCHAR(255) NOT NULL,
    position VARCHAR(50) NOT NULL
    )
    """
game_table_creation_query = """CREATE TABLE IF NOT EXISTS video games (
    id UUID PRIMARY KEY,
    appid VARCHAR(255) UNIQUE NOT NULL,
    title VARCHAR(255) NOT NULL,
    kind VARCHAR(255),
    is_free BOOLEAN DEFAULT FALSE,
    short_description TEXT,
    detailed_description TEXT,
    builders VARCHAR(255),
    publishers VARCHAR(255),
    value VARCHAR(255),
    genres VARCHAR(255),
    classes VARCHAR(255),
    release_date VARCHAR(255),
    platforms TEXT,
    metacritic_score FLOAT,
    suggestions INTEGER
    )
    """

user_games_query = """CREATE TABLE IF NOT EXISTS user_games (
    id UUID PRIMARY KEY,
    username VARCHAR(255) NOT NULL,
    appid VARCHAR(255) NOT NULL,
    shelf VARCHAR(50) DEFAULT 'Wish_List',
    score FLOAT DEFAULT 0.0,
    assessment TEXT
    )
    """
recommendation_table_creation_query = """CREATE TABLE IF NOT EXISTS user_recommendations (
    id UUID PRIMARY KEY,
    username VARCHAR(255),
    appid VARCHAR(255),
    similarity FLOAT
    )
    """

game_tags_creation_query = """CREATE TABLE IF NOT EXISTS game_tags (
    id UUID PRIMARY KEY,
    appid VARCHAR(255) NOT NULL,
    class VARCHAR(255) NOT NULL
    )
    """

# Working queries to create tables
engine.delete_table('user_recommendations')
engine.delete_table('user_games')
engine.delete_table('game_tags')
engine.delete_table('video games')
engine.delete_table('customers')

# Create tables
engine.create_table(user_table_creation_query)
engine.create_table(game_table_creation_query)
engine.create_table(user_games_query)
engine.create_table(recommendation_table_creation_query)
engine.create_table(game_tags_creation_query)

# Guaranteeing every row of every dataframe has a novel ID
if 'id' not in users_df.columns:
    users_df['id'] = [str(uuid.uuid4()) for _ in range(len(users_df))]
if 'id' not in games_df.columns:
    games_df['id'] = [str(uuid.uuid4()) for _ in range(len(games_df))]
if 'id' not in user_games_df.columns:
    user_games_df['id'] = [str(uuid.uuid4()) for _ in range(len(user_games_df))]
if 'id' not in user_recommendations_df.columns:
    user_recommendations_df['id'] = [str(uuid.uuid4()) for _ in range(len(user_recommendations_df))]
if 'id' not in game_tags_df.columns:
    game_tags_df['id'] = [str(uuid.uuid4()) for _ in range(len(game_tags_df))]

# Populates the 4 tables with knowledge from the dataframes
engine.populate_table_dynamic(users_df, 'customers')
engine.populate_table_dynamic(games_df, 'video games')
engine.populate_table_dynamic(user_games_df, 'user_games')
engine.populate_table_dynamic(user_recommendations_df, 'user_recommendations')
engine.populate_table_dynamic(game_tags_df, 'game_tags')

# Testing if the tables have been created and populated accurately
print(engine.test_table('customers'))
print(engine.test_table('video games'))
print(engine.test_table('user_games'))
print(engine.test_table('user_recommendations'))
print(engine.test_table('game_tags'))

Deploying a FastAPI Software on Render

We now have the primary half of our undertaking deployed on render, and it’s time to arrange our FastAPI software. To do that, we’re going to make use of Render’s Internet Software internet hosting service, which is able to permit us to deploy our FastAPI App as an online software that may be accessed by exterior providers. If we needed to construct a full-stack software, we may then permit our entrance finish to ship requests to the FastAPI software on Render and return knowledge to the consumer. Nonetheless, as a result of we’re not fascinated with constructing a front-end element presently, we’ll as an alternative work together with our App by the Swagger docs.

Containerizing our Software with Docker

We’ve arrange our FastAPI undertaking in an area surroundings, however now we have to switch it, with all of the code, dependencies, and environmental variables, to a container on Render. This could possibly be a frightening problem. Fortuitously, Docker handles all of the difficult items and permits us to do exactly that with a easy configuration file and a few instructions. For many who haven’t used Docker, there’s a nice tutorial right here. The transient overview is that Docker is a instrument that simplifies the method of deploying and managing purposes by permitting us to bundle our software with all its dependencies as a picture after which deploy that picture to a service like Render. On this undertaking, we use DockerHub as our picture repository, which serves as a central version-controlled storage space for our picture, which we will then pull into Render.

Our general stream for this undertaking will be considered like this FastAPI app operating domestically → A ‘Snapshot’ is taken with Docker and saved as a Docker Picture → That Picture is pushed to DockerHub → Render pulls this picture and makes use of it to spin up a Container that runs the applying on a Render Server. Getting began with this course of, which we’ll stroll by subsequent, requires having Docker Desktop put in. Docker has a simple set up course of which you may get began on right here: https://www.docker.com/merchandise/docker-desktop/

Moreover, in case you don’t have one already, you’ll want a Docker Hub account as it will function the repository to avoid wasting Docker Photographs to after which Pull them into Render. You may create a Docker Hub right here: https://hub.docker.com/.

Constructing a Docker Picture

To create a Docker Picture for our undertaking, first ensure Docker Desktop is operating; if it isn’t, you’ll doubtless get an error when attempting to create a Docker picture. To make sure it’s operating, open the Docker Desktop software out of your search bar or desktop, click on on the three dots within the backside left as proven beneath, and make sure you see the Inexperienced dot adopted by ‘Docker Desktop is operating’.

Subsequent, we have to inform Docker find out how to construct our picture, which is finished by defining a Dockerfile. Our Dockerfile will be seen in Determine 9. We reserve it in our top-level listing, and it supplies the directions that inform Docker find out how to bundle our software into a picture that may be deployed on a unique piece of {hardware}. Let’s stroll by this file to know what it’s doing.

1. FROM: Selecting Base Picture: The primary line in our Dockerfile specifies what base picture we need to use to then lengthen for our software. On this case, we’re utilizing the python:3.13-slim-bullseye picture, which is a light-weight Debian-based picture that can function the bottom for our software.
2. WORKDIR: Altering Work Listing: Right here we’re setting the default listing inside our container to /app
3. RUN: Checking for updates to system dependencies
4. COPY: Coping necessities.txt file, it’s essential that necessities.txt is updated and incorporates all libraries required for the undertaking, or the Picture gained’t run accurately after we attempt to spin it up
5. RUN: Putting in our necessities.txt file
6. COPY: Copy our total undertaking from our native listing to /app, which we created in step 2
7. RUN: Making a logs listing at /app/logs
8. EXPOSE: Doc that the port we’ll be exposing is port 8000
9. ENV: Units our Python path to /app
10. CMD: Runs our FastAPI app utilizing Uvicorn, units our app to the one outlined in src.essential:app, runs our app on port 8000

With our Dockerfile outlined, we now have a set of directions that we may give to Docker to containerize our software into a picture that we will then push to Docker Hub. We will now do that with a few instructions from our VS Code terminal, proven beneath. Every of those strains must be run individually within the VS Code terminal from the highest listing of your undertaking.

1. First, we construct our Docker picture, which is able to doubtless take a minute or two. On this case, we’re naming our picture ‘recommendersystem’
2. Subsequent, we tag our picture, the syntax right here is image_name user_name/docker_hub_folder:image_name_on_dockerhub
3. Lastly, we push our picture to Dockerhub once more specifying the user_name/docker_hub_folder:image_name_on_dockerhub

docker construct -t recommendersystem .
docker tag recommendersystem seelucas/fastapi_tutorial:fastapi_on_render
docker push seelucas/fastapi_tutorial:fastapi_on_render

After that is accomplished, we must always be capable to log in to DockerHub, navigate to our undertaking, and see that we have now a picture whose title matches what we gave it within the earlier 3 instructions, on this case, fastapi_on_render.

Pulling Docker Picture to Render

Now we have now our Docker Picture on DockerHub, and it’s time to deploy that Picture on Render. This may be accomplished by navigating to the identical undertaking that we created our database in, “fastapi-test”, deciding on “New”, within the high proper, after which deciding on “Internet Service” as our FastAPI app will probably be deployed as a Internet Software.

As a result of we’re deploying our picture from Dockerhub, we specify that our Supply Code is an Current Picture, and as proven in Determine 11, we paste the Dockerhub Listing path to the Picture we need to deploy into ‘Picture URL’ in Render. We then get a notification that it is a non-public picture, which suggests we’ll have to create a Dockerhub Entry token that we will then use to securely pull the picture from DockerHub into Render.

Fortuitously, making a DockerHub Entry token is easy; we navigate to our DockerHub account -> Settings → Private Entry token. The display ought to seem like Determine 12. we offer an entry token title, expiration date, and permissions. Since we’re pulling the picture into Render, we solely want learn entry relatively than write or delete, so we choose that.

Lastly, deciding on ‘Generate’ will generate our token, which we then want to repeat over to render and enter as proven in Determine 13.

As soon as we’ve chosen ‘Add Credential’ as proven above, it is going to then load for a minute because the credentials are saved. We’ll then be taken again to the earlier display, the place we will choose our credentials to make use of to hook up with DockerHub. On this case, we’ll use the tutorial credentials we simply created and choose Join. We’ll then have established a connection that we will use to tug our Docker Picture from DockerHub to Render for Deployment.

On the following web page, we proceed with establishing our Render Internet applicaiton by deciding on the free choice after which importantly, on Environmental Variables, we copy and paste our .env file. Whereas we don’t use all of the variables on this file, we do use the ‘Internal_Database_Url’, which is the URL that FastAPI will search for in our essential.py file. With out this, we gained’t be capable to connect with our database, so it’s essential that we offer this. Be aware: for testing, we beforehand used the ‘External_Database_Url’ as a result of we have been operating the script from our native machine, which is exterior to our Render surroundings; nevertheless, right here each the Database and Internet Server are in the identical Render surroundings, so we use the Internal_Database_Url in essential.py.

After coming into our environmental variables, we then select ‘Deploy Internet Service’.

The service will take a few minutes to deploy, however then you need to get a notification like beneath that the service has deployed with a render hyperlink on high that we will entry at.

Navigating to this hyperlink will take us to the Good day World technique, if we add/docs to the top of it, we’ll be taken to the swagger docs in Determine 17. Right here we will take a look at and guarantee our FastAPI Internet Software is linked to our database by utilizing the Fetch All Customers technique. We will see beneath that this does certainly return knowledge.

Lastly, we need to verify if our consumer suggestions system is dynamically updating. In your earlier API name, we will see that there’s a consumer ‘user_username’ in our database. Utilizing the Fetch Really helpful Sport technique with this username, we will see the highest match is appid = B08BHHRSPK.

We replace our customers’ appreciated video games by selecting a random one from our video games appid = B0BHTKGN7F, which seems to be ‘The Elder Scrolls: Skyrim Boardgame’, and leveraging our user_games POST technique.

Including a recreation to our consumer video games desk is meant to routinely set off the recommender pipeline to rerun for that consumer and generate new suggestions. If we navigate to our console, we will see that it seems to have occurred as we get the brand new consumer suggestions generated message proven beneath.

If we navigate again to our Swagger docs, we will strive the fetch suggestion technique once more, and we see in Determine 21 that we certainly do have a unique listing of suggestions than the one earlier than. Our Recommender Pipeline is now routinely updating as customers add extra knowledge and is accessible past our native surroundings.

Wrapping Up:

On this undertaking, we’ve proven find out how to arrange and deploy a suggestion system leveraging a FastAPI interplay layer with a PostgreSQL database to generate clever board recreation suggestions for our customers. There are additional steps we may take to make this method extra sturdy, like implementing a hybrid suggestion system as we achieve extra consumer knowledge or enabling consumer tagging to seize extra options. Moreover, though we didn’t cowl it, we did make the most of a GitHub workflow to rebuild and push our Docker picture at any time when there’s a brand new replace to our essential department, and this code is obtainable in .github/workflows. This helped to vastly speedup improvement as we didn’t should manually rebuild our Docker picture at any time when we made a small change.

I hope you loved studying and that this helps you construct and deploy your tasks with FastAPI.

LinkedIn: https://www.linkedin.com/in/lucas-see-6b439188/

Electronic mail: [email protected]

Figures: All pictures, except in any other case famous, are by the creator.

Hyperlinks:

1. Github Repository for Venture: https://github.com/pinstripezebra/recommender_system
2. FastAPI Docs: https://fastapi.tiangolo.com/tutorial/
3. Docker Tutorial: https://www.youtube.com/watch?v=b0HMimUb4f0
4. Docker Desktop Obtain: https://www.youtube.com/watch?v=b0HMimUb4f0
5. Docker Hub: https://hub.docker.com/