🌤️ Day 1: Setting Up the AWS Weather Dashboard Application

Check out the project on GitHub: AWS Weather Dashboard Repository

In this post, I’ll walk you through setting up a weather dashboard that fetches weather data from the OpenWeather API and displays it on a dashboard. The app uses AWS services like S3 for data storage and AWS CLI for managing AWS resources. I’ve built this app using Python, and it integrates seamlessly with the AWS Cloud.

You can choose to run the app either locally or inside a Docker container.

🛠️ Prerequisites

Before you get started, make sure you have the following:

• 🐍 Python 3.x or Docker installed on your machine.

• 🛠️ AWS CLI installed and configured with your AWS credentials.

• 🌍 OpenWeather API Key for fetching weather data.

• ☁️ AWS Account with access to S3.

Here’s a step-by-step guide on writing and running the weather_dashboard.py script for your Weather Dashboard project:

🚀 Write the Script: weather_dashboard.py

1. Imports and Initialization:

   • os, json, boto3, requests: Import necessary libraries. boto3 is for interacting with AWS S3, requests is for API calls, and json is used for handling JSON data.

   • dotenv: Loads environment variables from a .env file.

   • datetime: Used to add timestamps to the file names when saving weather data.

import os
import json
import boto3
import requests
from botocore.exceptions import ClientError
from dotenv import load_dotenv
from datetime import datetime

2. Load Environment Variables:

   • Load variables like the OpenWeather API key and S3 bucket details from the .env file.

# Load environment variables from .env file
load_dotenv()

3. Class Setup:

   • Create a class WeatherDashboard that will hold methods to fetch weather data, create an S3 bucket (if it doesn’t exist), and upload weather data to S3.

class WeatherDashboard:
    def __init__(self):
        self.aws_bucket_name = os.getenv("AWS_BUCKET_NAME")
        self.aws_region = os.getenv("AWS_REGION")
        self.openweather_api_key = os.getenv("OPENWEATHER_API_KEY")
        self.s3_client = boto3.client("s3", region_name=self.aws_region)

4. Creating the S3 Bucket if it Doesn’t Exist:

   • The create_bucket_if_not_exists method checks if the specified S3 bucket exists. If not, it creates the bucket.

def create_bucket_if_not_exists(self):
    try:
        self.s3_client.head_bucket(Bucket=self.aws_bucket_name)
        print(f"Bucket '{self.aws_bucket_name}' already exists.")
    except ClientError as e:
        if e.response['Error']['Code'] == '404':
            print(f"Bucket '{self.aws_bucket_name}' does not exist. Creating...")
            self.s3_client.create_bucket(
                Bucket=self.aws_bucket_name,
                CreateBucketConfiguration={'LocationConstraint': self.aws_region}
            )
            print(f"Bucket '{self.aws_bucket_name}' created successfully.")
        else:
            raise

5. Fetch Weather Data:

   • The fetch_weather method calls the OpenWeatherMap API and retrieves the weather data for a given city.

def fetch_weather(self, city):
    url = f"http://api.openweathermap.org/data/2.5/weather?q={city}&appid={self.openweather_api_key}"
    try:
        response = requests.get(url)
        response.raise_for_status()
        print(f"Weather data fetched for city: {city}")
        return response.json()
    except requests.exceptions.RequestException as e:
        print(f"Failed to fetch weather data: {e}")
        return None

6. Save Weather Data to S3:

   • The save_to_s3 method saves the fetched weather data to an S3 bucket as a JSON file with a timestamped name.

def save_to_s3(self, data, city):
    file_name = f"{city.lower()}_weather_{datetime.now().strftime('%Y%m%d_%H%M%S')}.json"
    try:
        self.s3_client.put_object(
            Bucket=self.aws_bucket_name,
            Key=file_name,
            Body=json.dumps(data),
            ContentType="application/json",
        )
        print(f"Weather data for {city} saved to S3 as '{file_name}'.")
    except ClientError as e:
        print(f"Failed to upload weather data to S3: {e}")

7. Run the Script:

   • The run method brings everything together: it creates the bucket, fetches weather data, and uploads it to S3.

def run(self, city):
    print("Initializing Weather Dashboard...")
    self.create_bucket_if_not_exists()
    weather_data = self.fetch_weather(city)
    if weather_data:
        self.save_to_s3(weather_data, city)

8. Main Execution Block:

   • This part allows you to run the script from the command line, where the user inputs a city name.

if __name__ == "__main__":
    city_name = input("Enter the city name: ").strip()
    dashboard = WeatherDashboard()
    dashboard.run(city_name)

🖥️ Running the Script Locally

1. Install Dependencies:

Make sure you've installed the necessary libraries. If you haven’t done that yet, use this command:

pip install -r requirements.txt

2. Set Up Environment Variables:

Create a .env file in the project root directory:

AWS_BUCKET_NAME=your_bucket_name_here
AWS_REGION=your_aws_region_here
OPENWEATHER_API_KEY=your_openweather_api_key_here

3. Run the Script:

After setting up everything, you can run the script by executing the following command:

python3 src/weather_dashboard.py

You’ll be prompted to enter a city name, and the script will fetch weather data for that city and save it to your AWS S3 bucket.

🐳 Running the Script with Docker

To run this script inside a Docker container, follow these steps:

1. Create a Dockerfile:

In your project root, create a Dockerfile that defines how the app is built and run in Docker.

# Use the official Python image from Docker Hub
FROM python:3.8-slim

# Set the working directory in the container
WORKDIR /app

# Copy the requirements.txt file into the container
COPY requirements.txt /app/

# Install the dependencies
RUN pip install --no-cache-dir -r requirements.txt

# Copy the rest of the app into the container
COPY . /app/

# Set the environment variable to point to the .env file
ENV ENV_FILE=.env

# Run the weather_dashboard.py script
CMD ["python", "src/weather_dashboard.py"]

2. Build the Docker Image:

Run the following command to build your Docker image:

docker build -t weather-dashboard .

3. Run the Docker Container:

You can run the Docker container with the following command, ensuring the .env file is correctly loaded:

docker run --env-file .env weather-dashboard

This will start the container and run the script inside it.

🚀 Automating Deployment with GitLab CI/CD

To automate the deployment of your Docker image, you can push it to GitLab’s Docker registry.

1. Tag the Docker Image:

Tag your Docker image with the GitLab registry URL:

docker tag weather-dashboard registry.gitlab.com/your-username/your-repository/weather-dashboard:latest

2. Log in to GitLab’s Docker Registry:

docker login registry.gitlab.com

3. Push the Docker Image:

Push the tagged image to GitLab’s Docker registry:

docker push registry.gitlab.com/your-username/your-repository/weather-dashboard:latest

4. Set Up GitLab CI/CD Pipeline:

Create a .gitlab-ci.yml file in your project’s root directory to automate the build and deployment process.

stages:
  - deploy

deploy:
  stage: deploy
  image: docker:latest
  services:
    - docker:dind
  script:
    - docker pull registry.gitlab.com/your-username/your-repository/weather-dashboard:latest
    - docker run --env-file .env registry.gitlab.com/your-username/your-repository/weather-dashboard:latest

This CI/CD pipeline will:

• Pull the Docker image from GitLab’s registry.

• Run the container with the appropriate environment variables.

Now your Weather Dashboard script is set up, and you can easily run it locally, inside Docker, or automate the deployment using GitLab. Enjoy coding and leveraging AWS for cloud-based solutions!

What's Next?

This is just Day 1 of the 30-Day DevOps Challenge. In the upcoming days, I’ll be exploring more DevOps projects, diving deeper into automation, infrastructure as code, continuous integration and delivery (CI/CD), cloud services, and much more. Stay tuned for more hands-on projects and valuable insights that will help you level up your DevOps skills!