Website Detective: Python Flask, MongoDB and Celery

Photo by Priscilla Du Preez on Unsplash

“There are two ways to write error-free programs; only the third one works”

Back to my server logs, I see lots of entries that made me curious; Who is checking out a simple, plain website? I needed to know more about my website visitors.

Who accessed my Page?

Just like a curious kid, photo by Joseph Rosales on Unsplash
  1. Creating a simple Flask application
  2. Adding a task using Celery
  3. Performing CRUD operations on MongoDB

Flask — a python micro web framework

What I like about Flask is the simplicity. It also comes with batteries-included, which helps a long way. You can get things done quickly without having to worry about structural complexity. I’ll use it to run a simple http web server. It’s advised not to use Flask in production. You can switch to Gunicorn or uWSGI instead when deploying your Flask app.

Python Celery — task queue

Celery is an open source asynchronous task queue. Task queues are used as a mechanism to distribute work across threads or machines. Celery requires a message broker to communicate between workers and the Flask app, and to store task results. A common choice RabbitMQ, but you can also use Redis or MongoDB. I chose MongoDB for this demonstration.

MongoDB — document-oriented database

A general purpose, document-based, distributed database built for modern application developers and for the cloud era. I will be using MongoDB to store the access logs and task results.

The Code

The snippets below explains in brief, the app implementation.

Main — app/__init__.py

This comprise of the main part of the app. Some external file such as app/constant.py and app/task.py was created to keep the code clean and for simplicity. Remember DRY & KISS? from software design principles.

# set Celery Broker URL
app.config['CELERY_BROKER_URL'] = MONGODB_CON_STR

app.config['CELERY_RESULT_BACKEND'] = MONGODB_CON_STR

Celery Task — app/task.py

The task.py contains a celery task. The main reason for using celery is to asynchronously run check_who_and_where() function. This will prevent lagging and waiting on the user end.

View Logs

The logs are stored in side MongoDB and can be accessed through mongo shell or running the view_logs() function. You can also use a MongoDB GUI like Robo 3T if you despise command line but I doubt it… haha, you came this far.

# simple data stored
# db.page_access_log.findOne({})
data = {
"browser": "chrome",
"city": "Dallas",
"country_code": "US",
"country_name": "United States",
"date": "Sat, 06 Jun 2020 08:29:28 GMT",
"ip": "ip.ad.dre.ss",
"latitude": 32.8477,
"longitude": -96.7025,
"org": "ZNET",
"page": "profile",
"platform": "windows",
"region": "Texas",
"region_code": "TX"
}

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store