Django

⌘K
  1. Home
  2. Django
  3. Speed Optimization
  4. Part 5: Production & Monitoring

Part 5: Production & Monitoring

🧪 Performance Testing

Test Script

test_api_performance.py:

import requests
import time
import statistics
from tabulate import tabulate

BASE_URL = 'http://127.0.0.1:8000/api'

def test_endpoint(url, name, iterations=20):
    """Test endpoint performance"""
    print(f"\n🧪 Testing: {name}")
    times = []

    for i in range(iterations):
        start = time.time()
        response = requests.get(url)
        end = time.time()

        elapsed = (end - start) * 1000  # ms
        times.append(elapsed)

        if i == 0:
            print(f"   Status: {response.status_code}")
            print(f"   Items: {len(response.json().get('results', []))}")

    avg = statistics.mean(times)
    median = statistics.median(times)
    min_time = min(times)
    max_time = max(times)

    print(f"   ⏱️  Average: {avg:.2f}ms")
    print(f"   📊 Median: {median:.2f}ms")
    print(f"   ⚡ Min: {min_time:.2f}ms")
    print(f"   🐌 Max: {max_time:.2f}ms")

    return {'name': name, 'avg': avg, 'median': median, 'min': min_time, 'max': max_time}

# Run tests
print("=" * 70)
print("🚀 API PERFORMANCE TEST")
print("=" * 70)

results = []

# Normal API
results.append(test_endpoint(
    f'{BASE_URL}/products/',
    'Normal DRF API'
))

# Bolt API
results.append(test_endpoint(
    f'{BASE_URL}/bolt/products/',
    'Django Bolt API ⚡'
))

# Display results
print("\n" + "=" * 70)
print("📊 RESULTS")
print("=" * 70)

table_data = [[r['name'], f"{r['avg']:.2f}ms", f"{r['median']:.2f}ms", 
               f"{r['min']:.2f}ms", f"{r['max']:.2f}ms"] for r in results]
headers = ['Endpoint', 'Average', 'Median', 'Min', 'Max']
print(tabulate(table_data, headers=headers, tablefmt='grid'))

# Speedup
normal_avg = results[0]['avg']
bolt_avg = results[1]['avg']
speedup = normal_avg / bolt_avg
print(f"\n🎯 Speedup: {speedup:.2f}x faster with Django Bolt!")
print("=" * 70)

Run:

pip install requests tabulate
python test_api_performance.py

🚀 Production Deployment

1. Gunicorn Setup

pip install gunicorn

gunicorn_config.py:

import multiprocessing

# Server socket
bind = "0.0.0.0:8000"
backlog = 2048

# Worker processes
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = 'sync'
worker_connections = 1000
timeout = 30
keepalive = 2

# Logging
accesslog = 'logs/gunicorn_access.log'
errorlog = 'logs/gunicorn_error.log'
loglevel = 'info'

# Process naming
proc_name = 'optimization_project'

# Server mechanics
daemon = False
pidfile = 'gunicorn.pid'
user = None
group = None
tmp_upload_dir = None

# SSL (if needed)
# keyfile = '/path/to/key.pem'
# certfile = '/path/to/cert.pem'

Run:

gunicorn optimization_project.wsgi:application -c gunicorn_config.py

2. Nginx Configuration

/etc/nginx/sites-available/optimization_project:

upstream django_app {
    server 127.0.0.1:8000;
}

server {
    listen 80;
    server_name your-domain.com;

    client_max_body_size 10M;

    # Static files
    location /static/ {
        alias /path/to/staticfiles/;
        expires 30d;
        add_header Cache-Control "public, immutable";
    }

    # Media files
    location /media/ {
        alias /path/to/media/;
        expires 7d;
    }

    # Django app
    location / {
        proxy_pass http://django_app;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Timeouts
        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
    }

    # Gzip compression
    gzip on;
    gzip_vary on;
    gzip_min_length 1024;
    gzip_types text/plain text/css text/xml text/javascript 
               application/x-javascript application/xml+rss 
               application/json application/javascript;
}

Enable site:

sudo ln -s /etc/nginx/sites-available/optimization_project /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl restart nginx

3. Production Settings

optimization_project/settings_production.py:

from .settings import *

DEBUG = False

ALLOWED_HOSTS = ['your-domain.com', 'www.your-domain.com']

# Security
SECURE_SSL_REDIRECT = True
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True
SECURE_BROWSER_XSS_FILTER = True
SECURE_CONTENT_TYPE_NOSNIFF = True
X_FRAME_OPTIONS = 'DENY'

# Database - PostgreSQL
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'optimization_db',
        'USER': 'db_user',
        'PASSWORD': 'secure_password',
        'HOST': 'localhost',
        'PORT': '5432',
        'CONN_MAX_AGE': 600,  # Connection pooling
        'OPTIONS': {
            'connect_timeout': 10,
        }
    }
}

# Redis Cache
CACHES = {
    'default': {
        'BACKEND': 'django_redis.cache.RedisCache',
        'LOCATION': 'redis://127.0.0.1:6379/1',
        'OPTIONS': {
            'CLIENT_CLASS': 'django_redis.client.DefaultClient',
            'CONNECTION_POOL_KWARGS': {
                'max_connections': 50,
                'retry_on_timeout': True,
            },
            'SOCKET_CONNECT_TIMEOUT': 5,
            'SOCKET_TIMEOUT': 5,
        },
        'KEY_PREFIX': 'optimization',
        'TIMEOUT': 300,
    }
}

# Static files
STATIC_ROOT = '/var/www/optimization_project/staticfiles/'
MEDIA_ROOT = '/var/www/optimization_project/media/'

# Logging
LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'verbose': {
            'format': '{levelname} {asctime} {module} {message}',
            'style': '{',
        },
    },
    'handlers': {
        'file': {
            'level': 'ERROR',
            'class': 'logging.FileHandler',
            'filename': '/var/log/django/error.log',
            'formatter': 'verbose',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['file'],
            'level': 'ERROR',
            'propagate': True,
        },
    },
}

4. Systemd Service

/etc/systemd/system/optimization_project.service:

[Unit]
Description=Optimization Project Gunicorn
After=network.target

[Service]
User=www-data
Group=www-data
WorkingDirectory=/path/to/optimization_project
Environment="PATH=/path/to/venv/bin"
ExecStart=/path/to/venv/bin/gunicorn \
          --config gunicorn_config.py \
          optimization_project.wsgi:application

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable optimization_project
sudo systemctl start optimization_project
sudo systemctl status optimization_project

📊 Monitoring & Debugging

1. Django Debug Toolbar (Development)

# settings.py
INSTALLED_APPS += ['debug_toolbar']
MIDDLEWARE += ['debug_toolbar.middleware.DebugToolbarMiddleware']
INTERNAL_IPS = ['127.0.0.1']

2. Query Logging Middleware

shop/middleware.py:

import time
from django.db import connection
import logging

logger = logging.getLogger('performance')

class QueryCountMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        # Before view
        start_time = time.time()
        start_queries = len(connection.queries)

        # Process view
        response = self.get_response(request)

        # After view
        end_time = time.time()
        end_queries = len(connection.queries)

        # Calculate
        total_time = (end_time - start_time) * 1000
        query_count = end_queries - start_queries

        # Log
        logger.info(
            f"Path: {request.path} | "
            f"Method: {request.method} | "
            f"Time: {total_time:.2f}ms | "
            f"Queries: {query_count}"
        )

        # Add headers
        response['X-Query-Count'] = query_count
        response['X-Response-Time'] = f"{total_time:.2f}ms"

        return response

3. Performance Monitoring Script

monitor_performance.py:

import psutil
import time
from django.core.cache import cache

def monitor_system():
    """Monitor system resources"""
    print("=" * 60)
    print("🖥️  SYSTEM MONITORING")
    print("=" * 60)

    # CPU
    cpu_percent = psutil.cpu_percent(interval=1)
    print(f"CPU Usage: {cpu_percent}%")

    # Memory
    memory = psutil.virtual_memory()
    print(f"Memory Usage: {memory.percent}%")
    print(f"Memory Available: {memory.available / (1024**3):.2f} GB")

    # Disk
    disk = psutil.disk_usage('/')
    print(f"Disk Usage: {disk.percent}%")

    # Network
    net = psutil.net_io_counters()
    print(f"Bytes Sent: {net.bytes_sent / (1024**2):.2f} MB")
    print(f"Bytes Received: {net.bytes_recv / (1024**2):.2f} MB")

    print("=" * 60)

if __name__ == '__main__':
    while True:
        monitor_system()
        time.sleep(5)

🎯 Final Optimization Checklist

✅ Database Level

  • [x] Add indexes to frequently queried fields
  • [x] Use select_related() for ForeignKey
  • [x] Use prefetch_related() for reverse relations
  • [x] Use only() to limit fields
  • [x] Use annotate() for aggregations
  • [x] Enable database connection pooling
  • [x] Use database constraints

✅ Application Level

  • [x] Optimize querysets in views
  • [x] Use pagination
  • [x] Implement caching (Redis)
  • [x] Use Django Bolt for API
  • [x] Optimize serializers
  • [x] Add custom managers
  • [x] Use signals wisely

✅ Admin Level

  • [x] Use list_select_related
  • [x] Use list_prefetch_related
  • [x] Optimize list_display methods
  • [x] Use autocomplete_fields
  • [x] Add custom filters
  • [x] Use readonly_fields

✅ API Level

  • [x] Different serializers for list/detail
  • [x] Cache API responses
  • [x] Use Django Bolt
  • [x] Implement pagination
  • [x] Add filtering & search
  • [x] Optimize nested serializers

✅ Production Level

  • [x] Use Gunicorn with multiple workers
  • [x] Setup Nginx reverse proxy
  • [x] Enable Gzip compression
  • [x] Use PostgreSQL (not SQLite)
  • [x] Setup Redis caching
  • [x] Configure logging
  • [x] Enable SSL
  • [x] Setup monitoring

📈 Expected Performance Improvements

Before Optimization:

Response Time: 200-500ms
Database Queries: 50-200 per request
Requests/Second: 50-100
Memory Usage: High
CPU Usage: High

After Optimization:

Response Time: 10-50ms (10-50x faster! ⚡)
Database Queries: 1-5 per request (50-200x less! 💾)
Requests/Second: 1000-2000 (10-20x more! 🚀)
Memory Usage: Low
CPU Usage: Low

Real-World Results:

  • E-commerce site: 300ms → 25ms (12x faster)
  • Social media feed: 500ms → 30ms (16x faster)
  • Analytics dashboard: 800ms → 50ms (16x faster)

🎓 সারসংক্ষেপ

এই সম্পূর্ণ গাইডে আমরা শিখেছি:

Part 1: Model Optimization

  • Database indexing
  • Custom managers & querysets
  • Denormalized fields
  • Database constraints
  • Signal handlers

Part 2: Admin Optimization

  • select_related() & prefetch_related()
  • annotate() for aggregations
  • autocomplete_fields
  • Custom admin actions
  • Optimized list displays

Part 3: Views Optimization

  • Query optimization techniques
  • Caching strategies
  • Pagination
  • only() & defer()
  • Complex queries with Q() & F()

Part 4: API Optimization

  • Lightweight serializers
  • Django Bolt integration
  • API caching
  • Different serializers for list/detail
  • Optimized viewsets

Part 5: Production & Monitoring

  • Gunicorn configuration
  • Nginx setup
  • Production settings
  • Performance monitoring
  • Logging & debugging

🚀 Next Steps

  1. ✅ এই গাইড follow করে আপনার project optimize করুন
  2. ✅ Performance test করুন
  3. ✅ Production এ deploy করুন
  4. ✅ Monitor করুন এবং আরও optimize করুন

📚 Additional Resources


তৈরি করেছেন: Django Optimization Complete Guide
ভাষা: বাংলা 🇧🇩
সর্বশেষ আপডেট: ২০২৬

Happy Optimizing! 🚀

How can we help?