Today I’ll show you how to create your custom prometheus metric for your django application. Prometheus is an awesome tool to monitor our stack, specially with its integration with grafana and its cool dashboards.
The example metric is for a recent use case. I wanted to measure my website visitors favourite pages, and where they came from.
I wanted to achieve this without tracking, cookies, paid services or having to use Google Analytics. Since I was already using a Prometheus/ Grafana suite to have visibility of my platform health, I decided to use it for this goal as well.
This was the end result, on grafana:
First step: having django-prometheus up and running!
Just follow the instructions on the project readme, they should be just:
pip install django-prometheus
INSTALLED_APPS = [ ... 'django_prometheus', ... ] MIDDLEWARE = [ 'django_prometheus.middleware.PrometheusBeforeMiddleware', # All your other middlewares go here, including the default # middlewares like SessionMiddleware, CommonMiddleware, # CsrfViewmiddleware, SecurityMiddleware, etc. 'django_prometheus.middleware.PrometheusAfterMiddleware', ]
And lastly, on your
urlpatterns = [ ... url('', include('django_prometheus.urls')), ]
Also, if you are deploying your code using gunicorn or similar(and you should if you are deploying to a production environment!), pay attention. You need to declare multiple ports since the workers could block each other while trying to service the Prometheus scrape request. This is also very well explained in the django-Prometheus documentation.
PROMETHEUS_METRICS_EXPORT_PORT_RANGE = range(8001, 8050)
Second step: adding our custom django Prometheus metric
This is the actual first step in creating our custom prometheus metric on our django app. register the new metric.
Now it might be a good time to refer to the official Prometheus documentation about metrics. After a good read, we can register it:
Our custom metric will count the number of requests by view, path and referer.
The general idea is to first create a new Metrics class, that inherits from
django_prometheus.Metrics. On that we will register own actual metric:
- we want it to be a Counter, because it will be increasing over time.
- we will unoriginally name it
- and add a description that will be useful for documentation purposes.
- lastly, our metric will have three labels:
refererso that we can group and filter by them.
Last but not least, do not forget to call the parent method so that we register the remaining metrics.
from django_prometheus.conf import NAMESPACE from django_prometheus.middleware import Metrics from prometheus_client import Counter class CustomMetrics(Metrics): def register(self): self.requests_total_by_view_path_referer = self.register_metric( Counter, "django_http_requests_total_by_view_path_referer", "Count of requests by view, path, referer.", ["view", "path", "referer"], namespace=NAMESPACE, ) return super().register()
Third step: creating our custom PrometheusMiddleware
Now that we have our shiny metric, we need to measure it. To do so, we will create two classes that inherit from
PrometheusBeforeMiddleware – they will contain our actual metric collection logic.
PrometheusAfterMiddleWare has a bunch of methods where we can add our logic. These methods are called on different moments of the request/response lifecycle, and so receive different parameters:
- For example,
the process_view method()receives the
Take a look at the
django-prometheus existing code to figure out the best place to put your metric, putting it closer to similar metrics. For this
seemed like a good candidate because the view name has been resolved by then.
from django_prometheus.middleware import Metrics, PrometheusBeforeMiddleware, PrometheusAfterMiddleware ... class AppMetricsBeforeMiddleware(PrometheusBeforeMiddleware): metrics_cls = CustomMetrics class AppMetricsAfterMiddleware(PrometheusAfterMiddleware): metrics_cls = CustomMetrics def _get_referer_name(self, request): referer_name = "<unnamed referer>" if hasattr(request, "META"): if request.META is not None: if request.META.get("HTTP_REFERER") is not None: referer_name = request.META.get("HTTP_REFERER") return referer_name def process_response(self, request, response): self.label_metric(self.metrics.requests_total_by_view_path_referer, request, view=self._get_view_name(request), path=request.path, referer=self._get_referer_name(request)).inc() return super().process_response(request, response)
The actual implementation is very simple:
- Label the metric with the view name, the
- To get the view we use the
django-prometheusbuilt-in _get_view_name function,
- To get the referer we adapt the aforementioned function’s logic to get the referer field from the HTTP request header dict, and lastly,
- We get the path from the django request object.
- To get the view we use the
- We finally increment the resulting metric
- Call the parent method to get all the other metrics.
Last step: plugging everything together
This is the easiest step! We just need to replace
django-prometheus middleware with our custom middleware in django settings!
Assuming your custom Middleware is in
MIDDLEWARE = [ 'MyApp.custom_metrics.AppMetricsBeforeMiddleware', # .. all other middleware 'MyApp.custom_metrics.AppMetricsAfterMiddleware' ]
If you go to your
/metrics endpoint you should see it:
Then is can be exported to prometheus so we can do all the cool things with grafana:
Although it has some tricks, creating your custom Prometheus metric for your Django application is not very difficult.
For my web analytics use-case, it would be trivial to get the user-agent field to know how my visitors are reading my site, if they are registered users or not, etc.