While I’m playing with Javascript and React, I’m also going on with using Python and Flask to build a backend. To have meaningful statistics about my app usage and what to optimize, I wanted to centralize logging somewhere.
Context
When I read Python Microservices Development, one of the options for logging was Graylog, but the description to use it was for Graylog 2, and Graylog 3 was just out.
Furthermore, the documentation for Docker was explaining how to build with an older version of Docker Compose that didn’t support link, but it’s not working anymore!
So it makes sense to have a full tutorial, from MongoDB set up to ElasticSearch.
MongoDB setup
Let’s start with MongoDB set up. It’s very easy as long as we add an init.sh script that will create the database for Graylog:
mongo admin -u $MONGO_INITDB_ROOT_USERNAME -p $MONGO_INITDB_ROOT_PASSWORD --eval "db.getSiblingDB('graylog').createUser({user: 'graylog', pwd: 'xxxxx', roles: [{role: 'readWrite', db: 'graylog'}]});"
We can now modify out Docker image to launch the script at start up:
ADD init.sh /docker-entrypoint-initdb.d/init.sh ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] EXPOSE 27017 CMD ["mongod"]
This will properly set up the MongoDB the first time the image is run.
Graylog setup
Let’s now move to Graylog set up through Docker Compose.
We start by generating another password and its SHA hash:
pwgen -N 1 -s 96 echo -n yourpassword | shasum -a 256
We fill this inside the Docker Compose configuration:
graylog: image: graylog/graylog:3.0 environment: GRAYLOG_PASSWORD_SECRET: xxxx GRAYLOG_ROOT_PASSWORD_SHA2: xxxx GRAYLOG_HTTP_EXTERNAL_URI: http://192.168.0.1:9000/ GRAYLOG_MONGODB_URI: mongodb://graylog:xxxx@nosql:27017/graylog GRAYLOG_ELASTICSEARCH_HOSTS: http://elasticsearch:9200 depends_on: - nosql - elasticsearch ports: - 9000:9000 - 12201: 12201
Set GRAYLOG_HTTP_EXTERNAL_URI to the address from which you will serve the Graylog service. Otherwise, you will get a blank page!
We add the rest of the configuration as well:
nosql: image: my-nosql volumes: - local_storage_nosql:/data/db environment: # provide your credentials here - MONGO_INITDB_ROOT_USERNAME=root - MONGO_INITDB_ROOT_PASSWORD=xxx elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:6.6.2 environment: - http.host=0.0.0.0 - transport.host=localhost - network.host=0.0.0.0 - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
Launching and using Graylog
After launching Graylog, you can go at http://192.168.0.1:9000 to start using Graylog. It is possible to use graypy or pygelf, I’m going to show how to use the first former, but the latter works just as well with Graylog 3.
Let’s create a small Python script to log something:
import logging import graypy my_logger = logging.getLogger('test_logger') my_logger.setLevel(logging.DEBUG) handler = graypy.GELFUDPHandler('192.168.0.1', 12201) my_logger.addHandler(handler) my_logger.debug('Hello Graylog2.')
But if you look at Graylog webpage, you won’t see anything. The reason is that you need to create an input first.
In the admin panel, go to System->Inputs and see that you don’t have any existing input. To create one, select GELF UDP in the drop-down and click on the now available Launch new input. Check Global to start the input on all nodes and launch your script again.
You should now see messages coming in.

Conclusion
The configuration is quite straightforward, but if you don’t have all the pieces, then you won’t see the results or the logs coming in. Now you can start logging and explore Graylog further!