Last updated: December 15, 2023
Docker has improved how we develop, package, and deploy applications. It allows us to encapsulate logic and dependencies in a container ready to run on any platform. However, each container produces logs, which can be challenging to manage. Logs give a record of events happening in a system and are very useful when debugging. Thus, we need ways to access the container logs. Docker logs tail is a command that allows us to follow specific log outputs of our container.
In this article, we’ll talk a lot about Docker logs. You’ll learn how to display Docker logs in your console. Also, you’ll learn tips and tricks related to displaying logs. Lastly, this article will discuss how to configure Docker to send logs to SolarWinds® Papertrail™.
Let’s learn first about the importance of Docker logs.
What Are Docker Logs?
Docker logs are any output data the container writes to STDOUT (standard output) or STDERR (standard error). These logs are helpful for debugging and tracing issues in your application running in a container. However, if you run a container in detached mode, you can’t see the logs in your console. This is because detached mode refers to running a container in the background of your terminal. Therefore, you won’t see any logging or other output from your Docker container. To display these logs, you can use the command docker logs with different options like –tail.
Docker logs are important for developers and DevOps engineers. Logs are most useful when debugging or analyzing a problem because they offer insights into what went wrong. Thus, developers and engineers can solve issues faster.
In addition, you can apply trend analysis to Docker logs to detect anomalies. For example, SolarWinds Observability SaaS (formerly known as SolarWinds Observability) offers this feature for any set of data, including logs. This lets you detect anomalies faster and shift from reactive to proactive monitoring.
The next section goes into detail on how to open a stream of Docker logs.
How to Tail Docker Logs?
Imagine we’re running a container and want to access the logs for this container. How can we accomplish this task?
First, we can use the following command to check for the currently running containers:
docker ps -a This command prints a list of the running containers. You’ll see information about the containers, like their IDs, used images, start commands, how long the containers have been running, and many other variables.
For us, the most important parameter is the container ID, which we will use in the next step.
CONTAINER ID IMAGE COMMAND CREATED STATUS
fcae22fa4144 xyz/my-image:latest "/opt/bin/entrypoint-" 2 hours ago Up 2 minutes
155206af2f03 abc/my-image:latest "/entrypoint.sh mysql" 2 hours ago Up 2 minutes Now that we’re sure our container is running, let’s use the container ID to see all of its logs.
Enter the following command in your command-line interface (CLI), replacing <container ID> with your container’s ID:
docker logs <container ID>Although this will show us the logs, it won’t allow us to view continuous log output. In Docker jargon, we refer to creating a continuous stream of log output as tailing logs. To tail the logs for our container, we can use the follow option.
docker logs --follow <container ID>Where Are Docker Logs Stored by Default?
Docker logs are stored in a JSON file on the Docker host by default. The default log file directory for Docker is /var/lib/docker/containers/ on the host machine. Each container’s logs are stored in a separate directory under this location. The naming format of the log files is container_id-json.log, where container_id is a unique identifier for each container. Below is an example file path, but you’ll replace the container id with yours.
/var/lib/docker/containers/<container_id>/<container_id>-json.log Next, let’s explore more exciting tricks related to displaying Docker logs.
Other Docker Logging Tricks
Here are three useful logging tricks you can access through your CLI.
#1: Display Only the Latest Lines
In some cases, you don’t want to see all logs for a container. Perhaps something happened and you want to quickly verify the latest 100 logs for your container. In this case, you can use the tail option to specify the number of logs you want to see:
docker logs --tail 100 <container ID>#2: Stream Logs Until a Specific Point in Time
Docker provides the option to only stream logs from a specific period of time. For example, logs written during the first three seconds when the container was active can tell you if the container started successfully. In this case, you don’t have to create a never-ending stream of logs. Here, you can use the until option in combination with the follow option. The until option allows you to specify a time span for which the container should print logs to your CLI.
docker logs --follow --until=3sNote that you can use different notations to designate the timescale. For example, to see the logs for the last 30 minutes, use the following command:
docker logs --follow --until=30m#3: Stream Logs From a Specific Point in Time
The opposite action is also possible with Docker CLI options. Let’s say you want to see the logs from a specific point in time up until now. The since option helps with this task.
docker logs --since 2023-12-02 <container ID>Note that the accepted format here is YYYY-MM-DDTHH:MM:SS. This means you can specify a very accurate timestamp from which you want to display logs, or a less specific timestamp, as shown in the example above.
Alternatives to Docker Logs Tail
Using the docker logs tail command to track your logs isn’t the only way you can track your logs. There are other ways and other docker commands you can use. They include:
- Directly accessing the log files: by knowing where logs are stored by default, you navigate to the location. For example, this is how you can use the tail command to view the logs of your container.
tail -f /var/lib/docker/containers/<container_id>/<container_id>-json.log
- Using Docker desktop GUI: in the docker desktop, you navigate to the containers section, select your container, and then click on the logs tab to view and tail the logs.
- Using docker-compose: if you’re managing multiple containers, you can use docker-compose -f <service_name> to view and tail logs.
- Use logging drivers: docker supports various logging drivers like Syslog, journald, or fluentd that you can configure to tail and view logs.
- Use third-party logging tools: you can leverage tools like Logspout, Logstash, or Papertrail to collect, tail, and analyze logs from your containers.
How to Log Docker Logs to Papertrail
In this section, we’ll give you a simple example of how you can configure Docker to send logs to Papertrail.
First of all, you can run a logspout container, which allows you to configure an address to send logs to. The below example starts a logspout container configured to send logs to Papertrail.
docker run --restart=always -d \
-v=/var/run/docker.sock:/var/run/docker.sock gliderlabs/logspout \
syslog+tls://logsN.papertrailapp.com:XXXXXAlternatively, you can configure the syslog driver to tell the container where to send logs. Make sure to change the syslog-address property to your custom Papertrail address.
docker run --log-driver=syslog
--log-opt syslog-address=udp://logsN.papertrailapp.com:XXXXX
image-nameYou can find more information in Papertrail tutorial.
Next, let’s dive into best practices for logging in Docker.
Frustration-free log management. (It’s a thing.)
Aggregate, organize, and manage your logs with Papertrail4 Best Practices When Logging in Docker
Here’s a shortlist of four best practices related to logging in Docker containers, which you can apply to your organization.
Export Logs to Persistent Storage
You can easily create and destroy containers. However, every time a container restarts, you lose all the data it holds. Therefore, never store application-specific data in your container.
For the same reason, you should take good care of your logs. Logs can be stored persistently to a volume, but it’s even better to store them long-term. For example, you can pipe logs to a local hard drive or you can send them to a log management platform. Both options allow you to save your logs long-term and use them for future analysis.
Consider Using a Logging Container
A logging container helps you scale your logging. The idea is that you pipe your logging output from multiple containers to a logging container. Next, your logging container takes care of saving the logs to persistent storage or a log management platform.
Also, you can spin up multiple logging containers to scale your logging service when you decide to host more containers. It’s a flexible and easy solution for handling log output.
Log Data to Standard Text: Output Channels
To be able to aggregate logs from your containers, you need to make sure the applications running in those containers log data to STDOUT or STDERR. Those are both standard channels for logging output messages or error messages. Docker is configured to automatically pick up data from both outputs. If you log data to a file inside your container, you risk losing all this data when the container crashes or restarts. Therefore, if you don’t want to lose any important logging data, it’s important to log to STDOUT or STDERR.
Log Data as JSON Format
Docker supports the JSON logging format, and logging data in this format is recommended. You might wonder why. The reason is that Docker itself stores logs as JSON files; therefore, it’s optimized to handle JSON data.
For this reason, many Node.js logging libraries, such as Bunyan or Winston, prefer to log data using the JSON format.
Now, let’s summarize what we’ve learned from this article.
Final Words on Docker Logs
As you’ve seen, Docker provides multiple CLI options to display logs in different formats. For example, you can display logs for a specified point in time. In addition, the follow option is one of the most used Docker options because it allows developers to live tail logs for specific containers.
Lastly, you can configure Docker to transport logs to a log management platform, such as Papertrail. This also helps to visualize your logs. Papertrail allows you to easily monitor logs and provides developers with the ability to create alarms to warn them in case an anomaly is detected.
Want to give it a try? You can sign up for a free trial of Papertrail now and see just how it can work for you.
This post was written by Michiel Mulders. Michiel is a passionate blockchain developer who loves writing technical content. Besides that, he loves learning about marketing, UX psychology, and entrepreneurship. When he’s not writing, he’s probably enjoying a Belgian beer!