As a developer, you’ll always face challenges with your applications. Some of these challenges arise from the server after deploying code. It can sometimes be difficult to know what the problem is, especially if the app is running as expected in your development environment. This is something no one can escape. These errors are bound to happen to anyone, regardless of their level of experience as a software engineer or developer.
In this post, I’ll walk you through some of the best tips you can use to log error messages. These tips can help you solve error issues you might face in DigitalOcean. DigitalOcean is among the top five web hosting platforms to serve your web apps. In my experience, companies and developers alike seem to love it, so if you haven’t used it yet, it probably won’t be long before you do.
Is Logging Apps on DigitalOcean a Big Deal?
It’s definitely a big deal, and it’ll help you in the long run. With the monitoring DigitalOcean provides, you can monitor your Droplet performance and even receive alerts when there are issues in your DigitalOcean infrastructure. However, monitoring applications isn’t as easy.
Even if you’re not currently facing challenges with the apps you’ve deployed on DigitalOcean, you’ll likely experience some issues as time goes on. These issues might be caused by increased traffic as more of your customers use the app. I bet you’re likely having some issues already and need a good resource to find some help.
Working With journalctl to View Logs
journalctl is a command tool used to view logs collected by systemd. The journalctl system service collects messages from the kernel and other resources. If you can use this tool, you’ll be able to view all the logs in the system. By default, when you run journalctl, journalctl echoes messages by showing the oldest first. You can change this by running the command with -r, like this: $ journalctl -r. You’ll then be able to view logs beginning with the latest log.
You may also want to monitor new log messages, which you can do by passing -f in front: $ journalctl -f. Not all log messages are useful, so you can filter the logs by passing an option –until to see log messages through a specified period or –since to show logs after a specific period. To show log messages between a certain date range, you can combine the two options like this: $ journalctl –since “2020-08-01 20:24:00” –until “2020-09-10 20:30:00”. This combination will show the logs between the provided dates and times.
Using doctl Command-Line Interface (CLI)
The command-line interface (CLI) in DigitalOcean is called doctl. You can find the documentation of the tool here. Here are some of the things you can access using doctl:
doctl allows you to interact with the DigitalOcean API via the command line. It supports most functionality found in the control panel. You can create, configure, and destroy DigitalOcean resources like Droplets, Kubernetes clusters, firewalls, load balancers, database clusters, domains, and more.
Using doctl, you can manage the DigitalOcean infrastructure from your local computer. With DigitalOcean’s doctl command-line tool, you can access the infrastructure and control your applications as if you’re using the actual control panel. In other ways, doctl gives you full access to interact with DigitalOcean’s control panel.
Logging With NGINX
When you’ve deployed your app on DigitalOcean under the NGINX server, the log messages can be found in the root of the server. You’ll need to go to the following folders: /var/log/nginx/access.log and /var/log/nginx/error.log. “But how can I get to see the actual logs?” you may ask. Well, the first step is knowing where your log files are. Once you locate your logs, you can use commands like cat to open the file. In this case, you already know where the files are located, so let’s say you wanted to open the error.log folder.
You can run a command like this: $ cat ./var/log/nginx/error.log. You’ll be able to view what’s contained in the error log file. You can even go further and filter the log file to get specific errors you want to see. If you want to see errors with the keyword “error,” you can simply use the grep command. The command can look like this: $ grep “error” ./var/log/nginx/error.log. This command will search the error.log file. The command will echo to the terminal all the lines in the file with an error keyword. This can be helpful, especially if the file you’re working with is large.
Logging Apache Errors
The same basic concept applies for Apache logs. The only thing you should be aware of is that the Apache errors are written to the error.log file, which you can find here: /var/log/apache2/error.log. You’ll need to run the grep command and point to that path (like this: $ cat ./var/log/apache2/error.log and $ grep “error” ./var/log/nginx/error.log.) to filter the log messages. To find lines containing “error” and create a new file from your search output, for example, run grep using this command: $ grep “error” ./var/log/nginx/error.log > new_output.log. The file created, new_output.log, contains your new search results. If you’re looking for other Apache error logs, there are additional log files here: /var/log/apache2/access.log.
As a rule of thumb, you can find application log files in the /var/log directory in DigitalOcean. If you’re searching for log data, this is the first place you should look.
Logging MySQL Logs From Your App
If you notice your application is running slow but you’re not sure why, the best place to start is to look at the database log messages. Slow queries to the database will cause your app to crawl, resulting in a suboptimal user experience and frequent calls to support. In DigitalOcean, you can check the log messages in the MySQL directory under /var/log/mysql/ and in /var/log/mysql/mysql-slow.
By default, slow error log configuration is in a commented status. You’ll need to uncomment these settings or add them if they aren’t in the mysql.cnf file. This can vary a bit depending on the different versions of MySQL, so you may need to check your configurations. Here’s how you can open the configuration file: /$ sudo /etc/mysql/mysql.d/msqld.cnf. You’ll see lines you need to change before they start receiving MySQL log messages from the server.
You can use vim or nano to open the configuration file, and you’ll see the lines. Edit them to look like this:
. . .
slow_query_log = 1
slow_query_log_file = /var/log/mysql/mysql-slow.log
long_query_time = 2
log-queries-not-using-indexes
. . .After you’ve uncommented the lines and saved the file, restart MySQL and you’re good to go. You can restart MySQL with the command sudo systemctl restart mysql. Just like the previous tips I gave in this article, you’ll need to open the file using the command $ sudo less /var/log/mysql/mysql-slow.log to find the errors you’re looking for. Since we’re looking for the queries taking too long to execute, we use the “less” in the command to open the file in read-only mode.
Logging Your App by Using a Programming Language
Most back-end programming languages offer support for logging. Some languages use libraries or plug-ins to log error messages. For example, Python has a logging module you can use for log messages.
Here’s an example of logging an app built with the Python programming language:
import logging
logging.basicConfig(filename="log_file.log", level=logging.WARN,format="%(asctime)s:%(levelname)s:%(message)s")
class Payments:
def __init__(self, item, price):
self.item = item
self.price = price
logging.warning(f'Paid ${self.price} for {self.item}')
Payments("jordan air force shoes.",120)This demonstrates how you can log messages using Python. Obviously, you’ll need to log messages such as exceptions. If you run the code above, you’ll see a new file created named log_file.log, as specified in the settings. From there, you can find log messages like this:
#output
2020-09-06 16:17:36,221:WARNING:Paid $120 for jordan air force shoes.Similarly, you can also log apps developed in other programming languages. For example, if you’re using Node.js as your back-end language, you can use a package called winston to log messages from your app. Here’s an example of this:
const winston = require('winston');
const logger = winston.createLogger({
level: 'info',
format: winston.format.json(),
defaultMeta: { service: 'user-service' },
transports: [
// - Write all logs with level `error` and below to `error.log`
// - Write all logs with level `info` and below to `combined.log`
new winston.transports.File({ filename: 'error.log', level: 'error' }),
new winston.transports.File({ filename: 'log-combined.log' }),
],
});As you might have guessed, the error messages from the logger will be stored in error.log, and the other types of logs from different levels will be in the log-combined.log file because we haven’t specified the logging level.
Conclusion
No matter what error you’re trying to troubleshoot, you’ll need to see the log files. Finding and searching the logs is the first step to solving any problem you’re having with applications on the DigitalOcean server. With the distributed systems popular with cloud applications, it’s not always obvious where to start troubleshooting. It can be hard to understand the interactions between different systems when using application-specific commands or searching the logs one at a time. This is where a cloud-based logging service like SolarWinds® Papertrail™ can help.
Papertrail bring logs from multiple services and applications together and lets you search them from a single UI. It offers a tail feature similar to the journalctl -f. command across your logs. This feature is extremely useful if you’re trying to get the root of a problem and want to test out different behaviors. Additionally, Papertrail lets you livestream logs and search and filter live log data. You can even save your searches as you troubleshoot and create alerts right from the search bar. Sign up for a free trial of Papertrail if you want to see this for yourself.
This post was written by Mathews Musukuma. Mathews is a software engineer with experience in web and application development. Some of his skills include Python/Django, JavaScript, and the Ionic framework. Over time, Mathews has also developed interest in technical content writing.