Using Python How Do You Print Log Entries for a Given Time Range?

0 Comments

Problem scenario
Using Python, you want to parse a log file. You want to print out entries that have a datetime stamp that are within 24 hours of a given date.

The log file is in this format:

Sep 18 07:28:11 server1 sshd[29284]: Received disconnect from 115.52.17.109 port 46970:11: Bye Bye [preauth]
Sep 18 07:28:11 server1 sshd[29284]: Disconnected from 115.52.17.109 port 46970 [preauth]
Sep 18 07:28:11 server1 sshd[29282]: Failed password for root from 51.10.7.109 port 24844 ssh2
Sep 18 07:28:13 server1 sshd[29287]: pam_unix(sshd:auth): authentication failure; …

Continue reading “Using Python How Do You Print Log Entries for a Given Time Range?”

How Do You Set a Date-Time Value with a Default Year in Python?

0 Comments

Problem scenario
You are printing out some dates with a Python program. You are using the datetime module and the strptime method. The year is always defaulting to 1900 because the log entries have no year designated in them. How do you seed the dates with a specific year of your choice?

Solution
Here is an example of how to seed the dates with the year 2021 (but it assumes you have a file name auth.log with dates without years in it):

import datetime
from datetime import datetime, …

Continue reading “How Do You Set a Date-Time Value with a Default Year in Python?”

How Do You Troubleshoot the Python Problem “ValueError: time data … does not match %m”?

0 Comments

Problem scenario
You are trying to parse a log file with Python. The date entries use abbreviated spellings of months — not integer month values. You get an error like this:

“ValueError: time data ‘Jul 15 06:10:32’ does not match format ‘%m %d %H:%M:%S'”

What should you do?

Solution
Replace the “%m” with “%b”.

import datetime
log_reader = open(‘auth.log’, …

Continue reading “How Do You Troubleshoot the Python Problem “ValueError: time data … does not match %m”?”

How Do You Troubleshoot the Python Problem “TypeError: ‘module’ object is not callable”?

0 Comments

Problem scenario
You run a Python program with datetime, but you get the error “TypeError: ‘module’ object is not callable.” What should you do?

Solution
Even though you may use “import datetime” at the top of the program, you should use “from datetime import datetime, timezone” at the top. If you used the syntax “datetime.datetime…”, you will need to remove one of the “datetime.” strings from the code.

…

Continue reading “How Do You Troubleshoot the Python Problem “TypeError: ‘module’ object is not callable”?”

How Do You Write a Python Program to Extract Lines of a Specific Date?

0 Comments

Problem scenario:
You have a log like this:

Sep 18 07:28:11 server1 sshd[29284]: Received disconnect from 125.52.17.109 port 46970:11: Bye Bye [preauth]
Sep 18 07:28:11 server1 sshd[29284]: Disconnected from 125.52.17.109 port 46970 [preauth]
Sep 18 07:28:11 server1 sshd[29282]: Failed password for root from 51.12.19.109 port 24844 ssh2
Sep 18 07:28:13 server1 sshd[29287]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=119.29.155.214 user=root
Sep 18 07:28:13 server1 sshd[29282]: Failed password for root from 51.12.19.109 port 24844 ssh2
Sep 18 07:28:14 server1 sshd[29282]: Received disconnect from 51.12.19.109 port 24844:11: [preauth]
Sep 18 07:28:14 server1 sshd[29282]: Disconnected from 51.12.19.109 port 24844 [preauth]
Sep 18 07:28:14 server1 sshd[29282]: PAM 2 more authentication failures; …

Continue reading “How Do You Write a Python Program to Extract Lines of a Specific Date?”

How Do You Troubleshoot a “SyntaxError: invalid syntax” Error in Python?

0 Comments

Problem scenario
You do not know why you are getting the error. The line has an “l” (a lowercase “L”) or a numeral one (“1”) in it.

You may have identified that the line of code is different in one file from another. You cannot tell what is different about your code or why you are getting this error. What should you do?

Solution
Enter vi or open Notepad++ to view the code.

…

Continue reading “How Do You Troubleshoot a “SyntaxError: invalid syntax” Error in Python?”

How Do You Troubleshoot The Message “no module named pandas”?

0 Comments

Problem scenario
From a Python command prompt or when you run a Python program, you get the error message “no module named pandas”. You have used pip and pip3 to install pandas. What should you do?

Solution
Use “python3” instead of “python” to ensure you are using Python 3.

You may also want to see this posting.

…

Continue reading “How Do You Troubleshoot The Message “no module named pandas”?”

How Do You Write a Python Program to Parse a Log File and Return Lines with the Word “Failed”?

0 Comments

Problem scenario
You want to retrieve lines from a log file with the word “failed”. How do you do this with Python?

Solution

Assuming the log file is named “auth.log”, this will work:

log_reader = open(‘auth.log’, ‘r’)
for line in log_reader:
lower_line = line.lower()
if lower_line.find(“failed”) != -1:
print(line) …

Continue reading “How Do You Write a Python Program to Parse a Log File and Return Lines with the Word “Failed”?”

What is a Recommended Practice for Handling Asynchronous Requests with Long-Running Processes with Flask?

0 Comments

Problem scenario
You are designing a Flask application with Python running some long-running processes. You want to handle the requests asynchronously. The individual requests will have a duration of more than one hour. What is the recommended way of handling this?

Solution
Use a task queue like Celery or RQ (Redis Queue).

Sources:
https://stackoverflow.com/questions/42790362/how-to-handle-long-sql-query-in-flask

Flask API app for long running process? …

Continue reading “What is a Recommended Practice for Handling Asynchronous Requests with Long-Running Processes with Flask?”

How Do You Troubleshoot “ImportError: No module named ‘pandas'”?

0 Comments

Problem scenario
From a Python command line, you get this message:

Traceback (most recent call last):
File “”, line 1, in
ImportError: No module named ‘pandas’

What should you do?

Solution
Prerequisite
You need pip to be installed. If you need assistance, see this posting.

Procedures
Run one of these commands:

pip install pandas

pip3 install pandas …

Continue reading “How Do You Troubleshoot “ImportError: No module named ‘pandas’”?”