Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [parsing]

42 questions
133
votes
9 answers

How to remove empty/blank lines from a file in Unix (including spaces)?

How do I remove empty/blank (including spaces only) lines in a file in Unix/Linux using the command line? contents of file.txt Line:Text 1: 2:AAA 3: 4:BBB 5: 6:CCC 7: 8:DDD output desired…
Michael Ellick Ang
  • 1,839
  • 3
  • 13
  • 15
76
votes
5 answers

How do I grep through binary files that look like text?

I have binary files that should be text (they're exported logs), but I can't open it with less (it looks ugly - it looks like a binary file). I found that I could open it with vi and I can cat it (you'll see the actual logs), but what I'd really…
Robyn Smith
  • 861
  • 1
  • 7
  • 5
73
votes
12 answers

Do you have any useful awk and grep scripts for parsing apache logs?

I can use log analyzers, but often I need to parse recent web logs to see what's happening at the moment. I sometimes do things like to figure out top 10 ips that request a certain file cat foo.log | grep request_to_file_foo | awk '{print $1}' | …
deadprogrammer
  • 1,661
  • 7
  • 24
  • 25
8
votes
7 answers

Get list of user-agents from nginx log

I have nginx log file, and I want to find out market share for each major version of browsers. I am not interested in minor versions and operating systems. I would like to get something like this: 100 IE6 99 IE7 20 IE8 200 FF2 300 FF3 I know how…
Željko Filipin
  • 223
  • 1
  • 3
  • 12
7
votes
2 answers

Parseable NGINX accesslog files with delimiters

The default NGINX format is this: log_format combined '$remote_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent"'; Which is a bit hard to parse. I am…
Kasper Grubbe
  • 301
  • 4
  • 9
5
votes
3 answers

Discover intended Foreign Keys from JOINS in scripts

I'm inheriting a database that has 400 tables and only 150 foreign key constraints registered. Knowing what I do about the application and looking at the table columns, it's easy to say that there ought to be a lot more. I'm afraid that the current…
Jason Kleban
  • 786
  • 3
  • 8
  • 19
4
votes
4 answers

AWS - Script to create EC2 snapshots and automatically rename them

I'm currently trying to setup a script (using AWS CLI from an Ubuntu server) that will do the following: Create a snapshot of every existing volume. Those volumes already have a NAME tag (Like SERVER1, SERVER1DATA, SERVER2, SERVER2DATA etc). Rename…
Nicolas
  • 53
  • 1
  • 1
  • 4
4
votes
2 answers

PowerShell - Add multiple strings to the same element in an array

So I have a array that I need to search through line by line and divide up by interfaces. My code loops through this file line by line. I want to split up the interfaces by the "!" character and add the the strings to an element in an array so I can…
runcmd
  • 255
  • 2
  • 4
  • 11
4
votes
1 answer

How to handle many daily emails from cron efficiently?

With each new server comes at least one more daily crontab email. The FreeBSD servers even send two: a daily and a security run output. Thanks to my obsession with virtualisation the number of emails I need to read (and then often discard without…
drumfire
  • 1,699
  • 3
  • 14
  • 21
4
votes
1 answer

Parsing iometer results

How do your visualise your iometer results?
romant
  • 526
  • 5
  • 21
3
votes
0 answers

I am looking for software that can parse emails and expect emails within a grace period

I manage a diverse set of backup systems, the only commonality is email notifications, There status being Failed, Success, Success with exceptions and unknown(where the email isn't sent). Case 1) Expect email from system3@example.net every day, at…
Jacob Evans
  • 7,636
  • 3
  • 25
  • 55
3
votes
1 answer

Is there a way to determine the log format if given a ".log" file?

Is there a way to determine the kind of log (so that it can be parse correctly) if I have no prior information about the type (for eg. syslog, apache log, IIS log) of log it is? I am trying to write a Grok filter for the logs but I have no idea what…
shruti gupta
  • 33
  • 1
  • 4
3
votes
2 answers

Scripts or tools to parse email message to remove attachment of particular name

I have configured Redmine email integration and while it's awesome, a major annoyance is that people have signatures that include a company logo, which then gets posted to every ticket they update via email. I know this is not a perfect solution but…
tacos_tacos_tacos
  • 3,220
  • 16
  • 58
  • 97
3
votes
3 answers

How can I limit the output of the whois command?

How do I get only the owner/orgname from a PTR whois lookup? I don't need all the other entires, just the part about ownership (google in this case). Edit /usr/bin/whois 74.125.236.52 or any ip get OrgName # # Query terms are…
Rajeev
  • 251
  • 4
  • 10
2
votes
2 answers

Ansible parsing a JSON output

I'm using URI in order to get a JSOn output to a register called "vchosts" TASK [debug]…
Vidanez
  • 21
  • 1
  • 2
1
2 3