How to watch logs in real time via Terminal?

Question

This is a basic question - in the movie "The Social Network" there are several scenes when the young Facebook staff is watching the PHP/Apache server logs on in their TERMINAL in real time.

I'm familiar with how to do this in a RUBY/RoR environment - but with a standard LAMP Apache/PHP environment, how to do actively monitor your server's actions in real time? I'm guessing there's an easy way to do this in Terminal.

I haven't seen this movie. Does anyone have a screenshot or a clip which shows the details of what the Facebook staff are doing in the movie? — Stefan Lasiewski, Oct 19 '10 at 23:02

score 48 · Answer 1 · answered Oct 19 '10 at 09:42

48

maybe they use tail -f on the access log?

answered Oct 19 '10 at 09:42

Dror

601
5
10

2

Wouldn't this get unmanageable really fast for large sites like Facebook where there are thousands of requests every second? – Vilx- Oct 19 '10 at 09:49
I assume so, but in that case viewing the live access log will not help in anyway at all... – Dror Oct 19 '10 at 09:53
3

tail -f can always be filtered through grep if you're looking for specific requests... that can reduce the volumes – Mark Baker Oct 19 '10 at 09:58
3

`tail -f` is the no. 1 reason for putting at least minimal Cygwin on a windows box! – Daniel Earwicker Oct 19 '10 at 10:21
1

@Daniel I think GnuWin32 also has a tail command – Jader Dias Oct 19 '10 at 10:51
1

for that speed problem ... `tail -f /log/any.log | while read line; do echo $line; sleep 3; done` but this wouldn't be "realtime" anymore – theist Oct 19 '10 at 13:58

score 17 · Answer 2 · edited Nov 08 '15 at 21:00

17

Or even something like logstalgia ;-)

edited Nov 08 '15 at 21:00

dreftymac

453
6
15

answered Oct 19 '10 at 09:48

weeheavy

4,039
1
27
41

3

Wow that's awesome. And it might be more then just eye candy. – Stefan Lasiewski Oct 19 '10 at 23:39

score 8 · Answer 3 · answered Oct 19 '10 at 10:10

8

or install and use multitail to specify filters and coloring :)

answered Oct 19 '10 at 10:10

onik

997
3
7
20

score 6 · Answer 4 · answered Oct 23 '10 at 16:28

If you are looking for a real-time Apache log viewer and analyzer, I would definitely recommend GoAccess.

http://goaccess.prosoftcorp.com/

You just run it as (no conf needed):

goaccess -f /var/log/apache2/access.log -s -b

score 3 · Answer 5 · answered Oct 16 '13 at 23:01

3

Another thing you can try is the "watch" command which can repetitively run a grep command on any log file you want. Using approriate watch command switches you can even highlight changes in log files in real time.

For example:

watch -d -n5 uptime

answered Oct 16 '13 at 23:01

mk_gocs

31
3

score 2 · Answer 6 · answered Oct 16 '13 at 22:24

2

GoAccess 0.6.1 should be at least one of the tools in your bag. Open Source. Fast and STABLE.

For Rhel/ Centos 6.4

yum install glib2 glib2-devel glibc make geoip
yum install ncurses-devel
Download GoAccess - http://goaccess.prosoftcorp.com/download
Untar it and cd into it
./configure
make
make install

Finally: goaccess -f /var/log/httpd/access.log (or wherever your access logs are [maybe even /etc/httpd/logs/access_log]

answered Oct 16 '13 at 22:24

Chris

149
5

geoip make error on CENTOS 6 – Mike Castro Demaria Jan 28 '15 at 16:28

score 1 · Answer 7 · answered Oct 19 '10 at 15:49

1

This is a basic question - in the movie "The Social Network" there are several scenes when the young Facebook staff is watching the PHP/Apache server logs on in their TERMINAL in real time.

For a large system such as Facebook, you got to wonder how much artistic license (aka bullsh*t) was given in the making of the movie. In an environment with thousands of systems and thousands of clusters, watching server logs scrolling on a terminal is not necessarily the #1 monitoring activity.

In an environment like that (or any production environment with several dozen nodes), you'd have a NOSC of sort with indicators, and only when something is flagged as being in a state of crapping out that one would go and actively look at the logs (probably filtered for significant events and messages.)

answered Oct 19 '10 at 15:49

luis.espinal

439
3
6

For a small enough site, watching the logs is kind of fun, and can give you a general feel for how well your expectations are being met. In any case, the question just uses the movie as inspiration, asking "How can I do something like that?" instead of "How did they do that?" – eswald Oct 19 '10 at 18:28
Well, one thing is to keep a terminal open with tail -f scrolling down. It could get very expensive bandwith-wide though. It can get very expensive CPU-wise *at the client* if you are, say, scrolling the logs on a remote terminal being displayed through VNC or Remote Desktop. Other solutions involve sending log output to syslog (and that having it send remotely to a syslog server), or even sending batches of log lines (compressed and via UDP) to a monitoring client. The later might involve loss of *some lines* since it's UDP. It might be acceptable in some conditions, though. – luis.espinal Oct 19 '10 at 19:59

score 0 · Answer 8 · edited Jul 20 '12 at 18:39

0

you could write a polling script pretty easily, jsut compare the timestamp or the filesize and if changed, do a tail -n1

edited Jul 20 '12 at 18:39

Lucas Kauffman

16,818
9
57
92

answered Oct 19 '10 at 09:54

jambox

101
1

score 0 · Answer 9 · answered Oct 19 '10 at 22:41

While I'm thrilled that The Social Network used mostly accurate jargon and terminology, some of the stuff they did just for show.

I'm no server expert but I saw Zuckerberg have a terminal open with a ping command running and I can't think of any reason he would be randomly doing that. (This was after the time that they already had the server up and running).

I did find the Python server hacking scene very amusing though.

Stefan Lasiewski · Answer 10 · 2010-10-19T23:01:43.673

0

Logtool will also make nice colorized logs. Apache can write it's logs to a central NFS server, or can write logs using syslog, and syslog can send the log data to a central syslog server.

Click on the links for more detailed explanations.

tail -F /export/syslog/log/apache/access.log | logtool

edited Oct 19 '10 at 23:01

answered Oct 19 '10 at 22:55

Stefan Lasiewski

22,949
38
129
184

score 0 · Answer 11 · answered Oct 22 '10 at 03:46

0

You could also use the realtime log viewer in something like LogLogic and filter our what you did not want to see. This way you could collect more than apache logs, you could get router logs, ftp logs, etc.

answered Oct 22 '10 at 03:46

BillRoth

57
3

score 0 · Answer 12 · answered May 06 '16 at 20:47

0

Apachetail also provides a real-time view/analysis of Apache logs by connecting directly to your Apache server from a windows desktop app. No software needs to be installed on the Apache server.

answered May 06 '16 at 20:47

Peter Jamsmenson

43
1
8

How to watch logs in real time via Terminal?

12 Answers12

Linked

Related