Linux palvelimena – Homework 2 – Running processes and system status

Assignment

The second batch of homework consisted of four main assignments:

1) Install munin.

2) Examine a process of your choice with tools taught in class: top, iotop, netstat, lsof, ps. Finish by killing the process.

3) Stress the system with the ‘stress’ tool. Measure the stress brought on different resources.

4) Examine data collected by munin.

Sources

Unless explicitly stated otherwise, these exercises are based on lessons in Linux palvelimena ICT4TN003-4: http://terokarvinen.com/2011/aikataulu-linux-palvelimena-ict4tn003-2

1) Install munin

man munin – “Munin is as group of programs to gather data from hosts, graph them, create html-pages, and optionally warn contacts about any off-limit values”

First thing’s first. Let’s find and install munin with:

sudo apt-cache search munin

antero@xub12:~$ sudo apt-cache search munin
munin - network-wide graphing framework (grapher/gatherer)
munin-common - network-wide graphing framework (common)
munin-node - network-wide graphing framework (node)
mailping - monitor email service availability and functioning
mumble-django - Mumble-Server web interface
munin-java-plugins - network-wide graphing framework (java plugins for node)
munin-libvirt-plugins - Munin plugins using libvirt
munin-plugins-extra - network-wide graphing framework (user contributed plugins for node)
virt-goodies - A collection of helpful virtualisation related tools

The first result looks like the correct one. Let’s install it with:

sudo apt-get install munin

antero@xub12:~$ sudo apt-get install munin
<<Wall of output ensues, here's the tail>>
Setting up munin (1.4.6-3ubuntu3.1) ...
Adding system user `munin' (UID 115) ...
Adding new group `munin' (GID 124) ...
Adding new user `munin' (UID 115) with group `munin' ...
Not creating home directory `/var/lib/munin'.
Setting up munin-node (1.4.6-3ubuntu3.1) ...
Initializing plugins..done.
munin-node start/running, process 9010
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
antero@xub12:~$

Why is it not creating home directory /var/lib/munin? I sense problems in the near future. Nevertheless, let’s assume it is now gathering data silently.

(At this juncture I restarted the computer.)

2) Examine a process of your choice with tools taught in class: top, iotop, netstat, lsof, ps. Finish by killing the process.

Now then, time to select a process to observe. I’ve been thinking of trying the Spotify Linux beta, so let’s see about installing that. I decided to choose it because of two things:

1) It’s quite heavy on p2p connections, so it might provide interesting netstat data.

2) It allows me to pipe in to some music.

Spotify has kindly provided users with detailed installation instructions, found in: http://www.spotify.com/fi/download/previews/

First we add Spotify’s repository into /etc/apt/sources.list:

antero@xub12:~$ sudo nano /etc/apt/sources.list

(must be run as su since otherwise there’s no permission to write to the file)

Added the repository in the end of the file:

## Spotify repository
deb http://repository.spotify.com stable non-free

I want to verify the download packages, so I did as instructed:

sudo apt-key adv –keyserver keyserver.ubuntu.com –recv-keys 94558F59

antero@xub12:~$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 94558F59
[sudo] password for antero: 
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --secret-keyring /tmp/tmp.lvtxBUkOTA --trustdb-name /etc/apt/trustdb.gpg --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyserver keyserver.ubuntu.com --recv-keys 94558F59
gpg: requesting key 94558F59 from hkp server keyserver.ubuntu.com
gpg: key 94558F59: public key "Spotify Public Repository Signing Key <operations@spotify.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)

Everything seems to be in order, as far as I can tell. After that the program is just a simple apt-get update and install away. Let’s look for it, and install it:

antero@xub12:~$ sudo apt-cache search spotify
spotify-client - Spotify desktop client
spotify-client-gnome-support - Transitional package for spotify-client
spotify-client-qt - Transitional package for spotify-client

antero@xub12:~$ sudo apt-get install spotify-client

<<Wall of text>>
Setting up spotify-client (1:0.8.4.103.g9cb177b.260-1) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place

Let’s start it up from the Applications menu. Here’s a screenshot of the eventual result:

top

man top: “top – display Linux tasks”

Spotify is now playing random tracks until forever. Time to see what top has to say about that. Let’s run it:

antero@xub12:~$ top

Pressing shift-m shows running processes organized by memory use, from most hungry to least. Shift-p does the same for processor time. Affixed here are screenshots of the system’s CPU and memory usage while Spotify is playing music:

It appears to using 5% of cpu resources. What a hog. Next up is the RAM usage view:

Using 3.4% of the computer’s random access memory. I’d say that’s within reasonable limits.

netstat

man netstat: “netstat – Print network connections, routing tables, interface statistics, masquerade connections, and multicast memberships”

Time to check out netstat, with the -pea flags and -n, with no sudo.

antero@xub12:~$ netstat -pena --inet

Here is a screenshot of netstat’s output for your convenience:

Spotify has 3 TCP ports listening, one established TCP connection and 3 UDP connections.

For future reference, just add -n with the other flags, resulting in “netstat -pena –inet”. As far as mnemonics go, “penan internet” is quite dumb, but still helpful.

ps

man ps: “ps – report a snapshot of the current processes”

Next up is ps, started with the waux parameters. Note how there’s no ‘-‘ before the flags – the command is from the days of yore, when hyphens were nowhere to be seen. (As taught in class.) To avoid a wall of output, we pipe ‘grep spotify’.

antero@xub12:~$ ps waux |grep spotify
antero    7715  5.2  3.4 447844 105272 ?       Sl   17:59   1:14 spotify
antero   10095  0.0  0.0   4368   836 pts/2    S+   18:22   0:00 grep --color=auto spotify

Well, the program is certainly running. Interestingly, grep is also shown, since I used it just then.

lsof

man lsof: “lsof – list open files”

I wonder which files Spotify has open. We use the ‘lsof’ command here, which, if invoked without bells and whistles, lists WAY too much output to be pasted here. Let’s grep it a bit.

antero@xub12:~$ lsof |grep spotify

Still too many lines for comfort. Let’s just add a bunch of |grep -v’s for words present in the most numerous entry types. (-v excludes lines with the keyword present.)

antero@xub12:~$ lsof |grep spotify |grep -v '.so' |grep -v 'cache' |grep -v 'pipe' |grep -v 'inode'

iotop

man iotop: “iotop – simple top-like I/O monitor”

Next up is iotop, which is used to monitor I/O activity. It’s not installed by default in Xubuntu 12.04. Here’s the installation output:

antero@xub12:~$ sudo apt-get install iotop
[sudo] password for antero: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  iotop
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 22.2 kB of archives.
After this operation, 123 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu/ precise/universe iotop i386 0.4.4-4 [22.2 kB]
Fetched 22.2 kB in 0s (37.7 kB/s)
Selecting previously unselected package iotop.
(Reading database ... 156655 files and directories currently installed.)
Unpacking iotop (from .../iotop_0.4.4-4_i386.deb) ...
Processing triggers for man-db ...
Setting up iotop (0.4.4-4) ...

‘man iotop’ tells that the program is a “simple top-like I/O monitor”, so let’s try running it without any parameters.

antero@xub12:~$ iotop
Netlink error: Operation not permitted (1)
iotop requires root or the NET_ADMIN capability.

Foiled again! Good thing we have sudo.

Odd. I would have expected Spotify’s disk cache to show up. Gazing upon iotop’s manpage I found a flag that makes it only show processes with actual I/O activity. However, with just -o the window was mostly blank, with the occasional process briefly flashing up and disappearing. I added -b, which is batch mode, with which iotop outputs a feed of activity, which eliminates the flashing. Here’s the output:

antero@xub12:~$ man iotop
antero@xub12:~$ sudo iotop -o
antero@xub12:~$ man iotop
antero@xub12:~$ sudo iotop -ob
Total DISK READ:       0.00 B/s | Total DISK WRITE:       0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
Total DISK READ:       0.00 B/s | Total DISK WRITE:       0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
Total DISK READ:       0.00 B/s | Total DISK WRITE:       0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
Total DISK READ:       0.00 B/s | Total DISK WRITE:       0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
 7718 be/4 antero      0.00 B/s  499.92 K/s  0.00 %  0.00 % spotify
Total DISK READ:       0.00 B/s | Total DISK WRITE:       0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
Total DISK READ:       0.00 B/s | Total DISK WRITE:       0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
Total DISK READ:       0.00 B/s | Total DISK WRITE:      27.23 K/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
  266 be/3 root        0.00 B/s    3.89 K/s  0.00 %  4.10 % [jbd2/sda6-8]
Total DISK READ:       0.00 B/s | Total DISK WRITE:       0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
Total DISK READ:       0.00 B/s | Total DISK WRITE:       0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
Total DISK READ:       0.00 B/s | Total DISK WRITE:       0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
Total DISK READ:       0.00 B/s | Total DISK WRITE:       0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
Total DISK READ:       0.00 B/s | Total DISK WRITE:       0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
^Cantero@xub12:~$

killall

man killall: “killall – kill processes by name”

Everything must come to an end. It’s time to kill Spotify. The obvious way would be to killall spotify. Let’s see if that works.

antero@xub12:~$ killall spotify

I no longer hear music. Also, the Spotify icon is gone from the taskbar and tray. So that’s that.

stress

man stress: “stress – tool to impose load on and stress test systems”

Enough with peace and tranquility, time for some stress. Oh, but we do need to install it first.

antero@xub12:~$ stress
The program 'stress' is currently not installed.  You can install it by typing:
sudo apt-get install stress
antero@xub12:~$ sudo apt-get install stress
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  stress
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 17.0 kB of archives.
After this operation, 76.8 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu/ precise/universe stress i386 1.0.1-1build1 [17.0 kB]
Fetched 17.0 kB in 0s (92.1 kB/s) 
Selecting previously unselected package stress.
(Reading database ... 156682 files and directories currently installed.)
Unpacking stress (from .../stress_1.0.1-1build1_i386.deb) ...
Processing triggers for install-info ...
Processing triggers for man-db ...
Setting up stress (1.0.1-1build1) ...
antero@xub12:~$

Just running ‘stress’ doesn’t do anything except print info on what the user is expected to do. It also gives an example of a 10 second test, let’s try that.

Example: stress –cpu 8 –io 4 –vm 2 –vm-bytes 128M –timeout 10s.

antero@xub12:~$ stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M --timeout 10s
stress: info: [13621] dispatching hogs: 8 cpu, 4 io, 2 vm, 0 hdd
stress: info: [13621] successful run completed in 10s

Well, that was unimpressive. The laptop became slightly noisier for a while, though.

The assignment told us to measure the stress imposed on the system, but man stress doesn’t really shed light into this. I assume munin will take care of that. In order to get a clearer reading, I’ll just run a 25 second test.

antero@xub12:~$ stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M --timeout 25s
stress: info: [13685] dispatching hogs: 8 cpu, 4 io, 2 vm, 0 hdd
stress: info: [13685] successful run completed in 25s
antero@xub12:~$

I took a break from tinkering with technology, not realizing that it might make munin graphs harder to read. Let’s see.

munin’s data

(Munin proved to be a bit too much for me to handle at this point.)

I think I ought to munin-update first.

antero@xub12:~$ munin-update
munin-update: command not found

Right. The manpage *does* list such a command. Apparently it’s not completely up-to-date. There does exist, however, munin-check.

antero@xub12:~$ munin-check
Check /var/cache/munin/www
check /var/lib/munin/datafile
check /var/lib/munin/limits
check /var/lib/munin/localdomain
check /var/lib/munin/munin-graph.stats
check /var/lib/munin/munin-update.stats
Check miscelaneous
# /var/lib/munin/datafile : Wrong permissions (664 != 644)
# /var/lib/munin/limits : Wrong permissions (664 != 644)
# /var/lib/munin/munin-graph.stats : Wrong permissions (664 != 644)
# /var/lib/munin/munin-update.stats : Wrong permissions (664 != 644)
# /var/lib/munin/plugin-state : Wrong owner (munin != nobody)
# /etc/munin/plugin-conf.d : Wrong permissions (750 != 755)
Check done.  Please note that this script only checks most things,
not all things.

Please also note that this script is very new and may be buggy.

You don’t say. Googling “Wrong permissions (664 != 644)” yields one result, a pastebin paste with similar output. Unfortunately, it is of little help. sudoing the command doesn’t help either.

Bash autocomplete does list various interesting programs though:

antero@xub12:~$ munin
munin-check           munin-cron            munindoc              munin-node            munin-node-configure  munin-run

Perhaps we need to configure a node or something. I guesstimate it might help.

According to http://manpages.ubuntu.com/manpages/natty/man8/munin-check.8.html munin-check should be run as superuser, and it checks required permissions. However, it doesn’t work for me:

antero@xub12:~$ sudo munin-check
Check /var/cache/munin/www
check /var/lib/munin/datafile
check /var/lib/munin/limits
check /var/lib/munin/localdomain
check /var/lib/munin/munin-graph.stats
check /var/lib/munin/munin-update.stats
Check miscelaneous
# /var/lib/munin/datafile : Wrong permissions (664 != 644)
# /var/lib/munin/limits : Wrong permissions (664 != 644)
# /var/lib/munin/munin-graph.stats : Wrong permissions (664 != 644)
# /var/lib/munin/munin-update.stats : Wrong permissions (664 != 644)
# /var/lib/munin/plugin-state : Wrong owner (munin != nobody)
# /etc/munin/plugin-conf.d : Wrong permissions (750 != 755)
Check done.  Please note that this script only checks most things,
not all things.

Please also note that this script is very new and may be buggy.

munin-check’s manpage says -f fixes permissions:

antero@xub12:~$ sudo munin-check -f
Check /var/cache/munin/www
check /var/lib/munin/datafile
check /var/lib/munin/limits
check /var/lib/munin/localdomain
check /var/lib/munin/munin-graph.stats
check /var/lib/munin/munin-update.stats
Check miscelaneous
# /var/lib/munin/datafile : Wrong permissions (664 != 644)
# /var/lib/munin/limits : Wrong permissions (664 != 644)
# /var/lib/munin/munin-graph.stats : Wrong permissions (664 != 644)
# /var/lib/munin/munin-update.stats : Wrong permissions (664 != 644)
# /var/lib/munin/plugin-state : Wrong owner (munin != nobody)
# /etc/munin/plugin-conf.d : Wrong permissions (750 != 755)
Check done.  Please note that this script only checks most things,
not all things.

Please also note that this script is very new and may be buggy.

Yes, it may be buggy, I get it. Others have noticed this behaviour as well: https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=834055

A braver man would go on setting permissions manually, but even running the risk of looking like a lazy student, I prefer to wait for the developers to fix their tools.

*edit* Munin does create graphs in browser-friendly form without any specific configuration, which can be found in /var/cache/munin/www

Advertisements

About a1100320

IT student, musician, gamer. Beep boop.
This entry was posted in Linux palvelimena ICT4TN003-4. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s