Linux And High Io Wait

blogger /

When you look at the CPU activity of your computer, one of the parameters is the iowait. This value shows how much time your CPU wastes while it is waiting for I/O operations for complete. These include disk read/write operations, network, IPC, etc. Is this behavior a problem and, if so, what causes it and how to fix it? One one of the popular Unix-related forums one “genius” wrote:

The iowait “problem” is funny. It’s like when people complain that Linux is “using all my memory”. Yeah, no shit. You should be upset if you are copying files and your computer is /not/ in 100% iowait.

In reality, 100% iowait indicates that there is a problem and in most cases – a big problem that may even lead to data loss. Essentially, there is a bottleneck somewhere in the system. Maybe one of your disks is getting ready to die; or, perhaps, the NIC firmware is having problems with the latest kernel upgrade you installed. The troubleshooting process starts with the potentially more serious possibility: bad disk.

Take a quick look at /etc/messages, /etc/dmesg, /etc/boot.log and any other system log files. You are looking for disk I/O errors, failed read/write operations, bad sectors – anything that indicates a hardware problem with a disk. If you don’t find anything, look for IRQ and disk controller errors. Also look for memory errors and kernel panics. The three most likely culprits of high iowait are: bad disk, faulty memory and network problems.

If you still see nothing relevant, it is time to test your system. If possible, kick all the users off the box, shut down Web server, database and any other user application. Log in via command line and stop XDM.

Open three shell windows: run “top” in one, “iostat -x 1? in the other and “find /etc -type f -print” in the third. Make sure you can see all three windows at the same time. This is a simple test that should generate some I/O activity on the system disk. Repeat this process for other disks. If you see iowait hovering near 100%, chance are you have a problem but we don’t know what it is yet. However, now we do know that network is probably not the cause.

deathstar:/ # iostat -x 1
Linux 2.6.5-7.201-default (deathstar) 12/20/08

avg-cpu: %user %nice %sys %iowait %idle
2.83 0.42 1.45 9.11 86.20

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
hda 40.63 66.34 27.45 6.04 936.50 581.23 468.25 290.61 45.32 2.42 72.16 2.22 7.42
hdc 0.01 0.00 0.01 0.00 0.03 0.00 0.02 0.00 4.02 0.00 1.17 1.17 0.00
sda 0.09 2.32 4.15 1.33 71.56 29.23 35.78 14.62 18.37 0.65 118.49 6.39 3.51
sdb 3.47 0.00 1.90 0.00 15.32 0.01 7.66 0.01 8.08 0.74 391.31 5.68 1.08
fd0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 2.00 0.00 45.00 45.00 0.00

deathstar:/ # top
top – 21:28:28 up 1:22, 2 users, load average: 0.09, 0.14, 0.16
Tasks: 77 total, 1 running, 76 sleeping, 0 stopped, 0 zombie
Cpu(s): 2.8% us, 1.3% sy, 0.4% ni, 86.2% id, 9.1% wa, 0.1% hi, 0.0% si
Mem: 508644k total, 503612k used, 5032k free, 34052k buffers
Swap: 1020088k total, 458980k used, 561108k free, 16012k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 root 16 0 640 56 28 S 0.0 0.0 0:05.14 init
2 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
3 root 5 -10 0 0 0 S 0.0 0.0 0:00.09 events/0
4 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 khelper

Next step, lets stress out your CPU but not the disks. The command below will try to create an endless zip file in /dev/null. This generates no disk activity, but loads the CPU. Continue running “top” and “iostat -x 1? in the other two windows.

cat /dev/zero | bzip2 -c > /dev/null

If you see high CPU load but low iowait, we can eliminate CPU issues, IRQ conflicts, and faulty memory. Just to be on the safe side, let’s test memory anyway:

deathstar:/ # free
total used free shared buffers cached
Mem: 508644 503504 5140 0 37036 48968
-/+ buffers/cache: 417500 91144
Swap: 1020088 516196 503892

This server has 508644Kb of RAM. Use the corresponding value for the following test:

deathstar:/ # dd if=/dev/hda2 bs=508644 of=/backups/memtest count=1050
1050+0 records in
1050+0 records out

deathstar:/ # md5sum /backups/memtest ; md5sum /backups/memtest ; md5sum /backups/memtest
04762ff36b2231aac75754ab9c1a564a /backups/memtest
04762ff36b2231aac75754ab9c1a564a /backups/memtest
04762ff36b2231aac75754ab9c1a564a /backups/memtest

The three MD5 values above should be identical. If they are not – your system has a faulty RAM chip.

When you have eliminated hardware problems as possible causes of high iowait, the next step is to review firmware and drivers. You are particularly interested in disk controller firmware: unstable performance and no error messages are the signs of a firmware problem. Try really hard to remember if you made any system changes recently, especially something that required a reboot – like kernel upgrade, for example. If this is the case, roll back the upgrade or search for upgrade firmware. You should grab a copy of Sysinfo (free 30-day trial) to help you identify makes and models of your disks, controllers, etc.

While your disks and controllers may be tip-top, your may have a problem with a filesystem. Even if you see high iowait when accessing any filesystem, you should still check out the partition where /var is mounted and swap – if there is a problem, it will manifest itself regardless of what your system is doing. But here you will run into a little problem: fsck will not scan a mounted partition and you cannot unmount /var. Let’s say these are your partitions:

deathstar:/ # more /etc/fstab
/dev/hda2 / reiserfs acl,user_xattr 1 1
/dev/hda1 swap swap pri=42 0 0

You need to fsck /dev/hda2 because this is where your /var is mounted. Download KNOPPIX or Ubuntu LiveCD, boot from CD (without installing) and “fsck /dev/hda2? from there. If everything looks clean, shut down your system, take the CD out and boot normally. The next step is to check out swap. If you just run fsck on the swap partition, it will fail:

deathstar:/ # fsck /dev/hda1
fsck 1.34 (25-Jul-2003)
fsck: fsck.swap: not found
fsck: Error 2 while executing fsck.swap for /dev/hda1

You need to disable swap on /dev/hda1 before you can scan it. Before you can do this, you need to add another swap area: you cannot run without any swap space. So, to add swap on the fly, create a swap file (1Gb in this example):

deathstar:/ # dd if=/dev/zero of=/swapfile bs=1024 count=1048576
1048576+0 records in
1048576+0 records out

deathstar:/ # chmod 600 /swapfile

deathstar:/ # ls -lash /swapfile
1.1G -rw——- 1 root root 1.0G Dec 20 22:48 /swapfile

Now you can set up and activate the new swap file:

deathstar:/ # mkswap /swapfile
Setting up swapspace version 1, size = 1073737 kB
deathstar:/ # free
total used free shared buffers cached
Mem: 508644 500996 7648 0 38912 147332
-/+ buffers/cache: 314752 193892
Swap: 1020088 521784 498304
deathstar:/ # swapon /swapfile
deathstar:/ # free
total used free shared buffers cached
Mem: 508644 502232 6412 0 39400 147392
-/+ buffers/cache: 315440 193204
Swap: 2068656 521784 1546872

Now we need to deactivate the original swap partition. This operation may take a couple minutes to complete:

deathstar:/ # swapoff /dev/hda1
deathstar:/ # free
total used free shared buffers cached
Mem: 508644 501624 7020 0 31712 10416
-/+ buffers/cache: 459496 49148
Swap: 1048568 167032 881536

The next step is to create a standard filesystem on the old swap partition, so that fsck has something to scan:

deathstar:/ # mke2fs -c /dev/hda1
mke2fs 1.34 (25-Jul-2003)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
127744 inodes, 255024 blocks
12751 blocks (5.00%) reserved for the super user
First data block=0
8 block groups
32768 blocks per group, 32768 fragments per group
15968 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376
Checking for bad blocks (read-only test): done
Writing inode tables: done
Writing superblocks and filesystem accounting information: done

The previous operation already ran fsck and so, if you see no errors, you can now re-activate your original swap space and remove the temporary swap you created:

deathstar:/ # mkswap /dev/hda1
Setting up swapspace version 1, size = 1044574 kB
deathstar:/ # swapon /dev/hda1
deathstar:/ # swapoff /swapfile
deathstar:/ # rm /swapfile
deathstar:/ # free
total used free shared buffers cached
Mem: 508644 503172 5472 0 33668 9256
-/+ buffers/cache: 460248 48396
Swap: 1020088 156300 863788

Anothe command commonly used for analyzing system bottlenecks is vmstat. The following example runs vmstat five times at 2-second intervals:

deathstar:~ # vmstat -S M 2 5
procs ———–memory———- —swap– —–io—- –system– —-cpu—-
r b swpd free buff cache si so bi bo in cs us sy id wa
0 0 15 174 70 58 0 0 189 50 5 6 1 3 94 1
0 0 15 174 70 58 0 0 0 0 1005 35 4 0 96 0
0 1 15 174 70 58 0 0 0 258 1515 45 0 6 88 7
0 0 15 173 71 58 0 0 0 194 1083 24 0 1 83 16
0 0 15 173 71 58 0 0 0 0 1003 19 0 0 100 0

Explanation of vmstat columns:

(a) procs is the process-related fields are:

* r: The number of processes waiting for run time.
* b: The number of processes in uninterruptible sleep.

(b) memory is the memory-related fields are:

* swpd: the amount of virtual memory used.
* free: the amount of idle memory.
* buff: the amount of memory used as buffers.
* cache: the amount of memory used as cache.

(c) swap is swap-related fields are:

* si: Amount of memory swapped in from disk (/s).
* so: Amount of memory swapped to disk (/s).

(d) io is the I/O-related fields are:

* bi: Blocks received from a block device (blocks/s).
* bo: Blocks sent to a block device (blocks/s).

(e) system is the system-related fields are:

* in: The number of interrupts per second, including the clock.
* cs: The number of context switches per second.

(f) cpu is the CPU-related fields are:

These are percentages of total CPU time.

* us: Time spent running non-kernel code. (user time, including nice time)
* sy: Time spent running kernel code. (system time)
* id: Time spent idle. Prior to Linux 2.5.41, this includes IO-wait time.
* wa: Time spent waiting for IO. Prior to Linux 2.5.41, shown as zero.

If you failed to identify the cause of the iowait problem, you should consider the possibility that there is no problem: perhaps your system is handling extra load and running short on resources. Take a look at the running processes and see what’s eating up memory. Perhaps you upgraded an application and now it is using more RAM, which leads to high swapping, which leads to high disk activity, which leads to high iowait.

The solutions are simple:

1. Install more RAM
2. Move swap to another disk or – even better – move it to another disk on a separate controller.
3. Move user applications to another disk/controller and specify default log locations outside of the system disk.

Red hat linux introduced lokkit first

blogger /

The IT sector has proved to be extremely rewarding and monetarily beneficial to a large number of people. Understanding these advantages there are a large number of people who tend to be more inclined to join this booming industry. There is no dearth of job opportunities here as it is one of those industries that have gained momentum not just in a few select countries but across the globe. The job opportunities for people are vast as they can join retail and manufacturing departments, BPO’s that are thriving today, IT industries etc. The fact that mechanisms of the IT sector and knowledge of the same is essentially required in our daily lives as well, makes it a sector that has even more demand.

Understanding the benefit of this industry, there are a number of institutes that train people to join the IT sector. One such institute is jetking. Jetking is an acclaimed institute and is considered to be the best in the field of IT training. This is an institute that has tied up with Red hat linux , providing numerous benefits to those who train for this renowned institute. This gives a cutting edge to students passing out from jetking as the IT sectors prefer students from this esteemed institute as their primary preference.

The Red hat linux is extremely beneficial and has undergone a name change since its establishment in 1993. The year 2004 saw this shift taking place and red hat linux was renamed as red hat enterprise linux. The enterprising unit also benefitted a great deal after this change was made and has been progressing ever since. Linux’s saw a number of changes after this linux was introduced. This was the first linux that brought lokkit into picture making configuration extremely convenient and one of the most prominent computer tools.

The Red hat linux is extremely simple to use and works fast as well. Installing the same is also extremely convenient. It uses the most favored and the best systems and options available and maintains pretty high standards. Another reason why it is preferred over many others is because it presents itself in a very simple manner as well.

Top Ten Linux Interview Questions

blogger /

By having the perfect answer the Top Ten Linux Interview Questions you can ace any Linux Interview. I’ve been working as a Freelance Linux Consultant for the past seven years. Over all those years, I’ve been in more than a dozen of roles, attended tens of face-to-face interviews and probably close to a hundred of phone interviews.

Few painfully obvious things struck me during my time:

1) Your encyclopedic knowledge of Linux is far more likely to get you a job than your actual problem solving skills.
2) Technical questions asked during phone interviews and even face-to-face interviews tend to be fairly basic. Employers rarely ever go into detailed technical scenarios.
3) You will be asked almost the same technical questions at almost all interviews you attend.

This means that even if you feel your knowledge of Linux is limited at the moment, make sure to know detailed answers to the following Top Linux Interview Questions and you double your chances of getting the job.

You are virtually guaranteed that at least half of the questions asked will sound very similar to the following (I tried to provide the simplest ways to answer these as well, for full answers please refer to http://www.linuxinterviewquestions.com):

1) What is the difference between TCP and UDP?

The basic difference is that TCP establishes a connection before sending data and this allows it to control the dataflow and guarantee that all packets get delivered. UDP simply chucks datagrams onto the wire and if some get lost or arrive in bad order there’s no way to request a resend. However UDP has low network overhead so some services such as DNS resolution, SNMP, DHCP, RIP and VOIP use UDP for its speed and any errors are usually dealt with on the application layer rather than network layer.

2) What is the TCP hand shake?

TCP requires three packets to set up a socket connection, before any user data can be sent. This is called the tree way TCP handshake. First the requester sends a SYN packet and expects a SYN-ACK packet, to which the initiator replies with ACK packet plus the first chunk of user data. From there on the TCP connection is established and two sides exchange user data using features such as message acknowledgment, retransmission and timeout.

3) How does DNS resolution work?

A client application requests an IP address from the name server usually by connecting to UDP port 53. The name server will attempt to resolve the FQDN based on its resolver library, which may contain authoritative information about the host requested or cached data about that name from an earlier query. If the name server does not already have the answer, it will turn to root name servers to determine the authoritative for the FQDN in question. Then, with that information, it will query the authoritative name servers for that name to determine the IP address.

4) What is an MX record?

MX record numerically ranks the mail servers you would prefer to receive email for a domain. The MX record with the lowest number is preferred over the others, but you can set multiple email servers with the same value for simple load balancing.

5) Describe Linux boot-up sequence

BIOS reads the MBR where Boot Loader sits, Boot Loader reads Kernel into memory, Kernel starts Init process, Init reads inittab, executes rc.sysinit, the rc script than starts services to reach the default run level and once this is done the last thing that gets run is the rc.local script.

6) What is an INODE?

All files have its description stored in a structure called ‘inode’. The inode contains info about the file-size, access and modification time, permission and so on. In addition to descriptions about the file, the inode contains pointers to the data blocks of the file.

7) How do you search for a pattern and than replace it in an entire file?

Using Sed or in Vi editor the search usually involves character ‘s’ slash the pattern to be searched, slash the pattern to replace it with, slash ‘g’ which stands for entire file.

8) How do you list and flush all IPtables?

Using the iptables command with -L switch first to see all the rules currently present in memory and than -F to flush them.

9) How do you list compiled-in Apache modules?

Run the httpd daemon as a command with -l parameter.

10) What is a zombie?

Zombie processes can be identified in the output of ‘ps’ by the presence of ‘Z’ in the STAT column. Zombies are child processes whose parent process died

without reaping its children. Zombies can’t be killed with the usual KILL signal.

Lamp, Lamr.what – Your Options Are As Wide As The Linux Web Hosting Service Itself

blogger /

So, whats your need LAMP, LAMR or any other variety of Linux web hosting services?
Need explanation? Right then. Here we go.

They are the solution stacks that enable any Linux web hosting service to effectively fructify an effective combination of computer software with database and server.

LAMP, the most popular of the lot, comes with following specifications:

L: Linux
A: Apache
M: MySQL
P: PHP/Perl/Python

While LAMR expands to,

L: Linux
A: Apache
M: MySQL
R: Ruby on Rails

For other variants, the meaning of essentialities gets accommodated with certain alterations like M instead of MySQL means Mod Perl or Mod Python. P instead of PHP/Perl/Pyhton designates PostgreSQL (another form of data base language).

Bur then whether LAMP, LAMR or any other varieties, Linux web hosting services constitute meaning for millions world-wide in more ways than just by their names. The credentials lie in the effectiveness of OpenSource programming architecture.

The OpenSource architecture offers a secure and stable solution for hosting a website across the span of internet and the ever-so-dreadful threats of spams, viruses, and malwares. Thanks to the stringent coding regime and regular reviews conducted by the peers world-wide. For every programming loophole (if there is any what so ever) there lies a surety of immediate cure. The entire condition directs towards one statement: Linux web hosting is safe. And several web hosting service providers world-wide are taking credits for superior hosting services by successfully implementing LAMP/LAMR or other OpenSource Linux-based frame work to their clients in ned of business or personal web hosting solution.

Netcraft (the prominent British Internet monitoring company) says eight of the ten most reliable web hosting companies in the world run Linux on their web servers. However, it is up to you to believe in the credentials or discard them.

Burgeoning population of Linux web hosting service providers do have certain points to prove on the positives of this robust web hosting service variety of course.

Learn more Advantages of Using Linux Dedicated Server

blogger /

Linux servers are best for the people who are looking for the best service at a reasonable price. Once you start using Linux servers, you will find that they are the perfect way for running a business or website.

There are many people who think twice when they hear about Linux. This is because most of them do not have any knowledge on this amazing software. If you now how to use it in the right way, you will be able to obtain many benefits form it. Most people also have a general rule to stick to a brand or product they know. If you are one such person, a Linux dedicated server could be the best option for you to go though. These servers will not only help you to save your hard-earned money, but also give you more stability than other options available in the market today.

Using Linux servers will also give more security to your website. Limited use is one such advantage that plays an important role in the security of your site. This will help you to stop users from adding spyware and viruses your website. With the help of these dedicated servers, you may not have to use a graded virus protection for your site. The server providers will also take care of all the simple and major problems that may occur on your website.

When you are using dedicated servers, of Linux, you will get immediate assistance if any problem occurs. You just have to contact the customer support desk of the service provider and tell them about your problems for getting quick and effective assistance. Apart from these, there are also many other benefits, which you can enjoy by using dedicated servers. As you will have individual control on your website, you can also easily modify the content or products on your site whenever required. This is also one of the main reasons why most of the website owners prefer dedicated servers.

Although Linux Dedicated Server are new in the market, they where able to obtain a great reputation within a short time. This is mainly because of the quality of the products and services they provide. When your website is running on the net with the help of this dedicated server, you do not have to worry about any problems.

Author Resource:

The Author is a professional writer, presently writing for Cpanel dedicated servers and Cheap Dedicated Hosting.