[WBEL-users] Problem with grep

[Robert Heller]

At Sat, 14 Oct 2006 13:01:00 +0200 Benedikt Carda <carda at two-wings.net> wrote:

> 
> Dear All,
> 
> I am a linux user since around ten years but this problem I still 
> haven't had and taking the poor results I found when researching in 
> mailing list archives and google, it seems that I am the first one ever 
> having the following problem. It is simple but not easy to solve:
> 
> The programs "grep" and "egrep" hang whenever started. "They hang" means 
> that it is not possible to kill the process ("kill -9 PID-number", and 
> "killall grep" or "killall egrep" do not show any result). As a result 

Before you 'kill' them, what state are the processes in? Are they in 'D'
state?  Are you using WBEL 3 or 4?  Are you using NFS file systems? Are
you using an SMP kernel?

It *sounds* like the programs are landing in an I/O 'hang' state, which
means you have some deeper problems.  There are several possibilities:

There is a known problem relating generally to RHEL 3 / 2.4.21 SMP
kernels, memory size, NFS, and ACLs -- the kernel runs out of a
particular sort of buffer and fails to allocate more.  This causes file
I/O operations on NFS mounted file systems to 'hang' -- only a reboot
fixes it.  It happens when the system uses enough memory to get close to
or to actually use swap space -- if you never get close to needing swap
space, you probably won't see this.  I am not sure if this was ever
fixed.  It is *only* an issue with RedHat's 'patched' kernels for RHEL 3
(2.4.21-<mumble>.EL) and only the SMP ones. Any disk I/O program is
affected: ls, grep/egrep, find, etc.

Another possiblity is almost any disk I/O *hardware* problem -- are your
100% sure you don't have an intermittent disk problem?  Check
/var/log/messages for disk I/O error reports!  Make use of the SMART
API.

> more and more <defunct> processes are listed when making a "ps aux" with 
> no possibility to get rid of these processes. Furthermore "grep" and 
> "egrep" are tools constantly used by other components of the system. 
> E.g. my backup program relies on grep, furthermore nearly every init 
> script needs grep. This results in not being able to shutdown or boot 
> the system, as when it comes to the first init script that is using grep 
> the whole init process just stands still at that point forever.
> 
> I already tried to de- and reinstall the grep-rpm. Did not work, same 
> problem. As it does not look like a hardware error for me (everything 
> else beside grep is working fine) it may be that a package that is used 
> by grep is corrupted. So what I would need as information:
> 
> * How can I get a list of RPMs that are needed by grep in order to 
> reinstall them (as there may be a corrupted file causing this).

rpm -qR grep

Also, try doing using the verify function of RPM -- this will find altered
(broken, corrupted) files:

rpm -V grep
or
rpm -Va

man rpm for all sorts of documentation.

> * Is there any possibility to have yum reinstall all RPMs of a system 
> (leaving the configuration alone of course) in order to be sure that the 
> problem doesn't result from a broken binary?

I don't think yum can be told to force re-install packages.  You may
have to use rpm directly to do that.  Run the rpm verify and manually
re-install any 'broken' RPMS.  You can use yum to 'download-only' any
RPMs you don't have on-hand (eg on your install CDs).

> * Did anyone else have this problem before and knows a solution?
> 
> Thanks in advance,
> Benedikt.
> 
> 
> _______________________________________________
> Whitebox-users mailing list
> Whitebox-users at beau.org
> http://beau.org/mailman/listinfo/whitebox-users
> 
>                                                                                            

-- 
Robert Heller             -- 978-544-6933
Deepwoods Software        -- Linux Installation and Administration
http://www.deepsoft.com/  -- Web Hosting, with CGI and Database
heller at deepsoft.com       -- Contract Programming: C/C++, Tcl/Tk