[WBEL-users] High loads of CPU all of a sudden

[Scott Silva]

Alon spake the following on 8/29/2006 12:21 AM:
> Hi All,
>  
> I'm runnig WB3.0 Respin 3 on a DELL 1850 PE machine.
> I have a single website that was running fine till few days ago.
> I'm getting high CPU loads randomly and I can't figure out what is the
> cause.
> I didn't detect any attacks from outside of the server, so this is of
> course confusing.
>  
> I decided to look into the File system as a possible problem area.
> I tried to run fsck and got the following:
>  
> / # fsck
> fsck 1.32 (09-Nov-2002)
> e2fsck 1.32 (09-Nov-2002)
> /dev/sda2 is mounted. 
>  
> WARNING!!!  Running e2fsck on a mounted filesystem may cause
> SEVERE filesystem damage.
>  
You should really pay attention to warnings like this!!

> Do you really want to continue (y/n)? yes
>  
> Group descriptors look bad... trying backup blocks...
> fsck.ext3: Invalid argument while checking ext3 journal for /
>  
> / #
> ----------------------------------------------------------------------------------------------------------------------
> After running this and getting the error above,. I can't go anywhere:
> See the following lines that came right after I ran the fsck:
>  
>  
> 
> / # mount
> -bash: mount: command not found
>  
> / # su -
> su: error while loading shared libraries: libpam.so.0: cannot open
> shared object file: No such file or directory
>  
> / # whoami
> -bash: whoami: command not found
>  
> / # uptime
> -bash: /usr/bin/uptime: No such file or directory
>  
> / # cd
>  
> /root # whoami
> -bash: whoami: command not found
>  
> ----------------------------------------------------------------------------------------------------------------------
>  
>  
> Now that can't be good.
>  
> Any suggestions? pointers?
> I have a live production server running,.. but I'm scared if something
> will happen to it.
> I can format a new server and transfer all the data over, but do I need
> do this?
>  
> Any thoughts about this will be appreciated.
>  
> Thanks,
>  
> -Alon.
Can you have this server down while you work on it?
You have a few options.
Try to fix it.
Reinstall to this hardware AFTER a good backup.
Install a new server and migrate over.

If you need to do keep running, but do not have extra hardware, you could get
VMWare for free now, and install a new server as a virtual machine on
different hardware (like a workstation or something), migrate the data over,
and now you have a temporary server in place as you work on the original.

But backup quickly, just in case!

-- 

MailScanner is like deodorant...
You hope everybody uses it, and
you notice quickly if they don't!!!!