C-ICAP cannot allocate memory

This forum is for all copfilter support related questions in English.

C-ICAP cannot allocate memory

Postby mgnwong » 21 May 2013 23:05

Hi everyone,

IPCOP 2.0.91beta3 was working fine for months with Green and Red setup, when suddently today C-ICAP is hanging on the http requests (I'm able to ping outside though). When I disable the C-Icap service, internet navigation returns to normal but of course without the url and blacklist filtering.

Checked the file /var/log/copfilter/default/opt/c_icap/var/log/system.log for more details and this is what i found:

Mon May 20 16:25:48 2013, 16328/3076396736, Environment open failed: Cannot allocate memory
Mon May 20 16:25:48 2013, 16328/3076396736, Environment open failed: Cannot allocate memory
Mon May 20 16:25:48 2013, 16328/3076396736, Environment open failed: Cannot allocate memory
Mon May 20 16:25:48 2013, 16328/3076396736, Environment open failed: Cannot allocate memory
Mon May 20 16:25:48 2013, 16328/3076396736, open db domains.db: Cannot allocate memory
Mon May 20 16:25:48 2013, 16328/3076396736, open db urls.db: Cannot allocate memory
Mon May 20 16:25:48 2013, main proc, Child 16328 did not exit normally.Mon May 20 16:25:48 2013, main proc, signaled with signal:11
Mon May 20 16:25:48 2013, main proc, Possibly a term signal received. Monitor process going to term all children
Mon May 20 16:25:48 2013, 16329/3076396736, Environment open failed: Cannot allocate memory
Mon May 20 16:25:48 2013, 16329/3076396736, Environment open failed: Cannot allocate memory
Mon May 20 16:25:48 2013, 16329/3076396736, Environment open failed: Cannot allocate memory
Mon May 20 16:25:48 2013, 16329/3076396736, Environment open failed: Cannot allocate memory
Mon May 20 16:25:48 2013, 16329/3076396736, open db domains.db: Cannot allocate memory
Mon May 20 16:25:48 2013, 16329/3076396736, open db urls.db: Cannot allocate memory


Seems domains.db and urls.db are the ones keeping c-icap from starting because of a memory issue. Has this happenned to anyone before?

I've seen a similar problem with the "Child did not exit normally" error message in this post, but the solution was a reinstall of copfilter.
viewtopic.php?f=9&t=696

Quickly checked the memory limits and the current memory usage

root@ipcop:/var/log/copfilter/default/opt/c_icap/var/log # ulimit -a

core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 7553
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 7553
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

root@ipcop:/var/log/copfilter/default/opt/c_icap/var/log # cat /proc/meminfo

MemTotal: 970968 kB
MemFree: 496196 kB
Buffers: 7712 kB
Cached: 256956 kB
SwapCached: 2740 kB
Active: 323112 kB
Inactive: 125724 kB
Active(anon): 173044 kB
Inactive(anon): 11160 kB
Active(file): 150068 kB
Inactive(file): 114564 kB
Unevictable: 0 kB
Mlocked: 0 kB
HighTotal: 73672 kB
HighFree: 232 kB
LowTotal: 897296 kB
LowFree: 495964 kB
SwapTotal: 262136 kB
SwapFree: 254232 kB
Dirty: 1624 kB
Writeback: 0 kB
AnonPages: 182160 kB
Mapped: 12068 kB
Shmem: 32 kB
Slab: 19324 kB
SReclaimable: 5444 kB
SUnreclaim: 13880 kB
KernelStack: 752 kB
PageTables: 676 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 747620 kB
Committed_AS: 362208 kB
VmallocTotal: 122880 kB
VmallocUsed: 1760 kB
VmallocChunk: 79936 kB
DirectMap4k: 8184 kB
DirectMap4M: 901120 kB

root@ipcop:/var/log/copfilter # df -h
Filesystem Size Used Avail Use% Mounted on
tmpfs 475M 44K 475M 1% /dev
/dev/sda1 755M 638M 79M 89% /
/dev/sda2 146G 14G 126G 10% /var/log
tmpfs 475M 4.0K 475M 1% /tmp
shm 475M 0 475M 0% /dev/shm

It doesnt seem like a memory allocation problem. Do you have any ideas on this?
Any help or suggestion on where to start is greatly appreciated.

Thanks.

mgnwong
mgnwong
 
Posts: 3
Joined: 11 Oct 2012 19:37

Re: C-ICAP cannot allocate memory

Postby FischerM » 23 May 2013 13:28

Hi!

First thought:

Stop/start 'C-ICAP' from console in debug mode and post output here.

Code: Select all
/var/log/copfilter/default/opt/c_icap/etc/init.d/copfilter_c-icap stop

Code: Select all
/var/log/copfilter/default/opt/c_icap/etc/init.d/copfilter_c-icap debug


HTH
Matthias
FischerM
Site Admin
 
Posts: 544
Joined: 09 Dec 2009 19:24
Location: Rheinbach

Re: C-ICAP cannot allocate memory

Postby mgnwong » 08 Jun 2013 00:31

Hello FischerM,

Thanks for the reply, and sorry for not replying before. I thought I would receive an e-mail when someone replied, but now I see it wasn't enabled in the Forum User Control Panel.
I started 'C-ICAP' in debug mode and got an output. I waited for about 15 minutes and the file got considerably big.
Don't really know how to attach a file in a post in this forum, so i uploaded the output to this link: https://www.dropbox.com/s/3aewzioo139e2ss/Debug.txt

Seems by the resulting output everything is Okay. Any thoughts?

Thanks in advance.

mgnwong
mgnwong
 
Posts: 3
Joined: 11 Oct 2012 19:37

Re: C-ICAP cannot allocate memory

Postby karesmakro » 09 Jun 2013 14:13

Hello mgnwong,

this debug output does not provide c-icap with activated url filter! Could you please activate the url filter with such a few categories and repeat the debug mode (there is no need to wait 15 minutes in debug mode, it produces a lot of output ... post the start of c-icap, to see what is loaded (until the "Waiting for a request" message) and the last 50 lines, where c-icap crashes)

regards
karesmakro
Site Admin
 
Posts: 1280
Joined: 09 Dec 2009 21:17

Re: C-ICAP cannot allocate memory

Postby mgnwong » 11 Jun 2013 00:29

Hi Karesmakro,

Im sorry, you are right. This is the output we are looking for: https://www.dropbox.com/s/y432v4jy5qhnsn3/debug2.txt

The first error is in
Code: Select all
line 517: open db domains.db: Cannot allocate memory

and then theres
Code: Select all
line 652: Child 678 died ...

right after trying to create the shared memory.


mgnwong
mgnwong
 
Posts: 3
Joined: 11 Oct 2012 19:37

Re: C-ICAP cannot allocate memory

Postby karesmakro » 12 Jun 2013 13:42

Hi,

can you tell me please the c-icap version you are using? Is it the latest with clamav 0.97.8?
If not, try one of the newer c-icap versions. I know, there was some bugs in it.
Did you already checked your memory with memtest?
I saw, that you are using a lot of categories from the url filter. Did you tried to decrease the choice of categories?

regards
karesmakro
Site Admin
 
Posts: 1280
Joined: 09 Dec 2009 21:17


Return to English Copfilter Support

Who is online

Users browsing this forum: No registered users and 1 guest

cron