1Henning Heinz  27.08.2010 12:38:56  Low RAM means lower values are better in this case

But if you heavily use XPages then you will need higher values someday. Afaik Domino will not use more than 2 GB of memory on 32 Bit so the 1,7 GB are not too bad. It seems that you have a memory leak which basically means that something in your Domino setup is eating up your memory until you have no more. If you would go 64Bit your machine could run a bit longer but if it is a memory leak you will run out of memory sometime too.

On Linux 64 Bit Domino 32 Bit will have a full 2 GB range which is better than what you currently have. But if you have 500 users on a machine those will have to work with 2 GB either so memory should not really be a problem.

I run some Domino server on Debian Linux and they are running fine. Running out of memory imho hardly ever has something to do with the OS you use (supported or unsupported).

2Sean Cull  27.08.2010 12:45:36  Thanks Henning

There is nothing clever on the machine and it would never have that many users. There are bespoke applications but nothing very clever and certainly no bespoke java or API code.

Are you running 852 ?

3Jeremy Hodge  27.08.2010 14:00:46  Troubleshooting the Memory Leak

Sean, one of the things I learned (from Philippe Riand) through my recent foray into diagnosing my speed problem was about the YourKit java profiler. You can download a 15 day trial at yourkit.com. Install it on your linux server, integrate it into domino by by running the integration wizard, selecting server type #11, saying NO to all the options, add an extra option for listen=YOURLOCALIP:10001 (replace YOURLOCALIP with your machines local IP address_ and then take the configuration file it gives you, add it as the value of a JavaOptionsFile parameter in your notes.ini, similar to this (Your path to the options file will most likely be different):

JavaOptionsFile=/opt/ibm/lotus/notes/latest/linux/YourKitOptionsFile.txt

the contest of the options file will be something like this:

-agentpath:/home/YourKit/yjp-9.0.7/bin/linux-x86-32/libyjpagent.so=listen=10.10.10.102:10001

Then if you have your firewall running (either on the box or externally) you'll need to open TCP/UDP 10001 (not sure if its tcp or udp so I opened both).

Restart Domino, then on your local machine, install the windows version of YourKit, and connect to your server's IP and port 10001, and you will be able to see what is running in the JVM, and what your memory leak using by doing some memory profiling.

Hope this helps - feel free to email or bleedyellow sametime me with questions.

Jeremy

4Henning Heinz  02.09.2010 11:59:49  Yes, I have updated 2 so far

Yes I have updated two servers to 8.5.2 ( 3 and 4 Gb RAM). The rest is waiting for the SMTP fix and language packs. I do not run on Amazon EC2 but dedicated servers. So far both seem to run fine (Debian Lenny / Squeeze) and free memory is plenty.

What a pity that you had to gov back but indeed it was difficult to follow this topic because your server was down quite often.

If you updated from a previous version you could check if your notes.ini contain settings that are obsolete. Also Ubuntu runs fine in a VM like the free VirtualBox so you could test without killing your live server.

I am sorry that I have no concrete advise for you. When I updated I deleted many files and directories leaving theold Domino installation at the minimum level (basically only keeping my nsf and the server id and notes.ini).

Jeremy's way to debug this issue reads quite interesting. Thank you.

5Sean Cull  02.09.2010 21:24:43  on 852 but not ec2

Hi Henning, sorry about the outages, I'm on 852 on a physical server but the ec2 server still missbehaves ( { Link } )

I have been mad busy but I am going to try working on it tomorrow again. Thought I had it sussed but no

Sean

6Rasmus  04.10.2010 9:32:32  Problem solved?

Did you solve this problem?

I'm now experiencing the same problem on my Domino server, and are kinda clueless atm.

7Sean Cull  04.10.2010 10:15:54  No

@Rasmus,

no, we moved to a much larger server ( part of a pre planned project ) but that didn't solve the problem. I also get the problem on both Centos and Ubuntu.

I think it is application related but I haven't had the chance to did into it further.

Our current workaround is to restart the Domino server daily using a program . Program name = server Command line = -c "restart server" ( with quotes )

The problem usually stops Xpages but not general HTTP stuff. Sometimes it stops the whole server.

When I get time I will look to see if any of the http threads are staying open for long periods of time using

tell http show thread state

Possible issues include an endless loop ?

What OS are you running and do you have bespoke applications or just commercial / Open NTF applications ?

Sean

8d jackson  14.10.2010 17:01:07  8.5.2 errors

we are having the same problems on both 32 and 64 machines running 8.5.2 which are both craching regularly

we also get 'java.lang.OutOfMemoryError: native memory exhausted' along with your 'HTTP JVM: The XPages runtime engine faced an OutOfMemoryError '

does anyone know if this is a specific issue with 8.5.2

d jackson

9peter  10.12.2010 21:29:13  8.5.2 errors

Check HTTPJVMMAXHEAPSIZE in notes.ini. After upgrade to 8.5.2 it reverts back to 64MB from 256MB.

10Sean Cull  13.12.2010 9:22:23  HTTPJVMMAXHEAPSIZE

@9 Thanks Peter, I had already tried every combination.

We are trying a hotfix at the moment, there are some known issues but IBM will only deal with users on a one by one basis.

Sean