oracledba.ru

[jf]

Egor,

I've left some posts before and I can see that you implemented some of the suggestions. Thank you very much for being so responsive.

Now, I have used orasrp in a performance improvement project and have success with it. In this last project that I did, I ran into a situation where it was taking an incredibly long time to generate the .html output.

The file is 40+Million lines, > 2GB in size.

First, I tried it on my laptop (1.6GHz, 1GB RAM, WinXP Pro, lots of virtual memory). I verified that psyco was on. The job ran for 8 hours, did not encounter any problems, and then I killed it.

When I got home, I ran it again overnight. Didn't finish. I also ran it on another machine (WinXP Pro, psyco on, 1.2GHz, 512 RAM, 2GB virtual memory). This one ran for 2 days and never finished. Before I killed it, I can see that it was consuming *very little* CPU but there was a *lot* of disk IO. Memory use would go up and down in the 400MB-500MB range.

So then I brought the file to one of my old Solaris machine -- Ultra 60, 2 CPUs, 1.5GB RAM. First I gunzipped the file and ran it. It ran overnight and didn't finish. While it was running, I did a truss and I can see that it was reading the file in 8K chunks. So it was definitely doing something but it was taking incredibly long. I could see the disk light blinking like crazy. The Solaris prstat also showed CPU at around 10%.

Then, I had this idea that maybe, what is causing it was that the reads were very slow. I happened to have a gzipped format of the file so I did this:

gunzip -c tracefile.trc.gz | ./orasrp --stdin > tracefile.trc.html

After that command, I can see via truss that reads from stdin was at 5K but the screen was scrolling much faster. prstat shows CPU at 40-70%. gunzip was sleeping most of the time probably because orasrp could not keep up with it. I was also doing a truss on gunzip and it would wake up once in a while to uncompress some more.

This was uncouraging. I think the CPU used by orasrp was now up because now it was no longer being held back by reads from disk... essentially, the pipe to stdin was making orasrp read from memory!

So I let it run like that and it finished in 4+ hrs... still long but it finished.

Now, I said earlier that this file is 40+Million lines long, > 2GB in size. I have anoter file that if 800+MB, 11+Million lines long. orasrp on Windows processed that in 6+minutes. So the problem file is 4x bigger but the time it takes to run is so much more... it won't even finish in most situations.

Any comments? Has any one encountered this before?

Egor, if you want the trace file, we can take this off line.

Thanks.

jf

[jf]

btw, orasrp version is 2.1

[egorst]

[jf]

[egorst]