XYSeries and out of memory issue

[vrsarav]

Hi All,
I'm trying to add huge data more than a million to XYSeries[]. Means i need multiple XYSeries with each having millions of data. The data comes from a file.
Whenever the file size is about 40 MB it fails with a out of memory error. I just came up with a test code to verify this.

Code: Select all

double time = 50.0;
for (i = 0; i < totalSeries; i++) {
			xySeries[i] = new XYSeries("Test"+i,false);
			for (j = 0; j < noOfTimes; j++) {
				xySeries[i].add(time, i,false);
			}
		}

"noOftime" variable will be multiples of 24000
"totalSeries" can be from 1 to 128

Our ideal set up has totalSeries as 72 and noOftime as 24000*10. It was good for this set up. But when i increase noOfTime to 24000*15 it fails with out of memory.

My Query:
Ideally i load all the data to RAM and then plot it. But as per this scenario, the memory is exponentially increasing as the data sample count increases. Lot of discussion threads suggested using partial view of data than viewing complete data, but then i have to get the data from flat file. To load partial data and keep track of rest of the data when moving forward and backward in the chart might be challenging. So any idea how can i approach this scenario? Btw, I'm using XYDataset and XYLIneAndShapeRenderer for the chart.

Thanks,
Saravanan

[paradoxoff]

My first try would be to replace the XYSeriesCollection with something that consumes less memory. Try the DefaultXYDataset which is based on arrays.
Second, I would omit the XYLineAndShapeRenderer and go for the SamplingXYLineAndShapeRenderer described at the end of this thread (which is a recommended reading anyways)

[vrsarav]

Thanks Paradoxoff. You've always been a tremendous help to this group.
I have to look into defaultXYDataset yet. Let me try that and let u know.
As far as SamplingXYLineAndShapeRenderer i did have a quick later earlier. My requirement is to have have all the data points visible, but with SamplingXYLineAndShapeRenderer i'm not sure whether thats possible and even it is it's little distorted compared to XYLineAndShapeRenderer, which looks really smooth. But i'll look into this both and will give it a try.

Thanks for u'r reply. I appreciate it.