Chart draws slowly after a while

[Jorne]

Hi there,

First of all thanks for reading trough my post.

I already read the topic http://www.jfree.org/phpBB2/viewtopic.php?f=3&t=18592. And i know they have similar problem and they solved it they say but i can't get the Fast classes implemented into my application and like the creator says. he has written these classes specific for his application. So there are methods and variables which give errors to my application.

The problem:
- We are adding live data gotten from the engine of a vehicle to a chart created by JFreeChart. Which works nicely except for one thing. After a while the dataset is getting very big and every point/line will get redrawed everytime i call the add()-method. after a while the dataset will be so big that this will take a lot of time which will slow the entire proces of the diagnostic meaning of our application. because we won't get the live data anymore. It could be that in real life the RPM go from 1300-2500 RPM but in our application the live data jumps from 1300-2500 RPM suddenly just because in between the car is going from 1300-2500 RPM the chart will be drawing all the points in the dataset and will lose all the data between 1300-2500 RPM. I'm not sure if this makes sense but the short explanation is just after a while the chart can't follow up as fast anymore as in the beginning becuase it draws every point again and again.
Our axises you can assume they are stay static. They get a fixed range and no need to re-create these every time.

Solution:
- In my opinion there is a 'simple' solution of only drawing the last point added because the panel or the area on which the points get drawn won't change anyway. i use the method add(double x, double y). is there a possibility to do this? Keep in mind i'm a beginner in the usage of graphics and paint methods and all this from Java so try to be precise as possible in which classes i have to extend and which method i have to override.

- i also read in the above thread (link) that you have a notify option for the add()-methods. But in my opinion this is not the solution for my application because i need the data to show up live i have to now every single second what is happening in the engine. and if i understand it correctly this notify will only draw each 1000 points or something added which won't give me live data.

I really hope someone can help me here because i'm stuck with my back against the wall. I have been trying things for 3 days now and i haven't been able to come 1% closer to the solution.
Thanks for reading through my post and trying to help me.

[paradoxoff]

- What type of dataset and renderer are you using? From your post, I assume that the dataset might by an XYSeriesCollection or a TimeSeriesCollection.
- Have you read the thread that you mentioned until the end, where I have described the SamplingXYLineAndShapeRenderer?
- Do you need symbols, lines, or both?
- What is the maximum time range that you want to show with the mentioned resolution ? If it is only a couple of hours, i.e. < 10k of data points, the SamplingXYLineAndShapeRenderer and an efficient dataset implementation (NOT the ones I mentioned above) should work.

[Jorne]

Hi Paradox,

Thanks for replying i'm going to give you the best explanation on those questions as possible.

- What type of dataset and renderer are you using? From your post, I assume that the dataset might by an XYSeriesCollection or a TimeSeriesCollection.

Indeed it's a XYSeriesCollection. Not a TimeSieriesCollection. And i'm using the XYLineAndShapeRenderer(lines true, shapes false). But i could also use XYLineAndShapeRenderer(lines false, shapes true). I tried both but always same mistake.

- Have you read the thread that you mentioned until the end, where I have described the SamplingXYLineAndShapeRenderer?

I have indeed read the thread until the end and i have used the SamplingXYLineAndShapeRenderer It will delay the problem for like a minute or two but then it still starts to slow down alot.

- Do you need symbols, lines, or both?

Like above i can use one of the two. Preferably lines but it's not 100% necessarily.

- What is the maximum time range that you want to show with the mentioned resolution ? If it is only a couple of hours, i.e. < 10k of data points, the SamplingXYLineAndShapeRenderer and an efficient dataset implementation (NOT the ones I mentioned above) should work.

Well Time range is only like a couple of minutes. But you need to understand that i get data incoming every 20ms sometimes even twice each 20ms and all this data will be processed and drawed in the chart. So you can imagine after 1 minute off running my diagnostic application I will have approximately 3000 points. this is if i get 1 value each 20ms but like i said most of the time its 2 values each 20ms so let's say it gets 5000 points at least in 1 minute. After 2mins it's 10000 points 3 minutes 15000 etc. And sometimes this chart will run for 1 minute or 2 minutes and then the slow drawing is not that big of a deal (but still prefer not to have any slow drawing) but if it needs to run for like 5-10mins it will slow down so hard that the definition of 'live' data from my application is gone.

I've found the Draw method in XYPlot.java and i see that they are using buffered image here. So what i was thinking is that if it is possible to override this method and make sure that the gridlines and axises and all that kind of stuff only gets drawn one time (unless the chart resized then everything has to be drawed again). and then if i can use the bufferedimage to draw to points and lines on the buffered image in the background and only draw the last point added in the foreground so on my application this won't slow the application down anymore.

Are is this impossible to achieve?

[paradoxoff]

As stated in the mentioned thread, a "dynamic" XYSeriesCollection and an XYLineAndShapeRenderer are not suitable for either dynamic data or large dataset, and definitely not for a combination of both as in your case.
So 100k of data (100 Hz data update frequency, total time 1000 sec or 16 min) rendered as a line with a "visual update frequency" of 0.5 sec would be ok?
Your suggested approach sounds feasible. Instead of extending XYPlot, it might by simpler to create your own Plot type. See for example the org.jfree.chart.plot.FastScatterPlot.

[Jorne]

So 100k of data (100 Hz data update frequency, total time 1000 sec or 16 min) rendered as a line with a "visual update frequency" of 0.5 sec would be ok?

If you mean that the 100k data points will be drawed within 0.5 secs. Then yes i think this would be ok for me. I'm not sure tho i would have to test this out.

Your suggested approach sounds feasible. Instead of extending XYPlot, it might by simpler to create your own Plot type. See for example the org.jfree.chart.plot.FastScatterPlot.

Yes i'm trying to override the draw function of XYPlot by extending it but there are some private functions (getRendererIndices, getDatasetIndices) that can't be used outside the original XYPlot class as well as the private variables:

Code: Select all

/** Storage for the datasets. */
    private Map<Integer, XYDataset> datasets;

    /** Storage for the renderers. */
    private Map<Integer, XYItemRenderer> renderers;

So i'm trying to get this sorted for overriding the draw method. But i will also look into the FastScatterPlot. So to add the points to my FastScatterPlot i will need to create a two dimensional array (float[][]) and then add points to it and everytime i get an x and y value i add it to the array and use the setData() function of scatterplot and it will update.
But then i have to make the array big enough.

Correct me if i'm wrong. I'll try this out

[paradoxoff]

But i will also look into the FastScatterPlot. So to add the points to my FastScatterPlot i will need to create a two dimensional array (float[][]) and then add points to it and everytime i get an x and y value i add it to the array and use the setData() function of scatterplot and it will update.
But then i have to make the array big enough.

Don't do it this way, because then the FastScatterPlot will draw the entire array again if you provide a new data array with the setData method, as opposed to only draw the last data point.
I mentioned the FastScatterPlot not because I thought that it may solve your performance issue, but as an example of how a "lightweight" plot that behaves like an XYPlot could be programmed.
You certainly need a way to store all of the data, but in order to draw only the last data point, you need to tell the plot that a new data point has arrived, for example by an addPoint(double x, double y) method.
One potential drawback of the FastScatterPlot is that it does not use any optimizations mentioned in the "optimization thread", such as:
- check which data points are within the visible range of the axes, and draw only those
- don't draw data points that won't be visible

Before you write w new plot class, check whether example below is fast enough.
I have used the following approach to make that thing fast:
- the axes have a fixed size. Thus, the plot does not need to spend time to determine the optimal data ranges
- the dataset has DomainOrder.ASCENDING as DomainOrder. This makes it easier to find the data items that are within the value range of the axes. In this example, this is onmly of minor importance since all data points are visible.
- Use of the SamplingXYLineAndShapeRenderer. This avoid the rendering of points that won't be visible anyways.

Code: Select all

package jfree;

import java.awt.Color;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.geom.Ellipse2D;
import javax.swing.JFrame;
import org.jfree.data.xy.AbstractXYDataset;
import org.jfree.chart.ChartPanel;
import org.jfree.chart.JFreeChart;
import org.jfree.chart.axis.DateAxis;
import org.jfree.chart.axis.NumberAxis;
import org.jfree.chart.plot.XYPlot;
import org.jfree.chart.renderer.xy.SamplingXYLineRenderer;
import org.jfree.chart.renderer.xy.SamplingXYLineAndShapeRenderer;
import java.util.Random;
import org.jfree.data.DomainOrder;
import javax.swing.Timer;

public class AutoUpdatingDatasetDemo {

    public static void main(String[] args) {
        int max = 100000;
        int delay = 10;
        AutoUpdatingDataset dataset = new AutoUpdatingDataset("Auto Data", max, delay, 250);
        DateAxis dateAxis = new DateAxis("Date");
        NumberAxis rangeAxis = new NumberAxis("Value");
        long now = System.currentTimeMillis();
        dateAxis.setAutoRange(false);
        rangeAxis.setAutoRange(false);
        SamplingXYLineAndShapeRenderer renderer = new SamplingXYLineAndShapeRenderer(false, true);
        renderer.setSeriesShape(0, new Ellipse2D.Double(-2,-2, 4, 4));
        renderer.setSeriesPaint(0, Color.blue);
        
        XYPlot plot = new XYPlot(dataset, dateAxis, rangeAxis, renderer);
        plot.setDomainMinorGridlinesVisible(true);
        plot.setDomainPannable(true);
        JFreeChart chart = new JFreeChart(plot);
        JFrame frame = new JFrame("Auto Update Demo");
        ChartPanel cp = new ChartPanel(chart);
        cp.setMouseWheelEnabled(true);
        frame.getContentPane().add(cp);
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.pack();
        frame.setVisible(true);
        dateAxis.setLowerBound(now);
        dateAxis.setUpperBound(now + max*delay);
        rangeAxis.setLowerBound(-1.2);
        rangeAxis.setUpperBound(1.2);
        dataset.start();
    }
}

class AutoUpdatingDataset extends AbstractXYDataset implements ActionListener {

    private String name;
    private int max;
    private long delay;
    private long visualDelay;
    private double[][] values;
    private int cursor = -1;
    long first;
    private Timer timer;
    private Random random = new Random();
    private long lastEvent;
    private long period;
    private double yDataMax = Math.PI*10;

    AutoUpdatingDataset(String name, int max, int delay, int visualDelay) {
        this.name = name;
        this.max = max;
        this.delay = delay;
        this.visualDelay = visualDelay;
        this.values = new double[max][2];
        timer = new Timer(delay, this);
        lastEvent = System.currentTimeMillis();
       
    }

    public DomainOrder getDomainOrder() {
        return DomainOrder.ASCENDING;
    }

    public int getSeriesCount() {
        return 1;
    }

    public Comparable getSeriesKey(int series) {
        return name;
    }

    public int getItemCount(int series) {
        return cursor;
    }

    public double getYValue(int series, int item) {
        return values[item][1];
    }

    public double getXValue(int series, int item) {
        return values[item][0];
    }

    public Double getY(int series, int item) {
        return new Double(getYValue(series, item));
    }

    public Double getX(int series, int item) {
        return new Double(getXValue(series, item));
    }

    public void actionPerformed(ActionEvent ae) {
        if (cursor >= max - 1) {
            timer.stop();
            return;
        }
        cursor++;
        long now = System.currentTimeMillis();
        values[cursor][0] = now;
        values[cursor][1] = Math.sin(yDataMax/max*cursor);
        if (now - lastEvent > visualDelay) {
            lastEvent = now;
            fireDatasetChanged();
        }
    }

    public void start() {
        timer.start();
    }
}