Lazy loading of tables in JSF – working with large amount of data

The challenge working with a large amount of data in user-presentation is to serve the user an easy and effective way of getting to the information he/she is looking for. If you have a table with 50.000 rows, you would in some way need to chop the information in smaller pieces, for example having multiple pages displaying 50 rows each, and then having a way of navigating through the pages.

But still, there are some considerations to do this in a way that opts for performance. If the user selects do display page 2, meaning rows 50-100, wouldn’t you like to fetch only those, not all the 50.000 rows? And like in JSF, if you have multiple components (dataTables, dataScrollers) that are dependent on/using the same data, you would like that the data is only retrieved once, not one time for each component. Doing web applications, being able to limit the amount of data that is fetched from the backend is a key to success when it comes to performance and operability issues.

My example is based on Cagatay Civici’s post http://cagataycivici.wordpress.com/2006/07/10/jsf_datatable_with_custom_paging/,
so I will not go through the technical solution and how the “lazy” data model works once more, I suggest you read his post for that. But I give a complete example of how to use his PagedListDataModel, and in addition how to make sure the data is retrieved only once by using phaseListeners.

To keep it simple, let’s imagine that we have a Dictionary in the backend, and a jsp page with static links with letters like ‘A’, ‘B’, ‘C’ and so on throughout the alphabet. Pressing on one of the letters would display all the words that starts with it in the dataTable, and we can use a dataScroller to navigate through multiple pages.

This is my jsp, in which I’ve removed the columns of the dataTable as they are of no importance. Note that the dataTable is set with a valueBinding, and that the value of it is pointing to a dataModel. Also note that we do not render the table if no one has selected a letter. I also just give one link as an example of how to update the property we can refer to as the “selected letter”:

<h:commandLink styleClass="myLink" value="A">
	<t:updateActionListener value="A" property="#{myDataModelHandler.selectedLetter}"></t:updateActionListener>
</h:commandLink>
<h:panelGrid id="mainPanel" width="100%">
	<t:dataTable
	  id="htmlTableView"
	  value="#{myDataModelHandler.activeDataModel}" rendered="#{myDataModelHandler.selectedLetter != null}"
	  binding="#{myHtmlDataTableHandler.htmlDataTable}"
	  headerClass="tableViewHeader"
	  var="row">
	</t:dataTable>
	<t:panelGrid id="footerPager">
		 <t:dataScroller id="ds5"
				for="htmlTableView"
				fastStep="5"
				styleClass="scroller"
				paginator="true"
				paginatorMaxPages="5"
				paginatorTableClass="paginator"
				paginatorActiveColumnStyle="font-weight:bold;"
				rowsCountVar="rowsCount"
				renderFacetsIfSinglePage="true"
			>
			<f:facet name="previous">
				<t:graphicImage id="datascrollerPrevious" value="/images/left-one-step.gif" border="1" styleClass="dataScrollerImage"/>
			</f:facet>
			<f:facet name="next">
				 <t:graphicImage id="datascrollerNext" value="/images/right-one-step.gif" border="1" styleClass="dataScrollerImage"/>
			</f:facet>
			<f:facet name="first">
				<t:graphicImage id="datascrollerFirst"
				value="/images/left.gif" border="1" styleClass="dataScrollerImage"/>
			</f:facet>
			<f:facet name="last">
				 <t:graphicImage id="datascrollerLast"
				 value="/images/right.gif" border="1" styleClass="dataScrollerImage" />
			</f:facet>
		 </t:dataScroller>
	 </t:panelGrid>
</h:panelGrid>

This is the handler (backing bean) that holds the htmlDataTable:

public class MyHtmlDataTableHandler
{
    private HtmlDataTable htmlDataTable;
    public MyHtmlDataTableHandler()
    {

    }

    public HtmlDataTable getHtmlDataTable()
    {
        if(htmlDataTable == null)
        {
            log.debug("htmlDataTable == null");
            HtmlDataTable dataTable = new HtmlDataTable();
            dataTable.setRows(50);
            setHtmlDataTable(dataTable);
        }
        return htmlDataTable;
    }

    public void setHtmlDataTable(HtmlDataTable htmlDataTable)
    {
            this.htmlDataTable = htmlDataTable;
    }
}

You can see that I have modified the getter to create a new instance if the instance it holds is null. It’s important to set the number of rows here, or it will fail when trying to build the lazy data model.The dataModel we use for our dataTable is created and set before the RENDER_RESPONSE phase, as mentioned earlier this is to avoid that the data itself is retrieved multiple times. To be able to do this we need to use a phaseListener, so we implement it in our constructor of the dataModelHandler. Note that we have references here to the dataTableHandler as well as a util for helping us creating a paged data model. As you can see in the phaseListener beforePhase()-method, if no letter has been selected, we don’t create or set the model.

public class MyDataModelHandler
{
	private String selectedLetter;
	private PagedListDataModel activeDataModel;

	private MyHtmlDataTableHandler myHtmlDataTableHandler; //BEAN_REFERENCE SET IN FACES-CONFIG
	private PagedDataModelUtil pagedDataModelUtil; //BEAN_REFERENCE SET IN FACES-CONFIG

	public MyDataModelHandler()
	{
		LifecycleFactory lifecycleFactory = (LifecycleFactory) FactoryFinder.getFactory(FactoryFinder.LIFECYCLE_FACTORY);
		Lifecycle lifecycle = lifecycleFactory.getLifecycle(lifecycleFactory.DEFAULT_LIFECYCLE);

		lifecycle.addPhaseListener
		(
			new PhaseListener()
			{
				public void afterPhase(PhaseEvent arg0)
				{
				}
				public void beforePhase(PhaseEvent arg0)
				{
					if(selectedLetter != null)
					{
						setActiveDataModel(pagedDataModelUtil.createPagedDataModel(selectedLetter, getMyHtmlDataTableHandler().getHtmlDataTable()));
					}
				}	

				public PhaseId getPhaseId()
				{
					return PhaseId.RENDER_RESPONSE;
				}
			}
		);
	}

	public String getSelectedLetter()
	{
		return selectedLetter;
	}

	public void setSelectedLetter(String selectedLetter)
	{
		this.selectedLetter = selectedLetter;
	}

	public PagedListDataModel getActiveDataModel()
	{
		return activeDataModel;
	}

	public void setActiveDataModel(PagedListDataModel activeDataModel)
	{
		this.activeDataModel = activeDataModel;
	}

	public MyHtmlDataTableHandler getMyHtmlDataTableHandler()
	{
		return myHtmlDataTableHandler;
	}

	public void setMyHtmlDataTableHandler(MyHtmlDataTableHandler myHtmlDataTableHandler)
	{
		this.myHtmlDataTableHandler = myHtmlDataTableHandler;
	}

	public PagedDataModelUtil getPagedDataModelUtil()
	{
		return pagedDataModelUtil;
	}

	public void setPagedDataModelUtil(PagedDataModelUtil pagedDataModelUtil)
	{
		this.pagedDataModelUtil = pagedDataModelUtil;
	}
}

The pagedDataModel we create is an instance of PagedListDataModel which is exactly the same as in Cagatay Civici’s post:

public class PagedListDataModel extends DataModel
{
  private int rowIndex = -1;
  private int totalNumRows;
  private int pageSize;
  private List list;

  public PagedListDataModel(List list, int totalNumRows, int pageSize)
  {
    setWrappedData(list);
    this.totalNumRows = totalNumRows;
    this.pageSize = pageSize;
  }

  public boolean isRowAvailable()
  {
    if(list == null)
      return false;
    int rowIndex = getRowIndex();
    if(rowIndex >=0 && rowIndex < list.size())
      return true;
    else
      return false;
  }

  public int getRowCount()
  {
    return totalNumRows;
  }

  public Object getRowData()
  {
    if(list == null)
      return null;
    else if(!isRowAvailable())
      throw new IllegalArgumentException();
    else
	{
      int dataIndex = getRowIndex();
      return list.get(dataIndex);
    }
  }

  public int getRowIndex()
  {
    return (rowIndex % pageSize);
  }

  public void setRowIndex(int rowIndex)
  {
    this.rowIndex = rowIndex;
  }

  public Object getWrappedData()
  {
    return list;
  }

  public void setWrappedData(Object list)
  {
    this.list = (List) list;
  }

}

&#91;/sourcecode&#93;
As you see from the beforePhase()-method in the phaseListener I use a util to create the pagedDataModel. In my application I have a index on the back, but this can of course differ from your situation. I will therefore refer to the backend as "service", and whatever you use that holds and gets the data should not matter.
&#91;sourcecode language='java'&#93;
public class PagedDataModelUtil
{
	public PagedListDataModel createPagedDataModel(String selectedLetter, HtmlDataTable htmlDataTable)
	{
			int startPosition = htmlDataTable.getFirst();
			int dimensionToUse = htmlDataTable.getRows();
			Service service = new Service();
			int totalListSize = service.getTotalNumberOfHits(selectedLetter);

			List<Word> words = new ArrayList<Word>();
			words.addAll(service.getHits(selectedLetter, startPosition, dimensionToUse ));
			PagedListDataModel dataModel = new PagedListDataModel(words, totalListSize, dimensionToUse);
			return dataModel;
	}
}

So, we have now a table that retrieves only the data the user wants to see, and the data is not retrieved multiple times. Fine!But then, what about sorting?? Let’s say that one of the columns we have in our table is length, meaning the length of the word in number of characters. What if I want to sort by this?You could do this as well. I assume you have a header in your table, giving the labels for each column. You can create a commandLink on each label, and if you press one of them you set that value into a property, for example “sortedBy” in the dataModelHandler.First, the link:

 			<t:column>
				<f:facet name="header">
					<t:commandLink id="sortBySomethingLink" value="myColumnLink">
						<t:updateActionListener property="#{myDataModelHandler.sortedBy}" value="theFieldNameIWantToSortOn"></t:updateActionListener>

					</t:commandLink>
				</f:facet>
				<h:outputText value="#{someObject.someValue}" />

			</t:column>

We see that this link updates the property sortedBy in the MyDataModelHandler with the value specified. In the phaseListener, when we call the createPagedDataModel, we now need to pass this property.In other words we modify the createPagedDataModel() method to add support for sorting. We have a new parameter called sortedBy, and use StringUtils to test whether we should bring this one into play or not. If it is null or “” (blank), we don’t sort. We could also keep the sortedBy parameter as a property in our PagedDataModelUtil, in this way we can have an ascending / descending functionality. If the sortedBy parameters equals the sortedBy property, we flip the ascending boolean, if not we sort ascending by default:

private String sortedByLast = "";
private boolean ascending = true;

public PagedListDataModel createPagedDataModel(String selectedLetter, HtmlDataTable htmlDataTable, String sortedBy)
{
			int startPosition = htmlDataTable.getFirst();
			int dimensionToUse = htmlDataTable.getRows();
			int totalListSize = service.getTotalNumberOfHits(selectedLetter);

			List<Word> words = new ArrayList<Word>();
			if(StringUtils.isNotBlank(sortedBy))
			{
				if(sortedBy.equalsIgnoreCase(sortedByLast))
					ascending = !ascending;
				else
					ascending = true;

				words.addAll(service.getHitsSorted(selectedLetter, startPosition, dimensionToUse, sortedBy, ascending));
			}
			else
				words.addAll(service.getHits(selectedLetter, startPosition, dimensionToUse ));

			PagedListDataModel dataModel = new PagedListDataModel(words, totalListSize, dimensionToUse);
			return dataModel;
}

Of course, this puts the responsibility of sorting on the backend service. Somewhere we need to do the sorting. 🙂 If you use a database and have a high number of rows/items, this might be a performance issue. In my application I use Lucene (index), and even with a very high number of rows I don’t have any problems with performance. But doing the sorting in the backend and getting the first 25 items e.g. is anyway faster than retrieving all the items and then do sorting.Ok, that was all this time!