Friday, 18 April 2014

Spring Batch Tutorial

Introduction

Spring batch is a lightweight framework that provides a solid foundation on which to build robust and scalable batch applications. It provides developers with a set of tried and tested patterns that solve common batch problems and allows developers to focus more on the business requirement and less on complex batch infrastructure. Spring batch contains a variety of out of the box configurable components that can be used to satisfy many of the most common batch use cases. Extensive XML configuration and an extensible programming model mean that these components can be customised and used as building blocks to quickly deliver common batch functionality.
This tutorial will show you how to build a very simple batch application to read fixed length data from a flat file and write it to a database table. This is a common batch use case and should be sufficient to demonstrate some of the fundamental concepts of Spring batch and provide you with a foundation on which to build more complex batch applications.

Sample Application

The sample batch application described in this tutorial uses a H2 in memory database so that you can download the sample code and run it without having to set up a database server. The sample job is run as an integration test so once you grab the code you can have a working batch job up and running in a matter of minutes. The rest of this post will take you through a step by step guide describing all components in the sample batch job provided.

Project Structure

The diagram below shows the project structure of our sample batch application. Each component is described in detail below.
Figure 1.0 - Sample Application Project Structure

Batch Job Definition

This import-accounts-job-context file contains the XML definition of our batch job and the components it uses. Each part of the job definition is described in detail below.
1:  <?xml version="1.0" encoding="UTF-8"?>  
2:  <beans xmlns="http://www.springframework.org/schema/beans"  
3:         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
4:         xmlns:batch="http://www.springframework.org/schema/batch"  
5:         xsi:schemaLocation="http://www.springframework.org/schema/beans   
6:                                  http://www.springframework.org/schema/beans/spring-beans-3.0.xsd  
7:                                  http://www.springframework.org/schema/batch   
8:                                  http://www.springframework.org/schema/batch/spring-batch-2.1.xsd">  
9:    
10:    
11:       <job id="importAccountData" xmlns="http://www.springframework.org/schema/batch">       
12:            <step id="parseAndLoadAccountData">  
13:                 <tasklet>  
14:                      <chunk reader="reader" writer="writer" commit-interval="3" skip-limit="2">  
15:                           <skippable-exception-classes>  
16:                                <include class="org.springframework.batch.item.file.FlatFileParseException" />  
17:                           </skippable-exception-classes>  
18:                      </chunk>  
19:                 </tasklet>                 
20:            </step>  
21:       </job>  
22:         
23:       <bean id="reader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">  
24:            <property name="resource" value="file:#{jobParameters['inputResource']}" />  
25:            <property name="linesToSkip" value="1" />  
26:            <property name="lineMapper">  
27:                 <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">  
28:                      <property name="lineTokenizer">  
29:                           <bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">  
30:                                <property name="names" value="ACCOUNT_ID,ACCOUNT_HOLDER_NAME,ACCOUNT_CURRENCY,BALANCE" />  
31:                                <property name="delimiter" value="," />  
32:                           </bean>  
33:                      </property>  
34:                      <property name="fieldSetMapper">  
35:                           <bean class="com.blog.samples.batch.AccountFieldSetMapper" />  
36:                      </property>  
37:                 </bean>  
38:            </property>  
39:       </bean>  
40:         
41:       <bean id="writer" class="com.blog.samples.batch.AccountItemWriter">  
42:            <constructor-arg ref="dataSource" />  
43:       </bean>  
44:    
45:  </beans>  

Line 11 - Batch job element defines a batch job which is the top level configurable batch component and acts as a container for one or more batch steps.  The id attribute is used to uniquely identify the batch job and is used later by the JobLauncher to invoke the job.
Line 12 - A batch step is a component that represents a specific independent phase of a batch job. In our sample application we define a single step that parses data from a flat file and loads that data into the database. Our step is given the unique identifier parseAndLoadAccountData.
Line 13 - Spring batch provides Tasklets as an extension point that allow developers to handle processing inside a batch step. A tasklet is a Java class that implements the tasklet interface and is written to implement custom logic within a step. The tasklet is then invoked by Spring Batch at runtime.
Line 14 - For read/write uses cases Spring batch uses chunk oriented processing. Items are read one at a time by an item reader and are aggregated into a collection, or 'chunk' of a specified size. When the number of read items in the chunk reaches the specified limit, the contents of the chunk are sent to the item writer and written to the target data source. The size of the chunk is configured as a commit limit on the chunk definition. The diagram below describes the sequence of events and components used for chunk processing.

Figure 2.0 - Chunk Oriented Processing
A chunk is configured by specifying the following
  • Item Reader - component that reads data from a specified data source. Common data sources include flat files, XML files, database tables, JMS etc.
  • Item Writer - component that writes data to a target data source in chunks. Common data sources are the same as described in the item reader above. 
  • Commit Interval - Value specifies chunk size for the batch step, in other words, the number of items that are aggregated and written by the item writer in a single commit.  
  • Skip Limit - Number of erroneous records that can be skipped before a batch job fails. In our sample application we set the skip limit to 2, meaning that if 2 erroneous records are encountered the batch process will continue. If a third erroneous record is found the batch job will terminate. 
Line 15 to 17 - The Skippable Exception Class element like the skip limit attribute above, provides a means of ensuring the batch application is more robust. You can define a list of exceptions, that if encountered during processing, will be ignored by Spring Batch. In our sample application we have chosen to ignore FlatFileParseExeptions.
Line 23 - The FlatFileItemReader is one of the reader components that Spring Batch provides out of the box. Reading flat files is a common batch use case so Spring Batch provides a convenience class that be easily configured to satisfy this requirement. I've described this configuration in detail below.
Line 24 - The resource attribute refers to the input file to be processed. In this instance we set the input file as a job parameter using the following notation #{jobParameters['inputResource']}. In order to set component attributes as job parameters, the class must support late binding by setting the scope attribute to step (line 23)
Line 25 - The linesToSkip attribute indicates the number of lines that should be ignored by the reader before actual processing begins. In our example we've ignored the first line of the file as this is a header row.
Line 26 - The lineMapper attribute defines the configuration of the component that will perform the line reads. In this instance we use a Spring Batch implementation called DefaultLineMapper which requires a line tokenizer component to split the line contents into individual fields.
Line 29 - Spring Batch provides an out of the box implementation of the tokenizer called DelimitedLineTokenizer that is configured with a list of field names.
Line 30 - The DelimitedLineTokenizer splits the line into tokens that are later referenced by the token names defined.
Line 31 - The delimiter attribute specifies the delimiter used to tokenize each line of the input file. In this instance our input file is comma delimited.
Line 34 & 35 - The fieldSetMapper attribute refers to the custom class AccountFieldSetMapper that takes a Field Set and maps the fields to instance variables on the Account  POJO (descried later).
Line 41 - The writer component is responsible for writing data items, in this case Account POJOs to the database. When the reader has reached the commit limit it passes a chunk of read items to the writer component so that they can be written to the database in a single transaction.

Field Set Mapper

On line 35 of the job definition above we referenced an AccountFieldSetMapper object for mapping the field set provided by the line tokenizer into our Account domain model. The AccountFieldSetMapper is defined below.
1:  package com.blog.samples.batch;  
2:    
3:  import org.springframework.batch.item.file.mapping.FieldSetMapper;  
4:  import org.springframework.batch.item.file.transform.FieldSet;  
5:  import org.springframework.validation.BindException;  
6:    
7:  import com.blog.samples.batch.model.Account;  
8:    
9:  /**  
10:   * Account field mapper takes FieldSet object for each row in input   
11:   * file and maps it to an Account model object  
12:   *   
13:   */  
14:  public class AccountFieldSetMapper implements FieldSetMapper<Account>  
15:  {  
16:         
17:       /**   
18:        * Map provided fieldset to Account POJO using keys defined in the names   
19:        * attribute of the DelimitedLineTokenizer object  
20:        */  
21:       public Account mapFieldSet(FieldSet fieldSet_p) throws BindException  
22:       {  
23:            Account account = new Account();  
24:            account.setId(fieldSet_p.readString("ACCOUNT_ID"));  
25:            account.setAccountHolderName(fieldSet_p.readString("ACCOUNT_HOLDER_NAME"));  
26:            account.setAccountCurrency(fieldSet_p.readString("ACCOUNT_CURRENCY"));  
27:            account.setBalance(fieldSet_p.readBigDecimal("BALANCE"));  
28:         
29:            return account;  
30:       }  
31:  }  
As you can see this class implements the FieldSetMapper interface and provides an implementation of the mapFieldSet method that maps fields from the FieldSet to our Account model object. Individual fields are referenced using the keys defined in the names property of the DelimitedLineTokenizer we defined earlier.

Item Writer

On line 41 of the job definition above we referenced an AccountItemWriter object for writing data to the database. The AccountItemWriter is defined below.
1:  package com.blog.samples.batch;  
2:  import java.util.List;  
3:  import javax.sql.DataSource;  
4:  import org.springframework.batch.item.ItemWriter;  
5:  import org.springframework.jdbc.core.JdbcTemplate;  
6:  import com.blog.samples.batch.model.Account;  
7:    
8:  /**  
9:   * Class takes Account model objects created in item reader and makes   
10:   * them available to writer to persist in the database  
11:   *   
12:   */  
13:  public class AccountItemWriter implements ItemWriter<Account>  
14:  {  
15:    
16:       private static final String INSERT_ACCOUNT = "insert into account (id,accountHolderName,accountCurrency,balance) values(?,?,?,?)";  
17:       private static final String UPDATE_ACCOUNT = "update account set accountHolderName=?, accountCurrency=?, balance=? where id = ?";  
18:       private JdbcTemplate jdbcTemplate;  
19:    
20:       /**  
21:        * Method takes a list of Account model objects and uses JDBC template to either insert or  
22:        * update them in the database  
23:        */  
24:       public void write(List<? extends Account> accounts_p) throws Exception  
25:       {  
26:            for (Account account : accounts_p)  
27:            {  
28:                 int updated = jdbcTemplate.update(UPDATE_ACCOUNT, account.getAccountHolderName(), account.getAccountCurrency(),   
29:                                  account.getBalance(), account.getId());  
30:                 if (updated == 0)  
31:                 {  
32:                      jdbcTemplate.update(INSERT_ACCOUNT, account.getId(), account.getAccountHolderName(),   
33:                                account.getAccountCurrency(), account.getBalance());  
34:                 }  
35:            }  
36:       }  
37:    
38:       public AccountItemWriter(DataSource dataSource_p)  
39:       {  
40:            this.jdbcTemplate = new JdbcTemplate(dataSource_p);  
41:       }  
42:  }  
The AccountItemWriter class implements the ItemWriter interface and provides an implementation of the write method. The write method is invoked by Spring Batch with a list of objects read by the item reader component. The number of items in the list, or chunk size is dictated by the commit-interval attribute on the item reader definition. As you can see above we use a jdbcTemplate to persist the list of account objects one at a time. Note that Spring batch will perform a single commit once all items in the chunk have been written as this is substantially more performant than one commit per object. This is particularly significant when writing large datasets.

Framework Component Configuration

We've already defined our batch job but in order to run that job we'll need to configure a number of Spring Batch framework components. In this tutorial these are loaded as part of our integration test (test-context.xml) but in real world application we'd likely deploy our batch job in a web container and load these components on container start up. These components are described below.
1:  <?xml version="1.0" encoding="UTF-8"?>  
2:  <beans xmlns="http://www.springframework.org/schema/beans"  
3:         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
4:      xmlns:jdbc="http://www.springframework.org/schema/jdbc"  
5:         xsi:schemaLocation="http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-3.0.xsd  
6:                      http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">  
7:    
8:         
9:       <jdbc:embedded-database id="dataSource" type="H2">  
10:            <jdbc:script location="/create-account-table.sql"/>  
11:       </jdbc:embedded-database>  
12:    
13:       <bean class="org.springframework.jdbc.core.JdbcTemplate">  
14:            <constructor-arg ref="dataSource" />  
15:       </bean>  
16:         
17:       <bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">  
18:            <property name="dataSource" ref="dataSource" />  
19:       </bean>  
20:    
21:       <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">  
22:            <property name="transactionManager" ref="transactionManager" />  
23:       </bean>  
24:    
25:       <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">  
26:            <property name="jobRepository" ref="jobRepository" />  
27:       </bean>  
28:    
29:  </beans>  
Lines 9 to 11 - We define an in memory database for persisting the job meta data and Account 'business data'. In a real world application we'd use a proper RDBMS like MySQL or Oracle.
Lines 13 to 15 - A JdbcTemplate is required by the item writer component to write Account data to the database. We also use the JdbcTemplate in our unit test to check the job ran as expected.
Lines 17 to 19 - A transaction manager is defined and takes in the data source we defined above.
Lines 21 to 23 - A JobRepository is required so that Spring batch can maintain job state by serializing and persisting important job metadata. The JobRepository is transactional and requires the transaction manager we defined above.  
Lines 25 to 27 - A JobLauncher is required so that we can invoke our batch job from the integration test.

Batch Test Data

We have 2 input files to run as part of our integration test. The accounts.txt file below contains 10 valid records.
 ACCOUNT_ID,ACCOUNT_HOLDER_NAME,ACCOUNT_CURRENCY,BALANCE  
 1234567,Riain McAtamney,STG,3233.43  
 5494032,Gary Jonston,STG,32329.45  
 4324324,Colm Toale,STG,5435.80  
 2436513,Gary Gallagher,STG,43234.54  
 6242345,Connor Smith,EUR,5342.32  
 5435432,Ruairi Digby,EUR,4322.13  
 6543523,Steve Jones,EUR,5643.54  
 5431245,Peter Murray,STG,4324.13  
 6546556,John Collins,STG,54354.43  
 7654654,Sean Molloy,STG,32133.22  
The accountsError.txt file below contains 8 valid and 2 invalid records and will allow us to test the skip-limit attribute on the item reader.
 ACCOUNT_ID,ACCOUNT_HOLDER_NAME,ACCOUNT_CURRENCY,BALANCE  
 1234567,Riain McAtamney,STG,3233.43  
 5494032,Gary Jonston,STG,32329.45  
 4324324,Colm Toale,STG,5435.80  
 2436513,Gary Gallagher,STG,43234.54  
 6243345,Connor Smith,EUR,5xxx342.32  
 5435432,Ruairi Digby,EUR,4322.13  
 6543523,Steve Jones,EUR,5643.54  
 5431245,Peter Murray,STG,432XX4.13  
 6546556,John Collins,STG,54354.43  
 7654654,Sean Molloy,STG,32133.22  

Batch Integration Test

The final step is to write an integration test to run our batch job. The test is defined as follows.
1:  package com.blog.samples.batch.test;  
2:    
3:  import org.junit.Assert;  
4:  import org.junit.Before;  
5:  import org.junit.Test;  
6:  import org.junit.runner.RunWith;  
7:  import org.springframework.batch.core.Job;  
8:  import org.springframework.batch.core.JobParametersBuilder;  
9:  import org.springframework.batch.core.launch.JobLauncher;  
10: import org.springframework.beans.factory.annotation.Autowired;  
11: import org.springframework.beans.factory.annotation.Value;  
12: import org.springframework.core.io.Resource;  
13: import org.springframework.jdbc.core.JdbcTemplate;  
14: import org.springframework.test.context.ContextConfiguration;  
15: import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;  
16:    
17:  @RunWith(SpringJUnit4ClassRunner.class)  
18:  @ContextConfiguration(locations = { "/import-accounts-job-context.xml", "/test-context.xml" })  
19:  public class ImportAccountsIntegrationTest  
20:  {  
21:    
22:       @Autowired  
23:       private JobLauncher jobLauncher_i;  
24:       @Autowired  
25:       private Job job_i;  
26:       @Autowired  
27:       private JdbcTemplate jdbcTemplate_i;  
28:       @Value("file:src/test/resources/input/accounts.txt")  
29:       private Resource accountsResource;  
30:       @Value("file:src/test/resources/input/accountsError.txt")  
31:       private Resource accountsErrorResource;  
32:         
33:       @Before  
34:       public void setUp() throws Exception  
35:       {  
36:            jdbcTemplate_i.update("delete from account");            
37:       }  
38:    
39:       @Test  
40:       public void importAccountDataTest() throws Exception  
41:       {  
42:            int startingCount = jdbcTemplate_i.queryForInt("select count(*) from account");  
43:            jobLauncher_i.run(job_i, new JobParametersBuilder().addString("inputResource", accountsResource.getFile().getAbsolutePath())  
44:                                                                             .addLong("timestamp", System.currentTimeMillis())  
45:                                                                             .toJobParameters());  
46:    
47:            int accountsAdded = 10;  
48:            Assert.assertEquals(startingCount + accountsAdded, jdbcTemplate_i.queryForInt("select count(*) from account"));  
49:       }  
50:    
51:       @Test  
52:       public void importAccountDataErrorTest() throws Exception  
53:       {  
54:            int startingCount = jdbcTemplate_i.queryForInt("select count(*) from account");  
55:            jobLauncher_i.run(job_i, new JobParametersBuilder().addString("inputResource", accountsErrorResource.getFile().getAbsolutePath())  
56:                                                                             .addLong("timestamp", System.currentTimeMillis())  
57:                                                                             .toJobParameters());  
58:    
59:            int accountsAdded = 8;  
60:            Assert.assertEquals(startingCount + accountsAdded, jdbcTemplate_i.queryForInt("select count(*) from account"));  
61:       }  
62:  }  
Line 18 - Import the job and infrastructure component definitions required to run the job.
Line 22 to 31 - Injected infrastructure dependencies and file resources required to run the batch job.
Line 33 to 37 - Set-up method runs before test and clears down database.
Line 43 - Here we use the JobLauncher to run our import account data job by loading the accounts.txt file. A JobParametersBuilder object is used to pass in the input file and time stamp as job parameters.
Line 48 - Query the database to get the Account table row count and ensure all rows have been successfully persisted.
Line 55 - Use the JobLauncher to run our import account data job by loading the accountsError.txt file. A JobParametersBuilder object is used to pass in the input file and time stamp as job parameters.
Line 60 - Query the database to get the Account table row count and ensure that only 8 rows have been successfully persisted, as we would expect the two invalid rows were skipped.

Sample Code

 You can get the sample code for this post on github at https://github.com/briansjavablog/spring-batch-tutorial. Feel free to experiment with the code and as usual comments/questions are welcome.

Friday, 31 January 2014

Web Resource Optimisation with WRO4J

Its been a while since I've posted anything so I'm going to ease myself back in with a post on web resource optimisation, and in particular my recent experience optimising a Java web application using WRO4J.

What is Web Resource Optimisation?

Rich internet applications are now common place, delivering a slick, responsive user experience that in the past was only possible with thick client applications. Modern web applications are typically composed of many JavaScript and CSS resources, so we need to consider ways of optimising how theses resources are served to the client. The objective of web resource optimisation like any other type of optimisation, is to increase application performance and ultimately improve user experience.

In this post I'll create a very simple Spring MVC app with 2 JSP views that import some CSS and JavaScript resources. Both views will import the same resources, the difference being that one will manage resources using a traditional approach, while the other will use WRO4J. We'll take a look at how these resources are configured and the performance implications of both approaches. The sample application is available to download at the bottom of this post.

Standard Web Resource Configuration

We'll start off by creating a very simple JSP view that imports 11 JavaScript and 6 CSS files. The JSP is defined as follows.
 <%@taglib uri="http://www.springframework.org/tags" prefix="spring" %>  
 <!DOCTYPE html>  
 <html lang="en">  
 <head>  
      <!-- JavaScript Resources -->  
      <script type="text/javascript" src="<spring:url value="/resources/js/lib/jquery/jquery-1.8.3.js"/>"></script>  
      <script type="text/javascript" src="<spring:url value="/resources/js/lib/bootstrap.js"/>"></script>  
      <script type="text/javascript" src="<spring:url value="//resources/js/lib/data-tables/1.9.4/media/js/jquery.dataTables.js"/>"></script>  
      <script type="text/javascript" src="<spring:url value="/resources/js/lib/underscore/underscore.js"/>"></script>  
      <script type="text/javascript" src="<spring:url value="/resources/js/lib/underscore/underscore.string.js"/>"></script>  
      <script type="text/javascript" src="<spring:url value="/resources/js/lib/jquery.bootstrap-growl.js"/>"></script>  
      <script type="text/javascript" src="<spring:url value="/resources/js/lib/datepicker/js/bootstrap-datepicker.js"/>"></script>  
      <script type="text/javascript" src="<spring:url value="/resources/js/lib/jquery/plugins/jquery.ui.widget.js"/>"></script>  
      <script type="text/javascript" src="<spring:url value="/resources/js/lib/jquery/plugins/jquery.iframe-transport.js"/>"></script>  
      <script type="text/javascript" src="<spring:url value="/resources/js/lib/jquery/plugins/jquery.fileupload.js"/>"></script>  
      <script type="text/javascript" src="<spring:url value="/resources/js/lib/jquery.slimscroll.js"/>"></script>    
      <!-- CSS resources -->       
      <link href="<spring:url value="/resources/css/lib/bootstrap/3.0.0/css/bootstrap.min.css"/>" rel="stylesheet" media="screen">  
      <link href="<spring:url value="/resources/css/lib/bootstrap/3.0.0/css/bootstrap-theme.min.css"/>" rel="stylesheet" media="screen">  
      <link href="<spring:url value="/resources/css/lib/font-awesome/4.0.0/css/font-awesome.css"/>" rel="stylesheet" media="screen">       
      <link href="<spring:url value="/resources/js/lib/data-tables/1.9.4/media/css/demo_table.css"/>" rel="stylesheet" media="screen">  
      <link href="<spring:url value="/resources/js/lib/datepicker/css/datepicker.css"/>" rel="stylesheet" media="screen">        
      <link href="<spring:url value="/resources/css/lib/jquery-ui-1.10.3.custom.min.css"/>" rel="stylesheet" media="screen">  
 </head>  
 <body>                  
      <div class="container">  
           <H1>Not much to see here, just a test page with a bunch of JavaScript imports. No optimisation here!</H1>       
   </div>        
 </body>  
 </html>  
We can load this view by navigating to http://localhost:8080/wro4j-sample/standardView in the sample application. Before loading this view open chrome dev tools (CTRL+SHFT +I) and switch to the Network tab (you can of course use Firebug or any other dev tool for this).


When the view loads we’ll see that there are 18 individual HTTP requests to pull resources from the server (17 for the JavaScript/CSS and 1 for the HTML view itself). This results in 1.1MB of resources loading in 1.01 seconds (these metrics can be seen in the foot of the chrome dev tools window in the screenshot above). Note that individual page load times will vary depending on your machine spec, Tomcat memory settings etc.

Options for Web Resource Optimisation 

Now we’re going to look at how we can optimise these resources. In particular we’ll be looking at
  • Resource Minification
  • Resource Merging
  • Resource Compression
  • Resource Caching

Resource Minification
Resource minification is a process where by all non-essential characters are removed from a resource without impacting its functionality. Non-essential characters typically include white space, comments and new line characters. Resource minification can substantially reduce the size of a file and therefore the amount of time taken to retrieve that resource from the server. In the optimised example later you’ll see minification being applied to both JavaScript and CSS resources.

Resource Merging
Resource Merging is a process where by multiple distinct resources are merged into a single file. In our example above we have 11 separate JavaScript files - merging these resources into a single JavaScript file will reduce the number of HTTP requests from 11 to just 1. Resource merging reduces the load on the server and in conjunction with resource minification (described above) will reduce the amount of time taken to load the page resources. In the optimised example later you’ll see how WRO4J provides granular control over what resources are merged together.

Resource Compression
Resource compression is a process whereby resources are compressed on the server prior to being sent to the browser. GZIP compression is such an approach and substantially reduces the resource size, resulting in faster page loads on the browser. In order for GZIP compression to work the browser must tell the server that it would like a compressed resource using a HTTP header like Accept-encoding:gzip. If the server supports GZIP compression it will compress whatever resources were requested and return them to the client. The client must then decompress the resources on the client side before using them. If the server doesn't support GZIP compression it will return the request resources in their standard format. In the optimised example later you’ll see how GZIP compression can be enabled in WRO4J.

Resource Caching
Resource caching is a process where by resources are served from memory as opposed to being retrieved from the file system. When a resource request reaches the server it will first check its cache for that resource and if its available it will return the resource to the client. If the requested resource isn't available in the cache the server will retrieve the resource from disk, add it to the cache and then return it to the client. Retrieving resources from the cache is preferable as it reduces load on the server and reduces the response time for the client. Resource caching is particularly desirable when used in conjunction with the optimisation methods discussed above. Resource merging, minification and compression can be performed on the first request for a resource before it is added to the cache. Subsequent requests for the same resource will return the merged, minified and compressed resource straight from the cache.

Configuring Resource Optimisation with WRO4J

Below is a step by step guide to configuring WRO4J in a Spring web application.

POM Configuration

Add the following entry to your POM so that the required WRO4J resources are available to the application.
 <dependency>  
    <groupId>ro.isdc.wro4j</groupId>  
    <artifactId>wro4j-core</artifactId>  
    <version>1.6.3</version>  
 </dependency>  

Filter Definition

Define a filter and filter mapping in your web.xml file for WRO4J. These are required to tell WRO4J to handle incoming HTTP requests for resources.
 <filter>  
    <filter-name>WebResourceOptimizer</filter-name>  
    <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>  
    <init-param>  
       <param-name>targetBeanName</param-name>  
       <param-value>wroFilter</param-value>  
    </init-param>  
    <init-param>  
       <param-name>targetFilterLifecycle</param-name>  
       <param-value>true</param-value>  
    </init-param>  
 </filter>  
 <filter-mapping>  
    <filter-name>WebResourceOptimizer</filter-name>  
    <url-pattern>/wro/*</url-pattern>  
 </filter-mapping>  
The delegating filter proxy delegates off to a Spring managed bean that we'll define later. This allows us to configure WRO4J using standard Spring property management. Note that the target bean name must match the Spring managed bean that is defined in the Spring configuration file. Also, the URL pattern defined in the filter mapping tells WRO4J to handle all incoming requests for URLs matching the pattern /wro/*

WRO.xml Configuration

In the webapp/WEB-INF directory create a wro.xml file. This file will contain grouping definitions that tell WRO4J what resources you want to merge together and how they should be named.
 <?xml version="1.0" encoding="UTF-8"?>  
 <groups xmlns="http://www.isdc.ro/wro"  
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
         xsi:schemaLocation="http://www.isdc.ro/wro wro.xsd">  
   
   <!-- CSS resources -->  
   <group name="bundledCss">  
     <css>/resources/css/lib/bootstrap/3.0.0/css/bootstrap.css</css>  
     <css>/resources/css/lib/bootstrap/3.0.0/css/bootstrap-theme.css</css>  
     <css>/resources/css/lib/font-awesome/4.0.0/css/font-awesome.css</css>  
     <css>/resources/js/lib/data-tables/1.9.4/media/css/demo_table.css</css>  
     <css>/resources/js/lib/datepicker/css/datepicker.css</css>    
     <css>/resources/css/lib/jquery/jquery.fileupload.css</css>    
     <css>/resources/css/lib/jquery-ui-1.10.3.custom.css</css>           
   </group>  
   
   <!-- JS resources -->  
   <group name="bundledJs">  
     <js>/resources/js/lib/jquery/jquery-1.8.3.js</js>  
     <js>/resources/css/lib/bootstrap.js</js>  
     <js>/resources/js/lib/data-tables/1.9.4/media/js/jquery.dataTables.js</js>  
     <js>/resources/js/lib/underscore/underscore.js</js>  
     <js>/resources/js/lib/underscore/underscore.string.js</js>    
     <js>/resources/js/lib/jquery.blockUI.js</js>  
     <js>/resources/js/lib/jquery.bootstrap-growl.js</js>    
     <js>/resources/js/lib/datepicker/js/bootstrap-datepicker.js</js>     
     <js>/resources/js/lib/jquery/plugins/jquery.ui.widget.js</js>  
     <js>/resources/js/lib/jquery/plugins/jquery.iframe-transport.js</js>  
     <js>/resources/js/lib/jquery/plugins/jquery.fileupload.js</js>   
     <js>/resources/js/lib/jquery.slimscroll.js</js>   
     <js>/resources/js/lib/jquery-ui-1.10.3.custom.js</js>   
   </group>  
   
 </groups>  
You'll can see from the configuration above that we've merged
  • 11 JavaScript resources into a single group that can be referenced by the name bundledJS. 
  • 7 CSS resources into a single group that can be referenced by the name bundledCss 

WRO4J Spring configuration

 <?xml version="1.0" encoding="UTF-8" standalone="no"?>  
 <beans xmlns="http://www.springframework.org/schema/beans"  
     xmlns:context="http://www.springframework.org/schema/context"  
     xmlns:mvc="http://www.springframework.org/schema/mvc"  
     xmlns:p="http://www.springframework.org/schema/p"  
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
     xmlns:util="http://www.springframework.org/schema/util"  
     xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd  
                         http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd  
                         http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd  
                         http://www.springframework.org/schema/util  
                         http://www.springframework.org/schema/util/spring-util-3.0.xsd">  
   
        
      <mvc:annotation-driven/>  
   
      <!-- Setup spring to pull in @Controller, @RequestMapping, etc -->  
      <context:component-scan base-package="com.blog.samples.wro4j" />  
   
      <!-- Handles HTTP GET requests for /resources/** by efficiently serving up static resources in the ${webappRoot}/resources directory -->  
      <mvc:resources mapping="/resources/**" location="/resources/" />  
   
      <!-- Resolves views selected for rendering by @Controllers to .jsp resources in the /WEB-INF/views directory -->  
      <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">  
           <property name="prefix" value="/WEB-INF/views/" />  
           <property name="suffix" value=".jsp" />  
      </bean>  
        
      <mvc:view-controller path="/" view-name="/testViewNoOptimisation"/>       
   
      <!-- WRO config -->  
      <bean id="wroFilter" class="ro.isdc.wro.http.ConfigurableWroFilter">  
       <property name="properties" ref="wroProperties"/>  
      </bean>  
        
      <bean id="wroProperties" class="org.springframework.beans.factory.config.PropertiesFactoryBean">   
        <property name="location" value="classpath:wro.properties" />   
      </bean>   
             
 </beans>  
   
You'll notice that only 2 pieces of WRO4J configuration are required.
  • A configurable filter that will be invoked by the DelegatingFilterProxy we defined earlier in web.xml. Make sure that the bean name matches the targetBeanName attribute define in the filter definition in web.xml. 
  • A PropertiesFactoryBean for loading the wro4j properties file and making it available to the ConfigurableWroFilter mentioned above.

WRO4J Property Configuration 

WRO4J has the following configuration options. These are explained below.
 debug=false  
 gzipEnabled=true  
 cacheGzippedContent=true  
 jmxEnabled=false  
 mbeanName=wro  
 cacheUpdatePeriod=100  
 modelUpdatePeriod=0  
 disableCache=false  
 encoding=UTF-8  
 managerFactoryClassName=ro.isdc.wro.manager.factory.ConfigurableWroManagerFactory  
 preProcessors=cssUrlRewriting,semicolonAppender,jsMin  
 uriLocators=servletContext,uri,classpath  
  • debug - debug set to true allows you to add ?minimise=false to the request URL. This is pretty handy during development when debugging JavaScript.   
  • gzipEnabled - flag enables or disables gzip compression
  • cacheGzippedContent - when this is enabled resources will be compressed only once and added to the cache. Subsequent requests for the same resource will be taken from the cache. If this flag is disabled every resource requested will be compressed. Note that disabling this setting will result in slower response times
  • jmxEnabled - defines whether or not WRO4J is exposed to JMX console
  • mbeanName - Name given to mBean object exposed to JMX console
  • cacheUpdatePeriod - defines how often (in seconds) the cache is flushed
  • modelUpdatePeriod - defines how often (in seconds) the bundle definitions are flushed in wro.xml
  • disableCache - option for use in debug mode only results in every request being processed from scratch. This is useful when debugging.
  • encoding - type of encoding used. Default is UTF8
  • managerFactoryClassName - required in order to configure pre and post processors (see below)
  • preProcessors - a comma separated list of processors to run during resource pre processing. The configuration above uses the following pre processors
    • cssUrlRewriting - when CSS resources are merged the relative URLs are broken (links to images for example). This pre processor rewrites URLs so that they refer to the appropriate resources.
    • semiColonAppender - adds a semi colon to the end of each JavaScript file if its missing. This avoids issue when multiple JavaScript files are merged.
    • cssMin - runs cssMinification to remove non essential characters such as comments and white space
    • jsMin - runs JSMin utility to remove non essential characters such as comments and white space

Using WRO4J in the JSP

Now that we've configured WRO4J lets take a look at how it's used in our sample application.  We'll create another simple JSP view that imports the same resources (11 JavaScript and 6 CSS) as before. This time however, we'll use WRO4J to serve the resources. The JSP is defined as follows.
 <%@taglib uri="http://www.springframework.org/tags" prefix="spring" %>  
 <!DOCTYPE html>  
 <html lang="en">  
   <head>  
     <link rel="stylesheet" type="text/css" href="/wro4j-sample/wro/bundledCss.css" />  
     <script type="text/javascript" src="<spring:url value="/wro/bundledJs.js"/>"></script>         
   </head>  
   
   <body>                       
      <div class="container">  
           <H1>Not much to see here, just a test page with a bunch of JavaScript imports. Now with resource optimisation!</H1>            
      </div>     
   </body>  
   
 </html>  
A couple of things to note here
  • 11 JavaScript references from the original view (defined earlier) have been replaced with a single reference to /wro/bundledJs.js. 
  • 6 CSS references from the original view (defined earlier) have been replaced with a single reference to /wro/bundledCss.js.
  • All JavaScript and CSS requests now have the pattern /wro/* so that they are serviced by the WRO filter we defined earlier. 
  • The name of the JavaScript and CSS bundles (bundledJs.js & bundledCss.css) are taken from the group names we defined in wro.xml

Testing the Optimised View

We can load this view by navigating to http://localhost:8080/wro4j-sample/optimisedView in the sample application. Before loading this view open chrome dev tools (CTRL+SHFT +I) and switch to the Network tab.
When the view loads we’ll see that there are now 3 HTTP requests to pull resources from the server, 1 for the bundled JavaScript, 1 for the bundled CSS and 1 for the HTML view itself. 
This results in 245KB of resources loading in 316 ms (these metrics can be seen in the foot of the chrome dev tools window in the screenshot above). Note that individual page load times will vary depending on your machine spec, Tomcat memory settings etc.

Simple View vs WRO4J Optimised View

So what are the results when we compare the standard JSP view with the WRO4J optimised view? Remember that the resources referenced in both views are exactly the same, the only difference being the way they are served. The results are impressive
  • fewer HTTP requests (reduced from 18 to 3)
  • smaller message payload returned from the server (reduced from 1.1MB to 245KB)
  • reduced load time for resources (reduced from 1.01 seconds to 316ms)
The examples used in this post are primitive but they do demonstrate the fundamentals of web resource optimisation and the associated benefits. Ultimately, web resource optimisation should result in faster loading, more responsive applications and a better user experience.

Sample Code

Sample code for this post is available at https://github.com/briansjavablog/wro4j-sample. Feel free to experiment with the code and as always, comments/questions are welcome.   

Tuesday, 15 October 2013

Before & After Gif Transition

A friend has just released a cool new app for creating 'before and after' animated GIFs using your Android phone. Take a look... https://play.google.com/store/apps/details?id=com.dallapps.lite



Monday, 22 April 2013

Spring AOP Tutorial

Introduction
Application logic can be broken into 2 distinct areas, core business logic and cross cutting concerns. Business logic is code written to satisfy a functional requirement, while a cross cutting concern is 'utility' logic that is agnostic to any specific business process and required by many parts of the application. Examples include logging, transaction management, performance monitoring and security. While none of these address a functional requirement, they remain fundamental parts of the application runtime.

Cross cutting concerns present 2 main challenges.
  1. They tend to be 'scattered' across the application which can lead to considerable duplicate code. Logging or performance monitoring is a good example. 
  2. They become tangled with application business logic and result in code that is difficult to maintain because there is no clear Separation of Concerns.  
Aspect Oriented Programming (AOP) aims to address these challenges by providing a means of modularising application logic, so that each module addresses a distinct concern. Take performance monitoring as an example - rather than have performance monitoring logic littered across the application, AOP provides a means of modularising this logic, and applying it to various parts of the application at runtime. This provides a clear Separation of Concerns as performance monitoring logic is no longer tangled with business logic throughout the application.

Key Terms
The detailed inner workings of AOP is well beyond the scope of this post. However, I've provided a brief definition of key terms that are referred to throughout this tutorial.

Aspect - A self contained module that attempts to address a specific cross cutting concern. In this tutorial  the Aspect will contain performance monitoring functionality.
Advice - Logic that is invoked at a specific point in a programs execution.
Join Point - A specific point in a programs execution where advice is applied. Note that advice can be applied before and after a Join Point.
Pointcut - An expression that identifies Join Points matching certain criteria. For example we can define a Pointcut that identifies all public methods in a package. These specific points in program execution (JoinPoints) have advice applied to them at runtime.    

How does it work?
If Advice (performance monitoring logic in this case) is completely separate from the business logic, how is it actually called? AOP takes care of this at runtime by wrapping the target object inside a proxy. The proxy intercepts incoming requests to the target object and calls the performance monitor 'utility' logic before it calls the target object. The proxy can also invoke utility logic after the target object has been called. Figure 1.0 below shows the steps involved.

Figure 1.0 - AOP Proxy
Sample Code
This sample code described in this tutorial will demonstrate some of Springs AOP capabilities by showing you how to build a simple performance monitor. Springs AOP support can be set up with either XML configuration or annotations. The XML approach is typically used in older applications running Spring 2.5, whereas newer applications running Spring 3.X typically use annotations. For completeness this tutorial will cover both approaches.

Project Structure
I've create 2 separate projects, one using XML configuration and one using annotations. The project structure is identical for both - the only difference is the configuration. To keep things simple both applications will be kicked off via a main method and call a simple Service. We'll then use AOP to apply performance monitoring to the service call. The structure of both projects is shown below.
Figure 2.0 - Project Structure

I'll start off by explaining the components that are identical in both the XML and annotation driven approaches. Then I'll move on to explaining the configuration specific to each approach. Full source code for both projects will be available to download at the bottom of this post.    

Service Interface
CalculationService.java is an interface that defines the simple service method shown below. Note that programming to interfaces is regarded as good practice, but has even more significance when using Springs AOP support. In order for Spring to use the dynamic proxy approach described earlier (see figure 1.0), it is expected that the target object (CalculationServiceImpl in this case) implements an interface. Its also possible to use AOP on a class that doesn't implement an interface. In this case CGLIB is used to proxy the target object.
1:  package com.blog.samples.aop;  
2:    
3:  /**  
4:   * Dummy service interface  
5:   *  
6:   */  
7:  public interface CalculationService  
8:  {  
9:       /**  
10:        * A dummy method  
11:        *  
12:        * @param employee_p  
13:        * @return  
14:        */  
15:       public void dummyServiceMethod();  
16:  }  

Service Implementation
CalculationServiceImpl.java implements the service interface defined above. The dummyServiceMethod simply logs a message and calls the randomSleep method to sleep the current thread for a random period of time. This allows us to simulate varying response times from the service method which will useful for testing our performance monitor later.
1:  package com.blog.samples.aop;  
2:    
3:  import org.apache.log4j.Logger;  
4:    
5:  /**  
6:   * Dummy service calculates implementation implements dummyServiceMethod.  
7:   *  
8:   */  
9:  public class CalculationServiceImpl implements CalculationService  
10:  {  
11:       private static final Logger logger_c = Logger.getLogger(CalculationServiceImpl.class);  
12:    
13:       /**  
14:        * dummyServiceMethod simply logs a message and calls randomSleep to  
15:        * sleep the current thread for a random period of time. This allows us  
16:        * to simulate varying response times from the service method which is  
17:        * useful for testing our performance monitor  
18:        *  
19:        * @param employee_p  
20:        * @return  
21:        */  
22:       public void dummyServiceMethod()  
23:       {  
24:            logger_c.debug("Doing some service stuff here...");  
25:    
26:            /* Sleep thread for random period so as to vary service execution time */  
27:            randomSleep();  
28:       }  
29:    
30:       /**  
31:        * Sleep thread for random period  
32:        */  
33:       private void randomSleep()  
34:       {  
35:            try  
36:            {  
37:                 Thread.sleep((long)(Math.random() * 1000));  
38:            }  
39:            catch (InterruptedException ie_p)  
40:            {  
41:                 logger_c.error("Error occurred sleeping thread", ie_p);  
42:            }  
43:       }  
44:  }  

Test Harness
RunTest.java uses a main method to load the Spring configuration and invokes the test service 50 times. We call the service 50 times as it helps demonstrate some of the statistics that can be gathered by the performance monitor defined later.
1:  package com.blog.samples.aop;  
2:    
3:  import org.apache.log4j.Logger;  
4:  import org.springframework.context.support.ClassPathXmlApplicationContext;  
5:    
6:  public class RunTest  
7:  {  
8:       private static final Logger logger_c = Logger.getLogger(RunTest.class);  
9:    
10:       public static void main (String [] args)  
11:       {  
12:            logger_c.debug("loading spring application context");  
13:            ClassPathXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext("classpath:spring-config.xml");  
14:    
15:            /* get been from application context */  
16:            CalculationService calculationService = (CalculationService)applicationContext.getBean("calculationService");  
17:    
18:            /* call service 50 times so that we can see aggregated performance statistics for service call */  
19:            for(int i=0; i<50; i++)  
20:            {  
21:                 /* invoke dummy service */  
22:                 calculationService.dummyServiceMethod();  
23:            }  
24:    
25:            /* close down spring application context */  
26:            applicationContext.stop();  
27:       }  
28:  }  
Line 13 - load Spring configuration from src/main/resources/spring-config.xml
Line 16 - load calculation service from bean factory
Lines 19 to 23 - invoke calculation service 50 times so that we can gather a variety of metrics using the performance monitor (defined later).

Spring Configuration (XML Approach)
This section will detail the components required to enable AOP using traditional XML configuration. The Calculation Service bean, performance monitor bean and AOP configuration are defined in spring-configuration.xml below.
1:  <?xml version="1.0" encoding="UTF-8"?>  
2:  <beans xmlns="http://www.springframework.org/schema/beans"  
3:                xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
4:                xmlns:context="http://www.springframework.org/schema/context"  
5:                xmlns:aop="http://www.springframework.org/schema/aop"  
6:                xsi:schemaLocation="http://www.springframework.org/schema/context  
7:                                    http://www.springframework.org/schema/context/spring-context-2.5.xsd  
8:                                    http://www.springframework.org/schema/beans  
9:                                    http://www.springframework.org/schema/beans/spring-beans-2.5.xsd  
10:                                   http://www.springframework.org/schema/aop  
11:                                   http://www.springframework.org/schema/aop/spring-aop-2.0.xsd">  
12:    
13:    
14:       <!-- Calculation Service bean definition - this is the bean that we want to monitor performance on -->  
15:       <bean id="calculationService" class="com.blog.samples.aop.CalculationServiceImpl" />  
16:    
17:    <!-- ================================================================== -->  
18:    <!--         A O P C O N F I G B E A N S                        -->  
19:    <!-- ================================================================== -->  
20:    
21:      <!-- Performance Monitor bean definition - this bean contains performance monitoring logic  
22:           like starting and stopping the monitoring, logging statistics etc -->  
23:      <bean id="performanceMonitor" class="com.blog.samples.aop.PerformanceMonitor" />  
24:    
25:      <!-- AOP configuration -->  
26:      <aop:config>  
27:    
28:        <!-- Name of the Aspect we're defining -->  
29:        <aop:aspect ref="performanceMonitor">  
30:    
31:          <!--  
32:            the Pointcut expression specifies where the advice (performance monitoring code) will  
33:            be applied. Pointcut expressions are very flexible and can be as specific or as generic  
34:            as you like. In this instance we configured the Pointcut expression so that it covers  
35:            all methods in the com.blog.samples.aop.CalculationService class. However, we could  
36:            have configured it so that advice is only applied to a single Service method, or even  
37:            opened it up to all classes within the com.blog.samples.aop package. Obviously this  
38:            level of flexibility isn't required for our trivial example, but is very useful in  
39:            large enterprise applications.  
40:           -->  
41:          <aop:pointcut  
42:               id="serviceMethod"  
43:               expression="execution(* com.blog.samples.aop.CalculationService.*(..))"/>  
44:    
45:          <!--  
46:               <aop:before ... /> is used to specify the advice that should be run before a Pointcut  
47:               method - invoke the startMonitor method to start timing method execution before the Pointcut  
48:               pointcut-ref - the point cut reference specifies the point of execution in the code where the  
49:                              advice should be run. In this case its before any public method in the  
50:                              Calculation Service.  
51:          -->  
52:          <aop:before  
53:               method="startMonitor"  
54:               pointcut-ref="serviceMethod"/>  
55:    
56:          <!--  
57:               <aop:after ... /> is used to specify the advice that should be run after a Pointcut  
58:               method - invoke the stopMonitor method to stop timing method execution after the Pointcut  
59:               pointcut-ref - the point cut reference specifies the point of execution in the code where the  
60:                              advice should be run. In this case its after any public method in the  
61:                              Calculation Service.  
62:          -->  
63:          <aop:after  
64:               method="stopMonitor"  
65:               pointcut-ref="serviceMethod"/>  
66:    
67:          <!--  
68:               method - invoke the log method to log method execution metrics after the Pointcut  
69:               pointcut-ref - the point cut reference specifies the point of execution in the code where the  
70:                              advice should be run. In this case its after any public method in the  
71:                              Calculation Service.  
72:          -->  
73:          <aop:after  
74:               method="log"  
75:               pointcut-ref="serviceMethod"/>  
76:    
77:          <!--  
78:               <aop:after-throwing ... /> is used to define the advice to be run after an exception is thrown at a Pointcut.  
79:               method - invoke the stopMonitor method to stop timing method execution after an exception is thrown at the Pointcut  
80:               pointcut-ref - the point cut reference specifies the point of execution in the code where the  
81:                              advice should be run. In this case its after any public method in the  
82:                              Calculation Service.  
83:          -->  
84:          <aop:after-throwing  
85:               method="stopMonitor"  
86:               pointcut-ref="serviceMethod"/>  
87:    
88:          <!--  
89:               method - invoke the log method to stop timing method execution after an exception is thrown at the Pointcut  
90:               pointcut-ref - the point cut reference specifies the point of execution in the code where the  
91:                              advice should be run. In this case its after any public method in the  
92:                              Calculation Service.  
93:          -->  
94:          <aop:after-throwing  
95:               method="log"  
96:               pointcut-ref="serviceMethod"/>  
97:    
98:        </aop:aspect>  
99:      </aop:config>  
100:    
101:    <!-- ***************************** -->  
102:    
103:  </beans>  

Performance Monitor (XML Configuration Approach)
The AOP configuration above uses a PerformanceMonitor class to apply advice at specified Pointcucts. The class is defined below and uses the JAMon API to provide basic performance monitoring functionality to captures and log performance metrics.
1:  package com.blog.samples.aop;  
2:    
3:  import java.util.Date;  
4:  import org.aspectj.lang.JoinPoint;  
5:  import org.apache.log4j.Logger;  
6:  import com.jamonapi.Monitor;  
7:  import com.jamonapi.MonitorFactory;  
8:    
9:  /**  
10:   * Performance monitor use the Jamon library to provide basic performance  
11:   * monitoring and logging functionality.  
12:   *  
13:   */  
14:  public class PerformanceMonitor  
15:  {  
16:       private static final Logger logger_c = Logger.getLogger(PerformanceMonitor.class);  
17:       private final String MONITOR = "PERFORMANCE_MONITOR";  
18:       private Monitor monitor_i;  
19:    
20:       /**  
21:        * Start monitor  
22:        */  
23:       public void startMonitor()  
24:       {  
25:            monitor_i = MonitorFactory.start(MONITOR);  
26:       }  
27:    
28:       /**  
29:        * Stop monitor  
30:        */  
31:       public void stopMonitor()  
32:       {  
33:            monitor_i.stop();  
34:       }  
35:    
36:       /**  
37:        * get last access  
38:        *  
39:        * @return Date  
40:        */  
41:       public Date getLastAccess()  
42:       {  
43:            return monitor_i.getLastAccess();  
44:       }  
45:    
46:       /**  
47:        * get call count  
48:        *  
49:        * @return int  
50:        */  
51:       public int getCallCount()  
52:       {  
53:            return (int) monitor_i.getHits();  
54:       }  
55:    
56:       /**  
57:        * get average call time  
58:        *  
59:        * @return double  
60:        */  
61:       public double getAverageCallTime()  
62:       {  
63:            return monitor_i.getAvg() / 1000;  
64:       }  
65:    
66:       /**  
67:        * get last call time  
68:        *  
69:        * @return double  
70:        */  
71:       public double getLastCallTime()  
72:       {  
73:            return monitor_i.getLastValue() / 1000;  
74:       }  
75:    
76:       /**  
77:        * get maximum call time  
78:        *  
79:        * @return double  
80:        */  
81:       public double getMaximumCallTime()  
82:       {  
83:            return monitor_i.getMax() / 1000;  
84:       }  
85:    
86:       /**  
87:        * get minimum call time  
88:        *  
89:        * @return double  
90:        */  
91:       public double getMinimumCallTime()  
92:       {  
93:            return monitor_i.getMin() / 1000;  
94:       }  
95:    
96:       /**  
97:        * get total call time  
98:        *  
99:        * @return double  
100:        */  
101:       public double getTotalCallTime()  
102:       {  
103:            return monitor_i.getTotal() / 1000;  
104:       }  
105:    
106:       /**  
107:        * log statistics  
108:        *  
109:        * @param joinPoint_p  
110:        */  
111:       public void log(JoinPoint joinPoint_p)  
112:       {  
113:            StringBuffer sb = new StringBuffer();  
114:    
115:            sb.append("\n");  
116:            sb.append("*======================================");  
117:            sb.append("\n");  
118:            sb.append("*    PERFORMANCE STATISTICS    *");  
119:            sb.append("\n");  
120:            sb.append("*======================================");  
121:            sb.append("\n");  
122:            sb.append("* Method Name: " + joinPoint_p.getSignature().getName());  
123:            sb.append("\n");  
124:            sb.append("* Execution Date: ").append(this.getLastAccess());  
125:            sb.append("\n");  
126:            sb.append("* Last Execution Time: ").append(this.getLastCallTime()).append(" sec");  
127:            sb.append("\n");  
128:            sb.append("* Service Calls: ").append(((this.getCallCount())));  
129:            sb.append("\n");  
130:            sb.append("* Avg Execution Time: ").append(this.getAverageCallTime()).append(" sec");  
131:            sb.append("\n");  
132:            sb.append("* Total Execution TIme: ").append(this.getTotalCallTime()).append(" sec");  
133:            sb.append("\n");  
134:            sb.append("* Min Execution Time: ").append(this.getMinimumCallTime()).append(" sec");  
135:            sb.append("\n");  
136:            sb.append("* Max Execution Time: ").append(this.getMaximumCallTime()).append(" sec");  
137:            sb.append("\n");  
138:            sb.append("*======================================");  
139:    
140:            logger_c.info(sb.toString());  
141:       }  
142:  }  

Line 23 - Method starts monitor. This is called before the service method is invoked.
Line 31 - Method stops monitor. This is called after the service method completes.
Line 41 to 101 - Series of utility methods gather various metrics from the Jamon monitor.
Line 111 - Method logs performance statistics gathered by monitor. The JoinPoint argument provides access to the target object - in this instance we use the JoinPoint to access the name of the target method being invoked.

Spring Configuration (Annotation Driven)
This section shows how AOP can be configured using annotations. You'll notice that this approach is less verbose than the XML approach described earlier. Note that Pointcuts are no longer defined using XML, instead Spring will look for beans annotated with @Aspect and register them as aspects.
1:  <beans xmlns="http://www.springframework.org/schema/beans"  
2:       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
3:       xmlns:aop="http://www.springframework.org/schema/aop"  
4:       xsi:schemaLocation="http://www.springframework.org/schema/beans  
5:                                http://www.springframework.org/schema/beans/spring-beans-3.0.xsd  
6:                                http://www.springframework.org/schema/aop  
7:                                http://www.springframework.org/schema/aop/spring-aop-3.0.xsd ">  
8:    
9:       <!-- this registers beans annotated with @Aspect -->  
10:       <aop:aspectj-autoproxy />  
11:    
12:       <!-- Calculation Service bean definition - this is the bean that we want to monitor performance on -->  
13:       <bean id="calculationService" class="com.blog.samples.aop.CalculationServiceImpl" />  
14:    
15:       <!-- Performance Monitor bean definition - this bean contains performance monitoring logic  
16:            for starting and stopping the monitoring, logging statistics etc -->  
17:       <bean id="performanceMonitor" class="com.blog.samples.aop.PerformanceMonitor" />  
18:    
19:  </beans>  

Performance Monitor (Annotation Driven)
There are a number of significant differences between the Performance monitor defined below, and the one we defined earlier (using the XML approach). Here AOP is configured using class level meta data in the form of annotations.The @Aspect annotation tells Spring that this class contains advice and should be applied to specified Pointcuts at runtime. The Pointcuts are defined using method level annotations and have the same format as the ones defined earlier.
1:  package com.blog.samples.aop;  
2:    
3:  import java.util.Date;  
4:    
5:  import org.aspectj.lang.JoinPoint;  
6:  import org.aspectj.lang.annotation.After;  
7:  import org.aspectj.lang.annotation.Aspect;  
8:  import org.aspectj.lang.annotation.Before;  
9:  import org.apache.log4j.Logger;  
10:    
11:  import com.jamonapi.Monitor;  
12:  import com.jamonapi.MonitorFactory;  
13:    
14:  /*  
15:   * @Aspect tells the Spring framework that this class contains advice that should  
16:   * be applied to one or more specified Pointcuts at runtime  
17:   */  
18:  @Aspect  
19:  public class PerformanceMonitor  
20:  {  
21:       private static final Logger logger_c = Logger.getLogger(PerformanceMonitor.class);  
22:       private final String MONITOR = "PERFORMANCE_MONITOR";  
23:       private Monitor monitor_i;  
24:    
25:       /*  
26:        * @Before tells the Spring framework that this method should be invoked before the specified Pointcut.  
27:        * The Pointcut expression here is identical to the one we used in the XML configuration example  
28:        */  
29:       @Before("execution(* com.blog.samples.aop.CalculationService.dummyServiceMethod())")  
30:       public void startMonitor()  
31:       {  
32:            monitor_i = MonitorFactory.start(MONITOR);  
33:       }  
34:    
35:       /*  
36:        * @After tells the Spring framework that this method should be invoked after the specified Pointcut.  
37:        * The Pointcut expression here is identical to the one we used in the XML configuration example  
38:        */  
39:       @After("execution(* com.blog.samples.aop.CalculationService.dummyServiceMethod())")  
40:       public void stopMonitor()  
41:       {  
42:            monitor_i.stop();  
43:       }  
44:    
45:       /**  
46:        * get last access  
47:        *  
48:        * @return Date  
49:        */  
50:       public Date getLastAccess()  
51:       {  
52:            return monitor_i.getLastAccess();  
53:       }  
54:    
55:       /**  
56:        * get call count  
57:        *  
58:        * @return int  
59:        */  
60:       public int getCallCount()  
61:       {  
62:            return (int) monitor_i.getHits();  
63:       }  
64:    
65:       /**  
66:        * get average call time  
67:        *  
68:        * @return double  
69:        */  
70:       public double getAverageCallTime()  
71:       {  
72:            return monitor_i.getAvg() / 1000;  
73:       }  
74:    
75:       /**  
76:        * get last call time  
77:        *  
78:        * @return double  
79:        */  
80:       public double getLastCallTime()  
81:       {  
82:            return monitor_i.getLastValue() / 1000;  
83:       }  
84:    
85:       /**  
86:        * get maximum call time  
87:        *  
88:        * @return double  
89:        */  
90:       public double getMaximumCallTime()  
91:       {  
92:            return monitor_i.getMax() / 1000;  
93:       }  
94:    
95:       /**  
96:        * get minimum call time  
97:        *  
98:        * @return double  
99:        */  
100:       public double getMinimumCallTime()  
101:       {  
102:            return monitor_i.getMin() / 1000;  
103:       }  
104:    
105:       /**  
106:        * get total call time  
107:        *  
108:        * @return double  
109:        */  
110:       public double getTotalCallTime()  
111:       {  
112:            return monitor_i.getTotal() / 1000;  
113:       }  
114:    
115:       /*  
116:        * @After tells the Spring framework that this method should be invoked after the specified Pointcut.  
117:        * The Pointcut expression here is identical to the one we used in the XML configuration example  
118:        */  
119:       @After("execution(* com.blog.samples.aop.CalculationService.dummyServiceMethod())")  
120:       public void log(JoinPoint joinPoint_p)  
121:       {  
122:            StringBuffer sb = new StringBuffer();  
123:    
124:            sb.append("\n");  
125:            sb.append("*======================================");  
126:            sb.append("\n");  
127:            sb.append("*    PERFORMANCE STATISTICS    *");  
128:            sb.append("\n");  
129:            sb.append("*======================================");  
130:            sb.append("\n");  
131:            sb.append("* Method Name: " + joinPoint_p.getSignature().getName());  
132:            sb.append("\n");  
133:            sb.append("* Execution Date: ").append(this.getLastAccess());  
134:            sb.append("\n");  
135:            sb.append("* Last Execution Time: ").append(this.getLastCallTime()).append(" sec");  
136:            sb.append("\n");  
137:            sb.append("* Service Calls: ").append(((this.getCallCount())));  
138:            sb.append("\n");  
139:            sb.append("* Avg Execution Time: ").append(this.getAverageCallTime()).append(" sec");  
140:            sb.append("\n");  
141:            sb.append("* Total Execution TIme: ").append(this.getTotalCallTime()).append(" sec");  
142:            sb.append("\n");  
143:            sb.append("* Min Execution Time: ").append(this.getMinimumCallTime()).append(" sec");  
144:            sb.append("\n");  
145:            sb.append("* Max Execution Time: ").append(this.getMaximumCallTime()).append(" sec");  
146:            sb.append("\n");  
147:            sb.append("*======================================");  
148:    
149:            logger_c.info(sb.toString());  
150:       }  
151:  }  

Running the Test Harness
At this point I've covered all the various components and explained how AOP can be configured using both XML and annotations. Its time to run the test harness and see the performance monitor at work. Simply kick off the main method in RunTest.java and you should see the Calculation Service invoked 50 times, with performance metrics logged for each call. For brevity the extract below shows the end of the log.
1:  DEBUG: [Apr-16 18:37:12,945] samples.aop.CalculationServiceImpl - Doing some service stuff here...  
2:  INFO : [Apr-16 18:37:13,493] samples.aop.PerformanceMonitor -   
3:  *======================================  
4:  *    PERFORMANCE STATISTICS    *  
5:  *======================================  
6:  * Method Name: dummyServiceMethod  
7:  * Execution Date: Tue Apr 16 18:37:12 BST 2013  
8:  * Last Execution Time: 0.063 sec  
9:  * Service Calls: 49  
10:  * Avg Execution Time: 0.5139387755102041 sec  
11:  * Total Execution TIme: 25.183 sec  
12:  * Min Execution Time: 0.024 sec  
13:  * Max Execution Time: 0.988 sec  
14:  *======================================  
15:  DEBUG: [Apr-16 18:37:13,493] samples.aop.CalculationServiceImpl - Doing some service stuff here...  
16:  INFO : [Apr-16 18:37:14,176] samples.aop.PerformanceMonitor -   
17:  *======================================  
18:  *    PERFORMANCE STATISTICS    *  
19:  *======================================  
20:  * Method Name: dummyServiceMethod  
21:  * Execution Date: Tue Apr 16 18:37:13 BST 2013  
22:  * Last Execution Time: 0.548 sec  
23:  * Service Calls: 50  
24:  * Avg Execution Time: 0.51462 sec  
25:  * Total Execution TIme: 25.731 sec  
26:  * Min Execution Time: 0.024 sec  
27:  * Max Execution Time: 0.988 sec  
Various performance statistics have been logged, including minimum, maximum and average execution times. These metrics were logged every time the service method was invoked, and without adding cross cutting monitoring logic to the Calculation Service.

Summary
This tutorial has demonstrated how AOP can be used to encapsulate a cross cutting concern, apply it to business logic at runtime, and ensure a clear separation of concerns. Although the performance monitor example is trivial, it demonstrates a realistic use case for AOP and could form the basis for a more complete performance monitoring utility. Full source code is available here for the XML and annotation driven approaches, so feel free to download and play around with it. As always, comments or questions are welcome below.

Tuesday, 9 April 2013

jQuery Tables

Introduction
This post will show you how to create rich data grids using the DataTables plugin for jQuery. DataTables is a flexible framework that allows you to create grids using a variety of data sources. This tutorial will show you how DataTables can be used with DOM and JavaScript data sources.

DOM Data Source
The simplest way to use DataTables is with a DOM data source. This approach simply 'decorates' an existing HTML table and is ideal for enhancing plain HTML tables created using template languages like JSP or Velocity.

Make Model Body Shape Engine Size (Cubic Capacity) Horse Power
BMW 320d Saloon 2000 181
BMW 330d Coupe 3000 231
BMW 335d Estate 3000 286
BMW 335i Saloon 3000 306
BMW M3 Coupe 4000 415
BMW 520d Saloon 2000 181
BMW 530d Saloon 3000 231
BMW 535d Estate 3000 286
BMW 535i Saloon 4000 306
BMW M5 Saloon 5000 508

To turn this rather dull looking HTML table into a rich data grid we simply decorate the table on document load.
1:  <script type="text/javascript">  
2:         $(document).ready(function() {  
3:                 $('#sampleGrid').dataTable({        
4:                       "bJQueryUI": true,        
5:                       "iDisplayLength": 4,  
6:                       "sPaginationType": "full_numbers"  
7:                 });            
8:         } );  
9:   </script>  
10:  <table id="sampleGrid">  
11:       <thead>  
12:            <tr>  
13:                 <th>Make</th>  
14:                 <th>Model</th>  
15:                 <th>Body Shape</th>  
16:                 <th>Engine Size (Cubic Capacity)</th>  
17:                 <th>Horse Power</th>  
18:            </tr>  
19:       </thead>  
20:       <tbody>  
21:            <tr>  
22:                 <th>BMW</th>  
23:                 <th>320d</th>  
24:                 <th>Saloon</th>  
25:                 <th>2000</th>  
26:                 <th>181</th>  
27:            </tr>  
28:            <tr>  
29:                 <th>BMW</th>  
30:                 <th>330d</th>  
31:                 <th>Coupe</th>  
32:                 <th>3000</th>  
33:                 <th>231</th>  
34:            </tr>  
35:            <tr>  
36:                 <th>BMW</th>  
37:                 <th>335d</th>  
38:                 <th>Estate</th>  
39:                 <th>3000</th>  
40:                 <th>286</th>  
41:            </tr>  
42:            <tr>  
43:                 <th>BMW</th>  
44:                 <th>335i</th>  
45:                 <th>Saloon</th>  
46:                 <th>3000</th>  
47:                 <th>306</th>  
48:            </tr>  
49:            <tr>  
50:                 <th>BMW</th>  
51:                 <th>M3</th>  
52:                 <th>Coupe</th>  
53:                 <th>4000</th>  
54:                 <th>415</th>  
55:            </tr>  
56:            <tr>  
57:                 <th>BMW</th>  
58:                 <th>520d</th>  
59:                 <th>Saloon</th>  
60:                 <th>2000</th>  
61:                 <th>181</th>  
62:            </tr>  
63:            <tr>  
64:                 <th>BMW</th>  
65:                 <th>530d</th>  
66:                 <th>Saloon</th>  
67:                 <th>3000</th>  
68:                 <th>231</th>  
69:            </tr>  
70:            <tr>  
71:                 <th>BMW</th>  
72:                 <th>535d</th>  
73:                 <th>Estate</th>  
74:                 <th>3000</th>  
75:                 <th>286</th>  
76:            </tr>  
77:            <tr>  
78:                 <th>BMW</th>  
79:                 <th>535i</th>  
80:                 <th>Saloon</th>  
81:                 <th>4000</th>  
82:                 <th>306</th>  
83:            </tr>  
84:            <tr>  
85:                 <th>BMW</th>  
86:                 <th>M5</th>  
87:                 <th>Saloon</th>  
88:                 <th>5000</th>  
89:                 <th>415</th>  
90:            </tr>  
91:       </tbody>  
92:  </table>  
Line 3 - reference the HTML table by id (#sampleGrid) and invoke the DataTables plugin.
Line 4 - apply jQueryUI theme to the grid
Line 5 - set the default number of rows for display.
Line 6 -  set up the pagination control in the footer.
Line 10 to 92 - table definition that we're decorating. Note that the id must be set so that it can be referenced by the plugin on document load)

As you can see below, the resulting data grid is a vast improvement over the plain HTML table and provides some really cool functionality out of the box.

Make Model Body Shape Engine Size (Cubic Capacity) Horse Power
BMW 320d Saloon 2000 181
BMW 330d Coupe 3000 231
BMW 335d Estate 3000 286
BMW 335i Saloon 3000 306
BMW M3 Coupe 4000 415
BMW 520d Saloon 2000 181
BMW 530d Saloon 3000 231
BMW 535d Estate 3000 286
BMW 535i Saloon 4000 306
BMW M5 Saloon 5000 415

JavaScript Array Data Source
DataTables can also be used to create grids declaratively by passing the data as a JavaScript array. This is pretty flexible as it allows you to take data from any source, convert it to a JavaScript array and populate the grid. For example, you might take data keyed by the user or from the server side  and convert it to a JavaScript array, before passing it to the grid. In the sample code below I've hard coded an array of values.
1:  <table id="sampleGridJavaScriptDataSource">  
2:  </table>  
3:  <script type="text/javascript">  
4:         $(document).ready(function() {  
5:                 var someData = [                           
6:                                  [ "Colm", "Toale", "12/04/1981", "Male", "Engineer" ],  
7:                                  [ "Gary", "Gallagher", "12/04/1981", "Male", "Project Manager" ],  
8:                                  [ "Gary", "Jonston", "18/01/1979", "Male", "Doctor" ],  
9:                                  [ "Riain", "McAtamney", "02/07/1985", "Male", "Managing Consultant" ],  
10:                                 [ "Gareth", "Burnside", "07/01/1979", "Male", "Lorry Driver" ],  
11:                                 [ "Connor", "Smyth", "11/03/1986", "Male", "Bank Manager" ],  
12:                                 [ "John", "Flemming", "19/07/1976", "Male", "Accountant" ],  
13:                                 [ "Claire", "Jones", "21/11/1980", "Male", "HR Manager" ],  
14:                                 [ "Sarah", "Nolan", "10/08/1972", "Female", "Receptionist" ],  
15:                                 [ "Steve", "Scott", "27/08/1984", "Male", "Taxi Driver" ]  
16:                               ]  
17:                 $('#sampleGridJavaScriptDataSource').dataTable( {  
18:                      "bJQueryUI": true,        
19:                      "iDisplayLength": 4,  
20:                      "sPaginationType": "full_numbers",  
21:                      "aaData": someData,  
22:                      "aoColumns": [  
23:                           { "sTitle": "Forename" },  
24:                           { "sTitle": "Surname" },  
25:                           { "sTitle": "Date Of Birth" },  
26:                           { "sTitle": "Gender" },  
27:                           { "sTitle": "Occupation" }                           
28:                      ]  
29:                 } );       
30:            } );  
31:  </script>  
Line 1 - Simple HTML table definition with id attribute set
Line 5 to 16 - Hard coded array of values used to populate the grid. In a real application this data would come from another source (user interface, server side etc.)
Line 17 - Define the grid by applying the DataTables plugin to the table skeleton we defined on line 1
Line 18 to 20 - This is the same basic configuration that we applied to our first grid. See DOM data source above.
Line 21 - Pass JavaScript array as data source. In our case this data is hard coded but in a real application it could come from any source.
Line 22 to 27 - Column definitions for the grid. Note that these were not required in the earlier example because we were using an existing HTML table as our data source