Batch jobs with Spring batch framework is very common and popular. This tutorial assumes that
1) You have set up you Java, Maven, and Eclipse
Once you have followed the following steps, you should have a project structure as shown below.
Step 1: Create a new Maven project with the command shown below.
1 | c:\Temp\Java\projects>mvn archetype:generate -DgroupId=com.mytutorial -DartifactId=simpleBatch |
Step 2: Import it into eclipse IDE via File –> import –> Existing Maven Projects.
Step 3: Update the pom.xml file so that it looks like as shown below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.mytutorial</groupId> <artifactId>simpleBatch</artifactId> <version>1.0-SNAPSHOT</version> <packaging>jar</packaging> <name>simpleBatch</name> <url>http://maven.apache.org</url> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> </properties> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>3.8.1</version> <scope>test</scope> </dependency> <!-- Spring dependencies --> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-beans</artifactId> <version>3.2.0.RELEASE</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-core</artifactId> <version>3.2.0.RELEASE</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-context</artifactId> <version>3.2.0.RELEASE</version> </dependency> <!-- Spring batch dependencies --> <dependency> <groupId>org.springframework.batch</groupId> <artifactId>spring-batch-core</artifactId> <version>2.1.9.RELEASE</version> </dependency> </dependencies> </project> |
Step 4: Define the reader and writer files as shown below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | package com.mytutorial.reader; import java.util.ArrayList; import java.util.Arrays; import java.util.List; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.NonTransientResourceException; import org.springframework.batch.item.ParseException; import org.springframework.batch.item.UnexpectedInputException; public class SimpleReader implements ItemReader<String> { // creates an unmodifiable list String[] itemArray = new String[] { "Java", "Spring", "Hibernate" }; // creates a modifiable list List<String> items = new ArrayList<String>(Arrays.asList(itemArray)); public String read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException { if (!items.isEmpty()) { return getNextItem(); } return null; } // using items directly without lock is not thread safe private synchronized String getNextItem() { return items.remove(0); } } |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | package com.mytutorial.writer; import java.util.List; import org.springframework.batch.item.ItemWriter; public class SimpleWriter implements ItemWriter<String> { public void write(List<? extends String> items) throws Exception { for (String item : items) { // prefix each item with "my-" final String prefix = "My_"; item = prefix + item; System.out.println(Thread.currentThread() + " writes: " + item); } } } |
Step 5: Define the batch job and wire the reader and writer via the Spring context file as shown below.
src/main/resources/applicationBatchContext.xml
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:p="http://www.springframework.org/schema/p" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:util="http://www.springframework.org/schema/util" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.1.xsd http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-2.0.xsd"> <!-- define the job repository --> <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean"> </bean> <!--define the launcher and pass the jobRepository as setter injection --> <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher"> <property name="jobRepository" ref="jobRepository" /> </bean> <!-- multi-threading --> <bean id="taskExecutor" class="org.springframework.core.task.SimpleAsyncTaskExecutor"> <property name="concurrencyLimit" value="3" /> </bean> <bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" /> <job id="simpleJob" xmlns="http://www.springframework.org/schema/batch"> <step id="simpleStep"> <tasklet task-executor="taskExecutor"> <chunk reader="simpleReader" writer="simpleWriter" commit-interval="1"> </chunk> </tasklet> </step> </job> |
Step 6: Finally, the test class to bootstrap “applicationBatchContext.xml”.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 | package com.mytutorial; import org.springframework.batch.core.Job; import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.JobParametersInvalidException; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException; import org.springframework.batch.core.repository.JobRestartException; import org.springframework.context.support.ClassPathXmlApplicationContext; public class SimpleBatchTest { public static void main(String[] args) throws JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException, JobParametersInvalidException { ClassPathXmlApplicationContext appContext = null; try { appContext = new ClassPathXmlApplicationContext("classpath:applicationBatchContext.xml"); // get the launcher JobLauncher jobLauncher = (JobLauncher) appContext.getBean("jobLauncher"); // get the job to run Job job = (Job) appContext.getBean("simpleJob"); // run jobLauncher.run(job, new JobParameters()); } finally { appContext.close(); } } } |
Run the SimpleBatchTest java file right clicking on it and the Run As –> Java Application.
Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | 24/10/2014 4:12:58 PM org.springframework.context.support.AbstractApplicationContext prepareRefresh INFO: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@3597a37c: startup date [Fri Oct 24 16:12:58 EST 2014]; root of context hierarchy 24/10/2014 4:12:58 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions INFO: Loading XML bean definitions from class path resource [applicationBatchContext.xml] 24/10/2014 4:12:58 PM org.springframework.beans.factory.support.DefaultListableBeanFactory registerBeanDefinition INFO: Overriding bean definition for bean 'simpleJob': replacing [Generic bean: class [org.springframework.batch.core.configuration.xml.SimpleFlowFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null] with [Generic bean: class [org.springframework.batch.core.configuration.xml.JobParserJobFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null] 24/10/2014 4:12:58 PM org.springframework.beans.factory.support.DefaultListableBeanFactory registerBeanDefinition INFO: Overriding bean definition for bean 'simpleReader': replacing [Generic bean: class [com.mytutorial.reader.SimpleReader]; scope=step; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=false; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in class path resource [applicationBatchContext.xml]] with [Root bean: class [org.springframework.aop.scope.ScopedProxyFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in BeanDefinition defined in class path resource [applicationBatchContext.xml]] 24/10/2014 4:12:58 PM org.springframework.beans.factory.support.DefaultListableBeanFactory registerBeanDefinition INFO: Overriding bean definition for bean 'simpleWriter': replacing [Generic bean: class [com.mytutorial.writer.SimpleWriter]; scope=step; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=false; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in class path resource [applicationBatchContext.xml]] with [Root bean: class [org.springframework.aop.scope.ScopedProxyFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in BeanDefinition defined in class path resource [applicationBatchContext.xml]] 24/10/2014 4:12:58 PM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@44e06940: defining beans [jobRepository,jobLauncher,taskExecutor,transactionManager,org.springframework.batch.core.scope.internalStepScope,org.springframework.beans.factory.config.CustomEditorConfigurer,org.springframework.batch.core.configuration.xml.CoreNamespacePostProcessor,simpleStep,simpleJob,simpleReader,simpleWriter,scopedTarget.simpleReader,scopedTarget.simpleWriter]; root of factory hierarchy 24/10/2014 4:12:58 PM org.springframework.batch.core.launch.support.SimpleJobLauncher afterPropertiesSet INFO: No TaskExecutor has been set, defaulting to synchronous executor. 24/10/2014 4:12:58 PM org.springframework.batch.core.launch.support.SimpleJobLauncher$1 run INFO: Job: [FlowJob: [name=simpleJob]] launched with the following parameters: [{}] 24/10/2014 4:12:58 PM org.springframework.batch.core.job.SimpleStepHandler handleStep INFO: Executing step: [simpleStep] Thread[SimpleAsyncTaskExecutor-2,5,main] writes: My_Java Thread[SimpleAsyncTaskExecutor-1,5,main] writes: My_Spring Thread[SimpleAsyncTaskExecutor-3,5,main] writes: My_Hibernate 24/10/2014 4:12:58 PM org.springframework.batch.core.launch.support.SimpleJobLauncher$1 run INFO: Job: [FlowJob: [name=simpleJob]] completed with the following parameters: [{}] and the following status: [COMPLETED] 24/10/2014 4:12:58 PM org.springframework.context.support.AbstractApplicationContext doClose INFO: Closing org.springframework.context.support.ClassPathXmlApplicationContext@3597a37c: startup date [Fri Oct 24 16:12:58 EST 2014]; root of context hierarchy 24/10/2014 4:12:58 PM org.springframework.beans.factory.support.DefaultSingletonBeanRegistry destroySingletons INFO: Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@44e06940: defining beans [jobRepository,jobLauncher,taskExecutor,transactionManager,org.springframework.batch.core.scope.internalStepScope,org.springframework.beans.factory.config.CustomEditorConfigurer,org.springframework.batch.core.configuration.xml.CoreNamespacePostProcessor,simpleStep,simpleJob,simpleReader,simpleWriter,scopedTarget.simpleReader,scopedTarget.simpleWriter]; root of factory hierarchy |
Running more than once can throw JobIsAlreadyRunningException. So, you need to supply different JobParameters as shown below.
1 2 3 4 5 | JobParameters jobParameters = new JobParametersBuilder() .addLong("time",System.currentTimeMillis()).toJobParameters(); jonLauncher.run(job, jobParameters); |
The above was run with 3 threads and commit-interval of 1. Experiment by changing the “commit-interval” (aka chunk size) within the step and the “concurrencyLimit” within the SimpleAsyncTaskExecutor in the spring config file.
