Monday, April 05, 2010

How to pass filename to FlatFileItemReader with JobParameters in Spring Batch?



 The follow config was tested and succeeded on Spring Batch 2.0.4.
<bean id="flatFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
  <property name="resource"  value="file:#{jobParameters['input.file.name']}" />
  <property name="lineMapper" ref="lineMapper"/>
</bean>



Java code:

try {
//launcher.run(job, new JobParameters());
JobParametersBuilder jpBuilder = new   JobParametersBuilder().addString("input.file.name", "files/input/test.txt");

 jpBuilder.addString("output.file.name", "files/output/"+System.currentTimeMillis()+"_test.txt");
launcher.run(job, jpBuilder.toJobParameters());
} catch (Exception e) {
e.printStackTrace();
}


Another config used FileSystemResource for resource likes the follow.

<property name="resource">
   <bean class="org.springframework.core.io.FileSystemResource" scope="step" autowire-candidate="false">
     <constructor-arg value="#{jobParameters['input.file.name']}" />
   </bean>
</property>



For ItemWriter, the configuration almost same.
NOTE:
The 'file:' is mandatory if the resource is a file needed to be load from specified path.
The  scope="step" is very important. It seems the file cannot be found if it is missed.
From the reference, I found 'Any bean that uses late-binding must be declared with scope="step".'.


The follow code is is used by SpringBatch 1.x.
Just like the reference metioned, you need to define a listener for step to the proxy to have the access to the StepExecution.

<property name="listeners">
  <list>
    <ref bean="inputFile" />
  </list>
</property>

<bean id="inputFile" class="org.springframework.batch.core.resource.StepExecutionResourceProxy">

  <property name="filePattern" value="file://%file.name%"/>
</bean>

<bean id="myFileItemReader" class="org.springframework.batch.item.file.FlatFil eItemReader">
  <property name="resource" ref="inputFile" />
</bean>


Java Code:

JobParametersBuilder builder = new JobParametersBuilder();
builder.addString("file.name", "files/input/test.txt");

JobExecution jobExecution = getJobLauncher().run(getJob(), builder.toJobParameters());

1 comment:

  1. mainDir - FileA
    processDir - FileB, FileC, FileD



    1. Check file in mainDir. If present, check files in processDir. If not present, copy FileA to processDir.
    2. Check file in mainDir. If present, check files in processDir. If present, check for mapping.
    - Creating a mapping object
    mappingObj1(FileA_Name, FileB_Name)
    mappingObj1(FileA_Name, FileC_Name)
    mappingObj1(FileB_Name, FileD_Name)






    FeedDef
    1|LEM| |TradeRef
    1|LEM| |TradeDate
    2 RAVEN LOH
    SystemMapping
    1,2
    1,3
    2,4



    ----------------------------------------------------------------------------------------------------------------
    Hashmap fdMap
    for(String fdLine : fdLines)
    {
    String[] fd = fdLine.split("|");
    FeedDef fdObj = new FeedDef();
    fdObj.setSourceSystemName(fd[1]);
    fdObj.setSourceSystemSiteCode(fd[2]);
    fdMap.put(fd[0], fdObj);
    }


    HashMap
    List sysMappingList
    for(String smLine : smLines)
    {
    String[] sm = smLine.split("|");
    for(String sm : smList)
    {
    SystemMapping sysMapping;
    sysMapping.setSourceSystemId((sm[0]);
    sysMapping.setSourceSystemName((fdMap.get(sm.[0]).getSourceSystemName()));
    sysMapping.setSourceSystemSiteCode((fdMap.get(sm.get(0)).getSourceSystemSiteCode()));

    sysMapping.setTargetSystemId((sm[1]);
    sysMapping.setTargetSystemName((fdMap.get(sm.[1]).getSourceSystemName()));
    sysMapping.setTargetSystemSiteCode((fdMap.get(sm.get(1)).getSourceSystemSiteCode()));

    sysMappingList.add(sysMapping);
    }
    }

    FileToProcess - sourceFileName, targetFileName
    ArrayList fileToProcessList
    File[] mainDirFiles
    if(null != mainDirFiles)
    {
    move files to processDir
    File[] processDirFiles;
    if(null != processDirFiles)
    {
    for(File mdFile : mainDirFiles)
    {
    for(File pdFile : processDirFiles)
    {
    for(SystemMapping sysMaping : sysMappingList)
    {
    if(mdFile.getName().contains(sysMaping.getSourceSystemName() && mdFile.getName().contains(sysMaping.getSourceSystemSiteCode()))
    {
    if(pdFile.getName().contains(sysMaping.getTargetSystemName() && pdFile.getName().contains(sysMaping.getTargetSystemSiteCode()))
    {
    FileToProcess fileToProcess = new FileToProcess();
    fileToProcess.setSourceFileName(mdFile.getName());
    fileToProcess.setTargetFileName(pdFile.getName());
    fileToProcessList.add();
    }
    }
    else if(pdFile.getName().contains(sysMaping.getSourceSystemName() && pdFile.getName().contains(sysMaping.getSourceSystemSiteCode()))
    {
    if(mdFile.getName().contains(sysMaping.getTargetSystemName() && mdFile.getName().contains(sysMaping.getTargetSystemSiteCode()))
    {
    FileToProcess fileToProcess = new FileToProcess();
    fileToProcess.setSourceFileName(mdFile.getName());
    fileToProcess.setTargetFileName(pdFile.getName());
    fileToProcessList.add();
    }
    }
    }

    }
    }
    } else
    {
    don't start job
    }
    } else
    {
    don't start job
    }


    For(FileToProcess fileToProcess : fileToProcessList){

    try {
    JobParametersBuilder jpBuilder = new JobParametersBuilder().addString("source.file.name", fileToProcess.getSourceFileName());
    jpBuilder.addString("target.file.name", fileToProcess.getTargetFileName());
    launcher.run(job, jpBuilder.toJobParameters());
    } catch (Exception e) {
    e.printStackTrace();
    }




    The follow config was tested and succeeded on Spring Batch 2.0.4.





    }








    ReplyDelete