dynamic - spring batch : Read flat file which is getting changed continously -

dynamic - spring batch : Read flat file which is getting changed continously -

i have requirement, have read flat text file continuously changing. let's assume have file 100 lines read using flatfilereader in batch , process lines. 1 time again when step gets called let's after 30 sec, there 110 lines. in case batch should read line 101. know there 'linestoskip' parameter in reader can define @ start of batch not dynamically. file defined in batch configuration should reloaded 1 time again on phone call of step(step continuous process).

any thought this?

thanks niraj

i suggest next approach:

wrap reader step listener , utilize before , after hooks. create sure both step , flatfileitemreader bean defined @ step scope.

in before step read lastly line processed count persistence (file/db/etc) , place on stepexecutioncontext. use value placed on stepexecutioncontext set linestoskip in flatfileitemreader using spel in after step take current writecount , skipcount execution context sum value before step. persist value next execution

your listener similar 1 below

@component public class linecursorlistener implements steplistener { @beforestep public exitstatus beforestep(stepexecution stepexecution){ int curser = 0;//read persistence stepexecution.getexecutioncontext().put("linestoskip", curser); homecoming stepexecution.getexitstatus(); } @afterstep public exitstatus afterstep(stepexecution stepexecution){ int nextcurser= stepexecution.getwritecount() + stepexecution.getskipcount(); nextcurser =nextcurser + stepexecution.getexecutioncontext().getint("linestoskip"); // persistence nextcurser homecoming stepexecution.getexitstatus(); } }

your job xml similar

<batch:job> ... <batch:step id="processcsv"> <batch:tasklet transaction-manager="transactionmanager"> <batch:chunk reader="somefilereader" writer="writter" commit-interval="10" /> </batch:tasklet> <batch:listeners> <batch:listener ref="linecurserlistener" /> </batch:listeners> </batch:job> <bean id="somefilereader" scope="step" class="org.springframework.batch.item.file.flatfileitemreader" > ... <property name="linestoskip" value="#{stepexecutioncontext['linestoskip']}" /> <property name="linemapper"> ... </property> </bean>

i presenting spring batch point of view issue, guess need watch out concurrency issue related file read/write

dynamic spring-batch filereader