java - How to put data from server to Kinesis Stream -

java - How to put data from server to Kinesis Stream -

i new kinesis. reading out documentation found can create kinesis stream info producer. using kcl read info stream farther processing. understand how write kcl application implemeting irecordprocessor .

however first stage how set info on kinesis stream still not clear me. have aws api need implementation accomplish this.

scenarios: have server contineously getting info various sources in folders. each folder containing text file rows containing required attributes furhter analytical work. have force these info kinesis stream.

i need code below below class putdata method wil used out in kinesis stream

public class set { amazonkinesisclient kinesisclient; put() { string accesskey = "my access key here" ; string secretkey = "my secret key here" ; awscredentials credentials = new basicawscredentials(accesskey, secretkey); kinesisclient = new amazonkinesisclient(credentials); kinesisclient.setendpoint("kinesis.us-east-1.amazonaws.com", "kinesis", "us-east-1"); system.out.println("starting set application"); } public void putdata(string filecontent,string session) throws exception { final string mystreamname = "clickstream"; putrecordrequest putrecordrequest = new putrecordrequest(); putrecordrequest.setstreamname(mystreamname); string putdata = filecontent; putrecordrequest.setdata(bytebuffer.wrap(putdata.getbytes())); putrecordrequest.setpartitionkey("session"+session); putrecordresult putrecordresult = kinesisclient.putrecord(putrecordrequest); system.out.println("successfully putrecord, partition key : " + putrecordrequest.getpartitionkey() + ", shardid : " + putrecordresult.getshardid()); system.out.println(filecontent); system.out.println("sequence number: "+putrecordresult.getsequencenumber()); system.out.println("data has been set successfully"); } }

however reading file source folder server , design should utilize phone call putdata record on kinesis stream. need infinite loop , reading files , or framework improve care of fault tolerance , single point of failure . help appreciated.

briefly: need improve technique set regularly generated info kinesis stream info generated @ regular interval server. thanks

so seems using... http://docs.aws.amazon.com/awsjavasdk/latest/javadoc/com/amazonaws/services/kinesis/amazonkinesisclient.html

specific method want follows.

you need stream name, record, , stream key. http://docs.aws.amazon.com/awsjavasdk/latest/javadoc/com/amazonaws/services/kinesis/model/putrecordresult.html

but seems have this?

you need programme running tailing server log file , when ever there new line force this.

but info sit down 24 hours. need worker programme consume info , place in other aws resource.

java amazon-web-services rabbitmq amazon-kinesis