druid - What kafka.common.OffsetOutOfRangeException means -


i'm trying load data through apache kafka , getting error:

kafka.common.offsetoutofrangeexception: offset 1003786 out of range @ kafka.log.log$.findrange(log.scala:46) @ kafka.log.log.read(log.scala:264) @ kafka.server.kafkarequesthandlers.kafka$server$kafkarequesthandlers$$readmessageset(kafkarequesthandlers.scala:112) @ kafka.server.kafkarequesthandlers$$anonfun$2.apply(kafkarequesthandlers.scala:101) @ kafka.server.kafkarequesthandlers$$anonfun$2.apply(kafkarequesthandlers.scala:100) @ scala.collection.traversablelike$$anonfun$map$1.apply(traversablelike.scala:206) @ scala.collection.traversablelike$$anonfun$map$1.apply(traversablelike.scala:206) @ scala.collection.indexedseqoptimized$class.foreach(indexedseqoptimized.scala:34) @ scala.collection.mutable.arrayops.foreach(arrayops.scala:34) @ scala.collection.traversablelike$class.map(traversablelike.scala:206) @ scala.collection.mutable.arrayops.map(arrayops.scala:34) @ kafka.server.kafkarequesthandlers.handlemultifetchrequest(kafkarequesthandlers.scala:100) @ kafka.server.kafkarequesthandlers$$anonfun$handlerfor$3.apply(kafkarequesthandlers.scala:40) @ kafka.server.kafkarequesthandlers$$anonfun$handlerfor$3.apply(kafkarequesthandlers.scala:40) @ kafka.network.processor.handle(socketserver.scala:296) @ kafka.network.processor.read(socketserver.scala:319) @ kafka.network.processor.run(socketserver.scala:214) @ java.lang.thread.run(thread.java:724)

what exception mean , how can fix it?

offsetoutofrangeexception indicates client has requested range no longer available on server.
happen topic log not exists anymore based on retention policy in kafka setup.
if using simpleconsumer need handle offsetoutofrange exception in code. ideally consumer should issue offsetrequest latest/earliest offset available on server, , use in fetchrequest (as parameter)


Comments

Post a Comment

Popular posts from this blog

hibernate - How to load global settings frequently used in application in Java -

1) in application have table stores settings detail used heavily every functionality.basically conditions set in table checked. now doing making db call every , fetch details , check condition.this has resulted in lot of db calls , repetitive calls same condition.there quite lot of condition records in table.we need change make perform better. 2) in previous project had same requirement have configuration table such settings configured,there implemented load configuration table @ start of application , store them in system.setproperty(name,value)..and later use system.getproperty(name) retrieve it.in case settings changed update in system property.this worked charm. however,in current case(point 1 above) don't think prev way best way because lot of records present in table having lot of columns. can suggest way how achieve such kind of behavior in java? of them can think of is 1) using system.setproperty explained. 2) using caching ehcache - standalone or hibernate(as u...
Read more

python 3.x - Mapping specific letters onto a list of words -

can please suggest approach write piece of codes automatically maps letter in letter_str onto dic_key (a dict key string type contains dashes match length of words in word_lst)? so, mapping occurs if letter appears in every words in list @ same position no matter how many words in list. if no letter appears @ position words in word list new_dic_key '----'. please see examples below thanks word_lst = ['ague', 'bute', 'byre', 'came', 'case', 'doze'] dic_key = '----' letters_str ='abcdefghijklmnopqrstuvwxyz' new_dic_key = '---e' if word_list = ['bute', 'byre'] new_dic_key = 'b--e' or word_list = ['drek', 'drew', 'dyes'] new_dic_key = 'd-e-' if words in word_list of same length code give want: word_list = ['drek', 'drew', 'dyes'] cols = [] in range(len(word_list[0])): cols.append([]) word in ...
Read more

objective c - Ownership modifiers with manual reference counting -

what happens when use objective-c onwership modifiers ( __strong, __weak, __unsafe_unretained ) manual reference counting? of them modify behavior of ignored? clang -e -dm shows definitions of built-in macros: $ clang -e -dm -x objective-c /dev/null | egrep "weak|strong|unsafe" #define __strong #define __unsafe_unretained #define __weak __attribute__((objc_gc(weak))) as can see, __strong , __unsafe_unretained defined empty macros, , __weak defined (as understand it) relevant in (no longer supported) garbage-collected environment. so in short: ownership modifiers ignored manual reference counting. makes sense because your responsibility maintain reference counts. the situation different properties: @property(strong) id x; // same "retain" @property(weak) id y; // syntax error mrc @property(unsafe_unretained) id z; // same "assign" and sake of completeness: arc ownership modifiers built-in macros de...
Read more