Sign in

I have a kafka consumer which after resume, it’s picking up from last committed offset and sequentially consuming from there. Even with several calls to seekToEnd().

Logs

//before pause 2021-10-09 15:44:18,569 [fx-kafka-consumer-0-C-1] INFO c.x.f.r.k.HarvestConsumer - check raw msg size 1 on [17] @ [38012024115] 2021-10-09 15:44:28,573 [fx-kafka-consumer-0-C-1] INFO c.x.f.r.k.HarvestConsumer …

i was using PreparedStatement to load some data out of sql database, which has been working fine for quite some time. Till recently, for one day’s data, the query is not able to finish (hang on reading the response from database)

"stackTrace": [ { "methodName": "socketRead0", "fileName": "SocketInputStream.java", "lineNumber": -2…

when I am writing a comparator using lamba, java throws out this exception

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.IllegalArgumentException: Comparison method violates its general contract! at java.util.TimSort.mergeLo(TimSort.java:777) at java.util.TimSort.mergeAt(TimSort.java:514) at java.util.TimSort.mergeCollapse(TimSort.java:441) at java.util.TimSort.sort(TimSort.java:245) at java.util.Arrays.sort(Arrays.java:1512) at java.util.ArrayList.sort(ArrayList.java:1464) at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:392)…

i have set up a program to consume from a kafka topic.

it has been running fine, till ~ 25% of the messages (out of 2/3mil rows) are consumed, there would be a lot of reblance started happening.

i have tried to increase the `

max.poll.interval.ms

and reduce the `


TreeMap could be used to sort the map by value

Map<Character, Integer> sorted = new TreeMap<>((o1, o2) -> comparator.compare(map.get(o2), map.get(o1))); sorted.putAll(map);

however, there is a caveat there. if the values…


there are times even the application is requesting for parallel processing, it could stick to one single thread without rotation.

for example, the case, https://lwpro2.dev/2021/06/11/jdk-spliterator-with-parallel-processing/.

this could be the case…


I have created a fixed size Spliterator to split the collection into fixed size. https://github.com/openjdk/jdk/pull/2907 It's working as expected. With a collection of size greater than the threshold, it would trySplit and generate a new Spliterator. And if the stream is parallel, it would pass the spliterator to different threads.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store