Saya terus mendapatkan FETCH_SESSION_ID_NOT_FOUND. Saya tidak yakin mengapa itu terjadi. Adakah yang bisa menyenangkan saya di sini apa masalahnya dan apa dampaknya bagi konsumen dan broker.
Log Server Kafka:
INFO [2019-10-18 12:09:00,709] [ReplicaFetcherThread-1-8][] org.apache.kafka.clients.FetchSessionHandler - [ReplicaFetcher replicaId=6, leaderId=8, fetcherId=1] Node 8 was unable to process the fetch request with (sessionId=258818904, epoch=2233): FETCH_SESSION_ID_NOT_FOUND.
INFO [2019-10-18 12:09:01,078] [ReplicaFetcherThread-44-10][] org.apache.kafka.clients.FetchSessionHandler - [ReplicaFetcher replicaId=6, leaderId=10, fetcherId=44] Node 10 was unable to process the fetch request with (sessionId=518415741, epoch=4416): FETCH_SESSION_ID_NOT_FOUND.
INFO [2019-10-18 12:09:01,890] [ReplicaFetcherThread-32-9][] org.apache.kafka.clients.FetchSessionHandler - [ReplicaFetcher replicaId=6, leaderId=9, fetcherId=32] Node 9 was unable to process the fetch request with (sessionId=418200413, epoch=3634): FETCH_SESSION_ID_NOT_FOUND.
Log Konsumen Kafka:
12:29:58,936 INFO [FetchSessionHandler:383] [Consumer clientId=bannerGroupMap#87e2af7cf742#test, groupId=bannerGroupMap#87e2af7cf742#test] Node 8 was unable to process the fetch request with (sessionId=1368981303, epoch=60): FETCH_SESSION_ID_NOT_FOUND.
12:29:58,937 INFO [FetchSessionHandler:383] [Consumer clientId=bannerGroupMap#87e2af7cf742#test, groupId=bannerGroupMap#87e2af7cf742#test] Node 3 was unable to process the fetch request with (sessionId=1521862194, epoch=59): FETCH_SESSION_ID_NOT_FOUND.
12:29:59,939 INFO [FetchSessionHandler:383] [Consumer clientId=zoneGroupMap#87e2af7cf742#test, groupId=zoneGroupMap#87e2af7cf742#test] Node 7 was unable to process the fetch request with (sessionId=868804875, epoch=58): FETCH_SESSION_ID_NOT_FOUND.
12:30:06,952 INFO [FetchSessionHandler:383] [Consumer clientId=creativeMap#87e2af7cf742#test, groupId=creativeMap#87e2af7cf742#test] Node 3 was unable to process the fetch request with (sessionId=1135396084, epoch=58): FETCH_SESSION_ID_NOT_FOUND.
12:30:12,965 INFO [FetchSessionHandler:383] [Consumer clientId=creativeMap#87e2af7cf742#test, groupId=creativeMap#87e2af7cf742#test] Node 6 was unable to process the fetch request with (sessionId=1346340004, epoch=56): FETCH_SESSION_ID_NOT_FOUND.
Detail Cluster:
Broker: 13 (1 Broker : 14 cores & 36GB memory)
Kafka cluster version: 2.0.0
Kafka Java client version: 2.0.0
Number topics: ~15.
Number of consumers: 7K (all independent and manually assigned all partitions of a topic to a consumers. One consumer is consuming all partitions from a topic only)
apache-kafka
Hrishikesh Mishra
sumber
sumber
max.incremental.fetch.session.cache.slotsJawaban:
Meningkatkan nilai
max.incremental.fetch.session.cache.slots. Nilai default adalah 1K, dalam kasus saya, saya telah meningkatkannya menjadi 10K dan diperbaiki.sumber
Saya telah meningkatkannya pada awalnya dari 1K ke 2K, dan pada langkah kedua dari 2K ke 4K, dan selama batasnya tidak habis, tidak ada tampilan kesalahan:
Sepertinya bagi saya seperti kebocoran sesi oleh konsumen tak dikenal tertentu, saya belum mencoba batas 10K, tetapi membaca jawaban Hrishikesh Mishra, saya pasti akan melakukannya. Karena, meningkatkan batas juga mengurangi frekuensi kesalahan, jadi pertanyaan untuk mengidentifikasi kelompok konsumen individu yang membuka terlalu banyak sesi pengambilan secara bertahap, disebutkan di sini . , mungkin tidak relevan pada akhirnya.
sumber