Skip to main content

Apache Pulsar 2.9.4

What's Changed​

  • [fix][broker] Fix uncompleted future when get the topic policies of a deleted topic #18824
  • [fix][broker] Fix delete system topic clean topic policy #18823
  • [fix][broker] Fix duplicated schemas creation #18701
  • [improve][broker] System topic writer/reader connection not counted #18603
  • [fix][offload] Fix numerical overflow bug while reading data from tiered storage #18595
  • [fix][broker] DnsResolverUtil.TTL should be greater than zero #18565
  • [fix][offload] Fix memory leak while Offloading ledgers #18500
  • [fix][client] Fix multi-topic consumer stuck after redeliver messages #18491
  • [fix][client] Avoid redelivering duplicated messages when batching is enabled #18486
  • [fix][client] Fix failover/exclusive consumer with batch cumulate ack issue. #18454
  • [fix][client] Fix possible npe #18406
  • [fix][client] Fixes batch_size not checked in MessageId#fromByteArrayWithTopic #18405
  • [fix][sec] Upgrade jackson-databind to 2.13.4.2 to get rid of CVE-2022-42003 #18394
  • [improve][broker] System topic writer/reader connection not counted. #18369
  • [fix][client] Fix exception when calling loadConf on a ConsumerBuilder that has a KeySharedPolicy #18345
  • [fix][client] Support LocalDateTime Conversion #18334
  • [improve][cli] Fix NPE in admin-CLI topic stats command #18326
  • [fix][broker] In the trimDeletedEntries method release the removed entry #18305
  • [fix][client] Fix NPE of MultiTopicsConsumerImpl due to race condition #18287
  • [fix][broker] fix delete_when_subscriptions_caught_up doesn't work while have active consumers #18283
  • [fix][ml] Persist correct markDeletePosition to prevent message loss #18237
  • [fix][pulsar-client] Fix pendingLookupRequestSemaphore leak #18219
  • [fix][monitor] fix metrics string encoding #18138
  • [fix][function] Fix invalid metric type "gauge " #18129
  • [fix][fn] fix function failed to start if no typeClassName provided in FunctionDetails #18111
  • [fix][sec] File tiered storage: upgrade jettison to get rid of CVE-2022-40149 #18105
  • [fix][broker] Fixed delayed delivery after read operation error #18098
  • [fix][sec] Upgrade protobuf to 3.19.6 to get rid of CVE-2022-3171 #18086
  • [fix][broker]Cache invalidation due to concurrent access #18076
  • [fix][admin] Fix NPE when get OffloadThreshold on namespace. #18061
  • [fix][client][txn] Use PulsarClient HashWheelTimer to schedule producer batch trigger task #18058
  • [improve][broker]consumer backlog eviction policy should not reset read position for consumer #18037
  • [fix][broker] Update the log print content of createSubscriptions #18024
  • [fix][sec] File tiered storage: upgrade jettison to get rid of CVE-2022-40149 #18022
  • [fix][sec] Upgrade JacksonXML to 2.13.4 #18020
  • [fix][broker] Fix the order of resource close in the InMemoryDelayedDeliveryTracker #18000
  • [improve][broker] Avoid unnecessary creation of BitSetRecyclable objects #17998
  • [fix][broker] Fix getPositionAfterN infinite loop. #17971
  • [fix][broker]unify time unit at dropping the backlog on a topic #17957
  • [improve][client] Refactor SchemaHash to reduce call of hashFunction in SchemaHash #17948
  • [fix][broker]Fix mutex never released when trimming #17911
  • [fix][broker] Fix the broker shutdown issue after Zookeeper node crashed #17909
  • [improve][broker]Allow to configure and disable the size of lookahead for detecting fixed delays in messages #17907
  • [fix][admin] returns 4xx error when pulsar-worker-service is disabled and trying to access it #17901
  • [improve][ML] Print log when delete empty ledger. #17859
  • [fix][pulsar-client] Fix pendingLookupRequestSemaphore leak when channel inactive #17856
  • [improve][broker]Improve PersistentMessageExpiryMonitor expire speed when ledger not existed #17842
  • [fix][broker] Fix NPE when ResourceGroupService execute scheduled task. #17840
  • [refactor][java] Unify the acknowledge process for batch and non-batch message IDs #17833
  • [fix][proxy] Fix refresh client auth #17831
  • [fix][cli] Check numMessages after incrementing counter #17826
  • [fix][broker] add return for PersistentMessageExpiryMonitor#findEntryFailed #17816
  • [improve][broker] Make MessageRedeliveryController work more efficiently #17804
  • [Improve][Auth]Update authentication failed metrics report #17787
  • [fix][sec] Bump snakeyaml to 1.32 for CVE-2022-38752 component/security #17779
  • [fix][broker] ManagedLedger: move to FENCED state in case of BadVersionException #17736
  • [fix][client] Unwrap completion exception for Lookup Services #17717
  • [fix][broker] Fix namespace backlog quota check with retention. #17706
  • [fix][metadata] Cleanup state when lock revalidation gets LockBusyException #17700
  • [fix][broker] Fix executeWithRetry result is null #17694
  • [fix][common] Fix parsing partitionedKey with Base64 encode issue. #17687
  • [fix][metadata] Set revalidateAfterReconnection true for certain failures #17664
  • [fix][build] duplicate entry when merging services #17659
  • [fix][broker] Unregister topic policy listener if managed ledger close failed #17652
  • [fix][broker] Prevent Automatic Topic Creation during namespace deletion #17609
  • [improve][pulsar-proxy] Update proxy lookup throw exception type #17600
  • [improve][loadbalance] added loadBalancerReportUpdateMinIntervalMillis and ignores memory usage in getMaxResourceUsage() #17598
  • [improve][broker]Issue 17588: Allow deletion of a namespace that was left in deleted status #17592
  • [fix][broker] Topic policy reader can't recover when get any exception. #17562
  • [fix][broker] Fix potential exception causing the reader to be unable to continue reading #17556
  • [improve][txn] Implementation of Delayed Transaction Messages #17548
  • [fix][admin] Add SNI header when tlsHostnameVerification is not enabled #17543
  • [fix][client]Fix scheduledExecutorProvider not shutdown #17527
  • [fix][broker]Consumer can't consume messages because there has two sames topics in one broker #17526
  • [fix][broker] Fix the replicator unnecessary get schema request for BYTES schema #17523
  • [fix][tiered-storage] Don't cleanup data when offload met Metastore exception #17512
  • [fix][broker] Multiple consumer dispatcher stuck when unackedMessages greater than maxUnackedMessages #17483
  • [fix][broker] Extract additional servlets to the default directory #17477
  • [improve][broker] Improve cursor.getNumberOfEntries if isUnackedRangesOpenCacheSetEnabled=true #17465
  • [fix][sec] bump snakeyaml to 1.31 fix CVE-2022-25857 #17457
  • [improve][txn] Add getState in transaction for client API #17423
  • [improve][broker] Using handle instead of handleAsync to avoid using common pool thread #17403
  • [fix][broker] Fix issue where leader broker information isn't available after 10 minutes #17401
  • [fix][storage]fix OpAddEntry release error when exception in ManagedLedgerInterceptor #17394
  • [fix][broker] fix can not revoke permission after update topic partition #17393
  • [fix][client] Fix the message present in incoming queue after go to DLQ #17326
  • [fix][client] Fix reach redeliverCount client can't send batch messags #17317
  • [fix][client] Fix reach redeliverCount client can't send messages to DLQ #17287
  • [fix][schema]ledger handle leak when update schema #17283
  • [fix][broker] Fix broker cache eviction of entries read by active cursors #17273
  • [fix][broker]ManagedLedger metrics fail cause of zero period #17257
  • [fix][broker]Persist cursor info error when cursor close #17255
  • [enh][broker] Add metrics for entry cache insertion eviction #17248
  • [fix][tool] Using int instead of long in python scripts #17215
  • [fix][broker] Fix out of order data replication #17154
  • [fix][broker] Pass subscriptionName to auth service #17123
  • [fix][common]Fix presto sql avro decode error when publish non-batched msgs #17093
  • [fix][security] Bump PostgreSQL version to 42.4.1 #17066
  • [fix][ML] Fix offload read handle NPE. #17056
  • [fix][client] Fixed cnx channel Inactive causing the request fail to time out and fail to return #17051
  • [fix][broker] Fix schema does not replicate successfully #17049
  • [fix][broker] Fix calculate avg message per entry #17046
  • [fix][flaky-test]ManagedCursorMetricsTest.testCursorReadWriteMetrics #17045
  • [fix][broker] Increment topic stats outbound message counters and update rate after messages have been written to the TCP/IP connection #17043
  • [fix][broker] remove exception log when access status.html #17025
  • [fix][tiered-storage] move the state check forward #17020
  • [fix][client] Release semaphore before discarding messages in batchMessageContainer #17019
  • [fix][txn] fix ack with txn compute ackedCount error #17016
  • [fix][broker] fix broker unackmessages become a negative number #17003
  • [improve][broker]Remove unnecessary lock on the stats thread #16983
  • [fix][broker]Prevent StackOverFlowException in SHARED subscription #16965
  • [improve][broker] Improve naming for delete topic error #16965
  • [broker][monitoring][fix] fix pulsar_subscription_msg_ack_rate #16866
  • [improve][broker] Expose topic level storage write and read rate metrics #16842
  • [fix][client] fix PatternTopicsChangedListener blocked when topic removed #16842
  • [fix][broker] ManagedCursor: mark delete no callback when create meta-ledger fail #16841
  • [fix][proxy] Fix client service url #16834
  • [improve][broker] clean the empty topicAuthenticationMap in zk when revoke permission #16815
  • [fix][client] Remove redundant check for chunked message TotalChunkMsgSize in ConsumerImpl #16797
  • [fix][client] Fix load trust certificate #16789
  • [fix][broker] Avoid IllegalStateException while client_version is not set #16788
  • [fix][connector] Fixed kafka source config for pulsar-io-kafka when consumerConfigProperties="" #16731
  • [fix][broker] Fix consumer does not abide by the max unacks limitation for Key_Shared subscription #16718
  • [fix][function] Fix python instance not process zip file correctly #16697
  • [fix][broker] Retry to delete the namespace if new topics created during the namespace deletion #16676
  • [improve][connector] add reader config to pulsar-io-debezium and pulsar-io-kafka-connect-adaptor #16675
  • [fix][broker] Fix consumer does not abide by the max unacks limitation for Shared subscription #16670
  • [fix][broker] fix stats-internal with option -m cause active ledger recover then close #16662
  • [fix][broker] Fixed deadlock in key-shared dispatcher #16660
  • [fix][client] Fix ReconsumeLater will hang up if retryLetterProducer exception #16655
  • [fix][authorization] Fix multiple roles authorization #16645
  • [fix][client ]Skip creating a subscription replication snapshot if no messages have been published after the topic gets activated on a broker #16618
  • [improve][client] Send CloseConsumer on client timeout #16616
  • [improve][client] Add message key if exists to deadLetter messages #16615
  • [fix][client]Fix newLookup TooManyRequestsException message #16594
  • [fix][function] Fixed error when user starts with the pulsar functions local runner #16565
  • [fix][broker] The configuration loadBalancerNamespaceMaximumBundles is invalid #16552
  • [improve][test] Reduce the time consumption of BacklogQuotaManagerTest #16550
  • [fix][python client] Better Python garbage collection management for C++-owned objects #16533
  • [fix][txn] fix pattern sub filter transaction system topic #16533
  • [fix][security] Upgrade to Jetty to 9.4.48.v20220622 to get rid of CVE-2022-2047 #16520
  • [fix][txn] Allow producer enable send timeout in transaction #16519
  • [fix][java-client] Make DeadLetterPolicy deserializable #16513
  • [fix][broker] Fixed error when delayed messages trackers state grows to >1.5GB #16490
  • [fix][pulsar-broker] Fix RawReader hasMessageAvailable returns true when no messages #16443
  • [fix][common] Fix setManagedLedgerOffloadedReadPriority not work. #16436
  • [fix][broker] fix No such ledger exception #16420
  • [fix][broker] Skip reading more entries for a pending read with no more entries #16400
  • [improve][broker] Recycle OpReadEntry in some corner cases #16390
  • [fix][broker] Fix RawReader out of order #16390
  • [fix][flaky-test] Fix failed test NonPersistentTopicE2ETest.testGCWillDeleteSchema #16381
  • [fix][broker] Expose timestamp field for SchemaData&SchemaInfo #16380
  • [improve][package] Improve the package download process #16365
  • [fix][proxy] Do not preserve host when forwarding admin requests. #16342
  • [improve][broker] Use shared broker client scheduled executor provider #16338
  • [fix][client] Ensure ack-timeout task gets re-scheduled when there are exception in the final stage #16337
  • [improve][java-client] Improve performance of multi-topic consumer with more than one IO thread #16336
  • [improve][java-client] Support passing existing scheduled executor providers to the client #16334
  • [fix][broker] fix select broker is none #16316
  • [improve][broker] Exclude the Netty Reactive Stream from asynchttpclient #16312
  • [fix][client] Fixed deadlock when checking topic ownership #16310
  • [fix][txn] Fix append txn message is lower than lowWaterMark decrease PendingWriteOps #16266
  • [fix][txn] Fix TopicTransactionBuffer ledger apend marker throw ManagedLedgerAlreadyClosedException #16265
  • [fix][transaction] Cmd-Subscribe and Cmd-Producer will not succeed even after 100 retries #16248
  • [improve][broker] Avoid go through all the consumers to get the message ack owner #16245
  • [improve][broker] Reduce the consumers list sort by priority level #16243
  • [fix][broker]fix npe when invoke replaceBookie. #16239
  • [fix][broker]Fix getInternalStats occasional lack of LeaderInfo again #16238
  • [improve][java-client] Replace ScheduledExecutor to improve performance of message consumption #16236
  • [fix][broker] Fix NPE when drop backlog for time limit. #16235
  • [improve][broker][PIP-149]Make getList async #16221
  • [fix][broker]fix bug: fail to expose managed ledger client stats to prometheus if bookkeeperClientExposeStatsToPrometheus is true #16219
  • [fix][broker] Fix compaction subscription acknowledge Marker msg issue. #16205
  • [fix][broker] Fix passing incorrect authentication data #16201
  • [fix][function] Ensure bytes is a well-formed UTF-8 byte sequence when decode the FunctionState bytes to string #16199
  • [fix][admin] Fix get non-persistent topics issue in Namespaces. #16170
  • [improve][java-client] Only trigger the batch receive timeout when having pending batch receives requests #16160
  • [fix][sql] Fix messageQueue release message issue. #16155
  • [fix][txn]Fix race in TransactionMetaStoreHandler #16147
  • [fix][txn] Fix NPE when ack message with transaction at cnx = null #16142
  • [improve][broker] Use LinkedHashSet for config items of type Set to preserve elements order #16138
  • [improve][broker] Use OrderedExecutor instead of OrderedScheduler for consumer dispatch #16115
  • [fix][broker][monitoring] fix message ack rate #16108
  • [fix][broker] Fix NPE when get /admin/v2/namespaces/public/default/maxTopicsPerNamespace #16076
  • [fix][Java Client] Fix thread safety issue of LastCumulativeAck #16072
  • [improve][broker] Add config to allow deliverAt time to be strictly honored #16068
  • [fix][broker] Avoid AuthenticationDataSource mutation for subscription name #16065
  • [improve][broker] Avoid reconnection when a partitioned topic was created concurrently #16043
  • [fix][txn] Ack the same batch message different batchIndex with transaction #16032
  • [fix][client] Remove producer when close producer command is received #16028
  • [fix][admin] Fix missing response type in swagger definitions #16022
  • [fix][broker] Fix create client with TLS config #16014
  • [fix][broker] Removing log4j-1.2-api from dependencies #15991
  • [fix][tests] TieredStorageConfigurationTests - clear system properties #15957
  • [fix][admin] Fix producer/consume permission can’t get schema #15956
  • [improve][client] Extracted interface for EntryCacheManager #15933
  • [improve][client] Add classLoader field for SchemaDefinition #15915
  • [fix][broker] Avoid contended synchronized block on topic load #15883
  • [fix][tiered-storage] Fix the wrong secret key name get from env #15814
  • [fix][broker] Make deleteTopicPolicies serialized is executed when close topic. #15811
  • [broker][monitoring] add message ack rate metric for consumer #15674
  • [fix][storage] Autorecovery default reppDnsResolverClass to ZkBookieRackAffinityMapping #15640
  • [fix][broker][functions-worker] Ensure prometheus metrics are grouped by type #15558
  • [fix][broker] Fixed key-shared delivery of messages with interleaved delays #15409
  • [fix][function]Pulsar Functions: allow a Function<GenericObject?> to access the original Schema of the Message and use it #14847
  • [improve][security] Add load multiple certificates support #14798
  • [improve][client] Optimize pause when creating sub consumers in multi-topic consumer #14566
  • [improve][broker] refactor ManagedLedger cacheEvictionTask implement #14488
  • [improve][metadata-store] Fix metadata cache inconsistency on doing refresh #14283
  • [fix][broker] Fix NPE of internalExpireMessagesByTimestamp #14243
  • [fix][broker]Prevent StackOverFlowException in KEY_SHARED subscription #14121
  • [fix][txn] Set TC state is Ready after open MLTransactionMetadataStore completely. #13957
  • [improve][broker] Do not create missing topic when loading namespace #13948
  • [improve][broker] Use shared executors for broker and geo-replication clients #13839
  • [improve][broker]Updating dependencies to get rid of CVEs brought in with kafka and log4j-1.2 libs #13726
  • [improve][broker]Support dynamic update cache config #13679
  • [improve][broker] Recycler should use io.netty.recycler.maxCapacityPerThread instead of io.netty.recycler.maxCapacity.default as maxCapacityPerThread configuration in Netty 4.1.x #13563
  • [fix][broker]Fix zk-node leak of admin path #12972
  • [fix][broker] Only refresh metadata if path is already in cache after write. #12896
  • [improve][broker] Broker extensions to allow operators of enterprise wide cluster better control and flexibility #12536
  • [improve][client] Support passing existing executor providers to the client #12037
Was this helpful?