Skip to main content

Apache Pulsar 2.8.4

Broker​

  • [improve][broker] Do not print the log that the topic does not exist 13535
  • [fix][broker] Fix NPE when nextValidLedger is null 13975
  • [fix][broker] Fix NPE when ReplicatedSubscriptionsController sends marker message with deduplication enabled 14017
  • [fix][broker] Fix avg-messagePerEntry metrics for consumer 14330
  • [improve][broker] Validate rack name when setting the bookie rack info 14336
  • [improve][broker] Optimize find nics process. 14340
  • [fix][broker] Fix NPE when subscription is already removed 14363
  • [fix][broker] Fix Broker HealthCheck Endpoint Exposes Race Conditions 14367
  • [improve][broker] Optimize RawReader#create when using Compactor 14447
  • [fix][broker] Fix the latest message of the compacted topic cannot be read 14449
  • [fix][broker] Fix producerFuture not completed in ServerCnx#handleProducer 14467
  • [fix][broker] Fix NPW in ManagedLedgerImpl 14481
  • [improve][broker] Support shrink in ConcurrentLongHashMap 14497
  • [fix][broker] Fixed wrong behaviour caused by not cleaning up topic policy service state. 14503
  • [improve][broker] Optimize memory usage: support to shrink for pendingAcks map 14515
  • [fix][broker] Cancel offload tasks when managed ledger closed 14744
  • [improve][broker] Reduce unnecessary expansions for ConcurrentLong map and set 14562
  • [fix][broker] Fix metadata store deadlock when checking BacklogQuota 14634
  • [improve][broker] Support shrink for map or set 14663
  • [fix][broker] Fix lost message issue due to ledger rollover 14664
  • [fix][broker] Set splitNamespaceBundle with readonly=false 14680
  • [fix][broker] Fixed duplicated delayed messages when all consumers disconnect 14740
  • [fix][broker] Fix wrong state for non-durable cursor 14869
  • [fix][broker] Improve error logging for topic not found 14892
  • [fix][broker] Fix topic policy reader close bug 14897
  • [fix][broker] Return if reset in progress 14978
  • [fix][broker] Fix potential NPE in Replicator 15003
  • [fix][broker] Fix race condition in updating lastMarkDeleteEntry field 15031
  • [fix][broker] Fix potential to add duplicated consumer 16826
  • [fix][broker] Follow up on race condition fixes in ManagedCursorImpl #15031 15067
  • [cleanup][broker] Remove useless code to avoid confusion in OpReadEntry#checkReadCompletion 15104
  • [fix][broker] Cancel fencedTopicMonitoringTask when topic closed normally. 15202
  • [fix][broker] Fix MessageDeduplication#inactiveProducers may not be persistence correctly 15206
  • [fix][broker] Fix race condition between timeout and completion in OpAddEntry 15233
  • [fix][broker] Fix problem at RateLimiter#tryAcquire 15306
  • [improve][broker] Use shrink map for message redelivery. 15342
  • [improve][broker] Support shrink for ConcurrentSortedLongPairSet 15354
  • [improve][broker] Close connection if a ping or pong message cannot be sent 15382
  • [cleanup][broker] Fix typo in enum name and handle closing of the channel properly since writeAndFlush is asynchronous 15384
  • [fix][broker] Avoid heartbeat topic to offload. 15393
  • [fix][broker] Fix deadlock in broker after race condition in topic creation failure 15570
  • [fix][broker] fix calculation in getNumberOfEntriesInStorage 15627
  • [fix][broker] Fix NPE when set AutoTopicCreationOverride 15653
  • [fix][broker] Fix creating producer failure when set backlog quota 15663
  • [fix][broker] Fix NPE when put value to RangeCache 15707
  • [fix][broker] Fix can not enable system topic if AutoUpdateSchemaEnabled=false 15759
  • [fix][broker] Fix wrong unit of NIC speed on Linux 15770
  • [fix][broker] Fix NPE in MessageDeduplication 15820
  • [fix][broker] Fix NPE when ledger id not found in OpReadEntry 15837
  • [fix][broker] When skipping updating mark delete position, execute callback with executor to prevent deadlock 15971
  • [fix][broker] Fix NPE when drop backlog for time limit. 16235
  • [fix][broker]Fix getInternalStats occasional lack of LeaderInfo again 16238
  • [fix][broker] Fix RawReader out of order 16390
  • [fix][broker] Skip reading more entries for a pending read with no more entries 16400
  • [fix][broker] Fix No such ledger exception 16420
  • [fix][broker] Fix RawReader hasMessageAvailable returns true when no messages 16443
  • [fix][broker] Fix stats-internal with option -m cause active ledger recover then close 16662
  • [fix][broker] Fix passing incorrect authentication data 16840
  • [fix][broker] ManagedCursor: mark delete no callback when create meta-ledger fail 16841
  • [improve][broker] Avoid reconnection when a partitioned topic was created concurrently 16867
  • [improve][broker] Recycle OpReadEntry in some corner cases 16869
  • [fix][broker] Fix compaction subscription acknowledge Marker msg issue. 16918

Security​

  • [fix][sec] Use provider's canLookupAsync method for AuthorizationService 11777
  • [improve][sec] Optimize the logic of allowing namespace operation 13090
  • [improve][sec] Full-support set SSL provider, ciphers and protocols 13740
  • [fix][sec] Role with namespace produce authz can also get topics 13773
  • [improve][sec] Full-support SSL provider, ciphers, and protocols for broker service and proxy service 15034
  • [improve][sec] Add load multiple certificates support 14798
  • [fix][sec] Fix handling single role and non-jwt-token in MultiRolesTokenAuthorizationProvider 14857
  • [fix][sec] Use tlsCertRefreshCheckDurationSec instead of 0 for refresh value 15075
  • [improve][sec] Skip unnecessary DNS resolution when creating AuthenticationDataHttp instance 15221
  • [improve][sec] Improve skipping of DNS resolution when creating AuthenticationDataHttp instance 15228
  • [fix][sec] Fix MultiRolesTokenAuthorizationProvider authorize issue 15454
  • [fix][sec] Fix grant all permissions but can't list topic 15501
  • [improve][sec] Switch to rely on Netty for Hostname Verification 15824
  • [fix][sec] Fix create client with TLS config 16014
  • [fix][sec] Avoid AuthenticationDataSource mutation for subscription name 16065
  • [improve][sec] Improve get the basic authentication config 16526
  • [fix][sec] Fix multiple roles authorization 16645
  • [fix][sec] Add timeout of sync methods and avoid call sync method for AuthoriationService (#15694) 16831
  • [improve][sec] Support for get token from HTTP params 16871
  • [improve][sec] Adapt basic authentication configuration with prefix 16935

Admin​

  • [improve][admin] Add corresponding get command for namespace 12322
  • [fix][admin] Fix deleting namespace will fail when system topics exist 15040
  • [fix][admin] Fix validateGlobalNamespaceOwnership wrap exception issue 14612
  • [fix][admin] Fix wrong prompt exception when getting the non-persistent topic list without GET_BUDNLE permission 14638
  • [fix][admin] Fixed 404 error msg not returned correctly using http lookup 14677
  • [improve][admin] Provide an accurate error message when set autoTopicCreation 14684
  • [fix][admin] Fix cannot delete namespace with system topic 14730
  • [fix][admin] Fix NPE in PulsarAdminBuilder when the service is not set 14769
  • [fix][admin] Fix missing response type in swagger definitions 16022

Function​

  • [fix][fn] Fixing get functions for output topic and serde classname 14103
  • [improve][fn] Pass configured metricsPort to k8s runtime 14502
  • [fix][fn] fix some IOExceptions when create functions from package URL 14553
  • [fix][fn] handle NPE when getLeader returns null 15058
  • [improve][fn] Add KeyStore support in WebSocket, Function Worker HTTPS Servers 15084
  • [fix][function] Check executor null when close the FileSource 15247
  • [improve][fn] Provide default error handler for function log appender 15728
  • [fix][function] Fix error when user starts with the pulsar functions local runner 16565
  • [fix][function] Fix python instance not process zip file correctly 16697

IO​

  • [fix][io] throw exceptions when Kafka offset backing store failed to start 14491
  • [fix][io] Handle Kafka sinks that return immutable maps as configs 14780

SQL​

  • [fix][sql] Fix PulsarRecordCursor deserialize issue. 14615
  • [fix][sql] Fix messageQueue release message issue. 16155

Offloader​

  • [fix][offloader] Fix incorrect entryId in warn log when reading entry from tiered storage 14685
  • [improve] TieredStorage: add debug information 14907
  • [feat][offloader] Add pure S3 provider for the offloader 15710
  • [fix][offloader] Fix setManagedLedgerOffloadedReadPriority not work. 16436

Proxy​

  • [improve][proxy] Log warning when opening connection to broker fails 14710
  • [refactor][proxy] Refactor Proxy code and fix connection stalling by switching to auto read mode 14713
  • [improve][proxy] Configure Netty DNS resolver to match JDK DNS caching setting, share DNS resolver instance in Proxy 15219
  • [fix][proxy] Fix proxy connection leak when inbound connection closes while connecting is in progress 15366
  • [fix][proxy] Fix DNS server denial-of-service issue when DNS entry expires 15403
  • [improve][proxy] Remove unnecessary blocking DNS lookup in LookupProxyHandler 15415
  • [fix][proxy] Do not preserve host when forwarding admin requests. 16342
  • [fix][proxy] Fix client service URL 16834

Monitor​

  • [improve][monitor]: add metrics for pulsar web service thread pool 14742
  • [improve][monitor] add message ack rate metric for consumer 15674
  • [fix][monitor] Fix bug: fail to expose managed ledger client stats to prometheus if bookkeeperClientExposeStatsToPrometheus is true 16343
  • [improve][monitor] Expose topic level storage write and read rate metrics 16855

CLI​

  • [fix][cli] Remove the trust certs check 14764

Test & Other​

  • [improve][test] Use Awaitility to replace Thread.sleep for pulsar-zookeeper-utils module. 11017
  • [fix][test] Fix flaky test in PersistentStreamingDispatcherBlockConsumerTest 12943
  • [fix][build] Fix NoClassDefFoundError: com/google/inject/AbstractModule in pulsar-io/batch-data-generator and Jcloud offloader 14150
  • [fix][test] Fix flaky AdminApiSchemaTest#testSchemaInfoApi 14508
  • [fix][test] Fix NamespacesTest execution order 14552
  • [fix][test] Fix AdminApiTest.testNamespaceSplitBundleConcurrent 14565
  • [fix][test] Fix flaky test MemoryLimitTest#testRejectMessages (#14220) 14628
  • [fix][ci] Upgrade zlib version to 1.2.12 14964
  • [improve][build] Ignore case when obfuscating passwords in configuration scripts 15077
  • [fix][build] Fix LICENSE files for branch-2.8 15261
  • [Improve][doc] Add config of IO and acceptor threads in proxy 15340
  • [improve][build] Configure DLog Bookie, Pulsar, and Admin clients via pass-through config 15818
  • [improve][build] Allow pulsar_tool_env.sh PULSAR_MEM to be Overridden 15868
  • [fix][test] TieredStorageConfigurationTests - clear system properties 15957
  • [fix][doc] Update/fix Swagger Annotation for param: authoritative 16222
  • [improve][test] Verify the authentication data in the authorization provider 16900

Dependency Updates​

  • [improve][build] Upgrade snakeyaml version to 1.30 13722
  • [fix][build] Remove --illegal-access errors resulting from Google Guice (upgrade to 5.0.1 and JClouds to 2.4.0) 13810
  • [fix][build] Remove --illegal-access errors resulting from Google Guice - Pulsar IO, Offloaders and Pulsar SQL - Bump Guice to 5.1.0 14300
  • [fix][build] Upgrade jackson and jackson-databind (2.13.2.1) to get rid of CVE-2020-36518 14871
  • [improve][build] Remove log4j for CVE-2022-23307 15109
  • [improve][build] Upgrade Netty to 4.1.76.Final, Netty Tcnative, grpc and protobuf 15212
  • [improve][build] Use grpc-bom to align grpc library versions 15234
  • [improve][build] Upgrade BookKeeper to 4.14.5 15581
  • [improve][build] Upgrade Netty to 4.1.77.Final and netty-tcnative to 2.0.52.Final 15646
  • [improve][build] Upgrade JClouds to 2.5.0 [15649](https://github.com/apache/pulsar/pull/15649
  • [fix][build] Tiered storage: Upgrade Hadoop to 3.3.3 to get rid of CVE-2022-26612 15660
  • [fix][build] Upgrade to Jetty to 9.4.48.v20220622 to get rid of CVE-2022-2047 16520
  • [improve][broker] Upgrade log4j2 version to 2.18.0 (#16884) 16914
Was this helpful?