Skip to content

feat: Bump Druid to 37.0.0#414

Merged
NickLarsenNZ merged 3 commits into
mainfrom
druid-37.0.0
Jun 19, 2026
Merged

feat: Bump Druid to 37.0.0#414
NickLarsenNZ merged 3 commits into
mainfrom
druid-37.0.0

Conversation

@NickLarsenNZ

@NickLarsenNZ NickLarsenNZ commented Jun 18, 2026

Copy link
Copy Markdown
Member

Part of stackabletech/docker-images#1490

Bump to Druid 37.0.0, which requires S3 via TLS. So we need to use the minio-tls pre-rendered chart.

Druid 37.0.0 requires S3 via TLS. So we need to use the minio-tls pre-rendered chart.
@NickLarsenNZ

Copy link
Copy Markdown
Member Author

I believe this to be good (it was failing until stackabletech/druid-operator#832 was merged/built/published).

image

Task log

2026-06-19T07:30:19,117 INFO [[index_kafka_earthquakes_ecfc7af154be638_pkjodnek]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.StreamAppenderator - Persisted stats: processed rows: [3032536], persisted rows[67856], sinks: [73], total fireHydrants (across sinks): [154], persisted fireHydrants (across sinks): [3]
2026-06-19T07:30:19,857 INFO [task-runner-0-priority-0] org.apache.druid.segment.realtime.appenderator.StreamAppenderator - Marking ready for non-incremental async persist due to reasons[[(estimated) bytesCurrentlyInMemory[44739610] is greater than maxBytesInMemory[44739242]]].
2026-06-19T07:30:19,858 INFO [task-runner-0-priority-0] org.apache.druid.segment.realtime.appenderator.StreamAppenderator - Persisted rows[67,250] and (estimated) bytes[44,374,610]
2026-06-19T07:30:19,873 INFO [[index_kafka_earthquakes_ecfc7af154be638_pkjodnek]-appenderator-persist] org.apache.druid.segment.IndexMergerV9 - Building segment with IndexSpec[IndexSpec{bitmapSerdeFactory=RoaringBitmapSerdeFactory{}, dimensionCompression=lz4, stringDictionaryEncoding=Utf8{}, metricCompression=lz4, longEncoding=longs, complexMetricCompression=null, autoColumnFormatSpec=null, jsonCompression=null, segmentLoader=null}]
2026-06-19T07:30:20,083 INFO [[index_kafka_earthquakes_ecfc7af154be638_pkjodnek]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.StreamAppenderator - Flushed in-memory data for segment[earthquakes_2021-01-01T00:00:00.000Z_2022-01-01T00:00:00.000Z_2026-06-19T07:30:15.438Z] spill[3] to disk in [224] ms (19,602 rows).
2026-06-19T07:30:20,129 INFO [[index_kafka_earthquakes_ecfc7af154be638_pkjodnek]-appenderator-persist] org.apache.druid.segment.IndexMergerV9 - Building segment with IndexSpec[IndexSpec{bitmapSerdeFactory=RoaringBitmapSerdeFactory{}, dimensionCompression=lz4, stringDictionaryEncoding=Utf8{}, metricCompression=lz4, longEncoding=longs, complexMetricCompression=null, autoColumnFormatSpec=null, jsonCompression=null, segmentLoader=null}]
2026-06-19T07:30:20,573 INFO [[index_kafka_earthquakes_ecfc7af154be638_pkjodnek]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.StreamAppenderator - Flushed in-memory data for segment[earthquakes_2022-01-01T00:00:00.000Z_2023-01-01T00:00:00.000Z_2026-06-19T07:30:17.507Z] spill[1] to disk in [487] ms (47,648 rows).
2026-06-19T07:30:20,579 INFO [[index_kafka_earthquakes_ecfc7af154be638_pkjodnek]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.StreamAppenderator - Flushed in-memory data with commit metadata [AppenderatorDriverMetadata{segments={index_kafka_earthquakes_ecfc7af154be638_0=[SegmentWithState{segmentIdentifier=earthquakes_1973-01-01T00:00:00.000Z_1974-01-01T00:00:00.000Z_2026-06-19T07:29:23.283Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1976-01-01T00:00:00.000Z_1977-01-01T00:00:00.000Z_2026-06-19T07:29:24.125Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1960-01-01T00:00:00.000Z_1961-01-01T00:00:00.000Z_2026-06-19T07:29:21.791Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1957-01-01T00:00:00.000Z_1958-01-01T00:00:00.000Z_2026-06-19T07:29:21.592Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2007-01-01T00:00:00.000Z_2008-01-01T00:00:00.000Z_2026-06-19T07:29:53.838Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1954-01-01T00:00:00.000Z_1955-01-01T00:00:00.000Z_2026-06-19T07:29:21.197Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1996-01-01T00:00:00.000Z_1997-01-01T00:00:00.000Z_2026-06-19T07:29:40.260Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1977-01-01T00:00:00.000Z_1978-01-01T00:00:00.000Z_2026-06-19T07:29:24.311Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1980-01-01T00:00:00.000Z_1981-01-01T00:00:00.000Z_2026-06-19T07:29:25.956Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1988-01-01T00:00:00.000Z_1989-01-01T00:00:00.000Z_2026-06-19T07:29:31.476Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1969-01-01T00:00:00.000Z_1970-01-01T00:00:00.000Z_2026-06-19T07:29:22.891Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1970-01-01T00:00:00.000Z_1971-01-01T00:00:00.000Z_2026-06-19T07:29:22.991Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2000-01-01T00:00:00.000Z_2001-01-01T00:00:00.000Z_2026-06-19T07:29:44.321Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1979-01-01T00:00:00.000Z_1980-01-01T00:00:00.000Z_2026-06-19T07:29:24.950Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1987-01-01T00:00:00.000Z_1988-01-01T00:00:00.000Z_2026-06-19T07:29:30.757Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1982-01-01T00:00:00.000Z_1983-01-01T00:00:00.000Z_2026-06-19T07:29:27.076Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1990-01-01T00:00:00.000Z_1991-01-01T00:00:00.000Z_2026-06-19T07:29:33.362Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1985-01-01T00:00:00.000Z_1986-01-01T00:00:00.000Z_2026-06-19T07:29:28.970Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1993-01-01T00:00:00.000Z_1994-01-01T00:00:00.000Z_2026-06-19T07:29:36.957Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1991-01-01T00:00:00.000Z_1992-01-01T00:00:00.000Z_2026-06-19T07:29:34.260Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2018-01-01T00:00:00.000Z_2019-01-01T00:00:00.000Z_2026-06-19T07:30:09.510Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1994-01-01T00:00:00.000Z_1995-01-01T00:00:00.000Z_2026-06-19T07:29:38.080Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2021-01-01T00:00:00.000Z_2022-01-01T00:00:00.000Z_2026-06-19T07:30:15.438Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1997-01-01T00:00:00.000Z_1998-01-01T00:00:00.000Z_2026-06-19T07:29:41.370Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1967-01-01T00:00:00.000Z_1968-01-01T00:00:00.000Z_2026-06-19T07:29:22.395Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2014-01-01T00:00:00.000Z_2015-01-01T00:00:00.000Z_2026-06-19T07:30:03.938Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1955-01-01T00:00:00.000Z_1956-01-01T00:00:00.000Z_2026-06-19T07:29:21.389Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1981-01-01T00:00:00.000Z_1982-01-01T00:00:00.000Z_2026-06-19T07:29:26.325Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2022-01-01T00:00:00.000Z_2023-01-01T00:00:00.000Z_2026-06-19T07:30:17.507Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2003-01-01T00:00:00.000Z_2004-01-01T00:00:00.000Z_2026-06-19T07:29:47.697Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2017-01-01T00:00:00.000Z_2018-01-01T00:00:00.000Z_2026-06-19T07:30:08.350Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2006-01-01T00:00:00.000Z_2007-01-01T00:00:00.000Z_2026-06-19T07:29:52.140Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1984-01-01T00:00:00.000Z_1985-01-01T00:00:00.000Z_2026-06-19T07:29:28.192Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2020-01-01T00:00:00.000Z_2021-01-01T00:00:00.000Z_2026-06-19T07:30:13.466Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1968-01-01T00:00:00.000Z_1969-01-01T00:00:00.000Z_2026-06-19T07:29:22.818Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2009-01-01T00:00:00.000Z_2010-01-01T00:00:00.000Z_2026-06-19T07:29:57.185Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2012-01-01T00:00:00.000Z_2013-01-01T00:00:00.000Z_2026-06-19T07:30:01.054Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1965-01-01T00:00:00.000Z_1966-01-01T00:00:00.000Z_2026-06-19T07:29:22.093Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1956-01-01T00:00:00.000Z_1957-01-01T00:00:00.000Z_2026-06-19T07:29:21.490Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1953-01-01T00:00:00.000Z_1954-01-01T00:00:00.000Z_2026-06-19T07:29:20.985Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1975-01-01T00:00:00.000Z_1976-01-01T00:00:00.000Z_2026-06-19T07:29:23.795Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1964-01-01T00:00:00.000Z_1965-01-01T00:00:00.000Z_2026-06-19T07:29:22.003Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2019-01-01T00:00:00.000Z_2020-01-01T00:00:00.000Z_2026-06-19T07:30:11.493Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1950-01-01T00:00:00.000Z_1951-01-01T00:00:00.000Z_2026-06-19T07:29:19.904Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1978-01-01T00:00:00.000Z_1979-01-01T00:00:00.000Z_2026-06-19T07:29:24.726Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1961-01-01T00:00:00.000Z_1962-01-01T00:00:00.000Z_2026-06-19T07:29:21.866Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1958-01-01T00:00:00.000Z_1959-01-01T00:00:00.000Z_2026-06-19T07:29:21.689Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2011-01-01T00:00:00.000Z_2012-01-01T00:00:00.000Z_2026-06-19T07:30:00.026Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1995-01-01T00:00:00.000Z_1996-01-01T00:00:00.000Z_2026-06-19T07:29:39.124Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1992-01-01T00:00:00.000Z_1993-01-01T00:00:00.000Z_2026-06-19T07:29:35.184Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1966-01-01T00:00:00.000Z_1967-01-01T00:00:00.000Z_2026-06-19T07:29:22.189Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1952-01-01T00:00:00.000Z_1953-01-01T00:00:00.000Z_2026-06-19T07:29:20.495Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1998-01-01T00:00:00.000Z_1999-01-01T00:00:00.000Z_2026-06-19T07:29:42.273Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1963-01-01T00:00:00.000Z_1964-01-01T00:00:00.000Z_2026-06-19T07:29:21.922Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2001-01-01T00:00:00.000Z_2002-01-01T00:00:00.000Z_2026-06-19T07:29:45.244Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1971-01-01T00:00:00.000Z_1972-01-01T00:00:00.000Z_2026-06-19T07:29:23.012Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2004-01-01T00:00:00.000Z_2005-01-01T00:00:00.000Z_2026-06-19T07:29:49.326Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2015-01-01T00:00:00.000Z_2016-01-01T00:00:00.000Z_2026-06-19T07:30:05.667Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1974-01-01T00:00:00.000Z_1975-01-01T00:00:00.000Z_2026-06-19T07:29:23.591Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2010-01-01T00:00:00.000Z_2011-01-01T00:00:00.000Z_2026-06-19T07:29:58.209Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1999-01-01T00:00:00.000Z_2000-01-01T00:00:00.000Z_2026-06-19T07:29:43.265Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1962-01-01T00:00:00.000Z_1963-01-01T00:00:00.000Z_2026-06-19T07:29:21.904Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1951-01-01T00:00:00.000Z_1952-01-01T00:00:00.000Z_2026-06-19T07:29:20.291Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2013-01-01T00:00:00.000Z_2014-01-01T00:00:00.000Z_2026-06-19T07:30:02.655Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2002-01-01T00:00:00.000Z_2003-01-01T00:00:00.000Z_2026-06-19T07:29:46.185Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1983-01-01T00:00:00.000Z_1984-01-01T00:00:00.000Z_2026-06-19T07:29:27.306Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1959-01-01T00:00:00.000Z_1960-01-01T00:00:00.000Z_2026-06-19T07:29:22.238Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2005-01-01T00:00:00.000Z_2006-01-01T00:00:00.000Z_2026-06-19T07:29:50.818Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1986-01-01T00:00:00.000Z_1987-01-01T00:00:00.000Z_2026-06-19T07:29:29.645Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1972-01-01T00:00:00.000Z_1973-01-01T00:00:00.000Z_2026-06-19T07:29:23.165Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2008-01-01T00:00:00.000Z_2009-01-01T00:00:00.000Z_2026-06-19T07:29:55.562Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_1989-01-01T00:00:00.000Z_1990-01-01T00:00:00.000Z_2026-06-19T07:29:32.288Z, state=APPENDING}, SegmentWithState{segmentIdentifier=earthquakes_2016-01-01T00:00:00.000Z_2017-01-01T00:00:00.000Z_2026-06-19T07:30:06.814Z, state=APPENDING}]}, lastSegmentIds={index_kafka_earthquakes_ecfc7af154be638_0=earthquakes_2022-01-01T00:00:00.000Z_2023-01-01T00:00:00.000Z_2026-06-19T07:30:17.507Z}, callerMetadata={nextPartitions=SeekableStreamEndSequenceNumbers{stream='earthquakes', partitionSequenceNumberMap={KafkaTopicPartition{partition=1, topic='null', multiTopicPartition=false}=379764, KafkaTopicPartition{partition=0, topic='null', multiTopicPartition=false}=378974, KafkaTopicPartition{partition=3, topic='null', multiTopicPartition=false}=380967, KafkaTopicPartition{partition=2, topic='null', multiTopicPartition=false}=374837, KafkaTopicPartition{partition=5, topic='null', multiTopicPartition=false}=385200, KafkaTopicPartition{partition=4, topic='null', multiTopicPartition=false}=385358, KafkaTopicPartition{partition=7, topic='null', multiTopicPartition=false}=386689, KafkaTopicPartition{partition=6, topic='null', multiTopicPartition=false}=374950}}}}] for segments: earthquakes_2021-01-01T00:00:00.000Z_2022-01-01T00:00:00.000Z_2026-06-19T07:30:15.438Z, earthquakes_2022-01-01T00:00:00.000Z_2023-01-01T00:00:00.000Z_2026-06-19T07:30:17.507Z
2026-06-19T07:30:20,580 INFO [[index_kafka_earthquakes_ecfc7af154be638_pkjodnek]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.StreamAppenderator - Persisted stats: processed rows: [3051545], persisted rows[67250], sinks: [73], total fireHydrants (across sinks): [156], persisted fireHydrants (across sinks): [2]

@NickLarsenNZ NickLarsenNZ enabled auto-merge (squash) June 19, 2026 07:38
@NickLarsenNZ NickLarsenNZ merged commit f4eea48 into main Jun 19, 2026
2 checks passed
@NickLarsenNZ NickLarsenNZ deleted the druid-37.0.0 branch June 19, 2026 07:38
@NickLarsenNZ NickLarsenNZ moved this from Development: Waiting for Review to Development: Done in Stackable Engineering Jun 19, 2026
@lfrancke lfrancke moved this from Development: Done to Done in Stackable Engineering Jun 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

3 participants