GitHub / apache/paimon / commits
Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
| SHA | Message | Author | Date | Stats |
|---|---|---|---|---|
| 0af297be | [python] Fix GenericVariant UUID decoding for Java compatibility and support ... |
Juntao Zhang <j****g@f****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| 567c8401 | [spark] Fix bucketed scan reporting for unsupported timestamp precision (#9216) |
Xiangyi Zhu <8****i@u****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| 158982cd | [python][ray] Fix update_by_row_id failure on empty datasets (#9217) |
XiaoHongbo <x****b@a****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| 8db20324 | [spark][flink] Support bucket-level compaction for fixed-bucket tables (#9219) |
sanshi <4****8@u****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| f5bb59de | [python] Add tests for Data Evolution BLOB writes on branches (#9221) |
XiaoHongbo <x****b@a****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| 8b1364a3 | [python] Update paimon-rust to speed up native-plan CI (#9235) |
XiaoHongbo <x****b@a****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| 350a13ed | [core] Validate deletion vector mode when resetting table options (#9237) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| dc52c87f | [spark] Add option for Hive-style dynamic partition writes (#9222) |
Kerwin Zhang <x****k@a****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| 4dba8f84 | [format] Validate blob file footer and index metadata (#9223) |
Wenchao Wu <6****1@u****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| fa3c0967 | [spark] Fix missing data for delete hiding remaining records (#9224) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| c80f8a68 | [flink] Fix scan partitions with partial partition spec (#9225) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| 0b129229 | [core] Close every writer and shut both executors down when one close fails (... |
ZIHAN DAI <9****K@u****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| 53a549a8 | [service] Fix KvQueryClient hang when query location lookup fails on retry (#... |
Eunbin Son <5****s@u****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| cf09697e | [core] Support DATE, TIMESTAMP and TIMESTAMP_NTZ in variant shredding schema ... |
Juntao Zhang <j****g@f****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| fec6be9e | [core] Parallelize primary-key sorted-index scan (#9231) |
QuakeWang <4****g@u****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| 796b698b | [core] Fix incremental diff scan missing first snapshot (#9234) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| cdf2fb10 | [core] Speed up row-id manifest sorting (#9135) |
YeJunHao <4****8@u****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| 072097d4 | [core] Optimize ordinary manifest merging with raw Avro blocks (#9213) |
YeJunHao <4****8@u****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| c55209ab | [core] Support arbitrary time granularity for chain table delta computation (... |
Juntao Zhang <j****g@f****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| 55d3511b | [flink][spark] Support manifest sort in compact_manifest (#9083) |
umi <5****r@u****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| 8f46a628 | [common] Compare maps of different representations without throwing (#9211) |
ZIHAN DAI <9****K@u****m>
Committed by: GitHub <n****y@g****m> |
14 days ago | |
| 54e64e36 | [core] Fix stale file index cleanup (#9085) |
huangxiaoping <1****5@q****m>
Committed by: GitHub <n****y@g****m> |
15 days ago | |
| 087cbd26 | [docs] Sync procedure documentation (#9101) |
huangxiaoping <1****5@q****m>
Committed by: GitHub <n****y@g****m> |
15 days ago | |
| 191bb2c1 | Bump org.apache.logging.log4j:log4j-api from 2.25.4 to 2.25.5 (#9215) |
dependabot[bot] <4****]@u****m>
Committed by: GitHub <n****y@g****m> |
15 days ago | |
| ed26b66a | [python] Avoid Arrow offset overflow when merging read batches (#9209) |
XiaoHongbo <x****b@a****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| 887e5ed3 | [spark] Prevent table deletion for self referencing RTAS (#9139) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| ea293d58 | [common] Fix missing rows for null range bitmap indexes (#9140) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| c721d2c5 | [core] Support latest snapshot delta scan in batch (#9142) |
wangwj <h****j@g****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| e0cc46c8 | [core] Fix incorrect file indexes table range filtering (#9176) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| fe081c02 | [flink] Avoid redundant HTTP BLOB existence probes (#9181) |
wangwj <h****j@g****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| dd1b91d5 | [core] Avoid repeated sorted-index source lookups (#9187) |
QuakeWang <4****g@u****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| b6e0e78e | [core] avoid reading stale blobs during fallback read. (#9208) |
Faiz <w****9@a****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| a398e4e5 | [iceberg] Exclude NaN from manifest partition bounds (#9188) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| 17e9ad33 | [core] Fix dynamic bucket allocation across partitions (#9193) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| 3da36f2f | [python] Support create_partitions in RESTCatalog (#9198) |
Jiajia Li <p****a@a****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| c9eb26e8 | [spark] Do not display an unknown partition statistic as a number (#9200) |
Dapeng Sun(孙大鹏) <d****p@a****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| c2aea2ba | [docs] Clarify snapshot record count semantics (#9202) |
wangwj <h****j@g****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| 66156311 | [common] Fix incorrect date for casting pre epoch time (#9203) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| 30c96ae7 | [common] Fix like optimization for escaped patterns (#9205) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| 81a3f271 | [python] Fix pyjindo advanced config setting in pypaimon (#9206) |
timmyyao <1****1@q****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| d69a7e65 | [spark] Retry data evolution compaction after merge conflicts (#9197) |
Jingsong Lee <j****0@g****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| 87eb0445 | [core][format] Support Arrow bundle writes (#9100) |
jianguotian <8****4@q****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| 6cda45e6 | [core] Allow first-row merge engine for primary-key managed BLOB tables. (#9201) |
Wenchao Wu <6****1@u****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| 42b81c3d | [core] Refactor conflict detection by table type (#9196) |
Jingsong Lee <j****0@g****m>
Committed by: GitHub <n****y@g****m> |
16 days ago | |
| 4425a598 | [core] Add manifest Avro writer (#9194) |
YeJunHao <4****8@u****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| 35f5d892 | [spark] Introduce reassign_row_id procedure (#9192) |
Xiangyi Zhu <8****i@u****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| 83f7409c | [flink] Avoid overlapping sorted index key ranges (#9189) |
YeJunHao <4****8@u****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| c249758d | [test] Fix URI-based temporary paths (#9191) |
Jingsong Lee <j****0@g****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| 9f8bcdc5 | [core] Optimize data evolution compaction planning (#9177) |
Jingsong Lee <j****0@g****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| 10efb03f | [spark] Add Format Table partition operation tests and clearer errors (#9119) |
Dapeng Sun(孙大鹏) <d****p@a****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| 15a454b2 | [core] Register every bucket in bucketList so overflow rows spread (#9180) |
ZIHAN DAI <9****K@u****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| a3907798 | [flink] Fix CSV delimiter fallback for format tables (#9178) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| 3b7d606a | [core] Parallelize primary-key full-text search (#9184) |
QuakeWang <4****g@u****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| 527f2eec | [core] Fix invalid length when reading spilled data (#9183) |
tsreaper <t****6@g****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| d2a22e09 | [core] Fix level 1 inflation in universal compaction (#9185) |
tsreaper <t****6@g****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| 7ab3947e | [core][flink] Remove file-based privilege system (#9186) |
Jingsong Lee <j****0@g****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| f2a59c20 | [core] Unify manifest Avro readers (#9182) |
YeJunHao <4****8@u****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| ea855002 | [spark] Improve compaction logs in CompactProcedure (#9169) |
Zouxxyy <z****y@a****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| 233abc64 | [common] Finish the lookup-store close chain instead of stopping at the first... |
ZIHAN DAI <9****K@u****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| 711cc647 | [python] Fix locale-sensitive Date header and nonce replay on retry (#9173) |
Dapeng Sun(孙大鹏) <d****p@a****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| c9960797 | [core] Fix IS NAN predicate failing with file index (#9143) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| 87de4a72 | [iceberg] Omit unknown bounds in Iceberg manifest partition summary (#9152) |
Eunbin Son <5****s@u****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| 1ceeeb60 | [iceberg] Warn when the Iceberg metadata committer factory is missing from th... |
Andreas Bube <a****e@t****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| a5ed6adf | [core] Restore the interrupt status in the two JDBC catalog paths that drop i... |
ZIHAN DAI <9****K@u****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| b0f89c92 | [core] Release every reader in SortMergeReaderWithMinHeap.close() (#9163) |
ZIHAN DAI <9****K@u****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| 30677e4f | [spark] Fix IndexOutOfBoundsException for vector_search with too few argument... |
jackylee <q****1@g****m>
Committed by: GitHub <n****y@g****m> |
17 days ago | |
| 7c820f3d | [core] Compact adjacent overlapping blob ranges together (#9175) |
YeJunHao <4****8@u****m>
Committed by: GitHub <n****y@g****m> |
18 days ago | |
| a05e8f98 | [format] Support delimiter as fallback key of csv.field-delimiter (#9170) |
Zouxxyy <z****y@a****m>
Committed by: GitHub <n****y@g****m> |
18 days ago | |
| 98db99dc | [spark] Respect blob-compaction.enabled in Spark compact procedure (#9165) |
jackylee <q****1@g****m>
Committed by: GitHub <n****y@g****m> |
18 days ago | |
| e3cd6808 | [python] Align exact VARIANT integer and decimal types (#9168) |
XiaoHongbo <x****b@a****m>
Committed by: GitHub <n****y@g****m> |
18 days ago | |
| faacb7e1 | [flink] Fix flaky Lumina data evolution index refresh IT (#9153) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
18 days ago | |
| 4b0d6589 | [python] Add efficient VARIANT path get and replace (#9147) |
XiaoHongbo <x****b@a****m>
Committed by: GitHub <n****y@g****m> |
18 days ago | |
| 8fff1ff5 | [core] Rename to ProjectedManifestEntry | JingsongLi <j****0@g****m> | 18 days ago | |
| 42857c25 | [core] Remove unused manifest scan file size | JingsongLi <j****0@g****m> | 18 days ago | |
| e0e7ac9d | [spark] Fix MERGE INTO silently skipping WHEN NOT MATCHED BY SOURCE actions (... |
Kerwin Zhang <x****k@a****m>
Committed by: GitHub <n****y@g****m> |
18 days ago | |
| 79a8d165 | [core] Add block-based manifest Avro reader (#9160) |
Jingsong Lee <j****0@g****m>
Committed by: GitHub <n****y@g****m> |
18 days ago | |
| 70d114db | [core] Refactor compact file identifiers | JingsongLi <j****0@g****m> | 18 days ago | |
| b8527c49 | [spark][test] Use explicit Paimon provider in feature tests (#9145) |
Zouxxyy <z****y@a****m>
Committed by: GitHub <n****y@g****m> |
18 days ago | |
| 17bc7645 | [python] Fix row-id updates for Arrow columns over 2 GiB (#9146) |
umi <5****r@u****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| e274652d | [lance] Reject unsupported VARIANT and BLOB types in LanceFileFormat schema v... |
Eunbin Son <5****s@u****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| 4156dd3e | [core] Remove configurable manifest format (#9154) |
YeJunHao <4****8@u****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| 7cd451fa | [python] Keep native planning for Blob descriptor reads (#9144)a |
XiaoHongbo <x****b@a****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| 158d83cb | [python] Reuse streams for coalesced blob reads (#9133) |
XiaoHongbo <x****b@a****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| 3c43be6d | [lance] Reject unsupported element types nested in ARRAY (#9102) |
jackylee <q****1@g****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| b1841c3a | [core] Fix stale default value in sort-compaction.range-strategy description ... |
jackylee <q****1@g****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| dc30c49c | [core] Reject Variant type for primary key and partition key (#9105) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| eed76ed7 | [format][python] Reject duplicate MAP BLOB keys (#9107) |
QuakeWang <4****g@u****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| a5431b32 | [common] Fix timestamp partition values being written but not readable (#9115) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| 6208af5b | Fix #9113 Exception when reading Paimon table via Hive proxy user in … (#9116) |
zhenye zhang <z****3@1****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| db1b5c7c | [flink] Fix flaky test duplicate nullable PK IT (#9117) |
Arnav Balyan <6****n@u****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| 79a8b644 | [iceberg] Fix Iceberg metadata after a rollback (#9036) |
Jiajia Li <p****a@a****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| b2599b08 | [core] Fix file index filtering with deletion vectors (#9088) |
huangxiaoping <1****5@q****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| 796faf2f | [common] Fix concurrent remote stream leak in CachingSeekableInputStream (#9098) |
Vova Kolmakov <w****n@g****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| 288f9777 | [rest] Do not replay a POST the server cannot absorb twice (#9122) |
Dapeng Sun(孙大鹏) <d****p@a****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| 1f15a041 | [flink] Fix nullable primary key test across Flink versions (#9118) |
QuakeWang <4****g@u****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| cfd07816 | [core] Spell out what a negative partition statistic means (#9120) |
Dapeng Sun(孙大鹏) <d****p@a****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| b297d907 | [core] Carry the row count and byte size a format table writer already counte... |
Dapeng Sun(孙大鹏) <d****p@a****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| b8c9383d | [core] Honor query-level search-mode overrides (#9128) |
CrownChu <4****u@u****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| 6dafb0c3 | [core] Fix partition timestamp pattern replacement for prefix variable names ... |
Juntao Zhang <j****g@f****m>
Committed by: GitHub <n****y@g****m> |
19 days ago | |
| 3d10846a | [core] Add file index system table (#9138) |
sanshi <4****8@u****m>
Committed by: GitHub <n****y@g****m> |
19 days ago |