Picodata S3R
Amazon S3-compatible object storage for Lakehouse. Fast, horizontally scalable, disaster-tolerant metadata. Reed-Solomon erasure coding for data.
Key capabilities
-
Amazon S3 compatibility: applications and SDKs work without code changes -
Metadata stored in database tables; instant reads of object attributes, small objects and directory listings -
Transactional catalog integrity: strong consistency for reads and writes -
Erasure coding within a data center, asynchronous replication across data centers -
Multi-tenancy: each tenant has its own domain, certificate and quotas -
One security perimeter with the Picodata database: shared users, roles and grants
Architecture
-
Metadata consistency
Users, tenants, buckets and access keys live in global tables and are agreed through the cluster-wide Raft group. The object catalog is sharded, and each replicaset agrees on its part through its own Raft group.
-
Erasure coding and replication
Within a data center, objects are protected by Reed-Solomon erasure coding, and checksums are verified on every read. Across data centers, data is shipped asynchronously while metadata is replicated by the database, so an object is readable at any site as soon as it is written.
-
Small objects and directories in memory
An object of up to 4 KiB is stored in its metadata row, and directories are stored as rows of their own. Reading a small object or listing a directory is an in-memory index lookup.
-
Scale with cores
In conventional object stores, scale is capped by a single PostgreSQL server. S3R scales out by adding database instances.
-
Catalog reconciliation and repair
An additional mechanism reconciles blocks against metadata, both within a site and across sites. Data can be partially recovered after correlated failures.
-
Management built into the plugin
The console, the management API, quotas, Prometheus metrics and Grafana dashboards ship with the plugin. There are no separate management services to deploy.
S3R compared with MinIO and Apache Ozone
| Capability | MinIO | Apache Ozone | Picodata S3R |
|---|---|---|---|
| Metadata storage and consistency | xl.meta on the drives of the erasure set; read-after-write on xfs, zfs or btrfs | RocksDB in Ozone Manager, replicated with Ratis; strong consistency is stated | Database tables with synchronous replication; linearizable per key |
| Metadata scaling | No central service; with several pools, every GET and HEAD queries each pool | Ozone Manager scales vertically only | Sharded across replicasets; grows with the number of database instances |
| Small objects and listing | Inlined into xl.meta; a listing walks every set of every pool and merges the results | The general path through containers; open issues on listing large directories | An object of up to 4 KiB is stored in its metadata row; the cost of a listing page does not grow with the directory |
| Conditional writes and versioning | Yes | No; conditional writes are planned | Yes |
| Redundancy and cross-site replication | Erasure coding in sets of up to 16 drives. Buckets and sites are replicated, asynchronously by default. | 3 replicas or RS 3+2, 6+3, 10+4; snapshots and DistCp across sites | Reed-Solomon erasure coding within a data center. Across data centers, metadata is replicated synchronously and data asynchronously. |
| Reconciliation and repair | Healing on read; the background scanner samples objects, and bitrot checking is off by default | Container replicas checked every 5 minutes, checksums every 7 days; a lost container needs manual intervention | Checksums on every read; blocks, objects and directories reconciled within and across sites |
| Hadoop and Lakehouse engines | Through S3; the community docs cover Spark and Hive | Native ofs:// and o3fs://; Hive, Impala, Spark and Iceberg documented | Access through S3, with efficient support for Iceberg and Delta Lake. |
| Support | Community repository unmaintained since 2026 | Apache community project | Vendor-supported |
Picodata S3R: object storage for Lakehouse at scale
Picodata S3R brings S3-compatible object storage and a distributed database together in one cluster. Metadata lives in Picodata tables and scales with the number of database instances; data is protected by Reed-Solomon erasure coding within a data center and replicated across data centers. Strong consistency, instant reads of small objects and directories, and catalog reconciliation come from the architecture itself, not from additional services.
The plugin supports versioning and object lock, lifecycle rules, storage classes with transitions between disk tiers, multipart upload and conditional writes. Amazon S3 compatibility is backed by a conformance suite in which all 412 selected cases pass. Applications, SDKs and tools that work with MinIO switch to S3R without code changes.
With the MinIO community edition no longer maintained, and with metadata services that only scale vertically, choosing an object store has become a long-term risk decision. Picodata S3R answers it with a product built for large installations and supported by the team that develops it.