Compression is almost done: it's quite thoroughly tested, the only remaining issue is a problem with copygc fragmenting existing compressed extents that only breaks accounting.
NFS export support is almost done: implementing i_generation correctly required some new transaction machinery, but that's mostly done. What's left is implementing a new kind of reservation of journal space for the new, long running transactions.
Allocation information (currently just bucket generation numbers & priority numbers, for LRU caching) needs to be moved into a btree, and we need to start persisting actual allocation information so we don't have to walk all extents at mount time.
Just moving the existing prios/gens to a btree will be a significant improvement - besides getting us incrementally closer to persisting full allocation information, the existing code is a rather hacky mechanism dating from the early days of bcache and has recently been the source of an annoying bug due to the design being a bit fragile, and it'll be a performance improvement since it'll get rid of the last source of forced journal flushes.
When we're using compression, we end up wasting a fair amount of space on internal fragmentation because compressed extents get rounded up to the filesystem block size when they're written - usually 4k. It'd be really nice if we could pack them in more efficiently - probably 512 byte sector granularity.
On the read side this is no big deal to support - we have to bounce compressed extents anyways. The write side is the annoying part. The options are:
- Buffer up writes when we don't have full blocks to write? Highly problematic, not going to do this.
- Read modify write? Not an option for raw flash, would prefer it to not be our only option
- Do data journalling when we don't have a full block to write? Possible solution, we want data journalling anyways
Inline extents - good for space efficiency for both small files, and compression when extents happen to compress particularly well.
Full data journalling - we're definitely going to want this for when the journal is on an NVRAM device (also need to implement external journalling (easy), and direct journal on NVRAM support (what's involved here?)).
Would be good to get a simple implementation done and tested so we know what the on disk format is going to be.
(Will not be implemented by the lead developer.)
- It is possible to add bcache functionality to bcachefs. If there is someone who wishes to implement this it would be an ok addition. However using it as a filesystem should still be better.
End user documentation needs a lot of work - complete man pages, etc.
bcachefs-tools needs some fleshing out in the --help department
- Write a tool to benchmark tail-latency.
Benchmark bcachefs performance on different configurations: (Note that these tests will have to be repeated in the future.)
- SSD + HDD
Update the website / Documentation
- Ask questions (so and we can update the documentation / website).
- "Seed devices", hard-readonly devices that are CoWed from on write (btrfs has this; useful for base devices for virtualization, among other things).
- Nonce-misuse-resistant authenticated encryption, such as AES-SIV or HS1-SIV (Closes potential hole regarding nonce reuse and "external" snapshots, as might happen to VMs or systems with externally-managed storage like iSCSI).
- Some form of "secure delete" functionality. (However, see this LWN article regarding implementation strategies and pitfalls).
- A simplified userspace API with no hierarchy, only blobs identified by unique integer keys (eternaleye thinks this might be useful for object-capability systems, such as Robigalia).
- An API like the above, but supporting multiple streams per blob, possibly with string identifiers (needs further examination, intent is to match the needs of CephFS for OSD backends).
- More advanced caching algorithms; one potentially-relevant paper is Pannier: A Container-based Flash Cache for Compound Objects.
- "Asymmetrical" compression algorithms, that support only decompression (XZ is a nice candidate here, and would be a very good match for some seed device use cases).
- RAID-6 with parity 3 or greater - could potentially use Andrea Mazzoleni's technique for generating Cauchy matrices compatible with Linux' current RAID-5 and RAID-6 formats, providing a clean upgrade path.
- "Inline" forward error correction, possibly using a fountain code like RaptorQ.
- Support Trusted/Encrypted kernel keyring keys, in order to take advantage of TPMs.
- Support for multiple key slots.
- Ponder the ramifications of (and safe defaults for) compression in the presence of encryption.
- Swap file support.
- Support (a subset of?) the Ext attributes denoting special behaviors:
+cfor compressed files
+Cfor disabling copy-on-write
+efor extent-based storage (always set? btrfs doesn't set it...)
+Efor displaying that a file is encrypted
+Nfor displaying that a file's contents are inlined into the inode
+sfor files that should be securely erased on delete
+ufor files that should permit being "undeleted"
- Others seem less relevant, but may be worth investigating.