Roadmap
The Java and Python libraries evolve together: each feature is released in both, with the same concepts. Feedback and contributions are welcome: open an issue on Asgarde Java or Asgarde Python.
1.3.0 (released in Java, planned in Python)
Section titled “1.3.0 (released in Java, planned in Python)”Enrichment of the Failure model and integration with the Beam native error handling, without breaking change.
| Feature | Java / Kotlin | Python |
|---|---|---|
Richer Failure: exception type, message and stack trace computed from the exception, and a timestamp |
✅ 1.3.0 (Failure) | #7 |
Failures to BigQuery: conversion to Beam Rows (Java) or dicts (Python) with a documented schema, and a BigQuery recipe |
✅ 1.3.0 (Write the failures) | #8 |
Beam native error handling: bridge to the Beam ErrorHandler (BadRecord), and any Beam transform in the Python composer (with_exception_handling) |
✅ 1.3.0 (Beam native error handling) | #9 |
2.0.0 (later)
Section titled “2.0.0 (later)”A major version for the platform, not for the Failure model:
- Java 11 bytecode, when the minimum supported Beam version is raised to
2.74.0or later (Beam requires Java 11 since this version). - Kotlin 2 for the extensions, whose metadata can’t be read by Kotlin 1.x projects.
Ideas, on demand
Section titled “Ideas, on demand”Not planned: they’ll be considered if users need them.
- Custom
Failuretype (Java): inject aFailurefactory once for all the steps. It needs a generic composer, a breaking and complex change. - Serializer per step for the input element: mostly covered by the origin element, which already lets you choose the format (JSON, id…) to replay from the start.
- Ready-to-use failure sinks module (BigQuery, GCS, Pub/Sub): a sink module would bring the GCP dependencies; the 1.3.0 conversion plus a one line recipe with the Beam IOs covers most needs.
Delivered
Section titled “Delivered”| Version | Highlights |
|---|---|
| 1.2.0 | Origin element in the failures (lazy evaluation) |
| 1.1.0 (Java) | Failure handling guarantees and failure metrics |
| 1.0.0 | Not tied to a Beam version anymore: Beam is provided (Java) or a minimum version (Python), tested every week against the latest Beam release. The Python 1.0.0 also brings the guarantees and the metrics |
