Skip to content

Roadmap

The Java and Python libraries evolve together: each feature is released in both, with the same concepts. Feedback and contributions are welcome: open an issue on Asgarde Java or Asgarde Python.

1.3.0 (released in Java, planned in Python)

Section titled “1.3.0 (released in Java, planned in Python)”

Enrichment of the Failure model and integration with the Beam native error handling, without breaking change.

Feature Java / Kotlin Python
Richer Failure: exception type, message and stack trace computed from the exception, and a timestamp ✅ 1.3.0 (Failure) #7
Failures to BigQuery: conversion to Beam Rows (Java) or dicts (Python) with a documented schema, and a BigQuery recipe ✅ 1.3.0 (Write the failures) #8
Beam native error handling: bridge to the Beam ErrorHandler (BadRecord), and any Beam transform in the Python composer (with_exception_handling) ✅ 1.3.0 (Beam native error handling) #9

A major version for the platform, not for the Failure model:

  • Java 11 bytecode, when the minimum supported Beam version is raised to 2.74.0 or later (Beam requires Java 11 since this version).
  • Kotlin 2 for the extensions, whose metadata can’t be read by Kotlin 1.x projects.

Not planned: they’ll be considered if users need them.

  • Custom Failure type (Java): inject a Failure factory once for all the steps. It needs a generic composer, a breaking and complex change.
  • Serializer per step for the input element: mostly covered by the origin element, which already lets you choose the format (JSON, id…) to replay from the start.
  • Ready-to-use failure sinks module (BigQuery, GCS, Pub/Sub): a sink module would bring the GCP dependencies; the 1.3.0 conversion plus a one line recipe with the Beam IOs covers most needs.
Version Highlights
1.2.0 Origin element in the failures (lazy evaluation)
1.1.0 (Java) Failure handling guarantees and failure metrics
1.0.0 Not tied to a Beam version anymore: Beam is provided (Java) or a minimum version (Python), tested every week against the latest Beam release. The Python 1.0.0 also brings the guarantees and the metrics