Join today’s leading executives online at the Data Summit on March 9th. Register here.
Let the OSS Enterprise newsletter guide your open source journey! Sign up here.
ThatDot, a startup headquartered in Portland, Oregon that offers a complex event processing (CEP) platform to capture the full value of streaming data for advanced AI and ML applications, has released open source software to help developers and data pipeline engineers build high volume, real-time event processing workflows at scale.
Officially dubbed Quine, the solution combines event streaming and graph data technologies to connect to existing data streams and build data into a stateful graph. Then, it analyzes this graph for user-specified “standing queries” and streams results out to trigger real-time event-driven workflows.
Quine accelerates processing
The offering comes as the answer to event processing frameworks such as Flink. Ryan Wright, the cofounder of ThatDot, notes that these previous generation solutions come with various limitations and spend enormous time — on the scale of months — and effort to build complicated event-driven architectures that only work on short time windows of in-memory data and miss out on the bigger picture.
Quine, on the other hand, uses a handful of queries to transform the tedious data engineering process into an afternoon job. It can eliminate batch processing, multi-level joins, and other time-consuming and outdated processes that drag down and stall analysis on streaming data. This way, data pipeline engineering teams can easily interpret high-volume event data streams, innovate and ship products faster and use the emerging Graph AI tools driving the next wave in machine learning.
The company, with its early access launch partners and community members, has also created pre-built application functions called “recipes” to help data pipeline engineers with multiple event stream use-cases. This includes Blockchain Real-time Tag Propagation to trace money laundering, CDN cache efficiency analysis to continuously monitor CDN logs to materialize cache efficiency and generate alerts, and Kubernetes event observability to ingest Kubernetes events and calculate state by component, pod, and service for alerts and root cause traces.
Previous application
ThatDot created Quine streaming graph solution in 2014 and has since been leveraging it as part of its software portfolio. In 2015, Wright had also led a team of researchers and developers on the DARPA Transparent Computing program and used Quine to create new capabilities for finding and stopping Advanced Persistent Threats (APTs). Now, the company is taking it to data pipeline engineers worldwide.
“The decision to open-source the Quine streaming graph underscores ThatDot’s conviction that the best infrastructure software thrives within an open and diverse community of contributors, and that well-made software freely available benefits everyone,” the company notes on Quine’s website. The solution can also be accessed on GitHub.
