1. Providing real-time, enriched context for agents
1.1. Pub/Sub AI Inference SMT (GA): You can now run inference on messages streamed through Pub/Sub. Data practitioners can choose any models available on Gemini Enterprise Agent Platform. Pub/Sub makes the inference call and appends the result to each message before sending it downstream, bringing Pub/Sub’s simplicity together with the Gemini Enterprise’s fully managed tools.
1.2. Pub/Sub Bigtable subscriptions (Preview): Stream Pub/Sub data directly to Bigtable. Pub/Sub Bigtable subscriptions directly materialize event data from a Pub/Sub topic into a Bigtable table, eliminating the need for custom pipelines and dramatically simplifying your streaming architecture. For instance, you can easily ingest vector embeddings into Bigtable to power semantic search workloads.
1.3. BigQuery continuous queries stateful data processing (Preview): BigQuery continuous queries can now perform complex correlations between multiple data streams using JOINs and calculate metrics over consistent time intervals with tumbling window aggregations. This enables sophisticated analysis, such as calculating 30-minute averages or correlating events across different streams, directly as data is ingested into BigQuery. Furthermore, you can integrate AI directly into your data pipelines by calling generative functions like AI.GENERATE_TEXT, as well as materialize continuous query SQL results into BigQuery tables or export them to operational sinks like Bigtable, Spanner, and Pub/Sub for real-time reverse ETL.
2. Direct agents to manage your resources
2.1. Model Context Protocol (MCP) support for Pub/Sub, Managed service for Apache Kafka, Bigtable and BigQuery (GA): Your agents can manage Pub/Sub,Managed service for Apache Kafka services, and BigQuery using fully managed MCP endpoints. Agents can also publish messages to Pub/Sub.
2.2. ADK integration (GA): Your agents can interact with your real-time data stored in Pub/Sub, Bigtable, BigQuery, or other Google Cloud services using pre-built ADK integrations. Developers can build agents acting on real-time context without having to implement complex configurations or plumbing.
3. Combine multi-agent systems with your data processing
3.1. Event-driven autonomous agents: As agents become core to our workflows, real-time data pipelines must evolve to incorporate them directly into the stream. We have enabled this capability by treating agentic logic as a first-class citizen within the Dataflow pipeline. You can now incorporate your agent code using the Agent Development Kit (ADK) and deploy it as a specialized node using the RunInference transform and the new ADKAgentModelHandler. Key advantages of this approach include:
-
- Massive scalability: Leverage Dataflow’s architecture to process high velocity events upstream and keep hundreds of agents sessions active simultaneously, each driven by specific incoming events.
- Pre-processing power: Dataflow handles the heavy lifting of complex data enrichment, delivering a “ready-to-act” context directly to the agent so it can focus on reasoning.
3.2. Dataflow Unified embeddings Sinks: We are introducing unified embedding generation directly within the data stream to eliminate “context lag”. You can now transform incoming data into high-dimensional vectors at low latency using Dataflow. These real-time embeddings are then seamlessly materialized into our expanded suite of high-throughput vector sinks, which now includes Cloud Spanner (featuring its new built-in vector search) and AlloyDB, providing you with an up to date vector database for semantic search needs as well as for your autonomous agents making RAG calls with an instantly searchable and perfectly synchronized long-term memory. This feature works with both remote and local models, for example Gemma.
As we continue to build out the platform, customers can expect to see even tighter integrations and more powerful capabilities. We look forward to seeing what you build with these new capabilities.
