Build Smarter Apps: Tool Calling & AI Orchestration Explained

Teachers
Mike Grabowski
CEO & Founder
@
Callstack

Generating text is the easy part. The hard part is turning AI output into reliable application behavior.

This episode shows what becomes possible when on-device models in React Native can do more than chat. Using the AI SDK support across the React Native AI family, you can combine structured outputs and tool calling to build complex, production-ready features that stay maintainable and predictable.

Tool calling lets a model interact with functions and APIs you provide, so it can fetch real data and perform actions. Structured outputs let you constrain responses to a defined schema, which makes the output type safe and directly usable without fragile parsing.

Together, these capabilities move AI from a UI demo to something you can trust inside real app logic.

Tool calling: connecting models to real data and actions

Tool calling enables on-device models to invoke external functions and APIs. In practice, this is how you bridge model reasoning with live data sources and app capabilities. The episode walks through simple and more advanced tool calls, showing how the same AI SDK API applies even when the underlying model changes.

A key takeaway is that tool calling is not limited to a single predefined action. You can expose any API you control, from small utilities to domain-specific operations, and let the model incorporate the results into its response.

From simple tools to app workflows

The examples move from a basic getCurrentTime utility to a more realistic scenario: checking an agenda using calendar data. This is where tool calling stops being a toy and starts looking like application behavior.

The episode also highlights an important implementation detail: when you surface results to users, time and date handling must respect localization and time zones. Tool calls can return correct data in the wrong format if you do not normalize it for the user.

Letting the model decide when to call tools

Beyond explicit tool calls, the episode shows a more flexible pattern: asking higher-level questions and allowing the model to choose which tools it needs. Instead of requesting raw data, you can ask for a decision, and the model can call tools like a calendar to analyze context before answering.

This is the core of orchestration. You are not writing a fixed flow. You are defining capabilities, and the model decides how to use them.

Structured outputs: type-safe AI results

Structured outputs constrain model responses to a specific format, such as a JSON schema with defined fields and types. This removes the need to parse freeform text and reduces failure modes in production.

The episode demonstrates multiple structured output shapes, including objects with typed fields, enums, booleans, and arrays. The practical benefit is that the output can feed directly into UI and business logic. You can generate forms, extract information, produce UI components, and build complex data structures with full type safety.

Building maintainable, production-ready AI features

These capabilities are not separate tricks. Tool calling and structured outputs work together to produce AI features that are reliable and maintainable. Instead of asking a model to guess everything in text, you constrain what it returns and give it the tools it needs to fetch facts and perform actions.

If you are building offline-first on-device AI experiences, this episode shows the patterns that make those experiences fit real production requirements.

Insights

Learn more about AI

Building a Software Factory: Write Specs an Agent Can Build From — cover

Nov 6Article

Building a Software Factory: Write Specs an Agent Can Build From

Learn how to research requirements, resolve open decisions, and write testable specs that AI coding agents can implement without the planning conversation.

 Large-Scale Migrations With Coding Agents: Planning, Verification, and Rollback — cover

Nov 4Article

Large-Scale Migrations With Coding Agents: Planning, Verification, and Rollback

Plan codebase migrations with coding agents using reviewable requirements, verification checks, staged rollouts, and defined rollback criteria.

Conversational Banking: From Customer Questions to Authorized Actions — cover

Nov 2Article

Conversational Banking: From Customer Questions to Authorized Actions

Explore conversational banking architecture through a card-dispute example, covering tool access, authorization, execution, observability, and human handoff.

How AI Agents Can Cut Manual QA by 30% — cover

Oct 30Article

How AI Agents Can Cut Manual QA by 30%

Explore an agentic QA workflow targeting 30% less manual effort across mobile, web, and TV, with test selection, failure triage, and validation.

Building a Software Factory: The Night My Agents Waited for Me — cover

Oct 28Article

Building a Software Factory: The Night My Agents Waited for Me

Learn how permissions, review limits, and missing product decisions stalled two overnight coding agent runs, and how their maintainer revised the workflow.

Your AI Coding Agent Still Needs an Architecture — cover

Oct 22Article

Your AI Coding Agent Still Needs an Architecture

Use explicit contracts, behavior tests, and dependency checks to guide AI coding agents and review their changes through a TypeScript favorites example.

cover

Oct 15Live Stream

Who Left the Agents Running? React Native Software Factories

Join Kewin Wereszczyński and Szymon Chmal to explore what can go wrong in a React Native software factory overnight and the controls that keep agents on track.

cover

Oct 14Podcast

When Coding Gets Faster, What Breaks Next?

Matt Hargett explains how AI moves bottlenecks across testing, deployment, infrastructure, support, and the rest of software delivery.

cover

Oct 12Podcast

AI Coding Agents and Hermes With Tzvetan Mikov

Tzvetan Mikov discusses AI coding agents, context switching, low-level engine optimization, reviewing specs and tests, and his plans for Hermes X.

cover

Oct 12Podcast

State Machines for AI Agents With David Khourshid

David Khourshid explains state machines for AI agents, human approval steps, tracing, and handling invalid model outputs in agent workflows.

cover

Oct 7Talk

Automating Mobile QA with Cloud Agents

Michał Pierzchała shows how cloud-based QA agents can test real iOS and Android user flows and bring screenshots, recordings, and logs into pull requests.

cover

Oct 1Live Stream

Can Jev Make AI Agent Decisions Faster?

Explore Jev with Kewin Wereszczyński and Szymon Chmal: how it works, its approach to faster agent decisions, and possible local alternatives.

cover

Mar 19Tutorial

The Hard Way vs. The React Native AI SDK Way: Stop Writing Custom Modules for Every Model

Learn how React Native AI uses a unified provider model to switch between cloud and on-device LLMs without rewriting application logic, enabling reliable AI experiences online and offline.

cover

Mar 12Tutorial

Build Smarter Apps: Tool Calling & AI Orchestration Explained

Learn how on-device models in React Native can call tools, interact with external APIs, and return structured outputs you can use directly in application logic.

cover

Mar 5Tutorial

MLC LLM + React Native: On-Device AI Without the Pain

Learn how to run third-party on-device LLMs in React Native using MLC LLM. Choose your models, run them efficiently across platforms, and keep a unified JavaScript API with React Native AI.

cover

Feb 24Tutorial

Intelligent Fallbacks with Apple Intelligence in React Native

Learn how to use Apple’s built-in Foundation Models in React Native apps, reduce memory usage, and enable fast, fully on-device AI with React Native AI Apple.

cover

Feb 10Tutorial

What Is the React Native AI SDK? A Complete Intro & Quickstart

Learn why on-device AI matters for React Native apps, how local LLMs behave offline, and what problems React Native AI is designed to solve from day one.

cover

Jan 29Tutorial

Implementing an Android TurboModule with Kotlin

Learn how to implement a React Native TurboModule on Android, from the TypeScript spec and Codegen to the Kotlin implementation and package registration.

cover

Jan 19Tutorial

Working with Different Threads in Swift TurboModules

Learn how JavaScript, native module, background, and main threads interact in React Native TurboModules, and how to use GCD queues to run async work safely in Swift.

cover

Jan 9Tutorial

How to Add Type-Safe Constants to Swift TurboModules

Learn how to expose typed, immutable constants from a Swift TurboModule using Codegen, and get full end-to-end type safety between native and JavaScript.

cover

Dec 30Tutorial

Adding Event Emitters to Your TurboModule in Swift

Learn how to emit native events from a Swift TurboModule and subscribe to them in JavaScript, including proper memory management and Codegen integration.

cover

Dec 22Tutorial

Writing Your First TurboModule in Swift

Learn how to build a fully working TurboModule in Swift, integrate it with Codegen, and bridge it through Objective‑C inside an Expo app.