Start Where the Data Lives: Best Practices for Taking IBM i Into AI

October 5, 2026 Doug McMaster Ron Venzin and Gregg Rohaly

The AI conversation has arrived at the IBM i shop, and in most cases, it did not come in through the computer room door. It came down from the boardroom, or over from finance, or sideways from a sales vice president who watched a demo at a conference and wants to know why the company cannot ask its own order history a question in plain English. The IBM i manager is then handed a question that sounds technical but is strategic: can our core platform play in this, or do we need to replace it to get there?

That is the wrong question, and answering it the wrong way is expensive. If IT says the platform cannot do AI, the very next meeting is about replacing the ERP, which is a far bigger risk and a far bigger bill than adopting AI around the system that already runs the business. The better question is where the data you already own justifies AI, and how to get there without anybody worrying about, say, order processing.

The pressure is real, and the numbers show it. In Fortra’s 2026 IBM i Marketplace Survey, IBM i skills displaced cybersecurity as the number one concern for the first time since 2017, and AI and machine learning was the biggest mover among top concerns, jumping from 30 percent of respondents in 2025 to 42 percent in 2026. Those two findings are not unrelated. Shops are short on people who understand twenty-year-old+ RPG, and they are hoping AI helps close that gap. It can. But only if the right foundational steps are taken, and taken in the right order.

Here is what we have learned about taking those first steps, drawn from the same hybrid cloud, disaster recovery, and managed services work we have written about in You Are Much More Than Power Systems, And So Are We and From Migration To Maturity: The Cloud Reality For IBM i Shops.

Stop Conflating Three Different Things Called “AI”

When an IBM i shop hears “AI,” it usually lumps three distinct ideas together, and that confusion stalls projects before they start. There is AI that runs on the platform, AI that reads platform data, and AI that helps people work on the platform. Each has different infrastructure requirements, different risks, and different payback periods.

On the first front, the hardware story has changed materially. Power10 and Power11 processors carry on-chip matrix math acceleration for inferencing, and IBM’s Spyre Accelerator, a 75-watt PCIe card with 32 AI cores, became generally available for Power11 in December 2025, with up to 16 cards per Power server. IBM positions Spyre for low-latency, on-premises inference and lists compatibility with Red Hat OpenShift AI, Red Hat AI Inference Server, and watsonx.data. The practical nuance for IBM i people: that software stack runs on Linux, so the model lives in a Linux partition sitting next to your IBM i LPAR on the same box, not inside QSYS. That is actually good news. Inference stays close to the data, behind the same physical and network boundary, without a GPU farm in somebody else’s datacenter.

On the second front, IBM Bob, the company’s AI coding agent, gained real IBM i depth on June 24, when IBM shipped the Bob Premium Package for i. It is a separate add-on to Bob rather than part of the base product, and it connects Bob directly to a live system: developers read source members from QSYS, edit them, and compile and test from inside the session, with no export-edit-import loop. Its IBM i skills span RPG II and III through ILE RPG, CL, DDS, COBOL for i, and Db2 for i SQL, covering fixed-to-free conversion, OPM-to-ILE migration, business rules extraction, documentation, and RPGUnit test generation. A separate Database Mode targets Db2 for i query work and can build an entity relationship diagram straight from the catalog of a live schema.

Be candid about what IBM i is – and what it is not. The IBM i platform, underpinned by Power Systems hardware, is not an AI model training platform. Its real value in an enterprise AI architecture is as a trusted source for inference and retrieval, leveraging some of the enterprise’s most valuable and current transactional data. That is not a limitation – it is a highly practical and valuable role for the platform.

Sequence By Risk, And Start Read-Only

The highest-value first use cases for IBM i shops are read-only: summarization, anomaly detection, documentation, and natural-language query. They deliver measurable value without touching the transaction path. We recommend sequencing adoption in four patterns, lowest risk first, and naming the business metric for each before any code is written.

Pattern four should be treated as a “deliberate” final-stage architecture – not as a prohibited approach. Embedding a model directly into a live transaction path introduces the highest level of performance, reliability, security, and operational risk, and therefore requires rigorous testing, governance, observability, and a well-defined rollback strategy.

This is particularly important in IBM i environments, where applications can be highly database-intensive and sensitive to additional network round trips, latency, locking, and transaction response times. An AI inference call that adds two seconds to a reporting workflow may be an acceptable performance tradeoff. Introducing that same synchronous dependency into an order-entry transaction can materially impact throughput, response time, and availability – and potentially turn an AI integration issue into a production incident.

The business driver: Sequencing by risk is how you earn a second AI budget. The first project has to produce a defensible number within a quarter, and it has to do so without a single anxious phone call from the order desk.

Treat Db2 For i (Or ERP Application of Choice) As The Asset, And Do The Unglamorous Work

Most midmarket IBM i shops are sitting on twenty or thirty years of consistent, integrity-enforced transactional history. That is a genuine competitive moat. A mid-sized distributor with three decades of clean order data can often get useful forecasting and anomaly detection faster than a larger competitor whose data is scattered across a dozen acquired systems.

But the moat has to be drained before anyone can use it. The unglamorous work is where AI projects on IBM i succeed or fail:

Six- and ten-character field names that mean nothing to a language model, and often nothing to the newer members of your own team.
Packed decimal fields, seven-digit CYYMMDD dates, and flag fields whose meaning lives only in someone’s head.
Decades of undocumented business rules encoded in data values rather than in code.

The practical fix is to build a semantic layer first: SQL views with descriptive column names, documented business meaning, and consistent date handling, sitting over the physical files. That layer pays for itself twice, once for AI retrieval and again for every reporting and integration project that follows. And remember that retrieval-augmented patterns rarely need the whole library. You are indexing curated extracts and documents, not every file in every schema.

The business driver: Data quality is the one AI advantage a midmarket firm can own outright. It cannot be bought from a hyperscaler, and it is already on your disk or flash.

Put Your DR Copy To Work

Here is an ROI argument many IBM i shops miss. If you already pay for a high availability or disaster recovery environment, you own a standing, current copy of production data that is not carrying transaction load. For many AI read workloads, that replica is the right target. It isolates analytical and inference query load from production entirely.

The guardrails matter. Enforce read-only access. Understand your replication lag, because a model summarizing yesterday’s exceptions is fine and a model promising real-time inventory from a lagging copy is not. Above all, AI workloads must never compromise recovery point objectives, recovery time objectives, or failover readiness. Whether this works cleanly depends on your replication method, so validate it with whoever owns the recovery runbook before anyone points a model at it.

The business driver: DR infrastructure is typically funded as a necessary cost of resilience—an insurance policy that protects the business but does not directly contribute to revenue or operating efficiency. Copurposing that infrastructure as an analytics and AI read tier changes the budget equation. The organization can fund one infrastructure investment that serves two purposes: disaster recovery when needed, and analytics and AI workloads every day.

For the CFO, the value is not simply higher utilization. It is the ability to justify existing DR spend through additional business outcomes without requiring a separate, equivalent infrastructure investment for analytics and AI. That can reduce incremental capital requirements, improve the return on existing infrastructure, and make future DR capacity investments easier to defend because they support productive workloads as well as resiliency.

There is also an operational payoff. Because the DR environment is continuously exercising compute, storage, networking, replication, and data-access paths, it is being validated continuously rather than only during an annual DR exercise. The result is infrastructure that both produces business value today and strengthens recovery readiness for tomorrow.

Decide Where Inference Lives Before You Pick an Architecture

Every AI architecture decision for an IBM i shop is really a data movement decision. For example, Db2 for i datasets are typically large, sensitive, and constantly changing. Most shops land on one of three patterns, or a hybrid of them:

Inference local to Power. On-chip acceleration or Spyre in a Linux partition adjacent to IBM i. Data never leaves the frame.
Inference in a hosted private cloud. Models run in a managed environment adjacent to the IBM i LPAR, inside a controlled boundary, without the customer buying and running the accelerator hardware.
Inference in a hyperscaler AI service. Fed by curated extracts, with the broadest model choice and the least control over where the data travels.

None of these is universally the best. What is universally true is that moving decades of Db2 for i data to a cloud model, repeatedly, can be a recurring cost line and not a one-time project. Token charges and egress fees are operating expenses that compound with adoption. We wrote earlier this year in IT Jungle about the cost shock pattern in cloud transformation, where the surprises come from everything around the IBM i core. AI is the newest way to recreate that pattern if the architecture is chosen by default instead of by design.

The business driver: Architecture chosen badly becomes a permanent monthly bill. Getting the inference location right early is the difference between a predictable total cost of ownership and a line item the CFO starts asking about every month.

Govern It The Way IBM i Governs Everything Else

IBM i shops have both an advantage and a blind spot when it comes to AI governance. The advantage is object-level authority, user-profile-based security, and journaling and auditing that have been built into the platform for decades. The blind spot is weak native federation with cloud identity systems and the audit expectations that come with AI services.

The failure modes are predictable. A service account is created with *ALLOBJ, or something close to it, to feed an AI pipeline and is never tightened. Identities are duplicated across platforms. The audit trail stops cold at the IBM i boundary. And the first question every auditor will ask, which is what data was sent to a model, whether it was retained, and whether it trained anything, has no documented answer.

Four practices close most of the gap:

Inventory the AI use is already happening informally. Developers pasting production RPG into public chat assistants is the most common blind spot we see.
Give AI pipelines narrowly scoped profiles that read from semantic views, never the underlying physical files, and nothing more.
Log prompts, responses, and model versions for anything touching financial or customer data, and feed IBM i audit journals into enterprise observability instead of leaving them isolated.
Where data sensitivity demands it, keep the model and the data inside the same controlled boundary with private AI hosting.

The business driver: In regulated midmarket sectors, an ungoverned AI pilot is an audit finding waiting to happen. And the IBM i team that brings a governed AI proposal to the table controls the platform’s narrative. The team that stays quiet gets AI policy written around it.

Use AI To Narrow The Skills Gap, Not To Hide It

The skills argument cuts both ways, and IT Jungle readers will see through anyone who pretends otherwise. AI genuinely accelerates code comprehension and documentation, which is the single hardest part of onboarding a new developer onto a thirty-year-old application. Having an assistant explain what a 4,000-line RPG program does, and produce the documentation nobody wrote in 1998, is real, measurable value. It is also exactly the job IBM built the Bob Premium Package for i around, with business rules extraction and technical documentation generation among its named capabilities.

The limits are just as real. AI-generated RPG still requires someone who can judge whether it is right. AI does not carry operational accountability, does not own the recovery plan, and does not answer the pager at 2 a.m. when a subsystem hangs during month-end close. AI plus a shrinking internal team is still a single point of failure. AI plus an accountable managed operations team is a durable model.

The business driver: The retiring administrator is the board-level risk in most IBM i shops. AI is a partial mitigation. An accountable operations partner is the rest of it.

Make The First Project Pay For The Second

Skip the framework. Start with one bounded, read-only use case with a measurable 90-day outcome. Good candidates include critical-application code comprehension and documentation, or anomaly detection across job logs, message queues, and subsystem activity – with human approval required for any action.

Define the architecture before you start: data source, retrieval layer, model and hosting location, user interface, and accountability for each component. That last item is often overlooked – and is critical to moving from pilot to production.

Most importantly, start with the business problem, not “AI.” Focus on measurable outcomes such as fewer order exceptions, improved inventory accuracy, shorter close cycles, reduced service-ticket volume, or fewer audit-preparation hours. Solve the business problem first; the AI investment follows.

Questions To Ask Before You Commit

Whether you build it yourself or engage a services provider, get clear answers to these before the first model call:

Who owns the outcome, not just the infrastructure?
Where does our data go, what happens to it there, and does it train anything?
How is every AI interaction with our data logged and audited?
What is the recurring cost at ten times the pilot volume?
Who is accountable when a model misbehaves at 2 a.m., and how does it affect our recovery objectives?

Extend What Works

At CloudSAFE, we have watched the same adoption pattern play out in cloud for years: customers prove the model with backup, then disaster recovery, then production, building trust at each step. AI follows the same curve. Prove it on a non-production or DR-adjacent workload, measure it in business terms, then expand. Our IBM i and Power heritage, combined with private AI hosting and managed public cloud under one operating model, means the architecture decision is driven by your data and your risk tolerance, not by whatever a single provider happens to sell.

IBM i is not an obstacle to AI adoption. It is the data asset that makes AI adoption worth doing. The first step does not require replacing the system that runs your business. It requires deciding, deliberately, where to start as we outlined in this article.

Doug McMaster is chief executive officer at CloudSAFE. Ron Venzin is chief development officer at the company, and Gregg Rohaly is business development manager.

This content was sponsored by CloudSAFE.

SOURCES

Fortra, IT Initiatives & Trends: 2026 IBM i Marketplace Survey Results

IBM Community, Announcing General Availability of IBM Spyre Accelerator for Power (December 2025)

IBM Newsroom, IBM Introduces the Spyre Accelerator for Commercial Availability (October 7, 2025)

IBM, DbToo: A Db2 for i SDK for connecting to various services

IT Jungle, New DbToo SDK Hooks RPG And Db2 For i To External Services (August 25, 2025)

IBM, SDK for Db2: A comprehensive guide to using AI with IBM i

IBM Bob Blog, Premium Package for i is now available (June 26, 2026)

IBM Bob Docs, IBM Bob Premium Package for i: Skills

IBM, Announcing the IBM Bob Premium Package for i (July 9, 2026)

This Issue Sponsored By

Table of Contents

Content archive

Recent Posts

Subscribe

Pages

Search